On this page
The short answer
Enforce retrieval access before restricted passages enter model context. Evaluate query filters, document fetches, snippets, citations, and cached results under the caller’s actual identity. An assistant refusing to quote a private document is not sufficient if the application already supplied that document to an unauthorized request.
What to take away
- Use authorization at retrieval and fetch boundaries.
- Inspect returned context, not only final answers.
- Test changed permissions and cached evidence.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Define the caller and protected resource
OWASP’s authorization guidance emphasizes explicit permission checks. Rubrex recommends applying them to each retrieval surface instead of relying on the answer prompt. The query service, document store, snippet endpoint, and citation destination may have different access paths.
Create two synthetic teams with a shared public handbook and separate private appendices. A team-one user may retrieve the handbook and their own appendix. Give every document a harmless unique marker so the evaluator can identify a restricted passage even if the generated answer paraphrases it.
Evidence: OWASP: Authorization Cheat Sheet [1]
Probe the boundary before and after search
Issue a normal search, then request a known private document identifier directly. Repeat through a citation link and a snippet-preview endpoint if those surfaces exist. A search filter can be correct while a direct fetch still exposes the content. Retain the caller identity and resource decision at each path.
Change the user’s team after one authorized query and repeat the request using a warmed cache. The cached content should be reevaluated or scoped under the chosen access contract. Document how permission changes propagate; do not silently assume every cache entry updates at the same instant.
Inspect model context and metadata leakage
Capture the actual retrieved context supplied to the model in the isolated fixture. Check titles, snippets, counts, and citation metadata as well as full text. A document’s existence or title may be restricted even when its body is withheld, depending on the product rules.
Include a legitimate public query and a newly authorized private query after a controlled permission grant. These controls verify useful access in both directions. Report unauthorized context delivery separately from final-answer leakage, because a model that declines to repeat restricted content has not repaired an upstream access failure.
| Case | Evidence to retain | Acceptance condition |
|---|---|---|
| Direct private ID | Fetch decision and context | Unauthorized document withheld |
| Permission revoked | Policy version and warmed cache | Old access not reused |
| Public handbook | Returned evidence | Permitted retrieval succeeds |
Release retrieval with a documented scope contract
Accept the two-team fixture when protected markers and restricted metadata never enter unauthorized context through the tested paths, while permitted evidence remains available. Bind checks to trusted caller context rather than a model-generated team identifier. Preserve access decisions with document and policy versions.
Re-run direct fetch, citation, and warmed-cache cases when adding a retrieval component or changing permissions. This article focuses on the pre-context boundary; answer-quality and citation-faithfulness tests still apply after authorized evidence is retrieved. Passing one layer should not be used to claim the other has been evaluated.
Limits of the evidence
The teams, documents, and markers are synthetic. This proposed access fixture does not prove complete tenant isolation or immediate revocation across every distributed component. Permission semantics and propagation guarantees must be defined for the actual application.
Common questions
Can the model enforce document permissions?
The retrieval and document services should enforce them before content reaches the model.
Should citation links be tested too?
Yes. A correct search filter does not establish that direct document and citation endpoints are protected.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
Questions or corrections? Write to Rubrex. Read our editorial approach.