On this page
The short answer
After deleting a document in a controlled test corpus, verify that it disappears from every eligible retrieval path and derived context. Source deletion alone does not establish that chunks, summaries, or caches stopped influencing answers.
What to take away
- Trace every derived representation.
- Distinguish deletion from ineligibility.
- Test caches and parent lookups.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Document the deletion mechanism
Azure explicitly distinguishes source change detection from deletion detection. Your test should name the actual configured mechanism rather than assuming physical source removal automatically updates search. Map the source ID to child chunks, parent records, embeddings, graph artifacts, and cached retrieval results. Decide what the application promises after deletion and when that promise becomes effective. Operational retention and backups may follow different policies; do not confuse them with whether content remains eligible for live answers.
Create a uniquely identifiable source
Author a disposable policy document containing a distinctive fictional project code and rule that appear nowhere else in the test corpus. Ask a question that requires that rule and confirm that the source is retrievable before deletion. Then remove or mark it deleted through the normal supported path. Use a related surviving document as a control. The post-deletion expectation is that the unique rule is no longer supplied as approved evidence, while the surviving source remains available.
Probe independent retrieval paths
Check direct document lookup, semantic and lexical search, expanded parent retrieval, and any graph or summary path your application uses. Warm a retrieved-context cache before deletion, then repeat the original query after the expected invalidation point. Keep the returned IDs and source revisions rather than relying only on the final answer. A model might avoid repeating the deleted fact by chance even while the prohibited source remains in its context, so evidence inspection is essential.
Define safe behavior during propagation
If deletion is asynchronous, test the documented pending state and the application’s response before propagation completes. A tombstone or eligibility filter may be needed to prevent serving known-deleted content while a background index job catches up. Verify the configured behavior rather than prescribing one implementation for every system. Record source deletion time, acknowledgment time, last observed stale evidence, and first confirmed clean read. Failed probes should remain visible and should not be mistaken for successful removal.
Preserve a regression record without republishing data
Keep synthetic IDs and structural evidence for the deletion test. For real sensitive deletions, retain only the audit information allowed by the organization’s policy rather than copying the deleted content into unrestricted evaluation logs. Exercise reindexing and cache restoration with the fixture to check that old artifacts cannot resurrect the source. Treat any stale derived representation as a separate remediation item with a named owner and a concrete retest path. A source recreated under the same filename should receive explicit revision lineage. Test whether old child records or cached references are incorrectly attached to the replacement, even when the displayed document title remains unchanged.
Limits of the evidence
This is a synthetic deletion-validation procedure, not a guarantee of legal erasure, backup removal, or absence of model memorization. Test your actual retention commitments separately.
Common questions
Does a missing source file prove successful removal?
No. Search chunks, derived artifacts, and warm caches may still contain usable copies.
Can the final answer alone verify deletion?
No. Inspect supplied evidence because the model may omit a still-retrievable source on a particular run.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
- Change and delete detection using indexers Microsoft · Living reference · Maintained documentationReviewed: change tracking and explicit deletion detection requirements. Accessed October 16, 2026.
Questions or corrections? Write to Rubrex. Read our editorial approach.