On this page
The short answer
Test incremental indexing by comparing the expected document and chunk manifest with the searchable state after each change. A successful job status is useful operational evidence, but it does not prove every intended record is correct.
What to take away
- Use a stable expected-record manifest.
- Replay retries and interrupted runs.
- Check content versions, not counts alone.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Define the expected searchable state
Azure describes indexer change tracking and reset operations. Build a test independent of the job’s success flag: for each source revision, define expected searchable IDs, content hashes, eligibility metadata, and derived chunk relationships. Counts alone are weak because one missing record and one unexpected duplicate can cancel out. Keep the source snapshot and expected manifest versioned together so a later evaluator can distinguish an indexing defect from a deliberately changed corpus.
Evidence: Run or reset indexers [1]
Author a small mutation sequence
Start with fictional records A and B. Add C, edit A, delete B, and repeat the update request for C. The expected final state contains the edited A and a single C, with B unavailable to eligible retrieval. If one document expands into several chunks, specify which old chunks must disappear when its new text becomes shorter. The criterion checks identities and revisions, not simply that two parent documents exist after the job finishes.
Test interrupted and repeated work
Stop a controlled indexing run between source processing and completion, then resume it using the system’s supported workflow. Observe whether retries create duplicate chunks or leave half-updated parent-child relationships. Preserve job IDs, source revisions, acknowledgments, and the final searchable manifest. A transient error can be acceptable if recovery reaches the correct state under the agreed policy. Silent partial completion should fail even when most documents are searchable and the overall job later reports success.
Validate content and retrieval separately
First verify that each expected record exists with the intended fields and revision. Then run a small set of evidence probes to confirm that the searchable representation is usable. A record can be present but contain empty extracted text, stale permissions, or a vector generated from the previous content. Conversely, a retrieval miss does not automatically mean the record is absent. This two-stage check helps the owner choose between ingestion repair, representation repair, and search tuning.
Use discrepancies as release evidence
Classify differences as missing, unexpected, duplicate, stale, or structurally incomplete. Assign a remediation path for each class and rerun the full mutation sequence after a fix. Do not repair a test corpus by hand and then claim that incremental recovery worked. Record any manual intervention explicitly. Before promoting the pipeline change, keep the old index available or establish a tested rebuild path so a failed incremental run cannot leave the application dependent on an unexplained mixed state. Include a no-op update with unchanged content. It should preserve the intended searchable state without accumulating duplicate derived records, which provides a useful control for retry and idempotency behavior.
Limits of the evidence
The mutation sequence is authored and was not executed against a live index. Atomicity, retry behavior, and supported recovery operations depend on the chosen backend.
Common questions
Why not compare document totals?
Equal totals can hide a missing record paired with an unexpected or duplicate one.
Should retries count as failures?
Judge them by documented recovery behavior and the final state, while retaining their cost and operational impact.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
- Run or reset indexers Microsoft · Living reference · Maintained documentationReviewed: indexer change tracking, reset and resynchronization distinctions. Accessed October 16, 2026.
Questions or corrections? Write to Rubrex. Read our editorial approach.