Test whether chunk overlap helps retrieval or adds noise

State what overlap must repair

Azure describes overlap as a way to preserve continuity across text chunks. Your evaluation needs a narrower hypothesis: a condition and its exception are being separated, so retrieval returns only half the rule. Mark the two source spans and identify which questions require both. Also retain questions supported by a single intact span. Without those controls, a test composed entirely of boundary problems makes increased overlap look useful by construction rather than showing its actual tradeoff.

Evidence: Chunk large documents for RAG and vector search [1]

Author a repeated-text example

Use a fictional warranty document with an eligibility sentence at the end of one chunk and the exclusion at the start of the next. Produce zero-overlap and overlapping variants. For another question, repeat an unrelated marketing paragraph near both chunks. This tests whether overlap crowds out the exclusion when retrieval selects similar neighboring results. The acceptance rule requires the answer to include the eligibility condition and exclusion; citing two overlapping copies of the same sentence does not satisfy both.

Measure marginal evidence rather than chunk count

For every returned context, calculate how much distinct relevant source text is present. Use original source offsets to merge overlapping spans before counting them. Record how many retrieved chunks contribute no new evidence. Compare answer acceptance at equal context-token budgets and separately record index size and ingestion work. A higher count of relevant-looking chunks can coexist with less useful context if near duplicates occupy the available slots. Keep both outcomes visible instead of averaging them into one score.

Keep overlap conditional on document structure

Adopt an overlap change when it repairs the targeted boundary failures and does not displace evidence for other question groups. If the same failures disappear after heading-aware splitting, compare that simpler alternative before increasing overlap everywhere. Long narrative documents and short structured policies may need different configurations. Preserve the rejected examples as regression tests, including the case where duplicate neighbors displaced an exception. That record makes the decision about evidence, rather than a default percentage copied into every ingestion pipeline.

Audit repeated text after normalization

Overlap can become difficult to recognize after a parser changes whitespace, removes headers, or joins hyphenated words. Keep a mapping from normalized chunk text back to original source positions where possible. When exact offsets are unavailable, manually review suspected duplicates in the small trial set rather than presenting a similarity threshold as ground truth. Two passages that look almost identical may contain different dates or an important negation. Deduplication must preserve those differences. Track accidental loss of unique qualifiers as its own failure instead of crediting every removed passage as useful compression.

Common questions