Test PDF retrieval across headings, footnotes, and reading order

Treat the PDF as a structured source

Microsoft’s layout documentation describes structure-aware extraction and location metadata. Your PDF test still needs to establish that those features preserve meaning on your files. A PDF can contain a text layer whose order differs from the visual page, or an image whose words require recognition. Record file type and extraction path. Do not compare a clean digital PDF with a degraded scan and attribute their different outcomes entirely to the retriever or embedding model.

Evidence: Document Layout skill [1]

Author a reading-order trap

Create a fictional two-column safety handbook. The left column describes indoor use; the right describes outdoor restrictions. Place a footnote below both columns applying only to the outdoor paragraph. Ask an indoor question and an outdoor exception question. The fixture passes only if each answer uses its applicable column and qualifier. Add a heading continued on the next page to test whether chunking retains the section identity instead of attaching the following paragraph to a nearby unrelated heading.

Trace evidence through each representation

Save the original file hash, page image reference, extracted blocks, block order, and indexed chunk. When the expected passage is absent, identify whether recognition, layout ordering, heading association, or chunk assembly first lost it. Search cannot repair text that was never indexed correctly. Conversely, a faithful extraction does not prove that ranking finds it. Evaluate retrieval only after confirming that a complete, eligible evidence bundle exists in the searchable representation.

Test citations at the page boundary

A citation should locate the evidence supporting the claim, not merely open the file. Verify page numbers against the viewer’s numbering convention, especially when front matter uses separate labels. Check that a cited excerpt crossing pages still contains the qualifying sentence. An answer can be correct while its page link points to a nearby but unsupported location. Record content support and citation-location accuracy separately so the remediation targets generation, provenance mapping, or document-viewer integration appropriately.

Choose a bounded format acceptance rule

Define which PDF patterns the application supports and route unsupported cases for review rather than silently indexing empty or scrambled text. Include wide tables, rotated pages, repeated headers, and image-only scans when they occur in the intended corpus. Keep representative fixtures small enough to inspect visually after parser changes. A successful file upload or extraction job is not the release gate; require preserved critical facts, usable source locations, and accepted answers on the agreed layout patterns. Include a file whose visible page label differs from its physical page index. That control can expose citation-link offsets that otherwise look like unsupported generation.

Common questions