Evaluate retrieval from tables with row and column context

Define the evidence unit around a cell

Microsoft’s Document Layout skill describes extracting structure from documents. Table evaluation should check whether the representation preserves the relationships your task needs, rather than merely whether it outputs text. Define an evidence bundle containing table title, row and column headings, cell value, units, and any qualifying note. Retain original page or table coordinates for inspection. A flattened row can look readable while silently attaching its value to a heading from the next column.

Evidence: Document Layout skill [1]

Create a crossed-column example

Author a table of fictional regional revenue with columns for 2025 actual and 2026 forecast, measured in thousands of units. Ask for the 2025 result for the north region. Include a footnote excluding one discontinued product from the forecast only. The acceptance rule requires the actual column, correct region, and correct unit. An answer using the forecast number, interpreting thousands as individual units, or applying the forecast-only footnote to actual results should fail for different recorded reasons.

Test parsing and retrieval independently

First inspect whether the extracted table retains the required labels and relationships. Then query the indexed representation and check whether the full evidence bundle reaches the context. Include questions by row name, period, and a paraphrase of the measure. If the parser dropped the unit, retrieval cannot recover it from a cell-only chunk. If the bundle exists but ranking returns another region, inspect search and entity matching. This separation prevents a table-structure defect from becoming an endless embedding-tuning exercise.

Check calculations as additional claims

For an answer that sums rows or compares periods, preserve the source operands and the calculation. A correct arithmetic operation on the wrong cells is still incorrect. Decide whether subtotals and totals can be combined without double counting. Add a fixture with a missing value and another with a dash that means unavailable rather than zero. The evaluator should reject an invented numeric result when the representation does not support the operation or its assumptions.

Set a table-specific release gate

Require every critical numerical answer to retain source coordinates, applicable labels, and units in its evidence record. Test a parser or chunking change on wide tables, merged headers, continued tables, and footnotes before broad reindexing. Keep an allowed abstention path when a required cell relationship cannot be established. The next engineer should be able to inspect the original table and identify exactly which relationship was lost, rather than guessing from a plausible-looking number in the answer. Repeat the fixture after column reordering to check that the pipeline relies on labels and structure rather than an accidental column position.

Common questions