On this page
The short answer
Evaluate table extraction at the cell, structure, and task-result levels. Correct text in the wrong row or under the wrong header can be more harmful than a visible OCR error. Preserve source coordinates or cell references and define how merged headers, blank cells, units, and repeated page headers should be represented.
What to take away
- Score text and structure separately.
- Preserve units, blanks, zeros, and merged headers.
- Check downstream row relationships.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Define a structural reference
PubTabNet documents table HTML structure and cell annotations, with resources for structure-sensitive evaluation. That illustrates why table recognition includes relationships as well as text. This guide does not download its data or reuse its historical benchmark results.
Choose the representation your application needs: a grid, row records, hierarchical headers, or another explicit schema. Preserve row and column identity, merged-cell spans, units, and source-page references. A flattened list of cell strings cannot by itself establish which number belongs to which product or reporting period.
Evidence: PubTabNet: Annotation structure and TEDS resources [1]
Test realistic structural difficulty
Include multi-level headers, merged cells, blank versus zero values, footnotes, repeated headers, and tables split across pages where relevant. Keep the same table in different rendered layouts only when the content remains equivalent and reviewers verify that relation.
Separate recognition errors from structural association errors. A digit can be recognized correctly yet attached to the wrong row after an omitted blank cell. Define acceptable normalization for spacing and number formats, but do not normalize away units or sign characters that alter the task’s meaning.
Use a row-shift fixture that exposes hidden errors
Consider an invented table with products Cedar and Harbor, quantities two and three, and unit prices ten and twenty. The intended row totals are twenty and sixty. An extraction that swaps the quantities can preserve every source number while producing the wrong product records.
A proposed acceptance check verifies each product-to-quantity-to-price relationship and the derived totals. A bag-of-values match would miss the defect. Include a blank cell and a zero cell as separate cases when the workflow distinguishes missing data from an observed zero.
| Case or record | Expected check | Decision |
|---|---|---|
| Correct numbers; quantities swapped | Product associations are wrong | Reject the extracted records |
| Blank cell versus zero | Missing and observed values differ | Preserve the distinction |
| Cedar 2×10; Harbor 3×20 | Expected totals 20 and 60 | Use as a relationship control |
Combine diagnostics with downstream acceptance
Report cell-content matches, structural relationships, and complete-table acceptance separately. If a structure score is used, retain its implementation and explain what it does not test, such as unit semantics or a downstream calculation. Keep unparseable tables visible.
For the real workflow, execute only the permitted deterministic calculations against the extracted records and compare with reviewed reference outcomes. Preserve the original extraction so a repair to postprocessing can be distinguished from a model change. A table passing arithmetic checks can still contain a wrong label, so use several independent acceptance conditions.
Limits of the evidence
The product table and arithmetic are authored illustrations. No table-recognition system or TEDS implementation was evaluated. Structural references can contain annotation errors, and a finite set of layouts cannot establish performance on every document style.
Common questions
Can OCR accuracy measure table quality?
It measures part of the problem. Row, column, header, and unit associations still need explicit checks.
Is a structural similarity metric enough?
No. It can diagnose layout reconstruction, while the application may also require semantic units, labels, and correct downstream calculations.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
- PubTabNet: Annotation structure and TEDS resources IBM Research · Living reference · Maintained documentationReviewed: README table HTML/cell annotation structure and TEDS resource link; no benchmark data downloaded. Accessed October 16, 2026.
Questions or corrections? Write to Rubrex. Read our editorial approach.