On this page
The short answer
Evaluate numerical reasoning by checking the quantities selected, the operation performed, the units, and the final tolerance. A close number with the wrong unit can still fail the task. Define rounding and missing-data rules before scoring, and use independent arithmetic to verify references and illustrative test cases.
What to take away
- Define tolerances from the task, not library defaults.
- Check selected quantities and units.
- Handle zero, missing, and nonfinite values explicitly.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Write the numeric contract before comparing strings
Python’s math.isclose documents separate relative and absolute tolerances and their comparison rule. Those defaults are software behavior, not a recommended acceptance tolerance for every business calculation. Choose tolerances from the task and retain the raw value and unit.
Define whether the expected result is exact, rounded, approximate, or a range. For monetary extraction, an exact decimal representation may be required; for an estimated measurement, a justified tolerance may be appropriate. Do not accept a unit conversion merely because the digits resemble the reference.
Evidence: Python: math.isclose [1]
Check inputs and operations separately
Record source quantities, intended formula, required conversions, and the reference calculation. Distinguish a reasoning error from a source-reading or formatting error. A correct multiplication of the wrong subtotal is not a successful end-to-end answer.
Include negative values, zero denominators, missing inputs, percentages, and scale labels such as thousands where relevant. Define how the system should respond when the calculation is not supported. A model that invents an absent input to produce a neat number should fail the evidence contract even if the arithmetic is internally consistent.
Use explicit worked tolerance cases
For an illustrative conversion, two and a half hours equals 150 minutes. Returning 2.5 minutes fails the unit relationship. For a separate approximate task with expected value 100, an absolute tolerance of 0.5 and zero relative tolerance accepts 100.4 and rejects 100.6. These are authored arithmetic controls.
Near zero, a relative tolerance alone can be inappropriate because the permitted absolute difference shrinks with magnitude. Choose an absolute tolerance where the task requires it. Reject NaN or infinity unless the output contract explicitly permits and interprets them; do not let unusual numeric values silently bypass comparisons.
| Case or record | Expected check | Decision |
|---|---|---|
| 2.5 hours converted to 150 minutes | Value and unit relationship correct | Pass the conversion control |
| Expected 100; absolute tolerance 0.5 | 100.4 accepted;100.6 rejected | Illustrative boundary test |
| NaN in ordinary numeric field | No meaningful finite result | Reject under the stated contract |
Report the reason behind every numeric miss
Keep value correctness, unit correctness, evidence support, and formatting as separate criterion outcomes. Report complete task acceptance only when the required combination passes. Preserve the formula and normalized values so a reviewer can recompute the result.
Compare candidates on the same references and tolerance policy. If a tolerance changes, rescore saved outputs and label that policy change instead of calling it a model improvement. Use systematic error categories to decide whether the next fix belongs in retrieval, calculation tools, prompt instructions, or downstream validation.
Limits of the evidence
All calculations are illustrative controls, not model results or financial advice. Appropriate precision depends on the application. Floating-point behavior, decimal requirements, unit conversion, and incorrect reference values can each affect acceptance.
Common questions
Should I use exact string matching?
Use it only when formatting is part of the contract. Otherwise compare parsed values with defined units, rounding, and tolerance rules.
Can a correct final number hide a reasoning error?
Yes. Inspect selected inputs and required evidence where the workflow needs an auditable calculation, especially when errors can cancel by coincidence.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
- Python: math.isclose Python Software Foundation · Living reference · Maintained documentationReviewed: documented relative/absolute tolerance formula and nonfinite-value behavior. Accessed October 16, 2026.
Questions or corrections? Write to Rubrex. Read our editorial approach.