On this page
The short answer
Evaluate rare failures with two complementary records: targeted tests that try to expose the failure mechanism and representative observations that estimate its frequency. Keep their denominators separate. Zero observed failures in a finite sample is evidence about that sample, not proof that the failure cannot occur.
What to take away
- Use discovery tests and prevalence samples separately.
- Preserve mechanism families and repeated-trial counts.
- Report nonzero uncertainty after zero observed events.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Separate discovery from frequency estimation
SciPy’s proportion-interval documentation provides exact and Wilson methods for binomial outcomes. A statistical interval can describe sampling uncertainty under its assumptions, but it cannot make a deliberately enriched challenge set representative of customer traffic.
Define the failure precisely and record the observation unit. For a permission boundary, a task may fail if any unauthorized state change occurs. For a long conversation, several correlated turns may belong to one opportunity for failure. Counting those turns as independent observations can exaggerate the evidence available about a rare event.
Evidence: SciPy: BinomTestResult.proportion_ci [1]
Build targeted cases around the mechanism
Create controlled scenarios for the boundary you care about: conflicting instructions, missing approvals, shared-state races, or a changed permission context. Include legitimate controls so a system that blocks all work does not look successful. Record what was attempted and the final observable state.
Group variants by underlying mechanism. One thousand paraphrases of one injection are not one thousand independent discoveries about all prompt-injection risks. Report unique mechanisms, configuration coverage, repeated trials, and unresolved paths. A reproduced critical failure can be decision-relevant even when its production frequency remains unknown.
Keep zero-event arithmetic honest
For an illustrative independent sample with zero failures in 100 opportunities, the exact one-sided 95% upper bound is 1 minus 0.05 raised to the power 1/100, approximately 2.95%. This calculation is not a production reliability result and assumes identical independent opportunities.
The bound explains why zero events does not mean a zero rate. If observations are clustered by customer or incident, use an analysis that reflects the dependence or narrow the claim. If the sample excludes the risky workflow entirely, no interval around its aggregate result answers the missing boundary question.
| Case or record | Expected check | Decision |
|---|---|---|
| Zero failures in 100 independent cases | One-sided 95% exact upper bound≈2.95% | Do not claim zero risk |
| Enriched attack suite | Selection is intentionally nonrepresentative | Report capability coverage, not traffic prevalence |
| Reproduced unauthorized write | Observable contract violation | Apply the agreed blocking action |
Choose the next evidence deliberately
Use the targeted suite to identify and repair mechanisms, then retest original and legitimate cases. Use representative sampling to estimate operating frequency where the product decision requires it. Preserve the split between these purposes when creating dashboards or progress summaries.
Set the response to a confirmed failure before testing: pause a capability, require review, or continue a bounded investigation according to the product’s agreed policy. Do not keep collecting easy successes merely to dilute a reproduced boundary failure in a larger denominator. Record remaining uncertainty and the owner of each untested path.
Limits of the evidence
The zero-event bound is a mathematical illustration, not measured AI performance. Its independence and identical-opportunity assumptions may fail in real workflows. Targeted tests can find important defects without estimating their prevalence or proving their absence elsewhere.
Common questions
Does one rare failure invalidate every score?
It does not erase unrelated evidence, but it can block a release when it violates a non-negotiable requirement. Keep that decision separate from an average.
Can more repeated trials help?
Yes, for instability on the tested case. Distinguish repeated attempts from new independent tasks and from coverage of new failure mechanisms.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
- SciPy: BinomTestResult.proportion_ci SciPy · Living reference · Maintained documentationReviewed: exact and Wilson binomial proportion interval methods and returned bounds. Accessed October 16, 2026.
Questions or corrections? Write to Rubrex. Read our editorial approach.