On this page
The short answer
Test cross-document questions with an explicit evidence chain. Verify that every required source is retrieved and that the answer combines them using a supported relationship rather than a plausible guess.
What to take away
- Label every necessary evidence link.
- Check joins between source entities.
- Include missing-link controls.
Sources checked . Research synthesis and Rubrex recommendations. Examples and local experiments are labeled; they are not client results.
Define the required evidence chain
RAGBench frames RAG evaluation around explainable aspects of retrieved evidence and answers. Your cross-document fixture should specify which facts must be combined and why the join is valid. A shared product name may be insufficient if the sources concern different versions or regions. Store stable entity and revision identifiers where available. Distinguish a question requiring two sources from a question that could be answered by either of two interchangeable sources; those are different coverage obligations.
Evidence: RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems [1]
Author a two-step compatibility case
Create one fictional catalog stating that device D-4 uses connector C-2, and another approved guide stating which adapters support C-2. Ask which adapter can be used with D-4. The answer requires both sources and the connector relationship. Add a distractor about device D-5 and an adapter guide for an obsolete connector revision. The acceptance criterion requires the valid D-4-to-C-2 link and the applicable adapter rule, not simply two citations mentioning similar equipment.
Measure retrieval completeness before synthesis
For each question, record whether all mandatory evidence groups reach the final context. Score individual group recall and complete-chain availability separately. An average that finds one of two sources for every question can look acceptable while no question is fully answerable. Include a control where one link is intentionally absent from the eligible corpus. The correct response should state the missing relationship or ask for more information rather than completing the chain from background assumptions.
Inspect the join made by the answer
Require the answer to use compatible identifiers, dates, units, and scopes across documents. A judge should identify which source establishes each premise and which rule permits the conclusion. If the application performs a structured join, retain its key values. If it reasons in text, preserve the relevant passages and explanation for review. A correct-looking adapter name without a traceable supported join is insufficient evidence that the workflow can handle adjacent cases reliably.
Choose the repair from the broken link
If a required source is missing, investigate query expansion, candidate retrieval, or corpus eligibility. If both sources are present but the answer combines mismatched entities, test synthesis and entity resolution. If the source relationship is itself ambiguous, the correct repair may be better source data or a clarification path. Keep a failure ledger organized by missing source, invalid join, stale revision, and unsupported inference. That makes a complex RAG error actionable without treating every failure as a need for more context. Add a valid one-source control containing the complete relationship. The evaluator should not demand multiple citations when one applicable source genuinely establishes every required premise and the task imposes no independent-source requirement.
Limits of the evidence
The compatibility chain is fictional and no multi-hop benchmark was executed. Valid joins require task-specific knowledge and cannot be inferred from citation count alone.
Common questions
Does retrieving two sources prove multi-hop success?
No. The sources must provide all required premises and be combined through a valid relationship.
What if one link is unavailable?
The expected behavior should identify the gap rather than invent the missing relationship.
Sources & further reading
Primary sources behind this briefing. A source’s findings apply to its own study conditions; publication on arXiv does not establish peer review.
- RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems arXiv / paper authors · 2024 · PreprintReviewed: abstract and benchmark task framing; no benchmark execution. Accessed October 16, 2026.
Questions or corrections? Write to Rubrex. Read our editorial approach.