Research & field notes

Research library — page 5

Read a guide, follow its sources, and check the limits before applying a finding. Search the full library from the research desk, or browse by topic.

RAG systems3 min read

Monitor RAG index freshness with inspectable probes

Use versioned sentinel documents and content probes to distinguish source age, ingestion delay, cached evidence, and stale answers.

Research briefing · Sources included

RAG systems3 min read

Version a RAG corpus for reproducible evaluation

Record corpus manifests, document revisions, transformations, and index lineage so an evaluation can reproduce the evidence actually available.

Research briefing · Sources included

RAG systems3 min read

Evaluate retrieval from tables with row and column context

Test table evidence with headers, row identity, units, footnotes, and time periods so correct cells do not produce incorrect answers.

Research briefing · Sources included

RAG systems3 min read

Test PDF retrieval across headings, footnotes, and reading order

Check PDF reading order, headings, footnotes, scanned text, and page anchors before treating a retrieved excerpt as complete evidence.

Research briefing · Sources included

RAG systems3 min read

Test spreadsheet retrieval with units, formulas, and sheet identity

Evaluate spreadsheet evidence with workbook and sheet identity, units, formula provenance, cached values, and correct row-column relationships.

Research briefing · Sources included

RAG systems3 min read

Evaluate multilingual retrieval across query and document languages

Compare query and document language pairs, analyzer behavior, named entities, and answer support without hiding weaker languages in one average.

Research briefing · Sources included

RAG systems3 min read

Test retrieval of acronyms and ambiguous entity names

Build retrieval cases for shared acronyms, aliases, product codes, and unresolved names while preserving the user’s intended entity.

Research briefing · Sources included

RAG systems3 min read

Retrieve numerical evidence with the correct unit and period

Test numerical answers with source operands, units, time periods, rounding, and explicit handling of missing or incompatible values.

Research briefing · Sources included

RAG systems3 min read

Evaluate questions that require evidence from multiple documents

Evaluate questions requiring multiple sources by labeling required evidence groups, valid joins, missing links, and unsupported conclusions.

Research briefing · Sources included

RAG systems3 min read

Test follow-up questions against the correct conversation context

Test pronouns, corrections, entity switches, and unresolved references before rewriting a follow-up into a standalone retrieval query.

Research briefing · Sources included

RAG systems3 min read

Separate RAG conversation memory from durable knowledge

Test session isolation, user corrections, durable-source authority, and memory expiration without promoting conversation claims into approved knowledge.

Research briefing · Sources included

RAG systems3 min read

Test RAG cache invalidation when source documents change

Test response and retrieved-context caches against document revisions, eligibility changes, model settings, and warm-cache update scenarios.

Research briefing · Sources included

RAG systems3 min read

Choose public RAG datasets without assuming they match your product

Compare dataset scope, evidence labels, answer format, licensing, and contamination risks before adding public examples to a RAG evaluation.

Research briefing · Sources included

RAG systems3 min read

Use BEIR as a retrieval diagnostic with a local task set

Use the original BEIR benchmark and official tooling with explicit corpus, query, relevance, cutoff, and local-task comparison records.

Research briefing · Sources included

RAG systems3 min read

Trial Ragas metrics against a small human-reviewed fixture

Test Ragas faithfulness and context-precision behavior against authored edge cases before adopting an aggregate score as a release gate.

Research briefing · Sources included

RAG systems3 min read

Trial DeepEval for RAG with inspectable case-level evidence

Evaluate documented DeepEval contextual metrics on fixed RAG fixtures, preserving expected outputs, retrieved context, judge settings, and failures.

Research briefing · Sources included

RAG systems4 min read

Validate RAG metric input contracts before choosing evaluation tooling

Validate question, answer, retrieved context, reference evidence, and source identities before comparing RAG evaluation metrics or choosing tools.

Research briefing · Sources included

RAG systems3 min read

Evaluate corrective RAG retrieval decisions against a fixed baseline

Evaluate CRAG-inspired retrieval correction with branch labels, fixed baselines, external-source controls, and complete accounting for extra search work.

Research briefing · Sources included

RAG systems3 min read

Evaluate self-reflective RAG decisions without trusting self-scores

Test Self-RAG retrieval and reflection decisions with independent answer labels, fixed controls, and explicit separation from generic self-critique loops.

Research briefing · Sources included

RAG systems3 min read

Test entity resolution in a GraphRAG index

Evaluate aliases, shared names, entity types, and source provenance before trusting a GraphRAG index to connect facts across documents.

Research briefing · Sources included