How to evaluate an LLM application before release

What the guidance says

Anthropic’s January 2026 guide breaks agent tests into tasks, repeated runs, scoring rules, and the test environment. It also separates what an agent says from what it actually does. For example, an agent may say it saved a record when the record never changed. Score the result as well as the conversation. This is advice from a model vendor, not proof that one method works for every system.

Evidence: Demystifying evals for AI agents [1]

Choose the decision your test must support

Write down the release question in one sentence. For example: can our support assistant explain the account policy without promising a refund it cannot approve? This gives the test a clear user, task, and risk. A vague goal such as “be more helpful” can reward a smooth answer that is wrong.

Decide how much of the workflow to test. One answer may be enough for a summary tool. A chat assistant may need a whole conversation. If an agent edits a record, check the saved record too. Save the model version, prompt, search settings, and tool permissions. A change to any of these can change the result.

Build a small test set you can review

Rubrex recommends three groups of cases: common tasks, edge cases, and known failures. For each case, write the expected behavior and why the case belongs in the test. Hold back some cases while tuning the system. Use the first test run to find unclear requirements before using scores to support a launch.

Use code to check exact rules, such as required fields or allowed record IDs. Use a scoring guide for meaning, missing details, and honest uncertainty. Ask reviewers to score a small shared set on their own. Discuss where they disagree before trusting an automated judge with more cases. Keep cases that cannot be scored in the report and explain why.

• Save the original version and every answer, including failed requests.
• Run the changed version on the same inputs and limits.
• Inspect cases that got better and cases that got worse.
• Give each failure that blocks release a named owner.

Example: an assistant that explains policy

Suppose an assistant must cite the current policy, explain exceptions, and avoid promising approval. It may meet the first two rules but still fail by promising a refund. Score each rule so the team can find the problem. Also record whether the answer passed every required rule.

Now suppose a new prompt fixes citations but makes more unsupported promises. That is a mixed result. Test a change aimed at the promises, then rerun the earlier cases to check for new failures. This is an example test plan, not a result from a Rubrex client.

Give the next engineer enough to rerun the test

Share the cases, scoring rules, system versions, and original answers. Group failures by cause and explain what is still unknown. Include steps for rerunning the test and adding a failure found after launch. Another engineer should be able to check the decision without needing the author’s slide deck. When the evidence is too weak to decide, name the missing data and the next test.

Record what each score means

Choose measures that answer the release question. For a support assistant, Rubrex recommends tracking wrong answers, unsupported claims, completed tasks, and the cost of a useful answer. State what each count covers. Ten failed answers out of 100 answers means something different from ten failed sessions out of 100 sessions. Include requests that produced no answer.

One test checks one requirement. A test suite checks several requirements and ways the system can fail. Keep your application results separate from public model benchmark scores. Write a short decision record with the old and new versions, cases tested, score definitions, and failures that block release. Name the person who owns the next test.

Common questions