AI model comparison / The decision desk
Choose a task. Inspect the evidence. Compare the tradeoffs.
A practical model shortlist starts with the work you need done.
01 / Start with the task
Highest observed means in six Epoch AI benchmark tables. These are source results, not Rubrex tests or universal winners. Coverage and evaluation settings vary.
Reimplement complete programs from their behavior and documentation.
Run started: .
Uses a large agent budget: up to seven days and 10 billion tokens per attempt. This is not a quick coding-assistant test.
Source & test conditions ↗Answer difficult multiple-choice questions in biology, chemistry, and physics.
Run started: .
Measures a specific question set. It does not establish scientific research ability or accuracy on your documents.
Source & test conditions ↗Solve expert-written mathematics problems with access to Python.
Run started: .
These are v2 results with tool access. Do not combine them with the original question set or tool-free results.
Source & test conditions ↗Answer short factual questions, scored using Epoch AI’s proportion-correct measure.
Run started: .
Epoch encourages a best guess in this setup. This is neither a hallucination rate nor a test of useful abstention or private-document RAG.
Source & test conditions ↗Track progress in the Earthborne Rangers game environment.
Run started: .
Game-agent performance is a narrow signal. It does not directly measure business workflow completion or tool authorization.
Source & test conditions ↗Resolve software issues under Epoch AI’s evaluation setup.
Run started: .
This source has limited recent-model coverage. Agent scaffolds, budgets, and evaluation subsets can differ from other SWE-bench leaderboards.
Source & test conditions ↗In this snapshot, Claude Opus 5.5 (max) has the highest observed mean on MirrorCode; GPT-6 Astra (max) leads the included GPQA Diamond runs. Use the task-specific results below to select candidates, then test them on your own examples at a fixed budget.
02 / Explore the evidence
Reimplement complete programs from their behavior and documentation. Uses a large agent budget: up to seven days and 10 billion tokens per attempt. This is not a quick coding-assistant test.
Read the original methodology ↗Showing 9 of 9 matching evaluation runs. Ordered by mean score, highest first. ± is one standard error in percentage points, where supplied.
Scroll the table horizontally on smaller screens.
| Model and setting | Mean score | Standard error | Run started |
|---|---|---|---|
| Claude Opus 5.5 (max)Anthropic | 77.4% | ± 8.3 pp | Sep 22, 2026 |
| Claude Fable 5.1 (high)Anthropic | 73.3% | ± 9.2 pp | Sep 10, 2026 |
| Claude Fable 5 (high)Anthropic | 63.9% | ± 10.4 pp | Aug 10, 2026 |
| GPT-6 Astra (high)OpenAI | 46.7% | ± 11.9 pp | Aug 30, 2026 |
| Claude Opus 4.7 (high)Anthropic | 31.1% | ± 9.0 pp | Aug 12, 2026 |
| GPT-5.6 Sol (high)OpenAI | 20.0% | ± 9.2 pp | Aug 10, 2026 |
| GPT-5.4 (high)OpenAI | 15.6% | ± 7.7 pp | Aug 10, 2026 |
| GPT-5.5 (high)OpenAI | 10.0% | ± 6.0 pp | Aug 10, 2026 |
| Gemini 3.1 Pro PreviewGoogle DeepMind | 8.9% | ± 4.6 pp | Aug 10, 2026 |
03 / Build your shortlist
Compare model families using their highest observed mean score in each test. The exact setting appears below every result. This view does not equalize reasoning effort or compute budgets.
A missing score is unknown, not zero. Higher means can have overlapping uncertainty. These results are a shortlist signal, not proof of superiority on your workload.
04 / Make the budget explicit
A dated sample from OpenRouter’s public catalog. Prices are USD per million tokens at the listed base tier. The example uses 10,000 input tokens and 2,000 total billed output tokens for one request.
Scroll the table horizontally on smaller screens.
| Model | Input / 1M | Output / 1M | Example request |
|---|---|---|---|
| OpenAI: GPT-6 Luna ↗Higher-context price tiers apply | $0.1 | $0.5 | $0.0020 |
| Anthropic: Claude Haiku 5.5 ↗Higher-context price tiers apply | $0.1 | $0.5 | $0.0020 |
| Z.ai: GLM 5.3 Flash ↗ | $0.0232 | $1.2 | $0.0026 |
| DeepSeek: DeepSeek V4.1 Flash ↗Higher-context price tiers apply | $0.15 | $0.6 | $0.0027 |
| Z.ai: GLM 5.3 ↗ | $0.079 | $4.8 | $0.0104 |
| Mistral: Mistral Large 4 ↗ | $0.68 | $2.09 | $0.0110 |
| Google: Gemini 3.8 Flash ↗ | $0.75 | $3.75 | $0.0150 |
| StepFun: Step 5 Preview ↗ | $1 | $2.7 | $0.0154 |
| Meta: Muse Spark 1.3 ↗ | $1.25 | $4.25 | $0.0210 |
| MoonshotAI: Kimi K3 ↗ | $0.73 | $10 | $0.0273 |
| Qwen: Qwen3.8 Max (0902) ↗ | $2 | $6 | $0.0320 |
| SpaceXAI: Grok 4.7 ↗Higher-context price tiers apply | $2 | $6 | $0.0320 |
| OpenAI: GPT-6 Sol ↗Higher-context price tiers apply | $2 | $10 | $0.0400 |
| Anthropic: Claude Sonnet 5 ↗ | $2 | $10 | $0.0400 |
| OpenAI: GPT-6.1 Sol ↗Higher-context price tiers apply | $2 | $10 | $0.0400 |
| OpenAI: GPT-6.1 Sol Pro ↗Higher-context price tiers apply | $2 | $10 | $0.0400 |
| Anthropic: Claude Sonnet 5.5 ↗ | $2 | $10 | $0.0400 |
| Google: Gemini 3.1 Pro Preview ↗Higher-context price tiers apply | $2 | $12 | $0.0440 |
| Anthropic: Claude Opus 5.5 ↗ | $4 | $20 | $0.0800 |
| OpenAI: GPT-6 Astra ↗Higher-context price tiers apply | $10 | $50 | $0.2000 |
| Anthropic: Claude Fable 5.1 ↗ | $10 | $50 | $0.2000 |
This is a token estimate, not cost per successful task. Extra reasoning tokens, retries, tools, search, media, routing/provider differences, and higher-context tiers can change the bill. Cache and batch discounts are excluded. Verify the linked model listing before purchase.
05 / Model guides
Explore version changes, documented capabilities, and practical evaluation plans. Guides have their own review dates, separate from the benchmark snapshot above.
Sources checked
Understand DeepSeek V4.1 Flash’s October update, API alias behavior, and reported coding results. Plan a controlled test before changing production traffic.
Read the model guide →Sources checked
Review Mistral Large 4’s open-weight release, reasoning modes, and dated API specifications. Compare hosted and self-hosted options with a practical trial.
Read the model guide →Sources checked
Understand Qwen3.8 Max’s dated snapshots, multimodal inputs, and regional API differences. Build a reviewable trial for document work and longer agent tasks.
Read the model guide →Sources checked
Check GLM-5.3’s required reasoning mode, documented input limits, and differences from Flash. Evaluate coding changes using a fixed budget and reviewer rubric.
Read the model guide →Sources checked
Compare current OpenAI GPT-6 models with dated API prices, context limits, and independent benchmark results. A practical selection guide for B2B teams.
Read the model guide →Sources checked
Review Claude Opus 5.5 with dated pricing and independent planning metrics. Learn how to test coding quality, reviewer effort, and application permissions.
Read the model guide →Sources checked
Review Meta Muse Spark 1.3 with dated math benchmarks, API pricing, context limits, audio caveats, and the difference between Standard and Contributor tiers.
Read the model guide →Sources checked
Examine Kimi K3 with independent science and math metrics, API prices, and context limits. Learn how to evaluate long tasks without confusing model and agent.
Read the model guide →Sources checked
Evaluate Gemini 3.8 Flash using dated science and math results, context limits, introductory pricing, and practical migration and multimodal checks.
Read the model guide →06 / Evidence you can trace
222 model families appear across these six tables. We import Epoch AI’s internal-run mean scores, retain exact settings, and show supplied standard errors. We convert fractions to percentages and sort results. We do not average unrelated benchmarks into a new rating.
Epoch AI data & attribution ↗Sources retrieved Oct 12, 2026. This is a saved snapshot, not a live feed. The run date in each row can be older than the download date. New models enter different tests at different times; a missing result does not mean poor performance.
Download the current source export ↗Reasoning effort, tools, scaffolds, and compute budgets affect results. A small mean-score gap is not a statistical significance test. Before switching models, evaluate task success, tail latency, failure severity, and total cost under your own operating conditions.
Read the model migration guide →These benchmark tables do not measure API speed or user preference. Artificial Analysis provides performance comparisons; Arena provides preference-based comparisons. Their tests answer different questions, so their numbers are not merged into this table.
Explore speed at Artificial Analysis ↗Benchmark data: Epoch AI, Capabilities & Benchmarking, used under CC BY 4.0. Selected, reformatted, and converted to percentages by Rubrex. Original benchmark creators and methods are linked in each category. No endorsement is implied. API price data: OpenRouter public model catalog, retrieved on the same date.
The source export can change after this snapshot. We publish no provider-sponsored ranking and make no claim that these scores establish production reliability.
Before you choose
Independent evaluation takes time, and each benchmark covers a different set of models. We leave missing results empty. We do not infer a score from another model in the same family or from a launch claim.
Only if it meets your quality bar. Compare cost per successful task, including retries and human review. A low token price can still produce an expensive workflow.
They can help build a shortlist. You still need representative documents, answerability cases, citation checks, tool boundaries, and tests of the complete application.
Build a RAG evaluation →From a shortlist to a decision
Bring one use case. We can define the quality bar, establish a baseline, and investigate the failures.
Explore the AI Reliability Sprint →