| Benchmark | o1 preview | Claude 3.5 Sonnet | Llama 3.1 405B | GPT 4o | o1 mini | Qwen2.5 72B | Claude 3 Opus | Gemini 1.5 Pro | DeepSeek V2.5 | Llama 3.1 70B | Mistral Large 2 | GPT 4o mini | Gemini 1.5 Flash | Llama 3.1 8B | GLM 4 Plus | Yi Large | DeepSeek V4.1 Flash | Qwen3.8 Flash Next | GLM 5.3 | GLM 5.3 Flash | DeepSeek V4 Pro 0813 | DeepSeek V4 Flash 0731 | MiMo V2.6 Pro | MiMo V2.6 Flash | MiMo V2.5 Pro | Kimi K3 | Claude Opus 5 | GPT 5.6 Sol | Claude Fable 5 | GPT 6 Astra | Claude Fable 5.1 | Qwen3.8 Max | Qwen3.8 27B | Qwen3.6 Plus | Qwen3.6 27B | Qwen3.6 35B A3B | Claude Opus 5.5 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Agents' Last Exam% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.8 | — | 28.5 | — | 25.7 | 25.2 | 31.6 | 27.6 | 13.2 | 27.6 | 28.6 | 26.7 | 25.7 | 34.2 | — | — | — | — | — | — | — |
| Arena Elo | 1258 | 1268 | 1225 | 1287 | 1230 | 1249 | 1240 | 1255 | — | 1180 | 1224 | 1207 | 1201 | 1140 | — | 1195 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Automation Bench v1.0.6% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 54.8 | — | 48.8 | — | 43.2 | 37.7 | 53.1 | 52.3 | 16.0 | 46.7 | 50.3 | 45.8 | 46.2 | 52.0 | — | — | — | — | — | — | — |
| BabyVision (with tools)% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 89.6 | — | — | — | — | — | — | — | — | 85.7 | 94.1 | 88.9 | — | — | — | — | — | — | — | — | — |
| BBH% | — | 93.1 | 93.0 | 83.1 | — | 86.8 | 93.7 | 89.2 | — | 81.6 | 78.9 | 75.0 | 75.0 | 62.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| Chartography (with tools)% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.9 | — | — | — | — | — | — | — | — | 68.1 | 84.0 | 79.9 | — | — | 88.4 | — | — | — | — | — | 89.0 |
| Codeforces (Rating) | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3471 | — | — | — | 3348 | 3289 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| CursorBench 4.0% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 46.6 | 41.7 | — | — | 51.8 | — | — | — | — | — | 57.8 |
| CyberGym% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 88.1 | — | 84.5 | — | 83.3 | 76.7 | 94.0 | 95.1 | 40.0 | 80.0 | — | 84.5 | — | — | — | — | — | — | — | — | — |
| DeepSWE v1.1% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 74.2 | 58.7 | 66.9 | 63.4 | 62.7 | 54.4 | 71.9 | 67.9 | 19.0 | 67.5 | 74.0 | 73.0 | 70.0 | 74.0 | — | 56.6 | 42.2 | — | 13.3 | — | — |
| ExploitBench% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 47.9 | 25.3 | 16.6 | 32.2 | 70.0 | 78.5 | 78.0 | 100.0 | 83.0 | — | — | — | — | — | — |
| ExploitGym% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 15.3 | — | 15.0 | — | 5.4 | 1.8 | 17.8 | 6.0 | 0.1 | 8.1 | 22.1 | 33.7 | 28.4 | 42.4 | 30.4 | — | — | — | — | — | — |
| FrontierCode v1.1 (Main)% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 48.0 | 47.5 | — | 53.3 | 50.3 | — | — | — | — | — | 54.4 |
| GDPVal 2.1 (AA) | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1600 | — | — | — | — | — | 1673 | — | 1107 | 1524 | 1708 | 1588 | 1595 | 1542 | 1735 | — | — | — | — | — | 1846 |
| GPQA Diamond% | 78.3 | 65.0 | 51.1 | 53.6 | 60.0 | 49.0 | 50.4 | 46.2 | 48.0 | 41.7 | 48.0 | 40.2 | 38.6 | 30.4 | — | — | 90.9 | 91.7 | 88.1 | — | 92.4 | 89.9 | — | — | — | 92.9 | 93.4 | 94.1 | — | — | — | 92.6 | 89.2 | 90.4 | 87.8 | 86.0 | — |
| GSM8K% | — | 96.4 | 96.8 | 95.8 | — | 95.8 | 95.0 | 91.7 | 91.0 | 95.1 | 93.0 | 87.0 | 86.2 | 84.5 | — | 89.0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| HLE% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 36.8 | 35.9 | — | — | — | — | — | — | — | 43.5 | 56.3 | 44.5 | — | — | — | 43.6 | 30.8 | 28.8 | 24.0 | 21.4 | — |
| HLE (with tools)% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 63.9 | — | 62.5 | 55.3 | 60.0 | 51.5 | — | — | — | 59.8 | 63.6 | — | — | 57.2 | 65.6 | 56.2 | — | 50.6 | — | — | 67.7 |
| HumanEval% | 92.4 | 92.0 | 89.0 | 90.2 | 92.4 | 86.6 | 84.9 | 84.1 | 89.0 | 80.5 | 92.0 | 87.2 | 74.3 | 72.6 | 81.0 | 80.0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| IFEval% | — | 88.0 | 88.6 | 84.0 | — | 84.0 | 85.0 | — | 84.6 | 87.5 | — | 80.5 | — | 80.0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 94.3 | — | — | — |
| JobBench% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 45.8 | 55.7 | — | — | — | — | 62.0 | 61.2 | 25.0 | 54.3 | 65.7 | 45.4 | 57.4 | — | — | 53.4 | 33.4 | — | 21.8 | — | — |
| LiveCodeBench v6% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 91.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 90.3 | 87.1 | 83.9 | 80.4 | — |
| MATH% | 94.8 | 71.1 | 73.8 | 76.6 | 90.0 | 83.1 | 60.1 | 67.7 | 72.7 | 67.9 | 68.6 | 70.2 | 54.9 | 51.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| MathArena Apex% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 65.6 | — | — | — | 65.3 | 58.6 | — | — | — | 65.6 | — | — | — | — | — | — | — | — | — | — | — |
| MMLU% | 90.8 | 88.7 | 87.3 | 88.7 | 85.2 | 86.1 | 86.8 | 85.9 | 80.6 | 83.6 | 84.0 | 82.0 | 78.9 | 69.4 | 86.0 | 84.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| MMLU-Pro% | 79.2 | 78.0 | 73.3 | 72.6 | 74.0 | 71.1 | 68.0 | 75.8 | 69.4 | 66.4 | 61.2 | 58.5 | 67.3 | 48.3 | 68.0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 88.5 | 86.2 | 85.2 | — |
| NL2Repo% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 65.4 | 48.1 | 58.0 | 56.3 | 61.5 | 54.2 | — | — | — | 58.0 | 75.3 | 56.8 | — | — | — | 55.9 | 42.3 | 37.9 | 36.2 | 29.4 | — |
| OSWorld 2.0 (partial)% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 74.0 | — | — | — | 80.7 | — | — | — | — | — | 81.8 |
| OSWorld-Verified% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 82.0 | 80.8 | — | 84.8 | 83.4 | 83.0 | 86.0 | — | — | — | 84.3 | 62.5 | 63.9 | — | — |
| ProgramBench% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 20.3 | — | 19.0 | — | 15.5 | — | 26.5 | 26.0 | 12.5 | 17.5 | 37.0 | 23.0 | 33.0 | — | — | — | — | — | — | — | — |
| SEC Bench Pro% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 62.8 | — | — | — | 56.4 | 30.9 | 66.3 | 47.5 | 17.7 | — | — | 74.3 | — | 85.4 | — | — | — | — | — | — | — |
| SWE-bench Pro% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 62.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | 67.7 | 61.7 | 56.6 | 53.5 | 49.5 | — |
| Terminal Bench 2.1% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 90.6 | — | 88.2 | 84.3 | 87.9 | 82.7 | 89.9 | 87.6 | 65.2 | 88.3 | 89.1 | 88.8 | 84.3 | 89.9 | 91.4 | 86.6 | 73.0 | — | — | — | — |
| Terminal Bench 3.0% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30.0 | — | 28.3 | — | 11.8 | 7.6 | — | — | — | 17.7 | 43.3 | 34.4 | — | — | — | — | — | — | — | — | — |
| Terminal Bench 4.0% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.2 | — | 37.9 | — | 12.4 | 7.0 | 34.9 | 28.8 | 1.5 | 12.6 | 51.8 | 39.9 | 42.4 | 57.9 | 55.8 | — | — | — | — | — | 66.4 |
| Terminal-Bench-Science 0.1% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.0 | 22.4 | — | 64.6 | 52.6 | — | — | — | — | — | 58.7 |
| Toolathlon-verified% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 76.9 | 73.6 | 49.1 | 76.5 | 80.6 | 74.9 | 77.9 | — | 77.8 | — | — | — | — | — | — |
| ZeroBench-main (with tools)% | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 49.0 | — | — | — | — | — | — | — | — | 41.0 | 52.0 | 53.0 | — | — | — | — | — | — | — | — | — |
Figures are taken from public model cards, technical reports and vendor announcements, and are approximate where sources disagree. Results from different benchmark versions or tool-use settings never share a column: Terminal Bench 2.1, 3.0 and DeepSWE v1.1 are distinct benchmarks, and HLE with tools is separate from HLE without tools. Pick up to 5 models above — type to search by name or provider. Benchmarks none of them published are left out, and the best result in each row is shown in bold.