The five AI agent benchmarks worth reading in October 2026 are SWE-bench Pro (coding), Terminal-Bench (real work in a shell), OSWorld 2.0 (using a computer), tau-bench (customer conversations, and reliability across retries) and BFCL (tool calling). Each one tests a different skill.

Three famous ones are near or past their ceiling: SWE-bench Verified, GAIA and WebArena. I read every score below from each official board on 9 October 2026.

Why care if you never write code? Agent builders use these boards to choose models. So the scores shape the agents you end up using.

Abstract grey and orange isometric blocks joining into one connected structure
Each benchmark tests one slice of an agent's job. None of them covers the whole thing.

The scoreboard on 9 October 2026

Scores come from each official board, read on 9 October 2026. Each board uses its own metric, so never compare across rows.

BenchmarkWhat it testsTop score todayHuman baselineStatus
SWE-bench Pro (public)Fixing real repo issuesMuse Spark 1.1, 61.50% ±3.10 (marked NEW)Not statedActive
Terminal-Bench 4.0Hard jobs in a terminalOpus 5.5 (max), 64.8% ±3.1 (22 Sep 2026)Not statedActive
OSWorld 2.0Long workflows on a computerClaude Opus 5 (max, v2026.08.08), 31.4%Not stated (humans need about 1.6 hours per task)Active
tau-bench bankingSupport chats over about 700 documentsQwen 3.8 Max, 55.2% pass^1 (4 Aug 2026)Not statedActive
BFCL V4Tool calls and their argumentsClaude-Opus-4-5-20251101 (FC), 77.47%Not statedBoard last updated April 2026
SWE-bench Verified500 filtered Python issuesClaude 4.5 Opus (high), 76.80% (17 Feb 2026)Not statedRetired by OpenAI
GAIAMulti-step assistant questionsOps-Agentic-Search-2.0, 93.69% (22 Sep 2026)92% (2023 paper)Past human baseline
WebArenaTasks on working websitesDeepSeek V3.2, 74.3% (Feb 2026)78.24% (2023 paper)Close to human baseline

1. SWE-bench Pro: can it fix real code?

SWE-bench Pro is the coding board to quote now. OpenAI recommends it in place of SWE-bench Verified, which it stopped reporting in February 2026 as "increasingly contaminated" (details below).

Scale AI built Pro from 1,865 tasks across 41 professional repositories. The public set has 731 instances from repos under strong copyleft licenses such as GPL, a legal deterrent against training on them. A private set of 276 comes from 18 proprietary startup codebases. Reference fixes average 107.4 lines of code across 4.1 files.

Scoring is strict. A task counts only if the patch makes the new fail-to-pass tests pass and every pass-to-pass test still passes.

The public leaderboard, as of 9 October 2026:

  1. Muse Spark 1.1: 61.50% (±3.10), marked NEW
  2. gpt-5.4 (xHigh): 59.10%
  3. Muse Spark: 55.00%
  4. claude-opus-4-6 (thinking): 51.90%
  5. gemini-3.1-pro (thinking): 46.10%

One oddity I noticed: the page's intro still says "top models score around 23% on the SWE-Bench Pro public set", while its own table tops out at 61.50%. The GPT-6 and Claude Opus 5 families seen on other boards are not on it yet.

If a vendor still quotes Verified in late 2026, ask for its Pro number. What OpenAI Codex Means for Agent Builders goes further on coding agents.

2. Terminal-Bench: can it do real work in a shell?

Terminal-Bench, from Stanford, Harbor and the Laude Institute, drops the agent into a real terminal with a job to finish. It ships fast: 1.0 on 19 May 2025, 2.0 on 7 November 2025, 2.1 on 6 May 2026, 3.0 on 30 July 2026 and 4.0 on 28 August 2026.

Version 3.0 brought 74 tasks across 7 domains, built on real artifacts such as database snapshots and CAD files. The best models scored about 34% then (GPT-5.6 Sol in Codex 34.4%, Fable 5 in Claude Code 33.8%). Version 4.0 removed 8 saturated tasks (solved 5 out of 5 times), fixed 19 and set a flat 8-hour agent timeout.

RankModelAgentResolution rateDateCost
1Opus 5.5 (max)Claude Code64.8% ±3.122 Sep 2026$4.7k
2Sonnet 5.5 (max)Claude Code61.8% ±2.928 Sep 2026$7.3k
3GPT-6 Astra (max)Codex58.2% ±2.83 Sep 2026$3.3k
3GPT-6.1 Sol (max)Codex58.2% ±3.129 Sep 2026$0.6k
5Fable 5.1 (max)Claude Code57.9% ±3.81 Sep 2026$6.2k
10Grok 4.7 (xhigh)Grok Build37.6% ±3.521 Sep 2026$3.7k

Read the cost column first. GPT-6.1 Sol (max) ties for third at about one eighth of the leader's listed cost, on 1.5B tokens against 8.0B.

Every score is a model plus the agent software wrapped around it, which the board lists in its own column (see our guide to open-source AI agent frameworks). Where does Grok land? Grok 4.7 (xhigh) in Grok Build sits 10th at 37.6% (on tau-bench's banking domain below, Grok 4.5 scores 47.9% pass^1, 0.8 points behind Claude Opus 5). What Grok costs per plan and per API token is in my Grok pricing explainer.

3. OSWorld 2.0: can it use a computer for an afternoon?

OSWorld, from the XLANG Lab at the University of Hong Kong, makes the agent operate a full computer. Version 1.0 (2024) had 369 tasks: humans completed 72.36%, the best model 12.24% at launch. It became OSWorld-Verified on 28 July 2025, then OSWorld 2.0 arrived on 26 June 2026.

The new version has 108 long-horizon workflows across 7 professional domains, and humans need a median of about 1.6 hours each (69.6% of tasks take a skilled user more than an hour). Claude Opus 4.7 at max thinking averaged 318 tool calls per task, against about 30 in 1.0. One task: file an expense claim through a legacy portal using receipts, Gmail and a bank site, then react to a new email mid-task.

Results, binary completion at 500 steps (partial score in brackets):

  1. Claude Opus 5 (max, v2026.08.08): 31.4% (68.3%)
  2. Claude Opus 5 (xhigh): 30.2%
  3. GPT-5.6 Sol (max): 27.3% (62.7%)
  4. Claude Opus 4.8 (max, v2026.06.24): 20.6% (54.8%)
  5. GPT-5.5 (xhigh): 13.9%

The leader earns a 68.3% partial score but finishes 31.4% of workflows outright. GPT-5.6 Sol (max) used 36.1K output tokens per task against 103K for Claude Opus 5 (max), about a third, for 27.3% against 31.4%.

The paper's diagnosis is blunt: agents "lose track of constraints, miss information that arrives mid-task, guess rather than ask the user, and skip verification". I'd pin that sentence above any agent design doc. Computer-use agents are one branch of the family in types of AI agents explained.

4. tau-bench: can it serve a customer, every time?

Sierra built τ-bench (tau-bench) around a chat: the agent talks to a simulated customer, calls tools and follows a policy. It is graded on the final database state. The June 2024 paper introduced pass^k to measure "the reliability of agent behavior over multiple trials". Back then gpt-4o succeeded on under 50% of tasks, and its pass^8 in retail was under 25%.

pass^k, worked through

pass^k is the chance the agent succeeds on the same task in all k tries. The top retail entry, Qwen3.5-397B-A17B (2 March 2026), scores 84.4% at pass^1. Require success in all four tries and it drops to 59.6%. Same model. Same tasks.

What tau2 and tau3 added

tau2-bench (June 2025) added telecom and "dual control", where the user also acts. tau3 (March 2026) added banking, with a knowledge base of about 700 documents, plus a voice mode. The older domains sit near the top: best pass^1 is 84.0% in airline (Claude Opus 4.5, 26 February 2026) and 97.8% in telecom (Qwen3.5-397B-A17B, 2 March 2026).

Banking results, Sierra-run submissions dated 4 August 2026, four trials each:

Modelpass^1pass^4
Qwen 3.8 Max55.2%35.1%
Claude Opus 548.7%32.0%
Grok 4.547.9%32.0%
GPT-5.6 Sol46.9%27.8%
GPT-5.5 (5 May 2026)44.6%29.9%

Grok 4.5 ties Claude Opus 5 on pass^4. Choosing a support agent? Read this board next to our list of the best AI agents for customer support.

5. BFCL: does it call the right tool with the right arguments?

The Berkeley Function Calling Leaderboard checks whether a model picks the right function and fills its arguments. Categories cover agentic tasks (web search, memory), multi-turn, single-turn, hallucination (relevance and irrelevance) and format sensitivity.

Top overall accuracy: Claude-Opus-4-5-20251101 (FC) 77.47%, Claude-Sonnet-4-5 (FC) 73.24%, Gemini-3-Pro-Preview (Prompt) 72.51%, GLM-4.6 (FC thinking) 72.38%. The cost column lists $298.47 for Gemini-3-Pro-Preview and $4.64 for GLM-4.6, for nearly the same score.

The catch is age. The board says "Last Updated: 2026-04-12", so nothing newer is on it. The basics of tool calls are in AI agent tool use explained.

Three benchmarks you can stop quoting

These three still get quoted. Each is now retired, past its human baseline or close to it.

Earlier versions of this post listed AgentBench and ToolBench, both 2023 research benchmarks. I replaced them with boards that had new entries in 2026. One more frozen chart: METR's time-horizon page, which tracks the task length (in human expert time) at which an agent is predicted to succeed half the time, says it "is no longer actively updated". Its last update was 8 September 2026.

SWE-bench Verified

The original SWE-bench (2023) drew 2,294 problems from real GitHub issues in 12 popular Python repositories, and Claude 2 solved 1.96%. OpenAI published Verified in August 2024 after expert review of 1,699 problems, three experts each.

On 23 February 2026, OpenAI wrote that "SWE-bench Verified is increasingly contaminated". State of the art had moved "from 74.9% to 80.9% in the last 6 months". An audit of a 27.6% subset that models often failed found at least 59.4% had flawed tests that reject functionally correct fixes. Frontier models could also reproduce the original human-written fix, or verbatim problem details, for certain tasks.

The swebench.com board (bash-only setting, mini-SWE-agent) still shows Claude 4.5 Opus (high) on top at 76.80% (17 February 2026). Nothing on it is newer than February 2026.

GAIA

GAIA (Mialon et al., 2023) has 466 questions at three levels: "human respondents obtain 92% vs. 15% for GPT-4 equipped with plugins". The test-set leaderboard is now led by Ops-Agentic-Search-2.0 at 93.69% (22 September 2026), ahead of CustomGPT.ai Research Lab v44 at 93.36% and Co-Sight Pro v1.0.1 (ZTE-AICloud) at 93.02%.

All three beat the human figure. Most leaders are multi-model systems, and only Level 3, at 85.71%, has clear room left.

WebArena

WebArena (Zhou et al., 2023) runs working websites in four domains: e-commerce, a social forum, collaborative software development and content management. The best GPT-4 agent scored 14.41%; humans scored 78.24%. The leaderboard sheet now tops out at 74.3% (DeepSeek V3.2, WebTactix, February 2026), ahead of OpAgent at 71.6%, ColorBrowserAgent at 71.2% and OpenAI Operator at 58.1% (January 2025). Several rows are marked "Self-reported".

How to read any agent leaderboard: five checks

  1. Model plus agent setup. The 64.8% belongs to Opus 5.5 (max) inside Claude Code.
  2. Version and date. The best Terminal-Bench 3.0 score at launch was 34.4%. The 4.0 leader shows 64.8% on a revised task set, so check which version a number comes from, and when.
  3. Error bars. GPT-6 Astra (max) and GPT-6.1 Sol (max) both show 58.2%, and Fable 5.1 (max) shows 57.9% ±3.8. With margins of about ±3 points, ranks 3 to 5 can be a tie.
  4. Retries. The same tau-bench retail entry goes from 84.4% at pass^1 to 59.6% at pass^4. If a board reports single tries only, expect the repeat number to be lower.
  5. Cost and tokens. Princeton's "AI Agents That Matter" warns of "a narrow focus on accuracy without attention to other metrics" and finds "SOTA agents are needlessly complex and costly". It proposes "jointly optimizing" cost and accuracy. On Terminal-Bench 4.0, GPT-6.1 Sol (max) gives up 6.6 points to the leader at $0.6k against $4.7k.

What to measure in production is a separate list: see AI agent evaluation metrics.

What no benchmark tells you

No board has your inbox, your CRM or your refund policy. The Princeton paper also found "many agent benchmarks have inadequate holdout sets, and sometimes none at all", so agents "take shortcuts and overfit to the benchmark". Test on your own task with your own data; our step-by-step evaluation framework shows how.

Benchmarks also assume the API answers. Our AI API reliability report shows that in the 90 days to 9 October 2026, the Claude API posted 56 status-page incidents (22 major or critical). The OpenAI API posted 21 (4 major or critical), with 226 hours of an open incident on at least one component.

Safety is missing too. Our AI agent incidents database logged 63 incidents from 15 February 2024 to 6 October 2026. Coding agents are the largest group (31, or 49%), and 32% (20 of 63) involved prompt injection. The patterns are in AI agent failure modes and AI agent failures: lessons from 2026.

Before you commit, Can AI do this task? shows how much of a task an agent can take on today and what your time on it costs. The token counter compares cost per call across models, with prices from our daily LLM API price history.

Where Gravity fits

Builders on Gravity pick the model and tools for each agent they publish; boards like these are one input. As a user, you describe a recurring task in plain words and pick an agent that is already built, so what you judge is whether it finishes your task, week after week. Gravity is in public beta: Free is $0 with no card, and Mini Pilot is $15 a month. Explore agents.

Frequently asked questions

What are the most important AI agent benchmarks in 2026?

As of 9 October 2026, the five worth reading are SWE-bench Pro (coding), Terminal-Bench 4.0 (terminal work), OSWorld 2.0 (computer use), tau-bench (customer conversations) and BFCL (tool calling). SWE-bench Verified, GAIA and WebArena are still quoted, but each is retired, past its human baseline or close to it.

Which AI model scores highest on agent benchmarks right now?

It depends on the board. As of 9 October 2026, Opus 5.5 (max) in Claude Code leads Terminal-Bench 4.0 at 64.8% (dated 22 September 2026), Claude Opus 5 (max, v2026.08.08) leads OSWorld 2.0 at 31.4%, and Qwen 3.8 Max leads tau-bench banking at 55.2% pass^1 (run 4 August 2026). No single model tops all five boards.

Why did OpenAI stop using SWE-bench Verified?

On 23 February 2026, OpenAI said SWE-bench Verified is increasingly contaminated. Its audit of a 27.6% subset that models often failed found at least 59.4% of those problems had flawed tests that reject correct fixes, and frontier models could reproduce original fixes or problem details for certain tasks. OpenAI now recommends SWE-bench Pro instead.

What does pass^k mean in tau-bench?

pass^k is the chance an agent succeeds on the same task in all k tries, so it measures consistency across repeated runs. On tau-bench retail, the top entry, Qwen3.5-397B-A17B, scores 84.4% on a single try (pass^1) but 59.6% when it must succeed in all four tries (pass^4).

Is GAIA still a useful benchmark?

It is less useful than it was. In 2023, humans scored 92% against 15% for GPT-4 with plugins; as of 9 October 2026, the top system, Ops-Agentic-Search-2.0, averages 93.69%. Only Level 3 (85.71% for the leader) has clear room left, and most top entries are multi-model systems.

Do AI agent benchmark scores predict real-world performance?

They predict it only partly. A board shows what a model does inside one agent setup, at one budget, on one date, and it never sees your data or your edge cases. Run the agent on your own recurring task several times and count how many runs it finishes; AI agent success metrics explains what to track.

Sources