Agentic benchmarks
Quantitative agentic performance on real-world, tool-using, multi-step tasks. Every score is sourced and citable — see the provenance notes below before comparing across rows, harness configurations, or protocol versions.
Frequently asked questions
- What is BrowseComp?
- BrowseComp (Wei et al., OpenAI, April 2025) is a 1,266-question benchmark that tests an agent's ability to persistently browse the web to locate hard-to-find, entangled facts — the answers are short and easily verified, but finding them requires real, sustained search rather than a single lookup. Scores are harness-dependent: Anthropic, OpenAI, and Google DeepMind each disclose the tool configuration used (single-agent vs. multi-agent), and only same-harness scores are directly comparable. This page also tracks OSWorld-Verified and its successor OSWorld 2.1, real computer-use tasks completed via a live Ubuntu virtual machine — see the column headers and sourcing notes below.
- Which models are currently tracked?
- 19 models: GPT-6 Astra, Kimi K3, GPT-5.5 Pro, Claude Opus 4.8, Claude Sonnet 5, Gemini 3.1 Pro, GPT-5.5, MiniMax M3, DeepSeek V4-Pro, Claude Sonnet 4.6, Solar Pro 4, Claude Fable 5.1, Claude Fable 5, Claude Haiku 4.5, Claude Opus 5.5, Claude Sonnet 5.5, Gemini 3.5 Flash, GPT-5.3-Codex, GPT-5.4 mini.
- Which model scores highest on BrowseComp?
- GPT-6 Astra, at 91.5% as of 2026-09.
- How often is the data updated?
- This page was last rebuilt October 7, 2026 from Modelglass's agentic benchmark registry. Scores are appended as new evaluations are verified — an existing result is never edited or deleted, only superseded by a newer entry.
- Where does the score come from?
- Every score below links to its source (the ↗ icon next to each result) and shows an icon for how it was obtained: 🏢 vendor-reported (the model developer's own announcement), 📊 public leaderboard (a public, independently-maintained evaluation run), 🔬 independent evaluation (a credible third party ran the benchmark), or 📄 published paper (a peer-reviewed or arXiv result). See the sourcing notes below the table for BrowseComp/OSWorld-Verified harness and protocol caveats, and the note on GAIA's retirement as this page's former anchor benchmark.
Who this is for
This page is built for teams building or evaluating autonomous agents that need to act, not just answer — persistently searching the web for hard-to-find facts, or operating a real desktop environment via mouse and keyboard. BrowseComp's own creators describe it as analogous to programming competitions for coding agents — an imperfect but useful stress test for a specific capability; OSWorld-Verified plays the same role for computer-use agents completing real, multi-step desktop tasks.
What a BrowseComp score means
BrowseComp is the percentage of 1,266 questions — each with a short, verifiable answer that is deliberately hard to find — that an agent resolves by persistently searching the web (BrowseComp paper ↗). Human trainers abandoned roughly 71% of these after two hours. The bands below are pegged to the benchmark's own reported difficulty ladder, not to natural gaps between today's models — the six currently tracked here all sit in the top band.
| Score range | What it means | Best used for |
|---|---|---|
| ≥ 75% | Resolves 3 in 4 or more of BrowseComp's buried-fact questions — many of which human trainers gave up on after two hours. Where current frontier agents cluster. | Reliable autonomous fact-finding across many independent sources. At this level the score no longer separates models well — the disclosed harness (single- vs multi-agent) and the OSWorld-Verified computer-use column do more of that work. |
| 50–75% | Roughly the level of the first dedicated deep-research agents — OpenAI's Deep Research scored 51.5% at BrowseComp's 2025 release. | Sustained multi-step search, but drops a meaningful fraction of the harder fact-chains — supervise long research tasks and spot-check conclusions. |
| 25–50% | Completes a minority of the entangled-fact searches. | A research assistant a person directs and verifies, not an autonomous researcher. |
| 10–25% | Roughly the level of a strong reasoning model with basic browsing but no dedicated research loop — OpenAI's o1 scored ~10%. | Single-hop lookups and shallow search; not multi-source investigation. |
| < 10% | Near the floor — where models without a genuine browsing loop land. GPT-4o with browsing scored ~2% at release. | No autonomous web-research capability; a person has to do the searching. |
Before you rely on a band
- Harness-dependent, heavily — single-agent and multi-agent configurations differ by several points on the same model, and only same-harness scores compare (the warning above). Multi-agent scores run systematically higher.
- A browsing benchmark, not a general one — its authors liken it to programming-competition benchmarks for coding agents: a stress test for one capability (persistent search for buried facts). It explicitly sidesteps generating long answers and resolving ambiguity.
- Live web — answers can drift as the underlying pages change, and labs re-run older models against updated contamination blocklists, which shifts previously reported scores.
- Narrow task shape — short, verifiable answers to fact-finding questions. Says nothing about acting on what's found, or the multi-step desktop work the OSWorld-Verified column covers.
- Approaching saturation at the frontier — every model currently tracked here scores 76% or above, so this number alone no longer separates them; weigh it alongside OSWorld-Verified and the disclosed harness.
| Model | |||
|---|---|---|---|
| GPT-6 Astra openai/gpt-6-astra | 91.5% 🏢 ↗ 2026-09 | — | — |
| Kimi K3 moonshot/kimi-k3 | 91.2% 🏢 ↗ 2026-07 | 84.8% 🏢 ↗ 2026-07 | — |
| GPT-5.5 Pro openai/gpt-5.5-pro | 90.1% 🏢 ↗ 2026-04 | — | — |
| Claude Opus 4.8 anthropic/claude-opus-4-8 | 88.5% 🏢 ↗ 2026-05
· multi-agent | 83.4% 🏢 ↗ 2026-05 | — |
| Claude Sonnet 5 anthropic/claude-sonnet-5 | 86.6% 🏢 ↗ 2026-06
· multi-agent | 81.2% 🏢 ↗ 2026-06 | — |
| Gemini 3.1 Pro google-deepmind/gemini-3.1-pro | 85.9% 🏢 ↗ 2026-02
· search + python + browse | — | — |
| GPT-5.5 openai/gpt-5.5 | 84.4% 🏢 ↗ 2026-04 | 78.7% 🏢 ↗ 2026-04 | — |
| MiniMax M3 minimax/m3 | 83.5% 🏢 ↗ 2026-06 | 70.1% 🏢 ↗ 2026-06 | — |
| DeepSeek V4-Pro deepseek/deepseek-v4-pro | 83.4% 🏢 ↗ 2026-04 | — | — |
| Claude Sonnet 4.6 anthropic/claude-sonnet-4-6 | 76.2% 🏢 ↗ 2026-06 | 78.5% 🏢 ↗ 2026-06 | — |
| Solar Pro 4 upstage/solar-pro-4 | 49.2% 🏢 ↗ 2026-08 | — | — |
| Claude Fable 5.1 anthropic/claude-fable-5-1 | — | — | 80.7% 🏢 ↗ 2026-09 |
| Claude Fable 5 anthropic/claude-fable-5 | — | 85.0% 🏢 ↗ 2026-06 | — |
| Claude Haiku 4.5 anthropic/claude-haiku-4-5 | — | 50.7% 🏢 ↗ 2025-10 | — |
| Claude Opus 5.5 anthropic/claude-opus-5-5 | — | — | 81.8% 🏢 ↗ 2026-09 |
| Claude Sonnet 5.5 anthropic/claude-sonnet-5-5 | — | — | 80.1% 🏢 ↗ 2026-09 |
| Gemini 3.5 Flash google-deepmind/gemini-3.5-flash | — | 78.4% 🏢 ↗ 2026-05 | — |
| GPT-5.3-Codex openai/gpt-5.3-codex | — | 64.7% 🏢 ↗ 2026-02 | — |
| GPT-5.4 mini openai/gpt-5.4-mini | — | 72.1% 🏢 ↗ 2026-03 | — |
See how benchmark data like this can power a cost-aware routing setup
See how →Why we track this
We track BrowseComp because it's a genuine differentiator between models — not a vanity metric. Used alongside the other benchmarks on this page and pricing data elsewhere in the registry, it's designed to help you find the most appropriate model for your specific use case, not just the highest-scoring one.
Sourcing and comparability
Source types: 🏢 vendor-reported scores come directly from the model developer and may use proprietary evaluation setups. 📊 leaderboard scores are from public, independently-maintained evaluation runs. 🔬 independent scores are from credible third-party evaluations. 📄 paper scores are from peer-reviewed or arXiv publications.
What is BrowseComp: BrowseComp (Wei et al., OpenAI, April 2025) tests an agent's ability to persistently browse the web to locate hard-to-find, entangled facts. Scores are harness-dependent — Anthropic, OpenAI, and Google DeepMind each disclose the tool configuration used (e.g. single-agent vs multi-agent), and only same-harness scores are directly comparable.
What is OSWorld-Verified: OSWorld-Verified (Xie et al., 2024) evaluates an agent's ability to complete real-world computer tasks — editing documents, browsing the web, managing files — by controlling a live Ubuntu virtual machine via mouse and keyboard. Protocol details (screen resolution, action-step cap, run count) materially affect scores; labs have retroactively corrected historical numbers after harness bug fixes, so only matching-protocol scores are comparable.
What is OSWorld 2.1: OSWorld 2.0 (Yuan et al., 2026) is a harder successor of 108 long-horizon computer-use tasks, graded against weighted checkpoints. Labs moved to it from OSWorld-Verified in mid-2026, so current models only have scores on 2.0. The two OSWorld columns aren't comparable with each other. OSWorld 2.0 has had several task releases, and its authors and the labs say results across releases aren't comparable, so this column records only the 2026-09-10 release (v2.1). It shows the partial score (mean checkpoint credit); the stricter pass rate (every checkpoint satisfied) is in each score's source notes. All current figures are vendor-reported: no independent board covers this release yet.
GAIA is legacy — not shown in this table
This table previously tracked GAIA (Mialon et al., 2023). It was replaced in 2026-07 as the anchor benchmark because no major lab self-reports a bare-model GAIA score for any current-gen model — a research pass found none, across the official leaderboard, every major vendor's own materials, and the paused Princeton HAL leaderboard. The registry keeps 2 historical GAIA entries (GPT-4 and GPT-4 Turbo, scored against GAIA's original 2023 paper) for the historical record, but since neither has a BrowseComp or OSWorld-Verified score, they aren't shown in this current-gen-focused table.
Data is growing: This table is sourced from Modelglass's agentic benchmark registry, a registry of agentic-task benchmark results. Scores are append-only — once recorded, a result is never edited or deleted.