Science benchmarks

Sourcing and comparability

Source types: 🏢 vendor-reported scores come directly from the model developer and may use proprietary evaluation setups. 📊 leaderboard scores are from public, independently-maintained evaluation runs. 🔬 independent scores are from credible third-party evaluations. 📄 paper scores are from peer-reviewed or arXiv publications.

HLE variant warning: HLE (no tools) and HLE (with tools) are separate evaluations on the same question corpus and must not be compared directly. With-tools scores are systematically 10–20 percentage points higher than no-tools scores for the same model. GPT-4o and Claude 3.5 Sonnet entries show only no-tools scores from the original January 2025 benchmark paper — they are historical baselines, not current performance figures.

Vendor vs. leaderboard gap: Some models have both a vendor-reported score and a CAIS leaderboard score for the same benchmark. These differ because vendor announcements often use extended thinking, optimised prompting, or internal test sets, while the CAIS AI Dashboard uses standardised conditions across all models. The table shows the most recent entry per benchmark — check the source badge and click the ↗ link to see which evaluation applies. For Anthropic models the typical gap is 7–14 percentage points.

ARC-AGI-2 compute sensitivity: ARC-AGI-2 scores are far more compute-sensitive than HLE. The same model can score 2–4× higher with extended thinking tokens or a higher-compute configuration vs. the default API. The scores shown here are from the ARC Prize Foundation's "We tested every major AI reasoning system" evaluation at ARC-AGI-2's May 2025 launch, which used matched compute settings across models. Only 4 of the 12 tracked models were included in that evaluation — the remaining cells show "—". Human average on ARC-AGI-2 is approximately 66%.

Data is growing: This table is sourced from modelglass-science, a registry of science and reasoning benchmark results. Scores are append-only — once recorded, a result is never edited or deleted.