SWE-BENCH-PRO
30 days · UTC
Synchronizing with global intelligence nodes...
Claude Mythos posts record SWE-bench numbers, but it’s gated; tighten your evals and fix your AI test blind spots
Anthropic’s Claude Mythos preview claims record SWE-bench results, but it isn’t publicly available and public leaderboards don’t reflect it yet. A de...
SWE-Bench Pro leaderboard: small gains at the top, big contexts, and mostly self-reported results
A new SWE-Bench Pro leaderboard shows top code models clustered around 0.55–0.58, with large contexts and self-reported scores. The updated [SWE-Benc...
Coding-agent benchmarks are wobbling—trust results only after your own cross-context checks
SWE-Bench-style coding scores are spiking, but contamination and self-reported leaderboards mean you should trust results only after your own verifica...
E2E agentic benchmarks replace SWE-bench; Gemini 3.1 favors deliberation
Agentic coding benchmarks are shifting toward end-to-end app-building tests as SWE-bench Verified is being phased out, while Google’s Gemini 3.1 Pro t...
Coding agents: smarter context and sequential planning beat model-only upgrades
Third‑party tests show Bito’s AI Architect lifted a Claude Sonnet 4.5 agent to 60.8% on SWE‑Bench Pro by adding MCP‑delivered codebase intelligence—up...
E2E coding agents: 27% pass, cheaper scaling, and safer adoption
A new end-to-end benchmark, [ProjDevBench](https://arxiv.org/html/2602.01655v1)[^1] with [code](https://github.com/zsworld6/projdevbench)[^2], reports...