GPT-6 Astra Scores 77.3% on a Browser-Agent Benchmark, Claude Opus 5 Gets 50.5%

OpenAI's GPT-6 Astra reportedly scored 77.3% on a browser-agent capability benchmark — informally referred to in early coverage as "Browser Use Benchmark v2" — measuring how well an AI agent can autonomously operate a web browser: navigating between pages, filling out forms, and completing multi-step tasks without a human clicking anything in between. Anthropic's Claude Opus 5 scored 50.5% on the same task suite. That's a roughly 27-percentage-point gap — about a 53% relative improvement — reported around September 6, 2026, days after Astra's own launch.

We don't have a stable, citable URL for this specific benchmark, and explainx.ai has not reproduced the run itself — so what follows treats the number as a reported figure worth taking seriously, not a settled fact. What's more useful than repeating the headline is explaining what this class of benchmark actually tests, why the specific comparison model matters here, and what a builder deciding between these two models should actually do this week.

TL;DR

What a browser-use benchmark actually tests

A browser-use or computer-use benchmark is a meaningfully different evaluation from a coding benchmark or a pure text-reasoning eval like MMLU or GPQA. Instead of scoring a single text output against a reference answer, it drops an agent into a real or simulated browser session and scores whether a multi-step goal actually got completed end to end.

That typically breaks down into a few distinct sub-skills, each of which can fail independently:

• Navigation — finding the right page or menu path, including recovering when a link is broken, a redirect fires, or the site layout doesn't match training-time expectations.
• Form-filling — reading field labels correctly, entering the right data in the right field, and handling validation errors without giving up or looping.
• Multi-step task completion — chaining several of the above together toward one goal (book a flight, file an expense report, compare prices across tabs) where an error in step 3 can silently invalidate steps 4 through 10.
• Dynamic page state — pop-ups, loading spinners, infinite scroll, and JavaScript-rendered content that a model reading a static screenshot or DOM snapshot has to interpret correctly in the moment.

This is why explainx.ai has treated computer-use scores as their own category rather than folding them into general intelligence claims — see the OSWorld and AutomationBench breakdown in our Astra vs Fable 5.1 comparison, where Astra also leads on GUI-driving tasks (72.6% on OSWorld 2.0 Offline, 41.4% vs 31.4% on AutomationBench). A model can reason brilliantly in text and still fumble a browser task by clicking the wrong element or losing track of which tab it's in — that's a UI-grounding and long-horizon-state problem, not a pure intelligence problem, and it's exactly why labs run separate benchmarks for it.

What the 77.3% vs 50.5% gap means in practice

Read the two numbers as failure rates, not just success rates, and the practical stakes get clearer. A 50.5% success rate means the agent fails roughly half the time it attempts a task in this suite. A 77.3% success rate means it fails on about one in four attempts. That's roughly a 3x reduction in outright failure rate — not just "22.8 points better," but a categorically different reliability tier for anything that has to run unattended.

That distinction matters more as task chains get longer. If a browser agent needs to complete 10 sequential steps and each step has an independent chance of failure, the overall success rate compounds multiplicatively. A model that fails one step in four will complete a 10-step chain far less often than a model that fails one step in two — the gap between 77.3% and 50.5% per-task widens dramatically once you're chaining multiple browser actions toward one goal, which is exactly the shape of most real RPA-style or research-agent workloads: log in, navigate to a report, filter by date, export, verify the export matches expectations.

For anyone evaluating models to build a web-automation agent, an RPA replacement, or a research agent that needs to browse live sites, this is the number that should carry more weight than a general intelligence leaderboard position. Astra's own launch coverage already showed a model can trail on the aggregate Intelligence Index (61, behind Fable 5.1's 66) while leading decisively in a specific capability lane — computer use looks like another one of those lanes.

The honest caveat: which Claude model is this actually comparing against?

Here's the detail that's easy to miss in a headline gap this large: Claude Opus 5 is not Anthropic's current flagship model. Opus 5 launched on July 24, 2026 at $5/$25 per million tokens, positioned as "near Fable 5 intelligence at half the price" — a strong mid-tier option at the time, not Anthropic's top computer-use performer. Anthropic has since shipped two more flagship generations: Claude Fable 5 in August, and Claude Fable 5.1 on September 1-2, 2026 — days before this browser-use report even surfaced.

Fable 5.1 already publishes its own computer-use numbers: 77.9% partial-credit and 41.7% strict on OSWorld 2.0, per Anthropic's own announcement. Those aren't directly comparable to the 77.3%/50.5% browser-use figures above — different benchmark, different scoring methodology, different task set — but the point stands regardless of exact numeric overlap: a browser-use comparison run against Opus 5 is benchmarking Astra against a model Anthropic itself has since superseded twice over. That doesn't make the 77.3% score fake, but it does mean the "27-point gap" headline is likely larger than a same-generation Astra-vs-Fable-5.1 comparison would show. Before treating this as a settled "Astra crushes Claude at browser agents" verdict, it's worth asking: was this run against the newest available Claude model, or the one still floating around comparison charts from two months ago?

Why skepticism is the right default here

explainx.ai has direct, documented precedent for treating launch-era and post-launch benchmark claims about GPT-6 Astra specifically with caution. Within days of Astra's September 3 launch, OpenAI quietly revised Astra's published hallucination rate from 4.2% down to 2%, then reverted it back toward the original figure — without a prominent changelog entry marking the change. In the same window, a separate cybersecurity comparison score was flagged for using a reasoning tier that isn't actually available to paying customers, inflating a comparison figure beyond what a real customer could reproduce.

Neither of those incidents is evidence that the 77.3%/50.5% browser-use numbers are wrong. But they're a documented, dated reason not to treat any single launch-week benchmark table — from any lab, on any capability — as final. A benchmark score is a snapshot of one run, under one harness, at one point in time, and this specific benchmark doesn't yet have the kind of independent aggregator coverage (Artificial Analysis, ARC Prize's own leaderboard) that other Astra numbers have received. Until an independent source reproduces or contextualizes it, hold the 77.3%/50.5% gap as directionally interesting rather than a precise, load-bearing figure for a procurement decision.

What to actually test before picking a model for a browser agent

A single benchmark row — this one included — should never be the sole basis for choosing a model to build a production browser agent. Before committing:

1. Pull 15-20 real tasks from your actual target sites, not generic examples. Browser agents fail differently on a bank's MFA flow than on a public e-commerce checkout, and a general benchmark averages over site types you may never touch.
2. Test login and multi-factor flows specifically. These are disproportionately represented in real failure reports because they combine dynamic state, security friction, and time-sensitive tokens — exactly the conditions a browser-use benchmark tries to simulate but can't fully replicate for every site.
3. Measure recovery, not just success. A model that fails cleanly and reports "I could not complete step 4, here's why" is more useful in production than one that silently proceeds on bad state. Neither the 77.3% nor the 50.5% figure tells you which failure mode you're getting.
4. Check cost per completed task, not just per-token pricing. A cheaper model that needs three retries to finish a task can cost more than a pricier model that finishes in one pass — Astra vs Fable 5.1's cost-per-task comparison is a useful template for running that math yourself.
5. Re-run your test against whichever model is each lab's current flagship, not whichever one shows up first in search results — as this story's Opus-5-vs-Astra framing shows, a model comparison is only as fresh as its weakest side.
6. Watch for guardrail behavior, not just task completion. If your agent will operate with real credentials on real sites, its behavior around unexpected navigation targets matters as much as its raw completion rate — see what actually goes wrong when browser agents get open-ended navigation.

If you're building on browser-use or a similar open harness rather than a vendor's own computer-use API, the model-selection question is exactly the same — the harness just gives you more control over how navigation and retries are handled around whichever model you plug in.

Honest limitations

• We do not have a stable, verifiable URL for "Browser Use Benchmark v2" as a named, established benchmark suite, and are not linking to one to avoid citing a source we can't confirm.
• explainx.ai has not independently reproduced this benchmark run. The 77.3%/50.5% figures are reported numbers, not numbers we generated ourselves.
• The comparison model matters. As covered above, Claude Opus 5 is not Anthropic's current flagship — Claude Fable 5.1 is, and it reports different (though not directly comparable) computer-use figures.
• A single benchmark should never be the sole basis for a model choice, regardless of which lab publishes it or how large the reported gap is.
• Benchmark figures from any lab, including this one, can shift after initial reporting — see the documented Astra hallucination-rate revision above as a concrete, dated precedent from the same model, the same week.

Related on explainx.ai

• GPT-6 Astra launch: every benchmark, pricing, and the ARC-AGI harness caveat
• GPT-6 Astra vs Claude Fable 5.1: which model wins where
• OpenAI changed GPT-6 Astra's benchmark numbers after launch — twice
• Claude Opus 5 launch: near Fable 5 at half the price
• Claude Fable 5.1 and Mythos 5.1: benchmarks, pricing, safeguards
• The viral "AI agent browsed NSFW content" claim: the real browser-agent guardrails lesson
• GPT-6 Astra scores 95% on a robot control task, up from Fable 5.1's 40%
• macOS harness: browser-use's Mac agent control tool
• How to read an AI benchmark and not get fooled
• AI benchmarks: the complete guide

Benchmark figures in this post reflect reports circulating as of September 6-7, 2026. explainx.ai has not independently reproduced the referenced browser-use benchmark. Verify current model versions, scores, and pricing against each lab's own documentation before making a model-selection or procurement decision.