Vals AI has published a new cyber benchmark, and the main result is simple: a restricted frontier model leads, but a model anyone can download is close behind. On the CyberOffense leaderboard, OpenAI's gated GPT-6 Sol Daybreak gave the best result on two simulated company networks. The open-weight DeepSeek V4.1 Flash 1017 followed for about a tenth of the cost per run.
This post covers what Vals AI measured, the scores in full, where the headline and the table differ, and what the result does and does not prove. The leaderboard page is marked as updated October 23, 2026. Press coverage began spreading the result on November 6.
Update (November 7, 2026): UK AISI timeline puts the Vals result in context
On November 7, outlets including The Decoder recapped the same Vals AI leaderboard without new scores. Their framing is still useful because it ties today's gap to a longer government measurement.
In a July 17, 2026 post, the UK's AI Security Institute (AISI) said leading open-weight models trailed the closed cyber frontier by four to seven months, down from six to ten months across much of 2025. On narrow tasks, AISI compared GLM-5.2 (June 2026) to Opus 4.6 (February 2026) and DeepSeek V4-Pro to Opus 4.5 (November 2025). On longer-horizon cyber ranges, AISI said GLM-5.2 matched Opus 4.5 instead.
AISI's cost illustration matches the economic story in Vals AI's table. For a 100M-token range run, it estimated roughly $1.19 on DeepSeek V4-Pro versus about $85 on Opus 4.5 or 4.6, with GLM-5.2 at about $46. AISI notes the open-model costs are estimates because it did not use first-party providers. Vals AI's own under $3 versus about $22 Daybreak comparison is a different harness and range, but the direction is the same: cheap tokens let open models run long.
The policy point AISI and Vals AI share is not that open models have already matched the latest gated release on every task. On AISI's 32-step range "The Last Ones," its July write-up reports GLM-5.2 reaching step 7 and Opus 4.6 reaching step 11 before stalling, so the closed model was still ahead. The warning is temporal: capabilities that lived behind monitoring and access controls show up in downloadable weights months later, and that window has shortened.
For defenders, that is why NCSC guidance on baselines and AI-enhanced defence matters alongside red-team exercises. If you already read the scores below, treat this update as the timeline layer: Vals AI shows how close the gap is on its ranges today; AISI shows the gap has been compressing for months.
TL;DR: the CyberOffense result in one table
| Question | Answer (source: Vals AI) |
|---|---|
| What was tested? | Seven AI models, attacking two simulated company networks alone |
| Who led? | GPT-6 Sol Daybreak, offered only through OpenAI's restricted Daybreak cyber program |
| Who was close? | DeepSeek V4.1 Flash 1017, an open-weight model |
| Cost gap | Under $3 per run for DeepSeek on the first network, about $22 for Daybreak |
| Did any model finish everything? | No. Nobody completed the final objectives on the two-network range |
| Biggest caveat | Simulated networks, three runs per model on one range, and Vals AI's own scoring |
Agents that act on your behalf need rules. See how explainx helps secure your agents →
What Vals AI actually tested
Most offensive-security benchmarks use short puzzles, such as a single capture-the-flag task. Vals AI argues that real intrusions take hours and chain many dependent steps. So it built two "cyber ranges": isolated copies of realistic enterprise networks with servers, workstations, user accounts and planted security gaps.
The two ranges are:
- Nova: a large Windows network with several connected Active Directory domains and 21 objectives. The agent starts on its own workstation with the address of a public web application. It must exploit the app, gain administrator rights, map the network, reuse passwords and move across domains.
- Lighthouse: a campaign across two separate corporate networks with 31 objectives. Information found in the first network is the only way into the second. Later stages need simulated phishing, Active Directory attacks, trust abuse between domains and data extraction.
Every model ran inside the open-source Pi agent on an isolated Kali Linux machine that could reach only the model provider and the range. Each run started from a fresh copy of the range. It ended when the agent finished, hit its token budget or reached a 48-hour limit.
Scoring avoids trusting the model. Each objective has a hidden marker, such as a file on a target machine. After a run, Vals AI searches the recorded command output for markers from a private answer key. Models are compared at equal compute: only objectives completed within 500 million cumulative tokens count.
The full scores
These are the numbers on the leaderboard page as we read it on November 6, 2026.
| Model | Nova (of 21) | Lighthouse (of 31) |
|---|---|---|
| GPT-6 Sol Daybreak | 15.7 average of 3 runs (range 12 to 19) | 27 |
| DeepSeek V4.1 Flash 1017 | 16.3 average of 3 runs (range 15 to 18) | 21 |
| Kimi K3.1 Pro | 13 | 20 |
| Meta Wonder | 7 | 28 |
| Grok 4.7 | 12 | 16 |
| Kimi K3 | 9 | 10 |
| GLM 5.3 Flash | 5 | 5 |
Vals AI says both ranges count equally in the overall score, averaging the share of objectives completed on each. Using that stated method, our own arithmetic gives about 81% for Daybreak and about 73% for DeepSeek V4.1 Flash 1017. The other models land lower, with Meta Wonder near 62% and GLM 5.3 Flash near 20%. These percentages are our calculation, not a published Vals AI column.
Where the headline and the table do not line up
Two details deserve care before you quote this result.
First, the key takeaway says DeepSeek came "close behind" Daybreak. On Nova, DeepSeek's three-run average (16.3) is actually higher than Daybreak's (15.7). The gap comes from Lighthouse, where Daybreak completed 27 objectives and DeepSeek 21.
Second, Meta Wonder completed 28 of 31 on Lighthouse, one more than Daybreak, but only 7 of 21 on Nova. It ranks lower overall because the two ranges count equally. Only Daybreak and DeepSeek have three-run averages on Nova. The other models show a single figure, so differences between them are less certain.
Why the cost gap matters
Vals AI reports that DeepSeek scored level with Daybreak on the first network for under $3 per run, against about $22 for Daybreak. Treat these as Vals AI's own figures.
The cost story has a second half. Daybreak was efficient: it reached 25 of 31 objectives on Lighthouse inside 100 million tokens. DeepSeek completed nothing in its first 25 million tokens and made most of its progress after 100 million, spending two to four times as many tokens. Because its tokens are cheap, a long run still cost under three dollars.
This is the practical lesson for anyone reading benchmark charts. A test that capped compute early would have ranked DeepSeek far below its real reach. Vals AI's data shows models kept completing objectives as they used more compute, at least for the stronger ones.
What a successful run looks like
On the harder network, Vals AI says GPT-6 Sol Daybreak broke into the first company, took administrator control, stole passwords and used them to reach a second company. It completed 27 of 31 objectives in under three hours.
That is the pattern defenders worry about: a long chain where each step depends on the last. Nova and Lighthouse reward planning, remembering what was found earlier, and recovering from dead ends. It is not a single exploit.
The deepest stages stayed hard. On the two-network range, no model completed the final objectives, which require carrying stolen access from the first network through to the last target. Only one run in the entire set reached the final stage of the single-network range.
How the weaker models failed
Vals AI notes that more compute did not help the weaker models. Grok 4.7, Kimi K3 and GLM 5.3 Flash each stalled well before their budgets ran out and never recovered. Every run in the set ended on its own, not at the token or time limit. So each score marks where a model ran out of ideas, not where the evaluators stopped it.
GLM showed the clearest dead end. On one range it spent roughly 36 hours and a thousand turns, much of it sleeping and polling for a background job that never returned, and still finished at five objectives. For builders, that is a reminder that agent loops can burn time without progress.
For more on how agents spend tokens, see our harness token study.
The open-weight problem
Vals AI's own summary states the policy point plainly: cyber capability that was limited to trusted-access AI is now available to anyone, and once weights are public, no provider can monitor or stop how the model is used.
Restricted access is the current safety strategy at several labs. OpenAI gates Daybreak. Anthropic runs a tiered cyber verification program. Both approaches assume the strongest cyber capability stays behind a login. Vals AI's result suggests that assumption is weakening, at least on this measure.
DeepSeek's model has its own history on explainx.ai. We covered its open-weights release under an MIT license, the Artificial Analysis cyber index claim, and a separate report on bug-hunt traces being deleted. The Vals AI result is a different measurement: full multi-step campaigns instead of an index score.
The defender side
Vals AI also argues this is a window for defenders. The same models that find a way into a network find the weaknesses that let them in, and open weights let defenders test without waiting for a provider's approval.
There is evidence for that already. Cloudflare described a vulnerability discovery and remediation workflow built on OpenAI's Daybreak and Codex. OpenAI has also described GPT-6 Cyber in a honeypot test, where out-of-bounds behavior stayed under one percent.
Pressure from outside is rising as well. A group of scientists has asked for air-gapping of cyber-capable AI models, and Bill Gates has warned that AI cyberattacks will surprise people next year.
Limits of the result
Treat the numbers with care.
- Simulation, not reality. The ranges are isolated and built for testing. Real networks add monitoring, patching and human responders.
- Few runs. Only two models show three-run averages, and only on Nova. Ranges like 12 to 19 for Daybreak show real run-to-run variance.
- Vendor-side scoring. Vals AI runs and scores everything. Its method is detailed, and it says it reviewed every run and found no shortcuts, but this is not an independent replication.
- One agent harness. Every model ran in the Pi agent. A different harness could change the ranking.
- Restricted model naming. The leaderboard lists "GPT-6 Sol Daybreak", the variant offered through OpenAI's Daybreak program. See our GPT-6 Sol launch coverage for the public model.
What to do with this
If you run agents or defend a network, three actions follow from the result.
- Assume long, cheap, autonomous runs. A model that makes progress only after 100 million tokens still costs a few dollars. Do not rely on an attacker giving up early.
- Test yourself first. Use a gated or open model against your own staging network. Fix chains, not single bugs: credential reuse, domain trusts and exposed web apps were the path in these ranges.
- Limit what your own agents can reach. Coding agents with broad shell access are the same kind of tool, aimed at your machine. AgentBeam is the agent security platform from the explainx.ai team that stops AI agents before they take dangerous actions. For a comparison of a cyber model and a runtime control, read our GPT-6 Cyber vs AgentBeam breakdown.
Related misuse reports are worth reading next: a cartel using Claude, Grok and Kimi against Latin American financial targets and our running tracker of agent incidents at /felony-bench.
Bottom line
Vals AI's CyberOffense shows a narrower gap between gated and open models than many policy discussions assume. The leader is still a restricted model, and the hardest stages remain unsolved. But a freely downloadable model reached most of the same objectives for about a tenth of the cost, and it improves the longer you let it run.
Check the live leaderboard and Vals AI's GPT-6 Sol model page before you quote a number, because rankings change as new models and ranges are added.
Related reading
- DeepSeek V4.1 Flash 1017: open weights, MIT license, benchmarks and pricing
- DeepSeek V4.1 Flash and the Artificial Analysis cyber index: claimed vs verified
- Cloudflare and OpenAI Daybreak: vulnerability discovery and remediation
- Anthropic cyber verification program: three tiers
- Scientists' open letter on air-gapping cyber-capable AI
- GPT-6 Cyber vs AgentBeam
- Kimi K3.1 Pro open weights, benchmarks and price
Scores and costs are taken from the Vals AI leaderboard as read on November 6, 2026, and may change as Vals AI updates it.