DeepSeek V4.1 Flash 1017 Scores 57% on the Artificial Analysis Cyber Index: What the Number Measures

Artificial Analysis says an updated open-weight DeepSeek model is now the strongest publicly available model on its new defensive-security benchmark. According to the firm's X post on October 23, 2026, DeepSeek V4.1 Flash 1017 scores 57% on the Artificial Analysis Cyber Index, the highest of any model with a public API and second overall to OpenAI's restricted-access GPT-6 Sol (Daybreak Blue, max) at 69%. It reportedly costs $0.42 per task against $1.77 for GPT-6 Sol.

We checked the live Cyber Index page, which says DeepSeek V4.1 Flash 1017 (Max) scores the highest with 57, followed by Grok 4.7 (Xhigh) and MiMo-V2.6-Pro at 56 each. We could not read the chart values behind it, so the 69% GPT-6 Sol score, the per-task costs and the per-evaluation breakdown below are as reported by secondary summaries and are not confirmed by us. We also verified the method: Artificial Analysis publishes a Cyber Index methodology page, and this post explains what the 57% does and does not tell you.

TL;DR: the claim and what backs it

What the Cyber Index tests

The methodology page describes the index as testing "agentic cyber defense work: discovering vulnerabilities in a codebase, reproducing and validating them, and patching them without breaking existing functionality." It is an equally weighted average of three evaluations, each run on Artificial Analysis' open-source agent harness, Stirrup:

Two details deserve emphasis. First, the index measures defense only: every task starts with source access, and no task asks a model to build a working exploit. Second, it excludes incident response, writing secure new code, targets without source such as compiled software or live servers, and exploit realization. A high score means a model is good at audit-and-patch work on source code, not that it is a good attacker or a safe one.

Artificial Analysis also says it records cases where a model or provider declines a task on safety grounds and reports them separately from the score. That is the mechanism behind a claim that surfaced alongside the result: refusals, not capability, held back some proprietary models. We did not see the per-model refusal counts, so treat that as unverified. For background on why guardrails and defenders collide, see our report on Artemis Orion-1 and guardrails that block defenders, and Anthropic's views on open-weight cyber capabilities in GLM-5.3.

Where the jump comes from

The reported 16-point improvement over the earlier V4.1 Flash, from 41% to 57%, comes mostly from CyberGym-E2E-AA, which rose from 23% to 76%. That is a very large move on one test. Posts on X cited in the summary pointed out that the other two evaluations show negligible or even negative change, which would make 1017 a model that got much better at one workflow, writing proof-of-concept inputs and patching memory-safety bugs in C/C++, rather than across the board. We cannot confirm the per-evaluation numbers, but the shape fits a targeted post-training update.

It also fits what we saw at release. Our DeepSeek V4.1 Flash 1017 launch coverage covers the Terminal-Bench gain from 31.2 to 42.3, the MIT license and the pricing. A separate tester reported odd behavior on a bug-hunt benchmark, which we cover in the claimed-versus-verified post on the cheating allegation. Those two stories are worth reading together with this score: a model that is strong at patching tasks is also one whose agent traces deserve auditing.

Cost per task is the more useful number

Raw index points flatter expensive models. The reported cost per task, $0.42 here, is what matters if you run a scanner across many repositories. At that price DeepSeek is about four times cheaper than the leader, and a cheaper rival, MiMo-V2.6-Pro, reportedly scored 56% at $0.18 per task. If those numbers hold, the practical frontier for defensive code work is open-weight and inexpensive, while the top score sits behind restricted access.

That restricted access is itself the story. GPT-6 Sol's Daybreak Blue configuration is gated for defenders, in line with OpenAI's staged approach to cyber capability described in the pause over cyber-critical risk. An open-weight model near the top reopens the debate about who gets strong defensive tooling, and who else can use the same weights for other purposes.

How to read a number like 57%

Percentages on agentic benchmarks hide what the agent actually did. On CWE-Bench-AA a pass means two things at once: the exploit no longer works and legitimate behavior still works, checked by a deterministic verifier inside the same sandbox. On CyberGym-E2E-AA a pass needs three stages: a crash from the proof-of-concept input, a patch that removes the crash, and the project's functionality tests still passing. Those are strict, and they are the reason scores on the full defensive loop are lower than scores on finding bugs alone. DeepsecBench-AA is different: it grades findings against an expert golden set using F2, which weights recall over precision, so a noisy model that flags many real bugs can score well while costing triage time.

Averaging these three into a single index is convenient but blends very different skills. A team that mainly needs memory-safety patching should look at the CyberGym-E2E-AA column; a team doing web application review should look at DeepsecBench-AA. If the reported pattern is right, DeepSeek's strength is concentrated in the first, which is useful for C and C++ maintainers and less so for a Node.js shop.

What this means for open-source maintainers and security teams

Maintainers receive a growing flow of AI-generated reports and patches, as seen in OpenAI's Patch the Planet work. A cheap open-weight model that can reproduce a bug and propose a tested patch lowers the cost of triage, but it also lowers the cost of flooding maintainers with plausible but wrong reports. The sensible workflow is to require a reproducer and passing tests with every machine-generated submission, which is exactly what the CyberGym-style scoring rewards.

Caveats before you rely on it

1. Vendor-independent, but single-source. Artificial Analysis runs the evaluations; the results are not DeepSeek's own. But the only source is Artificial Analysis itself, and we confirmed just the headline score.
2. One run per task. CWE-Bench-AA and CyberGym-E2E-AA use pass@1 with a single repeat, so scores carry sampling noise. Close gaps, such as 56 versus 57, are not meaningful.
3. Judge dependence. DeepsecBench-AA relies on an LLM judge; the methodology names a GPT-5.6 Sol (high) judge, which is a choice that could shape scores.
4. Harness effects. Results use Artificial Analysis' Stirrup harness, and the firm notes its numbers are not directly comparable to the original benchmark leaderboards.
5. Procurement risk. US civilian agencies were ordered to remove DeepSeek products on October 19; see our DHS procurement ban report. Strong scores do not change compliance rules.
6. Offense is not measured. Nothing here shows the model cannot be used to attack. Defensive strength and offensive misuse come from the same skills.

How to check and use it

• Open the Cyber Index methodology and the model page on Artificial Analysis to confirm the current table, refusal counts and cost per task.
• Run your own repositories through a small pilot rather than trusting a leaderboard: pick 10 known historical bugs and measure find, reproduce and patch rates, plus whether the patch passes your tests.
• Log agent traces and diff the files an agent touches, since patch-quality scores do not reveal side effects.
• If you run agents with write access to code, a runtime guard helps; AgentBeam is the agent security platform from the explainx.ai team that stops AI agents before they take dangerous actions.

Why it matters

Three things stand out. A cheap open-weight model is competitive on defensive security work with systems gated for safety reasons. A benchmark that records refusals separately exposes how much "capability" depends on what a model will agree to do. And gains concentrated in one sub-test remind us that a single index number hides a lot. The right reading is: promising, cheap, worth a pilot, and not yet a reason to retire your scanner.

Figures are as reported on October 23, 2026 and may change when Artificial Analysis updates its leaderboard.

Related reading

• DeepSeek V4.1 Flash 1017: open weights, MIT license, benchmarks and pricing
• DeepSeek V4.1 Flash 1017 cheating claim, claimed vs verified
• DHS orders DeepSeek removed from federal civilian agencies
• Artemis Orion-1 and guardrails that block defenders
• Anthropic on GLM-5.3 cyber capabilities
• OpenAI pauses frontier RL over cyber-critical risk