OpenClaw Deleted 400K Lines of AI-Written Tests. The test-audit Skill Shows How

Update — September 25, 2026: Anthropic's claude.ai speedup uses the same measurable-goal principle. See How Anthropic made Claude.ai faster.

OpenClaw deleted around 400,000 lines of its own tests and code coverage barely moved. Maintainer Peter Steinberger posted the result on September 24, 2026, with a blunt diagnosis: "Modern models just love writing tests for every tiny change, even if they aren't useful." What made it possible is a public agent skill, test-audit, that anyone can copy.

This is the reverse of the usual AI-coding worry. The problem here is not that agents skip tests. It is that they write so many that the suite becomes a cost center, and nobody can tell which tests still protect anything.

TL;DR: the skill and the result

Why do AI agents write so many useless tests?

Two patterns show up in the thread. Coding agents add a test for every small change because tests are an easy way to appear thorough, and models are rewarded for verifying their work. And when the agent changes code, it often rewrites the tests to match, which David Cramer of Sentry pointed out defeats their purpose: "the reason for those tests in the first place wasn't to prevent regressions."

Steinberger agreed it is "the worst" and added that OpenClaw keeps tests the agent does not see until release prep, specifically to catch regressions the agent may have papered over. That is a useful design idea on its own: hold out some verification from the loop that is optimizing against it.

The cost is not abstract. To a commenter asking how adding a test can ever be bad, Steinberger answered: "If you pay for CI you will understand." Slow, flaky, redundant suites burn minutes on every pull request. Our post on how Anthropic scaled test impact analysis covers the same pressure from the infrastructure side: as agents write most of the code, CI volume grows faster than anything else.

What is inside the test-audit skill?

The skill's description says to invoke it "whenever writing, changing, reviewing, or sweeping tests." It has three operating modes.

Mode 1: Authoring gate

Before any test is added or changed, four questions must have answers:

1. What observable behavior, invariant or independent contract does it protect?
2. What credible regression makes it fail?
3. Why does existing coverage not already catch that failure?
4. Does it require a production seam (an export, flag or wrapper) that no production caller needs?

The last question is the sharpest. Agents routinely export a private function or add a flag purely so a test can reach it, which turns test-only scaffolding into permanent production surface.

Mode 2: Audit

A read-only sweep that reports evidence before editing. The skill splits discovery into parallel lanes: core and packages, plugins under extensions/, UI and apps and scripts, and cross-cutting patterns.

Mode 3: Campaign

For pruning a whole subsystem. The skill requires reading a CAMPAIGN.md first, and the handoff report lists root cause, removed categories, production simplifications, false positives retained with justification, proof run, production versus test lines of code, and named follow-ups.

What counts as a junk test?

The skill gives a rejection checklist. A test matching one of these fails the gate unless a retention rule applies.

What tests does the skill tell you to keep?

The retention bar matters as much as the deletion list, because an aggressive pruning agent will delete real guards otherwise. The skill preserves tests that independently enforce public API, plugin SDK, protocol, config, migration, storage, security, platform defaults, generated cross-language code, packages, releases or architecture contracts. It also keeps observable call ordering, tests that fail on baseline (to be investigated as a possible product bug), and source inspection where it is the cheapest independent guard.

One line is worth pinning to a wall: "Static or slow is not a deletion reason."

How do I make an agent actually finish the job?

Steinberger's practical advice from the replies:

• Give an ambitious, measurable goal. "If you just tell the agent to clean up, it will stop far too early. Give it an ambitious goal. Try 'remove 20% of the least useful tests while maintaining code coverage within 2%.'"
• Use a stronger model. Asked whether the agent hit the target or stopped early, he said "ever since ~GPT 5.6 the agent no longer stops. 5.5 still needed /goal."
• Do not worry about skill length. A commenter noted the skill is verbose compared with guidance to keep skills concise. His reply: "It doesn't matter."

The measurable goal doubles as a guardrail. Coverage staying within 2% is the check that the agent deleted redundancy rather than protection.

A copy-paste version for your own repo

Drop this into .agents/skills/test-audit/SKILL.md or adapt it to your agent's skill format, then tighten it to your codebase:

---
name: test-audit
description: Invoke whenever writing, changing, reviewing, or pruning tests.
---

Before adding or changing a test, answer:
1. What observable behavior does it protect?
2. What credible regression makes it fail?
3. Why doesn't existing coverage catch that?
4. Does it need a production seam no production caller uses?
If any answer is missing, do not add the test.

Reject: assertion-free tests, self-comparisons, expected values
computed by the code under test, mocks that implement the asserted
behavior, string greps of source, tests that break on behavior-
preserving refactors.

Keep: public API, protocol, config, migration, security, release
contracts. Slow or static is not a reason to delete.

And a campaign prompt:

Audit the tests under packages/billing. Report candidates with evidence
first, no edits. Then remove the least useful 20% while keeping line
coverage within 2%. Run the smallest owning tests after each batch.
Never edit tests while the test runner is running.

That last sentence comes from the skill: "Never edit source or tests while Vitest is running in the checkout."

What are the limits and risks?

• Coverage is a weak proxy. Staying within 2% says lines are still executed, not that behavior is still protected. Use it alongside the four-question review, not instead of it.
• Mutation testing is the stronger check. If you can run it on the touched modules, a jump in surviving mutants means you deleted a real guard.
• The skill is written for OpenClaw's layout. Paths like extensions/ and commands like scripts/run-vitest.mjs will not exist in your repo.
• Heavy pruning needs a human sign-off. The skill's own handoff report is designed for review, not blind merge.

What this means for what you build or pay

• Add the gate at write time. It is cheaper to refuse a test than to delete it later, and most of the skill's value is the four questions.
• Set a CI-cost baseline. Track minutes per pull request before and after a pruning campaign so the win is measurable.
• Hold out some tests. Keep verification the coding agent cannot see, as OpenClaw does for release prep.
• Pair it with skills discipline. See our guide to what agent skills are and how AGENTS.md-style guidance shapes what an agent does by default.

How do you measure whether a pruning campaign worked?

Deleting lines is not the goal. Track four numbers before and after: CI minutes per pull request, flaky-test rate, time to first failing signal on a real regression, and the count of production-only exports or flags that existed just for tests. If CI time drops and the regression-detection time holds steady, the campaign removed cost without removing protection. If detection slows, restore the batch and tighten the retention rule.

Related reading on explainx.ai

• What are agent skills? A complete guide
• OpenClaw 2.0 release
• Top OpenClaw claws and skills
• Agentic coding is straining CI: how Anthropic scaled test impact analysis
• Uncle Bob does not review AI code, he builds a gauntlet
• Claude Code hooks: automate actions on tool calls
• OpenClaw Foundation nonprofit

Official sources: the test-audit SKILL.md in the OpenClaw repository and Peter Steinberger's thread on X, September 24, 2026.

Figures come from Steinberger's post and the skill file as of September 25, 2026; the skill may change after publication.