Design Docs Are All You Need: SMART Turns Specs Into Regenerable ML Perf Code

ML performance modeling is hostile terrain for long-lived code. New MoE layouts, attention variants, and accelerator generations invalidate yesterday's abstractions, so frameworks accrue patch-on-patch debt until a rewrite looks cheaper than another refactor. On September 4, 2026, researchers from DeepMind, MIT, Stanford, and Google posted arXiv 2609.05364 — Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool — arguing that coding agents are now fast enough that regenerating the library from prose beats incrementally patching it.

The system is called SMART. Main holds almost no hand-authored implementation: a DAG of self-contained design docs is the source of truth; coding sub-agents regenerate Python from those docs; humans only edit the docs. That is the same markdown-as-contract instinct behind DESIGN.md and agent markdown files — pushed to the extreme where code is a build artifact.

TL;DR

What problem is SMART solving?

Performance-modeling code sits between algorithms that churn weekly and hardware that churns every generation. Traditional maintenance patches the last abstraction until the structure is fragile. The paper names a sharper failure mode for agent-written software: incremental generation debt — the gap between a codebase that has been repeatedly patched by humans or agents and the codebase you would regenerate from today's requirements if a full rewrite were cheap.

SMART's bet: once regen is ~hours and ~$100, that rewrite is cheap. Drive incremental debt toward zero on every version bump instead of forever paying interest on patches.

That maps directly onto loop engineering: stop being the thing that hand-merges agent diffs into a rotting tree; put the durable intent somewhere else and let the loop rebuild.

How the regeneration workflow works

Figure 1 in the paper (summarized on alphaXiv) is an agent harness, not a single chat:

1. Dependency discovery — read-only agents scan the design-doc collection and infer which docs depend on which (e.g. hardware topology before a scheduler that consumes it).
2. Topological orchestration — an orchestrator walks the resulting DAG and assigns a coding sub-agent per doc.
3. Bounded context — each sub-agent sees only its doc plus interfaces from dependencies, fighting context-window myopia (locally plausible, globally inconsistent code).
4. Targeted feedback — failures point at the ambiguous doc, not a random stack of patches. Humans fix prose; another regen follows.

Humans never "fix the generated library in place" as the durable act. If the code is wrong, the design doc was underspecified — the same discipline spec-driven UI asks of DESIGN.md, applied to systems software.

Worked examples are the real prompt engineering

Abstract rules leave too much room for agents to invent shapes and costs. SMART's docs are opinionated around step-by-step worked examples: concrete tensor shapes, intermediate values, closed-form cost expressions, and tables that act as reconciliation anchors. Those anchors become unit tests for the regenerated code — in-context demonstrations in the Brown et al. few-shot sense, but checked into the repo as the product.

For builders writing DESIGN.md or architecture markdown today: a token list without examples is under-specified. A table that says "given shape X across Y devices, communication cost is Z" is what agents can regenerate against.

The Op IR: SymPy costs, fast vs slow roll-up

Under the docs sits a stable intermediate representation so the algorithm composition can churn without rewriting chip pricing:

The IR is the "API surface" between regenerable code and durable intent — analogous to how a good skill freezes a contract while the implementation can be rewritten.

Correctness and cost numbers that matter

Round-off parity on a frontier serving model is the paper's proof that design-doc-as-source is not just a blog-post workflow — it holds for serious ML systems co-design tooling.

What builders should change about DESIGN.md and harnesses

You probably will not regenerate an entire production monolith tomorrow. You should steal SMART's invariants for agent-heavy repos:

1. Make markdown the durable edit surface

Keep CLAUDE.md / AGENTS.md / DESIGN.md / ARCHITECTURE.md as the things humans change under review. Treat large agent-written trees as rebuildable when the domain churns (perf models, generated SDKs, schema binders, UI from DESIGN.md).

2. Prefer worked examples over vibes

Every critical doc should include at least one numeric or structural worked example that can become a test. Vague "be careful with sharding" prose is how regeneration fails.

3. DAG the docs; bound the sub-agents

Do not paste the whole monorepo into one context. Discover dependencies, topologically schedule subagents, and pass only local interfaces — the harness pattern explainx.ai covers in agent harness guides and loop engineering.

4. Price a clean regen vs another patch

If a full rebuild is hours and tens-to-hundreds of dollars, stop treating forever-forward patches as the default. That is the economic argument behind SMART — and behind why skills that encode recipes beat one-off prompts that accumulate debt.

5. Fix the doc when the agent is wrong

Harness feedback should name the underspecified design node, not encourage drive-by edits to generated sources that then diverge from the docs.

Practical starter checklist

1. Split high-churn modules into a small doc DAG (docs/perf/*.md or similar) with explicit depends_on links.
2. Add one worked example + expected cost/shape per leaf doc.
3. Wire a harness that regenerates one package at a time (orchestrator + sub-agents), with tests generated from those examples.
4. Keep UI intent in DESIGN.md templates; keep systems intent in architecture/perf docs — same source of truth idea, different schema.
5. Cap spend and iterations the way loop engineering caps dollar budgets.

Limitations (read the paper, don't cargo-cult)

• SMART targets high-churn, formally checkable modeling code — not every CRUD app.
• Regeneration still needs a stable IR and strong examples; vague docs fail loudly (a feature, not free lunch).
• ~$100 and hours assume capable frontier coding models and a mature orchestrator; your harness quality dominates.
• The paper proposes generalizability (compilers, scientific tools, cloud infra) as outlook, not a shipped product for those domains.

Related on explainx.ai

• DESIGN.md templates for AI agents
• Google Labs DESIGN.md open spec
• Vercel design.md: spec-driven UI
• Agent markdown files complete guide
• Loop engineering for coding agents
• What is an agent harness?
• What are agent skills?
• How to build your first agent skill

Primary sources: arXiv:2609.05364 · alphaXiv abstract

Paper claims (doc counts, regen time/cost, DeepSeek-V3 round-off match, author list) reflect arXiv 2609.05364v1 and alphaXiv coverage as of September 11, 2026; verify against the PDF before citing in production decisions.