From ReAct Loop to Production Harness: DAG Planning, Tiered Memory, Budget Pressure

A single agent loop — call the model, parse a tool call, execute it, feed the result back, repeat — works fine on a demo. It falls over the moment a task has real dependency structure, a cost ceiling, or more than one thing that can go wrong at once.

Bruno Gonçalves, publishing on Data For Science's Substack, wrote the clearest 2026 walkthrough of exactly that upgrade: "Building an Advanced Agentic Harness" (July 15, 2026), which went to Hacker News at 23 points and 15 comments. It takes a "basic" ReAct-style loop and rebuilds it as eight small, composable primitives — the same shape Claude Code, Devin, and Cursor wrap around their own model calls.

We've covered what an agent harness is and the four-layer prompt/context/loop/harness stack at the concept level already. This post goes one level deeper — the specific architectural pattern for turning a working prototype into something you'd actually run in production, with the code shapes, the honest limitations, and the parts of the Hacker News thread worth taking seriously.

TL;DR — the eight primitives at a glance

The air-campaign framing: why "just the pilot" doesn't scale

Gonçalves opens with an analogy worth keeping: a naive agent loop is a lone pilot in a dogfight jet — skilled, but limited to what one person, alone, in one cockpit, can perceive and decide in real time. A real air campaign doesn't replace the pilot. It wraps the pilot in mission planners, parallel independent sorties, fuel budgets with hard bingo (return-to-base) calls, flight recorders, and after-action reviews.

None of that structure makes any single pilot better at flying. It makes the campaign faster, safer, debuggable, and measurable — properties a lone cockpit can't produce no matter how good the pilot is. The article's central claim is that Claude Code, Devin, and Cursor do the same thing to their underlying model calls, and that you can build the same wrapping yourself with primitives small enough to read in one sitting.

The central engineering question the piece answers: how do you turn a single LLM call into a reliable system that can plan, act, recover, and prove it did the right thing?

The running example: a city-comparison agent

The article's test case is a "city comparison agent" — given N cities, produce a report with population, timezone, and a narrative summary of each. It's a well-chosen toy problem because it has three properties real production tasks share:

1. Per-city lookups are independent — they decompose cleanly into parallel tool calls.
2. The final report depends on all of them finishing — a real dependency edge, not just a list of steps.
3. Results are programmatically checkable — every requested city must appear in the output, which is a free, deterministic pass/fail test.
4. Tools have wildly different costs — dictionary lookups are near-free; LLM-backed summarization and aggregation calls are not, which creates realistic budget pressure instead of a flat, uniform cost per step.

That last property matters more than it looks. A toy example with uniform-cost tools hides the exact problem multi-dimensional budgets exist to solve.

Primitive 1 — typed tools instead of string-parsed arguments

Every tool declares its arguments as a Pydantic model. One model definition drives four things at once:

• Runtime argument validation before the tool ever executes
• The JSON Schema that Anthropic- and OpenAI-style tool-use APIs expect
• Documentation — Field(..., description=...) text the planner reads when deciding what to call
• A cost_hint the budget system uses for accounting

from pydantic import BaseModel, Field

class LookupPopulationArgs(BaseModel):
    city: str = Field(..., description="City name, e.g. 'Paris'")
    cost_hint: float = 0.001  # near-free dictionary lookup

The stated design principle: "a bad plan should fail fast, at the validation layer, not deep inside a database query." Validating arguments before execution means a hallucinated or malformed tool call is caught before it can trigger a wasted, potentially side-effecting call — a much cheaper failure mode than discovering the bad input three network hops deep.

Primitive 2 — the plan is a DAG, not one action at a time

Instead of asking the model for one action per turn — the loop-engineering pattern most tutorials teach — the harness asks the Planner for the entire dependency graph up front, expressed as JSON nodes with deps lists.

A ready_nodes() scheduler function returns every node whose dependencies are already satisfied. The executor runs everything that's ready right now, concurrently, level by level, via asyncio.gather, until nothing is left or no forward progress is possible.

Because the planner is itself an LLM call, it can hallucinate structure — a deps list referencing a node ID that doesn't exist, or a circular dependency. Every plan is validated before any execution begins, not discovered mid-run.

Is a DAG actually better than a free-running loop?

This is where the Hacker News thread pushed back hardest. Commenter budududuroiu argued for giving the model a REPL loop with tools injected as callable functions instead — letting the model write code that loops or exits early, rather than being constrained to a fixed graph structure decided before execution starts.

Both sides have real evidence behind them. Academic work on this exact tradeoff — "From Agent Loops to Structured Graphs: A Scheduler-Theoretic Framework for LLM Agent Execution" — frames it precisely: free-running "propose one tool call, execute, repeat" loops are flexible but degrade as interaction history grows, with success rates plateauing well short of what production revenue-critical workflows need. Full-horizon planning (making the whole plan up front, as the article does) matches step-by-step planning accuracy on well-defined, data-centric tasks while spending fewer tokens — because the planner reasons once instead of re-reasoning at every step. LangChain's own plan-and-execute writeup reaches the same conclusion from the product side: drastically fewer LLM calls, and an inspectable, loggable, human-reviewable plan before execution starts.

The honest read: DAG-first planning wins when subtasks are genuinely independent and dependencies are known ahead of time — exactly the city-comparison shape. Code-as-plan / REPL loops win when the task's structure isn't knowable until you're partway through it. explainx.ai has covered this same tension from the product-launch side in our graphs vs. loops orchestration debate — it is a decades-old state-machine-vs-imperative-code argument resurfacing under new vocabulary, not a settled question either side has won outright.

Primitive 3 — level-synchronous parallel execution

A semaphore caps max concurrent tool calls — the article uses 5 — to avoid rate-limit storms or cost spikes from a wide plan. Synchronous tool functions run through asyncio.to_thread, so tools never need to be rewritten as async def just to fit the executor.

This is explicitly not a full dynamic scheduler with work-stealing or priority queues. The article's argument: level-synchronous parallelism captures most of the practical win for workloads where each node is a hundred-millisecond-to-seconds API call. Sequential wall time is roughly the sum of every node's latency; parallel wall time is roughly the maximum latency in the current level, plus the aggregation step at the end. For the city-comparison task, that's the difference between waiting for N cities one at a time and waiting for the slowest one.

Primitive 4 — tiered memory under a hard budget

Naive agents dump the full chat history and every tool output into every prompt. That wastes tokens and — the article is blunt about this — measurably degrades model performance, because irrelevant text dilutes attention on the actual goal. "Context should be actively assembled, not passively accumulated."

The harness splits memory into three tiers:

• Working memory — always in context: the goal, a plan summary, and the last few results
• Episodic memory — outcomes of past runs, retrieved only when a past task looks similar to the current one
• Semantic memory — background facts, retrieved the same way but not tied to any specific run

Memories are assembled by pulling the top-k most similar items to the current goal, under a hard character budget — with episodic memories prioritized over semantic ones, on the reasoning that past mistakes are usually more actionable than generic facts. Truncation is made explicit rather than silent when the budget runs out — the harness tells you it cut something, instead of quietly dropping it.

For similarity, plain Jaccard word-overlap is free but fails on paraphrase — "famous landmarks in France" and "Paris is known for the Eiffel Tower" share almost no words despite meaning nearly the same thing. Real sentence embeddings (384-dimensional, all-MiniLM-L6-v2) map paraphrases to nearby vectors instead. The harness tries embeddings first and falls back to Jaccard if the embedding model isn't available — a pragmatic degrade-gracefully choice rather than a hard dependency.

This is the piece of the article HN commenter floatrock engaged with most substantively, in a thread that started skeptical. dominotw called harnesses, skills, and memory systems "totally useless in practice" — real practitioner fatigue with framework hype that's worth naming, not dismissing. floatrock's reply reframed the generalizable point: the toy example genuinely is overengineered for its own complexity, but the reason subagent/harness structure exists isn't that large context windows can't technically fit the data — it's context protection. Tokens from a subtask that pollute the main context window degrade the primary agent's attention even when there's technically room. The actual skill is knowing when task complexity warrants that isolation versus when a single agent is enough.

Primitive 5 — two-tier verification: cheap filters before expensive judges

Deterministic structural checks run first and are essentially free — for the city agent, that's confirming every requested city actually appears in the report. Only a report that survives that tier escalates to an LLM judge for subjective quality, which costs real tokens.

A deliberately incomplete report — missing one requested city — fails at the deterministic tier with an actionable reason string, and zero LLM tokens spent on judging it. The article names this "a robust pattern behind most production eval pipelines: cheap filters first, expensive judges on survivors only" — and it separates concerns cleanly: the Worker produces, the Critic evaluates, and the generator never grades its own homework.

Primitive 6 — Planner / Worker / Critic instead of one prompt

A single do-everything prompt confuses objectives — planning constraints bleed into writing style — and is hard to test or swap in isolation. The harness splits into three roles:

• Planner — receives the goal and tool schemas, returns validated DAG JSON
• Worker — just executes the DAG
• Critic — receives the goal and the finished report, returns a verdict

The Planner's system prompt has the live tool catalog spliced directly in, so it can only ever reference tools that actually exist — no hallucinated tool names to catch downstream. All three roles go through the same LLMProvider.complete(..., role=...) interface, built on a pluggable LLMProvider base class with a deterministic MockProvider for testing. That separation matters during development: it lets you tell "is my orchestration wrong?" apart from "is the model planning badly?" — and swapping the planner's model, or mocking the critic entirely, becomes a one-line change instead of a prompt rewrite.

The article draws the same comparison worth making explicit here: AutoGPT-style planners, SWE-agent-style workers, and LLM-as-judge evaluators already operate this way in production systems today — the harness stays minimal enough to read in one sitting while mirroring how real systems are actually structured.

Primitive 7 — multi-dimensional budgets and a single pressure scalar

A single max_steps counter — the pattern in most basic harnesses, including the simple retry loop — hides real constraints. You can run out of tokens with steps left. You can be under budget but rate-limited on tool calls. A hung network call can burn real wall-clock time without incrementing any step counter at all.

BudgetMulti tracks tokens, tool calls, wall time, and estimated dollars simultaneously. A run stops when any dimension is exhausted, not just when the step counter hits zero.

Its most useful output is a single "pressure" scalar — the maximum utilization ratio across all four dimensions. You're limited by whichever resource runs out first, "exactly like real billing." Pressure drives graceful degradation directly:

This is the article's answer to the skepticism champagnepapi raised on Hacker News — that all this tooling is really an attempt to impose determinism on inherently non-deterministic LLMs, and that there are too many edge cases for that to ever be fully reliable. Budget pressure doesn't make the harness deterministic. It makes degradation deterministic — the harness fails predictably and cheaply instead of unpredictably and expensively, which is a materially different (and more achievable) goal.

Error classification: not every failure deserves a retry

A small classify_error() function maps errors into four classes, each with its own recovery policy:

• Transient (rate limits, timeouts) — exponential backoff with jitter, so a fleet of agents doesn't retry in lockstep, then retry
• Validation errors (tool misuse) — feed the structured error back to the LLM so it can self-correct its arguments
• Unknown entity (missing information) — retrying is actively harmful; a hallucinated city name fails identically every time, so the right move is to re-plan without it, not retry blindly
• Policy violations — fatal; halt immediately

Recovery policy follows from the error's class instead of blind universal retries. It's a small function, but it's the difference between a harness that retries a doomed call five times and one that recognizes the call was never going to work.

Primitive 8 — tracing that's boring on purpose

An append-only structured event log captures identity (step ID plus a parent ID linking worker steps back to the plan node that spawned them), semantics (role, action), economics (latency, tokens, cost, budget-pressure snapshot at write time), and critic verdicts where applicable.

The schema is deliberately flat and boring — dumpable to a file, shippable to OpenTelemetry or LangSmith, or plottable directly. The article's claim: "you don't need a proprietary format to get real observability, just a sufficient schema." Example plots show per-step latency by role (LLM-backed summarize/aggregate nodes dominate wall time versus flat dictionary lookups — the concrete reason parallelism matters here), budget pressure rising monotonically with a sharp jump at the aggregation step, and token usage skewing heavily toward the Worker on this particular task.

The orchestrator and re-planning loop

The full orchestrator: build context from memory → ask the Planner for a DAG → hand the DAG to the Worker, recording a trace event and charging budget per tool call → check for failures → verify with pressure-aware degradation → store the outcome in episodic memory → return a bundled result (report, verdict, DAG, trace, budget).

It adds one behavior beyond a naive retry loop: if execution fails and the error classifies as missing-information, the harness sends the failure context back to the Planner for an informed re-plan — up to a configurable max_replans — instead of blindly retrying the same doomed plan.

The gotcha: never trust the LLM about its own node names

The article shares a real bug worth repeating verbatim, because it's the kind of thing that only shows up once you swap the mock for a real model. Their mock planner always named the capstone aggregation node aggregate, by convention. An early orchestrator version looked that node up by its literal ID string. Switching to a real model broke it silently — real models often named the same node aggregate_report (matching the tool name) or invented their own ID entirely.

The fix: resolve the capstone node by which tool it calls, not by node ID, and make the planner prompt's instructions about the capstone node explicit rather than assumed. The lesson generalizes past this one bug — "never take the LLM at its word, not even about node names." Anything the model names, IDs, or labels is a hint, not a contract, until your harness validates it independently.

What the article admits it doesn't solve

To its credit, the piece doesn't oversell itself. Stated gaps, directly:

• Memory is in-process — not persisted to a real vector store like Chroma, Weaviate, or pgvector.
• Tool output is trusted as instructions in this demo. Production systems must sandbox untrusted tool output as data, to guard against prompt injection — treating a tool's return value as something to parse, not something to obey.
• Irreversible actions have no human-approval gate. Not implemented in the demo; a production harness handling anything destructive needs one.
• Token accounting estimates from character counts rather than reading real usage metadata from the model SDK — a reasonable shortcut for a teaching example, a real gap for cost-sensitive production use.
• One clean demo proves the harness CAN work, not that it reliably DOES work across cases. The article explicitly defers reliability-at-scale to a future post on building a proper eval harness.

That last point is also the strongest thread running through the Hacker News comments. shostack connected it to a real, recurring tension: hand-tuned harness and context-engineering setups tend to break every time a new model ships, and Anthropic's own guidance has at points pushed the other direction — trust the frontier model, ditch prior scaffolding investment rather than re-tuning it around a stronger model. The general HN consensus lands in a sober place: the eight primitives here — typed tools, DAG planning, tiered memory, a verification hierarchy, split roles, budget pressure, tracing — are a genuinely useful, well-reasoned checklist for going from prototype to production-shaped. They are tools and patterns, not a reliability guarantee. The unresolved question, which the article itself defers, is evaluation — proving the harness works across many cases, not just one clean demo.

Where this fits if you're building your own

If you're starting from our building-your-first-harness sequence — define success, write verification, build the simplest loop, add a hard exit, add context management, add failure handling, instrument it — this article is what steps 5 through 7 look like once you actually need them at production scale, rather than a prototype. Don't add DAG planning, tiered memory, or a Planner/Worker/Critic split on day one. Add them when a single loop with a max_steps counter has demonstrably stopped being enough — when you can point at a specific failure (context bloat, runaway cost, a re-plan that should have happened instead of a fifth retry) that one of these eight primitives directly fixes.

Related reading

• What Is an Agent Harness? The Complete Guide — the six-component harness model this post builds on
• Cloudflare OS — zero-trust agent workspace (Aug 2026) — where a production harness meets Gatekeepers and sandboxed Gadgets
• Agent Harness Engineering: When the Model Stays Fixed — the Terminal-Bench evidence that harness changes beat model upgrades
• Context vs Prompt vs Loop vs Harness Engineering: The Four-Layer Stack — where DAG planning and budget pressure sit in the full stack
• Graphs vs. Loops: The Agentic AI Orchestration Debate — the DAG-vs-imperative-loop argument from the product-launch side
• Pi: Mario Zechner's Minimal Agent Harness — the opposite end of the spectrum, a harness that ships primitives instead of a fixed pipeline
• What Is Loop Engineering? — the five-component loop model this harness sits inside
• Top 10 Closed-Source and Open-Source Agent Harnesses (2026) — how Claude Code, Devin, Cursor, and open alternatives implement these same primitives in production
• Building an Advanced Agentic Harness — Bruno Gonçalves, Data For Science — the primary source for this post
• "From Agent Loops to Structured Graphs" — arXiv:2604.11378 — the scheduler-theoretic framework behind the DAG-vs-loop tradeoff

Article details, code shapes, and the Hacker News discussion referenced above reflect the piece as published July 15, 2026 and its HN thread; verify current state at the source before citing exact figures.