GLM-5.2 vs Claude Fable 5: Kilo Code's Planning Benchmark Shows a Near-Tie at 1/10th the Price

Why Planning Benchmarks Matter More in 2026

Standard benchmarks evaluate what a model knows. Planning benchmarks evaluate whether a model can design — whether it can take an open-ended problem, make reasonable engineering trade-offs, and produce a specification precise enough for autonomous execution.

This distinction matters because the AI coding workflow has shifted. In 2024, developers used models for autocomplete and chat. In 2026, agentic coding tools like Kilo Code, Claude Code, and OpenCode execute multi-step plans autonomously — and the quality of the initial plan determines whether the execution succeeds or degrades into correction loops.

A model that generates correct code but produces brittle plans forces the developer to act as a human spec-reviewer, catching the implicit assumptions the model did not state. Kilo Code's benchmark directly tests for this: can the model produce a plan that another model — with no human in the loop — can execute correctly on the first attempt?

The result landed within a rounding error: Fable scored 9.1, GLM-5.2 scored 9.0. Both plans converged on the same architectural decisions for the same reasons. The only difference? One model spelled out an edge case the other left implicit.

The price difference, however, is not a rounding error. GLM-5.2 lists at $1.40 per million input tokens and $4.40 per million output tokens. Fable 5 lists at $10 and $50 — roughly ten times the cost.

The Benchmark: What Kilo Code Actually Tested

Kilo Code ran a planning task — not a multiple-choice benchmark like MMLU or a code generation sweep like SWE-bench, but a genuinely hard specification problem: turning vague requirements into a spec that another model could build from without guessing.

Planning benchmarks measure a different capability than standard evaluation suites. A model that scores well on knowledge recall or code completion can still produce plans with gaps, contradictions, or implicit assumptions that derail execution. Planning tasks test whether the model can reason about trade-offs, anticipate edge cases, and produce a specification precise enough that a downstream model — which cannot ask clarifying questions — will build the right thing.

Kilo Code's rubric specifically rewarded explicitness: if a constraint mattered for correct execution, the plan had to state it. Leaving it implicit — even if a human engineer would infer it — counted as a deduction.

Both models received:

• The same prompt
• The same task description
• The same evaluation rubric

Fable's plan, which won Kilo's previous frontier round, was the baseline. GLM-5.2's plan was evaluated fresh against the identical criteria.

Where Both Models Made the Same Hard Calls

Kilo Code's team noted that both models converged on the same architectural decisions for the planning task — the same judgments on the calls that matter:

• Environment variables kept out of the rollout hash — both models recognized that including environment state in a deployment hash would cause false cache misses
• Fast SHA-256 for API keys — both chose a fast cryptographic hash over a slow password hash for API key storage, understanding the throughput requirements
• Unknown-flag lookups cached — both models identified caching for unknown-flag lookups as a performance-critical optimization

The convergence on these decisions is notable because none of them were explicitly specified in the prompt. Both models independently derived the same engineering trade-offs from the same vague requirements.

Each of these decisions represents a genuine design fork with meaningful consequences:

• Environment in rollout hash: Including process.env in a deployment hash sounds defensive — you want to catch config drift. But doing so means every environment variable change, including innocuous additions, invalidates every cached deployment. Both models correctly identified that the cost of false cache misses outweighs the benefit of catching environment drift at deploy time, and that environment-specific configuration should be handled by a separate mechanism.

• SHA-256 vs password hash for API keys: A naive plan might reach for bcrypt or argon2 for API key hashing, following database password best practices. But API keys are validated on every request — sometimes hundreds per second. A bcrypt hash at cost 10 takes ~10ms to verify. SHA-256 takes nanoseconds. Both models recognized that the correct threat model for API keys is theft of the hashes file, not brute-force recovery, and that throughput requirements rule out slow hashes entirely.

• Unknown-flag caching: Feature-flag systems that serve millions of requests need to handle lookups for flags that do not exist. Without caching, every unknown flag lookup hits the database or control plane. Both models identified that caching these negative lookups — with a short TTL — is the standard performance optimization, not an afterthought.

The One Place Fable Edged Ahead

The 0.1-point gap came down to a single difference in specification explicitness.

Fable 5 spelled out a create-time cache trap — a constraint that certain cache entries must be invalidated or avoided at creation time. GLM-5.2's plan left the same constraint implicit, assuming the builder would infer it from the cache architecture.

In a planning evaluation where the rubric rewards explicitness — because the plan is meant to be executed by another model without guessing — Fable's extra clarity earned the margin.

Both plans were structurally equivalent. Fable's was simply more didactic about a single edge case.

The Price Gap Is the Headline

Farhan summarized the result on X in a line that captured the developer reaction:

"The score gap is noise. The price gap is the whole story. 9.0 planning at a tenth of the cost changes what you can afford to run on every task instead of saving the frontier for special occasions."

This framing matters because planning tasks — especially in agentic coding workflows — are not one-off invocations. A developer running an iterative agent loop may call the model dozens or hundreds of times per session. At Fable pricing, those calls add up fast. At GLM-5.2 pricing, they become background noise.

Even at conservative token counts, the cost differential changes agent architecture decisions — making it viable to run frontier-quality planning on every loop iteration instead of reserving it for the initial plan step.

What This Means for Developers

Kilo Code was explicit that one task, one run is not proof that GLM-5.2 plans at Fable's level across the board. It is a single data point.

But it is a significant data point, for three reasons:

1. The planning gap is closing. If the best open-weight model is within 0.1 points of the frontier on a genuinely hard spec-writing task, the "open weights are good at recall, bad at reasoning" stereotype needs updating. Planning has been the last stronghold of closed frontier models — the argument that open models can generate text and code, but cannot architect systems. This benchmark suggests that advantage is narrowing.

2. Open weights change availability. Fable 5 has been subject to export control restrictions — the U.S. blocked it globally on June 12. GLM-5.2 is fully open. Open weights come with an availability story that closed models cannot currently match. Developers in regions affected by export controls, or teams that need air-gapped deployments, simply cannot rely on closed frontier models for planning tasks. See our earlier coverage: GLM-5.2 Beats Fable 5 on Reasoning — 24 Hours After the U.S. Export Ban

3. Cost changes behavior. When frontier-quality planning costs 10x less, it changes what you plan for. Agentic coding workflows that previously reserved expensive frontier calls for initial planning can now afford to re-plan at every loop iteration. This has concrete architectural implications: teams can move from a single-plan-then-execute pattern to an iterative plan-execute-replan loop without blowing their inference budget.

Practical Model Selection Strategy

For teams building agentic coding workflows, the tiered approach that makes sense today:

The key takeaway: planning is no longer a reason to default to closed frontier models. Kilo Code's benchmark is one run, but it aligns with the broader trend — GLM-5.2 already beats Fable 5 on BridgeBench Reasoning, and developer rankings place it at parity with Opus 4.8. Planning was the last capability where open-weight models lagged. If that gap is closing, the economic case for defaulting to frontier models weakens further.

For developers running agent harnesses, GLM-5.2 is already integrable into every major tool. See our step-by-step guide: How to Run GLM 5.2 in Claude Code, Pi, OpenCode & Every Harness

Related Reading

• GLM-5.2 Beats Fable 5 on Reasoning — 24 Hours After the U.S. Export Ban
• How to Run GLM 5.2 in Claude Code, Pi, OpenCode & Every Harness
• GLM-5.1 on Hugging Face & How to Run It (Ollama, vLLM)
• Kimi K2.7-Code: Moonshot AI's 1T-Parameter Open Coding Powerhouse
• OpenRouter Fusion API: Fable-Level AI at Half the Price

Kilo Code published the planning benchmark results via their official X account on June 19, 2026. Model pricing is as listed by Anthropic and Zhipu AI as of the same date. Single-run evaluations are directional, not statistically significant.