LangChain Open SWE Router: 64% Cheaper Coding Threads, 973-Thread A/B Test

LangChain says it cut the median cost of a coding-agent thread by 64 percent by putting a model router inside Open SWE, its open source coding agent, without a measurable drop in merged pull requests. The numbers come from a 973-thread A/B test described in LangChain's October 1, 2026 post, How to Build a Model Router in the Harness, and began circulating widely on October 13. This is a first-party result on the team's own traffic, labeled by LangChain as a proof of concept, but it is unusually transparent about method, which makes it useful to copy.

LangChain's post gives unusually detailed method and statistics to evaluate, which is why it is worth reading closely.

TL;DR: the results

How does the Open SWE router work?

LangChain describes four steps. First it studied its tasks: a week of Open SWE threads from LangSmith traces, labeled by an LLM classifier. New features were 22 percent, bug fixes 17 percent, and test or no-op runs 16 percent. Feature work threads were longer and costlier, while testing and release threads were short and cheap. That variation suggested many tasks did not need frontier intelligence.

Second, it picked three models along the Artificial Analysis cost-versus-intelligence curve, using chart data as of September 9, 2026: fast GLM-5.3-Flash (xhigh), balanced GPT-5.6 Sol (medium), and performance GPT-6 Astra (low). The fast tier is an open model, which LangChain presents as a sign open models have crossed a threshold.

Third, it built the router as middleware. It runs on the thread's first human message and has three parts: a base prompt telling a classifier to pick the least expensive model likely to complete the task, plain-language criteria for each tier, and a classifier model. The classifier now runs on Jev, a decision model LangChain says made classification almost 50 times faster; our guide to wiring Jev into an agent pipeline covers that tool. The router picks once at the start of the thread and the choice sticks.

Fourth, it tracked outcomes with merged pull requests as the main success metric plus thumbs up or down feedback, both logged in LangSmith.

What did the A/B test show?

Half of 973 threads went through the router and half always used GPT-6 Astra. Quality did not measurably change, and cost dropped sharply. The median routed thread cost $0.94 versus $2.61. The cost ladder between tiers was steep: median $0.097 on fast, $1.50 on balanced and $2.88 on performance, roughly a 30-times spread from cheapest to priciest.

Engineers' feedback pointed the same way. When simple requests ran on Astra, they left comments such as "pretty expensive for this query." LangChain also ran the opposite control, with half of threads always on the fast model. It ended that test within a day because engineers flagged low output quality and it disrupted their work. That is an important data point: the savings come from routing, not from simply buying a cheaper model.

Does the tier mix explain the saving?

A quick sanity check shows the numbers hang together. Using LangChain's tier medians and split, a back-of-envelope blended cost is: 56 percent of threads at $1.50, 34 percent at $0.097 and 10 percent at $2.88. That works out to roughly 0.84 + 0.033 + 0.288, or about $1.16 per thread. That is a weighted average of tier medians, which is not the same statistic as the reported $0.94 overall median, so the two should not match exactly; but they land in the same range, which is reassuring. It also shows where the money goes: the balanced tier handles most threads and drives most of the bill, so improving the balanced model's price or accuracy matters more than the performance tier, which handles only one thread in ten.

The other lesson is about the fast tier. At about ten cents a thread it is cheap enough that even a modest hit rate would pay off, yet LangChain's fast-only trial failed on quality. The router's value is therefore in the classifier's judgment about which tasks are safe to send down, not in the existence of a cheap model. A classifier that sends one hard task in ten to the fast tier will quietly cost more in rework than it saves.

What are people asking?

Is the router itself expensive? LangChain says it moved the classifier to Jev, which made classification almost 50 times faster; the post does not publish the classifier's own per-thread cost, so include it in your own math.

Does it work outside coding? The principle should transfer to any agent with a varied task mix, but LangChain only tested Open SWE. Support, research and data-analysis agents have different failure costs and need their own outcome metrics.

Why not just train a bigger, general router? A general router benefits from more data, while a harness router benefits from task context. They are not mutually exclusive: a learned router could sit inside a harness that supplies domain signals.

Can I try it? Open SWE is an MIT-licensed, open source asynchronous coding agent built on Deep Agents and LangGraph, and the model routing middleware is documented by LangChain. Its repository says the project is under active development, so expect breaking changes, and that the maintainers are not currently accepting issues or external contributions, though forking is allowed. Coding runs happen in isolated per-thread Linux sandboxes with powerful tools and network access, so the maintainers recommend least-privilege credentials and restricted repository access. Expect to write your own tier criteria, since the shipped ones are tuned to Open SWE.

What does Open SWE actually do? It investigates a codebase, edits code, runs validation and opens or updates pull requests. It can also review pull requests, monitor them, diagnose CI failures and run scheduled tasks, and work can start from a dashboard, GitHub issues or Slack. That breadth is why the task mix is so varied, and why a router has room to save money.

Why you should still be careful

1. The control was the most expensive option. LangChain itself notes the result is unsurprising given that the baseline was its priciest model. A fairer comparison for many teams is against a single mid-tier model.
2. Not significant is not equal. A p-value of 0.49 on merged PRs means the test could not distinguish the arms at this sample size, not that quality is identical.
3. It is one team's tasks. The criteria are written for Open SWE's task mix, so the router is, in LangChain's words, deeply coupled to the work it handles.
4. No mid-thread re-routing. A thread that starts as a question and becomes a bug fix stays on the model chosen at the start.
5. Merged PRs are a proxy. They measure outcomes the team accepted, not code quality over time.

Router in the harness versus a gateway

The post's design argument is the most reusable part. A gateway sees a request and a model list. A harness knows the system prompt, tools, repository and the user's role, so it can judge difficulty. LangChain frames routing as context engineering, analogous to feature engineering: decide what the router should see. That fits how products like the Cursor Router and Cognition's Devin Fusion present their own savings, and echoes the results in AT&T's 56 percent coding-cost cut.

How to copy this in your own agent

1. Pull a week of traces and label tasks by type. Look at cost and turn counts as proxies for complexity.
2. Choose three models along the cost-intelligence curve, ideally from different providers, with one open model as the fast tier.
3. Write tier criteria from your own tasks, not from a generic benchmark.
4. Add routing middleware. LangChain says it has released model routing middleware that takes a base prompt, tiers and criteria and picks a model at the start of each thread.
5. Measure outcomes before routing. Define success: merged PRs, passing tests, user feedback.
6. A/B test against both controls. Compare against your strongest model and against your cheapest, as LangChain did.
7. Watch the tail and the cache. Switching models mid-thread discards the prompt cache, so re-routing costs more than it looks for synchronous agents.

For model background, see our comparisons of GPT-6 Astra against Claude Fable 5.1 and the GPT-6 Astra launch pricing.

What is next for the router

LangChain lists four directions: benchmarking on DeepSWE or similar coding suites, routing subagents (which currently pick models independently), mid-thread re-routing when a conversation changes character, and refining criteria with signals such as user frustration found in traces. Each is a place where a naive router could still leave money or quality on the table.

What this means for what you build or pay

If your coding agent sends every task to one frontier model, this is the cheapest experiment you can run this month. Even a conservative outcome, such as a 30 percent saving with flat acceptance rates, would matter at scale. But verify on your own traffic; do not extrapolate 64 percent from someone else's mix of work.

Related reading

• How to wire Jev into your agent pipeline
• Cursor Router
• AT&T cut AI coding costs 56%
• Devin Fusion cuts coding costs 39%
• GPT-6 Astra vs Claude Fable 5.1

Figures are from LangChain's published post as of October 13, 2026.