Castform + Neon: A 4B Open Model Matches GPT-5.6 Sol at 1/100th the Cost

A 4-billion-parameter open-weight model reportedly matched GPT-5.6 Sol's retrieval accuracy on a search task — while costing about 100x less per request. That's the headline claim from a joint blog post published August 5, 2026, by Neon (the Postgres database company, now part of Databricks) and Castform, an RL post-training startup, authored by Pranav Aurora, Ying Hang Seah, and Angel Pan. The post, titled "How Castform + Neon Beats Frontier Models on Price and Efficiency," is a vendor case study — not an independent benchmark — but it's a clean illustration of a pattern explainx.ai has been tracking all year: small, task-specific fine-tuned models closing the gap with frontier models on narrow jobs, at a fraction of the cost. It sits alongside our coverage of DeepSeek V4 Pro's pricing disruption and Intology's Locus post-training beating Qwen3's own team as another data point in the same story: the frontier-model tax on narrow tasks is getting harder to justify.

TL;DR

What is Castform?

Castform is a startup that specializes in reinforcement-learning post-training for AI agents. Its pitch, per cofounder Ying Hang Seah, is direct:

"Most teams' best training data is just sitting in their databases. The problem is that turning raw data into something usable is hard, and letting agents read, search, and mutate data cheaply at scale requires advanced infra. Pointing Castform at Neon skips both."

The framing Castform uses is that a "good agent" needs two separate things to work: context (the right tools to find the right data) and a model capable of deciding what to search for and when. In their telling, Neon's Lakebase Postgres and its search extensions solve the context half of that equation; Castform's RL post-training solves the model half. Neither piece alone gets you a cheap, accurate search agent — you need both wired together.

This mirrors a broader industry shift we've covered under RL post-training beating human-tuned baselines, where automated or lightweight RL pipelines are increasingly closing gaps that used to require either a much bigger model or a specialist research team.

What is Neon Lakebase Search?

Lakebase Search is Neon's hybrid text-and-vector search layer built directly on top of Postgres. It combines two retrieval primitives:

• lakebase_text — traditional BM25 keyword search
• lakebase_vector — vector similarity (embedding) search

The two result sets are merged using Reciprocal Rank Fusion (RRF), a standard technique for combining rankings from different retrieval methods into a single relevance-ordered list. This isn't a new idea — hybrid BM25 + vector search has been standard practice in production RAG systems for a couple of years now, something our RAG vs MCP comparison covers in more depth. What's notable here is where it's deployed: directly inside Postgres on Neon, rather than a separate vector database bolted onto the application stack.

Per the post, Neon's most-downloaded extension circa 2022 was pgvector — the embedding-search era of RAG. The claim is that agentic search moved past that single-shot embedding lookup starting around 2025, into multi-hop retrieval, where a model plans, searches, evaluates results, and searches again in a loop. Every one of those loop iterations, when driven by a frontier model, calls the frontier model again — which is where the cost problem starts.

How much cheaper is this than using GPT-5.6 Sol directly?

Per Neon and Castform's own numbers: about 100x cheaper per request. The post frames it against a specific baseline — a typical multi-turn agentic search request against GPT-5.6 Sol reportedly takes more than 10 seconds and costs roughly $0.03 end-to-end, because each loop iteration in a multi-hop search re-invokes the frontier model. Small open-weight models are cited as roughly 100x cheaper per token than frontier models like Sol, but they've historically lagged badly on raw capability — which is exactly the gap RL post-training is meant to close for a narrow task.

The chart in the original post ("Comparison of Castform fine-tune and frontier models by inference cost and mean evaluation reward") plots the Castform-tuned model in the high-reward, low-cost corner — roughly comparable reward to GPT-5.6 Sol while landing about two orders of magnitude cheaper on the cost axis. We're describing that chart qualitatively rather than reproducing its exact coordinates, since those weren't disclosed with enough precision to reprint responsibly.

This is the same fundamental dynamic we covered when DeepSeek V4 Pro undercut GPT-5.5 pricing by 34x: a much cheaper model doesn't need to beat the frontier model everywhere — it just needs to be "good enough" on the task that actually matters, at a price low enough to change the unit economics of running it at scale.

How Castform's training pipeline actually works

The post describes a four-stage pipeline, all running against Neon:

1. Corpus storage — the target company's documents live in Postgres on Neon.
2. Synthetic task generation — Castform generates synthetic question/ground-truth/answer training triples using the lakebase_text and lakebase_vector search primitives against that corpus.
3. RL training — every rollout's search tool call hits Lakebase Search on Neon live, so the model is trained against the real retrieval system it will use in production, not a simulated stand-in.
4. Production inference — the trained model uses the exact same search tool call at inference time that it used during training.

That last point matters more than it might look: train/production parity for the tool interface removes a common failure mode in RL-trained agents, where a model learns to exploit quirks of a training-time simulator that don't exist in the real tool.

The reward function

The post shares illustrative code for the reward design. A run_tool function performs the hybrid BM25 + vector search against Neon and merges results with RRF. A reward function then scores the model's output on three separate axes:

• Retrieval — did the model retrieve the right source document?
• Citation — did it cite the correct chunk within that source?
• Correctness — did it land on the right final answer?

Scoring on all three, rather than just final-answer correctness, is a reasonable design choice — it catches a model that gets the right answer for the wrong reason (a known failure mode in RL-trained retrieval systems), which matters if you plan to trust the citations an agent produces, not just its final text.

A worked training example

The post's illustrative example uses GitLab's public, open-source company handbook (handbook.gitlab.com) as the training corpus. From a source document —

"Trains booked through Navan will be paid by GitLab travel card. Train rides must be standard cabin class with 14 day booking lead time."

— Castform's pipeline extracts a ground truth ("Train rides must be standard cabin class with a 14 day booking lead time") and generates a synthetic question a real employee might plausibly ask:

"When booking a rail trip in Navan, what are the rules for how early I need to reserve it and which seating level I'm expected to choose?"

The synthetic question deliberately paraphrases away from the source document's exact wording ("standard cabin class" becomes "which seating level," "14 day booking lead time" becomes "how early I need to reserve it"), which is a sensible test of retrieval — it forces the model to search on meaning rather than pattern-match on surface phrasing.

Do I need my own training data to do this?

Yes, unambiguously. This isn't a general-purpose model swap — it's fundamentally an RL post-training exercise against your own corpus and your own tools. The synthetic task generation step depends entirely on having a searchable body of documents to generate question/answer pairs from, and the RL rollouts depend on live access to that same search infrastructure during training. If your organization doesn't have a well-structured corpus and the infrastructure to search it cheaply at RL-training scale (thousands of parallel rollouts, each making multiple search calls), this approach doesn't have anything to plug into.

That's also the underlying reason Neon's specific infrastructure claims matter to the pitch, not just as marketing color.

Infrastructure claims worth noting

Three Neon-specific claims the post makes, worth separating from the accuracy/cost headline:

• Autoscaling absorbs bursty RL training load. Neon's dynamic compute is credited with handling the bursty workload of thousands of parallel RL rollouts — each firing multiple search tool calls — without Castform needing to provision fixed peak capacity around the clock. Compute scales down during idle periods between training runs.
• Branching for isolated rollout state. Neon's database branching feature is framed as enabling isolated database state per training rollout — relevant for future work on stateful agents (ones that can mutate data, not just read it), beyond the read-only search case demonstrated here.
• Time-travel queries for debugging. Neon's point-in-time query capability is cited as useful for inspecting exactly what state an agent encountered during a specific training rollout — a debugging aid for catching reward hacking or broken tool calls after the fact.
• Full training observability. Castform says it provides reward curves per training step and per-task trace inspection, letting engineers catch problems like broken tools or reward hacking during a run rather than after it ships.

None of these are independently verified either — they're operational claims from the same vendor post — but they're the kind of infrastructure detail that's at least checkable against Neon's own public product documentation, unlike the accuracy benchmark itself.

Is this reproducible or generalizable beyond GitLab's clean handbook corpus?

This is the honest open question, and it's the one Hacker News commenters raised most pointedly on the linked thread. GitLab's public handbook is a genuinely good demo corpus — it's well-structured, publicly available (so readers can inspect it themselves), and written to be internally consistent. That also makes it close to a best-case scenario for retrieval.

Real enterprise documentation rarely looks like that. Internal wikis accumulate contradictory versions of the same policy, outdated pages nobody archived, inconsistent formatting across teams, and documents that assume context a search system doesn't have. None of that is addressed in the vendor post, and it's a reasonable thing to stay skeptical about before assuming this result transfers to a messier real-world corpus.

The broader HN discussion also connected this to a wider 2026 trend: a shift away from routing every agentic task through a monolithic frontier model, toward smaller, task-specific fine-tuned models for cost efficiency — a pattern also visible in other RL-post-training approaches this year, including Intology's Locus post-training pipeline for Qwen3. Whether that pattern holds up depends heavily on how well "clean benchmark corpus" results transfer to production data — which is precisely the part no vendor blog post can settle on its own.

What people are asking

Is this a real independent benchmark?

No. It's a vendor blog post co-authored by the two companies involved, describing their own evaluation setup. The 100x cost figure and the "matches GPT-5.6 Sol" accuracy claim are self-reported. Treat this the same way you'd treat any vendor case study: a credible, specific claim worth tracking, not a confirmed third-party result. Our guide to reading AI benchmarks without getting fooled is a useful checklist for this exact situation.

Why compare against GPT-5.6 Sol specifically?

Sol is OpenAI's current flagship frontier tier, covered in our GPT-5.6 Sol, Terra, Luna launch breakdown. Comparing against the flagship rather than a cheaper tier like Terra or Luna makes the cost gap look larger — a fairer apples-to-apples comparison might have used Terra, which OpenAI itself positions as the lower-cost, GPT-5.5-class tier. That's a framing choice worth noting, not necessarily a flaw.

Is RL post-training the same as fine-tuning?

Related but not identical. Standard fine-tuning typically trains on a fixed labeled dataset. RL post-training here trains the model through live rollouts — the model actually calls the search tool, gets real results back from Neon, and is scored on the outcome, adjusting its behavior based on that reward signal rather than a static label. Our prompt engineering vs. fine-tuning vs. RAG guide and grounding vs. fine-tuning decision guide both cover where fine-tuning fits relative to retrieval-based approaches; RL post-training on live tool calls is a step further than either.

Does a 4B model beating a frontier model on accuracy make sense?

Only within a narrow task scope, and that scope matters. A 4B model isn't outperforming Sol at general reasoning, coding, or open-ended conversation — it's matched specifically on one retrieval task, after being RL-trained against the exact data and tools it will be evaluated on. Frontier models are still favored for breadth; small RL-tuned models win on cost when the task is narrow, well-defined, and has a clear reward signal — which retrieval, with its retrieval/citation/correctness reward structure, happens to be a good fit for.

Related reading

• GPT-5.6 Sol, Terra, and Luna: OpenAI preview launch explained
• DeepSeek V4 Pro disrupts AI pricing: 34x cheaper than GPT-5.5
• Intology's Locus: automated post-training beats Qwen3's own team
• RAG vs MCP: the complete guide to context-aware AI systems
• Prompt engineering vs. fine-tuning vs. RAG
• Grounding, RAG, and fine-tuning: a decision guide
• How to read an AI benchmark and not get fooled
• Neon — official product site
• Castform — official product site

This post summarizes claims made in a joint Neon and Castform blog post published August 5, 2026. All benchmark figures, cost comparisons, and infrastructure claims are self-reported by the vendors and have not been independently verified as of publication. Treat the 100x cost figure and the accuracy-parity claim accordingly.