Coinbase says a small open-weight model it post-trained for under $100 beat a frontier model at one job: catching fraud on its Onramp payments product. In Part 3 of its Owning Intelligence series, dated October 8, 2026, the company describes post-training Qwen3.5-9B with reinforcement learning, and reports that it exceeded Claude Opus 4.5 on every detection metric it tracked. CEO Brian Armstrong summarized the result on X as a fine-tuned open-weight model outperforming frontier models "at much lower cost and speed" for a specialized task. This post covers the method, the numbers, and the caveats. We could not load the Coinbase page directly (it returned an access error to our fetcher), so figures below come from search excerpts of the post and secondary coverage, and are marked as company-reported.
TL;DR
| Question | Answer |
|---|---|
| Who | Coinbase, for its Onramp fraud detection |
| Model | Qwen3.5-9B, open weights, post-trained |
| Training | Reinforcement learning with SkyRL on Anyscale; one NVIDIA RTX PRO 6000 96GB GPU; reported total cost under $100 |
| Benchmark | 16,140 Onramp transactions, 7,293 users, 813 confirmed fraud cases (company-reported) |
| Result | Beat Opus 4.5 on all four metrics, with 9.6% higher F1; 55% lower end-to-end latency in online inference |
| Production | In A/B testing, reported 30% fewer fraudulent transactions and 22% lower fraud dollar value |
| Caveat | Proprietary benchmark; no independent replication |
What was built
The agent is not a replacement for Coinbase's existing machine-learning risk models. Per the coverage, it sits on top of them and reviews transactions that have already cleared those systems, looking for fraud the first layer missed. That placement matters: it explains why a small model can add value, since it only has to handle a narrow, hard residual slice with a lot of labeled history.
Post-training used historical fraud outcomes as the reward signal. Because the ground truth (was this payment later confirmed fraudulent?) is deterministic, the reward needs no human judge or model judge, which sidesteps one of the weaker parts of most RL-for-LLMs setups. Coverage says rewards balanced fraudulent and legitimate examples, the standard defense against a model that learns to approve everything.
The numbers, with attribution
According to the coverage of Coinbase's post:
- On the benchmark, the tuned model led on precision, recall, F1 and dollar-weighted recall, with margins reported between 7.5 and 35.4 percentage points over baselines.
- Against Opus 4.5 specifically, it was 9.6% higher on F1.
- Online latency fell by 55%.
- Training cost under $100 on a single 96 GB workstation-class GPU.
- A/B test: 30% fewer fraudulent transactions and a 22% lower dollar value of fraud.
The surprising part: newer models did worse
Separate coverage of the same series says Coinbase replayed the 16,140 transactions with fixed guidance and the same risk-classification policy, changing only the model version. In each pair, the newer model caught less fraud: Opus 5 versus Opus 4.5, Sonnet 5 versus Sonnet 4.6, GPT-5.6 versus GPT-5.4. Reported effects: Sonnet's recall dropped 22.2 percentage points and dollar-weighted recall 22.9; GPT-5.6's precision rose 11.5 points while recall fell 20.7; Opus slipped by less than a point of recall.
The pattern, if it holds, is plausible and well known: newer models can be more conservative or follow instructions differently, so a prompt tuned for the older model can behave differently on the newer one. It is a reminder to treat model upgrades as changes requiring evaluation. It is not proof that the newer models are worse in general. Our own comparison of small current models is in the small models comparison.
Claimed vs verified
- Verified: that Coinbase published the series post and that its CEO posted the summary claim.
- Company-reported, not reproducible: all benchmark numbers. The data are private transactions.
- Not shown: how prompts for the frontier baselines were tuned versus how much effort went into the specialist. A fair comparison needs equal tuning effort; prompt-only baselines usually trail a purpose-trained model.
- Not shown: performance on new fraud patterns that appear after training. Fraud adapts, so a model trained on history can decay.
Reading the metrics: why recall and dollars matter
Fraud screening is a trade between two kinds of error. Precision asks: of the payments the model flagged, how many were truly fraud? Recall asks: of all the fraud that happened, how many did the model flag? F1 blends the two. Dollar-weighted recall counts each missed fraud by its value, so missing one large payment hurts more than missing several small ones.
A model can look better on one axis while being worse where it counts. The GPT-5.4 versus GPT-5.6 comparison in the coverage is the textbook case: precision rose, so flagged payments were more often real fraud, but recall fell by about twenty points, so many frauds slipped through. In a payments business a missed fraud is a direct loss and often a chargeback, while a false flag costs a review or a declined customer. The right operating point depends on those costs, which is why Coinbase reports dollar-weighted recall and why any team copying this should define its own cost matrix before comparing models.
Base rates matter too. In the reported benchmark, 813 of 16,140 transactions were fraud, about 5 percent, which is far higher than real-world traffic. Benchmarks are often enriched with positives so they are informative, but that means precision on the benchmark overstates what you would see in production, where legitimate payments vastly outnumber fraud. The A/B test, which runs on live traffic, is the more meaningful of the two evidence types, and it is also company-reported.
Why newer models can regress on a fixed prompt
The replay result is the part most likely to be quoted, so it deserves care. Holding the guidance and policy fixed while swapping the model isolates one variable, but it also freezes a prompt written for an older model. Newer models are often trained to be more cautious, to follow the letter of instructions more literally, or to hedge on borderline cases. For fraud triage that can show up as a higher bar before saying "fraudulent", which raises precision and lowers recall, the exact pattern reported for GPT-5.6.
There are three reasonable conclusions, and none is "newer models are worse". First, prompts and policies are coupled to a model version and should be re-tuned at each upgrade. Second, evaluation sets built from your own outcomes catch regressions that public benchmarks cannot. Third, a purpose-trained model is insulated from vendor-side changes, because you own the weights and decide when to move. That last point is the strategic argument in the series title, owning intelligence: control over a critical decision function rather than renting it from a provider that updates on its own schedule.
The cost is operational. You take on training, evaluation, deployment and retraining, plus the risk that your specialist silently decays as fraud tactics change. Teams without labeled history, an objective reward or ML engineering capacity will usually do better with a frontier model plus careful evaluation.
Why a 9B model can win here
- Narrow task, rich labels. Fraud outcomes are logged for years. That is the ideal shape for supervised or RL post-training.
- Objective reward. No judge noise.
- Latency and cost matter. A small model on owned hardware answers faster and costs far less per call, which supports real-time screening.
- Data advantage. The specialist has seen the company's own failure modes; a general model has not.
This is the same direction we see in other recent work: post-trained small models beating larger ones on a defined task, as in Castform's open model beating GPT-5.6 on retrieval and an 8B RL model on ALFWorld. For the basics, see our fine-tuning guide.
What this means if you build
- Re-run your evals on every model upgrade. Keep a frozen replay set of real cases, like Coinbase's 16,140.
- Report precision and recall separately, and weight by dollars when errors differ in cost.
- Try a specialist before scaling a prompt. If you have thousands of labeled outcomes and a deterministic reward, a small post-trained model may beat a prompted frontier model at a fraction of the cost.
- Keep the frontier model for the long tail. Specialists are brittle off-distribution.
- Plan for drift. Schedule retraining and monitor recall on fresh confirmed cases.
- Guard the agent. Models that make financial decisions need limits on actions; a tool like AgentBeam from the explainx.ai team is built to stop agents before they take dangerous actions.
What people are asking
Is this open source? The base model, Qwen3.5-9B, is open-weight. Coinbase's fine-tuned weights, data and harness are not published in the coverage we saw. Armstrong said the company built an internal harness employees can use to spin up and train their own models, such as for payments risk review.
Did a 9B model really beat Opus? On one private task, per the company. That is meaningful evidence for the specialization strategy and weak evidence about general capability.
Could we do the same? If you have labeled outcomes, an objective reward and the engineering to run RL, probably for a narrow problem. SkyRL and similar open frameworks lower the barrier; the hard part is the data and the evaluation.
Related reading
- Small models comparison: Haiku 5.5, GPT-6 Luna, DeepSeek V4.1 Flash and others
- Castform: RL post-training beats GPT-5.6 on retrieval
- EvoHarness: an 8B RL model on ALFWorld
- What is fine-tuning? A complete guide
- CoreWeave model distillation with a GLM teacher
- Qwen censorship audit
- Primary: Coinbase, Owning Intelligence Part 3
Numbers are company-reported and drawn from coverage on October 13, 2026; we could not load the original page.