AI21 says it turned NVIDIA's open Nemotron 3.5 Super VL into three specialized enterprise models in three weeks, and that each one matches or beats frontier models on its task at a fraction of the cost. The company's announcement is dated October 15, 2026. The headline most people saw, "beats Claude Opus 5.5 at 33x lower cost," is directionally what AI21 claims, but one number in circulation is wrong: social summaries quote 88.5 percent payroll accuracy, while AI21's own post says 72 percent.
This post separates what AI21 claims from what can be checked, and what it means if you are deciding between renting a frontier model and owning a tuned open one. It builds on our coverage of the Nemotron 3.5 Super VL release.
TL;DR: claimed vs verified
| Claim | Source | Status |
|---|---|---|
| Three specialized models built in three weeks | AI21 blog | Self-reported by the vendor |
| Deel payroll: 72% accuracy, 8 points above Opus 5.5, 33x cheaper per query | AI21 blog, Emet by Deel benchmark | Self-reported; no independent run; no cost methodology published |
| Deel payroll: 88.5% accuracy | Social posts and aggregator summaries | Not in the AI21 post; treat as a misquote until AI21 shows it |
| Gong: matches Opus 5.5 on speed and accuracy at a small fraction of the cost | AI21 blog, citing Gong's own evaluations | Self-reported; no dollar figure |
| Alice: attack resilience up 41 points, beating GPT-6.1 Sol | AI21 blog | Self-reported; measured on Alice's own RL gym |
What AI21 actually did
AI21 used the open Nemotron 3.5 Super VL as the base rather than training from scratch. NVIDIA describes it as an open multimodal model for long-running agents that need to see, reason and act, built to be post-trained on an organization's own data and policies. AI21 sells this as "Lab-as-a-Service," where its Forward-Deployed Researchers build the recipe and the custom evaluations.
The three partners show three different goals: cutting cost without losing quality (Gong), raising accuracy past the frontier on a proprietary task (Deel), and hardening against attacks while keeping data in-house (Alice).
Deel: the 72 versus 88.5 percent question
Deel runs a payroll question-answering model across more than 40,000 companies and 150-plus countries. AI21's post says frontier models often leave part of an answer out, and "in payroll, though, a partial answer is a wrong answer."
The recipe was supervised fine-tuning on pruned generated rollouts, followed by SDPO (self-distillation policy optimization), where the teacher gets hindsight by conditioning on detailed instructions about the desired behavior. Training data and the benchmark came from Emet by Deel, an evaluation lab.
The result, in AI21's words: accuracy "rose to 72% on Emet's 8-country payroll benchmark evaluation, sitting 8 points above Claude Opus 5.5, and costing 33x less per query." That makes Opus roughly 64 percent on the same test, if the 8 points are percentage points. The "88.5%" version of the story, and the "8% better" phrasing that goes with it, do not match the primary post. We could not find where 88.5 comes from, so do not cite it.
A second caveat from the post itself: 72 percent is not a guarantee of correct payroll guidance. Nearly three in ten answers still fail the benchmark.
Gong: parity at lower cost
Gong's Data Extractor reviews calls, emails and web conferences and updates CRM fields such as deal stage and health scores, at tens of millions of interactions a month. AI21 says it built a training set in two weeks, used a mix of supervised fine-tuning and DPO, and built evaluation gates on the CRM edits.
The claim, from Gong's own evaluations, is parity with Opus 5.5 on speed and accuracy "at a small fraction of the cost." No cost figure is given, so there is nothing to verify beyond the direction.
Alice: a 41-point attack-resilience gain
Alice builds RL gyms where customers train agents against security threats. AI21 applied supervised fine-tuning and DPO, then GRPO online reinforcement learning, using Alice's "Rabbit Hole" dataset built from threat-actor intel and more than 150 staff security researchers and expert-network members. The target was indirect prompt injection in Slack and GitHub.
AI21 reports a 41-point improvement in attack resilience with no harm to overall task performance, outperforming frontier models such as GPT-6.1 Sol. This is resistance on Alice's own tested attacks, not a general immunity. For background on why this class of attack matters, see our guide to indirect prompt injection.
The four training methods in plain language
AI21's post name-drops several techniques. Here is what each one does, so you can judge the recipes:
- SFT (supervised fine-tuning): the model is trained on example inputs paired with the answers you want. For Deel, those examples were pruned generated rollouts, meaning model outputs filtered down to the good ones.
- DPO (direct preference optimization): the model is shown pairs of answers, one preferred and one rejected, and nudged toward the preferred style. Gong and Alice used it after SFT.
- SDPO (self-distillation policy optimization): a teacher copy of the model sees extra hindsight, here detailed instructions about the desired behavior, and the student learns to produce that behavior without being given the instructions. This is how AI21 says it baked "give the complete answer unprompted" into the weights.
- GRPO (online reinforcement learning): the model generates several attempts, a reward signal scores them, and the policy shifts toward higher-scoring attempts. Alice used it inside its RL gym once the model was good enough to produce varied, useful attempts.
The common thread is that the work is mostly about data and evaluation, not exotic architecture. AI21 itself says the first step at Deel was to test whether the base model already had the knowledge, and found it did: the training mainly changed behavior and level of detail. That matches the broader lesson that fine-tuning is best at changing how a model answers, not at teaching it large amounts of new facts.
Rent versus own: the argument behind the announcement
AI21 frames the results as a case for "sovereign specialized intelligence": companies owning a tuned model rather than renting a general one. The reasons it lists are cost, control over data, and a differentiating edge. These are real trade-offs, but they come with costs the announcement does not price: building the evaluation set, retraining when rules change (payroll rules, as the post notes, change regularly), hosting and monitoring the model, and security review. A frontier API absorbs all of that into a per-token price.
The honest comparison therefore has two parts. First, the per-query cost gap, which AI21 puts at 33x for Deel. Second, the fixed cost of the project, which is not stated. Whether the saving pays back depends on volume, and the post stresses volume: tasks with "up to millions of queries a month."
Why the numbers deserve caution
- Partner-defined tasks. Each evaluation was designed with or by the partner for one narrow use case. A model tuned on a task usually beats a generalist on that task. The interesting question is the cost-adjusted gap, not the win itself.
- No cost methodology. "33x less per query" depends on the price assumed for Opus 5.5, the tokens produced, and whether the open model's hosting cost is counted. None is published in the post.
- No independent replication. There is no public model release, dataset or harness for these three models, so outsiders cannot rerun the tests.
- Moving baselines. Opus 5.5 was run as a general model. A frontier model given the same fine-tuning data or a long, tuned prompt could narrow the gap.
- Vendor interest. AI21 sells the post-training service the post advertises.
None of that makes the results implausible. It is consistent with a pattern we have covered before: Coinbase's fine-tuned Qwen3.5 9B on fraud, and the Paper Instruments Ultramarine claim. In each case a narrow, repetitive, well-specified task favors a small tuned model.
What this means for what you build or pay
The practical rule of thumb: specialization pays when the task is repeated at high volume, the "right answer" is well defined and checkable, and you can afford to build an evaluation set.
| Situation | Better fit |
|---|---|
| Millions of similar queries a month, clear scoring | Post-trained open model |
| Broad, changing tasks, few examples | Frontier model via API |
| Strict data-control or sovereignty needs | Open model you host |
| No trustworthy evaluation set yet | Build the evaluation first, then decide |
If you want to try the approach, our fine-tuning guide covers the basics, and the token-cost playbook from Microsoft covers cheaper alternatives such as routing and caching that need no training. Our piece on the AI verification tax is a reminder that a cheaper wrong answer is still wrong.
How to check claims like this yourself
- Find the vendor's primary post, and read the exact number and its unit (points versus percent).
- Ask who ran the evaluation and whether it is public.
- Look for the cost basis: price per million tokens, tokens per query, hosting.
- Run a 50-to-100 example sample from your own data against both models.
- Treat any number you cannot find in the primary source as unverified, as with the 88.5 percent figure here.
What to watch next
Watch for a published evaluation from Emet by Deel, a Gong or Deel statement on production use, AI21 releasing weights or recipes, and independent tests of Nemotron 3.5 Super VL fine-tunes. We will update this post if any of them appear.
Related reading
- Nemotron 3.5 Super VL: NVIDIA's open 120B multimodal agent model
- Coinbase's Qwen3.5 9B fraud agent beats a frontier model
- Paper Instruments Ultramarine 1 Flash: claimed vs verified
- Claude Opus 5.5 launch: benchmarks and pricing
- What is fine-tuning? A complete guide
- What is indirect prompt injection?
Figures are as stated by AI21 on October 15, 2026 and have not been independently verified.