Kev's Real Numbers: Inside the Open-Source Jev Clone's 0.8B/4B/9B Family

Six weeks ago, explainx.ai covered Kev — Jared Palmer's open-source alternative to TypeSafe AI's Jev — through nothing but digest headlines: a Kev-0.5B built to run on a MacBook Pro, then an unsourced "Kev-8B" follow-up that explainx.ai's own reporting flagged as unverifiable, since "no linked source article was available describing Kev-8B's specific architecture, training method, or benchmark results."

That source has now arrived, and it tells a different story than the headlines did. Kev's GitHub repository and a 77-point Hacker News launch thread describe a documented three-model family — Kev-0.8B, Kev-4B, and Kev-9B — with training code, frozen evaluation suites, a local API server, and benchmark numbers run directly against Jev. There is no 8B model in the released lineup. This post corrects the earlier record and covers what the primary source actually shows: how Kev works, how it stacks up against Jev, what its own creator says it's bad at, and what the Hacker News crowd argued about it.

TL;DR: What people are asking

Correcting the record: what "Kev" actually is

The confusion up to this point wasn't unreasonable — explainx.ai's coverage of six Jev clones shipped in 48 hours was itself built from digest headlines with no primary source to check against, a limitation that post was explicit about at the time. Kev's README removes that ambiguity. It describes itself as "small Jev-like decision models you can train and run yourself," positioned directly against Jev's own architecture and API shape — Kev's local server implements the same request format as TypeSafe's System One API, down to matching field names, so TypeSafe's own Python SDK works against a local Kev server without modification.

The three released sizes target different tradeoffs, per the README:

Each pair is development / test accuracy, and lower Brier score is better calibration. Kev's README recommends Kev-4B as the default starting point, Kev-9B when calibration matters more than memory footprint, and Kev-0.8B when the deployment target genuinely can't spare the extra parameters.

How Kev actually works: LoRA adapter plus pointer head

This is the part the earlier headline-only coverage couldn't get to, and it's the most technically interesting piece of the README. Each Kev checkpoint is not a full fine-tune — it's a rank-16 LoRA adapter plus a small pointer head sitting on top of a frozen Qwen3.5 base model. The base weights never move during training; only the adapter and head do.

At inference, the state text and each question get tokenized into one sequence: the state, followed by a question block with its options, followed by a <decide> token. The pointer head scores each option's hidden state against the question's <decide> hidden state, and a softmax over those scores produces the answer probabilities. Because <decide> is the last token in its block, it can attend to the full option list before producing a score — the mechanism that lets Kev report calibrated probabilities across an arbitrary number of options rather than a single top pick.

Question isolation — making sure one question in a batched request can't "see" another question's options or instructions — works differently depending on the base model's architecture, and Kev's README is unusually specific about this:

• On attention-only models (the earlier Qwen3-based generation), each question gets its own attention mask and its position IDs restart right after the shared state, so the model computes the state once and each question is blind to every other question and to future tokens.
• On Qwen3.5, which mixes attention layers with recurrent Gated DeltaNet layers that ignore attention masks entirely, isolation instead comes from running each question as its own row — state plus that one question — with the server caching the shared state prefix and reusing it across rows. Kev's own parity tests report the two approaches produce probabilities within 4e-6 of each other on attention-only models, confirming the row-based method isn't a lossy workaround.

The three question types Kev exposes — matching Jev's own primitives — are noul (yes/no probability), choice (up to 255 named options with descriptions), and score (an ordered rating scale). A single request can mix all three against the same input text, which is the same design Jev launched with.

What the numbers actually show, beyond the headline

Kev's README doesn't stop at the development-set table above — it's notably candid about where the comparison to Jev breaks down. Directly under the benchmark table, it states: "Kev-9B trails Jev by about 4.5 points on the new-source development set. We don't know which datasets Jev was trained on, so this isn't a controlled comparison of the two architectures." That's a meaningfully more honest framing than the "rival Jev" language attached to the earlier unsourced "8B" headline.

Two external test sets, built by other projects and included as-is, complicate the picture further:

• On SemIf's 144 authored decisions, Jev leads clearly: 0.965 vs Kev-9B's 0.917.
• On scienthoon's 900 support tickets, Kev-9B actually beats Jev on routing accuracy (0.952 vs 0.897) and is close to even on tone (0.911 vs 0.914).

That split matters more than either number alone. It suggests Jev's edge isn't uniform across task types — a smaller, self-trainable model can outperform a hosted vendor model on the specific domain it's been evaluated against, which is a genuinely useful data point for anyone deciding between the two rather than assuming Jev's previously fact-checked speed and cost claims also imply a fixed accuracy gap.

"Just a classifier with new branding," or something structurally different?

The Hacker News thread attached to Kev's launch — 77 points, 41 comments as of this writing — spent a meaningful share of its discussion on the same question explainx.ai has covered separately for Jev itself: is this category actually new, or is it classic classification with a fresh vocabulary?

One commenter (Oras) called it "vibe coders" rediscovering classifiers. Another (akkad33) asked directly how the model handles an arbitrary, user-defined option set instead of a fixed label space a traditional neural network would need baked in at training time — the answer, per the architecture section above, is the pointer head: it scores whatever options are passed in the request against the question's decide token, so the same trained checkpoint handles a 2-option or a 200-option request without retraining, unlike a classifier's fixed softmax output layer. A defender (BoorishBears) pushed back hard on the "nothing new" framing, arguing the comparison to BERT-era classifiers undersells what changes when option sets are dynamic and zero-shot rather than fixed at train time.

A separate, more technical objection (hbarka) questioned the "Jev-like" framing on training grounds: Jev is reportedly built with RLCD, while Kev's own README states plainly that its adapter and pointer head are trained with cross-entropy on labeled examples, on top of a Qwen3.5 base trained with standard RLHF-style methods. That's a legitimate distinction Kev's README doesn't paper over — "Jev-like" describes the API shape and question types, not an identical training method.

What people said they'd actually build with it

Beyond the categorization debate, the thread surfaced concrete use cases that line up with ground explainx.ai has already covered for Jev itself, worth reading alongside this post rather than in isolation:

• Coding-agent verifiers. Commenter NitpickLawyer sketched a pattern of generating a plan, then a per-step verifier classification — "is this following project conventions," "is this touching files from another task" — run after each agent action rather than folding verification into the main model's own output. That's functionally the same pattern explainx.ai described for Jev as cheap verification checkpoints in agent pipelines, just running on a self-hosted Kev checkpoint instead of a hosted Jev call.
• Tool-call risk gating. vidarh raised classifying bash tool calls as safe or unsafe before execution — the identical middleware pattern covered in explainx.ai's Jev agent-routing integration guide, where a noul check flags destructive commands before they run.
• Support and ticket routing, the scienthoon benchmark's actual domain, and the use case where Kev-9B's numbers above are strongest.
• Spam filtering, raised by jwr as a comparison point against 27B-class general models the commenter already uses for the same task, wondering whether Kev's much smaller footprint holds up on accuracy for that specific job — an open question Kev's README doesn't directly answer, since spam filtering isn't one of its published eval categories.
• A local NPC decision system for a 2D roguelike (saejox), cited specifically because a purpose-built small model avoids hand-writing a classic state machine — a use case with no Jev-parity comparison, since it's outside anything TypeSafe has published on.

Fine-tuning without losing what Kev already knows

For teams whose categories don't match Kev's public training data, the README documents kev.train --init_from, which loads a released checkpoint's LoRA adapter and pointer head before training continues on new labeled data, rather than starting from the base Qwen3.5 model cold. The stated reason: starting from the base model discards everything the released checkpoint already learned about the noul/choice/score question format itself, not just domain knowledge.

The README backs this with one concrete before/after: a user's fine-tune on 836 support-tool decisions, trained from the base model, scored 0.33 on Kev's own held-out evaluation set — a collapse, not a modest regression. The same 836 examples trained with --init_from instead kept 0.83 on that same evaluation set and reached 0.88 on the new support-tool domain. That's the strongest single data point in the README for treating Kev's released checkpoints as a starting point for domain adaptation rather than either using them unmodified or training a bespoke model from scratch.

Honest limitations, per Kev's own documentation

Kev's README is unusually direct about where the models fall short — worth reading in full rather than through a marketing lens:

• Calibration degrades sharply on new sources. Kev-4B assigns at least 90% confidence to a wrong answer on 8.2% of new-source development questions (Kev-9B: 7.5%). The README's own instruction is blunt: "Test it on your own data before choosing a probability threshold."
• Fine-tuning measurably erodes date arithmetic. The untrained Qwen3.5-9B base scores 0.82 on deadline-policy questions; Kev-9B, after the standard training recipe, drops to 0.72 on the same task category — a documented regression from training itself, tracked as issue #8 in the repository.
• Knowledge questions lag Jev by a wide margin — Kev's MMLU score of 0.74 against Jev's reported 0.90, the largest gap in the README's own comparison table.
• Apple Silicon serving is slow for the Qwen3.5 generation specifically. Kev-9B takes roughly 2 seconds per five-question request on an M5 in bf16, against about 300ms for the equivalent Qwen3-based Kev-8B from the prior generation — because Qwen3.5's Gated DeltaNet layers have no fast inference kernels on Apple GPUs yet, forcing PyTorch's slower reference implementation.
• Option order can change the answer, and Kev's own question-isolation mechanism doesn't prevent that — it only isolates questions from each other, not options within one question from each other.
• The local server has no authentication by default and binds to 127.0.0.1 — fine for local development, a real gap to close before exposing it beyond a single machine.

What this means for builders

If you were holding off on evaluating Kev because the only available coverage was an unsourced "8B" headline, that specific reason no longer applies — the project is real, documented, and its own benchmark numbers are candid about where it trails Jev and where it doesn't. The practical next step matches what explainx.ai has recommended for Jev itself: pick the size that fits your latency and hardware budget (Kev-4B as the sane default per the README), and test it against your own representative task rather than either the development-set table above or the scienthoon numbers where Kev happens to win — Kev's own documentation makes clear those results don't generalize automatically to a task you haven't measured.

For teams specifically weighing self-hosted control against a hosted API, Kev's --init_from fine-tuning path is the more distinctive part of the story: a released checkpoint you can adapt to your own categories in a few hundred labeled examples, without TypeSafe's waitlist, pricing, or previously scrutinized benchmark claims in the loop at all.

Related on explainx.ai

• TypeSafe AI launches Jev: a "System One Model" that never hallucinates
• Six Jev clones shipped in two days — the original, unsourced Kev-0.5B coverage this post corrects and supersedes
• Jev Playground and JevBench: what TypeSafe AI actually claimed — the post that first flagged the unsourced "Kev-8B" figure as unverifiable
• Is Jev's 200x-faster, 400x-cheaper claim actually true?
• Jev vs. XGBoost and BERT: is a System One Model actually new? — the same "is this really new" debate this post's Hacker News section covers for Kev
• How to wire Jev into your agent pipeline for routing decisions
• Using Jev as cheap verification checkpoints in agent pipelines
• Bespoke Nimble: a 9B model hit 90% on Jev, built in days
• Official source: jaredpalmer/kev on GitHub

This post is sourced to Kev's public GitHub README (jaredpalmer/kev) and its associated Hacker News launch thread as of September 21, 2026, and supersedes explainx.ai's earlier digest-only reporting on an unverified "Kev-8B" figure. Benchmark numbers, limitations, and quoted commenter arguments are drawn directly from those primary sources; verify current figures against the live repository before citing them elsewhere, since Kev's own PLAN.md indicates active, ongoing experimentation.