Contrastive Language Model (CLM-8B): An Open System One Model That Matches Jev at Up to 9x Lower Latency

Most of the AI news cycle is about bigger models that think longer. CLM goes the other way: a small model that decides fast.

On September 24, 2026, researcher Jacky Kwok (Stanford CS PhD, Berkeley EECS) introduced the Contrastive Language Model, describing CLM-8B as "an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions." It performs on par with TypeSafe's Jev across computer use, gaming and tool calling at up to 9x lower latency, and, with light fine-tuning, it is used as a verifier that lifts agentic coding benchmarks to 81.6% on DeepSWE and 87.6% on Terminal-Bench 2.1.

We read the README, the charts and the benchmark setup, because the headline numbers come with caveats worth knowing. If you are new to the category, start with what a System One model is and how Jev works.

TL;DR

What a "System One model" is, in one paragraph

Psychologists describe System One as fast, intuitive judgment and System Two as slow, deliberate reasoning. A System One model is a model built for the fast kind of decision: pick one of a fixed set of options quickly and cheaply, instead of generating a long chain of thought. TypeSafe's Jev popularized the label; see our launch coverage and the fact check on its speed and cost claims. CLM is an open contender with a different training recipe.

How CLM works

The README lays out the idea clearly.

1. Two encoders. CLM trains a state encoder and an action encoder.
2. Contrastive training. Using InfoNCE, each state is pulled toward the ground-truth action that was taken and pushed away from all others in the batch.
3. Zero-shot classification. At deployment, given a current state and a set of candidate actions, CLM scores each action by how well its embedding aligns with the state's embedding and picks the highest.

A "typed question" in the API is just a state plus a closed set of options with descriptions. A softmax over the scores is the answer distribution. The same mechanism ranks best-of-N agent trajectories, routes tools, shortlists retrieval candidates and answers typed decisions "with no per-task setup."

Why disaggregation is the speed trick

Because the state and action encoders are independent, their embeddings can be cached and reused separately. If the state changes continuously but the set of actions is fixed (a game, a tool router, a fixed menu of intents), you embed each action once and only re-embed the new state. Inference then costs one embedding per fresh text plus a dot product per cached candidate. That is why the speedups are largest when there are many candidate actions (WikiRacing) or when actions are reused across states (the T-Rex game), as the README says.

Model anatomy

Each encoder is a frozen LLM backbone plus a 20M-parameter trainable projection head. In the reference serving setup, the encoder is Qwen3-8B served through vLLM in pooling mode, and the CLM head (about 75 MB) is downloaded on first run. That means most of the compute is a standard embedding forward pass; the "model" that makes the decision is small.

The data recipe

CLM-8B is trained in three stages:

• Pre-training on 60M Nemotron Q&A pairs, giving broad question-to-answer alignment.
• Mid-training on 30M synthetic hard negatives, which teach it to distinguish near-miss options from correct ones.
• Post-training on 1M agentic trajectories, adapting it to states and actions from tool use, browsing and coding.

Hard negatives are the important detail. A contrastive model that only sees easy wrong answers learns shallow features; hard negatives force it to attend to what distinguishes the right action.

Zero-shot results: CLM-8B vs Jev

Figure: zero-shot latency and success rate, CLM-8B vs Jev. Source: the Contrastive-LM GitHub repository (Apache 2.0), reproduced with attribution.

Reading the chart:

The "up to 9x" claim is the T-Rex row. Across the other tasks, speedups range from roughly 1.6x to 4x. And "on par" is fair but not identical: Jev is a bit more accurate on tool calling (99.2% vs 95.2%) and WikiRacing (30/30 vs 26/30). CLM wins on speed everywhere. Several of these benchmarks are tiny (5 trials, 30 trials), so treat the success columns as directional.

The agentic results: CLM as a verifier

Figure: DeepSWE and Terminal-Bench 2.1, success rate and verifier latency, CLM (Qwen3-8B, fine-tuned) vs Jev. Source: the Contrastive-LM GitHub repository (Apache 2.0), reproduced with attribution.

This is the flashier claim, so read the setup carefully.

Best-of-N verification. For each task, a strong coding model produces several candidate solutions (Opus 5 for DeepSWE, Fable 5 for Terminal-Bench 2.1). CLM or Jev acts as the verifier that picks the best one.

Important reading notes:

• The gain is over pass@1, not over nothing. Picking the best of several attempts lifts DeepSWE from 73.7% to 81.6% and Terminal-Bench 2.1 from 84.0% to 87.6%. That is a genuine verifier effect of about 8 and 3.6 points, not a model beating frontier models from scratch.
• Jev scored below pass@1 on both (71.1% and 83.1%), which the authors say shows it "fails to serve as an effective verifier for these long-horizon tasks." That is a claim about the untuned Jev in this setup, not about Jev's zero-shot strengths.
• The sets are small. The README says evaluation used 38 held-out DeepSWE tasks and 30 held-out Terminal-Bench 2.1 tasks, so 81.6% is 31 of 38, and 87.6% is roughly 26 of 30. A change of one or two tasks moves the percentage by several points.
• CLM was fine-tuned; Jev was not. The comparison is CLM "with lightweight fine-tuning" against Jev, and the README says the DeepSWE result is task-disjoint. Fine-tuning on the benchmark's own distribution is a fair method for a verifier but limits how far to generalize.
• "SOTA" is the authors' label. Compare against other verifier and reranking approaches before treating it as settled.

Latency is measured on an H100 for these verifier runs, and the win is large: 79 ms versus 449 ms means you can verify many candidates inside a human-scale wait.

Scaling laws

The authors report that test InfoNCE loss decreases predictably as a power law with training compute, dataset size, projection-head size and encoder size. They note these dimensions should be scaled jointly, and that scaling the encoder yields the strongest gains. Experiments use the Nemotron DQA dataset on a held-out set. Predictable scaling matters because it lets teams forecast whether a bigger encoder or more data is worth the cost. The fits and figures are in the authors' blog post, which is hosted on Notion and did not render for our text fetch, so we rely on the README for these details.

How to try it

From the README:

git clone https://github.com/Contrastive-LM/CLM.git && cd CLM
pip install -r requirements.txt

# 1. encoder: Qwen3-8B via vLLM (pooling)
vllm serve Qwen/Qwen3-8B --served-model-name qwen3-8b --runner pooling \
     --enable-prefix-caching --max-model-len 2048 --gpu-memory-utilization 0.35 --port 8090

# 2. CLM API (downloads the 75 MB reference head on first run)
clm-serve --port 8700 --emb-url http://127.0.0.1:8090/v1/embeddings

Requirements: Linux, Python 3.10+, an NVIDIA GPU. A typed question in Python:

from clm import CLMClient, Choice, Noul, Score

client = CLMClient()
r = client.system_one(
    state="Customer: my invoice was charged twice and nobody answers the phone!",
    questions={
        "urgency": Noul(instructions="Is this urgent?"),
        "department": Choice(instructions="Which team should handle this?",
                             criteria={"billing": "Charges, invoices, refunds",
                                       "technical": "Bugs and outages"}),
    },
)

The README's example prints a probability that the statement is true, a probability per department option, and reports about 58 ms of latency. The API is described as TypeSafe-compatible, so code written against the Jev-style interface may need few changes. There is also a playground captured against a real server on one RTX 4090, and a fine-tuning tutorial for your own data.

Where CLM fits, and where it does not

Good fits:

• Routing: intents, tools, queues, model selection. See Jev-style agent routing.
• Verification checkpoints in agent pipelines. See cheap verification checkpoints with Jev.
• Reranking best-of-N outputs.
• Real-time decisions in games and UI agents.
• Classification with changing labels, where you can rewrite option text instead of retraining.

Poor fits:

• Open-ended generation. CLM chooses among options; it does not write.
• Long-horizon planning on its own. It verifies and routes; it does not plan.
• Tasks where the candidate set is unknown or enormous and cannot be enumerated.
• Anything needing calibrated probabilities without checking. Softmax outputs from a contrastive head are scores, not guarantees. Validate calibration on your data. See where Jev actually fails for the kinds of failure this class of model can have.

How it compares with the Jev ecosystem

Since Jev launched, the ecosystem has produced clones and alternatives: six Jev clones in two days, Kev, an open-source clone on Qwen3.5, OpenJev in the browser, Laya on MLX and Bespoke Nimble-9B. CLM differs in that it is a new training recipe (contrastive, disaggregated) rather than a clone of Jev's architecture, and it is Apache 2.0. Tooling around System One models is also maturing: see LangChain's Jev agent evals benchmark and LangSmith scoring production traces with Jev.

For the benchmarks themselves, see our notes on DeepSWE, Terminal-Bench 2.0, and why benchmark isolation matters.

What we could not verify

• The authors' blog post. It is on Notion and did not render for our fetch, so architecture and scaling details come from the README.
• Independent replication. The numbers are self-reported on small held-out sets.
• Head-to-head with other rerankers or reward models on the same tasks.
• Behavior under distribution shift beyond the tested benchmarks.

Bottom line

CLM is a well-documented, open, and very fast take on System One decision-making. The speed claims hold in the published charts; the accuracy claims are "on par" zero-shot and a real but modest verifier lift on small benchmarks. If you build agents that make many cheap decisions, such as routing, verifying or reranking, it is worth a benchmark against your current LLM calls. Run it on your own tasks before you trust the leaderboard numbers.

Details reflect the Contrastive-LM GitHub repository and the author's X thread on September 24, 2026. Figures are reproduced from the project's Apache 2.0 repository with attribution; results are self-reported.

Related reading

• What is a System One model?
• How does Jev work? RLCD explained
• TypeSafe AI launches Jev
• Jev speed and cost claims: a fact check
• Where Jev actually fails
• Cheap verification checkpoints in agent pipelines
• Six Jev clones in two days
• Official: CLM on GitHub, CLM data and models on Hugging Face