How GLM-5.3-Flash Scores a Closed Choice in One Token

Most of what a product asks a model is a closed choice: which queue gets the ticket, which clause type a sentence is, whether a message should be held for review. GLM-5.3-Flash can answer that kind of question from a single token. On September 24, 2026, Johannes Hötter (VP Growth) and Marko Rosenmüller, PhD (Technical Lead AI) at Edgeless Systems published a Privatemode writeup that forces the model to emit one option index, then reads the log probabilities of every legal index at that position. They ran the setup against TypeSafe's Jev and Convai's Laya. This guide explains the method, the serving details that change the numbers, and how to read their benchmark.

Jev is a System One Model: a model whose job is a typed decision with a probability on each option, trained for that job. The mechanism behind Jev, including RLCD, is already covered on explainx.ai. This post does not re-derive it. The new piece is the other direction: leave a general chat model as it shipped, and make one forward pass behave like a decision model for a fixed list.

Figures below are Privatemode's, as of September 24, 2026. explainx.ai did not re-run the 29 datasets.

TL;DR

Why a closed question does not need a JSON object

A generative model never writes text directly. Given a prompt, it produces a distribution over the vocabulary. Ordinary decoding picks a token, appends it, and repeats. A routing call that returns a JSON object pays for every token of that object, and a reasoning model may write hundreds of tokens before the object starts. You also do not get a probability for each legal answer unless you ask for one in prose and then hope the prose is honest.

Hötter and Rosenmüller put the point directly: "it's unnecessary to have the LLM predict the whole JSON object, as we already know its shape." The shape is fixed. The only unknown is which option the model prefers. If the options are numbered, that unknown is one token.

Specialized decision models exist because high-volume routing, moderation, and clause classification made that decode loop expensive. Jev's launch is the hosted version of that idea, and the category explainer covers why a small output space changes latency. Laya is the small on-device version, including the MLX build. The Privatemode result is a third implementation: same product shape, no decision-model training, one token from a chat model you may already be serving.

How one forward pass scores the options

Their procedure has three steps. None of them is a new model.

Number the options. Put the state, the question, and the options in JSON. Each option carries an index. The instruction tells the model to answer with choice_index: and an index. Names stay in the prompt so the model can read them. The token it must produce is the index, which is a short, tokenizer-stable target.

Prefill the assistant turn. The prompt ends at choice_index:. The next token should be an option index. On vLLM's OpenAI-compatible API they call /chat/completions with continue_final_message and add_generation_prompt: false, so the server continues the prefilled assistant message instead of opening a fresh one. That same chat request is what lets them place images beside the text.

Read the row, then renormalize. The sampled token is a weak summary of that position. A sample of one draw hides a 60/40 split and a 99/1 split the same way if both draws land on the winner. They read the probabilities of the option-index tokens at that one position, renormalize so those options sum to 1, and pick the max. Tokens outside the option list are discarded by that renormalization, which is why "none of these" is impossible unless you add it as an option.

Their illustrative routing figure, which they label as illustrative logits, shows payments at 62.1 percent, complaints at 37.7 percent, and technical at 0.2 percent, with a confidence of 0.39. They define that confidence so 1 means certain and 0 means the options are equally likely. Their library README says the same endpoints in prose: 0 when probability is spread evenly, 1 when it sits on one option, and it measures how sure the model is, which is a separate question from whether the choice is correct. explainx.ai is not filling in a formula they did not publish. Use the number as a review threshold the way they describe it, and calibrate the threshold on your own labels.

They set max_tokens to 1 and temperature to 0. Temperature 0 is still not bit-reproducible. Their words: temperature 0 "removes the randomness from sampling, but batching and floating-point arithmetic still keep a forward pass on a busy server from being bit-reproducible." GLM-5.3-Flash and Jev each changed up to 3.5 percent of answers between identical runs. They treat smaller gaps as noise. If you gate a workflow on a 1-point accuracy delta between two of these systems, you are inside that band.

The prompt shape you can paste

This is the shape in their figure, written as valid JSON. The working client, including image handling and the token-id cache, is edgelesssys/privatemode-decisions. Treat the block as the contract a coding agent should implement, and check the repo before you copy request fields onto a server you have not read.

Answer with choice_index: and the index of the single best option.

{
  "state": "I was charged twice for my order.",
  "question": "Which team?",
  "options": [
    {"index": 0, "name": "payments"},
    {"index": 1, "name": "complaints"},
    {"index": 2, "name": "technical"}
  ]
}

The assistant message is prefilled, and it stops at the colon:

choice_index:

Any endpoint that will continue a prefilled assistant turn and return logprobs for specific token ids can host this. A generic endpoint that only returns the sampled text cannot. You would be back to trusting one draw.

The sketch below is the scoring step a reader can adapt. It is a softmax over the option logprobs you already fetched. It is not a client for a particular server, and it does not know your tokenizer.

import math

def score_options(logprobs: dict[str, float]) -> tuple[str, dict[str, float]]:
    """logprobs maps option name -> logprob of that option's index token.

    Renormalize over the options only, then take the max.
    Fetching those logprobs is the server-specific part; see the
    privatemode-decisions repo for the vLLM request that does it.
    """
    peak = max(logprobs.values())
    weights = {name: math.exp(value - peak) for name, value in logprobs.items()}
    total = sum(weights.values())
    probs = {name: weight / total for name, weight in weights.items()}
    choice = max(probs, key=probs.get)
    return choice, probs

What the sketch leaves out, on purpose, is everything their post says you have to get right on the wire: which token id spells "12", which response field still contains that id after a vocabulary mask, and how the assistant prefill is flagged so the server does not throw it away.

The vLLM details that change the probabilities

They implemented the steps for GLM-5.3-Flash on vLLM inside Privatemode. Four details are load-bearing. Naming them is enough to implement against their repo. Linking a vendor marketing page is not.

allowed_token_ids masks the vocabulary down to the option-index tokens and sets every other logit to negative infinity. They set it. They also call it a guardrail, which means the method still makes sense if you renormalize over the option ids yourself and the mask is absent. The mask stops the model from emitting a comma or a newline as the one token you asked for. It does not create the distribution.

top_logprobs is the distribution before that mask. A leading space, or some other formatting token, can occupy the top slots. Options that fall off the returned list then look like probability 0, which is a reporting bug, not a model belief. logprob_token_ids returns logprobs for the exact ids you ask for. That is the field they tell you to read. If your gateway only exposes top logprobs, check that every option id is actually present before you treat a missing id as zero.

Digit tokenization is model-specific. GLM-5.3-Flash has a single token for "12". Another model may split "12" into "1" and "2", and then max_tokens: 1 cannot name option 12 at all. They tell you not to ship a local tokenizer and hope it matches the server. Ask the server: a /completions call with echo returns the tokenization of the model that is actually loaded. Their library caches those ids and expires the cache because an alias can move to a different tokenizer. If you pin glm-flash-latest in a client and the weights behind the alias change, every index above 9 is a place the bug hides.

logprob_token_ids on their Privatemode deployment returns at most 128 entries. A question with 151 options is sent twice, identically: 128 ids on the first response, the other 23 on the second, then merged. Both requests run the same forward pass, so the merged row is the distribution one request would have returned, up to the run-to-run noise above. The cost of that workaround shows up in latency, which the CLINC150 numbers below make concrete.

The ceiling past that workaround is the tokenizer, not the API page size. They put it at 191 option indexes that GLM-5.3-Flash spells as a single token. Option 192 is a different project: either a model whose tokenizer keeps larger integers intact, or a scheme that spends more than one token, which gives up the "one forward pass" claim.

How GLM, Jev, and Laya compared on their benchmark

The benchmark lives in edgelesssys/privatemode-decisions-benchmark: methodology, frozen dataset specs, harness, and raw runs. Every number in this section is theirs. explainx.ai did not recompute it.

They compared three systems on 29 public labeled datasets: GLM-5.3-Flash on Privatemode with the one-token setup, TypeSafe Jev, and Convai Laya (421 million parameters, run locally). The sets cover intent routing, sentiment, topic classification, moderation, entailment, question answering, legal text, and scanned documents, in English and German, with 2 to 151 options. All three received the same state, the same option names, the same order, and the same instruction. They did not tune the prompt on these datasets. Jev and Laya ran at default settings. Each dataset was run twice.

The head-to-head on accuracy is the 28 text datasets. GLM and Jev each win 10. On the other 8, the two sit within one percentage point, which their figure caption treats as the same result because that is the scale of run-to-run variation. The median gap is 0.7 percentage points in Jev's favor. The chance estimate is a two-sided Wilcoxon signed-rank test on the per-dataset differences, p = 0.64. They also say two equally accurate systems would show a gap at least that large in about 6 of 10 comparisons. Laya's median gap is 13 to 15 percentage points worse than the hosted pair, with p below 0.001.

Their repository summary compresses a related score: normalized accuracy of 0.585 for the GLM setup, 0.574 for Jev, and 0.422 for Laya, where 0 means always guessing a dataset's most common label and 1 means perfect, averaged across datasets. That is a different aggregation from the win counts. It points the same way. It is still their number.

Option count moves accuracy more than the choice between Jev and GLM. On TREC, going from 6 question types to 42, Jev goes from 92.1 percent to 85.6 percent, GLM-5.3-Flash from 91.2 percent to 79.6 percent, and Laya from 88.4 percent to 51.2 percent. Those three finer-set figures are explicitly ordered Jev, GLM, Laya in the prose.

MASSIVE is the case to be careful with. The task is the same questions labeled twice, once as 18 scenarios and once as 59 intents. They say Jev and GLM scores rose in English and in German at the finer label set, and Laya's score dropped. The figure prints about 83.0, 82.4, and 44.6 for English at the finer granularity, and about 79.2, 77.9, and 22.0 for German. The caption says the numbers on the right are accuracy at the finer granularity. The prose does not assign each of those three floats to a named system in a sentence, so this post will not invent the assignment. The claim that is explicit: Jev and GLM stay close, and Laya falls.

When "faster" is the closer datacenter

Latency was measured separately, one request at a time, because timings under load measure the queue. Privatemode is hosted in the EU and Jev in the US, so they ran four datasets from Germany and from the US at the same time.

From Germany, Privatemode's median is 180 ms, with a band of 173 to 263 ms that they describe as the 10th to 95th percentile. Jev's median is 264 ms, band 235 to 331 ms. From the US the order reverses: Jev at 164 ms (142 to 227) and Privatemode at 299 ms (271 to 402). The model call the user feels includes the network. A procurement slide that crowns one of these "the fast one" without a region is describing a datacenter.

That is the useful correction to a slogan like "Jev is faster." On this measurement, Jev is faster from the US, and the GLM setup is faster from Germany, by roughly the same kind of gap. explainx.ai's fact-check of Jev's speed and cost claims already argued that headline multipliers need a matched task. This benchmark is a matched task, and the latency result is mostly geography.

Laya has no comparable hosted latency or price in their table because they ran it locally. Local is a different product: you pay in hardware and in the accuracy gap above, and you pick up the option-name budget described later. Ollaya is the local runtime that bundles Laya with other decision models if that is the deployment you actually want.

What a million decisions cost, and where the tokens go

On cost, Jev is cheaper in their table. One million text decisions cost about EUR 62 with GLM-5.3-Flash and about EUR 16 with Jev, at each service's list prices, median over the 28 text datasets. Scanned documents are excluded from that pair because Jev cannot read them. About four times is the right verbal summary of 62 against 16. "Several times cheaper" is the phrasing that showed up when people asked why anyone would skip Jev. Both describe the same pair of list prices.

Most of the gap is the price per input token. Some of it is packaging. Jev adds roughly 270 fixed tokens and about 10 per option. The GLM prompt adds about 55 fixed tokens and about 20 per option. Below about 21 options, GLM sends fewer tokens than Jev. Above that, it sends more. The trend they state: GLM starts about 219 tokens below Jev and adds about 11 more per option. Two endpoints of that line are in the post. SST-2, with 2 options, is 127 tokens versus 317. LEDGAR, with 100 options, is 2,040 versus 1,207. The outlier they flag at 13 options is SCOTUS, where long court opinions split differently under the two tokenizers, so a single dataset can leave the trend line without disproving it.

If your option lists are short routing labels, the GLM prompt is the smaller payload and the higher price per token still loses. If your option lists are long, Jev's packaging and its price both point the same way on text. Neither statement is a reason to skip the measurement on your own prompts. Their prompt was not tuned, and a fatter instruction on the GLM side would move the crossover.

Reasoning helps, and it spends the savings

As a control, they let the same GLM-5.3-Flash reason before it answered, on all 29 datasets. It was more accurate in every option-count band. At 2 options the reasoned run is 89.9 percent against 85.5 percent for the one-token setup. Between 21 and 80 options it is 82.0 percent against 79.2 percent. The reasoned run writes hundreds of tokens and costs about EUR 350 per million decisions, against about EUR 62 for the one-token path.

That is the trade the one-token method is for. If a wrong route is expensive and the volume is small, pay for the reasoning trace. If you are classifying every ticket, the one-token path is the one that stays in the same cost band as a decision model, and the reasoned path is a different budget.

A second control, embedding similarity with no decision model, picks the option closest to the text. It landed between 45.9 percent and 72.8 percent depending on the option-count band. It is a real baseline: some of these datasets are easy enough that a nearest-label embedding looks competent. It is also the floor you should beat before you tell yourself the logprob trick is doing work. On the bands where embeddings already sit in the low 70s, a 2-point gap between Jev and GLM is hard to feel in production. On the bands where embeddings sit near 46 percent, the decision procedure is the product.

Images are the capability Jev does not have here

The state can be a picture. GLM-5.3-Flash is vision-capable, so a scanned invoice, a photo of a damaged parcel, or a screenshot goes into the same prompt, and the answer is still one token with a probability per option. On RVL-CDIP, 1,600 scanned business documents in 16 classes, they report 70.2 percent. Jev and Laya are text-only in this comparison, so 70.2 percent is a score only one system posted.

The image adds about 1,350 input tokens. A million document decisions cost about EUR 270. That is a different invoice from the EUR 62 text figure, and it is the honest price of the feature people will cite when they prefer the GLM path. Seventy percent on 16 document classes is useful for a first-pass sort. It is a weak number if a misfiled contract is the failure mode you cannot absorb. Read it as "this system can see, and this is the accuracy they measured," and put a review threshold on the confidence score before you auto-file anything.

Where the method breaks

Option budgets differ by system. Laya's option names share 192 tokens. That fits banking77's 77 intents and does not fit CLINC150's 151. The GLM setup can represent 151 options only by splitting logprob_token_ids across two requests, as described above. On CLINC150 they report GLM at 87.5 percent and Jev at 78.4 percent, with latency of 719 ms against 249 ms. The accuracy edge and the latency penalty are the same workaround. Past 191 single-token indexes, GLM-5.3-Flash as they configured it stops being a one-token decider.

Labels are noisy, so 100 percent is the wrong ceiling. When the systems agree and the dataset disagrees, the label is often one of two defensible answers. They cite banking77 pairs such as get_physical_card versus order_physical_card, and declined_transfer versus failed_transfer. About 17 percent of banking77 examples fall in that category, so the ceiling they state is about 85 percent. A leaderboard gap inside that residue is a labeling argument. Where Jev actually fails is the companion point for the hosted model: a type-valid answer can still be the wrong type-valid answer. The one-token GLM setup has the same shape of error, because the output space was constrained to the options you supplied. Constraint removes malformed JSON. It does not remove a bad taxonomy.

Renaming options is a brittle test, and they say so. They re-ran every dataset with each option replaced by a synonym. On BoolQ, true / false became correct / wrong, and GLM-5.3-Flash lost 20 points. Jev and Laya each lost under 3. They warn that the rename also changes the meaning of the question, so it does not isolate memorization. A 20-point drop is still a practical warning: if your product team renames "payments" to "billing ops" over a weekend, remeasure. The index is what the model emits. The name is what it reads. Those are different strings, and GLM moved more when the strings moved.

The distribution cannot abstain. Because probabilities are renormalized over the options you passed, the model will pick one. Add an explicit "none of these" option when abstaining is a legal answer. Otherwise a confident wrong route looks like a confident right one.

When to use Jev, this GLM setup, or Laya

Use the one-token GLM path when three conditions overlap. You already serve an LLM. The product asks closed questions all day. And at least one of these is true: the input includes images, you want the call to stay on a model you already host, or the option list changes faster than you want to train or retune a classifier. Pointing the same request at any vLLM-served model is the portability they describe. Open weights are the other reason they give for staying on that path.

Use Jev when the workload is text, the volume is high, and the list price matters. On this benchmark the accuracy is tied inside noise, the latency tie goes to geography, and the cost does not. About EUR 16 against about EUR 62 per million text decisions is the gap that survives after you throw out the "faster" claim. Jev remains a hosted system with an unpublished architecture. If that is unacceptable, the GLM setup is the alternative this post is about, and a local runtime is the other.

Use Laya, or a local bundle such as Ollaya, when the decision has to stay on device and the accuracy gap is acceptable. On this benchmark that gap is large: a median of 13 to 15 points, and a collapse as the option list grows (TREC at 42 options, MASSIVE at the finer intents, CLINC150 not representable inside the 192-token name budget). Local cost can still win if your alternative is a metered API and your labels are easy. It does not win by pretending the accuracy table was flat.

Skip all three when the answer is a paragraph, a patch, or a plan. A one-token index cannot be those things. The reasoned GLM control is the reminder that you can spend your way back to a normal chat completion, at about EUR 350 per million, when the decision deserves a trace.

What people asked after the writeup

The Hacker News thread under the post stayed small. The useful objections are about the method.

"Why not just use Jev? It is faster and cheaper." The post's answer, matched to the tables above: speed is similar once you account for region, Jev is several times cheaper on text, and the GLM setup adds vision plus the ability to aim the same trick at any vLLM model. Open weights are the other reason. If your constraint is the EUR per million text decisions and your users sit near Jev's region, that objection wins. If your constraint is a scanned document or a model you already operate, it does not.

"Is Jev just open weights plus post-training?" Treat that as speculation. What is public about the training story, and what is still undisclosed, is the subject of the how Jev works explainer. A rumor about the base checkpoint leaves the list-price gap and the vision gap in the table as they are.

A person who has used Jev said it is sensitive to phrasing and sometimes an outlier against other models, which is not the same as being more correct. That is a usage report. It is not a row in the 29-dataset benchmark, and it should not be graphed next to the Wilcoxon result. It does line up with something the benchmark itself shows on the GLM side: BoolQ's synonym rename moved GLM by 20 points. Phrasing is part of the task. Freeze the option strings the way you freeze a label set.

What explainx.ai did not verify

explainx.ai did not re-run the 29 datasets, did not call Privatemode or Jev for this article, and did not measure the 180 ms or EUR 62 figures independently. The latency bands, the cost medians, the TREC drops, the RVL-CDIP score, and the confidence example are cited from the September 24, 2026 writeup and the benchmark repo that backs it. Where a chart prints three percentages and the prose names the systems in a different breath, this post says so, as with the finer MASSIVE numbers.

What you can check without trusting the leaderboard is the mechanism. Number the options. End the prompt at choice_index:. Request logprobs for those index token ids. Renormalize. Argmax. If your server cannot return the ids you ask for, you do not have this method yet, regardless of how the benchmark came out.

Related reading

• How Jev works: RLCD and the single forward pass
• TypeSafe's Jev launch and the System One framing
• What a System One Model is
• Jev's speed and cost claims, fact-checked
• Where Jev actually fails
• Ollaya, a local runtime for decision models
• Laya on MLX, an on-device Jev alternative
• Primary sources: Privatemode, "Turn GLM-5.3-Flash into a Jev-like System One model" · privatemode-decisions · privatemode-decisions-benchmark

Benchmark figures, latency bands, and list prices in this piece are Privatemode's, as of September 24, 2026. explainx.ai did not re-run the 29 datasets. Tokenizer limits and API field names follow that writeup and the privatemode-decisions repository; confirm them against the server you actually call before you depend on a 191-option ceiling or a 128-id page size.