On November 10, 2026, Hugging Face published Decision Models in Hugging Face, standardizing how agents route tickets, score urgency, and estimate yes/no risk without begging a chat model for valid JSON.
The post lands as Fastino GLiDE competes on the Decision Index and labs ship System-One style models like Jev. Hugging Face''s move is infrastructure: one /v1/systemone contract across Transformers and llama.cpp.
TL;DR
| Piece | What it does |
|---|---|
| State + questions | Input context and named decision fields |
| choice / score / noul | Typed primitives with probabilities |
| DecisionPipeline | pipeline("decision", model=...) in Transformers |
| systemone HTTP API | Shared JSON schema on ports 8000 (Transformers) or 9931 (llama.cpp) |
| Compatible LLMs | Label-logit scoring without token generation |
The problem with JSON classification prompts
Support routers often prompt an LLM: "Return JSON with department and urgency." That pattern breaks in production:
- Models hallucinate keys or ignore enums.
- Constrained decoding (Outlines, XGrammar) fixes syntax but still burns tokens generating braces.
- You only needed probabilities over five departments.
Hugging Face''s blog contrasts that with autotrust/GEV-26B-Decide, returning billing at 0.998 probability with zero output tokens in the REST example.
That is the same economic story as semantic caching and token inflation: stop paying generation prices for classification work.
Primitives: choice, score, noul
Choice
Pick one label from a runtime-defined set. Probabilities must sum to 1. Confidence normalizes how far the top probability sits above uniform guessing.
Use for intent routing, model selection, or tool allowlists.
Score
Ordered scales (low / medium / high) return an expectation over indices, not just the modal bin. A distribution {low: 0.10, medium: 0.26, high: 0.72} yields score 1.7, communicating lean-high urgency even when high is not certain.
Use for priority queues where stakeholders want graduated SLA, not binary flags.
Noul
Binary questions return Bernoulli parameter p for Yes. The application sets thresholds (escalate if p > 0.9).
Use for refund detection, PII presence, or safety triggers.
Soft green bars forming a discrete probability distribution
Versus classification and zero-shot pipelines
| Approach | Labels at runtime | Multiple questions | Native ordered scores |
|---|---|---|---|
| Fixed-head classifier | No | No | Rare |
| Zero-shot NLI | Yes | Awkward | Hacky |
| Decision model | Yes | Yes | Yes |
Zero-shot models like BART-MNLI already score entailment per label, but they lack a multi-question schema and confidence semantics tuned for agents.
Transformers serving path
Install serving extras:
pip install "transformers[serving]"
transformers serve
POST to http://localhost:8000/v1/systemone with state, questions, and criteria objects. Responses nest answers with typed payloads identical to the Python pipeline("decision") example in the blog.
AutoModelForDecision and DecisionConfig mirror other Transformers heads, so Hub models can declare model_type: jev (or successors) and inherit standard loaders.
llama.cpp parity for edge routing
The blog runs:
llama serve -hf ggml-org/Kev-4B-GGUF
Then POST to port 9931. Interface parity matters for local agent stacks where a 4B decision model routes traffic to 70B generators.
Pair with vLLM vs SGLang only on the generation tier; decision models should stay small and colocated with your queue consumer.
Training ecosystem and benchmarks
Hugging Face argues multiple architectures (joint schema heads, LLM readouts, edge variants) can coexist if they honor the same JSON contract. That unlocks:
- Hub cards listing supported primitives per model.
- llama.cpp and Transformers consuming the same GGUF or safetensors weights.
- Benchmarks like GLiDE''s Decision Index targeting interchangeable runtimes.
Read our open-weight licenses explainer before you ship autotrust/GEV-26B-Decide weights — some decision checkpoints carry research-only terms.
When to adopt decision models in your stack
- High-volume routing (support, fraud, moderation) where JSON validity is on your critical path.
- Multi-field triage on one ticket (department + urgency + refund) without three separate prompts.
- Edge deployments that cannot afford 2k-token JSON examples in context every call.
Keep full LLMs for customer-facing prose. Use decision models as typed sensors feeding Bedrock AgentCore or Raindrop memory.
Limits to plan for
- Calibration: High confidence does not mean correct — monitor offline confusion matrices.
- Schema drift: Changing criteria strings shifts logits; version your criteria like API fields.
- Multimodal states: Blog focuses on text JSON states; verify whether your candidate model supports images before routing screenshot tickets.
Confidence math: when 0.99 is not "certain"
The blog defines choice confidence as a normalized distance from uniform guessing. That means 0.98 confidence can still leave 2% mass on dangerous wrong departments — fine for triage, not for autonomous refunds.
Set business thresholds per primitive:
| Primitive | Example threshold | Action |
|---|---|---|
| choice | confidence < 0.6 | Route to human |
| score | expectation > 1.8 on 0–2 scale | Escalate SLA |
| noul | p > 0.95 | Auto-tag fraud review |
Log calibration curves monthly. Decision models can be overconfident on tail prompts just like generative models hallucinate.
Migration guide from JSON-router prompts
Week 1: Shadow-run decision models alongside existing LLM JSON routers. Compare disagreements.
Week 2: Replace department classification only; keep generative model for customer replies.
Week 3: Add score primitives for SLA fields previously parsed from strings.
Week 4: Decommission JSON-schema prompts and delete regex validators that no longer run.
Track p99 latency. Decision models should cut time-to-first-token to near single forward pass latency, especially on Kev-4B-GGUF class weights.
Using compatible LLMs without a decision head
Transformers can score label logits from autoregressive weights without emitting JSON. That is ideal when you already fine-tuned Qwen or Llama on support tickets but lack a Jev checkpoint.
Caveat: label strings must appear in tokenizer vocabulary with stable token IDs. Weird subword splits skew probabilities.
Hub discovery and licensing
The blog teases Hub cards listing decision support. Until search filters land, grep Hub for AutoModelForDecision compatibility and read license tags (guide).
GEV-26B-Decide is powerful but heavy for edge routers — start with Kev-4B for colocated queues.
Observability: log distributions, not only argmax
Store full probability maps in your event bus (redacted for GDPR). Analysts can replay why billing won vs technical, improving prompts and criteria text without retraining.
Pair with Bedrock tokenomics lake patterns if you host routers on AWS and need per-department cost.
Testing matrix before production
| Test | Pass criteria |
|---|---|
| Criteria rename | Change label string; ensure probabilities shift smoothly |
| Adversarial state | Prompt injection in ticket body does not flip department |
| Missing field | Model returns calibrated uncertainty, not crash |
| Load test | 1k RPS on 4B router without GPU OOM |
Relationship to agent memory and tools
Decision outputs make clean tool triggers: if department=billing, load billing MCP tools only. That complements Raindrop memory where structured state beats long chat logs.
For Felony Bench class incidents, never auto-act on noul fraud signals — use them to prioritize human review.
Origins: Jev, GLiDE, and the Decision Index
TypeSafe AI''s Jev popularized System-One routing. Fastino''s GLiDE pushed Decision Index leaderboard claims (our claim check). Hugging Face''s November post does not pick a winner — it standardizes wires so any checkpoint can compete on application ergonomics, not secret JSON prompts.
That is the same playbook Hugging Face used for diffusers and transformers pipelines: opinionated UX, pluggable weights.
For education teams, decision models also simplify grading rubrics in AI bootcamp exercises: students inspect probability vectors instead of parsing malformed JSON at 2 a.m.
When to keep generative models in the loop
Decision models should not write customer-facing empathy paragraphs. Keep Fable / Opus / Carbon generators downstream, fed by structured decisions (department, urgency, refund likelihood).
This separation of concerns reduces alignment failures where a classifier tone-deafly apologizes while labeling a ticket low urgency.
FAQ deep dive
Is systemone the final API name? Hugging Face uses /v1/systemone today, evolving from Jev / System One naming. Pin versions in clients.
Can I mix decision and chat models in one request? Architecturally yes: decision pass routes, chat pass composes. Avoid single models doing both without latency budgets.
Does output_tokens = 0 break billing meters? Some providers bill input-only for classifier passes — confirm with your vendor.
What about multilingual tickets? Test criteria translations; label descriptions shift logits. Maintain per-locale calibration.
Will OpenAI or Anthropic ship native decision endpoints? Unknown. Hugging Face''s open contract lets you swap hosts without rewriting app code if they do.
Export decision logs to your data warehouse with the same retention policy as chat logs. Regulators may ask why you routed a healthcare ticket to sales, not only what the model said afterward. Store criteria text versions next to every log row for reproducibility. When you A/B criteria wording, run chi-squared tests on department distributions before declaring winners; small datasets lie. Publish calibration dashboards to support leads so they trust automation enough to stop manual double-triage. Decision APIs fail gracefully when criteria are malformed; add JSON schema validation in your API gateway. Version your state JSON shapes like any other public API. Breaking changes deserve semver tags in your router service. Announce deprecations in your internal changelog. Support engineers should read decision logs before they re-open tickets with generative rewrites.