A Jev-like LLM wrapper using logprobs — including vision — explained

TL;DR: Developer Allan Riordan Boll published a minimal Jev-like wrapper that scores webcam frames by asking a vision LLM lettered multiple-choice questions with logprobs: true and max_completion_tokens: 1. Hacker News (~101 points) debated whether it beats TypeSafe Jev on cost, latency, and calibration — but agreed the pattern is the real API unlock for intent-style classifiers.

The pattern in one paragraph

Jev formalizes structured decisions: state + questions → probabilities over choices. Boll's hack skips a bespoke decision endpoint and uses any chat model:

1. Format State, Question, and Options [A]…[T] (2–20 options).
2. Instruct: "Answer with the letter of the best option only."
3. Request one output token with logprobs / top_logprobs (OpenAI Responses or Chat Completions; llama.cpp chat).
4. Convert letter logprobs to normalized probabilities; map back to booleans, enums, or ordinal scores.

OpenAI documents similar ideas in their logprobs cookbook; Boll's twist is packaging it like Jev JSON plus attachments for images.

Why logprobs feel like "Star Trek doors"

HN user TeMPOraL joked seriously: sci-fi interfaces infer intent before acting — automatic doors open when the computer is confident you mean to enter, not when you breathe near a sensor.

The logprob wrapper is a crude intent layer:

if person_within_10m:
  if P(intent == "will pass through" | camera_frame) > threshold:
    open_door()

You are not running full chain-of-thought; you are sampling one decisive token and reading alternatives. That is fast enough for ambient UX if you accept model cost and false opens.

Real automatic doors fail on weather, latency, and privacy — same constraints apply when every frame hits a cloud API.

Boll's webcam demo (September 2025–26)

On Allan's blog (posted September 25, 2026), the script:

• Captures OpenCV webcam frames.
• JPEG-encodes to base64 data URLs in attachments.
• Asks parallel questions per frame: person visible?, plant?, indoors/outdoors?, brightness score?
• Runs one background worker so preview stays smooth.

Reported throughput:

Specialized CV models beat this on efficiency; the win is flexibility — change a condition by editing English, not retraining YOLO heads.

API differences he handles

• OpenAI: /responses with include: message.output_text.logprobs, top_logprobs: 20, reasoning.effort: none.
• llama.cpp: /chat/completions with logprobs: true, top_logprobs: 1024, temperature: 0.
• top_p: 1 so pruning does not hide option letters.

Shared state prefix can be KV-cached on backends that support it — critical if you ask many questions per frame.

JSON schema: Jev + attachments

Boll's example payload:

• state: instruction string (or JSON) describing what to judge.
• questions: map of named items with type choice, noul (boolean-ish), or score (ordinal criteria).
• attachments: image paths or data URLs — not in stock Jev docs today; his local extension.

Community project jevper wraps official Jev for OpenAI-compatible hosts; Boll's script is host-agnostic without Jev training.

Normalization edge cases (worth copying)

When an expected letter is missing from top_logprobs:

• If no letter appears, fail loud.
• Otherwise cap missing mass using the lowest returned logprob so omitted options cannot silently win.

That matters for Unicode variants, lowercase letters, or models that prefer "A)" tokens — production wrappers should map token strings to options.

Jev vs logprobs vs Lichen — how to choose

HN jampekka linked Mushroom-Systems/lichen as independent evidence that logprob-style pipelines can win accuracy and speed on Jev's own benchmarks — pushback to "Jev is nothing but an API breakthrough" and subsidy pricing theories.

explainx.ai's deeper Jev internals live in How does Jev work (RLCD) and cheap verification checkpoints — use Jev where calibrated gates matter in agent loops; use logprob wrappers for rapid prototyping and vision until numbers prove otherwise.

Limitations HN surfaced

1. Tail latency — real-time end-of-utterance detection may still prefer Jev's first-token SLA (CROON_tv comment).
2. Calibration — letter logits are not guaranteed well-calibrated probabilities unless you fit or RLCD-train; see where Jev actually fails.
3. Context for doors — still images miss motion intent; V-JEPA2-style video encoders may sit upstream (HN suggestion).
4. Cost at scale — three sequential API calls per frame without batching burns tokens; batch questions only if your provider supports parallel tool-free requests safely.
5. Grammar decoding — some HN readers asked if JSON schema decoding equals Jev; similar structure, different probability readout.

When to use this in production agents

Good fits:

• Router nodes — pick support queue, severity, or tool with explicit options.
• Moderation pre-filters before expensive reasoning.
• Multimodal sanity checks — "does this screenshot contain a payment card?"

Poor fits:

• Legal/medical decisions needing audited calibration without measurement.
• High-frequency control loops on remote APIs.
• Safety-only reliance — pair with injection tests and tool sandboxes.

Batching, KV cache, and cost math

Boll’s demo issues one HTTP request per question per frame. Throughput (~1 FPS on Gemma 4 12B locally) is bounded by:

• Prefill cost — encoding image + long state every time unless the server caches prefix KV across questions.
• Sequential Q — three questions means three prefills if state+image is duplicated and only the question suffix changes.

Production pattern:

1. Put stable instructions and image in a shared prefix; append Question k as the only varying tail (providers differ on whether this hits cache — test with your host’s cache hit metrics).
2. Where the API allows, batch multiple letter-classification prompts that share attachments — some gateways (including Respan-style products) optimize multi-behavior forwards explicitly; logprob DIY does not unless you engineer it.
3. Compare $/decision = (input_tokens + output_tokens) × price / decisions; Jev’s free output and Span-01’s $0.02/M marketing only matter once tokens per gate are fixed.

At ~0.2 FPS on gpt-6-luna, cloud vision logprob scoring is a prototype tool, not a loading dock door controller — latency and egress privacy dominate.

Fireworks, grammar decoding, and jevper

HN noted Fireworks AI grammar support as a way to force JSON or single-letter outputs without manual logprob parsing. That overlaps Jev’s structured API but still may not expose full top-k logprobs for calibrated probabilities.

jevper routes to Jev-compatible backends if you want official response shapes on self-hosted infra. Boll’s script is lower-level — you own normalization bugs when tokenizers split "A" oddly.

Choose grammar when you need schema guarantees; choose logprobs when you need probabilities over a small discrete set; choose Jev / Span-01 when you want someone else to maintain RLCD / RLAIF calibration.

Quick start sketch (conceptual)

You do not need the full webcam loop to try the core idea:

1. Pick an OpenAI-compatible server (http://localhost:8060/v1 for llama.cpp).
2. Send one user message with text prompt + optional image_url content parts.
3. Set max_completion_tokens: 1, logprobs: true, top_logprobs: 20+.
4. Parse top_logprobs[0] for letters A–D.
5. softmax over present letters; pick argmax or threshold noul probability.

For Gemma 4 GGUF + mmproj download URLs and uv run webcam.py, use Boll's post — versions drift; always pin model hashes.

Connection to explainx.ai Jev cluster

• Respan Span-01 vs Jev — hosted behavior classifier launch (Sep 2026).
• Six Jev clones in two days — ecosystem frenzy after Jev GA.
• Kev open-source clone — weights you can self-host.
• LLMs repeat 96% with Jev confident errors — why probability ≠ truth.
• Ollaya — local decision runtime angle.

Bottom line

Allan Boll's logprob Jev wrapper is not magic — it is disciplined use of logits on general models, extended to vision with an attachments field. It democratizes structured decisions the way Jevper democratizes Jev endpoints.

Before you ship Star Trek doors powered by gpt-6-luna, measure latency, churn under lighting change, and cost per million frames — then compare against hosted Jev, Span-01, or fine-tuned small classifiers on your hardware. The winning stack is whichever keeps false opens and cloud spend below your facility manager's patience threshold.

Related reading

• Allan Riordan Boll — Jev-like wrapper including vision (Sep 25, 2026)
• Hacker News — single-function Jev-like wrapper (~101 pts)
• OpenAI logprobs cookbook
• How does Jev work?

Example code and model names come from Boll's blog; verify API fields against your provider's current docs.