Microsoft Amplifier UI-Use and Decision Models: A Bounded Browser Agent With Jev and Decision-1 Backends

Microsoft has put four new repositories into the open under its Amplifier agent ecosystem, and the two main ones read like a design document for cautious browser agents. amplifier-bundle-ui-use adds a bounded controller for browser and computer use, and amplifier-lib-decisions defines typed contracts for "decision models" that answer constrained questions. Provider repos for Jev and local decision services complete the set. All carry an experimental label, MIT license and, when we looked on October 22, 2026, two commits and zero stars.

That makes this a release to read for ideas, not one to deploy. It is also a notable data point: a big-tech lab shipping agent-control code that assumes the page, the model and the user can each be wrong. Below we explain the design, the decision-model angle, the limits Microsoft lists itself, and where it fits among recent work on Jev decision models and local decision runtimes.

TL;DR: the four repos

How the UI-use loop works

The README describes a loop that runs per iteration: observe the page, check whether the goal is already met, select an observed candidate, authorize and execute it, then verify the visible result. "DONE alone is never sufficient," it says, meaning a model claiming success must be backed by an observable condition. Callers supply success criteria such as visible text, an exact URL or a field name and value. Step, cycle and time budgets are finite, cancellation propagates, and stale evidence forces re-observation.

Failure handling is deliberately conservative. If the outcome of an action is unknown, the controller stops without replaying it or retrying with a new model. Effects are not rolled back. Repeated ineffective choices are detected even when the page's DOM is replaced; with an ordinary fallback configured, the controller asks it for a different action before dispatching again, and continued repetition returns control to a planner.

The safety design

Several choices stand out as a checklist for anyone building a browser agent:

• Opaque candidate IDs. Model outputs are IDs of observed candidates, not raw selectors, code or coordinates, so a model cannot invent an element or inject script.
• Untrusted observations. Page content is treated as untrusted, which matters for prompt injection. An ordinary model writes text only after a field has been chosen.
• Explicit origins and approval. The local browser creates a private context, requires explicit origins and approval, and rejects unknown candidates. Cross-origin redirects stay blocked. Opening a new origin requires exact URL and origin permission, off by default.
• Recheck after approval. Evidence is rechecked after human approval and again in the same JavaScript turn as the DOM dispatch.
• Privacy in receipts. Public progress receipts omit page contents, screenshots and typed text.
• No ambient capability. Installation opens no browser and performs no inference, and the default include installs no browser SDK, decision SDK or desktop executor.

For related risks, see our MCP security guide and the Pixelleak post on agents and GitHub screenshots.

What decision models add

amplifier-lib-decisions is a small library for "bounded predicate, choice and ordinal score requests." A DecisionRequest supplies one state and named typed questions, and each provider must see explicit allow_external_state=True before sending state anywhere. Results preserve the requested and reported model, usage, request ID, latency and a per-question outcome: answered, refused, invalid, unavailable or abstained. Missing answers stay missing, and "providers cannot invent execution authority."

The library imports no vendor SDK, owns no credentials or endpoints, and lists three initial implementations: OpenAI Decisions, TypeSafe Jev and Microsoft Decision-1 through the Azure OpenAI provider. A shared System One codec serves Jev and Decision-1. Two honesty notes in the README are worth quoting in spirit: confidence is provider specific and not calibrated across vendors, and a model alias is not an immutable checkpoint. For self-hosted setups, a zero API charge excludes hardware and electricity, and does not erase hosted planning or fallback costs.

In the UI-use bundle, a decision selector can replace ordinary model selection for choosing which observed element to act on. Fallback to ordinary selection is allowed when a backend is missing or abstains, but a model's "blocked" vote ends the task unless the caller explicitly recovers. Thresholds of probability 0.8 and margin 0.15 are called experimental and vendor specific. If you want background on why single-choice classifiers matter here, our Jev versus XGBoost and BERT comparison and the Respan SPAN-01 behavior classifier cover the model side.

Limits Microsoft states itself

The README is unusually frank. DOM observation covers the visible top document, labels and controls, not the full accessibility tree, and frames and shadow DOM need later work. Dispatch uses DOM click, input and change events, not trusted native input, so some apps will not respond. Truncated coverage prevents decision selection. Downloads, popups and dialogs are not interactive workflows. Visible success criteria do not prove hidden business correctness. The computer lane is limited to local X11 with OCR (Tesseract) and click or wait actions.

On evaluation it advises reporting verified success before latency, preserving failures, comparing on identical fixtures and executors, and noting that a held-out task used for debugging is no longer untouched. The included triage fixture, a standalone support-ticket exercise needing at least 24 actions, is described as a development exercise, not a held-out benchmark.

How to try it, carefully

The README gives an installation shape: add behaviors/ui-use.yaml to an Amplifier host, install the optional browser and decisions extras with uv, and run Playwright's browser install. A minimal tool configuration lists allowed origins and selection mode standard. Decision mode requires a provider to register a named backend, and the tool's standard mode always permits an ordinary run. Our advice: use a disposable environment, an allowlist containing only a local test origin, human approval on, and the triage fixture as your first task. Record the installed revision, as the README itself recommends.

Where it sits in the decision-model landscape

Decision models, small systems that return a constrained answer such as yes, no or a pick from a list, have become a theme this month. Our comparison of Clairic, Jev, OpenAI Decisions, Clef and Microsoft Decision-1 lays out the vendors. What the Amplifier libraries add is a neutral wrapper: one request and result shape that any of those providers can implement, so a consumer such as the UI-use controller binds to a named backend rather than to a vendor.

That design has a practical benefit. Because the library owns neither credentials nor endpoints, switching provider is a configuration change, and because every result records the requested and reported model, an audit can see when an alias silently pointed to a different checkpoint. The README warns that an alias is not an immutable checkpoint, and self-hosted providers can attach a transport-verified checkpoint and explicit cost metadata.

Design lessons for your own agent

Even if you never touch Amplifier, five ideas transfer to any browser or desktop agent:

1. Constrain the action space. Let the model pick from an enumerated list of observed candidates instead of writing selectors or code.
2. Verify, do not trust. Require an observable success condition for every task and never accept the model's own "done."
3. Stop on uncertainty. When you cannot tell whether an action took effect, halt instead of replaying it, since replaying a payment or deletion is worse than pausing.
4. Bound everything. Cap steps, cycles, time and internal model calls, and make cancellation propagate.
5. Keep receipts private. Log what happened without storing typed text, screenshots or page content that may contain secrets.

What we did not verify

We read the READMEs for ui-use and lib-decisions only. We did not run the code, inspect the Jev or local-decisions provider repositories in detail, or test any claim against a live site. Nothing in the READMEs reports accuracy numbers, so there is no benchmark result to repeat here, and Microsoft has not published an announcement tying the repos to a product. Treat the post as a reading of intent and design, and check the repositories for changes, since commit history is still only two entries deep.

What people are asking

Is this Microsoft's answer to computer-use agents from Anthropic and OpenAI? The repos do not claim that. They provide a controller and contracts inside Amplifier; whether Microsoft ships a product on them is unstated.

Why decision models for UI tasks? Choosing among observed elements is a bounded classification problem, where a small constrained model can be cheaper and more auditable than free-form generation. That is a hypothesis the README treats as experimental.

Can I rely on the star count? Zero stars and two commits just reflect a brand-new repo from a major organization, not low quality or interest; they do mean no community track record yet.

Related reading

• Jev prompt injection detector
• Ollaya: Ollama for decision models
• Jev vs XGBoost and BERT classifiers
• Respan SPAN-01 behavior classifier
• MCP security guide
• Claude background computer use
• Pixelleak: agents and GitHub screenshots

Details come from the READMEs on October 22, 2026; interfaces are explicitly subject to change.