Microsoft Mage-VL-Cowork: A 4B Open Model That Clicks and Codes, 66.0% on OSWorld-Verified

Microsoft has put a new open-weight computer-use agent on Hugging Face: Mage-VL-Cowork, a 4B-parameter model that decides turn by turn whether to click through a screenshot or to run code in a terminal. Microsoft's model card reports 66.0% on OSWorld-Verified under a 50-step budget, which it describes as the best result among 4B models, and the weights are released under Apache-2.0.

The idea is simple and practical: most desktop agents work only through pixels, so every screenshot burns thousands of tokens and a spreadsheet edit turns into dozens of mouse clicks. Mage-VL-Cowork keeps the GUI for visual work and drops to a shell for structured edits. This post walks through what Microsoft published, the numbers and where they come from, how to try it safely, and what the release does not prove.

TL;DR: the questions people ask first

What Mage-VL-Cowork actually is

The model card calls it "a full training recipe to learn hybrid computer use from evolved harnesses." Underneath, it is a fine-tune of the intermediate Mage-VL 4B checkpoint, which pairs a from-scratch vision encoder (Mage-ViT, a 16 by 16 patch grid with 3D RoPE) with a Qwen3-4B language backbone. Microsoft lists a 32,768-token active context window, with the architecture supporting up to 262,144.

Each turn the model emits a three-part structured output: a <think> rationale, a one-line Action: summary, and a <tool_call> that is either a GUI action in normalized [0, 999] coordinates or a bash command in a persistent shell and Python environment. There is no external router or second model deciding which channel to use. That decision is learned.

To keep long tasks affordable, the card describes an asymmetric working memory. The three most recent screenshots and four full responses stay in context, while everything older is folded into single-line Action: summaries placed after the system prompt. If you have used coding agents that compact history, this is the same idea applied to screenshots.

How was it trained?

Microsoft's recipe has three parts, and the second one is arguably the most reusable.

Microsoft Mage-VL-Cowork 4B computer use model scoring 66.0% on OSWorld-Verified against larger models Image: Microsoft

1. Evolve the teacher harness before collecting data

A strong teacher does not automatically produce clean training data. Microsoft used an authoring agent to refine the scaffold around the teacher model: the tool interfaces, system prompt and pre-installed terminal environment. It removed overlapping tools such as xdotool and verbose accessibility-tree dumps and pre-installed Python libraries such as python-pptx, openpyxl, pandas, pypdf and pillow.

The card reports a controlled comparison with the GPT-5.4 teacher fixed, on 672 held-out multi-application tasks. The pruned GUI-plus-coding harness gave the teacher 92.0% success in 25.0 mean steps, against 89.6% in 35.33 steps with accessibility data added. The bigger effect showed up in the student: a 4B model trained on the cleaner trajectories reached 37.6%, against 25.1% for the one trained on the cluttered harness, a 12.5 point gap. The lesson, in Microsoft's words, is that "more tools do not automatically make a better agent", especially for small students.

2. Mage-CUA-Forge: workflow-first task synthesis

Instead of running an open-ended agent loop to invent each training task, Mage-CUA-Forge writes a reusable generation workflow once per atomic skill. Per the card, three student-feedback rounds produced 293,806 tasks across 6,915 atomic skills (38.24% multi-application, 278 file categories) and kept 318,837 verified hybrid trajectories, 68.19% GUI tool calls and 31.81% terminal calls.

On cost, Microsoft says that with the same generator model the workflow-first approach cut tokens per task from 1,276K to 85K (down 93.3%) and recurring spend from $0.906 to $0.158 per task (down 82.6%). That is a claim about Microsoft's own pipeline, but it is a useful data point for anyone building synthetic agent environments.

Mage-CUA-Forge training pipeline that evolves the teacher harness for a hybrid GUI and terminal agent Image: Microsoft

3. A five-stage specialize-then-unify curriculum

Rather than learning shell access as a workaround for weak visual grounding, the model first builds two strong standalone skills and then merges them:

Total nominal compute was 7.79 ZFLOPs on 32 NVIDIA B200 GPUs plus 1,120 CPU sandbox slots for the RL stage. Progress on OSWorld-Verified went from 45.56% after GUI SFT, to 55.84% after hybrid SFT, to 66.04% after online RL. The RL stage also made the agent leaner: model calls per task fell from 21.57 to 18.77 and output tokens from 7,374 to 5,499.

The numbers, and where they come from

All of the following are from Microsoft's model card, not from an independent evaluation. The 66.0% is the average of two full passes (65.33% and 66.75%) on 361 OSWorld-Verified tasks, excluding Google Drive tasks, using Microsoft's own MAGEHybridAgent harness at 1920 by 1080 resolution.

Two things stand out. First, the "best at 4B" claim is true within the table, but a 9B model (ScaleCUA-Qwen3.5-9B at 68.7) scores higher, and frontier proprietary models sit 9 to 13 points above. Second, 66.0% is still below the 72.4% human reference in the same table, so read the whole table, not the headline. Its strongest categories are OS tasks (87.5%) and multi-application workflows (62.7%), where the card says shell access turns long click sequences into short, verifiable ones.

On the browser side, the card reports 58.6% on WebVoyager-Static, a deterministic offline version of the benchmark with 643 tasks across 15 sites, graded on both navigation traces and backend database state. That score is for the SFT checkpoint before RL, and it trails the browser-only specialist Fara1.5-4B at 68.0%. Microsoft's point is that the generalist covers both full desktops and web in one small model.

The efficiency story

Against GPT-5.4-mini, Mage-VL-Cowork is less accurate (66.0% against 69.6%) but cheaper to run per task: similar interaction turns (18.77 against 18.24 calls) with about 55.6% fewer output tokens (5,499 against 12,397). If you are running thousands of automation tasks on your own hardware, that ratio may matter more than three points of accuracy.

The card also shows two side-by-side demos against Qwen3.6-27B in GUI-only mode. In LibreOffice Calc, merging ranges, labeling, adding a chart and saving took Mage-VL-Cowork 39 seconds in 6 steps by manipulating the workbook with Python, against 9 minutes 56 seconds in 39 steps for the 27B model. In GIMP, the 4B model finished in 1:58, while the 27B model stopped at 7:02 with a file that failed verification. These are two hand-picked demos, so treat them as illustrations of the mechanism, not as averages.

How to try it

Microsoft's quick start uses SGLang, and there is a pinned container (docker.io/xiaoyizhang/mage-vl-cowork:0.1.0 with SGLang 0.5.2 and Transformers 5.7.0). A minimal Transformers setup, per the card, is:

pip install "transformers>=4.57.3" accelerate pillow torch torchvision qwen-vl-utils

Then load microsoft/Mage-VL-Cowork with trust_remote_code=True. The model uses a custom llava_onevision2 architecture with bundled modeling files and a Qwen2.5-VL-style processor, so do not assume it drops into standard Qwen3-VL classes. The repo also contains an osworld_eval/ folder with scripts to reproduce the OSWorld-Verified run, which is the first thing worth doing if you plan to rely on the number.

Practical advice from the way the card is written:

1. Reproduce before you trust. Run the provided OSWorld harness on a small slice and compare with the reported 65 to 67% band.
2. Sandbox everything. The agent has a persistent shell. Give it a disposable VM or container, no saved logins, no network access you do not need.
3. Give it the libraries it was trained with. The training environment pre-installed openpyxl, python-pptx and friends, so the shell channel works best when those exist.
4. Expect English-only behavior. The Hugging Face metadata lists English as the language.

If you are building agents rather than benchmarking them, our guides on loop engineering for coding agents and agent harnesses cover the surrounding scaffolding that Microsoft's own result suggests matters as much as the model.

What the release does not tell you

• No independent verification yet. The model went up on October 30, 2026 with almost no downloads, and the numbers are the authors' own. OSWorld has already had scoring disputes across the field, so wait for third-party reproductions.
• A 50-step budget and a custom harness. Scores depend heavily on the harness. Microsoft's MAGEHybridAgent settings (three screenshots, three history turns) are part of the result.
• Safety is mostly your job. A model that can run bash on your desktop can delete files or exfiltrate data if prompted by hostile page content. The card includes a Responsible AI section, but a benchmark pass rate is not a security guarantee. For agents that act on real machines, add a policy layer such as AgentBeam, the agent security platform from the explainx.ai team that blocks risky agent actions before they run.
• Headline tables mix sources. Some rows are reproduced, some "reported scale", some one-run evaluations. Compare like with like.

Where it fits among open computer-use agents

October has been busy for open GUI agents. Alibaba's OpenGCUA stack and the MAI-Gym 9B GUI agent RL platform focus on generalist training and reinforcement-learning environments, OpenMobile 2 targets phones, and Work2Skill turns human demonstrations into reusable skills. On the product side, Cursor now drives desktop apps and the iOS simulator and Microsoft itself is pushing local and cloud hybrid agents in Windows.

What Mage-VL-Cowork adds is a concrete, reproducible argument that a small model can match much larger GUI-only agents if it is allowed to use the terminal, and that cleaning the teacher's toolset can matter more than adding tools. If you are comparing small models more broadly, see our small models comparison.

What this means for builders

For people building agents, three takeaways stand out. First, channel choice is a trainable skill: if your agent has both a GUI and a shell, measure how often it picks the right one rather than assuming a frontier model will. Second, token cost per task is a first-class metric next to success rate, and RL on verifiable rewards visibly improved it here. Third, an Apache-2.0 4B model that fits on one GPU makes it realistic to run private desktop automation without sending screenshots to a cloud API, which matters for regulated workplaces.

For everyone else, the story is that computer-use agents are moving from giant proprietary models toward small open ones that run on a workstation. Whether this particular checkpoint holds up outside Microsoft's harness is the open question, and the next few weeks of community tests will answer it.

Related reading

• Alibaba OpenGCUA: open stack for generalist computer-use agents
• MAI-Gym 9B GUI agent RL platform
• OpenMobile 2: open mobile agent stack
• Cursor computer use for desktop apps
• Microsoft Windows hybrid intelligence
• Work2Skill: demonstrations to reusable agent skills
• Small models comparison, October 2026

Details reflect the Hugging Face model card as of October 30, 2026. Benchmarks are self-reported by Microsoft and may change as the card is updated.