Back to list
Nov 02, 2026
3.4K
0
0
Open Source

Microsoft Mage-VL-Cowork: 4B Computer-Use Model, Apache-2.0

Microsoft's 4B Mage-VL-Cowork joins GUI actions and bash in one policy under Apache-2.0, reporting 66.0% on OSWorld-Verified (50 steps).

#Microsoft#Mage-VL-Cowork#Computer Use#Vision-Language Model#Open Source
Microsoft Mage-VL-Cowork: 4B Computer-Use Model, Apache-2.0
AI Summary

Microsoft's 4B Mage-VL-Cowork joins GUI actions and bash in one policy under Apache-2.0, reporting 66.0% on OSWorld-Verified (50 steps).

Introduction

On October 30, 2026, Microsoft's Mage team published Mage-VL-Cowork on Hugging Face, a 4B-class vision-language model for computer use released under Apache-2.0. Its defining design is one policy with two action channels: screenshot-driven GUI actions (computer_use) and persistent shell and Python execution (bash), with no external orchestrator deciding between them. The model card reports 66.0% on OSWorld-Verified, measured on 361 tasks with a 50-step budget, and calls that the best result among 4B models.

Those figures come from the authors, not from a third party. This review keeps the evaluation conditions attached to each number and checks the model card's own comparison table, because the headline reads differently once you see the neighboring rows.

Feature Overview

One policy, two channels. The GUI channel offers 11 primitives (click, double_click, triple_click, move, drag, type, keypress, scroll, wait, screenshot, terminate) in normalized 0 to 999 coordinates. The bash channel runs shell commands and Python, and returns text without using up an image slot. Each turn is a structured <think> rationale, a one-line Action: summary and a tool call. In the card's example, asked to merge spreadsheet cells in an existing .xlsx file, the model chose openpyxl through bash instead of opening LibreOffice Calc. Its working memory keeps the 3 most recent screenshots and 4 full responses, and folds older turns into one-line action summaries.

PropertyValue (model card unless noted)
BaseAn intermediate Mage-VL 4B checkpoint; Mage-ViT encoder with a Qwen3-4B-Instruct-2507 backbone
Size9.48 GB in BF16 across 2 safetensors shards; Hugging Face metadata lists 4,741,793,792 parameters
Context32,768 tokens in active use; architecture supports up to 262,144
LicenseApache-2.0 for the model; the linked GitHub repo (microsoft/Mage) shows an MIT license in GitHub metadata
Training compute7.79 ZFLOPs nominal over 5 stages; 32 NVIDIA B200 GPUs plus 1,120 CPU sandbox slots for RL

Training recipe. The card describes five stages: GUI-oriented midtraining, GUI agent SFT, coding and terminal SFT, hybrid SFT, and online hybrid reinforcement learning (asynchronous GRPO with verifiable rewards on 1,988 tasks). Its data engine, Mage-CUA-Forge, synthesizes 293,806 tasks across 6,915 atomic skills and retains 318,837 verified hybrid trajectories, of which 68.19% of tool calls are GUI and 31.81% terminal. The authors report that pruning overlapping tools from the teacher's harness (such as xdotool and accessibility-tree dumps) lifted a 4B student from 25.1% to 37.6% on 672 held-out multi-application tasks, using a GPT-5.4 teacher. They also report that the workflow-first synthesis cut recurring cost from $0.906 to $0.158 per task with the same generation model. On OSWorld-Verified, the stages move from 45.56% (GUI SFT) to 55.84% (hybrid SFT) to 66.04% (after RL).

OSWorld-Verified, with conditions. The protocol is 361 held-out tasks excluding Google Drive, a 50-step budget, the MAGEHybridAgent harness and a 1920x1080 desktop. The 66.0% is the average of two full passes, 65.33% and 66.75%. The table below copies selected rows from the card, with columns built from its header (Params, Steps, then the five task groups and Overall).

ModelParamsStepsOSOfficeDailyProfessionalWorkflowOverall
Mage-VL-Cowork4B5087.564.965.066.362.766.0
ScaleCUA-Qwen3.5-9B9B5079.270.674.389.548.168.7
GPT-5.4-mini (single run)not listed5079.268.076.669.463.369.6
Fara1.5-4B4B5025.07.629.512.26.613.9

Three readings follow. First, the 66.0% does not top the table: the 9B open model lists 68.7% and the GPT-5.4-mini reference lists 69.6%, while the card's human reference is 72.4. The "best among 4B models" claim holds against the one other 4B entry in that table, Fara1.5-4B at 13.9%. Muse Glimmer-30B lists 65.9 overall with no step count. Second, the 87.5% OS score is the highest in that column and the 62.7% workflow score trails GPT-5.4-mini's 63.3. The card gives no per-category task counts, so per-column scores should be read with caution. Third, other baselines in the table use a 100-step budget, so not every row is directly comparable.

Efficiency and demos. After RL the model averages 18.77 model calls and 5,499 output tokens per task, against 7,374 tokens for the hybrid-SFT stage. The card's GPT-5.4-mini (xhigh) reference averages 18.24 calls and 12,397 tokens, so the authors say Mage-VL-Cowork uses 55.6% fewer output tokens at a similar number of turns. In an author-selected side-by-side against Qwen3.6-27B working GUI-only, a Calc task finished in 0:39 over 6 steps versus 9:56 over 39 steps, and a GIMP task finished in 1:58 while the 27B run stopped at 7:02 with a failed verification.

Usability Analysis

This is a research-grade agent, not a drop-in assistant. The model uses a custom llava_onevision2 architecture, so loading requires trust_remote_code=True with transformers 4.57.3 or newer. The card's recommended serving path is an SGLang Docker image published under the Docker Hub namespace xiaoyizhang rather than a Microsoft-branded registry, so teams may prefer to build their own container from the card's direct SGLang command. The example uses one GPU, bfloat16 and a 32,768-token context, with SGLANG_MAX_PIXELS set to preserve full 1080p screenshots. The 66.0% configuration uses temperature 0.7, top_p 0.8 and a 4,096-token response cap, along with the card's verbatim system prompt.

Safety guidance is explicit. The card says the agent can run arbitrary shell commands and is designed and evaluated for isolated, disposable sandboxes. It recommends human confirmation for irreversible actions, no mounted credentials, and outbound network allow-listing to limit indirect prompt injection.

Pros and Cons

The strengths are the unified GUI-plus-shell policy, a permissive license with a bundled evaluation harness, and a small download. The limits are scores that sit below larger and closed references in the same table, a browser result that a specialist beats, and the operational risk of an autonomous shell agent. The browser figure (58.6% on WebVoyager-Static) belongs to the SFT checkpoint rather than the final model, and Fara1.5-4B scores 68.0% there.

Outlook

The card positions the work as a step toward on-device agents, and the repository includes the evaluation package for reproducing the OSWorld run. The technical report is published as a PDF in the GitHub repository. What to watch next is independent reproduction of the two-run protocol and behavior on tasks outside the authors' synthetic environments.

Conclusion

Mage-VL-Cowork is a well-documented open recipe for hybrid computer use at a small size, and its channel-routing idea is practical: use code where code is faster and the GUI only where pixels are required. Its 66.0% is a respectable vendor-reported score, not a leading one. It suits researchers and agent builders who can run it in a sandbox and want a compact baseline they can inspect and re-run.

Editor's Verdict

Mage-VL-Cowork earns a solid recommendation as a compact, open baseline for hybrid computer-use agents.

The strongest case for paying attention: a single 4B-class policy that chooses between GUI actions and bash on each turn, with no external router or orchestrator. Reinforcing that, a permissive Apache-2.0 license plus a bundled OSWorld evaluation package, so others can re-run the reported setup — practical value rather than just headline appeal. The broader signal worth registering is straightforward: the distinguishing idea is channel routing inside one policy, so the agent can edit a spreadsheet through Python in bash and use the GUI only where pixels are required. On the other side of the ledger, one constraint is real rather than a marketing footnote: a headline score that trails a 9B open model in its own table (68.7%) and the 69.6% GPT-5.4-mini reference run. It should factor into any serious decision. Layered on top of that, the 4B-class claim comes from the authors' own table, where the only other 4B entry scores 13.9% — which narrows the set of teams for whom this is an obvious yes.

For agent researchers and teams building desktop-automation prototypes who can run it inside isolated sandboxes, this is a serious evaluation candidate, not just a curiosity to bookmark.

Advertisement

Pros

  • A single 4B-class policy that chooses between GUI actions and bash on each turn, with no external router or orchestrator
  • A permissive Apache-2.0 license plus a bundled OSWorld evaluation package, so others can re-run the reported setup
  • A small footprint, with 9.48 GB of BF16 weights in two safetensors shards
  • Lower output-token use than the GPT-5.4-mini reference run (5,499 versus 12,397 tokens per task) at a similar number of model calls
  • A detailed, disclosed training recipe, including task counts, per-stage compute and harness ablations

Cons

  • A headline score that trails a 9B open model in its own table (68.7%) and the 69.6% GPT-5.4-mini reference run
  • The 4B-class claim comes from the authors' own table, where the only other 4B entry scores 13.9%
  • A custom architecture that needs trust_remote_code=True and a pinned SGLang container hosted outside a Microsoft registry
  • An autonomous shell-plus-GUI agent that the card itself says must run only in isolated sandboxes
Advertisement

Comments0

Key Features

1. Released October 30, 2026 on Hugging Face by Microsoft's Mage team under Apache-2.0; 9.48 GB in BF16 (2 safetensors shards) 2. One vision-language policy with two channels: computer_use (11 GUI primitives) and bash (persistent shell and Python), no external router 3. Mage-ViT encoder with a Qwen3-4B-Instruct-2507 backbone; 32,768-token active context; working memory of 3 screenshots and 4 full responses 4. Five-stage curriculum ending in online hybrid RL; Mage-CUA-Forge synthesized 293,806 tasks across 6,915 atomic skills 5. Author-reported 66.0% on OSWorld-Verified (361 tasks, 50-step budget, two-run average), 87.5% on Operating System tasks, 62.7% on Multi-Application Workflows 6. Designed for isolated sandboxes; requires trust_remote_code=True and a custom llava_onevision2 architecture

Key Insights

  • The distinguishing idea is channel routing inside one policy, so the agent can edit a spreadsheet through Python in bash and use the GUI only where pixels are required
  • The reported 66.0% is a two-run average (65.33% and 66.75%) over 361 tasks with a 50-step budget, so the evaluation conditions matter as much as the number
  • The headline score does not top its own table, since a 9B open model lists 68.7% and the GPT-5.4-mini reference lists 69.6%, so the claim is about 4B-class efficiency
  • The frontier proprietary rows in the same table sit well above it, from 73.1% (Kimi-K2.6) to 78.7% (GPT-5.5), so the result is a small-model milestone rather than a frontier one
  • Harness design matters as much as the model, because pruning overlapping tools from the teacher harness lifted the 4B student from 25.1% to 37.6% on 672 held-out multi-application tasks, per the authors
  • The weights are meant for isolated sandboxes, since the card says the agent can run arbitrary shell commands and recommends no mounted credentials, human confirmation and network allow-lists
  • The browser result belongs to the SFT checkpoint, and a browser-only 4B specialist scores higher (68.0% versus 58.6%) on the same WebVoyager-Static benchmark

Was this review helpful?

Share

Twitter/X
Advertisement