Skip to main content
ToolPotion
AI Advisor
Submit a Tool (Free)

Mage-VL-Cowork

LLMs#78AI Models & LLMsOpen source · Web

Mage-VL-Cowork is a 4B parameter multimodal hybrid computer-use agent developed by Microsoft. It integrates screenshot-grounded GUI interaction and persistent code/terminal execution within a single vision-language policy. This model excels at complex computer tasks, demonstrating best-in-class performance for its scale and pushing the frontier of on-device intelligence for autonomous desktop agents.

Visit Website
Share
Mage-VL-Cowork screenshot
Mage-VL-Cowork screenshot

Mage-VL-Cowork is a sophisticated 4B parameter multimodal hybrid computer-use agent developed by the Microsoft Mage Team. It is designed to bridge the gap between pixel-based computer-use agents (CUAs) and those with naive access to both GUI and terminal tools. The model achieves hybrid competence by combining screenshot-grounded GUI interaction (`computer_use`) and persistent code/terminal execution (`bash`) within a single vision-language policy, eliminating the need for an external orchestrator.

Read More

The model's training involves a progressive five-stage specialize-then-unify curriculum, starting from an intermediate 4B Mage-VL checkpoint. This curriculum builds strong standalone visual grounding, GUI execution, and coding/terminal competence before unifying them through Hybrid SFT and asynchronous Online Hybrid RL. A key aspect of its development is the evolution of the teacher harness, which prunes redundant action routes and equips domain libraries, leading to significant student distillation gains.

Mage-VL-Cowork also leverages Mage-CUA-Forge, a workflow-first synthesis engine that generates a vast number of tasks across atomic skills while drastically cutting recurring per-task generation costs. This approach reduces token consumption and expenditure compared to standard agent-first construction methods.

At 4B parameters, Mage-VL-Cowork achieves 66.0% on OSWorld-Verified under a 50-step budget, marking the best performance among models of its size. It demonstrates strong execution on Operating System tasks (87.5%) and Multi-Application Workflows (62.7%). The model's efficiency is further enhanced by online reinforcement learning, which adds significant accuracy improvements while simultaneously reducing mean model calls and output tokens. Its asymmetric working memory allows it to manage long-horizon desktop tasks by compressing earlier turns and retaining recent visual and response data.

Mage-VL-Cowork is released under the Apache-2.0 License and is intended for deployment in isolated, disposable sandbox environments due to its autonomous capabilities. It supports custom multimodal architecture and requires specific configurations for inference, including a unified two-channel contract for GUI and terminal interactions.

  • CategoryAI Models & LLMs
  • Pricing ModelOpen source
  • DeploymentCloud
  • PlatformsWeb

  • Hybrid Computer Use: Combines GUI interaction and terminal execution within a single policy.

  • 4B Parameters: Achieves best-in-class performance at the 4B scale.

  • Multimodal Architecture: Image-Text-to-Text model based on llava_onevision2 architecture.

  • Five-Stage Training Curriculum: Progressive specialize-then-unify approach for comprehensive skill development.

  • Evolved Teacher Harnesses: Amplifies student distillation and improves efficiency.

  • Workflow-First Environment Synthesis (Mage-CUA-Forge): Reduces task generation cost and token consumption.

  • Internalized Turn-Level Channel Routing: Dynamically selects between GUI and bash for optimal execution.

  • High OSWorld-Verified Performance: Achieves 66.0% on OSWorld-Verified tasks.

  • Strong OS & Multi-Application Workflow Execution: 87.5% on OS tasks, 62.7% on Multi-Application Workflows.

  • Online Reinforcement Learning: Improves accuracy by +10.16 pp and reduces model calls/output tokens.

  • Asymmetric Working Memory: Manages long-horizon tasks with folded memory and active visual window.

  • Unified Checkpoint: Handles visual grounding, GUI manipulation, programmatic operations, and channel switching.

  • Apache-2.0 License: Open-source license for usage and distribution.

  • Custom Multimodal Architecture: Uses llava_onevision2 with Qwen2_5_VLProcessor.

  1. Install Dependencies: Set up the required environment, including SGLang, PyTorch, and Transformers.

  2. Serve with SGLang: Launch the GPU inference server using the provided Docker container or a custom environment.

  3. Configure Multimodal Requests: Render prompts with AutoProcessor.apply_chat_template and send base64 image payloads to SGLang's /generate endpoint.

  4. Apply Sampling Parameters: Use recommended settings like temperature 0.7, top_p 0.8, and max_new_tokens 4096.

  5. Implement System Prompt & Tool Schema: Use the exact unified two-channel contract for computer_use and bash.

  6. Perform Offline Transformers Inference: Utilize the model for inference with provided examples.

  7. Reproduce OSWorld-Verified Evaluation: Launch the SGLang runtime and run the OSWorld-Verified evaluation harness.

  8. Deploy Safely: Always run the agent in an isolated sandbox environment with scoped authorization and zero credential exposure.

  • Autonomous Desktop Automation
  • GUI Interaction
  • Programmatic File Editing
  • System Operations
  • Web Environment Interaction
  • Software Engineering Tasks
  • On-Device Intelligence
  • Research and Development

Popular AI Tools Like Mage-VL-Cowork

DeepSeek-V4-Flash-Vision-Exp is an experimental multimodal model in the DeepSeek-V4 family, enhancing visual understanding capabilities. It builds on the DeepSeek-V4-Flash…

AI Models & LLMs

Ranked #70 in LLMs

View Tool

Experimental multimodal model in the DeepSeek-V4 family, enhancing visual understanding capabilities. It builds on the DeepSeek-V4-Flash…DeepSeek-V4-Flash-Vision-Exp is an experimental multimodal model in the DeepSeek-V4 family, enhancing visual understanding capabilities. It builds on the DeepSeek-V4-Flash…

AI Models & LLMs
Ranked #70 in LLMs

LiquidAI LFM2.5-2.6B is a hybrid model designed for on-device deployment, offering efficient inference and agentic capabilities. It supports multiple languages and is optimized…

AI Models & LLMs

Ranked #26 in LLMs

View Tool

Hybrid model designed for on-device deployment, offering efficient inference and agentic capabilities. It supports multiple languages and is optimized…LiquidAI LFM2.5-2.6B is a hybrid model designed for on-device deployment, offering efficient inference and agentic capabilities. It supports multiple languages and is optimized…

AI Models & LLMs
Ranked #26 in LLMs

OmniParser is a method for parsing user interface screenshots into structured elements. It enhances the ability of vision language models like GPT-4V to generate actions on…

AI Models & LLMs
View Tool

Method for parsing user interface screenshots into structured elements. It enhances the ability of vision language models like GPT-4V to generate actions on…OmniParser is a method for parsing user interface screenshots into structured elements. It enhances the ability of vision language models like GPT-4V to generate actions on…

AI Models & LLMs
See the Top 5 AI Models

Command A+ is a Mixture of Experts model with 25B active and 218B total parameters, designed for complex reasoning, vision, and multilingual tasks across 48 languages, providing…

AI Models & LLMs
View Tool

Mixture of Experts model with 25B active and 218B total parameters, designed for complex reasoning, vision, and multilingual tasks across 48 languages, providing…Command A+ is a Mixture of Experts model with 25B active and 218B total parameters, designed for complex reasoning, vision, and multilingual tasks across 48 languages, providing…

FeaturedAI Models & LLMs
See the Top 5 AI Models

Spark-X2.6-4B is a compact, general-purpose language model designed for on-device deployment. It offers improved coding, multilingual, and reasoning capabilities, making it ideal…

AI Models & LLMs
View Tool

Compact, general-purpose language model designed for on-device deployment. It offers improved coding, multilingual, and reasoning capabilities, making it ideal…Spark-X2.6-4B is a compact, general-purpose language model designed for on-device deployment. It offers improved coding, multilingual, and reasoning capabilities, making it ideal…

AI Models & LLMs
See the Top 100 LLMs

MiniMax-M3.1-Flash is a multimodal mixture-of-experts model designed for coding, agentic workflows, and cybersecurity. It supports text, image, and video inputs, offering improved…

AI Models & LLMs

Ranked #84 in LLMs

View Tool

Multimodal mixture-of-experts model designed for coding, agentic workflows, and cybersecurity. It supports text, image, and video inputs, offering improved…MiniMax-M3.1-Flash is a multimodal mixture-of-experts model designed for coding, agentic workflows, and cybersecurity. It supports text, image, and video inputs, offering improved…

AI Models & LLMs
Ranked #84 in LLMs

LLaVA

AI GitHub Repos

LLaVA is a visual instruction tuning model that combines large language and vision capabilities. It aims to achieve GPT-4V level performance, enabling multimodal understanding and…

AI Models & LLMs
View Tool

AI GitHub Repos

Visual instruction tuning model that combines large language and vision capabilities. It aims to achieve GPT-4V level performance, enabling multimodal understanding and…LLaVA is a visual instruction tuning model that combines large language and vision capabilities. It aims to achieve GPT-4V level performance, enabling multimodal understanding and…

AI Models & LLMs
See the Top 250 AI GitHub Repos

gptme is a provider-agnostic, local-first AI agent that runs in any terminal. It offers a powerful CLI for coding, knowledge work, and autonomous agent tasks, supporting numerous…

AI Coding Assistants

Ranked #219 in AI Agents

View Tool

Provider-agnostic, local-first AI agent that runs in any terminal. It offers a powerful CLI for coding, knowledge work, and autonomous agent tasks, supporting numerous…gptme is a provider-agnostic, local-first AI agent that runs in any terminal. It offers a powerful CLI for coding, knowledge work, and autonomous agent tasks, supporting numerous…

AI Coding Assistants
Ranked #219 in AI Agents
1 of 8