Computer Use MCP Server
Let agents see and operate macOS apps on your Mac: read an app's accessibility tree, screenshot its window, then click, type, scroll, drag, paste and set values. Runs locally on macOS 13 or later.
What is the Computer Use MCP Server?
The Computer Use MCP server gives AI agents structured, permission-aware access to Computer Use through the Model Context Protocol. With 11 pre-built actions, agents can read, create, and update Computer Use data on behalf of authorized users.
Willow ships the Computer Use MCP server as part of an enterprise control plane. Every call runs behind SSO (Okta, Azure AD), enforces RBAC and least-privilege at runtime, writes to a full audit trail, and integrates with Splunk and Loki for SIEM visibility. Connect from Claude Desktop, Claude Code, Cursor, ChatGPT, VS Code, n8n, or any custom agent. Install once, distribute org-wide, and see exactly how Computer Use is being used by every AI agent in your stack.
Tools
list_apps
List apps on this Mac: every running app plus the apps installed in the Applications folders. Returns id (bundle identifier), displayName, isRunning and path.
get_app_state
Launch the app if needed, then return its front window's accessibility tree and a screenshot. Each tree line starts with the element_index other tools take. Call this before interacting with an app and again after actions, since indexes only apply to the latest state.
click
Click an element by element_index, or a point by x/y screenshot coordinates. Left single clicks on pressable elements use the accessibility press action; everything else moves the real mouse.
drag
Drag with the left mouse button from one point to another, in screenshot coordinates.
paste
Paste text into the focused field through the clipboard, then restore the user's clipboard. Prefer this for multiline or formatted text.
perform_secondary_action
Invoke an accessibility action other than a plain click, such as "Show Menu", "Increment", "Cancel" or "Raise". Only actions listed for the element under "secondary actions" in get_app_state work.
press_key
Press a key or key combination in the app, using xdotool syntax: "a", "Return", "Tab", "Escape", "Up", "super+c", "ctrl+shift+t", "KP_0".
scroll
Scroll an element (by element_index), a point (by x/y), or the window's center, by a number of pages.
select_text
Select text inside an editable element, or place the cursor before or after it. Use prefix and suffix to pick one of several matches.
set_value
Set the value of a settable element (text fields, sliders, checkboxes) directly, without typing. Elements marked settable in get_app_state accept this.
type_text
Type literal text into the focused element with the keyboard. "\n" presses Return, which may submit a form or send a message.
Customize Tools
Edit descriptions, modify arguments, select tools, or add new ones
Set Up Your Computer Use MCP Server in Minutes
Add the following configuration to your MCP client. Authentication is handled via OAuth. Compatible with Claude Desktop, Claude Code, Cursor, ChatGPT, VS Code, n8n, and any MCP-compatible agent.
Claude Desktop
{
"mcpServers": {
"willow-willow-computer-use": {
"type": "http",
"url": "https://<org>.mcp-s.com/mcp/mcp/willow-computer-use"
}
}
}Cursor
{
"mcpServers": {
"willow-willow-computer-use": {
"type": "http",
"url": "https://<org>.mcp-s.com/mcp/mcp/willow-computer-use"
}
}
}Claude Code
claude mcp add willow-willow-computer-use --transport http https://<org>.mcp-s.com/mcp/mcp/willow-computer-usen8n
{
"url": "https://<org>.mcp-s.com/mcp/mcp/willow-computer-use",
"method": "POST"
}Or click "Install with Willow" above to set up automatically with SSO and RBAC preconfigured.
Enterprise Governance for Computer Use
Willow adds the layer Computer Use and every other SaaS doesn't ship out of the box: every call runs behind SSO (Okta, Azure AD), enforces RBAC and least-privilege at runtime, writes to full audit logs, and detects shadow AI usage across your stack. One MCP gateway. Any agent. Every tool.
Computer Use MCP Server FAQ
What is the Computer Use MCP server?
The Computer Use MCP server is a Model Context Protocol implementation that lets AI agents like Claude, Cursor, and ChatGPT read and write Computer Use data through a standardized interface. Willow hosts and governs this server so enterprises can roll it out without a security review backlog.
How is Willow's Computer Use MCP server different from the official one?
The official Computer Use MCP server is scoped to a single user's account and does not include enterprise governance. Willow's version adds SSO, RBAC, audit logging, shadow AI detection, and centralized control over which actions agents can take across the entire org.
Which AI clients work with the Computer Use MCP server?
Claude Desktop, Claude Code, Cursor, ChatGPT, VS Code with MCP support, n8n, and any custom agent built with OpenAI Agents SDK, LangChain, Vercel AI SDK, or Anthropic SDK.
Is the Computer Use MCP server secure? How does Willow handle authentication?
Every call runs behind your existing SSO (Okta, Azure AD). Per-user OAuth scopes the agent to exactly what that user can do in Computer Use, nothing more. No credentials reach the LLM. Every action writes to an audit trail.
Can I limit which Computer Use actions agents can take?
Yes. Willow lets you scope agents to specific actions, specific projects, or specific environments. Toggle actions on or off in the dashboard, or enforce policy via infrastructure-as-code through GitHub.
How do I detect shadow Computer Use MCP servers in my org?
Willow's browser extension and discovery service surface unmanaged MCP servers, skills, and AI agents across the org. If a developer installed an unapproved Computer Use MCP locally, you'll see it.
What does the Computer Use MCP server cost?
Pricing depends on org size and deployment model (SaaS, dedicated cloud, self-host). See withwillow.ai/pricing or contact sales for a quote.
How do I install the Computer Use MCP server with Willow?
Install via the Willow Connect Panel in one click, or paste the JSON snippet above into your Claude Desktop, Cursor, or Claude Code config. SSO and RBAC inherit from your existing Willow setup.
Compare Willow MCP Gateway
See how Willow stacks up against other MCP platforms on governance, security, and enterprise readiness.
Your agents are already in the wild.
Give them a Basecamp. Go from AI chaos to AI work, in minutes.