In short: Vercel Labs released agent-desktop, an open-source tool that lets coding agents like Claude Code use your computer desktop. Instead of interpreting the screen pixel by pixel, it reads the operating system’s accessibility tree — so every button and field is a direct “address” for the agent.
Why it matters
- Screenshot-based control is slow and error-prone: the agent has to guess “there’s probably a button around here” from pixels.
- Because agent-desktop reads the window as a list of elements, when Claude Code says “click that address”, it just works, reliably.
- Works on Windows, macOS and Linux.
One thing to watch
As with any tool that controls your desktop, it’s safest to try it in a test environment first. I’ve put the setup steps and a sample use case in the guide.
Free PDF guide
Setup and first use, step by step, in a 3-page guide.
Source: github.com/vercel-labs/agent-desktop. Current as of November 15, 2026.



