drawgent connects your own Claude Code, Codex or opencode (your install, login, config and repo) to an Excalidraw whiteboard. Ask for a diagram in the chat panel, or write AGENT: … next to the part of a drawing you want changed. The agent looks at the canvas (screenshot + scene), edits it live, checks the result, and marks the note DONE.

How it works#

drawgent architecture: the browser, the drawgent server, your coding agent and its MCP canvas tools

Quick start#

Prerequisite: one of claude, codex or opencode installed and logged in. The Claude and Codex bridges also need Node.js ≥ 18 (npm).

drawgent setup claude        # once per agent: claude | codex | opencode
cd ~/my-repo
drawgent up                  # new agent session in this repo + canvas in your browser
drawgent up --attach         # or: pick one of your running sessions and connect the canvas to it

Example :)#

A hand sketch on the canvas (left) and the clean diagram the agent drew from it (right)

`drawgent setup <agent>`#

Checks everything once, fails with the exact fix when something is missing, and writes ~/.config/drawgent/config.toml:

The agent CLI. Your claude / codex / opencode on PATH.
Login. claude auth status, codex login status or opencode auth list.
ACP bridge.
- opencode speaks ACP itself (opencode acp).
- Claude Code and Codex use the official ACP adapters. They are installed once into ~/.cache/drawgent/adapters (~60 MB), without their bundled agent binaries, and pointed at your CLI (CLAUDE_CODE_EXECUTABLE, CODEX_PATH).
- Setup then verifies the ACP handshake.
Canvas tools for attached sessions. Only Codex needs a change: codex mcp add drawgent -- drawgent mcp. Claude and opencode get the tools at attach time.
Headless Chrome for the renderer. Setup uses your Chrome/Chromium if you have one. Otherwise it proposes:
- downloading Chrome Headless Shell (Chrome for Testing, ~120 MB, no sudo) into ~/.cache/drawgent/chrome, and telling you exactly which system libraries are missing, if any;
- installing Chromium with your package manager (apt, snap, dnf, pacman, zypper, apk, brew, or nix without sudo).
Non-interactive: --chrome download | system | /path/to/chrome.

drawgent up refuses to start until setup succeeded for that agent, or if something setup recorded disappeared.

`drawgent up`#

Runs in the current directory (the workspace):

starts the editor + API on 127.0.0.1:7300 (next free port if taken);
starts a new session of your agent over ACP, working in the workspace;
opens your browser. On a headless box it prints the ssh -L command instead.

The scene is kept in .drawgent/scene.json, which is git-ignored automatically.

`drawgent up --attach [id]`#

Connects the canvas to a session you already run. Without an id it lists the sessions it finds (current directory first) and lets you pick one:

agent	discovery	how the canvas reaches it
Claude Code	`claude agents --json` (interactive and background sessions)	fork: a new session carrying the full conversation, driven by drawgent over ACP. Your terminal session is left untouched
opencode	opencode servers listening locally: start the TUI with `opencode --port 4096` (or `opencode serve`)	live: messages go into your running session (you see them in your TUI); drawgent adds its MCP tools to that server at runtime; replies, tool calls and prompts you type in the TUI are mirrored into the chat panel
Codex	sessions in `~/.codex/sessions`	live: `codex queue --thread <id>`; replies are mirrored from the session's rollout file. The session needs the drawgent MCP (registered by setup, loaded when Codex starts)

Using the canvas#

Chat panel (right side): send a request, watch replies and tool calls stream in, approve permission prompts, press Stop to cancel a turn.
On the canvas: write a text starting with AGENT: next to or inside a shape, or draw an arrow from the note to a shape. It fires about 2.5 s after you stop typing, with its position, what it points at and what is nearby. The agent resolves it into a green DONE: … note. Edit it back to AGENT: to send it again.
Laser zones: pick Excalidraw's laser (K) and circle or scribble over part of the diagram. The trace stays on the canvas as a red outline, and the chat panel opens ready for your instruction, with a chip listing what is under the zone (🔴 Laser zone · API, Redis ×). Your next message goes to the agent together with the zone (bounds and covered elements), and the trace disappears when the agent is done. Strokes within 4 s of each other form one zone; × discards it.
Prompts are queued and run one at a time. A queued note that was already handled is skipped.

excalidraw.com rooms#

drawgent up --room 'https://excalidraw.com/#room=<id>,<key>'

drawgent joins the room as a collaborator (🤖 Agent, with a cursor that follows its edits). Humans can stay on excalidraw.com: their AGENT: notes reach your agent, and its edits appear there live.
Traffic is end-to-end encrypted with the room key. Empty rooms are loaded from and saved to excalidraw's Firestore storage.
The local editor mirrors the room and still has the chat panel.

Other commands#

drawgent mcp: stdio MCP server with the canvas tools. Agents launch it; it finds the running drawgent up by itself.
drawgent serve: low-level server with explicit agents (--agent claude,codex,opencode as set up, or name=command for any ACP agent); for scripts and containers.
Options for up / serve: --port, --room, --token (API bearer + ?token= in the URL), --permissions canvas|ask|all (default canvas: drawing tools auto-approved, anything else asked in the chat panel), --data, --settle-ms.

Optional: Docker#

docker compose up --build runs a canvas server (drawgent + Chromium, no agents), e.g. to host a shared canvas or a room bridge on a server. Agents are never bundled: they always run with your own setup.

Build from source#

npm ci && npm run build      # editor + renderer pages -> dist/ (embedded into the binary)
cargo install --path .       # or: cargo build --release

Agent tools (MCP)#

get_scene, get_screenshot (vision; zoom with element_ids), add_elements (Excalidraw skeletons; arrows bind by id and are routed edge-to-edge), add_mermaid (auto-layout), update_elements (labels follow shapes, bound arrows re-route), delete_elements, clear_canvas, list_instructions, resolve_instruction, set_status.

API#

GET /api/health · GET /api/scene · GET /api/screenshot?ids=&padding=&max= · POST|PATCH|DELETE /api/elements · POST /api/mermaid · POST /api/clear · GET /api/instructions · POST /api/instructions/{id}/resolve · POST /api/status · POST /api/chat {agent?, text} · WS /ws (browser sync + chat events)

Tests#

cargo test                          # fractional indices, room crypto/framing
node scripts/smoke.mjs [url]        # chat turn + AGENT: note against a running drawgent
node scripts/e2e-browser.mjs [url]  # real browser: chat panel + note typed on the canvas
node scripts/room-e2e.mjs           # fresh excalidraw.com room ↔ drawgent, both directions
node scripts/laser-e2e.mjs          # laser zone → chat → agent edits only that zone (needs setup claude)

Layout#

src/ (Rust):

main.rs: CLI.
setup.rs, config.rs, chrome.rs: setup, config, renderer install.
attach.rs: session discovery and picker.
agents.rs + acp.rs: ACP driver (new / fork sessions).
live.rs: live opencode / Codex drivers.
hub.rs: routing, notes, chat log.
scene.rs: store and edit operations.
renderer.rs: Chrome over CDP.
mcp.rs: MCP server.
room.rs: excalidraw.com client.
fractional.rs, geometry.rs, el.rs: helpers.

web/: editor (main.jsx, chat.jsx, laser.js) and renderer page (render.jsx).

Limits#

The renderer needs Chrome (a native renderer is planned).
Claude "attach" is a fork, because Claude Code has no public way to inject into a running terminal session.
Codex live attach is implemented, but was not yet tested against a logged-in Codex.
One scene per workspace. Images/files are not synced.

Configure Feed