soundcraft/docs/control-protocol.md

Control protocol

SoundCraft can be driven completely by other programs: scripts, test harnesses and AI agents. Every menu item, button, mouse gesture and keystroke maps to a command (or to synthetic input), and every command is reachable from:

Surface	How
Desktop app control channel	soundcraft --control <port> (0 = pick a free port; printed as SOUNDCRAFT_CONTROL_PORT=…)
CLI against a running app	soundcraft-cli app --port <port> <method or command id> [JSON]
Headless CLI	soundcraft-cli run --demo --cmd 'mix.volume={"track":"Kick","db":-6}' --bounce out.wav
MCP (headless)	soundcraft-cli mcp [--demo]
MCP (bridged to the app)	soundcraft-cli mcp --connect <port>

Wire format

Newline-delimited JSON over TCP on 127.0.0.1. One request per line, one reply per line:

{"id": 1, "method": "engine.execute", "params": {"command": "track.new", "params": {"count": 2, "format": "Stereo"}}}
{"id": 1, "ok": true, "result": {"tracks": [12, 13]}}

Errors: {"id": 1, "ok": false, "error": "bad parameters for mix.volume: no track named Nope"}.

Methods

Method	Params	What it does
engine.execute (alias command)	{command, params}	Run a command programmatically. Never opens a dialog; empty params use defaults or the current selection.
engine.commands	{filter?}	Every command: id, label, menu path, shortcut, params doc, enabled + reason. Includes UI-layer commands (window.*).
engine.parity	{}	Parity report against the incumbent's menu catalog.
session.inspect	`{detail?: "summary"	"full"}`
ui.inspect	{}	UI state (window, panels, dialogs), window size, playing/position, audio device, and edit_layout (timeline rect and every track row's rect, for clicking).
ui.set	any UiState fields	e.g. {"window": "Mix", "narrow_mix": true}.
ui.menu.list	{}	Every catalog menu path and the command that implements it (or null).
ui.menu.invoke	{path} or {id}	Like clicking the menu item: may open its dialog (e.g. "Track > New...").
ui.click	{x, y, button?, count?, cmd?, shift?, alt?, ctrl?}	Synthetic click (window points).
ui.drag	{x, y, to_x, to_y, steps?}	Synthetic drag with the primary button.
ui.move	{x, y}	Move the pointer.
ui.key	{key, cmd?, shift?, alt?, ctrl?}	Key press (Space, Enter, A, F7, =…).
ui.text	{text}	Type text into the focused field.
ui.screenshot	{path?}	PNG of the window (base64 in png_base64 when no path).
app.quit	{}	Quit.

Positions and references

• Tracks: id (number) or name ("Kick").
• Clips: id (from session.inspect).
• Positions: samples (96000), seconds ({"seconds": 2.5}), or a time string in any timebase, optionally prefixed: "bars_beats:5|1|000", "min_secs:0:12.500", "timecode:00:00:10:00", "samples:48000".
• Ranges: start + end (or start + length); omitted = the current edit selection.

Examples

P=$(soundcraft --demo --control 0 | sed -n 's/SOUNDCRAFT_CONTROL_PORT=//p')   # or read stderr
soundcraft-cli app --port $P mix.solo '{"track":"Bass"}'
soundcraft-cli app --port $P transport.play
soundcraft-cli app --port $P ui.screenshot '{"path":"/tmp/shot.png"}'
soundcraft-cli app --port $P ui.menu.invoke '{"path":"Track > New..."}'

Offscreen (no window): cargo run -p soundcraft-ui-egui --example ui_shot -- out.png script.jsonl, where each script line is a request ({"method": …, "params": …}), {"shot": "path.png"} or {"steps": n}. Use --system-theme light|dark|none to supply the initial OS appearance, and script events such as {"system_theme": "dark"} to change it on subsequent frames.

Appearance defaults to Dark. ui.theme {"mode":"system"} follows the appearance supplied by the native or web integration at launch and on later frames, while the saved choice stays System. When the integration cannot report an appearance, System uses Dark. Manual Light and Dark stay fixed when the OS appearance changes. An offscreen render without --system-theme supplies no OS appearance and therefore resolves System to Dark.