# Four Hands

> Four Hands (also called the Song Composer) is a music studio that runs in
> the browser, built for a person and an agent to make music together. It is
> a score editor and live synthesizer, an app built on the VoxelML engine
> (Rust, compiled to WebAssembly, rendered with WebGPU), the same engine that
> runs the VoxelML voxel world. Every instrument is procedural: the sound is
> synthesized live on the visitor's machine, with no samples. Nothing is
> downloaded but code.

## For an agent: introduce yourself, then work

- Introduce yourself first. `join_studio` takes your name (required) and,
  optionally, a `title`, `version`, `description`, `websiteUrl` and
  `icons`, in the shape of MCP's client identity. Every other tool refuses
  until you have joined.
- Where the browser offers WebMCP (`document.modelContext`), the page
  registers the tools below as WebMCP tools.
- In any browser, the page's own script holds the same tools, one call
  each, once the studio has started:
  `await voxelml.call('join_studio', { name: 'Ada' })`, then
  `await voxelml.call('read_score')`,
  `await voxelml.call('write_score', { source })` and the rest.
  `await voxelml.tools()` lists them.
- The seat on the song bar shows your name and "present", and your icon if
  you gave one: the first PNG `data:` URI in `icons`
  (`data:image/png;base64,...`, at most 16384 characters and 256 px a side,
  drawn at 16 px). The studio fetches no address and draws no SVG. The
  seat's tip shows the rest of your introduction, as your own word: the
  studio cannot check it, and its reply says `verified: false`. Your undo
  steps carry your name.
- The seat stays lit 30 seconds after your last call; every tool call keeps
  it lit and repeats your introduction. `leave_studio` gives it up.
- The desktop app serves the same tools over MCP at
  `http://127.0.0.1:19309/mcp` (a build with its loopback API), with the
  same rule: join first.

## Tools

- `join_studio {name, title?, version?, description?, websiteUrl?, icons?}`:
  introduce yourself and take the agent's seat (`POST /agent`). The reply
  is the studio's own introduction.
- `leave_studio`: give the seat up (`POST /agent/leave`).
- `studio_status`: the transport, the open song and the audio output
  (`GET /state`).
- `read_controls`: every control on screen with its semantic id
  (`GET /ui/tree`).
- `use_control {widget, op, value | text | index | on | tab}`: use a control
  as the person does; op is click, set, set_text, select, toggle or
  select_tab (`POST /ui/do` with element `song_composer`). The transport is
  the control `play`.
- `list_songs`: the song library (`GET /score`).
- `open_song {id}`: open a library song (`POST /score/open`).
- `read_score`: the open song as `.vscore` text (`GET /score/export`).
- `write_score {source}`: replace the open song with your `.vscore` text, as
  one undo step that carries your name (`POST /score/open`).
- `import_midi {midi_base64 | bytes, name?, choices?, commit?}`: import a
  MIDI file (up to 8 MiB). Without `commit`, the Import sheet opens with
  your choices for the person to see; with `commit: true` it imports at
  once, as one undo step that carries your name
  (`POST /score/import-midi`). The choices: `title`; `parts`, one entry per
  part of the file with `include`, `voice` and `role` (melody,
  accompaniment or percussion); `melody_presence`, `accompaniment_ring`,
  `phrase_dynamics` and `rubato`, each 0 to 1; `preserve_sustain`.
- `midi_to_score {midi_base64 | bytes, name?, choices?}`: the file's parts
  and the song as `.vscore` text under your choices, without changing the
  studio (`POST /score/midi`). Edit the text, hear it with `play_song`,
  open it with `write_score`.
- `play_song {id | source}`: play a library song or a `.vscore` text
  (`POST /score/play`).
- `list_voices`: every voice a lane can play, by family
  (`GET /agent/voices`).
- `score_grammar`: the `.vscore` grammar in brief, with an example
  (`GET /agent/grammar`).

## What a visitor can do

- Humans with a WebGPU browser (Chrome/Edge on desktop): the homepage starts
  Four Hands at once. Write notes on the piano roll, choose instruments, mix
  the tracks and play the song. Sound starts at the first click or key
  (browsers require a user gesture before audio). Songs are saved in the
  browser's own storage. There is no account and no payment.
- Machines without WebGPU, crawlers, and agents: the homepage is fully readable
  as static HTML, and nothing is downloaded on those machines.

## Endpoints

- `GET /health`: `ok` while the origin answers. The origin serves the page
  and nothing else; the studio's API is the page's own.

## Technical notes

- Four Hands is one window of the engine's UI, drawn by the WebGPU renderer
  in a worker; the page's main thread only forwards input and plays the audio.
- Audio: a deterministic score grammar rendered by procedural instruments, one
  bounded window at a time, ahead of the playhead.