Tracking and relay protocol #

Phone to desktop #

Send UTF-8 JSON datagrams, at most 4096 bytes, to the desktop LAN IPv4 address on UDP 49984:

{
  "schema": "ghost.face/1",
  "session": "12345678-abcd-4321-abcd-123456789abc",
  "sequence": 42,
  "tracked": true,
  "head": [0, 0, 0, 1],
  "gaze": [0, 0],
  "blink": [0, 0],
  "recenter": false
}

Field	Contract
session	New identifier when tracking starts; 8–64 ASCII letters, digits, hyphens. The app uses a UUID.
sequence	Nonnegative safe integer, increasing each frame.
tracked	Boolean; false frames only need schema, session, sequence, tracked.
head	Quaternion [x,y,z,w], face relative to camera; norm within 0.02 of 1.
gaze	Horizontal/vertical radians, head-local combined eye direction, finite within ±π.
blink	Anatomical left/right coefficients in 0…1.
recenter	Optional; exact true uses this frame as the new center.

The receiver accepts private IPv4 and loopback, pins the first sender IP, and ignores non-increasing sequences in the same session while receiving. After a stale interval an earlier sequence can be accepted again. Session changes reset calibration. Release phone clears the peer lock. Face loss neutralizes output; silence is stale after 1200 ms, checked every 100 ms.

At most once every 400 ms, reply to the sender's source port:

{ "schema": "ghost.face/ack1", "sequence": 42 }

Keep the sending socket open for replies. “PC receiving” depends on acknowledgments. UDP is lossy, unencrypted, and unauthenticated. Sender locking only avoids accidental mixing. Use a trusted LAN.

Local HTTP #

Base URL: http://127.0.0.1:8766. POST requires Origin http://127.0.0.1:8766 or http://localhost:8766, exact Content-Type application/json, and at most 2048 body characters.

Endpoint	Method	Behavior
/api/pose	GET	SSE events pose, phone, controller, camera; initial status and pose on connection.
/api/pose	POST	All seven pose fields; clamps manual head/gaze/blinks and yields to active native input.
/api/phone	GET / POST	Phone status and known tuning fields; enabled, selected, release, recenter.
/api/controller	GET / POST	Native status; enabled, sensitivity, deadzone, behavior, invert, reset.
/api/settings	GET	Saved settings snapshot and warning.
/api/settings	POST	action: save, revert, or defaults. Save persists current live settings.

Pose fields are yaw/pitch/roll in degrees, dimensionless gazeX/gazeY, blinkL/blinkR in 0…1. Manual POST limits are yaw ±30°, pitch/roll ±20°, gaze ±1. Phone gain is applied before broadcasting and can exceed these limits.

Settings live in data/tracking-settings.json with version 1. Unknown keys and invalid values are normalized or excluded. Calibration and LAN addresses are not saved. The HTTP relay is local software, not a public remote-control API.

Avatar state #

GET /api/avatar returns the selected id, per-avatar values and saved snapshot. POST accepts id, settings (a partial behavior update), and action (save/revert/defaults). It has the same local Origin/JSON checks and a 4096-character body limit. Avatar events on /api/pose carry shared state to local renderers. Persisted behavior lives in data/avatar-settings.json; tracking/calibration remain separate. See AVATARS.md.