# machine

Small, spec'd agent loops with an attached evaluator.

A **machine** is code: what it is for (its intent), how it is instructed, which
model and tools it uses, a **rubric** that says what good output looks like,
and an **evaluator** that assigns a grader to every criterion of that rubric.
Each execution is a **run**, stored with its full trajectory and, once graded,
a scorecard. Runs are what later layers will optimize against: fine-tuning on
good runs, revising rubrics, and splitting a machine into specialized
sub-machines.

**New here? Start with [docs/getting-started.md](docs/getting-started.md):**
install, talk to a machine, write your own, specialize it, measure and
improve it, and change how a model reasons.

## Using a machine

```bash
bun src/cli.ts pirate                 # talk to it in Pi's terminal UI
bun src/cli.ts julia/fast --continue  # resume the last conversation with it
```

`machine <name>` opens the machine in [Pi](https://pi.dev)'s interactive UI.
The model sees exactly the machine's instructions and tools, with its model
(Tinker ids, `openrouter/...`, or any Pi provider). Tools run in this process
against the current directory, which is the session's workspace (commands
isolated by bubblewrap, with the directory writable). Pi's own tools, your
`~/.pi` setup, and context files stay out. Every exchange is stored as a run
with its full trajectory, so interactive use feeds the same grading and
training as suites do. In the session:

- `/score` grades the last exchange against the machine's rubric
- `/good`, `/bad` mark it; `/correct <note>` records what should change
- `/machine` shows the machine, its criteria and tools, and the session's spend

`--eval` grades every exchange as it finishes. Judges default to
Inkling-Small on Tinker; set `MACHINE_JUDGE_MODEL` to grade elsewhere.

## Primitives

| Primitive | What it is |
|---|---|
| `defineRubric` | Criteria with descriptions and weights, in their own file |
| `defineEvaluator` | One grader per criterion; TypeScript rejects a missing or extra one |
| `check(name, fn)` | Deterministic grader |
| `judge()` | Model grader; uses the workspace harness and judge model (the run's model unless one is set). Retries a reply without a usable score twice |
| `jev()` | Graders backed by [Jev](https://docs.typesafe.ai/api), TypeSafe's decision model: `noul` (yes/no), `score` (ordered levels), `choice` (options with scores). Much cheaper than a model judge; one request per run for all of a client's questions. Needs `TYPESAFE_API_KEY` |
| `defineMachine` | Name, intent, instructions, model, rubric, evaluator, tools |
| `judgePanel(rubric, ids)` | Model graders that score several criteria in one judge call; for when a call per criterion costs too much |
| `counterpart` | Another machine that plays the other side of a conversation (e.g. a simulated user); each case is then a whole conversation |
| `defineChild` | A machine that specializes a parent: inherits its criteria and graders, instructions, tools, and model, and adds its own |
| `Harness` | The agent loop for one task: `chatCompletionsHarness` (Tinker, Ollama, OpenRouter, any OpenAI-compatible endpoint), `piHarness` (Pi), `opencodeHarness`, `fxHarness` (libfx, Vercel AI Gateway), `scriptedHarness` (tests) |
| `Budget` | A spending limit charged for every run and every judge call; refuses new work once reached |
| `Sandbox` | Where a run's tools act. `localSandboxes` gives each run its own working directory and, with bubblewrap, isolates its commands (see below) |
| `Store` | Runs and named chat sessions, addressed by readable names; every write is an upsert. `diskStore` writes private JSON files |
| `createWorkspace` | Binds machines to a store, sandboxes, and a default harness; `run`, `chat`, `evaluate` |

## A machine

```text
machines/chatbot/
  rubric.ts    what good looks like
  machine.ts   instructions, model, and the grader for each criterion
```

```ts
// machines/chatbot/rubric.ts
export const rubric = defineRubric({
  answers: { description: "Addresses what the user actually asked…", weight: 3 },
  concise: { description: "Uses as few words as the question needs.", weight: 1 },
});

// machines/chatbot/machine.ts
export default defineMachine({
  name: "chatbot",
  intent: "Answer one person's questions in a running conversation, plainly and honestly.",
  instructions: "…",
  model: "Qwen/Qwen3.8-27B",
  rubric,
  evaluator: defineEvaluator(rubric, {
    answers: judge(),
    concise: check("words", (run) => wordCount(run.output) <= 150),
  }),
});
```

Use the cheapest grader that is reliable for a criterion: a `check` when code
can decide it, Jev for a simple question about the text, and a `judge` when it
takes open-ended reasoning:

```ts
const j = jev(); // TYPESAFE_API_KEY
evaluator: defineEvaluator(rubric, {
  in_french: j.noul(),                                   // the criterion's description is the question
  tone: j.score(["rude", "neutral", "warm"]),
  length: j.choice({ short: { score: 1 }, long: { score: 0 } }),
  accurate: judge(),
}),
```

Before relying on a cheap grader for a criterion, measure it against the
machine's own grader on stored runs:

```bash
bun src/cli.ts agree coder does_task --with jev      # or --with judge
```

`agree` reports agreement, Cohen's kappa, the false-pass rate (a bad run
passed: the error that poisons training data and rewards the wrong behavior)
and false-fail rate (a good run failed: wasted samples), precision, bits of
information per label, and the share of bad samples a filter would keep at
10% and 50% base pass rates, at several thresholds. It reuses stored
reference grades, lists the runs where the graders disagree, and counts runs
where the candidate is unsure (0.3 to 0.7) and should escalate to a stronger
grader. Jev's scores vary slightly between identical calls, so unsure runs can
land on either side of 0.5.

Nested machines live in nested directories and are named with `/`, for
example `machines/chatbot/triage/` is `chatbot/triage`.

## Watching machines

`machine dashboard` serves a live view of the workspace on 127.0.0.1:4242:
what is running now and each step it takes, every stored run with its
scorecard, reasoning, and tool calls, each machine's recent quality and spend,
and fine-tunes under `.machine/training/` with their loss curves and
checkpoints. Every process that runs a machine appends events to
`.machine/events.jsonl`; the dashboard follows it, so nothing else has to run.

## Child machines

A child machine specializes its parent within the parent's constraints:

```ts
// machines/julia/parametric/machine.ts
import julia from "../machine.ts";

export default defineChild(julia, {
  name: "julia/parametric",
  intent: "Write Julia that is parametric: type parameters let one definition serve every element type.",
  instructions: instructionsFrom(import.meta.url), // appended to the parent's
  rubric,                                          // only the child's own criteria
  evaluator: defineEvaluator(rubric, { parametric: onProgram("parametric", isParametric) }),
});
```

A child inherits every criterion of its parent with its grader, and cannot
redefine or drop one: what the parent requires, every child is graded on.
It also inherits the parent's instructions (its own follow them), tools (it
may add more), and model (unless it names its own). Because the family
shares inherited criteria and a model, it can be measured and trained as
one:

```bash
bun src/cli.ts tree julia                 # each machine, what it inherits, what it adds
bun src/cli.ts suite julia --tree         # every machine's cases, then the inherited criteria across all of them
bun src/cli.ts export julia --tree --out julia.jsonl   # passing runs of the whole family, one dataset for the shared model
```

`optimize` on a child rewrites only the child's own `instructions.md`; the
optimizer sees the parent's instructions as fixed.

### A machine that writes its own specialist

`machine specialize` has an author model (default `thinkingmachines/Inkling`)
write a child for a focus, and the cases to rehearse it on:

```bash
machine specialize dev --focus "Python datetime and timezone bugs" \
  --model thinkingmachines/Inkling-Small --slug datetime --budget 2
machine suite dev/datetime --trials 4 --effort high --budget 2     # rehearse
machine train dev/datetime --best-of 4 --effort high --budget 3    # fine-tune on the best rehearsals, gate on held-out
```

The author sees the parent's job, rubric, instructions, tools, and source,
and writes `machine.ts` (`defineChild`), `rubric.ts` (1 to 3 new criteria),
and `instructions.md`. Nothing it writes is trusted: the child must
typecheck, load as a child on the requested model, and its check graders
must run on sample runs of the parent; failures go back to the author
(two retries by default), and a check that scores every sample run alike, or reads only
the files a run started with, gets a second look. Then it writes cases in
batches over its own topic list: each case's test must fail on the starting
files and pass with the author's reference fix, in the sandbox runs use, or
the case is discarded. Kept cases split into `cases.json` and
`holdout.json`. For `dev`, a child that grades with `hiddenTests` also gets
held-back edge-case tests per case (`hidden.json`, never shown to the agent),
which must fail alone before the fix and pass with it.
`m.specialize({ focus, model, budget })` is the SDK form; `--reuse-cases`
rewrites only the child. Review what it wrote before training: a criterion
that cannot be met on some cases shows up in rehearsal's failure reasons, and
a scoped description fixes it.

## CLI

Runs on [Bun](https://bun.sh), which executes the TypeScript directly.

```bash
bun install
bun src/cli.ts run chatbot                        # interactive, in Pi (--continue resumes)
bun src/cli.ts run chatbot --eval "Explain LoRA in two sentences."
bun src/cli.ts runs chatbot
bun src/cli.ts eval chatbot <run-id>
bun src/cli.ts suite red --harness opencode -v      # run and grade machines/red/cases.json
bun src/cli.ts compare red --harness opencode --trials 5   # working tree vs HEAD, 5 trials each
bun src/cli.ts compare red --ref <commit> --cases holdout.json --trials 3  # on held-out cases
bun src/cli.ts converse <machine> 3 --eval       # one conversation with its counterpart, printed live
```

## Machines that talk to each other

A machine can name a **counterpart**: another machine that plays the other
side of a conversation, such as a simulated user with a hidden scenario.

```ts
counterpart: {
  machine: simulatedUser,                          // another machine
  until: (run) => isDone(run),                     // the conversation ends after this turn
  maxTurns: 8,
},
```

Each case is then the counterpart's starting point, and running it is a whole
conversation: the counterpart speaks first, they alternate, and it ends when
`until` holds or after `maxTurns`. Every turn of each side is an ordinary
stored run (tagged with its `conversation`), and the machine's last turn,
whose history is the whole conversation, is the run that is graded. So
`suite`, `compare`, `optimize`, and `export` work on conversations unchanged,
and `--model` swaps only the machine's model, never the counterpart's.
`machine converse <machine> [case]` runs one and prints it as it happens; with
`--eval` it grades both sides.

## Changing a machine

There is one copy of each machine. Change it in place and let git hold the
versions:

1. Edit the machine (its instructions, model, or tools).
2. `machine compare <machine> --trials 5` runs the working tree's version and
   the version at `--ref` (default `HEAD`) on the same cases, interleaved.
3. Commit if it wins; `git checkout -- machines/<machine>` if it does not.

The old version is loaded from git into a temporary directory. Both versions
are graded with the current rubric and evaluator, so the comparison measures
the change in behavior and nothing else; `compare` notes when the rubric has
changed since the ref, or when the machine has not changed at all. Each run
records its `revision` (a commit, or `<commit>+working-tree`).

Models are not deterministic, so use `--trials` to run each case several
times: `compare` reports each version's mean and pass rate, and the mean
difference with its standard error over cases. A difference within about two
standard errors of zero is not evidence that either version is better.

## Optimizing instructions

A machine declares its instructions an optimization target by keeping them in
`instructions.md` beside `machine.ts`:

```ts
instructions: instructionsFrom(import.meta.url),
```

`machine optimize <machine> --trials 3 --rounds 2` then runs the suite, hands
the failures to the built-in `optimizer` machine (purpose, tools, rubric,
current instructions, each failing case with what the machine did and why the
graders failed it, and the passing cases), runs the suite again with its
proposed instructions, and keeps the proposal only if it scores higher. A kept
proposal is written to `instructions.md` and left uncommitted: commit to
accept it, `git checkout` to reject it. The optimizer only proposes; the
measurement decides, and you approve. It may rewrite instructions, never the
rubric.

A rewrite must also win by at least two standard errors (`--min-z`); a
smaller gain is indistinguishable from trial noise and is discarded.

`optimize` only ever shows the optimizer `cases.json`. Keep a few cases it
never sees in `holdout.json` (or pass `--holdout <file>`): a proposal is kept
only if it scores higher on `cases.json` and no lower on the held-out cases.
A rewrite that wins on its own failures but loses on unseen cases has
memorized them, and is discarded. Without a held-out set `optimize` warns and
falls back to `cases.json` alone. `compare --cases holdout.json` runs the
same check by hand.

## Training a machine in place

A machine **version** is what decides how it behaves and is graded: its own
instructions, its model, and its rubric. Versions are numbered in
`machines/<name>/versions.json`, checked in beside the machine, and every run
records the version that made it. The machine's files stay the truth:
`instructions.md`, `model.json` (a machine that declares
`model: modelFrom(import.meta.url, "<default>")` takes its committed model
from there), and `rubric.ts`.

```bash
machine versions dev                      # list versions; the first call records the current one as v1
machine train dev --samples 4 --trials 3 --budget 5
machine propose dev --model Qwen/Qwen3.6-35B-A3B --trials 3 --budget 1   # another model, as a candidate
machine commit dev                        # the newest pending candidate, only if it passed the gate
machine revert dev                        # the previous live version
```

`train` never changes the machine. It samples the cases in `cases.json`
(`--samples` runs each, default 0 to use stored runs), grades whatever is
pending, and fine-tunes the live model's base on Tinker with the good runs
(`--min-score`, default 1; the best `--best-of` per case, default 2; never a
held-out case; only runs by the live model unless `--any-model`, which
distills from other models' runs). Each step of a tool-using run becomes its
own training example, with the context the model saw at that step and the
tools it was offered; steps whose tool calls all failed are left out. On
Tinker a run's later prompts hold its earlier steps with their reasoning, so
for Inkling each step's prompt is a token prefix of the whole run (checked
against the endpoint's prompt tokens), and a run is trained as one example
with every good step a target, at a fraction of the tokens. Inkling's prompt
carries its thinking effort: training renders at the effort runs sampled at
(`--effort`; the endpoint's default is high, 0.9), and its learning rate
defaults to 2e-4. It then
runs the checkpoint and the live version on `holdout.json` (`--trials` each;
stored graded runs of the live version are reused) and prints a report: the
data, the losses, each criterion before and after, and the decision. The
result is a pending **candidate**.

`commit` makes a candidate the live version only if it passed the promotion
gate: a higher mean on held-out cases by at least `--min-z` (default 2)
standard errors of the per-case difference, the margin `optimize` uses. It
refuses otherwise and prints the report; `--force` overrides. Committing
writes `model.json` (the checkpoint and its base model, which prices it) and
a new version; `revert` writes back the previous one. `--budget` caps
sampling, grading, training, and the comparison together. Training runs
`scripts/train_sft.py` with `$MACHINE_PYTHON`, a Python with tinker-cookbook
installed; its output goes to `train.log` in the run's directory under
`.machine/training/`, where the dashboard charts it.

## SDK

The same loop from code, shaped like an agent SDK: `run()` streams messages
and can be awaited for the finished run.

```ts
import { formatReport, machine } from "machine";

const dev = await machine.load("dev", { judgeModel: "openrouter/thinkingmachines/inkling-small:free" });

for await (const message of dev.run("The tests fail. Fix calc.py.", { files: { "calc.py": "…", "test_calc.py": "…" } })) {
  if (message.type === "tool_call") console.log("→", message.name);
  if (message.type === "result") console.log(message.run.version, message.run.output);
}

const chat = dev.session({ autoGrade: true, cwd: "/tmp/demo-project" }); // multi-turn, one working directory
await chat.send("Add a --shout flag to greet.py");
await chat.close();

await dev.grade();                                     // runs never graded, or graded under an older rubric
const candidate = await dev.train({ budget: 5, samples: 4, trials: 3 });
console.log(formatReport(candidate.report));           // the live machine is unchanged
await candidate.commit();                              // throws CommitRefused unless it won on held-out cases
await dev.revert();                                    // back to the previous version
```

`machine.new(spec)` does the same for a machine defined in code, keeping its
versions under `.machine/versions/`. `m.update({ instructions, model })`
records a new version by hand, `m.versions()` lists them, `m.pending()` lists
runs due for grading, and `m.optimize({ budget })` returns a candidate that
rewrites the instructions, committed the same way.

## Example machines

- `chatbot`: answers questions in a running conversation. Graded on answering
  the question and honesty (judged), concision, and no filler (checked).
- `pirate`: answers anything as a golden-age pirate. Graded on still
  answering the question, pirate voice, naming nothing modern (modern things
  get period names: a phone is a "talkin' slate"), and seafaring metaphor
  where it fits (judged), and one to four pirate emoji (checked).
- `red`: calls its `red_mentioned` tool exactly once when the message
  contains "red" as a discrete word (any case, no letter or digit directly on
  either side), and never otherwise. That rule is the whole rubric, checked
  deterministically from the input and the trajectory; the reply text is not
  graded. `cases.json` holds near misses such as "bored", "redo", "Fred", and
  "I read it".
- `coder`: writes a program with `write_file` and runs it with `run` in the
  run's sandbox. `run` refuses unless the sandbox is isolated. Graded on
  whether it wrote and successfully ran a file (checked from the real tool
  results) and whether the program's actual output does the task (judged
  against the tool results).
- `dev`: a development assistant for an existing project. Lists, searches,
  reads, and edits files (`edit_file` replaces one exact snippet), writes new
  ones, and runs things with `run`. It answers questions, explains, and plans,
  and changes files only when asked. Graded on whether a command passed after
  its last change (checked from the tool results), whether it did what was
  asked, whether it stayed on task, and whether it acted on the request:
  editing only for a change, answering or asking for questions and unclear
  input like "test" (judged). Its cases are small Python projects in
  `cases.json` and `holdout.json`, including questions and requests to run
  and report. `cd your-project && machine dev --workspace ~/code/machine`.
- `assistant`: a general-purpose assistant in the folder you start it in
  (`cd somewhere && machine assistant --workspace ~/code/machine`, or
  `demo/assistant.sh`). It answers questions, researches, reads and writes
  files, and runs commands, with dev's file tools and `run` (isolated: no
  network, only the folder) plus `web_fetch`, which runs in the machine
  process: http and https only, private, loopback, link-local, and internal
  addresses refused (on every redirect too), a 15 s timeout, scripts and
  styles stripped, and 20,000 characters a call with an `offset` to read on.
  It has no search tool: no web search API is configured. A sibling of
  `dev`, not a child: dev's `verified` and `focused` are about code. Graded
  on `helpful`, `grounded` (claims trace to tool results and cite the file
  or URL; judges see whole fetched pages), dev's own `acts_on_request`
  (judged), and `concise` (word count; an empty reply scores 0). Its cases
  cover file questions, page fetches, a refused localhost fetch, a one-word
  request, writes and edits, run and report, and general knowledge.
- `machinist`: makes and improves machines, in Pi (`machine machinist`, or
  `demo/machinist.sh`). Its tools run in the machine process: read-only
  `list_machines`, `show_machine`, and `recent_runs`; `specialize`,
  `rehearse`, and `train`, which spend money, each with a required `budget`
  (at most $10 a call, or `MACHINIST_MAX_BUDGET`); and `commit` (through the
  promotion gate) and `revert`. Spending and changing tools run only in an
  interactive session, and only after the person agreed: their latest message
  names the budget, or says yes to a cost the machinist stated. Graded on that
  (checked from the conversation), and on grounding its claims in tool
  output, choosing sensible specializations, and reporting results honestly
  (judged).
- `wordmath`: solves word problems; the answer on its last line is checked
  against a known answer. `wordmath/fullwidth` must also reason in fullwidth
  letters and digits (checked from the captured reasoning), and its
  `transformReasoning` rewrites reasoning into fullwidth when runs are
  exported, so `export wordmath/fullwidth --from wordmath` turns the parent's
  correct runs into training data for thinking that way.
- `julia` and its children: writes a Julia program and runs it with
  `run_julia` in the isolated sandbox. The parent requires that the program
  runs (checked), does the task, and is idiomatic (judged). Each child adds
  a specialty: `julia/parametric` (type parameters, checked), `julia/generic`
  (no concrete argument types, checked; shown on several input types,
  judged), `julia/fast` (times its work, checked; written for speed, judged),
  and `julia/inline` (only short-form or `@inline` functions, checked).
  Juliaup installs under your home directory, which the sandbox hides, so
  `run_julia` mounts only the Julia installation read-only, never the rest
  of `~/.julia`.

Choose the agent loop with `--harness` (or `MACHINE_HARNESS`):

| Harness | Needs | Models |
|---|---|---|
| `tinker` (default) | `TINKER_API_KEY` | Tinker models and fine-tuned checkpoints, e.g. `Qwen/Qwen3.8-27B` |
| `pi` | `pi` on PATH | Tinker ids as above (or `tinker/<id>`) with `TINKER_API_KEY`, or Pi's own `provider/model` ids |
| `opencode` | `opencode` on PATH, `bun add @opencode-ai/sdk`, and the provider's key (e.g. `OPENROUTER_API_KEY`) | opencode's `provider/model` ids, e.g. `--model openrouter/thinkingmachines/inkling:free` |
| `openrouter` | `OPENROUTER_API_KEY` | OpenRouter ids, e.g. `--model thinkingmachines/inkling-small` |
| `fx` | `bun add libfx`, `AI_GATEWAY_API_KEY` | Vercel AI Gateway ids, e.g. `--model anthropic/claude-sonnet-5.5` |
| `openai` | `--base-url` or `OPENAI_BASE_URL`, optional `OPENAI_API_KEY` | Whatever the endpoint serves (Ollama, OpenRouter, vLLM) |
| `echo` | nothing | Replies with the input; free dry run of the plumbing |

Free Inkling (`thinkingmachines/inkling:free` and `inkling-small:free` on
OpenRouter) is served only to agentic harnesses, so call it through
`--harness pi` or `--harness opencode` with `OPENROUTER_API_KEY` set; the
plain `openrouter` harness gets a 403. The pi harness turns on Pi's own
OpenRouter attribution headers (`HTTP-Referer: https://pi.dev`) for
OpenRouter models, which is what admits it, while keeping Pi offline so no
install telemetry is sent. Free models count as $0 against the budget. To
grade for free too, pass `--judge-model
openrouter/thinkingmachines/inkling-small:free` (or set
`MACHINE_JUDGE_MODEL`):

```bash
bun src/cli.ts suite pirate --harness pi \
  --model openrouter/thinkingmachines/inkling-small:free \
  --judge-model openrouter/thinkingmachines/inkling-small:free
```

Prompts and outputs sent to the free endpoint are logged and used by Thinking
Machines to improve its models, so keep private material off it.

The opencode harness runs one local `opencode serve` per CLI invocation, gives
each run a fresh session in the run's sandbox, and deletes the session after
the run. Runs use a custom `machine` agent whose one-line prompt replaces
opencode's coding-agent prompt, so the model sees only the machine's
instructions plus a short environment block (model id, working directory,
platform, date) that opencode always adds. The server runs with a private,
empty config home and opencode's switches for project config, Claude Code
instruction files, skills, plugins, and sharing turned off, so your own
opencode setup never leaks into machine runs. Pass extra opencode config (for
example a custom `provider`) with `opencodeHarness({ config })`. Machine tools
are served to opencode over MCP from the machine process (127.0.0.1 only,
behind a per-run bearer token), so they run with the run's sandbox as on any
other harness; only that run's tools are enabled.

The pi harness runs `pi --mode json` with a private, empty agent directory,
so your `~/.pi` settings, keys, extensions, skills, and context files never
load, and the model sees exactly the machine's instructions. Machine tools
are registered in a generated per-run extension that forwards each call to
the machine process (127.0.0.1, per-run bearer token), so they run in the
run's sandbox as on any other harness.

`--eval` grades each run. Judge graders call a model, so they cost money on a
paid harness. On `tinker` and `pi` the default judge is
`thinkingmachines/Inkling-Small`; pick another with `--judge-model` or
`MACHINE_JUDGE_MODEL`.

Judges on `tinker` think at `reasoning_effort` medium (`--judge-effort`;
`--effort` sets it for runs). Lower effort is cheaper but can grade worse:
check a judge's effort with `agree --with judge --judge-effort` before
lowering it.

### Spending

On `tinker` and `pi`, every command runs under a budget, $1 unless you pass
`--budget <usd>` (or `--budget none`). Each run and each judge call is
charged at Tinker's listed prices (`src/pricing.ts`), with prompt-cache
hits at the cached rate (Tinker caches prefixes automatically, and tool
loops resend their whole conversation, so most of a long run's input is
cached; Inkling's temporary discount is ignored, so estimates err high), new work is refused once the limit is reached,
and the command prints what it spent. Runs already in flight finish, so a
command can overshoot by the cost of `--concurrency` runs. Usage at an
unknown price is refused under a budget; price a fine-tuned `tinker://`
checkpoint as its base with `--price-as <model>`.

`suite`, `compare`, and `optimize` run up to `--concurrency` runs at once
(default 4). A run that fails after the harness's retries (429s, 5xx, and
network errors are retried with backoff) is reported and left out of the
scores instead of ending the command.

### Fine-tuning data

`machine export <machine> [--tree] [--min-score 1] --out data.jsonl` writes
graded runs that score at least `--min-score` as chat conversations (the
instructions the run was given, the input, and its assistant turns with tool
calls and results), the format tinker-cookbook's `FromConversationFileBuilder`
reads. The files hold prompts and outputs: read them before training.
For rejection sampling, `--best <k>` keeps each case's best k runs, `--since`
limits to one sampling batch, and `--cases cases.json` keeps held-out cases
out. `--per-turn` writes one example per assistant message, each ending at
it, and every turn of a conversation, for trainers such as
`scripts/train_sft.py` that learn only each example's last assistant message.

Runs, sessions, and sandboxes are stored under `.machine/` in the workspace
directory. They contain your prompts and outputs: `.machine/` is git-ignored,
and files are written readable only by you.

## Not yet

- Training methods beyond SFT on good runs (RL against the rubric, DPO from good and bad pairs)
- Rubric drafting, and approval of workspace edits through the CLI
- Cloud sandboxes (Modal) and a cloud store
- Restoring fx conversations from libfx checkpoints; history is passed as a transcript for now

## Sandboxes

Each run gets a working directory under `.machine/sandboxes/`, removed after
the run. Tools write and read files there (`writeFile`, `readFile`, confined
to the directory, symlinks included) and run commands with `exec`.

Where [bubblewrap](https://github.com/containers/bubblewrap) works (Linux with
unprivileged user namespaces), commands run isolated: no network, a read-only
system, `/home`, `/root`, `/tmp`, and other user-data mounts hidden except the
run's own directory, an empty environment (no API keys), and the whole process
tree killed on timeout. `sandbox.isolated` says whether that is in force;
tools that run model-written code should refuse when it is false. Without
bubblewrap the sandbox still scrubs the environment, but commands can reach
your files and the network.

Bubblewrap shares your kernel and does not cap memory or CPU. For stronger
isolation, run code in a remote sandbox (Modal is planned) behind the same
`Sandbox` interface.

## Development

```bash
bun test
bun run typecheck
```

Tests use scripted harnesses and fake endpoints only; they never call a model.