Eve + Tinker recipes #

End-to-end Tinker model training and serving with Eve.

This repository contains small, deterministic recipes for adapting a model to agent work, promoting it only after validation, and exercising the promoted checkpoint through Eve's real session surface.

The recipes are deliberately closed. Each one has generated train, validation, and hidden test splits; a strict JSON contract; a shared LoRA training and promotion harness; and Eve evals derived from the hidden split.

Recipes #

Recipe	Behavior	Target	Status
capability-placement	Route capabilities into Eve's project structure	{kind, path}	trained and live-tested
context-selection	Select the smallest sufficient context set for a turn	{include: [...]}	runnable recipe
action-authorization	Choose execute, draft, ask, or refuse from explicit authority	{decision, next}	runnable recipe

The current trained result belongs only to capability-placement. The other recipes are complete data, training, serving, and eval examples; this repository does not claim model quality before someone runs their training gates.

The loop #

Every recipe uses the same path:

1. generate deterministic data with disjoint slugs across splits;
2. measure the base model on validation;
3. train a LoRA adapter with Tinker's native SDK;
4. evaluate the candidate on validation;
5. promote only if absolute accuracy and improvement gates both pass;
6. open the hidden test split only after promotion;
7. serve the promoted checkpoint through Eve;
8. run strict evals against Eve's durable HTTP session surface.

Private run receipts and checkpoint pointers are mode 0600 under artifacts/ and .tinker/. Both directories are ignored by Git.

Install #

Requirements:

• pnpm and a system Node version that can run it; the project installs its own pinned Node 24.18.1 for Eve;
• Python 3.11 or newer;
• uv;
• TINKER_API_KEY for live sampling or training.

pnpm install
pnpm setup:python
pnpm check

The Python environment lives at .generated/venv; .venv is a compatibility symlink. This avoids Eve 0.29.5 copying roughly five gigabytes of Tinker/PyTorch dependencies into every development snapshot.

Choose a recipe #

EVE_RECIPE selects the instructions, generated eval fixture, and private promoted checkpoint pointer. It defaults to capability-placement.

EVE_RECIPE=context-selection pnpm inspect
EVE_RECIPE=action-authorization pnpm eval:list

Without a promoted checkpoint for the selected recipe, both Eve and the native bridge use the base Qwen/Qwen3.5-4B model. Set TINKER_MODEL to test another base model or checkpoint without changing promotion state.

Generate and inspect data #

All checked-in JSONL and Eve fixtures are generated from recipe code:

pnpm data:write
uv run python -m training.data --all --check
uv run python -m training.run --recipe context-selection --dry-run

Measure and train #

Export TINKER_API_KEY, then choose one recipe:

uv run python -m training.run --recipe context-selection --baseline
uv run python -m training.run --recipe context-selection --train

The shared default promotion gate requires at least 80% exact validation accuracy and a 30-point gain over the base model. capability-placement raises its absolute gate to 100% because it is a small exact task. A passing run writes .tinker/context-selection.json; a failed run preserves its private receipt without changing the active checkpoint. Use --help to change steps, rank, batch size, learning rate, or gate thresholds.

capability-placement has validation-derived calibration: it repeats skill, library, and subagent three times in its effective training stream. The other recipes use balanced data by default.

Run through Eve #

The development command starts the native Tinker bridge and Eve, then opens Eve's TUI:

EVE_RECIPE=capability-placement pnpm dev

Use the headless development server when another process owns the client:

EVE_RECIPE=capability-placement pnpm dev:headless

For the built server:

EVE_RECIPE=capability-placement pnpm build
EVE_RECIPE=capability-placement pnpm start

The build writes an ignored runtime contract containing the selected recipe, model, and bridge URL. pnpm start refuses to launch if those values drift, so a checkpoint cannot silently run under another recipe's instructions.

From a second SSH session:

pnpm chat
pnpm ask -- 'Place this capability in an eve project. Capability: Add a typed callable action named `lookup_invoice` that the model can invoke at runtime. Return the canonical placement.'

The clients wait for /eve/v1/info, which avoids opening a stale TUI while the native sampler is still starting. With no EVE_URL, they probe the built server on port 3000 and then the development server on port 2000.

Run the selected recipe's smoke and held-out eval suite against a live development server:

pnpm smoke
pnpm eval

For a recipe without a promoted checkpoint, these commands evaluate the base model and may fail by design. A failed strict smoke is behavior evidence, not a broken serving path.

Why the native bridge exists #

Training and native evaluation use Qwen 3.5's qwen3_5_disable_thinking renderer. Tinker's hosted OpenAI-compatible endpoint fixes this model to its default thinking template and rejects reasoning_effort; it therefore cannot preserve the adapter's token contract. bridge/server.py samples through Tinker's native client with the exact training renderer, then exposes a small loopback OpenAI chat-completions surface to Eve.

The bridge buffers each native completion before emitting SSE. It is a local integration surface, not internet-facing or high-throughput serving. Its bind host is restricted to loopback because the process holds the outbound Tinker credential.

Add a recipe #

A recipe has three authored parts:

1. a module in training/recipes/ that generates Case objects and validates its target;
2. an instructions.md under recipes/<recipe-id>/;
3. a registry entry in training/recipes/registry.py and agent/lib/recipe.ts.

Then run pnpm data:write, add recipe-specific tests and examples, and inspect the dry-run contract before spending Tinker tokens. recipes/README.md contains the full checklist.

Security boundary #

All eleven Eve framework tools are disabled, including shell, filesystem, web, delegation, skill loading, and human input. These recipes train model decisions without quietly giving a small adapter workstation authority. The bridge holds TINKER_API_KEY; the startup script removes it from Eve's process environment. Both servers bind to loopback by default, and the authored HTTP channel refuses production use until its placeholder auth policy is replaced.

License #

MIT. See LICENSE.