Test execution #
test.sh is the pre-commit gate. With no arguments it selects
Python end-to-end scenarios affected by the change and runs Go end-to-end tests
for CLI changes. ./test.sh --full runs all end-to-end scenarios and the web
check. Manual tests under test/manual are opt-in.
Gate and resource limits #
test/quick.sh runs formatting, type, lint, generated-code,
and Go vet checks before the suites. It also builds each
workspace package with warnings as errors, and the gate
runs each package's own tests as a suite. The gate builds a daemon snapshot and
launches suites concurrently. Its end-to-end selection and web checks depend on
the changed paths or --full. See test.sh for that selection and
Gleam lint for lint profiles.
The snapcompact tests in gleam test render frames through albedo-render. The
daemon looks for it in priv/bin, then on PATH. The gate installs it with
native/render/install.sh before its suites. When
gleam test runs alone and finds no renderer, its entry point,
test/albedo_test.gleam, runs that script first. The
script needs cargo from nix develop. Run it again after changing
native/render, because gleam test does not rebuild a renderer that is
already installed.
test/resources.py calculates concurrency limits from CPU
and available memory. It reserves one quarter of memory and gives half of the
rest to end-to-end lanes. A lane is one test on the shared daemon, two exclusive
daemons, and one Go scenario. E2E_LANE_MEMORY is what one lane adds to the
gate's peak, and the plan rejects a host where one lane does not fit.
ALBEDO_TEST_*_JOBS variables can override the calculated limits. These limits
control concurrency, not process memory allocations.
Measure the gate before changing a cost in the plan:
- Set every
ALBEDO_TEST_*_JOBSvariable to the plan under test, such as CI'stest plan:line with a different lane count. - Put
TMPDIRandALBEDO_TEST_TMPunder/dev/shm, as CI does. - Run each suite of
./test.sh --fullin its own cgroup, for example throughsystemd-run --user --scope, and sample each cgroup'smemory.current. - Compare the gate's peak at two or more lane counts.
A cgroup counts the detached daemons and the tmpfs scratch that a sum of the suite's process tree misses. Gleam lint runs in test/quick.sh before the suites start, so its peak does not add to theirs. See Gleam lint memory.
The suites start together, so the gate's peak memory is what they all hold at
that moment. CI's workflow puts TMPDIR and
ALBEDO_TEST_TMP under /dev/shm, so scratch files use the same memory. Do not
let a test start background work that it does not check. For example,
dev-proxy.test.mjs loads only the server
options from vite.config.ts. With the config's plugins, Vite pre-bundles the
client's dependencies when its server starts, and CI's fresh node_modules has
no cache for that work.
Expected reports in gleam test #
Some tests provoke failure paths on purpose, and the daemon code they drive logs those failures. A passing run prints these lines, each from the named test:
listener was killed,listener did not start: port taken, andgiving the listener upcome froma_bounded_listener_stops_after_its_failed_restarts_testin listener_test.kernel_cleanup_failedwithplugin cleanup did not finish within 1.5 scomes fromhard_timeout_requires_explicit_reset_testin runtime_test. Its cell holds the kernel's event loop, so plugin cleanup cannot start.owner_lostwithkernel ownership cleanup was not confirmedcomes from tests that end a kernel's owner:owner_loss_*in owner_supervision_test andruntime_killed_with_a_queued_reload_result_retires_the_kernel_testin runtime_ownership_test. Only the owner can forget the kernel's record. The last test also printsreload for a stopped runtime:with either that reason orkernel lost before its processes could be supervised, depending on whether the driver handles the stop or the owner's death first.spare warmup outcome is unknowncomes froman_unknown_spare_allocation_retains_budget_without_blocking_stop_test, andkernel cleanup late-cleanup: unconfirmedfromlate_cleanup_success_keeps_another_generations_failure_and_retry_test, both inruntime_ownership_test.
This list covers the listener and kernel lifecycle reports. When one of them comes from a different test, or a new one appears, find the test that printed it before treating it as expected.
Fixture teardown #
test/e2e/harness.py owns test daemons and fixtures.
peer_daemon(name, prepare=None) starts a separate daemon for a test.
finish_test cleans up peers and registered resources. Use the shared harness
for daemon scenarios. See storage migrations for upgrade
coverage.
A test daemon stops when the suite that started it exits, through
ALBEDO_PARENT_PID. A daemon that a scenario installs as a service drops that
variable, so the scenario's own cleanup must stop it. When the gate ends,
test/reap-daemons.sh waits for detached kernels to
finish their grace period, then stops any process still running from the gate's
snapshot and fails the gate, naming each process's ALBEDO_HOME. Fix the
scenario that owns that home. Run the script on the scratch root to clear
processes left by an interrupted run. That also stops the daemons of a gate
still running in another checkout.
Mixed-version moves #
move_versions_test.py runs this
checkout's daemon beside an older reference daemon. Both run as peer daemons.
Moves are real ones, over the loopback ssh that move_test.py uses. Each source
session has a transcript, kernel cells, a memory note, a work item, a vent, a
delivered webhook letter, a schedule, and a workspace link.
- A move from the reference to this daemon is the supported path. The scenario finishes the checklist, then checks the carried data and runs a cell and a turn on the target.
- A move from this daemon to the reference must be refused cleanly. The scenario
adds a column to
sessionsfirst, so the refusal does not depend on how far the pin lags. The reference predatesPOST /transfers/check, so the preflight must still offer the move. The move must then fail at staging withtransfer_target_olderand the reference's detail, leaving no receipt or staging directory there. The scenario then checks for no session on the reference, and a source session that keeps its schedule and takes input.move_test.pycovers the check against a target of this build.
The rules these scenarios check are in moves between versions.
test/reference-daemon.sh pins the reference
commit in REFERENCE. Albedo has no release tags, so the pin is a commit on
main. The script extracts it with git archive, builds it with that commit's
own test/snapshot-daemon.sh and its CLI, and prints the directory. It fetches
the commit first when a shallow clone lacks it. Builds are cached under
ALBEDO_REFERENCE_CACHE, which defaults to ~/.cache/albedo-reference, one
directory per commit. A cold build took 37 s on a developer machine with warm
Hex and Go module caches. A cached run returns at once. CI keeps the directory
in its albedo-reference cache, keyed by the script, so ./test.sh --full
builds it once per pin.
Bump the pin when a commit that users run lands, such as one deployed to another host, or when the reference falls back past a transfer protocol change.
- Set
REFERENCEto the full commit hash. - Run
nix develop -c python3 test/e2e/run.py move_versions_test.py downgrade_test.py. - Report a failure of the supported move as an incompatibility between the two builds. Do not loosen an assertion to make it pass.
reference_support.py builds the reference
on first use. Its run_reference points a test daemon's next boot at the
reference daemon and CLI. downgrade_test.py
uses it to restart a home this checkout upgraded on the reference build, and
checks that the reference reads a session this checkout wrote. A rollback across
a breaking migration is not supported, so bump the pin past each new breaking
migration. See downgrades.
areas.toml selects the file for changes to transfer
code, adapters, migrations, and the core tables in conversation.gleam and
mail.gleam. A schema change in another owner reaches it only through
./test.sh --full. Run it directly after adding a column or table that a
transfer carries.
Offline Antigravity scenarios #
antigravity_test.py drives the real adapter
through a private copy of the runner's sealed daemon. The
fixture replaces the streaming transport's
destination function and the non-streaming HTTP binding in that copy and
reseals it before boot. Production endpoints and profile settings remain
unchanged.
The route guard accepts only the exact inference URL, the dummy bearer token, and the fixture's marked home. The compiled destination is a numeric route on the loopback fake provider. Other streaming destinations and non-streaming provider HTTP calls are refused. The safety probe checks refusals before socket acquisition, including malformed headers.
Run nix develop -c python3 test/e2e/run.py antigravity_test.py for adapter
scenarios. They cover tool execution, title-request separation, usage, restart
replay, cancellation, and conditional signature fallback. Fixture signatures
check preservation, not Google's cryptographic validation. OAuth, Google TLS,
live quotas and acceptance of genuine signatures require separate live checks.
Manual web review #
Use the UI review kit for seeded browser reviews, scenario helpers, evidence capture, and private-resource cleanup. These manual checks are separate from the automated gate.