End-to-end test harness #
e2e/run.sh is the supported suite, lane and native pytest selection entry.
It enters the pinned .#e2e Nix shell when needed. Python packages come from
that shell; ordinary suites do not install packages or create a virtualenv.
Use the CI/VM admission procedure before running builds or real-service tests.
e2e/run.sh one -- e2e/linecov/tests/t_git_hooks.py
e2e/run.sh selftest
e2e/run.sh standard
e2e/run.sh full
e2e/run.sh linecov --no-coverage
e2e/run.sh linecov --no-coverage -- e2e/linecov/tests/t_git_hooks.py
e2e/run.sh linecov --no-coverage -- -k expression
e2e/run.sh integration loopback
e2e/run.sh integration bsky
e2e/run.sh integration oauth-domains
e2e/run.sh feature-crawl parallel
e2e/run.sh feature-crawl flows cms
e2e/run.sh stress -- -k expression
e2e/run.sh minimal-build
e2e/run.sh feature-matrix --full
Linecov uses stock pytest/xdist in three separately accounted phases:
parallel modules (loadfile), pooled modules (loadgroup, at most four workers),
then exclusive tests without xdist. The frozen full-selection floors are
334, 38 and 19. Each phase reports its own receipt; no aggregate parser
combines the three phases. Native paths, node IDs, -k and -m follow --.
The complete selection must collect successfully and contain tests before its
populated phases run. Empty selections, collection errors, lost workers,
skips and xfails fail the run. A surviving server stops later phases.
The root conftest.py translates EXCLUSIVE, POOL_LEASE and POOL_SPLIT
module attributes, assigns worker accounts before imports and verifies actual
execution. Supported pytest calls pass e2e/pytest.ini explicitly. Integration
retains its fixtures, markers and independent submodule dispatch; miscellaneous
and standalone integration directories are excluded from ordinary discovery.
standard accounts for harness controls, formatting, Clippy, concurrency shape,
JavaScript, Rust crates, setup, browser crawl, integration loopback and the three
linecov phases. full also accounts for stress, integration bsky, feature matrix
and minimal-build. Setup failure prevents downstream tests. Other lane failures
remain visible in the final summary and cannot make later nonexecution green.
setup.sh defines shell/interpreter helpers when sourced, and builds binaries,
prepares companion references and checks runtime prerequisites when executed.
coverage.sh owns the separate instrumented target, profile merge and report.
Run e2e/run.sh linecov for coverage; --no-coverage uses the plain target.
Coverage uses the same canonical feature definition as ordinary builds.
A failed report is a failure, never permission to publish an older report.
ci/suite-outer.sh validates CI refs, prepares the disposable container and drops
privileges before checking the actual UID and capabilities. Only matching
standard-* and full-* push refs are accepted; test-* and full-test-* are
explicitly refused. lane_accounting.py reads tool receipts and rejects missing,
duplicate or inconsistent results.
The browser crawl keeps its stdlib worker pool and owned process-group cleanup.
lib/live_server.py owns server lifecycle and exact replacement cleanup; tests
still drive real servers, browsers and PDS accounts. The optional Iroh crawl has
separate dependencies and remains an explicit opt-in.
See TESTING.md for verification scope and FEATURE-MATRIX.md for feature matrix details. Product behavior tests are preserved separately from harness controls; passing accounting controls is not a product-suite result.
Useful diagnostic labs, mutation arms, release-artifact inspection and standalone
integration scripts are parked under misc/. Normal discovery excludes that
whole directory. Standalone integration entry remains run.sh integration GROUP;
the scripts execute against the original behavior-test directories. Labs need
their documented prepared container/UID environment; release inspection needs a
release artifact. They are opt-in diagnostics, not standard/FULL lanes.
File map #
The list covers every tracked harness, support and parked file. The explicit
behavior-test membership, frozen at the M3 base, and
integration test-data/ are excluded. python3 e2e/check-readme-map.py checks
set equality; the harness regression also checks that a scratch rename fails.
e2e/FEATURE-MATRIX.mde2e/README.mde2e/TESTING.mde2e/__init__.pye2e/browser_preflight_selftest.pye2e/check-feature-matrix.she2e/check-readme-map.pye2e/ci/suite-outer.she2e/concurrency-model-shape.she2e/conftest.pye2e/coverage.she2e/feature-crawl/README.mde2e/feature-crawl/__init__.pye2e/feature-crawl/api.pye2e/feature-crawl/config.pye2e/feature-crawl/coverage_map.pye2e/feature-crawl/crawl.pye2e/feature-crawl/diagnostics.pye2e/feature-crawl/flows/__init__.pye2e/feature-crawl/flows/acl.pye2e/feature-crawl/flows/auth.pye2e/feature-crawl/flows/cli.pye2e/feature-crawl/flows/cms.pye2e/feature-crawl/flows/collab.pye2e/feature-crawl/flows/git.pye2e/feature-crawl/flows/iroh.pye2e/feature-crawl/flows/oauth.pye2e/feature-crawl/flows/publish.pye2e/feature-crawl/flows/routing.pye2e/feature-crawl/flows/security.pye2e/feature-crawl/flows/sharing.pye2e/feature-crawl/flows/shell.pye2e/feature-crawl/flows/sites.pye2e/feature-crawl/flows/smoke.pye2e/feature-crawl/iroh_crawl.pye2e/feature-crawl/iroh_support.pye2e/feature-crawl/pages/__init__.pye2e/feature-crawl/pages/dashboard.pye2e/feature-crawl/pages/login.pye2e/feature-crawl/report.pye2e/feature-crawl/scope_selftest.pye2e/feature-crawl/server_log_selftest.pye2e/feature-crawl/server_probe_selftest.pye2e/feature-crawl/steps.pye2e/feature-matrix-features.pye2e/feature-matrix.she2e/flow_timeout_selftest.pye2e/git_bootstrap_selftest.pye2e/integration/.gitignoree2e/integration/Caddyfilee2e/integration/OAUTH-DOMAINS.mde2e/integration/conftest.pye2e/integration/hugo/conftest.pye2e/integration/import_smoke.pye2e/integration/iroh-sync/README.mde2e/integration/iroh-sync/conftest.pye2e/integration/keycloak/conftest.pye2e/integration/keycloak/setup.mde2e/integration/oauth_domains.pye2e/integration/publish/.testenv.examplee2e/integration/publish/README.mde2e/integration/publish/conftest.pye2e/integration/setup.mde2e/integration/sse/conftest.pye2e/integration/start_caddy.she2e/integration/support.pye2e/integration/test_standard_site_refresh.py— product OAuth regression added after the frozen inventorye2e/lane-floors.enve2e/lane_accounting.pye2e/lib/__init__.pye2e/lib/browser.pye2e/lib/build_status.pye2e/lib/collab.pye2e/lib/config.pye2e/lib/custom-pds.jsone2e/lib/custom_pds.pye2e/lib/features.enve2e/lib/git_bootstrap.pye2e/lib/git_state.pye2e/lib/gitdaemon.pye2e/lib/gitshim.pye2e/lib/harness.pye2e/lib/httpc.pye2e/lib/install-reference.she2e/lib/irohlib.pye2e/lib/live_server.pye2e/lib/pool_purge.pye2e/lib/ports.pye2e/lib/public_runtime.pye2e/lib/real_pds.pye2e/lib/secret_mapping.pye2e/lib/shell_test_env.pye2e/lib/standard_site_records.pye2e/lib/test_audit_retirement.pye2e/lib/test_build_status.pye2e/lib/test_commit_observer.pye2e/lib/test_compact_oracle.pye2e/lib/test_consolidated_entry.pye2e/lib/test_covenv.pye2e/lib/test_crawl_arguments.pye2e/lib/test_crawl_cancellation.pye2e/lib/test_crawl_silent.pye2e/lib/test_custom_oauth_isolation.pye2e/lib/test_custom_pds.pye2e/lib/test_e2e_shell.pye2e/lib/test_git_failure_diagnostics.pye2e/lib/test_git_state.pye2e/lib/test_helper_target.pye2e/lib/test_install_reference.pye2e/lib/test_iroh_observer.pye2e/lib/test_lane_accounting.pye2e/lib/test_live_server_crash.pye2e/lib/test_live_server_replacement.pye2e/lib/test_minimal_build.pye2e/lib/test_owned_readiness.pye2e/lib/test_parked_discovery.pye2e/lib/test_pds_companion.pye2e/lib/test_pool_purge.pye2e/lib/test_process_state.pye2e/lib/test_pytest_phases.pye2e/lib/test_readme_map.pye2e/lib/test_receive_push_settle.pye2e/lib/test_render_declaration.pye2e/lib/test_runner_arguments.pye2e/lib/test_scaffold_request.pye2e/lib/test_secret_mapping.pye2e/lib/test_shared_python.pye2e/lib/test_staging_observer.pye2e/lib/test_stress_full.pye2e/lib/test_stress_maintenance.pye2e/lib/test_stress_oracle_lock.pye2e/lib/test_suite_naming.pye2e/lib/test_workflow_drift.pye2e/lib/test_workflow_pin_gate.pye2e/lib/wsclient.pye2e/linecov/.gitignoree2e/linecov/artifacts.pye2e/linecov/gaps.pye2e/migration/README.mde2e/migration/process-state.jsone2e/migration/process_state.pye2e/minimal-build/cases.jsone2e/minimal-build/migration.jsone2e/minimal-build/ordinary-cases.jsone2e/minimal-build/run.pye2e/minimal-build/run.she2e/minimal-build/test_minimal.pye2e/misc/devshell_hook_arms.she2e/misc/integration/iroh-sync/run-tests.she2e/misc/integration/keycloak/run.she2e/misc/integration/publish/run-tests.she2e/misc/lab/pw-loader-lab.she2e/misc/lab/uid-drop-lab.she2e/misc/reference_target_dir_arms.she2e/misc/release-carries-no-debug-trackers.she2e/product-test-files.txte2e/pytest.inie2e/run.she2e/sabotage-command-gate.pye2e/sabotage-write-gate.she2e/selftest.she2e/setup.she2e/stress/__init__.pye2e/stress/lib.pye2e/test_staging_observers.pye2e/workflow-pin-gate-test.she2e/workflow-pin-gate.sh
Offline mutation controls and fix-verification tools now live in the coordination
repository's e2e-utils.
That directory documents --repo, preserved original hashes, working controls and
historical controls whose removed subjects or unmet prerequisites prevent a verdict.
The reviewed copy landed before source removal (coordination PR420, mergea9576e66).