Something went wrong. Try again.
the cms for static site generators on atproto
Something went wrong. Try again.
lichen.page justfile
15 kB · 288 lines
Just
at main
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289# The PR gate, as commands rather than prose.## `just gate` is the whole pre-PR check: fmt, clippy, tests. It replaces a paragraph in# AGENTS.md that told you which crates to lint and which feature flags to pass — a list# that could (and did) drift from the code it described.## just gate fmt + clippy (scoped to what you changed) + tests# just crates print which crates the gate would lint, and why# just e2e ARGS... run e2e/run.sh in the repository e2e Nix shell# just gate-all the full-workspace sweep (pre-release, not per-PR)## Every recipe runs cargo directly; run them inside `nix develop`.
# the ONE feature set for both clippy and tests. most of this codebase is feature-gated, so a# bare command lints and tests none of the git/atproto/hugo/mdbook code — the gap that let an# E0407 under --features mdbook reach main. cargo also fingerprints artifacts per feature# selection, so using one set here means the dependency graph is built once, not twice.# ONE-BUILD-1: the value now lives in e2e/lib/features.env and is READ here, so the gate and the# e2e lanes cannot drift. The old literal was lichen-server/git,lichen-server/atproto,# lichen-cms/hugo,lichen-cms/mdbook -- which left lichen-server's OWN hugo/mdbook OFF while the# e2e suite built them ON, so cargo re-fingerprinted the whole graph between the gate and the# lanes. That is the "built once, not twice" above, extended past clippy-vs-test to the suite.FEATURES := trim(shell('. ./e2e/lib/features.env; printf "%s" "$LICHEN_FEATURES"'))# THEIR feature model (42cc1996) now lives in features.env, so the value is lichen-cli/* rather# than lichen-server/*. The READ stays ours: a literal here and a literal there is exactly the# drift that re-fingerprinted the graph between the gate and the lanes.
# what to diff against when working out which crates changedBASE := env_var_or_default("GATE_BASE", "review/main")
# lichen-tauri is excluded everywhere: its glib-sys build script needs GTK system libs the# devshell does not provide, so clippy cannot run it at all here.EXCLUDED := "lichen-tauri"
default: @just --list
# Run an e2e lane or focused test with every argument forwarded in order.e2e *args: env -u LD_LIBRARY_PATH nix develop .#e2e --command e2e/run.sh {{args}}
# The pre-PR gate. Zero warnings is the standard; a PR that adds one is not ready.# `fmt-check`, NOT `fmt`. This dependency used to be `fmt` — `cargo fmt --all`, which REWRITES the# tree and therefore CANNOT FAIL. A gate whose formatting third repairs the thing it is checking# passes on unformatted code and reports success, and in CI it did exactly that. (`workflow-pins`# is on the CI lane's gate too; that recipe is not on this base yet.)## `fmt-check` FIRST, and the order is load-bearing rather than alphabetical. The shape script# plants banned tokens into real tracked files and restores them under a trap; if the check ran# AFTER it and the script died between a plant and its restore, the formatting check would# measure a PLANTED tree.## TODAY THAT IS LATENT, NOT ACTIVE, and the distinction is measured rather than assumed:# `lichen-lean-adversary` appended the shape script's own token to a tracked `.rs` file and ran# `cargo fmt --all --check`, which exited 0. Every current plant is either into a `.md`, which# rustfmt never reads, or a trailing `//` comment line, which rustfmt does not reformat — so an# interrupted control would leave a planted tree but would NOT produce a formatting failure.# The order still earns its place: it costs nothing and it defends the first plant that is# misformatted Rust. Please do not reorder this list to tidy it.gate: fmt-check workflow-pin-check process-state-check concurrency-model-shape clippy test-js test @echo "gate: ok"
process-state-check: python3 e2e/migration/process_state.py
workflow-pin-check: bash e2e/workflow-pin-gate.sh
fmt: cargo fmt --all
# Checks formatting and changes NOTHING. This is what `gate` depends on.## `gate` used to depend on `fmt`, i.e. `cargo fmt --all` with no `--check`, so running# the gate REFORMATTED the tree it was measuring. Two consequences, both bad:## - the result stopped being reproducible. After one run the tree is "the named rev# PLUS whatever fmt changed", which has no name. That is how a pure-shell gate run# came to measure a tree carrying 8 modified tracked files including a test file —# a result its adversary correctly refused.# - the gate could not FAIL on formatting. It silently fixed it instead, so every# green gate credited on this host included an edit nobody reviewed. Measured at# 0b9dc6a0: `cargo fmt --all --check` on a clean tree exited 1 with 19 hunks# across 8 files, all absorbed this way. They are committed in the parent commit.fmt-check: #!/usr/bin/env bash set -uo pipefail # THE LANE PRINTS ITS OWN EVIDENCE OF A PASS (orchestrator ruling 2f6d3503). # `cargo fmt --all --check` says NOTHING when it passes, so rc=0 is indistinguishable from a # recipe that ran nowhere -- which is not hypothetical: under JUST-1 this lane exited 127 for # four pushes while the gate read as green, and a grader looking only at the lane's rc could # not have told a real pass from an absent one. A pass must therefore assert what it checked. n=$(git ls-files -- '*.rs' 2>/dev/null | wc -l) case "$n" in ''|0) echo "fmt-check: CANNOT READ -- git listed $n tracked .rs files; refusing to report a" >&2 echo " pass over an empty or unreadable file set." >&2 exit 95 ;; esac echo "fmt-check: cargo=$(command -v cargo) rustfmt=$(command -v rustfmt || echo via-cargo) files=$n" cargo fmt --all --check rc=$? if [ "$rc" -eq 0 ]; then echo "fmt-check: PASS -- $n tracked .rs files checked, 0 formatting diffs" else echo "fmt-check: FAIL rc=$rc -- $n tracked .rs files checked" >&2 fi exit "$rc"
# Clippy over the crates you touched PLUS everything that depends on them.## clippy lints only the packages you name — dependencies are built with plain rustc — so# linting just the touched crate would miss a warning your change caused in a consumer.# The dependent set is derived from cargo metadata, never hand-maintained.clippy: #!/usr/bin/env bash set -euo pipefail # NO BASE MEANS LINT EVERYTHING, NOT LINT NOTHING. Scoping is a SPEED optimisation for local # work, where `review/main` resolves; it is not the definition of the check. A checkout that # cannot name a base (every CI clone) must fall back to the whole workspace and SAY SO — # linting nothing there would be a silent pass on the lint third of the gate. if ! crates="$(just _scoped-crates)"; then all="$(just _all-lintable-crates)" # NAME THEM. "the WHOLE workspace" is a claim the next reader trusts instead of counting. echo "clippy: no resolvable base — linting every lintable crate: ${all//$'\n'/ }" args=(); for c in $all; do args+=(-p "$c"); done cargo clippy "${args[@]}" --all-targets --features {{FEATURES}} exit 0 fi if [ -z "$crates" ]; then echo "clippy: base resolved and no crate sources changed — nothing to lint" exit 0 fi args=(); for c in $crates; do args+=(-p "$c"); done # `--features pkg/feat` is only legal when `pkg` is one of the SELECTED packages, so the # flags follow the highest selected layer: the executable forwards to the server, # which forwards SSG and git features to the CMS feats="" if grep -qxF lichen-cli <<<"$crates"; then feats="{{FEATURES}}" elif grep -qxF lichen-server <<<"$crates"; then feats="lichen-server/git,lichen-server/atproto,lichen-server/hugo,lichen-server/mdbook" elif grep -qxF lichen-cms <<<"$crates"; then feats="lichen-cms/git,lichen-cms/hugo,lichen-cms/mdbook" fi echo "clippy: ${crates//$'\n'/ }${feats:+ (--features $feats)}" if [ -n "$feats" ]; then cargo clippy "${args[@]}" --all-targets --features "$feats" else cargo clippy "${args[@]}" --all-targets fi
# Frontend regressions run production JavaScript with virtual clocks and controlled responses.# THEIRS (42cc1996), restored by hand: our side had deleted this recipe long ago (0d8527b7 recorded# that our gate omitted it while the rest of the lineage ran it), so the 3-way merge kept OUR# deletion while I was taking THEIR `gate:` line -- and `just --list` refused with# error: Recipe `gate` has unknown dependency `concurrency-model-shape`# The YAML/justfile parse check is what caught it; a push would have failed the gate outright.concurrency-model-shape: e2e/concurrency-model-shape.sh
test-js: node --test crates/lichen-cms/tests/*.test.cjs
# Tests at the SAME feature set clippy used, so the tests cover the code that was just linted.# (At default features this command compiles no git/atproto/hugo/mdbook code at all.)# BOTH INVOCATIONS RUN, AND BOTH OUTCOMES ARE REPORTED.## These were two plain recipe lines, and `just` ABORTS A RECIPE AT THE FIRST FAILING LINE. So# whenever lichen-cms/lichen-server was red — which it was in CI all day on 2026-09-02 — THE# ENTIRE lichen-flat-file SUITE SILENTLY NEVER RAN. Not skipped, not reported: absent. The log# showed exactly one `test result:` line and no mention of cap-std, and nothing said so.## What was lost is the point. That second invocation carries `fs_pins_on_every_backend`, whose own# reason for existing is to make "a silently-untested backend loud" — an instrument built against# silence, silenced by the line above it. It also carries the four Landlock denial arms, which are# the only things in this repository that prove confinement ENFORCES rather than merely reporting# itself available.## NOT FIXED BY REORDERING. Putting flat-file first would have surfaced the arms and left the hole# exactly where it is, one line further down — evidence obtained, defect preserved, and it would# have read as a repair. Ruled by `lichen-orchestrator` 2026-09-02.## The outcomes start at `did-not-run` and are printed from an EXIT trap, so an invocation that# never happens is DISTINGUISHABLE FROM ONE THAT PASSED even if this recipe dies partway. "Absent"# and "green" must never again be the same output.test: #!/usr/bin/env bash set -uo pipefail cms=did-not-run flat=did-not-run summary() { echo "test: lichen-cms + lichen-server ${cms}" echo "test: lichen-flat-file (cap-std) ${flat}" } trap summary EXIT
cargo test -p lichen-cli -p lichen-cms -p lichen-server --features {{FEATURES}} --no-fail-fast cms_rc=$? [ "$cms_rc" -eq 0 ] && cms=ok || cms="FAILED (exit $cms_rc)"
# lichen-flat-file: the conformance suite must run on BOTH resolution backends (openat2 + # cap-std) — the crate's `fs_pins_on_every_backend` refuses to pass on one backend unless # told so, which is what makes a silently-untested backend loud. It runs REGARDLESS of the # result above, because a red in another crate is not a reason to stop testing this one. cargo test -p lichen-flat-file --features cap-std --no-fail-fast flat_rc=$? [ "$flat_rc" -eq 0 ] && flat=ok || flat="FAILED (exit $flat_rc)"
# fail if EITHER failed; the summary has already named which [ "$cms_rc" -eq 0 ] && [ "$flat_rc" -eq 0 ]
# The full sweep — pre-release or a specifically planned e2e pass, not per-PR.gate-all: just fmt-check cargo clippy -p lichen-cms -p lichen-server -p lichen-cli -p lichen-iroh -p lichen-shell -p lichen-yrs \ --all-targets --features {{FEATURES}} just test-js cargo test -p lichen-cli -p lichen-cms -p lichen-server --features {{FEATURES}} --no-fail-fast cargo test -p lichen-flat-file --features cap-std --no-fail-fast LICHEN_FLAT_FILE_ONE_BACKEND=1 cargo test -p lichen-flat-file --no-default-features --no-fail-fast
# Show the scoping decision: what changed, and what that expands to.crates: #!/usr/bin/env bash set -euo pipefail echo "base: {{BASE}}" echo "touched: $(just _touched-crates | tr '\n' ' ')" echo "linting: $(just _scoped-crates | tr '\n' ' ')"
# ── internals ────────────────────────────────────────────────────────────────────────────
# Workspace crates with a source file changed vs BASE, plus anything uncommitted — a gate you# run before committing has to see the working tree, not just what is already in history.# AN UNRESOLVABLE BASE IS "THE QUESTION COULD NOT BE ASKED", NOT "NOTHING CHANGED". BASE defaults# to `review/main`, and `review` is a REMOTE NAME in our own worktrees: a fresh CI checkout has no# such ref. The previous version swallowed that with `|| true`, so merge-base failed on EVERY push,# the set was always empty, and `just clippy` printed "no crate sources changed — nothing to lint"# and exited 0 — the lint third of the per-push gate inert while printing a line that reads like a# correct scoping decision. Exit 9 says the instrument could not reach its subject; callers that can# proceed without scoping handle it (see `clippy`)._touched-crates: #!/usr/bin/env bash set -euo pipefail if ! git rev-parse --verify --quiet "{{BASE}}^{commit}" >/dev/null; then echo "just: BASE '{{BASE}}' does not resolve, so changed crates cannot be derived." >&2 echo " This is NOT 'nothing changed' — it is 'the question could not be asked'." >&2 echo " Set GATE_BASE to a ref this checkout has (e.g. GATE_BASE=HEAD~1), or use a" >&2 echo " full-workspace lint. Callers that can proceed without scoping handle exit 9." >&2 exit 9 fi base="$(git merge-base HEAD "{{BASE}}")" { git diff --name-only "$base"...HEAD git diff --name-only HEAD || true git ls-files --others --exclude-standard || true } | sed -n 's#^crates/\([^/]*\)/.*#\1#p' | sort -u
# The touched crates plus their transitive dependents, from cargo metadata rather than a table# in a doc: a hand-maintained list is one refactor away from being wrong, silently._all-lintable-crates: #!/usr/bin/env bash set -euo pipefail cargo metadata --no-deps --format-version 1 \ | jq -r '.packages[].name' \ | grep -vxF "{{EXCLUDED}}" | sort -u
_scoped-crates: #!/usr/bin/env bash set -euo pipefail touched="$(just _touched-crates)" [ -n "$touched" ] || exit 0 # "<dependency> <dependent>" for workspace-internal edges only edges="$(cargo metadata --no-deps --format-version 1 \ | jq -r '.packages[] as $p | $p.dependencies[]? | select(.path != null) | "\(.name) \($p.name)"')" set="$touched" # fixpoint: keep pulling in dependents until nothing new appears while :; do added=0 while read -r dep dependent; do [ -n "$dep" ] || continue if grep -qxF "$dep" <<<"$set" && ! grep -qxF "$dependent" <<<"$set"; then set="$set"$'\n'"$dependent"; added=1 fi done <<<"$edges" [ "$added" -eq 1 ] || break done grep -vxF "{{EXCLUDED}}" <<<"$set" | sort -u