Contributing to Pareidolia #

Development environment #

The blessed workflow is Nix:

nix develop                 # drops you into a shell with toolchain + tools
just                        # lists workflow recipes
just test                   # everything CI runs, plus what doesn't gate it
just ci-fast                # just what CI runs on every push

The Rust toolchain version is pinned in flake.nix and mirrored by rust-version in the root Cargo.toml. Bump them together.

Without Nix #

If you don't use Nix, you'll need the following installed manually. The Nix devShell provides all of these automatically.

Tool	What it's for	Install
Rust 1.75+	Compiling everything	rustup install 1.75.0 && rustup default 1.75.0
rustfmt	Format checks (just fmt-check)	rustup component add rustfmt
clippy	Lint (just lint)	rustup component add clippy
cargo-nextest	Test runner used by every test recipe	cargo install cargo-nextest --locked
cargo-sweep	Stale-artifact GC (just sweep)	cargo install cargo-sweep --locked
cargo-mutants	Mutation testing (just mutants)	cargo install cargo-mutants --locked
just	Recipe runner (everything in justfile)	cargo install just --locked or your distro's package manager
pkg-config	Native dep discovery (needed in M2+)	distro package (e.g. apt install pkg-config)

CI's lanes (.tangled/workflows/ci-*.yml) are deliberately thin wrappers around just recipes, so anything you can run locally with just is exactly what CI runs.

Build-cache hygiene #

Pareidolia deliberately structures its build to avoid the kind of target/ explosion documented in mnemosyne#2 (81.9 GiB reclaimed in one cargo clean). The relevant rules:

1. One integration-test binary, workspace-wide #

Every file in a Cargo tests/ directory compiles to its own statically-linked binary, each linking the entire crate graph. With many test files this multiplies into gigabytes of redundant artifacts.

Rule: all cross-crate integration tests live in crates/integration-tests/tests/integration.rs, organised as modules under tests/integration/. Adding a new top-level tests/*.rs file anywhere in the workspace is a regression. See matklad's post for the rationale.

2. No build.rs files #

Build scripts that consult external state (databases, network, generated code) trigger full recompiles and leave stale .rlib artifacts behind. Pareidolia has zero build scripts and intends to keep it that way. If you find yourself reaching for one, raise an issue first — there's almost always an alternative (env!, include_bytes!, vendored generated code).

3. No examples/, no doctests #

Both compile as separate binaries. Every crate sets doctest = false in its [lib] section; we don't ship examples/. If you have a runnable snippet worth testing, write it as a real test in the integration-tests crate.

4. Workspace-deduplicated dev-dependencies #

Every shared dep is declared once in the root [workspace.dependencies] and referenced via workspace = true from member crates. Prevents linking multiple versions of the same crate.

5. cargo-sweep for stale artifact GC #

Cargo never deletes stale hashed artifacts. Run periodically:

just sweep                  # drops artifacts untouched for 14 days

cargo-sweep is included in the dev shell.

6. CI disables incremental compilation #

The rustc incremental cache (target/debug/incremental/) grows unbounded and is pure waste in CI (which always does fresh-ish builds). All CI lanes export CARGO_INCREMENTAL=0. Local builds keep incremental on for iteration speed.

7. Shared CARGO_TARGET_DIR (optional) #

If you set CARGO_TARGET_DIR to a user-wide location, the dev shell respects it. We don't force this because of lock-contention edge cases, but if your disk is under pressure across many Rust projects, it's worth setting.

Testing philosophy #

See the project plan (in commit history / README evolution) for the full testing strategy. The short version:

• Property tests (proptest) wherever inputs have a meaningful generator and properties are invariant.
• Golden tests for the ML pipeline — known images produce known embeddings within tolerance.
• One integration binary (see above) for cross-crate flows.
• NixOS VM tests under nix/tests/ for the real-PAM-real-systemd story. This is the test layer that justifies daily-driving the module.
• Mutation testing (cargo-mutants) gates releases, not PRs. Surviving mutants in enrollment, matching, ipc, pam must be addressed before a release tag.

If a test passes when its target code is replaced with unimplemented!(), it isn't a test. Mutation testing exists to surface that case.

Hardware-dependent tests #

Tests that require real hardware (a V4L2 camera node, eventually a CUDA device, etc.) are marked #[ignore = "requires …"] so the default test runner skips them. CI never runs them — it has no hardware. Locally:

just test-features          # runs v4l2 + inference unit tests (no hardware)
just test-hardware          # runs only the ignored / hardware-gated tests

If you touch a hardware-facing module (pareidolia_core::camera::v4l2, the inference backends, etc.), run just test-hardware on your machine before requesting review. The PR description should call out which physical devices and model files the tests were exercised against.

Model weights #

ONNX model weights are pinned in nix/models.nix and fetched on demand by Nix (fixed-sha256, sandbox-safe). The devShell exports PAREIDOLIA_TEST_MODEL / PAREIDOLIA_TEST_SCRFD / PAREIDOLIA_TEST_ARCFACE pointing at the materialised files, so just test-hardware inside nix develop runs the model-loading tests with no manual setup.

Outside Nix, fetch the files manually (see nix/models.nix for the upstream URLs) and set the env vars yourself.

The pinned models can also be materialised standalone:

nix build .#scrfd-10g       # → ./result symlink to the .onnx
nix build .#arcface-glintr100

Feature flags #

pareidolia-daemon and pareidolia-cli default to a set of features that make a normal cargo build (or nix build) produce working binaries with camera capture and inference wired in:

• v4l2 — Linux V4L2 capture (needs libv4l + bindgen prerequisites in the build environment).
• inference — ONNX Runtime inference (needs libonnxruntime; the devShell sets ORT_DYLIB_PATH to nixpkgs's .so).

cargo build --no-default-features opts out of both — what CI uses, since its Nixery image doesn't ship libv4l, libonnxruntime, libclang, or kernel headers. CI threads this through via the CARGO_ARGS env var that the justfile recipes read.

pareidolia-core itself keeps both features opt-in (default features are empty) so consumers that only need the pipeline abstractions (mocks, format conversions, IPC types) don't pay the build cost.

Note that [workspace.dependencies] sets default-features = false on every internal crate. This is load-bearing: cargo --workspace unifies features across the dep graph, and without that setting the daemon's default-on v4l2 would transitively re-enable itself for every dependent crate (integration tests, PAM module) and negate the CI opt-out. Crates that genuinely need another internal crate's default features can flip default-features = true back on in their own dep entry.

UI architecture #

Pareidolia today exposes a CLI. The long-term plan does not include a GUI in any milestone, but we keep the codebase ready for one (or for any other UI — TUI, RPC client, Wayland session helper) by enforcing one rule:

Decisions live in pareidolia-core. Rendering lives in pareidolia-cli (and any future UI crate). They communicate via structured outcome types.

In practice that means:

1. Anything that can be decided without printing belongs in core. "Which IR camera should we use given these args and this hardware snapshot?" is a pure function of its inputs; it returns a typed enum and prints nothing. Example: [pareidolia_core::camera::select::resolve_ir_camera].
2. CLI subcommands are thin formatters. They parse args into a shape the core function understands, call the core function, and translate the structured result into stderr / stdout text. The match arm that picks between two output strings is rendering; the match arm that picks between two operations is a decision and belongs in core.
3. Long-running operations report progress as structured events rather than printing inline. (We don't have a full event-stream abstraction yet — the enroll loop still prints directly — but per-iteration state goes into typed structs as we touch each command.)
4. Errors carry structure, not just strings. Variant-rich error enums (StoreError, CliError) let UIs map specific failure modes to specific recovery flows. New errors that boil down to "user did something wrong with this specific remediation" should be their own variant, not a Config(String).

What we do not do #

• No interactive prompts. They add modal state, break under script(1) / piping, and a GUI couldn't reuse them anyway. Where the resolver would need to ask the user a question — "you have two IR cameras, which one?" — we emit an [Ambiguous]-style outcome and let the UI layer decide how to surface the choice (CLI prints a warning and falls back; GUI shows a dropdown).
• No spinners, colour, or TUI widgets. Plain text only. Survives piping, scriptability, log-capture, and accessibility tooling.
• No pareidolia init-style wizards. Setup is one command (enroll) plus optional config-file edits; a wizard is more lines of code than docs.

Worked example: IrCameraResolution #

[crates/core/src/camera/select.rs] is the canonical instance of the pattern. Read it first when extending another subcommand.

• The enum has one variant per state a UI might need to render differently (explicit, opt-out, no-hardware, auto-detected, ambiguous), not just Some / None.
• The resolver is a pure function that takes &[CameraInfo] (not an internal call to enumerate()), so unit tests can drive it with synthetic snapshots and a future caller can pass a cached or user-filtered list.
• The CLI's match in crates/cli/src/commands/enroll.rs is purely about how each variant renders, not what each variant means.

Things that already follow the pattern #

• pareidolia_core::camera::{enumerate, CameraInfo} + the CLI's pareidolia cameras subcommand (data in core, table renderer in CLI with its own unit tests).
• pareidolia_core::enrollment::{MatchOutcome, DualMatchOutcome, SingleVerdict} + the CLI's print_single_outcome / print_dual_outcome in pareidolia test.
• pareidolia_core::enrollment::{AuthMode, EnrollmentReadiness, EnrollmentSessionSummary} — added alongside this section.

Things that don't yet (and can move when touched) #

• list.rs, remove.rs — small, mostly rendering already; revisit if a richer UI for either lands.
• Enroll-load disposition (existing-vs-new), next-id allocation — could move into core helpers but the payoff is tiny.

Security #

What Pareidolia matches #

Pareidolia performs face-identity matching via the ArcFace model on embeddings extracted from each captured face. The model produces a 512-dimensional vector per face; cosine similarity between vectors clusters by identity. The matcher picks the highest-similarity enrolled sample and grants auth when that similarity clears a configurable threshold (0.6 by default).

This is not liveness detection. ArcFace by itself cannot distinguish a real face from a sharp photograph of one — that is an architectural property of every single-modality identity-matching system without an added anti-spoof model.

Defense mode: dual-modality authentication #

The recommended mode whenever the user has IR samples enrolled is dual-modality: pass --rgb-camera and --ir-camera together to both pareidolia enroll and pareidolia test. With both pools populated, [Enrollment::authenticate_dual] requires both modalities to clear the threshold independently before granting auth. The daemon (M5+) will default to dual mode automatically whenever the enrollment supports it.

Dual-modality auth defeats the common presentation attack — a printed photo or a face displayed on a phone/laptop screen — without any additional ML model:

• Phone and laptop screens emit visible light but almost no near-infrared. In an IR illuminator's beam such a screen is just a dim rectangle, and SCRFD typically does not detect a face in it.
• Printed photos vary in NIR reflectivity by pigment, but most prints either fail face detection in IR or produce IR embeddings too distant from the enrolled IR samples to clear the threshold.

This is a partial defense, not bulletproof. Specifically, it does not defeat:

• NIR-tuned print attacks (photos printed with pigments chosen for skin-like NIR reflectivity).
• 3D masks built with realistic skin-like NIR behaviour.
• A photograph illuminated by an NIR source matching the camera illuminator's spectrum.

A learned, model-based PAD (presentation-attack detection) is still a future milestone and would be required to defend against the high-effort attacks above. A model-free PAD layer exists today as defense-in-depth — see below.

Liveness detection (model-free PAD) #

Liveness — "is this a real face present now, not a photo or a screen?" — is an axis orthogonal to identity matching, implemented in pareidolia_core::liveness as a set of pluggable checks combined by a policy. It's additive hardening on top of dual-modality, not a replacement: on a NIR sensor, screens are largely defeated already by simply not registering a face in IR (SCRFD vetoes the IR side), and liveness raises the bar against the cases that do get a face.

Composition. Each LivenessCheck returns a 0..1 confidence-live score and declares a CheckRole:

• Evidence ("I see signs of life") combines by noisy-OR — more applicable evidence can only raise confidence, and a single check of score s contributes exactly s (no inflation; this rules out a weighted average, which would scale one weak signal up to full range).
• Veto ("I see signs of a spoof") caps the result multiplicatively: combined = evidence × ∏(veto). A confident veto drives the score to ~0 regardless of evidence. A veto-only policy reduces to the vetoes' verdict (the evidence factor is a neutral 1.0 when no evidence check applies); "nothing applicable at all" scores 0 and so fails closed at any positive threshold.

A check returns NotApplicable (contributing nothing, capping nothing) when it can't run on the frames it got — so a strobed-only check on a constant sensor, or any check on hardware it doesn't support, is purely additive and never breaks a working config. This is also why a strobed illuminator is structurally more secure: it has strictly more applicable checks, so a higher achievable confidence ceiling.

Checks that exist:

• NIR reflectance gate (veto, single-frame): screens emit ~no near-IR (dark/flat under the sensor → vetoed); skin has a characteristic NIR reflectance and sub-surface scattering, so a real face is bright, structured, and diffusely smooth. Combines face-region brightness, structural contrast, and a Laplacian-based smoothness proxy by taking their minimum (an attack need fail only one). Measures the detected face region, not the whole frame.

A dead end worth recording: raw inter-frame motion magnitude was tried and removed. Empirically it is anti-correlated with liveness — a hand-waved photo or a flickering screen changes more than a still live face — matching the PAD literature's lesson that liveness is in signal structure, not magnitude. The intended replacement is a planar-homography motion-residual check (rigid/planar vs 3D), not yet built.

Tuning is mandatory and data-driven. The checks' thresholds ship as guesses and liveness defaults to observe-only: --liveness-threshold 0.0 computes and logs the score but never rejects. You must profile real live-vs-spoof captures on your own sensor and conditions before raising the threshold, via pareidolia calibrate-liveness (capture labeled batches per attack type → report per-feature separation → suggest data-derived thresholds). A profile is only valid for similar capture conditions (notably: indoor vs daylight, since ambient near-IR shifts every brightness-based feature).

Honest scope. Model-free liveness stops casual attacks (a phone, a plain printed photo) on a given sensor; it will not stop a determined attacker who tailors the artifact (NIR-reflective mask, NIR-tuned print, NIR-leaking display). Evaluate per-attack-type (ISO/IEC 30107-3 APCER/BPCER), not as a single accuracy number. The framework's value is that a learned anti-spoof model drops in later as just another LivenessCheck.

Family resemblance and identity discrimination #

ArcFace clusters by identity, and biological relatives occupy regions of the embedding space that are closer together than unrelated people. A child, sibling, or parent may produce a similarity above the default 0.6 threshold against a relative's enrollment — this is identity confusion, not spoofing, and exists independently of liveness or the choice of single-vs-dual modality. Remedies:

• Raise --threshold to be stricter at the cost of more false-rejects on poses or lighting the user didn't enroll for.
• Re-run pareidolia enroll to add more sample variety to the user's own pool. Best-of-N matching is monotonically improving in N, so this raises the bar against close-relative false-accepts without making the user's own auth more fragile.

Defense matrix #

Attack	Single (RGB only)	Dual (RGB + IR)	+ Liveness (NIR gate, once tuned)
Phone or laptop screen showing a photo	passes	rejected (SCRFD vetoes the IR side)	rejected (dark/flat in IR)
Printed photo on paper	passes	likely rejected	gate may also veto (wrong NIR reflectance)
NIR-tuned printed photo	passes	may pass	may pass
3D mask with skin-like NIR	passes	may pass	may pass
Live family member	may pass	may pass	may pass (identity confusion, not spoofing)
Same person, hardware variation	passes	passes	passes

The liveness column assumes --liveness-threshold has been raised from its observe-only default of 0.0 after tuning (see Liveness detection); at the default it logs but never rejects.

pareidolia test mirrors the modes available to the daemon, so it's useful for sanity-checking which attacks your current enrollment would let through before exposing it to PAM.

Source control #

The canonical workflow uses Jujutsu (jj). The repository is also a Git repo for compatibility, but please prefer jj commands when working locally.

License #

By contributing you agree that your contributions are dual-licensed under MIT and Apache-2.0, matching the project license.