Production and federation plan #
This document is the completion checklist for turning cpds from a local,
single-account protocol implementation into a reliable Personal Data Server
that can participate in the public AT Protocol network. A checked box must be
backed by code, an automated local test, and the operational evidence named in
that section. Passing the current test suite alone does not imply production
readiness.
The intended first production topology remains deliberately small: one hosted account, one process, SQLite, and a filesystem blob store behind an HTTPS reverse proxy. Multi-account hosting and object storage are useful extensions, but they are not prerequisites for a production-grade personal server. The repository, sync, identity, authorization, recovery, and operational contracts are prerequisites.
The normative references for this work are the AT Protocol specifications for repositories, sync, event streams, accounts, DIDs, handles, and OAuth. Canonical Lexicons and language-neutral test vectors come from the reference repository. The ATrium adoption evaluation defines which Rust building blocks CPDS will compose, the trust boundaries around them, and the fixture and dependency gates required before they carry production data.
Current baseline #
The current implementation already provides:
- zero-configuration loopback startup with one development account;
- SQLite persistence and atomic filesystem publication for blobs;
- password hashing, one-time OAuth authorization codes, S256 PKCE, and hashed opaque bearer tokens;
- public record create/get/list/delete and blob upload/get endpoints;
- official-fixture-matched schema-independent AT Protocol data-model encoding, native DAG-CBOR CID links/bytes, and deterministic record CIDs;
- a
did:webdocument with an atproto signing key and PDS service endpoint; - an extensive alpha Spaces implementation with granular scopes, signed commits, DPoP-bound space credentials, CAR recovery, and authenticated notifications;
- local unit, adapter-contract, and router-level integration tests.
The complete baseline gate is:
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo test --all-targets --locked
cargo deny --locked check
The public repository and OAuth implementations are intentionally smaller than the network protocols they resemble. They must not be advertised as federating until the federation gates below pass.
Gap and acceptance matrix #
1. Local and network-safe configuration #
Current gap: development values are implicit in every environment. A process can listen publicly while retaining a development password, signing key, HTTP origin, relative data paths, or static test client registration.
Completion evidence:
2. Durable repository-v3 state #
Current state: public creates, updates, and deletes now use a signed repository-v3 MST and an atomic SQLite write batch. Schema-independent data validation and canonical encoding match the official interoperability fixtures. The remaining work includes per-Lexicon schema validation, apply-writes semantics, and the broader official fixture and multi-write failure matrix.
Completion evidence:
Implemented foundation: schema version 9 includes content-addressed repository
blocks, one signed head per account, ordered operation rows, a durable replay stream,
ordered per-commit block membership for CAR diff/firehose recovery, and a
public/Space per-record blob-reference index with durable upload and deletion
states. Version 7 generalizes the outbox so lifecycle and commit events share
one constrained sequence without inventing commit roots or revisions for
lifecycle rows; version 8 adds a constrained durable account status with
active as the backward-compatible upgrade default. Version 9 repairs skipped
legacy stream-table upgrades while preserving issued sequence numbers, and adds
indexed OAuth credential expiry cleanup. Bounded repository-block
collection uses ATrium's exported reachable graph and preserves current heads
plus retained sync-event membership before deleting historical blocks. Fresh
databases create all schema generations in one transaction; version 1 through 8
upgrades preserve existing rows, version 5 blobs become available in place,
version 6 commit rows move into the constrained stream schema without
renumbering, and version 7 accounts become active in place. Reference-index
migrations use the shared ATrium scanner and all migrations roll back on
incompatible state. Current databases recompute both indexes at startup and fail
closed on drift. Historical version-2 outbox rows intentionally have no guessed
block membership and replay as signed #sync reset points. New versioned outbox
payloads retain the prior revision and MST root required for inductive firehose
events. The public write path stages ATrium blocks in memory, filters legacy
backfills to the reachable graph, persists incremental write sets, verifies
block bytes against their CIDs on every read and write, and publishes the whole
mutation with a SQLite head compare-and-swap.
Tests cover transaction rollback, block corruption, concurrent creates,
restart monotonicity, signature verification, legacy-row backfill, and startup
rejection of reachable-block corruption, SQL/MST record-index drift, or retained
event metadata, sequence, inductive-chain, block-membership, and head drift.
3. Public sync surface and durable event stream #
Current state: getBlob, getLatestCommit, getRepoStatus, full getRepo,
and proof-oriented getRecord use canonical generated request types. CAR
responses delegate traversal, proof extraction, and v1 framing to ATrium,
reverify the locally stored head signature, and enforce a 64 MiB writer limit.
Interoperability tests reopen the archives through ATrium, verify the signed
root, read included records, and prove a record archive omits an unrelated
value block. Incremental getRepo exports retain the current signed head as the
CAR root while including only blocks assigned to repository revisions newer
than since; they fail closed when a legacy event lacks exact block membership.
Every new mutation has a durable ordered outbox row and its exact ATrium write
set.
getBlocks permits at most 1,000 requested CIDs, proves each is reachable from
the current signed graph before accessing the shared block table, and returns
an ATrium CAR with the reference implementation's empty roots array.
listBlobs queries the atomically maintained reference index populated through
ATrium's typed and legacy blob-reference parser, provides bounded CID
pagination, and never lists unattached uploads. subscribeRepos now backfills
and polls the durable outbox over binary WebSockets, serializes generated ATrium
#identity, #account, #commit, and #sync models as the two
concatenated DAG-CBOR objects required by the event-stream specification, and
returns FutureCursor as a stream error. On the first repository commit, the
same transaction assigns consecutive identity, active-account, and commit
events in the recommended order. The identity event includes the canonical
stored handle; local mode derives it from the validated local document and
production substantiates it before binding. Retained historical events are not
rewritten. Historical rows and commit diffs above two megabytes become signed
#sync reset points. The transport limits handshakes by peer, caps global and
per-peer live connections and inbound frames, and disconnects stalled consumers
after a bounded send.
The transactional outbox keeps a configurable bounded window; stale consumers
receive generated #info/OutdatedCursor and incremental exports fail closed
when their block history is no longer retained. listRepos exposes initialized
signed heads through generated ATrium types and the reference time/DID cursor.
listReposByCollection reuses that stable cursor and returns only repositories
whose current records contain the requested canonical NSID and whose account is
active. getRepoStatus, listRepos, and getSession report durable account
state through generated ATrium models. Authenticated activation and deactivation
atomically append idempotent #account events; inactive states suppress public
records, blobs, exports, and future repository commits, while SQLite rechecks
state in commit and blob-publication statements to close authorization races.
describeRepo exposes the canonical DID document and collection inventory
through generated ATrium types and reports the hosted handle as correct only
after the deployment-mode identity gate succeeds. listMissingBlobs
uses ATrium's current and legacy blob scanner to group unresolved references by
CID and return a representative record URI through the generated Lexicon shape.
An explicit operator command now asks bounded configured relays to crawl the
PDS using ATrium's generated request type. A separate offline, storage-verified
operator command applies host account states and reversals with atomic retained
events. Periodic production handle checks now publish confirmed
handle.invalid and recovery transitions. Secure data erasure, DID-document
lifecycle, and crawl metrics remain absent. One fake relay now
carries the served identity through crawl, gap repair, and resumed live ingestion
using the same public endpoints a network relay uses. The separate in-process
EventSink remains best-effort and is not part of relay delivery.
Completion evidence:
4. Identity, handle, account state, and relay discovery #
Current gap: the server derives and validates its canonical local did:web
document, resolves public handles and DIDs, verifies its production identity
bidirectionally before binding, exposes generated resolveHandle, emits initial
identity and active-account events, durably implements user activation and
deactivation, and reports verified repository identity and hosting status. It
periodically revalidates the hosted handle and emits durable invalidation and
recovery events. It does not track later DID document/key/service transitions,
securely erase deleted account data, support did:plc operation workflows, or
exercise the production resolver against injected fake DNS/HTTPS/PLC services.
Completion evidence:
5. AT Protocol OAuth profile #
Current gap: the local browser flow has useful OAuth primitives, but network OAuth requires pushed authorization requests (PAR), dynamically fetched client metadata, DPoP with server nonces, refresh-token rotation, revocation, and profile-complete metadata. Current bearer tokens are not suitable for public deployment.
Completion evidence:
6. Storage lifecycle, backup, and recovery #
Current gap: production scheduling, retention, off-host transfer, and separate
secret protection remain operator work. SQLite has versioned migrations and an
explicit durability policy, filesystem blobs use verified crash-durable
publication, cpds backup captures the database and CID-verified blobs behind a
write barrier, and cpds verify runs the complete startup proof without
collection or socket bind.
The bundled SQLite version must remain at 3.51.3 or newer (or an official fixed backport) before WAL is enabled. Earlier versions contain the documented WAL-reset corruption race under concurrent writers/checkpoints.
Completion evidence:
7. HTTP and process hardening #
Current gap: all current request classes are bounded for the supported single-process topology. A future multi-process deployment would need to move principal budgets to a shared limiter.
Completion evidence:
8. Observability and operations #
Current gap: request tracing is structured but console-only. There are no service-level indicators, metrics, alert thresholds, deployment artifact, or rehearsed operator runbook.
Completion evidence:
Local testing architecture #
All protocol behavior must be testable locally and deterministically. Tests must not require the public PLC directory, DNS, a public relay, or an existing Bluesky account.
- Pure unit tests cover parsers, canonical encoders, CIDs, TIDs, MST operations, commit signing/verification, cursor logic, scopes, OAuth/DPoP, configuration, and state transitions. Time, randomness, and DNS/HTTP lookup are injected at these boundaries.
- Adapter contracts run identical behavior against memory/test and production adapters: blob stores, block stores, outboxes, identity resolvers, client metadata resolvers, clocks, and event subscribers.
- SQLite component tests use temporary on-disk databases, close/reopen them, run migrations, force conflicts, and validate invariants directly. In-memory SQLite alone is insufficient for WAL/restart behavior.
- Router integration tests send complete HTTP requests through the Axum router and assert status, headers, JSON/CBOR/CAR bodies, authorization, size limits, and observable committed state.
- Process integration tests spawn the compiled binary with an isolated temporary data directory and dynamically allocated loopback listener. They test startup failure, readiness, real TCP/WebSocket behavior, signals, and restart. Child processes always receive explicit cleanup guards.
- Federation end-to-end tests run a fake DNS/HTTPS identity service, OAuth client, relay, and consumer. The relay crawls the PDS, verifies its identity, subscribes, validates every repository block and signature, intentionally disconnects, resumes, detects a gap, and repairs through a full CAR export.
- Interop tests consume pinned upstream, language-neutral fixtures with provenance and licensing recorded. Generated local fixtures never replace upstream vectors for wire compatibility.
- Fault and concurrency tests exercise write races, clock rollback, database contention, slow stream consumers, full queues, I/O failures, retries, abrupt shutdown, corruption, backup, and restore.
Tests that bind loopback sockets are still local tests. They may require a test runner environment that permits loopback binding, but never public network access.
Sequenced implementation #
Each item below should normally be one commit: one observable behavior, its tests, and the smallest documentation update needed to keep claims accurate. Dependencies may reveal a smaller prerequisite commit, but unrelated items must not be bundled.
Phase A — make the existing server safe to operate #
- Add explicit local/production configuration modes and exhaustive config unit tests.
- Canonicalize the public origin and derive all externally visible URLs from it; add metadata/URL integration tests.
- Add versioned SQLite migrations and on-disk migration/restart tests.
- Apply and verify SQLite durability/contention pragmas with tests.
- Harden filesystem blob publication and recovery with fault-oriented adapter tests.
- Add body, timeout, concurrency, and basic authentication rate limits with boundary tests.
- Add liveness/readiness and graceful shutdown process tests.
Phase B — build the public repository as one atomic domain #
- Pin
atrium-crypto, run official signature fixtures, and remove superseded AT Protocol multikey and signing code. - Pin
atrium-apiwith default features disabled and migrate standard public XRPC wire types endpoint by endpoint. - Implement an atomic SQLite block-store adapter for
atrium-repo; prove MST, DAG-CBOR, CID, commit, and CAR output against official fixtures. - Store exact canonical DAG-CBOR record blocks and validate blob references. Schema-independent data-model validation, native CID-link/byte encoding, exact official-fixture bytes/CIDs, a one-megabyte encoded-record bound, typed reference ownership/metadata validation, and atomically maintained public/Space reference indexes are complete. Per-Lexicon validation remains. Blob publication is durably staged and orphan collection is bounded, configurable, and crash-resumable. Repository-block collection is also bounded and retains both the current ATrium graph and the durable sync replay window.
- Add signed repository-v3 commit construction and verification around the ATrium repository mechanism and explicit CPDS identity trust policy.
- Persist block store, repository head, operation log, and event outbox in one transaction.
- Implement
putRecord,applyWrites, and swap constraints; move existing create/delete through the same transaction. - Add restart, race, rollback, clock-skew, malformed-import, resource-bound, and corruption tests for the full repository mutation path. Startup now uses ATrium's MST enumerator to verify signed heads, every reachable record block, and exact SQL index parity before binding; remote-import and broader fault coverage remain.
Phase C — expose verifiable sync and relay integration #
- Implement and test repository status/latest-commit/block/record endpoints.
- Implement and verify full/incremental CAR repository exports. Complete for locally generated versioned repository history; legacy rows without block membership fail closed.
- Build durable outbox sequence allocation, retention, and restart behavior. Allocation and replay survive restart; bounded retention and canonical stale cursor recovery are complete.
- Implement CBOR event frames and WebSocket
subscribeReposwith backfill. Generated#commit/#syncmodels, real-WebSocket backfill, live delivery, future-cursor errors, and restart recovery are complete. - Add identity/account/sync/info events and slow-consumer behavior. Initial identity/account, commit, sync, and outdated-cursor info events are complete, and a real stalled consumer proves bounded disconnect and replay. User activation/deactivation and offline host enforcement now emit retained account-state events; later identity changes remain.
- Add the fake-relay federation harness, gap detection, and CAR repair test.
The explicit generated-payload relay crawl operation and its fake HTTP relay
contract are complete. A relay-style real-WebSocket test now forces an
expired cursor, repairs through
getRepo, validates the signed MST with ATrium, resumes live, and validates the next inductive event. The combined fake relay now receives the crawl, validates the served DID/handle/key, and carries that identity into the repair and resumed live commit.
Phase D — complete network identity and OAuth #
- Introduce deterministic DID/handle resolver ports backed by
atrium-identityand CPDS-owned safe transport/cache adapters. The shared resolver, bounded production transport, generatedresolveHandle, injected orchestration-policy contract, and pure negative tests are complete. Fake DNS/HTTPS/PLC wire services must still exercise the production adapters. - Enforce bidirectional identity and account-state gates. Production startup identity verification plus durable user activation/deactivation, offline host transitions, periodic invalid-handle/recovery events, and read/write/event account-state gates are complete.
- Add hardened dynamic client metadata resolution.
- Add PAR storage/endpoints and one-time request URI handling.
- Add OAuth DPoP nonce, proof, replay, and token binding.
- Add rotating refresh tokens, session revocation, cleanup, and CORS/security headers.
- Drive a profile-complete local client flow with
atrium-oauthwhile retaining focused server-side negative tests.
Phase E — recovery, operations, and release proof #
- Add backup/restore verification and account export/import tests. Startup
verification of each signed ATrium graph, reachable record block, and SQL
record index is complete. It also verifies every stored blob byte and rebuilds
the retained event chain through ATrium's generated models and CAR writer,
including sequence, inductive-link, block-membership, and latest-head checks.
cpds verifyruns that complete proof against existing restored storage, checks configured account identity, and exits without maintenance or a listener.cpds backupholds writes across SQLite's native snapshot and verified blob copy; a process test restores that output to different paths and runs the verifier. A populated-state test additionally restores and reads a signed repository, referenced blob/index, retained event, and CAR export. Production backup scheduling, separately protected secrets, and account export/import remain. - Add metrics, redaction tests, correlation, and operator diagnostics.
- Add the deployment artifact and install/upgrade/rollback runbook.
- Add dependency policy, CI, federation interop, fault, concurrency, and soak gates.
- Rehearse a clean network deployment: identity and handle validation, OAuth client login, relay crawl, live event ingestion, disconnect/resume, full repair, backup, restore, upgrade, rollback, and key/session revocation.
Release gates #
cpds may describe itself as federation-capable only when Phases A through D
pass locally and the fake relay verifies the entire repository/event chain. It
may describe itself as production-grade only when Phase E also passes and the
following evidence is recorded for the release candidate:
- all checked items in this document have a test or named operational proof;
- formatting, Clippy, unit, component, router, process, federation, interop, fault, concurrency, and restore suites pass from a clean checkout;
- no endpoint or metadata document advertises an unimplemented capability;
- a clean deployment and upgrade preserve a valid repository and stream cursor;
- a backup restores to a different directory and re-verifies every CID, MST root, commit signature, blob, account state, and pending event;
- an independent local consumer can discover the account and reconstruct its complete current public record set from only the documented network surface.