# Interfaces The internal contract between the build step, the serving core, the shells, and the verifier. This file is normative for implementation: modules may only assume what is written here about each other. It describes internals and is not part of the public API surface documented in [ARCHITECTURE](ARCHITECTURE.md). ## Language and dependency rules - Plain ESM JavaScript, no compile step. Types are JSDoc, checked with `npm run typecheck`. - `src/serve/**` and `src/worker/**` import **nothing** — no npm packages, no `node:` builtins. Web-standard globals only (`Request`, `Response`, `URL`, `TextEncoder`, `atob`, `JSON`). - `src/node/**` may import `node:` builtins only. - `src/build/**` and `src/verify/**` may import anything; they run offline in Node. - `src/message.js` belongs to no subsystem: it imports nothing, and `src/build/**`, `src/verify/**`, and `bin/**` may import it without depending on each other. It exists because `catch (err)` binds `unknown` everywhere and `verify` states in its own header that it does not import the build. - `src/dispatch.js` belongs to no subsystem either: it is the CLI's argv router, imports only `src/message.js`, and reaches every subcommand through `import()` at routing time. It must never gain a static import of a subcommand — see [CLI](#cli). - Tests use `node:test` and `node:assert/strict`. ## Validating lexicon source Two independent validators exist and **they do not agree**. Run both. `lexiconDocumentSchema.parse()` from `@atproto/lex-document` (a transitive dependency, already installed) is the official TypeScript parser and the one the reference stack actually uses. It is the right in-process gate for the build. It is not sufficient on its own. Its permission schema is `{type, resource}` intersected with an **open dictionary** of arbitrary keys to scalars-or-arrays, so it accepts any field name with any scalar shape inside a permission. `goat lex lint` has concrete Go structs instead and rejects things the TypeScript parser waves through — verified: `"aud": ["*"]` parses fine under `lexiconDocumentSchema` and fails under `goat` with `cannot unmarshal array into ... aud of type string`. `goat` is right; `aud` is a single string. This matters most exactly where the stakes are highest. Permission sets drive the OAuth consent screen, and they are the document type the generic parser checks least. Bluesky's own permission sets (`app.bsky.authFullApp`, `chat.bsky.authFullChatClient`) are the reference for correct shape: `lxm` is an array of NSIDs, and a permission set uses `"inheritAud": true` rather than naming an audience, because the set does not know the audience — the requesting client supplies it. So: `corpora build` gates on `@atproto/lex-document` (in-process, no external binary), and the publish loop in [PUBLISHING](PUBLISHING.md) keeps `goat lex lint` as a second opinion. Neither subsumes the other. ## `dist/` layout `corpora build` emits exactly this tree. Every shell consumes it; nothing else is contractual. ``` dist/ did.json # DID document repo.car # full repo CAR bytes commit.json # sync.getLatestCommit response body describeRepo.json # repo.describeRepo response body records/.json # repo.getRecord response body, one per lexicon proofs/.car # sync.getRecord response bytes, one per lexicon list/page-1.json # repo.listRecords response body, page 1 (page-N for later pages) manifest.json # build metadata data.js # generated ESM module embedding all of the above ``` `` is the literal NSID (e.g. `com.example.thing.json`). NSIDs are already filename-safe. ## `dist/data.js` — the embedded bundle A generated, dependency-free ESM module with one default export. Byte payloads are embedded as base64 strings and decoded at module load by an inlined helper using `atob`, which exists in both Node >= 18 and workerd. The decoded values are `Uint8Array`. ```js /** @type {import('../src/serve/types.js').CorporaData} */ export default { did: 'did:web:lexicon.example.com', rev: '3lxyz...', // commit rev (TID) commitCid: 'bafyrei...', // commit CID, string collection: 'com.atproto.lexicon.schema', didDoc: { /* did.json, parsed */ }, commit: { cid: 'bafyrei...', rev: '3lxyz...' }, describeRepo: { /* describeRepo.json, parsed */ }, order: ['com.example.thing'], // rkeys in DESCENDING order — see below records: { 'com.example.thing': { uri, cid, value } }, proofs: { 'com.example.thing': Uint8Array }, repoCar: Uint8Array, manifest: { /* manifest.json, parsed */ }, } ``` `manifest.json` carries build metadata, not response bodies: ```json { "corporaVersion": "0.1.0", "did": "did:web:lexicon.example.com", "collection": "com.atproto.lexicon.schema", "rev": "3lxyz...", "commitCid": "bafyrei...", "mstRoot": "bafyrei...", "signingKey": "did:key:zQ3sh...", "didDocHash": "sha256-...", "nsids": ["com.example.thing"], "authorityDomains": { "_lexicon.example.com": ["com.example.thing"] }, "sources": { "com.example.thing": "sha256-..." }, "recordCids": { "com.example.thing": "bafyrei..." }, "totalBytes": 12345 } ``` `didDocHash` is the sha256 of the DID document **as served** — `JSON.stringify` with no whitespace and no trailing newline, the bytes `/.well-known/did.json` returns — not of the pretty-printed `dist/did.json` file. It is what `corpora verify --expect-did-doc ` compares against, and it exists to be published away from the serving host; see [BUILD](BUILD.md#emitted-bytes-that-are-contractual). No serving shell reads it. `manifest.json` must be byte-stable for identical inputs and key — no timestamps, no absolute paths, keys sorted. Reproducibility is an M1 acceptance criterion. ### `order` is a contract between build and serve A reference PDS returns `listRecords` in **descending** rkey order by default — the opposite of MST order. Verified live against `jarrett.app` on 2026-08-03: 25 records, `io.atcr.tag` first and `io.atcr.authFullApp` last. So `corpora build` must emit `order` — and `list/page-N.json` — in descending rkey order. The handler treats that sequence as canonical `reverse=false` order and never sorts. Emit it ascending and Corpora silently disagrees with a real PDS over the same repo, which is exactly the class of near-miss this project is supposed to eliminate. `order` holds rkeys, not records. The bundle carries each record body exactly once, under `records`, and the handler pages over `order` per request rather than serving a prebuilt page shape. `list/page-N.json` still exists in the static tree for readers walking `dist/` directly; the bundle does not embed it, because doing so meant a second copy of every record — about 27% of `data.js` and 21% of the heap a Worker holds. `test/build/build.test.js` guards the invariant structurally and `test/worker/bundle-size.test.js` guards it by size. `reverse=true` is served by reading `order` from the other end, not from a mirrored copy: a record at index `i` is at `length - 1 - i` reversed, and cursors translate through the same arithmetic. ## `corpora/serve` ```js /** * @param {CorporaData} data * @param {{ maxAge?: number, didDocMaxAge?: number, landing?: boolean }} [opts] * @returns {(req: Request) => Response} */ export function createHandler(data, opts) ``` Synchronous, no I/O, no async. Routing table and response shapes are specified in [RESOLUTION](RESOLUTION.md#the-http-surface). Behavioural requirements: - `GET` and `HEAD` only; anything else is `405` with an XRPC error body. - Unknown paths and unknown NSIDs are `404` XRPC errors: `{"error":"...","message":"..."}`. - Query parameters are validated: `repo`/`did` must equal `data.did`, `collection` must equal `data.collection`, `rkey` must be a known NSID. Wrong values produce the same error names *and status codes* a reference PDS returns (`RepoNotFound`, `RecordNotFound`, `InvalidRequest`) — verified live against `jarrett.app` on 2026-08-03. Note the asymmetry, which is real rather than an oversight: `sync.*` and `describeRepo` declare `RepoNotFound` in their lexicons, while `repo.getRecord` and `repo.listRecords` declare no repo error and surface the same condition as `InvalidRequest`. An unknown rkey is **400 `RecordNotFound`**, matching the reference, not 404. A wrong `collection` on `listRecords` returns `{"records":[]}`, again matching the reference. One divergence is deliberate. A real PDS answers `sync.getRecord` for an *absent* rkey with 200 and a non-existence proof CAR — verified: 2,259 bytes, comparable to an inclusion proof. Corpora returns 404 `RecordNotFound`, a declared error of the same lexicon. Note that precomputing these is not actually impossible, as one might assume. Absent *keys* are unbounded, but the exclusion proofs they map to are not: an exclusion proof is the MST path to the gap where the key would sit, and a tree over N records has O(N) such gaps. You could emit one per gap and route any unknown rkey to the right one. It is not worth it. Nothing on the resolution path asks for an NSID it does not expect to exist; a resolver meeting an unpublished lexicon needs to fail, and 404 fails it. Building a gap-routing table to return a cryptographically precise "no" to a question nobody asks would add the most intricate code in the project to serve a case with no consumer. Revisit only if a real client turns out to distinguish the two. - CAR responses use `content-type: application/vnd.ipld.car`. - JSON responses use `content-type: application/json; charset=utf-8`. - `/` is an HTML landing page for humans, `text/html; charset=utf-8`, rendered from `data` once at handler construction — never per request, and never from anything the request supplies. It is the only response that is composed rather than replayed, so everything it interpolates is HTML-escaped and every link it prints is followed by a test. `{ landing: false }` drops the route back to the standard 404. `/favicon.svg` and `/favicon.ico` return the same SVG bytes with `image/svg+xml`; a browser asking for `.ico` unprompted gets a working icon rather than a 404, since renderers dispatch on content type, not extension. - `did.json` is served with `cache-control: public, max-age=300`, as is `/`, whose content follows the deploy; everything else `public, max-age=86400, immutable`. - No redirects, ever, on any path. - CORS: `access-control-allow-origin: *` on every response, plus an `OPTIONS` preflight answer. ## `corpora/node` ```js /** @param {(req: Request) => Response} handler @returns {import('node:http').RequestListener} */ export function createRequestListener(handler) /** @param {CorporaData} data @param {{port?: number, host?: string}} [opts] */ export function serve(data, opts) // → Promise<{ url, port, server, close() }> ``` ## `corpora/worker` ```js export function createWorker(data) // → { fetch(req): Response } ``` ## `corpora/verify` Verifies an authority independently of the code that built it — by fetching over HTTP, never by reading build internals. ```js /** * @param {{ origin: string, did?: string, sourceDir?: string, expectDidDoc?: string }} opts * @returns {Promise} */ export async function verifyAuthority(opts) ``` `VerifyReport` is `{ ok: boolean, checks: Array<{name, ok, detail}>, nsids: string[] }`. `src/verify/index.js` is the only file `package.json` maps to `corpora/verify`; it also exports `verifyProof`, `authorityDomain`, and `authorityDomains`. Alongside it, `src/verify/format.js` holds the verifier's pure formatting and comparison helpers — `canonical`, `byKey`, `short`, `bytes`, `equalBytes`, `rkeyOf`, `sameOrigin`, `describeType`, `errCode` — and is **internal**: not in `exports`, imported only by `src/verify/**`, and importing nothing but `src/message.js`. The split is what lets those functions be tested as units without the package's public surface growing to accommodate a test. A helper earns a place there by being a total function of its arguments; anything that fetches, reads the filesystem, resolves DNS, verifies a signature, or touches report state stays in `index.js`. ## CLI `bin/corpora.js` is a shebang and one line: `process.exit(await dispatch(process.argv.slice(2)))`. The routing lives in `src/dispatch.js`, which is importable and therefore testable. ```js /** * @param {string[]} argv process.argv.slice(2) * @param {CommandTable} [commands] defaults to the real four * @returns {Promise} the exit code; dispatch never exits the process */ export async function dispatch(argv, commands) ``` Each subcommand lives in its owning module and exports `run(argv)` returning an exit code. A `run` that resolves to nothing is dispatched as `0`. The table's values are `() => import(...)` thunks and exactly one is called per invocation: `corpora verify` must not load the build. ``` corpora build --lexicons --out --did [--key-file |CORPORA_SIGNING_KEY] [--service ] [--dns-check strict|cutover|off] [--no-dns-check] [--rev ] [--against ] [--force] [--quiet] corpora verify --origin [--did ] [--source ] [--json] corpora serve --dist [--port 2583] [--host 127.0.0.1] corpora keygen [--out ] ``` ## Local end-to-end The whole point is being able to run the real resolution chain against localhost. See [LOCAL_E2E](LOCAL_E2E.md), owned by the verify/e2e work.