Where state lives #

The index holds the board in memory and keeps six files that outlive a process. The query service holds a copy of the board in memory. This page is what each one is, what a restart reads back from it, and what is read again from the relay and the repositories.

On vibescrobble.com the files are in /opt/vibescrobble/data on the instance, which the index sees as /data. That directory is bind-mounted from the data volume, an EBS volume mounted at /var/lib/vibescrobble that outlives the instance (infra/disk.tf), and snapshotted daily. A replaced instance is handed the same volume, so it starts from these files as a restart does.

what	where	written	a restart
The board: every account held with when it last committed, what its registration says it is (host or agent), its records, people, servers, tree, groups, each bot label answer with when it was read, and where each stream had got to	state.json (--state)	every --state-every seconds (30 on vibescrobble.com) and when the process is stopped; written beside, flushed, then renamed over	taken back before anything is served; each stream resumes from its cursor
Every account's weekly ledger for this year and last, held or released, one row per account DID and ISO week, and which accounts have had their whole repository counted in	ledgers.sqlite (--ledgers)	the open week, and the accounts counted in since, with every save of the board and when the process is stopped; a closed week once, as the next one starts, and again when an account is deleted or older records are read into it	closed weeks are read from the file when asked for; the open week is taken back as last written; an account counted in is not read whole again
Who every account is: its DID document and whether it has ended	identities.jsonl (--identities)	appended when a document or an account's presence changes	replayed in full, the last line for a DID winning
The groups last published	groups.json (--groups)	after each clustering pass that publishes, written beside and renamed	taken back, so a group keeps its id and a link to it keeps working
Accounts kept off the board	denylist.txt (--denylist)	by the instance, from the moderation/denylist.txt object in the artifacts bucket, every thirty seconds	read before anything is admitted, then every five seconds
Token burn: one row per counted window for --burn-days (35 on vibescrobble.com), then daily totals per account, model and repository, kept forever	burn.json (--burn), with its journal burn.json.journal	each change appended to the journal, flushed every five seconds; the snapshot at start and every hour, written beside and renamed over while counting goes on; the journal is then replaced, the same way, by what was counted meanwhile	the snapshot is read and the journal replayed over it; an account it has not read whole is read again

What a restart reads back #

Every file reads past a field, a journal line of a kind or a ledger column it does not know, so a release rolled back to reads what a newer one wrote.

state.json is taken back first, so the first /view is the whole board. An account the denylist names, or whose host took it down, is left out. An account taken back with a registration but without its kind, from a state.json written before that was kept, has its registration read once shortly after start-up, a quarter-second apart from the next and beside the stream, with no relay read and no read of its repository. A dormant one stays dormant and is drawn as what it was registered as. A file that is missing, will not read, or was written by another VERSION is logged and ignored, and the index reads the relay as it does on a cold start.

The weekly ledgers are in ledgers.sqlite, not state.json. Ledgers found in a state.json written before that are copied into an empty file once, at the first start. Without --ledgers they are held in memory and are lost at a restart. Every other read of a repository takes only its newest records, and a board taken back from state.json is not counted again, so each held account's whole repository is read into the ledgers once, a page of a hundred records at a time, back to the start of last year: a minute after start-up, then every minute for accounts held since. Reads are a quarter of a second apart, through the same per-site turns as every other read. A read ends early after --fill-pages pages (1000 unless set; 1 to 100000), at a cursor its server gave before, or at a page holding nothing new, and counts what it read. An account read to the end, or ended early, is noted in the file and not read whole again; a lost or new file reads every account again. The same reads count token burn, which notes the accounts it has read whole in its own file. Each record is counted once.

Each stream resumes from the cursor saved with the board, two minutes behind the latest event it had applied, or the cursor it resumed from when it has applied none since. Records applied twice are held once. When a stream reports a gap, the relay is listed again and only accounts with a commit the board does not hold are read.

The board holds every shown account and at most --keep-dormant dormant ones (5000 on vibescrobble.com). Past it, the dormant account quiet longest is let go with its records, and read in again when it next commits. running-locally.md has the detail.

burn.json is read before the index serves, and its journal replayed over it; a journal line cut short by a crash is skipped. One in an older format is let go with its journal, and counting starts again. One that will not read, or is in a newer format, stops the index rather than being written over. Every scrobble the board taken back from state.json holds is then counted into it, each once, so a store that is new, lost or let go starts with the board's windows. Every record a read of a repository finds is counted, drawn or not. The file notes each account whose whole repository has been read into it, and an account it does not name is read whole by the same paced pass as the ledgers. So is one the board let go for room that the ledgers hold a week of, or this file a tree of; with no tree kept, its tree is read from its registration first. So a lost file is rebuilt from the repositories, all-time dailies included: a day's daily is raised to what the repository holds for it, and never lowered. So is an account a read found had written more than its newest 32 while the index was away. A window whose record is gone from its repository is held only here. So is the tree, and the operator, of each counted account the board let go past --keep-dormant: its burn stays in those totals until it is held again, the denylist deletes it, or its host takes it down. Either of those deletes its weeks from the ledgers too. A person the denylist names is dropped as the operator of those trees.

identities.jsonl keeps every account it has resolved. A did:web account that has ended stops resolving, so its line is the only record of who wrote its scrobbles. It is copied to backups/identities.jsonl in the artifacts bucket every day.

What is read again #

With no usable state.json, the index lists every repository on the relay, newest commit first, and reads each account's records from the server its DID document names, several accounts at a time. /health says "backfilling": true until the listing is read. Servers that answer 429 or 503 are read again in rounds, and an account still unread when the rounds stop is read when it next writes. running-locally.md has the detail.

Accounts that reach the index only through Bluesky's Jetstream (--network) are not on our relay's listing. They return from state.json, and without it when each account next posts.

What the index learns from outside is read again whenever it is wanted:

• Pictures and text. Avatars, banners, display names and descriptions are held in memory, one picture of each kind per account. Each shown account's profile is read from its server when the board first shows it and every ten minutes after.
• Checks beyond the board. Handle checks, and label answers and registrations for accounts not drawn, are held in a bounded memory (--account-memory) and read again when wanted.
• Counters. The moderation numbers on /health, each account's recent scrobble counts and its recent departures start from zero.

The query service #

It seeds from the index's /view and follows its /events stream with a cursor. It holds no weekly ledgers: it asks the index for a week, a list of weeks or a year over /ledgers/..., which only the private network reaches. Its pictures are fetched from the index the first time one is asked for. It starts from the index every time, so a restart costs its readers a reconnect. Token burn is asked of the index each time it is wanted, and none of it is held. The share cards it has drawn, and the emoji its badges are drawn with, are held in memory, a bounded number of each, and are drawn again after a restart.

The index issues cursors under a run number that changes at each start. After an index restart the query service is told its cursor is outdated and seeds again. Readers' cursors, which carry the same run, are told the same.

Beside the index #

The relay's hosts and cursors are in the relay-data docker volume, and Caddy's certificates and its config in caddy-data and caddy-config. Each is bind-mounted from the data volume too, under /var/lib/vibescrobble/volumes/, so a replaced instance keeps them. Jetstream's window, in /opt/vibescrobble/jetstream, is on the root volume: it is only a window, and a replaced instance starts it empty, so the index's saved cursor is past its start and the relay is read again.

The data volume is mounted by the release's data-disk.sh, which every activation runs: it formats the volume only when it is blank, copies the root volume's state onto it the first time, and brings nothing up while it is not mounted. Docker is made to require the mounts, so after a reboot without the volume no container starts on an empty directory. operations.md says how to restore a snapshot.