at-record
gotta store my records
Star
Fork
Watch
at-record deploy CLUSTER.md
3.2 kB · 66 lines
Markdown
at main

Moving crate to the riotbyte cluster #

The droplet path (deploy/AUTOPUSH.md, crate-pull.timer, deploy/publish.sh) is replaced by a namespace on the riotbyte DOKS cluster. This is a rebuild, not a transfer: the droplet is on the "My Team" DigitalOcean account and the cluster is on the riotbyte one.

Why the hostname does not change #

crate.mokkenstorm.dev stays. The atproto OAuth client_id is https://crate.mokkenstorm.dev/client-metadata.json, so moving to a riotbyte domain would invalidate every grant every user has already given and force all of them to re-authorise. The riotbyte terraform therefore looks the zone up as a data source rather than adopting it: it routes one hostname without owning a personal domain that hosts other things.

What terraform creates #

projects/crate/ in the riotbyte infra repo, reusing the shared modules/project/modules/environment:

  • the crate-production namespace
  • a crate_production database and user on the shared DO Postgres, with DATABASE_URL injected into the app secret
  • a crate-owned ghcr-pull-secret, so pulls stop riding on a personal account
  • app-env from the Crate - Production 1Password item in the Infrastructure vault
  • a namespace-scoped kubeconfig for CI
  • the Cloudflare tunnel ingress route and DNS record, generated from the module's project_domains output

k8s/apps/crate-production.yaml in that repo carries the Deployment and Service. Note the shape differs from the other apps: strategy: Recreate and a startupProbe with a 20 minute budget, because boot runs a synchronous PDS backfill before the listener binds.

Prerequisites, all outside this repo #

  1. REGISTRY_USER / REGISTRY_TOKEN as tangled repo secrets. Without them the deploy workflow has never published an image, which is why production has been serving a build from 2026-08-03.
  2. KUBECONFIG_B64 as a tangled repo secret, from module.crate.kubeconfig.
  3. A Crate - Production item in the Infrastructure vault with sections app (secret key base, store key), oauth (client jwk) and discogs (consumer key, consumer secret).

The Cloudflare trap #

The droplet is reached by a DNS-only A record. The cluster is behind a Cloudflare Tunnel, and the OAuth path must not sit behind the proxy/WAF without a skip rule for /client-metadata.json and /jwks.json, with Rocket Loader and Auto Minify off (they rewrite the Lustre bundle and break it). The riotbyte terraform manages always_use_https and ssl per zone but none of these, so they need adding rather than inheriting. Avoiding exactly this is why the alpha used a DNS-only record; a tunnel is a regression until those rules exist.

Data #

The droplet's Postgres holds the appview index, which is derivable: it is rebuilt from the PDS and the firehose. A pg_dump and restore is the fast path; a deliberate rebuild-from-PDS is the honest drill and the one ADR 0002's disposability argument leans on. Either way, remember the PG15+ GRANT ALL ON SCHEMA public step on the new database.

Nothing else on the droplet is stateful: the image holds compiled code and the secrets arrive from the app secret at runtime.