Social graph sync #

Social graph sync keeps a user's follow graph identical across the apps they choose among Bluesky, Tangled, Sifa, Grain and Semble. A follow or unfollow on one app is mirrored on the others, existing follows are merged when sync is turned on, and people who join an app later are followed when they show up. It is a first-party feature, not a user-built automation.

Plan and requirements: docs/plans/2026-09-14-001-feat-social-graph-sync-plan.md. Code: lib/sync/. The owner manages it at /u/{handle}/social-graph-sync, reached from a panel at the top of their own profile. Public description: /sync. API: /api/sync.

The safety rule #

Only an observed unfollow event deletes a follow. Backfill, the waiting list and the sweep can only add follows. The rule is enforced by the queue row kind: the only code that inserts an unfollow row is the delete branch of the Jetstream handler in lib/sync/events.ts. Deleting one of two duplicate follow records for the same subject is cleanup, not an unfollow, so it propagates nothing.

The residual risk this accepts: a delete event missed during an outage longer than the sync lookback window (24 hours by default) is never recovered. The sweep drops the stale mirror row, but it never deletes on the other apps.

State #

Six tables, all cascading from users (lib/db/schema.ts):

Table	Holds
sync_sets	One row per set: apps, enabled, paused, auth-failure streak, rate-limit park, current backfill run and its total.
sync_follow_mirror	One row per follow record the owner has on an app in the set, keyed by rkey.
sync_waiting	Subjects that could not be followed on an app because they have no profile there yet.
sync_queue	Every PDS write sync will make, in event order.
sync_backfill_tasks	Listing work for backfill and sweep runs, with a resumable page cursor.
sync_activity	What sync did; the page lists it and the pacer sums its points.

Why the follow mirror is required. Jetstream delete commits carry only the collection and rkey, never the record, so an unfollow's subject is only knowable from a mirror of the owner's own follow records. The mirror also answers "already following" locally and identifies sync's own writes: the runner generates the record's rkey and inserts a created_by_sync mirror row before calling the PDS, so the Jetstream echo of that write finds its rkey already mirrored and propagates nothing. A retry reuses the same rkey, so a write that timed out after succeeding cannot create a second record.

Flow #

flowchart TB
  SH[Shard subscriptions: synced users, follow collections] --> H[Handler: database only, synchronous]
  GL[Global subscription: Tangled, Sifa, Grain profile creates] --> H
  H --> Q[(Write queue)]
  BF[Backfill listing and union step] --> Q
  SW[Daily sweep] --> Q
  Q --> R[Runner: one row per lane, pacing, re-verify]
  R --> PDS[(Owner's PDS)]

1. Subscriptions. The Jetstream manager accepts injected partition sources (lib/jetstream/consumer.ts). Sync contributes shard subscriptions keyed sync:<n>, whose members are every enabled user hashed to that shard (paused users included), plus profile create cells on the global subscription. A membership change reconfigures the shard's socket in place and resumes from the in-memory cursor.
2. Handler. Runs synchronously inside dispatch, in one transaction, so queue insertion order equals event order. A follow create mirrors the record and queues a follow on every other app in the set. A follow delete removes the mirror row, cancels queued follows for the subject, purges its waits, and queues an unfollow for each mirrored record on the other apps. A profile create queues follows for every owner waiting on that subject.
3. Runner. A lane is one (owner, app) pair and runs its oldest non-terminal row first, so an unfollow and a later re-follow apply in order. Before a follow it re-checks the mirror (still followed somewhere, not already followed on the target) and the subject's profile. No profile puts the subject on the waiting list. At most eight lanes run at once.
4. Backfill. Enabling sync or adding an app lists the owner's follows on each unseeded app in short resumable bursts. When the run's listings are done, one guarded union step queues a follow for every (subject, app) pair the mirror lacks. A lane never writes a follow on an app whose most recent listing is not done.
5. Sweep. Daily per owner: re-list follows, propagate records the handler missed, drop mirror rows whose record is gone (only after a complete listing, and only rows last seen before it started), and re-check waiting subjects: every two days for Bluesky and Semble, weekly for the others.

Pacing, failures and pauses #

• Budget. Sync spends at most SYNC_HOURLY_POINTS and SYNC_DAILY_POINTS per account (defaults 1,500 and 15,000) of the PDS write budget of 5,000 per hour and 35,000 per day. A create costs 3 points, a delete 1. An exhausted budget waits; it never disables anything. Sync never touches the automation rate limiter.
• 429. Parks the whole account until the ratelimit-reset the PDS reported, without spending an attempt.
• 5xx, network, unknown profile. Retry with backoff, up to five attempts.
• 401. Parks the lane and counts toward a streak; the set pauses after five in a row, or immediately when the OAuth session row is gone. A paused user stays subscribed, so their events keep queuing in order. The OAuth callback resumes the set, drains the queue, and queues a sweep. Pending rows of a set paused for 30 days are cancelled by the cleanup job.

Configuration #

Variable	Default	Purpose
SYNC_SHARD_COUNT	1	Jetstream subscriptions synced users are hashed into. Jetstream v2 accepts 10,000 DIDs per subscription. Raising it is safe: a new shard resumes from the oldest sync cursor.
SYNC_JETSTREAM_MAX_LOOKBACK_US	24h	Resume window for sync shards. Bounds how long an outage can be before unfollows in the gap are lost.
SYNC_HOURLY_POINTS	1500	PDS write points sync may spend per account per hour.
SYNC_DAILY_POINTS	15000	PDS write points sync may spend per account per day.

The instance allow and block lists apply: an app whose follow or profile collection is blocked is not offered. A profile collection listed in NSID_REQUIRES_DIDS is not watched live, so late joiners on that app wait for the sweep.

Adding an app #

Sync reads the follow-target registry. Add the target as described in docs/follow-actions.md, then add its key to apps.items.knownValues in lexicons/run/airglow/graphSync.json; the parity test fails until both agree. An app whose profile collection is its own is watched live for late joiners automatically.

Local development #

The local PDS is not crawled by the public Jetstream, so live events for the test accounts never arrive on their own. Inject them into the running dev server with the admin-only endpoint, which is refused on non-local instances:

# alice follows a subject on Bluesky: write the record, then dispatch its event
pnpm tsx lib/scripts/dev-jetstream-event.ts --as <admin-did> --did <alice-did> \
  --collection app.bsky.graph.follow --op create --subject <subject-did> --write

# the matching unfollow
pnpm tsx lib/scripts/dev-jetstream-event.ts --as <admin-did> --did <alice-did> \
  --collection app.bsky.graph.follow --op delete --rkey <rkey> --write

# a profile create for a late joiner, dispatched from a record you supply
pnpm tsx lib/scripts/dev-jetstream-event.ts --as <admin-did> --did <subject-did> \
  --collection sh.tangled.actor.profile --op create --rkey self --record '{}'

--write needs a live Airglow OAuth session for the account, so sign in as it first.