Something went wrong. Try again.
bulleted.app spaces node bug
Something went wrong. Try again.
4.4 kB · 84 lines · Created 2mo ago
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485# Space bullets are deleted by the ProvisionalReconciler
Space nodes disappear ~15 minutes after they are written. The write path marks themprovisional, nothing can ever confirm them, and a background sweep then concludes thewrite never landed and deletes them.
## The mechanism
1. You write a bullet into a space. The write path stores the row with the space coordinate and `provisional = 1`, `firehose_seq = NULL` (`src/writer.rs:4045`). The writer is careful here — it threads `space` through explicitly, calling these helpers "the last place a space write could silently be indexed as public."2. Nothing ever clears that flag from ingest. A space is deliberately *not* broadcast — `docs/spaces.md`: "A space does not use the firehose — the whole point is that its contents are not broadcast." So the confirmation the flag waits for cannot arrive.3. 15 minutes later (`PROVISIONAL_TIMEOUT`, `src/backfill.rs:817`) the sweep picks the row up. The query is **global, not scoped by space** — and `src/queries.rs:1394` says so on purpose: "The one query here that is not scoped by `space`, and deliberately so."4. The reconciler groups by `(space, did)` but then calls `self.backfiller.backfill_repo(&did)` (`src/backfill.rs:917`) — the single shared `RepoBackfiller` built for the **public** plane in `src/main.rs:307` and handed to the reconciler at `src/main.rs:432`. There is no `.in_space(space)`. It walks the public PDS repo.5. Space records aren't in the PDS repo, so `seen == false` (`src/backfill.rs:962`). The guard at line 971 is `!seen && walk.complete && !walk.truncated` — and `complete` is just `!truncated` (line 602), so an ordinary repo passes.6. `drop_phantom` deletes the row (`src/backfill.rs:972`), logging *"dropped a provisional row the PDS does not have: the write never landed."*
Because it routes through the indexer's delete path, it also takes the co-keyed note, tag,mention and FTS rows, and **orphans the children** — which is why it reads as the outlinegutting itself rather than one bullet vanishing.
## The proof
A temporary test: a provisional row in space `did:plc:authority/app.bulleted.space/self`,with the member's public repo walking cleanly to its end.
```REPRO REPORT: ReconcileReport { examined: 1, confirmed: 0, dropped: 1, unresolved: 0, failed: 0 }REPRO SURVIVORS: 0```
The bullet is deleted. The test was reverted; the tree is clean.
Why this shipped: all three existing reconciler tests (`src/backfill.rs:1373`, `1409`,`1464`) insert `space = ''`. The space case was never exercised.
## Why it's intermittent rather than constant
A space sync *does* clear the flag — `src/space_sync.rs:237` applies its commit`.in_space(coordinate)`. So rows survive if a `/space` visit re-indexes them inside the15-minute window. Write some bullets, navigate away for 15 minutes, and the sweep wins.That fits "disappear after a while, I have to start fresh." A truncated or unreachablerepo also spares them, so it will not reproduce uniformly across accounts.
## Confirm it on prod
```shgrep "the write never landed" <log> # WARN, names did + rkeycurl -s <host>/_metrics | grep bulleted_reconcile_dropped```
A nonzero `bulleted_reconcile_dropped` climbing while `bulleted_reconcile_confirmed` staysflat is the signature.
## On fixing it
The obvious repair — make the reconciler space-aware — **is not available to a worker.**Reading a space needs a member's live OAuth session through the three-hop`getDelegationToken` -> `getSpaceCredential` chain, and this app holds no tokens by design.That is the same constraint that stops subscriptions being renewed by a worker.
So the fix is to make the sweep decline to judge what it cannot see: skip non-defaultspaces when grouping in `sweep()`, so it does not walk the wrong repo at all, plus a`space == DEFAULT_SPACE` guard before `drop_phantom` as the correctness backstop. Spacerows would then stay provisional indefinitely — which is already the accepted, documentedstate for `note`, `mirror` and `outline` rows (`src/writer.rs:4222`: "There is noreconciler for a provisional mirror"). A permanently-pending style is a cosmetic bug; thisis data loss.
Worth noting the deliberate-looking comment at `src/queries.rs:1394` is reasoning purelyabout index efficiency, and it is what made the missing space predicate read asintentional.