fih

hardened bluesky app fork with microcosm

Fork

fih MICROCOSM_PLAN.md

13 kB · 218 lines

Markdown

at main

· microcosm: custom/algorithmic feeds without the AppView · 4mo ago

e50c9189

Plan: replacing the Bluesky API with microcosm services #

TL;DR — the hard truth #

The app makes ~91 app.bsky.* read/hydrated queries (timeline, post threads, profiles, author feeds, feeds, search…). None of the three microcosm services provide these. They serve atproto primitives, not the hydrated "View" objects the UI is built on:

Service	Serves	Replaces in this app
Slingshot	`com.atproto.repo.getRecord`, `resolveHandle`, by-URI record fetch, MiniDoc identity, service resolution, a `hydrateQueryResponse` proxy	the ~12 `com.atproto.repo.`/`sync` record reads* + identity resolution
Constellation	backlink index: counts + lists of who-liked / who-reposted / who-replied / who-follows / who-quotes	the interaction counts & lists that today come baked into hydrated `app.bsky.*` views
Spacedust	WebSocket firehose of interactions (a "notifications firehose")	a live notifications stream, and live counters

Conclusion: microcosm gives the ingredients, not the meal. There is no "getTimeline" or "getPostThread" primitive. To be independent of api.bsky.app we must build a client-side hydration layer that assembles app.bsky.*-shaped View objects from {Slingshot records + Slingshot identities + Constellation counts}. That is, in effect, writing a mini-AppView inside the client.

This is a real project, not a config swap. Below is a staged plan that yields working increments instead of a big-bang rewrite.

Current architecture (what we're replacing) #

All API calls go through @atproto/api's AtpAgent (agent.app.bsky.*, agent.com.atproto.*). The agent is created in src/state/session/agent.ts (BskyAppAgent extends AtpAgent).
Reads are routed to the AppView via service proxying: every agent sets the atproto-proxy: <did>#bsky_appview header (BLUESKY_PROXY_HEADER in src/lib/constants.ts), so app.bsky.* queries hit api.bsky.app (PUBLIC_APPVIEW / PUBLIC_APPVIEW_DID).
Logged-out / public reads use createPublicAgent() → PUBLIC_BSKY_SERVICE (public.api.bsky.app).
Reads/writes are wrapped in @tanstack/react-query hooks under src/state/queries/**. This is the seam to intercept — the UI consumes hooks, not the agent directly, so we can swap the data source per-hook behind the same hook signature.

API surface inventory (from `agent.*` call sites in `src/`) #

~91 app.bsky.* read queries (get*, search*) → no microcosm equivalent; need hydration layer
~11 com.atproto.repo.* (getRecord + applyWrites/putRecord/createRecord/deleteRecord)
1 com.atproto.sync.getRepo
~20 com.atproto.server.* (auth/account — these go to the PDS, never the AppView; unaffected)
writes (putRecord/applyWrites/createRecord) go to the user's PDS, also unaffected by AppView choice

Key files #

src/lib/constants.ts — PUBLIC_APPVIEW, PUBLIC_APPVIEW_DID, PUBLIC_BSKY_SERVICE, BLUESKY_PROXY_HEADER
src/state/session/agent.ts — agent construction + configureProxy
src/state/queries/** — ~all read hooks (interception layer)
record reads to convert to Slingshot:
- src/features/liveNow/index.tsx:268
- src/state/queries/messages/actor-declaration.ts:128
- src/state/queries/threadgate/index.ts:119
- src/state/queries/postgate/index.ts:58

What each microcosm service maps to #

Slingshot — record + identity reads ✅ clean drop-in #

com.atproto.repo.getRecord → same lexicon, compatible response ({uri, cid, value}). Point record reads at slingshot.microcosm.blue instead of the PDS/AppView.
blue.microcosm.repo.getRecordByUri → ergonomic at-uri fetch (no repo/collection/rkey split).
com.atproto.identity.resolveHandle → handle→DID, bi-directionally verified.
blue.microcosm.identity.resolveMiniDoc → {did, handle, pds, signing_key} — everything needed to then talk to the right PDS for writes.
com.bad-example.proxy.hydrateQueryResponse → can fetch records referenced by paths in any upstream XRPC response (partial hydration helper; experimental).

Constellation — interaction graph ✅ verified working (see `constellation.openapi.yaml`) #

Source strings map directly to bsky interactions:

likes on a post: source=app.bsky.feed.like:subject.uri
reposts: source=app.bsky.feed.repost:subject.uri
replies: source=app.bsky.feed.post:reply.parent.uri
quotes: source=app.bsky.embed.record:record.uri (and recordWithMedia)
followers of a DID: subject=<did>&source=app.bsky.graph.follow:subject
blocks: source=app.bsky.graph.block:subject
getBacklinksCount → like/repost/reply counts
getBacklinkDids → "liked by" / "followers" lists (DIDs → resolve via Slingshot)

Spacedust — live interactions ✅ websocket #

/subscribe?wantedSources=...&wantedSubjectDids=<myDid> → a notifications stream (someone liked/replied/followed you), replacing app.bsky.notification.* polling for the live case.

Staged plan #

Phase 0 — client scaffolding (no behavior change) #

Add src/lib/microcosm/ with typed clients generated/handwritten from the 3 specs:
- constellation.ts (from constellation.openapi.yaml, verified)
- slingshot.ts (from slingshot.json)
- spacedust.ts (websocket)
All requests send User-Agent: fih (microcosm client; scan@scanash.com) per microcosm etiquette.
Add src/env hosts: CONSTELLATION_URL, SLINGSHOT_URL, SPACEDUST_URL (default to public instances).
No wiring yet — just the clients + a manual test screen.

Phase 1 — route record + identity reads through Slingshot (low risk) #

Replace the 4 com.atproto.repo.getRecord call sites with a Slingshot-backed helper (falls back to the PDS on miss). These are isolated and already com.atproto-shaped, so this is a near-drop-in.
Route handle/DID resolution through Slingshot resolveMiniDoc where the app resolves identities outside the logged-in agent.
Win: record reads no longer depend on api.bsky.app. Fully behind a flag.

Phase 2 — interaction counts/lists from Constellation (high value, isolated) #

Add hooks: useBacklinkCount(subject, source), useLikedBy(uri), useRepostedBy(uri), useFollowers(did) backed by Constellation.
First surface them as additive UI (e.g. real-time counts, "who liked" lists) — this works alongside the bsky AppView and immediately does something the stock client can't (cross-lexicon backlinks).
Then optionally use them to override the counts in hydrated views.

Phase 3 — the hydration layer (the actual AppView replacement) #

Status: profile hydration DONE + wired; post hydration built, not yet wired.

src/lib/microcosm/hydrate.ts assembles Views client-side:

✅ hydrateProfile(identifier) → ProfileViewDetailed (MiniDoc + profile record
- Constellation follower count). Wired into useProfileQuery behind the flag, with AppView fallback. Verified live: full profile incl. working CDN avatar. Known gaps (partial by nature): followsCount (needs listRecords), viewer state, verification.
✅ hydratePost(uri) → PostView (record + author basic-view + Constellation like/repost/reply/quote counts). Wired into usePostQuery behind the flag. Reconstructs image and external-link embed views from blob refs; verified the feed_thumbnail/feed_fullsize CDN URLs resolve (200 JPEG).
✅ hydrateProfileBasic(did) → ProfileViewBasic (post author embedding).
✅ viewer state (did I like/repost): reconstructed from Constellation did-filtered backlinks -> like/repost record rkey -> at-uri. Wired into hydratePost via agent.did. Verified live.
✅ Image CDN swapped from Bluesky's cdn.bsky.app to the community-run cdn.blueat.net (configurable via IMG_CDN_URL); same URL scheme, verified.
✅ quote-record embeds: recursively hydrate the quoted post into a recordView; handles recordWithMedia (quote + image/external). Depth-bounded. Also fixed the quote link source (app.bsky.feed.post:embed.record.uri, was wrong → counts were 0).

Remaining for Phase 3 (follow-ups, non-blocking):

video embeds (need the video service)
thread reply-tree assembly (replaces app.bsky.unspecced.getPostThreadV2)

Original Phase 3 scope notes:

PostView ← Slingshot getRecord(post) + Slingshot MiniDoc(author) + Constellation counts(likes/reposts/replies) + viewer-state (likes/reposts by me, via Constellation did filter on my DID).
ProfileView ← Slingshot MiniDoc + the profile record + Constellation follower/following counts.
ThreadView ← post + Constellation replies (app.bsky.feed.post:reply.parent.uri) → recurse.
Wire getProfile / getPostThread / single-post hooks to this behind the flag.
This is the bulk of the work and where "build a mini-AppView" lives.

Phase 4 — feeds & timeline (hardest; may stay hybrid) #

Status: following-feed builder DONE (not yet wired to the home screen).

✅ src/lib/microcosm/feed.ts buildFollowingFeed(viewerDid): lists the viewer's follows from their PDS, fans out per-author listRecords of recent posts (Slingshot doesn't proxy listRecords → go to each author's PDS via MiniDoc), merges by createdAt, hydrates the top slice. Added slingshot.listRecords (PDS-routed) helper. Verified end-to-end: a real merged timeline, newest-first. CAVEAT: N+1 fan-out is slow (~14s for 15 follows in a quick test). Needs caching + parallelism tuning before wiring to the home feed; viable for the concept, not a perf-ready timeline yet. NOT wired into usePostFeedQuery.
Following timeline (original notes): read my app.bsky.graph.follow records → fan-out fetch recent posts per follow → merge client-side. Expensive; viable for modest follow counts, needs caching/windowing.
Algorithmic / custom feeds (getFeed): DONE, and NOT locked behind the AppView (earlier assessment was WRONG). A custom feed is produced by a feed generator service, not Bluesky. getFeed is just: call the generator's getFeedSkeleton (post URIs) + hydrate. We do both ourselves: buildCustomFeed(feedUri) reads the generator record → resolves its #bsky_fg endpoint via Slingshot → calls getFeedSkeleton directly on the generator → hydrates the URIs with hydratePost. Verified end-to-end on the Discover feed (resolved discover.bsky.app, hydrated real posts) in ~2.3s, no AppView. This is faster + better than the fan-out following feed since the generator does the algorithm. NOT yet wired into usePostFeedQuery.
Search (searchActors/searchPosts): no microcosm equivalent → keep, or integrate a separate search index later.

Phase 5 — live notifications via Spacedust #

Status: live-interactions hook DONE (additive); feed replacement is follow-up.

✅ useLiveInteractions() subscribes over websocket to Spacedust for like/repost/reply/follow events targeting the current user's DID; exposes a live count + recent interactions. Verified: event shape {kind,link:{operation,source,source_record,subject}} and that one wantedSubjectDids=<myDid> subscription catches both follows (subject=DID) and likes/etc (subject=at://DID/post/...). Self-interactions filtered out.
Follow-up: wire into a live unread badge; optionally replace the poll-based listNotifications feed entirely (large — it does grouping + hydration).

Phase 6 — writes & auth (mostly already independent) #

com.atproto.server.* and com.atproto.repo.put/applyWrites already target the user's PDS, not the AppView — independent of this work once OAuth/app- password login resolves the PDS (Slingshot MiniDoc gives the PDS URL).

Honest assessment of "fully independent" #

Achievable for: record reads (Slingshot), identity (Slingshot), interaction counts/lists (Constellation), live notifications (Spacedust), writes/auth (PDS, already independent), single-post/profile/thread views (hydration layer), the following-feed (PDS fan-out), and custom/algorithmic feeds (feed-generator skeleton + hydration — NOT AppView-locked).
The actual remaining gaps: full-text search (searchActors/searchPosts need a search index — genuinely a separate service microcosm doesn't provide), video embeds (need the video service), and followsCount (listRecords aggregation). Feeds are NOT a gap.
Effort: Phases 0–2 are a few focused sessions and give big, demoable wins. Phase 3 is the real build. Phases 4–5 are research-heavy.

Recommended first move #

Phase 0 + Phase 2 (Constellation): scaffolding + a real "who liked this / live like count" feature sourced entirely from microcosm. Proves the whole pipe end-to-end, ships something the stock client can't do, and de-risks Phase 3.