ghostoff
Migrating content using the Ghost API to Offprint / Standard Site on atproto
Star
Fork
Watch
ghostoff docs session-summary.md
5.0 kB · 75 lines
Markdown
at main

Building GhostOff: from Ghost CMS to Offprint in one session #

GhostOff is a small TypeScript tool that exports public posts from a Ghost CMS and uploads them to Offprint using the AT Protocol and Standard.site lexicons. This post walks through the decisions and gotchas from the first build session.

Starting point #

The goal was straightforward: take content that lives in Ghost and make it available on Offprint, an atproto-based long-form publishing platform. Ghost exposes a read-only Content API, and Offprint's underlying data format is a set of atproto records (site.standard.document plus app.offprint.document.article). So the task reduces to:

  1. Fetch posts from Ghost.
  2. Download and re-host images as atproto blobs.
  3. Convert Ghost HTML into Offprint blocks.
  4. Write the records to a PDS.
  5. Keep track of what was uploaded so reruns are idempotent.

Stack choices #

We picked TypeScript because the atproto ecosystem has first-class tooling there. For the atproto side we used the atcute packages: @atcute/client for XRPC, @atcute/password-session for password-based auth, and @atcute/standard-site plus @atcute/lexicons for schemas and runtime validation.

For DOM parsing we used linkedom, which is light and Node-friendly. For image resizing we used sharp. For the CLI surface we used commander and dotenv.

Mapping the two formats #

Ghost posts come back as HTML with a JSON representation of the Ghost editor's cards: headings, paragraphs, galleries, embeds, code blocks, and custom HTML. Offprint expects a block array: app.offprint.block.text, heading, bulletList, orderedList, codeBlock, image, imageGrid, webEmbed, blockquote, and so on.

The converter walks the body of the post and maps top-level elements:

  • <p> -> block.text
  • <h1>-<h3> -> block.heading
  • <ul> / <ol> -> block.bulletList / block.orderedList
  • <pre><code> -> block.codeBlock
  • <figure class="kg-gallery-card"> -> block.imageGrid
  • <img> / single-image <figure> -> block.image
  • <blockquote data-bluesky-uri="..."> -> block.webEmbed
  • <hr> -> block.horizontalRule

Inline formatting (<strong>, <em>, <a>, <code>) is turned into byte-offset facets on each block.text plaintext string, matching the app.offprint.richtext.facet format.

The biggest format surprise was that site.standard.document.content is a single open-union object ({ $type: "app.offprint.content", items: [...] }), not an array of block objects. Once we fixed that, validation passed cleanly.

Images and blobs #

atproto stores images as blobs, with a 1 MB limit per blob. The tool downloads each referenced image into a local cache under ghostoff-export/assets, then runs it through sharp to resize and compress if it's over about 900 KB. We deliberately stay under the limit to avoid edge cases. The resulting blob reference is reused across posts and across reruns via the state file.

Some image formats commonly found in Ghost content (favicon.ico, remote SVGs) are not supported by sharp, so those are logged and skipped rather than failing the whole post.

Authentication and rate limits #

We authenticate with a password session against the user's PDS (for the test migration, https://selfhosted.social). The session is persisted locally so reruns can resume without re-logging in.

Network calls use a small retry helper that looks at Retry-After and RateLimit-* headers. XRPC calls go through @atcute/client's retryFetchHandler, and the raw blob upload has its own retry loop for 429 / 5xx responses.

Dry-run mistakes #

The dry-run mode was useful, but it taught us a lesson: don't reuse the real state file for dry runs. The first dry run wrote placeholder AT-URIs like at://dry-run/... into the state file, and the subsequent real run tried to update those records. We fixed that by only saving state during real uploads and by ignoring any cached records whose URI doesn't start with at://did:.

The result #

Running the finished script against https://atprotocol.dev uploaded all 26 public posts to the test publication at https://ghostoff.offprint.app. Each post now has:

  • a site.standard.document record pointing to the test publication,
  • a cover image blob (when present),
  • the post body as app.offprint.content blocks,
  • an app.offprint.document.article record linking the two.

You can inspect one of the records directly on the PDS:

https://selfhosted.social/xrpc/com.atproto.repo.getRecord?repo=did:plc:bbj62ki3gmlsjfzsr3dxbkna&collection=site.standard.document&rkey=3mpnnekkw2n2l

The "W for ATProto" post demonstrates the full pipeline: paragraphs, a Bluesky embed converted to block.webEmbed, a two-image gallery as block.imageGrid with a caption, and link facets inside the text blocks.

What's next #

See docs/next-steps.md for ideas ranging from better embed handling and CLI polish to tests and OAuth support.