GET /api/unfurl?url=https://example.com/post fetches that page on Cloudflare's side (browsers aren't allowed to read other sites' pages) and returns a tidy JSON summary of what the page says about itself:

the usual: title, description, site name, language, canonical address, best icon, image, video, theme colour, dates, licence, generator;
atproto: at:canonical, at:alternate, at:author, at:me, custom at:{namespace}:{property} tags (the at-tags convention), plus the older site.standard.document / site.standard.publication links, <link rel="alternate" href="at://…">, Semble's atprotocol:creator and Bluesky's oEmbed link; and the site's own /.well-known/atproto-did (the whole site is an account) and /.well-known/site.standard.publication;
fediverse: fediverse:creator, the ActivityPub alternate link;
IndieWeb: rel=me, webmention, micropub, indieauth, hub, syndication, in-reply-to, and a light reading of microformats (first h-entry, the representative h-card);
feeds (RSS, Atom, JSON Feed), the oEmbed link, JSON-LD (headline, author, dates, image, video), Open Graph and Twitter cards.

The summary shape is described at the top of parse.js and in site/lib/unfurl.js (the browser side, with helpers that turn it into plain-language rows). Two additions beyond the basic shape: manifest (the page's web-app manifest address) and indieweb.endpoints (pingback, microsub, authorization, token, self). If the address is a picture, video, PDF or 3D model rather than a page, the answer is just { ok, url, contentType, kind }. Failures are { ok: false, error } with a 4xx/5xx status.

Files #

parse.js — pure reading of a page (no network, no libraries) plus the check of which addresses are allowed. Runs the same in the Worker and in Node.
index.js — the Worker: routes /api/unfurl, fetches carefully, caches.

Limits and safety #

Only public http(s) sites: no IP addresses, localhost, *.local, *.internal, single-word names, passwords in the address, or odd ports. Every redirect is checked again, at most 5.
Nothing from the visitor is passed on (no cookies or headers); our own fixed user agent Fieldwork-LinkPreview/1.0.
6 seconds in total for the page; at most 384 KB read, then the download is cancelled. The .well-known checks and the manifest get 3 seconds and 2 KB / 64 KB. At most about 4 requests are in flight at once and about 10 in total.
Page text is cleaned and cut (titles 300 characters, descriptions 1000, other text 500, lists 20); only http(s) web addresses and strictly shaped at:// addresses survive. Whoever shows the result must show it as text.
Answers are public data: access-control-allow-origin: *, cached for six hours by the Cloudflare cache (successes only). There is no way to force a refetch from outside; fresh in the browser client only skips the browser's own remembered copy.
Free plan: 10 ms of CPU per request (waiting on the network doesn't count). Reading is a single pass with hard caps, which took roughly 1 to 6 ms for a 384 KB page in Node once warmed up; a cold start or an unusually dense page could go over. That would fail only that one lookup. Workers can't fetch IP-literal addresses, which also keeps the refusal above honest.

How it deploys #

wrangler.jsonc at the repo root names this script (main) and gives the static files a binding (ASSETS). run_worker_first: ["/api/*"] means only /api/* runs the script; every other path is served as a plain static file exactly as before, without running any code. Workers Builds deploys it with the rest of the site; preview branches get a preview version.

How to test #

Reading and refusing: node tools/smoke/run.mjs unfurl (part of the smoke suite; the local test server answers /api/unfurl from pretend pages using the real parser, so no network is needed).
Config and bundling: npx wrangler@4 deploy --dry-run --outdir /tmp/out.
The real thing, locally: npx wrangler@4 dev, then open http://localhost:8787/api/unfurl?url=https://example.com/.