Attachments #

Summary #

Users can attach images and PDFs to a message. Images are sent to vision-capable models as images. PDFs go through a file ingester that extracts their text, and the text is sent to the model. Attachments are blobs referenced from the message record: on the user's PDS for spaces users, and in a local blob store for fallback users. Extracted text is saved as its own blob, referenced from the file part, so a PDF is only processed once. PDF text extraction ships as the first ingester plugin, @scn-chat/plugin-pdf-text.

Motivation #

Vision and PDF support were on the original phase 1 list. Keeping the file bytes and extracted text as blobs in the user's own repo keeps attachments under the user's control like the rest of the conversation. Making extraction a plugin means OCR, Word documents, or other formats can be added later without core changes.

Design #

Upload flow #

1. The web UI sends the file to POST /api/attachments as the raw request body, with its Content-Type and an X-Filename header. Attachments are uploaded before the message is sent, so the user sees progress and errors early.
2. The server checks the MIME type and size. Images must be PNG, JPEG, WebP, or GIF, up to 20 MB. Other files are accepted when an ingester matches their type, up to 50 MB. These limits match the lexicon. The PDS may enforce a lower limit, and its error is passed through.
3. The server stores the bytes through the blob store below and gets back an atproto blob reference.
4. For non-image files, the matching ingester extracts text. The text is stored as a second text/plain blob.
5. The response carries the part to put in the message: an imagePart, or a filePart with name and extracted: { text, method }.

GET /api/conversations/:skey/blobs/:cid serves one of the signed-in owner's blobs, with its MIME type, X-Content-Type-Options: nosniff, Cache-Control: private, max-age=31536000, immutable since the URL names the blob's CID, Vary: Cookie so another account signed in on the same browser doesn't read it from the cache, and Content-Disposition: inline for an image or a PDF, so the browser shows it, a PDF in its own viewer, and attachment for anything else. For spaces users it calls com.atproto.space.getBlob for the conversation, and the PDS refuses any blob not referenced from a record in that space. The server passes that refusal on as 404, with no check of its own. For fallback users it serves the file from the owner's blob directory, or 404 when there is none. Shared conversations serve blobs through the sharing spec's route.

The web UI includes the returned parts in the message it sends. The server does not track which blobs a user uploaded. For spaces users, the PDS rejects a record that references a blob the account never uploaded, and for fallback users a blob either exists in the owner's directory or does not.

Blob store #

apps/server/src/blobs/ has one interface with two implementations, each bound to one user's DID like a RecordStore. createChatServices gives each user's ChatService the one for their storage mode, and callers use the service's putBlob and readBlob:

interface BlobStore {
  put(bytes: Uint8Array, mimeType: string): Promise<BlobRef>
  get(conversationUri: string, cid: string): Promise<{ bytes: Uint8Array; mimeType: string }>
}

• Spaces users. put calls com.atproto.repo.uploadBlob on the user's PDS with their OAuth session, which needs the blob permission in the permission set. get calls com.atproto.space.getBlob for the conversation.
• Fallback users. Bytes are written under DATA_DIR/blobs/<did>/<cid>, one directory per owner, with the CID computed the way a PDS computes it: CIDv1, raw codec, SHA-256. The same bytes therefore get the same CID if the user later moves to a spaces PDS, so their records' blob references stay valid after migration.

A PDS keeps an uploaded blob only if a record references it within a time window. An attachment that is never sent is cleaned up by the PDS for spaces users, and by a daily sweep of unreferenced files older than a day for fallback users, sweepLocalBlobs, which asks the local record store whether any record still references a blob.

Blobs are not cached on the server. A turn fetches the images it sends to the model from the blob store each time, and holds them in memory only for that turn.

Ingesters #

In @scn-chat/plugin-api:

interface Ingester {
  id: string
  accepts: string[]            // MIME types, e.g. ['application/pdf']
  priority?: number            // higher wins when several match, default 0
  method: 'text' | 'ocr'       // written to extracted.method
  ingest(file: { bytes: Uint8Array; mimeType: string; name?: string }): Promise<{ text: string }>
}

The highest-priority ingester that accepts the MIME type runs. An OCR plugin can therefore register as a fallback for scanned PDFs, or take over PDFs entirely with a higher priority. If no ingester accepts a non-image type, the upload is refused with 415.

@scn-chat/plugin-pdf-text uses unpdf to extract the text of every page, joined with page breaks. A PDF with no extractable text, such as a scan, fails with a clear message saying the PDF has no text layer.

Sending attachments to the model #

When the chat-turns spec builds the model's messages:

• An imagePart becomes an AI SDK file part with the image bytes and MIME type, if the resolved model has the vision capability. If it does not, the turn fails before calling the model with an error saying the model cannot read images. The web UI prevents this by disabling image attachments for such models.
• A filePart with extracted text becomes a text part: the file name as a heading, followed by the extracted text.
• A filePart without extracted text is left out, with a note in the text telling the model a file was attached but could not be read.

Scope Boundaries #

• No OCR. That is a future plugin.
• No sending PDFs to models as native PDF input.
• No attachments on assistant messages. Image generation is a future plugin.
• No audio or video.
• No attachments in encrypted conversations.
• No limit on how much extracted text is sent, beyond the model's own context limit.

Edge Cases and Decisions #

• Extraction runs at upload time, not at send time, so a failure shows up before the user sends the message.
• Fallback-mode blobs use the same CID scheme as a PDS, so migration to a spaces PDS keeps blob references valid.
• Spaces users' blobs are fetched from their PDS on every turn that needs them, with no server-side cache, keeping the PDS the only store of their attachments.
• A failed ingestion returns 422 with the ingester's message.
• The blob routes take an optional type query parameter naming an image MIME type or application/pdf, because a PDS's getBlob does not report the type. Any other type it names is ignored, so a request can never have a blob served inline as, say, HTML.
• Local blobs keep their MIME type and size in a small JSON file beside the bytes.

Acceptance Criteria #

• Uploading a PNG returns an imagePart whose blob can be fetched back through the conversation's blob route once the message is sent.
• The blob route returns 404 for a CID not referenced in the conversation, and for another user's conversation.
• The blob route serves a PDF inline as application/pdf when the request names that type, and any other non-image file as a download.
• Uploading a PDF with text returns a filePart with extracted text stored as a text/plain blob, with method text.
• A PDF without a text layer fails with a message saying it has no text.
• An unsupported MIME type is refused with 415.
• Files over the size limits are refused with 413.
• Fallback-mode blobs get the same CID a PDS would compute for the same bytes.
• When several ingesters accept a type, the highest priority runs.
• An image sent to a model without vision fails the turn before any model call.
• Extracted text reaches the model as a text part headed by the file name.
• The blob route returns the PDS's refusal as 404 for a blob not referenced in the conversation, without reading the conversation's records.
• No blob is written to server storage for a spaces user.
• The fallback sweep removes unreferenced blobs older than a day and keeps referenced ones.