STAR — STreaming ARchives

STAR serializes key-value data pairs into a flat stream, with header metadata. Application profiles can define additional data validations and restrictions.

Introduction

The STAR format (STreaming ARchive) stores sequences of key-value data pairs. The format is simple and flexible: a single required $type key in the header metadata lets applications associate additional data restrictions and validations for their own use-cases.

Compared to CAR ([[car]]): STAR is not content addressed and has no format-level CIDs ([[cid]]), contains data pairs instead of individual blocks, and makes the order of pairs explicitly significant. STAR can losslessly encode any CAR with less than one byte per block overhead, see Application Profiles below for an example.

Format

A STAR file begins with four magic bytes, followed by a length-prefixed metadata map with a $type key, and then zero or more data pairs. Each pair consists of two length-prefixed byte sequences, "key" and "value".

|------ header ------| |-------------------- data pairs --------------------|
[ magic | len | meta ] [ len | key | len | val ] [ len | key | len | val ] …

Magic

The magic bytes are 0x2A 0x6C 0x01 0x00: ascii *l ("star lite"), a fixed version 1, and a null byte to make four. STAR's magic bytes are always distinct from the first bytes of a CAR ([[car]]) file.

Length-prefixing

All length prefixes in STAR are encoded as unsigned variable-length integers ([[varint]]). The integer specifies the number of following bytes to read, not including the integer itself. Length prefixes must not contain superfluous trailing 0x00 bytes: the only number that can end in 0x00 is 0.

Metadata

The header metadata must be a DRISL-encoded ([[drisl]]) map with the exact size of its length prefix. The map's only reserved and required key is "$type", whose value must be a string. The meaning of $type is described below under Application Profiles. Arbitrary DRISL metadata can be included at other keys in the map.

Data pairs

Keys and values are both length-prefixed sequences of zero or more bytes. Exactly one key and one value must be present in each data pair. Duplicate keys and empty keys and values are permitted, but may be restricted by an Application Profile. Pairs are ordered: serializers and parsers must emit data pairs in the exact order they are provided/encoded.

Additional considerations

Any parser error must be treated as fatal for the entire archive.

There is no data-pair-level integrity verification in STAR, and early truncation at a data pair boundary cannot be detected by parsers. See Application Profiles below for use-cases that require these properties.

STAR encoding is deterministic where data pair contents and ordering is deterministic.

Like CAR ([[car]]), no format-level size limits are defined for header metadata or data-pair keys and values. However, parsers may impose reasonable maximums and are encouraged to provide configurable lower limits.

Application profiles

TODO

Not stable yet!

STAR is a new format, and while it has applications headed for production soon, be prepared for breaking spec changes while its DASL maturity remains Hot New Stuff. Additional backgound, resources, and open questions:

STAR-lite, an earlier version of this format specifically for atproto public data repository exports. It is not part of the atproto specs but is actively used on the network, to be superseded by STAR.
Initial public STAR feedback of the first attempt to adapt STAR-lite for atproto spaces, leading to its current flexible shape.
Proposal for STAR, tangled issue introducing STAR to DASL.
Presentation slides introducing STAR at the October 2026 CID Congress.
Interop test vectors for implementors.
Soon: a link to a summary of questions still to be settled, when that exists.

Appendix: Media Type

The media type for STAR is TBD! Maybe application/vnd.dasl.star? When is the right time to seek an iana assignment?

The conventional file extension for STAR is .star.