zig-matroska
A sans-I/O reader and writer for Matroska and WebM in Zig.
Star
Fork
Watch
2 1 0
Files
.forgejo
LICENSES
spec
src
tests
tools
.gitignore
build.zig
build.zig.zon
build.zig.zon.nix
flake.lock
flake.nix
package.nix
README.md
REUSE.toml
README.md

zig-matroska #

A sans-I/O reader and writer for Matroska and WebM in Zig 0.17: the EBML underneath them (RFC 8794), demuxing a stream into tracks, frames, cues, chapters, tags and attachments, and muxing frames back into a file (RFC 9559).

The API documentation, generated from the doc comments in the source, is at https://jeff.jcollie.page/zig-matroska/.

What "sans-I/O" means here #

Nothing in the library opens a file, reads a socket or seeks. The demuxer is a state machine the caller pushes bytes into; the muxer writes to a std.Io.Writer, which is only an interface. Where Matroska needs to move around a file — to read the index after the Clusters, or to fill in sizes at the start once they are known — the library asks rather than does it.

  • Demuxer.next(bytes, eof) takes whatever the caller has and returns how much it consumed and one event: the header, the tracks, a frame, the cues, and so on. When it needs more it says how much (need_more); when it wants to be somewhere else it says where (seek).
  • Muxer.finish() returns patches — runs of octets to write over what was written at given offsets — for the Segment size, the Duration and the SeekHead. The file is valid before they are applied, just less useful.

matroska.io has the adapters for when the bytes do come from std.Io: FileSource and ReaderSource drive a Demuxer from a file or a stream, and applyPatches writes a Muxer's patches to a file. They are the only part of the library that does I/O.

Reading #

const matroska = @import("matroska");

var d: matroska.Demuxer = .init(gpa, .{ .seekable = true });
defer d.deinit();
var src: matroska.io.FileSource = .init(gpa, io, file);
defer src.deinit();

while (true) switch (try src.next(&d)) {
    .tracks => |tracks| for (tracks) |t| std.debug.print("{d}: {s}\n", .{ t.number, t.codec_id }),
    .frame => |f| if (f.whole()) consume(f.track, f.timestamp_ns, f.data),
    .eof => break,
    else => {},
};

Or by hand, with no std.Io at all:

while (true) {
    const step = try d.next(buf[start..end], at_eof);
    start += step.consumed;
    switch (step.event) {
        .need_more => |n| {}, // keep buf[start..end], make it at least n bytes, call again
        .seek => |offset| {}, // move the source to offset, drop the buffer, call again
        .frame => |f| {},     // f.data borrows buf
        .eof => break,
        else => {},
    }
}

What the demuxer handles:

  • Every block shape: SimpleBlocks, BlockGroups (duration, references, additions, discard padding), and all three lacings — Xiph, EBML and fixed — each laced frame reported as its own event.
  • Unknown sizes on the Segment and Clusters, as live streams write them, ended by the first element the schema does not allow inside them.
  • Frames larger than the buffer. A block over Options.max_inline comes in pieces, each saying where it belongs in its frame, so that memory use is bounded by the caller rather than by the file.
  • The index. With seekable set it follows the SeekHead to Cues, Chapters, Tags and Attachments written after the Clusters before reading the Clusters; seekTime then seeks by the Cues, and seekOffset hunts forward for the next Cluster from anywhere, which is also how to carry on past damaged data.
  • Content encodings: header stripping is reported as a prefix rather than copied in, and content.decode undoes zlib.
  • Metadata: Info, Tracks (video, colour and HDR metadata, projection, audio, block addition mappings), Cues, Chapters (nested editions and atoms), Tags (nested simple tags and targets) and Attachments, each as a typed structure.

Malformed input is an error and never a crash; the demuxer is fuzzed to keep it that way.

Writing #

var m: matroska.Muxer = .init(gpa, &file_writer.interface, .{ .doc_type = .webm });
defer m.deinit();
const video = try m.addTrack(.{ .number = 0, .type = .video, .codec_id = "V_VP9",
    .video = .{ .pixel_width = 1280, .pixel_height = 720 } });
try m.writeHeader();
try m.writeFrame(.{ .track = video, .timestamp_ns = 0, .data = frame, .keyframe = true });
const patches = try m.finish();
try matroska.io.applyPatches(io, file, 0, patches);

The muxer gathers each Cluster in memory so it is written with a known size, cuts Clusters at video keyframes and when they grow too long or too large, writes Cues for the keyframes, and reserves room at the front for a SeekHead that finish fills in. Frames that carry a duration, references, additions or discard padding go in BlockGroups; the rest in SimpleBlocks. writeLaced packs several frames into one block with whichever lacing is smallest.

Mode.live writes a file that needs no patches at all — unknown Segment size, no SeekHead, Duration or Cues, and optionally unknown-size Clusters written as the frames arrive — for a pipe or a live stream.

DocType.webm refuses codecs and elements the WebM profile does not have, checking every element it writes against the schema.

The schema #

src/schema.zig is generated from the specification's own EBML schema, spec/ebml_matroska.xml (vendored from the CELLAR working group's repository), and spec/ebml_header.xml, which writes out the EBML header elements of RFC 8794 in the same schema format. It gives every element its ID, type, parent, defaults, range, and the specification's definition as a doc comment. zig build schema regenerates it; zig build test fails if the committed copy and the schema disagree. The generator uses zxml, as a lazy dependency, so nothing that only uses the library ever fetches it.

Building #

$ nix develop
$ zig build test            # unit, fixture and round-trip tests, and the schema check
$ zig build test --fuzz     # fuzz the reader, the demuxer and a mux round trip
$ zig build oracle          # check the muxer against mkvmerge, mkvinfo and ffprobe
$ zig build coverage        # kcov line coverage, in zig-out/coverage
$ zig build docs-serve      # serve the API documentation on localhost:8000
$ zig build example -- --frames file.mkv   # mkvdump: what is in a file

The tests check the demuxer against files written by ffmpeg and mkvmerge in tests/fixtures, frame by frame against what ffprobe says is in them — timestamps, keyframe flags, sizes and CRCs — fed a byte at a time and a megabyte at a time, with and without seeking. tools/fixtures.py made them. zig build oracle goes the other way: tools/samples.zig writes files with the muxer and tools/oracle.py checks that mkvmerge and mkvinfo accept them and that ffprobe finds exactly the packets that went in.

To use it from another project:

$ zig fetch --save git+https://git.jcollie.dev/jeff/zig-matroska.git

const matroska = b.dependency("matroska", .{ .target = target, .optimize = optimize }).module("matroska");

Where it lives #

The repository is at https://git.jcollie.dev/jeff/zig-matroska:

$ git clone https://git.jcollie.dev/jeff/zig-matroska.git

It is mirrored on Tangled at https://tangled.org/jcollie.dev/zig-matroska.

It is on Radicle as rad:z2F992PFX6CPAhurX8h58jnZrhs4n, which is the only thing a peer needs to find it — a Radicle repository is discoverable by its identifier and by nothing else:

$ rad clone rad:z2F992PFX6CPAhurX8h58jnZrhs4n

License #

MIT, except spec/ebml_matroska.xml, which is the Matroska specification's schema under CC-BY-4.0, and src/schema.zig, generated from it, under both. See REUSE.toml.

References cited #