zig 0.17 driver for libhegel, with the harness parts for stateful property tests

README.md

hegel-zig #

a zig 0.17 driver for libhegel, the property-testing engine behind Hegel (MIT, by Antithesis; hegel-rust describes itself as based on Hypothesis). libhegel generates test cases, shrinks failures and replays them. this repo wraps its C API in the loop a zig property needs, and carries the harness parts that zat, burner-redis and docket each wrote once: the libhegel build, an operation tally, an echo mode for crashes, and a throwaway redis-server to diff against.

install #

needs zig 0.17, and cargo and git the first time a build needs libhegel.

zig fetch --save=hegel https://tangled.org/zzstoatzz.io/hegel-zig/archive/<commit>.tar.gz

mark the dependency .lazy = true and gate it behind an option, so nothing but the property test pays for it:

if (b.option(bool, "hegel", "build the property tests (builds libhegel once)") orelse false) {
    if (b.lazyDependency("hegel", .{ .target = target, .optimize = optimize })) |hegel| {
        const pbt = b.addExecutable(.{
            .name = "pbt",
            .root_module = b.createModule(.{
                .root_source_file = b.path("pbt/machine.zig"),
                .target = target,
                .optimize = optimize,
                .imports = &.{.{ .name = "hegel", .module = hegel.module("hegel") }},
            }),
        });
        const run = b.addRunArtifact(pbt);
        run.addPassthruArgs();
        b.step("pbt", "run the state machine property test").dependOn(&run.step);
    }
}

zig build pbt -Dhegel then builds libhegel if it has not been built, and runs the harness.

use #

const hegel = @import("hegel");

const Rule = enum { put, sum };
var tally: hegel.Tally(Rule) = .{};

fn property(_: void, case: *hegel.TestCase) !void {
    var toy: Toy = .{};
    tally.case();
    var machine = try case.machine(Rule, &.{ 2, 1 }, 20);
    defer machine.deinit();
    while (try machine.next()) |rule| {
        tally.step(rule);
        switch (rule) {
            .put => {
                const key = try case.index(4);
                const value = try case.int(u8, 0, 255);
                case.note("put(k{d}, {d})", .{ key, value });
                toy.put(key, value);
            },
            .sum => {
                case.note("sum()", .{});
                if (toy.sum() != toy.recompute()) return case.fail("sum() disagrees with a recompute");
            },
        }
    }
    tally.finish(toy.count());
}

pub fn main() !void {
    const outcome = hegel.run(.{}, {}, property);
    tally.print();
    try outcome;
}

the whole program is examples/stale_cache.zig. a failure prints the shrunk history and a blob that replays it:

FAILED: sum() disagrees with a recompute
minimal history:
put(k0, 0)
sum()
put(k0, 1)
sum()
a throwaway redis-server
const with_redis = b.addSystemCommand(&.{"sh"});
with_redis.addFileArg(hegel.namedLazyPath("with-redis"));
with_redis.addArtifactArg(pbt);
with_redis.addPassthruArgs();

starts a redis-server with no persistence in a temporary directory, runs the harness with HEGEL_REDIS_SOCKET set, and removes both when it exits. -p <port> before the command also listens on TCP and sets HEGEL_REDIS_URL. for differential tests against the real thing.

a crash
HEGEL_ZIG_ECHO=1 zig build pbt -Dhegel

prints every case's notes as they are made. a panic in the code under test kills the process before libhegel can shrink, so the history has to be read off the end of the output.

design #

  • the engine stays libhegel — generation, shrinking and replay are the hard part and already exist. this is the loop, the draws and the report.
  • libhegel is built by the build — scripts/libhegel.sh builds one pinned hegel-rust revision under ~/.cache/hegel-zig and zig caches the result. the pin moves together with the wrapper, because hegel.h is translated at build time and the C API is still changing. rust is needed once.
  • opt-in in every consumer — a lazy dependency behind -Dhegel, so zig build test in a project that uses this needs neither rust nor this repo.
  • the tally is part of the result — a run that found nothing prints what it exercised. a quiet run over an empty state space is otherwise indistinguishable from a quiet run over a real one.
  • only as wide as its harnesses — the API grows when one of the projects above needs a draw, not before.

develop #

zig build test                    # runs the example; it must find the planted bug
sh scripts/libhegel.sh <rev>      # build libhegel for a hegel-rust revision, print the checkout

-Dhegel-include=<dir> -Dhegel-lib=<libhegel_c.a> use a libhegel built elsewhere.

docs #

docs/ holds the method: how a harness is built, checked and turned into reviewable fixes, written from the three campaigns that produced this repo.

used by #

  • zat — the MST against a key-value model, including partially loaded trees
  • burner-redis — the store against redis-server, command by command
  • docket — the worker loop under a virtual clock, and its memory backend against redis-server

license #

MIT. libhegel is MIT, by Antithesis.