struthio
A small configuration language for Zig: ZON with let, if and templates over a context, evaluated to a struct, always finishing.
Star
Fork
Watch
17 1 0
Files
.forgejo
LICENSES
src
tests
tools
tree-sitter-struthio
.gitignore
build.zig
build.zig.zon
build.zig.zon.nix
CONTRIBUTING.md
flake.lock
flake.nix
package.nix
README.md
REUSE.toml
README.md

struthio #

A small configuration language for Zig 0.17 whose programs evaluate to a Zig struct. A program is ZON with expressions in it: it can read a context the caller hands it, bind names with let, choose with if and switch, compute with the usual operators, template strings with ${...}, merge structs with ++, and call a handful of built-in functions and any the caller adds. It cannot loop or recurse, so every program finishes, in time proportional to its length.

const struthio = @import("struthio");

const Config = struct {
    name: []const u8,
    port: u16 = 8080,
    tls: bool = false,
    log_level: enum { debug, info } = .info,
    tags: []const []const u8 = &.{},
};

const Context = struct { hostname: []const u8, prod: bool, debug: bool };
const ctx: Context = .{ .hostname = "web1", .prod = true, .debug = false };

var diag: struthio.Diagnostic = .{};
const config = struthio.evalSource(Config, gpa,
    \\.{
    \\    .name = "web-${ctx.hostname}",
    \\    .port = if (ctx.prod) 443 else 8080,
    \\    .tls = ctx.prod,
    \\    .log_level = if (ctx.debug) .debug else .info,
    \\    .tags = .{ "app", if (ctx.debug) "debug" },
    \\}
, &ctx, .{}, &diag) catch |err| {
    std.log.err("config: {f}", .{&diag}); // "line:column: message"
    return err;
};
defer config.deinit();
// config.value is a Config: .{ .name = "web-web1", .port = 443, .tls = true, ... }

The API documentation is generated from the doc comments and published at https://jeff.jcollie.page/struthio/.

The name is the genus of the ostrich, which is a bird that cannot fly — as a struthio program cannot loop.

Why #

Configuration that somebody other than the program's author edits wants more than ZON — the same file for staging and production, a hostname spliced into a path — and much less than a general-purpose language. Three properties follow:

  • It always finishes. There is no loop syntax and no way to define a function, and a let can only name what came before it, so nothing can refer to itself. What a program builds is bounded by Limits — string length, list length, and the work of comparing and decoding — since ++ on a value built with ++ doubles it.
  • An error is an error, with a position. A syntax error, a type error (1 + "a"), a missing field or an out-of-range integer comes back as an error and a Diagnostic saying line:column: message — or file:///path:line:column: message when the program's files have names. A decoding error names the path, .listen[1].port: -1 does not fit in u16, and points at the field in the program that wrote it.
  • It does no I/O. A program reads its source, its context, what the host's functions return, and the files it includes — which the host reads for it, when the program is parsed. Nothing else.

The language #

Any ZON file is a program, and decodes to what std.zon.parse would make of it, with two exceptions: integers are 64-bit signed, so a literal that does not fit an i64 is an error; and ${ in a string starts an interpolation (write \$ for a literal $).

// Comments are Zig's.
let host = ctx.hostname;                // `let` binds a name, once
let base = .{ .port = 8080, .tls = false };

base ++ .{                              // `++` merges structs
    .name = "web-${host}-${ctx.region}",            // templated string
    .port = if (ctx.prod) 443 else base.port,
    .tls = ctx.prod and ctx.cert_path != null,
    .cert = ctx.cert_path orelse "/etc/ssl/default.pem",
    .log_level = if (ctx.debug) .debug else .info,  // enum literal
    .workers = @max(1, ctx.cpus / 2),
    .tags = .{ "app", @lower(ctx.env) },
    .admin = if (@contains(ctx.users, "root")) "root",  // no `else`: left out
    .backend = if (ctx.cache) |c| .{ .redis = c.url } else .memory,
    .pool = switch (ctx.cpus) {                     // ranges and lists
        1...4 => 8,
        5...16, 32 => 32,
        else => 64,
    },
}

Values #

Kind Written Notes
null null
bool true, false
int 42, -7, 0xff, 0o17, 0b101, 1_000, 'a' i64; overflow is an error
float 1.5, 1e-3, 0x1p4, inf, nan f64
string "a\n${x}", \\ multiline lines escapes are Zig's plus \$
enum literal .name, .@"two words" an enum tag, or a union variant with no payload
list .{ 1, 2, 3 } also a tuple, array or slice
struct .{ .a = 1, .b = 2 } also a union: .{ .variant = payload }

.{} is both an empty list and an empty struct.

Expressions #

From loosest to tightest, as in Zig:

Operators
or bool operands only, short-circuit
and bool operands only, short-circuit
== != < <= > >= do not chain; == compares deeply, and 1 == 1.0
orelse the left side unless it is null
+ - ++ ++ joins strings, joins lists, and merges structs (right side wins)
* / % / truncates on ints; % takes the sign of the left side
! - prefix
x.name x[i] x.? x.len field, index, unwrap an optional, length of a string or list

There is no truthiness: if and the boolean operators want a bool.

if (cond) a else b is an expression whose branches extend as far right as they can. if (optional) |x| a else b binds x to the value when it is not null. As the value of a struct field or a list item, an if may leave out its else, and when its condition is false the field or item is left out — so the struct's default applies, or the list is one shorter.

switch (x) { cases => value, ..., else => value } chooses the first prong with a case that matches, trying them in the order written; else is taken only when none does, wherever it is written. A case is a value, matched with ==, or an inclusive range lo...hi, matched with <= at both ends, and a prong may list several separated by commas. A prong can capture with |name| after its =>: matched by its tag, a tagged union from the context gives its payload, and anything else gives the value switched on.

switch (ctx.backend) {
    .memory => "none",
    .redis => |r| r.url,
}

A switch that matches nothing, with no else, is an error — except as the value of a struct field or a list item, where, like an if without else, it leaves the field or item out. The same literal written as two cases is a syntax error; nothing else about the cases is checked until the switch is evaluated, since a program's values have no types to check them against.

In a multiline \\ string, interpolation works and escapes do not; a literal ${ there is ${"$"}{.

Names #

  • ctx is the caller's context.
  • let name = expr; at the start of a program binds name for everything after it. Names cannot be rebound or shadowed, as in Zig.
  • |name| in an if binds for the then branch, and in a switch prong for that prong.

Every let is evaluated, in order. One that fails is only an error if it is used, so let cert = ctx.cert.?; is harmless in a program that reads cert only when ctx.cert != null.

Built-in functions #

Function
@lower(s), @upper(s), @trim(s) ASCII case; trims spaces, tabs and line ends
@len(x) bytes of a string, items of a list, fields of a struct
@contains(hay, needle) substring of a string, or member of a list
@startsWith(s, p), @endsWith(s, p)
@join(list, sep) items written as ${} would write them
@replace(s, from, to), @split(s, sep)
@min(...), @max(...) of two or more numbers or strings, or of one list
@int(x), @float(x), @string(x) conversions; @int truncates and parses "0x10"
@include("path") the value of another file; see below

Including files #

@include("path") stands for the value of the program in another file:

let defaults = @include("defaults.struthio");
defaults ++ .{ .port = 8443 }

The included file is a program of its own. It sees ctx and the host's functions, but none of the names of the file that includes it, and its own lets are not seen outside it.

Files are named by file: URIs, and the path is a string literal holding a URI reference, resolved against the URI of the file it is written in. So it is usually a relative path — defaults.struthio, ../common/tls.struthio — but it can be an absolute one, /etc/app/base.struthio, or a whole URI, file:///etc/app/base.struthio. Whatever it resolves to must be a file: URI with an empty host or localhost. A program with no filename of its own can include only absolute paths and URIs, having nothing to resolve a relative path against.

A file is read and parsed once however many times it is included, and a file that includes itself, directly or not, is a syntax error.

Files are read by the host, when the program is parsed, so a program evaluated many times reads them once. A host that gives no way to read them does not support @include, and a program using it does not parse.

Using it from Zig #

Program.parse parses once; Program.eval(T, gpa, &ctx, &diag) evaluates against a context and decodes the result into T. evalSource does both. T may be struthio.Value for the undecoded result, and a field of type Value takes whatever the program wrote there.

The context is a pointer to anything, read by reflection only as far as the program reads it: ints, floats, bools, enums, optionals, []const u8 strings, slices, arrays and tuples, structs, tagged unions, and pointers to any of these. A field of a type the language has no value for — an allocator, a function — is an error only if a program reads it, so a context can be a program's real state. Pass {} for no context.

Decoding follows ZON: a field left out takes its default or is an error, an unknown field is an error, integers are range-checked, and an int is accepted for a float. The result lives in its own arena, strings included, and outlives the program and the context; deinit frees it.

Host functions are called like built-ins:

fn env(call: *struthio.Call, args: []const struthio.Value) struthio.EvalError!struthio.Value {
    if (args[0] != .string) return call.fail("@env needs a string", .{});
    const map: *const Env = @ptrCast(@alignCast(call.context.?));
    return if (map.get(args[0].string)) |v| .{ .string = v } else .null;
}

const functions = [_]struthio.Function{
    .{ .name = "env", .min_args = 1, .max_args = 1, .context = &my_env, .call = env },
};
var program: struthio.Program = try .parse(gpa, source, .{ .functions = &functions }, &diag);

Names and argument counts are checked when parsing. A host function that needs the context asks for it with call.ctx().

Which fields were read. Program.evalReads and evalSourceReads take one more argument, a *struthio.ContextReads(Context) — a std.EnumSet of the context's top-level fields — and fill it in with the fields the program read, whether it succeeded or failed:

var reads: struthio.ContextReads(Context) = .empty;
const config = try struthio.evalSourceReads(Config, gpa, source, &ctx, .{}, &reads, &diag);
if (!reads.contains(.foreground)) {
    // No other value of ctx.foreground can change the result, as long as
    // the other fields stay as they are.
}

A field counts only if evaluating the program read it: in the branch of an if or a switch that was taken, in an included file, or through a let that was used. A let nothing uses reads nothing, even one that failed. Reading ctx other than as ctx.name — let c = ctx;, @string(ctx) — reads every field, and so does a host function calling call.ctx().

Filenames are file: URIs. Options.filename names the program's own file, so that diagnostics name it and relative includes have something to be resolved against. It must be a file: URI with an empty host or localhost — a bare path, another scheme or another machine fails the parse with error.InvalidFilename. zig-uri, which struthio uses for this and re-exports as struthio.uri, makes one from a path:

const main = try struthio.uri.Uri.fromPath(gpa, "/etc/app/main.struthio", .{});
defer main.deinit();
const filename = try main.toText(gpa, .{}); // file:///etc/app/main.struthio

Including files needs a reader from the host, Options.include. It is always handed a file: URI in one spelling — file:///path, with an empty host, no query or fragment, and . and .. resolved — and returns the file's contents or an error; allocating them from the arena it is given ties them to the program's lifetime. Where the files come from, and which paths to refuse, is up to it:

fn read(context: ?*anyopaque, arena: std.mem.Allocator, file_uri: []const u8) anyerror![]const u8 {
    const io: *const std.Io = @ptrCast(@alignCast(context.?));
    const u = try struthio.uri.Uri.parse(arena, file_uri);
    const path = try u.toPath(arena, .{});
    return std.Io.Dir.cwd().readFileAlloc(io.*, path, arena, .limited(1 << 20));
}

var program: struthio.Program = try .parse(gpa, source, .{
    .filename = filename,
    .include = .{ .context = @constCast(&io), .read = read },
}, &diag);

A file that cannot be read fails the parse with error.Include, and the diagnostic names its URI and the error.

Limits (Options.limits) bound expression nesting, the length of any string or list, the number of values comparison and decoding may visit, and the number of different files a program may include. The defaults are generous for configuration; tighten them for programs from less trusted hands.

Formatting #

struthio fmt lays programs out the way zig fmt lays out Zig, so that a struthio file sitting beside Zig looks like it belongs there:

$ struthio fmt config.struthio conf/   # rewrite in place; names what changed
$ struthio fmt --check .               # names what is not formatted, and fails
$ struthio fmt --stdin < a.struthio    # formatted to standard output

A directory is searched for *.struthio files, leaving out anything whose name starts with .. A file is replaced all at once, keeping its permissions, and only if formatting changed it. A program that does not parse is reported as file:line:column: error: message and left alone.

The rules are Zig's:

  • Four spaces of indent, single spaces around binary operators and after commas, and none inside parentheses or brackets.

  • The trailing comma decides. A struct or list literal, a call's arguments, and a switch's prongs go on one line — .{ .a = 1, .b = 2 } — unless the last of them is followed by a comma. Then each goes on a line of its own, with a comma after it. A comment inside, or anything inside that cannot be on one line, has the same effect, and so does a trailing comma after a switch prong's cases.

  • A line break written after a binary operator is kept, with what follows indented once more. One written anywhere else is not.

  • A multiline string starts on a line of its own, one indent in, and what follows it starts on the next:

    .motd =
    \\Welcome to ${ctx.hostname}.
,

  • Comments stay where they were: at the end of the line they were on, or on a line of their own. A run of blank lines becomes one.

Strings, numbers and names are written as they were, and so is everything inside a ${ }. Formatting changes only the space between tokens and the trailing commas, never what a program means. The fuzz targets check exactly that: every program the parser accepts formats to one that parses to the same tree and evaluates to the same value, and formatting it again changes nothing.

From Zig it is struthio.format, which needs no context or host functions, and accepts any function name:

var diag: struthio.Diagnostic = .{};
const formatted = try struthio.format(gpa, source, .{}, &diag);
defer gpa.free(formatted);

struthio.outline parses the same way and returns the program as a tree of Spans — a byte range, what kind of thing it is (a literal, a call, a field, a name, and so on), the span it sits inside, and the name it binds or calls, if any — which is what the language server folds, selects and finds names by. What is inside a string's ${ } is outlined too, inside the string's span:

const spans = try struthio.outline(gpa, source, .{}, &diag);
defer gpa.free(spans);

For a program that does not parse yet, struthio.Tokenizer splits source into tokens, as the parser does.

Building and testing #

The development shell has Zig 0.17.0 and the rest of the tooling:

$ nix develop
$ zig build test --summary all         # unit, golden and fuzz-seed tests
$ zig build run -- fmt --check .       # the struthio command, built and run
$ zig build run -- lsp                 # the language server, on stdio
$ zig build fuzz --fuzz                # Zig's coverage-guided fuzzer
$ zig build fuzz-run -- --seconds 300  # a reproducible fuzzing loop
$ zig build docs-serve                 # read the API docs at localhost:8000
$ nix build                            # the package, tests run in the sandbox

Its one dependency is zig-uri, with z46 beneath it. zig build fetches them; the Nix package is handed them through build.zig.zon.nix, which zon2nix --17 --nix=build.zig.zon.nix build.zig.zon regenerates whenever build.zig.zon changes.

tests/golden/ holds whole programs beside the ZON they must evaluate to (.zon) or the diagnostic they must fail with (.err). The ones that evaluate are also checked to be formatted already.

Editor support #

struthio lsp is a language server: an editor starts it and talks to it with the Language Server Protocol over its standard input and output. It reports what is wrong with each open program as you type, formats a program as struthio fmt does, folds it, and grows a selection outward one expression at a time. It knows where each name is bound, so it can say what a name is when you hover over it, jump to where it is bound, find every place it is used, rename it, complete names as you type them, and warn about a let that nothing uses.

The diagnostics come from the real parser, so they cover more than the grammar: a name used but never bound, a name bound twice, a file that cannot be included. An @include is read from the editor if that file is open there, and from disk if not, and every open program is checked again whenever any of them changes. A mistake inside an included file is reported on the first line of the program that includes it, naming the file, line and column.

Folding covers each struct or list literal, function call, switch and multiline string that spans more than one line, and each run of two or more comment lines. A fold ends on the line before the closing bracket, so the bracket stays visible. Growing a selection goes from a name, to the expression it is in, to the field, to the literal, and so on out to the whole program. Both need the program to parse; while it does not, only comments fold and a selection stays where it is.

Hovering over a name bound by let shows the let, with the comment on the lines just above it, so a comment written there documents the name. Hovering over a |capture| says what it holds, over a built-in function says what it does, and over ctx, inf or nan says what they are.

Going to the definition of a name jumps to the let or |capture| that binds it, and going to the definition of an @include opens the file it includes. Highlights and references find every place a name is written, or every call of a function. Renaming a name bound by let or |capture| changes every place it is written, and is refused if the new name is already in use where the old one is seen, since a name cannot be bound twice; a new name that is not an identifier, or is a keyword, is written as @"...". The outline of a program lists its lets and, nested inside them, the fields of their struct literals. Names inside a string's ${ } count like any other. All of these need the program to parse.

A let that nothing uses is a warning, which an editor may show by fading it, with a quick fix that removes it. An unknown name gets a quick fix for each name in scope that is spelled almost the same — hsot for host — or for ctx.

Completion does not, since a program being typed seldom parses. After @ it offers the built-in functions, with what each does, and the host functions called anywhere in the open programs. Elsewhere it offers the let names and captures in scope where the cursor is, ctx, the constants and the keywords. It offers nothing in a comment, in a string's text, or after a ., where a field's name goes. Inside a built-in function's parentheses, signature help shows its parameters, with the one being typed marked.

Two things it cannot know, and so does not check:

  • The host's functions. Which @names a program may call depends on the program that runs it, so every @name in a file is taken to be a function taking any number of arguments. A misspelled function name is not caught.
  • What a program evaluates to. That depends on the context it is given, so nothing is evaluated: a type error such as 1 + "a" shows up only when the program runs. For the same reason the fields of ctx are not known, and nothing is completed after ctx..

tree-sitter-struthio/ is a tree-sitter grammar for struthio programs, files ending .struthio, with queries for highlighting (highlights.scm), scopes (locals.scm) and folding (folds.scm). It parses everything the real parser accepts to the same shape, and is a little more lenient than it about meaning — an if without else anywhere, a name that is not defined — so that a program being edited still has a tree.

The generated parser is committed under tree-sitter-struthio/src/, which is where editors that build grammars from a repository look for it. Nix builds it into a shared library with the queries beside it:

$ nix build .#tree-sitter-struthio
$ ls result
parser  queries  tree-sitter.json

Neovim #

vim.filetype.add({ extension = { struthio = "struthio" } })
vim.treesitter.language.add("struthio", { path = "/path/to/result/parser" })
-- and result/queries/*.scm copied to queries/struthio/ on the runtimepath

-- The language server. Neovim 0.11 or later.
vim.lsp.config("struthio", {
    cmd = { "struthio", "lsp" },
    filetypes = { "struthio" },
    root_markers = { ".git" },
})
vim.lsp.enable("struthio")

-- Folding from the language server.
vim.api.nvim_create_autocmd("FileType", {
    pattern = "struthio",
    callback = function()
        vim.wo.foldmethod = "expr"
        vim.wo.foldexpr = "v:lua.vim.lsp.foldexpr()"
    end,
})

Neovim's own mappings then reach most of it: K hovers, CTRL-] goes to a definition, grr lists references, grn renames, gra offers the quick fixes, gO lists the outline, CTRL-X CTRL-O completes, CTRL-S in insert mode shows a signature, and vim.lsp.buf.format() formats. Neovim 0.12 also uses the server's selection ranges: in visual mode, an grows the selection one level and in shrinks it. Completion as you type, and highlighting the name under the cursor, are for you to turn on:

vim.api.nvim_create_autocmd("LspAttach", {
    callback = function(args)
        local client = vim.lsp.get_client_by_id(args.data.client_id)
        if client.name ~= "struthio" then return end
        vim.lsp.completion.enable(true, client.id, args.buf, { autotrigger = true })
        vim.api.nvim_create_autocmd({ "CursorHold", "CursorHoldI" }, {
            buffer = args.buf,
            callback = vim.lsp.buf.document_highlight,
        })
        vim.api.nvim_create_autocmd("CursorMoved", {
            buffer = args.buf,
            callback = vim.lsp.buf.clear_references,
        })
    end,
})

Without the language server, formatprg does the formatting instead. gggqG then formats the whole buffer; it has to be the whole buffer, since a few lines of a program are not a program:

vim.api.nvim_create_autocmd("FileType", {
    pattern = "struthio",
    callback = function() vim.bo.formatprg = "struthio fmt --stdin" end,
})

Helix #

Helix (25.07 or later) wants the language, where its grammar comes from, and the queries. The first two go in ~/.config/helix/languages.toml:

[[language]]
name = "struthio"
scope = "source.struthio"
file-types = ["struthio"]
comment-tokens = ["//"]
indent = { tab-width = 4, unit = "    " }
language-servers = ["struthio"]
auto-format = true

[language-server.struthio]
command = "struthio"
args = ["lsp"]

[[grammar]]
name = "struthio"
source = { git = "https://git.jcollie.dev/jeff/struthio.git", rev = "<commit>", subpath = "tree-sitter-struthio" }

rev is the commit to build the grammar from; for a checkout of your own, source = { path = "/path/to/struthio/tree-sitter-struthio" } takes its place. The language server needs struthio on the PATH. It formats on save, and :format formats by hand. To format without it, replace language-servers with formatter = { command = "struthio", args = ["fmt", "--stdin"] }. Helix completes and shows signatures as you type, hovers with space k, goes to a definition with gd, lists references with gr, renames with space r, offers the quick fixes with space a, lists the outline with space s, and selects every place the name under the cursor is written with space h. It grows a selection (Alt-o) by its tree-sitter grammar rather than the language server, and does not fold.

The queries are not part of the grammar as far as Helix is concerned. They live in its runtime directory under the language's name, and they are the ones in queries/helix/, not queries/:

$ mkdir -p ~/.config/helix/runtime/queries/struthio
$ cp tree-sitter-struthio/queries/helix/*.scm ~/.config/helix/runtime/queries/struthio/
$ mkdir -p ~/.config/helix/runtime/grammars
$ hx --grammar fetch     # only for a `git` source
$ hx --grammar build
$ hx --health struthio
...
Tree-sitter parser: ✓
Highlight queries: ✓

queries/helix/ holds the same patterns as queries/ with the captures renamed to Helix's scopes. A Helix theme has no number, boolean or property, for instance, and colors those as constant.numeric, constant.builtin.boolean and variable.other.member instead. A capture with a name the theme does not know is left uncolored.

Three things that trip this up:

  • hx --grammar build shells out to cc, which a NixOS system has no reason to provide. Run it from nix develop in a struthio checkout, which has a compiler. Alternatively, skip the build entirely: copy the parser Nix built to where Helix would have put it, cp result/parser ~/.config/helix/runtime/grammars/struthio.so.
  • runtime/grammars/ has to exist. Helix links the parser into it and will not create it.
  • It fetches and builds every configured grammar, not just this one, unless use-grammars = { only = ["struthio"] } is at the top of languages.toml. Failures for the bundled grammars are unrelated; look for the struthio line.

Working on the grammar #

In the development shell:

$ cd tree-sitter-struthio
$ tree-sitter generate   # after changing grammar.js; commit what it writes to src/
$ ./helix-queries.sh     # after changing queries/; commit what it writes to queries/helix/
$ tree-sitter test       # the corpus in test/corpus/ and the highlighting test
$ nix flake check        # all of that, and that src/ and queries/helix/ are up to date

The highlighting queries follow the convention that when several patterns match a node the last one wins. Neovim, Helix since 25.07, and the tree-sitter command line all work that way.

Where this lives #

The canonical repository is on my Forgejo instance, and Tangled carries a mirror of it:

git clone https://git.jcollie.dev/jeff/struthio.git

It is also on Radicle as rad:z3X2SbfjxqgqQMcbe33XhyzFUe1xC, which is the only name that finds it there — a peer-to-peer repository has no host to browse:

rad clone rad:z3X2SbfjxqgqQMcbe33XhyzFUe1xC

All three serve the same history. CI and the published documentation come from the Forgejo repository.

Report issues and send pull requests to the Tangled mirror. See CONTRIBUTING.md for how to set up, test and submit a change.

License #

MIT; see LICENSES/MIT.txt. The project follows REUSE.

References cited #