A local Git merge queue with content-bound gate verdicts

Fork

ocaml-merge-queue doc configuration.md

74 kB · 1187 lines

Markdown

at main

Thomas Gazagnaire · merge-queue: Work.S.wakes and Beside's woken go: no work counts the run's wake file · 2d ago

844ee938

Configuration #

The rules are one file, mq.toml, on a branch of its own, mq/config. It is a branch like any other: a rule changes by a land into it.

mq/config is an orphan branch of the repository, as gh-pages holds a site beside the code it documents. Its name is the ref refs/heads/mq/config, which reserves the prefix: no branch can then be called plain mq, since Git cannot hold both refs. On the first machine, mq init creates it, with the branch HEAD named as the whole DAG. It adds one check when it recognises how the project already checks itself, and lists it on that branch. That is the repository's own pre-commit hook when it has one. Otherwise it is the build system's own check (dune build @check for a Dune project, cargo check for Cargo, go vet ./... for Go, make check for a Makefile with that target). Otherwise it adds none. In a repository with a remote it adds [sync] too (Sync). A DAG with no check is a valid configuration, and each land then states it in its outcome (no checks configured) as a fact. On every later machine, mq init after git clone finds origin/mq/config and adopts it rather than creating a second one, and sets up each branch the rules name that the clone lacks from origin's, tracking it, as git switch staging does in a clone. It adopts rules that do not load too, and ends unavailable with next: mq doctor, whose remedy lands the fix. A machine takes a later value of origin/mq/config only by fast-forward, as git merge --ff-only does, honouring merge.verifySignatures: a copy that does not hold the value it last saw there, a rewind or a read that lags a push, moves nothing and is named, and the next push sends the newer rules back; a divergence is refused, naming it. In a repository where mq has run, the rules branch deleted with Git is not a queue mq init never set up. The journal records the landings: each note on a landed tree names the commit of the rules that judged it (rules=), and a landing into the rules branch names the commit it moved them to. Every command, mq init among them, is then refused with the newest of these, the rules last read here, and next: git branch mq/config COMMIT, the command that restores them; mq init creates no second set over them. A repair of rules that do not load is judged under them, so the commit restored is the repair's own, never the one that broke them. The first mq push creates on the remote whatever branches of the DAG it does not hold yet. One copy describes every branch:

[platforms]
supported = ["Darwin arm64", "Linux x86_64"]

[check.build]
command = "dune build @check"
read = ["~/.opam"]
write = ["~/.cache/dune"]
[check.test]
command = "dune runtest"
read = ["~/.opam"]
[check.suite]
command = "./scripts/suite.sh"
network = true
wall = 1800
platforms = ["Darwin arm64", "Linux x86_64"]

[branch.staging]
checks = ["pre-commit", "build", "test"]

[branch.main]
takes = "staging"
checks = ["pre-commit", "pre-push", "build", "test", "suite"]

[branch."release/*"]
checks = ["build", "test", "suite"]

[bounds]
check_wall = 3600
check_grace = 30
wait_wall = 7200
run_start = 300
hook_wall = 30
resolve_wall = 600
remote_bytes = 17179869184
remote_stall = 60
store_log = 67108864
max_batch = 128
trace_wall = 2
slot_overdraft = 1
check_memory = 4294967296

[resolve]
command = "mq resolve"
default = "stop"
models = ["deepseek-v4p1-flash"]
max_tokens = 32768
read = ["~/.config/model"]
network = true

[hook]
command = "./scripts/tracker.sh"

[cone]
command = "mq ocaml cone"

[lock]
command = "dune pkg lock"
paths = ["dune.lock"]

[generated]
files = ["*.opam"]
command = "dune build @install"

The [cone] section names the build adapter's answer to what a queued land's change can affect. A run asks it for a queued tip, from the repository's top, with MQ_TARGET, MQ_TARGET_TIP, MQ_LAND_TIP (the queued tip) and MQ_LAND_SINCE (the tip it was queued from: the land's own commits are those after it, whatever a rebase of the target replaced under them) set, bounded by bounds.hook_wall. Its first line is whole, or the directories the change can affect, separated by spaces. A second line reads P1 P2 ... names, as gitignore patterns, the paths of the target's tree the answer was read from; once the target moves, the run asks again only when the move changes a path one of them matches or the queued tip's merge base with the target, and with no such line at every move. The run also sets MQ_LAND_TIPS: the asked tip, then every other tip queued into the target with no answer that holds on the target's tip and none the call before is still keeping, each as TIP:SINCE, separated by spaces. A command that answers them all prints, for each, a line tip TIP and its cone's line, as it answers it, then the one reads line; the run keeps every answer printed, even by a call stopped at its bound, so one call answers the queue. A command that prints one answer answers MQ_LAND_TIP alone, as before. Each ask is a task, mq task cone, which keeps each answer under the repository's mq/cone-answers directory, named by the command, the queued tip and the tip it was queued from, so a run started again, as an install of mq starts one, asks no cone it was answered, and a land dequeued and queued again at the same tip asks none either. An answer no queued land keys is removed once nothing has written it for a week. Lands into one target whose cones are disjoint are checked at once. With no [cone], a land is the whole tree, and lands into one target are checked one at a time. bounds.hook_wall is the deadline of one call of the command: an ask past it, or a command that fails, keeps the whole tree for that tip, and no run waits for an answer (runs).

The rule a cone answers is set overlap. A land writes the paths its commits change; its check reads everything the build of its scopes reads. Two lands conflict when one's writes meet the other's reads, and a cone is the land's writes together with every directory whose build reads them, so two lands whose cones are disjoint write nothing the other reads.

mq ocaml cone takes the change from the queued tip's merge base with the target's tip, the commits the land replays, and reads the dune graph from the target's tree: the change's packages and every package whose stanzas use one of their libraries. A dune file the change touches adds the packages of the libraries its stanzas name at the queued tip, since a use the land adds is not in the target's graph and the check builds it. Its second line reads dune, dune-project, dune-workspace and *.opam, the graph's files. The answer is kept under the repository's mq/cone directory, keyed by the target's dune and opam files and the land's change, never by the executable, so a target move that leaves the graph alone moves no checkout and reads no index, and an install of mq answers nothing again. Asks on one target tip share one reading of its graph: the tip's graph files are read from Git's objects, and the project index keeps each top-level directory's part by the object ids of the graph files under it. The checkout a part not yet read is scanned from is a worktree of the target's tip under mq/cone-target holding the graph's files alone, made the tip's once and kept while every directory and graph file it read stats the same. One call answers every land the run names (MQ_LAND_TIPS) on that checkout. The rules of the graph's dune files are read once a graph and kept beside the project index's parts, keyed by the graph files' paths and object ids, so a call for a land queued after the others reads no dune file; each land's project index is the one an ask of it alone builds, its roots the directories of its change. An ask after the first lists no directory of the tree but those of the packages no ask on the tip has scanned.

The edges' checks. A land along an edge is a batch: every commit the branch taken holds and the target lacks, each of them checked already by its own land, or its own batch, into that branch. The edge's checks run on the batch's tree as any land's do, and a check that reads its change, as mq ocaml does, reads the batch's paths. A land into the entry branch runs the tests of the packages it touches and of those that read one of their libraries directly, mq ocaml suite --direct: a library most of the tree reaches costs its land its own readers, not the tree, and a red in the land's own package is found at the land, never by the edge. Past its check's wall the suite goes on while its build ends processes, as every check does ("A check", below): it runs to its verdict, so no test of the land's scope is left to a serve no land waits on (Delivery at full speed). The edge's suite runs the batch's whole cone, so it confirms its lands' tests and adds the packages that reach a change through another. A cone is a function of the paths a change writes, so the batch's is the union of its commits' cones: a batch that touches one package of fifty runs that package's tests and its readers', and a batch that touches a library half the tree reads runs it and that half. Google's TAP runs a change's affected tests and no others, and Bazel's target determination reruns only a target whose action keys changed. A root file that every rule reads, a workspace's env or a project's version, is the whole tree; a root file whose change no rule reads, dune's cache setting or a package's opam metadata, is no scope at all (mq ocaml's build_neutral). Main's lint compares the findings of the batch's touched files with the target's, as staging's lands did. The bisection of a red edge probes each prefix under the same checks, so a probe reads the prefix's cone, never more than the batch's.

What no cone sees, a test reading a file no rule declares or a tool the build takes from the machine, the whole tree sees. A branch that takes the checked branch, under checks that read every rule whatever changed, keeps that guarantee:

[check.suite-whole]
command = "mq ocaml suite --whole"
wall = 14400

[check.lint-whole]
command = "mq ocaml lint --whole"
wall = 14400

[branch.full]
takes = "main"
report = true
checks = ["suite-whole", "lint-whole"]

Its land moves only full, to main's tip when that tree passed, as Chromium's last known good revision moves. No land reads it and no edge takes it, so a red there holds no work back: the red is an event the hook delivers, charged to main's tip and naming the red check and its log. A red serve into a branch that reports is not bisected, since each probe would be one more run of checks that read the whole tree; the edge waits until main moves. A branch that reports is served whenever main holds commits full lacks and no serve of it is under way, whatever else waits. Its serve is of the reporting class, which is guaranteed one build slot: when it holds none, the next slot freed is its own, before any land's, and it takes more only when no fix, land or prediction waits for one (runs, "The work classes"). Each of its checks runs to its verdict. The whole-tree checks cost the queue one slot under load and never wait for it to idle. A serve queued into a branch that reports checks the tip it was queued at, through the end of a run and an install, so a main that moves faster than one whole-tree run still has its tips checked, one serve behind; the next serve takes main's newest tip. Its checks carry their own wall, since one of them reads the tree whole. report = true says the branch only reports; a branch that reports takes from another, and waits for no dependency's edge: it reads what its source locks, which the dependency's branch paired with its source holds already (commands, "The edges").

A [check.NAME] section names the command that runs the check in the tree being checked; it passes when the command exits 0. A command that exits 75, EX_TEMPFAIL (sysexits.h), says the machine could not run it, its first words on standard error saying why: that says nothing of the tree, so the check is not run again at once, no bisection names a culprit by it, and its land stays queued for a later run, which counts it as an attempt on the tree, as a check that could not start is counted (runs, "A check the machine stopped"): nothing the next round reads differs. A check whose supervisor another process stops with TERM or INT, an operator's stop or the stop of a run that read the command as left (runs), is a task that keeps no answer: its group is stopped, its log names that signal and the process that sent it, and the run that reads it starts the check again, an attempt it neither counts nor waits on. A command that exits 79 and names at least one failure (What a red names) says it fails only as the target's tip does, every failure it names being the tip's: the run is inherited, and its land goes ahead, not charged, since it makes the target no worse (outcomes, landed). A command that exits 80 says it is red and that its errors name no scope of the change in play, its check tree or the machine around it at fault, its last line the first error, as mq ocaml says it (commands): the run is no result on the tree, the check runs once more on its check worktree's build directory set aside, and a second such run stalls the check and charges no entry (runs, "A red outside the change"). 79 and 80 lie outside every convention a command exits by for another reason: sysexits.h's 64 to 78, Cmdliner's 123 to 125 (125 is also mq's own defect), the shell's 126 and 127 and the signals' 128 and above. A run that exits 79 naming no failure is failed, as any other status is. Its wall, bounds.check_wall by default, is given to its command as MQ_DEADLINE and stops no run: a run stops once it makes no progress for bounds.stall seconds, the slot it took its first progress. A run whose processes ran within the last bounds.stall seconds (one of its group or their descendants with no child under way consumed CPU time for at least a hundredth of the time between two readings, or one gained the time of a child it waited for, as the kernel counts them), or one of whose build processes ended within them, as dune's trace records it (a rule's action or a test), is making progress and is never stuck; one that made none that long is sent TERM, then KILL bounds.check_grace seconds later, and ends stuck (runs, "A check the machine stopped"). Its priority, "background" by default, runs the command and every process it starts 19 nice steps below the run, as nice -n 19 runs a program, so a check yields the cores to interactive work, mq status among it, and takes them back as soon as that work waits: a check is never held off a core that no other work wants. "normal" runs it at the run's own priority. mq's own commands keep their caller's priority. A system that refuses the change, as a sandbox may, runs the check at the run's priority, and the check's log ends with mq: the check ran at the run's priority: and why. Its repair, a command line, repairs the check's confirmed red on an entry's own land before that red is the entry's verdict: its commit is a candidate the land checks, and once the land's checks pass on it the branch's next, the entry queued again at it (runs, "A repair"); with none, the red is the verdict. It is a red bump's first repairer too, before the model's (Commands, "When a dependency moves"). A check runs in a worktree of the tree under check, HEAD detached at the replayed commit, with an environment built from a fixed list and never inherited from the caller: PATH, HOME, LANG, MQ_CHECK (the check's name), MQ_REPOSITORY (the repository's common Git directory, whose run lock a mq land inside the check would wait for: such a land into that repository is refused, and into any other proceeds), MQ_SOURCE (the branch landed: the work branch, or for the serve of an edge the branch the target takes), MQ_TARGET (the branch landed into), MQ_TARGET_TIP (the target's commit the tree was replayed onto), MQ_SCOPE_TIP (set only on a probe of the target's own tip, whose HEAD is MQ_TARGET_TIP: by a red edge's bisection, the range's head; by a land red under its check's inputs, the land's candidate. A check scoped by its change builds that commit's change from MQ_TARGET_TIP on the tip's tree; its result is stored under that commit too, so it is never read as the tip's own), MQ_DEADLINE (its wall as an RFC 3339 time), MQ_STALL (bounds.stall, the seconds mq lets the run make no progress past MQ_DEADLINE before it stops it, so a wait of the check's own, for a lock another check holds, lasts until the later of the two: mq ocaml waits for the target's verdict that long), MQ_JOBS (the jobs its builds run at: its share of the cores, the machine's cores times the slots the check may use when it starts, its own and those free, over mq.slots, so that a check alone takes all the cores and the checks side by side share them, over the share of a core one of its processes used, the CPU of the processes of the check's runs over the last day over their wall, as their build figures sum them; the share itself when no run of the check left build figures; at least 1. A check whose tests wait on processes, Git and the disk runs more of them at once than one that compiles, and either spends its share; mq ocaml passes it to dune as -j), MQ_TRACE_DIR, a directory of the run's own where it leaves the trace of each build it runs, and MQ_TRACE_REPORT, the file beside its log an mq command it runs writes its span tree to (trace_wall below). When the check ends, mq reads each trace, copies it beside the check log as <log>.trace-<name>, writes the rules the builds ran and the tests the shared cache restored, then each test they ran with its time, one JSON object a line, to the run's trace beside the log (check-...-<invocation>.trace, kept as long as the log), reports the measured build cost in the log, then removes the check worktree's trace directory. The run's check event records what mq stats reads of it: its tests' times as a histogram, its tests of ten seconds or more, and the rules it ran although an earlier run that had ended before its command started recorded their digests, found against the traces of the runs the journal holds, which mq reads once a process and then as each run ends. A check with inputs is also given, for each, MQ_INPUT_NAME, the commit its result is keyed by (empty when it has none), and MQ_REPO_NAME, the absolute path of its repository, NAME being the input's name in upper case. Dune's _build/trace.csexp is one, recorded with DUNE_TRACE=+cache for the shared cache's answers. The checks of a land run side by side, up to mq.slots at once, each in a worktree of its own taken from the pool, its build directory kept from that check's last use: each finds the replayed commit's files and nothing a run before it wrote, but what the build directory holds, as git clean -dx would leave it but for that directory. The rerun of a failure runs in the same worktree and also carries MQ_CHECK_RERUN, a key no run carried before: a build system answers a second run of a tree from its record of the first, a recorded failure included, and an action keyed on the variable executes again instead (dune's cram pages, whose recorded output a rerun would replay, declare it with (deps (env_var MQ_CHECK_RERUN)); dune records no failed action, so a failed test executes again without it, and a test that passed is answered from the build directory; in each phase, @check as @runtest, mq ocaml builds again only the build that failed, keyed as the first run was: a staged suite (mq ocaml suite --direct) builds its stages again in their order, the ones before the red stage, which passed, answered from the build directory, and the ones after it only once the red stage passes on its rerun, as the first run would have built them, so a red is known once the red stage alone is built again, never by the whole suite. In that build it sets the variable for one build, the aliases of the cram pages that failed, and before it, unless its first run recorded failed pages alone, builds the stage again without the variable, finding there the pages that failed when no record of the first run is left, so no page that passed executes again, and names on standard error each failure the rerun does not repeat, N flaked and passed on its rerun: NAME, ...). A rerun stopped before it has built its red stage again leaves the first run's failures standing, since no rerun has passed them. A first run carries none, so the build directory and the build cache answer it, a green as a red: dune stores a page's transcript whatever it says, so a red one execution left there, of this tree or of another with the same inputs, is answered to every later build that reads them. A red answered so is no observation of this run, so mq ocaml executes again, under a key no build carried before, each failed cram page that its first run's dune trace does not record running, and that execution is the first run's verdict on it; a page the build ran keeps its red for the rerun, and a build whose trace does not record dune's exit leaves every failure standing. The run after a pass that a repair's candidate asks (runs, "A repair") carries MQ_CHECK_RERUN and MQ_CHECK_AGAIN, the files the repair rewrote that the red's log named, separated by spaces: with no failure of its first run recorded, mq ocaml builds the cram pages among them under the variable, which executes each again, and the rest of the phase keyed as the first run was, which the build directory answers. A rule whose verdict reads commit messages, authors or trailers is not a check, since a check runs on a tree and its result is reused for every commit of that tree: such a rule belongs in the hook's queue-time refusal. takes makes an edge of the DAG, served as its source moves: a scheduled run serves it on the landing that moves the source, once no serve of it is under way, and serves it again at the end of one whose source moved meanwhile (runs). No key sets an interval: a rule is a condition on the queue's state, and the build slots bound what the serves cost. A release series has checks and no edge (Review and releases). read, write and network declare what a check needs beyond its worktree, as a path relative to it, under ~/ or absolute. inputs declares the other repositories a check's verdict reads, a table from a name ([a-z][a-z0-9_]*) to the repository's path, relative to the repository's top, under ~/ or absolute: the commit read of each is part of the key its result is stored under, so a move of any of them runs the check again (A check that reads other repositories).

vet lists checks among a branch's checks that each entry into it runs on its replay onto the branch's tip, beside the batches, so that an entry red on its own is kept out of every batch: vet = ["fmt", "lint-touched"] under [branch.staging] checks each entry's own commits, replayed onto staging's tip as the run found it, with those two checks, in the build slots no land holds, and its results are kept by that tree, so a land onto a tip that did not move reuses them. No entry waits for a vet: one whose vet is out rides as any other. A red there stops the batch that holds the entry, and goes to the repairs at the entry's land (runs, "A repair"). mq land runs them in the worktree that has the branch out before it queues, and a red there queues nothing (mq land; --force skips it). A name that is not one of the section's checks is refused, naming it. With no vet, the default, nothing is vetted. A target with batches off (bounds.max_batch = 1) vets nothing (A vet).

after lists checks among a branch's checks that its lands leave to the edges that take it: after = ["suite-touched"] under [branch.staging] lands each entry once its other checks pass, and each serve of main, which takes staging, runs suite-touched on the batch of every commit staging holds and main lacks. A land into the branch passes its checks less its after, and a serve along an edge out of it passes the target's checks, then each check of the source's after the target does not list, so the target holds only trees those checks passed and the source never waits for them. A red serve names its culprits by what its log names, and each culprit's repair lands into the source expedited (A check left to the edge). A name that is not one of the section's checks or that its vet lists is refused, naming it, and so is an after on a branch that only branches that report take, or none. With no after, the default, every check runs before the land.

drain lists checks among a branch's checks that a drain's lane runs before it moves the branch (A drain): drain = ["check"] under [branch.staging] builds the chain of every queued entry once; the branch's other checks run after, where the branches that take it run theirs, a test red at a tip repaired forward. A chain that changes the package the [install] executable is built in runs every check of checks whatever drain names, so a change to mq meets its own suite before the branch it installs from moves. A name that is not one of the section's checks is refused, naming it. With no drain, the default, a drain's lane runs the checks every land into the branch runs.

[platforms] supported names, once, the platforms the repository supports, each as uname -sm prints it ("Darwin arm64", "Linux x86_64"): the columns of mq checks, and the platforms whose results a gate counts. With no [platforms] section every platform is supported. A check is portable unless its section names platforms: its verdict cannot depend on the platform, so one confirmed pass on any supported platform passes it for every machine, and a result on a platform the rules do not support counts for nothing. platforms lists the platforms a check's verdict must hold on, each one of supported: a land passes it only once each of them has a confirmed pass of the tree under the check's definition, whichever machine recorded it, and a result on another platform is not required. The platform is part of what a check requires and never of the key its result is stored under (Check results), so listing it or removing it reruns nothing. A platforms that names a platform supported lacks, a platforms with no [platforms] section, an empty list, a platform named twice and a name that is not uname -sm's two words are refused when the rules are read, naming the check and the platform.

A confirmed red on any supported platform holds a portable check, even beside a pass on another: the check is then not portable, and the outcome says so, naming the two results with their platforms and machines (check suite is not portable: passed on Darwin arm64 (laptop), failed on Linux x86_64 (build-host)), so that the check's section lists its platforms or the check is fixed. A land on a machine whose platform a check lists runs the check when its own platform has no result; a land on any machine whose check still needs a pass on a platform it is not ends unavailable, naming the platforms still missing, since another machine's result decides it and nothing in the change does. A check runs with the run's rights, as a Git hook does (What mq trusts), and only a machine whose mq.confine is auto enforces them (How a check is confined). [resolve] takes the same three keys for the resolver, which such a machine confines as it confines a check. The bounds are in seconds of the host awake: a sleep of the host moves every bound a run holds later by its length (design, "A sleep of the host is an event"). MQ_CLOCK names a file holding a count of seconds, as a harness that plays the clock sets it: an mq given it reads every instant, and counts every bound and wait, that many seconds ahead of the host, reading the file again at each reading of the clock and, while it waits, at each write to it the kernel reports, never on a tick, so a waiting mq consumes no CPU. A larger count is that much time passing, so a bound it passes ends at once in every mq that reads the file; a missing file counts 0, a count smaller than one read before changes nothing, and a file that holds no count ends the command, naming it. check_wall, check_grace, wait_wall, run_start and hook_wall default to the values shown; wait_wall bounds a caller's wait for a lock, mq slot's wait and a dependency's preparation; a check's wait for a build slot has no bound of its own. Every deadline is the deadline of one external call (a check's wall, the cone command's, the resolver's, a hook's): a round is bounded by what it does, so there is no silence bound and no count of passes (design). remote_bytes, in bytes, bounds what one session with a remote reads, its listing of branches, the pack a pull fetches and the status a push reads back, 16 GiB by default, with the objects decoded to at most eight times that: a remote that sends more is refused, naming the bound, and nothing moves. remote_stall, in seconds, 60 by default, bounds how long a session with a remote waits on a remote that makes no progress, where the user's own configuration states no bound. Over SSH, as OpenSSH's client reads ssh_config(5): ConnectTimeout bounds the connection and the server's identification string, and a server silent for ServerAliveInterval is sent a keepalive, the session ending once more than ServerAliveCountMax went unanswered; where ssh_config sets no ConnectTimeout it is remote_stall, and where it sets no positive ServerAliveInterval the interval is remote_stall divided by one more than the count (3, OpenSSH's default, unless ssh_config sets it), so a server that stops answering ends the session remote_stall after it last spoke; and since a server whose command hangs still answers keepalives, the channel itself ends once it moves under 1 byte a second over remote_stall, as an HTTP transfer does. Over HTTP and HTTPS, as curl's low-speed check ends a transfer: a transfer under http.lowSpeedLimit bytes a second for http.lowSpeedTime seconds (git-config(1), or GIT_HTTP_LOW_SPEED_LIMIT and _TIME) ends, and where Git's configuration sets neither the limit is 1 byte a second over remote_stall; the connection is bounded at 10 s. Over git://, the connection's reads and writes end under 1 byte a second over remote_stall. Each low-speed window starts with the session's first bytes, so a connection, a key exchange or an authentication spends none of it. A session that ends so leaves the remote unreached, its operation unavailable and naming the remote and why (commands), and wait_wall still bounds a whole session. store_log, in bytes, bounds the write-ahead log of the journal's index, 64 MiB by default, two of the largest commits mq makes: a commit that leaves the log at 1000 frames copies it back into the index and cuts it to no bytes, and a log past the bound is a copy-back a reader kept from running, which mq doctor reports. check_memory, in bytes, bounds the resident size of each process of a check's group, 4 GiB by default: while the check runs, mq reads its group every half second and sends a process over the bound TERM, then KILL check_grace seconds later, and the check's log names the process, its command, its resident size and the bound. max_batch, a count, bounds the lands one batch checks as one tree, 128 by default: the entries a branch admits at once, their changes commuting, and the entries behind them that overlap no land under way ride in one batch once they are more than the build slots check at once, landed in one move, a red bisected over them (A batch); 1 checks each land alone. A round that starts no task and writes nothing ends the run, and a red tree is not checked again under the same rules, since the results on the candidates' trees are what a round reads. stall, in seconds, 120 by default, is how long a check run past its wall may make no progress before it is stuck (above). slot_overdraft, a count, 1 by default, is the build slots past mq.slots that an expedited land (mq land --expedite) or a land into the rules branch takes when every slot is held by serves, lands and predictions it ranks above, so that it never waits behind work under way; 0 holds every check to mq.slots. Each overdraft journals an overdrawn event naming the slot, and each prediction or probe refused the last free slot, left to a land queued whose checks have not begun while the run says one is, a slot-kept event. A command prints, as it exits, its span tree on standard error when asked, mq status --why, or when it is slow against its own history on this machine: its wall is past trace_wall, 2 by default, and in a band of powers of two two or more above the band of its median wall there, at least twice that median. The machine is loaded at all times, so a command that always takes 5 s there takes them quietly, and one that takes 20 s where it took 5 says where they went. Each command keeps its walls in the repository's mq/walls directory, a count for each band, and one with no history yet prints nothing. The tree is one line a span, indented under the span it opened within, with its milliseconds, its name and what it waited on (a lock and its holder's pid, the index and the bytes of its log, a child and its pid and arguments), under a first line naming the command, its wall and the median it is past. MQ_TRACE_WALL, in seconds, takes the history's place for one process, as a harness whose output must not depend on the machine's speed sets it: the command prints past that wall, 0 every time, and keeps no history. MQ_TRACE_REPORT names a file that takes the span tree in place of standard error: mq sets it for every command a run starts (a check, the resolver, the lock command) to the command's log with the extension .spans, so that an mq command a run starts leaves its span tree beside its log and the reason mq reads from its standard error is the command's own last line. resolve_wall has no default, and the resolver is refused without it. --wait-for is refused when no edge leads from the target to its branch. A top-level entry names the entry branch when more than one named branch takes no other.

Other repositories are no rule either. A repository names the ones it builds with in its own tree, in its lock (The lock), and a submodule's gitlink is Git's record, which mq reads and never lands into (Several repositories). A top-level name is the name the repository's branches take in its dependencies (app/fix-ttl for name = "app"); it is required once a lock declares a dependency, and the rules of two consumers on one machine must not share it.

A check named after a Git hook is that hook. pre-commit or pre-push in a branch's list needs no section: mq runs the same hook script Git would, with no copy and no parallel gate. A section is written for it only to grant it a path or the network, or to set its priority. A branch's list is how hooks are chosen per branch, which Git alone cannot do: staging runs the pre-commit hook, and main adds pre-push. mq finds the script as Git does, in the directory core.hooksPath names or else in hooks/ of the Git directory, and resolves that with ocaml-git, never by running git. An interop test holds the two equal, with git rev-parse --git-path hooks/pre-commit as the oracle. A core.hooksPath inside the work tree (.githooks) resolves against the check worktree, so the hook comes from the tree being checked, like any check. The hook runs with nothing staged, so a hook that reads only staged files has nothing to check and needs a whole-tree mode to be useful here.

A rule is changed like code. mq new looser-lint mq/config adds a worktree that holds mq.toml. Its author edits and commits, and mq land lands it. A land into mq/config runs one check, rules, which mq itself provides, so a [check.rules] section and a branch that lists rules are refused. It passes when the file loads, every branch it names exists, every check it names resolves at the tip of each branch it is required on, and a branch a bump serves whose tree locks dependencies (.mqmodule) has a repairer: a [repair] section, or a [resolve] command with bounds.resolve_wall. Without one, a red bump's repair request would wait for nobody, so the rules check refuses it, naming the branch. A check that resolves and then fails at some tip is that tip's own red, recorded the first time it runs there and charged to no change. git log mq/config is the history of the rules, a team that wants rule changes reviewed sends them through its forge like any branch, and mq status BRANCH prints the rules in force for that branch.

Rules on the rules branch that do not load end every command but mq doctor unavailable, naming mq.toml at its commit and the line, with next: mq doctor, which names the remedy. A land into the rules branch needs no rule of the rules it replaces, as it is never put to the hook: mq new NAME mq/config hands the branch out whatever it holds, and mq land NAME lands the fix under the rules check alone. The doctor's remedy is that land: the mq.toml of the commit the rules last loaded at, the newest of the branch's reflog or else of its history, landed into the rules branch. Every machine that follows the rules takes it as a fast-forward; mq never rewinds the rules branch.

A journal record or a record under refs/mq/ that this mq cannot read, one another mq wrote, is passed over and named by every reader (commands.md, "What a reader passes over"). Nothing is read with a default, and the rules check refuses an mq.toml the landing mq cannot load in full. In the repository that builds mq, each land into the install branch is asked mq doctor --state of the live queue before it lands (install.md, "The rules"), so the mq it installs meets no rule it cannot load and no record it cannot read.

Four properties follow from keeping the rules apart from the trees they judge.

A change never chooses the rules that judge it. A product change cannot touch mq/config, so nothing a change edits alters the rules that judge it.
A broken rule can be fixed. When a check main requires is itself broken, the land that removes it goes into mq/config and passes one check, the validation. Rules kept in the product tree would have to pass the rule they repair.
One copy governs every branch. A release series cut months ago is governed by [branch."release/*"] as it reads today, with its checks run by the scripts of its own tree. A series that needs other rules gets a section of its own, landed like any rule.
A round decides under the rules it read. Each round reads the rules at the rules branch's tip with the rest of its reading, and each result names the rules it was checked under. A rule that lands governs the next round, and since results are stored by tree, check and build configuration, only a check the new rule adds or redefines is run again. A pull that brings new rules sends this machine's unpublished landings back through the queue, checked under the rules the next round reads, which on a machine that follows mq/config are the pulled ones. Published landings are never judged again.

Keeping the rules apart gives one guarantee up. A target's compare-and-swap no longer sees a rule change, since the rules are not in the target's tree. Inside one machine, what keeps a land under one set of rules is the round: every rules read of a round uses the commit of the rules branch its reading took, and a move needs a pass under those rules. Between machines, a rule change is detected only as any concurrent change is: mq push finds the remote's mq/config moved and names mq pull, and a machine learns of a rule another machine landed at its next pull. Nothing judges a published landing again when the rules change: a landing published under rules the team has since changed is accepted, and only the note on its tree, which names the commit of the rules that judged it, says which rules those were.

Zuul's config projects keep the policy apart from the change in the same way, and so does Mergify, which reads the default branch's file. bors read bors.toml from the change being merged, which let a change choose its rules. The rules are a plain branch, where Gerrit keeps a project's configuration on refs/meta/config, outside refs/heads/. A plain git clone fetches no ref kept there, and git branch, git log and a forge's branch list do not show one.

Each machine lands under the rules it chooses. A machine reads the rules from the branch its Git configuration names as mq.rules, mq/config by default; the key chooses the rules and places no state. A teammate can therefore run another DAG: a laptop that lands into main under fast checks, beside a build host whose DAG is staging -> main -> release/* with the edges. A personal DAG is a personal branch (git branch my-rules mq/config, then git config mq.rules my-rules). What a team accepts from whom is the team's workflow, as it is for any push. mq/config is not a branch of the DAG, and nothing lands from it into another branch. mq push and mq pull carry it with the DAG's branches, so machines that follow it share one set of rules, and mq drop refuses it. A machine that writes the agreed copy is best left on the agreed rules, since what it publishes is what its own rules checked. mq advises this and enforces nothing.

On a machine with a scheduler, the run reads the rules branch that mq schedule start recorded, and never the repository's configuration. Every caller there shares one configuration file, and one agent that tries a looser DAG must not change the rules for all the others.

How a check runs (the script, the build command) lives in the product tree and is taken from the tree being checked. A check of tree T must come from T, since a script changes with the code it tests. Which checks run, and which command starts them, come from the rules. What mq trusts says what that leaves open.

The rules also name the resolver ([resolve], Conflicts): its command, its default for a land no terminal typed, its layout, what its worktree holds of the tree: "conflicts", the default, the paths in conflict alone, every other one in the index with the skip-worktree bit, as Git's mergetool reads only the conflict, or "tree", the whole tree, for a command that reads more of it, and what it asks, models, the models asked at once, in order, and max_tokens, the most tokens a model may write in one request, its reasoning included. mq gives the command the last two as MQ_RESOLVE_MODELS, the names separated by spaces, and MQ_RESOLVE_MAX_TOKENS when the rules set them; mq resolve reads them, and without them asks deepseek-v4p1-flash-1017 alone for at most 32768. A models that names no model, or a name with a space, does not load. They may name a repairer the same way, [repair] with a command, which it requires, models and max_tokens, given as MQ_REPAIR_MODELS and MQ_REPAIR_MAX_TOKENS, and the resolver's three grants: what answers a red bump's repair request after the red check's own repair (Commands, "When a dependency moves"), and an entry's own red no check's repair answers (Runs, "A repair"). Without [repair] the resolver is the model's repairer: the [resolve] command, given MQ_RESOLVE_TASK=repair and the repair's brief on its standard input, with its own model, token bound and grants, bounded by the red check's wall and bounds.resolve_wall, which it needs here as it does for a conflict. A repository whose rules name either never waits on a person for a dependency's move; with neither, and no repair on the red check, each request is journaled repair-failed, saying no section names a repairer, for the hook to tell one. The rules also name one hook, a command that receives every event and may refuse a branch when it is queued or handed out. A hand-out is mq new's: once the worktree is made, and before its path is printed, the handed-out event (the branch, its base, its description's fields, the worktree and where it came from) is put to the hook in the same way, and a refusal makes mq new remove the worktree, the branch it created and that branch's record, so a tracker can claim the issue the branch's Bug field names, or refuse it, with no command of its own. The refusal is the hook's verdict on the queued event's line, given before the pin is written and bounded by bounds.hook_wall: exit 0 admits, any other exit refuses with the hook's first line of standard error as the reason, and a hook past its bound ends the land unavailable. A land into the rules branch is never put to it, so the land that fixes a broken hook always lands; nor is a serve of a branch the rules name: its commits were put to the hook when they landed into it, so mq land main --onto full costs no hook call however many commits it carries, and its queued event is journaled at once. The queued event the journal then holds is the line the hook judged, byte for byte, and the hook's environment says which call it is: MQ_HOOK=verdict at queue or hand-out time, MQ_HOOK=delivery otherwise. judges names the event kinds the hook gives a verdict on, queued and handed-out, both by default; an event of a kind it does not name is not put to it as a verdict, so a hook that admits every hand-out says judges = ["queued"] and mq new starts no process for it. A hook that only judges says deliver = false in its section: it is asked for its verdicts and is delivered no event, so a land starts it once, and mq status counts no pending delivery, those written before the key included; the delivery's position stays where it stood, so removing the key delivers from there what the journal still holds. A hook that acts on a few types names them, as a webhook names the events it subscribes to (GitHub's events, Stripe's enabled_events): deliver = ["outcome.landed", "culprit"] puts to it the events of those types alone, each name a CloudEvents type after merge-queue., and the rules check refuses a name no event is written as. The delivery passes an event of another type with no call, it waits behind no event of its subject, and mq status counts it in no pending delivery; naming a type later delivers its events from then on, not those passed before. Otherwise every event is delivered by the task mq task hook, under its lease, given the records the hook owes that are due, oldest first (design, "The delivery"): one CloudEvents JSON line a call on standard input, at least once, with the same id and bytes on every redelivery, acknowledged only by exit 0, each call bounded by bounds.hook_wall. Each event is delivered on its own, as GitHub, Stripe and Knative deliver a webhook, and the events of one subject reach the hook in journal order, as a Kafka consumer reads one key's records in order: an event waits while an earlier event of its branch is held, and an event with no branch waits for none. The task answers with one hook-delivered record naming the events the hook acknowledged and those it held: refused, or behind a refused event of their subject. A held event is put again once a run starts after that answer, and on no schedule. A delivery call is given MQ_HOOK_ANSWER, the path of an empty file in which the hook may write its answer, one word and its argument a line, as a GitHub Actions step writes $GITHUB_OUTPUT; mq reads the file once the call ends and deletes it. A final line, with a refusal, says no retry can change it: the event is recorded once, as a hook-failed event naming it, its reason and its calls, and passed over. A file with any other line is no answer, and its call is a refusal. Each refusal is recorded, as a hook-refused event naming the event, its call, its reason and its log; a hook-refused or hook-delivered event is not itself put to the hook. A call past its bound or that cannot start is recorded as a refused one is. mq status says hooks N pending while N events the hook owes are due and the hook's task does not hold its lease (mq/lease/hook), with the events the hook held and the oldest of them, its reason and when it was refused, and the events passed over that mq doctor --repair has not delivered since. No delivery delays a land: the delivery is a task beside the lands' tasks, which a land and a run each start as they begin, as they end and as each of their rounds writes an event the hook takes, and a drop as it ends, whenever an event is due; none waits for it. A call a signal stops keeps its input in mq/logs/, and its standard error ends with the stop. The hook runs in the repository's main checkout with the caller's environment, unconfined, whatever mq.confine says. A tracker is such a hook, and it reads the fields mq new stored in the branch's description, from the event or with mq field NAME KEY. mq itself has no word for a tracker, and --field Issue=123 is a key and a value to it. A description is never pushed, so a published history carries no tracker id unless its author writes one. A public repository that must refuse a private tracker's identifiers says so in its hook: it reads a message's trailers as Git parses them (git interpret-trailers --parse) and asks the tracker whether a token is one of its issues. A pattern decides nothing, since a prefix that names a tracker's issues names packages and English words as well.

One machine's facts are not rules. They live in that machine's Git configuration and are set with git config.

Key	What it sets
`mq.rules`	the branch that holds this machine's rules (default `mq/config`)
`mq.pool`	the pool's directory (default `<repository>.worktrees`, beside the repository), one directory however it is spelled: mq reads it with no trailing slash and as realpath(3) resolves it when it exists, and compares every path under it the same way
`mq.poolBudget`	the disk the parked worktrees may use; a hand-out evicts them, least recently used first, to keep within it, and a worktree in use is outside it
`mq.freeSpace`	the free space the volume must keep; a hand-out under it evicts the parked worktrees, and ends `unavailable` with the measured free space and floor when the volume stays under it with none left
`mq.slots`	how many builds the repository runs at once
`mq.generations`	read by the code as how many generations of each dependency the pool keeps beside the ones a check reads (default 4); a generation is a worktree locked with `git worktree lock` and reaped when no worktree links it, never by count (The state, Where the code stands)
`mq.checkLogs`	the disk the logs of ended checks may use (default 512M, two hours of eight lanes' checks); a writer of a log deletes the oldest past it, and the logs of a check under way in any process are outside it
`mq.private`	a repository on this machine whose content must not reach this one, one key each (Several repositories)
`mq.dependencies`	where `mq init` clones a consumer's dependencies this machine lacks (default the parent of the main checkout; A fresh clone)
`mq.confine`	whether checks and the resolver run confined: `off` (default), as a Git hook runs, or `auto` where the machine can (How a check is confined)
`mq.evidenceKey`	the Ed25519 private key, a PEM file, that signs this machine's evidence (Evidence)
`mq.evidenceName`	the package this machine's evidence names, a word of letters, digits, `.`, `_` and `-`; set exactly when `mq.evidenceKey` is

Each key is read as git config --get reads it, from every file Git reads, the last value counting, except mq.private, read as git config --get-all reads it: every value counts, so a repository's own declaration adds to a global one rather than hiding it. A path is read as git config --type=path reads it and is absolute; a size or a count as git config --type=int reads it, a size not negative and a count positive. A value that is not one is refused, naming the key, the value and the file that set it, with next: git config --file FILE --unset-all KEY, which leaves the value an earlier file sets, or the default; a value set in the environment is in no file, and its refusal names no next.

The remote a branch is published to is the one git push would send it to, from Git's own configuration (mq push), and the scheduler's registration is the operating system's. mq has no config command: the shared rules are edited with an editor and landed, and the local facts are Git's.

The pool's stock #

mq keeps worktrees parked at the tip of each branch the rules name, so that mq new takes one whose files are the tip's and writes only what its branch changes. The restock is a task, mq task restock, under the lease restock, which no run waits for. A run starts it while a place of a branch's stock holds no parked worktree: each worktree parked under .idle/BRANCH/ takes the first place of that branch's stock that holds none (design.md, "The machine's interface"). The task reads each branch's stock and, while one is short of its target, takes one step at a time: it moves a parked worktree off the tip there by the diff, or adds one worktree at the tip, journaled as parked with the reason restock (a landed worktree parked by its tidy has the reason landed); it ends once every stock is at its target. A hand-out that finds nothing parked adds its own, and its handed-out event says added: nothing parked.

The [pool] section's stock is the target's floor, 2 when the rules have no [pool] section, 0 for none. The hand-outs on a branch in the last ten minutes raise its target, up to stock_max, 8 when unset (or stock, when that is higher), never under stock: a burst of lanes finds worktrees parked, and a peer that asks for hand-outs makes the run park at most stock_max worktrees a branch ahead of them.

[pool]
stock = 2
stock_max = 8

The pool's volume #

The pool's worktree adds and removes and the checks' _build trees make and delete hundreds of thousands of files an hour, and so does the build system's cache. On macOS, fseventsd logs every file event of a volume whose root holds no .fseventsd/no_log, and XProtect assesses each new executable; on a busy host the three spend cores between them. The pool and the build cache share one dedicated APFS volume whose root holds .fseventsd/no_log, with .noindex in its name so that Spotlight passes over it. They share it because a build restores each cached file into a worktree by cloning it (clonefile(2), dune's copy storage mode), and a clone cannot cross volumes: from a cache on another volume every restore copies the file's bytes. A dune restoring 54803 rules from a cache on the startup disk took 182 s and 199 s into a pool worktree on its own volume, and 82 s and 99 s into a clone on the cache's volume, interleaved on one host. The build cache's directory is a link onto the volume, so every build on the host, a check's or a lane's, restores by clone with no setting of its own:

diskutil apfs addVolume <container> APFS mq -mountpoint /tmp/mq.noindex
mkdir -p /tmp/mq.noindex/.fseventsd /tmp/mq.noindex/dune-cache
touch /tmp/mq.noindex/.fseventsd/no_log
mv ~/.cache/dune ~/.cache/dune.old && ln -s /tmp/mq.noindex/dune-cache ~/.cache/dune
git config mq.pool /tmp/mq.noindex/<repository>

and dune's configuration, ~/.config/dune/config, keeps the copy mode, in which a restore is a clone and a build that writes into its output never writes into the cache:

(cache enabled)
(cache-storage-mode copy)

<container> is the APFS container of the startup disk, the APFS Container that diskutil info / names. The log stops once fseventsd next mounts the volume: unmount and mount it again (diskutil unmount mq && diskutil mount mq). mq doctor reads the volume the pool is on, the topmost directory on the pool's device, and names a pool on a volume fseventsd logs as a volume finding, and a build cache on another device than the pool, where every restore copies, as another.

The cache on the volume grows with every build on the host, and dune bounds it only when told to: dune's configuration has no trimming policy. Eight lanes and six check slots grew it by 1.5 GB a minute, to 177 GB in seven hours, until the volume fell under mq.freeSpace and every hand-out was refused while evicting every parked worktree could not help. The [pool] section's cache_size bounds it, in bytes, 60 GB (60000000000, dune's GB) when unset:

[pool]
cache_size = 60000000000

The trim is a task, mq task trim, under the lease trim. A run starts it once the cache entries a check added are owned by no tree and configuration a train or a target's tip still reads (design.md, "The machine's interface"). The task reads the cache's size, the bytes of the files no build directory links to as dune cache size counts them, and over the bound runs dune cache trim --size=BOUNDB with DUNE_CACHE_ROOT set to the cache's root (dune keeps its entries under the root's db), its output in mq/logs/cache-trim.log, and journals a trimmed event with the cache's bytes before and after and the bound; a trim that leaves the cache over its bound says it is still over it, with the bytes freed or "nothing freed". The task holds no run lock, its lease, trim, names no deadline, and no run waits for it, so no land waits behind a trim. A trim costs what dune's own does, which reads every entry and every metadata entry of the cache: 3440 s of wall at load 72 for a cache of 325000 entries and 1.47 million metadata entries. One trim of a cache runs at a time, whichever repository's queue or hand starts it: the task holds an exclusive flock on mq-trim.lock in the cache's directory from its first count to its last, dune's processes inheriting it, and beside a trim that holds it runs no dune and prints the repository and process the lock's file names. The pool's reading under mq.freeSpace and the doctor's are dune's count under a wall: a hand-out refused for mq.freeSpace names the cache's size and bound when the cache is over it and dune counted it within ten seconds, and mq doctor names a cache over its bound as a volume finding.

Each check worktree under the pool's .check/ keeps its _build between checks, so that the next check of its target starts warm. Its files are links into the cache or outputs only it holds, and a cold one restores from the cache in seconds, so the worktrees' builds are bounded too: every five minutes into a scheduled run, the run reads the bytes of each _build no other directory links to, and while they total more than the [pool] section's check_build_size, half of cache_size when unset, it sets aside under .reap/, for the reap to delete, the _build of the worktrees no check runs in, least recently used first, until the rest are within it. A worktree a check runs in, or is being taken by one, keeps its _build. The reading walks a _build, one stat per file, only when nothing records its bytes: a walk writes them into _build/.mq-bytes, every later reading reads that record and stats no file of the build, and a check that ends in the worktree, however it ends, removes the record, so the next reading walks that build once again. A cleared _build takes its record along and counts nothing. A worktree a check runs in is never walked, its build being written: it counts its record, or nothing until its check ends. Parsimoni's .check/main/.1.suite alone held 50170 files, walked by every reading of every five minutes before the record. A cleared event names each worktree cleared with its bytes, the pass's wall from its reading of the builds to the event, and how many build directories it walked.

[pool]
check_build_size = 30000000000

Where the state lives #

mq keeps two places. The repository's common directory holds mq/, the records a run and a command read and write, each a file or a directory of small files; the pool holds every tree mq checks out, builds in and deletes, on the pool's volume, so that a build there restores from the cache by clone and the trees' file events stay off the startup disk.

Where	What
`mq/journal`, `mq/index.N.db`	the journal and the index folded from it, one file per schema identity
`mq/run.lock`, `mq/lease/`, `mq/.lock`, `mq/slots.`, `mq/resolvers.*`	the locks, leases and slots of the runs and checks
`mq/requests/`, `mq/schedule`, `mq/last-run`, `mq/wake`, `mq/walls`	the requests to the next round, the scheduler's records, the commands' walls
`mq/logs/`, `mq/run.log`, `mq/sync.log`	the check logs and their traces, bounded by `mq.checkLogs`, and the tasks' logs
`mq/cone/`, `mq/cone-answers/`, `mq/cone-graph`, `mq/cone-target/`	the cones asked and their answers, and the checkout of the graph files alone that a cone reads
`mq/tip-failures/`	the target tip's verdicts, one small file each
`mq/tip-lint/TARGET/TIP/`	the lint readings of the tip of TARGET a check last compared with, one small file each: a reading kept for a newer tip of TARGET removes those of every other
`mq/tip-red`	the runs of suites and of their comparisons at commits each target holds or may yet hold, from which each read derives the tests red at its tip (mq ocaml)
`.idle/`, `.stock/`	the parked worktrees and those being parked
`.check/TARGET/.N.CHECK`	the check worktrees
`.tip/TARGET/TIP/.N.CHECK`, `.tip/TARGET/.N.CHECK.lock`	the trees a check builds the target's tip in, under the target and tip that own them, and their slots' locks (mq ocaml)
`.deps/`, `.resolve/`	the dependencies' generations and the resolver's worktrees
`.reap/`	the trees set aside for the reap (`mq task reap`)

Sync #

A machine reaches the agreed copy when someone types mq pull and mq push, or mq sync, which is the one then the other. The [sync] section, which takes no key, has the queue do it, the same on every machine that reads these rules:

[sync]

A round starts mq sync whenever a branch the rules name, the rules branch or a remote-tracking copy of one holds another commit than it did when the last sync started, and no sync runs: a landing is pushed as it lands, and a pull runs once the copy moved, as a fetch moves it. No sync polls the remote: what another machine publishes reaches this one at its next sync, which a move here, a fetch or a typed mq sync starts. The sync is a task beside the lands' tasks, in a process of its own and a session of its own, its output in mq/sync.log, its start and process in mq/synced and the commits it started from in mq/synced-tips. Neither waits for the other: a pull and a push take no run lock and run no check. A freeze stops the sync with everything else: no run starts it, and a sync typed by hand fetches but moves no branch and pushes nothing, each refusal naming the freeze (commands.md, mq freeze). A round reads whether a sync is due when the run starts, each time its work appends to the journal, as a landing does, and once more as the run's work ends; a sync that ends wakes the scheduler's job, so a landing made while it ran starts the next one. A commit made with Git on the rules branch changes mq/wake (the reference-transaction hook changes it when a branch the rules name or the rules branch moves), and the run that wake starts publishes it: a run lasts seconds and waits on no check. A sync's own push moves the copy it pushes to, so it leaves one more sync to find nothing to do. Whatever starts it, a sync holds mq/sync.lock from its pull's fetch to its push's end, as a typed pull and push each hold it, and a consumer's push holds each dependency's while it publishes the dependency: a second sync, or a consumer's sync that publishes this repository, waits for the first to end rather than read the remote while it publishes. A remote that stops answering ends the sync's pull or push within bounds.remote_stall, unavailable and naming the remote, so the lock is released and the next run starts a sync again. A machine with no scheduler registered syncs only when typed.

A repository with a remote wants the rule: the queue is what publishes its landings, as a merge queue that lands on the remote publishes as it lands, and a landing the agreed copy lacks is an incident, not a setting (invariants, liveness 12). mq init writes [sync] into the rules it creates when the repository has a remote. Without the rule, mq status says so on a line of its own, in the alert colour, with the rule to add (sync overdue: the rules have no [sync] section: nothing publishes landings to origin but a typed mq push · add [sync] to mq/config), and mq doctor names it with the same remedy; each branch whose copy lacks a landing is then overdue since that landing, from the landing on. The time is the branch's own: the first line of its reflog after its copy last changed, so a commit made with Git on the rules branch, which no landing record names, is overdue from that commit, never from the copy's older change. With the rule, a branch is overdue once a push since its landing was not taken, or once its copy lacks the landing past bounds.run_start, or at once on a machine with no scheduler registered, with why no sync published it: none did (no sync published it within bounds.run_start (5m)), or its last push was not taken, refused or its remote out of reach, with the push's own words (its last push was not taken: origin could not be reached: ...). A push refused on a divergence no pull ends is journalled once for that pair of values, and the line names it with the command that ends it until the branch or its copy moves; each sync's push reads the remote again, which is how it sees the divergence end.

Each repository syncs on its own, the dependency as the consumer. A consumer's push refuses a lock that names a dependency commit the dependency's agreed copy lacks (commands), and its pull refuses one no landing record maps; either goes through at a later sync of the consumer, once the dependency's own has published or pulled it. No sync pulls or pushes a dependency on a consumer's behalf.

The lock #

A consumer's dependencies are part of its tree, so each branch, each commit and each release series says what it builds with. The lock, a file at the root of the tree, holds one section per dependency, in Git's configuration syntax (git-config(1), "CONFIGURATION FILE"). Its filename is .mqmodule. The url is the repository identity and fetch location, in Git's remote syntax, including local paths. Section names label dependency links. Relative local sources resolve against the declaring repository's root. One function classifies every source through nox-git's remote codec; mq never guesses that different mirrors or renamed URLs are the same identity. The declared path binds the source to a local checkout whose Git common directory supplies its run lock. Commit reachability proves availability, never identity or authenticity.

[dependency "lib"]
	path = vendor/lib
	url = https://example.org/lib.git
	commit = <the 40 hexadecimal digits of a commit of lib>
	back = site

Key	What it says	Written by
`path`	where the dependency's symbolic link stands in a checkout of the consumer, relative to its root	the consumer's author
`url`	repository identity and source a fresh clone fetches from, including local paths	the consumer's author
`commit`	the commit of the dependency that goes with this commit of the consumer	mq's lock commit, at every land whose paired branch's tip differs from it (Several repositories)
`back`	optional: the path, in the dependency's tree, of a tracked relative link that names this consumer	the consumer's author

The section's name labels a link in this consumer; another consumer may use the same label for a different URL. Two declared links with one URL that resolve to different local Git common directories are refused at the land that meets both, naming the two paths. A section with a key missing, a path that leaves the tree, a commit that is not a full commit id or two sections with one path are refused at mq new and mq land, naming the file, the section and the key. A lock is read from a tree with Git alone (git config --blob REV:.mqmodule --list); mq reads and writes it through nox-git's configuration codec, and an interoperability test holds the bytes it writes equal to those git config -f .mqmodule dependency.NAME.commit C writes.

The development loop reads the link at path, never the lock: the link names whatever the developer has there, and a check, a fresh clone and a release read the lock. A check's build reads the dependency through the link at path as the development loop does, so Dune, Make or Cargo build it with no adapter beyond that path, and a build that needs the commit itself reads the lock. A build system's own lock file and a .gitmodules for Git's submodule tools are derived from the lock, and choose no commit of their own.

The [lock] section of the rules names how: command, run by /bin/sh -c at the root of a worktree of each lock commit mq writes, whose dependencies' paths read the commits it selects, and of each land's replayed commits whose lock moves no dependency, since the command reads the consumer's tree too, bounded by bounds.check_wall, with MQ_TARGET set; and paths, the files and directories, relative to the root, that the command derives. The lock commit then holds each of paths as the command left it (a file with its executable bit, a link, a directory, or nothing), and nothing else the command wrote: a landed tree's lock and what it derives never disagree, and nobody regenerates them by hand. A command that does not exit 0 ends the land unavailable, naming its log, relock-COMMIT.log beside the checks' logs. A path that leaves the tree, the lock itself, a section without a command or with no path is refused when the rules are read. With no [lock], the lock commit changes the lock alone.

Generated files #

A file a build writes from the tree's sources and the tree also commits, as dune build @install writes each .opam from dune-project, has one producer: its generator. The [generated] section names them: files, path patterns relative to the root in the syntax of .gitattributes (gitattributes(5), "PATTERN FORMAT"), and command, the generator, run by /bin/sh -c at the root of a worktree of a replayed commit. A replay whose pick conflicts in these files alone is regenerated rather than held (Conflicts, "Generated files"). A section without a command, with no file or with an empty pattern is refused when the rules are read. With no [generated], every conflict is a conflict.

The install #

The repository that builds mq names, in an [install] section, how the scheduled run keeps the mq it runs built from a branch it checks (Installing from the queue's own branch): branch, a branch the rules name by its own name; check, the check that builds the executable and starts it, a [check.NAME] section that no branch lists, so the check worktree it runs in is its own; executable, the file that check leaves, a path inside the tree; and trial, how long after an install a run that could not run or ended as a defect rolls it back, 30m when unset. A section that lacks one of the first three, names a branch or a check the rules do not, a check a branch lists, or a path that leaves the tree is refused when the rules are read. With no [install], the run installs nothing.

[check.install]
command = "dune build bin/main.exe @test/startup"

[install]
branch = "staging"
check = "install"
executable = "_build/default/bin/main.exe"
trial = "30m"