Compaction #
Compaction changes the conversation projected into a model request. The durable
transcript remains the source for text history, but tool-output image payloads
can be elided. The strategy interface and request preparation live in
compaction.gleam. The
context inspector reports the prepared request and its compaction
observation.
Request preparation #
A Strategy supplies project and compact. Projection reads saved strategy
state without writing it or calling a model.
prepare projects the history,
applies notes layers, and compacts when the strategy says the projection is due
or the request forces compaction. It compacts at most once per request, then
projects again. Errors propagate to the caller.
triggered triggers a forced
compaction regardless of capacity. Otherwise it compares the request's size with
a percentage of known capacity. Without capacity it does not trigger
automatically.
Every strategy sizes through Context.sizing (compaction.Sizing); none turns
lengths into tokens itself. A request's size is the provider's count when its
route has a count endpoint (Upstream.count), else the reported input and
output of the request it continues plus an estimate of what came after, else an
estimate of the whole request. The loop
keeps how many inputs the measured request carried and their input chain, and a
request continues it only while it starts with the same inputs. Rows are priced
from the reported usage of the calls that wrote and first sent them
(requests.row_costs), else estimated.
size is the one module that estimates: a
crude length measure times a correction each response teaches for its route and
model, priced high until a route has reported three times and held within one to
four times the crude measure. Each observation's size says how much of it was
counted, reported or estimated, with the correction the estimate used (see
the context inspector).
Each strategy's observation names where it compacts as trigger_tokens, in the
same measure as the observation size's tokens. For rolling, snapcompact
and coding-lcm it is their share of the window rounded up to a whole token,
the boundary triggered compares
against; for persistent-agent it is foldTokens, or 85% of a smaller window.
The full and summary context views both carry it, and it is null until a request
has been observed. The context meter draws compaction at that point and counts
the tokens left until it, and draws neither when the strategy reports none.
Clients must not derive the point from the window themselves.
persistent-agent and coding-lcm size their tails, chunk checks, pressure
relief and merge guards the same way. persistent-agent starts a fold only on
the trusted part of a request's size: what providers measured, and estimates
once their correction has settled from reported usage. A request sized only by a
provisional estimate, such as the first after an upgrade, never folds, and its
observation says so. A request that still overflows comes back as a
ContextTooLong refusal, and loop forces
one compaction and resends it.
Strategies #
The /compact command can retain the active strategy or switch to
snapcompact, rolling, coding-lcm, or persistent-agent. The command
describes an idle session and preserves the transcript. See
commands/extension.gleam.
snapcompact
can represent earlier conversation as rendered frames or serialized text, with a
verbatim tail. Its memory extension exposes transcript.grep and
transcript.read to retrieve transcript rows. Its defaults are an 80% trigger,
10% tail, and 20% archive budget. When maxFrames is unset, the frame budget
depends on capacity and the provider's declared image limits, with an upper
bound of 80 frames. Each provider's ImageLimits carries a VisionProfile
(geometry family, whether images ride inline, and a frame cap for a provider
that states no image count); snapcompact reads only that, never a provider or
model name. Settings validation requires
0 < tailPercent < triggerPercent < 100 and 0 < archivePercent < 100.
Snapcompact's Gleam extension owns archive normalization and pagination. It
expands tabs to four spaces and folds adjacent CR/LF runs into full-block cells.
Escape removal consumes whole Unicode scalars; malformed legacy bytes become
replacement characters. The scanner retains limited CSI and OSC handling,
including OSC termination by ESC followed by any character, rather than acting
as a terminal emulator. Frames count scalars, including combining marks and
joiners, independently. Nonpositive advance, width, or row count, or a frame
with no cells, returns no pages. render_frames writes every missing chunk to
one scratch directory and renders them in one
sdk/renderer run. Input names sort
in chunk order. sdk/renderer finds the binary, sets the 30-second timeout and
runs it through
albedo_ports.
The other strategies have their own state and configuration in
rolling,
coding-lcm, and
persistent-agent.
Their implementation, rather than snapcompact's defaults, determines their
behavior. When the model reads no images, snapcompact runs rolling's strategy as
its text fallback, and that strategy keeps its summary in rolling's table under
rolling's storage handle. See
public features own their tables.
coding-lcm and persistent-agent share the LCM summary graph. The
lcm-memory extension
follows forks and rewinds for that graph through a
FollowPlugin, whichever strategy is active.
A fork copies the nodes whose whole source span it keeps. A rewind drops the
nodes that reach past the cut, and their complete children become the frontier
again. See graph.follow.
The graph's tables belong to lcm-memory. coding-lcm and persistent-agent
declare its owner, so the graph's functions run under its storage handle
whichever strategy calls them.
persistent-agent consolidates memories while they take more than
memoryTokens and stops once they fit it. Six tenths of memoryTokens only
sizes how much one pass tries to free; consolidation does not chase it. A
memory's size is a size value: the
provider's count of the request with the memories in it, past the request
without them, when the route has a count endpoint; else the output tokens,
thinking included, that the call named by the memory's written_by reported;
else the size module's labelled estimate. Whether to consolidate is decided on
trusted sizes only: measured ones, and estimates once their correction has
settled from reported usage. Memories written before sizes were linked, priced
with a provisional correction, never start a consolidation by themselves. A pass
sizes the whole frontier, the batches it takes, and once the memories it leaves.
consolidation.plan
picks the batches oldest first: four consecutive memories of one level merge
into one a level up, and adjacent pairs merge only when no such group is left.
It counts each batch as saving half of itself and stops once that covers the
excess, so older life grows lossier and the newest memories keep their detail. A
compaction makes one pass of merges side by side, at most four, and leaves the
rest to the next fold; another pass runs only to fall back to smaller batches
after refusals. A memory a merge wrote during this compaction is not merged
again, alone or in a batch, until the next compaction, so a memory climbs at
most one level per compaction. The trailing level-0 memories, grouped in fours
from the oldest of them, end in the newest group of written memories, whole or
still forming. A batch touching it is merged only when that group is the whole
frontier and the compaction has not merged anything yet, so recent memories stay
as the agent wrote them even when that leaves the memories over memoryTokens
until a later compaction.
Each merge prompt asks for a length relative to the memories it joins, never a
number: about half as long, or a third when half would pass half the
summarizer's output limit. When half would still leave the memories over
memoryTokens, even counting half of every older memory still free to merge,
the pass's merges share by size what memoryTokens leaves beside the memories
they do not merge, asking for a third or a quarter, never less. It also asks for
a cue to each detail it leaves out. The joined memories stay in the graph as the
merge's children, so lcm.grep, lcm.describe and lcm.expand still reach
them and their rows. The length is prompt guidance; nothing truncates the reply.
A merge cut off at its output limit, or one whose size is not a tenth below the
memories it replaces, is discarded. The round then falls back from groups of
four to pairs, a refused pair ends it, and no fold asks for that batch again for
a day.
Everything an LCM request shows before its verbatim tail follows from the rows
the graph covers. When the tail continues a turn whose user message a fold took,
that message stays verbatim before the tail until the next fold, even after a
newer message arrives. A new message therefore only appends, so the next request
keeps the provider's cache and reuses its reported count. See
uncovered_of.
Summary failures and progress #
reply_text rejects an empty
summarizer reply and detects replies that stopped before finishing. A long reply
cut at its output limit is kept as a Summary with cut_off set, and its text
ends with a closing note for the reader. Code decides on the flag, never on the
note. A summary combined from parts is cut_off when any part was, even when
the reply that combined them finished. persistent-agent discards a
consolidation merge marked cut_off, because the merge lost its end and would
replace complete memories. LCM graph nodes and pending memories store the flag
in a cut_off column, and a forced persistent-agent fold re-forms any cut-off
leaf from its rows. Nodes written before that column were marked once, when it
was added, by whether their text ends with the note.
A Summary also names the provider request whose reply is its text, in
written_by: the provider_requests row the loop recorded for the call, or the
call that joined a summary's parts. Text kept verbatim has none. Graph nodes and
pending memories store it in a written_by column, so what a memory cost to
write is read from that call's reported usage rather than from its text. Nodes
written before the column stay unlinked, and nothing guesses a link for them. A
transfer shifts the column with the requests it moves.
Compaction exposes progress stages and updates through ProgressUpdate.
Consumers must handle failure as well as progress. A compacted request is a
projection, not a replacement for the durable transcript.
The notes layer can rewrite notes when newly evicted history extends beyond its saved cut. A summarizer failure keeps the previous notes and adds a failure observation. Configuration, storage reads, or saving rewritten notes can still fail preparation after the strategy commits.
Image retention #
images.elide_evicted can replace evicted
tool-output images with a marker while retaining their text. Unreferenced
payloads are then released. User uploads are excluded from that cleanup. The
same module implements age-based tool-image expiry. Payload reads and inserts
live in sdk/images. A transcript row
or a saved summary does not guarantee continued access to an image payload.