Prompt cache TTL table #
The TTL table supplies initial cache-warmth estimates. Provider request
measurements can replace those estimates. The table also supplies a clock and
price inputs to cache warming. Its values are priors, not
guarantees about a provider's current behavior.
cache_ttl owns the types, merge, and
lookup.
Format #
Each entry has an id, a match, a policy, a clock, evidence and source
metadata, and optional tiers, read multiplier, and survival estimate. A tier
contains seconds and an optional write multiplier. match can restrict
extension, host, and model with case-insensitive glob patterns. An absent field
matches anything. The accepted shapes and values are defined by entry_decoder
and its helper decoders in
cache_ttl.gleam. The shipped data is
priv/cache-ttl.json.
Refresh and Fixed entries with a tier provide a clock. Evict and Unknown
do not. lifetime uses the shortest explicit cache mark when present, otherwise
the first clock tier. for_call resolves the profile's extension and the
endpoint host before lookup. The first matching entry wins.
Layers #
cache_ttl.gleam reads the shipped
table, $ALBEDO_HOME/cache-ttl-remote.json, and $ALBEDO_HOME/cache-ttl.json,
in that order. Later layers replace an entry with the same ID in place. New IDs
precede earlier layers. Entries merge before full decoding, so an invalid
override can shadow an earlier entry and then be skipped. Duplicate IDs
invalidate a layer.
Reload #
Files are limited to 1 MiB.
layered_table caches parsed
layers by size and modification time. A malformed revision retains the layer's
last good entries and reports an error. Transient read errors are retried. A
missing optional layer contributes no entries. See resolve and parse in
layered_table, and accept in
cache_ttl.gleam. The context-limits
table uses the same layer reader.
cacheTtl.url in extension settings selects the remote source. null disables
fetching. refreshHours defaults to 24, and zero disables background refresh.
table triggers a stale refresh. reload explicitly fetches and reports
failure, retaining the previous copy on failure. A successful replacement
invalidates the caches even if the file revision appears unchanged.
cache_ttl.gleam and
sdk/cached_fetch.gleam
implement this path.
HTTP API #
GET /models?view=cache-policy returns paginated entries, layer reports, a
matching entry, and a continuation when needed. The lookup uses extension,
host, and model parameters, defaulting absent values to empty strings.
Provider parameters are rejected for this view. See the route in
http_resources.gleam and its
contract in openapi.yaml.