cache #

In-memory caches with a time-to-live, shared between Eio fibers.

A service that looks up the same remote data many times per minute, such as DNS answers, account records or OAuth metadata, wants to keep each answer for a while and then fetch it again. Cache binds keys to values that expire after a time-to-live (TTL). Each entry's TTL is the cache's base TTL plus a random jitter, so entries written together do not all expire at the same moment and send a burst of recomputations to the backend. Expiry is lazy: a lookup drops the expired entry it finds, and gc sweeps the rest. Every operation takes an Eio.Mutex, so fibers can share one cache.

Installation #

Install with opam:

$ opam install cache

If opam cannot find the package, it may not yet be released in the public opam-repository. Add the overlay repository, then install it:

$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install cache

Usage #

A cache with string keys #

Cache.String is ready to use:

let lookup env =
  let cache = Cache.String.v ~clock:(Eio.Stdenv.clock env) () in
  Cache.String.set cache "user:42" "Alice";
  match Cache.String.get cache "user:42" with
| Some data -> ()
| None -> ()

Computing a missing value #

get_or_compute returns the cached value, or computes, stores and returns it. The lock is not held while the function runs, so two fibers that miss the same key at once may both compute it; the last one to store wins.

(* Returns cached value or computes and caches it *)
let data cache =
  Cache.String.get_or_compute cache "user:42" (fun () -> "Alice")

Other key types #

Cache.Make builds a cache for any key type with equality and a hash:

module IntCache = Cache.Make (struct
  type t = int

  let equal = Int.equal
  let hash = Hashtbl.hash
end)

let cache clock = IntCache.v ~clock ~base_ttl:300.0 ~jitter:0.1 ()

API #

Every cache module provides:

v ~clock ?base_ttl ?jitter (), which creates a cache with a base TTL of 60 seconds and a jitter of 0.2 by default. The jitter j is a fraction of the base TTL b: each entry's TTL is drawn uniformly between b(1 − j/2) and b(1 + j/2), so the defaults give TTLs between 54 and 66 seconds;
get, set and remove;
get_or_compute, which computes and stores a missing value;
gc, which removes expired entries, and clear, which removes all of them;
stats, which returns the number of entries and the number of unexpired ones.

Licence #

ISC. See LICENSE.md.