cache #

In-memory caches with a time-to-live, shared between Eio fibers.

Programs that ask a remote service the same question every few seconds (a DNS resolver, an OAuth client fetching provider metadata) usually want to keep the answer for a minute and ask again later. Cache keeps each value for a time-to-live (TTL) and forgets it afterwards.

The TTL of an entry is the cache's base TTL plus a small random jitter. Without the jitter, a batch of entries written in the same second would expire in the same second too, and the backend would see all the recomputations at once. Nothing runs in the background: get drops an expired entry when it meets one, and gc removes the others. Each operation holds an Eio.Mutex, so one cache can be shared by several fibers.

Installation #

Install with opam:

$ opam install cache

The package may not be in the public opam-repository yet. In that case, add the overlay first:

$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install cache

Usage #

A cache with string keys #

Cache.String needs no functor application:

let lookup env =
  let cache = Cache.String.v ~clock:(Eio.Stdenv.clock env) () in
  Cache.String.set cache "user:42" "Alice";
  match Cache.String.get cache "user:42" with
| Some data -> ()
| None -> ()

Computing a missing value #

get_or_compute returns the cached value when there is one. Otherwise it calls the function, stores the result and returns it. The lock is released while the function runs, so two fibers that miss the same key at the same time can both call it, and the value stored last is the one that stays.

(* Returns cached value or computes and caches it *)
let data cache =
  Cache.String.get_or_compute cache "user:42" (fun () -> "Alice")

Other key types #

For other keys, apply Cache.Make to a module with equal and hash:

module IntCache = Cache.Make (struct
  type t = int

  let equal = Int.equal
  let hash = Hashtbl.hash
end)

let cache clock = IntCache.v ~clock ~base_ttl:300.0 ~jitter:0.1 ()

API #

Cache.String and every Cache.Make instance have the same signature. v ~clock ?base_ttl ?jitter () creates an empty cache. The base TTL b defaults to 60 seconds and the jitter j to 0.2, a fraction of b: each entry lives for a time drawn uniformly between b(1 - j/2) and b(1 + j/2), so with the defaults an entry lasts between 54 and 66 seconds.

get, set and remove work as in a hash table, and get_or_compute is described above. gc removes the expired entries and clear removes all of them. stats returns a pair: the number of entries, then how many of those have not expired yet.

Licence #

ISC. See LICENSE.md.