A System One model, such as TypeSafe's Jev, does not generate text: it answers named questions about a state with typed judgements. A Choice picks one option of a set and gives each option's probability, a Noul gives the probability that the answer to a yes/no question is yes, and a Score places the state on ordered, described levels. One request carries a state and every question asked about it.

typesafe holds checked, inert values: requests, questions, answers, token usage and the API's refusals, each with a JSON codec. It enforces the documented request rules (at most 255 Choice options, 2 to 10 Score levels, 64 levels of nesting, finite numbers, UTF-8, a 32 MiB body) before anything is sent, and checks that a response answers each question asked. It performs no I/O.

typesafe-eio sends the requests over requests-eio, retrying a failed one under llm-eio's policy, and backs the typesafe command.

Security #

Sending text to TypeSafe publishes it to a third party. The typesafe command refuses a file (a state, a questions file or a batch) that lives in a git repository whose remote belongs to a private owner, including a local clone of one, and standard input when it runs in such a repository, unless --allow-private is given.

The API key is sent only as an Authorization: Bearer header and never printed: -v names where the key came from, not the key. A key file that its group or others may read is refused, with the chmod that fixes it.

Install #

opam install typesafe-eio

Usage #

Library #

A request is built and checked without a network, and refused, with the rule it breaks, when the API would refuse it:

# let text = Typesafe.Content.text ;;
val text : string -> Typesafe.Content.t = <fun>
# let request =
    let ( let* ) = Result.bind in
    let* department =
      Typesafe.Question.choice
        (text "Which team should handle this?")
        [ ("billing", Some (text "Payments, invoicing, refunds"));
          ("technical", Some (text "Bugs, outages, integrations")) ]
    in
    Typesafe.Request.v ~state:(text "My payouts have been failing.")
      [ ("department", department) ] ;;
val request : (Typesafe.Request.t, string) result = Ok <abstr>
# Result.map Typesafe.Request.to_string request ;;
- : (string, string) result =
Ok
 "{\"state\":\"My payouts have been failing.\",\"model\":\"jev-latest\",\"questions\":{\"department\":{\"type\":\"choice\",\"instructions\":\"Which team should handle this?\",\"criteria\":{\"billing\":\"Payments, invoicing, refunds\",\"technical\":\"Bugs, outages, integrations\"}}}}"
# Typesafe.Question.score (text "How severe?") [ text "Cosmetic" ] ;;
- : (Typesafe.Question.t, string) result =
Error "a Score has 2 to 10 levels, not 1"
# Typesafe.Request.v ~state:(text "x") [] ;;
- : (Typesafe.Request.t, string) result = Error "no question to ask"

Sending it with the Eio client, and reading the answer:

let ask request =
  Eio_main.run @@ fun env ->
  Eio.Switch.run @@ fun sw ->
  let clock = Eio.Stdenv.clock env in
  let session =
    Requests_eio.v ~sw ~mono:env#mono_clock ~clock (Eio.Stdenv.net env)
  in
  let key =
    Result.get_ok (Typesafe.Api_key.of_string (Sys.getenv "TYPESAFE_API_KEY"))
  in
  let client = Typesafe_eio.v ~clock session key in
  match Typesafe_eio.ask client request with
  | Error e -> prerr_endline (Typesafe.Error.to_string e)
  | Ok r -> (
      match Typesafe.Response.answer r "department" with
      | Some (Typesafe.Answer.Choice c) when c.confidence >= 0.5 ->
          print_endline c.choice
      | _ -> print_endline "ask a person")

Typesafe_eio.ask_all answers a list of requests in order, with a bounded number in flight.

Command line #

The key is read from --key-file PATH, then from TYPESAFE_API_KEY, then from $XDG_CONFIG_HOME/typesafe/token (by default ~/.config/typesafe/token), which must have mode 0600.

Questions are data: a JSON object mapping each question id to a question, in the API's own shape.

$ cat triage.json
{
  "department": {
    "type": "choice",
    "instructions": "Which team should handle this?",
    "criteria": {
      "billing": "Payments, invoicing, refunds",
      "technical": "Bugs, outages, integrations",
      "sales": "Pricing, upgrades, new accounts"
    }
  }
}
$ typesafe ask --questions triage.json -s 'Help! My payouts have been failing for 3 days.'
department: choice billing (p=0.87, confidence 0.80)
jev-1.13.0: 349 input tokens, 38 output tokens
$ typesafe ask --questions triage.json --json ticket.txt
$ typesafe ask --questions triage.json --dry-run ticket.txt

ask prints one line per answer, or the response as received with --json; the model that answered and the tokens counted go to standard error. --json-state reads the state as JSON, an object or an array.

batch reads JSON Lines, one item per line: an id, a state and, optionally, the item's own questions; an item without questions is asked those of --questions. Every item is checked before the first request is sent. It prints one JSON line per item, in order, with the response or the error, and keeps at most --jobs requests in flight.

$ typesafe batch --questions triage.json -j 8 tickets.jsonl > answers.jsonl

--dry-run prints the request bodies and sends nothing; it reads no key and refuses no file. --model (or TYPESAFE_DEFAULT_MODEL) names another model, such as jev-preview or a versioned id; --endpoint (or TYPESAFE_ENDPOINT) another server; --long-context raises the context limits to 262k tokens. The exit status is 0 when every request was answered, 1 when a batch item was not, and 123 when nothing was sent or the one request was not answered.

Tests #

The codec tests decode the examples of the TypeSafe API reference, copied verbatim, and check the documented request rules at their bounds. The client and the command are tested against a local fake of the API. The interop test decodes answers recorded from the live API; run TYPESAFE_API_KEY=... REGEN=1 dune build @traces with a key to record them again.

TypeSafe publishes Python and JavaScript SDKs; this package covers the same text questions (Choice, Noul, Score) for OCaml, without the Box question or images.

License #

ISC. See LICENSE.md.

Clone this repository

typesafe #