The usual way to read a CSV file into records is to split lines by hand, look columns up by index and raise an exception on a bad value, with no row or column to point at; writing the records back means doing it all again the other way round. nox-csv has you describe a row once, as a codec in the style of jsont, and uses that description to decode a file into records and to encode records into a file. The codec's column names are matched against the header once per file, so decoding a row needs no further lookup.

The streaming functions use bytesrw: of_reader reads from a Bytes.Reader.t and to_writer writes to a Bytes.Writer.t, which lets CSV sit in a pipeline with other bytesrw codecs. of_file and of_string return the decoded rows, while fold_file and fold_reader go through them without building the list.

A decode error carries a source location from nox-loc and a path to the value that failed, giving the row by index and the column by name.

Installation #

Install with opam:

$ opam install nox-csv

If opam cannot find the package, it may not yet be released in the public opam-repository. Add the overlay repository, then install it:

$ opam repo add samoht https://tangled.org/gazagnaire.org/opam-overlay.git
$ opam update
$ opam install nox-csv

The package installs the library nox-csv, whose module is Csv. It needs OCaml 5.1 or later. The package nox-csv-eio installs the library of the same name, whose module Csv_eio reads, folds and writes the files an Eio.Path.t names, keeping the core free of Eio.

Usage #

type point = { x : float; y : float; label : string }

let point_codec =
  Csv.(Row.(
    obj (fun x y label -> { x; y; label })
    |> col "x" Field.float ~enc:(fun p -> p.x)
    |> col "y" Field.float ~enc:(fun p -> p.y)
    |> col "label" Field.string ~enc:(fun p -> p.label)
    |> finish
  ))

(* Decode a string. Errors carry source locations and row/column context. *)
let csv = "x,y,label\n1.0,2.0,A\n3.5,4.5,B\n"

let points =
  match Csv.of_string point_codec csv with
  | Ok points -> points
  | Error e -> failwith (Csv.Error.to_string e)

let () =
  assert (points =
    [ { x = 1.0; y = 2.0; label = "A" };
      { x = 3.5; y = 4.5; label = "B" } ])

(* Streaming decode from a bytesrw reader. *)
let sum =
  let r = Bytesrw.Bytes.Reader.of_string csv in
  match Csv.of_reader point_codec r with
  | Ok rows -> List.fold_left (fun acc p -> acc +. p.x) 0.0 rows
  | Error e -> failwith (Csv.Error.to_string e)

let () = assert (sum = 4.5)

(* Encode to a bytesrw writer. Raises [Csv.Invalid_utf8_encode] if any
   field string contains malformed UTF-8. *)
let encoded =
  let buf = Buffer.create 256 in
  let w = Bytesrw.Bytes.Writer.of_buffer buf in
  Csv.to_writer point_codec points w;
  Buffer.contents buf

let () = assert (encoded = "x,y,label\n1,2,A\n3.5,4.5,B\n")

The decoding functions return errors as Csv.Error.t values, so a program such as an editor or a report that collects errors can look inside them. Csv.Error.kind gives the kind of error and Csv.Error.to_string the full message. The reading functions whose name ends in _exn raise Csv.Error instead:

let error_kind =
  match Csv.of_string (Csv.col "x" Csv.Field.float) "x\nbad\n" with
  | Ok _ -> failwith "expected an invalid float"
  | Error e -> Csv.Error.kind e

let () =
  assert (Csv.Error.string_of_kind error_kind =
    "invalid value \"bad\": not a float")

API #

lib/csv.mli is the reference, which dune build @doc renders with odoc.

Field codecs: Field.string, int, int64, float, bool, option, nullable_float and nullable_int. Field.v builds a custom codec and Field.map transforms one; Field.decode, Field.decode_exn and Field.encode run one on a single field.
The Row builder: obj, col, col_at, col_block and finish. Named and positional columns take a ?dec_absent default, and col_block reads strided groups of columns.
Reading: of_file, of_string and of_reader, with _exn variants. Writing: to_string, to_writer and to_file, which replaces a file whole or not at all. Folding: fold_file and fold_reader.
Csv_eio, in nox-csv-eio, has of_file, of_file_exn, fold_file and to_file on an Eio.Path.t, reading and writing through Eio flows, and writing through an atomic publication.
decode, decode_exn and encode work on rows already split into fields, header first, and resolve, decode_row, encode_row and encode_header on one row at a time against a header.
Value reads and writes untyped rows with no codec: Value.of_string, Value.of_reader and their _exn twins, Value.to_string and Value.to_writer with the codec writers' quoting rule, and Value.records, each data row paired with the header's column names.
col, update_col and delete_col query and update single columns, and col_names and col_count describe a codec.
Csv.Loc, Csv.Meta, Csv.Path and Csv.Error are the location and error modules of nox-loc, and the Sort module names the three sorts of value (Row, Field, Header) that errors refer to.
The encoder raises Invalid_utf8_encode when a field is not well-formed UTF-8.

For untrusted input, the functions that read bytes and the folds take ?max_rows and ?max_cols, which bound the number of data rows and the width of the header. Neither has a limit by default. The byte readers and writers take ?sep for a separator other than a comma.

A decode reads only the columns its codec names and skips the others, so a wide file costs no more than the columns in use. UTF-8 is checked on every header name, since resolving the codec reads them all, and in a data row only on the columns the codec reads.

References #

RFC 4180, the CSV format.
jsont, the JSON codec library whose design this follows.
bytesrw, composable byte stream readers and writers.
nox-loc, source locations and errors.

License #

ISC. See LICENSE.md.