small gleam coding and (not yet) persistent agent daemon with a detachable cli forked from okami.mom/albedo
albedo robot-docs workspace-packages.md
12 kB · 218 lines
Markdown
at main

Workspace packages #

A workspace package is a Gleam package in this repository that albedo depends on and that other projects could use. Each one lives in its own directory under packages/, with its own gleam.toml, manifest, sources, tests, and Nix derivation.

Current packages #

Six packages hold albedo's general BEAM helpers. Each module lives under its package's name, so the module albedo_process/memo is the atom albedo_process@memo.

  • albedo_protect: attempt and guarded answer a callback's crash as a value, rescue and raise catch and raise one again unchanged, and finally runs cleanup however a callback ends. Its catches live in albedo_protect_ffi.erl.
  • albedo_clock: system and monotonic clocks in explicit units, send_after and sleep through delay_ms, calendar parts, utc_iso, and parse_rfc3339. set_scale and scale set and read the VM's timer scale; the package reads no environment. See clocks.
  • albedo_bytes: albedo_bytes compacts, validates and searches binaries for protocol decoders, and albedo_bytes/scalars, digest, ids, zlib, term, and term_codec hold scalar-bounded text, hashes and encodings, random and seeded ids, bounded inflation, term ordering, and safe external-term decoding.
  • albedo_env: albedo_env.lookup tells an unset variable from an empty one, and albedo_env/log, file_status, and file_revision log a message, describe a path without following a final link, and read or set a file's size and modification time.
  • albedo_process: albedo_process labels processes and names callbacks, albedo_process/actor_call makes monitored calls, memo keeps expiring values under the registered owner albedo_process_memo, and lock holds a mutex per key under the registered owner albedo_process_lock.
  • albedo_ports: albedo_ports opens programs through ports with typed settings and events, selective reads, and the deadline kill, and albedo_ports/subprocess runs a bounded command with an environment allowlist.

Two packages implement AWS protocols for the Bedrock extension.

  • aws_sigv4: AWS Signature Version 4 for services other than S3. sign takes a Request, Credentials, a region, a service, and the signing time, and returns the headers to send. The Bedrock extension signs through wire.signed_headers, which adds Bedrock's signed content-type.
  • aws_event_stream: the bounded incremental decoder for application/vnd.amazon.eventstream. feed and finish return a typed FrameError, and describe words it. The Bedrock Converse stream decodes through it in converse_http.gleam.

Dependency direction #

Albedo depends on packages. A package never imports albedo. The compiler enforces this: a path dependency compiles with only its own declared dependencies, so an albedo/... import fails to build.

A package may depend on another workspace package with a path dependency such as albedo_protect = { path = "../albedo_protect" }. The graph has no cycles:

albedo_protect      (none)
albedo_env          (none)
albedo_clock     -> albedo_protect
albedo_bytes     -> albedo_protect
albedo_process   -> albedo_protect, albedo_clock
albedo_ports     -> albedo_protect, albedo_clock
aws_sigv4           (none)
aws_event_stream    (none)

A package cannot call the SDK or any other albedo module. Take what the package needs as a parameter, such as the signing time, depend on another workspace package, or write the small helper inside the package. aws_event_stream keeps its own small compact and UTF-8 check rather than depending on albedo_bytes, so an AWS client takes no albedo helpers with it. Prefer Gleam. A direct @external to an OTP function such as erlang:crc32 or binary:copy is acceptable. Add an .erl file only when Gleam cannot express the binding. Use only Hex packages that albedo already depends on, and add none without the maintainers' agreement.

Keep the public API shaped for an outside caller. Return typed errors rather than strings, and keep albedo policy, such as Bedrock's signed content type, in the albedo module that calls the package. A package does not read albedo's environment variables either: albedo reads ALBEDO_TEST_TIME_SCALE and passes the scale to albedo_clock.set_scale.

How albedo consumes a package #

The root gleam.toml names each package as a path dependency, as it does the patched copy under vendor/. gleam deps download records it in the root manifest.toml with source = "local". Gleam compiles the package into the root build/ and recompiles it when its source changes.

The erlang-shipment, the test snapshot from test/snapshot-daemon.sh, and the code-path snapshot in boot_code each take every application on the code path. A package's application ships and loads with no extra step.

A package's modules keep their own names on the VM. The module aws_sigv4 is the atom aws_sigv4, and albedo_bytes/scalars is albedo_bytes@scalars, not albedo@.... Erlang has one module namespace, so every module, Erlang binding, and registered process name a package adds must not collide with any other in the release. Keep them all under the package's name: a package's Erlang binding takes a name such as albedo_protect_ffi, since the Gleam module albedo_protect owns the atom albedo_protect, and its registered owners take names such as albedo_process_lock. priv/boot-modules.txt and priv/ready-modules.txt list modules by atom, so a module moved into a package must be renamed there. test/manual/record_boot_modules.py records every loaded module whatever its application.

The build digest hashes only the albedo application's ebin and priv. A change confined to a workspace package leaves a source checkout's digest unchanged, so the CLI does not see a rebuilt checkout as a different build, and keeps the running daemon with the old package code until it is restarted explicitly. A differing ALBEDO_BUILD label does not help: equal nonempty digests decide sameness. Nix builds are not affected in practice, because the digest also covers files that embed the daemon's store path, which changes when a package does. Restart the daemon after changing only a package until the digest covers every workspace application. Since the clock, protect, byte, environment, process, and port helpers moved into packages, a change to any of them falls in this gap too.

Nix #

workspace-packages.nix lists every directory under packages/ that holds a gleam.toml. workspace.nix builds each one on its own manifest with warnings as errors, checks its formatting, runs its tests, and installs its ebin and package-interface.json. Its source is rooted at packages/, and holds the gleam.toml and src of each workspace package its manifest locks beside its own, so a ../<name> path dependency resolves in the sandbox. default.nix exposes each as packages.<name> and checks.<name>, so nix build .#aws_sigv4 builds one and nix flake check tests all of them.

daemon.nix compiles the packages from source as path dependencies. Its source includes each package's gleam.toml and src only, so a change to a package's tests does not rebuild the daemon. gleam-deps.nix stages a manifest's Hex packages for the offline build and touches each path dependency's fingerprint, so gleam does not try to resolve versions over the network. The daemon, the packages, and albedo-lint share it.

A flake sees only files that git tracks. Add a new package's files to git before building it with Nix.

Checks #

test/packages.sh runs in each package directory. test/quick.sh runs its check mode, a format check and a build with warnings as errors. test.sh runs its test mode, the package's own tests, beside gleam test. nix fmt formats packages like the rest of the tree.

The default lint profile includes packages/ and excludes each package's build/. glinter's rules apply to packages. Most of albedo's own rules read only src/, because they enforce albedo's layering. The clock and text scalar rules also read each package's src/, so a package reads clocks through albedo_clock and cuts text through albedo_bytes/scalars. The process-wide registration checks read each package's src/ too, so a package's registered names and persistent terms are reviewed like the SDK's. The extension boundary rules treat a workspace package as a declared dependency: an extension may import its Gleam API, and albedo_extension_external still refuses an extension's @external binding to its Erlang modules.

Add a package #

  1. Create packages/<name>/ with a gleam.toml that sets name, version, description, licences, gleam, and target = "erlang". Name another workspace package it needs as <other> = { path = "../<other>" }. Add src/<name>.gleam, test/<name>_test.gleam with a gleeunit main, and a .gitignore holding /build.
  2. In nix develop, run gleam deps download in the package directory to write its manifest.toml. Do not edit a manifest by hand.
  3. Add <name> = { path = "packages/<name>" } to the root gleam.toml and run gleam deps download at the root.
  4. Move the albedo modules and their unit tests. Update callers to the package's API and delete the old modules.
  5. Search priv/*.txt, tests, and lint fixtures for the old module atoms and paths, and update them.
  6. Add the files to git, then run nix build .#<name>, test/packages.sh check, and the gate.

Nix, the gate, and the lint need no edit for a new package.

Naming and licence #

Name a package for what it implements, in snake case, not for albedo: aws_sigv4, not albedo_bedrock_sigv4. Its top-level module takes the same name. A package of general helpers whose plain name would be too broad for one module namespace, such as clock or bytes, takes the albedo_ prefix. Check Hex for the name before anyone publishes a package.

Packages carry the repository's AGPL-3.0-only licence in licences for now. The Nix derivation reads its meta.license from that field. Relicensing a package is the maintainers' decision.