small gleam coding and (not yet) persistent agent daemon with a detachable cli forked from okami.mom/albedo
albedo docs extensions.md
30 kB · 599 lines
Markdown
at main

Write an extension #

Use this guide to add capabilities through an extension. It walks through a complete extension, then describes the other contributions an extension can make. Read the extension reference, plugin contracts, and shared contribution rules before choosing an owner.

Follow the SDK import contract when choosing shared helpers, daemon imports, and native bindings. The extension API reference covers every public SDK function and explains how to build the searchable reference with exact signatures.

Python is part of the harness, not an extension. Every session has the python tool and a kernel, whatever extensions it enables, so an extension may rely on the kernel and on the Python bindings it and other extensions contribute. List in requires only the extensions whose features you use, and never python. See the harness base.

Extensions are compiled into Albedo. Adding a source file does not install it into an already running daemon, and the SDK is currently part of this repository rather than a separately versioned package.

The worked example: a session checklist #

The checklist extension keeps a per-session checklist. It is installed but not enabled by default. It makes one of each common contribution:

  • its own table, created when the extension is installed;
  • a Python binding, checklist.add, checklist.items, and checklist.check, backed by a host route;
  • a /checklist slash command;
  • a hook that holds a turn once while items are open;
  • a transfer adapter that carries its rows when a session moves to another daemon, and posts the moved session a note through the inbox;
  • a cleaner that deletes its rows with their session.

Besides the Gleam standard library and sqlight, it imports only the SDK, its own modules, albedo/transfer, and the albedo_bytes package. It passes every lint rule without an exception. Its files are:

File Owns
extension.gleam The declaration, the route, the command, and the hook
ledger.gleam The table, its schema, and every query
transfer.gleam The transfer adapter
checklist.py The model's Python binding
checklist_test.py One daemon scenario for each contribution
extensions.gleam The registration, outside the extension's directory
shelves.gleam The shelf and line clients show, outside the directory
api_docs_test.py The check that the model is told about every Python call

Read those files beside the sections below. The sections follow the order you would write them in.

Declare the extension #

Create src/albedo/harness/extensions/<name>/extension.gleam. An extension belongs to that one directory. Its constructor returns an extension.Extension:

pub fn extension() -> extension.Extension {
  extension.Extension(
    name: "checklist",
    description: "A per-session checklist the model keeps and is reminded of before it stops.",
    requires: [],
    plugins: [
      extension.ToolPlugin(instructions, [], ["checklist"], [
        #("checklist", fn(context: host.Context, method, arguments) {
          route(context.storage, context.session, method, arguments)
        }),
      ]),
      extension.CommandPlugin([add_command()]),
      extension.HookPlugin(remind),
      extension.TransferPlugin(transfer.adapter()),
      extension.CleanPlugin(ledger.forget),
    ],
    initialise: extension.storage_initialise(ledger.initialise),
    tables: ledger.tables,
    uses: uses,
  )
}

pub const uses = [capability.Inbox]

Keep the constructor free of I/O and of anything read from the environment. A sealed build caches the declarations it returns. Callbacks stored in plugins run later, at their host boundary.

Keep your own tables #

Declare the tables the extension creates in a pub const tables of string literals in a module under its directory, and pass that constant as tables. The albedo_table_ownership lint reads the constant, not the constructor call. Its SQL may name only those tables. Read a core table, such as sessions, only through a typed SDK operation that takes your storage handle. See extension storage.

The initializer is a fn(extension.Setup) -> Result(Nil, String). It runs inside the installation transaction, before any startup callback, every time the daemon starts. Make it idempotent, with CREATE TABLE IF NOT EXISTS and CREATE INDEX IF NOT EXISTS. Use extension.storage_initialise(initialise) for an initializer that needs only the handle, and extension.no_initialise when there are no tables. An extension whose tables refer to the inbox's calls setup.inbox.initialise(setup.storage) first, and declares Inbox.

The checklist's ledger.initialise creates checklist_items and an index on (session, done, id). Index every column a delete or a per-session read filters on. Session deletion runs the cleaner on the shared writer, where a full table scan holds up every other write. See foreign-key indexes.

Run SQL through sdk/storage, never sdk/store. Use storage.read for a read on a reader, storage.query for work on the writer, and storage.transaction inside a query for several statements that commit together. transaction takes a body whose error is a String. storage.transaction_with(db, failed, body) takes a body that fails with your own error type, and failed turns a failure to begin or commit into it. Any error of the body rolls the transaction back. ledger.add uses it to answer Full for a full checklist. In outline:

storage.query(store, fn(db) {
  use <- storage.transaction_with(db, Storage)
  use open <- result.try(count_open(db, session) |> result.map_error(Storage))
  use <- bool.guard(open >= max_open, Error(Full))
  insert(db, session, text) |> result.map_error(Storage)
})

Bound what each session can store, open and finished rows alike: a transfer carries every row, and the rows stay until the session is deleted. The checklist refuses a 51st open item and a text over 500 Unicode scalars. When an item is checked off, ledger.check deletes the session's oldest checked items past 50 in the same transaction, through the (session, done, id) index. Scalars are measured with albedo_bytes/scalars. The albedo_text_scalars lint refuses the grapheme functions of gleam/string, such as length, under src/.

To add a column later, call storage.add_columns(db, "table", [#(name, type)]) from the initializer, or add a MigrationPlugin. A transfer carries every column, and a daemon refuses a segment with a column its own table lacks, so a new column on a carried table also stops moves to older daemons. Bump the table's transfer version when the rows change meaning. A MigrationPlugin names itself and declares migration.Additive or migration.Breaking. An older daemon that lacks a breaking one refuses the home. See migrations.

Add a Python binding #

A Python binding is a module the kernel loads into every session that enables the extension. It has two halves.

  1. Write the module as priv/python/albedo_plugins/<module>.py. The kernel imports albedo_plugins.<module> for each name in a ToolPlugin's python_modules, then calls its setup(api). setup returns a dict of the names it adds to the model's namespace. The checklist module returns {"checklist": Checklist()}. Each method calls await host("checklist.add", {"text": text}), where host is api.host. albedo_api.load_plugins refuses a name another module already bound.
  2. Serve the calls with a host route, a #(namespace, handler) pair in the same ToolPlugin. A call whose method starts with namespace. reaches the handler, with a host.Context, the method, and the arguments as a Dynamic. Answer Ok(json), which the Python call returns, or Error(#(code, message)), which it raises as api.HostError with that message and a code attribute. Use rpc.serve to map a domain error to that pair and rpc.args to decode arguments.

The context's storage is the extension's handle, bound to the calling session, and session is that session's ID. Key every row by context.session, and check the session on every change, so one session cannot read or change another's rows by guessing an ID. ledger.check updates WHERE id=? AND session=? for this reason. A route that also needs the loaded workspace reads context.workspace. See host route context.

Tell the model about every call in the ToolPlugin's instructions. Name each one with its binding, as checklist.add(text), and name the error class setup returns. api_docs_test.py enables every installed extension on a real daemon. It fails for each public call in the kernel's namespace that the provider request never names, and for each shipped module with a setup that no extension loaded. It needs no list of plugins.

A route or module name that another enabled extension already contributes makes your extension inactive for the session, with a warning. Choose names after the extension.

Add a slash command #

CommandPlugin adds commands to the session's catalog. A command.Command chooses who may call it and what a call does:

  • model_callable: True adds a method to the Python commands object, such as await commands.greet("Ada"). Its run answers command.Data(json), which the call returns. That object comes from the commands extension, so list it in requires.
  • user_turn: True lets a person send the command as a session input. Its run answers command.Turn(display, text): the transcript shows display, and the model reads text as the person's message.
  • page: True, built with command.resource, names an HTTP resource operation that a ClientPlugin declares. See declare client operations.

A person reaches a command only through a user_turn input or a client operation. The daemon refuses an input that names a command without user_turn. It runs a user_turn command's run while it prepares the input, before it admits it, with a context that refuses every state change. So that run must not write anything, even to the extension's own tables.

The checklist's /checklist <item> is a user_turn command. It does not add the row itself. It submits a turn that asks the model to call checklist.add, and without an item it asks the model to review the list. Its model_callable is False, because the model has the Python binding.

An extension declares a command.Command(command.Invocation). Its run and skill_activation receive a command.Invocation: the session ID, a state callback that reaches session state through the session's actor, and the extension's handle in storage, bound to that session. The host builds it for each call, so read the handle there rather than capturing one.

A command receives no inbox, because commands rarely need one. A command acts on its own session through state, or by answering command.Turn. A user_turn command's run must not write at all. A call from the model that posts to a session belongs in a host route, whose host.Context.inbox the host gates on Inbox, as the run extension's jobs route posts a notice of a job result nobody read. A command that must post anyway builds its closure in a StartupPlugin, from the extension.Startup inbox, and returns it as extension.StartedCommand.

Add a hook #

A hook answers turn events. Declare it with HookPlugin:

extension.HookPlugin(remind)

The hook is a fn(hook.Context, hook.Event) -> hook.Verdict. Its context carries the session, the workspace the session loaded, and the extension's handle, bound to that session. Read them there. Do not capture a handle in the hook, because the composition is copied between processes. Set hook on a managed contribution instead only when the hook needs something preparation supplies, as the agents extension's hook needs Preparation.letters.

The checklist's remind answers hook.Stopping(held: 0, ..), the model ending a turn that no hook has held yet. With items open it answers a block that lists them. The loop commits the reason as input the model reads and runs another step. Every other event, and a turn hooks already held, gets hook.proceed, so the reminder cannot loop. held is the loop's count of holds in this turn by every hook, kept in memory only. If another hook holds the turn first, the checklist does not remind in that turn. A read failure becomes a note the person sees, and the turn proceeds.

A hook runs in the turn, so keep it to a bounded indexed read. See the hook reference for every event, the merge rules, and what the model and the person see.

Declare capabilities #

Some context fields carry authority beyond the extension's own session data, such as inbox, control, or session_state. List the ones the extension's code reads in a pub const uses of capability.Capability constructors, and pass it as uses. The host replaces each capability the extension does not declare with a stand-in that refuses. Pass uses: [] when the extension reads none.

The albedo_extension_capability lint flags a read that uses does not cover, and a declared capability the extension never reads. The checklist reads only inbox on the transfer.Activation its adapter receives, so it declares [capability.Inbox]. The storage handle, session ID, and workspace are plain data and need no declaration. See extension capabilities.

Carry rows when a session moves #

A session can move to another daemon. A TransferPlugin declares which of the extension's rows move with it. Without one, the rows stay behind. Build the adapter with transfer.declare(owner, tables, exclusions) from albedo/transfer, and set the fields you need with a record update:

transfer.Adapter(
  ..transfer.declare(
    "checklist",
    [Table("checklist_items", 1, "session IN (SELECT id FROM transfer_scope)")],
    [],
  ),
  shifts: [Shift("checklist_items", "id", RowId("checklist_items"))],
  activate:,
)
  • A Table's filter is a SQL condition over the table's rows. It may read the transfer_scope(id) view of the moving sessions, transfer_workspace(path), and transfer_images(hash). The version is checked on import. A target refuses a segment whose version it does not know. The target imports owners and each owner's tables in the order they are declared, so list a table after the tables its rows refer to.
  • An INTEGER PRIMARY KEY collides with the target's own rows. Declare a Shift with RowId(table) for it, and for every column that holds one of those numbers, so the target renumbers them after its own. Without it, the import fails with transfer_collision.
  • workspaces lists #(table, column) cells that hold a workspace path. The import rewrites them for the destination.
  • activate runs on the target as the import goes live, inside the activation's transaction. It receives a transfer.Activation, with the imported sessions, the extension's storage handle, and the daemon's inbox, and the transaction's connection. Post with inbox.post_in on that connection, so the input commits with the import. The checklist posts each imported session with open items a note, keyed input.Latest, so a retried activation replaces a note still waiting instead of adding another. It ignores a full inbox, because a courtesy note must not fail a move.

List a table that should not move in exclusions, with the reason. The transfer code does not read them, but transfer_coverage_test.py does: every table the daemon creates must be some owner's Table or some owner's Excluded, exactly once, so a new table cannot silently stay behind when its session moves. See moving a session.

Delete rows with the session #

A CleanPlugin deletes the extension's rows for a session inside the session delete's transaction. Every installed extension's cleaner runs, enabled for the session or not. The checklist's is:

pub fn forget(db: sqlight.Connection, session: String) -> Result(Nil, String) {
  storage.forget_session(db, ["checklist_items"], session)
}

storage.forget_session deletes the rows whose session column names the session. Name the tables as string literals so the lint can check them. An error rolls back the deletion.

Register the extension #

Import the constructor in src/albedo/harness/extensions.gleam and add it to the first list in declared(). Add its name to the second list, default_enabled, only if every session should have it. The checklist is installed but not enabled by default: its hook changes how every turn ends. A person enables it per session through /extensions, or for every session in the global defaults.

Clients show each built-in extension on a shelf with a one-line summary from shelves.gleam. An extension without an entry lands on a shelf guessed from its plugins, which for most features is reach. Add an entry for a built-in feature.

Compile and lint:

nix develop -c gleam check
nix develop -c test/gleam-lint.sh

Test it through the daemon #

checklist_test.py drives each contribution through the real daemon with a scripted provider: a cell that calls the binding, the command as a session input, the hook's single hold, a move to a peer daemon whose own items already use the moved ids, and deletion. The scripted model answers each user message with a Python cell chosen by a marker in the message. Enable the extension for a test's session with app.patch_session(session, {"selection": {"extensions": {"checklist": True}}}).

The shared harness supplies the daemons. Run a file from the repository root with a compiled snapshot and CLI:

nix develop
scratch=$(mktemp -d /tmp/albedo-checklist.XXXXXX)
ALBEDO_TEST_DAEMON=$(test/snapshot-daemon.sh "$scratch/daemon")
export ALBEDO_TEST_DAEMON ALBEDO_NO_BROWSER=1
go -C cli build -o bin/albedo ./cmd/albedo
python3 test/e2e/run.py checklist_test.py
test/reap-daemons.sh "$scratch"
exit

Rebuild the snapshot after every Gleam or Python change. Run the cleanup command even if the scenario fails. Name the file after the extension: ./test.sh selects an end-to-end file whose name shares a word with a changed path. Check that each scenario fails when its contribution is removed. Follow the test policy.

Read settings #

An extension reads its own section of extensions.json through sdk/settings. Declare the section once, register it with SettingsPlugin, and load it when a callback runs:

fn configuration() -> settings.Section(String) {
  let prefix =
    settings.text("prefix", "Greeting prefix", "Text before a name.", "Hello")
  settings.section(
    "greeting",
    "Greeting",
    settings.Tools,
    [settings.field(prefix)],
    {
      use value <- settings.value(prefix)
      decode.success(value)
    },
    fn(value) { Ok(value) },
  )
}

// In the plugin list:
extension.SettingsPlugin(settings.declaration(configuration()))

// In a route, command, or hook:
use prefix <- result.try(settings.load_section(configuration()))

settings.load_section validates the saved section and answers the default when the section or key is missing. Invalid saved settings answer an error instead of being replaced. Load settings in callbacks, not in the constructor, so a sealed build does not freeze one home's settings. A test that writes settings before the daemon boots uses prepare=lambda app: app.write_extensions(...) and is @exclusive. See persisted settings.

Choose the contributions #

Choose plugin types from the current Plugin definition in extension.gleam instead of copying an old inventory. Use ToolPlugin for static instructions, tools, Python modules, and host routes, CommandPlugin for static commands, and HookPlugin for a static hook. Use ManagedPlugin when a contribution must be prepared before use. Use OwnedManagedPlugin when allocated resources must survive preparation handoff safely. If a route needs only the values on host.Context, keep the route in ToolPlugin and do not add a managed contribution that only returns routes.

Keep schema, data migrations, deletion cleanup, and transfer behavior with the extension that owns the data. See migrations and extension lifecycle rules.

Connect host routes #

Use host.Route for Python-to-host calls. A route receives host.Context, a method string, and dynamically typed arguments. It returns JSON or an error pair. The context supplies the extension's storage handle, the session ID, the loaded workspace, and the providers from the active composition. Use its supplied callbacks for progress and captured user-command output. The providers, the captured output, and the other authority-bearing fields are capabilities the route's extension must declare in uses. Other session operations and durable observations use the typed SDK APIs.

Declare routes that need only context values in ToolPlugin. Follow the checklist route, which passes context.storage and context.session to its handler, and the work route, which also passes context.workspace. Use this loaded workspace instead of querying session settings or reading the process working directory. See the context snapshot contract.

Resolve providers from the supplied context. Do not capture the installation registry as a substitute for the active session composition. Keep HTTP service admission separate from session host routes. The extension HTTP contract describes caller permissions, conditional edits, and notifications.

Declare client operations #

Use ClientPlugin for human commands that read or change an HTTP resource. Start with client_api.operation_defaults, then override the bindings and result constraints the operation needs. Reuse text_field, integer_field, choice_field, and form_body when their defaults fit. Preserve custom labels, defaults, validators, and success statuses. Use a FieldKind variant when constructing a field directly. Build resource-page actions with client_api.actions and propagate its error before encoding the page. See client declaration checks.

Follow the work declaration and its page actions. Keep model host routes separate from HTTP operations. Read client API ownership for transport, recovery, and authorization boundaries.

Prepare session contributions #

Follow the schedule extension for a managed contribution that builds a command from the supplied store and session ID. Start from extension.empty() and fill the contribution fields you need.

For an OwnedManagedPlugin whose worker only observes session events, use worker.observer. Pass the preparation owner from the extension.Preparation it receives, the worker loop, the message constructor that wraps a session handle and event, and the startup wait in milliseconds. Wrap its owned result in Some and map its Error(Nil) to the extension startup message:

worker.observer(preparation.owner, run, Observed, 5000)
|> result.map(Some)
|> result.replace_error("the worker did not start")

observer builds the managed contribution from extension.empty(). Use worker.start when other callbacks need the worker inbox, then attach its lifecycle with extension.owned(contribution, started.adopt, started.close). Return Ok(None) only when preparation retains no resource or contribution. See the owned worker helper reference.

Keep event observers short. They run inside the session actor, so send work to its owner and return instead of blocking. Run command handlers outside the actor whose state operations they call. See plugin contracts and commands.

Give allocated resources an explicit cleanup owner. Make close callbacks idempotent. For owned preparation, establish lifecycle monitors before readiness and complete adoption before publication. Preserve saved selection when applying it fails. The shared lifecycle rules describe the required failure boundaries.

Report hooks #

Add a hook shows a hook that steers turns. A hook that only observes work must send the event to its owner and return hook.proceed promptly.

For calls inside Python cells, use albedo_plugins.hook. Use reported(name, fields) when method parameters map to hook arguments, or around(name, arguments, run) for an asynchronous callback. Execute with the rewritten arguments, not the original arguments. Use before and after when completion must be reported separately.

For command-hook configuration, inspect the parser in albedo_hook_command.py. The hook reference describes command execution and failure behavior. Command hooks are not a fail-closed security boundary.

Expose a JSON service #

For a buffered JSON API authenticated with the daemon token, import harness/sdk/service as http_service. Add its result to the extension's plugin list:

extension.ServicePlugin(http_service.json(65_536, service.dispatch))

Choose a body limit for the service's operations. Implement service.dispatch(daemon, path, request) with a Result(Response(mist.ResponseData), http_api.Failure) return value. Keep route matching, query and body decoding, and domain error mapping there. Follow the work service and reuse the decoders in harness/sdk/http_api.

Read the HTTP service contract for negotiation and admission behavior. Use extension.Service directly when routes need streaming, non-JSON responses, different authentication, or different body limits per route.

Register and check the extension #

Register the constructor as register the extension describes. Keep discovery read-only and ensure failed preparation leaves no callable contributions from the rejected bundle.

Exercise the externally visible change through the shared daemon harness. Cover the relevant activation, failure, and cleanup paths. Update the canonical feature page with the behavior, limits, and guidance callers need, then follow the repository checks.