small gleam coding and (not yet) persistent agent daemon with a detachable cli forked from okami.mom/albedo
albedo robot-docs imagegen.md
7.5 kB · 146 lines
Markdown
at main

Image generation #

The imagegen extension exposes a shared Python interface for generating and editing raster images. Driver extensions contribute ImagegenPlugin providers. Codex supplies the built-in driver through ChatGPT sign-in. The shared interface and the driver can be enabled independently.

extensions/imagegen/extension.gleam owns host routes, driver selection, reference resolution, and response bounds. imagegen.py owns local reference reads and output publication. See plugins for the driver contract.

Select a driver #

await imagegen.providers() returns the loaded drivers' names and labels. await imagegen.generate(prompt, provider=...) selects one by name. Omitting provider works only when exactly one driver is active. An absent or ambiguous selection fails without trying another driver. Reloading extensions updates the drivers captured for later host calls.

Driver discovery does not establish that an account is signed in or that the upstream service accepts its request. Codex selects credentials in the daemon through the existing account selection interface. Its configuration owns the image model setting separately from the reasoning model. Its configured default is gpt-image-2; this is not a model-availability guarantee. The production endpoint is fixed to https://chatgpt.com/backend-api/codex, and the transport refuses other destinations and does not follow redirects. Do not forward subscription credentials to an arbitrary image service.

Generate and inspect #

The Python call is:

result = await imagegen.generate(
    "A cut-paper illustration of a fox in snow",
    transparent_background=True,
    output="output/imagegen/fox.png",
)
show_image(result.path)

The result contains path, mime_type, width, height, bytes, request_id, and generation_id. Without an output path, the binding chooses output/imagegen/generated-<uuid>.png in the kernel's working directory. Output must be PNG and at most 5 MiB. Existing files and dangling output symlinks are refused. Publication uses a same-directory hard link to avoid replacing a destination created during generation.

Generating a file does not attach it to the conversation. show_image is a separate operation subject to the active reasoning model's viewing limits. Read the shipped image generation guidance for reference selection, prompting, inspection, and revisions. That skill grants no additional tools or permission to spend quota.

Reference images #

Pass at most five references to imagegen.generate(..., references=[...]). Local PNG, JPEG, and WebP files must total at most 5 MiB. The binding reads only regular files, checks the image bytes, and sends inline data to the daemon.

await imagegen.references(after=0, limit=50) discovers uploaded and tool-output images in the current session. Each entry includes a reference identifier, transcript sequence, image index, MIME type, width, and height. Pass the returned identifier to generate; do not invent identifiers or reuse another session's. The daemon resolves each identifier against the calling session and checks its stored image bytes. Each transcript reference is limited to 5 MiB.

Pages scan transcript rows, including rows without images. Continue with next_after while has_more is true, even when images is empty. The page limit is between 1 and 100 rows. Local references and transcript identifiers can be mixed, and their order is preserved.

Cancellation and recovery #

Generation can consume quota or incur provider charges. Do not automatically retry an interrupted, cancelled, or uncertain call, and do not change drivers as a fallback. Re-running the Python expression creates a new call identity and can submit another request. The admission map is not durable idempotency.

imagegen/calls owns bounded admission and cancellable workers. The kernel's existing host-call ledger owns replay decisions across reattachment. A worker ending locally cannot undo an already submitted upstream generation.

The extension's StartupPlugin starts one admission for the daemon's store and returns the OwnedManagedPlugin whose imagegen routes close over that opaque calls.Admission. Nothing finds the admission by name. Closing the store stops it. Each prepared composition takes a calls.Lease on its session, which the runtime adopts and closes with the composition.

imagegen.begin admits a call ID for the calling session and parks a coordinator for it. The admission holds at most eight calls for one session and 512 in total across the daemon. A begin for a session that no lease holds is refused. A parked coordinator gives up its slot after one minute. imagegen.generate claims the parked coordinator once as an opaque calls.Call and runs the driver in that coordinator's linked worker. A missing, repeated, cancelled, or expired begin is refused, and the call must not be replayed. The caller, the coordinator, or the admission ending stops the worker and reports an uncertain outcome. A request has 600 seconds before the coordinator reports the deadline.

A call belongs to its session, not to the composition that admitted it. An extension reload that keeps image generation prepares the replacement composition, and its lease, before it closes the previous one. A running call therefore survives the reload, including one awaited by a backgrounded cell. Cancellation through the replacement composition's routes reaches it.

When a session's last lease ends, the admission stops that session's calls with the uncertain "admission owner ended" outcome. Closing that lease returns once their coordinators have stopped, waiting at most five seconds for each. Removing image generation from a session and deleting the session both end its last lease. Removing image generation also changes the kernel's Python modules, so that reload is refused while a cell is running and otherwise replaces the kernel. Stopping the daemon closes the store, which stops the admission, and every coordinator then stops its worker with the same outcome.

Successful cancellation waits for the coordinator and request worker to stop before releasing their slot. It does not establish whether the upstream service accepted the request before the worker stopped.

The binding probes the destination before contacting the provider and validates the returned image before publication. A later storage error can leave complete bytes in a .imagegen-*.png temporary file. Inspect the reported recovery path and the destination before generating again. A retained copy does not establish that its bytes reached durable storage. Partial output must not be reported as a complete image.

Host and driver refusals raise ImagegenError. Local validation and filesystem failures raise Python exceptions. Preserve request identifiers when reporting an uncertain outcome. Provider request accounting has a separate limitation for these calls.