small gleam coding and (not yet) persistent agent daemon with a detachable cli forked from okami.mom/albedo
2.1 kB · 45 lines
Markdown
at main

Computer use #

The cua extension exposes native desktop automation to the Python kernel through cua-driver. Its extension declaration adds a module to the kernel every session has. The Python binding owns driver communication.

Driver and lifecycle #

cua.configure(executable=..., socket=...) can override driver settings. The binding also reads environment variables and the cua section of extensions.json, and resolves the executable when needed. It starts a persistent driver subprocess for the kernel. await cua.close() ends the driver session. Driver errors raise CuaError. Startup and calls have timeouts in _DriverClient.

Windows and observations #

await cua.apps() lists applications and await cua.windows(pid=None) returns window handles. await cua.window(...) resolves a window. await win.observe() captures an observation and attaches any screenshot returned by the driver. render, find, and one inspect its elements. await win.zoom(...) requests a cropped image. Tool-output images can be elided from later contexts. See Observation and Window and the image sweep.

Actions and permissions #

A window handle exposes actions including click, type_text, press_key, focus, and kill. The corresponding cua methods accept a target. The binding defaults window actions to background delivery and desktop actions to foreground delivery. Driver refusals propagate as CuaError. permissions, health, clipboard operations, recording methods, tools, and the raw call escape hatch are defined in cua.py. These operations act on the host desktop and clipboard, so their effects extend beyond the kernel.