Computer use #
The cua extension exposes native desktop automation to the Python kernel
through cua-driver. Its
extension declaration
adds a module to the kernel every session has. The
Python binding owns driver
communication.
Driver and lifecycle #
cua.configure(executable=..., socket=...) can override driver settings. The
binding also reads environment variables and the cua section of
extensions.json, and resolves the executable when needed. It starts a
persistent driver subprocess for the kernel. await cua.close() ends the driver
session. Driver errors raise CuaError. Startup and calls have timeouts in
_DriverClient.
Windows and observations #
await cua.apps() lists applications and await cua.windows(pid=None) returns
window handles. await cua.window(...) resolves a window. await win.observe()
captures an observation and attaches any screenshot returned by the driver.
render, find, and one inspect its elements. await win.zoom(...) requests
a cropped image. Tool-output images can be elided from later contexts. See
Observation and Window and the
image sweep.
Actions and permissions #
A window handle exposes actions including click, type_text, press_key,
focus, and kill. The corresponding cua methods accept a target. The
binding defaults window actions to background delivery and desktop actions to
foreground delivery. Driver refusals propagate as CuaError. permissions,
health, clipboard operations, recording methods, tools, and the raw call
escape hatch are defined in cua.py.
These operations act on the host desktop and clipboard, so their effects extend
beyond the kernel.