owi-cli
Command line interface for accessing the Open Web Index shards and slices.
Star
Fork
Watch
owi-cli docs AGENTS.md
5.8 kB · 164 lines
Markdown
at main

Agent Development Guide #

Streamlined instructions for AI agents working on the owilix codebase.

Agent Conventions (must follow) #

  • Always read docs/architecture.md before generating code
  • Do not modify files outside the active feature scope unless explicitly requested
  • Never hardcode secrets — use environment variables or config files
  • Prefer readability and modular changes over premature optimization
  • Ask before introducing new dependencies
  • Always run uv run pytest tests/ before considering a task complete
  • Use type hints and follow the coding guidelines in docs/CODING_GUIDELINES.md

Architecture Quick Reference #

Entry point and CLI wiring #

  • Entry: owilix/cli/__init__.py → cli_main() → Typer app
  • 7 sub-apps: remote, local, query, config, admin, plugin, batch (each a separate Typer app)
  • The Typer main callback calls create_context() which populates ctx.obj with a CLIContext
  • CLIContext (cli/_common/context.py) holds owi (OWIlixManager), console, format settings, etc.

Task wiring pattern #

CLI commands delegate to task functions via execute_command:

from owilix.cli._common.executor import execute_command
from owilix.core.tasks.remote import remote_pull

result = execute_command(ctx.obj, remote_pull, ctx.obj.owi, specifier, ...)
  • Task functions live in owilix/core/tasks/ — they are plain Python, accept a manager, return CommandResult
  • execute_command / CommandExecutor handles timing, exception categorisation, and log writing
  • Never call LexisRepository or OWILexisDatasetAPI directly from CLI code

Manager and repositories #

  • ctx.obj.owi is OWIlixManager (core/manager/manager.py)
  • manager.remote_data → AggregatedRepository (fans out to all configured LEXIS instances in parallel)
  • manager.local → LocalRepository
  • manager.session → OWILIXSession (lazy-loaded, wraps py4lexis.LexisSession)
  • Specifiers (lrz:latest, it4i:2024-03-01#30) are parsed by manager.parse_specifier()

Multi-datacenter routing #

  • Each entry in owilix.cfg under selected_remote is a separate LexisRepository instance
  • AggregatedRepository.list() dispatches to all repos in parallel, deduplicates by internalID
  • The best-performing repo wins when a dataset appears in multiple datacenters
  • A datacenter filter in the specifier restricts which repos are queried

Server context #

The LEXIS/OWS server is in active development and outside this team's control.

  • Treat all server responses defensively: validate status codes, never assume 2xx = success
  • Transient failures, unexpected status codes, and malformed error bodies are normal operating conditions — handle them gracefully with user-facing messages
  • Do not implement client-side workarounds that depend on undocumented server behaviour

Known fragilities (do not replicate) #

  • session.get_irods_client() calls the private mangled method _iRODS__check_access_token() — this is a known weakness, now guarded with try/except AttributeError (H4); do not copy this pattern
  • The legacy owilix/cmd/ package has been removed. Legacy command classes (BaseCommand, SubCommand) live in owilix/compat/ for plugin backward compatibility. New code should not import from owilix.compat — use owilix.core.types for CommandResult and owilix.core.manager.ui for UI utilities.
  • See docs/backlog.md for the full list of tracked issues

When implementing a feature #

  1. Read docs/architecture.md first
  2. Review current branch plan (if exists) in docs/branch/<branch-name>.md
  3. Plan the change (bullet points)
  4. Implement minimal working version
  5. Add or update tests in tests/ using pytest
  6. Run verification: uv run pytest tests/
  7. Summarize changes
  8. Store analysis documents and reports under docs/review/ if needed

When fixing a bug #

  1. Reproduce the issue
  2. Identify root cause
  3. Implement the smallest safe fix
  4. Add a regression test
  5. Run verification: uv run pytest tests/
  6. Summarize the fix

Branch Workflow #

Simple tasks (on current branch) #

  • Make changes directly
  • Document stable commits
  • Run tests: uv run pytest tests/

Complex features (new branch) #

  • Create feature branch: git checkout -b feature/<name>
  • Create branch document: cp docs/branch/_template.md docs/branch/<name>.md
  • Update branch document with:
    • Past: What's been completed
    • Present: Current work
    • Future: Planned implementation steps
  • Keep committing stable progress

Definition of Done #

  • All tests pass: uv run pytest tests/
  • No unrelated files were changed
  • No secrets were introduced
  • Changes align with architecture in docs/architecture.md
  • Code follows docs/CODING_GUIDELINES.md
  • Branch document updated (if applicable)

Key Documentation #

Document Purpose
docs/architecture.md Package structure, component diagram, command→repo→LEXIS flow, multi-datacenter routing, auth
docs/CODING_GUIDELINES.md Coding standards and conventions
docs/backlog.md Known issues, hardening items, and technical debt
docs/branch/_template.md Template for branch documentation

Testing #

Run tests with:

uv run pytest tests/

Test markers:

  • @pytest.mark.integration - Requires network, may be slow
  • @pytest.mark.slow - Long-running tests

Run subset:

# Skip slow tests
uv run pytest tests/ -m "not slow"

# Run only integration tests
uv run pytest tests/ -m integration

Environment Assumptions #

  • User is logged in with valid refresh token
  • Remote servers may be slow or unavailable
  • Use --limit flags to reduce data size in testing