Integration Test Plan #

Date: 2026-02-17 Scope: pytest -m integration for owi-cli

Current Inventory #

• Total integration tests: 26
• Collected with:
  • uv run pytest tests/ -m integration --collect-only -q
• Current distribution:
  • CLI integration/smoke: 7
  • Core repository integration: 3
  • fsspec integration: 14
  • DB executor integration: 2

Authentication Strategy (Reuse Existing .owi Token) #

Goal: avoid re-login prompts and reuse the token already available on the machine.

1. Verify token exists:
   • ls -l ~/.owi/.tokens/refresh_token
2. Export token for subprocess-based tests:
   • export PY4LEXIS_TOKEN="$(cat ~/.owi/.tokens/refresh_token)"
3. Ensure default OWI path is used (or set explicitly):
   • export OWS_OWI_PATH="$HOME/.owi"
4. Optional quick auth sanity check before running integration:
   • uv run owi config version
   • uv run owi remote ls all:latest --limit 1

Notes:

• OWIlixManager reads token from PY4LEXIS_TOKEN first, then ~/.owi/.tokens/refresh_token.
• Keep tokens out of logs and shell history where possible.

Execution Plan #

Run integration in phases so failures are isolated and reruns are fast.

Phase 0: Preflight (fast) #

• uv run pytest tests/owilix/cli/test_smoke.py -m "not integration" -q
• uv run pytest tests/owilix/core/fsspec/test_core_fsspec_unit.py -q

Phase 1: Fast Integration Smoke (target 2-5 min) #

• uv run pytest -m integration tests/owilix/cli/test_smoke.py tests/owilix/core/repository/test_integration.py::TestIntegration::test_config_version -v -s

Phase 2: API/Auth Integration (target 5-10 min) #

• uv run pytest -m integration tests/owilix/core/repository/test_integration.py::TestIntegration::test_remote_ls tests/owilix/cli/test_smoke.py::TestCLIIntegration::test_remote_ls_latest -v -s

Phase 3: fsspec Integration (target 10-20 min) #

• uv run pytest -m integration tests/owilix/core/fsspec/test_core_fsspec_integration.py -v -s

Phase 4: Heavy/Performance Integration (target 15-30+ min) #

• uv run pytest -m integration tests/owilix/core/fsspec/test_genquery_find.py::TestGenQueryFind::test_find_large_dataset_performance -v -s
• uv run pytest -m integration tests/owilix/core/db/test_executors.py -v -s

Optional Full Run #

• uv run pytest tests/ -m integration -v -s

Speed Optimization Strategy #

Keep integration real, but reduce redundant remote work.

Immediate (No Code Changes) #

• Run targeted subsets before full suite; only run full integration before release tags.
• Prefer smaller datasets for default integration path; reserve large dataset tests for nightly/manual.
• Use --maxfail=1 in pre-release gate to fail fast:
  • uv run pytest tests/ -m integration -v -s --maxfail=1
• Keep PY4LEXIS_TOKEN exported once per shell session.

Near-Term Test Refactors (Recommended) #

1. Shared authenticated session fixture (session scope):

   • Reuse one Lexis/OWI session across tests/modules.
   • Avoid repeated token reads and repeated auth handshake.

2. Shared dataset fixture registry:

   • Centralize known test dataset IDs in tests/conftest.py.
   • Distinguish:
     • SMALL_DATASET_ID for functional integration
     • LARGE_DATASET_ID for performance-only checks

3. Split marker taxonomy:

   • Keep integration for functional E2E.
   • Add integration_heavy for expensive tests.
   • Add integration_perf for throughput/perf assertions.
   • Then run release gate with:
     • -m "integration and not integration_heavy and not integration_perf"

4. Reduce redundant subprocess invocations:

   • Current CLI integration repeatedly executes uv run owi ....
   • For many assertion variants, use one command + multiple assertions on parsed output where feasible.

5. Add explicit test time budgets:

   • Use pytest-timeout or local timeout wrappers for known long-running tests.
   • Mark tests with expected runtime in docstrings.

Proposed Release Gates #

Gate A (required on merge to main) #

• Non-integration + selected fast integration:
  • uv run pytest tests/ -m "not integration" -q
  • uv run pytest -m integration tests/owilix/cli/test_smoke.py tests/owilix/core/repository/test_integration.py::TestIntegration::test_remote_ls -v -s

Gate B (required before tag/release) #

• Full functional integration excluding heavy perf:
  • uv run pytest tests/ -m "integration and not integration_heavy and not integration_perf" -v -s

Gate C (nightly/manual) #

• Heavy/performance integration:
  • uv run pytest tests/ -m "integration_heavy or integration_perf" -v -s

Risks and Controls #

• Token expiry during run:
  • Control: preflight remote command and fail early.
• Remote service instability:
  • Control: retries in fixtures for setup calls; avoid strict timing asserts on shared infra.
• Dataset drift:
  • Control: pin canonical dataset IDs and validate existence in a dedicated preflight test.

Backlog Items to Implement #

• Add tests/conftest.py integration fixtures for token/session/datasets.
• Introduce integration_heavy and integration_perf markers in pyproject.toml.
• Refactor tests/owilix/core/fsspec/test_genquery_find.py token loading (~/tmp/refresh_token.txt) to use shared OWI token path.
• Update docs/testing.md to include phased integration execution and gating.