broker/.ciagent/ARCHITECTURE.md

ARCHITECTURE — CoreCI Broker

Overview

One Open WebUI workspace tool, implemented as a single Python file (coreci_broker.py), embedding a stdlib HTTP server (ThreadingHTTPServer) bound to BROKER_BIND (default 127.0.0.1:8765). It serves a JSON API, a four-page admin web UI (static HTML/CSS/JS from coreci_broker_ui/), and a public static client module. SQLite (broker.db, mode 0600) is the source of truth; an append-only JSONL file (broker.audit.log, mode 0600) records every read and denied attempt. The broker is the trust root with no upstream auth; it reads the creds-coreci-coo KB through a pluggable adapter and never writes to it.

Two runtime modes share the same file: (a) Open WebUI workspace tool — a Tools class starts the server in a daemon thread at construction (__init__), behind a once-per-process idempotency guard (the OWUI loader has no activate() hook and re-execs modules on edit; paths resolve from env, never __file__); (b) standalone — python3 coreci_broker.py for development, testing, and direct operation.

Components

Name	Description	Boundaries	Depends on
Config bootstrap	Reads/writes the broker .env (spec §5.1 contract + documented additions); first-boot generation of BROKER_INSTANCE_PEPPER and BROKER_ADMIN_PASS (printed to the startup log once, written to .env and SQLite)	Owns the .env file and the broker_config table; never prints secrets after first boot	SQLite store
SQLite store	Tables: broker_config, app_registrations, user_allowlist, agent_registrations, admin_sessions + indexes (spec §5.2); mode 0600	Sole writer of broker.db; all SQL lives here	—
Audit writer/reader	Appends one JSON line per action (register/read/denied/login/logout/add_app/remove_app/add_user/remove_user); filtered query endpoint (since/until/user_id/uuid/action/limit)	Owns broker.audit.log; append-only; no rotation in v0.1	—
Hasher	Dual-mode slow hash: bcrypt when importable, else stdlib hashlib.scrypt; stored hashes format-prefixed ($2b$… / scrypt$…) so verification is format-aware	The only component with an optional import	—
Token engine	Server-side derivation: token = base64(sha256(pepper:UUID:user_id:agent_id:','.join(sorted(tags))))	Pure function of (pepper, inputs); never stores plaintext tokens	Hasher, Config
HTTP server + router	stdlib threaded HTTP; JSON body/param handling; {"error, message} contract with 400/401/403/404/409/500; static file serving (UI + client module)	No business logic; routes to admin/tool planes	All handlers
Admin plane	Session auth (cookie broker_session + HTTP Basic); login/logout/whoami; app_registrations CRUD (add/revoke/reinstate); user_allowlist CRUD; audit query	Admin-only; audited	Store, Audit
Tool auth plane	POST /api/broker/agents/register (allowlist gates, idempotent re-derive, TTL, pepper rule); App-auth verification (Authorization: App <UUID> <token> — UUID-scoped slow-hash scan + in-memory verified-token cache); GET /agents/<id>; state GET/PUT with identity enforcement	Register is callable without a valid token (gates are the allowlists); the cache is never authoritative	Store, Token engine, Hasher
Secrets plane	GET /api/broker/secrets/<path>; scope check path == "coreci/" + agent_id; creds-source adapter (FileCredsSource default | OwuiApiCredsSource)	The only reader of creds-coreci-coo; read-only	Tool auth plane, Creds adapter
Agent-state store	Backs GET/PUT /api/broker/agents/<id>/state; toggle BROKER_AGENT_STATE_BACKEND=internal (rows in broker.db) | owui (real per-agent KBs via OWUI API)	Broker-owned; tool-facing; identity-checked	Tool auth plane
Static UI	Four pages (login, app registrations, user allowlist, audit) + dashboard + persistent nav; server-rendered HTML, no build step	Read-only against the admin API	Admin plane
Client module	coreci_broker_client.py — dependency-free (stdlib urllib) auth client: ensure-registered, state get/put, secret read with 401→re-register→retry-once; served unauthenticated at /static/	Contains no secrets; importable by any OWUI tool	Broker API (external)
Reference tools	reference_tools/ — OWUI tool classes showcasing auth-first-then-use (e.g., proxmox reference: register → fetch coreci/proxmox → downstream call, safe dry-run default)	Templates for tool authors; not wired to live infra in tests	Client module

DataFlow

1. Admin: CEO logs in via /ui/login (or HTTP Basic) → session row in admin_sessions → manages allowlists via UI → every admin action audited.
2. Registration: tool calls POST /api/broker/agents/register with {agent_id, app_registration_uuid, user_id, tags} → broker checks app_registrations (active) and user_allowlist (active) → 403 on unknown/revoked → broker derives the token from the pepper + inputs → stores slow-hash in agent_registrations, sets TTL expiry → returns token (+ pepper when the row is created or the caller presented no valid token) → tool persists via PUT /agents/<id>/state.
3. Secret read: tool calls GET /api/broker/secrets/coreci/<agent_id> with Authorization: App <UUID> <token> → broker parses UUID, scans that tool's registration rows with the slow hash (in-memory cache after first verify) → 401 on failure → scope check path == coreci/<agent_id> → 403 on mismatch → creds adapter reads creds-coreci-coo → audit line (ok/denied/error) → JSON response.
4. Expiry: after BROKER_TOKEN_TTL the token verifies but the row is expired → 401 → tool re-registers (step 2) → new token overwrites token_hash → audit register.

BuildOrder

1. Config bootstrap + SQLite store + audit writer (P1)
2. Hasher + token engine (P1)
3. HTTP server + router + error contract + OWUI/standalone mounts (P1)
4. Admin plane + audit query + static UI (P2)
5. Tool auth plane + agent-state store (P3)
6. Secrets plane + creds adapter + client module + reference tools (P4)
7. Acceptance matrix, security review, README (P5)

Notes

• The in-memory verified-token cache maps (uuid, token) -> registration row; it is populated on first successful slow-hash verification and cleared on restart. SQLite remains authoritative; cache entries are re-validated against row status/expiry on every hit.
• Creds adapter contract: lookup by path string (coreci/<agent_id>) → JSON object. FileCredsSource reads a single JSON document mapping paths to secret objects (default for dev/tests, BROKER_CREDS_FILE). OwuiApiCredsSource calls the Open WebUI HTTP API with BROKER_OWUI_BASE_URL + BROKER_OWUI_API_KEY and is exercised in production; its exact KB-file layout is resolved in P4 research.
• This file is the init-time snapshot; it is refined at phase boundaries per ci-files discipline.

Distribution (v0.2 — cli-packaging)

v0.2 adds a distribution layer around the unchanged v0.1 broker: no broker-line changes (run_standalone, boot, load_config, _parse_env_file are consumed as-is). Three new components sit outside coreci_broker.py and are exercised by the same stdlib-unittest harness (in-process stub servers on ephemeral ports; subprocess boot tests against the built artifact).

Components (v0.2)

Name	Description	Boundaries	Depends on
CLI artifact build (scripts/build_cli.py + cli/__main__.py)	Stages coreci_broker.py + coreci_broker_client.py + coreci_broker_ui/ + the boot shim + a generated _cli_version.py, builds coreci-broker.pyz with stdlib zipapp (--compress, shebang #!/usr/bin/env python3), then a deterministic normalization pass (sorted entries, fixed timestamps) for byte-reproducible output; emits dist/coreci-broker-vX.Y.Z.pyz + sha256	Produces the artifact only; never publishes; owns no state; the shim materializes bundled UI/static assets into <state>/.coreci_broker_assets/<version>/{ui,static} at boot (temp+os.replace, overwrite-per-boot) and sets BROKER_UI_ROOT/BROKER_STATIC_ROOT only when neither process env nor the user .env defines them; --version is answered from the generated _cli_version.py before any broker import	Broker module surface (run_standalone, _parse_env_file) — import-only, zero edits
Release-asset pipeline (scripts/release.py + scripts/_env_secrets.py)	Builds release notes (milestone summary, phases, verification, CLI install/usage, docker usage, artifact sha256), create-or-get the gitea release by tag, attaches coreci-broker-vX.Y.Z.pyz via POST /api/v1/repos/{owner}/{repo}/releases/{id}/assets (multipart field attachment, Authorization: token <T>); idempotent re-attach; bounded retry (3) → release_pending escalation on exhaustion — never blocks tag+merge	Owns the forge API conversation for release assets only; token resolved exclusively from .env* files via the file-reading loader (never shell env; assert non-empty before any call — empty token → gitea 404 masking, v1.8); does not build the artifact, does not push images	CLI artifact build (attaches its output); gitea API (/api/v1/...)
Daemonless registry publish client (scripts/publish_image.py)	Once per milestone: pull the pinned python:3.11-slim base from Docker Hub (anonymous token exchange → OCI index → resolve linux/amd64 child manifest → config + layer blobs), append one deterministic broker layer carrying /app/coreci-broker.pyz (tar+gzip, fixed metadata, mtime=0), build a new config blob (diff_id appended; Entrypoint ["python3","/app/coreci-broker.pyz"], Cmd [], WorkingDir /state), and push git.coreci.dev/coreci/broker:<milestone-release-tag> + latest to the gitea container registry (Basic-auth token exchange at /v2/token, Bearer JWT on API calls, HEAD-skip + monolithic blob uploads, OCI image manifest PUT; explicit Location-following — stdlib urllib does not follow 307/308 on PUT)	No container runtime (D-019); base layer digests reused verbatim, base blobs re-uploaded (no cross-registry mount); records resolved base digests as provenance; failure follows bounded-retry → release_pending escalation — never blocks the milestone merge	CLI artifact build (ships its output in the layer); Docker Hub + gitea registry HTTP APIs (OCI dist-spec v2)
Dockerfile (documentation)	Self-contained docker-build path for users WITH docker: same base, in-image zipapp build of the same sources + shim, same entrypoint//state anchoring	Documentation + alternative build path only; the daemonless client is the executable source of truth for what the pipeline publishes; equivalence with the published image is required, byte-identity is not	CLI artifact build (same staging content, same shim)

DataFlow (v0.2 distribution)

1. CLI boot (packaged): ./coreci-broker.pyz → shim resolves the state dir (BROKER_ENV_PATH-anchored; CWD-relative defaults preserved) → materializes UI/static assets into the versioned assets dir → sets BROKER_UI_ROOT/BROKER_STATIC_ROOT (user env/.env overrides win) → coreci_broker.run_standalone() → first-boot .env generation, admin pass printed once, broker.db/broker.audit.log 0600 in the state dir, UI served at /ui/, client at /static/coreci_broker_client.py — behavior byte-identical to the unpackaged standalone mode.
2. Release attachment (per phase ship from the CLI-building phase): scripts/build_cli.py → reproducible dist/coreci-broker-vX.Y.Z.pyz (+sha256) → scripts/release.py → gitea create-or-get release by tag → multipart attach → notes body (CLI install/usage + docker usage + digest). Bounded retry; release_pending escalation on exhaustion; tag+merge is the completion gate.
3. Image publish (once per milestone, at the final ship): scripts/publish_image.py → anonymous Docker Hub pull (python:3.11-slim OCI index → amd64 child + blobs) → deterministic broker layer (contains the pyz) + new config → gitea registry auth (/v2/token, Basic <user>:<GITEA_TOKEN>) → blob upload (HEAD-skip, monolithic POST, explicit Location-following) → manifest PUT <milestone-release-tag> then latest → provenance digests recorded. Bounded retry; never blocks the milestone merge.

BuildOrder (v0.2 — appended to the v0.1 order above)

8. CLI build pipeline + boot shim + packaging/boot-parity tests (P1 — REQ-21)
9. Release-asset uploader + daemonless publish client + Dockerfile + stub tests (P2 — REQ-22/REQ-23)
10. Cumulative acceptance (v0.1 suite through the packaged artifact), security review, milestone ship: attach pyz to v0.2.3, publish image, merge (P3)

Notes (v0.2)

• Distribution components hold no secrets: the archive bundles only broker + UI + client code; the image ships no state; forge tokens live in .env* files loaded only by scripts/_env_secrets.py (blocked from shell env by config.security.bash_allowlist).
• All three pipeline scripts take injectable base URLs (stub-testable); live-forge interactions are read-only in tests and attempted-only at ship per the bounded-retry contract.
• Verified environment facts backing this section live in RESEARCH.md v0.2 (gitea 1.26.1 API shapes, OCI index resolution for python:3.11-slim, gitea /v2/token Basic-auth exchange, zipapp normalization, urllib 307/308 behavior).