# CoreCI Broker **TL;DR** — CoreCI Broker — single-file credential broker for autonomous agent tools (Open WebUI workspace tool / standalone). This file explains how to stand it up and who to contact; the machinery detail stays lower under Details. A single-file credential broker for autonomous agent tools on the Open WebUI host. Tools authenticate with derived tokens instead of shared secrets, the CEO gates access through UI-managed allowlists of app UUIDs and user IDs, every read lands in an append-only audit log, and secret reads are path-scoped to `coreci/` — one Python file, one SQLite DB, a four-page admin UI, zero pip installs, no build step. ## Quickstart (standalone) ```bash python3 coreci_broker.py ``` - Stdlib-only runtime, Python 3.11+. Zero pip installs: bcrypt when importable, stdlib scrypt otherwise (format-prefixed hashes). - First boot generates `.env`: a 32-byte base64 pepper plus a 32+-char admin password. The password is printed to the startup log as `BROKER_ADMIN_PASS=…` **exactly once** — save it immediately; it is never printed again. Lost pass: remove `BROKER_ADMIN_PASS` from `.env` and reboot — a new one prints once. - The server binds `127.0.0.1:8765`. Open `http://127.0.0.1:8765/ui/login` and log in with the printed pass. - Stop with Ctrl+C. Durable state is exactly three files: `broker.db`, `broker.audit.log`, `.env` — back them up. ## Configuration Read on boot from the `.env` at `BROKER_ENV_PATH`. Precedence: **process env → `.env` → defaults**. | Variable | Default | Purpose | |----------|---------|---------| | `BROKER_BIND` | `127.0.0.1:8765` | Listen address; the bind controls reachability | | `BROKER_ADMIN_USER` | `admin` | Admin login user name | | `BROKER_ADMIN_PASS` | generated first boot | Admin login password (32+ random chars) | | `BROKER_INSTANCE_PEPPER` | generated first boot | 32-byte base64 pepper; token derivation input, synced to SQLite | | `BROKER_ENV_PATH` | `.env` | Path of the `.env` file the broker reads/writes | | `BROKER_TOKEN_TTL` | `86400` | Tool token lifetime, seconds (24 h) | | `BROKER_SESSION_TTL` | `300` | Admin UI session cookie lifetime, seconds | | `BROKER_DB_PATH` | `broker.db` | SQLite database (mode 0600) | | `BROKER_AUDIT_LOG_PATH` | `broker.audit.log` | Append-only JSONL audit log (mode 0600) | | `BROKER_CREDS_SOURCE` | `file` | Credential source adapter: `file` (dev/tests) or `owui` (production) | | `BROKER_CREDS_FILE` | `data/creds.json` | `FileCredsSource` JSON document: paths → secret objects | | `BROKER_OWUI_BASE_URL` | — | Open WebUI base URL for the OWUI creds source | | `BROKER_OWUI_API_KEY` | — | Open WebUI API key (Bearer) for KB read access | | `BROKER_AGENT_STATE_BACKEND` | `internal` | Agent-state storage: `internal` (rows in broker.db) or `owui` | | `BROKER_MAX_AGENTS_PER_APP` | `50` | Cap on agent registrations per app UUID | | `BROKER_UI_ROOT` | `coreci_broker_ui` | Directory served for `/ui/*` pages | | `BROKER_STATIC_ROOT` | `.` | Directory served for `/static/*` (client module) | Production: `BROKER_CREDS_SOURCE=owui` with `BROKER_OWUI_BASE_URL` + `BROKER_OWUI_API_KEY` (0600 `.env` discipline). Keep `BROKER_BIND` on loopback unless you deliberately widen reachability. ## Running as an Open WebUI workspace tool Paste `coreci_broker.py` into Open WebUI as a workspace Tool (frontmatter `requirements:` stays empty — the runtime already ships everything). Notes that matter: - The server starts in `Tools.__init__` behind a once-per-process guard; the OWUI loader has no activate/deactivate hook, and disabling a tool does not stop a previously started server. - **After editing the broker tool source, restart Open WebUI.** The loader re-execs the module on source change, but the previously loaded server keeps its port — the new instance fails loud with a bind-error message telling you to restart, rather than silently serving old code. - Paths (DB, audit log, `.env`, UI roots) resolve from environment config, never from `__file__` — the OWUI loader deletes the temp file it execs. - Durable state: `broker.db` + `broker.audit.log` + `.env` — **back those three up**; SQLite is the source of truth, no HA, restore is a file copy. - `Tools.status()` reports the serving address; call it from a chat to verify the mount, or hit `/healthz` on the bind. ## Admin UI Four pages + dashboard, server-rendered on the same port as the API: - `/ui/login` — the CEO's bookmark. First-boot pass discipline: the printed pass is shown once; record it in your secret store on first login. - `/ui/app_registrations` — allowlist of tool app UUIDs: add, revoke, reinstate. - `/ui/user_allowlist` — allowlist of user IDs (optional email): same actions. - `/ui/audit` — filterable, read-only view of the audit log (no export in v0.1). - `/ui/dashboard` — landing page with persistent nav header and Logout. Allowlist semantics: **revoke keeps existing tokens working until TTL expiry — by design.** Revocation immediately blocks new registrations; already-issued tokens live at most one `BROKER_TOKEN_TTL` (24 h by default). ## Tool authors Fetch the public client module — unauthenticated, from `http:///static/coreci_broker_client.py` — into your workspace and import it. Flow: `ensure_registered(state)` → `put_state(state)` → `read_secret("coreci/", state)`. Stdlib-urllib-only (no httpx, no bcrypt), so any tool can import it; on a 401 it re-registers and retries once. State endpoints are the tool's only persistence channel — keep `state` there, not in local files. See `reference_tools/` for the proxmox-style §11 flow and a minimal auth-then-use showcase; downstream calls default to a safe dry-run. ## Security model - Derived token: `base64(sha256(pepper:UUID:user_id:agent_id:sorted_tags))` — 44 ASCII chars, no prefix, no challenge protocol. Stored only as a slow hash (`$2b$…` / `scrypt$…`), never in plaintext. - Allowlists are the gate: unknown or revoked app UUIDs and unknown users get 403 at registration; the token is the consequence, not the authority. - Every secret-read attempt is audited (ok/denied/error) with user, UUID, agent, and path. - Secret reads are path-scoped: a registration reads exactly `coreci/` — cross-path → 403, absent → 404. - The broker never writes to `creds-coreci-coo`; the CEO maintains that KB through the Open WebUI UI. - `broker.db`, `broker.audit.log`, and `.env` are mode 0600. - Binds `127.0.0.1` by default; trusted-network posture — no rate limiting by design (spec §14). ## Documented spec deviations Honest list; full detail in `.ciagent/PROJECT.md`: - **Line-cap amendments (D-016/D-017/D-018)**: the single-file constraint grew 800 → 1000 → 1200 → 1400 lines via the owner-approved §12.14 amendment pattern (readability over dense cramming). Current source: 1390 lines; AC14 enforces the cap. - **Dual-mode hashing (D-001)**: spec §6 mandates bcrypt; the runtime is stdlib-only, so scrypt fallback with format-prefixed hashes. - **Token verification (D-004)**: spec §7.5's in-memory `token_hash` index is impossible with salted slow hashes; the broker scans rows scoped to the presented UUID and keeps a non-authoritative verified-token cache. - **Pepper rule (D-005)**: the pepper is returned only on row creation or tokenless registration (the KB-loss recovery path); spec §9's snippet would raise `KeyError` on re-registration. - **Spec defects resolved**: §7.5 index, §6-vs-§9 (the broker derives authoritatively; the tool stores), §9's `KeyError` — see PROJECT.md §Context. - `BROKER_ADMIN_USER` defaults to `admin` (spec silent on a default). - Register cap `BROKER_MAX_AGENTS_PER_APP` (50) — a documented addition. - Reinstate actions audit as `add_app`/`add_user` (no separate audit action). - In-scope but absent secrets audit as `read` with `result: error` (404s are audited, not silent). ## Post-ship live-OWUI smoke checklist (owner-run) Before migrating any real tool (G-005; deployed host pinned v0.11.3, G-006): 1. Confirm the deployed Open WebUI version (expected **v0.11.3**) and that the KB read endpoints match the researched surface: `/api/v1/knowledge/{id}` and `/api/v1/files/{id}/data/content`. 2. Add the broker as a workspace tool; check `/healthz` on the bind. 3. CEO first-boot login at `/ui/login` with the printed pass. 4. Add one app UUID and one user to the allowlists via the UI. 5. In a scratch tool: import the client module from `/static/` → `ensure_registered` → read `coreci/proxmox` via the OWUI creds source. 6. Verify every step in `/ui/audit`. 7. Rollback = disable the tool + restore the three state files. No real tool migrates before this passes; the 90-day adoption criterion (proxmox tool on the client, direct KB reads removed) is due 2027-01-09. ## Tests ```bash timeout 300 python3 -m unittest discover -s tests ``` 295 tests, stdlib unittest only (~2 min; one skip where bcrypt is not importable). The AC1–AC14 acceptance matrix is green. ## Repository layout | Path | What it is | |------|------------| | `coreci_broker.py` | The broker: single file — server, store, audit, auth, UI serving | | `coreci_broker_client.py` | Public stdlib-only client module (served at `/static/`) | | `coreci_broker_ui/` | Server-rendered HTML for the four pages + dashboard | | `reference_tools/` | Tool-author templates (proxmox-style §11 flow + minimal showcase) | | `tests/` | Stdlib-unittest suite incl. the AC1–AC14 acceptance matrix | | `.ciagent/` | CI pipeline artifacts (project, requirements, roadmap, decisions) | ## CLI and Docker (v0.2) From v0.2.1 onward every gitea release attaches the broker packaged as a self-contained CLI zipapp: `coreci-broker-vX.Y.Z.pyz` (its sha256 is in the release notes). **CLI install + usage:** ```bash # from the release page assets: chmod +x coreci-broker-vX.Y.Z.pyz ./coreci-broker-vX.Y.Z.pyz --version # prints the release version ./coreci-broker-vX.Y.Z.pyz # boots the standalone broker # or: python3 coreci-broker-vX.Y.Z.pyz (Python 3.11+) ``` The artifact bundles the broker, the admin UI pages, and the client module — one file serves everything (UI at `/ui/`, client at `/static/coreci_broker_client.py`). First boot prints `BROKER_ADMIN_PASS=...` exactly once and creates `.env`, `broker.db`, and `broker.audit.log` mode 0600 in the state dir; state is anchored by `BROKER_ENV_PATH` (relative paths stay CWD-relative). The full `BROKER_*` env contract is in the Configuration section above. **Docker:** ```bash docker pull git.coreci.dev/coreci/broker: # also :latest docker run -p 8765:8765 -v broker-state:/state \ git.coreci.dev/coreci/broker: ``` State lives on the `/state` volume (`WORKDIR /state`; first boot generates `.env`, `broker.db`, and `broker.audit.log there) — mount a persistent volume or the admin pass and any registrations are lost with the container. The image is built and published once per milestone at the milestone release (see `scripts/publish_image.py` — the daemonless registry client; the `Dockerfile` documents the docker-user path and its equivalence mapping).