broker/.ciagent/PROJECT.md

PROJECT — CoreCI Broker

Name

CoreCI Broker

Core Value

A single canonical credential flow for autonomous agent tools. The broker mediates access to the creds-coreci-coo knowledge base: tools authenticate by a derived token (not a shared secret), the CEO manages a UI allowlist of approved tool UUIDs and user IDs, every read is audited to an append-only file, and per-tool scope is enforced by path (coreci/<agent_id> only). It replaces per-tool credential resolution (KB files, env vars, mode-0600 JSON) with one small, readable implementation surface: a single Python file, one SQLite DB, a four-page web UI.

Requirements

Validated

Source: Requirements Specification v0.5 (owner: coreci-coo, 2026-10-09; supersedes v0.4) — distilled into REQ-01..REQ-20 (shipped in v0.1). v0.2 additions validated from owner directives (2026-10-10) — distilled into REQ-21..REQ-23: CLI zipapp packaging, per-release artifact attachment with notes on gitea, once-per-milestone docker publication to the gitea container/package registry. The v0.5 spec's ambiguities were resolved by D-001..D-015 (init + clarify commits, v0.1); v0.2's packaging/distribution ambiguities are resolved as D-019..D-022 (this milestone's clarify stage):

• Dual-mode token hashing (bcrypt when importable, stdlib scrypt fallback; format-prefixed hashes)
• Pluggable creds-source adapter (file-based default; Open WebUI HTTP API for production)
• Agent-state backend toggle (internal default | owui)
• UUID-scoped token verification + in-memory verified-token cache (spec §7.5's token_hash index is impossible with bcrypt)
• Register/pepper semantics including the KB-loss recovery path
• A public, dependency-free client module served as a static asset so agents can bootstrap their own OWUI tools
• Spec §13 defaults: no audit rotation, single instance, 24h token TTL, CEO sole approver

Active

Milestone v0.2 (milestone/v0.2-cli-packaging) — CLI + distribution only, no functional changes to the broker: REQ-21 (zipapp CLI packaging), REQ-22 (release assets attached + notes on gitea), REQ-23 (docker container built once per milestone, published as a gitea package). Owner-locked inputs: zipapp .pyz form factor; docker publication starts with v0.2 (no retroactive v0.1 image); scope excludes deferred v2 candidates.

Out of Scope

See REQUIREMENTS.md §Out of Scope — federation, secret versioning, high availability, UI beyond the four pages + dashboard, audit rotation, multiple broker instances, approver agents, key rotation ceremony, HSM, rate limiting, admin MFA, notifications.

Constraints

• Single-file broker: coreci_broker.py under 1400 lines (amended from 800 via owner decisions D-016..D-018; excluding tests, UI HTML, client module, reference tools) — spec §12.14 as amended
• Stdlib-only runtime: no pip in the target dev environment; bcrypt is the sole optional import (dual-mode hashing); tests use unittest
• SQLite is the source of truth: single instance, no HA; back it up
• Bind default 127.0.0.1:8765; the bind address controls reachability
• Mode 0600 on broker.db and broker.audit.log
• The broker never writes to creds-coreci-coo — the CEO writes that KB directly through Open WebUI's UI
• Config contract = spec §5.1 plus documented additions: BROKER_ENV_PATH, BROKER_CREDS_SOURCE, BROKER_CREDS_FILE, BROKER_OWUI_BASE_URL, BROKER_OWUI_API_KEY, BROKER_AGENT_STATE_BACKEND
• Token form: base64(sha256(pepper:UUID:user_id:agent_id:sorted_tags)) — 44-char ASCII, no prefix; BROKER_TOKEN_TTL default 86400s
• Open WebUI runtime: bcrypt and httpx exist there; the client module must not depend on either (stdlib urllib) so any tool can import it

v0.2 (packaging/distribution) constraints:

• No pip / stdlib-only build path: the CLI artifact is a stdlib zipapp (.pyz); no wheel, no PyInstaller, no pip at build or runtime (D-009)
• No container runtime in the dev environment: no docker/podman/buildah available — the once-per-milestone publish runs daemonless via a stdlib registry client (D-019)
• Self-contained artifact: one .pyz bundles broker + UI assets + client module; asset roots resolved from env, never __file__
• Broker behavior frozen: first-boot semantics, API surface, and all v0.1 acceptance criteria remain byte-identical when driven through the packaged CLI
• Release secrets discipline: forge tokens resolved only from .env* files via the secrets-loader — never shell environment (v1.8 lesson)
• broker.db/broker.audit.log remain state-directory-local — the CLI boots in a runtime state dir (docker: a volume), never writes inside its own archive

Context

CoreCI estate: sibling projects researchx and nextcraft live on gitea (git.coreci.dev/coreci/*) and use this same ciagent pipeline. The broker runs on the Open WebUI host as one workspace tool (embedded HTTP server in a daemon thread; also runnable standalone). The creds-coreci-coo KB currently mediates four credentials (coreci/proxmox, coreci/gitea, coreci/executor TBD, plus the COO's own login — independent of the broker's BROKER_ADMIN_PASS). The existing proxmox_readonly_tool.py is updated post-broker to use the public client module; it lives outside this repo, so this repo ships the reference implementation of the tool-side bootstrap instead.

v0.2 context: milestone v0.1 shipped complete (all REQ-01..REQ-20, v0.1.5 on main, six gitea releases each carrying a broker-vX.Y.Z.tar.gz source archive, zero escalations). Owner directives for v0.2 (2026-10-10): package the broker as a CLI; attach the CLI to every release alongside notes on gitea; build a docker container once per milestone and publish it as a package in the gitea registry. The v0.1 release-attachment mechanism (sanctioned .env* token path, per-release asset upload) is the proven base REQ-22 extends.

Known spec defects resolved by clarification (recorded for traceability): §7.5's in-memory token_hash index is impossible with bcrypt salted hashes; §6 claims the tool derives a token every call while §9's flow uses the broker-returned token from the agent KB (the broker derives authoritatively; the tool stores); §9's snippet raises KeyError on data["pepper"] for re-registrations; the pepper chicken-and-egg (a token is needed to read agent state, but the pepper is needed to derive a token) is fixed by returning the pepper when the caller presents no valid token.

Key Decisions

ID	Decision	Rationale	Outcome
D-001	Dual-mode bcrypt/scrypt hashing, format-prefixed hashes	Spec §6 requires bcrypt; dev env is stdlib-only; OWUI runtime ships bcrypt	To be validated in P1
D-002	Creds-source adapter: file (default) | OWUI API	Spec never specified the physical read mechanism; both environments must work	To be validated in P4
D-003	Agent-state backend toggle, internal default	§9.1 endpoints are the only interface; internal storage is functionally identical without the admin-API dependency	To be validated in P3
D-004	UUID-scoped slow-hash scan + in-memory verified-token cache	§7.5's bcrypt token_hash index is impossible; header UUID bounds the scan; SQLite stays authoritative	To be validated in P3
D-005	Register re-derives; pepper returned on row-create or tokenless register	Broker holds pepper + all inputs; tokenless register is the only KB-loss recovery path	To be validated in P3
D-006	Public dependency-free client module at /static/ + reference tools	Owner directive: agents bootstrap their own tools within the OWUI workspace	To be validated in P4
D-007	Spec §13 defaults accepted	Owner confirmed: no rotation, single instance, 24h TTL, CEO sole approver	Binding for v0.1
D-008	Milestone v0.1; "v0.5" remains the spec revision label	computeMilestoneTag greenfield convention (researchx, nextcraft precedent)	Binding for v0.1
D-009	Stdlib-only runtime; unittest for tests	No pip in environment; dual-mode hashing is the sole optional import	Binding for v0.1
D-010	Single-project mode; gitea release target coreci/broker	Sibling convention on the same forge; first push deferred to a ship phase	Binding for v0.1
D-011	Empty root commit on main precedes the branch hierarchy	Branches cannot be created from an unborn HEAD; nextcraft precedent	Executed at init
D-012	Forge repo coreci/broker public, default branch main	Sibling convention (researchx/nextcraft both public); code ships no runtime secrets — pepper/admin pass generated at boot, .env.secrets gitignored	Binding for v0.1
D-013	Session cookie broker_session uses HttpOnly + SameSite=Lax	Spec silent on cookie attributes; HttpOnly blocks JS exfiltration, Lax preserves bookmark-login UX	Binding for v0.1
D-014	BROKER_CREDS_FILE defaults to data/creds.json	FileCredsSource dev/test default location; data/ already gitignored	Binding for v0.1
D-015	GET /api/broker/agents/<agent_id>: App-auth + identity match, metadata-only response (no token, no hash)	§7.4 auth/response unspecified; mirrors state-endpoint identity enforcement; token material never leaves the broker	Binding for v0.1
D-016	AC14 line cap amended: coreci_broker.py < 1000 lines (was < 800)	Owner-approved §12.14 amendment at P2: 11 lines of headroom vs ~275 lines of remaining handler surface; preserves the readable single-file intent without dense-cramming	Binding for v0.1
D-017	D-016 extended: AC14 cap < 1200 lines	Second §12.14 amendment at P3 (owner's D-016 pattern applied with new data): P3-01 lands at 990/1000 with ~150-180 lines of remaining broker surface (App-auth cache, agent state, creds adapter, secrets endpoint); mild compaction reclaims only ~50 — same readability-over-cramming tradeoff the owner chose at D-016; subject to review at the P5 owner approval gate	Binding for v0.1
D-018	D-017 extended: AC14 cap < 1400 lines — final projected amendment	Third §12.14 amendment, pre-P4 per the P3 verify planning signal: 1178/1200 after P3 with 21 lines headroom vs P4's ~105-140 line need (creds adapter + secrets endpoint, the LAST broker-surface additions); P5 adds only excluded artifacts (tests, README). Same owner pattern (D-016/D-017), flagged for the P5 owner approval gate	Binding for v0.1
D-019	Daemonless container publish: stdlib registry client (pull pinned base layers, append broker layer, push to gitea registry)	No docker/podman/buildah in the dev environment; gitea's package registry speaks OCI registry HTTP API v2 — fully automated once-per-milestone publish without a container runtime; stub-testable	Binding for v0.2
D-020	Attach coreci-broker-vX.Y.Z.pyz to every release from v0.2.1 onward	Consistent with v0.1's per-release broker-vX.Y.Z.tar.gz asset convention; every tagged release stays independently deployable	Binding for v0.2
D-021	Image tags git.coreci.dev/coreci/broker:<milestone-release-tag> + latest, pushed once per milestone	Version tag pins the release; latest gives docker users a sane default; once-per-milestone honors the owner directive	Binding for v0.2
D-022	Base image python:3.11-slim (pinned minor line)	Dev environment runs Python 3.11.2 — pinned minor matches tested stdlib behavior and stays reproducible	Binding for v0.2