CIAgent
0c15d3d0b2
M2 delivers the read-only MCP capability broker gateway and four Day-1 infrastructure adapters (Proxmox, SSH/Linux, GitHub, Gitea). 13 REQs (015-027) all pass. 656 tests green. M1 non-regression verified. MCP spec 2025-06-18 conformance verified (PROTOCOL.md + 7 tests). Defense-in-depth SSH (broker layer 1 + Relay Agent layer 2 + no-shell exec). Two-track LLM smoke (Track A mock-path P0 gate passes). CI: Gitea Actions (.gitea/workflows/ci.yml) with Postgres 16 + RLS verification. Phases shipped: P0 pre-execution v0.1.0 P1 Wave F — MCP gateway v0.1.1 P2 Wave G — Proxmox v0.1.2 P3 Wave H — SSH/Linux v0.1.3 P4 Wave I — Git adapters v0.1.4 P5 Wave J — SSE+smoke+UI v0.1.5 P6 Final — review+ship v0.1.6 ← milestone release ---ci--- phase: 6 milestone: v0.2 status: complete phase_role: final milestone_complete: true requirements: covered: [REQ-015, REQ-016, REQ-017, REQ-018, REQ-019, REQ-020, REQ-021, REQ-022, REQ-023, REQ-024, REQ-025, REQ-026, REQ-027] partial: [] ---/ci---
630 lines
75 KiB
Markdown
630 lines
75 KiB
Markdown
# Plan — Milestone 2 (v0.1, M2: MCP Layer & Day 1 Adapters)
|
||
|
||
Source spec: `.ciagent/steer-m2-spec.md` (v1.0, locked 2026-08-25, Sarah Chen). All 9 open questions resolved; spec delta applied to REQ-015/021/027 acceptance criteria, Section 5 constraints, Section 9 M2→M3 contract freeze.
|
||
M2 scope: **REQ-015..027 (13 REQs)**. MCP capability broker gateway + 4 Day-1 adapters (Proxmox, SSH/Linux via Relay Agent, GitHub, Gitea) + SSE streaming + token-bucket rate limiting + LLM smoke + Postgres 16 CI/RLS verification. M2 acceptance gate: spec §6 (15 items).
|
||
Architecture: `.ciagent/ARCHITECTURE.md` (M2 MCP broker section written). Decisions: `.ciagent/CLARIFY.md` (D-001..D-007; D-006 GitHub scopes, D-007 MCP transport). Research: `.ciagent/RESEARCH.md` (R-001..R-009, 7 flagged risks integrated below). Personas: `.ciagent/PERSONAS.md` (M2 roster: backend, data, frontend, lead, security, go-engineer for Wave H only). Requirements: `.ciagent/REQUIREMENTS.md` (REQ-015..027 with acceptance criteria + spec delta).
|
||
|
||
> **Note:** This PLAN.md overwrites the M1 PLAN.md. The M1 plan is preserved in git history (commit prior to M2 overwrite). M1 is COMPLETE; all 17 M1 REQs (001-014, 038, 039, 040) must remain passing through M2 (M1 non-regression — gate item 1).
|
||
|
||
---
|
||
|
||
## Phase mapping
|
||
|
||
M2 ships on the v0.1.x patch line (M1's previous minor). Phase 0 seeds `v0.1.0`; each execution phase ships a progressive patch; **the final phase's patch (`v0.1.6`) IS the M2 milestone release.** The milestone merge to `main` happens at the final phase.
|
||
|
||
| Phase | Wave | Branch | Patch tag | REQs covered |
|
||
|-------|------|--------|-----------|--------------|
|
||
| 0 | pre-execution | `phase/00-pre-execution` | `v0.1.0` | (this plan) |
|
||
| 1 | Wave F — MCP Gateway core | `phase/01-mcp-gateway` | `v0.1.1` | REQ-015, REQ-016, REQ-017, REQ-018, REQ-019, REQ-024 |
|
||
| 2 | Wave G — Proxmox adapter | `phase/02-proxmox-adapter` | `v0.1.2` | REQ-020, REQ-025 |
|
||
| 3 | Wave H — SSH/Linux adapter (Relay Agent) | `phase/03-ssh-adapter` | `v0.1.3` | REQ-021, REQ-026 (full) |
|
||
| 4 | Wave I — Git adapters | `phase/04-git-adapters` | `v0.1.4` | REQ-022, REQ-023, REQ-027 |
|
||
| 5 | Wave J — SSE integration + LLM smoke + adapter UI | `phase/05-sse-integration` | `v0.1.5` | REQ-017 integration, M2 gate item 8 (LLM smoke) |
|
||
| 6 | Final — Review + Audit + Ship | `phase/06-final-review-ship` | `v0.1.6` ← **M2 milestone release** | all M2 (sign-off) |
|
||
|
||
**Wave 0 prerequisites (not a spec REQ; MUST complete before Wave F):**
|
||
- **[G-011 P0] Build the CI/CD pipeline as an explicit Wave 0 task.** Create `.gitea/workflows/ci.yml` (Gitea Actions — the repo's forge is Gitea at `git.cloudinit.dev`; Gitea Actions is GitHub Actions-compatible, same YAML syntax + `secrets.*` context + service containers) with two jobs: `test-pglite` (default) + `test-postgres` (service container + `DB_MODE=pg`). Postgres 16 service container with `coreci_app` (no BYPASSRLS) + `migrator` (BYPASSRLS) role setup via `setup-ci-roles.sql` (R-009). GitHub smoke PAT as Gitea Actions repository secret (`secrets.GITHUB_SMOKE_PAT` — a GitHub API token for the Track-B real-GitHub adapter smoke, stored in Gitea's secret store). Caching for pnpm + go modules. Coverage upload. **This is M2 work, not an external prerequisite.** Without it, 9 of 15 gate items are unprovable.
|
||
- **[G-022 P1] Run the full M1 test suite against Postgres 16 in Wave 0.** After CI is provisioned, run the *entire* M1 test suite with `DB_MODE=pg`. Any M1 RLS failure (cross-tenant SELECT, missing `FORCE ROW LEVEL SECURITY`, wrong `WITH CHECK`) is an M1-regression P0 that blocks M2 until fixed. Wave 0 is the *first* real-RLS test of M1.
|
||
- **Real GitHub PAT available in CI** (ephemeral or test-org-scoped, stored as repository secret) for the Wave I real-target smoke + Wave J LLM smoke (gate item 8). Required for the optional real-GitHub smoke track (G-018); the P0 mock-path smoke does not depend on it.
|
||
|
||
---
|
||
|
||
## Grill fixes applied (G-011..G-022)
|
||
|
||
This plan was grilled (`.ciagent/GRILL.md`, verdict FAIL → auto-resolved at full autonomy with all 12 binding fixes applied). The 12 binding fixes (continuing from M1 GRILL's G-001..G-010) are integrated below and flagged inline as `[G-NNN]`. Summary:
|
||
|
||
- **G-011 (P0):** Build the CI/CD pipeline as an explicit Wave 0 task (not a "prerequisite"). Applied above.
|
||
- **G-012 (P1):** Own the M1 `AuditEventType` union extension as an explicit M1-file edit in Wave F (type widening, not "additive — no schema change" at the TS layer).
|
||
- **G-013 (P1):** Expand the cross-layer SSH test to a divergence matrix in Wave H (both-reject, both-accept, broker-rejects-Go-accepts, Go-accepts-broker-rejects). Tighten the Go deny list to include bare `rm` or validate `systemctl status` trailing args.
|
||
- **G-014 (P1):** Document the known closed-tool-set gaps in the Settings → Adapters UI help text in Wave J.
|
||
- **G-015 (P1):** Correct the INV-7 framing: the closed 9-tool registry (REQ-015) is the primary boundary; the write-blocklist is a backstop for adapter bugs. Document in Wave F PROTOCOL.md.
|
||
- **G-016 (P1):** Separate the write-blocklist into two enforcement models (method blocklist for Proxmox/Gitea; scope-via-403 for GitHub). Document GraphQL/PVE-GET-with-side-effects future risks in PROTOCOL.md.
|
||
- **G-017 (P1):** Add a 7th MCP conformance test over stdio transport in Wave F (`stdio-interop.test.ts`) — proves an external MCP client can connect.
|
||
- **G-018 (P0):** Add a `github-mock` adapter and a two-track LLM smoke in Wave J. Mock-path (canned repos) is the P0 gate; real-path (real GitHub) is optional/allow-failure.
|
||
- **G-019 (P0):** Harden `packages/llm-mock` pattern matching (regex set, not 2-word conjunction) + add retry policy for real-GitHub smoke in Wave J.
|
||
- **G-020 (P1):** Ship a documented `McpAdapter` interface in Wave F that both in-process adapters (G/I) and the WebSocket-backed SSH adapter (H) implement.
|
||
- **G-021 (P1):** Add M1-relay-WS regression test in Wave H + restructure the Go agent reader goroutine to dispatch on `type` before unmarshaling.
|
||
- **G-022 (P1):** Run the full M1 test suite against Postgres 16 in Wave 0. Applied above.
|
||
|
||
---
|
||
|
||
## Flagged risk mitigations (from RESEARCH R-001..R-009)
|
||
|
||
For each of the 7 flagged risks, the wave that addresses it and how:
|
||
|
||
| # | Risk | Wave | Mitigation |
|
||
|---|------|------|-----------|
|
||
| R-001 | Synthetic MCP `initialize`/`initialized` handshake for in-process adapters (lowest-confidence gate item 15) | **Wave F** | Implement a lightweight synthetic lifecycle exchange at adapter registration (broker → `{method:"initialize", params:{protocolVersion:"2025-06-18", capabilities:{tools:{listChanged:false}}}}`, adapter → `{capabilities:{tools:{}}}`). Produce `packages/mcp/PROTOCOL.md` + 6 conformance tests in `tests/mcp-conformance/` (gate item 15 artifact). |
|
||
| R-002 | PVEAuditor role introspection gap (PVE has no clean "what role does this token have" endpoint) | **Wave G** | Broker validates "token works for reads" (`GET /api2/json/version` + `GET /api2/json/nodes`) at submit, NOT "token lacks writes." Document the introspection gap in the Settings → Adapters UI help text ("Create a token with PVEAuditor role"). The broker write-method blocklist (reject POST/PUT/DELETE) is the load-bearing INV-7 boundary. Confidence 0.70 on this sub-point. |
|
||
| R-004 | GitHub fine-grained PAT scope introspection gap (no public API to list a fine-grained PAT's granted scopes) | **Wave I** | Submit-time: `github_pat_` prefix check (reject classic PATs) + `GET /user` (validates token + implicit `metadata:read`). Per-invocation: handle 403 with `X-Accepted-GitHub-Permissions` header (e.g., `actions=read` required) → HTTP 403 "insufficient scope" + `adapter.capability_invoked` with `result=failure` (NOT `write_rejected` — no write attempted). Document in UI. |
|
||
| R-005 | Gitea runtime verification (docs page required JS; findings from spec + GitHub-mirroring conventions) | **Wave I** | Verify `GET /api/v1/version`, `GET /api/v1/user/repos`, `GET /api/v1/repos/{owner}/{repo}/actions/runs` against a running Gitea 1.22+ AND a <1.22 instance during Wave I. Confirm `read:repository` scope behavior (≥1.22) and the `Authorization: token <token>` header. Confidence 0.72 → raise by runtime verification. |
|
||
| R-003 | Cross-layer SSH defense-in-depth test (two independent whitelist implementations: TS broker layer 1, Go agent layer 2) | **Wave H** | Add a cross-layer test asserting a non-whitelisted command (`rm -rf /`) is rejected by BOTH the broker (layer 1, TS) AND the Relay Agent `CheckCommand` (layer 2, Go). Keep the two implementations semantically in sync but code-independent (a bug in one doesn't bypass the other). |
|
||
| R-006 | 30s stream-not-opened timeout (orphan adapter calls if client never opens the SSE stream) | **Wave F** | Stream manager enforces a 30s "stream not opened" timeout: if `GET /api/mcp/stream/:correlationId` isn't called within 30s of `POST /api/mcp/invoke`, cancel the adapter call and delete the correlation context. Plus a 60s max-stream lifetime safety net for orphaned contexts. |
|
||
| R-009 | Wave 0 RLS assertions (M1 pen-test has placeholder `expect(true).toBe(true)` because PGlite doesn't enforce RLS on SELECT) | **Wave 0** | Deliver `packages/db/scripts/setup-ci-roles.sql` (`coreci_app` no BYPASSRLS, `migrator` BYPASSRLS). Parameterize the pen-test by `DB_MODE` (`pglite` default, `pg` in CI). Replace placeholder with real RLS WITH CHECK assertions gated on `DB_MODE=pg`. Two CI jobs: `test-pglite` + `test-postgres` (service container). |
|
||
|
||
**Additional research decisions integrated (D-M2-R001..R009, all above 0.6 threshold):** D-M2-R004 GitHub classic-PAT rejection by prefix; D-M2-R005 Gitea version-aware scope routing; D-M2-R006 SSE no-audit-on-client-cancel; D-M2-R007 token-bucket refund-on-tenant-fail + no-audit-on-429; D-M2-R008 `packages/llm-mock` import-guarding (eslint `no-restricted-imports` + build-time grep).
|
||
|
||
---
|
||
|
||
## User-Facing Surface (MVP/UX §1)
|
||
|
||
M2 ships **TWO user-facing surfaces** in the existing M1 dashboard (`apps/control-plane/app/(dashboard)/`). Non-UI surfaces (the MCP broker gateway, the 4 adapters, the `packages/llm-mock` CI-only smoke provider, the `mcp_adapters` Postgres table) are backend and verified by tests, not by user demonstration.
|
||
|
||
### Surface 1 — Settings → Adapters configuration UI (`/dashboard/settings/adapters`)
|
||
- **Adapter type picker:** Proxmox / GitHub / Gitea / SSH (the 4 Day-1 adapter types; closed set, no "custom adapter" option).
|
||
- **Per-adapter config forms:** type-specific fields:
|
||
- Proxmox: `host` (HTTPS URL), `PVEAuditor` token (secret), `allowSelfSigned` toggle (R-002 — common for customer PVE labs).
|
||
- GitHub: `host` (defaults to `api.github.com`), fine-grained PAT (secret). Help text: "Create a fine-grained PAT with `metadata:read` + `actions:read` minimum. Classic PATs (`ghp_`) are rejected."
|
||
- Gitea: `host` (customer Gitea URL), read-only token (secret), `allowSelfSigned` toggle. Help text: "Gitea ≥1.22 requires `read:repository` scope. Gitea <1.22 accepts any token (broker enforces write-method blocklist)."
|
||
- SSH: `hostname`, `port` (default 22), Relay registration token (secret; resolves to the M1 Relay Agent target). Help text: "Install the M1 Relay Agent on the target host first; paste the registration token here."
|
||
- **SecretProvider-backed credential entry:** all secrets enter via `SecretProvider.put` (INV-3); the DB stores only `secret_ref`. The UI never displays the raw secret after submit (redacted, edit-only).
|
||
- **Role/scope validation on submit (REQ-025, REQ-026, REQ-027):** the broker validates the token at submit time and returns a structured pass/fail. On failure → HTTP 422 with role/scope-violation error, no config persisted, UI shows the error inline.
|
||
- **"Test connection" button:** invokes the `test_connection` capability (REQ-016) through the closed tool registry; returns structured pass/fail within 5s (spec Journey 1 Step 7).
|
||
- **Audit event confirmation:** after a successful save, the UI shows "Saved (audit event `adapter.configured` appended)."
|
||
- **Multi-target support (REQ-024):** the config form supports multiple adapters of the same type (e.g., 2 Proxmox hosts). Each row has a `target_id` (display name) so the Test-Call UI's target picker can disambiguate.
|
||
- **PVEAuditor introspection gap (R-002):** the Proxmox config form's help text documents that the broker validates "token works for reads," not "token lacks writes," and the operator is responsible for creating a PVEAuditor-scoped token.
|
||
|
||
### Surface 2 — Test-Call UI (`/dashboard/test-call`)
|
||
- **Capability picker:** the closed 9-tool set from `GET /api/mcp/tools` (REQ-015). Tools are grouped by adapter type in the dropdown. Per-tenant policy may disable individual tools (greyed out in the picker).
|
||
- **Argument forms:** each tool's arguments are rendered from the tool's JSON Schema `inputSchema` (REQ-015) — required fields marked, type-validated on submit. Invalid args → inline validation error (mirrors the broker's HTTP 400 schema-validation error, Edge 4).
|
||
- **Target picker for multi-target tenants (REQ-024):** when a tenant has ≥2 adapters of the same type, the UI surfaces a target picker. If the user submits a same-type capability without selecting a target, the broker returns HTTP 400 "target required" (Edge 3) and the UI prompts for selection.
|
||
- **SSE stream consumer (REQ-017):** the UI opens an `EventSource` on `GET /api/mcp/stream/:correlationId` and renders events as they arrive (`id`, `event`, `data` fields per SSE spec). Terminal events `done` (completion) / `error` (failure) close the stream.
|
||
- **Staleness indicator for inventory calls:** for `list_*` capabilities (inventory, 60s TTL cache), the UI shows "cached Xs ago" when the result is served from cache. For live capabilities (non-`list_*`), no staleness indicator (fresh result). (Spec Journey 2 Step 6.)
|
||
- **Result rendering:** tool results render as JSON in a collapsible tree. Errors render with the `isError: true` flag surfaced.
|
||
|
||
### Non-UI surfaces (verified by tests, not user demo)
|
||
- The MCP broker gateway (`packages/mcp`): registry, router, write-blocklist, rate-limiter, stream-manager, translator, in-process transport.
|
||
- The 4 adapters (`packages/mcp/adapters/{proxmox,ssh,github,gitea}`).
|
||
- The `packages/llm-mock` CI-only LLM smoke provider (devDependency, import-guarded).
|
||
- The `mcp_adapters` Postgres table (tenant-scoped, RLS).
|
||
- The 5 gateway REST+SSE endpoints (M2→M3 contract freeze, spec §9).
|
||
|
||
---
|
||
|
||
## Happy Path (MVP/UX §2)
|
||
|
||
> **Given** a fresh M2 deployment with M1 complete (SSO, BYOM, Relay Agent, audit, RLS, secrets operational) and Wave 0 prerequisites met (CI Postgres 16 with role setup, real GitHub PAT available in CI),
|
||
> **when** Sam (Tenant Admin) configures and validates all four Day-1 adapters and runs a Test-Call,
|
||
> **then** the M2 acceptance gate (spec §6) passes: all 4 adapters respond to a test call, multi-target scoping is functional, read-only enforcement is verified at the broker (INV-7), LLM smoke passes, M1 non-regression.
|
||
|
||
### BDD steps (Given/When/Then) — written BEFORE EXECUTE
|
||
|
||
**Adapter config + validation (Journey 1):**
|
||
- [ ] **Given** Sam is signed in via WorkOS SSO (M1) as `admin`, **when** Sam navigates to Settings → Adapters and clicks "Add adapter", **then** the adapter type picker shows exactly 4 types (Proxmox, GitHub, Gitea, SSH).
|
||
- [ ] **Given** Sam selects "Proxmox" and enters `host` + a `PVEAuditor` token, **when** Sam submits, **then** the broker validates the token (`GET /api2/json/version` + `GET /api2/json/nodes` succeed — R-002), `SecretProvider.put` stores the token, the `mcp_adapters` row is inserted under `withTenant` + RLS, `adapter.configured` is appended to `audit_log`, and the UI confirms save in <5s.
|
||
- [ ] **Given** a Proxmox adapter is configured, **when** Sam clicks "Test connection", **then** the broker invokes `test_connection` through the closed tool registry, returns a structured pass/fail within 5s, and `adapter.test_connection.succeeded` (or `.failed`) is appended.
|
||
- [ ] **Given** Sam submits a Proxmox token that fails `GET /api2/json/version` (invalid token), **when** the broker validates, **then** submission returns HTTP 422 with role-violation error and no config is persisted (REQ-025).
|
||
- [ ] **Given** Sam submits a GitHub classic PAT (`ghp_...`), **when** the broker validates scope, **then** submission returns HTTP 422 "fine-grained PAT required" and no config is persisted (D-006).
|
||
- [ ] **Given** Sam submits a GitHub fine-grained PAT (`github_pat_...`) that passes `GET /user`, **when** the broker validates, **then** the config is persisted with `validated=true` (implicit `metadata:read`); `actions:read` is validated per-invocation (R-004).
|
||
- [ ] **Given** Sam submits a Gitea token for a Gitea ≥1.22 instance, **when** the broker calls `GET /api/v1/user/repos?limit=1` and it returns 403, **then** submission returns HTTP 422 "insufficient scope — `read:repository` required" (R-005).
|
||
- [ ] **Given** Sam submits a Gitea token for a Gitea <1.22 instance, **when** the broker validates, **then** the token is accepted (any valid token) with the broker-side write-method blocklist as the security backstop.
|
||
- [ ] **Given** Sam submits an SSH adapter config (hostname + Relay registration token), **when** the broker stores the token via `SecretProvider.put`, **then** the `mcp_adapters` row is persisted and `adapter.configured` is appended (REQ-026).
|
||
|
||
**Test-Call capability invocation (Journey 2):**
|
||
- [ ] **Given** a Proxmox adapter is configured and rate limits are not exceeded, **when** Sam invokes `proxmox.list_vms` (with `node` arg) via the Test-Call UI, **then** the broker mints a ULID correlation ID, returns `{correlationId, streamUrl}`, the UI opens an SSE stream, the adapter calls `GET /api2/json/nodes/{node}/qemu`, the stream emits `tool_result` events and terminates with `done`, and `adapter.capability_invoked` is appended with `correlation_id`.
|
||
- [ ] **Given** `proxmox.list_vms` was invoked within the last 60 seconds, **when** Sam invokes it again, **then** the broker returns the cached result with a "cached Xs ago" staleness indicator in the UI (inventory TTL cache, LRU).
|
||
- [ ] **Given** Sam invokes a non-`list_*` capability (e.g., `proxmox.get_node_metrics`), **when** the call completes, **then** the UI shows the fresh result with NO staleness indicator (live path).
|
||
- [ ] **Given** Sam invokes `github.list_repos` via the Test-Call UI, **when** the adapter calls `GET /user/repos?per_page=100` (real GitHub PAT), **then** the normalized repo list streams back and the UI renders it.
|
||
- [ ] **Given** Sam invokes `ssh.run_whitelisted_command` with `command: "uptime"`, **when** the broker validates (layer 1) and dispatches to the Relay Agent, **then** the agent's `CheckCommand` (layer 2) passes, `exec.Command("uptime")` runs (no shell), and the output streams back.
|
||
|
||
**Write rejection at broker (Edge 1, INV-7):**
|
||
- [ ] **Given** a malicious payload attempts `proxmox.shutdown_vm` via `POST /api/mcp/invoke`, **when** the request reaches the broker, **then** the broker's write-method blocklist rejects with HTTP 403, `adapter.write_rejected` is appended, and the adapter is NEVER invoked (verified by test per adapter at the M2 gate).
|
||
|
||
**SSH whitelist violation (Edge 7, defense-in-depth):**
|
||
- [ ] **Given** Sam attempts `ssh.run_whitelisted_command` with `command: "rm -rf /"`, **when** the broker validates (layer 1, 6-command subset), **then** the broker rejects with HTTP 403 + `adapter.write_rejected` (the command is not in the 6-command subset) — the Relay Agent is never reached. **And** the cross-layer test (R-003) asserts that IF the broker were bypassed, the Relay Agent `CheckCommand` (layer 2) would also reject.
|
||
|
||
**Multi-target disambiguation (Edge 3):**
|
||
- [ ] **Given** a tenant has 2 Proxmox adapters configured (target_a, target_b), **when** Sam invokes `proxmox.list_vms` without selecting a `target_id`, **then** the broker returns HTTP 400 "target required" with a list of available targets, and the UI surfaces the target picker.
|
||
|
||
**Rate limit (Edge 5):**
|
||
- [ ] **Given** Sam has exceeded 60 req/min (user bucket), **when** Sam invokes any capability, **then** the broker returns HTTP 429 with `Retry-After` header and no adapter call is made (REQ-019).
|
||
- [ ] **Given** the tenant has exceeded 300 req/min (tenant bucket), **when** any user in that tenant invokes a capability, **then** the broker returns HTTP 429 with `Retry-After` and refunds the user token (fairness, D-M2-R007).
|
||
|
||
**SSE client disconnect (Edge 8):**
|
||
- [ ] **Given** an SSE stream is open on `GET /api/mcp/stream/:correlationId` and the client disconnects mid-stream, **when** the broker detects the `EventSource` close (`req.signal` abort), **then** the in-flight adapter call is cancelled, the correlation context is deleted, and NO audit event is appended for the client-side cancellation (spec Edge 8).
|
||
|
||
**LLM smoke (M2 gate item 8, P0 — not deferrable):**
|
||
- [ ] **Given** CI starts the control plane with `packages/llm-mock` as the BYOM endpoint and a real GitHub adapter (real PAT, test-org-scoped), **when** the smoke test sends `POST /v1/chat/completions` with `tools=[github.list_repos]` and prompt "List my GitHub repositories.", **then** llm-mock returns `tool_calls:[{function:{name:"github.list_repos", arguments:"{}"}}]`, the broker translates to MCP `tools/call`, routes to the GitHub adapter, the adapter calls `GET /user/repos` (real GitHub), the broker translates the result to an OpenAI tool message, llm-mock synthesizes a grounded response, and the test asserts the response contains real repo names from the CI test org.
|
||
|
||
---
|
||
|
||
## UX Acceptance Criteria (MVP/UX §3)
|
||
|
||
Pass/fail criteria (all must pass for M2 ship):
|
||
|
||
- [ ] **Adapter config saves in <5s** with validation feedback (success or HTTP 422 role/scope-violation error inline).
|
||
- [ ] **"Test connection" returns in <5s** with structured pass/fail; `adapter.test_connection.{succeeded,failed}` audit event appended.
|
||
- [ ] **Test-Call SSE stream starts within 100ms** of `POST /api/mcp/invoke` returning `{correlationId, streamUrl}` (NFR: SSE chunk delivery <100ms).
|
||
- [ ] **Staleness indicator** shows "cached Xs ago" for cached inventory (`list_*`) calls; no indicator for live calls.
|
||
- [ ] **Write attempts rejected at broker with 403 before adapter invocation** (INV-7 — verified by a test per adapter at the M2 gate; the adapter is never invoked for write methods).
|
||
- [ ] **SSH non-whitelist commands rejected at broker (layer 1) AND Relay Agent (layer 2)** (defense-in-depth, R-003 cross-layer test).
|
||
- [ ] **Multi-target picker surfaces** when ≥2 same-type adapters are configured; submitting without `target_id` returns HTTP 400 "target required."
|
||
- [ ] **Rate limit 429 with `Retry-After`** when user >60/min OR tenant >300/min; no adapter call made on 429.
|
||
- [ ] **LLM smoke returns grounded response** with real repo names from the CI test org (P0 gate item 8 — not deferrable to M3).
|
||
- [ ] **M1 non-regression:** all M1 tests still pass (gate item 1).
|
||
- [ ] **Coverage ≥ 80%** on new M2 modules (`packages/mcp/**`, `packages/llm-mock`, adapter UI components) (gate item 3).
|
||
|
||
---
|
||
|
||
## Wave F — MCP Gateway core (Phase 1)
|
||
|
||
**Goal:** The MCP capability broker gateway: closed tool registry, adapter router, write-method blocklist enforcer (INV-7 at broker — the load-bearing safety boundary), token-bucket rate limiter, SSE stream manager, OpenAI↔MCP translator, in-process custom transport, synthetic MCP `initialize` handshake (R-001). `mcp_adapters` table with RLS. **No adapter implementations yet** (those are waves G/H/I) — Wave F ships the broker with stub adapters for testing.
|
||
|
||
**Depends on:** Phase 0, Wave 0.
|
||
**REQs covered:** REQ-015, REQ-016, REQ-017, REQ-018, REQ-019, REQ-024.
|
||
**Personas:** backend-engineer (broker modules), data-engineer (table + RLS), security-engineer (sign-off on write-method blocklist + INV-7 at broker).
|
||
**Patch tag:** `v0.1.1`. **Branch:** `phase/01-mcp-gateway`.
|
||
|
||
### Tasks
|
||
|
||
1. **`mcp_adapters` table + migration + RLS** (data-engineer)
|
||
- Migration `packages/db/migrations/0002_mcp_adapters.sql`: `mcp_adapters (id uuid PK, tenant_id uuid, adapter_type text CHECK in (proxmox,ssh,github,gitea), target_id text, config jsonb, secret_ref text, validated boolean, created_at timestamptz, updated_at timestamptz)`. UNIQUE `(tenant_id, adapter_type, target_id)`. RLS policy: tenant-scoped SELECT/INSERT/UPDATE/DELETE with `WITH CHECK (tenant_id = current_setting('app.tenant_id')::uuid)`. `ALTER TABLE ... FORCE ROW LEVEL SECURITY`.
|
||
- Verify against real Postgres 16 in CI (Wave 0 `DB_MODE=pg` job), not PGlite.
|
||
|
||
2. **`packages/mcp/registry.ts`** — closed 9-tool registry with JSON Schema inputSchemas (backend-engineer)
|
||
- 9 tools (REQ-015 frozen set): `proxmox.list_vms` (inventory, `{node: string}`), `proxmox.get_vm_status` (live, `{node: string, vmid: integer}`), `proxmox.get_node_metrics` (live, `{node: string}`), `ssh.run_whitelisted_command` (live, `{command: string}`), `github.list_repos` (inventory, `{}`), `github.get_recent_ci_runs` (live, `{owner: string, repo: string, per_page?: integer, status?: string}`), `github.get_workflow_run` (live, `{owner: string, repo: string, run_id: integer}`), `gitea.list_repos` (inventory, `{}`), `gitea.get_recent_ci_runs` (live, `{owner: string, repo: string, limit?: integer}`).
|
||
- Each tool: `{name, description, inputSchema}` (JSON Schema object with `type:"object"`). Export `MCP_PROTOCOL_VERSION = "2025-06-18"`.
|
||
- Per-tenant policy: `disabledTools: Set<toolName>` — may disable individual tools but never add new ones (closed registry). Registry metadata `isInventory: boolean` is the cache authority (NOT MCP `annotations`, which are advisory/untrusted per R-001).
|
||
- Argument validation: any tool call with args not matching `inputSchema` → HTTP 400 with schema-validation error (Edge 4) BEFORE adapter invocation.
|
||
|
||
3. **`packages/mcp/router.ts`** — adapter router (backend-engineer)
|
||
- Resolves `(tenant_id, adapter_type, target_id)` tuples to adapter instances (REQ-016). Reads `mcp_adapters` under `withTenant` + RLS. Routing errors (unknown adapter, target offline) → HTTP 404 with structured error.
|
||
- Multi-target scope disambiguation (REQ-024): if a tenant has ≥2 adapters of the same type and no `target_id` is provided, return HTTP 400 "target required" with a list of available targets (Edge 3).
|
||
|
||
4. **`packages/mcp/write-blocklist.ts`** — per-adapter write-method blocklist enforcer (security-engineer + backend-engineer) — **INV-7 BACKSTOP** [G-015, G-016]
|
||
- **[G-015] Framing correction:** The closed 9-tool registry (REQ-015) is the PRIMARY INV-7 boundary — `proxmox.shutdown_vm` is not a tool and cannot be routed. The write-method blocklist is defense-in-depth against adapter bugs (an adapter mistakenly constructing a non-GET). Security review must audit BOTH the registry (closed enumeration) AND the blocklist (method reject). Document this framing in `PROTOCOL.md` and the module docstring.
|
||
- **[G-016] Two enforcement models:** (a) **Method blocklist** (Proxmox/Gitea: pre-dispatch HTTP-method check — reject POST/PUT/DELETE/PATCH; REST-specific). (b) **Scope-via-403** (GitHub: runtime 403 + `X-Accepted-GitHub-Permissions` handling, per R-004 — NOT a pre-dispatch method check, because GitHub fine-grained PAT scopes are not introspectable). These are distinct mechanisms; do not conflate them.
|
||
- **[G-016] Future risks documented in PROTOCOL.md:** "The method blocklist is REST-specific. A future GraphQL adapter (not in M2) needs a different enforcement model (operation allowlist, not HTTP method — GraphQL uses POST for both queries and mutations). PVE has some GET endpoints with side effects; the 3 M2 endpoints (`/nodes`, `/nodes/{node}/qemu`, `/nodes/{node}/qemu/{vmid}/status/current`, `/nodes/{node}/status`) are verified read-only. An endpoint allowlist (only permit specific paths) is the M3+ evolution if the tool set grows."
|
||
- Per-adapter blocklist (spec §5): Proxmox POST/PUT/DELETE; SSH non-whitelist commands (6-command subset per REQ-021); GitHub scopes outside `metadata:read`+`actions:read` (validated per-invocation via 403 + `X-Accepted-GitHub-Permissions`); Gitea POST/PUT/DELETE/PATCH on all endpoints.
|
||
- 100% of write attempts rejected at broker with HTTP 403 + `adapter.write_rejected` audit event; adapter NEVER invoked. **Verified by a test per adapter at the M2 gate.**
|
||
- Order of enforcement (R-007): auth → tenant resolve → RBAC → **rate-limit check** → **write-method blocklist** → adapter resolve → invoke. Rate limit is the outermost gate; write-blocklist is the INV-7 gate.
|
||
|
||
5. **`packages/mcp/rate-limiter.ts`** — token-bucket, in-memory (backend-engineer)
|
||
- Per user (capacity=60, refill=1/sec) AND per tenant (capacity=300, refill=5/sec). Both must pass (AND logic). Refund user token on tenant-fail (fairness, D-M2-R007). O(1) check <5ms (NFR).
|
||
- `RateLimiter` interface `Promise`-returning now (M2 sync impl wrapped in Promise) so M3 can swap in `RedisRateLimiter` with no signature change (D-M2-R007). Redis migration path documented in code comments.
|
||
- On reject: HTTP 429 + `Retry-After: <seconds>` header (RFC 7231). Body `{error:"rate_limited", retryAfterSec}`. **No adapter call made. Do NOT audit 429s** (not adapter events; could amplify a flood — log at warn level instead).
|
||
|
||
6. **`packages/mcp/stream-manager.ts`** — SSE stream manager (backend-engineer)
|
||
- In-memory `Map<correlationId, CorrelationContext>`. Context: `{correlationId, tenantId, userId, adapterType, toolName, controller?, abortController, createdAt}`.
|
||
- ULID correlation IDs (26-char, lexicographically sortable) minted at `POST /api/mcp/invoke` (add `ulid` npm dep to `packages/mcp`). SSE event format: `id: <ulid>-<seq>\nevent: tool_result\ndata: {"content":[...],"isError":false}\n\n`. Terminal events `done`/`error`.
|
||
- **30s stream-not-opened timeout (R-006):** if `GET /api/mcp/stream/:correlationId` isn't called within 30s of `POST /invoke`, cancel the adapter call and delete the context. Plus 60s max-stream lifetime safety net.
|
||
- Client disconnect (Edge 8): on `req.signal` abort, cancel in-flight adapter call (AbortController), delete context, **NO audit event for client-side cancellation.** Guard with a `closed` flag (abort may fire after normal close).
|
||
- Backpressure: cap controller queue at 100 events; if exceeded, cancel with "client too slow."
|
||
|
||
7. **`packages/mcp/translator.ts`** — OpenAI↔MCP translation (backend-engineer)
|
||
- `tool_calls[i].function.name` → `params.name`; `JSON.parse(tool_calls[i].function.arguments)` → `params.arguments` (parsed JSON object — **pitfall:** OpenAI sends `arguments` as a JSON string; MCP expects an object; handle parse failures as protocol errors, not tool execution errors).
|
||
- MCP `result.content[].text + isError` → OpenAI tool message `{role:"tool", tool_call_id, content}`. `isError:false` → `content: result.content[0].text` (concatenate if multiple). `isError:true` → `content: "ERROR: " + result.content[0].text` (M2 convention; OpenAI has no native `isError`).
|
||
|
||
8. **`packages/mcp/transport/in-process.ts`** — in-process custom MCP transport with synthetic `initialize`/`initialized` handshake (R-001) (backend-engineer)
|
||
- JSON-RPC 2.0 messages (`tools/list`, `tools/call`) passed in-process between broker and TS adapter modules (no wire serialization, but shape must match). D-007.
|
||
- Synthetic lifecycle exchange at adapter registration (R-001 mitigation): broker → `{method:"initialize", params:{protocolVersion:"2025-06-18", capabilities:{tools:{listChanged:false}}}}`, adapter → `{capabilities:{tools:{}}}`. Cheap; produces clean conformance evidence.
|
||
|
||
9. **API routes** (backend-engineer + frontend-engineer for route handlers)
|
||
- `GET /api/mcp/tools` — list tools (MCP `tools/list` facade; returns 9-tool closed set; per-tenant disabled tools filtered).
|
||
- `POST /api/mcp/invoke` — invoke a capability; mints ULID, creates correlation context, kicks off adapter call async, returns `{correlationId, streamUrl}`. Order: auth → tenant → RBAC → rate-limit → write-blocklist → resolve → invoke.
|
||
- `GET /api/mcp/stream/[correlationId]/route.ts` — SSE stream (`runtime = "nodejs"`, `dynamic = "force-dynamic"` per R-006). `Content-Type: text/event-stream`, `Cache-Control: no-cache`, `Connection: keep-alive`.
|
||
- `POST /api/mcp/adapter` — configure an adapter (J1 Step 3). Validates role/scope at submit (REQ-025/026/027), `SecretProvider.put`, INSERT under `withTenant` + RLS, `adapter.configured` audit.
|
||
- `PATCH/DELETE /api/mcp/adapter/:id` — update/remove adapter config.
|
||
- All routes use M1 patterns: `requireAdmin`/`requireRead` auth guard (mirrors `apps/control-plane/app/api/byom/route.ts`), `withTenant`, `appendAudit`.
|
||
|
||
10. **Audit event types** (backend-engineer) [G-012]
|
||
- **[G-012] M1-file edit (type widening, NOT "additive — no schema change" at the TS layer):** Extend `AuditEventType` in `packages/db/src/audit.ts` (M1 source file) with `adapter.configured | adapter.test_connection.succeeded | adapter.test_connection.failed | adapter.capability_invoked | adapter.write_rejected`. The DB column (`audit_log.event_type`) is `TEXT` with no CHECK constraint, so no DB migration. But `appendAudit(client, event: AuditEvent)` is typed to `event.eventType: AuditEventType` — passing the new types is a TS compile error without the union extension. This is a backward-compatible type widening (M1 tests still pass).
|
||
- New event types: `adapter.configured`, `adapter.test_connection.succeeded`, `adapter.test_connection.failed`, `adapter.capability_invoked`, `adapter.write_rejected`. All hash-chained via M1 `appendAudit` (`packages/db/src/audit.ts`). Include `correlation_id` field for capability invocations.
|
||
|
||
11. **Multi-target scope disambiguation (REQ-024)** (backend-engineer)
|
||
- Broker returns HTTP 400 "target required" with a list of available targets when ≥2 same-type adapters exist and no `target_id` is provided (Edge 3).
|
||
|
||
12. **MCP conformance verification artifact (R-001)** (backend-engineer + security-engineer) [G-017]
|
||
- `tests/mcp-conformance/` (**7** tests, all must pass — gate item 15):
|
||
1. `tools-list.test.ts` — `GET /api/mcp/tools` returns 9 tools with `{name, description, inputSchema}` matching REQ-015 exactly. Snapshot the full `tools/list` response.
|
||
2. `tools-call-happy.test.ts` — mock adapter invocation returns MCP result shape `{content:[{type:"text",text}], isError:false}` via SSE.
|
||
3. `tools-call-error.test.ts` — mock adapter `isError:true` returns MCP error shape via SSE `error` terminal event.
|
||
4. `tools-call-invalid-args.test.ts` — args failing `inputSchema` → HTTP 400 schema-validation error (broker rejects before adapter).
|
||
5. `translator.test.ts` — OpenAI `tool_calls` ↔ MCP `tools/call` bidirectional translation, including `arguments` string→object parse and `isError`→content prefix.
|
||
6. `lifecycle.test.ts` — in-process custom transport synthetic `initialize`/`initialized` handshake preserves JSON-RPC 2.0 envelope.
|
||
7. **[G-017] `stdio-interop.test.ts`** — connects to the broker via **stdio transport** (the real-transport path used by the LLM smoke), issues a real `tools/list` JSON-RPC request over stdin/stdout, asserts the response is a valid JSON-RPC 2.0 envelope with the 9 tools, then issues a `tools/call` for a mock adapter and asserts the result shape. **This is the test that proves an external MCP client can connect** — moves the lowest-confidence axis (0.80) to evidence-backed. Optionally: run the official MCP inspector against the broker as a CI step.
|
||
- `packages/mcp/PROTOCOL.md` documenting: pinned spec version `2025-06-18` with links to the three spec pages (tools, transports, lifecycle); transports used (in-process custom, stdio for LLM smoke, REST facade + SSE for UI — NOT Streamable HTTP, compliant as custom transport); JSON-RPC 2.0 shapes preserved; OpenAI ↔ MCP translation contract; synthetic lifecycle handshake. **[G-015]** INV-7 framing (registry is primary, blocklist is backstop). **[G-016]** Two enforcement models + GraphQL/PVE-GET future risks.
|
||
- `MCP_PROTOCOL_VERSION = "2025-06-18"` constant exported from `packages/mcp` and asserted in the conformance test header.
|
||
|
||
13. **Stub adapters for testing** (backend-engineer) [G-020]
|
||
- **[G-020] `McpAdapter` interface** shipped in `packages/mcp/types.ts`: both in-process adapters (G/I) and the WebSocket-backed SSH adapter (H) implement this interface. The interface specifies: `tools/list() → Promise<Tool[]>`, `tools/call(name: string, args: Record<string, unknown>) → Promise<McpResult>`, and a registration mechanism. F's stubs implement it; G/H/I's real adapters implement it. The router (T3) accommodates both in-process module adapters and WebSocket-backed adapters (the SSH adapter wraps a WebSocket round-trip inside `tools/call`). **This makes F→G/H/I a contract handoff, not a code-reading exercise — enables real parallelism.**
|
||
- Minimal mock adapters (one per type: proxmox, ssh, github, gitea) that the broker can route to, for testing the broker in isolation (waves G/H/I plug in real adapters). Each stub: registers via synthetic `initialize`, responds to `tools/list` with its tool subset, responds to `tools/call` with a canned `{content:[{type:"text",text:"stub"}], isError:false}` or `isError:true` for error tests. All stubs implement `McpAdapter`.
|
||
|
||
### Must-haves (verify before ship)
|
||
- [ ] Closed tool registry: 9 tools with `name`, `description`, `inputSchema` (JSON Schema per MCP 2025-06-18).
|
||
- [ ] Write-method blocklist: 100% of write attempts rejected at broker with 403 + audit event, adapter never invoked (test per adapter type with stubs).
|
||
- [ ] Rate limiter: 60/min user + 300/min tenant enforced, HTTP 429 + `Retry-After`, refund-on-tenant-fail, no audit on 429.
|
||
- [ ] SSE stream: per-call, ULID correlation IDs, terminal events `done`/`error`, 30s stream-not-opened timeout (R-006), <100ms chunk delivery.
|
||
- [ ] Client disconnect (Edge 8): in-flight adapter call cancelled, no audit event for client-side cancellation.
|
||
- [ ] Multi-target: ≥2 same-type adapters without `target_id` → HTTP 400 "target required" with available targets list.
|
||
- [ ] MCP conformance artifact: `PROTOCOL.md` + 6 tests passing (gate item 15).
|
||
- [ ] Synthetic `initialize`/`initialized` handshake for in-process adapters (R-001).
|
||
- [ ] OpenAI↔MCP translator: `tool_calls` → `tools/call` (with `JSON.parse(arguments)`); `result.content + isError` → tool message.
|
||
- [ ] `mcp_adapters` table with RLS (tenant-scoped, verified against real Postgres 16 in CI per Wave 0).
|
||
- [ ] Audit events: `adapter.configured`, `adapter.test_connection.{succeeded,failed}`, `adapter.capability_invoked`, `adapter.write_rejected` — all hash-chained via M1 `appendAudit`.
|
||
- [ ] Coverage ≥ 80% on `packages/mcp`.
|
||
- [ ] M1 non-regression: all M1 tests still pass.
|
||
- [ ] Security-engineer sign-off on write-method blocklist + INV-7 at broker (blocks ship on P0/P1 finding).
|
||
|
||
---
|
||
|
||
## Wave G — Proxmox adapter (Phase 2)
|
||
|
||
**Goal:** Read-only Proxmox VE adapter with PVEAuditor role validation. 3 capabilities: `proxmox.list_vms` (inventory), `proxmox.get_vm_status` (live), `proxmox.get_node_metrics` (live).
|
||
**Depends on:** Wave F (broker).
|
||
**REQs covered:** REQ-020, REQ-025.
|
||
**Personas:** backend-engineer (adapter), security-engineer (sign-off on PVEAuditor validation + write-blocklist).
|
||
**Patch tag:** `v0.1.2`. **Branch:** `phase/02-proxmox-adapter`.
|
||
|
||
### Tasks
|
||
|
||
1. **`packages/mcp/adapters/proxmox/client.ts`** (backend-engineer)
|
||
- PVE API client (HTTPS, port 8006, cookie/token auth). Uses API Token auth (NOT ticket/cookie — stateless, no 2h expiry, no CSRF needed for tokens per R-002): `Authorization: PVEAPIToken=USER@REALM!TOKENID=UUID` header on every GET.
|
||
- `pveGet(host, token, path, allowSelfSigned)`: global `fetch` with `AbortSignal.timeout(10_000)` (10s upstream NFR, mirrors `packages/byom/src/validator.ts` pattern). `allowSelfSigned` per-adapter config flag → `https.Agent({rejectUnauthorized: false})` for customer PVE labs (R-002 pitfall).
|
||
- Unwrap `{data: <payload>}` response shape. Check both `!res.ok` AND `body.data === null` (PVE returns null data for some not-found cases). 5xx → HTTP 502/504 to caller (transient upstream error, NOT write rejection).
|
||
- Endpoints (R-002, verified): `GET /api2/json/nodes`, `GET /api2/json/nodes/{node}/qemu`, `GET /api2/json/nodes/{node}/qemu/{vmid}/status/current`, `GET /api2/json/nodes/{node}/status`. **Pitfall:** `/api2/json/qemu` is NOT a valid endpoint (qemu is under a node). No version branching needed (endpoints stable PVE 6.x–8.x, R-002).
|
||
|
||
2. **`packages/mcp/adapters/proxmox/adapter.ts`** (backend-engineer)
|
||
- Implements the 3 capabilities (GET endpoints only — never POST/PUT/DELETE):
|
||
- `proxmox.list_vms` (inventory): `GET /api2/json/nodes/{node}/qemu` (requires `node` arg; returns VMs on that node). `inputSchema: {node: string (required)}`.
|
||
- `proxmox.get_vm_status` (live): `GET /api2/json/nodes/{node}/qemu/{vmid}/status/current`. `inputSchema: {node: string, vmid: integer}`.
|
||
- `proxmox.get_node_metrics` (live): `GET /api2/json/nodes/{node}/status`. `inputSchema: {node: string}`.
|
||
- Registers via synthetic `initialize` handshake (Wave F transport). Responds to `tools/list` with the 3 proxmox tools; responds to `tools/call` with `{content:[{type:"text", text: JSON.stringify(normalizedResult)}], isError:false}`.
|
||
|
||
3. **`packages/mcp/adapters/proxmox/validate.ts`** (backend-engineer + security-engineer)
|
||
- PVEAuditor role validation at submit time (REQ-025, R-002): call `GET /api2/json/version` (any valid token) → token is valid; then `GET /api2/json/nodes` → token has at least `Sys.Audit` (read access). If both succeed → `validated=true`. If either fails → HTTP 422 with role-violation error, no config persisted.
|
||
- **R-002 documented gap:** PVE has no clean "what role does this token have" introspection endpoint. The broker validates "token works for reads," NOT "token lacks writes." True `PVEAuditor` enforcement is the operator's responsibility at token creation time. Document in the Settings → Adapters UI help text: "Create a token with PVEAuditor role. The broker validates read access; the write-method blocklist (POST/PUT/DELETE → 403) is the load-bearing safety boundary." Confidence 0.70 on this sub-point.
|
||
- Record the PVE version (from `GET /api2/json/version` during `test_connection`) in the `mcp_adapters.config` JSON column for diagnostics. No version branching (R-002).
|
||
|
||
4. **SecretProvider integration for Proxmox token (INV-3)** (backend-engineer)
|
||
- `SecretProvider.put(tenantId, "proxmox:<targetId>", token)` on config; `SecretProvider.get` on invocation. DB stores only `secret_ref`. Token never logged; `SecretValue.unwrap()` passed directly to the fetch `Authorization` header.
|
||
|
||
5. **Audit events for Proxmox adapter** (backend-engineer)
|
||
- `adapter.configured` (on save), `adapter.test_connection.{succeeded,failed}` (on "Test connection"), `adapter.capability_invoked` (on each capability call, with `correlation_id`, params hash, result status), `adapter.write_rejected` (if POST/PUT/DELETE somehow reached the broker — belt-and-suspenders). All via M1 `appendAudit`.
|
||
|
||
6. **Tests** (backend-engineer + security-engineer)
|
||
- Mock PVE API (no live Proxmox in CI — use a mock fetch responder). Validate the 3 capabilities call only GET endpoints. Validate write-method blocklist: POST/PUT/DELETE → 403 + `adapter.write_rejected` at broker (adapter never invoked). Validate PVEAuditor validation: token failing `GET /version` → 422; token failing `GET /nodes` (no `Sys.Audit`) → 422. Validate 5xx from PVE → HTTP 502/504 (not write rejection). Validate `allowSelfSigned` flag.
|
||
|
||
7. **Inventory TTL cache (60s, LRU) for `proxmox.list_vms`** (backend-engineer)
|
||
- In-memory cache keyed by `(tenantId, targetId, toolName, argsHash)`. `list_*` capabilities cached for 60s; live capabilities (`get_vm_status`, `get_node_metrics`) never cached. Cache hit → return cached result with staleness metadata (`cachedAt: timestamp`) so the SSE event / UI can show "cached Xs ago."
|
||
|
||
### Must-haves
|
||
- [ ] 3 capabilities call only GET endpoints (never POST/PUT/DELETE).
|
||
- [ ] PVEAuditor validation: token validated at submit (`GET /version` + `GET /nodes`); UI documents the introspection gap (R-002).
|
||
- [ ] Token stored via `SecretProvider.put`; DB holds only `secret_ref`.
|
||
- [ ] Write-method blocklist: POST/PUT/DELETE → 403 + `adapter.write_rejected` audit event at broker (adapter never invoked).
|
||
- [ ] Inventory cache: `list_vms` served from 60s TTL cache on repeat calls; staleness surfaced.
|
||
- [ ] Coverage ≥ 80% on `packages/mcp/adapters/proxmox`.
|
||
- [ ] PVE 7.x and 8.x supported (no version branching needed per R-002).
|
||
- [ ] Security-engineer sign-off on PVEAuditor validation + write-blocklist.
|
||
|
||
---
|
||
|
||
## Wave H — SSH/Linux adapter (Phase 3)
|
||
|
||
**Goal:** Read-only SSH/Linux adapter via M1 Relay Agent. `ssh.run_whitelisted_command` capability. Defense-in-depth: broker validates command (layer 1) + Relay Agent `CheckCommand` (layer 2). **Full REQ-026** (M1 shipped the hook; M2 plugs the adapter in).
|
||
**Depends on:** Wave F (broker), M1 Relay Agent.
|
||
**REQs covered:** REQ-021, REQ-026 (full).
|
||
**Personas:** go-engineer (Wave H only — Relay Agent integration), backend-engineer (TS SSH adapter module), security-engineer (sign-off on defense-in-depth).
|
||
**Patch tag:** `v0.1.3`. **Branch:** `phase/03-ssh-adapter`.
|
||
|
||
### Tasks
|
||
|
||
1. **`packages/mcp/adapters/ssh/whitelist-check.ts`** (backend-engineer + security-engineer)
|
||
- Broker-side whitelist validation (layer 1): validates `command` against the 6-command subset BEFORE dispatch to the Relay Agent (REQ-021, defense-in-depth layer 1).
|
||
- 6 commands (spec §7 Q3, conservative subset of M1's broader whitelist):
|
||
- `uptime` — exact match (no args).
|
||
- `df -h` — exact match.
|
||
- `free -m` — exact match.
|
||
- `systemctl status <svc>` — prefix `systemctl status ` + service name (regex `^[a-zA-Z0-9_.-]+$`, max 64 chars — **pitfall:** sanitize to prevent injection like `systemctl status nginx; rm -rf /`).
|
||
- `journalctl -n <N>` — `journalctl -n ` + integer 1-500 (regex `^journalctl -n ([1-9][0-9]{0,2}|500)$`).
|
||
- `systemctl list-units --type=service` — exact match.
|
||
- `validateSshCommand(command: string): {ok: boolean, reason?: string}`. **Independent TS implementation** from the Go `CheckCommand` (R-003: two independent codepaths so a bug in one doesn't bypass the other). Layer 1 (6 commands) is stricter than layer 2 (M1's broader whitelist) — correct defense-in-depth.
|
||
- On rejection: HTTP 403 + `adapter.write_rejected` audit event; Relay Agent never reached.
|
||
|
||
2. **`packages/mcp/adapters/ssh/adapter.ts`** (backend-engineer)
|
||
- TS SSH adapter module; MCP `tools/call` in-process (Wave F transport), then sends a `tool_call` WebSocket message to the M1 Relay Agent (downstream WebSocket — D-007: the MCP JSON-RPC layer is in-process; the WebSocket to the Relay Agent is downstream transport, does not affect MCP conformance).
|
||
- `ssh.run_whitelisted_command` (live): `inputSchema: {command: string}`. Broker validates (layer 1) → resolve adapter → send `tool_call` over WebSocket to the connected Relay Agent for `target_id` → receive `tool_result` → return MCP result.
|
||
- Target routing (R-003): reverse index `targetsByTenant: Map<tenantId, Map<targetId, WebSocket>>` built on the M1 `connectedAgents` registry in `ws-server.ts`. If target offline → HTTP 404 (do NOT queue the call). Verify target belongs to the same tenant (RLS — `withTenant` + targets table `tenant_id`).
|
||
|
||
3. **`apps/relay-agent/wsclient/handler.go`** (go-engineer) [G-021]
|
||
- **[G-021] Reader goroutine restructure:** the M1 reader goroutine (`apps/relay-agent/wsclient/client.go:182-203`) currently unmarshals every message as `pongMessage` and `continue`s on parse failure — `tool_call` messages are silently dropped today. M2 must restructure the reader goroutine to **dispatch on `type` field BEFORE unmarshaling into a specific struct**: read `type` from the raw JSON, route `pong` to the existing handler, route `tool_call` to the new handler. The heartbeat `pongArrived` signaling must not break.
|
||
- Add `tool_call` message handler. Handler receives `{type:"tool_call", callId, command, timeoutMs}`, calls `CheckCommand(command)` (M1 G-004 contract, layer 2 — **signature `CheckCommand(cmd string) error` UNCHANGED**), executes via `exec.Command` with split argv (**NO shell** — `exec.Command("systemctl", "status", "nginx")`, never `sh -c "..."` — third enforcement layer against shell injection).
|
||
- Returns `{type:"tool_result", callId, stdout, stderr, exitCode}` (success) or `{type:"tool_result", callId, error:"whitelist rejected: ...", exitCode:-1}` (CheckCommand rejection) or `{type:"tool_result", callId, error:"timeout after 10s", exitCode:-1}` (timeout).
|
||
- 9.5s `exec.Command` timeout (R-003: agent times out 0.5s before the broker's 10s timeout so the agent returns a timeout result before the broker gives up → SSE stream closes cleanly).
|
||
|
||
4. **`apps/relay-agent/main.go`** (go-engineer)
|
||
- Wire the `tool_call` handler into the WebSocket message router. M1 non-regression: the `register`/`ping`/`pong` paths must continue to work; the heartbeat loop must not break.
|
||
|
||
5. **Cross-layer SSH test — divergence matrix (R-003)** (security-engineer) [G-013]
|
||
- **[G-013] Divergence matrix** (not just both-reject — also both-accept and divergence cases):
|
||
(a) Both reject `rm -rf /` (existing — `rm` not in 6-command subset at broker; `rm` not in M1 whitelist at Go).
|
||
(b) Both accept `systemctl status nginx` (new — proves both layers agree on a valid command).
|
||
(c) Broker rejects `systemctl status nginx rm -rf /` (regex fails on spaces in service name) — assert Go **also** rejects (currently Go accepts because `rm` is not in the deny list and the prefix `systemctl status` matches). **Fix:** tighten the Go deny list to include bare `rm` OR validate `systemctl status` trailing tokens against `^[a-zA-Z0-9_.-]+$` (matching the broker's regex). Choose option (ii) — validate trailing tokens — to make the two layers semantically equivalent for the 6-command subset.
|
||
(d) Go accepts `systemctl status nginx$(curl evil)` (deny list misses `$()`) — assert broker rejects (regex fails). Document that `exec.Command` with split argv runs `nginx$(curl evil)` as a literal service name (no shell expansion), so the Go layer is saved by the no-shell third layer, but the divergence is real and must be documented.
|
||
- All 4 cases pass on both layers; Go deny list / trailing-token validation tightened per (c).
|
||
|
||
6. **M1-relay-WS regression test (G-021)** (go-engineer + security-engineer)
|
||
- **[G-021]** Test asserting `register`→`registered` and `ping`→`pong` still work after the `tool_call` case is added to `ws-server.ts` `handleMessage` switch and the Go reader goroutine is restructured. This is an M1-non-regression test for the shared M1 files Wave H edits.
|
||
|
||
6. **10s upstream timeout** (backend-engineer + go-engineer)
|
||
- Broker: `AbortController` 10s on the `tool_call` → `tool_result` round-trip. On timeout → SSE `error` terminal event with "upstream timeout" + HTTP 504 semantics.
|
||
- Agent: `exec.Command` 9.5s context timeout (R-003 — agent times out first).
|
||
|
||
7. **SecretProvider integration for SSH registration token (INV-3)** (backend-engineer)
|
||
- `SecretProvider.put(tenantId, "ssh:<targetId>", relayRegistrationToken)` on config. The SSH adapter uses the M1 Relay Agent's existing auth (the registration token resolves to a connected target; the broker routes `tool_call` to that target's WebSocket).
|
||
|
||
8. **Audit events for SSH adapter** (backend-engineer)
|
||
- `adapter.configured` (on save), `adapter.test_connection.{succeeded,failed}` (on "Test connection" — e.g., ping the target via `uptime`), `adapter.capability_invoked` (on each `ssh.run_whitelisted_command` call, with `correlation_id`, command hash, result status), `adapter.write_rejected` (on broker layer-1 rejection).
|
||
|
||
### Must-haves
|
||
- [ ] Broker validates `command` against 6-command subset BEFORE dispatch (layer 1).
|
||
- [ ] Relay Agent `CheckCommand` validates at execution (layer 2, M1 G-004 contract unchanged).
|
||
- [ ] **[G-013]** Cross-layer divergence matrix (R-003): all 4 cases pass — both-reject `rm -rf /`, both-accept `systemctl status nginx`, broker-rejects-Go-also-rejects `systemctl status nginx rm -rf /` (after Go tightening), Go-accepts-broker-rejects `systemctl status nginx$(curl evil)` (documented divergence).
|
||
- [ ] **[G-021]** M1-relay-WS regression test: `register`→`registered` + `ping`→`pong` still work after `tool_call` addition + reader goroutine restructure.
|
||
- [ ] `tool_call` WebSocket message type added to Relay Agent (clean extension of M1 protocol; `register`/`ping`/`pong` still work).
|
||
- [ ] `CheckCommand(cmd string) error` signature UNCHANGED (G-004 contract lock).
|
||
- [ ] No shell in Go executor (`exec.Command` with split argv).
|
||
- [ ] 10s broker timeout + 9.5s agent exec timeout.
|
||
- [ ] `ssh.run_whitelisted_command` returns whitelisted command output via SSE.
|
||
- [ ] Coverage ≥ 80% on `packages/mcp/adapters/ssh` + relay-agent additions.
|
||
- [ ] Security-engineer sign-off on defense-in-depth (blocks ship on P0/P1 finding).
|
||
- [ ] go-engineer persona removed after Wave H ships; `apps/relay-agent/**` territory reverts to backend-engineer for M2 follow-up.
|
||
|
||
---
|
||
|
||
## Wave I — Git adapters (Phase 4)
|
||
|
||
**Goal:** Read-only GitHub + Gitea adapters. 5 capabilities: `github.list_repos`, `github.get_recent_ci_runs`, `github.get_workflow_run`, `gitea.list_repos`, `gitea.get_recent_ci_runs`.
|
||
**Depends:** Wave F (broker).
|
||
**REQs covered:** REQ-022, REQ-023, REQ-027.
|
||
**Personas:** backend-engineer (adapters), security-engineer (sign-off on scope validation).
|
||
**Patch tag:** `v0.1.4`. **Branch:** `phase/04-git-adapters`.
|
||
|
||
### Tasks
|
||
|
||
1. **`packages/mcp/adapters/github/client.ts`** (backend-engineer)
|
||
- GitHub REST API client (fine-grained PAT, `Authorization: Bearer <token>`, `Accept: application/vnd.github+json`, `X-GitHub-Api-Version: 2022-11-28` — stable GA version per R-004). Global `fetch` with `AbortSignal.timeout(10_000)`.
|
||
- `ghGet(path, token, query?)`: rate limit handling — observe `x-ratelimit-remaining`; if 0, do NOT make the call, return HTTP 429 to caller with `Retry-After: <seconds until x-ratelimit-reset>`. If GitHub returns 429 (or 403 with `x-ratelimit-remaining: 0`), back off exponentially (1s, 2s, 4s, max 3 retries) then surface 429.
|
||
- 403 with `X-Accepted-GitHub-Permissions` header (e.g., `actions=read` required) → `GithubScopeError` → broker surfaces HTTP 403 "insufficient scope" + `adapter.capability_invoked` with `result=failure` (NOT `write_rejected` — no write attempted; this is a scope mismatch, R-004).
|
||
|
||
2. **`packages/mcp/adapters/github/adapter.ts`** (backend-engineer)
|
||
- 3 capabilities (GET endpoints only):
|
||
- `github.list_repos` (inventory): `GET /user/repos?per_page=100`. Normalize to `{id, name, full_name, owner, private, description, html_url, default_branch, updated_at}`. `inputSchema: {}` (first page; multi-page is M3, R-004).
|
||
- `github.get_recent_ci_runs` (live): `GET /repos/{owner}/{repo}/actions/runs?per_page={per_page}`. Normalize to `{total_count, runs:[{id, head_branch, status, conclusion, html_url, created_at, actor}]}`. `inputSchema: {owner: string, repo: string, per_page?: integer (default 30, max 100), status?: string, branch?: string}`.
|
||
- `github.get_workflow_run` (live): `GET /repos/{owner}/{repo}/actions/runs/{run_id}`. `inputSchema: {owner, repo, run_id: integer}`.
|
||
|
||
3. **`packages/mcp/adapters/github/validate.ts`** (backend-engineer + security-engineer)
|
||
- Fine-grained PAT validation (D-006, R-004):
|
||
1. **Detect classic vs fine-grained:** classic PATs start with `ghp_`/`gho_`/`ghu_`; fine-grained start with `github_pat_`. Reject classic PATs at submit → HTTP 422 "fine-grained PAT required" (D-006: classic `repo` scope grants write).
|
||
2. **Validate token works + `metadata:read`:** `GET /user` with token. 401 → invalid token (HTTP 422). 200 → token valid; all fine-grained PATs require `metadata:read` implicitly, so a successful `GET /user` implies `metadata:read`.
|
||
3. **`actions:read` validated per-invocation:** when `github.get_recent_ci_runs`/`get_workflow_run` is called, if GitHub returns 403 with `X-Accepted-GitHub-Permissions` indicating `actions=read` required → HTTP 403 "insufficient scope" + audit `adapter.capability_invoked` with `result=failure`. Submit-time best effort (R-004 gap documented in UI help text: "ensure the PAT has `actions:read`").
|
||
|
||
4. **`packages/mcp/adapters/gitea/client.ts`** (backend-engineer)
|
||
- Gitea REST API client (`Authorization: token <token>` — **pitfall:** Gitea uses `token` not `Bearer`, R-005). Base URL = customer's Gitea host (`https://gitea.example.com/api/v1/`). `allowSelfSigned` per-adapter flag (customer Gitea often self-signed).
|
||
|
||
5. **`packages/mcp/adapters/gitea/adapter.ts`** (backend-engineer)
|
||
- 2 capabilities (GET endpoints only — no `gitea.get_workflow_run` in M2, deferred to v1.2+ per Q2):
|
||
- `gitea.list_repos` (inventory): `GET /api/v1/user/repos?limit=50`. Normalize to `{id, name, full_name, owner, private, description, html_url, default_branch, updated_at}`. `inputSchema: {}`.
|
||
- `gitea.get_recent_ci_runs` (live): `GET /api/v1/repos/{owner}/{repo}/actions/runs?limit={limit}`. `inputSchema: {owner, repo, limit?: integer (default 30, max 50)}`. **Pitfall:** Gitea Actions may be disabled (`actions.ENABLED=true` in app.ini) → 404; surface as "Gitea Actions not enabled on this instance" (HTTP 502, not write rejection).
|
||
|
||
6. **`packages/mcp/adapters/gitea/validate.ts`** (backend-engineer + security-engineer)
|
||
- Version-aware validation (R-005, spec §7 Q6):
|
||
1. `GET /api/v1/version` → parse `version`, compare major.minor to `1.22` (semver-ish; compare as integers). Record version in `mcp_adapters.config`.
|
||
2. Gitea ≥1.22: `GET /api/v1/user/repos?limit=1` → 200 = `read:repository` ok; 403 = insufficient scope → HTTP 422 "insufficient scope — `read:repository` required".
|
||
3. Gitea <1.22: `GET /api/v1/repos/search?limit=1` → 200 = token valid (any token accepted; no read-only scope available). Broker-side write-method blocklist (POST/PUT/DELETE/PATCH) is the security backstop.
|
||
|
||
7. **`packages/mcp/adapters/gitea/version-check.ts`** (backend-engineer)
|
||
- Gitea version detection + scope routing (R-005). **Verify against a running Gitea 1.22+ AND a <1.22 instance during Wave I** (R-005 action: confirm `GET /api/v1/version`, `GET /api/v1/user/repos`, `GET /api/v1/repos/{owner}/{repo}/actions/runs` against real instances; confirm `read:repository` scope behavior and `Authorization: token <token>` header).
|
||
|
||
8. **SecretProvider integration for Git tokens (INV-3)** (backend-engineer)
|
||
- `SecretProvider.put(tenantId, "github:<targetId>", pat)` / `SecretProvider.put(tenantId, "gitea:<targetId>", token)`. DB stores only `secret_ref`. Tokens never logged.
|
||
|
||
9. **Rate limit handling for GitHub** (backend-engineer)
|
||
- Observe `X-RateLimit-Remaining`; back off on 429 (R-004). M2 broker's own token-bucket (60/min user) is well below GitHub's 5000/hour, so the GitHub limit is unlikely to bind unless many tenants share a token (they shouldn't — per-tenant tokens).
|
||
|
||
10. **Inventory TTL cache (60s, LRU) for `github.list_repos` and `gitea.list_repos`** (backend-engineer)
|
||
- Same pattern as Proxmox (Wave G). `list_*` cached 60s; live capabilities (`get_recent_ci_runs`, `get_workflow_run`) never cached. Staleness surfaced.
|
||
|
||
11. **Audit events for Git adapters** (backend-engineer)
|
||
- `adapter.configured`, `adapter.test_connection.{succeeded,failed}`, `adapter.capability_invoked`, `adapter.write_rejected` (for Gitea POST/PUT/DELETE/PATCH blocklist violations). All via M1 `appendAudit`.
|
||
|
||
12. **Tests** (backend-engineer)
|
||
- GitHub: validated via real-target smoke in CI (real GitHub PAT, Wave 0 prerequisite). Validate 3 capabilities call only REST GET endpoints. Validate classic PAT rejection (`ghp_` prefix → 422). Validate fine-grained PAT (`github_pat_`) → `GET /user` → 200 = valid. Validate rate limit handling (mock 429 + `x-ratelimit-remaining: 0`).
|
||
- Gitea: validated via mock + verify against running instance (R-005). Validate 2 capabilities call only GET endpoints. Validate version-aware scope routing (≥1.22 `read:repository`; <1.22 any token + write blocklist). Validate `Authorization: token <token>` header. Validate Gitea Actions disabled → 404 → "not enabled".
|
||
|
||
### Must-haves
|
||
- [ ] GitHub: fine-grained PAT only (classic PAT rejected by `github_pat_` prefix); `metadata:read` + `actions:read` minimum (D-006).
|
||
- [ ] GitHub: 3 capabilities call only REST GET endpoints; rate limit handling (`X-RateLimit-Remaining`, backoff on 429); per-invocation 403 + `X-Accepted-GitHub-Permissions` handling (R-004).
|
||
- [ ] Gitea: version-aware scope validation (≥1.22 `read:repository`; <1.22 any token with broker write-method blocklist).
|
||
- [ ] Gitea: verify against running instance during Wave I (R-005).
|
||
- [ ] Write-method blocklist: GitHub scopes outside read-only → 403; Gitea POST/PUT/DELETE/PATCH → 403 + `adapter.write_rejected` at broker.
|
||
- [ ] Tokens stored via `SecretProvider.put`; DB holds only `secret_ref`.
|
||
- [ ] Inventory cache: `list_repos` served from 60s TTL cache on repeat calls; staleness surfaced.
|
||
- [ ] Coverage ≥ 80% on `packages/mcp/adapters/github` + `gitea`.
|
||
- [ ] Security-engineer sign-off on scope validation.
|
||
|
||
---
|
||
|
||
## Wave J — SSE integration + LLM smoke + adapter UI (Phase 5)
|
||
|
||
**Goal:** SSE integration end-to-end, `packages/llm-mock` CI-only LLM smoke provider, Settings → Adapters UI + Test-Call UI. **The LLM smoke (M2 gate item 8, P0 — not deferrable) must pass before M2 ships.**
|
||
**Depends on:** Wave F (broker) + at least Wave I (GitHub adapter for real-target smoke).
|
||
**REQs covered:** REQ-017 integration (SSE to UI), M2 gate item 8 (LLM smoke).
|
||
**Personas:** frontend-engineer (UI), backend-engineer (llm-mock + SSE integration).
|
||
**Patch tag:** `v0.1.5`. **Branch:** `phase/05-sse-integration`.
|
||
|
||
### Tasks
|
||
|
||
1. **`packages/llm-mock/`** — CI-only mock LLM provider (backend-engineer) [G-019]
|
||
- `devDependency` (not a production dependency) — R-008. Implements OpenAI-compatible `/v1/chat/completions` accepting the `tools` parameter (OpenAI tool definitions from the broker's `GET /api/mcp/tools`).
|
||
- **[G-019] Hardened pattern matching** (regex set, not 2-word conjunction — tolerates prompt wording drift):
|
||
- `/(list|show|get|display).*\b(repo|repositor)/i` → `tool_calls:[{id:"call_1", type:"function", function:{name:"github.list_repos", arguments:"{}"}}]`.
|
||
- `/(recent|latest|last).*\b(run|ci|workflow)/i` → `tool_calls:[{function:{name:"github.get_recent_ci_runs", arguments:'{"owner":"...","repo":"..."}'}}]`.
|
||
- Tests assert the pattern matches "Show me my GitHub repositories", "List my repos", "Get repositories", "What are my recent CI runs" — wording-tolerant.
|
||
- Accepts follow-up `tool` role messages (broker's translated adapter result). On second call (with a tool message present): synthesize grounded text — parse repo names from the tool message content, return `"Your repos are: <names>"`.
|
||
- Deterministic — no randomness; the smoke test asserts specific repo names appear (R-008).
|
||
- Reuse BYOM request/response types from `packages/byom/src/types.ts` (D-001 contract).
|
||
|
||
2. **`apps/control-plane/app/(dashboard)/settings/adapters/page.tsx`** — Settings → Adapters UI (frontend-engineer) [G-014]
|
||
- Adapter type picker (4 types), per-adapter config forms, SecretProvider-backed credential entry (redacted after submit, edit-only), role/scope validation on submit (REQ-025/026/027), "Test connection" button (REQ-016), audit event confirmation, multi-target support (target_id per row), PVEAuditor introspection gap help text (R-002), GitHub fine-grained PAT help text (D-006), Gitea version-aware help text (R-005).
|
||
- **[G-014] Closed-tool-set gap documentation in UI help text** (sets operator expectations pre-ship so the "additions require spec amendment" gate is politically enforceable):
|
||
- SSH: "M2 supports 6 diagnostic commands (`uptime`, `df -h`, `free -m`, `systemctl status`, `journalctl -n`, `systemctl list-units`). `ps`, `ss`, `top`, `ip` are deferred to v1.2+."
|
||
- Proxmox: "`list_vms` requires a `node` argument. A `list_nodes` tool is deferred to v1.2+."
|
||
- GitHub: "`list_repos` returns up to 100 repos (first page). Pagination, PR lists, and issue lists are deferred to v1.2+."
|
||
- Gitea: "`get_workflow_run` is deferred to v1.2+."
|
||
- Server components read via the API gateway (never bypass RLS — D-005 pattern).
|
||
|
||
3. **`apps/control-plane/app/(dashboard)/test-call/page.tsx`** — Test-Call UI (frontend-engineer)
|
||
- Capability picker (closed 9-tool set from `GET /api/mcp/tools`, grouped by adapter type, per-tenant disabled tools greyed out). Argument forms (rendered from JSON Schema `inputSchema`, required fields marked, type-validated). Target picker for multi-target tenants (REQ-024). SSE stream consumer (`EventSource` on `GET /api/mcp/stream/:correlationId`, renders events as they arrive, terminal `done`/`error` close the stream). Staleness indicator for inventory calls ("cached Xs ago"). Result rendering (JSON tree, `isError` flag surfaced).
|
||
|
||
4. **SSE consumer in Test-Call UI** (frontend-engineer)
|
||
- `EventSource` opens on the `streamUrl` from `POST /api/mcp/invoke`. Renders `tool_result` events incrementally; terminal `done`/`error` close the stream. <100ms chunk delivery (NFR). On client disconnect (page close), the broker detects `req.signal` abort and cancels the in-flight adapter call (Edge 8).
|
||
|
||
5. **LLM smoke test — TWO-TRACK (M2 gate item 8)** (backend-engineer) [G-018, G-019]
|
||
- **[G-018] Two-track smoke** (mock-path is the P0 gate; real-path is optional/allow-failure — ensures the gate is reliable regardless of GitHub availability):
|
||
- **Track A — Mock-path smoke (P0 gate, runs ALWAYS, no external dependency):**
|
||
1. CI starts the control plane with `packages/llm-mock` as the BYOM endpoint + a `github-mock` adapter (deterministic canned-repo adapter, distinct from the broker stub — returns `[{"name":"coreci-test-repo-1"},{"name":"coreci-test-repo-2"}]`).
|
||
2. Test sends `POST /v1/chat/completions` with `tools=[github.list_repos definition]` and prompt "List my GitHub repositories."
|
||
3. llm-mock returns `tool_calls:[{function:{name:"github.list_repos", arguments:"{}"}}]`.
|
||
4. Broker's translator converts to MCP `tools/call` → broker routes to `github-mock` adapter → canned repo data.
|
||
5. Broker's translator converts the MCP result to an OpenAI tool message.
|
||
6. Test sends a second `POST /v1/chat/completions` with `messages=[original prompt, assistant tool_call, tool message]`.
|
||
7. llm-mock synthesizes a grounded response ("Your repos are: coreci-test-repo-1, coreci-test-repo-2"). Test asserts the response contains the canned repo names.
|
||
- **This track proves the OpenAI→MCP→adapter→result→synthesis integration with zero external dependencies. It is the P0 gate.**
|
||
- **Track B — Real-path smoke (optional, runs when `secrets.GITHUB_SMOKE_PAT` is available, `allow-failure` — does NOT block the gate):**
|
||
1. Same flow as Track A but against the real GitHub adapter (real PAT, test-org-scoped).
|
||
2. Test asserts the response contains real repo names from the CI test org.
|
||
3. **[G-019] Retry policy:** on 429/5xx/timeout, retry up to 3 times with exponential backoff (1s, 2s, 4s); on final failure, skip with a warning (the mock-path smoke is the gate, not this track).
|
||
- **This track proves real-target connectivity. It is the ideal, not the gate.**
|
||
- **P0 — not deferrable:** Track A (mock-path) must pass reliably in CI (no GitHub dependency). Track B (real-path) is optional/allow-failure. This ensures the M2 gate is reliable regardless of GitHub availability.
|
||
|
||
6. **Import guard (R-008)** (backend-engineer)
|
||
- `packages/llm-mock` is a `devDependency` of the CI test package (or `apps/control-plane` devDeps), NOT a `dependency`. `pnpm install --prod` excludes it.
|
||
- Eslint rule `no-restricted-imports` banning `@coreci/llm-mock` in `apps/control-plane/app/**` and `packages/mcp/**` (prod code paths). Allowed only in `tests/**` and `packages/llm-mock/**`.
|
||
- Build-time check: CI step greps the prod build output (`.next/` or `dist/`) for `llm-mock` and fails if found.
|
||
- **Pitfall:** the mock must not be imported transitively by a prod dependency. The broker talks to it over HTTP (as a BYOM endpoint), not via import.
|
||
|
||
### Must-haves
|
||
- [ ] `packages/llm-mock` is a `devDependency`; import-guarded against prod bundle (eslint + build-time grep).
|
||
- [ ] **[G-018]** Two-track LLM smoke: Track A (mock-path, `github-mock` adapter, canned repos) passes reliably in CI — **this is the P0 gate**.
|
||
- [ ] **[G-018]** Track B (real-path, real GitHub) runs when PAT available, `allow-failure` — does NOT block the gate.
|
||
- [ ] **[G-019]** llm-mock pattern matching hardened (regex set, tolerates wording drift); retry policy for real-GitHub track.
|
||
- [ ] **[G-014]** Settings → Adapters UI: closed-tool-set gap documentation in help text (SSH/Proxmox/GitHub/Gitea limitations).
|
||
- [ ] Settings → Adapters UI: adapter type picker (4 types), config forms, validation on submit (REQ-025/026/027), "Test connection" button, multi-target support, help text for introspection gaps (R-002, D-006, R-005).
|
||
- [ ] Test-Call UI: capability picker (9 tools), argument forms (JSON Schema validated), target picker for multi-target (REQ-024), SSE stream consumer, staleness indicator for inventory calls.
|
||
- [ ] SSE stream renders in Test-Call UI within 100ms of events.
|
||
- [ ] Coverage ≥ 80% on `packages/llm-mock` + UI components.
|
||
|
||
---
|
||
|
||
## Final Phase — Review + Audit + Ship (Phase 6)
|
||
|
||
**Goal:** Multi-persona code review + project health audit + milestone ship (`v0.1.6` release, merge to `main`).
|
||
**REQs covered:** all M2 (sign-off).
|
||
**Personas:** lead-developer (review + audit + ship), all personas (review).
|
||
**Patch tag:** `v0.1.6` ← **M2 milestone release**. **Branch:** `phase/06-final-review-ship`.
|
||
|
||
### Tasks
|
||
|
||
1. **Code review** (lead-developer + all personas)
|
||
- Review all changes in `milestone/v0.2` (or the M2 integration branch) since `main`. Auto-apply P0 fixes; flag P1+ for post-hoc. Security-engineer reviews INV-7 at broker, write-method blocklist per adapter, SSH defense-in-depth, fine-grained PAT scope validation, Gitea version-aware scope validation. go-engineer reviews Relay Agent `tool_call` additions + `CheckCommand` contract lock (Wave H only, then removed).
|
||
|
||
2. **Audit** (lead-developer)
|
||
- Reconstruction test: `git log` matches `.ciagent/` files (every decision, research finding, persona assignment traceable to a commit). File/branch/commit discipline verified (all commits on `phase/NN-*` branches with `---ci---` blocks; no direct commits to `main`).
|
||
- M2 gate items 1-15 all pass (spec §6).
|
||
|
||
3. **M2 acceptance gate verification** (lead-developer)
|
||
- Spec §6 — all 13 REQs (015-027) pass with Given/When/Then coverage.
|
||
- Mocks + real GitHub smoke + LLM smoke (P0 gate item 8) + INV-7 verified (per-adapter write-blocklist tests) + CI Postgres RLS verified (Wave 0) + M1 non-regression.
|
||
- Coverage ≥ 80% on new M2 modules; DB coverage ≥ 80% maintained on `packages/db`.
|
||
|
||
4. **MCP conformance verification artifact review (R-001)** (lead-developer + security-engineer)
|
||
- `packages/mcp/PROTOCOL.md` + 6 conformance tests in `tests/mcp-conformance/` reviewed and confirmed (gate item 15).
|
||
|
||
5. **Milestone ship** (lead-developer)
|
||
- Tag `v0.1.6` (final phase patch = M2 milestone release).
|
||
- Merge `phase/06` → `milestone/v0.1` (or `main` per branch hierarchy) → `main`.
|
||
- Create Gitea release (`https://git.cloudinit.dev/coreci/coreci-chat`) with full M2 summary.
|
||
|
||
6. **Complete** (lead-developer)
|
||
- Mark all M2 REQs (015-027) complete in `REQUIREMENTS.md`.
|
||
- Mark M2 complete in `ROADMAP.md`.
|
||
- Clear checkpoint.
|
||
|
||
---
|
||
|
||
## Wave ordering & parallelism
|
||
|
||
```
|
||
Phase 0 (this plan) ──▶ Wave 0 (prerequisites) ──▶ Wave F (gateway core)
|
||
│
|
||
┌───────────────────────────────────┼───────────────────────┐
|
||
│ │ │
|
||
├──▶ Wave G (Proxmox) ├──▶ Wave H (SSH) ├──▶ Wave I (Git adapters)
|
||
│ │ │
|
||
└───────────────────────────────────┴───────────────────────┘
|
||
│
|
||
▼
|
||
Wave J (SSE + LLM smoke + UI)
|
||
│
|
||
▼
|
||
Final (review + audit + ship)
|
||
```
|
||
|
||
- **F must complete first** (all adapters depend on the broker).
|
||
- **G, H, I can run in parallel after F** (different adapter territories; no cross-dependencies). Wave H reactivates the go-engineer persona for Relay Agent integration only.
|
||
- **J depends on F + at least I** (GitHub adapter for the real-target LLM smoke, gate item 8).
|
||
- **Final depends on all.**
|
||
|
||
---
|
||
|
||
## Test strategy
|
||
|
||
- **Unit:** vitest in `packages/mcp/**` + `apps/control-plane`; Go `testing` in `apps/relay-agent`.
|
||
- **Integration:** Postgres 16 container in CI (Wave 0); RLS + `withTenant` + audit tests against real Postgres (replaces PGlite-only verification, R-009). Two CI jobs: `test-pglite` (default) + `test-postgres` (service container + `DB_MODE=pg` + role setup).
|
||
- **Adapter validation:**
|
||
- Proxmox: mock PVE API (no live Proxmox in CI).
|
||
- SSH: mock + cross-layer test (R-003).
|
||
- Gitea: mock + verify against running instance during Wave I (R-005).
|
||
- GitHub: real-target smoke in CI (real PAT, Wave 0 prerequisite — gate item 7).
|
||
- **LLM smoke:** `packages/llm-mock` driving the full OpenAI→MCP→adapter→result→synthesis path against real GitHub (P0 gate item 8 — not deferrable).
|
||
- **MCP conformance:** `PROTOCOL.md` + 6 tests in `tests/mcp-conformance/` (R-001 artifact, gate item 15).
|
||
- **Cross-layer SSH test:** broker (layer 1) + Relay Agent (layer 2) both reject non-whitelisted commands (R-003).
|
||
- **Coverage gate:** ≥ 80% on new M2 modules (gate item 3); DB coverage ≥ 80% maintained (gate item 4).
|
||
- **M1 non-regression:** all M1 tests still pass (gate item 1).
|
||
|
||
---
|
||
|
||
## Decisions logged (to DecisionEngine)
|
||
|
||
- **D-M2-P001:** Wave 0 is a hard prerequisite for Wave F — CI Postgres 16 + role setup (`coreci_app` no BYPASSRLS, `migrator` BYPASSRLS) + real GitHub PAT. RLS assertions gated on `DB_MODE=pg` replace M1's PGlite-only placeholder. Confidence 0.90.
|
||
- **D-M2-P002:** Wave F ships the broker with stub adapters (not real adapters); real adapters land in waves G/H/I. This isolates broker verification (INV-7, rate-limit, SSE, conformance) from adapter upstream concerns. Confidence 0.85.
|
||
- **D-M2-P003:** G/H/I parallel after F; J depends on F + at least I (GitHub for real-target LLM smoke). go-engineer active for Wave H only, removed after. Confidence 0.85.
|
||
- **D-M2-P004:** v0.1.6 (final phase patch) IS the M2 milestone release; merge to `main` at the final phase. Tags run on v0.1.x patch line (M1's previous minor). Confidence 0.90.
|
||
|
||
All above the 0.6 threshold. No escalations. Pipeline proceeds to GRILL.
|
||
|
||
---
|
||
|
||
*End of M2 PLAN. M1 PLAN preserved in git history (commit prior to M2 overwrite).* |