From 85dce28ca3ec9febb4734d5cccff4ec539fb8c84 Mon Sep 17 00:00:00 2001 From: CIAgent Date: Tue, 25 Aug 2026 12:41:16 +0000 Subject: [PATCH] =?UTF-8?q?docs(PDLC):=20lock=20M3=20spec=20v1.2=20Final?= =?UTF-8?q?=20=E2=80=94=20Chat,=20Orchestration,=20Hardening?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 15 REQs (028-037, 041-045) with REQ-035 amendment (auto-promote at step 21). 3 personas (Devon/Sam/Casey), 5 journeys + J-infra, no orphan REQs. Decisions #1-#10 locked; OQ1-6 resolved, OQ7 open (control_state write-through). Migrations 0004-0006 (chat/workflow/posture), per-table for rollback isolation. STATE.md updated: M3 spec locked, implementation pending. ---ci--- phase: 3 stage: spec-lock milestone: v0.2-M3 spec: .ciagent/steer-m3-spec.md spec_version: v1.2 status: ready-for-handoff reqs: [028,029,030,031,032,033,034,035,036,037,041,042,043,044,045] req035_amended: true invariants: [INV-1,INV-2,INV-3,INV-4,INV-5,INV-6,INV-7,INV-8] release: none --- .ciagent/STATE.md | 20 +-- .ciagent/steer-m3-spec.md | 270 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 280 insertions(+), 10 deletions(-) create mode 100644 .ciagent/steer-m3-spec.md diff --git a/.ciagent/STATE.md b/.ciagent/STATE.md index 164054b..1963567 100644 --- a/.ciagent/STATE.md +++ b/.ciagent/STATE.md @@ -2,15 +2,15 @@ ## 1. Header (mandatory) Project: coreci-chat -Initiative: CoreCI Chat v0.1 — Read-Only Diagnostic MVP (M1 + M2 complete; next: M3 Chat/Orchestration/Hardening) +Initiative: CoreCI Chat v0.1 — M3 (Chat, Orchestration, Hardening) spec locked v1.2; M1+M2 shipped, M3 spec ready for engineering handoff Initiator: ciagent (autonomous, full autonomy) -Date (UTC): 2026-08-25T07:00:00Z (updated post-M2 audit) -Current Version: v0.1.6 (M2 milestone release complete; all 13 phases shipped to main + Gitea upstream) -System Health: GREEN — M1 complete (v0.0.1-v0.0.7, 17 REQs, 189 tests), M2 complete (v0.1.0-v0.1.6, 13 REQs, 656 tests), MCP 2025-06-18 conformance verified +Date (UTC): 2026-08-25T11:00:00Z (updated post-M3 spec lock v1.2) +Current Version: v0.1.6 (M2 milestone release complete; M3 spec v1.2 Final locked, not yet implemented) +System Health: GREEN — M1 complete (v0.0.1-v0.0.7, 17 REQs, 189 tests), M2 complete (v0.1.0-v0.1.6, 13 REQs, 656 tests), MCP 2025-06-18 conformance verified; M3 spec v1.2 Final ready for engineering handoff Raw Idea (≤ 3 sentences): M1 (read-only diagnostic MVP foundation) shipped: SSO, BYOM, Relay Agent, dashboard, audit, RLS, secrets. M2 (MCP Layer & Day 1 Adapters) shipped: MCP capability broker gateway, 4 adapters (Proxmox, SSH/Linux, GitHub, Gitea), SSE streaming, rate limiting, LLM smoke, CI pipeline (Gitea Actions). - Desired outcome: M3 (Chat, Orchestration, Hardening, REQ-028..037+041..044) to complete v0.1. + Desired outcome: M3 (Chat, Orchestration, Hardening, REQ-028..037+041..045) to complete v0.1 — spec locked v1.2, implementation pending. ## 2. Architecture State Active Layers (which exist and are stable): @@ -83,7 +83,7 @@ Out-of-Scope (hard): Write actions (v1.1), hosted LLM inference (never), Kuberne ## 5. Recent History & Quality Gates (last 1-2 milestones) Last Shipped: v0.2 M2 — 2026-08-25, 7 phases (P0 pre-execution → P5 Wave J SSE+smoke+UI → P6 final review+ship), 13 REQs (015-027), 656 tests, shipped to Gitea v0.1.0-v0.1.6 + releases #833-#839 -In Progress: Nothing — M2 milestone complete. Next: M3 (REQ-028..037, REQ-041..044, Chat/Orchestration/Hardening) not started. +In Progress: M3 spec v1.2 Final locked (2026-08-25) — 15 REQs (028-037, 041-045), ready for engineering handoff. Implementation not started. Sub-phases M3.a (chat inline), M3.b (orchestration+durability), M3.c (hardening+metering). Coverage Floor: 92.3% (packages/mcp, the M2 critical-path package; gate ≥80% per spec §6). packages/db 98.2%, packages/llm-mock 97%. Recent Incidents: none Known Tensions: @@ -106,12 +106,12 @@ Agent Assumptions: - PGlite is a sufficient dev/test substitute for Postgres 16 (RLS limitation documented, CI backstops) - The M2 acceptance gate can be verified via unit/integration tests + LLM smoke Track A without a prod deployment - Gitea releases fulfill the "distribution packages" requirement - - The next PDLC cycle is M3 (Chat, Orchestration, Hardening, REQ-028..037, REQ-041..044) + - The next PDLC cycle is M3 (Chat, Orchestration, Hardening, REQ-028..037, REQ-041..045) — spec v1.2 Final locked ## 7. Canonical State References (Version/Hash) Vision/Strategy doc: CoreCI Chat Vision v1.0 (referenced by spec, not in repo) Architecture document: .ciagent/ARCHITECTURE.md @ commit 0c15d3d (M2 milestone merge to main) -Last approved SPECs: .ciagent/steer-v0.1-spec.md v1.1 (M1, locked 2026-08-24) + .ciagent/steer-m2-spec.md v1.0 (M2, locked 2026-08-25) -Decision log: .ciagent/CLARIFY.md (D-001..D-007) + .ciagent/GRILL.md M1 (G-001..G-010) + M2 (G-011..G-022) -Invariants catalog: .ciagent/ARCHITECTURE.md §Architecture invariants (INV-1..INV-8) + M2 addition +Last approved SPECs: .ciagent/steer-v0.1-spec.md v1.1 (M1, locked 2026-08-24) + .ciagent/steer-m2-spec.md v1.0 (M2, locked 2026-08-25) + .ciagent/steer-m3-spec.md v1.2 (M3, locked 2026-08-25) +Decision log: .ciagent/CLARIFY.md (D-001..D-007) + .ciagent/GRILL.md M1 (G-001..G-010) + M2 (G-011..G-022) + M3 spec §Key Decisions #1..#10 (no new D-* IDs; REQ-035 amendment + Decisions #7-#10 baked into steer-m3-spec.md) +Invariants catalog: .ciagent/ARCHITECTURE.md §Architecture invariants (INV-1..INV-8) + M2 addition; M3 preserves all as written (session-batched Merkle is additive, not INV-4 amendment) Review artifacts: .ciagent/M1-REVIEW.md + .ciagent/M2-REVIEW.md + .ciagent/M2-VERIFY-P01.md \ No newline at end of file diff --git a/.ciagent/steer-m3-spec.md b/.ciagent/steer-m3-spec.md new file mode 100644 index 0000000..4a1afe9 --- /dev/null +++ b/.ciagent/steer-m3-spec.md @@ -0,0 +1,270 @@ +# CoreCI Chat — M3 Specification (Chat, Orchestration, Hardening) + +**Owner:** Sarah Chen (Sr. PM) +**Status:** Final (v1.2) +**Type:** Milestone +**Target Milestone:** v0.2 — M3 (Chat, Orchestration, Hardening) +**Locked:** 2026-08-25 + +> **Operating Principles for this Spec:** +> 1. **Incremental Delivery:** This spec defines net-new work only. Pre-existing systems and locked architectures (M1, M2) are referenced, not restated. +> 2. **Zero Ambiguity:** If a requirement cannot be translated into a pass/fail test by QA, it is incomplete and will be rejected by Engineering. +> 3. **Invariants preserved:** INV-1..INV-8 from `.ciagent/ARCHITECTURE.md` hold as written. M3 adds no invariant amendments (Q5 session-batched Merkle is additive, not a semantic change to INV-4). + +--- + +## 1. Objective + +M3 delivers a diagnostic copilot that lets operators ask natural-language questions about tenant infrastructure and receive cited, evidence-backed answers within the M3 acceptance gate. It ships chat UI with streamed tool execution (≤20 steps inline, auto-promoted to durable Trigger.dev workflows beyond), persistent conversation history, per-inference usage metering, a SOC2 posture page with live control probes, and Vanta evidence sync. Three personas benefit: Devon (Operator) gets the copilot; Sam (Admin) gets posture + usage visibility; Casey (Compliance Reader) gets read-only attestation surfaces. The milestone closes v0.1 with 15 REQs total and refines (does not rewrite) the acceptance gate. + +*Acceptance Gate:* A developer reading this can state: "M3 ships a multi-turn diagnostic copilot that streams tool execution, persists durable workflows, meters usage, and instruments SOC2 controls." + +--- + +## 2. Scope & Target Milestones + +### 2.1 In Scope (Explicit Additions) + +15 REQs in three clusters: + +**Chat & Orchestration (REQ-028..037):** chat UI surface, natural-language input, streamed response with inline citations, streamed tool traces, conversation history (first-class data model), LLM tool reasoning, multi-step workflows, ≤20-step inline limit (amended: auto-promote at 21), durable execution via Trigger.dev, workflow rejoin. New packages: `packages/orchestrator` (step counter, promotion logic), `packages/chat` (stream manager, citation builder). New schemas: `chat_sessions`, `chat_turns`, `workflow_events` (additive per `steer-m2-spec.md:162`). API contracts: `/api/chat/stream` (inline + durable rejoin via `Last-Event-ID`), `workflow.promoted` event (inline→durable handoff). + +**Hardening (REQ-041..045):** REQ-041 SOC2 posture page with live control probes (5-min default interval, configurable per tenant) + `control_state` cache table (write-through with probe transaction; Decision #8); REQ-042 Vanta evidence sync (1-h default, no signed attestations); REQ-043 usage metering (per-inference `usage` block from REQ-030 feeds the pipeline; aggregation 1-min default); REQ-044 usage dashboard (tenant-facing rendering); REQ-045 Redis-backed rate aggregation (cross-process coordination between API gateway and Trigger.dev workers, replacing M2's in-memory token bucket; fail-closed 503 if Redis down). + +**Architectural amendments:** REQ-035 amendment text baked in (§4). New audit event types: `chat.session.created`, `chat.turn.*`, `llm.inference.{requested,succeeded,failed}`, `tool.plan.{emitted,selected,rejected}`, `workflow.{started,step.*,promoted,completed,timeout,promotion_failed,rejoined}`, `control.probe.{succeeded,failed}`, `vanta.sync.{succeeded,failed}`, `rate.redis.unavailable`, `evidence.exported`, `authz.denied`. + +### 2.2 Out of Scope (Explicit Exclusions) + +Two categories. Anti-goals (vision spec §2.2 — never / post-MVP / v1.2+) vs. M4+ deferrals (no REQ slot in M3 set): + +**Anti-goals (vision §2.2, not M3 deferrals):** write actions on infrastructure (v1.1+, INV-7 holds); hosted LLM inference (never, INV-5 holds, planner split rejected per Phase 1 Q3); RAG / vector retrieval (v1.1+); SOC 2 final certification (post-MVP); custom RBAC roles (v1.2+); BYOK / customer-managed keys (v1.2+); multi-region deployment (MVP single-region); Windows support (not planned v1.x); fine-tuning (not planned); Kubernetes / ArgoCD / Helm; Slack / Teams / CLI / mobile clients (v1.1+); approval-gated remediation (v1.1+). + +**M4+ deferrals (no REQ slot exists in M3 — each item carries REQ-gap reasoning):** +- **Secret rotation** — `SecretProvider` interface exists (REQ-039, M1); rotation is an ops process. No REQ slot in 028-037/041-045. +- **Production deployment of M1+M2+M3** — ops track parallel to PDLC. M3 REQs satisfied by code shipping to `main` + Gitea releases. No REQ slot. +- **Adapter SSRF/CSRF guardrails** — tenant-supplied hostnames are trusted per M2 spec (`steer-m2-spec.md:172`); hardening requires a new REQ. No REQ slot. +- **OTel / RED metrics / distributed tracing** — observability stack. No REQ slot. REQ-041 is posture page, not instrumentation. +- **SBOM / supply-chain / signed releases** — No REQ slot. +- **Gitea Actions runner enablement** — M2 CI prereq, ops responsibility; blocks full M2 verification, not M3 REQ satisfaction. Listed as ship-blocker prereq in §6. +- **Track B GitHub smoke (`GITHUB_SMOKE_PAT`)** — M2 CI prereq, ops responsibility. Listed as ship-blocker prereq in §6. +- **LLM-as-executor over user-authored plans** — Phase 1 Q2 marked OUT; no REQ slot. +- **Async audit (decoupled via Trigger.dev)** — Phase 1 Q5 rejected; would amend INV-4 without justification. +- **Planner / executor BYOM split** — Phase 1 Q3 rejected without D-008; `byom_endpoints` is single-row-per-tenant. +- **Per-inference cost attribution as a billing signal** — REQ-043 meters; billing is not in scope (no billing REQ). + +### 2.3 Milestone Breakdown + +M3 is a single milestone. Sub-phases for delivery sequencing (not separate milestones): + +- **M3.a — Chat inline + citation + history (REQ-028..033):** chat UI, NL input, streaming + citations, tool traces, conversation history + lifecycle, planner. +- **M3.b — Orchestration + durability (REQ-034..037):** multi-step workflows, ≤20-step inline limit (amended), durable Trigger.dev execution, rejoin protocol. +- **M3.c — Hardening + metering (REQ-041..045):** SOC2 posture page + live probes + `control_state`, Vanta sync, usage metering + dashboard, Redis rate aggregation. + +Each sub-phase is independently shippable behind a feature flag; M3 final ship is the union of all three. + +*Acceptance Gate (M3 ship):* All 15 REQs PASS, ≥80% coverage on new packages, INV-1..INV-8 preserved as written, M1+M2 non-regression suite green, M3 acceptance gate demonstrably satisfied on dev environment (with durable-path SLO refinement from §6 anchored). + +--- + +## 3. Personas & User Journeys + +### 3.1 Personas + +- **Devon (Operator)** — Tenant user who invokes the diagnostic copilot. Technical sophistication: SRE / DevOps practitioner. Goal: ask a natural-language diagnostic question, see streamed tool execution, get a cited evidence-backed answer without hand-crafting curl commands or navigating the Test-Call UI. +- **Sam (Admin)** — Tenant administrator who configures adapters, BYOM endpoints, team membership, and reads posture + usage. Technical sophistication: platform admin. Goal: ensure controls are green, usage is within budget, evidence flows to Vanta. +- **Casey (Compliance Reader)** — Read-only stakeholder (auditor, GRC reviewer). Technical sophistication: compliance professional. Goal: view posture page + usage + audit log without write or admin powers. RBAC granularity: Casey sees all controls (posture, usage, audit) read-only; no probe-trigger, no config edit, no Vanta credential view. (New in M2; first explicit M3 journey in J3.) + +### 3.2 Happy Paths + +**Journey 1 (J1) — Devon asks an inline diagnostic question (≤20 steps).** + +1. Devon opens `/chat`, types "Why is the staging nginx fleet degraded?", submits. → Chat UI creates a `chat_session` row, emits `chat.session.created` audit event. *(Maps to REQ-028, REQ-029, REQ-032)* +2. Server begins streaming SSE on `/api/chat/stream`. First event: `turn.user` with the question. → Client renders user message. *(Maps to REQ-030)* +3. Orchestrator invokes planner via tenant BYOM. → Emits `llm.inference.requested` with `{turn_id, byom_endpoint_id, prompt_tokens_estimated, workflow_id=null}`. *(Maps to REQ-033, REQ-029)* +4. Planner returns tool-call plan. → Emits `tool.plan.{emitted,selected}`. Intermediate planner message stored `visible=false` in `chat_turns`. Server forwards tool trace to client. *(Maps to REQ-031, REQ-033)* +5. For each tool call: orchestrator invokes broker adapter router (one row = one step). → Emits `adapter.capability_invoked` (already in M2 audit). Increments step counter. *(Maps to REQ-031, REQ-034)* +6. Tool result returns; planner reasons again; loop continues until planner emits final answer or step 20. *(Maps to REQ-033, REQ-034)* +7. Server emits final assistant message with citations array per REQ-030 shape. `usage` block attached: `{prompt_tokens, completion_tokens, byom_endpoint_id, latency_ms, workflow_id, correlation_id}`. *(Maps to REQ-030, REQ-043)* +8. Server emits `llm.inference.succeeded` per BYOM call. Client closes SSE on `turn.assistant.final`. *(Maps to REQ-030, REQ-043)* + +*Testable Acceptance (BDD):* +- **Given** Devon is authenticated, tenant BYOM is configured, ≥1 adapter is configured, **when** Devon submits a question that triggers ≤20 tool calls, **then** the response streams with citations, all `adapter.capability_invoked` rows have `session_id` FK set, all `llm.inference.*` rows are present, and the final SSE event is `turn.assistant.final` within 5 min p95. +- **Given** J1 has just completed, **when** Devon reloads `/chat`, **then** the conversation appears in the history list with all turns and citations renderable. +- **Given** Devon submits a question, **when** the planner emits a tool call, **then** the SSE stream emits a `tool.trace` event before the corresponding `adapter.capability_invoked` row is written to `audit_log`. + +**Journey 1-durable (J1-durable) — Devon triggers a workflow that exceeds 20 steps or must survive restart.** + +1. Devon submits a question whose planner-execution loop is projected to exceed 20 steps (e.g., "Audit all 50 Proxmox nodes for CVE-2024-xxxx"). → Orchestrator detects projection ≥21 at step 20. *(Maps to REQ-034, REQ-035)* +2. At step 21, orchestrator auto-promotes: assigns `workflow_id` (ULID), creates Trigger.dev task, emits `workflow.promoted` event on the SSE stream, bridges handoff. *(Maps to REQ-035 amended, REQ-036)* +3. Client UI updates state to "Background workflow — running asynchronously" indicator upon receipt of `workflow.promoted`. *(Phase 2 Item 5 addition)* +4. Trigger.dev worker continues execution; each step emits `workflow.step.{started,completed,failed}` to `workflow_events` table; `audit_log` continues to receive per-row hash-chained events with `session_id` FK and new `workflow_id` column. *(Maps to REQ-036)* +5. On completion, worker emits `workflow.completed` audit event and closes the Trigger.dev task. Server emits `workflow.completed` SSE event to any connected client. *(Maps to REQ-036, REQ-037)* +6. Devon returns to dashboard → "Background workflows" card shows the completed workflow with `workflow_id`. *(Maps to REQ-037 fallback)* +7. Devon clicks the card → client opens new SSE connection to `/api/chat/stream?workflow_id=...&last_event_id=...` → server replays from `workflow_events` starting after `last_event_id` + tails live events. *(Maps to REQ-037)* + +*Testable Acceptance (BDD):* +- **Given** Devon submits a question projected to exceed 20 steps, **when** step 21 begins, **then** `workflow.promoted` is emitted on the SSE stream, `workflow_id` is assigned, and the Trigger.dev task is created within 1s. +- **Given** Trigger.dev is unreachable at step 21, **when** promotion is attempted, **then** the orchestrator halts and preserves the inline state, emits `workflow.promotion_failed` audit event, and returns an error to the client with retry guidance (legacy REQ-035 contract). +- **Given** a durable workflow completed 5 minutes ago, **when** Devon reopens it via dashboard card with `Last-Event-ID` set to event N, **then** the SSE stream replays events N+1 onward from `workflow_events` and the first replayed event arrives within 30s p95 of `workflow.completed` audit emission. +- **Given** a durable workflow is still running at TTL (24h), **when** the TTL elapses, **then** the workflow is marked timed-out, `workflow.timeout` audit event is emitted, partial state is preserved, and the workflow is not resumable. + +**Journey 2 (J2) — Sam runs onboarding + smoke.** *Existing M1 journey with M3 extensions — refer to `steer-v0.1-spec.md` for full text. M3 additions: smoke test exercises the J1 inline path and emits `llm.inference.*` audit events feeding REQ-043 metering. Maps to REQ-033, REQ-034, REQ-035, REQ-043, REQ-044.* + +**Journey 2-hardening (J2-hardening) — Sam configures and reviews posture + usage + Vanta.** + +1. Sam opens `/settings/posture`. → Posture page renders `{control_name, status, last_checked_at, evidence_ref}` matrix for {RLS, audit hash-chain, WorkOS SSO, BYOM reachability}. *(Maps to REQ-041)* +2. Server initiates live probes on 5-min interval (configurable per tenant). Each probe emits `control.probe.{succeeded,failed}`. *(Maps to REQ-041)* +3. Sam opens `/settings/usage`. → Dashboard renders per-tenant aggregates (tokens in/out, BYOM calls, latency percentiles) sliced by conversation/workflow. *(Maps to REQ-043, REQ-044)* +4. Sam configures Vanta integration: API key, sync interval (1h default). → Server schedules sync task; Vanta API key resolved via `SecretProvider` (INV-3). *(Maps to REQ-042)* +5. Vanta sync runs on schedule, pushes evidence bundle: probe results + config snapshots + control attestation records. *(Maps to REQ-042)* + +*Testable Acceptance (BDD):* +- **Given** Sam opens `/settings/posture`, **when** the page renders, **then** every row in the matrix has `last_checked_at` within the past 5 minutes and an `evidence_ref` pointing to a `control.probe.*` audit row. +- **Given** a probe fails (e.g., WorkOS API returns 5xx), **when** the failure is detected, **then** the posture page updates within 1 probe interval and the failure row links to the `control.probe.failed` audit event. +- **Given** Sam configures Vanta with a valid API key, **when** the scheduled sync runs, **then** an evidence bundle is pushed to Vanta and a `vanta.sync.{succeeded,failed}` audit event is emitted. +- **Given** Sam opens `/settings/usage`, **when** the dashboard renders, **then** per-tenant aggregates show `{prompt_tokens, completion_tokens, byom_call_count, latency_p50/p95/p99}` for the selected time window. + +**Journey 3 (J3) — Casey (compliance reader) reviews posture + usage + audit.** + +1. Casey logs in via SSO. RBAC resolves to `compliance-reader` role. → Casey has read access to `/settings/posture`, `/settings/usage`, `/audit` but no write or admin routes. *(Maps to REQ-041, REQ-042, REQ-043, REQ-044)* +2. Casey browses posture, usage, audit. All three pages render read-only. No edit, no config, no probe-trigger controls. +3. Casey exports evidence bundle (audit log slice + posture snapshot + usage slice) for offline review. *(Maps to REQ-042 evidence lineage)* + +*Testable Acceptance (BDD):* +- **Given** Casey is authenticated as `compliance-reader`, **when** Casey requests any write or admin endpoint, **then** the request is rejected with 403 and `authz.denied` audit event is emitted. +- **Given** Casey exports an evidence bundle, **when** the bundle is generated, **then** every included row has a verifiable provenance chain (audit hash-chain + Merkle session root) and the bundle itself is emitted as `evidence.exported` audit event. + +**Journey infra (J-infra) — Cross-process rate aggregation (REQ-045, no user persona).** + +1. API gateway receives a chat request. → Consults Redis-backed token bucket for tenant + global keys. *(Maps to REQ-045)* +2. Trigger.dev worker emits a step-completion event. → Worker also consults/updates Redis bucket to enforce cross-process rate limits on durable workflows. *(Maps to REQ-045)* +3. If bucket exhausted → request rejected with 429 + `Retry-After`; workflow step retried per Trigger.dev backoff or rejected if global ceiling hit. *(Maps to REQ-045)* + +*Testable Acceptance (BDD):* +- **Given** API gateway instance A and Trigger.dev worker instance W both serve tenant T, **when** T's per-tenant rate bucket is exhausted on A, **then** a subsequent request to W is rejected with 429 within 1s. +- **Given** Redis is unreachable, **when** rate lookup fails, **then** requests are rejected with 503 (fail-closed) and `rate.redis.unavailable` audit event is emitted. + +### 3.3 Failure & Edge Paths + +- **Edge 1 — BYOM unreachable mid-turn:** REQ-009 reject path. Server emits `llm.inference.failed` with `reason=byom_unreachable`, surfaces error to client, conversation state preserved. +- **Edge 2 — Hash-chain trigger fails on insert (INV-4):** INSERT rolls back, enclosing transaction rolls back, request returns 500; current operation has no audit row because the chain broke (preserved semantics). +- **Edge 3 — Planner emits malformed tool call:** Orchestrator rejects, emits `tool.plan.rejected`, requests planner retry. If 3 consecutive malformed plans → surface error to user, conversation state preserved. +- **Edge 4 — Tool call exceeds adapter capability (INV-7 write blocklist):** Broker rejects with `adapter.write_rejected` (already in M2). Conversation continues without that step. +- **Edge 5 — Durable workflow TTL timeout:** `workflow.timeout` audit event, workflow marked not resumable, partial state preserved. +- **Edge 6 — Vanta API failure during sync:** `vanta.sync.failed` audit event, retry with exponential backoff per Trigger.dev task config, posture page surfaces Vanta integration status. +- **Edge 7 — Probe fails (e.g., WorkOS 5xx):** `control.probe.failed` audit event, posture page status = `degraded`, `evidence_ref` points to failed probe. +- **Edge 8 — Redis unreachable:** Fail-closed 503, `rate.redis.unavailable` audit event. +- **Edge 9 — Client reconnect during inline SSE with no `Last-Event-ID`:** Server replays entire conversation state from `chat_turns` (full re-replay acceptable for inline streams). +- **Edge 10 — `workflow_id` lost client-side:** Dashboard "Background workflows" card surfaces all workflows for tenant; any persona with read access can rejoin by clicking the card. + +--- + +## 4. Functional Requirements + +| ID | Title | Journeys | Priority | Acceptance Criteria (Given/When/Then or explicit rules) | +| :--- | :--- | :--- | :--- | :--- | +| **REQ-028** | Chat UI surface | J1 | High | **Given** Devon is authenticated, **when** Devon navigates to `/chat`, **then** the UI renders an input surface + empty conversation state + streaming-capable message list. | +| **REQ-029** | Natural-language input | J1 | High | **Given** chat UI is open, **when** Devon submits a natural-language question, **then** server validates input length (≤4000 chars), creates `chat_session` row, emits `chat.session.created` audit event with `{session_id, tenant_id, user_id}`, returns first SSE event (`turn.user`) within 500ms p95. | +| **REQ-030** | Streamed response with citations | J1 | High | **1.** Every assistant message contains a `citations` array per the shape below. **2.** Each citation's `tool_invocation_id` is a valid FK to `audit_log.adapter.capability_invoked`. **3.** `usage` block on every assistant message (every BYOM call) with `workflow_id` (nullable) + `correlation_id`. **4.** Inline path: SSE completes within 5 min p95. **Citation message shape:** ```json { "message_id":"ulid","turn_id":"ulid","role":"assistant", "content":"nginx is down [c:1]...", "citations":[{"citation_id":"c:1","claim_text":"nginx is down", "tool_invocation_id":"ulid (FK audit_log adapter.capability_invoked)", "adapter_id":"ssh:web-server-01","capability":"ssh.run_whitelisted_command", "source_locator":{"type":"ssh_command_output","command":"systemctl status nginx", "output_snippet":"Active: inactive (dead)","output_full_ref":"audit_log.row_id"}}], "usage":{"prompt_tokens":1820,"completion_tokens":340, "byom_endpoint_id":"ulid","latency_ms":4200, "workflow_id":"ulid\|null","correlation_id":"ulid"}} ``` | +| **REQ-031** | Streamed tool traces | J1, J1-durable | High | **Given** orchestrator invokes a tool, **when** the broker adapter router begins execution, **then** SSE emits a `tool.trace` event with `{tool_invocation_id, adapter_id, capability, started_at}` before the `adapter.capability_invoked` audit row is written. | +| **REQ-032** | Conversation history (first-class data model) | J1 | High | **1.** `chat_sessions` and `chat_turns` tables exist (additive migration per `steer-m2-spec.md:162`). **2.** Retention: indefinite for `chat_sessions` and `chat_turns`. **3.** Devon can list all sessions for the tenant; clicking a session loads full turn history with citations and tool traces. **4.** Session lifecycle: soft-close on 30-min idle; hard-close on explicit user action or session age >7 days. Soft-close = read-only + audit log complete; not deleted. | +| **REQ-033** | LLM tool reasoning (planner) | J1, J2 | High | **1.** Every planner invocation goes through tenant BYOM endpoint (INV-5). **2.** Every planner invocation emits `llm.inference.{requested,succeeded,failed}` audit events. **3.** Intermediate planner messages (reasoning traces) stored in `chat_turns` with `visible=false`. **4.** Planner rejects malformed tool calls per Edge 3. | +| **REQ-034** | Multi-step workflows | J1, J1-durable | High | **Given** orchestrator is executing a workflow, **when** the planner emits a multi-step plan, **then** the orchestrator executes steps sequentially through the broker adapter router, each step increments the per-`workflow_id` counter, and each step emits `adapter.capability_invoked` audit event. | +| **REQ-035** | ≤20-step inline limit (AMENDED) | J1, J1-durable | High | **Given** step counter reaches 20 in an inline workflow, **when** step 21 is initiated, **then** the orchestrator auto-promotes the workflow to durable: assigns `workflow_id` (ULID), creates Trigger.dev task, emits `workflow.promoted` SSE event, bridges handoff. **Degraded mode:** if Trigger.dev unreachable at promotion time, halt and preserve per legacy contract, emit `workflow.promotion_failed` audit event, return error to client with retry guidance. | +| **REQ-036** | Durable execution | J1-durable | High | **1.** Durable workflows execute in Trigger.dev tasks with `workflow_id` correlation. **2.** Each step emits `workflow.step.{started,completed,failed}` to `workflow_events` table. **3.** `audit_log` continues per-row hash-chaining; new `workflow_id` column on audit rows; `session_id` FK still set. **4.** On completion: `workflow.completed` audit event + Trigger.dev task close. **5.** Hard TTL 24h → `workflow.timeout` + partial state preserved + workflow not resumable. **6.** `workflow_events` retention aligned with `audit_log` (90d min, 1y target); S3 WORM archive covers long-term. **7.** Trigger.dev task schema: input=`{session_id, user_turn_id, planner_plan, byom_endpoint_id}`; output=`{workflow.completed event payload}`; retry=exponential backoff ×3; idempotency key=`workflow_id`. | +| **REQ-037** | Workflow rejoin | J1-durable | High | **1.** Client reconnects via `/api/chat/stream?workflow_id=...&last_event_id=...`. **2.** Server replays from `workflow_events` starting after `last_event_id` + tails live events. **3.** Rejoin p95 ≤30s, anchored: `workflow.completed` audit emission → first replayed event receipt. **4.** Fallback: dashboard "Background workflows" card lists all workflows for tenant; clicking opens rejoin stream. | +| **REQ-041** | SOC2 posture page | J2-hardening, J3 | High | **1.** Page renders `{control_name, status, last_checked_at, evidence_ref}` matrix for {RLS, audit hash-chain, WorkOS SSO, BYOM reachability}. **2.** Live probes run on 5-min interval (configurable per tenant). **3.** Each probe emits `control.probe.{succeeded,failed}` audit event. **4.** Posture page never displays a status without an `evidence_ref` to a probe audit row. **5.** Live probes themselves are auditable (INV-4). **6.** `control_state` cache table (write-through with probe transaction, Decision #8) backs the page render for <500ms p95 render time. **7.** Casey RBAC: read-only; no probe-trigger, no config edit, no Vanta credential view. | +| **REQ-042** | Vanta evidence sync | J2-hardening, J3 | High | **1.** Scheduled sync (1-h default, configurable). **2.** Pushes evidence bundle: probe results + config snapshots + control attestation records. **3.** No signed attestations in M3 (avoids SOC2-cert anti-goal). **4.** Emits `vanta.sync.{succeeded,failed}` audit event per run. **5.** Retry with exponential backoff on failure. **6.** Vanta API key resolved via `SecretProvider` (INV-3). | +| **REQ-043** | Usage metering | J1, J2-hardening, J3 | High | **1.** Every BYOM call emits `llm.inference.{requested,succeeded,failed}` with `usage` block (REQ-030 schema). **2.** Metering pipeline aggregates per-tenant: `{prompt_tokens, completion_tokens, byom_call_count, latency_p50/p95/p99}` sliced by conversation/workflow. **3.** Aggregation interval: 1-min default (configurable). **4.** Failed inferences (no completion) are counted at zero tokens + recorded as failures. | +| **REQ-044** | Usage dashboard | J2-hardening, J3 | High | **Given** Sam or Casey opens `/settings/usage`, **when** the dashboard renders, **then** it displays metered data (REQ-043) for the selected time window with per-tenant aggregates + drill-down to conversation/workflow level. Casey sees all controls read-only; no edit, no config. | +| **REQ-045** | Redis-backed rate aggregation | J-infra | High | **1.** Rate limiter state in Redis (Valkey-compatible). **2.** API gateway + Trigger.dev workers share the same bucket state per tenant. **3.** If bucket exhausted: 429 + `Retry-After` for gateway requests; Trigger.dev backoff/reject for workflow steps. **4.** If Redis unreachable: fail-closed 503 + `rate.redis.unavailable` audit event. **5.** Bucket config: per-tenant + global keys; limits configurable per tenant. | + +--- + +## 5. Technical Constraints & NFRs + +- **INV-1 (API gateway ordering):** Every HTTP request hits API gateway first: auth → tenant resolve → RBAC → audit. M3 `/api/chat/stream` and `/api/chat/rejoin` MUST conform. +- **INV-2 (Tenant scoping):** Every DB query runs under `SET app.tenant_id` via `withTenant` transaction; RLS enforces scoping. M3 queries against `chat_sessions`, `chat_turns`, `workflow_events`, `control_state` MUST use `withTenant`. RLS policies for new tables ship in the M3 migration. +- **INV-3 (Secret resolution):** Every credential resolved via `SecretProvider.get`; never env/config/DB for tenant secrets. M3 scheduler tasks (Trigger.dev) for Vanta sync + posture probes MUST resolve tenant Vanta API key via SecretProvider, not env. +- **INV-4 (Audit hash-chain):** Every auditable event appended to `audit_log` with hash-chain; `UPDATE/DELETE` REVOKE'd; write failure halts enclosing operation. **M3 preserves INV-4 as written.** Per-row hash-chaining continues. Session-scoped Merkle root per `chat_session` row is additive and does not weaken INV-4. +- **INV-5 (BYOM routing):** Every LLM inference call routed to tenant's BYOM endpoint; unconfigured/unreachable → reject (REQ-009). **M3 enforces this for all roles (planner, executor, synthesis) — no D-008 split.** All N inference calls per turn hit the same `byom_endpoints` row. +- **INV-6 (Relay Agent outbound-only):** Unchanged from M2. M3 chat does not directly invoke Relay Agent; chat → broker → adapter → (optionally) Relay Agent. +- **INV-7 (Read-only by default):** Closed 9-tool registry is the primary boundary; write-method blocklist is the backstop. M3 orchestrator MUST refuse any tool call that resolves to a write capability, even if planner requests it. +- **INV-8 (PGlite RLS gap):** PGlite 0.5.7 doesn't enforce RLS on SELECT. App-layer `withTenant` + explicit `WHERE tenant_id` is primary in dev/test; RLS + FORCE RLS is prod backstop. M3 new tables (`chat_sessions`, `chat_turns`, `workflow_events`, `control_state`) MUST have explicit `tenant_id` columns + RLS policies + `withTenant` usage. + +**Performance NFRs:** +- Inline path: SSE completes within 5 min p95 (acceptance gate). +- Durable rejoin: p95 ≤30s from `workflow.completed` audit emission to first replayed event receipt (Phase 2 Item 6 anchor). +- Posture page render: <500ms p95 (backed by `control_state` write-through cache). +- Probe cadence: 5-min default interval (REQ-041). +- Vanta sync: 1-h default interval (REQ-042). +- Metering aggregation: 1-min default interval (REQ-043). + +**Schema additivity (per `steer-m2-spec.md:162` — "new tables" parenthetical explicitly permits new tables, not just columns):** +- New tables: `chat_sessions`, `chat_turns`, `workflow_events`, `control_state`. +- New `audit_log` columns: `session_id` (FK `chat_sessions`, nullable), `workflow_id` (nullable). +- New audit event types: see §2.1. + +**Migration shape (Decision #7 — per-table for rollback isolation, one PR):** +- `0004_chat_schema.sql` — `chat_sessions`, `chat_turns` + RLS; `audit_log.session_id` column. +- `0005_workflow_schema.sql` — `workflow_events` + RLS; `audit_log.workflow_id` column. +- `0006_posture_schema.sql` — `control_state` + RLS. + +**Anti-goal enforcement:** +- No write actions on infrastructure (closed 9-tool registry + write-method blocklist + INV-7). +- No hosted LLM inference (single BYOM endpoint, all roles, INV-5). +- No SOC2 final cert (no signed attestations in M3; REQ-042 pushes evidence only). + +**`control_state` cache table (Decision #8 — write-through):** +- `control_state` row updated in the same Postgres transaction as the `control.probe.{succeeded,failed}` audit event emission. Cache is always consistent with audit log at probe completion. +- **Implementation question (§7 Q1):** Engineering to confirm write-through is achievable in the chosen probe pipeline (likely Trigger.dev task per REQ-042 architecture). If a single transaction is not achievable, surface divergence handling in a v1.3 spec amendment. + +--- + +## 6. Milestone Plan & Release Gates + +**Test evidence required for Production Release (M3):** + +- [ ] Code coverage ≥80% on new packages (`packages/orchestrator`, `packages/chat`); existing M2 floor (92.3% on `packages/mcp`) maintained. +- [ ] All 15 REQs PASS (REQ-028..037, REQ-041..045) including REQ-035 amendment acceptance criteria. +- [ ] CI/CD pipeline GREEN (Gitea Actions, both `test-pglite` and `test-postgres` jobs). +- [ ] M1+M2 non-regression: full 656-test suite green. +- [ ] INV-1..INV-8 explicitly tested (new tests for INV-4 audit hash-chain preservation across M3 event types; new tests for INV-5 BYOM routing across planner/executor/synthesis; new tests for INV-2 RLS on `chat_sessions`/`chat_turns`/`workflow_events`/`control_state`). +- [ ] MCP 2025-06-18 conformance verified (M2 gate, non-regression). +- [ ] LLM smoke Track A passes (mock-path, P0 gate, no external deps). +- [ ] QA sign-off: 100% of J1, J1-durable, J2-hardening, J3 integration tests pass; J-infra (REQ-045) has unit + cross-process integration test demonstrating gateway ↔ Trigger.dev worker rate coordination. +- [ ] Security/compliance review: posture page + Vanta sync demonstrate evidence lineage (every rendered status has an audit row); RBAC denies Casey on all write/admin endpoints; audit log hash-chain integrity verified end-to-end. +- [ ] Production deployment is an ops track (not in this gate list); deploy to AWS us-east-1 follows the ops track's own gate. + +**Durable-path SLO refinement (gate clarification — does not rewrite `REQUIREMENTS.md:40`):** +> Inline path: p95 ≤5min. Durable path: rejoin p95 ≤30s from `workflow.completed` audit emission to first replayed event receipt; hard TTL 24h. `REQUIREMENTS.md:40` raw gate language stays untouched; this refinement documents the durable-path clarification locally in the M3 spec. + +**Operational prereqs (ship-blockers, NOT M3 REQ deliverables — ops track, parallel to PDLC):** +- Gitea Actions runner enablement on forge (blocks full M2 + M3 CI verification on `test-postgres` job). +- `GITHUB_SMOKE_PAT` provisioning as Gitea Actions secret (blocks Track B GitHub LLM smoke; Track A mock-path is the P0 gate and does not need this). + +--- + +## 7. Open Questions & Assumptions + +1. **Trigger.dev task schema for durable workflows** — RESOLVED. Workflow input=`{session_id, user_turn_id, planner_plan, byom_endpoint_id}`; output=`{workflow.completed event payload}`; retry policy=exponential backoff ×3; idempotency key=`workflow_id`. Baked into REQ-036 §4.7. +2. **M3 acceptance gate refinement (durable path SLO)** — RESOLVED. "Inline path: p95 ≤5min. Durable path: rejoin p95 ≤30s post-`workflow.completed` audit emission; hard TTL 24h." Baked into §6. `REQUIREMENTS.md:40` stays untouched; refinement documented locally in M3 spec §6. +3. **Chat session lifecycle** — RESOLVED. Soft-close 30-min idle; hard-close 7d or explicit. Read-only post-close, not deleted. Baked into REQ-032 §4. +4. **Workflow event retention vs audit log retention** — RESOLVED. Align `workflow_events` with `audit_log` (90d min, 1y target). S3 WORM archive covers long-term. Baked into REQ-036 §4.6. +5. **Citation rendering UX** — DEFERRED. Exact UI for inline `[c:1]` markers + citation popover. Defer to design review. Non-blocking for spec ratification. Proposed: inline superscript numbers; click expands to source adapter + tool invocation link + output snippet. +6. **Posture page RBAC granularity** — RESOLVED. Casey sees all controls (posture, usage, audit) read-only; no probe-trigger, no config edit, no Vanta credential view. Baked into §3.1 + REQ-041 §4.7 + REQ-044 §4. +7. **`control_state` cache invalidation strategy** — OPEN (engineering confirm during implementation). Spec default: write-through (same transaction as `control.probe.*` audit emission). Engineering to confirm feasibility in chosen probe pipeline; if not achievable, v1.3 amendment with divergence handling. + +--- + +## 8. Changelog + +| Version | Date | Author | What Changed | REQs Affected | +| :--- | :--- | :--- | :--- | :--- | +| v1.0 | 2026-08-25 | Sarah Chen (PM) | Initial M3 spec draft from Phase 3 generation | REQ-028..037, REQ-041..045 (with REQ-035 amendment) | +| v1.1 | 2026-08-25 | Sarah Chen (PM) | Open Questions 1-6 resolved and baked in (Trigger.dev task schema, durable SLO, session lifecycle, workflow retention, Casey RBAC); OQ5 deferred to design review | REQ-032, REQ-036, REQ-041, REQ-044 | +| v1.2 | 2026-08-25 | Sarah Chen (PM) | Engineering handoff: Decision #7 (per-table migrations `0004`/`0005`/`0006`), Decision #8 (`control_state` write-through cache), Decision #9 (Redis fail-closed 503), Decision #10 (usage on all BYOM calls); OQ7 opened (`control_state` write-through feasibility); ops prereqs documented in §6 | REQ-036, REQ-041, REQ-043, REQ-045 | + +--- + +**M3 spec v1.2 Final. Ready for engineering handoff.** \ No newline at end of file