coreci-chat/.ciagent/steer-m3-spec.md

# CoreCI Chat — M3 Specification (Chat, Orchestration, Hardening)

**Owner:** Sarah Chen (Sr. PM)
**Status:** Final (v1.2)
**Type:** Milestone
**Target Milestone:** v0.2 — M3 (Chat, Orchestration, Hardening)
**Locked:** 2026-08-25

> **Operating Principles for this Spec:**
> 1. **Incremental Delivery:** This spec defines net-new work only. Pre-existing systems and locked architectures (M1, M2) are referenced, not restated.
> 2. **Zero Ambiguity:** If a requirement cannot be translated into a pass/fail test by QA, it is incomplete and will be rejected by Engineering.
> 3. **Invariants preserved:** INV-1..INV-8 from `.ciagent/ARCHITECTURE.md` hold as written. M3 adds no invariant amendments (Q5 session-batched Merkle is additive, not a semantic change to INV-4).

---

## 1. Objective

M3 delivers a diagnostic copilot that lets operators ask natural-language questions about tenant infrastructure and receive cited, evidence-backed answers within the M3 acceptance gate. It ships chat UI with streamed tool execution (≤20 steps inline, auto-promoted to durable Trigger.dev workflows beyond), persistent conversation history, per-inference usage metering, a SOC2 posture page with live control probes, and Vanta evidence sync. Three personas benefit: Devon (Operator) gets the copilot; Sam (Admin) gets posture + usage visibility; Casey (Compliance Reader) gets read-only attestation surfaces. The milestone closes v0.1 with 15 REQs total and refines (does not rewrite) the acceptance gate.

*Acceptance Gate:* A developer reading this can state: "M3 ships a multi-turn diagnostic copilot that streams tool execution, persists durable workflows, meters usage, and instruments SOC2 controls."

---

## 2. Scope & Target Milestones

### 2.1 In Scope (Explicit Additions)

15 REQs in three clusters:

**Chat & Orchestration (REQ-028..037):** chat UI surface, natural-language input, streamed response with inline citations, streamed tool traces, conversation history (first-class data model), LLM tool reasoning, multi-step workflows, ≤20-step inline limit (amended: auto-promote at 21), durable execution via Trigger.dev, workflow rejoin. New packages: `packages/orchestrator` (step counter, promotion logic), `packages/chat` (stream manager, citation builder). New schemas: `chat_sessions`, `chat_turns`, `workflow_events` (additive per `steer-m2-spec.md:162`). API contracts: `/api/chat/stream` (inline + durable rejoin via `Last-Event-ID`), `workflow.promoted` event (inline→durable handoff).

**Hardening (REQ-041..045):** REQ-041 SOC2 posture page with live control probes (5-min default interval, configurable per tenant) + `control_state` cache table (write-through with probe transaction; Decision #8); REQ-042 Vanta evidence sync (1-h default, no signed attestations); REQ-043 usage metering (per-inference `usage` block from REQ-030 feeds the pipeline; aggregation 1-min default); REQ-044 usage dashboard (tenant-facing rendering); REQ-045 Redis-backed rate aggregation (cross-process coordination between API gateway and Trigger.dev workers, replacing M2's in-memory token bucket; fail-closed 503 if Redis down).

**Architectural amendments:** REQ-035 amendment text baked in (§4). New audit event types: `chat.session.created`, `chat.turn.*`, `llm.inference.{requested,succeeded,failed}`, `tool.plan.{emitted,selected,rejected}`, `workflow.{started,step.*,promoted,completed,timeout,promotion_failed,rejoined}`, `control.probe.{succeeded,failed}`, `vanta.sync.{succeeded,failed}`, `rate.redis.unavailable`, `evidence.exported`, `authz.denied`.

### 2.2 Out of Scope (Explicit Exclusions)

Two categories. Anti-goals (vision spec §2.2 — never / post-MVP / v1.2+) vs. M4+ deferrals (no REQ slot in M3 set):

**Anti-goals (vision §2.2, not M3 deferrals):** write actions on infrastructure (v1.1+, INV-7 holds); hosted LLM inference (never, INV-5 holds, planner split rejected per Phase 1 Q3); RAG / vector retrieval (v1.1+); SOC 2 final certification (post-MVP); custom RBAC roles (v1.2+); BYOK / customer-managed keys (v1.2+); multi-region deployment (MVP single-region); Windows support (not planned v1.x); fine-tuning (not planned); Kubernetes / ArgoCD / Helm; Slack / Teams / CLI / mobile clients (v1.1+); approval-gated remediation (v1.1+).

**M4+ deferrals (no REQ slot exists in M3 — each item carries REQ-gap reasoning):**
- **Secret rotation** — `SecretProvider` interface exists (REQ-039, M1); rotation is an ops process. No REQ slot in 028-037/041-045.
- **Production deployment of M1+M2+M3** — ops track parallel to PDLC. M3 REQs satisfied by code shipping to `main` + Gitea releases. No REQ slot.
- **Adapter SSRF/CSRF guardrails** — tenant-supplied hostnames are trusted per M2 spec (`steer-m2-spec.md:172`); hardening requires a new REQ. No REQ slot.
- **OTel / RED metrics / distributed tracing** — observability stack. No REQ slot. REQ-041 is posture page, not instrumentation.
- **SBOM / supply-chain / signed releases** — No REQ slot.
- **Gitea Actions runner enablement** — M2 CI prereq, ops responsibility; blocks full M2 verification, not M3 REQ satisfaction. Listed as ship-blocker prereq in §6.
- **Track B GitHub smoke (`GITHUB_SMOKE_PAT`)** — M2 CI prereq, ops responsibility. Listed as ship-blocker prereq in §6.
- **LLM-as-executor over user-authored plans** — Phase 1 Q2 marked OUT; no REQ slot.
- **Async audit (decoupled via Trigger.dev)** — Phase 1 Q5 rejected; would amend INV-4 without justification.
- **Planner / executor BYOM split** — Phase 1 Q3 rejected without D-008; `byom_endpoints` is single-row-per-tenant.
- **Per-inference cost attribution as a billing signal** — REQ-043 meters; billing is not in scope (no billing REQ).

### 2.3 Milestone Breakdown

M3 is a single milestone. Sub-phases for delivery sequencing (not separate milestones):

- **M3.a — Chat inline + citation + history (REQ-028..033):** chat UI, NL input, streaming + citations, tool traces, conversation history + lifecycle, planner.
- **M3.b — Orchestration + durability (REQ-034..037):** multi-step workflows, ≤20-step inline limit (amended), durable Trigger.dev execution, rejoin protocol.
- **M3.c — Hardening + metering (REQ-041..045):** SOC2 posture page + live probes + `control_state`, Vanta sync, usage metering + dashboard, Redis rate aggregation.

Each sub-phase is independently shippable behind a feature flag; M3 final ship is the union of all three.

*Acceptance Gate (M3 ship):* All 15 REQs PASS, ≥80% coverage on new packages, INV-1..INV-8 preserved as written, M1+M2 non-regression suite green, M3 acceptance gate demonstrably satisfied on dev environment (with durable-path SLO refinement from §6 anchored).

---

## 3. Personas & User Journeys

### 3.1 Personas

- **Devon (Operator)** — Tenant user who invokes the diagnostic copilot. Technical sophistication: SRE / DevOps practitioner. Goal: ask a natural-language diagnostic question, see streamed tool execution, get a cited evidence-backed answer without hand-crafting curl commands or navigating the Test-Call UI.
- **Sam (Admin)** — Tenant administrator who configures adapters, BYOM endpoints, team membership, and reads posture + usage. Technical sophistication: platform admin. Goal: ensure controls are green, usage is within budget, evidence flows to Vanta.
- **Casey (Compliance Reader)** — Read-only stakeholder (auditor, GRC reviewer). Technical sophistication: compliance professional. Goal: view posture page + usage + audit log without write or admin powers. RBAC granularity: Casey sees all controls (posture, usage, audit) read-only; no probe-trigger, no config edit, no Vanta credential view. (New in M2; first explicit M3 journey in J3.)

### 3.2 Happy Paths

**Journey 1 (J1) — Devon asks an inline diagnostic question (≤20 steps).**

1. Devon opens `/chat`, types "Why is the staging nginx fleet degraded?", submits. → Chat UI creates a `chat_session` row, emits `chat.session.created` audit event. *(Maps to REQ-028, REQ-029, REQ-032)*
2. Server begins streaming SSE on `/api/chat/stream`. First event: `turn.user` with the question. → Client renders user message. *(Maps to REQ-030)*
3. Orchestrator invokes planner via tenant BYOM. → Emits `llm.inference.requested` with `{turn_id, byom_endpoint_id, prompt_tokens_estimated, workflow_id=null}`. *(Maps to REQ-033, REQ-029)*
4. Planner returns tool-call plan. → Emits `tool.plan.{emitted,selected}`. Intermediate planner message stored `visible=false` in `chat_turns`. Server forwards tool trace to client. *(Maps to REQ-031, REQ-033)*
5. For each tool call: orchestrator invokes broker adapter router (one row = one step). → Emits `adapter.capability_invoked` (already in M2 audit). Increments step counter. *(Maps to REQ-031, REQ-034)*
6. Tool result returns; planner reasons again; loop continues until planner emits final answer or step 20. *(Maps to REQ-033, REQ-034)*
7. Server emits final assistant message with citations array per REQ-030 shape. `usage` block attached: `{prompt_tokens, completion_tokens, byom_endpoint_id, latency_ms, workflow_id, correlation_id}`. *(Maps to REQ-030, REQ-043)*
8. Server emits `llm.inference.succeeded` per BYOM call. Client closes SSE on `turn.assistant.final`. *(Maps to REQ-030, REQ-043)*

*Testable Acceptance (BDD):*
- **Given** Devon is authenticated, tenant BYOM is configured, ≥1 adapter is configured, **when** Devon submits a question that triggers ≤20 tool calls, **then** the response streams with citations, all `adapter.capability_invoked` rows have `session_id` FK set, all `llm.inference.*` rows are present, and the final SSE event is `turn.assistant.final` within 5 min p95.
- **Given** J1 has just completed, **when** Devon reloads `/chat`, **then** the conversation appears in the history list with all turns and citations renderable.
- **Given** Devon submits a question, **when** the planner emits a tool call, **then** the SSE stream emits a `tool.trace` event before the corresponding `adapter.capability_invoked` row is written to `audit_log`.

**Journey 1-durable (J1-durable) — Devon triggers a workflow that exceeds 20 steps or must survive restart.**

1. Devon submits a question whose planner-execution loop is projected to exceed 20 steps (e.g., "Audit all 50 Proxmox nodes for CVE-2024-xxxx"). → Orchestrator detects projection ≥21 at step 20. *(Maps to REQ-034, REQ-035)*
2. At step 21, orchestrator auto-promotes: assigns `workflow_id` (ULID), creates Trigger.dev task, emits `workflow.promoted` event on the SSE stream, bridges handoff. *(Maps to REQ-035 amended, REQ-036)*
3. Client UI updates state to "Background workflow — running asynchronously" indicator upon receipt of `workflow.promoted`. *(Phase 2 Item 5 addition)*
4. Trigger.dev worker continues execution; each step emits `workflow.step.{started,completed,failed}` to `workflow_events` table; `audit_log` continues to receive per-row hash-chained events with `session_id` FK and new `workflow_id` column. *(Maps to REQ-036)*
5. On completion, worker emits `workflow.completed` audit event and closes the Trigger.dev task. Server emits `workflow.completed` SSE event to any connected client. *(Maps to REQ-036, REQ-037)*
6. Devon returns to dashboard → "Background workflows" card shows the completed workflow with `workflow_id`. *(Maps to REQ-037 fallback)*
7. Devon clicks the card → client opens new SSE connection to `/api/chat/stream?workflow_id=...&last_event_id=...` → server replays from `workflow_events` starting after `last_event_id` + tails live events. *(Maps to REQ-037)*

*Testable Acceptance (BDD):*
- **Given** Devon submits a question projected to exceed 20 steps, **when** step 21 begins, **then** `workflow.promoted` is emitted on the SSE stream, `workflow_id` is assigned, and the Trigger.dev task is created within 1s.
- **Given** Trigger.dev is unreachable at step 21, **when** promotion is attempted, **then** the orchestrator halts and preserves the inline state, emits `workflow.promotion_failed` audit event, and returns an error to the client with retry guidance (legacy REQ-035 contract).
- **Given** a durable workflow completed 5 minutes ago, **when** Devon reopens it via dashboard card with `Last-Event-ID` set to event N, **then** the SSE stream replays events N+1 onward from `workflow_events` and the first replayed event arrives within 30s p95 of `workflow.completed` audit emission.
- **Given** a durable workflow is still running at TTL (24h), **when** the TTL elapses, **then** the workflow is marked timed-out, `workflow.timeout` audit event is emitted, partial state is preserved, and the workflow is not resumable.

**Journey 2 (J2) — Sam runs onboarding + smoke.** *Existing M1 journey with M3 extensions — refer to `steer-v0.1-spec.md` for full text. M3 additions: smoke test exercises the J1 inline path and emits `llm.inference.*` audit events feeding REQ-043 metering. Maps to REQ-033, REQ-034, REQ-035, REQ-043, REQ-044.*

**Journey 2-hardening (J2-hardening) — Sam configures and reviews posture + usage + Vanta.**

1. Sam opens `/settings/posture`. → Posture page renders `{control_name, status, last_checked_at, evidence_ref}` matrix for {RLS, audit hash-chain, WorkOS SSO, BYOM reachability}. *(Maps to REQ-041)*
2. Server initiates live probes on 5-min interval (configurable per tenant). Each probe emits `control.probe.{succeeded,failed}`. *(Maps to REQ-041)*
3. Sam opens `/settings/usage`. → Dashboard renders per-tenant aggregates (tokens in/out, BYOM calls, latency percentiles) sliced by conversation/workflow. *(Maps to REQ-043, REQ-044)*
4. Sam configures Vanta integration: API key, sync interval (1h default). → Server schedules sync task; Vanta API key resolved via `SecretProvider` (INV-3). *(Maps to REQ-042)*
5. Vanta sync runs on schedule, pushes evidence bundle: probe results + config snapshots + control attestation records. *(Maps to REQ-042)*

*Testable Acceptance (BDD):*
- **Given** Sam opens `/settings/posture`, **when** the page renders, **then** every row in the matrix has `last_checked_at` within the past 5 minutes and an `evidence_ref` pointing to a `control.probe.*` audit row.
- **Given** a probe fails (e.g., WorkOS API returns 5xx), **when** the failure is detected, **then** the posture page updates within 1 probe interval and the failure row links to the `control.probe.failed` audit event.
- **Given** Sam configures Vanta with a valid API key, **when** the scheduled sync runs, **then** an evidence bundle is pushed to Vanta and a `vanta.sync.{succeeded,failed}` audit event is emitted.
- **Given** Sam opens `/settings/usage`, **when** the dashboard renders, **then** per-tenant aggregates show `{prompt_tokens, completion_tokens, byom_call_count, latency_p50/p95/p99}` for the selected time window.

**Journey 3 (J3) — Casey (compliance reader) reviews posture + usage + audit.**

1. Casey logs in via SSO. RBAC resolves to `compliance-reader` role. → Casey has read access to `/settings/posture`, `/settings/usage`, `/audit` but no write or admin routes. *(Maps to REQ-041, REQ-042, REQ-043, REQ-044)*
2. Casey browses posture, usage, audit. All three pages render read-only. No edit, no config, no probe-trigger controls.
3. Casey exports evidence bundle (audit log slice + posture snapshot + usage slice) for offline review. *(Maps to REQ-042 evidence lineage)*

*Testable Acceptance (BDD):*
- **Given** Casey is authenticated as `compliance-reader`, **when** Casey requests any write or admin endpoint, **then** the request is rejected with 403 and `authz.denied` audit event is emitted.
- **Given** Casey exports an evidence bundle, **when** the bundle is generated, **then** every included row has a verifiable provenance chain (audit hash-chain + Merkle session root) and the bundle itself is emitted as `evidence.exported` audit event.

**Journey infra (J-infra) — Cross-process rate aggregation (REQ-045, no user persona).**

1. API gateway receives a chat request. → Consults Redis-backed token bucket for tenant + global keys. *(Maps to REQ-045)*
2. Trigger.dev worker emits a step-completion event. → Worker also consults/updates Redis bucket to enforce cross-process rate limits on durable workflows. *(Maps to REQ-045)*
3. If bucket exhausted → request rejected with 429 + `Retry-After`; workflow step retried per Trigger.dev backoff or rejected if global ceiling hit. *(Maps to REQ-045)*

*Testable Acceptance (BDD):*
- **Given** API gateway instance A and Trigger.dev worker instance W both serve tenant T, **when** T's per-tenant rate bucket is exhausted on A, **then** a subsequent request to W is rejected with 429 within 1s.
- **Given** Redis is unreachable, **when** rate lookup fails, **then** requests are rejected with 503 (fail-closed) and `rate.redis.unavailable` audit event is emitted.

### 3.3 Failure & Edge Paths

- **Edge 1 — BYOM unreachable mid-turn:** REQ-009 reject path. Server emits `llm.inference.failed` with `reason=byom_unreachable`, surfaces error to client, conversation state preserved.
- **Edge 2 — Hash-chain trigger fails on insert (INV-4):** INSERT rolls back, enclosing transaction rolls back, request returns 500; current operation has no audit row because the chain broke (preserved semantics).
- **Edge 3 — Planner emits malformed tool call:** Orchestrator rejects, emits `tool.plan.rejected`, requests planner retry. If 3 consecutive malformed plans → surface error to user, conversation state preserved.
- **Edge 4 — Tool call exceeds adapter capability (INV-7 write blocklist):** Broker rejects with `adapter.write_rejected` (already in M2). Conversation continues without that step.
- **Edge 5 — Durable workflow TTL timeout:** `workflow.timeout` audit event, workflow marked not resumable, partial state preserved.
- **Edge 6 — Vanta API failure during sync:** `vanta.sync.failed` audit event, retry with exponential backoff per Trigger.dev task config, posture page surfaces Vanta integration status.
- **Edge 7 — Probe fails (e.g., WorkOS 5xx):** `control.probe.failed` audit event, posture page status = `degraded`, `evidence_ref` points to failed probe.
- **Edge 8 — Redis unreachable:** Fail-closed 503, `rate.redis.unavailable` audit event.
- **Edge 9 — Client reconnect during inline SSE with no `Last-Event-ID`:** Server replays entire conversation state from `chat_turns` (full re-replay acceptable for inline streams).
- **Edge 10 — `workflow_id` lost client-side:** Dashboard "Background workflows" card surfaces all workflows for tenant; any persona with read access can rejoin by clicking the card.

---

## 4. Functional Requirements

| ID | Title | Journeys | Priority | Acceptance Criteria (Given/When/Then or explicit rules) |
| :--- | :--- | :--- | :--- | :--- |
| **REQ-028** | Chat UI surface | J1 | High | **Given** Devon is authenticated, **when** Devon navigates to `/chat`, **then** the UI renders an input surface + empty conversation state + streaming-capable message list. |
| **REQ-029** | Natural-language input | J1 | High | **Given** chat UI is open, **when** Devon submits a natural-language question, **then** server validates input length (≤4000 chars), creates `chat_session` row, emits `chat.session.created` audit event with `{session_id, tenant_id, user_id}`, returns first SSE event (`turn.user`) within 500ms p95. |
| **REQ-030** | Streamed response with citations | J1 | High | **1.** Every assistant message contains a `citations` array per the shape below. **2.** Each citation's `tool_invocation_id` is a valid FK to `audit_log.adapter.capability_invoked`. **3.** `usage` block on every assistant message (every BYOM call) with `workflow_id` (nullable) + `correlation_id`. **4.** Inline path: SSE completes within 5 min p95. **Citation message shape:** ```json { "message_id":"ulid","turn_id":"ulid","role":"assistant", "content":"nginx is down [c:1]...", "citations":[{"citation_id":"c:1","claim_text":"nginx is down", "tool_invocation_id":"ulid (FK audit_log adapter.capability_invoked)", "adapter_id":"ssh:web-server-01","capability":"ssh.run_whitelisted_command", "source_locator":{"type":"ssh_command_output","command":"systemctl status nginx", "output_snippet":"Active: inactive (dead)","output_full_ref":"audit_log.row_id"}}], "usage":{"prompt_tokens":1820,"completion_tokens":340, "byom_endpoint_id":"ulid","latency_ms":4200, "workflow_id":"ulid\|null","correlation_id":"ulid"}} ``` |
| **REQ-031** | Streamed tool traces | J1, J1-durable | High | **Given** orchestrator invokes a tool, **when** the broker adapter router begins execution, **then** SSE emits a `tool.trace` event with `{tool_invocation_id, adapter_id, capability, started_at}` before the `adapter.capability_invoked` audit row is written. |
| **REQ-032** | Conversation history (first-class data model) | J1 | High | **1.** `chat_sessions` and `chat_turns` tables exist (additive migration per `steer-m2-spec.md:162`). **2.** Retention: indefinite for `chat_sessions` and `chat_turns`. **3.** Devon can list all sessions for the tenant; clicking a session loads full turn history with citations and tool traces. **4.** Session lifecycle: soft-close on 30-min idle; hard-close on explicit user action or session age >7 days. Soft-close = read-only + audit log complete; not deleted. |
| **REQ-033** | LLM tool reasoning (planner) | J1, J2 | High | **1.** Every planner invocation goes through tenant BYOM endpoint (INV-5). **2.** Every planner invocation emits `llm.inference.{requested,succeeded,failed}` audit events. **3.** Intermediate planner messages (reasoning traces) stored in `chat_turns` with `visible=false`. **4.** Planner rejects malformed tool calls per Edge 3. |
| **REQ-034** | Multi-step workflows | J1, J1-durable | High | **Given** orchestrator is executing a workflow, **when** the planner emits a multi-step plan, **then** the orchestrator executes steps sequentially through the broker adapter router, each step increments the per-`workflow_id` counter, and each step emits `adapter.capability_invoked` audit event. |
| **REQ-035** | ≤20-step inline limit (AMENDED) | J1, J1-durable | High | **Given** step counter reaches 20 in an inline workflow, **when** step 21 is initiated, **then** the orchestrator auto-promotes the workflow to durable: assigns `workflow_id` (ULID), creates Trigger.dev task, emits `workflow.promoted` SSE event, bridges handoff. **Degraded mode:** if Trigger.dev unreachable at promotion time, halt and preserve per legacy contract, emit `workflow.promotion_failed` audit event, return error to client with retry guidance. |
| **REQ-036** | Durable execution | J1-durable | High | **1.** Durable workflows execute in Trigger.dev tasks with `workflow_id` correlation. **2.** Each step emits `workflow.step.{started,completed,failed}` to `workflow_events` table. **3.** `audit_log` continues per-row hash-chaining; new `workflow_id` column on audit rows; `session_id` FK still set. **4.** On completion: `workflow.completed` audit event + Trigger.dev task close. **5.** Hard TTL 24h → `workflow.timeout` + partial state preserved + workflow not resumable. **6.** `workflow_events` retention aligned with `audit_log` (90d min, 1y target); S3 WORM archive covers long-term. **7.** Trigger.dev task schema: input=`{session_id, user_turn_id, planner_plan, byom_endpoint_id}`; output=`{workflow.completed event payload}`; retry=exponential backoff ×3; idempotency key=`workflow_id`. |
| **REQ-037** | Workflow rejoin | J1-durable | High | **1.** Client reconnects via `/api/chat/stream?workflow_id=...&last_event_id=...`. **2.** Server replays from `workflow_events` starting after `last_event_id` + tails live events. **3.** Rejoin p95 ≤30s, anchored: `workflow.completed` audit emission → first replayed event receipt. **4.** Fallback: dashboard "Background workflows" card lists all workflows for tenant; clicking opens rejoin stream. |
| **REQ-041** | SOC2 posture page | J2-hardening, J3 | High | **1.** Page renders `{control_name, status, last_checked_at, evidence_ref}` matrix for {RLS, audit hash-chain, WorkOS SSO, BYOM reachability}. **2.** Live probes run on 5-min interval (configurable per tenant). **3.** Each probe emits `control.probe.{succeeded,failed}` audit event. **4.** Posture page never displays a status without an `evidence_ref` to a probe audit row. **5.** Live probes themselves are auditable (INV-4). **6.** `control_state` cache table (write-through with probe transaction, Decision #8) backs the page render for <500ms p95 render time. **7.** Casey RBAC: read-only; no probe-trigger, no config edit, no Vanta credential view. |
| **REQ-042** | Vanta evidence sync | J2-hardening, J3 | High | **1.** Scheduled sync (1-h default, configurable). **2.** Pushes evidence bundle: probe results + config snapshots + control attestation records. **3.** No signed attestations in M3 (avoids SOC2-cert anti-goal). **4.** Emits `vanta.sync.{succeeded,failed}` audit event per run. **5.** Retry with exponential backoff on failure. **6.** Vanta API key resolved via `SecretProvider` (INV-3). |
| **REQ-043** | Usage metering | J1, J2-hardening, J3 | High | **1.** Every BYOM call emits `llm.inference.{requested,succeeded,failed}` with `usage` block (REQ-030 schema). **2.** Metering pipeline aggregates per-tenant: `{prompt_tokens, completion_tokens, byom_call_count, latency_p50/p95/p99}` sliced by conversation/workflow. **3.** Aggregation interval: 1-min default (configurable). **4.** Failed inferences (no completion) are counted at zero tokens + recorded as failures. |
| **REQ-044** | Usage dashboard | J2-hardening, J3 | High | **Given** Sam or Casey opens `/settings/usage`, **when** the dashboard renders, **then** it displays metered data (REQ-043) for the selected time window with per-tenant aggregates + drill-down to conversation/workflow level. Casey sees all controls read-only; no edit, no config. |
| **REQ-045** | Redis-backed rate aggregation | J-infra | High | **1.** Rate limiter state in Redis (Valkey-compatible). **2.** API gateway + Trigger.dev workers share the same bucket state per tenant. **3.** If bucket exhausted: 429 + `Retry-After` for gateway requests; Trigger.dev backoff/reject for workflow steps. **4.** If Redis unreachable: fail-closed 503 + `rate.redis.unavailable` audit event. **5.** Bucket config: per-tenant + global keys; limits configurable per tenant. |

---

## 5. Technical Constraints & NFRs

- **INV-1 (API gateway ordering):** Every HTTP request hits API gateway first: auth → tenant resolve → RBAC → audit. M3 `/api/chat/stream` and `/api/chat/rejoin` MUST conform.
- **INV-2 (Tenant scoping):** Every DB query runs under `SET app.tenant_id` via `withTenant` transaction; RLS enforces scoping. M3 queries against `chat_sessions`, `chat_turns`, `workflow_events`, `control_state` MUST use `withTenant`. RLS policies for new tables ship in the M3 migration.
- **INV-3 (Secret resolution):** Every credential resolved via `SecretProvider.get`; never env/config/DB for tenant secrets. M3 scheduler tasks (Trigger.dev) for Vanta sync + posture probes MUST resolve tenant Vanta API key via SecretProvider, not env.
- **INV-4 (Audit hash-chain):** Every auditable event appended to `audit_log` with hash-chain; `UPDATE/DELETE` REVOKE'd; write failure halts enclosing operation. **M3 preserves INV-4 as written.** Per-row hash-chaining continues. Session-scoped Merkle root per `chat_session` row is additive and does not weaken INV-4.
- **INV-5 (BYOM routing):** Every LLM inference call routed to tenant's BYOM endpoint; unconfigured/unreachable → reject (REQ-009). **M3 enforces this for all roles (planner, executor, synthesis) — no D-008 split.** All N inference calls per turn hit the same `byom_endpoints` row.
- **INV-6 (Relay Agent outbound-only):** Unchanged from M2. M3 chat does not directly invoke Relay Agent; chat → broker → adapter → (optionally) Relay Agent.
- **INV-7 (Read-only by default):** Closed 9-tool registry is the primary boundary; write-method blocklist is the backstop. M3 orchestrator MUST refuse any tool call that resolves to a write capability, even if planner requests it.
- **INV-8 (PGlite RLS gap):** PGlite 0.5.7 doesn't enforce RLS on SELECT. App-layer `withTenant` + explicit `WHERE tenant_id` is primary in dev/test; RLS + FORCE RLS is prod backstop. M3 new tables (`chat_sessions`, `chat_turns`, `workflow_events`, `control_state`) MUST have explicit `tenant_id` columns + RLS policies + `withTenant` usage.

**Performance NFRs:**
- Inline path: SSE completes within 5 min p95 (acceptance gate).
- Durable rejoin: p95 ≤30s from `workflow.completed` audit emission to first replayed event receipt (Phase 2 Item 6 anchor).
- Posture page render: <500ms p95 (backed by `control_state` write-through cache).
- Probe cadence: 5-min default interval (REQ-041).
- Vanta sync: 1-h default interval (REQ-042).
- Metering aggregation: 1-min default interval (REQ-043).

**Schema additivity (per `steer-m2-spec.md:162` — "new tables" parenthetical explicitly permits new tables, not just columns):**
- New tables: `chat_sessions`, `chat_turns`, `workflow_events`, `control_state`.
- New `audit_log` columns: `session_id` (FK `chat_sessions`, nullable), `workflow_id` (nullable).
- New audit event types: see §2.1.

**Migration shape (Decision #7 — per-table for rollback isolation, one PR):**
- `0004_chat_schema.sql` — `chat_sessions`, `chat_turns` + RLS; `audit_log.session_id` column.
- `0005_workflow_schema.sql` — `workflow_events` + RLS; `audit_log.workflow_id` column.
- `0006_posture_schema.sql` — `control_state` + RLS.

**Anti-goal enforcement:**
- No write actions on infrastructure (closed 9-tool registry + write-method blocklist + INV-7).
- No hosted LLM inference (single BYOM endpoint, all roles, INV-5).
- No SOC2 final cert (no signed attestations in M3; REQ-042 pushes evidence only).

**`control_state` cache table (Decision #8 — write-through):**
- `control_state` row updated in the same Postgres transaction as the `control.probe.{succeeded,failed}` audit event emission. Cache is always consistent with audit log at probe completion.
- **Implementation question (§7 Q1):** Engineering to confirm write-through is achievable in the chosen probe pipeline (likely Trigger.dev task per REQ-042 architecture). If a single transaction is not achievable, surface divergence handling in a v1.3 spec amendment.

---

## 6. Milestone Plan & Release Gates

**Test evidence required for Production Release (M3):**

- [ ] Code coverage ≥80% on new packages (`packages/orchestrator`, `packages/chat`); existing M2 floor (92.3% on `packages/mcp`) maintained.
- [ ] All 15 REQs PASS (REQ-028..037, REQ-041..045) including REQ-035 amendment acceptance criteria.
- [ ] CI/CD pipeline GREEN (Gitea Actions, both `test-pglite` and `test-postgres` jobs).
- [ ] M1+M2 non-regression: full 656-test suite green.
- [ ] INV-1..INV-8 explicitly tested (new tests for INV-4 audit hash-chain preservation across M3 event types; new tests for INV-5 BYOM routing across planner/executor/synthesis; new tests for INV-2 RLS on `chat_sessions`/`chat_turns`/`workflow_events`/`control_state`).
- [ ] MCP 2025-06-18 conformance verified (M2 gate, non-regression).
- [ ] LLM smoke Track A passes (mock-path, P0 gate, no external deps).
- [ ] QA sign-off: 100% of J1, J1-durable, J2-hardening, J3 integration tests pass; J-infra (REQ-045) has unit + cross-process integration test demonstrating gateway ↔ Trigger.dev worker rate coordination.
- [ ] Security/compliance review: posture page + Vanta sync demonstrate evidence lineage (every rendered status has an audit row); RBAC denies Casey on all write/admin endpoints; audit log hash-chain integrity verified end-to-end.
- [ ] Production deployment is an ops track (not in this gate list); deploy to AWS us-east-1 follows the ops track's own gate.

**Durable-path SLO refinement (gate clarification — does not rewrite `REQUIREMENTS.md:40`):**
> Inline path: p95 ≤5min. Durable path: rejoin p95 ≤30s from `workflow.completed` audit emission to first replayed event receipt; hard TTL 24h. `REQUIREMENTS.md:40` raw gate language stays untouched; this refinement documents the durable-path clarification locally in the M3 spec.

**Operational prereqs (ship-blockers, NOT M3 REQ deliverables — ops track, parallel to PDLC):**
- Gitea Actions runner enablement on forge (blocks full M2 + M3 CI verification on `test-postgres` job).
- `GITHUB_SMOKE_PAT` provisioning as Gitea Actions secret (blocks Track B GitHub LLM smoke; Track A mock-path is the P0 gate and does not need this).

---

## 7. Open Questions & Assumptions

1. **Trigger.dev task schema for durable workflows** — RESOLVED. Workflow input=`{session_id, user_turn_id, planner_plan, byom_endpoint_id}`; output=`{workflow.completed event payload}`; retry policy=exponential backoff ×3; idempotency key=`workflow_id`. Baked into REQ-036 §4.7.
2. **M3 acceptance gate refinement (durable path SLO)** — RESOLVED. "Inline path: p95 ≤5min. Durable path: rejoin p95 ≤30s post-`workflow.completed` audit emission; hard TTL 24h." Baked into §6. `REQUIREMENTS.md:40` stays untouched; refinement documented locally in M3 spec §6.
3. **Chat session lifecycle** — RESOLVED. Soft-close 30-min idle; hard-close 7d or explicit. Read-only post-close, not deleted. Baked into REQ-032 §4.
4. **Workflow event retention vs audit log retention** — RESOLVED. Align `workflow_events` with `audit_log` (90d min, 1y target). S3 WORM archive covers long-term. Baked into REQ-036 §4.6.
5. **Citation rendering UX** — DEFERRED. Exact UI for inline `[c:1]` markers + citation popover. Defer to design review. Non-blocking for spec ratification. Proposed: inline superscript numbers; click expands to source adapter + tool invocation link + output snippet.
6. **Posture page RBAC granularity** — RESOLVED. Casey sees all controls (posture, usage, audit) read-only; no probe-trigger, no config edit, no Vanta credential view. Baked into §3.1 + REQ-041 §4.7 + REQ-044 §4.
7. **`control_state` cache invalidation strategy** — OPEN (engineering confirm during implementation). Spec default: write-through (same transaction as `control.probe.*` audit emission). Engineering to confirm feasibility in chosen probe pipeline; if not achievable, v1.3 amendment with divergence handling.

---

## 8. Changelog

| Version | Date | Author | What Changed | REQs Affected |
| :--- | :--- | :--- | :--- | :--- |
| v1.0 | 2026-08-25 | Sarah Chen (PM) | Initial M3 spec draft from Phase 3 generation | REQ-028..037, REQ-041..045 (with REQ-035 amendment) |
| v1.1 | 2026-08-25 | Sarah Chen (PM) | Open Questions 1-6 resolved and baked in (Trigger.dev task schema, durable SLO, session lifecycle, workflow retention, Casey RBAC); OQ5 deferred to design review | REQ-032, REQ-036, REQ-041, REQ-044 |
| v1.2 | 2026-08-25 | Sarah Chen (PM) | Engineering handoff: Decision #7 (per-table migrations `0004`/`0005`/`0006`), Decision #8 (`control_state` write-through cache), Decision #9 (Redis fail-closed 503), Decision #10 (usage on all BYOM calls); OQ7 opened (`control_state` write-through feasibility); ops prereqs documented in §6 | REQ-036, REQ-041, REQ-043, REQ-045 |

---

**M3 spec v1.2 Final. Ready for engineering handoff.**