coreci-chat/.ciagent/ARCHITECTURE.md

Architecture

Overview

CoreCI Chat v0.1 is a multi-tenant SaaS with a TypeScript control plane, a Go Relay Agent distributed to customer Linux hosts, and an OpenAI-compatible BYOM routing layer. The control plane runs in a single AWS region (us-east-1) and enforces tenant isolation via Postgres Row-Level Security. Every request flows through an API gateway that authenticates the session, resolves the tenant, enforces RBAC, and writes an immutable audit entry. The Relay Agent is an outbound-only WebSocket client that registers as a target and heartbeats to the control plane; in M2 it will receive MCP tool calls (SSH) and enforce a fixed command whitelist. All credentials live in AWS Secrets Manager (prod) or a local-encrypted fallback (dev) behind a SecretProvider interface — never env vars, config files, or DB columns.

The v0.1 wedge is read-only diagnostic: no write actions, no hosted inference, no remediation. Async durable execution is provided by Trigger.dev (instrumented in M3 for chat workflows; the runtime bootstrap lands in M1 Wave A so M3 plugs in cleanly).

Init-time invariants (preserved by the architecture-drift guard)

• Single git repository at ~/coreci-chat, branch hierarchy main → milestone/v0.1-bootstrap → phase/NN-*.
• .ciagent/ reference files at repo root (single-project mode).
• Release target: Gitea coreci/coreci-chat at https://git.cloudinit.dev.
• Autonomy: full (no HITL after clarify; decision threshold 0.6).

Architecture invariants (M1 → v0.1)

• Every HTTP request hits the API gateway first: auth → tenant resolve → RBAC → audit.
• Every DB query runs under SET app.tenant_id = ? via a session-scoped transaction; RLS policies enforce scoping. Cross-tenant queries return empty.
• Every credential is resolved via SecretProvider.get(tenantId, key); the provider abstracts AWS Secrets Manager (prod) and local-encrypted (dev/test).
• Every auditable event (prompt, tool call, SSH command, response) is appended to audit_log with a prev_hash → curr_hash chain; UPDATE/DELETE are REVOKE'd from the app role; a write failure halts the enclosing operation (REQ-038 Edge 7).
• Every LLM inference call is routed to the tenant's configured BYOM endpoint via an OpenAI-compatible /v1/chat/completions contract; unconfigured/unreachable → reject with actionable error (REQ-009).
• The Relay Agent makes only outbound connections (WebSocket to SaaS). No inbound firewall rules on customer hosts.

Components

apps/control-plane (TypeScript, Next.js App Router API routes + standalone services)

• Description: API gateway, tenant resolution, RBAC enforcement, BYOM registry/validation/routing, Relay Agent WebSocket server, audit writer, secret provider bootstrap, dashboard (Next.js). Hosts the admin dashboard in M1; the chat UI lands here in M3.
• Boundaries: All inbound HTTP/WebSocket enters here. Owns the Postgres connection and the SET app.tenant_id discipline. Owns the Trigger.dev runtime bootstrap.
• Depends on: packages/db, packages/auth, packages/audit, packages/secrets, packages/config, Trigger.dev, WorkOS, AWS Secrets Manager.

apps/relay-agent (Go)

• Description: Single static binary. Outbound WebSocket client to the control plane; registration (tenant/target/hostname/OS/IP/version); heartbeat + exponential-backoff reconnect (max 5 → alert); systemd service; SSH whitelist file format + enforcement hook (the SSH adapter plugs into the hook in M2 — no SSH execution in M1).
• Boundaries: No inbound ports. Reads its tenant registration token from a local config written by the install script (token resolves to a secret-manager reference, never the raw secret). All state is in-memory or on the control plane.
• Depends on: control-plane WebSocket endpoint; systemd; the fixed SSH whitelist file shipped alongside the binary.

apps/dashboard (Next.js, part of apps/control-plane in M1)

• Description: Tenant admin dashboard surfaced in the same Next.js app. M1 surfaces Relay Agent status (green/yellow/red), target hostname, last 100 log lines, BYOM config + validation, RBAC user/role management. Per-tenant view under RLS.
• Boundaries: Server components read via the same API gateway (no DB access bypassing RLS). Client components subscribe to a status WebSocket fan-out.
• Depends on: apps/control-plane API, packages/auth.

packages/db (TypeScript)

• Description: Postgres schema, migrations, RLS policies, the withTenant(tenantId, fn) helper that sets app.tenant_id and runs fn in a transaction. Tables: tenants, users, tenant_memberships (user × tenant × role), targets, byom_endpoints (URL in DB, key ref to secret manager), audit_log (append-only, hash-chain), invitations. All tenant-scoped tables carry tenant_id and an RLS policy.
• Boundaries: The ONLY module that opens a Postgres connection. App role has INSERT/SELECT on audit_log only; UPDATE/DELETE REVOKE'd. Schema migrations run via a gated migrator.
• Depends on: Postgres 16.

packages/auth (TypeScript)

• Description: WorkOS SSO/SAML integration, session management, tenant resolution middleware, RBAC role → route permission map (Admin/Operator/Viewer). Enforces RBAC at the API gateway from the first endpoint (REQ-005 pattern).
• Boundaries: Sessions are httpOnly cookies; tokens never logged. WorkOS is the only IdP in v0.1.
• Depends on: WorkOS SDK, packages/db.

packages/audit (TypeScript)

• Description: Append-only audit writer. Computes curr_hash = sha256(prev_hash || canonical_payload). Payload includes tenant_id, user_id, target_id (for SSH), timestamp, correlation_id, event_type, event_body. On write failure: throws AuditWriteHaltError → enclosing operation halts + admin alert (REQ-038 Edge 7).
• Boundaries: Never exposes UPDATE/DELETE. Read path is admin-export only (no query UI in MVP — spec §2.2).
• Depends on: packages/db.

packages/secrets (TypeScript)

• Description: SecretProvider interface with two impls: AwsSecretsManagerProvider (prod, us-east-1, KMS-backed) and LocalEncryptedProvider (dev/test, AES-256-GCM with a key from an env var only for the master key — the master key is the ONE allowed env var, everything else is provider-resolved). Every tenant credential (BYOM key, Proxmox token, SSH key, Git token) is stored/retrieved via provider.put(tenantId, name, value) / provider.get(tenantId, name).
• Boundaries: Secrets never logged, never written to DB columns, never in config files. The DB stores only a reference (e.g. aws-sm:coreci/<tenantId>/byom).
• Depends on: AWS SDK (prod), Node crypto (dev).

packages/config (TypeScript)

• Description: Typed config loading (env-aware: prod/staging/dev/test). Loads non-secret config only. The only env vars consumed here are infra-level (DATABASE_URL, WORKOS_API_KEY, AWS_REGION, SECRET_MASTER_KEY_DEV, TRIGGER_API_KEY). All tenant/customer secrets go through packages/secrets.
• Boundaries: Never reads tenant secrets directly.
• Depends on: nothing.

Data Flow

M1 happy path — Relay Agent registration (J2 steps 3–4)

Platform Lead ──install script──▶ customer Linux host
  detect-OS → install-binary → write-systemd-unit → register-target (writes tenant reg token to local config)
systemd ──start──▶ relay-agent (Go binary)
relay-agent ──outbound WebSocket (wss, tenant reg token)──▶ control-plane WS server
control-plane: auth token → resolve tenant (RLS) → INSERT target (hostname/OS/IP/version) → audit append
control-plane ──ack + target_id──▶ relay-agent
relay-agent ──heartbeat every Ns──▶ control-plane (updates last_seen → green status)
control-plane ──status fan-out──▶ dashboard (server-sent events / WS)

M1 happy path — BYOM configure + validate (J2 step 2)

Admin ──POST /api/byom (url, key)──▶ API gateway
  auth → tenant resolve (RLS) → RBAC (Admin only) → audit append (config event)
  secrets.put(tenantId, "byom", key) → returns ref
  INSERT byom_endpoints (tenant_id, url, secret_ref) — under RLS
  BYOM validator: fetch OpenAI-compatible /v1/chat/completions with a trivial test prompt
    on success → mark validated=true, audit append (validation ok), return green
    on failure → rollback insert, audit append (validation fail), return error details (Edge 11)

M3 (forward reference) — chat diagnostic (J1)

Operator ──prompt──▶ API gateway → auth → tenant → RBAC (Operator) → audit append (prompt)
  orchestrator (Trigger.dev task) → BYOM routing shim → POST tenant's BYOM /v1/chat/completions
    if unconfigured/unreachable → reject (REQ-009, Edge 1), audit append, halt
  LLM returns tool selection → MCP gateway → route to adapter (M2) → Relay Agent (SSH, M2)
    read-only enforced at gateway (403 on write) and Relay Agent (whitelist)
  tool output streams via SSE → UI trace panel → audit append (tool call + SSH command + output)
  LLM synthesizes cited answer → stream to UI → audit append (response)
  conversation persisted (tenant-scoped) → audit append

Build Order (M1 waves — each a vertical slice, testable + shippable)

1. Wave A — Foundations. Monorepo (pnpm): apps/control-plane, apps/relay-agent (Go module), apps/dashboard, packages/{db,auth,audit,secrets,config}. Postgres schema + RLS + withTenant. audit_log hash-chain + REVOKE UPDATE/DELETE. SecretProvider interface + AWS SM impl + local-encrypted impl. Trigger.dev bootstrap (runtime wired, no tasks yet). Covers REQ-038, REQ-039, REQ-040.
2. Wave B — Identity & RBAC. WorkOS SSO; session; tenant resolution middleware (SET app.tenant_id); RBAC at API gateway (Admin/Operator/Viewer → route permission map). First endpoint protected on day one. Covers REQ-001, REQ-002, REQ-003, REQ-004, REQ-005.
3. Wave C — BYOM. Tenant-scoped BYOM endpoint registry (URL in DB, key in secret manager); validate-on-save test inference call (OpenAI-compatible); routing shim (no inference in M1 — proxy contract + REQ-009 reject path). Covers REQ-006, REQ-007, REQ-008, REQ-009.
4. Wave D — Relay Agent. Modular install script (detect-OS/install-binary/write-systemd-unit/register-target; clean abort on unsupported OS). Go binary: outbound wss + tenant reg token, registration (tenant/target/hostname/OS/IP/version), heartbeat + exp-backoff (max 5 → alert), auto-reconnect, systemd unit w/ auto-restart. SSH whitelist file format + enforcement hook (no adapter yet — M2 plugs in). Covers REQ-010, REQ-011, REQ-012, REQ-013, (REQ-026 whitelist hook).
5. Wave E — Dashboard surfacing. WebSocket fan-out of agent status to dashboard; green/yellow/red; target hostname; last 100 log lines; per-tenant view under RLS. M1 gate demo: SSO → BYOM green → install → register → green dashboard. Covers REQ-014.