Praxis CI
813bd586d6
v0.3 milestone merged to main. Mastery scoring + competency rubrics + verifiable credentials (formative-tier) shipped. 13/13 REQ-IDs covered. Next milestone: v0.4 (operator tier — cohort dashboard + auth + Postgres). ---ci--- project: praxis phase: 2 milestone: v0.3 status: complete milestone_complete: true milestone_merged_to_main: true ---/ci---
481 lines
29 KiB
Markdown
481 lines
29 KiB
Markdown
# Praxis — Architecture (Research-Refined)
|
||
|
||
> **Status:** Research-refined (Phase 0 RESEARCH stage). Informed by `.ciagent/RESEARCH.md` — web-verified vendor catalogs, GitHub metadata, official docs.
|
||
|
||
## High-Level Topology
|
||
|
||
Three-tier architecture per PRD §7:
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────┐
|
||
│ Client (Android, iOS, Web, WhatsApp, USSD) │
|
||
│ - Voice I/O, cached scenarios, offline scenarios │
|
||
└────────────────┬─────────────────────────────────────────┘
|
||
│
|
||
┌────────────────▼─────────────────────────────────────────┐
|
||
│ Edge / Region (per market) │
|
||
│ - ASR + TTS (low-latency, local accent models) │
|
||
│ - Scenario runtime + role orchestration │
|
||
│ - Caching layer │
|
||
└────────────────┬─────────────────────────────────────────┘
|
||
│
|
||
┌────────────────▼─────────────────────────────────────────┐
|
||
│ Core Platform │
|
||
│ - LLM tutor (long-context, persona-aware, safety-tuned) │
|
||
│ - Scenario Authoring & Tagging │
|
||
│ - Mastery Rubric Engine │
|
||
│ - User state, progress, credentialing │
|
||
│ - Analytics │
|
||
└──────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
## LLM Foundation (D-003, D-020 — research-verified)
|
||
|
||
Open-weights models hosted via **Ollama Cloud direct API** (`https://ollama.com/api/chat` + `OLLAMA_API_KEY`) — no local daemon required for v0.1.
|
||
|
||
| Model | Verified status | Role | Context | Mode |
|
||
|-------|-----------------|------|---------|------|
|
||
| `gemma4:cloud` | ✅ Real, current (256K ctx, Text+Image, "Low Usage" tier) | Role-play fast path / persona turns | 256K | standard |
|
||
| `deepseek-v4-flash:cloud` | ✅ Real, current (1M ctx, 284B MoE / 13B active, "Medium Usage" tier) | Coaching debrief + scenario-branch decisions | 1M | **no-think** (latency); think/max-think reserved for offline analysis |
|
||
|
||
**Post-pilot cost-reduction path:** self-host `gemma4:e4b` (edge, native audio modality, 9.6GB) on partner hardware for the ≤$3/learner/month target. Architecture must keep the model-call layer swappable (D-020).
|
||
|
||
**Notable future option:** `gemma4:e2b`/`e4b` support Text+Image+Audio input — potential future Ollama-hosted ASR for cost reduction (not v0.1; dedicated Deepgram is lower-latency + more accent-robust).
|
||
|
||
## v0.1 Component Map (research-refined minimal viable voice loop)
|
||
|
||
```
|
||
Client: React + WebRTC (Pipecat client SDK)
|
||
│ audio in/out (WebRTC, UDP, sub-50ms)
|
||
▼
|
||
Pipecat server (Python)
|
||
├─ VAD: Silero
|
||
├─ STT: Deepgram Nova-3 (cloud, streaming, WebSocket)
|
||
├─ LLM: Ollama Cloud direct API (https://ollama.com/api/chat)
|
||
│ ├─ gemma4:cloud (role-play fast path)
|
||
│ └─ deepseek-v4-flash:cloud (debrief, no-think mode)
|
||
├─ TTS: Cartesia Sonic (cloud, ~120ms) ← behind interface
|
||
│ └─ fallback: Piper (self-hosted, ~80ms) ← R4 mitigation
|
||
├─ Scenario runtime: Pipecat Flows + YAML→Pydantic scenarios
|
||
├─ Guardrail layer: pluggable interface (v0.1: Customer Service ruleset)
|
||
└─ Learner state: SQLite (praxis.db, single-learner, no auth)
|
||
```
|
||
|
||
**v0.1 deliberately excludes:** edge-region split, multi-market deployment, caching layer, scenario authoring tools, mastery engine, credentialing, analytics, WhatsApp/USSD surfaces.
|
||
|
||
## Latency Budget (< 600ms end-to-end — research-revised)
|
||
|
||
| Segment | Budget | Source / note |
|
||
|---------|--------|---------------|
|
||
| Client capture + WebRTC uplink | ~50ms | WebRTC UDP, Canada region |
|
||
| ASR (Deepgram Nova-3 first partial) | ~250ms | Vendor claim; **R1: measure in Phase 1** |
|
||
| LLM first token (gemma4:cloud direct API) | ~200ms | **R3: measure in Phase 1** |
|
||
| TTS first audio (Cartesia Sonic) | ~120ms | Vendor/leaderboard; **R2: measure in Phase 1** |
|
||
| WebRTC downlink + playback | ~50ms | |
|
||
| **Total (all-cloud target)** | **~670ms** | ⚠️ Marginally over 600ms |
|
||
| **Total (Piper TTS mitigation)** | **~550ms** | R4: pre-stage Piper self-hosted on pilot server |
|
||
|
||
**R4 — single biggest v0.1 technical risk:** the all-cloud three-hop path likely lands ~670ms. The TTS service MUST sit behind an interface (D-014) and Piper-on-pilot-server MUST be pre-staged as the likely production v0.1 TTS. This is the first Phase 1 spike.
|
||
|
||
## Critical Risks to Engineer Around
|
||
|
||
1. **Accent robustness** — even a great LLM fails if ASR mishears the learner. Canadian English/French accents, code-switching.
|
||
2. **Hallucinated advice in safety-sensitive domains** — health, electrical. Domain-specific guardrails, escalation, disclaimers. (v0.1 uses Customer Service path, lower risk, but architecture must support the guardrail layer.)
|
||
3. **Cost per learner per month** must stay ≤ $3 in target markets. v0.1 Canada pilot relaxes this, but architecture must not bake in assumptions that violate it.
|
||
4. **Ollama model availability / cost** — `:cloud` variants imply hosted inference; verify pricing and rate limits at research phase.
|
||
|
||
## Deployment (v0.1)
|
||
|
||
- Single-region pilot (Canada)
|
||
- LLM via Ollama Cloud direct API (no local daemon)
|
||
- ASR via Deepgram cloud (North American endpoint)
|
||
- TTS: Cartesia cloud (quality benchmark) + Piper self-hosted on pilot server (R4 latency mitigation, likely production v0.1)
|
||
- Pipecat server on single pilot host (Python)
|
||
- Client: React web app (Pipecat client SDK, WebRTC transport)
|
||
- SQLite local file (`praxis.db`) on pilot host
|
||
|
||
## Open Architecture Questions (resolved by research)
|
||
|
||
| Question (from initial ARCHITECTURE.md) | Resolution |
|
||
|------------------------------------------|------------|
|
||
| Client framework | **React + WebRTC** via Pipecat client SDK (D-015) |
|
||
| Streaming transport | **WebRTC** (Pipecat); WebSocket dev fallback (D-016) |
|
||
| ASR/TTS provider | **Deepgram Nova-3** (ASR, D-013); **Cartesia Sonic** + Piper fallback (TTS, D-014) |
|
||
| Learner state store | **SQLite** confirmed (D-007 → 0.90) |
|
||
| Ollama deployment | **Ollama Cloud direct API** (D-020) |
|
||
| Scenario definition format | **YAML DSL → Pydantic → Pipecat Flows** (D-018) |
|
||
|
||
## Open Architecture Questions (remaining for PLAN stage)
|
||
|
||
- R1-R4 latency spikes (see Risks below) — first Phase 1 tasks
|
||
- Pipecat Flows schema mapping for the one branch point (escalate vs accept) in the refund scenario
|
||
- Guardrail ruleset concrete implementation (D-019) — system-prompt template + output filter
|
||
- SQLite schema for session log + progress + scenario state
|
||
- OLLAMA_API_KEY + DEEPGRAM_API_KEY + CARTESIA_API_KEY secret management (extend `config.secrets.scopes`)
|
||
|
||
---
|
||
|
||
## v0.2 Deployment Architecture (Proxmox LXC + Docker-in-LXC)
|
||
|
||
> **Status:** Research-refined (v0.2 RESEARCH stage). Informed by `.ciagent/RESEARCH.md` — Proxmox VE wiki, coreci script analysis, Docker/systemd ecosystem.
|
||
> **Decisions:** D-021 (LXC deploy), D-022 (Docker in LXC, nesting=1), D-023 (FastAPI StaticFiles), D-024 (infra-only keys), D-025/D-029 (build inside CT), D-026 (coreci secrets), D-027 (auto VMID), D-028 (Docker via apt), D-030 (vmbr0 DHCP).
|
||
|
||
### Docker-in-LXC Topology
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ Proxmox VE Host (PROXMOX_NODE) │
|
||
│ (D-026: secrets sourced from ~/coreci/.ciagent/ │
|
||
│ .env.secrets + praxis .ciagent/.env.secrets) │
|
||
│ │
|
||
│ Deploy operator runs: │
|
||
│ scripts/proxmox/lxc-deploy.sh │
|
||
│ ├─ stage-snippet.sh (upload hookscript to snippets) │
|
||
│ ├─ lxc-clone.sh (POST /nodes/{node}/lxc) │
|
||
│ ├─ lxc-config.sh (PUT /config + SSH lxc.env) │
|
||
│ ├─ lxc-start.sh (POST /status/start) │
|
||
│ └─ health-check.sh (poll /health:8789) │
|
||
│ │
|
||
│ ┌────────────────────────────────────────────────────┐ │
|
||
│ │ LXC Container (VMID: auto via pve_nextid, D-027) │ │
|
||
│ │ hostname: praxis │ │
|
||
│ │ memory: 4096MB rootfs: 16GB (bumped from 2/8) │ │
|
||
│ │ features: nesting=1 │ │
|
||
│ │ net0: bridge=vmbr0, ip=dhcp (D-030) │ │
|
||
│ │ hookscript: local:snippets/praxis-firstboot.sh │ │
|
||
│ │ lxc.environment: GITEA_TOKEN, DEEPGRAM_API_KEY, │ │
|
||
│ │ PRAXIS_PORT=8789, PRAXIS_HOST=0.0.0.0, ... │ │
|
||
│ │ │ │
|
||
│ │ post-start hook (runs on PVE host, pct exec → CT): │ │
|
||
│ │ 1. apt install docker.io docker-compose-v2 git │ │
|
||
│ │ 2. git clone praxis repo → /opt/praxis │ │
|
||
│ │ 3. install-service.sh (user + env + systemd unit) │ │
|
||
│ │ 4. systemctl start praxis │ │
|
||
│ │ → ExecStartPre: docker compose build │ │
|
||
│ │ → ExecStart: docker compose up (foreground) │ │
|
||
│ │ │ │
|
||
│ │ ┌──────────────────────────────────────────────┐ │ │
|
||
│ │ │ Docker daemon │ │ │
|
||
│ │ │ ┌────────────────────────────────────────┐ │ │ │
|
||
│ │ │ │ praxis container │ │ │ │
|
||
│ │ │ │ image: python:3.12-slim + deps + dist │ │ │ │
|
||
│ │ │ │ ports: 8789:8789 │ │ │ │
|
||
│ │ │ │ env_file: /etc/praxis/server.env │ │ │ │
|
||
│ │ │ │ volume: praxis-db → /app/data │ │ │ │
|
||
│ │ │ │ restart: unless-stopped │ │ │ │
|
||
│ │ │ │ │ │ │ │
|
||
│ │ │ │ uvicorn 0.0.0.0:8789 │ │ │ │
|
||
│ │ │ │ ├─ GET /health (FastAPI) │ │ │ │
|
||
│ │ │ │ ├─ POST /pipecat/webrtc (FastAPI) │ │ │ │
|
||
│ │ │ │ └─ GET / ... (StaticFiles client/dist)│ │ │ │
|
||
│ │ │ └────────────────────────────────────────┘ │ │ │
|
||
│ │ └──────────────────────────────────────────────┘ │ │
|
||
│ └────────────────────────────────────────────────────┘ │
|
||
│ │ │
|
||
│ vmbr0 (bridge) ──── DHCP ──── CT eth0 │
|
||
└───────────┬──────────────────────────────────────────────┘
|
||
│ <ct-bridge-ip>:8789
|
||
┌───────────▼───────────────────────┐
|
||
│ Operator / Learner (browser) │
|
||
│ http://<ct-ip>:8789 │
|
||
│ (direct access, no proxy/TLS) │
|
||
└───────────────────────────────────┘
|
||
```
|
||
|
||
### Image Build Pipeline (Multi-stage Dockerfile)
|
||
|
||
Two-stage build, Debian-slim bases, `python -m server` entrypoint:
|
||
|
||
```
|
||
Stage 1: client-builder (node:22-slim)
|
||
COPY client/package.json client/package-lock.json
|
||
RUN npm ci ← cached unless deps change
|
||
COPY client/
|
||
RUN npm run build ← tsc -b && vite build → client/dist/
|
||
|
||
Stage 2: server (python:3.12-slim)
|
||
RUN apt-get install gcc g++ libasound2-dev ← only if source compilation
|
||
COPY pyproject.toml
|
||
RUN pip install --no-cache-dir . ← pipecat-ai[deepgram,cartesia,piper,webrtc] + deps
|
||
COPY server/ scenarios/ db/
|
||
COPY --from=client-builder /app/client/dist ./client/dist
|
||
EXPOSE 8789
|
||
CMD ["python", "-m", "server"] ← calls uvicorn.run(app, host=HOST, port=PORT)
|
||
```
|
||
|
||
**Why Debian-slim (not Alpine):** numpy + pipecat-ai native extensions compile against glibc; musl wheels are less universally available. The ~50MB size saving of Alpine isn't worth the compatibility risk.
|
||
|
||
**Why `python -m server` (not `uvicorn server.__main__:app`):** Matches the existing entrypoint (`server/__main__.py:main()`) which reads `PRAXIS_HOST`/`PRAXIS_PORT` from env and calls `uvicorn.run(...)`. Single uvicorn process is correct for WebRTC/WebSocket (long-lived connections, not request-per-response).
|
||
|
||
### Secret Injection Chain
|
||
|
||
```
|
||
~/coreci/.ciagent/.env.secrets praxis/.ciagent/.env.secrets
|
||
PROXMOX_API_URL GITEA_TOKEN
|
||
PROXMOX_API_TOKEN DEEPGRAM_API_KEY
|
||
PROXMOX_NODE CARTESIA_API_KEY (empty, D-024)
|
||
PROXMOX_STORAGE OLLAMA_API_KEY (empty, D-024)
|
||
PROXMOX_TEMPLATE_VOLID
|
||
PROXMOX_TLS_SKIP_VERIFY
|
||
│ │
|
||
└────────┬───────────┘
|
||
▼
|
||
lxc-deploy.sh sources both
|
||
│
|
||
▼
|
||
lxc-config.sh (SSH to PVE host)
|
||
writes /etc/pve/lxc/<vmid>.conf:
|
||
lxc.environment: GITEA_TOKEN=<token>
|
||
lxc.environment: DEEPGRAM_API_KEY=<key>
|
||
lxc.environment: PRAXIS_PORT=8789
|
||
lxc.environment: PRAXIS_HOST=0.0.0.0
|
||
lxc.environment: OLLAMA_BASE_URL=https://ollama.com/v1
|
||
...
|
||
│
|
||
▼ (CT boots; systemd PID 1 has these env vars)
|
||
firstboot-hook.sh → pct exec install-service.sh
|
||
│
|
||
▼
|
||
/etc/praxis/server.env (root:praxis, chmod 0640)
|
||
GITEA_TOKEN=<token>
|
||
DEEPGRAM_API_KEY=<key>
|
||
PRAXIS_PORT=8789
|
||
...
|
||
│
|
||
▼
|
||
praxis.service (EnvironmentFile=/etc/praxis/server.env)
|
||
→ ExecStart: docker compose up
|
||
│
|
||
▼
|
||
docker-compose.yml (env_file: /etc/praxis/server.env)
|
||
│
|
||
▼
|
||
Docker container (os.environ)
|
||
→ server/__main__.py reads PRAXIS_HOST, PRAXIS_PORT, DEEPGRAM_API_KEY, ...
|
||
```
|
||
|
||
**.gitignore coverage:** `.env`, `.env.secrets`, `.env.*` are all gitignored in praxis (verified). No secrets are committed.
|
||
|
||
### CT Resource Sizing
|
||
|
||
| Resource | Coreci default | Praxis v0.2 | Rationale |
|
||
|----------|---------------|-------------|-----------|
|
||
| Memory | 2048 MB | **4096 MB** | Docker daemon (~200MB) + build peak (~1.2GB pip) + runtime (~500MB) + headroom |
|
||
| Rootfs | 8 GB | **16 GB** | Docker engine (~400MB) + build layers (~1.6GB) + final image (~1GB) + repo + apt + headroom |
|
||
| CPU cores | (default) | 2 | Sufficient for build + single-learner runtime |
|
||
| Swap | (default) | 0 | LXC swap is host swap; not needed for pilot |
|
||
|
||
Configured via `lxc-clone.sh` (`memory=${PROXMOX_MEMORY_MB:-4096}`, `rootfs=${storage}:16`) or env vars in the deploy script.
|
||
|
||
### Health-Check Path
|
||
|
||
```
|
||
lxc-deploy.sh
|
||
└─ health-check.sh <vmid>
|
||
│
|
||
├─ PRAXIS_HEALTH_URL set? → use directly
|
||
│
|
||
└─ else: pve_get /nodes/{node}/lxc/{vmid}/interfaces
|
||
│
|
||
├─ jq: .[] | select(.name != "lo") | (.inet? // .ip? // empty)
|
||
│ (NOT .hwaddr — P18 bug fix from coreci)
|
||
│
|
||
└─ health_url = http://<bridge-ip>:8789/health
|
||
│
|
||
└─ poll curl -fsS --connect-timeout 2 $health_url
|
||
for PRAXIS_HEALTH_TIMEOUT seconds (default 300s)
|
||
```
|
||
|
||
**Timing:** CT start → DHCP lease (~5s) → firstboot hook: apt install Docker (~90s) + git clone (~10s) + install-service + systemctl start (~120s: docker compose build + up) → uvicorn binds :8789 → health passes. Total: ~3-5 min. `PRAXIS_HEALTH_TIMEOUT=300` (5 min) covers this with margin.
|
||
|
||
### Firstboot Hook Sequence
|
||
|
||
```
|
||
Proxmox invokes hookscript at post-start phase (runs on PVE HOST):
|
||
$1 = VMID, $2 = phase
|
||
|
||
Phase: post-start
|
||
│
|
||
├─ 1. pct exec <vmid> -- apt-get install docker.io docker-compose-v2 git curl
|
||
│ (D-028: Docker via apt inside CT)
|
||
│
|
||
├─ 2. pct exec <vmid> -- git clone https://<GITEA_TOKEN>@git.cloudinit.dev/coreci/praxis.git /opt/praxis
|
||
│ (D-029: clone inside CT, self-contained)
|
||
│
|
||
├─ 3. pct exec <vmid> -- sh /opt/praxis/scripts/install-service.sh
|
||
│ │
|
||
│ ├─ create praxis user (useradd --system, add to docker group)
|
||
│ ├─ mkdir /var/lib/praxis/data /var/log/praxis /etc/praxis
|
||
│ ├─ write /etc/praxis/server.env from lxc.environment vars
|
||
│ ├─ install praxis.service systemd unit
|
||
│ └─ systemctl daemon-reload && enable praxis && restart praxis
|
||
│ │
|
||
│ ├─ ExecStartPre: docker compose build (TimeoutStartSec=300)
|
||
│ └─ ExecStart: docker compose up (foreground, Type=simple)
|
||
│
|
||
└─ 4. (hook exits 0; external health-check.sh polls /health:8789)
|
||
```
|
||
|
||
**Idempotency:** The hook checks if praxis is already installed + active before re-running (mirrors coreci's pattern at firstboot-hook.sh:82). Re-running `lxc-deploy.sh` against a healthy CT skips the hook entirely (P16 idempotency via `ct_exists` + `ct_running` + health-check).
|
||
|
||
### What's Reused Verbatim from CoreCI vs Adapted
|
||
|
||
| Component | Verdict | Notes |
|
||
|-----------|---------|-------|
|
||
| `api.sh` | **Verbatim** | REQ-DEPLOY-03. PVE REST helpers are project-agnostic. |
|
||
| `lxc-start.sh` | **Verbatim** | POST /status/start is identical. |
|
||
| `proxy/ct-exists.sh` | **Verbatim** | Used by lxc-deploy.sh idempotency; no proxy dependency in the helper. |
|
||
| `lxc-clone.sh` | Adapted | hostname=praxis, memory=4096, rootfs=16, features=nesting=1 (kept). |
|
||
| `lxc-config.sh` | Adapted | hookscript=praxis-firstboot.sh, lxc.environment vars for praxis. |
|
||
| `health-check.sh` | Adapted | /health (not /healthz), port 8789, PRAXIS_* env names, timeout 300s. |
|
||
| `rollback.sh` | Adapted | Remove proxy backend-remove (no proxy in v0.2). |
|
||
| `stage-snippet.sh` | Adapted | SNIPPET_NAME=praxis-firstboot.sh, praxis repo raw URL. |
|
||
| `timing.sh` | Adapted | Metric prefix: praxis_deploy_timing_. |
|
||
| `lxc-deploy.sh` | Adapted | Remove PROXY_VMID/BACKEND_DOMAIN steps; VMID=auto (D-027). |
|
||
| `firstboot-hook.sh` | **Heavy adaptation** | Docker install + git clone + compose build/up (not host-fetch binary). |
|
||
| `install-service.sh` | **Heavy adaptation** | praxis user (docker group), /etc/praxis/server.env, praxis.service (docker compose up). |
|
||
|
||
### v0.2 Deployment Risks (from RESEARCH.md)
|
||
|
||
| ID | Risk | Mitigation |
|
||
|----|------|------------|
|
||
| R-DEPLOY-01 | Pipecat wheel missing → source compilation OOM | Pre-test `docker build` locally; bump memory if needed |
|
||
| R-DEPLOY-02 | systemd TimeoutStartSec insufficient for build+up | Set 300-600s or split build into separate oneshot service |
|
||
| R-DEPLOY-03 | CT can't reach Gitea/apt mirrors | Validate internet access; fallback to host-clone+pct-push (D-025 hybrid) |
|
||
| R-DEPLOY-04 | Docker-in-LXC on ZFS rootfs | Check storage type; use local (directory) if ZFS |
|
||
| R-DEPLOY-05 | journald log flooding from compose up | Log rotation or StandardOutput=null for pilot |
|
||
| R-DEPLOY-06 | First-boot build > 5 min (NFR breach) | Pre-build on host + docker load fallback |
|
||
|
||
---
|
||
|
||
## v0.3 Architecture (Mastery Scoring + Competency Rubrics + VC + Cohort Dashboard)
|
||
|
||
> **Status:** Research-refined (v0.3 RESEARCH stage). Informed by `.ciagent/RESEARCH.md` v0.3 section.
|
||
> **Decisions:** D-031 (operator tier, overrides D-007 for operator surface), D-032 (mastery gate), D-033 (W3C VC 2.0), D-034 (k-anonymity), D-035 (IRT 1PL), D-036 (scenario library), D-037 (path structure), D-038..D-049 (clarify).
|
||
|
||
### Hybrid Storage Topology (D-031)
|
||
|
||
Learner-local state stays in SQLite (D-007 preserved); operator-tier state goes to a new Postgres service. The two stores never share a session and never join via cross-DB FKs (`learner_ref` is an opaque string in Postgres).
|
||
|
||
```
|
||
LXC Container (from v0.2, memory bumped 4GB → 6GB)
|
||
Docker daemon
|
||
├── praxis container (existing v0.2 + v0.3 additions)
|
||
│ ├─ uvicorn 0.0.0.0:8789
|
||
│ ├─ GET /health (v0.2)
|
||
│ ├─ POST /pipecat/webrtc (v0.2)
|
||
│ ├─ GET / ... StaticFiles (v0.2)
|
||
│ ├─ /api/operator/* NEW (v0.3 — operator auth gate)
|
||
│ ├─ /vc/verify/<id> NEW (v0.3 — public, unauthenticated)
|
||
│ ├─ SQLite /app/data/praxis.db (v0.2 + NEW v0.3 tables: learner_ability, mastery_progress)
|
||
│ └─ Postgres pool (asyncpg) (v0.3 — operator tier)
|
||
│
|
||
└── postgres container NEW (v0.3)
|
||
├─ postgres:16-slim
|
||
├─ pgdata named volume
|
||
├─ internal Docker network only (no published port)
|
||
├─ pg_isready healthcheck
|
||
└─ Tables: operators, issued_credentials, mastery_gate_events, cohort_aggregates, issuer_keys
|
||
```
|
||
|
||
### v0.3 Component Map (additions to v0.2)
|
||
|
||
```
|
||
Pipecat server (Python)
|
||
├─ ... (v0.2 voice loop unchanged) ...
|
||
├─ Rubric engine NEW (server/mastery/)
|
||
│ ├─ rubric_loader.py (rubrics/<skill>.yaml → Pydantic)
|
||
│ ├─ rubric_scorer.py (rule-based: signals → 1-5, deterministic — REQ-NFR-MAST-01)
|
||
│ ├─ evidence_extractor.py (LLM extracts quotes+signals, temp=0, JSON-schema)
|
||
│ └─ mastery_score.py (weighted mean + conjunctive floor + path gate)
|
||
├─ IRT engine NEW (server/mastery/irt.py)
|
||
│ ├─ 1PL/Rasch: P(success) = logistic(θ − b)
|
||
│ ├─ Bayesian θ update per session (<100ms — REQ-NFR-IRT-01)
|
||
│ └─ θ persisted to SQLite learner_ability (D-046)
|
||
├─ Scenario library NEW (server/scenarios/library.py)
|
||
│ ├─ scenarios/<path>/<id>.yaml + scenarios/index.yaml (semver, rubric_criteria mapping)
|
||
│ └─ AI variation review pipeline (_pending/ → expert review → library)
|
||
├─ Path engine NEW (server/paths/)
|
||
│ ├─ paths/<slug>.yaml (6-week structure, mastery gates — D-037)
|
||
│ └─ progression: current_week advances on gate-open (D-048)
|
||
├─ VC issuer NEW (server/vc/)
|
||
│ ├─ issuer.py (Ed25519, pynacl + canonicaljson + base58, eddsa-jcs-2022)
|
||
│ ├─ status_list.py (Bitstring Status List v1.0)
|
||
│ ├─ verification.py (public GET /vc/verify/<id> — D-043)
|
||
│ └─ issuer key in Postgres issuer_keys (encrypted at rest)
|
||
├─ Operator auth NEW (server/auth/)
|
||
│ ├─ SessionMiddleware (Starlette, itsdangerous-signed cookie — D-041)
|
||
│ ├─ argon2id passwords (argon2-cffi)
|
||
│ ├─ current_operator Depends
|
||
│ └─ slowapi 5/min login rate-limit
|
||
├─ Cohort aggregation NEW (server/cohort/)
|
||
│ ├─ on-session-end hook → k-anonymized aggregate upsert to Postgres (D-045)
|
||
│ └─ nightly reconciliation job (cron in praxis service)
|
||
└─ Operator API NEW (server/operator/)
|
||
├─ /api/operator/login, /api/operator/logout
|
||
├─ /api/operator/cohort (k-anonymized, ≥10 learners/cell — D-034)
|
||
└─ /api/operator/credentials (issued VCs, revocation)
|
||
|
||
Client (React)
|
||
├─ ... (v0.2 voice UI unchanged) ...
|
||
└─ /operator/* NEW (v0.3 — cohort dashboard UI, auth-gated — D-044)
|
||
```
|
||
|
||
### Mastery Scoring Flow (off the voice path)
|
||
|
||
```
|
||
Session end (server/session_recorder.py)
|
||
│
|
||
├─ 1. Evidence extraction (LLM, async, off-voice-path)
|
||
│ deepseek-v4-flash:cloud, temp=0
|
||
│ Input: session turns + scenario.rubric_criteria
|
||
│ Output (JSON-schema-validated): [{criterion_id, quote, signals: [...]}]
|
||
│ Guard: fuzzy-match quote vs transcript → reject+re-extract on mismatch (R-MAST-02)
|
||
│
|
||
├─ 2. Rule-based scoring (deterministic, no LLM — REQ-NFR-MAST-01)
|
||
│ rubric_scorer.py: signals → 1-5 level per criterion
|
||
│
|
||
├─ 3. Mastery Score (deterministic)
|
||
│ scenario_score = weighted_mean(levels, weights)
|
||
│ scenario_pass = scenario_score ≥ 3.0 AND every criterion ≥ 2 (conjunctive floor)
|
||
│ path MasteryScore = mean(scenario_scores for passing scenarios only)
|
||
│ path gate open = ≥3 distinct scenarios passed AND MasteryScore ≥ 3.5 (D-032)
|
||
│
|
||
├─ 4. IRT θ update (deterministic, <100ms — REQ-NFR-IRT-01)
|
||
│ θ ← θ + (outcome − P) × σ²/(σ² + 1); persist to SQLite learner_ability (D-046)
|
||
│
|
||
├─ 5. Progression (deterministic)
|
||
│ gate open → advance current_week (D-048)
|
||
│ week-final gate open → issue VC (REQ-MAST-03)
|
||
│ record mastery_gate_event in Postgres (REQ-NFR-MAST-02)
|
||
│
|
||
└─ 6. Cohort aggregation (async, k-anonymized)
|
||
on-session-end hook → upsert k-anonymized aggregate to Postgres (D-045)
|
||
nightly reconciliation reconciles 7-day windows
|
||
```
|
||
|
||
### VC Issuance + Verification Flow
|
||
|
||
```
|
||
Mastery gate opens (week-final)
|
||
├─ issuer.py: build payload {scenariosPassed, rubricScore, completedWeeks:6, evidence, validUntil:+3y}
|
||
│ canonicalize (JCS) → sign Ed25519 → store in Postgres issued_credentials
|
||
└─ Verification (third party): GET /vc/verify/<id> → fetch pubkey from verificationMethod URL
|
||
→ validate Ed25519 sig → check Status List → return {valid, status, issuer, mastery, verifiedAt}
|
||
```
|
||
|
||
### Postgres Schema (operator tier — D-040)
|
||
|
||
Tables: `operators` (id, username, password_hash argon2id), `issued_credentials` (id, learner_ref opaque-string, vc_payload_json, signature_b64, status, issued_at), `mastery_gate_events` (id, learner_ref, path, week, scenarios_passed_json, rubric_scores_json, gate_opened_at — REQ-NFR-MAST-02 audit), `cohort_aggregates` (path, week, window_start/end, metric, value, cell_suppressed — k-anon via write-time suppression, weekly partitions), `issuer_keys` (id, public_key Multikey, private_key_enc, status active|superseded). `gen_random_uuid()` in PG16 (no extension). No cross-DB FKs.
|
||
|
||
### CT Resource Sizing (v0.3 bump)
|
||
|
||
| Resource | v0.2 | v0.3 | Rationale |
|
||
|----------|------|------|-----------|
|
||
| Memory | 4096 MB | **6144 MB** | Postgres ~1GB + praxis ~2GB + build headroom (R-MT-01) |
|
||
| Rootfs | 16 GB | 16 GB | Postgres data on named volume, not rootfs |
|
||
| CPU | 2 | 2-4 | Postgres + praxis concurrent; 2 floor, 4 preferred |
|
||
|
||
### v0.3 Risks (from RESEARCH.md)
|
||
|
||
Top risks for PLAN: R-MAST-01 (N=3 thin for credential → label formative), R-AUTH-01 (Secure cookie + no-TLS pilot), R-MT-01 (Postgres resource contention), R-VC-01 (custom VC code ~200 LOC), R-MAST-02 (LLM hallucinated quotes → fuzzy-match guard), R-IRT-01 (cold-start θ → fall back to scenario.difficulty until ≥5 sessions). Full table in RESEARCH.md. |