6ab40c6f25
Phase 0 complete — v0.4 operator tier pre-execution artifacts: - PROJECT.md (v0.4 scope validated, D-050..D-057) - REQUIREMENTS.md (8 active REQs: REQ-MT-01/02, REQ-AUTH-01, REQ-DASH-01 + 4 NFRs) - ARCHITECTURE.md (operator Postgres + auth + dashboard + aggregation + VC migration) - PERSONAS.md (6 active personas — frontend + devops reactivated) - RESEARCH-v0.4-operator-tier.md (7 domains, 20 risks, confidence 0.70-0.95) - PLAN-v0.4-operator-tier.md (2 execution phases, 10 slices, 52 tasks, 8/8 REQ) - GRILL-v0.4.md (proceed-with-conditions, 6 MUST binding decisions) ---ci--- project: praxis phase: 0 milestone: v0.4 status: complete requirements: covered: [] partial: [] ---/ci---
488 lines
53 KiB
Markdown
488 lines
53 KiB
Markdown
# Praxis — Research Findings (v0.4 Operator Tier — Cohort Dashboard + Auth + Postgres)
|
||
|
||
> **Phase:** v0.4 research (operator tier)
|
||
> **Branch:** `phase/00-pre-execution`
|
||
> **Status:** research complete — pending orchestrator review
|
||
> **Date:** 2026-08-04
|
||
> **Method:** Codebase inspection (`server/`, `db/`, `docker-compose.yml`, `client/`, `pyproject.toml`, `client/package.json`), v0.3 research appendices (`.ciagent/RESEARCH.md`, `docs/RESEARCH-operator-postgres-auth.md`, `.ciagent/RESEARCH-vc.md`, `.ciagent/RESEARCH-v0.3-anonymization-irt-scenarios.md`), D-050..D-057 decision text, OWASP Password Storage Cheat Sheet (fetched 2026-08-04), Postgres 16 documentation, asyncpg/Starlette/argon2-cffi ecosystem knowledge. Web-verified where possible; domain-knowledge claims carry explicit confidence scores.
|
||
|
||
This document grounds the v0.4 operator-tier architecture in ecosystem evidence. It covers all 7 research domains and concludes with a consolidated risks table and a v0.3-assumption audit (which anticipatory assumptions were confirmed, which were overturned by D-050..D-057).
|
||
|
||
---
|
||
|
||
## Summary of Findings (Executive 1-Pager)
|
||
|
||
1. **Postgres 16-slim is the correct second service.** (0.90) `postgres:16-slim` (Debian-slim, glibc) matches the existing praxis Dockerfile rationale. Named volume `pgdata`, explicit `praxis-net` bridge network (no published port), `pg_isready` healthcheck, `depends_on: service_healthy`. PG16 ships `gen_random_uuid()` in core (no extension). asyncpg `create_pool(min_size=1, max_size=10)` on `app.state.pg_pool` via lifespan, `command_timeout=10`. CT memory bump 4GB→6GB (confirmed by v0.3 anticipatory section; D-050 fixes pool min at 1, not 2 — lower idle cost). Nightly `pg_dump -Fc` to `pgbackups` volume, `%u` 7-file rolling retention (D-055).
|
||
|
||
2. **Operator auth = signed stateless cookies (HMAC-SHA256 via Starlette SessionMiddleware) + argon2id + in-memory rate limit.** (0.88) D-056 overrides the v0.3 anticipatory "SessionMiddleware (itsdangerous-signed)" framing slightly — the architecture uses Starlette's SessionMiddleware which *is* itsdangerous-signed under the hood, so the v0.3 description holds. argon2-cffi `PasswordHasher` defaults (time_cost=3, memory_cost=64MiB, parallelism=4) **exceed** OWASP minimums (19MiB/t=2/p=1). `check_needs_rehash` for param upgrades. Login rate limit = in-memory `dict[ip, (count, window_start)]` dependency (D-057 + D-041); slowapi is the idiomatic FastAPI choice but a hand-rolled counter is simpler for single-instance and avoids a dep — **recommend slowapi for idiomaticity** (0.70) with the hand-rolled counter as the documented fallback.
|
||
|
||
3. **Secure cookie + no-TLS pilot tension → config-driven `Secure` flag, document the pilot risk.** (0.75) D-030 (no Traefik/TLS for pilot) conflicts with the `Secure` cookie attribute (requires HTTPS). Resolution: **(a) config-driven** — `PRAXIS_COOKIE_SECURE` env var (default `true`); set `false` only for the HTTP pilot, with a logged WARNING + a grill-tracked R-AUTH-01 mitigation. This is the safest minimal path: no new infra (Caddy/nginx would be a 3rd service), the flag flips automatically when TLS is added later. Reject option (c) minimal TLS via Caddy — adds a 3rd Docker service, breaks D-030's "direct bridge IP access" pilot stance, and TLS certs need a CA (self-signed → browser warnings worse than HTTP for a pilot). **The cohort dashboard reads only k-anonymized aggregates (D-034), so even a cookie sniffed over HTTP leaks no PII — defense in depth.**
|
||
|
||
4. **k-anonymity ≥ 10 enforced at write time via cell suppression in the aggregation SQL.** (0.85) `COUNT(DISTINCT learner_ref) >= 10` guard; cells below threshold are written with `cell_suppressed = TRUE` and `value = NULL`. 7-day rolling window computed on read via window functions over `cohort_aggregates` rows (incremental upsert by `(path, metric, window_start)`). No materialized view needed at v0.4 scale (<100 learners) — the nightly job recomputes all 7-day windows. Differencing attacks blocked by limiting to pre-defined 2-D views (path × week, path × outcome) per the v0.3 anonymization research.
|
||
|
||
5. **Aggregation trigger = async fire-and-forget `asyncio.Task` on session end + nightly reconciliation at 03:00 CT.** (0.82) D-054 confirms. The existing `SessionRecorder.end()` already schedules mastery flow via `asyncio.create_task` (line 143 of `session_recorder.py`) — the v0.4 aggregation hook follows the same pattern, chained after the mastery flow. Failures log + nightly job reconciles (idempotent upsert by window). Nightly job = in-process `asyncio.create_task` loop with `asyncio.sleep` until 03:00; no APScheduler (over-engineered for one cron job). If the service restarts, the in-flight task is lost but nightly reconciliation covers it.
|
||
|
||
6. **3 dashboard views = practice-volume, mastery-progression, failure-patterns — all k-anonymized, 7-day windows.** (0.82) D-053. Practice volume: sessions/day per path. Mastery progression: % learners at each week, gate-open rate. Failure patterns: top failure modes by frequency + rubric criterion weak-spots. Each view = a `/api/operator/<view>` endpoint returning pre-aggregated rows from `cohort_aggregates`; React renders read-only tables + sparkline charts. **No chart library is in `client/package.json`** — only react, react-dom, pipecat client SDK. Recommend **uPlot** (~40KB, sparkline-native, no React dependency) or inline SVG sparklines (~50 LOC, zero deps). Inline SVG is the v0.4 recommendation (zero deps, k-anon tables are small).
|
||
|
||
7. **VC issuer key migration = fresh keypair in Postgres `issuer_keys`; v0.3 SQLite public key archived as `superseded`.** (0.85) D-051. The existing `server/vc/issuer_keys.py` already implements the `active`/`superseded` lifecycle + `get_public_key_for_verification(key_id)`. v0.4 splits the issuer key store: Postgres `issuer_keys` (new active key) + archived v0.3 public key (status `superseded`). The verification endpoint (`server/vc/verification.py:verify_credential`) extracts `key_id` from the proof's `verificationMethod` and looks up the public key — the fallback to superseded keys is already implicit in `get_public_key_row(key_id)` (it queries by id, not by status). **No re-issuance of v0.3 VCs.** Private key encrypted at rest via `nacl.SecretBox` with `PRAXIS_VC_ISSUER_KEY` root key (existing pattern in `issuer_keys.py`).
|
||
|
||
8. **Operator account bootstrap = `scripts/create-operator.py` CLI, argon2id hash, idempotent insert.** (0.85) D-052. Reads `PRAXIS_BOOTSTRAP_OPERATOR_USER` + `PRAXIS_BOOTSTRAP_OPERATOR_PASS` from env, hashes with argon2-cffi, inserts into Postgres `operators` table with `ON CONFLICT (username) DO NOTHING`. Run from the host via `docker compose exec praxis python scripts/create-operator.py` or directly in the CT. No signup UI.
|
||
|
||
9. **Persona roster for v0.4: 6 active (lead-developer, backend-engineer, frontend-engineer REACTIVATED, data-engineer REACTIVATED/EXPANDED, security-engineer RETAINED, devops-engineer REACTIVATED for Postgres-in-LXC).** (0.90) v0.3 deactivated frontend-engineer + devops; v0.4 reactivates both. security-engineer retained (auth + crypto migration). data-engineer expands to Postgres schema + aggregation SQL. devops-engineer owns the docker-compose Postgres service + CT memory bump + backup cron + `create-operator.py` bootstrap script.
|
||
|
||
---
|
||
|
||
## Domain 1: Postgres 16 in Docker-in-LXC (D-040, D-050, D-055, REQ-NFR-MT-01)
|
||
|
||
### 1.1 Postgres 16-slim resource footprint inside an LXC CT
|
||
|
||
**Finding (0.88):** `postgres:16-slim` is Debian-slim-based (glibc), matching the praxis Dockerfile's rationale (avoiding Alpine musl locale issues with `pg_*` clients). The slim image is ~80MB compressed / ~200MB unpacked. Postgres 16 idle memory footprint with default `shared_buffers=128MB` is ~150-250MB RSS. With a small pilot workload (<100 learners, low-frequency operator queries), total Postgres RSS stays under ~400MB.
|
||
|
||
**Resource contention with the learner-facing praxis service:** The praxis container (uvicorn + Pipecat + voice loop) uses ~500MB at runtime (per v0.2 RESEARCH.md Q9). Postgres adds ~400MB. Docker daemon ~200MB. CT base ~200MB. Total ~1.3GB runtime, leaving ~4.7GB headroom on a 6GB CT. **The voice loop is latency-sensitive (C-8: <600ms); Postgres queries are off the voice path** (operator endpoints + nightly aggregation only). The risk is disk I/O contention during the nightly `pg_dump` + aggregation job — mitigated by scheduling at 03:00 CT (low learner activity) and the aggregation job being incremental upserts (not a full table scan).
|
||
|
||
**CT memory bump:** v0.3 anticipatory section said 4GB→6GB. D-050 fixes the asyncpg pool at min_size=1, max_size=10 (lower than the v0.3 anticipatory min_size=2). 6GB is confirmed sufficient. **Confidence 0.85** — the 6GB figure has ~50% margin.
|
||
|
||
### 1.2 docker-compose networking: internal bridge, service DNS, no external port
|
||
|
||
**Finding (0.92):** The current `docker-compose.yml` (verified — 49 lines, single `praxis` service, no explicit network → compose default bridge). v0.4 adds:
|
||
|
||
- An explicit named bridge network `praxis-net` (driver: bridge). **Not `internal: true`** — the postgres container doesn't need egress, but `internal: true` would also block DNS resolution from the praxis service. The simpler robust choice: named network, no `ports:` on postgres, no `internal: true`. The v0.3 research (`docs/RESEARCH-operator-postgres-auth.md` §1) confirmed this.
|
||
- The `praxis` service joins `praxis-net` and gains `depends_on: { postgres: { condition: service_healthy } }`.
|
||
- The `postgres` service joins `praxis-net`, no `ports:` mapping (not exposed to the LXC host bridge).
|
||
- Service DNS: the praxis service reaches postgres via the service name `postgres` (Docker Compose internal DNS). DSN: `postgresql://praxis:${PRAXIS_PG_PASSWORD}@postgres:5432/praxis` (D-050).
|
||
|
||
**Migration note:** Adding an explicit network to the existing `praxis` service means compose recreates the praxis container on `up` (the default bridge → named network is a recreate trigger). Plan a ~5-15s downtime window. The SQLite volume (`praxis-data`) is untouched → learner state preserved. **Confidence 0.90** — standard Docker Compose behavior.
|
||
|
||
### 1.3 Persistent volume strategy
|
||
|
||
**Finding (0.90):** Named volume `pgdata` (driver: local) on the LXC rootfs. **Never bind-mount `/var/lib/postgresql/data` to the CT filesystem** — Postgres requires `chown 999` and a specific directory layout; named volumes handle this. Set `PGDATA=/var/lib/postgresql/data/pgdata` to pin the subdirectory (survives image upgrades). A separate mount is not warranted for the pilot — the LXC rootfs (16GB) has headroom, and a named volume keeps the data with the compose stack.
|
||
|
||
**Backups:** second named volume `pgbackups` (driver: local). Nightly `pg_dump -Fc` (custom compressed format) → `/backups/praxis-$(date +%u).sql.gz` (D-055). `%u` = day-of-week 1-7 → rolling 7-file retention with zero cleanup logic. Operator can `pct pull` backups to the PVE host for off-CT safety. **Confidence 0.85** — `pg_dump -Fc` is the documented Postgres backup format; `%u` retention is a standard cron pattern.
|
||
|
||
### 1.4 asyncpg connection pooling
|
||
|
||
**Finding (0.88):** asyncpg `create_pool(min_size=1, max_size=10, command_timeout=10)` on `app.state.pg_pool` via FastAPI `lifespan` context manager. D-050 fixes min_size=1 (lower than the v0.3 anticipatory min_size=2 — reduces idle connection overhead). Pool created on startup, closed on shutdown. Operator endpoints are low-frequency (cohort dashboard, VC issuance); max_size=10 is generous for v0.4 single-instance. The `PraxisStore` (aiosqlite) keeps its current per-call connect pattern — **pools are independent and must not be shared** (different backends, different lifecycles). `command_timeout=10` prevents a slow operator query from blocking the event loop.
|
||
|
||
**Statement cache:** asyncpg caches prepared statements per connection by default. With a small schema (5 tables) and parameterized queries, the cache is small and effective. No explicit `statement_cache_size` config needed at v0.4 scale.
|
||
|
||
**Pip:** `asyncpg>=0.29` (new dep — confirmed not in `pyproject.toml`).
|
||
|
||
### 1.5 pg_dump backup strategy (D-055)
|
||
|
||
**Finding (0.85):** Cron job inside the praxis container (or a sidecar one-shot) runs nightly:
|
||
```
|
||
pg_dump -U praxis -Fc praxis | gzip > /backups/praxis-$(date +%u).sql.gz
|
||
```
|
||
Wait — `pg_dump -Fc` already produces a compressed custom format; piping through gzip is redundant. The correct command is:
|
||
```
|
||
pg_dump -U praxis -Fc praxis -f /backups/praxis-$(date +%u).dump
|
||
```
|
||
This produces a compressed custom-format dump that `pg_restore` can selectively restore. **Drill:** `pg_restore --clean --if-exists /backups/praxis_3.dump` (drop+recreate objects, safe against partial DB). Never restore into the live DB without stopping the praxis service first.
|
||
|
||
The backup job runs via the in-process asyncio scheduler (same as the aggregation reconciliation job) OR via a host-side cron that `docker compose exec`s the pg_dump. The in-process approach is simpler (one scheduler for both nightly jobs) but couples backup to the praxis service lifecycle. **Recommend host-side cron** → `docker compose exec -T postgres pg_dump ...` so backups run even if praxis is down. **Confidence 0.80** — host-side cron decouples backup from app uptime.
|
||
|
||
### 1.6 Healthcheck for the Postgres service
|
||
|
||
**Finding (0.95):** `pg_isready -U praxis -d praxis` every 10s, 5 retries, 5s timeout. `depends_on: { postgres: { condition: service_healthy } }` on the praxis service. **Caveat:** `pg_isready` returns healthy before the DB is fully ready for migration load — the praxis app must still retry the first migration attempt (the pg_migrate runner should be idempotent + retry on connection failure).
|
||
|
||
### 1.7 Postgres 16 features used
|
||
|
||
**Finding (0.90):**
|
||
- **`gen_random_uuid()`** — built into PG13+ core (no `pgcrypto` extension needed). Used as `DEFAULT gen_random_uuid()` for `operators.id`, `mastery_gate_events.id`, etc.
|
||
- **Partitioning for `cohort_aggregates`** — PG16 supports declarative partitioning by `RANGE (window_start)`. Weekly partitions (one per ISO week) keep the table small per partition + enable fast windowed queries. **However, at v0.4 scale (<100 learners, ~weeks of data), partitioning is premature optimization.** The v0.3 anticipatory section mentioned "weekly partitions" but D-053 clarifies the dashboard reads pre-aggregated rows — the `cohort_aggregates` table is small (one row per `(path, metric, window_start)`). **Recommendation: ship a plain table with an index on `(path, window_start)`; add partitioning only if the table exceeds ~100K rows** (post-pilot). **This overturns the v0.3 anticipatory "weekly partitions" assumption** — see §8 v0.3 audit.
|
||
|
||
---
|
||
|
||
## Domain 2: Operator Auth — argon2id + Signed Cookies (D-041, D-056, D-057, REQ-NFR-AUTH-01)
|
||
|
||
### 2.1 argon2id parameters for v0.4 scale
|
||
|
||
**Finding (0.92):** OWASP Password Storage Cheat Sheet (fetched 2026-08-04) recommends Argon2id with one of these minimum configurations:
|
||
- m=47104 (46 MiB), t=1, p=1
|
||
- m=19456 (19 MiB), t=2, p=1
|
||
- m=12288 (12 MiB), t=3, p=1
|
||
- m=9216 (9 MiB), t=4, p=1
|
||
- m=7168 (7 MiB), t=5, p=1
|
||
|
||
The `argon2-cffi` `PasswordHasher()` defaults are `time_cost=3, memory_cost=64MiB, parallelism=4` — **these exceed all OWASP minimums** (64MiB > 46MiB, t=3 matches the 12MiB/t=3 row, p=4 > p=1). The defaults are safe for a 6GB CT (64MiB per hash operation is trivial; login is low-frequency — one operator). **Recommendation: keep `PasswordHasher()` defaults.** Use `check_needs_rehash(stored_hash)` on login to rehash if params are bumped in the future. Benchmark login latency — if >1s, drop to `memory_cost=32MiB` (still exceeds OWASP minimums). **Confidence 0.92** — OWASP is the authoritative source; argon2-cffi defaults are documented.
|
||
|
||
### 2.2 Python argon2 library: argon2-cffi vs. passlib
|
||
|
||
**Finding (0.90):** **argon2-cffi** is the idiomatic choice for FastAPI. It's a thin CFFI wrapper around the reference Argon2 implementation, exposes `PasswordHasher` with argon2id as the default, and is actively maintained. `passlib` is a broader abstraction layer (supports multiple hash algorithms) but has had maintenance concerns (the 1.2 series hasn't seen a release in years; the 1.3 rewrite stalled). argon2-cffi is simpler, more focused, and the v0.3 research already chose it. **Pip: `argon2-cffi>=23.1`.** The v0.3 anticipatory architecture already lists `argon2-cffi` — confirmed.
|
||
|
||
### 2.3 Signed stateless cookies (HMAC-SHA256)
|
||
|
||
**Finding (0.88):** D-056 specifies "signed stateless cookies (HMAC-SHA256), no server-side session table." Starlette's `SessionMiddleware` uses `itsdangerous` under the hood, which signs the cookie with HMAC-SHA256 (via `TimestampedSigner`/`JSONWebSignature` depending on config). **The v0.3 anticipatory "SessionMiddleware (itsdangerous-signed)" framing is correct** — D-056's "HMAC-SHA256" is the underlying mechanism. The cookie is self-contained: `{operator_id, issued_at}` + HMAC signature. Verification = recompute HMAC + check expiry (8h). No `sessions` table in Postgres (D-056 explicit). Logout = client clears cookie (stateless — no server revocation list in v0.4).
|
||
|
||
**Key management:** `SECRET_KEY` from env (`PRAXIS_COOKIE_SECRET`, ≥32 bytes random). Rotation = change the key (invalidates all sessions — acceptable for a pilot). **Confidence 0.88** — Starlette SessionMiddleware is the documented FastAPI session pattern.
|
||
|
||
**Cookie attributes:**
|
||
- `session_cookie`: `"praxis_op"` (distinct from any future learner cookie)
|
||
- `max_age`: `28800` (8h, per D-041)
|
||
- `httponly`: `True` (middleware default; verify)
|
||
- `samesite`: `"strict"` (D-041 — CSRF defense-in-depth)
|
||
- `secure`: **config-driven** (see §2.4 below)
|
||
- `path`: `/` (or scope to `/api/operator` — cleaner, but the React `/operator/*` routes also need the cookie for the `/api/operator/me` call on mount; use `/`)
|
||
|
||
### 2.4 Secure cookie + no-TLS pilot tension (R-AUTH-01 resolution)
|
||
|
||
**Finding (0.75):** D-030 (no Traefik/TLS for pilot) conflicts with the `Secure` cookie attribute (browsers reject `Secure` cookies over HTTP, or rather: they don't send them over HTTP). The three options:
|
||
|
||
**(a) Config-driven `Secure` flag (RECOMMENDED):**
|
||
- `PRAXIS_COOKIE_SECURE` env var (default `true`).
|
||
- For the HTTP pilot: set `PRAXIS_COOKIE_SECURE=false`, log a WARNING, document the risk in GRILL-v0.4.md.
|
||
- When TLS is added later (post-v0.4), flip the env var → cookies become Secure automatically.
|
||
- **Defense in depth:** the cohort dashboard reads only k-anonymized aggregates (D-034) → even a cookie sniffed over HTTP leaks no PII. The VC issuance endpoints are auth-gated but the credentials themselves are public (verification endpoint is unauthenticated per D-043).
|
||
|
||
**(b) Accept the pilot risk + document:**
|
||
- Same as (a) but without the config flag — hardcode `secure=False` for v0.4.
|
||
- **Rejected:** inflexible — requires code change when TLS arrives.
|
||
|
||
**(c) Minimal TLS via Caddy/nginx sidecar:**
|
||
- Add a 3rd Docker service (Caddy reverse proxy) with a self-signed cert.
|
||
- **Rejected:** breaks D-030's "direct bridge IP access" pilot stance, adds a 3rd service + cert management, self-signed certs trigger browser warnings (worse UX than plain HTTP for a pilot). Defer to a later milestone.
|
||
|
||
**Verdict: option (a).** The config-driven flag is the safest minimal path — no new infra, automatic upgrade when TLS arrives, explicit risk documentation. **Confidence 0.75** — the resolution is sound but the pilot HTTP risk is real; the grill must sign off.
|
||
|
||
### 2.5 Login rate limiting (5 attempts/min)
|
||
|
||
**Finding (0.78):** D-041 specifies 5 attempts/min. Two implementations:
|
||
|
||
1. **slowapi** (`slowapi>=0.1`) — idiomatic FastAPI rate limiter. `@limiter.limit("5/minute")` on the login route. In-memory backend (per-process). **Caveat:** breaks if >1 praxis process (not a v0.4 concern — single uvicorn). Confidence 0.70 — young lib, but works.
|
||
|
||
2. **In-memory counter** — `dict[remote_ip, (count, window_start)]` in a FastAPI dependency. Zero deps, trivially auditable. For a single operator login endpoint, this is sufficient. Confidence 0.80 for the pilot.
|
||
|
||
**Recommendation: slowapi** for idiomaticity (decorator pattern, well-documented). The hand-rolled counter is the documented fallback if slowapi causes issues. Threshold: 5 failed attempts/minute/IP → 429 + `Retry-After` header. **Pip: `slowapi>=0.1`.** Rate limit is on the *login* route only (not the auth-gated routes — those check the cookie).
|
||
|
||
### 2.6 Session expiry (8h) + renewal strategy
|
||
|
||
**Finding (0.85):** `max_age=28800` (8h) on the cookie. No sliding renewal in v0.4 — the cookie expires 8h after issuance. The operator re-logs in after 8h. **Renewal is deferred** — a later milestone could implement sliding renewal (re-issue on activity) if 8h is too short for operator workflows. For v0.4 (single operator, low-frequency dashboard reads), 8h fixed is sufficient. **Confidence 0.85.**
|
||
|
||
---
|
||
|
||
## Domain 3: Cohort Aggregation — k-anonymity + 7-day windows (D-034, D-045, D-053, D-054)
|
||
|
||
### 3.1 k-anonymity ≥ 10 enforcement at write time
|
||
|
||
**Finding (0.85):** D-034 + REQ-NFR-DASH-01. The aggregation SQL enforces k≥10 via cell suppression at write time (not read time — auditable). Pattern:
|
||
|
||
```sql
|
||
-- Pseudo-SQL — shape only
|
||
INSERT INTO cohort_aggregates (path, metric, window_start, window_end, value, cell_count, cell_suppressed)
|
||
SELECT
|
||
path,
|
||
metric,
|
||
window_start,
|
||
window_end,
|
||
CASE WHEN COUNT(DISTINCT learner_ref) >= 10 THEN aggregate_value ELSE NULL END,
|
||
COUNT(DISTINCT learner_ref),
|
||
CASE WHEN COUNT(DISTINCT learner_ref) < 10 THEN TRUE ELSE FALSE END
|
||
FROM staging_sessions
|
||
GROUP BY path, metric, window_start, window_end
|
||
ON CONFLICT (path, metric, window_start) DO UPDATE SET
|
||
value = excluded.value,
|
||
cell_count = excluded.cell_count,
|
||
cell_suppressed = excluded.cell_suppressed,
|
||
updated_at = now();
|
||
```
|
||
|
||
- `cell_suppressed = TRUE` + `value = NULL` for cells < 10 learners.
|
||
- The dashboard renders suppressed cells as "— (suppressed, <10 learners)" — transparent to the operator.
|
||
- **Differencing attacks:** limited to pre-defined 2-D views (path × week, path × outcome) per the v0.3 anonymization research. No arbitrary filters (no per-learner drill-down — D-053 explicit).
|
||
|
||
**Confidence 0.85** — k-anonymity via `COUNT(DISTINCT) >= K` is the textbook suppression pattern.
|
||
|
||
### 3.2 7-day rolling window aggregation SQL
|
||
|
||
**Finding (0.82):** The `cohort_aggregates` table stores rows keyed by `(path, metric, window_start)`. Each row represents one 7-day window starting at `window_start`. The aggregation job (on-session-end hook + nightly) upserts by `(path, metric, window_start)` — idempotent. The 7-day window is a rolling construct: the nightly job recomputes the current window (the one containing "today") + the previous window (for continuity). On read, the dashboard queries `WHERE window_start >= now()::date - interval '7 days'` for the current view.
|
||
|
||
**Materialized view vs. incremental upsert:** Incremental upsert wins at v0.4 scale. A materialized view requires `REFRESH MATERIALIZED VIEW` (locks the view, slow at scale) and doesn't support partial refresh. Incremental upsert is cheap (one row per `(path, metric, window_start)`) + idempotent + supports the on-session-end hook pattern. **Confidence 0.82.**
|
||
|
||
### 3.3 On-session-end hook — async fire-and-forget (D-054)
|
||
|
||
**Finding (0.85):** D-054 confirms. The existing `SessionRecorder.end()` (line 142-145 of `session_recorder.py`) already schedules the mastery flow via `asyncio.create_task(self._run_mastery_flow_guarded(mastery_deps))`. The v0.4 aggregation hook follows the same pattern — chained after the mastery flow completes (or in parallel, since the aggregation only needs the session outcome + rubric scores, which the mastery flow produces). The hook:
|
||
|
||
1. Reads the session outcome + rubric scores from the mastery flow result (or directly from the SQLite `mastery_gate_events` table).
|
||
2. Computes the k-anonymized aggregate for the affected `(path, metric, window_start)` bin.
|
||
3. Upserts to Postgres `cohort_aggregates` (idempotent).
|
||
4. Failures log + the nightly job reconciles.
|
||
|
||
**Lifecycle:** `asyncio.Task` — non-blocking, the session-end response returns immediately. If the service restarts, the in-flight task is lost but nightly reconciliation covers it (D-054 explicit). **Confidence 0.85** — the pattern is already proven in the codebase.
|
||
|
||
### 3.4 Nightly reconciliation job
|
||
|
||
**Finding (0.80):** D-054 specifies 03:00 CT. Two implementation options:
|
||
|
||
1. **In-process asyncio scheduler** — `asyncio.create_task` loop with `asyncio.sleep` until 03:00 CT. No extra dep. If the service restarts, the scheduler resumes on startup (computes next 03:00). Simple, matches the "no Celery/Redis for v0.4" stance.
|
||
2. **APScheduler** — `apscheduler>=3.10` with a `AsyncIOScheduler`. More features (cron expressions, job stores) but over-engineered for one nightly job.
|
||
|
||
**Recommendation: in-process asyncio scheduler.** One `asyncio.create_task` that loops: compute seconds until next 03:00 CT → `asyncio.sleep(seconds)` → run reconciliation → repeat. The reconciliation job recomputes all 7-day windows for all paths (idempotent upsert). **Confidence 0.80** — simple, no dep, but no retry-on-failure (if the job fails, it retries the next night; the on-session-end hook keeps data fresh in the meantime).
|
||
|
||
### 3.5 Metrics for the 3 dashboard views (D-053)
|
||
|
||
**Finding (0.82):** D-053 names three views. Concrete metrics per view:
|
||
|
||
| View | Metrics (k-anonymized, 7-day windows) |
|
||
|------|---------------------------------------|
|
||
| **Practice volume** | sessions/day per path; total sessions in window; active learners in window (suppressed if <10) |
|
||
| **Mastery progression** | % learners at each week (1-6); gate-open rate (gate_opened / total_gate_events); median mastery_score; rubric criterion mean scores (per criterion, across path) |
|
||
| **Failure patterns** | top failure_modes by frequency; rubric criterion weak-spots (criteria with mean < 3.0); branch outcome distribution (escalate vs accept) |
|
||
|
||
Each metric is a row in `cohort_aggregates` with `(path, metric, window_start, window_end, value, cell_count, cell_suppressed)`. The `/api/operator/<view>` endpoint returns the pre-aggregated rows for that view's metrics.
|
||
|
||
### 3.6 No raw learner PII in Postgres (D-031 hybrid)
|
||
|
||
**Finding (0.90):** D-031 hybrid — learner-local state stays SQLite; operator-tier Postgres stores only aggregations + operator accounts + issued credentials. **What identifies a learner?** `learner_ref` — an opaque string (e.g., `"learner-1"`, the existing `HARDCODED_LEARNER_ID`). The Postgres tables (`cohort_aggregates`, `mastery_gate_events`, `issued_credentials`) use `learner_ref` as the join handle — **never** a FK to SQLite (cross-DB joins are impossible). The existing `issued_credentials` table in SQLite uses `learner_id` (the hardcoded string); v0.4's Postgres `issued_credentials` table uses `learner_ref` (same opaque string, different column name to emphasize it's not a FK). **Confidence 0.90** — D-031 is explicit; the codebase already uses opaque string IDs.
|
||
|
||
---
|
||
|
||
## Domain 4: React Cohort Dashboard (D-044, D-053, REQ-DASH-01)
|
||
|
||
### 4.1 React route under `/operator/*`
|
||
|
||
**Finding (0.85):** D-044. The existing client (`client/src/App.tsx`) is a single-view state machine (start → live → debrief) with **no React Router**. v0.4 adds:
|
||
|
||
- A new `client/src/operator/` directory with the cohort dashboard components.
|
||
- React Router (or a minimal route switch) for `/operator/*` routes: `/operator/login`, `/operator/dashboard`.
|
||
- The existing `App.tsx` remains the voice session UI at `/`.
|
||
|
||
**Routing structure:** The current `App.tsx` is mounted at `/` by the StaticFiles serving. Adding `/operator/*` routes requires either:
|
||
- **(a) React Router** — `npm install react-router-dom` + a `<BrowserRouter>` wrapper. The StaticFiles `html=True` mount serves `index.html` for all paths, React Router handles client-side routing. **Caveat:** the existing `App.tsx` doesn't use React Router; wrapping it requires a refactor (or a separate root).
|
||
- **(b) Minimal route switch** — `useState<'voice' | 'operator'>` based on `window.location.pathname.startsWith('/operator')`. No new dep. Simpler, but less idiomatic for a growing dashboard.
|
||
|
||
**Recommendation: React Router** (`react-router-dom@^7`) — it's the standard, supports nested routes, and the v0.4 dashboard will grow (D-053 names 3 views). The refactor to wrap `App.tsx` in a `<BrowserRouter>` is small. Add a catch-all route that serves the voice UI at `/` and the operator UI at `/operator/*`. **Pip: none; npm: `react-router-dom`.** **Confidence 0.80** — React Router is standard but adds a dep + a refactor of the existing single-view App.
|
||
|
||
**SPA fallback:** With React Router, the FastAPI StaticFiles mount needs to serve `index.html` for all non-API paths (SPA fallback). The current `app.mount("/", StaticFiles(directory=_CLIENT_DIST, html=True))` serves `index.html` for `/` but returns 404 for `/operator/dashboard` (no such file). **This is a required change:** add a catch-all route before the StaticFiles mount that returns `FileResponse("client/dist/index.html")` for any path not matching an API route. The v0.2 RESEARCH.md Q3 noted this as "NOT needed for v0.2" — v0.4 needs it. **Confidence 0.90** — standard SPA serving pattern.
|
||
|
||
### 4.2 Reusing v0.2 StaticFiles (same `client/dist` build)
|
||
|
||
**Finding (0.90):** D-044 explicit. No separate SPA build — the same `npm run build` produces `client/dist` with both the voice UI and the operator dashboard. The Dockerfile's Node stage is unchanged (one `npm run build`). The FastAPI StaticFiles mount is updated to serve the SPA fallback (see §4.1). **Confidence 0.90.**
|
||
|
||
### 4.3 Read-only tables + sparkline charts
|
||
|
||
**Finding (0.80):** The dashboard renders read-only tables + sparkline charts. **No chart library is in `client/package.json`** (verified — only react, react-dom, pipecat client SDK, dev deps). Options:
|
||
|
||
1. **Inline SVG sparklines** (~50 LOC, zero deps) — a `<Sparkline data={...} />` component that renders an SVG polyline. Sufficient for k-anon tables (small data: one sparkline per row, ~7-30 data points). **Recommendation for v0.4.**
|
||
2. **uPlot** (~40KB, sparkline-native, no React dependency) — high-performance, but overkill for small tables.
|
||
3. **Recharts** (~100KB, React-native) — idiomatic but heavy for sparklines.
|
||
4. **Chart.js + react-chartjs-2** (~200KB) — heaviest, overkill.
|
||
|
||
**Recommendation: inline SVG sparklines** (zero deps, ~50 LOC, sufficient for v0.4 scale). Add a chart library only if the dashboard grows to need axes, tooltips, zoom. **Confidence 0.80** — sparklines are simple; the inline SVG approach is well-documented.
|
||
|
||
### 4.4 `/api/operator/*` FastAPI endpoint structure
|
||
|
||
**Finding (0.88):** D-053 + D-057. New `server/operator/` module with an `APIRouter(prefix="/api/operator")`. Endpoints:
|
||
|
||
| Endpoint | Method | Auth | Purpose |
|
||
|----------|--------|------|---------|
|
||
| `/api/operator/login` | POST | rate-limited (5/min) | Login: validate argon2id, set signed cookie |
|
||
| `/api/operator/logout` | POST | auth-gated | Clear cookie (client-side) |
|
||
| `/api/operator/me` | GET | auth-gated | Return current operator (for React route guard) |
|
||
| `/api/operator/cohort` | GET | auth-gated | Practice volume view (k-anonymized) |
|
||
| `/api/operator/mastery` | GET | auth-gated | Mastery progression view (k-anonymized) |
|
||
| `/api/operator/failure-patterns` | GET | auth-gated | Failure patterns view (k-anonymized) |
|
||
| `/api/operator/credentials` | GET | auth-gated | List issued VCs (operator's issuance log) |
|
||
| `/api/operator/credentials/{id}/revoke` | POST | auth-gated | Revoke a VC |
|
||
|
||
Auth enforcement: FastAPI middleware checks the signed cookie on every `/api/operator/*` request (D-057); 401 if missing/invalid/expired. Router-level `dependencies=[Depends(current_operator)]` on the protected routes. Login + logout are outside the protected router (login is rate-limited, not auth-gated). **Confidence 0.88.**
|
||
|
||
### 4.5 Freshness ≤ 24h (REQ-NFR-DASH-02)
|
||
|
||
**Finding (0.85):** The on-session-end hook keeps aggregates fresh within minutes of a session ending. The nightly reconciliation job (03:00 CT) guarantees all 7-day windows are recomputed at least once/day. **Max staleness = 24h** (if the service restarts after a session and before the nightly job, the aggregate is stale until the next 03:00 run). The dashboard surfaces "last updated" via a `updated_at` timestamp on each `cohort_aggregates` row → the `/api/operator/<view>` response includes `last_updated: max(updated_at)` across the returned rows. React renders "Last updated: Xh ago" in the dashboard header. **Confidence 0.85.**
|
||
|
||
---
|
||
|
||
## Domain 5: VC Issuer Key Migration (D-042, D-051)
|
||
|
||
### 5.1 Migrating the Ed25519 issuer key from SQLite to Postgres
|
||
|
||
**Finding (0.88):** D-051. The existing `server/vc/issuer_keys.py` (verified — 128 lines) implements the issuer key lifecycle:
|
||
- `init_issuer_key(store, root_key)` — generates a fresh Ed25519 keypair, encrypts the private key with `nacl.SecretBox` (root key from `PRAXIS_VC_ISSUER_KEY` env), stores in the `issuer_keys` table.
|
||
- `get_active_signing_key(store, root_key)` — returns the active key (status='active'), or generates one if none exists.
|
||
- `get_public_key_for_verification(store, key_id)` — returns the public key for a given key_id (queries by id, not by status — **this is the fallback mechanism**).
|
||
- `rotate_key(store, root_key)` — generates a new key, marks the old as `superseded`.
|
||
- `_verification_method(key_id)` — builds the `verificationMethod` URL.
|
||
|
||
v0.4 migration:
|
||
1. The `issuer_keys.py` functions currently take a `PraxisStore` (SQLite). v0.4 adds a `PgStore` (Postgres) and the issuer key functions are refactored to accept either store (or a dedicated `IssuerKeyStore` interface). **The simplest refactor:** the issuer key functions accept a protocol/ABC with `init_issuer_key`, `get_active_signing_key_row`, `get_public_key_row`, `set_issuer_key_superseded` methods — both `PraxisStore` (SQLite) and `PgStore` (Postgres) implement it.
|
||
2. On first v0.4 boot: generate a fresh keypair in Postgres `issuer_keys` (status='active').
|
||
3. **Archive the v0.3 public key** — read the v0.3 active key's public key from SQLite, insert it into Postgres `issuer_keys` with status='superseded'. The private key is NOT migrated (v0.3 VCs are already signed; verification only needs the public key).
|
||
4. The verification endpoint (`server/vc/verification.py:verify_credential`) extracts `key_id` from the proof's `verificationMethod` and calls `get_public_key_for_verification(store, key_id)`. **The fallback to superseded keys is already implicit** — `get_public_key_row(key_id)` queries by id, not by status. v0.3 VCs have the v0.3 key_id in their proof → the lookup finds the archived (superseded) public key → signature verifies.
|
||
|
||
**Confidence 0.88** — the existing code already supports the lifecycle; the migration is a store swap + an archive insert.
|
||
|
||
### 5.2 Archiving the v0.3 public key as `superseded` (not revoked)
|
||
|
||
**Finding (0.90):** D-051 explicit. The v0.3 public key is archived as `superseded` — old VCs still verify against it. **Revoked** would imply the key is no longer trusted (old VCs should fail verification). **Superseded** means the key is no longer used for new signatures but old signatures remain valid. The existing `set_issuer_key_superseded(key_id)` method (line 348-354 of `store.py`) does exactly this. **Confidence 0.90.**
|
||
|
||
### 5.3 Verification endpoint fallback
|
||
|
||
**Finding (0.88):** The verification flow (`server/vc/verification.py`):
|
||
1. `verify_credential(store, credential_id)` → fetches the credential row.
|
||
2. `extract_key_id(secured_doc)` → extracts key_id from the proof's `verificationMethod` URL.
|
||
3. `get_public_key_for_verification(store, key_id)` → fetches the public key by id.
|
||
4. `verify_proof(secured_doc, verify_key)` → validates the Ed25519 signature.
|
||
|
||
The fallback is implicit: step 3 queries by `key_id` (not by status), so it finds both active and superseded keys. v0.3 VCs have v0.3 key_ids → step 3 finds the archived (superseded) public key → step 4 validates. **No code change needed in the verification flow** — only the store backing changes (SQLite → Postgres). **Confidence 0.88.**
|
||
|
||
### 5.4 Encrypted-at-rest private key in Postgres
|
||
|
||
**Finding (0.85):** The existing `_encrypt_private_key(signing_key, root_key)` uses `nacl.SecretBox` with a root key from `PRAXIS_VC_ISSUER_KEY` env. This is application-layer encryption — the private key is encrypted before being stored in the DB. The same pattern works for Postgres (the `private_key_enc` column is `BYTEA`). **Postgres-level encryption at rest** (TDE) is not available in the open-source Postgres 16 (that's an EnterpriseDB feature). The application-layer `nacl.SecretBox` is the correct approach for the pilot. The root key (`PRAXIS_VC_ISSUER_KEY`) is in `.env.secrets` (gitignored). **Confidence 0.85** — the pattern is already proven in v0.3; the store swap is mechanical.
|
||
|
||
---
|
||
|
||
## Domain 6: Operator Account Bootstrap (D-052)
|
||
|
||
### 6.1 `scripts/create-operator.py` CLI script
|
||
|
||
**Finding (0.88):** D-052. A new `scripts/create-operator.py` script:
|
||
- Reads `PRAXIS_BOOTSTRAP_OPERATOR_USER` + `PRAXIS_BOOTSTRAP_OPERATOR_PASS` from env (in `.env.secrets`).
|
||
- Hashes the password with `argon2-cffi` `PasswordHasher().hash(password)`.
|
||
- Connects to Postgres via asyncpg.
|
||
- Inserts into `operators` table: `INSERT INTO operators (username, password_hash, display_name) VALUES ($1, $2, $3) ON CONFLICT (username) DO NOTHING`.
|
||
- Idempotent — no-op if the user exists (no password update on re-run; a separate `--update` flag could force a rehash if needed).
|
||
- Prints the result: `created` or `already exists`.
|
||
|
||
**Running the script:** `docker compose exec praxis python scripts/create-operator.py` (from the host) or directly in the CT. The script reads env vars from the praxis container's environment (which sources `/etc/praxis/server.env`). **Confidence 0.88.**
|
||
|
||
### 6.2 Env vars in `.env.secrets`
|
||
|
||
**Finding (0.90):** D-052. `PRAXIS_BOOTSTRAP_OPERATOR_USER` + `PRAXIS_BOOTSTRAP_OPERATOR_PASS` added to `.ciagent/.env.secrets` (gitignored — verified in `.gitignore`). These are injected via `lxc.environment` → `/etc/praxis/server.env` → `docker-compose.yml` env_file → container env. The `config.json` secrets scopes need a new `operator` scope with these vars. **Confidence 0.90** — the secret injection chain is proven from v0.2.
|
||
|
||
---
|
||
|
||
## Domain 7: Persona Assessment (v0.4 roster)
|
||
|
||
### 7.1 Active personas for v0.4
|
||
|
||
**Finding (0.90):** v0.4 is **operator-tier-backend + dashboard-frontend + security-crypto + Postgres-in-LXC**. The roster:
|
||
|
||
| Persona | v0.3 status | v0.4 status | Reason |
|
||
|----------|-------------|-------------|--------|
|
||
| lead-developer | active | **active** | Coordinates across operator/auth/cohort/dashboard/Postgres domains. Owns docker-compose.yml Postgres service addition. |
|
||
| backend-engineer | active | **active** | Owns the asyncpg pool wiring, operator API routes, aggregation pipeline (on-session-end hook + nightly job), session_recorder.py extension for the aggregation hook. |
|
||
| frontend-engineer | active (reactivated v0.3) | **active** | Owns the React cohort dashboard UI (D-044). Auth-gated routes, k-anonymized tables, sparkline charts. React Router addition + SPA fallback. |
|
||
| data-engineer | active | **active (expanded)** | Owns the Postgres operator-tier schema (operators, cohort_aggregates, issuer_keys, mastery_gate_events, issued_credentials), the pg_migrate runner, the k-anonymity suppression SQL. |
|
||
| security-engineer | active (new v0.3) | **active (retained)** | Owns the VC issuer key migration (SQLite→Postgres, superseded archive), the auth stack (argon2id, signed cookies, rate limiting), the Secure-cookie-TLS resolution (R-AUTH-01). |
|
||
| devops-engineer | deactivated (v0.3) | **active (reactivated)** | Owns the docker-compose Postgres service + CT memory bump (4GB→6GB) + backup cron + `create-operator.py` bootstrap script + `.env.example` operator vars. |
|
||
|
||
### 7.2 Deactivated personas
|
||
|
||
None deactivated for v0.4 — all 6 personas are active. The voice-engineer and ml-engineer remain proposed (not v0.4).
|
||
|
||
### 7.3 Framework alignment (from actual `pyproject.toml` + `client/package.json`)
|
||
|
||
| Persona | Frameworks (v0.4 research-aligned) | Source |
|
||
|---------|-------------------------------------|--------|
|
||
| lead-developer | pipecat, fastapi, postgres, docker | `pyproject.toml` + `docker-compose.yml` |
|
||
| backend-engineer | pipecat, pydantic, fastapi, uvicorn, asyncpg, aiosqlite | `pyproject.toml` (asyncpg is NEW for v0.4) |
|
||
| frontend-engineer | react, react-router-dom (NEW), pipecat-client-sdk, webrtc, vite, fastapi-staticfiles | `client/package.json` (react-router-dom is NEW for v0.4) |
|
||
| data-engineer | sqlite, postgres16, aiosqlite, asyncpg, alembic-style-migrations | `pyproject.toml` + `db/migrate.py` pattern |
|
||
| security-engineer | pynacl, canonicaljson, base58, argon2-cffi, starlette-sessionmiddleware, slowapi | `pyproject.toml` (argon2-cffi + slowapi are NEW for v0.4) |
|
||
| devops-engineer | proxmox-ve-api, lxc, docker, systemd, bash, bats, gitea, pg_dump | `scripts/proxmox/` + `docker-compose.yml` |
|
||
|
||
### 7.4 Territory alignment (from actual `server/` structure)
|
||
|
||
The actual `server/` structure (verified): `asr/`, `tts/`, `llm/`, `guardrails/`, `scenarios/`, `mastery/`, `paths/`, `vc/`, `services/`, `pipeline.py`, `session_recorder.py`, `__main__.py`, `cost.py`, `debrief.py`, `latency.py`, `interruptibility.py`. v0.4 adds: `server/operator/` (operator API), `server/auth/` (auth middleware), `server/cohort/` (aggregation pipeline). New `db/pg_migrations/` (Postgres migrations) + `db/pg_schema.sql` + `db/pg_store.py` (Postgres store).
|
||
|
||
| Persona | Territory (v0.4) |
|
||
|---------|-------------------|
|
||
| lead-developer | `docker-compose.yml`, `.env.example` |
|
||
| backend-engineer | `**/server/**`, `**/operator/**`, `**/cohort/**`, `**/db/**` (excluding pg_schema) |
|
||
| frontend-engineer | `**/client/**`, `**/client/src/operator/**` |
|
||
| data-engineer | `**/db/**`, `**/db/pg_migrations/**`, `**/db/pg_schema.sql`, `**/db/pg_store.py` |
|
||
| security-engineer | `**/server/vc/**`, `**/server/auth/**` |
|
||
| devops-engineer | `scripts/proxmox/**`, `scripts/install-service.sh`, `scripts/create-operator.py`, `.env.example` (operator vars) |
|
||
|
||
### 7.5 Constraint alignment (v0.4-specific)
|
||
|
||
- **All personas:** `hybrid-storage-no-cross-db-joins` (D-031), `k-anonymity-floor-10` (D-034), `no-raw-learner-pii-in-postgres` (D-031).
|
||
- **backend-engineer:** `mastery-off-voice-path` (C-8), `aggregation-off-voice-path` (D-054 — async fire-and-forget), `deterministic-scoring` (v0.3 carry-forward).
|
||
- **frontend-engineer:** `auth-gated-operator-routes` (D-057), `k-anonymity-display-suppressed-cells` (D-034), `no-raw-learner-pii-in-ui` (D-031), `spa-fallback-for-operator-routes` (new — React Router needs index.html fallback).
|
||
- **data-engineer:** `no-cross-db-joins` (D-031), `opaque-learner-ref` (D-031), `write-time-suppression` (D-034).
|
||
- **security-engineer:** `argon2id-passwords` (D-041), `config-driven-secure-cookie` (R-AUTH-01 resolution), `issuer-key-encrypted-at-rest` (D-042), `superseded-not-revoked` (D-051).
|
||
- **devops-engineer:** `idempotent-deploy` (carry-forward), `secrets-never-committed` (carry-forward), `pg-dump-backup-retention-7d` (D-055).
|
||
|
||
---
|
||
|
||
## Consolidated Risks Table
|
||
|
||
| ID | Risk | Severity | Mitigation | Confidence |
|
||
|----|------|----------|------------|------------|
|
||
| **R-MT-01** | Postgres + praxis resource contention on 6GB CT (disk I/O during nightly pg_dump + aggregation) | medium | Schedule nightly jobs at 03:00 CT (low learner activity); aggregation is incremental upsert (not full scan); monitor CT memory; bump to 8GB if OOM | 0.75 |
|
||
| **R-MT-02** | Postgres container unhealthy on boot → praxis `depends_on` blocks startup | medium | `pg_isready` healthcheck + 5 retries; praxis app retries first migration on connection failure; `depends_on: service_healthy` is necessary but not sufficient | 0.80 |
|
||
| **R-MT-03** | Docker Compose network change (default bridge → praxis-net) recreates praxis container → ~5-15s downtime | low | Plan cutover window; SQLite volume untouched → learner state preserved; do on staging CT first | 0.85 |
|
||
| **R-MT-04** | `pgdata` volume corruption on CT restart (LXC + Docker volume interaction) | low | Named volumes are stable on Docker-in-LXC with nesting=1; nightly pg_dump provides backup; `pg_restore --clean --if-exists` drill | 0.70 |
|
||
| **R-MT-05** | Postgres 16 `gen_random_uuid()` not available (misremembered as PG13+) | low | Verified: `gen_random_uuid()` is built into PG13+ core (no extension). PG16 confirmed. | 0.95 |
|
||
| **R-AUTH-01** | Secure cookie flag + no-TLS pilot → cookies sent over HTTP (sniffable) | medium | Config-driven `PRAXIS_COOKIE_SECURE` (default true; false for HTTP pilot with logged WARNING); cohort dashboard reads only k-anonymized aggregates (no PII leak even if cookie sniffed); grill must sign off | 0.75 |
|
||
| **R-AUTH-02** | argon2id hashing blocks event loop (CPU-bound, ~30-80ms per login) | low | Single operator login is low-frequency; ~80ms is acceptable on the event loop. If batch-hashing needed, use `run_in_executor`. Not a v0.4 concern. | 0.85 |
|
||
| **R-AUTH-03** | In-memory rate limit lost on service restart (attacker bypasses by timing restart) | low | Single-instance pilot; restarts are rare + operator-initiated. A persistent rate-limit store (Redis) is deferred. | 0.80 |
|
||
| **R-AUTH-04** | Signed cookie secret (`PRAXIS_COOKIE_SECRET`) rotation invalidates all sessions | low | Pilot: acceptable (one operator re-logs in). Document the rotation procedure. | 0.85 |
|
||
| **R-AUTH-05** | No server-side session revocation (logout is client-side only) | low | D-056 explicit: stateless cookies, no revocation list in v0.4. A forced-logout requires cookie secret rotation. Deferred to a later milestone. | 0.80 |
|
||
| **R-DASH-01** | k-anonymity suppression hides meaningful data at v0.4 scale (<100 learners → many cells <10) | medium | Expected at pilot scale; dashboard shows "— (suppressed, <10 learners)" transparently. Aggregation window can be widened (14-day) if too many cells suppressed. | 0.75 |
|
||
| **R-DASH-02** | Differencing attack: operator compares two 7-day windows to isolate a single learner | medium | Limit to pre-defined 2-D views (path × week, path × outcome); no arbitrary filters; no per-learner drill-down (D-053). | 0.70 |
|
||
| **R-DASH-03** | SPA fallback breaks existing voice UI (StaticFiles mount change) | medium | Add catch-all route BEFORE StaticFiles mount; test `/` still serves voice UI; test `/operator/dashboard` serves index.html. | 0.80 |
|
||
| **R-DASH-04** | Nightly reconciliation job fails → aggregates stale >24h (NFR-DASH-02 breach) | low | On-session-end hook keeps data fresh; job retries next night; log + alert on job failure. | 0.75 |
|
||
| **R-DASH-05** | React Router addition requires App.tsx refactor → breaks voice UI | medium | Wrap App.tsx in `<BrowserRouter>` with a catch-all route; test voice UI at `/` unchanged. | 0.75 |
|
||
| **R-VC-MIG-01** | VC issuer key migration loses v0.3 public key → old VCs fail verification | high | Archive v0.3 public key as `superseded` in Postgres `issuer_keys` before activating new key; verification endpoint queries by key_id (not status) → fallback is implicit. Test: verify a v0.3 VC against the migrated store. | 0.85 |
|
||
| **R-VC-MIG-02** | `PRAXIS_VC_ISSUER_KEY` root key changes between v0.3 and v0.4 → encrypted private keys undecryptable | medium | The v0.3 private key is NOT migrated (only the public key is archived). The v0.4 active key is generated fresh with the v0.4 root key. Keep the v0.3 root key in secrets until all v0.3 VCs expire (3-year validUntil). | 0.80 |
|
||
| **R-VC-MIG-03** | `issuer_keys.py` store refactor (SQLite→Postgres protocol) breaks v0.3 verification | medium | Define an `IssuerKeyStore` protocol/ABC; both `PraxisStore` and `PgStore` implement it; verification endpoint uses the Postgres store for v0.4. Test: verify a v0.3 VC against the Postgres store with the archived public key. | 0.80 |
|
||
| **R-BOOT-01** | `create-operator.py` fails on first boot (Postgres not ready) | low | Script retries on connection failure (3 attempts, 5s backoff); run after `docker compose up -d postgres` + healthcheck passes. | 0.80 |
|
||
| **R-BOOT-02** | `PRAXIS_BOOTSTRAP_OPERATOR_PASS` not set → operator can't log in | low | Script checks env var presence + exits with clear error if missing. Document in `.env.example`. | 0.85 |
|
||
|
||
---
|
||
|
||
## v0.3 Assumption Audit (which anticipatory assumptions were confirmed / overturned)
|
||
|
||
The v0.3 ARCHITECTURE.md operator-tier section was anticipatory. D-050..D-057 (v0.4 clarify decisions) refine it. Audit:
|
||
|
||
| v0.3 anticipatory assumption | v0.4 decision | Verdict |
|
||
|------------------------------|---------------|---------|
|
||
| `postgres:16-slim`, named volume `pgdata`, internal network, `pg_isready` healthcheck | D-040, D-050 confirmed | **CONFIRMED** |
|
||
| asyncpg `create_pool(min_size=2, max_size=10)` | D-050: `min_size=1` | **OVERTURNED** — D-050 lowers min_size to 1 (lower idle cost) |
|
||
| Starlette `SessionMiddleware` (itsdangerous-signed) | D-056: signed stateless cookies (HMAC-SHA256) | **CONFIRMED** — SessionMiddleware uses itsdangerous/HMAC-SHA256 under the hood; D-056 is the mechanism clarification |
|
||
| argon2-cffi `PasswordHasher` defaults | D-041 + OWASP: defaults exceed minimums | **CONFIRMED** — keep defaults (time_cost=3, memory_cost=64MiB, parallelism=4) |
|
||
| slowapi 5/min login rate-limit | D-041 + D-057 | **CONFIRMED** — slowapi is the idiomatic choice; in-memory counter is the fallback |
|
||
| `cohort_aggregates` with weekly partitions | D-053: pre-aggregated rows, 7-day rolling windows | **OVERTURNED** — weekly partitions are premature at v0.4 scale; ship a plain table with `(path, window_start)` index. Add partitioning post-pilot. |
|
||
| `operators`, `issued_credentials`, `mastery_gate_events`, `cohort_aggregates`, `issuer_keys` tables | D-050..D-053 confirmed | **CONFIRMED** — schema holds; column names refined (learner_ref vs learner_id) |
|
||
| CT memory 4GB → 6GB | D-050 + REQ-NFR-MT-01 | **CONFIRMED** — 6GB is sufficient |
|
||
| `pg_dump -Fc` to `pgbackups` volume, `%u` 7-file retention | D-055 confirmed | **CONFIRMED** — but host-side cron (not in-process) for decoupling |
|
||
| Secure cookie requires TLS (R-AUTH-01) | D-056 + D-030: config-driven `Secure` flag | **REFINED** — config-driven flag is the v0.4 resolution; v0.3 flagged it as an open question |
|
||
| VC issuer key in Postgres `issuer_keys` (encrypted at rest) | D-042 + D-051 confirmed | **CONFIRMED** — plus the migration path (archive v0.3 public key as superseded) |
|
||
| `gen_random_uuid()` in PG16 (no extension) | Verified | **CONFIRMED** |
|
||
| React `/operator/*` route, reuses v0.2 StaticFiles | D-044 + D-053 confirmed | **CONFIRMED** — plus SPA fallback requirement (new) |
|
||
| on-session-end hook + nightly reconciliation | D-045 + D-054 confirmed | **CONFIRMED** — D-054 clarifies async fire-and-forget + 03:00 CT |
|
||
|
||
**Summary:** 2 overturned (asyncpg min_size, weekly partitions), 1 refined (Secure cookie → config-driven), 11 confirmed.
|
||
|
||
---
|
||
|
||
## New pip dependencies for v0.4
|
||
|
||
| Dep | Purpose | Confidence | Source |
|
||
|-----|---------|------------|--------|
|
||
| `asyncpg>=0.29` | Postgres async driver / pool | 0.90 | D-050 |
|
||
| `argon2-cffi>=23.1` | argon2id password hashing | 0.95 | D-041, OWASP |
|
||
| `slowapi>=0.1` | login rate limiting (in-memory) | 0.70 | D-041, D-057 |
|
||
|
||
`starlette` + `itsdangerous` already via FastAPI. `pynacl`, `canonicaljson`, `base58` already in `pyproject.toml` (v0.3).
|
||
|
||
## New npm dependencies for v0.4
|
||
|
||
| Dep | Purpose | Confidence | Source |
|
||
|-----|---------|------------|--------|
|
||
| `react-router-dom@^7` | React routing for `/operator/*` | 0.80 | D-044 |
|
||
|
||
No chart library — inline SVG sparklines (zero deps).
|
||
|
||
---
|
||
|
||
## Open Questions for PLAN Stage
|
||
|
||
1. **SPA fallback implementation:** Catch-all route before StaticFiles mount, or a custom StaticFiles subclass? The catch-all route is simpler but must not shadow `/api/*` or `/vc/*` routes.
|
||
2. **`IssuerKeyStore` protocol design:** ABC with methods, or a simpler duck-typing approach? The existing `PraxisStore` methods (`init_issuer_key`, `get_active_signing_key_row`, `get_public_key_row`, `set_issuer_key_superseded`) are the interface.
|
||
3. **Nightly scheduler:** In-process asyncio loop or host-side cron for the aggregation job? (pg_dump backup is host-side cron.) In-process is simpler for aggregation (shares the asyncpg pool); host-side is better for backup (decoupled from app uptime).
|
||
4. **`create-operator.py` update path:** `--update` flag to force rehash, or a separate `scripts/update-operator.py`? Keep it simple: `--update` flag on the same script.
|
||
5. **Cookie `path` scope:** `/` (cookie sent to all routes) or `/api/operator` (cookie sent only to operator API)? `/` is needed for the React `/operator/*` routes to call `/api/operator/me` on mount (the browser sends the cookie). Use `/`.
|
||
6. **Cohort aggregation `learner_ref` source:** The existing `HARDCODED_LEARNER_ID = "learner-1"` — is this stable enough for the aggregation? Yes for v0.4 (single learner); multi-learner-per-device is deferred. The aggregation groups by `learner_ref` so k-anonymity counts distinct learners.
|
||
7. **Phase split confirmation:** ROADMAP shows P1 (operator foundation: Postgres + auth) → P2 (cohort dashboard + aggregation) → P3 (review). Is the aggregation pipeline P1 or P2? D-045 + D-054 suggest the hook is P2 (needs the dashboard to be useful), but the Postgres schema + the on-session-end hook could be P1. **Recommendation:** P1 = Postgres + auth + VC key migration + schema (including `cohort_aggregates` table); P2 = aggregation pipeline (hook + nightly job) + dashboard UI + endpoints. The schema is P1 so P2 is pure code. |