Compare commits
45 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c9709e0f38 | |||
| 62bff575af | |||
| 8a43e95a4d | |||
| ee5be13c94 | |||
| 1da814e790 | |||
| cc7ec5d03f | |||
| 5c2829d9df | |||
| ec4648b37a | |||
| c5369b407b | |||
| 282f5ef150 | |||
| 3010bc4b96 | |||
| 35c4c386b5 | |||
| d8d2cebfc5 | |||
| b2a2a4023b | |||
| de431852c4 | |||
| 64e4842976 | |||
| a64733a262 | |||
| 0072689e4d | |||
| 3110be2f15 | |||
| bd5b0fee95 | |||
| 2e6d92dfa1 | |||
| e3c8cc7145 | |||
| 1e5ba6568a | |||
| f3f3746da7 | |||
| dd9bda27c9 | |||
| c80381c4df | |||
| 4238a06bca | |||
| 873d22069f | |||
| 5441e10a0e | |||
| 9fedeea767 | |||
| 6873ce6777 | |||
| c90bc7e618 | |||
| ff183b8fca | |||
| 08badd5ed6 | |||
| 9d8be6f466 | |||
| 6d6639b268 | |||
| 83246ed65b | |||
| 33f4c2d61d | |||
| 2c2f68e00e | |||
| 135ea21a61 | |||
| 1648467828 | |||
| 1a68d808fb | |||
| 2c68b44c1a | |||
| fb2db4ee97 | |||
| 485d86e117 |
+59
-13
@@ -6,6 +6,18 @@ Nextcraft is a TypeScript monorepo (pnpm workspaces + turborepo) with a Next.js
|
||||
|
||||
**v0.3 additions (Credential Engines):** real credential engines replace v0.2 mock inputs — a sandbox fabric (isolated per-learner coding environments via Linux user/mount/pid/net namespaces), a live build-telemetry pipeline (WebSocket ingest + SQLite-ordered event log), a process-trace grading engine, seeded per-learner variant task generation, and a voice-based oral defense (STT/TTS via a new provider-agnostic voice layer). **First real persistence introduced: SQLite** (`ai_service/telemetry/`, grading, variant, defense stores). Lab/Assessor/Proctor agents are re-grounded onto real telemetry/traces. **Identity/age-gating (KYC) deferred per founder directive** — no security engineer persona; secrets-hygiene checklist only.
|
||||
|
||||
**v0.4 additions (Distribution & Bootstrap CLI, founder directive D-016):** a new `apps/cli` package — the `nextcraft` bootstrap CLI (`doctor`/`bootstrap`/`verify`/`dev`) compiled to a self-contained linux x64 binary via **Node SEA** (probe-verified: Go/Rust absent, node v24.15.0 SEA-capable), installed by a repo-served one-liner script that resolves the latest Gitea release, downloads binary + sha256 sidecar, verifies, and installs to `~/.local/bin`. Every release from v0.4 onward attaches the binary + checksum as release assets (the "ongoing binaries" requirement). The CLI is a thin wrapper: all orchestration logic stays in `apps/ai-service/scripts/` (bootstrap.sh/dev.sh) — the CLI composes them via subprocess (A-202), duplicating nothing. Previously-planned v0.4 seams (real STT/TTS, KYC, design/sim envs, seq-lease) move to v0.5.
|
||||
|
||||
### v0.5 Research Conclusions (Phase 0 RESEARCH)
|
||||
|
||||
25. **D-040 Voice real path = `OpenAIAudioProvider` on the shared httpx pool (D-R01)** — implements the D-030 protocol: `transcribe()` = multipart `POST {AI_VOICE_BASE_URL}/audio/transcriptions` (`file` + `model`, response_format=json → `TranscriptSegment`), `synthesize()` = streaming `POST /audio/speech` (JSON body, raw byte stream, `input` ≤4096 chars). Carries `descriptor = VoiceDescriptor(mode="server", ...)` which defense.py:143 already prefers — mode selection drops in with zero API changes. Constructor takes the lifespan `httpx.AsyncClient` (D-017; read=300s already tolerates multi-minute clips). Factory signature becomes `voice_provider_from_settings(settings, http_client)`. Errors sanitized with key redaction (mirror `llm/openai_compat.py:_sanitize`).
|
||||
26. **D-041 Voice route fixes (D-R02)** — defense.py:189 fmt derivation must strip codec params (`"webm;codecs=opus"` → `webm`, else real STT 400s); ~10 MB audio guard (422/413) before provider call; TTS route media_type becomes format-aware (or force `response_format=wav`). Web: defense-session POSTs the recorded blob (FormData), engine-client gains the audio-answer variant.
|
||||
27. **D-042 Identity = 5th D-027 store + provider protocol (D-R03)** — `ai_service/identity/` (D-031): `IdentityProvider` protocol (submit/poll/verify), deterministic mock, SQLite store modeled on DefenseStore (WAL, FK on, portable columns). Stores **derived age_band** (`16-17`/`18+`, never raw DOB) + document **refs** (never raw docs); mock verdicts carry a `mock` marker so downstream never displays them as production-verified (A-304). PII hygiene pinned by caplog sentinel test. **Negative finding:** no age-gate UI or enrollment API exists today — the flow is built, not swapped (D-010's "visual flow" is vestigial: one mock field).
|
||||
28. **D-043 Gate composition (D-R04)** — api/-layer dependencies in order: G-5 allowlist (403 pilot guard, retained) → identity verdict (403 + verify-CTA payload) → rate caps (429). School 16+ gates variant generation + sandbox create + defense start; marketplace 18+ via reusable `require_verified_adult` demonstrated on one minimal gated route (marketplace has no backend today — the thin route proves the contract end-to-end). All middleware-layer, never in the manager.
|
||||
29. **D-044 Environment registry at the template/variant layer (D-R05)** — `TaskTemplate.environment: Literal["build","design","simulation"]` → carried through VariantRecord → API → TS `TaskVariant`; surfaces the existing-but-dead `test_command` field (Run/Test buttons stop hardcoding pytest). Manager/backend/workdir unchanged (D-024 untouched — A-307: an environment is a typing over starter files + command policy). Per-kind exec command policy at the api/ exec route (template-declared harness + generic file/nav commands, 422 on violation). Grading digest stays kind-agnostic by construction (features derive from event kinds — pinned with a digest test over a synthetic design-kind trace). Starter contents: design = SVG/HTML/schematic artifacts + validate/render harness (stack-designer c001/c002 already sanctioned); simulation = parameterized benchmark script + dataset files (stack-science, stack-operator).
|
||||
30. **D-045 Seq-ack protocol (D-R06/D-R07)** — new WS frame `{"type":"seq_ack","seq":N}` emitted per successful append from `IngestSession._append` (durable `latest_seq`; O(1); advisory — gap detection stays authoritative, G-3/G-4 unchanged). The capture agent's supervisor loop (which today discards all non-close frames) parses text frames and trims spool+pending to `seq > ack` under `_emit_lock` via atomic `Spool.rewrite` — closing the one-line replay-margin gap (`_flush_locked` pops N in-flight frames but `replay_margin()` requeues only `_last_sent`). An explicit spool bound is added (A-309's "spool cap exists" premise was factually wrong on disk). Regression test: mid-burst kill in the test_durability.py real-server harness (uvicorn + KillableProxy) — the exact scenario the P07 de-flake documented as uncovered.
|
||||
31. **D-046 v0.5 wave order (D-R08)** — P1 seq-lease (smallest, protocol-only, fixes transport before env phases add reconnecting producers), P2 voice, P3 identity, P4 environments, P5 final. Phase numbers renumbered accordingly (was 1=voice..4=seq-lease in the initial ROADMAP draft).
|
||||
|
||||
### Confirmed Technology Stack (v0.2)
|
||||
|
||||
| Technology | Version | Purpose |
|
||||
@@ -47,6 +59,16 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
7. **D-022 Monorepo integration** — zero-dependency shim `package.json` in apps/ai-service + `ai#*` turbo passthrough tasks (`cache:false, outputs:[]`) + root `ai:dev`/`ai:test` scripts + idempotent venv bootstrap.
|
||||
8. **D-023 Testing** — pytest-asyncio auto mode; TestClient `client.stream()` for SSE; httpx MockTransport for byte-exact provider parser tests; scripted mock provider incl. failure modes. Tests never call the cloud.
|
||||
|
||||
### v0.4 Architecture Decisions (from Research — Distribution & Bootstrap CLI)
|
||||
|
||||
18. **D-033 Binary toolchain = Node SEA (probe-verified)** — Go and Rust are absent from this box; node v24.15.0 ships SEA support (`--experimental-sea-config`, postject-free on linux via `cp node nextcraft && node sea-config` … blob injection with the system `dd`/`npx postject` if needed). CLI source lives in `apps/cli` (TypeScript, compiled to a single CJS bundle by esbuild, then SEA-injected into a copy of the node binary → `nextcraft-linux-x64`). Fallback if SEA breaks: python3 `zipapp` (3.11.2 available). No new toolchain deps beyond dev-scoped esbuild.
|
||||
19. **D-034 CLI = thin wrapper, orchestration stays in scripts/** — `nextcraft` composes `apps/ai-service/scripts/bootstrap.sh` and `scripts/dev.sh` equivalents via `spawn` with inherited stdio and timeout guards (A-202/A-209). doctor/bootstrap/verify implement only *checking* logic (prereqs, env template, health) — never re-implement installs. This keeps one source of truth for bootstrap semantics.
|
||||
20. **D-035 Install path = repo raw `install.sh` + Gitea latest-release API** — the one-liner `curl -fsSL <forge>/coreci/nextcraft/raw/main/scripts/install.sh | bash` resolves `GET /api/v1/repos/coreci/nextcraft/releases/latest`, downloads the `nextcraft-linux-x64` + `nextcraft-linux-x64.sha256` assets, verifies sha256 (`shasum -a 256`), installs to `~/.local/bin` (PATH hint), and degrades to printed source-bootstrap instructions when no binary asset exists or the platform mismatches (A-203/A-204/A-206).
|
||||
21. **D-036 Ongoing binaries = ship-workflow asset step** — the release pipeline (v0.3's `ShipWorkflow.createRelease` equivalent, executed as the ship step's asset stage) builds the binary + checksum and attaches both to every Gitea release from v0.4 onward (A-205). Token resolution stays `.env*`-only (D-006/D-014); binaries are linux x64 only for v0.4 (macOS arm64 deferred — unverifiable on this box).
|
||||
22. **D-037 CLI package layout** — `apps/cli` is a pnpm workspace package (`@nextcraft/cli`): `src/` (entry, commands/, checks/, lib/), `scripts/build-binary.mjs` (esbuild bundle → SEA inject), unit tests runnable via `pnpm --filter @nextcraft/cli test` (node:test, no new test framework). Root `package.json` gains `cli:*` passthrough scripts mirroring the `ai:*` pattern (D-022).
|
||||
23. **D-038 Network mode (v0.3.5)** — dev binds 0.0.0.0 (`AI_HOST`, default 0.0.0.0, revert via 127.0.0.1); CORS + WS-origin gates read `AI_CORS_ORIGINS` (default `*` — any origin, safe only because credentials are never enabled; explicit comma list restricts); the web client derives the API base URL from the browser hostname at runtime (`engine-base-url.ts`: `NEXT_PUBLIC_AI_SERVICE_URL` override → `http://${window.location.hostname}:8420` → `localhost` server-side). Hotfix also fixes: SEA direct-run detection (`require("node:sea").isSea()` — argv shape differs by invocation style), installer honesty gate (silent `--version` = hard fail), bootstrap venv recovery (poisoned partial `.venv` removal + distro-specific `apt install python3.XX-venv` hint), and doctor venv-capability probe with bootstrap preflight.
|
||||
24. **D-039 Single-port deploy + unattended dev (v0.3.6)** — only :8420 is reachable behind HAProxy, so the web app ships as a **static export** (`output: 'export'`, `NEXT_PUBLIC_AI_SERVICE_URL=self` → relative same-origin fetches) served by the ai-service itself (`AI_WEB_STATIC_DIR` StaticFiles mount, default off; `nextcraft dev` auto-wires it when `apps/web/out` exists). Daemon surface: `dev -d` (detached, `~/.nextcraft/run/<clone-hash>/dev.{pid,log}`), `stop`, `log [-n N|-f]`. Durable state (DB `~/.nextcraft/data/`, sandbox workdirs `~/.nextcraft/sandboxes/`) moves **out of the repo** (founder directive; `expanduser` validator makes `AI_DB_PATH=~/...` env overrides work). API routes beat the static mount; unknown paths serve the export's 404.html.
|
||||
|
||||
### v0.3 Architecture Decisions (from Research — Credential Engines)
|
||||
|
||||
9. **D-024 Sandbox isolation = Linux namespaces via `unshare`** — per-learner sandbox runs as a subprocess entered into fresh user+mount+pid+network namespaces (`unshare --user --map-root-user --mount --pid --fork --net`). Probe-verified on this box: in-namespace uid=0, **network fully isolated** (0 interfaces), learner writes land in a per-sandbox directory; proc-remount not permitted here but not required. Chosen because no container runtime (docker/podman/bwrap/firejail) exists on the box and there is no sudo. A `SandboxBackend` protocol abstracts the spawner so a future containerd/runc backend can replace namespace-spawning without touching callers.
|
||||
@@ -67,7 +89,7 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `ai_service/main.py` | FastAPI app factory, lifespan (httpx client pool, provider factory, SandboxManager + reaper loop, SQLite engine stores on app.state), CORS (localhost only, incl. PUT for file writes), /health | App entry | config, llm, agents, api, engines |
|
||||
| `ai_service/main.py` | FastAPI app factory, lifespan (httpx client pool, provider factory, SandboxManager + reaper loop, SQLite engine stores on app.state), CORS (localhost only, incl. PUT for file writes), /health; v0.3.6: optional StaticFiles mount of the exported web app when `AI_WEB_STATIC_DIR` is set (single-port deploy, D-039) | App entry | config, llm, agents, api, engines |
|
||||
| `ai_service/config.py` | pydantic-settings Settings (env_prefix="AI_", env_file, SecretStr key) | Configuration only | None |
|
||||
| `ai_service/api/` | Endpoints: chat.py (POST /v1/chat/stream, SSE), lab.py, assessment.py (POST /v1/assessment/evaluate v0.2 + POST /v1/assessment/grade v0.3), proctor.py, mentor.py, sandboxes.py (lifecycle + files/exec routes, G-5 abuse gates), telemetry.py (WS ingest + trace/gaps reads), variants.py (seeded per-learner variants), defense.py (defense loop, REQ-3-006); deps.py (DI) | Composes agents + sessions + engines; never imported by llm/ or agents/ | agents, llm, sandbox, telemetry, grading, variants, voice |
|
||||
| `ai_service/llm/` | types.py (Message; ChatDelta/ChoiceDelta removed in P3 — no consumers), base.py (LLMProvider protocol), openai_compat.py (ollama-cloud + local), mock.py (deterministic), factory.py | Never imports agents/ or api/ | config |
|
||||
@@ -84,16 +106,31 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `ai_service/sandbox/` | `backend.py` (SandboxBackend protocol), `unshare_backend.py` (userns/mount/pid/net spawner, D-024), `manager.py` (lifecycle: create/list/snapshot/destroy + concurrency guard D-032), `workdir.py` (per-sandbox fs layout) | Never imports api/ or agents/; spawns subprocesses only | config |
|
||||
| `ai_service/telemetry/` | `models.py` (TelemetryEvent, TraceSpan), `store.py` (TraceStore protocol + SQLite impl D-027), `ingest.py` (WebSocket /v1/telemetry/ingest, seq gap detection D-026) | Persistence; never imports agents/ | config |
|
||||
| `ai_service/grading/` | `features.py` (deterministic trace digest D-028), `engine.py` (rubric scoring orchestration), `store.py` (GradeStore) | LLM only via digest; never sees raw trace | llm, telemetry, prompts |
|
||||
| `ai_service/variants/` | `templates.py` (task template library), `generator.py` (seeded LLM instantiation D-029), `store.py` (VariantStore) | LLM via structured output | llm, grading |
|
||||
| `ai_service/voice/` | `base.py` (VoiceProvider protocol D-030), `browser.py` (native SR/TTS fallback descriptor), `mock.py` (deterministic; the real server STT/TTS provider is the v0.4 seam — GRILL CUT-1/G-7), `factory.py` (provider selection), `defense_store.py` (DefenseStore: transcripts + integrity signals, D-027) | Never imports agents/ or api/ | config |
|
||||
| `ai_service/telemetry/` | `models.py` (TelemetryEvent, TraceSpan), `store.py` (TraceStore protocol + SQLite impl D-027), `ingest.py` (WebSocket /v1/telemetry/ingest, seq gap detection D-026; v0.5 REQ-5-007/D-045: emits advisory `seq_ack` frames — highest-contiguous received seq per successful append; capture agent trims its spool to the ack, closing the replay-margin gap) | Persistence; never imports agents/ | config |
|
||||
| `ai_service/grading/` | `features.py` (deterministic trace digest D-028 — kind-agnostic by construction, pinned over design/sim traces in v0.5), `engine.py` (rubric scoring orchestration), `store.py` (GradeStore) | LLM only via digest; never sees raw trace | llm, telemetry, prompts |
|
||||
| `ai_service/variants/` | `templates.py` (task template library; v0.5 REQ-5-005/D-044: `environment: Literal[build,design,simulation]` registry + per-kind starter files + harness/test commands, G-15 shlex-roundtrip validation), `generator.py` (seeded LLM instantiation D-029), `store.py` (VariantStore; idempotent `_ensure_v05_columns` backfill for pre-v0.5 DBs) | LLM via structured output | llm, grading |
|
||||
| `ai_service/voice/` | `base.py` (VoiceProvider protocol D-030), `browser.py` (native SR/TTS fallback descriptor), `mock.py` (deterministic), `openai_audio.py` (v0.5 REQ-5-001, D-040: real server STT/TTS against OpenAI-compatible `/audio/transcriptions` + `/audio/speech` on the shared httpx pool — CUT-1/G-7 seam CLOSED), `factory.py` (provider selection by `AI_VOICE_PROVIDER`, G-11 boot-safe fallback to mock), `defense_store.py` (DefenseStore: transcripts + integrity signals, D-027) | Never imports agents/ or api/ | config |
|
||||
| `ai_service/identity/` | **v0.5 NEW (REQ-5-003/004, D-042/43):** `base.py` (IdentityProvider protocol: submit/poll/verify), `mock.py` (deterministic approve-on-policy mock; verdicts carry a `mock` marker, A-304), `store.py` (5th D-027 store: identity_record table — derived `age_band`, document **refs**, PII never stored raw); age-gate dependencies `require_verified_age`/`require_verified_adult` (gate composition D-043: G-5 allowlist → identity verdict → rate caps; mounted on variants/sandbox-create/defense-start + the G-18 marketplace stub), exposed via `api/identity.py` (`/v1/identity/*`) | Never imports agents/; api/ composes it via DI | config |
|
||||
| `ai_service/agents/examiner.py` | Seventh agent: oral defense examiner; streams over existing SSE, consumes process traces + emits integrity signals | reuses BaseAgent (D-018) | llm, prompts, telemetry |
|
||||
| `ai_service/data/*.db` | SQLite databases (telemetry/grades/variants/defenses) | gitignored | — |
|
||||
| `ai_service/data/*.db` | SQLite databases (telemetry/grades/variants/defenses) — **v0.3.6: default moved to `~/.nextcraft/data/nextcraft.db` (state out of the repo; AI_DB_PATH overrides, ~ expanded)** | outside repo (home) | — |
|
||||
| `scripts/sandbox-agent.py` | Tiny in-namespace capture process shipped into the sandbox; streams telemetry to ingest | standalone | stdlib only |
|
||||
|
||||
**Boundary additions:** `sandbox/`, `telemetry/`, `grading/`, `variants/`, `voice/` are engine modules — they never import `api/` (which composes them via DI) and never import `agents/` (agents call engines through narrow interfaces, not vice versa).
|
||||
|
||||
### apps/cli — Nextcraft Bootstrap CLI (v0.4 NEW)
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `src/index.ts` | Entry: arg parsing (no deps beyond node stdlib at runtime), command dispatch, `--help`/`--version`, exit-code contract (0 ok / 1 failure / 2 usage) | CLI surface only | commands/ |
|
||||
| `src/commands/` | `doctor.ts` (prereq checks + actionable errors), `bootstrap.ts` (pnpm install + scripts/bootstrap.sh wrapper + env template copy + key validation), `verify.ts` (health: venv imports, ports, env, build readiness), `dev.ts` (thin passthrough to scripts/dev.sh) | Compose checks/ + lib/; spawn scripts — never re-implement them | checks/, lib/ |
|
||||
| `src/checks/` | Pure check functions: `check-command.ts` (binary-on-PATH + version compare), `check-env.ts` (template diff, required/optional key classification) | Pure logic, unit-testable, no fs side effects at import | None |
|
||||
| `src/lib/` | `spawn.ts` (subprocess with timeout + inherited stdio), `log.ts` (✓/✗/warn output formatter) | Shared utilities | None |
|
||||
| `scripts/build-binary.mjs` | esbuild → CJS bundle → Node SEA injection → `dist/nextcraft-linux-x64` + sha256 sidecar | Build-time only | esbuild (dev dep) |
|
||||
| `scripts/install.sh` | The one-liner install script served from repo raw: Gitea latest-release resolve → download + checksum verify → ~/.local/bin; source-bootstrap fallback | Standalone POSIX sh | forge API |
|
||||
| `tests/` | node:test unit tests: command dispatch, check logic, env template diff, install-script shellcheck-style assertions | Fixtures only — never mutate repo state | src/ |
|
||||
|
||||
**Boundary rules:** the CLI never imports from `apps/web`, `packages/*`, or `ai_service` Python modules — it orchestrates them exclusively via subprocess/filesystem. Runtime deps: node stdlib only (no runtime npm deps; esbuild is dev-only). The binary embeds the bundle; `scripts/bootstrap.sh` remains the single source of bootstrap truth (D-034).
|
||||
|
||||
### apps/web — Next.js Application
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
@@ -105,7 +142,7 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
| `app/layout.tsx` | Root layout: theme provider, navigation shell, responsive container | All routes | packages/ui |
|
||||
| `components/` | Surface-specific components (learner/, marketplace/, employer/, admin/) plus shared chrome (navigation-shell, header/footer, role-switcher, theme-provider, breadcrumbs, dark-mode-toggle); v0.3 learner: build-surface, sandbox-terminal (read-only exec output), defense-session | App-level components | packages/ui |
|
||||
| `hooks/` | use-chat-stream.ts — SSE client hook: fetch + ReadableStream, byte buffering + frame reassembly, idempotent AbortController cleanup; use-sandbox-session.ts (v0.3) — sandbox lifecycle for the build session: create on task open, destroy on unmount, mid-start failure cleanup, 503/403/429 honest surfaces | Client components only | ai-service SSE / engine API |
|
||||
| `lib/` | sse.ts (shared SSE frame parser — CRLF normalization + `: ping` immunity, v0.2 G-1), breadcrumbs.ts, format.ts, engine-client.ts (v0.3: typed fetch client for /v1/sandboxes, files/exec, variants, grade, defense, traces) | Pure utilities | None |
|
||||
| `lib/` | sse.ts (shared SSE frame parser — CRLF normalization + `: ping` immunity, v0.2 G-1), breadcrumbs.ts, format.ts, engine-base-url.ts (v0.3.5: runtime API base — env override → browser hostname → localhost), engine-client.ts (v0.3: typed fetch client for /v1/sandboxes, files/exec, variants, grade, defense, traces) | Pure utilities | None |
|
||||
|
||||
### packages/ui — Shared Component Library
|
||||
|
||||
@@ -193,7 +230,13 @@ Composite/layout/theme components (navigation shell, tables, chat panels, graph
|
||||
|
||||
---
|
||||
|
||||
## Build Order (v0.3)
|
||||
## Build Order (v0.4)
|
||||
|
||||
1. **Bootstrap CLI core** — apps/cli package: doctor checks (node/pnpm/python3/git/unshare), bootstrap wrapper (pnpm install + scripts/bootstrap.sh + .env template + key validation), verify health check, dev passthrough; unit tests
|
||||
2. **Binary build + release pipeline** — esbuild bundle → Node SEA binary (`nextcraft-linux-x64`) + sha256 sidecar; install.sh one-liner (Gitea latest-release resolve + checksum verify + PATH install); release-asset upload wired into the ship flow (ongoing binaries from v0.4 onward)
|
||||
3. **Install docs + fresh-clone E2E** — README quickstart (one-liner → doctor → bootstrap → dev), CLI reference, fresh-clone end-to-end test proving a clean clone reaches a running stack
|
||||
|
||||
## Build Order (v0.3 — complete)
|
||||
|
||||
1. **Sandbox fabric** — SandboxBackend protocol + unshare namespace spawner + lifecycle manager (create/list/snapshot/destroy) + concurrency guard + per-sandbox workdir; isolation + resource-limit probes
|
||||
2. **Live build telemetry** — TelemetryEvent models + SQLite TraceStore + WebSocket ingest endpoint + seq gap detection + in-sandbox capture agent
|
||||
@@ -217,16 +260,19 @@ The v0.1 build order (monorepo → types → mock data → tokens → primitives
|
||||
|
||||
---
|
||||
|
||||
## Future Architecture (Post-v0.3, for reference)
|
||||
## Future Architecture (Post-v0.4, for reference)
|
||||
|
||||
v0.3 delivers the real credential engines; later milestones fill in the remaining platform:
|
||||
v0.4 delivers distribution (CLI + binary releases); later milestones fill in the remaining platform:
|
||||
|
||||
- **In-memory sessions → PostgreSQL + Drizzle/SQLModel** — SessionStore + v0.3 TraceStore/GradeStore/VariantStore/DefenseStore protocols swap SQLite→Postgres with no API changes
|
||||
- **userns subprocess sandboxes → containerd/runc backend** — D-024 `SandboxBackend` protocol swap; same lifecycle API
|
||||
- **Coding-IDE sandbox → design tool + simulation environments** — REQ-F-021 full scope (v0.4)
|
||||
- **Coding-IDE sandbox → design tool + simulation environments** — REQ-F-021 full scope (v0.5)
|
||||
- **Mock provider → per-agent model routing** — provider factory already selects by config; per-agent `AI_<AGENT>_MODEL` overrides
|
||||
- **No auth → real KYC + sessions** — **deferred per founder directive**; REQ-F-017 identity/age-gating lands post-v0.3 (v0.4+). Age-gating remains the v0.1 visual flow mockup
|
||||
- **No auth → real KYC + sessions** — **deferred per founder directive; moved to v0.5 with D-016**; REQ-F-017 identity/age-gating lands post-v0.4. Age-gating remains the v0.1 visual flow mockup
|
||||
- **Mock voice → real server STT/TTS (openai-audio provider)** — CUT-1/G-7 seam moved to v0.5 per D-016; VoiceProvider protocol is the drop-in point
|
||||
- **linux x64 binary → macOS arm64 + auto-update** — D-036 defers non-linux targets (unverifiable on this box); `nextcraft upgrade` (self-replace from latest release) is the natural v0.5+ follow-up
|
||||
- **Exec-telemetry seq-lease / replay-margin fix** — the P6-lesson one-line ACK gap moves to v0.5 per D-016
|
||||
- **No search → Semantic vector search (pgvector)** — Filter UI replaced with vector similarity search
|
||||
- **No payments → Payment processing** — Pricing page replaced with real subscription/payment flows
|
||||
|
||||
The monorepo structure (apps/web + apps/ai-service + packages/*) accommodates further apps without restructuring.
|
||||
The monorepo structure (apps/web + apps/ai-service + apps/cli + packages/*) accommodates further apps without restructuring.
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"phase": 7,
|
||||
"stage": "audit",
|
||||
"milestone": "v0.3",
|
||||
"phase_role": "final",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-09-12T20:55:00Z"
|
||||
}
|
||||
@@ -46,3 +46,71 @@ The credential-pipeline architecture (telemetry → trace → grade → defense)
|
||||
## Outcome
|
||||
|
||||
**GO** — all six binding decisions and both scope cuts applied to PLAN.md / REQUIREMENTS.md / ROADMAP.md / PROJECT.md before Phase 1 execution. No axis requires escalation (all resolvable at confidence ≥ 0.85). The milestone no longer claims resource enforcement it cannot deliver, and the no-auth abuse vector is closed at MVP scale.
|
||||
|
||||
---
|
||||
# Nextcraft v0.4 — GRILL.md (Adversarial Review Verdict)
|
||||
|
||||
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-CHANGES · **Confidence:** 0.83
|
||||
|
||||
## Summary
|
||||
|
||||
The distribution milestone is small, founder-directed (D-016, confidence 0.99), and additive (zero changes to the running credential pipeline). The plan's central risk: **Node SEA was probe-verified as a flag, not as a working build** — the v0.3 lesson (A-101: probe the mechanism, not the existence) applies. Second gap: a binary whose `--version` lies (stale package.json) would poison the "ongoing binaries" contract. Third: sed-based JSON parsing in install.sh is a fragility + integrity risk. Fourth: "ongoing binaries" has no enforcement mechanism beyond prose. All four closed by binding decisions G-101..G-104 below. No scope cuts required — the milestone is already minimal.
|
||||
|
||||
## Per-Axis Findings
|
||||
|
||||
| Axis | Verdict | Rationale |
|
||||
|------|---------|-----------|
|
||||
| Business case | PASS | Founder directive explicit + recorded (D-016). Evidence of need: live Gitea probe shows latest release v0.2.8 with ZERO assets; bootstrap requires repo archaeology (scripts found only via package.json spelunking). |
|
||||
| Scope | PASS | 5 REQs, 3 execution phases, one focused surface (apps/cli + scripts). Smallest milestone yet. macOS arm64 already cut (D-036, unverifiable here). |
|
||||
| Feasibility | CONCERN (fixed) | SEA flag exists on node v24.15.0, but no end-to-end SEA binary was built during RESEARCH. postject availability assumed (`npx postject` — needs npm registry reachability, unproven). Zipapp fallback requires python3 on target — an honest-degradation ladder, not a silent downgrade. → G-101. |
|
||||
| Honest versioning | CONCERN (fixed) | `--version` from package.json would print a stale hardcoded version inside a per-release binary — breaks upgrade detection + the one-liner's re-run-to-upgrade promise. → G-102. |
|
||||
| Install integrity | CONCERN (fixed) | sed/grep JSON parsing is brittle; a parse failure must never fall through to installing an unverified artifact. Exact asset-name matching + hard-degrade to source instructions. → G-103. |
|
||||
| Sequencing | PASS | P1 CLI (source-runnable) → P2 binary+pipeline → P3 docs+E2E matches dependency order; each phase ships independently. |
|
||||
| Cost/quota | PASS | Zero new paid infra; binaries built on-box; Gitea releases free. Dev-only esbuild dep. |
|
||||
| Risks | CONCERN (fixed) | Top 3: SEA end-to-end (→ G-101 live probe FIRST in P2), npm registry reachability for esbuild (→ proven by P1's pnpm install must-have), Gitea asset-upload token scope (→ live-proven at the v0.3.2 ship itself). |
|
||||
| Adoption/operability | PASS | Consumer = founder + future pilots; one command replaces README archaeology. Rollback trivial (rm ~/.local/bin/nextcraft). No server changes. |
|
||||
|
||||
## Binding Decisions (applied to PLAN.md)
|
||||
|
||||
- **G-101 (BINDING) — SEA live-build probe is the FIRST P2 action.** Task 2-1-01 builds a real binary before anything depends on it; the build script encodes the fallback ladder explicitly (SEA → zipapp with "requires python3" honesty). If SEA fails on this box, zipapp becomes primary with the docs stating the requirement — no silent claim of node-less operation.
|
||||
- **G-102 (BINDING) — Version stamping at build time.** `build-binary` accepts the shipping tag and stamps it into the bundle (`NEXTCRAFT_VERSION` replace); `--version` prints it; install E2E asserts the installed binary reports the tag it was downloaded from. A binary may never report a version it was not built as.
|
||||
- **G-103 (BINDING) — Install-script integrity hard-degrade.** install.sh matches assets by EXACT name (`nextcraft-linux-x64`, `nextcraft-linux-x64.sha256`); any parse/lookup/download failure degrades to source-bootstrap instructions (exit 0) — never installs unverified or name-approximate artifacts. Checksum mismatch = hard stop, exit 1, explicit do-not-run message. dash-safe POSIX sh, no jq.
|
||||
- **G-104 (BINDING) — Ongoing-binaries enforcement.** Every ship from v0.3.2 onward MUST run `scripts/release-assets.sh <tag>` after tag+merge (best-effort, non-blocking, `release_pending` escalation on failure — but attempted + logged every release). The final-phase audit gate includes "milestone release carries both assets" as a check. This makes the founder's "ongoing binaries" directive a pipeline property, not prose.
|
||||
|
||||
## Escalations
|
||||
|
||||
None. All four concerns resolved at confidence ≥ 0.85. No axis requires founder escalation (directive already explicit).
|
||||
|
||||
## Outcome
|
||||
|
||||
**GO** — G-101..G-104 applied to PLAN.md before Phase 1 execution. The milestone claims only what its probes prove, and the ongoing-binaries contract has an enforcement mechanism.
|
||||
|
||||
---
|
||||
|
||||
# v0.5 GRILL (Phase 0, 2026-09-13) — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease
|
||||
|
||||
**Verdict: GO-WITH-CHANGES · Confidence: 0.78** — CUT-3 + G-9..G-18 binding; applied to PLAN.md before P1 execution.
|
||||
|
||||
**Evidence:** every load-bearing plan claim verified against ground-truth code and held (fmt bug defense.py:189, descriptor flow defense.py:143, supervisor frame discard sandbox-agent.py:487-496, no spool cap, dead test_command field, one-line replay margin sandbox-agent.py:416-431, factory rejection). The research layer (D-040..D-046) sold no vapor; the misses were all *interaction* surface.
|
||||
|
||||
## Binding items (all applied to PLAN.md)
|
||||
|
||||
- **CUT-3**: MH-3e identity E2E downgraded from real-app uvicorn harness to `TestClient(create_app(...))` — identity is plain JSON HTTP; the uvicorn harness stays where transport matters (P1 durability, P4 in-ns exec).
|
||||
- **G-9**: `verified_pilot` conftest fixture MUST land with the identity gates (Wave 3-2) — variants/defense are ungated today; without the fixture the existing 413-test suite 403s en masse. MH-M6 reworded: regressions zero *for verified allowlisted pilot learners*.
|
||||
- **G-10**: engine-client must discriminate 403 reasons (`verify_cta` → new `VerifyRequiredError`; allowlist detail → existing `NotAllowlistedError`) — today every 403 collapses into NotAllowlistedError, which would render a verify-CTA as an allowlist lie.
|
||||
- **G-11**: misconfigured `openai-audio` must never crash the boot — lifespan catches the factory error, logs loudly, falls back to mock (descriptor honestly reads mock). Unattended deploy (D-039) survival rule.
|
||||
- **G-12**: client recording bound — auto-stop at 180s default + visible timer + timeslice; 413 renders "re-record" honestly (never silent answer loss).
|
||||
- **G-13**: identity submit caps — one active pending per learner (409 on resubmit) + per-learner rate cap (429); each submission becomes vendor money later.
|
||||
- **G-14**: spool-bound overflow is by-design gap creation — pinned: dropped-counter > 0 → replayed trace gapped → ungradable (never silently-truncated-but-gradable); worst-case spool ≈256MB documented against the 512MB G-2 sweep.
|
||||
- **G-15**: argv contract — templates validate shlex-roundtrip at definition (no quotes/globs); TS splits whitespace-only (no shlex in browsers); exec policy matches EXACT argv tokens (never prefix); `sh -c` passthrough disallowed for design/sim kinds (the digest-gaming vector).
|
||||
- **G-16**: `voice_tts_format` is a `Literal["mp3","wav","opus"]` enum, not a free string — it feeds a Content-Type.
|
||||
- **G-17**: MH-4e "seq-ack intact" was hope-shaped; merged into MH-4d as concrete assertions (stored seqs contiguous 0..N exactly once; spool ≤ ack margin; or cite the P1 suite where a fake agent runs).
|
||||
- **G-18**: the marketplace gated stub proves the gate then returns 501 + `stub: true` + mock markers — never a fabricated "applied" outcome (A-304 honesty house rule).
|
||||
|
||||
## Advisories (recorded)
|
||||
|
||||
a-6 ack on dedup'd appends too (tight margin); a-7 MH-1d 3× runs = one-time ship validation, not per-CI; a-8 manual voice probe must be executable-by-anyone-with-keys (fixture wav + fixed phrase, falsifiable asserts); a-9 413 Content-Length fast path before buffering; a-10 ROADMAP P4 "Depends On: 4" self-typo → fix to 0; a-11 TS `TaskVariant` fields required on the wire; a-12 identity insert-only growth fine at pilot scale; a-13 `recorder.start(1000)`; a-14 do NOT build ack batching unless a flood test shows drainer stall; a-15 provider must always carry `descriptor` (missing it would badge server as mock).
|
||||
|
||||
## Escalations
|
||||
|
||||
None. All four seams remain founder-locked via D-016 / REQ-5-001..007; every finding resolved at confidence ≥ 0.65.
|
||||
|
||||
+142
-128
@@ -2,13 +2,13 @@
|
||||
|
||||
## Persona Roster
|
||||
|
||||
> **v0.3 update (RESEARCH, lead-developer assessment):** backend-engineer territory extended to the new engine modules (telemetry/grading/variants persistence + APIs). New phase-relevant custom personas added: **sandbox-engineer** (Linux-namespace isolation infra) and **voice-engineer** (STT/TTS + Examiner agent audio pipeline). ai-engineer re-scoped to LLM/agents/prompts + grading/variant/voice *model-facing* logic. **security-auditor stays inactive** (KYC deferred per founder directive). frontend-engineer gains real-sandbox (read-only exec-output terminal frame, CUT-2/G-8 — interactive xterm relay is v0.4), live-telemetry, and live-defense surfaces.
|
||||
> **v0.5 update (RESEARCH, lead-developer assessment):** milestone = Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease. **Reactivated:** voice-engineer (real server STT/TTS, its v0.3 territory), sandbox-engineer (design/sim environment kinds + the seq-ack protocol on the fabric/agent it owns), frontend-engineer (defense audio upload, identity enrollment flow, kind-aware build surface). **New custom persona: identity-engineer** (KYC/identity domain — 5th store, provider protocol, age-gate dependencies, PII hygiene). **security-auditor re-activated (phase-specific)** for identity PII + age-gate bypass + audio upload attack surface (phases 1-4 review, final phase). ai-engineer light-touch (no LLM-facing work this milestone). backend-engineer retains settings/factory wiring + turbo/root scripts. cli-engineer inactive (v0.3.6 hotfix shipped; no CLI work planned). design-system-engineer/data-engineer inactive (one TS types extension only — data-engineer light-touch for variants/identity types).
|
||||
|
||||
### lead-developer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Coordinates task decomposition across web, AI service, engine, and sandbox territories; resolves conflicts between frontend, backend, AI, sandbox, and voice personas
|
||||
reason: Coordinates the four v0.5 seams across voice/identity/sandbox/telemetry territories; resolves wave-order (D-046: seq-lease first) and identity-gate composition (D-043) boundaries
|
||||
domain: coordination
|
||||
frameworks:
|
||||
- next.js
|
||||
@@ -27,97 +27,143 @@ territory:
|
||||
- "apps/ai-service/pyproject.toml"
|
||||
```
|
||||
|
||||
### frontend-engineer
|
||||
### voice-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Phase 6 real-engine learner-surface integration — read-only exec-output terminal frame (CUT-2, no interactive shell), file-tree/run/test controls, live telemetry panels, live voice defense UI, live grading display. Owns all page components, layouts, surface-specific UI.
|
||||
domain: frontend
|
||||
reason: v0.3 territory reactivated — owns REQ-5-001/002: OpenAIAudioProvider (D-040) on the shared httpx pool, factory signature change, defense route fixes (fmt strip, size guard, media_type), descriptor mode=server, web audio POST
|
||||
domain: ai-media
|
||||
frameworks:
|
||||
- httpx
|
||||
- fastapi
|
||||
- pytest
|
||||
- react
|
||||
- next.js
|
||||
- tailwindcss
|
||||
- lucide-react
|
||||
- recharts
|
||||
- react-flow
|
||||
constraints:
|
||||
- component-first
|
||||
- server-components-default
|
||||
- minimal-client-js
|
||||
- sse-client-buffering (buffer bytes, split frames on \n\n, join data: lines)
|
||||
- abortcontroller-cleanup (idempotent abort in effect cleanup)
|
||||
- fetch-lifecycle (typed engine-client calls, retry/teardown, cleanup)
|
||||
- mediarecorder-permission-ux (mic consent, graceful no-mic fallback)
|
||||
- responsive-all-breakpoints
|
||||
- dark-mode-support
|
||||
- provider-agnostic-protocol (D-030 drop-in; descriptor wins selection)
|
||||
- never-call-cloud-in-tests (MockTransport byte-contract pins)
|
||||
- key-redaction (mirror openai_compat _sanitize)
|
||||
- bounded-audio-in-memory (10MB guard before provider call)
|
||||
territory:
|
||||
- "apps/web/**"
|
||||
- "packages/ui/**"
|
||||
- "packages/mock-data/**"
|
||||
- "packages/types/**"
|
||||
- "apps/ai-service/ai_service/voice/**"
|
||||
- "apps/ai-service/tests/voice/**"
|
||||
- "apps/web/components/learner/defense-session.tsx"
|
||||
```
|
||||
|
||||
### data-engineer
|
||||
### identity-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Owns TS mock data layer schema and typed definitions + TS types for telemetry/trace/grade/variant/defense shapes the web surfaces consume. Does NOT own the Python corpus or engine stores — aligned by convention (D-021).
|
||||
domain: data
|
||||
reason: v0.5 custom persona (RESEARCH, D-042/43) — owns the new identity module: IdentityProvider protocol + mock, 5th SQLite store (DefenseStore pattern), /v1/identity router, age-gate dependencies (allowlist → identity → rate caps), derived age_band + document refs (PII minimal), caplog sentinel scrub test
|
||||
domain: identity
|
||||
frameworks:
|
||||
- typescript
|
||||
- fastapi
|
||||
- sqlmodel
|
||||
- pytest
|
||||
constraints:
|
||||
- schema-first
|
||||
- type-safe
|
||||
- migration-ready
|
||||
- mock-data-only
|
||||
- pii-never-stored-raw (refs + derived bands only, A-305)
|
||||
- pii-never-logged (caplog sentinel pin)
|
||||
- mock-verdict-honesty (mock marker rides every response, A-304)
|
||||
- gate-composition-order (G-5 allowlist first, identity second, 429 caps last)
|
||||
territory:
|
||||
- "packages/types/**"
|
||||
- "packages/mock-data/**"
|
||||
- "apps/ai-service/ai_service/identity/**"
|
||||
- "apps/ai-service/tests/identity/**"
|
||||
```
|
||||
|
||||
### sandbox-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: v0.3 territory reactivated — owns REQ-5-005/006 (template-layer env registry, starter contents, exec command policy) AND REQ-5-007 (seq-ack frame in ingest + agent spool trim, mid-burst regression test) — both live on the fabric/agent surfaces it built
|
||||
domain: infra
|
||||
frameworks:
|
||||
- python
|
||||
- linux-namespaces
|
||||
- pytest
|
||||
constraints:
|
||||
- stdlib-only-agent (AST-pinned sandbox-agent.py imports)
|
||||
- acks-are-advisory (gap detection stays authoritative; G-3/G-4 unchanged)
|
||||
- thread-safe-spool-trim (under _emit_lock, atomic Spool.rewrite)
|
||||
- digest-kind-agnostic (features derive from event kinds — pinned)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/sandbox/**"
|
||||
- "apps/ai-service/ai_service/telemetry/**"
|
||||
- "apps/ai-service/scripts/sandbox-agent.py"
|
||||
- "apps/ai-service/ai_service/variants/templates.py"
|
||||
- "apps/ai-service/tests/sandbox/**"
|
||||
- "apps/ai-service/tests/telemetry/**"
|
||||
```
|
||||
|
||||
### backend-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Owns apps/ai-service app shell, config, API endpoints (incl. WebSocket telemetry ingest), engine persistence (SQLite stores), scripts, and test harness. Extended for v0.3 engine modules.
|
||||
reason: Owns the settings surface both new providers hang off (voice_base_url/key/models, identity provider selection), factory + main.py lifespan wiring (voice factory gains http_client), .env.example documentation
|
||||
domain: backend
|
||||
frameworks:
|
||||
- fastapi
|
||||
- uvicorn
|
||||
- pydantic
|
||||
- pydantic-settings
|
||||
- httpx
|
||||
- pytest
|
||||
- sqlmodel
|
||||
- sqlalchemy
|
||||
- websockets
|
||||
- aiofiles
|
||||
- bash
|
||||
- turborepo
|
||||
- pnpm
|
||||
constraints:
|
||||
- provider-agnostic-boundaries (engine modules import nothing from agents/ or api/)
|
||||
- streaming-first
|
||||
- sqlite-first-persistence (protocol-wrapped stores, Postgres-ready, D-027)
|
||||
- secrets-via-env-only
|
||||
- mock-provider-in-tests
|
||||
- websocket-contract (typed envelopes, seq gap detection, D-026)
|
||||
- secrets-via-env-only (D-014; keys never in code or commits)
|
||||
- idempotent-scripts
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/main.py"
|
||||
- "apps/ai-service/ai_service/config.py"
|
||||
- "apps/ai-service/ai_service/api/**"
|
||||
- "apps/ai-service/ai_service/telemetry/store.py"
|
||||
- "apps/ai-service/ai_service/telemetry/ingest.py"
|
||||
- "apps/ai-service/ai_service/grading/store.py"
|
||||
- "apps/ai-service/ai_service/variants/store.py"
|
||||
- "apps/ai-service/ai_service/main.py"
|
||||
- "apps/ai-service/ai_service/voice/factory.py"
|
||||
- "apps/ai-service/scripts/**"
|
||||
- "apps/ai-service/package.json"
|
||||
- "apps/ai-service/tests/api/**"
|
||||
- "apps/ai-service/.env.example"
|
||||
- "package.json"
|
||||
- "turbo.json"
|
||||
```
|
||||
|
||||
### frontend-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Reactivated for v0.5 UX: defense-session audio POST (recorded blob upload), identity enrollment flow (submit → pending → verified states + verify-CTA surfaces), build-surface kind-awareness (variant environment + test_command), engine-client identity + audio functions
|
||||
domain: frontend
|
||||
frameworks:
|
||||
- react
|
||||
- next.js
|
||||
- tailwindcss
|
||||
constraints:
|
||||
- component-first
|
||||
- server-components-default
|
||||
- honest-state-surfaces (provider badge: mock vs browser vs server; unverified labels)
|
||||
territory:
|
||||
- "apps/web/**"
|
||||
- "packages/ui/**"
|
||||
- "packages/types/**"
|
||||
```
|
||||
|
||||
### security-auditor
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: true
|
||||
reason: v0.5 re-activated — identity PII (storage + logs), age-gate bypass review, audio upload attack surface (10MB guard, format confusion), exec command policy (per-kind allowlist), TTS media_type. Phases 1-4 reviews + final phase
|
||||
domain: security
|
||||
frameworks:
|
||||
- pytest
|
||||
- httpx
|
||||
constraints:
|
||||
- STRIDE-classified
|
||||
- pii-never-stored-raw
|
||||
- pii-never-logged
|
||||
- bounded-uploads
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/identity/**"
|
||||
- "apps/ai-service/ai_service/api/defense.py"
|
||||
- "apps/ai-service/ai_service/api/sandboxes.py"
|
||||
- "apps/ai-service/ai_service/voice/**"
|
||||
```
|
||||
|
||||
### ai-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Owns the LLM provider layer, agent framework, prompt library, structured outputs, and the model-facing logic of v0.3 engines — trace-digest→rubric grading prompts (grading/features.py+engine.py), variant instantiation (variants/templates.py+generator.py), and the Examiner agent. Owns the deterministic-mock corpora.
|
||||
reason: Light-touch v0.5 — no LLM-facing work (voice STT/TTS is media plumbing, not model work; examiner agent unchanged); guards the agent/engine boundaries the new surfaces touch
|
||||
domain: ai
|
||||
frameworks:
|
||||
- pydantic
|
||||
@@ -125,116 +171,84 @@ frameworks:
|
||||
- pytest
|
||||
constraints:
|
||||
- provider-agnostic-protocol
|
||||
- prompts-are-code
|
||||
- json-defensive-parsing
|
||||
- never-call-cloud-in-tests
|
||||
- delta-passthrough
|
||||
- llm-sees-digest-not-raw-trace (D-028)
|
||||
- seeded-variant-reproducibility (D-029)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/llm/**"
|
||||
- "apps/ai-service/ai_service/agents/**"
|
||||
- "apps/ai-service/ai_service/prompts/**"
|
||||
- "apps/ai-service/ai_service/corpus/**"
|
||||
- "apps/ai-service/ai_service/grading/features.py"
|
||||
- "apps/ai-service/ai_service/grading/engine.py"
|
||||
- "apps/ai-service/ai_service/variants/templates.py"
|
||||
- "apps/ai-service/ai_service/variants/generator.py"
|
||||
- "apps/ai-service/tests/llm/**"
|
||||
- "apps/ai-service/tests/agents/**"
|
||||
```
|
||||
|
||||
### sandbox-engineer
|
||||
### cli-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: true
|
||||
reason: v0.3 custom persona (RESEARCH) — owns the sandbox fabric: SandboxBackend protocol, unshare-based Linux user/mount/pid/net namespace spawner, per-sandbox workdir, resource limits, lifecycle manager, concurrency guard, and the in-sandbox capture agent. Probe-verified isolation on this box (D-024).
|
||||
domain: infra
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: v0.3 persona — CLI shipped complete (v0.3.6 daemon surface); no v0.5 CLI work planned. Reactivated if milestone work touches apps/cli.
|
||||
domain: cli
|
||||
frameworks:
|
||||
- python
|
||||
- linux-namespaces
|
||||
- asyncio
|
||||
- pytest
|
||||
- node
|
||||
- typescript
|
||||
- node:test
|
||||
- esbuild
|
||||
- node-sea
|
||||
- posix-sh
|
||||
constraints:
|
||||
- isolation-verified (probe must show in-ns uid=0, network isolated, writes to workdir only)
|
||||
- backend-protocol-swap (no containerd assumption; D-024)
|
||||
- resource-limits-enforced (cpu/mem/time quotas observable)
|
||||
- no-daemon (subprocess-only; no docker/containerd service)
|
||||
- capacity-guard (1-5 concurrent; 503 when full, D-032)
|
||||
- stdlib-only-runtime
|
||||
- thin-wrapper
|
||||
- timeout-every-spawn
|
||||
- fail-loud-exit-codes
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/sandbox/**"
|
||||
- "apps/ai-service/scripts/sandbox-agent.py"
|
||||
- "apps/ai-service/tests/sandbox/**"
|
||||
```
|
||||
|
||||
### voice-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: true
|
||||
reason: v0.3 custom persona (RESEARCH) — owns the voice layer: VoiceProvider protocol, STT/TTS against a compatible endpoint, deterministic mock (tests never call a voice API), browser-native fallback, and the media-path wiring consumed by the Examiner agent and assessment UI.
|
||||
domain: ai-media
|
||||
frameworks:
|
||||
- pydantic
|
||||
- httpx
|
||||
- pytest
|
||||
- web-mediarecorder
|
||||
constraints:
|
||||
- provider-agnostic-protocol (D-030)
|
||||
- never-call-voice-api-in-tests
|
||||
- browser-native-fallback (no-key path still functions)
|
||||
- bounded-turn-latency (conversational feel budget)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/voice/**"
|
||||
- "apps/ai-service/tests/voice/**"
|
||||
- "apps/cli/**"
|
||||
- "scripts/install.sh"
|
||||
- "scripts/release-assets.sh"
|
||||
```
|
||||
|
||||
### design-system-engineer
|
||||
```yaml
|
||||
active: true
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: Owns the shared component library, design tokens, and visual consistency. v0.3 duty: new primitives for the real build/assessment surfaces (terminal frame, telemetry status indicator, mic/record control, grade badge, defense transcript viewer).
|
||||
reason: No design-token or primitive work planned in v0.5 (existing primitives — MicControl, GradeBadge, TranscriptViewer — cover the voice surfaces); roster retained.
|
||||
domain: frontend
|
||||
frameworks:
|
||||
- tailwindcss
|
||||
- storybook
|
||||
- lucide-react
|
||||
constraints:
|
||||
- design-token-driven
|
||||
- wcag-aa-contrast
|
||||
- dark-mode-required
|
||||
- consistent-across-surfaces
|
||||
territory:
|
||||
- "packages/ui/**"
|
||||
```
|
||||
|
||||
### security-auditor
|
||||
### data-engineer
|
||||
```yaml
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: Identity/age-gating (KYC) deferred beyond v0.3 per founder directive (A-110) — no real auth or PII backend lands this milestone. Security coverage remains: verifier's STRIDE layer + Phase 7 secrets-hygiene checklist (keys absent from code/logs/commits/errors, localhost-only CORS, no PII in prompts). Sandbox isolation safety is owned by sandbox-engineer's probe-verified constraint.
|
||||
domain: security
|
||||
frameworks: []
|
||||
constraints: []
|
||||
territory: []
|
||||
reason: Light-touch via frontend-engineer territory (variants/identity TS type extensions); no schema/mock-data work beyond the two typed additions.
|
||||
domain: data
|
||||
frameworks:
|
||||
- typescript
|
||||
constraints:
|
||||
- schema-first
|
||||
- type-safe
|
||||
- dual-schema-sync (TS/Python changes made in both places)
|
||||
territory:
|
||||
- "packages/types/**"
|
||||
- "packages/mock-data/**"
|
||||
```
|
||||
|
||||
## Phase-Specific Personas
|
||||
|
||||
| Persona | Phases | Removed After |
|
||||
|---------|--------|---------------|
|
||||
| sandbox-engineer | 1 (primary), 2, 6 | persists while sandbox fabric exists |
|
||||
| voice-engineer | 5 (primary), 6 | persists while voice defense exists |
|
||||
| security-auditor | 1 (seq-ack/agent), 2 (audio upload), 3 (identity PII), 4 (exec policy), 5 (final review) | milestone complete |
|
||||
|
||||
All other active personas span the entire milestone. data-engineer and design-system-engineer are light-touch outside their phases.
|
||||
All other personas span the milestone. Deactivated personas receive no tasks.
|
||||
|
||||
## Territory Conflict Resolution
|
||||
|
||||
| Conflict | Resolution |
|
||||
|----------|------------|
|
||||
| frontend-engineer vs data-engineer (packages/types, packages/mock-data) | data-engineer owns type definitions and mock data schema (incl. new telemetry/grade/variant/defense TS types); frontend-engineer consumes them. |
|
||||
| frontend-engineer vs design-system-engineer (packages/ui) | design-system-engineer owns design tokens and primitive components (terminal frame, mic control, grade badge); frontend-engineer owns composite components and page-level UI. |
|
||||
| ai-engineer vs backend-engineer (grading/variants) | ai-engineer owns the model-facing files (features/engine/templates/generator = LLM logic + prompts); backend-engineer owns the persistence stores + API endpoints. Boundary: stores are pure SQLite; engine logic is pure compute. |
|
||||
| sandbox-engineer vs backend-engineer (sandbox/) | sandbox-engineer owns `ai_service/sandbox/**` + capture agent; backend-engineer owns the API route that composes `sandbox/manager.py` via DI. manager.py has a narrow typed interface consumed by api/. |
|
||||
| voice-engineer vs ai-engineer (Examiner agent) | ai-engineer owns `agents/examiner.py` + its prompt; voice-engineer owns `voice/**` (audio in/out). Examiner calls `voice/` through the `VoiceProvider` protocol — never imports concrete providers. |
|
||||
| ai-engineer vs data-engineer (mock duplication) | ai-engineer owns `ai_service/corpus/` (Python); data-engineer owns `packages/mock-data` (TS). Shared IDs/shapes aligned by convention (D-021). |
|
||||
| lead-developer vs any | lead-developer coordinates only, does not directly modify code files. |
|
||||
| voice-engineer vs backend-engineer (factory.py) | backend-engineer owns factory.py settings wiring + signature; voice-engineer owns openai_audio.py + its factory branch (provider construction), consumes config via DI |
|
||||
| sandbox-engineer vs frontend-engineer (build-surface) | sandbox-engineer owns engine-side kinds/templates/policy; frontend-engineer owns the web surface + client session hook; wire contract = VariantResponse TS types |
|
||||
| identity-engineer vs sandbox-engineer (gates) | identity-engineer owns the IdentityGate dependencies; sandbox-engineer owns the sandbox route they mount on (gate order D-043 is a joint review) |
|
||||
| security-auditor vs identity-engineer | identity-engineer implements; security-auditor reviews + may patch security defects directly in identity/ + api/defense.py (its territory) |
|
||||
| lead-developer vs any | lead-developer coordinates only, does not directly modify code files |
|
||||
+165
-421
@@ -1,438 +1,182 @@
|
||||
# Nextcraft v0.3 — PLAN.md
|
||||
# Nextcraft v0.5 — PLAN.md
|
||||
|
||||
## Overview
|
||||
|
||||
This plan covers execution phases 1-6 of milestone v0.3 (Credential Engines): the real engines that replace v0.2's mock inputs — a namespace-isolated sandbox fabric, live build telemetry over WebSocket + SQLite, a process-trace grading engine, seeded per-learner variant task generation, and an oral/voice defense with a seventh Examiner agent — plus re-grounding the Lab/Assessor/Proctor agents onto real engine inputs and wiring the v0.1 learner surfaces to the real build/defense/grading paths. Phases are strictly sequential (P1→P6); within each phase, Wave 1 tasks are parallelizable (no cross-file dependencies) and later waves depend on earlier ones.
|
||||
**Milestone:** v0.5 — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease
|
||||
**Tag line:** v0.4.x patches (P0 → v0.4.0 … P5 → v0.4.5 = milestone release)
|
||||
**Branch:** milestone/v0.5-real-voice-identity-envs
|
||||
|
||||
**Environment facts (apply throughout):** Python 3.11.2 via `python3 -m venv` (no uv, no system pip); pnpm 12.3.4 via corepack; turborepo; ai-service port **8420**; default model `gemma4:31b` (config via `AI_TUTOR_MODEL`); ollama-cloud base `https://ollama.com/v1` (OpenAI-compatible, Bearer auth); keys live only in gitignored `.ciagent/.env.secrets` (exported by `scripts/dev.sh`) — never in code, commits, or logs; all automated tests use the deterministic mock LLM and mock voice providers and **never call cloud or voice APIs**. New ai-service deps this milestone: `sqlmodel`, `sqlalchemy`, `websockets`, `aiofiles` (all PyPI-verified; added in Task 1-1-02). SQLite data (`apps/ai-service/ai_service/data/`) and sandbox dirs (`apps/ai-service/sandboxes/`) are already gitignored. **KYC/age-gating is deferred per founder directive (A-110)** — no identity work; age-gating stays the v0.1 visual flow mockup. **GRILL scope decisions (this revision):** real server STT/TTS (`OpenAIAudioProvider`) deferred to v0.4 — voice defense is mock+browser-first (CUT-1/G-7); the interactive xterm.js shell relay is deferred to v0.4 — the build panel is run/test-buttons + read-only exec output (CUT-2/G-8), so `@xterm/*` is NOT a v0.3 dependency; sandbox abuse control (per-learner caps + allowlist, G-5) ships even though KYC is deferred; sandbox resource limits are partially enforced (memory/CPU/wall-clock + workdir-size sweep; per-sandbox pids and hard disk quota are accepted gaps, G-1/G-2).
|
||||
**Environment facts (probe-verified, apply throughout):** python 3.11.2 (venv at apps/ai-service/.venv), node v24.15.0, pnpm 12.3.4, no docker/podman/sudo, `unshare` userns verified working, SQLite is the only persistence (D-027), tests NEVER call the cloud (mock providers + httpx MockTransport), secrets only in gitignored `.ciagent/.env.secrets` / `.env*` (D-006/D-014). Runtime state lives in `~/.nextcraft/` (D-039) — tests pin to tmp dirs; sandbox workdir fixtures use the bind-mount-safe `sandbox_dir` fixture (overlayfs /tmp breaks userns binds).
|
||||
|
||||
| Phase | Name | Requirements | Waves | Personas |
|
||||
|-------|------|-------------|-------|----------|
|
||||
| 1 | Sandbox fabric | REQ-3-001, 002 | 3 | sandbox-engineer, backend-engineer, ai-engineer (W1 lint only) |
|
||||
| 2 | Live build telemetry | REQ-3-003 | 4 | sandbox-engineer, backend-engineer, data-engineer |
|
||||
| 3 | Process-trace grading engine | REQ-3-004 | 3 | ai-engineer, backend-engineer |
|
||||
| 4 | Variant task generation | REQ-3-005 | 3 | ai-engineer, backend-engineer, data-engineer |
|
||||
| 5 | Oral / voice defense | REQ-3-006 | 4 | voice-engineer, ai-engineer, backend-engineer |
|
||||
| 6 | Agent re-grounding + learner surface integration | REQ-3-007, 008 | 5 | ai-engineer, frontend-engineer, design-system-engineer, data-engineer, backend-engineer, lead-developer |
|
||||
**Wave order (D-046, binding):** P1 seq-lease → P2 voice → P3 identity → P4 environments → P5 final. Seq-lease lands first: it fixes the transport before environment phases add reconnecting telemetry producers; it touches no stores, no web, no settings.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Sandbox Fabric
|
||||
|
||||
**Requirements:** REQ-3-001, REQ-3-002
|
||||
**Goal:** `SandboxBackend` protocol + `unshare`-based namespace spawner (D-024) + lifecycle manager with concurrency guard (D-032) + per-sandbox workdir; isolation and resources probe-verified on this box; `/v1/sandboxes` API live; deps + gitignore landed
|
||||
|
||||
### Wave 1: Foundations (parallel — no shared files)
|
||||
|
||||
#### Task 1-1-01: SandboxBackend protocol + unshare spawner + probe test
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-001, REQ-3-002
|
||||
- **Files:** `apps/ai-service/ai_service/sandbox/__init__.py`, `apps/ai-service/ai_service/sandbox/backend.py`, `apps/ai-service/ai_service/sandbox/workdir.py`, `apps/ai-service/ai_service/sandbox/unshare_backend.py`, `apps/ai-service/tests/sandbox/__init__.py`, `apps/ai-service/tests/sandbox/test_isolation.py`
|
||||
- **Action:** `backend.py`: `SandboxBackend` protocol + `SandboxSpec` (sandbox_id, learner_id, workdir, resource limits) + `SandboxHandle` (id, pid, workdir, created_at); `spawn(spec)`, `exec(handle, cmd)`, `snapshot(handle) -> Path`, `destroy(handle)`. `workdir.py`: per-sandbox layout under `apps/ai-service/sandboxes/<id>/` (workspace/ writable, snapshot() = recursive copy to `snapshots/<ts>/`) — no symlinks as the snapshot mechanism. `unshare_backend.py`: subprocess spawner — `unshare --user --map-root-user --mount --pid --fork --net` with the per-sandbox dir bind-mounted (`--bind <dir> /work`) and `chdir /work` (D-024); pipes for stdout/stderr; async wrappers. `test_isolation.py` — **re-verify box isolation properties (runs on this box, guarded by probe skip):** (a) `id -u` inside namespace prints `0`; (b) `ip link` inside namespace shows 0 usable interfaces (loopback-only/no carrier) — network isolated; (c) file written to `/work/inside.txt` lands at `sandboxes/<id>/workspace/inside.txt` on the host; (d) attempt to write outside the mount (e.g. host tmp path via bind) does not escape the per-sandbox dir; (e) `/proc` visibility degraded (proc-remount not permitted per A-101 — assert the probe documents this, not that it fails).
|
||||
- **Verify:** `pnpm ai:test` — `tests/sandbox/test_isolation.py` green on this box (probe-gated: skips with an explicit reason if userns unavailable); `lint` clean
|
||||
|
||||
#### Task 1-1-02: v0.3 dependencies + gitignore + config additions
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-001
|
||||
- **Files:** `apps/ai-service/pyproject.toml` (update), `apps/ai-service/ai_service/config.py` (update), `apps/ai-service/.env.example` (update), `apps/ai-service/scripts/bootstrap.sh` (update if needed), root `package.json` (no change), `turbo.json` (no change)
|
||||
- **Action:** Add pinned deps: `sqlmodel`, `sqlalchemy`, `websockets`, `aiofiles` to pyproject. `config.py` additions (env_prefix `AI_`): `AI_SANDBOX_DIR` (default `apps/ai-service/sandboxes`), `AI_SANDBOX_MAX_CONCURRENT` (default 5, D-032), `AI_SANDBOX_CPU_LIMIT` (default 1 core / cpu.max), `AI_SANDBOX_MEM_LIMIT_MB` (default 512), `AI_SANDBOX_PIDS_LIMIT` (default 256), `AI_SANDBOX_TIMEOUT_S` (default 1800), `AI_DB_PATH` (default `ai_service/data/nextcraft.db`), `AI_VOICE_BASE_URL` / `AI_VOICE_API_KEY` / `AI_VOICE_STT_MODEL` / `AI_VOICE_TTS_MODEL` (all **optional**, default empty — mock-first, D-030; documented in `.env.example` and README). Confirm `.gitignore` already covers `ai_service/data/` + `sandboxes/` (it does — v0.3 block present). Re-run bootstrap idempotently to install new deps.
|
||||
- **Verify:** `pnpm ai:bootstrap` re-installs cleanly (no-op venv, new wheels land); `python -c "import sqlmodel, sqlalchemy, websockets, aiofiles"` succeeds in the venv; settings parse with new keys unset
|
||||
|
||||
#### Task 1-1-03: Resource-limit probe documentation + harness ruff pass
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-001
|
||||
- **Files:** `apps/ai-service/README.md` (update: sandbox section + probe transcript), `apps/ai-service/pyproject.toml` (no change), `apps/ai-service/tests/sandbox/test_isolation.py` (no change)
|
||||
- **Action:** Record the A-101 probe transcript in README (verbatim commands + observed output from this box: `unshare --user --map-root-user --mount --pid --fork --net id -u` → `0`; `ip link` → loopback only; write containment). Document the v0.3 resource-limit mechanism choice: cgroup-v2 delegation via per-sandbox scope files is **not available** on this box without sudo → enforcement = subprocess-level (`ulimit`-equivalent via `preexec_fn`: RLIMIT_AS for memory, RLIMIT_CPU for CPU-seconds, RLIMIT_NPROC for pids) + hard wall-clock timeout kill in the manager. This is the locked v0.3 mechanism (D-024 + no-sudo constraint). Run ruff over the new tree; fix all findings.
|
||||
- **Verify:** `pnpm ai:lint` exits 0; README shows the probe transcript and the rlimit mechanism note
|
||||
|
||||
### Wave 2: Lifecycle manager (depends on Wave 1)
|
||||
|
||||
#### Task 1-2-01: Sandbox manager + concurrency guard + snapshots
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-001, REQ-3-002
|
||||
- **Files:** `apps/ai-service/ai_service/sandbox/manager.py`, `apps/ai-service/tests/sandbox/test_manager.py`
|
||||
- **Action:** `SandboxManager`: `create(learner_id) -> SandboxHandle` (guard: active count ≥ `AI_SANDBOX_MAX_CONCURRENT` → raise `PoolFullError` → API maps to **503**, D-032; no queue); `list() -> list[SandboxHandle]`; `get(id)`; `snapshot(id) -> Path` (delegates to workdir); `destroy(id)` (kill process tree, keep or purge workdir per flag); `reap_expired()` background hook for `AI_SANDBOX_TIMEOUT_S` which also performs a **workdir-size sweep**: any sandbox whose `workdir` exceeds `AI_SANDBOX_MAX_WORKDIR_MB` (new config, default 512) is snapshotted-then-destroyed and the event logged as an integrity signal (G-2 — soft disk cap, best-effort, not kernel-enforced); the sweep runs on the same timer as the timeout reaper. Enforce rlimits per spawner (Task 1-1-03: RLIMIT_AS + RLIMIT_CPU + RLIMIT_FSIZE=50MB as a cheap single-file disk guard (a-2); RLIMIT_NPROC noted as shared-per-host-uid, not relied on (G-1)) at exec time. Handle registry persisted **in-memory** (v0.3, single process; not a store — see D-019 precedent) with a clear note that handles are process-local. Startup reaper (a-1): on lifespan boot, scan `AI_SANDBOX_DIR` for workdirs whose recorded pid is dead and reap them, logging a warning. Narrow typed interface only — manager never imports api/ (boundary rule).
|
||||
- **Verify:** `pnpm ai:test` — test_manager covers create/list/destroy/snapshot, 6th create raises PoolFullError (503 path), destroy kills the namespace process (pid gone), snapshot dir exists with workspace contents, timeout reaper removes a stale handle
|
||||
|
||||
#### Task 1-2-02: Resource-limit enforcement test
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-002
|
||||
- **Files:** `apps/ai-service/tests/sandbox/test_resource_limits.py`
|
||||
- **Action:** Concrete enforcement probes (guarded like isolation tests): (a) spawn a process that allocates > `RLIMIT_AS` → assert it dies with MemoryError/killed within a bound; (b) spawn a CPU spinner past `RLIMIT_CPU` → assert SIGXCPU/kill; (c) single huge file > `RLIMIT_FSIZE` → assert write failure (a-2 partial disk guard); (d) wall-clock: spawn `sleep 9999` with a small manager timeout → reaper destroys it; (e) **disk sweep (G-2)**: write > `AI_SANDBOX_MAX_WORKDIR_MB` across many files → assert the manager sweep destroys the sandbox and logs the integrity signal. Assert limits are observable (handle reports its limit set). NOTE (G-1): per-sandbox `RLIMIT_NPROC` is shared at the host uid — the fork-bomb probe is documented as shared-budget behavior, NOT asserted as per-sandbox isolation.
|
||||
- **Verify:** `pnpm ai:test` — test_resource_limits green; limits proven enforced and observable
|
||||
|
||||
### Wave 3: API exposure (depends on Wave 2)
|
||||
|
||||
#### Task 1-3-01: Sandboxes API module
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-001, REQ-3-002
|
||||
- **Files:** `apps/ai-service/ai_service/api/sandboxes.py`, `apps/ai-service/ai_service/main.py` (update: include router + lifespan manager), `apps/ai-service/ai_service/api/deps.py` (update), `apps/ai-service/tests/api/test_sandboxes.py`
|
||||
- **Action:** DI exposes a singleton `SandboxManager`. Endpoints: `POST /v1/sandboxes {learner_id}` → 201 handle (503 when pool full); `GET /v1/sandboxes` → list; `GET /v1/sandboxes/{id}` → handle; `POST /v1/sandboxes/{id}/snapshot` → snapshot path; `DELETE /v1/sandboxes/{id}` → 204. All behind localhost CORS (A-008). Lifespan creates/destroys the manager; on shutdown destroys any live sandboxes (no orphans). **Abuse control (G-5, NOT KYC):** even in no-auth v0.3 the sandbox API enforces per-`learner_id` rate limiting (`AI_SANDBOX_MAX_PER_LEARNER`, default 1 active → 429) and a global create-rate cap (`AI_SANDBOX_CREATES_PER_MIN`, default 10 → 429); `learner_id` is validated against a server-side allowlist from config (`AI_LEARNER_ALLOWLIST`, default the single mock pilot id → unknown ids rejected 403). This ships in the no-auth milestone so a rogue local process can't exhaust shared NPROC/disk.
|
||||
- **Verify:** `pnpm ai:test` — test_sandboxes green (create→list→snapshot→delete roundtrip via TestClient; 6th create → 503; delete of unknown id → 404; abuse control (G-5): non-allowlisted learner_id → 403; >1 active sandbox for one learner → 429; burst of >10 creates/min → 429); manual probe: `curl -X POST localhost:8420/v1/sandboxes -d '{"learner_id":"l1"}'` returns a handle id
|
||||
|
||||
### Must-Haves (Phase 1)
|
||||
- [ ] Isolation probe test green on this box: in-namespace uid=0, network isolated (0 usable interfaces), host writes confined to the per-sandbox bind dir (A-101 re-verified as an automated test, not just research notes)
|
||||
- [ ] Resource limits enforced + observable: memory (RLIMIT_AS) + CPU (RLIMIT_CPU) rlimits kill violating processes; wall-clock reaper destroys stale sandboxes; disk usage capped by a periodic workdir-size sweep in the manager (soft cap, configurable `AI_SANDBOX_MAX_WORKDIR_MB`, default 512MB — NOT kernel-enforced); per-sandbox NPROC is shared across sandboxes at the host uid — documented, not relied on for isolation (G-1, G-2 — test_resource_limits green)
|
||||
- [ ] No cross-tenant access: sandbox A cannot read sandbox B's workdir (isolation test asserts containment)
|
||||
- [ ] Lifecycle API works end-to-end: create/list/snapshot/destroy via TestClient; pool full → **503** (D-032, no queue)
|
||||
- [ ] Snapshot produces a restorable directory copy under the sandbox's own snapshots/ dir
|
||||
- [ ] `pnpm ai:test` and `pnpm ai:lint` green; new deps (`sqlmodel`, `sqlalchemy`, `websockets`, `aiofiles`) installed via idempotent bootstrap
|
||||
- [ ] Boundary rules hold: `sandbox/` imports nothing from `api/` or `agents/`; only `api/sandboxes.py` composes the manager via DI
|
||||
- [ ] No docker/podman/sudo anywhere in the spawner path (D-024); `SandboxBackend` protocol is the only coupling to the spawner (containerd swap possible later)
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Live Build Telemetry
|
||||
|
||||
**Requirements:** REQ-3-003
|
||||
**Goal:** First real persistence (D-027: SQLite via SQLModel) with a `TraceStore` protocol; `TelemetryEvent` model with per-(learner,task) monotonic `seq`; WebSocket ingest endpoint (D-026) with gap detection; stdlib-only in-sandbox capture agent streams real sandbox activity into ai-service; trace retrievable by learner+task
|
||||
|
||||
### Wave 1: Models + stores + TS types (parallel — no shared files)
|
||||
|
||||
#### Task 2-1-01: Telemetry event models
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/telemetry/__init__.py`, `apps/ai-service/ai_service/telemetry/models.py`, `apps/ai-service/tests/telemetry/__init__.py`, `apps/ai-service/tests/telemetry/test_models.py`
|
||||
- **Action:** Pydantic/SQLModel `TelemetryEvent`: `learner_id`, `task_id`, `seq` (int, monotonic per (learner,task)), `kind` (`command` | `file_diff` | `run_result` | `test_result` | `activity` | `stdin` | `stdout`), `payload` (JSON), `ts` (datetime, monotonic-envelope), `sandbox_id`. `TraceSpan` derived view (ordered events for one (learner,task)). Validation: seq ≥ 0, kind enum, non-empty ids.
|
||||
- **Verify:** `pnpm ai:test` — test_models green (validation rules enforced, JSON payload roundtrip)
|
||||
|
||||
#### Task 2-1-02: TraceStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/telemetry/store.py`, `apps/ai-service/tests/telemetry/test_store.py`, `apps/ai-service/ai_service/data/.gitkeep`
|
||||
- **Action:** `TraceStore` protocol (D-027, Postgres-migration-ready): `append(event) -> None` (idempotent on (learner,task,seq) — at-least-once dedup), `get_trace(learner_id, task_id) -> list[TelemetryEvent]` (ordered by seq), `gaps(learner_id, task_id) -> list[int]` (missing seqs), `latest_seq(learner_id, task_id) -> int`, `list_tasks(learner_id) -> list[str]`, `close()`. `SQLiteTraceStore(SQLModel)`: single `telemetry_event` table, composite PK ((learner_id, task_id, seq)), indexes on (learner_id, task_id). Engine creation from `AI_DB_PATH`; `SQLModel.metadata.create_all` at app lifespan. Enable `PRAGMA journal_mode=WAL` + `synchronous=NORMAL` at engine creation (a-3) so concurrent ingest (writer) and trace reads (grader) don't hit `database is locked` under concurrent sandboxes.
|
||||
- **Verify:** `pnpm ai:test` — test_store green (append/ordered-get/dedup-on-retry/gap detection/latest_seq; tmp-path SQLite per test)
|
||||
|
||||
#### Task 2-1-03: TS types for telemetry/traces
|
||||
- **Persona:** data-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `packages/types/telemetry.ts` (new), `packages/types/index.ts` (update)
|
||||
- **Action:** TS `TelemetryEvent`, `TraceSpan`, `TelemetryKind` mirroring the Python model field-for-field (cross-referencing header, same string enums). Consumed by Phase 6 web surfaces; no runtime code.
|
||||
- **Verify:** `pnpm typecheck` passes; TS type keys match Python model keys exactly
|
||||
|
||||
### Wave 2: Capture agent + ingest (depends on Wave 1)
|
||||
|
||||
#### Task 2-2-01: In-sandbox capture agent (stdlib-only)
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/scripts/sandbox-agent.py`, `apps/ai-service/tests/sandbox/test_sandbox_agent.py`
|
||||
- **Action:** Tiny standalone process (D-031, **stdlib only** — no deps shipped into the namespace): wraps a shell inside the sandbox; captures commands, file diffs (mtime/content polling of `workspace/` at 250ms), run/test results, activity; assigns per-(learner,task) `seq`; buffers to a local spool file on disconnect (at-least-once, D-026); reconnects with **exponential backoff** and flushes spool in order; small WebSocket client implemented over raw `socket` (RFC6455 client handshake + frames — stdlib only, no `websockets` in-namespace). Configured via env baked at spawn (`NC_LEARNER_ID`, `NC_TASK_ID`, `NC_INGEST_URL`).
|
||||
- **Verify:** `pnpm ai:test` — unit tests with a loopback fake WS server: ordered seq emission, spool-on-disconnect, reconnect flush preserves order (no loss, dupes deduped server-side), no third-party imports in the file (asserted by AST scan)
|
||||
|
||||
#### Task 2-2-02: WebSocket ingest endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/telemetry/ingest.py`, `apps/ai-service/ai_service/api/sandboxes.py` (update: register WS route), `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/tests/api/test_telemetry_ingest.py`
|
||||
- **Action:** `WS /v1/telemetry/ingest` (D-026): accepts connections carrying learner_id/task_id/sandbox_id; validates + appends events to `TraceStore` (idempotent — server-side dedup on (learner,task,seq)); emits **gap warnings** when seq skips (logged + surfaced in a per-connection status); ping/pong keepalive. **Backpressure / flood control (G-3 — replaces silent drop):** bounded inbound queue; on overflow OR when a per-connection cap `AI_TELEMETRY_MAX_EVENTS_PER_TASK` (default 50000) is exceeded → **reject with a 1008 policy-violation close and mark the (learner,task) trace `INCOMPLETE_FLOODED`** (an integrity signal consumed by Proctor). Silent drop-oldest is FORBIDDEN because it corrupts grading input and is indistinguishable from trace-gaming. `GET /v1/telemetry/traces/{learner_id}/{task_id}` returns the ordered trace; `GET /v1/telemetry/gaps/{learner_id}/{task_id}` returns missing seqs.
|
||||
- **Verify:** `pnpm ai:test` — test_telemetry_ingest green (TestClient websocket: connect → send 3 events → trace retrievable ordered; resend event 2 → deduped; skip seq 5 → gap reported; unknown sandbox tolerated in v0.3 no-auth mode)
|
||||
|
||||
### Wave 3: Sandbox telemetry wiring (depends on Wave 2)
|
||||
|
||||
#### Task 2-3-01: Spawn sandboxes with the capture agent
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/sandbox/manager.py` (update), `apps/ai-service/ai_service/sandbox/unshare_backend.py` (update), `apps/ai-service/tests/sandbox/test_telemetry_wiring.py`
|
||||
- **Action:** `create()` gains optional `task_id`; when set the spawner copies `scripts/sandbox-agent.py` into the sandbox workdir, injects `NC_*` env, and launches the agent as a child of the namespace process (agent lifecycle tied to sandbox lifecycle; destroy kills the agent). No capture when task_id absent (pure shell sandbox).
|
||||
- **Verify:** `pnpm ai:test` — end-to-end on this box: create sandbox with task_id → run 2 commands via exec → events arrive at the ingest endpoint and land in SQLite in order
|
||||
|
||||
### Wave 4: Reliability probe (depends on Wave 3)
|
||||
|
||||
#### Task 2-4-01: Dropped-connection durability probe
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/tests/telemetry/test_durability.py`
|
||||
- **Action:** Integration probe: run a capture agent against ingest, kill the WS connection mid-stream (simulate network failure), keep generating events, reconnect, assert the SQLite trace contains **every** event exactly once in order (spool + dedup). Document at-least-once semantics + replay path in README.
|
||||
- **Verify:** `pnpm ai:test` — test_durability green; README documents semantics
|
||||
|
||||
### Must-Haves (Phase 2)
|
||||
- [ ] Real telemetry from a live sandbox arrives at ai-service: shell commands, file diffs, run/test results appear as ordered events in SQLite (end-to-end, no mocks)
|
||||
- [ ] Per-(learner,task) monotonic `seq`; gap detection reports missing seqs; replay yields the complete ordered trace
|
||||
- [ ] At-least-once proven: transient disconnect + reconnect loses no events; duplicates deduped server-side (durability probe green)
|
||||
- [ ] Capture agent is stdlib-only (AST-verified) and its lifecycle is tied to the sandbox (destroy kills it)
|
||||
- [ ] Trace retrievable by learner+task via `GET /v1/telemetry/traces/...`; unknown trace → 404
|
||||
- [ ] `TraceStore` protocol respected: no api/ code touches SQLite directly; `telemetry/` never imports `agents/` (D-027)
|
||||
- [ ] Flood control (G-3): burst past `AI_TELEMETRY_MAX_EVENTS_PER_TASK` → connection closed 1008 + trace marked `INCOMPLETE_FLOODED`; no silent event drop on overflow
|
||||
- [ ] `pnpm ai:test` green; `packages/types` telemetry TS types compile (`pnpm typecheck`)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Process-Trace Grading Engine
|
||||
|
||||
**Requirements:** REQ-3-004
|
||||
**Goal:** Deterministic feature computation over traces (D-028) → compact digest → LLM rubric scoring via existing D-020 JSON defense → validated structured scores stored in `GradeStore`; LLM never sees the raw trace; engine calibrated against v0.2 mock corpora so process quality separates paste-and-run from iterative debugging
|
||||
|
||||
### Wave 1: Features + grades store (parallel — no shared files)
|
||||
|
||||
#### Task 3-1-01: Deterministic trace digest (features)
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/grading/__init__.py`, `apps/ai-service/ai_service/grading/features.py`, `apps/ai-service/tests/grading/__init__.py`, `apps/ai-service/tests/grading/test_features.py`
|
||||
- **Action:** Pure compute module (D-028): `compute_digest(trace: list[TelemetryEvent]) -> TraceDigest`. Deterministic features: test pass/fail counts + final status; edit count; error/fix cycle count + mean fix latency; idle gaps (>Ns, count + total); command category histogram (build/test/file/nav/debug/other); session duration; first-test-pass offset. `TraceDigest` pydantic model — compact (bounded size, no raw commands), LLM-safe.
|
||||
- **Verify:** `pnpm ai:test` — test_features green over synthetic traces: paste-and-run trace (0 error/fix cycles, single test pass at end) vs iterative trace (many cycles) produce observably different digests
|
||||
|
||||
#### Task 3-1-02: GradeStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/grading/store.py`, `apps/ai-service/tests/grading/test_store.py`
|
||||
- **Action:** `GradeStore` protocol (D-027): `save(grade)`, `get(learner_id, task_id)`, `list_for_learner(learner_id)`, `close()`. SQLModel `GradeRecord`: learner_id, task_id, variant_seed (null until P4), digest (JSON), scores (JSON), verdict, model, created_at. PK (learner_id, task_id). Postgres-migration-ready.
|
||||
- **Verify:** `pnpm ai:test` — test_store green (save/get/list roundtrip, overwrite-on-regrade documented)
|
||||
|
||||
### Wave 2: Grading engine + calibration (depends on Wave 1)
|
||||
|
||||
#### Task 3-2-01: Rubric scoring engine
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/grading/engine.py`, `apps/ai-service/ai_service/prompts/grading.py` (new), `apps/ai-service/tests/grading/test_engine.py`
|
||||
- **Action:** `GradingEngine.grade(learner_id, task_id) -> GradeRecord`: **trace-completeness gate (G-4)** — first call `TraceStore.gaps()` + check the trace `INCOMPLETE_FLOODED` flag; if gaps are non-empty OR the trace is flagged incomplete → return `verdict=UNGRADABLE_TRACE_INCOMPLETE` (a first-class verdict, not an exception) surfacing the gap list; a credential is NEVER issued from a gapped/incomplete trace. Otherwise: load trace via `TraceStore` → `compute_digest` → render rubric prompt (`prompts/grading.py`: criteria + level anchors for process quality, correctness, debugging discipline, test usage; a-4: treat high edit/command churn with no test-progress as a process-quality negative) → LLM structured output through the **existing D-020 4-layer defense** (`agents/structured.py` reused — engine composes it, never duplicates it) → validate `RubricScore` model (per-criterion 0-4 + strengths + gaps + verdict) → persist via `GradeStore`. Grading depends on `llm/` + `telemetry/` + `prompts/` only (boundary). Mock provider scripts deterministic rubric JSON for tests, including the INCOMPLETE path.
|
||||
- **Verify:** `pnpm ai:test` — test_engine green (mock provider: digest-only prompt asserted — **raw trace string absent from prompt**; validated scores returned; malformed JSON exercises D-020 retry; unknown trace → error)
|
||||
|
||||
#### Task 3-2-02: Calibration against v0.2 mock corpora
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/trace_fixtures.py` (new), `apps/ai-service/tests/grading/test_calibration.py`
|
||||
- **Action:** Synthetic trace fixtures aligned with v0.2 `corpus/telemetry.py` + `corpus/artifacts.py` scenario IDs (strong/lazy/struggling builder archetypes). Assert grading separates them: strong archetype scores ≥ lazy archetype on process-quality criterion (mock provider maps digest shape → scripted scores; test asserts the ordering contract + that fixture IDs align with existing corpus IDs, D-021).
|
||||
- **Verify:** `pnpm ai:test` — test_calibration enforces the ordering contract
|
||||
|
||||
### Wave 3: Grading endpoint (depends on Wave 2)
|
||||
|
||||
#### Task 3-3-01: Assessment grade endpoint (real traces)
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/api/assessment.py` (update), `apps/ai-service/tests/api/test_grading.py`
|
||||
- **Action:** `POST /v1/assessment/grade {learner_id, task_id}` → `GradingEngine` → validated `RubricScore` JSON; `GET /v1/assessment/grade/{learner_id}/{task_id}` → stored grade; unknown trace → 404. Composed via DI (api/ owns wiring; engine knows nothing of FastAPI).
|
||||
- **Verify:** `pnpm ai:test` — test_grading green (grade roundtrip via TestClient with mock provider; 404 on unknown; GET after POST returns same scores)
|
||||
|
||||
### Must-Haves (Phase 3)
|
||||
- [ ] Engine emits structured rubric-aligned scores from a **real process trace** (not pre-baked input) — TestClient roundtrip green
|
||||
- [ ] Deterministic features computed in code (test pass/fail, edit count, error/fix cycles, idle gaps, command categories); LLM receives the **digest only** — test asserts the raw trace never reaches the prompt (D-028)
|
||||
- [ ] Scores distinguish process quality: iterative-debugging archetype out-scores paste-and-run on the process criterion (calibration contract test)
|
||||
- [ ] Grades persisted + retrievable by learner+task via GradeStore (SQLite, protocol-wrapped, D-027)
|
||||
- [ ] Boundary rules hold: `grading/` imports no `api/`/`agents/` internals except the shared D-020 structured defense; `pnpm ai:test` + `pnpm ai:lint` green
|
||||
- [ ] Incomplete-trace gate (G-4): gapped or `INCOMPLETE_FLOODED` trace → `verdict=UNGRADABLE_TRACE_INCOMPLETE` with the gap list; no credential issued from an incomplete trace (test green)
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Variant Task Generation
|
||||
|
||||
**Requirements:** REQ-3-005
|
||||
**Goal:** Template library with typed parameter slots (D-029) + seeded LLM instantiation + per-learner variant registry (SQLite) with difficulty-normalization anchors; two learners on the same competency get provably distinct, reproducible, auditable tasks
|
||||
|
||||
### Wave 1: Templates + store (parallel — no shared files)
|
||||
|
||||
#### Task 4-1-01: Task template library
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/variants/__init__.py`, `apps/ai-service/ai_service/variants/templates.py`, `apps/ai-service/tests/variants/__init__.py`, `apps/ai-service/tests/variants/test_templates.py`
|
||||
- **Action:** ≥3 initial task templates bound to existing competency IDs (D-021 alignment). `TaskTemplate`: id, competency_id, statement skeleton with `{slot}` placeholders, `ParameterSlot[]` (name, type: enum/int-range/string-set, allowed values), `rubric anchors` (difficulty normalization: expected feature envelope — e.g. expected edit-count band — used by grading context), starter-file scaffolds served to the sandbox. Seeded slot sampler is pure code (`random.Random(seed)`), fully reproducible.
|
||||
- **Verify:** `pnpm ai:test` — test_templates green (slot validation: bad value rejected; seeded sampling reproducible across runs; all templates bind to real competency IDs)
|
||||
|
||||
#### Task 4-1-02: VariantStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/variants/store.py`, `apps/ai-service/tests/variants/test_store.py`
|
||||
- **Action:** `VariantStore` protocol (D-027): `save(variant)`, `get(learner_id, template_id_or_task_id)`, `list_for_learner(learner_id)`, `list_by_template(template_id)`, `close()`. SQLModel `VariantRecord`: learner_id, task_id (the grading/telemetry task key), template_id, seed, params (JSON), statement (rendered), created_at. Unique (learner_id, template_id).
|
||||
- **Verify:** `pnpm ai:test` — test_store green (roundtrip, unique constraint, audit listing)
|
||||
|
||||
### Wave 2: Generator (depends on Wave 1)
|
||||
|
||||
#### Task 4-2-01: Seeded LLM variant generator
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/variants/generator.py`, `apps/ai-service/ai_service/prompts/variant.py` (new), `apps/ai-service/tests/variants/test_generator.py`
|
||||
- **Action:** `VariantGenerator.generate(learner_id, template_id) -> VariantRecord`: derive seed (`sha256(template_id|learner_id|milestone)` — reproducible, D-029); sample typed slots in code; render a fill prompt (statement skeleton + concrete slot values) → LLM via D-020 structured defense → unique task statement + starter files → validate → persist (seed + params + statement) via `VariantStore`. Cache: existing (learner,template) returns the stored variant (no duplicate work). Mock provider scripts deterministic statements per seed for tests.
|
||||
- **Verify:** `pnpm ai:test` — test_generator green: two different learner_ids → distinct statements for the same template; same learner twice → identical stored variant (reproducible); params JSON contains only schema-valid slot values; **fairness envelope (a-5):** two variants of one template compute digests within the template's expected feature envelope (comparable slot complexity/difficulty features) — "same bar" is testable, not asserted
|
||||
|
||||
### Wave 3: Variant endpoint + TS types (depends on Wave 2)
|
||||
|
||||
#### Task 4-3-01: Variant task endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/api/variants.py` (new), `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/tests/api/test_variants.py`
|
||||
- **Action:** `POST /v1/variants {learner_id, template_id or competency_id}` → generated (or cached) variant: statement, starter files, task_id, seed; `GET /v1/variants/{task_id}` → stored variant; `GET /v1/variants?learner_id=` → learner's variants. DI wiring in api/ only.
|
||||
- **Verify:** `pnpm ai:test` — test_variants green (generate→get roundtrip; cache hit on regenerate; distinct learners → distinct statements asserted at the API layer)
|
||||
|
||||
#### Task 4-3-02: TS types for variants (+ grades)
|
||||
- **Persona:** data-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `packages/types/variants.ts` (new), `packages/types/grading.ts` (new), `packages/types/index.ts` (update)
|
||||
- **Action:** TS `TaskVariant`, `VariantParams`, `RubricScore`, `GradeRecord` matching Python models (cross-referencing headers; same field names). Consumed by Phase 6 surfaces.
|
||||
- **Verify:** `pnpm typecheck` passes
|
||||
|
||||
### Must-Haves (Phase 4)
|
||||
- [ ] Two learners requesting the same competency receive **provably distinct** task variants (API-level test)
|
||||
- [ ] Seed derivation reproducible: same (template, learner) → same variant, served from cache without a second LLM call (D-029)
|
||||
- [ ] Variant seed + typed params persisted + auditable (VariantStore listing; proctoring cross-check path exists)
|
||||
- [ ] Difficulty normalization anchors present per template and shipped to the grader prompt context
|
||||
- [ ] Starter-file scaffolds defined per template (P6 wires them into the sandbox workdir)
|
||||
- [ ] `pnpm ai:test` + `pnpm typecheck` green; `variants/` imports no api/ (boundary)
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Oral / Voice Defense
|
||||
|
||||
**Requirements:** REQ-3-006
|
||||
**Goal:** `VoiceProvider` protocol with mock + browser fallback (D-030) + seventh `Examiner` agent streaming over existing SSE + `DefenseStore` persisting transcript + integrity signals. **Real server STT/TTS (`OpenAIAudioProvider`) is DEFERRED to v0.4 (with KYC, when there's a real key + real users)** — voice is mock-first (D-030) and the `/audio/*` real path could never be exercised in CI, so v0.3 proves the full defense *dialogue* + integrity-signal pipeline over mock + browser-native fallback only; the protocol seam keeps the real provider a drop-in later.
|
||||
|
||||
### Wave 1: Voice provider layer (parallel — no shared files)
|
||||
|
||||
#### Task 5-1-01: VoiceProvider protocol + mock provider + browser fallback
|
||||
- **Persona:** voice-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/voice/__init__.py`, `apps/ai-service/ai_service/voice/base.py`, `apps/ai-service/ai_service/voice/mock.py`, `apps/ai-service/ai_service/voice/browser.py`, `apps/ai-service/ai_service/voice/factory.py`, `apps/ai-service/tests/voice/__init__.py`, `apps/ai-service/tests/voice/test_mock.py`, `apps/ai-service/tests/voice/test_factory.py`
|
||||
- **Action:** `VoiceProvider` protocol mirroring `LLMProvider` (D-030): `transcribe(audio: bytes, fmt) -> TranscriptSegment` + `synthesize(text, voice) -> AsyncIterator[bytes]`. `MockVoiceProvider`: deterministic canned transcript (scripted per test), canned 1kHz-tone WAV bytes, scripted failure modes. `browser.py`: fallback **descriptor** (`sr_available: true`, endpoint hints) the web client uses to select browser-native `SpeechRecognition`/`speechSynthesis` when no server provider. `factory.py`: `AI_VOICE_PROVIDER=browser | mock` (default mock when no key). **`OpenAIAudioProvider` (real server STT/TTS) intentionally NOT built in v0.3 — deferred to v0.4**; the protocol is its future seam. `voice/` never imports `agents/` or `api/`.
|
||||
- **Verify:** `pnpm ai:test` — test_mock + test_factory green (deterministic transcribe/synthesize; failure modes; factory selects mock with empty key, browser when provider=browser; zero network calls)
|
||||
|
||||
#### Task 5-1-03: DefenseStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/voice/defense_store.py`, `apps/ai-service/tests/voice/test_defense_store.py`
|
||||
- **Action:** `DefenseStore` protocol (D-027): `start(defense)`, `append_turn(defense_id, turn)`, `finalize(defense_id, integrity_signals)`, `get(defense_id)`, `list_for_learner(learner_id)`, `close()`. SQLModel `DefenseRecord` (id, learner_id, task_id, status, created/finished_at) + `DefenseTurn` (defense_id FK, turn seq, role examiner|learner, text, ts, latency_ms) + integrity signals JSON on the record (long pauses, off-scope cadence markers — A-109).
|
||||
- **Verify:** `pnpm ai:test` — test_defense_store green (start→append turns→finalize→get roundtrip; ordered turns by seq)
|
||||
|
||||
### Wave 2: Examiner agent (depends on Wave 1)
|
||||
|
||||
#### Task 5-2-01: Examiner agent (seventh agent)
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/agents/examiner.py`, `apps/ai-service/ai_service/prompts/examiner.py` (new), `apps/ai-service/ai_service/agents/registry.py` (update: central registration, G-4 pattern), `apps/ai-service/tests/agents/test_examiner.py`
|
||||
- **Action:** `ExaminerAgent(BaseAgent)` (D-030/A-109): builds questions from learner transcript + trace digest + (P4) variant statement; probes understanding + challenges process choices ("why did you choose X at step N?"); streams questions over the existing SSE pipeline; `structured` verdict mode returns verdict + per-answer integrity signal list (long pause flags, off-scope answers) computed from turn metadata; calls voice **only through the `VoiceProvider` protocol** (PERSONAS conflict rule — never concrete providers). Session-scoped history reused from v0.2.
|
||||
- **Verify:** `pnpm ai:test` — test_examiner green (question stream references trace-digest facts; verdict structured output validates via D-020 defense; registry resolves all seven agents; mock-VoiceProvider wiring through protocol only — asserted by import scan in test)
|
||||
|
||||
### Wave 3: Defense endpoints (depends on Wave 2)
|
||||
|
||||
#### Task 5-3-01: Defense session + audio endpoints
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/api/defense.py` (new), `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/tests/api/test_defense.py`
|
||||
- **Action:** `POST /v1/defense/start {learner_id, task_id}` → creates DefenseRecord + streams the first examiner question (SSE, agent=examiner); `POST /v1/defense/{id}/answer` (multipart audio from browser MediaRecorder, or `{text}` for typed fallback) → STT via VoiceProvider → append learner turn → stream examiner follow-up (SSE) → TTS audio chunks over `GET /v1/defense/{id}/audio/{turn_id}`; `POST /v1/defense/{id}/finish` → verdict + integrity signals persisted; `GET /v1/defense/{id}` → full transcript + signals. Browser-fallback mode: when provider=browser, start returns the fallback descriptor instead of server audio.
|
||||
- **Verify:** `pnpm ai:test` — test_defense green (full loop with mock voice + mock LLM: start → answer(text) → answer(audio bytes) → finish → transcript retrievable with per-turn latency; unknown id → 404; `GET` signals present after finish)
|
||||
|
||||
### Wave 4: Examiner latency instrumentation (depends on Wave 3)
|
||||
|
||||
#### Task 5-4-01: Per-turn latency instrumentation (mock-based)
|
||||
- **Persona:** voice-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/tests/voice/test_latency.py`, `apps/ai-service/README.md` (update: conversational-budget doc + v0.4 voice note)
|
||||
- **Action:** Instrument per-turn latency (STT ms + LLM TTFT ms + TTS ms) recorded on each DefenseTurn; deterministic test over mock providers asserts instrumentation presence + `latency_ms` populated + budget constant defined (mock runs are near-instant — wall-clock asserted in v0.4 against a real endpoint). README documents the conversational-latency target as a v0.4 acceptance criterion (real STT/TTS deferred per CUT-1/G-7).
|
||||
- **Verify:** `pnpm ai:test` — test_latency green (latency_ms fields populated on every turn; budget constant defined); README documents the deferred real-voice acceptance probe
|
||||
|
||||
### Must-Haves (Phase 5)
|
||||
- [ ] Spoken defense runs end-to-end over HTTP with mock providers: start → answer (audio + typed fallback) → examiner follow-up streams → verdict + transcript persisted (automated)
|
||||
- [ ] Examiner is the seventh registered agent; streams over the existing SSE envelope (meta agent=examiner)
|
||||
- [ ] Instrumented per-turn latency fields populated on every DefenseTurn (STT ms + LLM TTFT ms + TTS ms); conversational budget named (A-109)
|
||||
- [ ] `VoiceProvider` protocol respected: examiner + api touch voice only via the protocol; mock-first — **no task requires a real voice key to pass**
|
||||
- [ ] Browser-native SR/TTS fallback descriptor returned when no server voice provider configured (mock/browser are first-class, D-030)
|
||||
- [ ] Real server STT/TTS (`OpenAIAudioProvider`) explicitly deferred to v0.4 (with real keys/users); the defense pipeline is fully proven over mock+browser — documented in README + release note
|
||||
- [ ] Optional future voice config noted for v0.4 (`AI_VOICE_BASE_URL` / `AI_VOICE_API_KEY`) in `.env.example` + README; keys only in gitignored `.ciagent/.env.secrets`; tests never call a voice API
|
||||
- [ ] Boundary rules hold: `voice/` imports no `agents/`/`api/`; `pnpm ai:test` + `pnpm ai:lint` green
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Agent Re-grounding + Learner Surface Integration
|
||||
|
||||
**Requirements:** REQ-3-007, REQ-3-008
|
||||
**Goal:** Lab/Assessor/Proctor consume real engine inputs (live telemetry, grading output, defense signals) with **no mock fallback in the learner path**; the v0.1 sandbox + assessment mockups become real — in-browser build/run (Run/Test buttons + read-only exec output, CUT-2 — no interactive shell), live telemetry panel, browser/typed voice defense, live grading; `pnpm build` + `pnpm typecheck` + full `pnpm ai:test` green
|
||||
**Note:** E2E verification runs against the real engines over HTTP with `AI_PROVIDER=mock` + `AI_VOICE_PROVIDER=mock` permitted (G-2 precedent) — the requirement is real engine plumbing (sandbox/telemetry/grading/defense over real endpoints, no corpus mocks in the learner path); a cloud outage must not block P6.
|
||||
|
||||
### Wave 1: Agent re-grounding (parallel — no shared files)
|
||||
|
||||
#### Task 6-1-01: Lab agent on live telemetry
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/agents/lab.py` (update), `apps/ai-service/ai_service/prompts/lab.py` (update: render real trace digest), `apps/ai-service/tests/agents/test_lab_live.py` (new)
|
||||
- **Action:** Lab consumes a **live trace digest** (grading/features `compute_digest` over `TraceStore` events) instead of `corpus/telemetry.py`. build_messages renders digest facts (recent commands, failing tests, idle). Mock-provider scripts assert digest-derived content. v0.2 corpus path removed from the agent (dormant corpus retained until Task 6-1-04 check).
|
||||
- **Verify:** `pnpm ai:test` — test_lab_live green: feedback references events actually present in a seeded SQLite trace (not corpus fixtures); no `corpus.telemetry` import in `agents/lab.py` (AST-asserted)
|
||||
|
||||
#### Task 6-1-02: Assessor agent on grading output
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/agents/assessor.py` (update), `apps/ai-service/ai_service/prompts/assessor.py` (update), `apps/ai-service/tests/agents/test_assessor_live.py` (new)
|
||||
- **Action:** Assessor consumes `GradeStore` output (validated `RubricScore` + digest) for learner+task instead of pre-baked artifacts/transcripts; renders strengths/gaps/verdict with rubric-anchored coaching framing. Structured output unchanged (D-020).
|
||||
- **Verify:** `pnpm ai:test` — test_assessor_live green: given a real stored grade, Assessor output reflects its scores; corpus artifact path gone from the agent (AST-asserted)
|
||||
|
||||
#### Task 6-1-03: Proctor agent on telemetry + defense signals
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/agents/proctor.py` (update), `apps/ai-service/ai_service/prompts/proctor.py` (update), `apps/ai-service/tests/agents/test_proctor_live.py` (new)
|
||||
- **Action:** Proctor consumes real integrity inputs: idle gaps + command cadence from the trace digest + defense integrity signals from `DefenseStore` → classified signals + coaching interventions (supportive tone retained). Cross-checks variant seed params (P4) for off-template work.
|
||||
- **Verify:** `pnpm ai:test` — test_proctor_live green: signals derived from seeded real trace + defense records; corpus proctor scenarios no longer imported (AST-asserted)
|
||||
|
||||
#### Task 6-1-04: Corpus dormancy + mockup removal verification
|
||||
- **Persona:** lead-developer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/telemetry.py`, `apps/ai-service/ai_service/corpus/artifacts.py`, `apps/ai-service/ai_service/corpus/` (README note), `apps/ai-service/tests/test_corpus_dormancy.py` (new)
|
||||
- **Action:** Verify no production code path imports `corpus/telemetry.py` or `corpus/artifacts.py` anymore (test scans imports across `agents/`, `api/`, engines). Retain files as Phase-3 calibration history with a header note marking them **dormant — v0.2 mocks, not used at runtime**; learner-context corpus stays (agents still need learner context). Disposes v0.2's G-5-class dead-code risk deliberately.
|
||||
- **Verify:** `pnpm ai:test` — test_corpus_dormancy green (zero runtime importers); suite otherwise unchanged
|
||||
|
||||
### Wave 2: Client plumbing + design primitives (parallel — no shared files)
|
||||
|
||||
#### Task 6-2-01: Sandbox build-panel engine client (run/test-only — no raw shell relay)
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `apps/web/hooks/use-sandbox-session.ts` (new), `apps/web/lib/engine-client.ts` (new), `apps/web/.env.example` (update)
|
||||
- **Action:** `engine-client.ts`: typed fetch client for `/v1/sandboxes` (create/destroy — learner_id from the v0.3 mock session constant, allowlisted server-side per G-5), `/v1/sandboxes/{id}/files` (workspace CRUD), `/v1/sandboxes/{id}/exec` (run/test), `/v1/variants`, `/v1/assessment/grade`, `/v1/telemetry/traces`, `/v1/defense/*`; base `NEXT_PUBLIC_AI_SERVICE_URL`. `use-sandbox-session.ts`: create sandbox+variant on task open → destroy on unmount (idempotent cleanup, AbortController pattern); 503 pool-full → user-facing "environment busy, retry" (D-032); 403/429 abuse-control surfaced honestly (G-5). **CUT-2 (G-8): NO raw interactive WS terminal relay (keystroke-level stdin/stdout) in v0.3** — the credential pipeline needs *process events* (from Run/Test + file edits), not a live shell; the interactive xterm relay is the most fragile real-time piece and is deferred to v0.4. The build panel is a **Run/Test output viewer** (exec results + telemetry pulse render), not an interactive shell. `@xterm/xterm` is therefore NOT a dependency in v0.3.
|
||||
- **Verify:** `pnpm install && pnpm typecheck` pass; hook unmount destroys the sandbox (manual probe: `curl localhost:8420/v1/sandboxes` shows count drop after navigation); RUN/TEST buttons produce streamed output + telemetry events in the trace
|
||||
|
||||
#### Task 6-2-02: New design primitives
|
||||
- **Persona:** design-system-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `packages/ui/src/primitives/terminal-frame.tsx` (new), `packages/ui/src/primitives/mic-control.tsx` (new), `packages/ui/src/primitives/grade-badge.tsx` (new), `packages/ui/src/primitives/telemetry-status.tsx` (new), `packages/ui/src/primitives/transcript-viewer.tsx` (new), `packages/ui/src/primitives/index.ts` (update), `packages/ui/src/index.ts` (update)
|
||||
- **Action:** Token-driven primitives: TerminalFrame (CUT-2: a read-only exec-output viewer chrome — streams Run/Test results, NOT an interactive shell), MicControl (record/stop with consent state + no-mic fallback state, MediaRecorder permission UX), GradeBadge (verdict rendering), TelemetryStatus (live event pulse / disconnected indicator), TranscriptViewer (examiner/learner turn list). Dark mode + WCAG AA; stories for each.
|
||||
- **Verify:** primitives import from `@nextcraft/ui`; Storybook stories render dark + light; `pnpm build` (ui package) passes
|
||||
|
||||
#### Task 6-2-03: Defense TS types
|
||||
- **Persona:** data-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `packages/types/defense.ts` (new), `packages/types/index.ts` (update)
|
||||
- **Action:** TS `DefenseSession`, `DefenseTurn`, `IntegritySignal`, `Verdict` mirroring P5 Python models (cross-referencing header).
|
||||
- **Verify:** `pnpm typecheck` passes
|
||||
|
||||
### Wave 3: Real build surface (depends on Wave 2)
|
||||
|
||||
#### Task 6-3-01: Sandbox mockup → real in-browser IDE
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `apps/web/app/(learner)/build/[competencyId]/page.tsx` (rewrite), `apps/web/components/learner/sandbox-terminal.tsx` (new), `apps/web/components/learner/file-tree.tsx` (new), `apps/web/components/learner/run-controls.tsx` (new), `apps/web/components/learner/lab-feedback-panel.tsx` (update: live trace)
|
||||
- **Action:** Replace the mockup with the real build environment (A-103, CUT-2): file tree (HTTP CRUD into the sandbox workdir via `/v1/sandboxes/{id}/files` routes added to api/sandboxes — read/write/list workspace files), syntax-highlight editor (existing), **Run**/**Test** buttons (exec in sandbox; results stream to a read-only TerminalFrame output panel — no interactive shell), starter files from the P4 variant scaffold. Lab panel posts `learner_id+task_id` → streams Lab feedback over the **live** trace (no scenario IDs). Telemetry sidebar shows live TelemetryStatus. Pool-full 503 → busy state with retry; 403/429 surfaced.
|
||||
- **Verify:** with ai-service up: open `/build/comp-01` → variant statement + starter files load → edit a file → **Run** executes the command in-sandbox and output renders in the panel → **Test** runs the test suite in-sandbox → Lab panel streams digest-derived feedback → telemetry status shows live events. Manual probe documented; `pnpm typecheck` green
|
||||
|
||||
### Wave 4: Live defense + grading surfaces (depends on Waves 2-3)
|
||||
|
||||
#### Task 6-4-01: Assessment mockup → live defense + live grading
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `apps/web/app/(learner)/defend/[competencyId]/page.tsx` (rewrite), `apps/web/components/learner/defense-session.tsx` (new), `apps/web/components/learner/assessor-results-panel.tsx` (update), `apps/web/components/learner/proctor-banner.tsx` (update), `apps/web/components/learner/oral-defense-interface.tsx` (rewrite or remove)
|
||||
- **Action:** Real assessment flow: **Start Defense** → POST `/v1/defense/start` → examiner question streams → learner answers via MicControl (MediaRecorder webm/opus → multipart POST) with typed fallback when mic denied or `provider=browser` (native `SpeechRecognition`/`speechSynthesis` path per fallback descriptor) → follow-ups stream → **Finish** → verdict + integrity signals panel (TranscriptViewer, GradeBadge) + **Grade My Work** → POST `/v1/assessment/grade` → structured rubric bars render. Proctor banner reads signals for task_id. Loading + error states throughout; no mock defense data remains in the learner path.
|
||||
- **Verify:** with ai-service up: full defense loop runs in-browser (typed fallback acceptable in CI-less manual probe; mic path exercised manually with permission granted); grade panel renders real rubric scores; `pnpm typecheck` green
|
||||
|
||||
### Wave 5: End-to-end verification (depends on Waves 3-4)
|
||||
|
||||
#### Task 6-5-01: Full learner-path E2E probe + green builds
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-007, REQ-3-008
|
||||
- **Files:** `apps/ai-service/tests/api/test_e2e_credential_flow.py` (new), `apps/ai-service/README.md` (update: E2E probe doc)
|
||||
- **Action:** Endpoint-level E2E (mock LLM/voice providers, real engines): create variant → create sandbox with task_id → ingest trace events via real sandbox exec → POST grade → start defense (typed answers) → finish → assert: trace persisted, grade stored + digest-linked, defense transcript + signals stored, Proctor/Assessor endpoints serve them. No corpus fixture anywhere in the flow (AST-asserted). Run `pnpm build` + `pnpm typecheck` + full `pnpm ai:test` at repo root; fix all failures before phase ship.
|
||||
- **Verify:** `pnpm ai:test` green incl. test_e2e_credential_flow; `pnpm build` + `pnpm typecheck` green; README E2E probe section documents the manual browser pass
|
||||
|
||||
### Must-Haves (Phase 6)
|
||||
- [ ] Lab/Assessor/Proctor operate on real inputs with **no mock fallback in the learner path** (AST-verified: no corpus telemetry/artifact/proctor imports in production paths)
|
||||
- [ ] Learner builds in-browser for real (CUT-2): Run/Test buttons execute in a namespace sandbox and stream output to a read-only panel; file tree CRUD works; starter files come from the learner's variant scaffold
|
||||
- [ ] Live telemetry: build activity streams to ai-service and the sidebar shows live status (TelemetryStatus); trace persisted in SQLite
|
||||
- [ ] Live defense: start → answer (mic or typed fallback) → examiner follow-ups → finish → transcript + integrity signals + verdict rendered; browser-native path works with no server voice key
|
||||
- [ ] Live grading: grade request returns structured rubric scores computed from the real trace digest; grade panel renders them
|
||||
- [ ] 503 pool-full surfaced honestly in UI; navigating away destroys the sandbox (no leaked sandboxes — `GET /v1/sandboxes` manual probe)
|
||||
- [ ] Examiner remains protocol-clean (voice only via `VoiceProvider`); module boundary rules hold across all new code
|
||||
- [ ] `pnpm build` and `pnpm typecheck` pass; full `pnpm ai:test` green; no cloud/voice calls in any automated test
|
||||
- [ ] Release-note input (for P7): v0.3 ships real engines; **identity/age-gating (KYC) remains deferred — age-gating is still a visual mockup** (A-110); sandbox scope is coding-IDE only (design tool/simulation deferred to v0.4, D-025)
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Final Review + Ship (no planned tasks)
|
||||
|
||||
Orchestrated by the SHIP stage, not this plan: multi-persona code review (correctness, testing, module boundaries, secrets hygiene — keys absent from code/logs/commits/errors, localhost-only CORS, no PII in prompts), project health audit (reconstruction, .ciagent/ discipline, branch/commit hygiene), then merge milestone → main, tag the final v0.2.x patch, create the Gitea release, mark all 8 v0.3 requirements complete.
|
||||
|
||||
**Release-note honesty:** the release note must state (a) Lab/Assessor/Proctor now run on real engine inputs (v0.2 mock-input caveat retired), (b) sandbox scope = coding IDE only — design tool + simulation environments deferred to v0.4 (D-025), (c) identity/age-gating (KYC) deferred per founder directive — age-gating remains the v0.1 visual flow mockup; abuse control (per-learner sandbox caps + server-side learner allowlist, G-5) ships in place of auth (A-110), (d) voice runs mock-first with browser-native fallback — real server STT/TTS deferred to v0.4 (CUT-1), (e) sandbox resource limits are partially enforced (memory/CPU/wall-clock kernel-enforced via rlimits; per-sandbox pids + hard disk quota are NOT — mitigated by a workdir-size sweep and per-learner caps; full enforcement requires cgroup delegation, deferred to the post-MVP containerd backend, D-024/G-1/G-2).
|
||||
|
||||
**Secrets-hygiene checklist (P7):** `.ciagent/.env.secrets` gitignored and never committed; `AI_VOICE_API_KEY` / `AI_TUTOR_API_KEY` referenced only via env; SQLite DBs + sandbox dirs gitignored; no keys in logs, error messages, or test fixtures.
|
||||
|
||||
**Disposal checks (G-5 class):** v0.2 dormant corpus files carry the dormant-header note (Task 6-1-04); any now-unused mock-data exports for the old sandbox/defense mockups (e.g. `aiTutorResponses`-class leftovers) must be removed or deprecated by review.
|
||||
**Research decisions D-040..D-046 are binding design contracts.** This plan operationalizes them; it does not re-litigate them.
|
||||
|
||||
---
|
||||
|
||||
## User-Facing Surface
|
||||
|
||||
The primary user-facing surface is the **learner build + defend flow** at `http://localhost:3000`, backed by the real engines in ai-service at `http://localhost:8420`:
|
||||
|
||||
- `/dashboard` — AI tutor chat (Coach/Tutor, streaming) + Mentor panel (unchanged from v0.2)
|
||||
- `/learn/[competencyId]` — byte viewer with streaming Tutor explanations (unchanged)
|
||||
- `/build/[competencyId]` — **real in-browser build environment**: file tree, syntax editor, **Run/Test buttons that execute in a namespace sandbox and stream output to a read-only panel** (CUT-2 — no interactive shell), live telemetry status, live Lab feedback, per-learner variant task statement
|
||||
- `/defend/[competencyId]` — **live oral defense + live grading**: Examiner voice/typed dialogue, transcript + integrity signals, real rubric scores from the process trace
|
||||
|
||||
The marketplace, employer, and admin surfaces are unchanged from v0.1/v0.2.
|
||||
1. **Voice defense with a provider badge** (`/defend/[competencyId]`): the learner's spoken answers upload as audio and are transcribed server-side when `AI_VOICE_PROVIDER=openai-audio` is configured; examiner questions play as server TTS audio; the mic control shows an honest badge — `server voice`, `browser voice`, or `mock` — derived from the provider descriptor (`VoiceDescriptor.mode`), and degrades visibly (browser/mock fallback) with keys absent.
|
||||
2. **Identity enrollment flow** (new `/enroll` learner route + marketplace surfaces): submit verification → pending state → verified/rejected state; verified learners proceed to variants/sandboxes/defense; unverified learners hitting gated routes see a structured verify-CTA (403 payload rendered as an actionable prompt, not a dead error). Mock verdicts are labeled `mock` everywhere they surface (A-304 honesty).
|
||||
3. **Environment-typed build flows** (`/build/[competencyId]`): design competencies open a design environment (SVG/HTML/schematic artifact starter files, Run = validator harness), simulation competencies open a simulation environment (benchmark script + dataset starter files, Run = bounded harness execution); the Run/Test buttons use the variant's real `test_command` instead of hardcoded pytest; the surface, file tree, editor, read-only output panel (CUT-2), and telemetry pulse are unchanged across kinds.
|
||||
4. **Invisible durability**: mid-connection kill of a build session loses nothing on reconnect (seq-ack protocol) — no visible UI, proven by tests.
|
||||
|
||||
## Happy Path
|
||||
|
||||
1. Learner opens `/build/comp-01` → a per-learner **variant statement** and starter files load; a namespace sandbox is created for the session
|
||||
2. Learner edits files in the tree and clicks **Run**/**Test** → commands execute in the sandbox and real output renders in the build panel; the telemetry sidebar pulses as events stream to ai-service and persist in SQLite
|
||||
3. The Lab panel streams feedback derived from the **live trace digest** (real commands, real failures)
|
||||
4. Learner opens `/defend/comp-01` → **Start Defense**: the Examiner streams an opening question ("Walk me through your build — why did you structure it this way?")
|
||||
5. Learner answers by voice (mic consent → MediaRecorder → STT) or typed fallback → examiner follow-ups probe the trace ("You hit three test failures before passing — what changed?"); TTS plays examiner audio (or browser speech in fallback)
|
||||
6. Learner finishes the defense → transcript + integrity signals appear; verdict renders in a GradeBadge
|
||||
7. Learner clicks **Grade My Work** → the grading engine computes the digest from the real trace, rubric-scores it, and the panel renders per-criterion bars + strengths/gaps/verdict
|
||||
8. Proctor banner shows integrity signals from the live trace + defense in coaching tone; Mentor panel on `/dashboard` can narrate the real outcome
|
||||
9. Navigating away destroys the sandbox (pool slot freed); killing ai-service shows inline error + retry states on every panel, with no crashes
|
||||
**Defense with real voice:** learner opens `/defend/cmp-*` → DefenseSession starts → examiner question streams (SSE) → learner speaks → MediaRecorder captures webm → POST `/v1/defense/{id}/answer` (multipart audio) → server strips codec param (`webm;codecs=opus` → `webm`), enforces ≤10MB, calls `OpenAIAudioProvider.transcribe` → transcript turn stored (STT latency recorded) → learner clicks the speaker icon on an examiner turn → GET `/v1/defense/{id}/audio/{turn_id}` → server TTS bytes stream back with correct media_type → verdict + integrity signals render unchanged.
|
||||
|
||||
**Identity-gated build:** learner completes `/enroll` (submit → mock provider verdict `verified`, age band 18+) → opens a design competency → POST `/v1/variants` passes the identity gate (verified ≥16) → variant carries `environment: "design"` + `test_command` → sandbox created (allowlist ✓ → identity ✓ → rate cap ✓) → starter files written → Run executes the validator harness in the namespace sandbox → telemetry streams with seq-acks → grade digest renders.
|
||||
|
||||
## UX Acceptance Criteria
|
||||
|
||||
1. Run/Test output visibly reflects the real sandbox execution (command round-trip to the sandbox, real stdout/stderr), not a replay animation
|
||||
2. The learner path contains **no mock engine data** — scenarios, canned artifacts, and scripted defense transcripts from v0.2 are gone from runtime
|
||||
3. Variant statements visibly differ between two learner sessions on the same competency
|
||||
4. Mic permission flow is graceful: consent prompt, recording indicator, no-mic/typed fallback, and browser-native speech path when no server voice key is configured
|
||||
5. Defense transcript renders turn-by-turn with latency shown; integrity signals render in coaching (supportive) tone
|
||||
6. Grade results render as structured per-criterion bars with verdict, from the real trace — not from final-output-only heuristics
|
||||
7. Pool-full (503) shows an honest "environment busy — retry" state; navigation/unmount destroys sandboxes with no leaks
|
||||
8. When ai-service is unreachable: inline error + retry on every panel — no crashes, no console errors, no blank UI
|
||||
9. All new UI uses design tokens, supports dark mode, meets WCAG AA contrast, responsive at 375px / 768px / 1280px
|
||||
10. `pnpm build` and `pnpm typecheck` pass with zero errors; `pnpm ai:test` green cloud-free and voice-key-free
|
||||
1. The mic control badge always tells the truth about which voice path is live (server/browser/mock) — never claims server when mock is wired.
|
||||
2. Unverified/under-age callers on gated routes get a 403 with an actionable verify-CTA payload (rendered as a prompt with a link to enrollment) — never a bare JSON error in the UI.
|
||||
3. Mock identity verdicts are visibly labeled `mock` in every surface that shows verification state.
|
||||
4. Design/sim environments are indistinguishable from build environments in surface mechanics (file tree, editor, Run/Test, output panel) — only starter contents and the Run command differ; no route changes, no new navigation.
|
||||
5. Run/Test buttons reflect the variant's `test_command` (no hardcoded pytest on a design competency).
|
||||
6. No durable state is written inside the repo (state in `~/.nextcraft/`; tests in tmp dirs).
|
||||
7. All existing accessibility baselines hold (WCAG AA contrast on new badge/CTA states).
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Seq-Lease + Replay Margin (REQ-5-007)
|
||||
|
||||
**Goal:** Close the one-line replay-margin/ACK gap (P07-documented): the capture agent requeues only `_last_sent` on detected disconnect while N frames may be in TCP flight — frames 1..N-1 are lost. Server seq-acks close it.
|
||||
|
||||
### Wave 1-1: ingest ack emission
|
||||
|
||||
- **Task 1-1-01** (sandbox-engineer): `telemetry/ingest.py` — emit `{"type":"seq_ack","seq":N}` after every successful `store.append` in `IngestSession._append` (N = durable `latest_seq` snapshot already computed; O(1), advisory — gap detection stays authoritative; G-3 flood semantics untouched). Unit test: appended seq N → ack frame N observed on the WS (TestClient portal pattern, `tests/api/test_telemetry_ingest.py`).
|
||||
- Files: `apps/ai-service/ai_service/telemetry/ingest.py`, `apps/ai-service/tests/telemetry/`
|
||||
- **MH-1a**: a test proves each successful append emits an ack carrying the post-append `latest_seq`.
|
||||
|
||||
### Wave 1-2: agent ack consumption + spool trim
|
||||
|
||||
- **Task 1-2-01** (sandbox-engineer): `scripts/sandbox-agent.py` — supervisor loop (currently discards all non-close frames) parses text frames; on `type == "seq_ack"` trims every spool/pending line with `seq <= ack` under `_emit_lock` via atomic `Spool.rewrite`; clears `_last_sent` if its seq ≤ ack; tolerates any frame interleaving (acks/gap_warning/rejected); keepalive pings (binary) unaffected. **Stdlib-only (AST-pinned).**
|
||||
- **Task 1-2-02** (sandbox-engineer): add an explicit spool bound (max lines, e.g. 4096 — documented; D-R07 correction: no cap existed) — oldest-beyond-bound dropped with a counter; **G-14: overflow is by-design gap creation — unit test proves dropped-counter > 0 → replayed trace exhibits gaps → grader/gap path marks it ungradable (never a silently-truncated-but-gradable trace); document the worst-case arithmetic (~64KB diff cap × 4096 lines ≈ 256MB, under but HALF the 512MB G-2 budget — the spool lives inside the swept workdir)**; document G-2+flood-cap as the outer bound.
|
||||
- Files: `apps/ai-service/scripts/sandbox-agent.py`, `apps/ai-service/tests/sandbox/test_sandbox_agent.py`
|
||||
- **MH-1b**: unit test — agent with a scripted WS that acks mid-drain trims its spool to `seq > ack` exactly (no over-trim, no under-trim), stays within the explicit bound, and overflow drops create honest gaps (ungradable, G-14).
|
||||
- **MH-1c**: `replay_margin()` behavior after acks: requeue window is bounded by unacked in-flight only (repeated reconnect/ack cycles never lose or duplicate a spooled line).
|
||||
|
||||
### Wave 1-3: mid-burst regression test (the real proof)
|
||||
|
||||
- **Task 1-3-01** (sandbox-engineer): extend `tests/telemetry/test_durability.py` with `test_midburst_disconnect_loses_nothing` — real uvicorn + KillableProxy; sever the connection **immediately after a rapid multi-frame send, WITHOUT waiting for server observation** (the exact scenario the P07 de-flake documented as uncovered); revive; assert every emitted seq stored exactly once, in order; assert spool trimmed to ≤ ack margin; keep `_await_events`/portal patterns (deterministic, no socket surgery).
|
||||
- Files: `apps/ai-service/tests/telemetry/test_durability.py`
|
||||
- **MH-1d**: the mid-burst test passes repeatedly (≥3 consecutive runs) with zero loss/dup outside the acked margin.
|
||||
|
||||
**Verification strategy P1:** `pnpm ai:test` (413+green), ruff, no web/TS changes, no settings changes. Existing durability + reconnect-flush suites stay green.
|
||||
|
||||
**Risks:** mid-burst determinism (mitigated: ack protocol is the fix; wait for server-side observation of the ack itself); `_emit_lock` reentrancy from supervisor thread (trim under the same lock as flush); frame-order tolerance.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Real Server Voice (REQ-5-001, REQ-5-002)
|
||||
|
||||
**Goal:** The `openai-audio` VoiceProvider — server STT/TTS for voice defense end-to-end when keys exist; mock/browser unchanged.
|
||||
|
||||
### Wave 2-1: settings + factory + provider
|
||||
|
||||
- **Task 2-1-01** (backend-engineer): `config.py` — add `voice_base_url: str = ""`, `voice_api_key: str = ""` (never logged), `voice_stt_model: str = "whisper-1"`, `voice_tts_model: str = "tts-1"`, `voice_tts_voice: str = "alloy"`, `voice_tts_format: Literal["mp3","wav","opus"] = "mp3"` (G-16: enum, not free string — it feeds a Content-Type), `voice_max_audio_mb: int = 10`; `.env.example` documents them (keys never in code/commits); unknown format value → fall back to default with a loud log (consistent with G-11).
|
||||
- **Task 2-1-02** (backend-engineer): `voice/factory.py` — `voice_provider_from_settings(settings, http_client)` (signature change); `openai-audio` branch constructs `OpenAIAudioProvider`; rejects selection when `voice_base_url`/`voice_api_key` empty with an actionable error for direct callers; **G-11: `main.py` lifespan catches the factory error, logs loudly (naming the missing vars), and falls back to the mock provider — a typo'd env in an unattended deploy must never crash the boot (D-039); descriptor then honestly reads `mock`**; lifespan passes `app.state.http_client` (state-injection preserved).
|
||||
- **Task 2-1-03** (voice-engineer): `voice/openai_audio.py` — `transcribe()`: multipart `POST {base}/audio/transcriptions` (`file` + `model`, response_format=json → `TranscriptSegment`); `synthesize()`: streaming `POST /audio/speech` (JSON body `model/input/voice/response_format`, raw byte chunks); `descriptor = VoiceDescriptor(mode="server", sr_available=True, tts_available=True, hint=...)`; errors sanitized with key redaction (mirror `llm/openai_compat.py:_sanitize`); reuses the shared httpx client (D-017; read=300s).
|
||||
- Files: `apps/ai-service/ai_service/config.py`, `voice/factory.py`, `voice/openai_audio.py`, `main.py`, `.env.example`
|
||||
- **MH-2a**: MockTransport byte-contract tests — STT: multipart fields + response parse → `TranscriptSegment`; TTS: JSON body + byte stream → concatenated chunks; failure pins 413/400/429/timeout → sanitized errors, NO key leak (pinned).
|
||||
- **MH-2b**: factory: `openai-audio` selected + configured → server-mode provider; selected + unconfigured → actionable rejection for direct callers AND app boot survives with mock fallback + loud log naming the fix (G-11); mock/browser unchanged; provider always carries a `descriptor` (a-15 — a missing descriptor would badge the server path as mock). Invert the v0.4 rejection test (`test_real_server_stt_tts_rejected_as_v04_seam`).
|
||||
|
||||
### Wave 2-2: defense route fixes + audio upload
|
||||
|
||||
- **Task 2-2-01** (voice-engineer): `api/defense.py` — fmt derivation strips codec params (`"webm;codecs=opus"` → `webm`; else real STT 400s); enforce `voice_max_audio_mb` BEFORE provider call (422 empty / 413 oversize; a-9: Content-Length fast path before buffering); TTS route media_type mapped from the `voice_tts_format` enum (G-16); descriptor now comes from the provider (mode=server flows to the client untouched).
|
||||
- **Task 2-2-02** (frontend-engineer): `engine-client.ts` — `answerDefense` audio variant (FormData: blob + filename + content-type); `defense-session.tsx` — POST the recorded blob instead of discarding it; **G-12: recording bound — auto-stop at a max duration (default 180s) with a visible timer, `recorder.start(timeslice)` for observable size (a-13); a 413 response renders as an honest "answer too long — re-record" prompt, never silent loss**; provider badge from descriptor (`server voice`/`browser voice`/`mock`) with visible degradation states.
|
||||
- Files: `apps/ai-service/ai_service/api/defense.py`, `apps/web/lib/engine-client.ts`, `apps/web/components/learner/defense-session.tsx`, `apps/web/tests/`
|
||||
- **MH-2c**: API test — webm;codecs=opus content-type reaches the provider as clean `webm`; oversize audio → 413 without provider call; empty → 422 (existing).
|
||||
- **MH-2d**: web tests — audio POST path builds correct FormData; badge reflects descriptor mode; auto-stop fires at the bound; 413 renders the re-record prompt (G-12).
|
||||
|
||||
**Verification strategy P2:** `pnpm ai:test`, ruff, `pnpm typecheck`, web tests, `pnpm build` (static export still emits). Manual cloud probe recipe documented in `.env.example` comments (never CI). Defense flow tests stay mock-only (cloud-free rule).
|
||||
|
||||
**Risks:** full audio bytes buffered in memory (bounded by the 10MB guard); TTS `input` ≤4096 chars (examiner questions are short — enforced with a guard + truncation error); factory signature change touches main.py lifespan (state-injection pattern preserved).
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Identity + Age-Gating (REQ-5-003, REQ-5-004)
|
||||
|
||||
**Goal:** Identity verification backend behind a provider protocol (mock-first); backend-enforced age gates composed with G-5.
|
||||
|
||||
### Wave 3-1: identity module core
|
||||
|
||||
- **Task 3-1-01** (identity-engineer): `ai_service/identity/base.py` — `IdentityProvider` protocol: `submit(learner_id, submission) -> submission_id`, `poll(submission_id) -> verdict {status, age_band, provider, mock, refs}`; `mock.py` — deterministic mock (approve-on-policy: age band from a scripted DOB field, reject scripted-bad); verdicts carry `mock: true` marker (A-304).
|
||||
- **Task 3-1-02** (identity-engineer): `ai_service/identity/store.py` — 5th D-027 store, DefenseStore pattern (WAL, `foreign_keys=ON`, portable columns, `@validates`): `identity_record` table — `id` (submission id, minted once), `learner_id` (indexed), `status: pending|verified|rejected`, `provider`, `provider_verdict` (JSON, mock-marked), `age_band` (derived `16-17`|`18+`, NEVER raw DOB), `document_refs` (JSON refs — raw documents NEVER stored), `submitted_at`, `verified_at`; insert-only + latest-per-learner lookup.
|
||||
- **Task 3-1-03** (identity-engineer): `main.py` lifespan — `app.state.identity_store` + `app.state.identity_provider` (state-injection overrides preserved); `config.py` — `identity_provider: str = "mock"`, identity store rides the same db_path.
|
||||
- **MH-3a**: store tests — insert/poll/latest/verdict provenance; constraints fire on invalid bands; same SQLite file (additive table, D-027 family).
|
||||
|
||||
### Wave 3-2: API surface + gates
|
||||
|
||||
- **Task 3-2-01** (identity-engineer): `api/identity.py` — `/v1/identity/submit` (submission → pending; G-13: one active pending per learner — resubmit while pending → 409 echoing the pending state; per-learner submit rate cap → 429), `/v1/identity/status/{learner_id}` (latest record + mock marker), `/v1/identity/verify/{submission_id}` (poll provider → verified/rejected transition); router mounted in main.py.
|
||||
- **Task 3-2-02** (identity-engineer): gate dependencies — `require_verified_age(min_age)` FastAPI dependencies; **composition order binding (D-043)**: G-5 allowlist (403 pilot guard) → identity verdict (403 + verify-CTA payload `{reason, min_age, current_status, verify_cta}`) → rate caps (429). Apply: school 16+ on variant generation (`api/variants.py`), sandbox create (`api/sandboxes.py`), defense start (`api/defense.py`); marketplace 18+ via `require_verified_adult` on ONE minimal gated route (`POST /v1/marketplace/apply` — G-18: honest stub; passes the gate composition then returns 501 with `stub: true` + mock-verdict markers, never a fabricated "applied" outcome).
|
||||
- **Task 3-2-03** (identity-engineer, G-9): conftest `verified_pilot` fixture — seeds an identity record (mock provider, band 18+ or 16+) for test learner ids + allowlist coverage in test settings; MUST land in the same wave as the gates or the existing variant/sandbox/defense suites 403 en masse (those routes are ungated today).
|
||||
- **Task 3-2-04** (security-auditor, phase-specific): PII review — sentinel scrub test: submit identity with sentinel PII strings → assert they appear in NO log record (caplog) and NO stored raw form (store inspection); API responses expose verdict + mock marker only.
|
||||
- Files: `apps/ai-service/ai_service/identity/**`, `api/identity.py`, `api/variants.py`, `api/sandboxes.py`, `api/defense.py`, `main.py`, `config.py`, `tests/identity/`, `tests/api/`
|
||||
- **MH-3b**: gate tests — verified 18+ passes all school gates; 16-17 passes school gates but 403s the marketplace route (honest stub response beyond the gate, G-18); unverified → 403 with verify-CTA payload; under-16 → 403 everywhere gated; allowlist rejection (403) still fires FIRST for non-pilot learner ids; identity submit: pending-resubmit → 409, rate cap → 429 (G-13); **all pre-existing variant/sandbox/defense API suites remain green under the `verified_pilot` fixture (G-9)**.
|
||||
- **MH-3c**: caplog sentinel test green (PII never logged/stored).
|
||||
|
||||
### Wave 3-3: web enrollment flow
|
||||
|
||||
- **Task 3-3-01** (frontend-engineer): `/enroll` route — submit → pending → verified/rejected states (honest, mock-labeled); engine-client identity functions; **G-10: client 403 discrimination — `verify_cta` present in the 403 payload → new `VerifyRequiredError` `{reason, min_age, current_status, verify_cta}`; allowlist detail → existing `NotAllowlistedError` (today engine-client.ts collapses every 403 into NotAllowlistedError — a verify-CTA would render as an allowlist lie)**; gated-route 403 CTA rendered as actionable prompt (link to `/enroll`); dashboard learner age badge reflects verified state.
|
||||
- Files: `apps/web/app/(learner)/enroll/`, `apps/web/lib/engine-client.ts`, `apps/web/components/`, `packages/types/`
|
||||
- **MH-3d**: web tests — identity client functions; CTA payload shape; enrollment states render; **403 discrimination: verify-CTA → VerifyRequiredError, allowlist detail → NotAllowlistedError (G-10)**. `pnpm build` emits the new route.
|
||||
- **MH-3e** (CUT-3): identity flow test via `TestClient` against real `create_app` (routers mounted, gates composed, real stores, mock providers) — unverified learner → variants POST → 403 verify-CTA → submit + verify (mock) → variants POST 200. No uvicorn harness (identity is plain JSON; the real-server harness stays where transport matters — P1/P4).
|
||||
|
||||
**Verification strategy P3:** `pnpm ai:test`, ruff, typecheck, web tests, `pnpm build`. PII caplog test is release-blocking (security-auditor sign-off).
|
||||
|
||||
**Risks:** self-asserted `learner_id` trust level (documented as pilot-scale — same as G-5 today; real auth is post-v0.5); shared-SQLite additive table (safe); mock-verdict honesty must ride every response (pinned).
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Design/Sim Environments (REQ-5-005, REQ-5-006)
|
||||
|
||||
**Goal:** Environment kinds at the template/variant layer; per-kind starter contents + exec policy; kind flows through telemetry; digest untouched.
|
||||
|
||||
### Wave 4-1: registry + wire types
|
||||
|
||||
- **Task 4-1-01** (sandbox-engineer): `variants/templates.py` — `TaskTemplate.environment: Literal["build","design","simulation"] = "build"` + per-kind `starter_files` + `harness_command`/`test_command` policy fields; **G-15: command fields validated at definition time (Python, where shlex exists) — must roundtrip `shlex.split` → whitespace-join → `shlex.split` identically (no quotes/globs/metachars; violation is a template-authoring bug caught in tests)**; add one design template (stack-designer c001/c002 — already sanctioned) + one simulation template (stack-science or stack-operator); `variants/generator.py` + `store.py` carry the field through `VariantRecord`.
|
||||
- **Task 4-1-02** (frontend-engineer): `api/variants.py` — `VariantResponse` gains `environment` + `test_command` (closes the dead-field gap); TS `packages/types/variants.ts` + engine-client types sync (dual-schema rule: both places, same change).
|
||||
- **MH-4a**: variant tests — design/sim templates generate kind-tagged variants with correct starter files + commands; command fields roundtrip the shlex validator (G-15); wire response carries both fields (a-11: required on the wire, TS required-field parity); TS types match Python field-for-field.
|
||||
|
||||
### Wave 4-2: exec command policy
|
||||
|
||||
- **Task 4-2-01** (sandbox-engineer + security-auditor): `api/sandboxes.py` exec route — per-kind command policy: **G-15: EXACT argv-token matching** against {template-declared harness/test argv[0]} ∪ a small generic file/nav set — never prefix/substring (trivially bypassed via flags/`-c` passthrough); `sh -c` passthrough DISALLOWED for design/sim kinds (the gaming vector: faking build-style test cycles into a kind-agnostic digest); violation → 422 naming the allowed set; policy table is code (reviewable, versioned); build-kind flows do not regress (existing tests green).
|
||||
- **MH-4b**: exec tests — design kind rejects pytest-style arbitrary commands not in policy (422); simulation kind accepts its declared harness; build kind flows unchanged.
|
||||
|
||||
### Wave 4-3: learner surface kind-awareness
|
||||
|
||||
- **Task 4-3-01** (frontend-engineer): `use-sandbox-session.ts` — `test()` uses `variant.test_command`; **G-15: TS splits on whitespace ONLY (no shlex in the browser — safe because templates validated quote-free at authoring, Task 4-1-01)**; `build-surface.tsx` — RunControls commands from the variant; starter-file materialization loop already kind-agnostic (verify against design/sim starter sets); honest busy/denied/error states carry over.
|
||||
- **MH-4c**: web tests — test command comes from the variant; Run button label/command per kind.
|
||||
|
||||
### Wave 4-4: telemetry + grading proof
|
||||
|
||||
- **Task 4-4-01** (sandbox-engineer): digest pin test — `compute_digest` over a synthetic design-kind trace (validator harness events) → same feature classes as build traces (kind-agnostic by construction — now pinned); telemetry capture agent unchanged (content-agnostic `_EVENT_KINDS` verified).
|
||||
- Files: `apps/ai-service/ai_service/variants/templates.py`, `generator.py`, `store.py`, `api/variants.py`, `api/sandboxes.py`, `grading/features.py` (tests only), `apps/web/hooks/use-sandbox-session.ts`, `apps/web/components/learner/build-surface.tsx`, `packages/types/variants.ts`
|
||||
- **MH-4d** (absorbs former MH-4e per G-17 — no hope-shaped must-haves): design-kind E2E in the real-server harness — design variant → sandbox → starter files → Run validator harness in-ns → telemetry flows → digest computes (mock LLM, real stores) with concrete assertions: stored seqs contiguous 0..N exactly once AND the agent's spool ends ≤ the ack margin (real agent; where a fake agent is used, cite the P1 suite as the ack/trim coverage instead of asserting).
|
||||
|
||||
**Verification strategy P4:** `pnpm ai:test`, ruff, typecheck, web tests, `pnpm build`. Grading digest diff vs build traces = zero behavioral drift (pinned).
|
||||
|
||||
**Risks:** starter-file materialization is client-driven (missing-file hazard — mitigate: starter sets are small + template-authored; document server-side materialization as a future seam); `test_command` is wire-visible (template-authored, not learner-authored — documented); dual TS/Python schema sync (rule enforced in review).
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Final Review + Ship (milestone release v0.4.5)
|
||||
|
||||
- **Task 5-1-01** (lead-developer → ci-review personas): multi-persona review of all v0.5 changes (correctness, testing, security, performance, maintainability) — P0s fixed in-phase.
|
||||
- **Task 5-2-01** (ci-audit): reconstruction test (git log ↔ .ciagent/ files), file/branch/commit discipline, tag hygiene; critical fixes in-phase.
|
||||
- **Task 5-3-01** (lead-developer → ci-ship): merge phase/05 → milestone → main; tag **v0.4.5** (milestone release); Gitea release with full summary + `nextcraft-linux-x64` + `.sha256` assets; delete milestone branches; mark requirements complete; clear checkpoint.
|
||||
|
||||
---
|
||||
|
||||
## Must-Haves (Milestone)
|
||||
|
||||
- **MH-M1**: Mid-burst disconnect loses zero events outside the acked margin (P1 regression test, repeatable).
|
||||
- **MH-M2**: `openai-audio` provider passes byte-contract STT/TTS tests with key-redaction pins; defense flow works server-side end-to-end (manual probe documented); mock/browser paths unchanged.
|
||||
- **MH-M3**: Identity flow (submit → pending → verified) works under mock; gates enforce 16+/18+ in the binding composition order with verify-CTA payloads; PII never stored raw or logged (caplog sentinel green).
|
||||
- **MH-M4**: Design + simulation environment kinds provisionable with per-kind starter contents + exec policy; grading digest proven kind-agnostic.
|
||||
- **MH-M5**: All gates green at every phase ship: `pnpm ai:test`, ruff, `pnpm typecheck`, `pnpm build`, web + cli tests; binary assets on every release (D-036).
|
||||
- **MH-M6**: v0.1 surface regressions: zero **for verified allowlisted pilot learners** (existing learner/marketplace/employer/admin flows unchanged except the additive enroll route + badges/CTAs; gating unverified/under-age ids is REQ-5-004's purpose, not a regression).
|
||||
|
||||
**Out of scope (guarded):** real KYC vendor integration (protocol only), real auth sessions (self-asserted learner_id documented as pilot-scale), marketplace backend beyond the one gated stub route, new isolation tech (D-024 unchanged), server-side starter materialization (documented as future seam).
|
||||
+58
-15
@@ -8,38 +8,79 @@ Nextcraft is an AI-native outcome school where graduates prove what they can bui
|
||||
|
||||
---
|
||||
|
||||
## Current Milestone: v0.3 — Credential Engines
|
||||
## Milestone v0.5 — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease (COMPLETE, shipped as v0.4.5)
|
||||
|
||||
**Scope:** Replace v0.2's mock engine inputs with real credential engines. Build the sandbox fabric (sandboxed IDE / design tool / simulation), the live in-environment build-telemetry pipeline, the process-trace grading engine, per-learner variant task generation, and the oral/voice defense with AI examiner. Lab/Assessor/Proctor agents move from mock inputs to real engine inputs; the six tutor agents operate on authentic telemetry and artifacts.
|
||||
**Scope (the four seams D-016 deferred out of v0.4, locked at v0.5 Phase 0 SPECIFY):**
|
||||
|
||||
1. **Real server STT/TTS** (CUT-1/G-7 seam): implement the `openai-audio` VoiceProvider against OpenAI-compatible `/audio/transcriptions` (STT) + `/audio/speech` (TTS) — the D-030 protocol drop-in; mock stays first-class for tests; browser fallback stays for the no-key path. Voice defense (Examiner flows) gains the real server path end-to-end.
|
||||
2. **KYC/identity + age-gating backend** (REQ-F-017): real identity verification behind a provider protocol (mock-first, D-014 pattern), age-gate enforcement (school 16+, marketplace 18+ with verified identity) replacing the v0.1 visual-only flow, session/learner identity wired into the API surface (the G-5 allowlist evolves toward real identity).
|
||||
3. **Design/simulation sandbox environments** (REQ-F-021 remainder): extend the D-024/D-025 namespace fabric beyond the coding IDE — design-tool and simulation environment types alongside the existing build environment, one lifecycle, per-type telemetry.
|
||||
4. **Exec-telemetry seq-lease / replay-margin fix**: the one-line ACK gap documented in the P6 lesson (P07 review): reconnect replay margin so at-least-once ingest acknowledges received seqs and the capture agent resumes from the ack — closing the documented gap.
|
||||
|
||||
**Success (milestone-level):** voice defense runs on real server STT/TTS when keys exist; age-gating is enforced by the backend (not a mockup page); the sandbox fabric provisions design/sim environments; the reconnect-replay gap is closed with a regression test.
|
||||
|
||||
## Prior Milestone: v0.4 — Distribution & Bootstrap CLI (COMPLETE, shipped as v0.3.4; hotfixes v0.3.5 fresh-box experience, v0.3.6 single-port unattended deploy)
|
||||
|
||||
**Scope (founder directive, 2026-09-12):** Streamline installing Nextcraft. Ship a bootstrap CLI with a single-liner install script, and publish release binaries on an ongoing basis for every release going forward.
|
||||
|
||||
**Delivered:** `nextcraft` CLI — `doctor` (prerequisite checks), `bootstrap` (deps + venv + env from templates + key validation), `verify` (health check), `dev` (thin passthrough to scripts/dev.sh); one-liner install script downloading the linux x64 binary from the latest Gitea release with sha256 + version integrity gates; binary release pipeline attached to every ship from v0.3.2 onward; install/quickstart documentation backed by a fresh-clone E2E test. All 5 requirements (REQ-4-001..005) complete.
|
||||
|
||||
**Status of v0.3:** Complete and shipped (v0.2.8). Credential engines live: namespace-isolated sandbox fabric, live build telemetry, process-trace grading, seeded variants, oral defense; real learner build/defense/grading surfaces.
|
||||
|
||||
**Status of v0.2:** Complete and shipped (v0.2.0). Six AI tutor agents live over mockengine inputs (D-015).
|
||||
|
||||
**Deferred from earlier plan:** REQ-F-017 (identity verification + 16+/18+ age-gating) is explicitly deferred to a later milestone per founder directive. Age-gating remains the v0.1-style visual flow mockup; no real KYC backend is built in v0.3.
|
||||
**Deferred from earlier plan:** REQ-F-017 (identity verification + age-gating KYC), real server STT/TTS, design/simulation sandbox environments, and the exec-telemetry seq-lease are all deferred to v0.5. Age-gating remains the v0.1-style visual flow mockup.
|
||||
|
||||
**Tech stack:** v0.1 TS monorepo (pnpm/turborepo, Next.js) + v0.2 Python FastAPI ai-service + new credential-engine services (sandbox fabric orchestrator, telemetry ingest, grading engine) in Python/TypeScript as determined at RESEARCH.
|
||||
|
||||
---
|
||||
|
||||
## Requirements (Validated)
|
||||
## v0.4 Requirements (Complete)
|
||||
|
||||
The following requirements have been validated during specification and are locked for milestone v0.3 (REQ-F-007..010 and REQ-F-021 activated from the deferred pool; REQ-F-017 deferred per founder directive):
|
||||
All 5 v0.4 requirements (REQ-4-001..005) are complete and shipped as v0.3.4:
|
||||
1. Bootstrap CLI — `nextcraft` executable with `doctor` / `bootstrap` / `verify` / `dev` (REQ-4-001, REQ-4-002)
|
||||
2. One-liner install — `curl | sh` fetching the linux x64 binary from the latest Gitea release with sha256 + version integrity verification (REQ-4-003)
|
||||
3. Ongoing release binaries — every release from v0.3.2 onward ships the CLI binary + checksum as release assets (REQ-4-004)
|
||||
4. Install documentation — README quickstart + CLI reference verified by a fresh-clone E2E test (REQ-4-005)
|
||||
|
||||
1. Sandbox fabric — sandboxed IDE, design tool, and simulation environments with isolated execution and lifecycle management (REQ-F-021)
|
||||
2. Live build telemetry — in-environment capture of process events (keystrokes, commands, file diffs, run/test results) streamed to ai-service (REQ-F-010)
|
||||
3. Process-trace grading engine — grade artifacts from their process traces, not just final output (REQ-F-007); feeds the Assessor agent real inputs
|
||||
4. Variant task generation — per-learner task variants so no two learners receive identical prompts (REQ-F-008)
|
||||
5. Oral/voice defense — AI examiner conducts spoken defense of submitted work (REQ-F-009); feeds the Proctor/Mentor agents
|
||||
6. Agent re-grounding — Lab/Assessor/Proctor consume real engine inputs (telemetry, traces, defenses) instead of v0.2 mocks
|
||||
7. Learner surface integration — wire the v0.1 sandbox + assessment mockups to the real engines (build/run in-browser, live telemetry, live defense)
|
||||
## v0.3 Requirements (Complete)
|
||||
|
||||
## v0.2 Requirements (Complete)
|
||||
|
||||
All 12 v0.2 requirements (REQ-2-001..012) are complete and shipped as v0.2.0. See REQUIREMENTS.md traceability matrix.
|
||||
All 8 v0.3 requirements (REQ-3-001..008) are complete and shipped as v0.2.8. See REQUIREMENTS.md traceability matrix.
|
||||
|
||||
## v0.1 Requirements (Complete)
|
||||
|
||||
All 28 v0.1 requirements (REQ-001..028) are complete and shipped as v0.1.0. See REQUIREMENTS.md traceability matrix.
|
||||
|
||||
## Clarified Assumptions (v0.5 CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
|---|-----------|------------|------------|
|
||||
| A-301 | Which STT/TTS endpoint for `openai-audio`? | **OpenAI-compatible `/audio/transcriptions` + `/audio/speech`** via `AI_VOICE_BASE_URL` (ollama-cloud does not expose audio endpoints; any OpenAI-compatible audio API works — the provider is endpoint-agnostic by config, D-014 pattern). Model via `AI_VOICE_MODEL` (default `whisper-1` STT / `tts-1` TTS class names, configurable). Keys env-only, never committed. | 0.75 |
|
||||
| A-302 | Does real voice replace browser fallback? | **No — layered.** `openai-audio` when configured, browser-native second-class fallback (CUT-1 keeps it first-class for the no-key path), deterministic mock for tests. The defense surface probes provider capability and degrades with a visible badge (mic mock vs browser vs server). | 0.85 |
|
||||
| A-303 | KYC vendor in v0.5? | **No vendor — protocol + mock-first backend.** `IdentityProvider` protocol (submit/poll/verify), deterministic mock (approve-on-policy), SQLite store. A real vendor (Stripe Identity/Persona/Onfido class) drops in later without API changes. Solo-founder economics: no vendor spend before pilot. | 0.85 |
|
||||
| A-304 | What counts as "verified 18+" for marketplace? | **Provider verdict = DOB-verified 18+; until a real vendor exists the mock verdict is explicit and marked `mock` in API responses** so downstream surfaces can label unverified state honestly (never display mock-verified as production-verified). | 0.80 |
|
||||
| A-305 | PII storage? | **Server-side only, minimal**: document refs + provider verdicts in SQLite; raw documents NEVER stored, NEVER logged (payload scrubbing pinned by test). Enrollment DOB stored as derived age-band, not raw DOB where possible. | 0.85 |
|
||||
| A-306 | Age-gate enforcement points? | **School enrollment (16+)**: identity submit → verify → age check at enrollment API. **Marketplace (18+ verified)**: gated marketplace routes check verified-identity verdict; unverified → 403 with verify-CTA payload. G-5 learner allowlist REMAINS as a pilot guard (identity doesn't replace sandbox rate caps). | 0.82 |
|
||||
| A-307 | What are "design" and "simulation" environments concretely? | **Same namespace fabric, typed starter contents + allowed commands.** `design`: canvas-style artifacts (files the learner edits: SVG/HTML/schematic text), Run = validator/renderer harness command; `simulation`: parameterized run harness (benchmark scripts + dataset files), Run = bounded harness execution. No new isolation tech — D-024 unchanged; the ENVIRONMENT is a typing over starter files + command policy. | 0.78 |
|
||||
| A-308 | Does the learner UI need new surfaces for design/sim? | **Reuse the build surface.** The existing /build flow accepts `environment` kind; the file tree, editor, Run/Test buttons, and read-only output panel (CUT-2) all carry over; only starter contents and command policy differ per kind. No new route groups; variants carry the kind. | 0.80 |
|
||||
| A-309 | Seq-lease protocol shape? | **Ingest acks highest-contiguous received seq per (learner, task) on the existing WS (JSON ack frame); the capture agent trims its spool to the ack and replays from there on reconnect.** Replay margin bounded (spool cap already exists). No new endpoint; G-3 flood semantics unchanged; acks are advisory hints, gap detection stays authoritative. | 0.80 |
|
||||
| A-310 | Where does identity live in the app? | **New `ai_service/identity/` module (D-031 pattern — extend ai-service, no new apps)**: provider protocol + mock, SQLite store (D-027 family), `/v1/identity/*` router. Web enrollment + marketplace surfaces call it via engine-client. | 0.85 |
|
||||
|
||||
## Clarified Assumptions (v0.4 CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
|---|-----------|------------|-------------|
|
||||
| A-201 | CLI language/toolchain for the binary? | **Probe-driven at RESEARCH** — Go → Rust → Node SEA → Python zipapp fallback chain; spec stays toolchain-agnostic so PLAN locks the probe-verified toolchain | 0.70 |
|
||||
| A-202 | Does bootstrap replace scripts/bootstrap.sh? | **No — reuse it.** CLI wraps existing `scripts/bootstrap.sh` + `scripts/dev.sh` via subprocess; zero orchestration logic duplicated in the CLI (thin passthrough pattern) | 0.85 |
|
||||
| A-203 | Where does the one-liner fetch the binary? | **Gitea latest-release API** (`/repos/{owner}/{repo}/releases/latest`) → download `nextcraft-linux-x64` + `.sha256` asset; repo raw serves `install.sh` as the stable URL | 0.80 |
|
||||
| A-204 | Install target + PATH? | **~/.local/bin** (XDG-style, no sudo), PATH hint printed when missing; `--dest` override flag | 0.85 |
|
||||
| A-205 | Binary "ongoing releases" scope? | **Every ship from v0.4 onward** attaches `nextcraft-linux-x64` + sha256 sidecar to the Gitea release — the ship workflow gains an asset step; retroactive binaries for old releases NOT required | 0.90 |
|
||||
| A-206 | No binary available yet / non-linux? | **Graceful degradation**: install script prints source-bootstrap instructions (git clone + scripts/bootstrap.sh) — never a hard fail | 0.88 |
|
||||
| A-207 | Checksum trust root? | **sha256 sidecar shipped as a release asset next to the binary** (same release, same channel); script verifies download against it. Signature/PKI out of scope for v0.4 (single forge, TLS transport) | 0.75 |
|
||||
| A-208 | Which prerequisites does doctor check? | node ≥18, pnpm ≥8, python3 ≥3.11, git, `unshare` availability (sandbox fabric needs it) — versions from the existing bootstrap tooling, not invented | 0.85 |
|
||||
| A-209 | Does `dev` manage multiple processes? | **No.** Thin passthrough to scripts/dev.sh only — the CLI stays bootstrap-scoped (D-016); orchestration remains in dev.sh | 0.82 |
|
||||
| A-210 | `.env.secrets` handling by bootstrap? | **Template copy only for `.env.example` → `.env`; secrets NEVER generated, NEVER committed; bootstrap validates presence of optional keys and warns (not blocks) when missing — mock-first providers keep the stack runnable** | 0.90 |
|
||||
|
||||
## Clarified Assumptions (v0.3 CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
@@ -133,6 +174,8 @@ The following remain deferred beyond v0.3 and will be activated in subsequent mi
|
||||
| D-013 | v0.1 prototype founder-agreed; D-001 business-logic gate unlocked | Founder approved starting v0.2 with AI Tutor Architecture, which constitutes agreement of the v0.1 prototype per D-001. Recorded at v0.2 SPECIFY. | Business logic authorized from v0.2 onward |
|
||||
| D-014 | Provider-agnostic LLM layer; ollama-cloud as initial provider | OpenAI-compatible client abstraction with pluggable providers: ollama-cloud (https://ollama.com/v1, default), local OpenAI-compatible endpoint, deterministic mock (tests/CI). Keys in gitignored .ciagent/.env.secrets, never in code or commits. | apps/ai-service llm package with 3 providers; default=ollama-cloud |
|
||||
| D-015 | All six agents implemented as real LLM services; engines mocked | Coach/Tutor/Mentor fully real. Lab/Assessor/Proctor are real LLM logic over mock inputs (simulated telemetry, pre-baked artifacts) since sandbox fabric, assessment engine, and identity verification are v0.3+. Consistent with v0.1's mock-data approach. | REQ-F-001..006 complete in v0.2; real engines deferred to v0.3+ |
|
||||
| D-016 | v0.4 = Distribution & Bootstrap CLI (founder directive supersedes previously-named v0.4 seams) | Founder directive 2026-09-12: focus this milestone on streamlining install, a bootstrap CLI with a one-liner install script, ongoing release binaries. Real server STT/TTS, KYC, design/simulation envs, seq-lease move to v0.5. | Milestone scope locked at SPECIFY; binary = CLI-only, linux x64 |
|
||||
| D-039 | Single-port deploy + unattended dev (v0.3.6 hotfix, founder directive) | Only :8420 reachable behind HAProxy; site was down (domain root hit the API 404). Web app = static export served same-origin by the ai-service (`AI_WEB_STATIC_DIR`); `dev -d`/`stop`/`log` daemonize ops; durable state (DB, sandboxes, pid/log) moves to `~/.nextcraft/` out of the repo. | One port serves UI+API; unattended deploy recipe in deploy/README.md |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,5 +1,51 @@
|
||||
# Nextcraft — REQUIREMENTS.md
|
||||
|
||||
## v0.5 Requirements (Complete — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease; shipped as v0.4.5)
|
||||
|
||||
### Real Server Voice
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-001 | `openai-audio` VoiceProvider: server STT (`/audio/transcriptions`) + TTS (`/audio/speech`) against an OpenAI-compatible endpoint via the existing D-030 protocol; provider selection by `AI_VOICE_PROVIDER` (+ base URL/key from env, never committed); deterministic mock stays first-class; browser fallback unchanged | critical | 2 | complete |
|
||||
| REQ-5-002 | Voice defense real path end-to-end: examiner dialogue answers transcribed server-side (audio upload → transcript), examiner questions spoken via server TTS (audio returned to the client); transcripts + integrity signals unchanged | critical | 2 | complete |
|
||||
|
||||
### Identity & Age-Gating
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-003 | Identity provider protocol + mock-first backend (REQ-F-017): verify-identity flow (submit → pending → verified/rejected with document refs), provider-agnostic (mock default; a real KYC vendor drops in later), PII stored server-side only, never logged | critical | 3 | complete |
|
||||
| REQ-5-004 | Age-gating enforced by the backend: school floor 16+ verified at enrollment, marketplace 18+ with verified identity — API surfaces reject under-age/unverified callers on gated routes (replaces the v0.1 visual-only flow; G-5 allowlist evolves toward real identity, allowlist remains as pilot guard) | critical | 3 | complete |
|
||||
|
||||
### Sandbox Environments
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-005 | Design + simulation sandbox environment types (REQ-F-021 remainder): extend the namespace fabric with environment kinds beyond the coding IDE (design-tool: canvas/editor surfaces with file artifacts; simulation: run/benchmark harnesses) — one lifecycle, one telemetry path, per-type starter contents + allowed commands | high | 4 | complete |
|
||||
| REQ-5-006 | Environment-typed learner surface: the build/defend flow accepts environment kind, telemetry captures per-kind events, grading digest stays kind-agnostic | high | 4 | complete |
|
||||
|
||||
### Telemetry Durability
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-007 | Exec-telemetry seq-lease/replay-margin fix: WS ingest acknowledges received seqs; capture agent resumes from the ack on reconnect (bounded replay margin) — closes the P6-lesson one-line ACK gap with a real-server regression test | high | 1 | complete |
|
||||
|
||||
## v0.4 Requirements (Complete — Distribution & Bootstrap CLI, shipped as v0.3.4)
|
||||
|
||||
### Bootstrap CLI
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-4-001 | `nextcraft` CLI (linux x64 binary): `doctor` command checking prerequisites (node, pnpm, python3, git, unshare) with actionable error messages | critical | 1 | complete |
|
||||
| REQ-4-002 | `bootstrap` command: pnpm install, ai-service venv + pinned deps, .env from templates, key validation, .env.secrets handling; `verify` health check (ports, imports, builds); `dev` thin passthrough to scripts/dev.sh | critical | 1 | complete |
|
||||
|
||||
### Distribution
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-4-003 | One-liner install script (`curl -fsSL <url> \| bash`): detects linux x64, resolves latest release from Gitea API, downloads binary + checksum, verifies sha256, installs to ~/.local/bin (PATH hint), degrades to source-bootstrap instructions when no binary | critical | 2 | complete |
|
||||
| REQ-4-004 | Binary release pipeline: reproducible linux x64 build script, sha256 checksum sidecar, upload as release assets on every ship from v0.4 onward (ongoing binaries requirement) | critical | 2 | complete |
|
||||
| REQ-4-005 | Install + quickstart documentation: README one-liner quickstart, CLI command reference, fresh-clone-to-running-stack end-to-end verification | high | 3 | complete |
|
||||
|
||||
## v0.3 Requirements (Credential Engines)
|
||||
|
||||
### Sandbox & Telemetry
|
||||
@@ -147,7 +193,7 @@
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.4+ | deferred (deferred from v0.3 per founder directive) |
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.5 | activated → complete (REQ-5-003/004) |
|
||||
| REQ-F-018 | Payment processing and subscription management | high | v0.3+ | deferred |
|
||||
| REQ-F-019 | Human tutor marketplace (third-party courses) | medium | v0.4+ | deferred |
|
||||
| REQ-F-020 | CIRR-style placement tracking and audit | medium | v0.5+ | deferred |
|
||||
@@ -170,7 +216,29 @@
|
||||
|
||||
## Traceability Matrix
|
||||
|
||||
### v0.3 (current milestone)
|
||||
### v0.5 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-5-007 | 1 | complete |
|
||||
| REQ-5-001 | 2 | complete |
|
||||
| REQ-5-002 | 2 | complete |
|
||||
| REQ-5-003 | 3 | complete |
|
||||
| REQ-5-004 | 3 | complete |
|
||||
| REQ-5-005 | 4 | complete |
|
||||
| REQ-5-006 | 4 | complete |
|
||||
|
||||
### v0.4 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-4-001 | 1 | complete |
|
||||
| REQ-4-002 | 1 | complete |
|
||||
| REQ-4-003 | 2 | complete |
|
||||
| REQ-4-004 | 2 | complete |
|
||||
| REQ-4-005 | 3 | complete |
|
||||
|
||||
### v0.3 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
|
||||
+57
-120
@@ -2,15 +2,19 @@
|
||||
|
||||
## Overview
|
||||
|
||||
**Milestone v0.3** — Credential Engines: Replace v0.2's mock engine inputs with real credential engines. Build the sandbox fabric (sandboxed IDE / design tool / simulation), the live in-environment build-telemetry pipeline, the process-trace grading engine, per-learner variant task generation, and the oral/voice defense with AI examiner. Lab/Assessor/Proctor agents move from mock inputs to real engine inputs.
|
||||
**Milestone v0.5 — COMPLETE (shipped as v0.4.5, 2026-09-13).** Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease: the four seams D-016 deferred out of v0.4. Server STT/TTS for voice defense (`openai-audio` VoiceProvider), identity verification + backend-enforced age-gating (REQ-F-017), design/simulation sandbox environments (REQ-F-021 remainder), and the exec-telemetry seq-lease/replay-margin fix.
|
||||
|
||||
**Deferred per founder directive:** REQ-F-017 identity verification + age-gating (real KYC backend) is deferred beyond v0.3. Age-gating remains the v0.1 visual flow mockup.
|
||||
**Milestone v0.4 — COMPLETE (shipped as v0.3.4, 2026-09-13; hotfixes v0.3.5 fresh-box, v0.3.6 single-port unattended deploy).** Distribution & Bootstrap CLI: `nextcraft` CLI (doctor/bootstrap/verify/dev) shipped as a linux x64 SEA binary, one-liner install script with checksum + version integrity gates, and binaries published on **every ongoing release** (v0.3.2 onward). v0.3.6 adds the single-port same-origin deploy (static export served by the ai-service on :8420) and unattended ops (`dev -d`/`stop`/`log`), with runtime state moved to `~/.nextcraft/`.
|
||||
|
||||
**Milestone v0.3** — Credential Engines: complete, shipped as v0.2.8 (2026-09-12). Real sandbox fabric, live build telemetry, process-trace grading, per-learner variants, oral defense, real learner surfaces.
|
||||
|
||||
**Deferred per founder directive (D-016):** REQ-F-017 identity verification + age-gating (real KYC backend), real server STT/TTS, design/simulation sandbox environments, and the exec-telemetry seq-lease are deferred to v0.5. Age-gating remains the v0.1 visual flow mockup.
|
||||
|
||||
**Prior milestone:** v0.2 (ai-tutor-architecture) — complete, shipped as v0.2.0, six tutor agents live over mock engine inputs (D-015).
|
||||
|
||||
**Milestone type:** Feature (new credential-engine services + real agent inputs)
|
||||
**Tag line:** v0.2.x (patches on the v0.2 line; milestone release as the final v0.2.x patch)
|
||||
**Branch:** milestone/v0.3-credential-engines
|
||||
**Milestone type:** Feature (voice provider + identity backend + new env types + ingest protocol fix)
|
||||
**Tag line:** v0.4.x (patches on the v0.4 line; milestone release as the final v0.4.x patch)
|
||||
**Branch:** milestone/v0.5-real-voice-identity-envs
|
||||
|
||||
---
|
||||
|
||||
@@ -18,14 +22,12 @@
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan complete; .ciagent/ files updated for v0.3 |
|
||||
| 1 | Sandbox fabric | complete | 0 | REQ-3-001, REQ-3-002 | Isolated per-learner sandbox environments provisioned (IDE / design / simulation); lifecycle API (create/destroy/snapshot); resource limits enforced; no cross-tenant access |
|
||||
| 2 | Live build telemetry | complete | 1 | REQ-3-003 | In-environment capture of process events (commands, file diffs, run/test results, keystroke-level activity) streamed to ai-service; reliable transport; per-learner trace persistence |
|
||||
| 3 | Process-trace grading engine | complete | 2 | REQ-3-004 | Grades artifacts from their full process traces (not just final output); emits structured rubric-aligned scores; feeds Assessor real inputs |
|
||||
| 4 | Variant task generation | complete | 1 | REQ-3-005 | Per-learner task variants generated so no two learners receive identical prompts; variant seed recorded for grading fairness |
|
||||
| 5 | Oral / voice defense | complete | 3 | REQ-3-006 | AI examiner conducts spoken defense of submitted work; STT → dialogue → TTS; transcript + integrity signals captured; feeds Proctor/Mentor |
|
||||
| 6 | Agent re-grounding + learner surface integration | complete | 2,3,4,5 | REQ-3-007, REQ-3-008 | Lab/Assessor/Proctor consume real engine inputs; v0.1 sandbox + assessment mockups wired to real engines (in-browser build/run, live telemetry, live defense) |
|
||||
| 7 | Final review + ship | in-progress | 6 | — | Code review clean; audit passes; milestone tagged (v0.2.x final patch); release created on Gitea |
|
||||
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan, grill complete; .ciagent/ files updated for v0.5 |
|
||||
| 1 | Seq-lease + replay margin | complete | 0 | REQ-5-007 | Ingest acks received seqs (`seq_ack` frame); capture agent trims spool to ack on reconnect (bounded margin); real-server mid-burst regression test closes the P07-documented gap |
|
||||
| 2 | Real server voice | complete | 0 | REQ-5-001, REQ-5-002 | `openai-audio` provider passes STT/TTS contract tests (MockTransport); voice defense runs server-side end-to-end when keys exist; mock/browser paths unchanged; suite green |
|
||||
| 3 | Identity + age-gating | complete | 0 | REQ-5-003, REQ-5-004 | Identity protocol + mock backend + verification flow API; gated routes enforce 16+/18+ (allowlist → identity → rate caps); PII hygiene pinned by caplog test |
|
||||
| 4 | Design/sim environments | complete | 0 | REQ-5-005, REQ-5-006 | Template-layer env registry (build/design/simulation); per-kind starter contents + exec policy; test_command surfaced; grading digest kind-agnostic (pinned) |
|
||||
| 5 | Final review + ship | complete | 1-4 | — | Code review clean; audit passes; milestone tagged (final v0.4.x patch); release with binary assets on Gitea |
|
||||
|
||||
---
|
||||
|
||||
@@ -33,147 +35,82 @@
|
||||
|
||||
### Phase 0: Pre-execution
|
||||
|
||||
**Goal:** Establish v0.3 specification, clarify ambiguities, research credential-engine architecture (sandbox isolation, telemetry transport, trace grading, variant generation, voice IO), create detailed plans.
|
||||
**Goal:** Lock the v0.5 specification (four deferred seams), clarify ambiguities, research the audio endpoint contract + KYC provider landscape + env-type design + the seq-lease protocol, plan waves, grill adversarially.
|
||||
|
||||
**Stages:** SPECIFY → CLARIFY → RESEARCH → PLAN → GRILL → MVP/UX CHECK → SHIP
|
||||
|
||||
**Deliverables:**
|
||||
- Updated .ciagent/config.json, PROJECT.md, REQUIREMENTS.md, ROADMAP.md, ARCHITECTURE.md, PERSONAS.md, PLAN.md
|
||||
|
||||
**Success criteria:** All .ciagent/ files updated for v0.3; phase 0 shipped as first v0.2.x patch.
|
||||
**Success criteria:** All .ciagent/ files updated for v0.5; phase 0 shipped as v0.4.0.
|
||||
|
||||
---
|
||||
### Phase 1: Seq-Lease + Replay Margin (REQ-5-007)
|
||||
|
||||
### Phase 1: Sandbox Fabric
|
||||
|
||||
**Goal:** Provision and manage isolated per-learner execution environments.
|
||||
|
||||
**Requirements:** REQ-3-001, REQ-3-002
|
||||
**Goal:** Close the reconnect-replay ACK gap from the v0.3 P6 lesson.
|
||||
|
||||
**Key deliverables:**
|
||||
- Sandbox orchestrator service: create/list/destroy/snapshot sandbox instances (coding IDE — design tool and simulation environments deferred to v0.4 per D-025)
|
||||
- Isolation boundary: per-learner Linux user/mount/pid/net namespace subprocess isolation (`unshare`, D-024); no cross-tenant filesystem/network access
|
||||
- Resource limits: CPU/memory/single-file-size quotas (rlimits) + wall-clock timeout reaper + best-effort workdir-size sweep; per-sandbox pids + hard disk quota accepted as v0.3 gaps (G-1/G-2)
|
||||
- Sandbox lifecycle API consumed by ai-service and the web learner surface
|
||||
- Ingest WS: ack frames carrying highest-contiguous received seq per (learner, task); capture agent tracks ack and resumes spool replay from acked position (bounded replay margin)
|
||||
- Real-server regression test: mid-burst kill → reconnect → no loss/dup exactly-once within margin (deterministic — waits for server-side observation like the P07 fix)
|
||||
|
||||
**Success criteria:**
|
||||
- A sandbox can be created, written to, snapshotted, and destroyed via API
|
||||
- Isolation verified: a sandbox cannot read another learner's data
|
||||
- Resource limits enforced and observable — enforcement mechanism: rlimits (memory/CPU) + wall-clock reaper + workdir-size sweep; per-sandbox pids and hard-disk-quota are accepted v0.3 gaps (no cgroup delegation/sudo on this box, G-1)
|
||||
**Success criteria:** regression test green on repeated runs; margin documented; flood/gap semantics (G-3/G-4) unchanged.
|
||||
|
||||
---
|
||||
### Phase 2: Real Server Voice (REQ-5-001, REQ-5-002)
|
||||
|
||||
### Phase 2: Live Build Telemetry
|
||||
|
||||
**Goal:** Capture in-environment process events and stream them to ai-service reliably.
|
||||
|
||||
**Requirements:** REQ-3-003
|
||||
**Goal:** The `openai-audio` VoiceProvider against OpenAI-compatible STT/TTS endpoints; voice defense real path end-to-end.
|
||||
|
||||
**Key deliverables:**
|
||||
- Telemetry capture agent (in-sandbox): commands, file diffs, run/test results, keystroke-level/activity events
|
||||
- Telemetry transport: durable, ordered, resumable stream to ai-service ingestion endpoint
|
||||
- Trace persistence: per-learner, per-task process traces stored for grading and proctoring
|
||||
- Transport hardening: retries, backpressure, exactly-once-or-at-least-once semantics documented
|
||||
- `ai_service/voice/openai_audio.py` — provider per D-030 protocol (STT: multipart upload → transcript; TTS: text → audio bytes); httpx via the app pool; timeouts; error mapping
|
||||
- `AI_VOICE_BASE_URL`/`AI_VOICE_API_KEY`/`AI_VOICE_MODEL` settings resolution (env-only, never committed); `AI_VOICE_PROVIDER=openai-audio|browser|mock` selection in factory
|
||||
- Defense answer route: real STT path (audio upload transcribed server-side when provider=openai-audio; browser/mock paths unchanged); examiner TTS question audio
|
||||
- Tests: MockTransport byte-contract tests (multipart shape, response parse, error modes); defense-flow tests stay mock-only (never call the cloud)
|
||||
|
||||
**Success criteria:**
|
||||
- Sandbox activity produces a complete ordered process trace in ai-service
|
||||
- Stream survives transient network failure without trace loss
|
||||
- Trace retrievable by learner+task ID for grading
|
||||
**Success criteria:** provider contract pinned by tests; voice defense works against a scripted audio endpoint; `pnpm ai:test` green; manual cloud probe documented.
|
||||
|
||||
---
|
||||
### Phase 3: Identity + Age-Gating (REQ-5-003, REQ-5-004)
|
||||
|
||||
### Phase 3: Process-Trace Grading Engine
|
||||
|
||||
**Goal:** Grade learner artifacts from their full process traces.
|
||||
|
||||
**Requirements:** REQ-3-004
|
||||
**Goal:** Real identity verification backend behind a provider protocol; backend-enforced age gates.
|
||||
|
||||
**Key deliverables:**
|
||||
- Trace analyzer: reconstructs build/decision timeline from a process trace
|
||||
- Grading engine: rubric-aligned scoring over the trace (process quality, not just final artifact)
|
||||
- Structured score output consumable by the Assessor agent
|
||||
- Calibration against v0.2 mock corpora to validate grading dimensions
|
||||
- `ai_service/identity/` — IdentityProvider protocol (submit/poll/verify), mock provider (deterministic), store (SQLite, D-027 family), API router (`/v1/identity/*`)
|
||||
- Age-gate enforcement: school 16+ (enrollment), marketplace 18+ verified (gated routes reject under-age/unverified with 403) — dependency-injected gate, allowlist (G-5) retained as pilot guard
|
||||
- PII hygiene: document refs stored, never raw docs in logs; secrets-hygiene checklist extended
|
||||
|
||||
**Success criteria:**
|
||||
- Engine emits structured rubric-aligned scores from a real process trace
|
||||
- Scores distinguish process quality (e.g., iterative debugging vs. paste-and-run)
|
||||
- Output feeds Assessor; replaces pre-baked artifact corpus inputs
|
||||
**Success criteria:** verification flow API green under mock; gated routes enforce ages in tests; zero PII in captured logs (pinned by test).
|
||||
|
||||
---
|
||||
### Phase 4: Design/Sim Environments (REQ-5-005, REQ-5-006)
|
||||
|
||||
### Phase 4: Variant Task Generation
|
||||
|
||||
**Goal:** Generate per-learner task variants so no two learners receive identical prompts.
|
||||
|
||||
**Requirements:** REQ-3-005
|
||||
**Goal:** Environment kinds beyond the coding IDE on the existing namespace fabric.
|
||||
|
||||
**Key deliverables:**
|
||||
- Variant generator: parameterized task templates → unique per-learner instances
|
||||
- Variant seed registry: record variant parameters for grading fairness and proctoring
|
||||
- Difficulty normalization: variants calibrated to equivalent difficulty
|
||||
- Environment-type registry in the sandbox fabric: `build` (existing), `design` (canvas/file artifacts), `simulation` (run/benchmark harness) — per-type starter contents + allowed exec commands, one lifecycle, one telemetry path
|
||||
- Learner surface + engine-client accept environment kind; telemetry events carry kind; grading digest stays kind-agnostic (D-028 unchanged)
|
||||
|
||||
**Success criteria:**
|
||||
- Two learners requesting the same competency receive distinct task variants
|
||||
- Variant parameters persisted and auditable
|
||||
- Grading engine scores variants equitably
|
||||
**Success criteria:** all three kinds provisionable + provable isolation; per-kind starter files land; kind flows through telemetry to the digest.
|
||||
|
||||
### Phase 5: Final Review + Ship
|
||||
|
||||
**Goal:** Code review, audit, milestone release with binary assets.
|
||||
|
||||
**Success criteria:** review P0s fixed in-phase; audit reconstruction clean; final v0.4.x patch tagged; Gitea release with `nextcraft-linux-x64` + `.sha256`; milestone merged to main; branches deleted.
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Oral / Voice Defense
|
||||
## v0.5 (Complete — Shipped as v0.4.5)
|
||||
|
||||
**Goal:** AI examiner conducts a spoken defense of the learner's submitted work.
|
||||
Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease: real server STT/TTS (`openai-audio` VoiceProvider with boot-safe fallback), the identity module (5th D-027 store, provider protocol + mock, D-043 age-gate composition on variants/sandboxes/defense/marketplace), design/simulation environment kinds at the template layer with per-kind exec policy, and the seq-ack protocol closing the P07 replay-margin gap. 6 phases (P0–P5). All 7 requirements (REQ-5-001..007) complete. Tags v0.4.0–v0.4.4 per phase, milestone release v0.4.5.
|
||||
|
||||
**Requirements:** REQ-3-006
|
||||
## v0.4 (Complete — Shipped as v0.3.4)
|
||||
|
||||
**Key deliverables:**
|
||||
- Voice pipeline: STT → defense dialogue (LLM examiner) → TTS
|
||||
- Examiner agent: probes understanding, challenges process choices from the trace
|
||||
- Transcript + integrity signals captured for Proctor/Mentor
|
||||
- Latency budget: defense feels conversational (bounded turn latency)
|
||||
Distribution & Bootstrap CLI: `nextcraft` CLI (doctor/bootstrap/verify/dev) as a self-contained linux x64 SEA binary, one-liner install with sha256 + version integrity gates, release-asset pipeline attaching binaries to every ongoing release (v0.3.2 onward), install/quickstart docs backed by a fresh-clone E2E test. 5 phases (P0–P4). All 5 requirements (REQ-4-001..005) complete. Tags v0.3.0–v0.3.3 per phase, milestone release v0.3.4.
|
||||
|
||||
**Success criteria:**
|
||||
- A spoken defense runs end-to-end (speak → examiner question → learner response → verdict)
|
||||
- Transcript + integrity signals persisted and consumable by Proctor
|
||||
- Turn latency within the documented budget
|
||||
## v0.3 (Complete — Shipped as v0.2.8)
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Agent Re-grounding + Learner Surface Integration
|
||||
|
||||
**Goal:** Move Lab/Assessor/Proctor to real engine inputs; wire learner surface to the real engines.
|
||||
|
||||
**Requirements:** REQ-3-007, REQ-3-008
|
||||
|
||||
**Key deliverables:**
|
||||
- Lab agent consumes live sandbox telemetry (replaces v0.2 mock telemetry)
|
||||
- Assessor agent consumes grading-engine output (replaces pre-baked artifacts)
|
||||
- Proctor consumes telemetry + defense integrity signals (replaces mock telemetry)
|
||||
- Learner sandbox mockup → real in-browser build/run; assessment mockup → live defense + live grading
|
||||
|
||||
**Success criteria:**
|
||||
- Lab/Assessor/Proctor operate on real inputs with no mock fallback in the learner path
|
||||
- Learner can build in-browser and see live telemetry + live feedback
|
||||
- Assessment surface runs a live defense and shows live grading
|
||||
- `pnpm build` and `pnpm typecheck` pass
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Final Review + Ship
|
||||
|
||||
**Goal:** Code review, audit, milestone release.
|
||||
|
||||
**Key deliverables:**
|
||||
- Multi-persona code review (correctness, testing, security, performance, maintainability)
|
||||
- Project health audit (reconstruction test, .ciagent/ file discipline, branch hygiene, commit discipline)
|
||||
- Milestone ship: merge milestone → main, tag final v0.2.x patch, create Gitea release
|
||||
|
||||
**Success criteria:**
|
||||
- Code review: P0 fixes applied, P1+ documented
|
||||
- Audit: all checks pass, project state reconstructable from git log
|
||||
- Ship: milestone tagged, branch merged to main, Gitea release created — release note states identity/age-gating (KYC) is deferred and age-gating remains a visual mockup
|
||||
- All v0.3 requirements marked complete
|
||||
|
||||
---
|
||||
Credential Engines: real sandbox fabric (Linux namespaces), live build telemetry
|
||||
(at-least-once/exactly-once), process-trace grading (G-4 gated), seeded per-learner
|
||||
variants (fairness anchors wired to grading), oral defense with integrity signals
|
||||
(mock-first voice, browser fallback), and real learner build/defense/grading surfaces.
|
||||
8 phases. All 8 requirements (REQ-3-001..008) complete. Tags v0.2.1–v0.2.7 per phase,
|
||||
milestone release v0.2.8.
|
||||
|
||||
## v0.2 (Complete — Shipped as v0.2.0)
|
||||
|
||||
|
||||
@@ -46,9 +46,9 @@
|
||||
"projects": [],
|
||||
"active_project": null,
|
||||
"milestone": {
|
||||
"version": "v0.3",
|
||||
"name": "credential-engines",
|
||||
"version": "v0.5",
|
||||
"name": "real-voice-identity-envs",
|
||||
"type": "feature",
|
||||
"branch": "milestone/v0.3-credential-engines"
|
||||
"branch": "milestone/v0.5-real-voice-identity-envs"
|
||||
}
|
||||
}
|
||||
@@ -20,6 +20,8 @@ dist/
|
||||
.next/
|
||||
.turbo/
|
||||
*.tsbuildinfo
|
||||
# Next.js static export (v0.3.6 build output, served by the ai-service)
|
||||
apps/web/out/
|
||||
|
||||
# Storybook
|
||||
storybook-static/
|
||||
|
||||
@@ -2,8 +2,86 @@
|
||||
|
||||
AI-native outcome school + marketplace — graduates prove what they can build, not what they can write.
|
||||
|
||||
## Quickstart
|
||||
|
||||
One-liner install (linux x64):
|
||||
|
||||
```sh
|
||||
curl -fsSL https://git.coreci.dev/coreci/nextcraft/raw/main/scripts/install.sh | sh
|
||||
```
|
||||
|
||||
That downloads the latest release's `nextcraft` CLI binary, verifies its sha256 checksum, and installs it to `~/.local/bin` (PATH hint printed if needed). Every release ships fresh binaries — re-run the one-liner to upgrade.
|
||||
|
||||
Then, from a clone of this repo:
|
||||
|
||||
```sh
|
||||
nextcraft doctor # check prerequisites: node >= 18, pnpm >= 8, python3 >= 3.11, git, unshare
|
||||
nextcraft bootstrap # pnpm install + ai-service venv + .env from template (idempotent)
|
||||
nextcraft verify # health check: venv imports, uvicorn, ports, env
|
||||
nextcraft dev # run the ai-service dev server on :8420 (web dev server: pnpm dev)
|
||||
```
|
||||
|
||||
The E2E test (`apps/cli/tests/fresh-clone-e2e.test.ts`) proves this exact sequence on a fresh clone.
|
||||
|
||||
### No binary / non-linux?
|
||||
|
||||
The installer degrades to printed source instructions. Manual equivalent:
|
||||
|
||||
```sh
|
||||
git clone https://git.coreci.dev/coreci/nextcraft.git && cd nextcraft
|
||||
pnpm install
|
||||
bash apps/ai-service/scripts/bootstrap.sh
|
||||
cp apps/ai-service/.env.example apps/ai-service/.env
|
||||
pnpm ai:dev
|
||||
```
|
||||
|
||||
## CLI reference (`nextcraft`)
|
||||
|
||||
| Command | What it does | Exit codes |
|
||||
|---------|--------------|------------|
|
||||
| `doctor` | Checks prerequisites on PATH: node >= 18, pnpm >= 8, python3 >= 3.11, git, unshare (sandbox fabric). Every ✗ prints a fix hint. | 0 all pass, 1 any fail |
|
||||
| `bootstrap` | Sets up the monorepo from a fresh clone: (1) locates the repo root, (2) `pnpm install`, (3) ai-service venv via `apps/ai-service/scripts/bootstrap.sh`, (4) copies `.env.example` → `.env` if absent, (5) warns on missing optional keys. Idempotent — safe to re-run. | 0 ok, 1 step failed |
|
||||
| `verify` | Health check: ai-service venv + `import ai_service`, uvicorn importable, `.env` present (warn-only), `AI_PORT` (default 8420) free, workspace `node_modules` present. | 0 ok, 1 failures |
|
||||
| `dev` | Thin passthrough to `apps/ai-service/scripts/dev.sh` (exports secrets from `.ciagent/.env.secrets` if present, runs uvicorn on :8420). Ctrl+C stops it. The web dev server is separate: `pnpm dev`. | child's exit code |
|
||||
| `dev -d` / `dev --detach` | Same, but as an **unattended daemon**: detached, output appended to `~/.nextcraft/run/<clone>/dev.log`, pidfile beside it. Refuses to double-start. Auto-serves the built web UI same-origin when `apps/web/out` exists. | 0 started, 1 already running |
|
||||
| `stop` | Stops the daemon started by `dev -d` (SIGTERM, SIGKILL after 5s), cleans the pidfile. | 0 stopped/clean, 1 kill failed |
|
||||
| `log` | Tails the daemon log: last 50 lines by default, `-n N` for more, `-f`/`--follow` to stream. | 0 ok, 1 no log yet |
|
||||
| `--help` / `-h` | Usage for the CLI or any command. | 0 |
|
||||
| `--version` | Prints the version this binary was built as (matches the release tag). | 0 |
|
||||
|
||||
Exit-code contract: `0` success, `1` check/step failure (hint printed), `2` usage error.
|
||||
|
||||
### Remote server / single-port deployment
|
||||
|
||||
`nextcraft dev` binds the API on **0.0.0.0:8420** (and `pnpm dev` serves the web app on all interfaces), so the stack works from other machines out of the box:
|
||||
|
||||
- Browse `http://<your-host>:3000` — the web app targets `http://<your-host>:8420` automatically (derived from the browser's hostname).
|
||||
- CORS admits any origin (`AI_CORS_ORIGINS=*` in `apps/ai-service/.env`). This is safe **only** because credentials are never enabled; to restrict, set an explicit list: `AI_CORS_ORIGINS=http://<your-host>:3000`.
|
||||
- To revert to loopback-only: `AI_HOST=127.0.0.1` in `apps/ai-service/.env`.
|
||||
- Security note: this is an unauthenticated dev API reachable from any network the box exposes. Mitigations that still apply: per-learner sandbox caps + global rate caps + learner allowlist (G-5), telemetry flood control (traces marked `INCOMPLETE_FLOODED` are refused by the grader). Expose only on trusted networks until identity/KYC lands (v0.5).
|
||||
|
||||
**Production behind one port (v0.3.6):** when only one port is reachable (e.g. behind HAProxy), build the web app as a static export and let the ai-service serve it same-origin on :8420 — UI and API on one port, no CORS, no mixed content:
|
||||
|
||||
```sh
|
||||
curl -fsSL https://git.coreci.dev/coreci/nextcraft/raw/main/scripts/install.sh | sh # fresh binary
|
||||
git pull --ff-only && pnpm install
|
||||
NEXT_PUBLIC_AI_SERVICE_URL=self pnpm build # emits apps/web/out
|
||||
nextcraft dev -d # daemon; auto-serves the UI from :8420
|
||||
nextcraft log -f # tail the daemon log
|
||||
```
|
||||
|
||||
Durable state (SQLite DB, sandbox workdirs, daemon pid/log) lives in `~/.nextcraft/` — never inside the repo. Full recipe incl. HAProxy timeouts and a systemd unit: [deploy/README.md](deploy/README.md).
|
||||
|
||||
## Docs
|
||||
|
||||
- [apps/cli/README.md](apps/cli/README.md) — CLI internals: build, binary pipeline, troubleshooting
|
||||
- [.ciagent/PROJECT.md](.ciagent/PROJECT.md) — product spec and milestone history
|
||||
- [.ciagent/ARCHITECTURE.md](.ciagent/ARCHITECTURE.md) — system architecture
|
||||
|
||||
## Status
|
||||
|
||||
**Milestone v0.1** — UI/UX Prototype (high-fidelity interactive, all mock data)
|
||||
**Milestone v0.5** — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease (shipped as v0.4.5)
|
||||
|
||||
Prior: v0.3 Credential Engines (shipped v0.2.8) · v0.2 AI Tutor Architecture (v0.2.0) · v0.1 UI/UX Prototype (v0.1.0)
|
||||
|
||||
Initialized via CIAgent v0.7.0
|
||||
@@ -2,6 +2,13 @@
|
||||
# Real keys live in .ciagent/.env.secrets (gitignored) and are exported by scripts/dev.sh
|
||||
|
||||
AI_PORT=8420
|
||||
# Network mode (v0.3.5, D-038): dev server binds 0.0.0.0 so remote machines can
|
||||
# reach the stack. Set to 127.0.0.1 to revert to loopback-only.
|
||||
AI_HOST=0.0.0.0
|
||||
# CORS + WS-origin policy: '*' (default) admits any origin — safe because
|
||||
# credentials are never enabled. Restrict with a comma list, e.g.:
|
||||
# AI_CORS_ORIGINS=http://nextcraft-1:3000
|
||||
AI_CORS_ORIGINS=*
|
||||
AI_PROVIDER=ollama-cloud
|
||||
AI_MODEL=gemma4:31b
|
||||
AI_OLLAMA_CLOUD_BASE_URL=https://ollama.com/v1
|
||||
@@ -9,8 +16,10 @@ AI_OLLAMA_CLOUD_API_KEY=
|
||||
AI_LOCAL_BASE_URL=http://localhost:11434/v1
|
||||
AI_JSON_MODE=auto
|
||||
|
||||
# Sandbox fabric (v0.3)
|
||||
AI_SANDBOX_DIR=sandboxes
|
||||
# Sandbox fabric (v0.3). v0.3.6: durable state defaults to ~/.nextcraft/ —
|
||||
# OUTSIDE the repo (the old repo-relative 'sandboxes' default polluted the
|
||||
# git tree). Set an absolute path (~/ works) or a repo-relative one only for
|
||||
# throwaway dev clones.
|
||||
AI_SANDBOX_MAX_CONCURRENT=5
|
||||
AI_SANDBOX_TIMEOUT_S=900
|
||||
AI_SANDBOX_MAX_WORKDIR_MB=512
|
||||
@@ -23,10 +32,45 @@ AI_SANDBOX_MAX_PER_LEARNER=1
|
||||
# global creates per rolling 60s window (in-memory) → 429 when exceeded
|
||||
AI_SANDBOX_CREATES_PER_MIN=10
|
||||
|
||||
# Persistence (SQLite)
|
||||
AI_DB_PATH=ai_service/data/nextcraft.db
|
||||
# --- v0.3 Voice (REQ-3-006, D-030) ---
|
||||
# 'mock' (default; no key needed — tests/dev) or 'browser' (client-native SR/TTS).
|
||||
# Real server STT/TTS ('openai-audio' + AI_VOICE_BASE_URL/AI_VOICE_API_KEY)
|
||||
# is deferred to v0.4 per GRILL CUT-1/G-7 — keys never in code or commits.
|
||||
# Persistence (SQLite). v0.3.6: default moved out of the repo to
|
||||
# ~/.nextcraft/data/nextcraft.db (~/ paths in env overrides are expanded).
|
||||
# Uncomment + edit ONLY to relocate:
|
||||
# AI_DB_PATH=~/.nextcraft/data/nextcraft.db
|
||||
|
||||
# Single-port deploy (v0.3.6): directory of the exported web app
|
||||
# (apps/web/out). When set, the UI is served from this same service on :8420
|
||||
# — build it with `NEXT_PUBLIC_AI_SERVICE_URL=self pnpm build`. The
|
||||
# `nextcraft dev` command auto-sets this when apps/web/out exists.
|
||||
# AI_WEB_STATIC_DIR=../../web/out
|
||||
# --- Identity (REQ-5-003, D-042) ---
|
||||
# 'mock' (default — deterministic, no vendor spend pre-pilot; verdicts carry
|
||||
# mock=True forever per A-304). A real KYC vendor drops in via the
|
||||
# IdentityProvider protocol without API changes.
|
||||
AI_IDENTITY_PROVIDER=mock
|
||||
# G-13: identity submit caps — one active pending per learner (409), and a
|
||||
# per-learner submit rate ceiling (429 over a rolling 60s window).
|
||||
AI_IDENTITY_SUBMITS_PER_MIN=3
|
||||
|
||||
# --- Voice (REQ-3-006 D-030; real server path REQ-5-001, D-040) ---
|
||||
# 'mock' (default; no key needed — tests/dev), 'browser' (client-native
|
||||
# SR/TTS), or 'openai-audio' (real server STT/TTS, live since v0.5).
|
||||
AI_VOICE_PROVIDER=mock
|
||||
# openai-audio requires BOTH (unconfigured → app boots, voice falls back to
|
||||
# mock with a loud log — G-11; the badge then honestly reports mock):
|
||||
# AI_VOICE_BASE_URL=https://your-audio-endpoint/v1
|
||||
# AI_VOICE_API_KEY=
|
||||
# Optional model/voice/format knobs (defaults shown):
|
||||
# AI_VOICE_STT_MODEL=whisper-1
|
||||
# AI_VOICE_TTS_MODEL=tts-1
|
||||
# AI_VOICE_TTS_VOICE=alloy
|
||||
# AI_VOICE_TTS_FORMAT=mp3 (enum: mp3 | wav | opus)
|
||||
# AI_VOICE_MAX_AUDIO_MB=10
|
||||
# Manual probe recipe (executable by anyone with the keys; never CI):
|
||||
# STT: curl -sS $AI_VOICE_BASE_URL/audio/transcriptions \
|
||||
# -H "Authorization: Bearer $AI_VOICE_API_KEY" \
|
||||
# -F file=@test/fixtures/answer.wav -F model=whisper-1 | jq -e '.text'
|
||||
# TTS: curl -sS $AI_VOICE_BASE_URL/audio/speech \
|
||||
# -H "Authorization: Bearer $AI_VOICE_API_KEY" \
|
||||
# -H 'Content-Type: application/json' \
|
||||
# -d '{"model":"tts-1","input":"Nextcraft","voice":"alloy"}' \
|
||||
# -o /tmp/probe.mp3 && file /tmp/probe.mp3 | grep -i audio
|
||||
|
||||
@@ -22,7 +22,7 @@ from __future__ import annotations
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, Request, UploadFile
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
@@ -40,6 +40,7 @@ from .deps import (
|
||||
get_voice_provider,
|
||||
get_voice_store,
|
||||
)
|
||||
from .identity import require_verified_age # noqa: E402 (gate dep, D-043)
|
||||
|
||||
router = APIRouter(prefix="/v1/defense", tags=["defense"])
|
||||
|
||||
@@ -90,7 +91,7 @@ def _voice_descriptor(settings) -> VoiceDescriptor:
|
||||
|
||||
Must-Have #6: browser mode returns BROWSER_FALLBACK_DESCRIPTOR so the
|
||||
web client selects native SpeechRecognition/speechSynthesis; mock mode
|
||||
returns the mock descriptor. (A v0.4 server provider would return
|
||||
returns the mock descriptor. (A real server provider returns
|
||||
mode="server" — the protocol seam.)
|
||||
"""
|
||||
if (settings.voice_provider or "mock").strip().lower() == "browser":
|
||||
@@ -103,6 +104,7 @@ def _voice_descriptor(settings) -> VoiceDescriptor:
|
||||
@router.post("/start", response_model=StartResponse)
|
||||
async def start_defense(
|
||||
body: StartRequest,
|
||||
request: Request,
|
||||
examiner: ExaminerAgent = Depends(get_examiner),
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
voice_provider=Depends(get_voice_provider),
|
||||
@@ -110,6 +112,19 @@ async def start_defense(
|
||||
variant_store=Depends(get_variant_store),
|
||||
settings=Depends(get_settings),
|
||||
) -> StartResponse:
|
||||
# v0.5 identity gate (D-043): allowlist first (G-5), then the school
|
||||
# 16+ verified verdict, before any defense machinery runs.
|
||||
if body.learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=(
|
||||
f"learner_id {body.learner_id!r} is not on the sandbox "
|
||||
"allowlist (G-5)"
|
||||
),
|
||||
)
|
||||
await require_verified_age(
|
||||
16, body.learner_id, request.app.state.identity_store
|
||||
)
|
||||
record = voice_store.start(
|
||||
DefenseRecord(
|
||||
id=f"dfn-{int(time.time() * 1000):x}-{body.learner_id[:8]}",
|
||||
@@ -161,6 +176,7 @@ async def answer_defense(
|
||||
examiner: ExaminerAgent = Depends(get_examiner),
|
||||
trace_store=Depends(get_trace_store),
|
||||
variant_store=Depends(get_variant_store),
|
||||
settings=Depends(get_settings),
|
||||
) -> AnswerResponse:
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
@@ -183,11 +199,36 @@ async def answer_defense(
|
||||
raw = await audio.read()
|
||||
if not raw:
|
||||
# Empty upload is a client error (422), not a provider crash
|
||||
# (500): validate before the provider call so every provider —
|
||||
# mock today, the v0.4 real one — sees the same contract.
|
||||
# (500): validate before the provider call so every provider
|
||||
# sees the same contract.
|
||||
raise HTTPException(status_code=422, detail="audio upload is empty")
|
||||
fmt = (audio.content_type or "audio/wav").split("/")[-1]
|
||||
segment = await voice_provider.transcribe(raw, fmt)
|
||||
max_bytes = settings.voice_max_audio_mb * 1024 * 1024
|
||||
if len(raw) > max_bytes:
|
||||
# D-041/G-12: bounded audio BEFORE the provider call — the
|
||||
# client renders this as an honest re-record prompt.
|
||||
raise HTTPException(
|
||||
status_code=413,
|
||||
detail=(
|
||||
f"audio exceeds {settings.voice_max_audio_mb}MB "
|
||||
"— re-record a shorter answer"
|
||||
),
|
||||
)
|
||||
# D-041: strip codec params — MediaRecorder sends
|
||||
# 'audio/webm;codecs=opus'; the bare extension is the provider
|
||||
# contract ('webm'), else real STT endpoints reject the multipart.
|
||||
fmt = (audio.content_type or "audio/wav").split("/")[-1].split(";")[0].strip()
|
||||
try:
|
||||
segment = await voice_provider.transcribe(raw, fmt)
|
||||
except RuntimeError as exc:
|
||||
# Provider failure is the 502 house pattern (assessment.py /
|
||||
# proctor.py), not a 500: a real endpoint outage (or the
|
||||
# default mock's unscripted queue — final-review cross-phase
|
||||
# P0) must surface as an honest upstream error. Both providers
|
||||
# raise RuntimeError with sanitized text (mock: MockVoiceFailure;
|
||||
# openai-audio: key-redacted _sanitize).
|
||||
raise HTTPException(
|
||||
status_code=502, detail=f"voice transcription failed: {exc}"
|
||||
) from exc
|
||||
stt_ms = int((time.perf_counter() - stt_started) * 1000)
|
||||
text = segment.text
|
||||
|
||||
@@ -246,6 +287,7 @@ async def defense_audio(
|
||||
turn_id: int,
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
voice_provider=Depends(get_voice_provider),
|
||||
settings=Depends(get_settings),
|
||||
):
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
@@ -258,7 +300,11 @@ async def defense_audio(
|
||||
async for chunk in voice_provider.synthesize(turn.text):
|
||||
yield chunk
|
||||
|
||||
return StreamingResponse(stream(), media_type="audio/wav")
|
||||
# G-16/D-041: the TTS format is a settings enum; the media_type maps
|
||||
# from it (was hardcoded audio/wav — wrong for every real format).
|
||||
return StreamingResponse(
|
||||
stream(), media_type=f"audio/{settings.voice_tts_format}"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{defense_id}/finish", response_model=FinishResponse)
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
"""Identity API + age-gate dependencies (REQ-5-003/004, D-042/43).
|
||||
|
||||
Flow (all under mock provider by default; A-304 mock markers ride every
|
||||
response so downstream surfaces never treat mock-verified as real):
|
||||
POST /v1/identity/submit submission → pending (G-13 caps first)
|
||||
GET /v1/identity/status/{lid} latest record + mock marker
|
||||
POST /v1/identity/verify/{sid} poll provider → terminal transition
|
||||
|
||||
Gate dependencies (D-043 binding composition order, mounted by the gated
|
||||
routes — variants/sandbox-create/defense-start for school 16+; one
|
||||
marketplace route for 18+ verified):
|
||||
allowlist (403, G-5 pilot guard) → identity verdict (403 + verify-CTA)
|
||||
→ rate caps (429, owned by the calling routes)
|
||||
|
||||
PII (A-305): raw DOB enters via the submission, is used to derive the
|
||||
band, and is NEVER stored or logged (caplog sentinel test pins it).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from ..config import Settings
|
||||
from ..identity.base import IdentityProvider, IdentitySubmission
|
||||
from ..identity.store import IdentityRecord, IdentityStore
|
||||
from .deps import get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1/identity", tags=["identity"])
|
||||
|
||||
|
||||
# -- DI ------------------------------------------------------------------------
|
||||
|
||||
|
||||
def get_identity_store(request: Request) -> IdentityStore:
|
||||
return request.app.state.identity_store
|
||||
|
||||
|
||||
def get_identity_provider(request: Request) -> IdentityProvider:
|
||||
return request.app.state.identity_provider
|
||||
|
||||
|
||||
# -- models ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class SubmitBody(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
#: ISO date. Validated AT THE BOUNDARY (D1 verifier fix): a malformed
|
||||
#: value would otherwise blow up as a 500 inside derive_age_band on the
|
||||
#: verify path — echoing the raw DOB into the traceback (A-305) and
|
||||
#: leaving a poisoned pending record that G-13 turns into a permanent
|
||||
#: learner lockout.
|
||||
date_of_birth: str = Field(description="ISO date; never stored or logged")
|
||||
document_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
@field_validator("date_of_birth")
|
||||
@classmethod
|
||||
def _validate_dob(cls, value: str) -> str:
|
||||
try:
|
||||
datetime.fromisoformat(value)
|
||||
except ValueError as exc:
|
||||
# 422 with the input scrubbed (A-305/D1 — the app-level
|
||||
# RequestValidationError handler redacts PII field inputs).
|
||||
raise ValueError("date_of_birth must be an ISO date (YYYY-MM-DD)") from exc
|
||||
return value
|
||||
|
||||
|
||||
class SubmitResponse(BaseModel):
|
||||
submission_id: str
|
||||
status: str
|
||||
#: A-304: honesty marker — a mock verdict is NEVER production-verified.
|
||||
mock: bool = True
|
||||
detail: str = ""
|
||||
|
||||
|
||||
class StatusResponse(BaseModel):
|
||||
learner_id: str
|
||||
status: str
|
||||
age_band: str | None
|
||||
mock: bool
|
||||
verified_at: str | None
|
||||
|
||||
|
||||
class VerifyResponse(SubmitResponse):
|
||||
age_band: str | None
|
||||
|
||||
|
||||
# -- G-13 submit caps ---------------------------------------------------------------
|
||||
|
||||
|
||||
class _SubmitRateLimiter:
|
||||
"""Per-learner submit rate cap (in-memory, process-local — the G-5
|
||||
creates-per-min pattern from the sandboxes route)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._window: dict[str, list[float]] = {}
|
||||
|
||||
def check(self, learner_id: str, per_min: int) -> None:
|
||||
now = time.monotonic()
|
||||
window = self._window.setdefault(learner_id, [])
|
||||
window[:] = [t for t in window if now - t < 60.0]
|
||||
if len(window) >= per_min:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail="identity submit rate exceeded — wait a minute",
|
||||
)
|
||||
window.append(now)
|
||||
|
||||
|
||||
_rate_limiter = _SubmitRateLimiter()
|
||||
|
||||
|
||||
# -- verification flow ------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/submit", response_model=SubmitResponse)
|
||||
async def submit_identity(
|
||||
body: SubmitBody,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
provider: IdentityProvider = Depends(get_identity_provider),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> SubmitResponse:
|
||||
# G-13: one active pending submission per learner — resubmit while
|
||||
# pending echoes the pending state (409), not a second submission.
|
||||
if store.count_pending_for_learner(body.learner_id) > 0:
|
||||
latest = store.latest_for_learner(body.learner_id)
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail={
|
||||
"reason": "submission_pending",
|
||||
"submission_id": latest.id if latest else None,
|
||||
"status": "pending",
|
||||
},
|
||||
)
|
||||
_rate_limiter.check(body.learner_id, settings.identity_submits_per_min)
|
||||
|
||||
submission = IdentitySubmission(
|
||||
learner_id=body.learner_id,
|
||||
date_of_birth=body.date_of_birth,
|
||||
document_refs=body.document_refs,
|
||||
)
|
||||
submission_id = await provider.submit(submission)
|
||||
|
||||
record = store.insert(
|
||||
IdentityRecord(
|
||||
id=submission_id,
|
||||
learner_id=body.learner_id,
|
||||
status="pending",
|
||||
provider="mock" if settings.identity_provider == "mock" else settings.identity_provider,
|
||||
document_refs=body.document_refs, # A-305: opaque handles, never contents
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
return SubmitResponse(
|
||||
submission_id=record.id, status=record.status, mock=record.mock
|
||||
)
|
||||
|
||||
|
||||
@router.get("/status/{learner_id}", response_model=StatusResponse)
|
||||
async def identity_status(
|
||||
learner_id: str,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
) -> StatusResponse:
|
||||
record = store.latest_for_learner(learner_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail="no identity record")
|
||||
return StatusResponse(
|
||||
learner_id=learner_id,
|
||||
status=record.status,
|
||||
age_band=record.age_band,
|
||||
mock=record.mock,
|
||||
verified_at=record.verified_at.isoformat() if record.verified_at else None,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/verify/{submission_id}", response_model=VerifyResponse)
|
||||
async def verify_identity(
|
||||
submission_id: str,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
provider: IdentityProvider = Depends(get_identity_provider),
|
||||
) -> VerifyResponse:
|
||||
record = store.get(submission_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail="no such submission")
|
||||
if record.status != "pending":
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"submission already {record.status}",
|
||||
)
|
||||
|
||||
verdict = await provider.poll(submission_id)
|
||||
# A-305: derive the band from the provider's verdict; the raw DOB never
|
||||
# entered the store and is not logged here.
|
||||
updated = store.mark_verified(
|
||||
submission_id, verdict.model_dump(), verdict.age_band
|
||||
)
|
||||
assert updated is not None # record existed a moment ago
|
||||
return VerifyResponse(
|
||||
submission_id=updated.id,
|
||||
status=updated.status,
|
||||
age_band=updated.age_band,
|
||||
mock=updated.mock,
|
||||
detail=verdict.detail,
|
||||
)
|
||||
|
||||
|
||||
# -- age-gate dependencies (D-043 composition) -------------------------------------
|
||||
|
||||
|
||||
def _verify_cta_payload(
|
||||
reason: str, min_age: int, record: IdentityRecord | None
|
||||
) -> dict:
|
||||
"""A-306/UX acceptance #2: an actionable 403 — never a bare error."""
|
||||
return {
|
||||
"reason": reason,
|
||||
"min_age": min_age,
|
||||
"current_status": record.status if record else "none",
|
||||
"verify_cta": "/enroll",
|
||||
}
|
||||
|
||||
|
||||
async def require_verified_age(
|
||||
min_age: int,
|
||||
learner_id: str,
|
||||
store: IdentityStore,
|
||||
) -> IdentityRecord:
|
||||
"""The identity half of the D-043 composition (allowlist runs FIRST in
|
||||
the calling routes; this is the second gate; caps come after).
|
||||
|
||||
School 16+ → min_age=16; marketplace 18+ verified → min_age=18.
|
||||
"""
|
||||
record = store.latest_for_learner(learner_id)
|
||||
if record is None or record.status != "verified":
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("identity_verification_required", min_age, record),
|
||||
)
|
||||
# D2 (verifier): FAIL CLOSED. Only canonical bands can pass — None,
|
||||
# unknown, or under-16 bands reject (the gate is the security boundary
|
||||
# for the future vendor and direct store writes; it never trusts a
|
||||
# band it does not recognize).
|
||||
if record.age_band not in ("16-17", "18+"):
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("identity_verification_required", min_age, record),
|
||||
)
|
||||
if min_age > 16 and record.age_band != "18+":
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("age_gate_18_plus", 18, record),
|
||||
)
|
||||
return record
|
||||
|
||||
|
||||
#: Convenience alias for the marketplace 18+ composition (D-043's
|
||||
#: `require_verified_adult` — direct calls use require_verified_age(18, ...)).
|
||||
require_verified_adult = require_verified_age
|
||||
|
||||
|
||||
# -- marketplace 18+ gated stub (G-18, REQ-5-004) ------------------------------------
|
||||
|
||||
marketplace_router = APIRouter(prefix="/v1/marketplace", tags=["marketplace"])
|
||||
|
||||
|
||||
class MarketplaceApplyBody(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
job_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
@marketplace_router.post("/apply", status_code=501)
|
||||
async def marketplace_apply_stub(
|
||||
body: MarketplaceApplyBody,
|
||||
request: Request,
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> dict:
|
||||
"""The ONE gated marketplace route (D-043): proves the 18+ verified
|
||||
composition end-to-end. G-18 honesty: after passing the gate it returns
|
||||
501 with explicit stub + mock markers — the marketplace backend does
|
||||
not exist yet; this route never fabricates an 'applied' outcome.
|
||||
"""
|
||||
if body.learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=f"learner_id {body.learner_id!r} is not on the sandbox allowlist (G-5)",
|
||||
)
|
||||
await require_verified_age(18, body.learner_id, get_identity_store(request))
|
||||
return {
|
||||
"detail": "marketplace applications are not live yet",
|
||||
"stub": True,
|
||||
"mock": True,
|
||||
}
|
||||
@@ -25,7 +25,7 @@ import time
|
||||
from collections import deque
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Response
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from ..config import Settings
|
||||
@@ -94,6 +94,10 @@ class SnapshotResponse(BaseModel):
|
||||
# -- abuse control (G-5; middleware layer, not auth) ---------------------------
|
||||
|
||||
|
||||
from ..variants.templates import get_template # noqa: E402
|
||||
from .identity import require_verified_age # noqa: E402 (gate dep, D-043)
|
||||
|
||||
|
||||
def _enforce_allowlist(learner_id: str, settings: Settings) -> None:
|
||||
if learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
@@ -139,10 +143,16 @@ def _check_global_create_rate(settings: Settings) -> None:
|
||||
@router.post("", status_code=201, response_model=SandboxResponse)
|
||||
async def create_sandbox(
|
||||
body: SandboxCreateRequest,
|
||||
request: Request,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> SandboxResponse:
|
||||
# v0.5 identity gate (D-043, REQ-5-004): allowlist (G-5) → identity
|
||||
# verdict (403 + verify-CTA) → caps (429) — the binding composition.
|
||||
_enforce_allowlist(body.learner_id, settings)
|
||||
await require_verified_age(
|
||||
16, body.learner_id, request.app.state.identity_store
|
||||
)
|
||||
_check_per_learner_cap(await manager.list(), body.learner_id, settings)
|
||||
_check_global_create_rate(settings)
|
||||
try:
|
||||
@@ -335,10 +345,54 @@ async def write_file(
|
||||
return {"path": body.path, "written": True}
|
||||
|
||||
|
||||
#: G-15 (REQ-5-005): per-kind exec command policy — EXACT argv[0] token
|
||||
#: matching, never prefix/substring (trivially bypassed via flags/-c
|
||||
#: passthrough). 'sh -c' passthrough is DISALLOWED for design/simulation
|
||||
#: kinds: the gaming vector would be faking build-style test cycles into a
|
||||
#: kind-agnostic digest. Build kinds keep v0.3 behavior (any command —
|
||||
#: the CUT-2 surface is Run/Test buttons, not a shell relay).
|
||||
#: python (bare) is deliberately absent — in-ns PATH resolves only python3
|
||||
#: (verifier P1); pip is absent (no network in the namespace).
|
||||
_GENERIC_FIRST_TOKENS = frozenset(
|
||||
{"ls", "cat", "pwd", "echo", "python3", "pytest"}
|
||||
)
|
||||
|
||||
|
||||
def _enforce_exec_policy(
|
||||
cmd: list[str], environment: str | None, allowed: set[str] | None = None
|
||||
) -> None:
|
||||
"""422 with the allowed set when a design/sim command is out of policy.
|
||||
|
||||
`allowed` defaults to the generic set; the exec route unions in the
|
||||
template's DECLARED harness argv[0] (a future non-python harness
|
||||
template must not reject its own Run command)."""
|
||||
if environment not in ("design", "simulation"):
|
||||
return # build kind: unchanged v0.3 semantics
|
||||
allowed = set(allowed) if allowed is not None else set(_GENERIC_FIRST_TOKENS)
|
||||
first = cmd[0] if cmd else ""
|
||||
if first in ("sh", "bash"):
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"shell passthrough is not allowed in a {environment} "
|
||||
f"environment; allowed commands: {sorted(allowed)}"
|
||||
),
|
||||
)
|
||||
if first not in allowed:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"command {first!r} is not allowed in a {environment} "
|
||||
f"environment; allowed commands: {sorted(allowed)}"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{sandbox_id}/exec", response_model=ExecResponse)
|
||||
async def exec_command(
|
||||
sandbox_id: str,
|
||||
body: ExecRequest,
|
||||
request: Request,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
) -> ExecResponse:
|
||||
try:
|
||||
@@ -349,5 +403,19 @@ async def exec_command(
|
||||
handle = manager._handles.get(sandbox_id) # noqa: SLF001
|
||||
if handle is None:
|
||||
raise HTTPException(status_code=404, detail=f"no live handle {sandbox_id!r}")
|
||||
# REQ-5-005 (G-15): resolve the sandbox's variant environment by its
|
||||
# task_id (manager side-table) and enforce the per-kind command policy
|
||||
# BEFORE execution.
|
||||
task_id = manager._task_ids.get(sandbox_id) # noqa: SLF001 - composition seam
|
||||
if task_id:
|
||||
variant = request.app.state.variant_store.get_by_task(task_id)
|
||||
if variant is not None:
|
||||
template = get_template(variant.template_id)
|
||||
declared = (
|
||||
{template.run_command.split()[0]} if template is not None else set()
|
||||
)
|
||||
_enforce_exec_policy(
|
||||
body.cmd, variant.environment, _GENERIC_FIRST_TOKENS | declared
|
||||
)
|
||||
result = await backend.exec(handle, body.cmd)
|
||||
return ExecResponse(**result.model_dump())
|
||||
|
||||
@@ -39,16 +39,28 @@ from .deps import get_trace_integrity, get_trace_store
|
||||
|
||||
router = APIRouter(prefix="/v1/telemetry", tags=["telemetry"])
|
||||
|
||||
#: Browser Origins allowed to open the ingest socket (A-008 mirror). The
|
||||
#: stdlib capture agent sends NO Origin header (it is not a browser) and
|
||||
#: Browser Origins allowed to open the ingest socket (A-008 mirror, D-038).
|
||||
#: The stdlib capture agent sends NO Origin header (it is not a browser) and
|
||||
#: stays allowed; a malicious page loaded in the learner's browser would
|
||||
#: carry an Origin and must not be able to poison/flood the trace. CORS
|
||||
#: middleware does NOT cover WebSocket upgrades, so this gate is explicit.
|
||||
_ALLOWED_WS_ORIGINS = frozenset(
|
||||
#: In network mode the configured CORS list governs (default '*' — any
|
||||
#: origin, since credentials are never used); an explicit list still rejects
|
||||
#: unlisted origins with 1008.
|
||||
_LOCAL_WS_ORIGINS = frozenset(
|
||||
{"http://localhost:3000", "http://127.0.0.1:3000", "http://localhost:8420"}
|
||||
)
|
||||
|
||||
|
||||
def _allowed_ws_origins(settings: object) -> frozenset[str]:
|
||||
configured = getattr(settings, "cors_origin_list", None)
|
||||
if configured is None:
|
||||
return _LOCAL_WS_ORIGINS
|
||||
if configured == ["*"]:
|
||||
return frozenset() # empty = wildcard = every Origin passes
|
||||
return frozenset(configured) | _LOCAL_WS_ORIGINS
|
||||
|
||||
|
||||
# --- WS ingest (D-026) ---------------------------------------------------------
|
||||
|
||||
|
||||
@@ -60,10 +72,12 @@ async def telemetry_ingest_ws(websocket: WebSocket) -> None:
|
||||
FastAPI; this shim is the only place the two layers meet.
|
||||
"""
|
||||
origin = (websocket.headers.get("origin") or "").strip()
|
||||
if origin and origin not in _ALLOWED_WS_ORIGINS:
|
||||
allowed = _allowed_ws_origins(getattr(websocket.app.state, "settings", None))
|
||||
if origin and allowed and origin not in allowed:
|
||||
# Same-origin dev pages (Next.js on :3000, the service itself on
|
||||
# :8420) pass; anything else is refused pre-accept. Non-browser
|
||||
# producers (the capture agent, tests) send no Origin and pass.
|
||||
# Wildcard (empty frozenset) passes every Origin in network mode.
|
||||
await websocket.close(
|
||||
code=1008, reason=f"origin {origin!r} not allowed for telemetry ingest"
|
||||
)
|
||||
|
||||
@@ -33,13 +33,29 @@ tests/api/test_variants.py asserts this end-to-end through the API.
|
||||
from datetime import datetime
|
||||
from typing import Self
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
from ..config import Settings
|
||||
from ..identity.store import IdentityStore
|
||||
from ..variants.generator import VariantGenerator
|
||||
from ..variants.store import VariantRecord, VariantStore
|
||||
from ..variants.templates import TaskTemplate, get_template, template_for_competency
|
||||
from .deps import get_variant_generator, get_variant_store
|
||||
from .deps import get_settings, get_variant_generator, get_variant_store
|
||||
from .identity import require_verified_age
|
||||
|
||||
|
||||
def _identity_store(request: Request) -> IdentityStore:
|
||||
return request.app.state.identity_store
|
||||
|
||||
|
||||
def _enforce_allowlist(learner_id: str, settings: Settings) -> None:
|
||||
"""G-5 pilot guard — allowlist runs FIRST in the composition (D-043)."""
|
||||
if learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=f"learner_id {learner_id!r} is not on the sandbox allowlist (G-5)",
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/v1/variants", tags=["variants"])
|
||||
|
||||
@@ -81,6 +97,9 @@ class VariantResponse(BaseModel):
|
||||
params: dict[str, str | int]
|
||||
statement: str
|
||||
starter_files: dict[str, str]
|
||||
#: REQ-5-005 (a-11): REQUIRED on the wire — always emitted.
|
||||
environment: str
|
||||
test_command: str
|
||||
created_at: datetime
|
||||
|
||||
|
||||
@@ -138,6 +157,9 @@ def _to_response(record: VariantRecord, competency_id: str) -> VariantResponse:
|
||||
params=dict(record.params),
|
||||
statement=record.statement,
|
||||
starter_files=dict(record.starter_files),
|
||||
# a-11: REQUIRED on the wire — the server always emits both (v0.5).
|
||||
environment=record.environment or "build",
|
||||
test_command=record.test_command or "pytest -q",
|
||||
created_at=record.created_at,
|
||||
)
|
||||
|
||||
@@ -148,12 +170,19 @@ def _to_response(record: VariantRecord, competency_id: str) -> VariantResponse:
|
||||
@router.post("", response_model=VariantResponse)
|
||||
async def generate_variant(
|
||||
body: VariantGenerateRequest,
|
||||
request: Request,
|
||||
generator: VariantGenerator = Depends(get_variant_generator),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> VariantResponse:
|
||||
"""The learner's variant for the resolved template — generated on the
|
||||
first request, cached (no LLM call) on every repeat: D-029 makes a
|
||||
regenerate a 200 of the SAME stored variant.
|
||||
|
||||
v0.5 identity gate (D-043, REQ-5-004): the school floor is 16+ verified.
|
||||
Composition order: G-5 allowlist (403) → identity verdict (403 + CTA).
|
||||
"""
|
||||
_enforce_allowlist(body.learner_id, settings)
|
||||
await require_verified_age(16, body.learner_id, _identity_store(request))
|
||||
template = _resolve_template(body)
|
||||
record = await generator.generate(body.learner_id, template.id)
|
||||
return _to_response(record, competency_id=template.competency_id)
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Service settings — pydantic-settings, env prefix AI_, .env support."""
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
|
||||
@@ -9,6 +10,15 @@ from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict
|
||||
# apps/ai-service/ (sandbox dir default is relative to the app, not the CWD)
|
||||
_SERVICE_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
# v0.3.6: durable runtime state lives OUTSIDE the repo (founder directive —
|
||||
# the repo is a git tree, not a data dir; old defaults under apps/ai-service/
|
||||
# polluted the working copy). ~/.nextcraft is the state root for DB + sandbox
|
||||
# workdirs; env overrides may use ~/ paths (expanded by the validator below).
|
||||
_STATE_ROOT = Path.home() / ".nextcraft"
|
||||
|
||||
# G-16: TTS response-format whitelist (feeds the TTS route's Content-Type).
|
||||
_TTS_FORMATS = ("mp3", "wav", "opus")
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_prefix="AI_", env_file=".env", extra="ignore")
|
||||
@@ -26,7 +36,7 @@ class Settings(BaseSettings):
|
||||
|
||||
# v0.3 sandbox fabric (REQ-3-001): root holding per-sandbox workdirs.
|
||||
# Relative paths resolve against the app dir (apps/ai-service/), not the CWD.
|
||||
sandbox_dir: Path = _SERVICE_ROOT / "sandboxes"
|
||||
sandbox_dir: Path = _STATE_ROOT / "sandboxes"
|
||||
|
||||
# D-032: single-box capacity, no queue — pool full → API maps to 503.
|
||||
sandbox_max_concurrent: int = 5
|
||||
@@ -63,7 +73,40 @@ class Settings(BaseSettings):
|
||||
return value
|
||||
|
||||
# D-027: SQLite path for telemetry/grades/variants/defenses stores.
|
||||
db_path: Path = _SERVICE_ROOT / "ai_service" / "data" / "nextcraft.db"
|
||||
# v0.3.6: default moved out of the repo to the home state root.
|
||||
db_path: Path = _STATE_ROOT / "data" / "nextcraft.db"
|
||||
|
||||
# v0.3.6 single-port deploy: directory of the exported Next.js app
|
||||
# (apps/web/out). When set, the app serves it at / (StaticFiles) so the
|
||||
# whole site — UI + API — answers on one port behind HAProxy; the web
|
||||
# build bakes NEXT_PUBLIC_AI_SERVICE_URL=self (relative same-origin
|
||||
# fetches). Empty default → no mount; dev/tests unchanged.
|
||||
web_static_dir: Path = Path("")
|
||||
|
||||
@field_validator("db_path", "sandbox_dir", "web_static_dir", mode="before")
|
||||
@classmethod
|
||||
def _expanduser_paths(cls, value: object) -> object:
|
||||
# Env-set paths may use ~ (e.g. AI_DB_PATH=~/.nextcraft/x.db);
|
||||
# pydantic Path does not expand it natively.
|
||||
if isinstance(value, str):
|
||||
return Path(value).expanduser()
|
||||
return value
|
||||
|
||||
@field_validator("voice_tts_format")
|
||||
@classmethod
|
||||
def _validate_tts_format(cls, value: str) -> str:
|
||||
# G-16 + G-11 consistency: unknown values NEVER crash the boot —
|
||||
# fall back to the default with a loud warning (the boot-survival
|
||||
# log lives in main.py's voice fallback; this validator normalizes).
|
||||
v = value.strip().lower()
|
||||
if v not in _TTS_FORMATS:
|
||||
logging.getLogger(__name__).warning(
|
||||
"AI_VOICE_TTS_FORMAT=%r is not one of %s — falling back to 'mp3'",
|
||||
value,
|
||||
_TTS_FORMATS,
|
||||
)
|
||||
return "mp3"
|
||||
return v
|
||||
|
||||
# G-3 flood control (NOT backpressure-by-silence): max events ingested per
|
||||
# (learner_id, task_id) trace before the WS endpoint closes the connection
|
||||
@@ -78,10 +121,54 @@ class Settings(BaseSettings):
|
||||
# `port` (A-004); only the host is configurable — never a second port.
|
||||
telemetry_ingest_host: str = "127.0.0.1"
|
||||
|
||||
# Voice provider selection (REQ-3-006, D-030): 'mock' (default — the
|
||||
# no-key path is first-class; tests never call a real voice API) or
|
||||
# 'browser' (browser-native SpeechRecognition/speechSynthesis fallback;
|
||||
# the descriptor tells the web client). The real server STT/TTS
|
||||
# ('openai-audio') is a v0.4 seam (GRILL CUT-1 / G-7) — AI_VOICE_BASE_URL
|
||||
# and AI_VOICE_API_KEY are documented in .env.example for that future.
|
||||
# v0.3.5 network mode (D-038): dev.sh binds 0.0.0.0 so remote browsers can
|
||||
# reach the stack; '*' (default) lets any origin call the API (safe ONLY
|
||||
# because credentials are never enabled — A-008). Set a comma-separated
|
||||
# origin list (e.g. 'http://nextcraft-1:3000') to restrict instead.
|
||||
# NOTE (v0.3.6): with the UI served same-origin from 8420 this is moot in
|
||||
# production (same-origin requests skip CORS); it stays for the two-port
|
||||
# dev topology.
|
||||
cors_origins: str = "*"
|
||||
|
||||
@property
|
||||
def cors_origin_list(self) -> list[str]:
|
||||
value = self.cors_origins.strip()
|
||||
if value == "*":
|
||||
return ["*"]
|
||||
return [o.strip() for o in value.split(",") if o.strip()]
|
||||
|
||||
# Identity provider selection (REQ-5-003, A-303): 'mock' (default —
|
||||
# deterministic, no vendor spend pre-pilot; verdicts carry mock=True
|
||||
# forever per A-304). A real KYC vendor drops in via the
|
||||
# IdentityProvider protocol without API changes.
|
||||
identity_provider: str = "mock"
|
||||
# G-13: identity submit caps — one active pending per learner (409 on
|
||||
# resubmit) and a per-learner submit rate ceiling.
|
||||
identity_submits_per_min: int = 3
|
||||
|
||||
# Voice provider selection (REQ-5-001, D-040): 'mock' (default — the
|
||||
# no-key path is first-class; tests never call a real voice API),
|
||||
# 'browser' (client-native SR/TTS; the descriptor tells the web client),
|
||||
# or 'openai-audio' (real server STT/TTS against an OpenAI-compatible
|
||||
# audio endpoint). openai-audio requires voice_base_url + voice_api_key;
|
||||
# when unconfigured the lifespan falls back to mock with a loud log
|
||||
# (G-11 — a typo'd env must never crash the unattended boot).
|
||||
voice_provider: str = "mock"
|
||||
|
||||
# Real server voice (D-040, A-301): endpoint-agnostic by config (D-014
|
||||
# pattern) — any OpenAI-compatible audio API works. Keys env-only,
|
||||
# never committed, never logged (mirrors ollama_cloud_api_key).
|
||||
voice_base_url: str = ""
|
||||
voice_api_key: str = ""
|
||||
voice_stt_model: str = "whisper-1"
|
||||
voice_tts_model: str = "tts-1"
|
||||
voice_tts_voice: str = "alloy"
|
||||
# G-16: whitelist, not free string — this feeds the TTS route's
|
||||
# Content-Type. A str + mode-after validator (NOT a pydantic Literal):
|
||||
# a Literal would raise ValidationError at Settings construction, before
|
||||
# main.py's G-11 fallback could catch it — crashing the unattended boot
|
||||
# on a typo'd env. Invalid values fall back to the default LOUDLY.
|
||||
voice_tts_format: str = "mp3"
|
||||
# A-302/D-041: upload guard before the provider call (webm/opus is
|
||||
# ~0.5-1MB/min, so 10MB tolerates very long answers).
|
||||
voice_max_audio_mb: int = 10
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
"""Identity engine package — provider protocol + store (REQ-5-003, D-042)."""
|
||||
|
||||
from .base import (
|
||||
AgeBand,
|
||||
IdentityProvider,
|
||||
IdentityStatus,
|
||||
IdentitySubmission,
|
||||
IdentityVerdict,
|
||||
)
|
||||
from .mock import MockIdentityProvider, derive_age_band
|
||||
from .store import IdentityRecord, IdentityStore, SQLiteIdentityStore
|
||||
|
||||
__all__ = [
|
||||
"AgeBand",
|
||||
"IdentityStatus",
|
||||
"IdentitySubmission",
|
||||
"IdentityVerdict",
|
||||
"IdentityProvider",
|
||||
"IdentityRecord",
|
||||
"IdentityStore",
|
||||
"SQLiteIdentityStore",
|
||||
"MockIdentityProvider",
|
||||
"derive_age_band",
|
||||
]
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Identity verification protocol (REQ-5-003, D-042, A-303).
|
||||
|
||||
Provider-agnostic like LLMProvider/VoiceProvider (D-014/D-030): a narrow
|
||||
protocol the identity API composes via DI, a deterministic mock, and a
|
||||
future real KYC vendor (Stripe Identity / Persona / Onfido class) that
|
||||
drops in without API changes. PII rules (A-305): the provider sees
|
||||
document REFERENCES, never raw documents; verdicts carry a mock marker
|
||||
(A-304) so downstream surfaces never display mock-verified as
|
||||
production-verified.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal, Protocol, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
AgeBand = Literal["16-17", "18+"]
|
||||
IdentityStatus = Literal["pending", "verified", "rejected"]
|
||||
|
||||
|
||||
class IdentitySubmission(BaseModel):
|
||||
"""What a learner submits: derived data + document refs only.
|
||||
|
||||
`date_of_birth` is a REAL date (the provider derives the age band) but
|
||||
raw DOB is NEVER persisted — only the derived band (A-305). Document
|
||||
refs are opaque handles (upload ids), never contents.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
learner_id: str = Field(min_length=1)
|
||||
date_of_birth: str = Field(description="ISO date; used to derive age_band, never stored")
|
||||
document_refs: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Opaque upload handles; raw documents are never stored",
|
||||
)
|
||||
|
||||
|
||||
class IdentityVerdict(BaseModel):
|
||||
"""Provider verdict — what gets stored + surfaced."""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
status: IdentityStatus
|
||||
age_band: AgeBand | None = None
|
||||
provider: str
|
||||
#: A-304 honesty: mock verdicts carry mock=True forever — downstream
|
||||
#: surfaces must never treat a mock verdict as production-verified.
|
||||
mock: bool = True
|
||||
detail: str = ""
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class IdentityProvider(Protocol):
|
||||
"""The KYC port: submit → (poll) → verdict. Never imports api/."""
|
||||
|
||||
async def submit(self, submission: IdentitySubmission) -> str:
|
||||
"""Start verification; returns a submission id (minted once)."""
|
||||
...
|
||||
|
||||
async def poll(self, submission_id: str) -> IdentityVerdict:
|
||||
"""Fetch the (possibly pending) verdict for a submission."""
|
||||
...
|
||||
@@ -0,0 +1,94 @@
|
||||
"""Deterministic mock identity provider (REQ-5-003, A-303).
|
||||
|
||||
Approve-on-policy: every submission verifies unless the caller scripts a
|
||||
rejection (by learner id) or the derived age band fails the floor
|
||||
(under-16 → rejected with an age detail). Verdicts are mock-marked (A-304)
|
||||
— the marker rides every verdict so no downstream surface can ever
|
||||
display mock-verified as production-verified. Never calls the network.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import itertools
|
||||
import secrets
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from .base import IdentitySubmission, IdentityVerdict
|
||||
|
||||
|
||||
def derive_age_band(date_of_birth: str, today: datetime | None = None) -> str:
|
||||
"""Derive the age band from an ISO date. Under-16 returns "under-16".
|
||||
|
||||
Pure + deterministic; used by the store test and the API alike.
|
||||
"""
|
||||
dob = datetime.fromisoformat(date_of_birth)
|
||||
now = today or datetime.now(UTC)
|
||||
age = now.year - dob.year - (
|
||||
(now.month, now.day) < (dob.month, dob.day)
|
||||
)
|
||||
if age < 16:
|
||||
return "under-16"
|
||||
if age < 18:
|
||||
return "16-17"
|
||||
return "18+"
|
||||
|
||||
|
||||
class MockIdentityProvider:
|
||||
"""Scriptable, deterministic; no network, no vendor calls.
|
||||
|
||||
Submission ids are minted UNIQUELY per submit() call (a monotonic
|
||||
counter + the per-process seed from `secrets`): the id is the PK of
|
||||
the insert-only IdentityStore, and a deterministic id derived from
|
||||
(learner_id, date_of_birth) collides on any resubmit-after-terminal
|
||||
(e.g. a rejected learner retrying with the same DOB) — the store
|
||||
surfaces IntegrityError and the API would 500 (cross-phase P0,
|
||||
final review). Uniqueness per call is the contract; determinism of
|
||||
VERDICTS (what tests actually pin) is preserved — poll() derives the
|
||||
band purely from the stored submission.
|
||||
"""
|
||||
|
||||
def __init__(self, reject_learners: set[str] | None = None) -> None:
|
||||
self._submissions: dict[str, IdentitySubmission] = {}
|
||||
self._reject_learners = reject_learners or set()
|
||||
# Per-process nonce: ids are opaque handles (A-305) — never
|
||||
# derived from PII. Counter + nonce keeps ids unique within and
|
||||
# across provider instances on one box.
|
||||
self._nonce = secrets.randbits(32)
|
||||
self._counter = itertools.count()
|
||||
|
||||
async def submit(self, submission: IdentitySubmission) -> str:
|
||||
submission_id = f"idc-{self._nonce:08x}{next(self._counter):08x}"
|
||||
self._submissions[submission_id] = submission
|
||||
return submission_id
|
||||
|
||||
async def poll(self, submission_id: str) -> IdentityVerdict:
|
||||
submission = self._submissions.get(submission_id)
|
||||
if submission is None:
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="unknown submission id",
|
||||
)
|
||||
if submission.learner_id in self._reject_learners:
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="scripted rejection (test)",
|
||||
)
|
||||
band = derive_age_band(submission.date_of_birth)
|
||||
if band == "under-16":
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="under 16 — the AI school floor is 16+ (COPPA avoidance)",
|
||||
)
|
||||
return IdentityVerdict(
|
||||
status="verified",
|
||||
age_band=band, # type: ignore[arg-type]
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="mock verdict — not production verification",
|
||||
)
|
||||
@@ -0,0 +1,214 @@
|
||||
"""IdentityStore — verification records (REQ-5-003, D-042, D-027 FIFTH store).
|
||||
|
||||
Insert-only + latest-per-learner lookup, modeled on the DefenseStore
|
||||
conventions: WAL + synchronous=NORMAL + busy_timeout + foreign_keys=ON
|
||||
pragmas at connect time, portable column types (str/datetime/JSON) for
|
||||
Postgres parity, @validates hooks for constraints sqlmodel's metaclass
|
||||
drops, tz-aware→naive→tz-aware boundary normalization.
|
||||
|
||||
PII contract (A-305): stores the DERIVED age_band (16-17 | 18+), NEVER a
|
||||
raw date of birth; document_refs are opaque handles, NEVER contents.
|
||||
Verdict provenance is audit data: every record carries provider + the
|
||||
mock marker (A-304) so downstream surfaces can label unverified state
|
||||
honestly.
|
||||
|
||||
Insert-only growth is fine at pilot scale (a-12): learner_id indexed,
|
||||
latest-per-learner lookup, no compaction pre-vendor.
|
||||
|
||||
Boundary (D-027): `identity/` never imports `agents/` / `api/`; this
|
||||
module imports config only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Protocol
|
||||
|
||||
from sqlalchemy import event, text
|
||||
from sqlalchemy.types import JSON, String
|
||||
from sqlmodel import Field, Session, SQLModel, create_engine
|
||||
|
||||
from ..config import Settings
|
||||
from .base import IdentityStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class IdentityRecord(SQLModel, table=True):
|
||||
"""One verification submission's lifecycle + verdict provenance."""
|
||||
|
||||
__tablename__ = "identity_record"
|
||||
|
||||
#: PK = the submission id minted once by the provider's submit().
|
||||
id: str = Field(primary_key=True)
|
||||
learner_id: str = Field(index=True)
|
||||
# Bare Literal annotations crash sqlmodel's column inference; explicit
|
||||
# sa_type + the validates hook below give the same contract
|
||||
# (VARCHAR column, Literal-rejected values — DefenseStore pattern).
|
||||
status: IdentityStatus = Field(default="pending", sa_type=String)
|
||||
provider: str
|
||||
#: Verdict provenance: the provider's raw verdict (mock-marked).
|
||||
verdict: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
#: DERIVED band only — raw DOB is never persisted (A-305).
|
||||
age_band: str | None = Field(default=None, sa_type=String)
|
||||
#: Opaque document handles — raw documents never stored (A-305).
|
||||
document_refs: list[str] = Field(default_factory=list, sa_type=JSON)
|
||||
submitted_at: datetime
|
||||
verified_at: datetime | None = Field(default=None)
|
||||
|
||||
@property
|
||||
def mock(self) -> bool:
|
||||
"""A-304: the mock marker rides every surface (record + API)."""
|
||||
return bool(self.verdict.get("mock", True))
|
||||
|
||||
def _validate(self) -> None:
|
||||
if not self.id or not self.learner_id:
|
||||
raise ValueError("id and learner_id must be non-empty")
|
||||
if self.status not in ("pending", "verified", "rejected"):
|
||||
raise ValueError(f"invalid identity status {self.status!r}")
|
||||
# D3 (verifier): a stored band must be canonical or None (pending).
|
||||
# The gate fails closed on anything else; the store refuses to
|
||||
# create it in the first place.
|
||||
if self.age_band is not None and self.age_band not in (
|
||||
"16-17",
|
||||
"18+",
|
||||
"under-16",
|
||||
):
|
||||
raise ValueError(f"invalid age_band {self.age_band!r}")
|
||||
|
||||
def _normalize(self) -> None:
|
||||
self.submitted_at = _as_utc(self.submitted_at)
|
||||
if self.verified_at is not None:
|
||||
self.verified_at = _as_utc(self.verified_at)
|
||||
|
||||
|
||||
def _as_utc(ts: datetime) -> datetime:
|
||||
"""SQLite stores naive; read paths re-label tz-aware UTC (D-027 pattern)."""
|
||||
if ts.tzinfo is None:
|
||||
return ts.replace(tzinfo=UTC)
|
||||
return ts
|
||||
|
||||
|
||||
def _sqlite_connect(dbapi_connection: object, _: object) -> None:
|
||||
"""Per-connection pragmas — mirrors the other D-027 stores."""
|
||||
cursor = dbapi_connection.cursor() # type: ignore[attr-defined]
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA synchronous=NORMAL")
|
||||
cursor.execute("PRAGMA busy_timeout=5000")
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close() # type: ignore[attr-defined]
|
||||
|
||||
|
||||
class IdentityStore(Protocol):
|
||||
"""Persistence contract for identity records."""
|
||||
|
||||
def insert(self, record: IdentityRecord) -> IdentityRecord:
|
||||
"""INSERT-ONLY: a duplicate id raises IntegrityError (surfaced, not
|
||||
swallowed — a submission id is minted once)."""
|
||||
...
|
||||
|
||||
def get(self, submission_id: str) -> IdentityRecord | None:
|
||||
"""Point lookup by submission id."""
|
||||
...
|
||||
|
||||
def latest_for_learner(self, learner_id: str) -> IdentityRecord | None:
|
||||
"""Newest record for the learner (or None)."""
|
||||
...
|
||||
|
||||
def count_pending_for_learner(self, learner_id: str) -> int:
|
||||
"""G-13: active pending submissions (cap = 1)."""
|
||||
...
|
||||
|
||||
def mark_verified(
|
||||
self, submission_id: str, verdict: dict[str, Any], age_band: str | None
|
||||
) -> IdentityRecord | None:
|
||||
"""Terminal transition (verified or rejected): stamp + store the
|
||||
provider verdict + derived band. Unknown id → None."""
|
||||
...
|
||||
|
||||
|
||||
class SQLiteIdentityStore:
|
||||
"""SQLite implementation of IdentityStore (D-027)."""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
self._db_path: Path = db_path if db_path is not None else Settings().db_path
|
||||
self._engine = create_engine(
|
||||
f"sqlite:///{self._db_path}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
event.listen(self._engine, "connect", _sqlite_connect)
|
||||
SQLModel.metadata.create_all(self._engine)
|
||||
|
||||
def insert(self, record: IdentityRecord) -> IdentityRecord:
|
||||
record._validate()
|
||||
record._normalize()
|
||||
with Session(self._engine) as session:
|
||||
session.add(record)
|
||||
session.commit() # IntegrityError SURFACES (insert-only, minted-once)
|
||||
session.refresh(record)
|
||||
return record
|
||||
|
||||
def get(self, submission_id: str) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
rec = session.get(IdentityRecord, submission_id)
|
||||
if rec is None:
|
||||
return None
|
||||
session.refresh(rec)
|
||||
rec.submitted_at = _as_utc(rec.submitted_at)
|
||||
if rec.verified_at is not None:
|
||||
rec.verified_at = _as_utc(rec.verified_at)
|
||||
return rec
|
||||
|
||||
def latest_for_learner(self, learner_id: str) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
# D4 (verifier): submitted_at alone can tie at microsecond
|
||||
# resolution — sqlite rowid breaks the tie deterministically
|
||||
# (last inserted wins, mirroring insert-only chronology).
|
||||
# rowid is a SQLite physical column, not a SQLModel field — it
|
||||
# rides the query as raw text.
|
||||
rec = (
|
||||
session.query(IdentityRecord)
|
||||
.filter(IdentityRecord.learner_id == learner_id)
|
||||
.order_by(
|
||||
IdentityRecord.submitted_at.desc(),
|
||||
text("rowid DESC"),
|
||||
)
|
||||
.first()
|
||||
)
|
||||
if rec is not None:
|
||||
rec.submitted_at = _as_utc(rec.submitted_at)
|
||||
if rec.verified_at is not None:
|
||||
rec.verified_at = _as_utc(rec.verified_at)
|
||||
return rec
|
||||
|
||||
def count_pending_for_learner(self, learner_id: str) -> int:
|
||||
with Session(self._engine) as session:
|
||||
return (
|
||||
session.query(IdentityRecord)
|
||||
.filter(
|
||||
IdentityRecord.learner_id == learner_id,
|
||||
IdentityRecord.status == "pending",
|
||||
)
|
||||
.count()
|
||||
)
|
||||
|
||||
def mark_verified(
|
||||
self, submission_id: str, verdict: dict[str, Any], age_band: str | None
|
||||
) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
rec = session.get(IdentityRecord, submission_id)
|
||||
if rec is None:
|
||||
return None
|
||||
rec.verdict = verdict
|
||||
rec.age_band = age_band
|
||||
rec.verified_at = datetime.now(UTC)
|
||||
rec.status = "verified" if verdict.get("status") == "verified" else "rejected"
|
||||
rec._validate() # D3: transitions validate like inserts
|
||||
session.commit()
|
||||
session.refresh(rec)
|
||||
return rec
|
||||
|
||||
def close(self) -> None:
|
||||
self._engine.dispose()
|
||||
@@ -25,6 +25,8 @@ from .api import (
|
||||
from .config import Settings
|
||||
from .grading.engine import GradingEngine
|
||||
from .grading.store import SQLiteGradeStore
|
||||
from .identity.mock import MockIdentityProvider
|
||||
from .identity.store import SQLiteIdentityStore
|
||||
from .llm import create_provider
|
||||
from .sandbox import SandboxManager, UnshareBackend
|
||||
from .telemetry.ingest import TraceIntegrityMap
|
||||
@@ -33,6 +35,7 @@ from .variants.generator import VariantGenerator
|
||||
from .variants.store import SQLiteVariantStore
|
||||
from .voice.defense_store import SQLiteDefenseStore
|
||||
from .voice.factory import voice_provider_from_settings
|
||||
from .voice.mock import MockVoiceProvider
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -116,12 +119,37 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
defense_store = SQLiteDefenseStore(db_path=settings.db_path)
|
||||
app.state.defense_store = defense_store
|
||||
if getattr(app.state, "voice_provider", None) is None:
|
||||
app.state.voice_provider = voice_provider_from_settings(settings)
|
||||
# G-11 (boot survival): a misconfigured real provider must never
|
||||
# crash the unattended deploy — fall back to mock loudly. The
|
||||
# mock provider's descriptor honestly reports mode='mock' so the
|
||||
# UI badge cannot lie about which path is live.
|
||||
try:
|
||||
app.state.voice_provider = voice_provider_from_settings(
|
||||
settings, app.state.http_client
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"voice provider %r unavailable (%s); falling back to mock "
|
||||
"— fix the AI_VOICE_* settings and restart",
|
||||
settings.voice_provider,
|
||||
exc,
|
||||
)
|
||||
app.state.voice_provider = MockVoiceProvider()
|
||||
if getattr(app.state, "examiner_agent", None) is None:
|
||||
from .agents.examiner import ExaminerAgent
|
||||
|
||||
app.state.examiner_agent = ExaminerAgent(app.state.provider, settings)
|
||||
|
||||
# Identity verification (REQ-5-003): 5th D-027 store (same SQLite
|
||||
# file) + mock-first provider (A-303). State-injection overrides
|
||||
# preserved — tests may pre-set either.
|
||||
identity_store = getattr(app.state, "identity_store", None)
|
||||
if identity_store is None:
|
||||
identity_store = SQLiteIdentityStore(db_path=settings.db_path)
|
||||
app.state.identity_store = identity_store
|
||||
if getattr(app.state, "identity_provider", None) is None:
|
||||
app.state.identity_provider = MockIdentityProvider()
|
||||
|
||||
# Grading persistence + engine (REQ-3-004): GradeStore from the same
|
||||
# SQLite file as traces (D-027), one GradingEngine singleton wired
|
||||
# through app.state — the engine receives its stores via constructor
|
||||
@@ -167,18 +195,21 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
grade_store.close()
|
||||
variant_store.close()
|
||||
defense_store.close()
|
||||
identity_store.close()
|
||||
await app.state.http_client.aclose()
|
||||
|
||||
app = FastAPI(title="Nextcraft AI Service", version="0.3.0", lifespan=lifespan)
|
||||
|
||||
# A-008: localhost-only CORS, no credentials. PUT is CONTRACT, not
|
||||
# trivia: the learner build surface writes workspace files with PUT
|
||||
# (engine-client writeFile) — v0.3 initially shipped without it and
|
||||
# every cross-origin Save failed preflight (caught in P7 review;
|
||||
# tests/api/test_cors.py pins the policy now).
|
||||
# A-008 + D-038: no-credentials CORS. Default '*' admits remote-browser
|
||||
# origins in network mode (safe only because allow_credentials stays
|
||||
# False — never enable credentials with a wildcard). AI_CORS_ORIGINS
|
||||
# restricts to an explicit list. PUT is CONTRACT, not trivia: the learner
|
||||
# build surface writes workspace files with PUT (engine-client writeFile)
|
||||
# — v0.3 initially shipped without it and every cross-origin Save failed
|
||||
# preflight (caught in P7 review; tests/api/test_cors.py pins the policy).
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://localhost:3000", "http://127.0.0.1:3000"],
|
||||
allow_origins=settings.cors_origin_list,
|
||||
allow_methods=["GET", "POST", "PUT", "DELETE", "OPTIONS"],
|
||||
allow_headers=["Content-Type"],
|
||||
allow_credentials=False,
|
||||
@@ -201,6 +232,74 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
app.include_router(telemetry_router)
|
||||
app.include_router(variants_router)
|
||||
app.include_router(defense_router)
|
||||
|
||||
# v0.5 identity (REQ-5-003/004): verification flow + age-gate deps
|
||||
# + the one marketplace 18+ gated stub (G-18).
|
||||
from .api.identity import marketplace_router
|
||||
from .api.identity import router as identity_router
|
||||
|
||||
app.include_router(identity_router)
|
||||
app.include_router(marketplace_router)
|
||||
|
||||
# A-305/D1: PII-safe 422s — FastAPI echoes the offending `input` in
|
||||
# validation errors by default; for the identity submit body that
|
||||
# leaks the raw DOB into responses + client logs. The handler scrubs
|
||||
# PII field inputs (scoped app-wide; harmless elsewhere).
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
def _scrub_pii_validation(request, exc): # type: ignore[unused-arg]
|
||||
pii_fields = frozenset({"date_of_birth"})
|
||||
scrubbed = []
|
||||
for err in exc.errors():
|
||||
err = dict(err)
|
||||
if err.get("loc") and err["loc"][-1] in pii_fields:
|
||||
# A-305/D1: never echo the submitted value (input) or the
|
||||
# ctx payload (carries the ValueError); the msg is the
|
||||
# constraint text and is safe.
|
||||
err["input"] = "[redacted]"
|
||||
err.pop("ctx", None)
|
||||
else:
|
||||
# FastAPI's default 422s are JSON-safe EXCEPT ctx payloads
|
||||
# carrying raw ValueError objects (pydantic model_validator
|
||||
# errors); strip ctx body-wide so non-PII routes keep their
|
||||
# 422 shape (msg + loc carry the meaning).
|
||||
ctx = err.get("ctx")
|
||||
if isinstance(ctx, dict):
|
||||
err["ctx"] = {
|
||||
k: v for k, v in ctx.items() if isinstance(v, (str, int, float, bool))
|
||||
}
|
||||
scrubbed.append(err)
|
||||
return JSONResponse(status_code=422, content={"detail": scrubbed})
|
||||
|
||||
app.add_exception_handler(RequestValidationError, _scrub_pii_validation)
|
||||
|
||||
# v0.3.6 single-port deploy: serve the exported web app (apps/web/out)
|
||||
# from the SAME origin as the API when AI_WEB_STATIC_DIR is set. Mounted
|
||||
# AFTER all routers, so /v1/*, /health, /docs win; StaticFiles(html=True)
|
||||
# then resolves / → index.html, /dashboard/ → dashboard/index.html. A
|
||||
# 404 handler below serves the export's 404.html for unknown paths so
|
||||
# browsers see the site's not-found page instead of FastAPI's JSON.
|
||||
# Default unset → no mount, dev/tests see the plain API app.
|
||||
if str(settings.web_static_dir) not in ("", "."):
|
||||
from fastapi.responses import FileResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
|
||||
web_dir = settings.web_static_dir
|
||||
if not web_dir.is_dir():
|
||||
raise RuntimeError(
|
||||
f"AI_WEB_STATIC_DIR is set but {web_dir} does not exist — "
|
||||
"build the web app first (pnpm build) or unset the setting"
|
||||
)
|
||||
|
||||
@app.exception_handler(404)
|
||||
async def _spa_404(request, exc): # type: ignore[unused]
|
||||
not_found_page = web_dir / "404.html"
|
||||
if not_found_page.is_file():
|
||||
return FileResponse(not_found_page, status_code=404)
|
||||
raise exc
|
||||
|
||||
app.mount("/", StaticFiles(directory=web_dir, html=True), name="web")
|
||||
return app
|
||||
|
||||
|
||||
|
||||
@@ -125,6 +125,9 @@ class SandboxManager:
|
||||
self._clock = clock or (lambda: datetime.now(UTC))
|
||||
self._handles: dict[str, SandboxHandle] = {}
|
||||
self._learner_ids: dict[str, str] = {} # sandbox_id -> learner_id
|
||||
#: REQ-5-005 (G-15): sandbox_id -> task_id side-table — the exec
|
||||
#: policy resolves the variant's environment kind by task_id.
|
||||
self._task_ids: dict[str, str | None] = {}
|
||||
self._lock = asyncio.Lock()
|
||||
self._integrity_events: list[SandboxIntegrityEvent] = []
|
||||
self._started = False
|
||||
@@ -172,6 +175,7 @@ class SandboxManager:
|
||||
handle = await self._backend.spawn(spec)
|
||||
self._handles[handle.id] = handle
|
||||
self._learner_ids[handle.id] = learner_id
|
||||
self._task_ids[handle.id] = task_id
|
||||
self._write_pid_marker(handle, learner_id)
|
||||
logger.info(
|
||||
"sandbox created: id=%s learner=%s task=%s",
|
||||
@@ -246,6 +250,7 @@ class SandboxManager:
|
||||
async with self._lock:
|
||||
handle = self._handles.pop(sandbox_id, None)
|
||||
learner_id = self._learner_ids.pop(sandbox_id, "unknown")
|
||||
self._task_ids.pop(sandbox_id, None)
|
||||
if handle is not None:
|
||||
await self._backend.destroy(handle)
|
||||
logger.info(
|
||||
|
||||
@@ -15,6 +15,9 @@ frame — no envelope:
|
||||
Server → client frames are typed status envelopes:
|
||||
|
||||
{"type": "ack_total", "count": N} — final flush summary, then close 1000
|
||||
{"type": "seq_ack", "seq": N} — advisory: durable latest_seq after
|
||||
each successful append (D-045; the
|
||||
agent trims its spool to seq > ack)
|
||||
{"type": "gap_warning", "missing_seqs": [...]} — seq skipped ahead
|
||||
{"type": "event_rejected", "detail": "..."} — one frame failed validation
|
||||
(seq echoed when parseable)
|
||||
@@ -349,6 +352,13 @@ class IngestSession:
|
||||
else:
|
||||
self._stored += 1
|
||||
self._seen.add(frame.seq)
|
||||
# Seq-ack (D-045, REQ-5-007): advisory hint carrying the durable
|
||||
# latest_seq AFTER this append — the capture agent trims its spool to
|
||||
# seq > ack on receipt, bounding the replay margin to the in-flight
|
||||
# window. Emitted on dedup'd appends too (a-6) so a replay flush
|
||||
# tightens the margin immediately. Gap detection stays authoritative
|
||||
# (_check_gap below); G-3 flood semantics untouched.
|
||||
await self._send_json({"type": "seq_ack", "seq": after})
|
||||
await self._check_gap(frame.seq)
|
||||
# SQLite appends are sync and fast; on a burst the drainer can hold
|
||||
# the loop between receives. Yield so the WS writer flushes the close
|
||||
|
||||
@@ -94,6 +94,8 @@ class VariantGenerator:
|
||||
params=dict(params),
|
||||
statement=statement,
|
||||
starter_files=dict(template.starter_files),
|
||||
environment=template.environment,
|
||||
test_command=template.test_command,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
self._store.save(record)
|
||||
|
||||
@@ -41,7 +41,7 @@ from pathlib import Path
|
||||
from typing import Any, Protocol
|
||||
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy import JSON, Index, UniqueConstraint
|
||||
from sqlalchemy import JSON, Index, String, UniqueConstraint
|
||||
from sqlalchemy.orm import validates
|
||||
from sqlmodel import Field, Session, SQLModel, create_engine, select
|
||||
|
||||
@@ -105,6 +105,12 @@ class VariantRecord(SQLModel, table=True):
|
||||
params: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
statement: str
|
||||
starter_files: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
#: REQ-5-005 (D-044): the environment kind rides the record to the API
|
||||
#: and TS client; defaults keep pre-v0.5 rows 'build'.
|
||||
environment: str = Field(default="build", sa_type=String)
|
||||
#: The variant's real test command (was a dead template field — v0.5
|
||||
#: surfaces it so the Run/Test buttons stop hardcoding pytest).
|
||||
test_command: str = Field(default="", sa_type=String)
|
||||
created_at: datetime
|
||||
|
||||
@validates("learner_id", "template_id", "task_id")
|
||||
@@ -215,6 +221,40 @@ class SQLiteVariantStore:
|
||||
self._engine = create_engine(f"sqlite:///{self._db_path}")
|
||||
sa.event.listen(self._engine, "connect", _sqlite_connect)
|
||||
SQLModel.metadata.create_all(self._engine)
|
||||
# v0.5 schema (D-044) added two columns to an existing table.
|
||||
# create_all does NOT ALTER existing tables: on a box with a
|
||||
# pre-v0.5 ~/.nextcraft/data/nextcraft.db, every variant read/write
|
||||
# would raise OperationalError("no such column: variant_record.
|
||||
# environment") — a silent total breakage of the variant path
|
||||
# (final-review P0, verified empirically). Backfill the missing
|
||||
# columns with the model defaults ('build' keeps pre-v0.5 rows
|
||||
# build-kind per the field contract; '' falls back to pytest at
|
||||
# the API seam, api/variants._to_response). Idempotent: the
|
||||
# PRAGMA table_info check makes re-runs no-ops.
|
||||
self._ensure_v05_columns()
|
||||
|
||||
def _ensure_v05_columns(self) -> None:
|
||||
"""Add v0.5 columns to a pre-v0.5 variant_record table (idempotent)."""
|
||||
from sqlalchemy import text
|
||||
|
||||
with self._engine.begin() as conn:
|
||||
columns = {row[1] for row in conn.execute(text("PRAGMA table_info(variant_record)"))}
|
||||
if "environment" not in columns:
|
||||
conn.execute(
|
||||
text(
|
||||
"ALTER TABLE variant_record ADD COLUMN environment "
|
||||
"VARCHAR DEFAULT 'build' NOT NULL"
|
||||
)
|
||||
)
|
||||
logger.info("variant store: backfilled 'environment' (pre-v0.5 schema)")
|
||||
if "test_command" not in columns:
|
||||
conn.execute(
|
||||
text(
|
||||
"ALTER TABLE variant_record ADD COLUMN test_command "
|
||||
"VARCHAR DEFAULT '' NOT NULL"
|
||||
)
|
||||
)
|
||||
logger.info("variant store: backfilled 'test_command' (pre-v0.5 schema)")
|
||||
|
||||
@contextmanager
|
||||
def _session(self) -> Iterator[Session]:
|
||||
|
||||
@@ -20,6 +20,31 @@ from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
#: REQ-5-005 environment kinds (D-044): one namespace fabric, typed starter
|
||||
#: contents + command policy. 'build' = the v0.3 coding IDE; 'design' =
|
||||
#: artifact editing with a validator/renderer harness; 'simulation' = a
|
||||
#: parameterized run harness (benchmark scripts + datasets).
|
||||
EnvironmentKind = Literal["build", "design", "simulation"]
|
||||
|
||||
|
||||
def validate_simple_argv(value: str) -> str:
|
||||
"""G-15: command fields must roundtrip shlex.split → join → split.
|
||||
|
||||
No quotes, no shell metachars — the TS client splits on whitespace only
|
||||
(no shlex in browsers), so anything quote-aware would split differently
|
||||
on the two sides. A violation is a template-AUTHORING bug caught here,
|
||||
at definition time, in Python where shlex exists.
|
||||
"""
|
||||
import shlex
|
||||
|
||||
parts = shlex.split(value)
|
||||
if not parts:
|
||||
raise ValueError("command must not be empty")
|
||||
joined = " ".join(parts)
|
||||
if shlex.split(joined) != parts:
|
||||
raise ValueError(f"command is not whitespace-joinable: {value!r}")
|
||||
return joined
|
||||
|
||||
SlotType = Literal["enum", "int_range", "string_set"]
|
||||
|
||||
|
||||
@@ -99,6 +124,12 @@ class TaskTemplate(BaseModel):
|
||||
rubric_anchors: RubricAnchors
|
||||
starter_files: dict[str, str] = Field(default_factory=dict) # path -> content
|
||||
test_command: str
|
||||
#: REQ-5-005 (D-044): the environment kind rides the variant through
|
||||
#: the API to the TS client; 'build' default keeps v0.3 behavior.
|
||||
environment: EnvironmentKind = "build"
|
||||
#: The kind's Run harness (design: validator/renderer; simulation:
|
||||
#: benchmark script). Defaults to the test_command for build kinds.
|
||||
harness_command: str = ""
|
||||
|
||||
@field_validator("statement_skeleton")
|
||||
@classmethod
|
||||
@@ -107,6 +138,16 @@ class TaskTemplate(BaseModel):
|
||||
raise ValueError("statement_skeleton needs at least one {slot}")
|
||||
return v
|
||||
|
||||
@field_validator("test_command", "harness_command")
|
||||
@classmethod
|
||||
def _simple_argv(cls, v: str) -> str:
|
||||
return validate_simple_argv(v) if v else v
|
||||
|
||||
@property
|
||||
def run_command(self) -> str:
|
||||
"""The Run button's command: kind harness when declared, else tests."""
|
||||
return self.harness_command or self.test_command
|
||||
|
||||
def render(self, params: dict[str, str | int]) -> str:
|
||||
"""Fill the skeleton with validated params."""
|
||||
for slot in self.slots:
|
||||
@@ -261,6 +302,134 @@ TEMPLATES: dict[str, TaskTemplate] = {
|
||||
},
|
||||
test_command="pytest -q",
|
||||
),
|
||||
# -- REQ-5-005 design environment (D-044): artifact editing with a
|
||||
# -- validator/renderer harness - same fabric, typed starter contents.
|
||||
"tpl-conversation-flow-design": TaskTemplate(
|
||||
id="tpl-conversation-flow-design",
|
||||
competency_id="stack-designer-c001",
|
||||
title="Conversational Flow Artifact",
|
||||
statement_skeleton=(
|
||||
"Design a conversational flow for a {persona} assistant helping "
|
||||
"users accomplish {goal}. Author the flow as a structured artifact "
|
||||
"with at least {turn_count} conversation turns, explicit fallback "
|
||||
"paths for misunderstandings, and an AI-transparency disclosure "
|
||||
"pattern. The flow must render validly (the harness validates "
|
||||
"structure) and read naturally end to end."
|
||||
),
|
||||
slots=[
|
||||
ParameterSlot(
|
||||
name="persona",
|
||||
type="enum",
|
||||
values=["travel-planner", "homework-tutor", "fitness-coach", "recipe-guide"],
|
||||
),
|
||||
ParameterSlot(
|
||||
name="goal",
|
||||
type="enum",
|
||||
values=["book-a-trip", "master-a-concept", "start-a-routine", "cook-a-meal"],
|
||||
),
|
||||
ParameterSlot(name="turn_count", type="int_range", lo=6, hi=12),
|
||||
],
|
||||
rubric_anchors=RubricAnchors(
|
||||
expected_edit_count_band=(3, 20),
|
||||
expected_min_test_runs=1,
|
||||
expected_error_fix_cycles_band=(0, 3),
|
||||
notes="Design kind: artifact quality + iteration cadence, not code depth.",
|
||||
),
|
||||
starter_files={
|
||||
"README.md": (
|
||||
"# Conversational Flow Design\n\n"
|
||||
"Edit flow.md - the structured flow artifact. python3 "
|
||||
"validate_flow.py checks structure (turn headings, fallback "
|
||||
"sections, a transparency disclosure) and reports issues.\n"
|
||||
),
|
||||
"flow.md": (
|
||||
"# Flow: your persona here\n\n"
|
||||
"## Turn 1\n- **AI:** (opening)\n- **User (expected):** ...\n\n"
|
||||
"## Fallback\n- (misunderstanding handling)\n\n"
|
||||
"## AI Transparency Disclosure\n- (disclosure pattern)\n"
|
||||
),
|
||||
"validate_flow.py": (
|
||||
"import re, sys\n"
|
||||
"text = open('flow.md').read()\n"
|
||||
"issues = []\n"
|
||||
"turns = len(re.findall(r'^## Turn', text, re.M))\n"
|
||||
"if turns < 3:\n"
|
||||
" issues.append(f'expected at least 3 turn sections, found {turns}')\n"
|
||||
"if not re.search(r'^## Fallback', text, re.M):\n"
|
||||
" issues.append('missing Fallback section')\n"
|
||||
"if not re.search(r'^## AI Transparency', text, re.M):\n"
|
||||
" issues.append('missing AI Transparency Disclosure')\n"
|
||||
"print('VALID' if not issues else 'ISSUES: ' + '; '.join(issues))\n"
|
||||
"sys.exit(0 if not issues else 1)\n"
|
||||
),
|
||||
},
|
||||
test_command="python3 validate_flow.py",
|
||||
environment="design",
|
||||
harness_command="python3 validate_flow.py",
|
||||
),
|
||||
# -- REQ-5-005 simulation environment (D-044): parameterized benchmark
|
||||
# -- harness with dataset generation.
|
||||
"tpl-sensor-benchmark": TaskTemplate(
|
||||
id="tpl-sensor-benchmark",
|
||||
competency_id="stack-orchestration-c011",
|
||||
title="Sensor Data Simulation Harness",
|
||||
statement_skeleton=(
|
||||
"Build a simulation harness for {sensor} readings over {duration_min} "
|
||||
"minutes at {sample_hz} Hz. Generate a synthetic dataset with a "
|
||||
"realistic noise profile, run the analysis pipeline, and print a "
|
||||
"metrics summary (mean, p95, anomaly count at {anomaly_sigma} sigma). "
|
||||
"The harness must be reproducible from the committed seed."
|
||||
),
|
||||
slots=[
|
||||
ParameterSlot(
|
||||
name="sensor",
|
||||
type="enum",
|
||||
values=["temperature", "vibration", "luminosity", "pressure"],
|
||||
),
|
||||
ParameterSlot(name="duration_min", type="int_range", lo=5, hi=60),
|
||||
ParameterSlot(name="sample_hz", type="int_range", lo=1, hi=10),
|
||||
ParameterSlot(name="anomaly_sigma", type="int_range", lo=2, hi=4),
|
||||
],
|
||||
rubric_anchors=RubricAnchors(
|
||||
expected_edit_count_band=(3, 25),
|
||||
expected_min_test_runs=2,
|
||||
expected_error_fix_cycles_band=(0, 3),
|
||||
notes="Simulation kind: pipeline correctness + reproducibility.",
|
||||
),
|
||||
starter_files={
|
||||
"README.md": (
|
||||
"# Sensor Simulation Harness\n\n"
|
||||
"Edit simulate.py - python3 simulate.py runs the full pipeline: "
|
||||
"generate, analyze, print metrics. pytest covers the analysis "
|
||||
"functions.\n"
|
||||
),
|
||||
"simulate.py": (
|
||||
"import random, statistics\n\n"
|
||||
"def generate(n=300, seed=42):\n"
|
||||
" rng = random.Random(seed)\n"
|
||||
" return [rng.gauss(20.0, 1.5) for _ in range(n)]\n\n"
|
||||
"def analyze(samples, sigma=3):\n"
|
||||
" mean = statistics.fmean(samples)\n"
|
||||
" stdev = statistics.pstdev(samples)\n"
|
||||
" anomalies = [s for s in samples if abs(s - mean) > sigma * stdev]\n"
|
||||
" p95 = sorted(samples)[int(0.95 * len(samples))]\n"
|
||||
" return {'mean': mean, 'p95': p95, 'anomalies': len(anomalies)}\n\n"
|
||||
"if __name__ == '__main__':\n"
|
||||
" print(analyze(generate()))\n"
|
||||
),
|
||||
"test_simulate.py": (
|
||||
"from simulate import generate, analyze\n\n"
|
||||
"def test_reproducible():\n"
|
||||
" assert generate() == generate()\n\n"
|
||||
"def test_metrics_shape():\n"
|
||||
" m = analyze(generate())\n"
|
||||
" assert set(m) == {'mean', 'p95', 'anomalies'}\n"
|
||||
),
|
||||
},
|
||||
test_command="pytest -q",
|
||||
environment="simulation",
|
||||
harness_command="python3 simulate.py",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ class VoiceDescriptor(BaseModel):
|
||||
"""Capability descriptor served to the web client (D-030).
|
||||
|
||||
The assessment UI reads this to decide HOW the learner speaks/hears:
|
||||
- `mode="server"` → server-side STT/TTS (v0.4 real provider seam)
|
||||
- `mode="server"` → server-side STT/TTS (openai-audio, live since v0.5)
|
||||
- `mode="browser"` → browser-native SpeechRecognition/speechSynthesis
|
||||
- `mode="mock"` → deterministic no-op path (tests / no-key dev)
|
||||
The descriptor never contains secrets — only capability hints.
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
"""Browser-native fallback descriptor (D-030, CUT-1 / G-7, REQ-3-006).
|
||||
|
||||
v0.3 has NO real server STT/TTS (deferred to v0.4 with KYC/keys — GRILL
|
||||
CUT-1). When the factory selects `browser` mode, the defense endpoints return
|
||||
this descriptor and the WEB CLIENT performs SpeechRecognition + speechSynthesis
|
||||
natively; the server persists text turns as usual.
|
||||
Browser-native SR/TTS is the no-key CLIENT-side path. When the factory
|
||||
selects `browser` mode, the defense endpoints return this descriptor and the
|
||||
WEB CLIENT performs SpeechRecognition + speechSynthesis natively; the server
|
||||
persists text turns as usual. Real server STT/TTS (openai-audio) is live
|
||||
since v0.5 — this descriptor is the no-key fallback.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -26,7 +27,7 @@ MOCK_DESCRIPTOR = VoiceDescriptor(
|
||||
sr_available=True,
|
||||
tts_available=True,
|
||||
hint=(
|
||||
"Deterministic mock voice (tests / no-key dev). Server STT/TTS "
|
||||
"endpoints serve canned responses; real server STT/TTS lands in v0.4."
|
||||
"Deterministic mock voice (tests / no-key dev). Real server "
|
||||
"STT/TTS is live since v0.5 (AI_VOICE_PROVIDER=openai-audio)."
|
||||
),
|
||||
)
|
||||
|
||||
@@ -1,24 +1,37 @@
|
||||
"""Voice provider factory (D-030, REQ-3-006).
|
||||
"""Voice provider factory (D-030; REQ-5-001 real path, D-040).
|
||||
|
||||
`AI_VOICE_PROVIDER = browser | mock` (default: mock — the no-key path is
|
||||
first-class). The real server provider (`openai-audio`) is a v0.4 seam and
|
||||
is REJECTED here with a clear error naming the deferral, so a stale env var
|
||||
can't silently pretend a real backend exists.
|
||||
`AI_VOICE_PROVIDER = mock | browser | openai-audio` (default: mock — the
|
||||
no-key path is first-class). `openai-audio` requires voice_base_url +
|
||||
voice_api_key: the factory raises `UnknownVoiceProviderError` with an
|
||||
actionable message for direct callers (tests), while the lifespan in
|
||||
main.py CATCHES it and falls back to mock with a loud log — a typo'd env
|
||||
must never crash the unattended boot (G-11), and the mock provider's
|
||||
descriptor then honestly reports mode='mock' so the UI badge cannot lie.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
|
||||
from ..config import Settings
|
||||
from .base import VoiceProvider
|
||||
from .mock import MockVoiceProvider
|
||||
from .openai_audio import OpenAIAudioProvider
|
||||
|
||||
|
||||
class UnknownVoiceProviderError(ValueError):
|
||||
"""Raised for a provider name outside the v0.3 contract."""
|
||||
"""Raised for a provider name outside the contract, or a real provider
|
||||
selected without its required configuration."""
|
||||
|
||||
|
||||
def voice_provider_from_settings(settings: Settings) -> VoiceProvider:
|
||||
"""Select the voice provider by settings (env `AI_VOICE_PROVIDER`)."""
|
||||
def voice_provider_from_settings(
|
||||
settings: Settings, http_client: httpx.AsyncClient | None = None
|
||||
) -> VoiceProvider:
|
||||
"""Select the voice provider by settings (env `AI_VOICE_PROVIDER`).
|
||||
|
||||
`http_client` is required for the `openai-audio` branch (D-017 shared
|
||||
pool); mock/browser ignore it.
|
||||
"""
|
||||
name = (settings.voice_provider or "mock").strip().lower()
|
||||
if name == "mock":
|
||||
return MockVoiceProvider()
|
||||
@@ -28,10 +41,27 @@ def voice_provider_from_settings(settings: Settings) -> VoiceProvider:
|
||||
# uses the descriptor for mic/speech). See browser.py.
|
||||
return MockVoiceProvider()
|
||||
if name in ("openai-audio", "openai", "server"):
|
||||
raise UnknownVoiceProviderError(
|
||||
"real server STT/TTS (OpenAIAudioProvider) is deferred to v0.4 "
|
||||
"(GRILL CUT-1 / G-7): set AI_VOICE_PROVIDER=mock or browser"
|
||||
if not settings.voice_base_url or not settings.voice_api_key:
|
||||
raise UnknownVoiceProviderError(
|
||||
"AI_VOICE_PROVIDER=openai-audio requires AI_VOICE_BASE_URL "
|
||||
"and AI_VOICE_API_KEY — set both, or use 'mock'/'browser'. "
|
||||
"(main.py falls back to mock when these are missing; the "
|
||||
"voice badge then honestly reports mock — G-11)"
|
||||
)
|
||||
if http_client is None:
|
||||
raise UnknownVoiceProviderError(
|
||||
"openai-audio requires the shared httpx client "
|
||||
"(voice_provider_from_settings(settings, http_client))"
|
||||
)
|
||||
return OpenAIAudioProvider(
|
||||
http_client=http_client,
|
||||
base_url=settings.voice_base_url,
|
||||
api_key=settings.voice_api_key,
|
||||
stt_model=settings.voice_stt_model,
|
||||
tts_model=settings.voice_tts_model,
|
||||
tts_voice=settings.voice_tts_voice,
|
||||
tts_format=settings.voice_tts_format,
|
||||
)
|
||||
raise UnknownVoiceProviderError(
|
||||
f"unknown AI_VOICE_PROVIDER {name!r}: use 'mock' or 'browser'"
|
||||
f"unknown AI_VOICE_PROVIDER {name!r}: use 'mock', 'browser', or 'openai-audio'"
|
||||
)
|
||||
|
||||
@@ -75,7 +75,7 @@ class MockVoiceProvider:
|
||||
|
||||
async def synthesize(self, text: str, voice: str = "default") -> AsyncIterator[bytes]: # noqa: ASYNC109 (protocol parity)
|
||||
# NOTE: protocol parity matters more than the async-generator purity
|
||||
# lint; the real provider seam (v0.4) will stream over HTTP.
|
||||
# lint; the real provider (openai_audio.py) streams over HTTP.
|
||||
self.synthesize_calls += 1
|
||||
if not text:
|
||||
raise MockVoiceFailure("cannot synthesize empty text")
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
"""OpenAI-compatible audio provider — real server STT/TTS (D-040, REQ-5-001).
|
||||
|
||||
One implementation serves any OpenAI-compatible audio endpoint (base_url is
|
||||
config; A-301 endpoint-agnostic by config, D-014 pattern). Raw httpx on the
|
||||
shared lifespan client (D-017; read=300s tolerates multi-minute clips).
|
||||
|
||||
Boundary rules (mirror llm/openai_compat.py):
|
||||
- voice/ imports nothing from agents/ or api/
|
||||
- api_key NEVER appears in exceptions, logs, or error messages
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import Literal
|
||||
|
||||
import httpx
|
||||
|
||||
from .base import TranscriptSegment, VoiceDescriptor
|
||||
|
||||
|
||||
class OpenAIAudioProvider:
|
||||
"""Server STT (`/audio/transcriptions`) + TTS (`/audio/speech`)."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
http_client: httpx.AsyncClient,
|
||||
base_url: str,
|
||||
api_key: str,
|
||||
stt_model: str = "whisper-1",
|
||||
tts_model: str = "tts-1",
|
||||
tts_voice: str = "alloy",
|
||||
tts_format: Literal["mp3", "wav", "opus"] = "mp3",
|
||||
) -> None:
|
||||
self._client = http_client
|
||||
self._base_url = base_url.rstrip("/")
|
||||
self._api_key = api_key
|
||||
self._stt_model = stt_model
|
||||
self._tts_model = tts_model
|
||||
self._tts_voice = tts_voice
|
||||
self._tts_format = tts_format
|
||||
# a-15: the descriptor is what defense.py prefers; a missing one
|
||||
# would badge the real server path as "mock".
|
||||
self.descriptor = VoiceDescriptor(
|
||||
mode="server",
|
||||
sr_available=True,
|
||||
tts_available=True,
|
||||
hint="server STT/TTS via AI_VOICE_BASE_URL",
|
||||
)
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
headers: dict[str, str] = {}
|
||||
if self._api_key:
|
||||
headers["Authorization"] = f"Bearer {self._api_key}"
|
||||
return headers
|
||||
|
||||
def _sanitize(self, exc: Exception) -> RuntimeError:
|
||||
text = str(exc)
|
||||
if self._api_key and self._api_key in text:
|
||||
text = text.replace(self._api_key, "[REDACTED]")
|
||||
return RuntimeError(f"voice provider error: {text}")
|
||||
|
||||
async def transcribe(self, audio: bytes, fmt: str) -> TranscriptSegment:
|
||||
"""STT: multipart upload (`file` + `model`) → TranscriptSegment.
|
||||
|
||||
`fmt` is a bare extension ('wav' | 'webm' | 'mp3') — the defense
|
||||
route strips codec params before this call (D-041).
|
||||
"""
|
||||
files = {"file": (f"answer.{fmt}", audio, f"audio/{fmt}")}
|
||||
data = {"model": self._stt_model, "response_format": "json"}
|
||||
try:
|
||||
resp = await self._client.post(
|
||||
f"{self._base_url}/audio/transcriptions",
|
||||
files=files,
|
||||
data=data,
|
||||
headers=self._headers(),
|
||||
)
|
||||
resp.raise_for_status()
|
||||
body = resp.json()
|
||||
except httpx.HTTPError as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
if not isinstance(body, dict):
|
||||
# P2 (verifier): a non-dict 200 body is a contract break —
|
||||
# context-wrap instead of a raw AttributeError.
|
||||
raise RuntimeError(
|
||||
"voice provider error: unexpected transcription response shape"
|
||||
)
|
||||
text = str(body.get("text", "")).strip()
|
||||
if not text:
|
||||
# 200 with an empty transcript is a provider contract break —
|
||||
# TranscriptSegment(min_length=1) would raise a bare pydantic
|
||||
# error; wrap it with provider context instead.
|
||||
raise RuntimeError("voice provider error: empty transcription")
|
||||
return TranscriptSegment(text=text)
|
||||
|
||||
def synthesize(
|
||||
self, text: str, voice: str = "default"
|
||||
) -> AsyncIterator[bytes]:
|
||||
"""TTS: JSON body → raw audio byte stream.
|
||||
|
||||
OpenAI's TTS caps `input` at 4096 chars; examiner questions are
|
||||
short, but enforce the guard so a long question fails loudly at the
|
||||
seam instead of as an opaque provider 400.
|
||||
"""
|
||||
return self._synthesize_stream(text, voice)
|
||||
|
||||
async def _synthesize_stream(
|
||||
self, text: str, voice: str
|
||||
) -> AsyncIterator[bytes]:
|
||||
if len(text) > 4096:
|
||||
raise RuntimeError(
|
||||
f"voice provider error: TTS input exceeds 4096 chars ({len(text)})"
|
||||
)
|
||||
payload = {
|
||||
"model": self._tts_model,
|
||||
"input": text,
|
||||
"voice": voice if voice != "default" else self._tts_voice,
|
||||
"response_format": self._tts_format,
|
||||
}
|
||||
try:
|
||||
async with self._client.stream(
|
||||
"POST",
|
||||
f"{self._base_url}/audio/speech",
|
||||
content=json.dumps(payload),
|
||||
headers={**self._headers(), "Content-Type": "application/json"},
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
async for chunk in resp.aiter_bytes():
|
||||
if chunk:
|
||||
yield chunk
|
||||
except httpx.HTTPError as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
@@ -1,28 +1,73 @@
|
||||
#!/usr/bin/env bash
|
||||
# Idempotent bootstrap: create venv + install deps.
|
||||
# Handles Debian systems without python3-venv/ensurepip via --without-pip + get-pip.
|
||||
# Handles Debian/Ubuntu systems without python3-venv/ensurepip via --without-pip + get-pip.
|
||||
# v2 (v0.3.5): recovers from a poisoned partial .venv left by a failed earlier
|
||||
# attempt, cleans before each retry, and dies with a distro-specific fix hint
|
||||
# when venv creation is impossible (e.g. missing python3.XX-venv package).
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
VENV="$APP_DIR/.venv"
|
||||
|
||||
venv_usable() {
|
||||
[ -x "$VENV/bin/python3" ]
|
||||
}
|
||||
|
||||
rm_broken_venv() {
|
||||
echo "bootstrap: removing broken partial .venv from a failed earlier attempt" >&2
|
||||
rm -rf "$VENV"
|
||||
}
|
||||
|
||||
mkdir -p "$HOME/.cache/ciagent"
|
||||
|
||||
if [ ! -x "$VENV/bin/python3" ]; then
|
||||
if python3 -m venv "$VENV" 2>/dev/null; then
|
||||
if venv_usable && [ ! -x "$VENV/bin/pip" ]; then
|
||||
# A usable python3 without pip means the --without-pip fallback half-ran and
|
||||
# the get-pip step never completed: start over cleanly.
|
||||
rm_broken_venv
|
||||
fi
|
||||
|
||||
if ! venv_usable; then
|
||||
if [ -d "$VENV" ]; then
|
||||
# Directory exists but no working python3: remains of a crashed venv create.
|
||||
rm_broken_venv
|
||||
fi
|
||||
if python3 -m venv "$VENV" 2>/tmp/venv-create.err; then
|
||||
:
|
||||
else
|
||||
# No ensurepip available — create bare venv and bootstrap pip separately.
|
||||
python3 -m venv --without-pip "$VENV"
|
||||
rm -rf "$VENV"
|
||||
if python3 -m venv --without-pip "$VENV" 2>>/tmp/venv-create.err; then
|
||||
:
|
||||
else
|
||||
rm -rf "$VENV"
|
||||
PYVER="$(python3 -c 'import sys; print("%d.%d" % sys.version_info[:2])' 2>/dev/null || true)"
|
||||
PKG="python3-venv"
|
||||
[ -n "$PYVER" ] && PKG="python${PYVER}-venv"
|
||||
echo "bootstrap: could not create a virtual environment." >&2
|
||||
echo " python3 reported:" >&2
|
||||
sed 's/^/ /' /tmp/venv-create.err >&2 || true
|
||||
echo " fix (Debian/Ubuntu): install the venv support package, then re-run nextcraft bootstrap:" >&2
|
||||
echo " apt install $PKG" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -x "$VENV/bin/pip" ]; then
|
||||
GET_PIP="$HOME/.cache/ciagent/get-pip.py"
|
||||
if [ ! -f "$GET_PIP" ]; then
|
||||
curl -sSf --max-time 60 https://bootstrap.pypa.io/get-pip.py -o "$GET_PIP"
|
||||
if ! curl -sSf --max-time 60 https://bootstrap.pypa.io/get-pip.py -o "$GET_PIP"; then
|
||||
rm -rf "$VENV"
|
||||
echo "bootstrap: get-pip.py download failed (no network?)." >&2
|
||||
echo " fix: restore network access and re-run nextcraft bootstrap" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
if ! "$VENV/bin/python3" "$GET_PIP" --quiet; then
|
||||
rm -rf "$VENV"
|
||||
echo "bootstrap: pip installation into the venv failed." >&2
|
||||
echo " fix: re-run nextcraft bootstrap (the venv was cleaned; this retry is safe)" >&2
|
||||
exit 1
|
||||
fi
|
||||
"$VENV/bin/python3" "$GET_PIP" --quiet
|
||||
fi
|
||||
|
||||
"$VENV/bin/pip" install --quiet --upgrade pip
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Dev server: export secrets (if present) then run uvicorn on :8420.
|
||||
# Dev server: export secrets (if present) then run uvicorn.
|
||||
# Binds 0.0.0.0 by default so the stack is reachable from other machines
|
||||
# (v0.3.5 network mode) — set AI_HOST=127.0.0.1 in .env to revert to loopback.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
@@ -7,7 +9,7 @@ REPO_ROOT="$(cd "$APP_DIR/../.." && pwd)"
|
||||
VENV="$APP_DIR/.venv"
|
||||
|
||||
if [ ! -x "$VENV/bin/uvicorn" ]; then
|
||||
echo "venv missing — run scripts/bootstrap.sh first" >&2
|
||||
echo "venv missing — run nextcraft bootstrap first (or: bash scripts/bootstrap.sh)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -22,5 +24,17 @@ if [ -f "$SECRETS" ]; then
|
||||
done < "$SECRETS"
|
||||
fi
|
||||
|
||||
ENV_FILE="$APP_DIR/.env"
|
||||
if [ -f "$ENV_FILE" ]; then
|
||||
while IFS='=' read -r key value; do
|
||||
case "$key" in
|
||||
AI_HOST|AI_PORT|AI_CORS_ORIGINS) export "$key=$value" ;;
|
||||
esac
|
||||
done < "$ENV_FILE"
|
||||
fi
|
||||
|
||||
HOST="${AI_HOST:-0.0.0.0}"
|
||||
PORT="${AI_PORT:-8420}"
|
||||
|
||||
cd "$APP_DIR"
|
||||
exec "$VENV/bin/uvicorn" ai_service.main:app --reload --port 8420
|
||||
exec "$VENV/bin/uvicorn" ai_service.main:app --reload --host "$HOST" --port "$PORT"
|
||||
@@ -84,6 +84,10 @@ class AgentConfig:
|
||||
command_timeout_s: float = 30.0
|
||||
backoff_base_s: float = 0.25
|
||||
backoff_max_s: float = 8.0
|
||||
#: Spool bound (G-14): explicit cap where none existed. Worst case
|
||||
#: ~64KB/line (diff cap) * SPOOL_MAX_LINES must stay well under the
|
||||
#: G-2 512MB workdir sweep: 4096 * 64KB = 256MB (half the budget).
|
||||
spool_max_lines: int = 4096
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for name in ("learner_id", "task_id", "ingest_url", "sandbox_id"):
|
||||
@@ -151,6 +155,20 @@ class Spool:
|
||||
os.replace(tmp, self._path)
|
||||
|
||||
|
||||
def _line_seq(line: str) -> int | None:
|
||||
"""Best-effort seq extraction from a spool line (None when unparseable).
|
||||
|
||||
The event's seq is a top-level wire field (`_next_event`). Used only for
|
||||
ack trimming; an unparseable line is retained (never dropped by the ack
|
||||
path — the overflow bound is the only dropper).
|
||||
"""
|
||||
try:
|
||||
seq = json.loads(line).get("seq")
|
||||
return seq if isinstance(seq, int) else None
|
||||
except (ValueError, AttributeError):
|
||||
return None
|
||||
|
||||
|
||||
# --------------------------------------------------------------- websocket codec
|
||||
|
||||
|
||||
@@ -337,13 +355,23 @@ class Agent:
|
||||
self._spool = Spool(config.spool_path)
|
||||
self._pending: deque[str] = deque()
|
||||
self._seq = 0
|
||||
self._emit_lock = threading.Lock() # serializes seq + spool + flush
|
||||
# RLock (D-3 verifier fix): emit() holds this across sends; a send
|
||||
# failure drops the conn, and _drop_conn -> replay_margin re-enters
|
||||
# the same lock. A plain Lock deadlocked the emitting thread there.
|
||||
self._emit_lock = threading.RLock() # serializes seq + spool + flush
|
||||
self._conn_lock = threading.Lock() # guards _conn swaps
|
||||
self._conn: WsConnection | None = None
|
||||
self._last_sent: str | None = None # one-line replay margin, see below
|
||||
self._stop = threading.Event()
|
||||
self._threads: list[threading.Thread] = []
|
||||
self._baseline: dict[str, tuple[int, int, str | None]] = {}
|
||||
# Spool bound (G-14): explicit cap where none existed. Worst case
|
||||
# ~64KB/line (diff cap) x SPOOL_MAX_LINES must stay well under the
|
||||
# G-2 512MB workdir sweep; overflow drops OLDEST lines with a
|
||||
# counter — by-design gap creation, so the trace goes ungradable
|
||||
# (G-4) instead of silently truncated-but-gradable.
|
||||
self._dropped_overflow = 0
|
||||
self._enforce_spool_bound_locked()
|
||||
self._resume_from_spool()
|
||||
|
||||
# -- durability ------------------------------------------------------
|
||||
@@ -400,9 +428,30 @@ class Agent:
|
||||
line = json.dumps(self._next_event(kind, payload))
|
||||
self._spool.append(line) # durable BEFORE any send attempt
|
||||
self._pending.append(line)
|
||||
self._enforce_spool_bound_locked()
|
||||
self._flush_locked()
|
||||
return json.loads(line)
|
||||
|
||||
def _enforce_spool_bound_locked(self) -> None:
|
||||
"""Drop OLDEST spooled lines past `spool_max_lines` (G-14).
|
||||
|
||||
Overflow is by-design gap creation: the dropped seqs become
|
||||
permanent gaps server-side, the gap path flags the trace, and the
|
||||
grader refuses it (G-4) — never a silently-truncated-but-gradable
|
||||
trace. Caller must hold `_emit_lock`. `_dropped_overflow` is the
|
||||
observable counter (surfaced in the final stop status event).
|
||||
"""
|
||||
lines = self._spool.read_all()
|
||||
overflow = len(lines) - self.config.spool_max_lines
|
||||
if overflow <= 0:
|
||||
return
|
||||
self._dropped_overflow += overflow
|
||||
self._spool.rewrite(lines[overflow:])
|
||||
# Pending may reference dropped lines; they replay as no-ops (server
|
||||
# dedup) but trimming them keeps the replay window honest.
|
||||
dropped = set(lines[:overflow])
|
||||
self._pending = deque(ln for ln in self._pending if ln not in dropped)
|
||||
|
||||
def _flush_locked(self) -> None:
|
||||
conn = self._current_conn()
|
||||
while self._pending and conn is not None:
|
||||
@@ -413,24 +462,57 @@ class Agent:
|
||||
self._drop_conn()
|
||||
return
|
||||
self._pending.popleft()
|
||||
self._last_sent = line # kept until a later send proves delivery
|
||||
if not self._pending and self._last_sent is not None:
|
||||
# Compact, but retain the most recently sent line: a send into a
|
||||
# silently-dead socket "succeeds" once at TCP level, so the last
|
||||
# line is only confirmed-sent once a later write works. Retention
|
||||
# is cheap; the server dedups on (learner, task, seq).
|
||||
self._spool.rewrite([self._last_sent])
|
||||
self._last_sent = line
|
||||
# D-045/D-1 (verifier fix): the spool is NEVER compacted below the
|
||||
# unacked window. A send into a silently-dead socket "succeeds" at
|
||||
# TCP level — those lines may be lost in flight — so they stay in
|
||||
# the spool until the server's seq_ack proves durable storage
|
||||
# (trim_to_ack is the ONLY spool shrinker besides the overflow
|
||||
# bound). Replays are harmless: the server dedups on
|
||||
# (learner, task, seq).
|
||||
|
||||
def replay_margin(self) -> None:
|
||||
"""Requeue the last-sent line after a detected disconnect."""
|
||||
"""Requeue every unacked spooled line after a detected disconnect.
|
||||
|
||||
D-045/D-1 (verifier fix): the pre-ack one-line margin could not cover
|
||||
a multi-frame in-flight window — a burst accepted by a dying socket
|
||||
popped N lines from pending while the spool had been compacted to the
|
||||
last one, permanently losing lines 1..N-1. The spool now retains
|
||||
everything unacked, so replay requeues the full unacked window;
|
||||
server-side dedup absorbs the duplicates.
|
||||
"""
|
||||
with self._emit_lock:
|
||||
if self._last_sent is not None and (
|
||||
not self._pending or self._pending[0] != self._last_sent
|
||||
):
|
||||
self._pending.appendleft(self._last_sent)
|
||||
self._spool.rewrite(list(self._pending))
|
||||
if self._pending:
|
||||
return # mid-flush caller holds the pending queue intact
|
||||
self._pending = deque(self._spool.read_all())
|
||||
self._last_sent = None
|
||||
|
||||
def trim_to_ack(self, ack_seq: int) -> None:
|
||||
"""Drop every spooled/pending line with seq <= ack_seq (D-045).
|
||||
|
||||
Advisory server hint: `seq_ack` carries the durable latest_seq, so
|
||||
everything up to and including it is stored server-side and dedup
|
||||
absorbs nothing on replay. Runs under `_emit_lock` — ordered against
|
||||
concurrent emit()/flush, and the rewrite is atomic (Spool.rewrite).
|
||||
Lines without a parseable seq are retained; the overflow bound is
|
||||
the only dropper of unparseable lines.
|
||||
"""
|
||||
with self._emit_lock:
|
||||
spooled = self._spool.read_all()
|
||||
kept_pending = [
|
||||
ln for ln in self._pending if (s := _line_seq(ln)) is None or s > ack_seq
|
||||
]
|
||||
kept_spooled = [
|
||||
ln for ln in spooled if (s := _line_seq(ln)) is None or s > ack_seq
|
||||
]
|
||||
if len(kept_pending) != len(self._pending) or len(kept_spooled) != len(spooled):
|
||||
self._pending = deque(kept_pending)
|
||||
self._spool.rewrite(kept_spooled)
|
||||
if self._last_sent is not None:
|
||||
s = _line_seq(self._last_sent)
|
||||
if s is not None and s <= ack_seq:
|
||||
self._last_sent = None
|
||||
|
||||
# -- connection supervision ------------------------------------------
|
||||
def _current_conn(self) -> WsConnection | None:
|
||||
with self._conn_lock:
|
||||
@@ -491,10 +573,33 @@ class Agent:
|
||||
continue
|
||||
if frame is None:
|
||||
continue
|
||||
opcode, _payload = frame
|
||||
opcode, payload = frame
|
||||
if opcode == 0x1: # server text frame — parse advisory envelopes
|
||||
self._handle_server_text(payload)
|
||||
continue
|
||||
if opcode == 0x8: # server close frame
|
||||
self._drop_conn()
|
||||
|
||||
def _handle_server_text(self, payload: bytes) -> None:
|
||||
"""Consume server->agent envelopes (advisory; never fatal).
|
||||
|
||||
`seq_ack` (D-045): the server's durable latest_seq — trims the
|
||||
spool/pending to `seq > ack`, bounding the replay margin to the
|
||||
in-flight window (REQ-5-007). Unknown/malformed frames are ignored:
|
||||
acks are hints; gap detection and flood semantics stay authoritative
|
||||
server-side.
|
||||
"""
|
||||
try:
|
||||
envelope = json.loads(payload.decode("utf-8"))
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
return
|
||||
if not isinstance(envelope, dict):
|
||||
return
|
||||
if envelope.get("type") == "seq_ack":
|
||||
ack_seq = envelope.get("seq")
|
||||
if isinstance(ack_seq, int) and ack_seq >= 0:
|
||||
self.trim_to_ack(ack_seq)
|
||||
|
||||
# -- workspace watcher ------------------------------------------------
|
||||
def _snapshot_workspace(self) -> dict[str, tuple[int, int, str | None]]:
|
||||
"""Map rel path -> (mtime_ns, size, text-or-None-if-too-large)."""
|
||||
@@ -641,7 +746,14 @@ class Agent:
|
||||
if self._stop.is_set():
|
||||
return
|
||||
try:
|
||||
self.emit("activity", {"state": "stopped", "spooled": len(self._pending)})
|
||||
self.emit(
|
||||
"activity",
|
||||
{
|
||||
"state": "stopped",
|
||||
"spooled": len(self._pending),
|
||||
"dropped_overflow": self._dropped_overflow,
|
||||
},
|
||||
)
|
||||
finally:
|
||||
self._stop.set()
|
||||
self._drop_conn()
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""CORS policy tests (A-008, P7 review regression).
|
||||
"""CORS policy tests (A-008, D-038 network mode).
|
||||
|
||||
v0.3 initially shipped `allow_methods` WITHOUT "PUT" while the learner
|
||||
build surface writes workspace files with PUT (engine-client writeFile) —
|
||||
@@ -6,11 +6,11 @@ every cross-origin Save failed preflight. These tests pin the policy so a
|
||||
future method-list edit fails loudly instead of silently breaking the
|
||||
headline flow.
|
||||
|
||||
Two-layer check:
|
||||
- preflight (OPTIONS + Access-Control-Request-Method) for every method the
|
||||
web client actually uses: GET/POST/PUT/DELETE;
|
||||
- actual cross-origin request echoes the localhost dev origin.
|
||||
Disallowed origins must NOT be granted (localhost-only, no credentials).
|
||||
v0.3.5 network mode (D-038): the default AI_CORS_ORIGINS='*' admits any
|
||||
origin (safe ONLY because credentials are never enabled); an explicit list
|
||||
restricts. Both modes are pinned here:
|
||||
- wildcard: remote origin gets the grant; credentials still never sent;
|
||||
- explicit: unlisted origins get no grant.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -18,9 +18,14 @@ from __future__ import annotations
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
ALLOWED_ORIGIN = "http://localhost:3000"
|
||||
REMOTE_ORIGIN = "http://nextcraft-1:3000"
|
||||
ALL_CLIENT_METHODS = ("GET", "POST", "PUT", "DELETE")
|
||||
|
||||
|
||||
def _allow_origin(resp) -> str | None:
|
||||
return resp.headers.get("access-control-allow-origin")
|
||||
|
||||
|
||||
def test_preflight_allows_every_method_the_web_client_uses(client: TestClient) -> None:
|
||||
for method in ALL_CLIENT_METHODS:
|
||||
resp = client.options(
|
||||
@@ -31,7 +36,7 @@ def test_preflight_allows_every_method_the_web_client_uses(client: TestClient) -
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 200, f"preflight {method} failed: {resp.status_code}"
|
||||
assert resp.headers["access-control-allow-origin"] == ALLOWED_ORIGIN
|
||||
assert _allow_origin(resp) in ("*", ALLOWED_ORIGIN)
|
||||
allowed = resp.headers["access-control-allow-methods"].split(", ")
|
||||
assert method in allowed, f"{method} missing from CORS methods: {allowed}"
|
||||
|
||||
@@ -39,13 +44,30 @@ def test_preflight_allows_every_method_the_web_client_uses(client: TestClient) -
|
||||
def test_cross_origin_get_echoes_allow_origin(client: TestClient) -> None:
|
||||
resp = client.get("/v1/sandboxes", headers={"Origin": ALLOWED_ORIGIN})
|
||||
assert resp.status_code == 200
|
||||
assert resp.headers.get("access-control-allow-origin") == ALLOWED_ORIGIN
|
||||
assert _allow_origin(resp) in ("*", ALLOWED_ORIGIN)
|
||||
|
||||
|
||||
def test_unknown_origin_gets_no_cors_grant(client: TestClient) -> None:
|
||||
resp = client.get("/v1/sandboxes", headers={"Origin": "https://evil.example"})
|
||||
assert resp.status_code == 200 # non-CORS requests still serve
|
||||
assert resp.headers.get("access-control-allow-origin") is None
|
||||
def test_wildcard_mode_grants_remote_origins(client: TestClient) -> None:
|
||||
"""D-038 default: '*' grants any origin — remote browsers work zero-config."""
|
||||
resp = client.get("/v1/sandboxes", headers={"Origin": REMOTE_ORIGIN})
|
||||
assert resp.status_code == 200
|
||||
assert _allow_origin(resp) in ("*", REMOTE_ORIGIN)
|
||||
|
||||
|
||||
def test_explicit_list_mode_denies_unlisted_origins(
|
||||
settings, monkeypatch, tmp_path
|
||||
) -> None:
|
||||
"""Explicit AI_CORS_ORIGINS restricts to the listed origins only."""
|
||||
from fastapi.testclient import TestClient as TC
|
||||
|
||||
from ai_service.main import create_app
|
||||
|
||||
restricted = settings.model_copy(update={"cors_origins": "http://localhost:3000"})
|
||||
app = create_app(restricted)
|
||||
with TC(app) as c:
|
||||
resp = c.get("/v1/sandboxes", headers={"Origin": "https://evil.example"})
|
||||
assert resp.status_code == 200 # non-CORS requests still serve
|
||||
assert resp.headers.get("access-control-allow-origin") is None
|
||||
|
||||
|
||||
def test_credentials_never_allowed(client: TestClient) -> None:
|
||||
|
||||
@@ -48,7 +48,19 @@ class ScriptedLLM(MockProvider):
|
||||
|
||||
@pytest.fixture()
|
||||
def app(tmp_path: Path):
|
||||
application = create_app(Settings(provider="mock", voice_provider="mock"))
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
voice_provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
application = create_app(settings)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
application.state.identity_store = identity_store # G-9
|
||||
llm = ScriptedLLM()
|
||||
application.state.provider = llm
|
||||
application.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
@@ -99,7 +111,20 @@ class TestBrowserFallback:
|
||||
def test_browser_mode_serves_browser_descriptor(self, tmp_path: Path) -> None:
|
||||
"""Must-Have #6: AI_VOICE_PROVIDER=browser → start returns the
|
||||
browser-native SR/TTS fallback descriptor (D-030), not 'mock'."""
|
||||
application = create_app(Settings(provider="mock", voice_provider="browser"))
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
application = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
voice_provider="browser",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity-b.db")
|
||||
seed_verified_identity(identity_store)
|
||||
application.state.identity_store = identity_store # G-9
|
||||
application.state.provider = ScriptedLLM()
|
||||
application.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
application.state.variant_store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
@@ -232,3 +257,107 @@ class TestFinishAndGet:
|
||||
def test_unknown_defense_404_on_all(self, client) -> None:
|
||||
assert client.post("/v1/defense/dfn-nope/finish").status_code == 404
|
||||
assert client.get("/v1/defense/dfn-nope").status_code == 404
|
||||
|
||||
|
||||
class TestServerVoiceRouteFixes:
|
||||
"""MH-2c (D-041/G-16/G-12): codec-strip, size guard, format-aware TTS."""
|
||||
|
||||
def test_webm_codec_params_stripped_for_provider(self, client, app) -> None:
|
||||
"""MediaRecorder sends 'audio/webm;codecs=opus' — the provider must
|
||||
see the bare 'webm' (D-041), else a real STT endpoint 400s."""
|
||||
received_fmts: list[str] = []
|
||||
|
||||
class ProbeVoice(MockVoiceProvider):
|
||||
async def transcribe(self, audio: bytes, fmt: str):
|
||||
received_fmts.append(fmt)
|
||||
return await super().transcribe(audio, fmt)
|
||||
|
||||
app.state.voice_provider = ProbeVoice(["clean fmt seen"])
|
||||
defense_id = _start(client)["defense_id"]
|
||||
resp = client.post(
|
||||
f"/v1/defense/{defense_id}/answer",
|
||||
files={"audio": ("answer.webm", b"\x1a\x45\xa3\xdf", "audio/webm;codecs=opus")},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert received_fmts == ["webm"], (
|
||||
f"codec params leaked to the provider: {received_fmts}"
|
||||
)
|
||||
assert resp.json()["question"]
|
||||
|
||||
def test_oversize_audio_413_before_provider_call(self, client, app) -> None:
|
||||
"""G-12/D-041: the guard fires before any provider call — the client
|
||||
renders an honest re-record prompt."""
|
||||
called = {"n": 0}
|
||||
|
||||
class ProbeVoice(MockVoiceProvider):
|
||||
async def transcribe(self, audio: bytes, fmt: str):
|
||||
called["n"] += 1
|
||||
return await super().transcribe(audio, fmt)
|
||||
|
||||
app.state.voice_provider = ProbeVoice(["x"])
|
||||
defense_id = _start(client)["defense_id"]
|
||||
settings = Settings()
|
||||
too_big = b"\x00" * (settings.voice_max_audio_mb * 1024 * 1024 + 1)
|
||||
resp = client.post(
|
||||
f"/v1/defense/{defense_id}/answer",
|
||||
files={"audio": ("answer.webm", too_big, "audio/webm")},
|
||||
)
|
||||
assert resp.status_code == 413
|
||||
assert "re-record" in resp.json()["detail"]
|
||||
assert called["n"] == 0, "provider must not be called for oversize audio"
|
||||
|
||||
def test_tts_media_type_maps_from_settings_enum(self, tmp_path: Path) -> None:
|
||||
"""G-16: media_type follows voice_tts_format (was hardcoded wav)."""
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
application = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
voice_provider="mock",
|
||||
voice_tts_format="opus",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity-opus.db")
|
||||
seed_verified_identity(identity_store)
|
||||
application.state.identity_store = identity_store # G-9
|
||||
application.state.provider = ScriptedLLM()
|
||||
application.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
application.state.variant_store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
application.state.grade_store = SQLiteGradeStore(db_path=tmp_path / "g.db")
|
||||
application.state.trace_integrity = TraceIntegrityMap()
|
||||
application.state.defense_store = SQLiteDefenseStore(db_path=tmp_path / "d.db")
|
||||
application.state.examiner_agent = ExaminerAgent(
|
||||
application.state.provider,
|
||||
Settings(provider="mock", voice_provider="mock"),
|
||||
)
|
||||
with TestClient(application) as c:
|
||||
defense_id = _start(c)["defense_id"]
|
||||
resp = c.get(f"/v1/defense/{defense_id}/audio/0")
|
||||
assert resp.status_code == 200
|
||||
assert resp.headers["content-type"].startswith("audio/opus"), (
|
||||
resp.headers["content-type"]
|
||||
)
|
||||
|
||||
def test_stt_provider_failure_is_502_never_500(self, client, app) -> None:
|
||||
"""Cross-phase P0 regression (final review): a provider failure on
|
||||
the audio-answer path (real endpoint outage — or the DEFAULT mock
|
||||
provider's unscripted queue, which every default-configured
|
||||
deployment hits on its first audio answer) must surface as an
|
||||
honest 502 per the assessment/proctor house pattern — never an
|
||||
unhandled 500. The transcript stays unaffected (typed answers
|
||||
still work)."""
|
||||
# Unscripted mock = exactly what create_app wires on default settings.
|
||||
app.state.voice_provider = MockVoiceProvider()
|
||||
defense_id = _start(client)["defense_id"]
|
||||
resp = client.post(
|
||||
f"/v1/defense/{defense_id}/answer",
|
||||
files={"audio": ("answer.webm", b"\x1a\x45\xa3\xdf" * 64, "audio/webm")},
|
||||
)
|
||||
assert resp.status_code == 502, resp.text
|
||||
assert "transcription failed" in resp.json()["detail"]
|
||||
# the defense is still alive for typed answers (no poisoned state)
|
||||
typed = client.post(f"/v1/defense/{defense_id}/answer", data={"text": "typed"})
|
||||
assert typed.status_code == 200, typed.text
|
||||
|
||||
@@ -111,8 +111,19 @@ async def test_full_credential_flow(tmp_path: Path) -> None:
|
||||
import httpx
|
||||
|
||||
port = _free_port()
|
||||
settings = Settings(provider="mock", voice_provider="mock", port=port)
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from tests.conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
voice_provider="mock",
|
||||
port=port,
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
app = create_app(settings)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
app.state.identity_store = identity_store # G-9
|
||||
app.state.provider = FlowLLM()
|
||||
app.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
app.state.grade_store = SQLiteGradeStore(db_path=tmp_path / "g.db")
|
||||
|
||||
@@ -43,9 +43,23 @@ BASE_SETTINGS: dict = {
|
||||
"sandbox_max_concurrent": 5,
|
||||
"sandbox_max_per_learner": 1,
|
||||
"sandbox_creates_per_min": 10,
|
||||
# G-9: allowlist widened to the suite roster (identity records seeded
|
||||
# per-app below); the gate tests live in test_identity.py.
|
||||
"learner_allowlist": __import__("tests.conftest", fromlist=["SUITE_LEARNERS"]).SUITE_LEARNERS,
|
||||
}
|
||||
|
||||
|
||||
def _seed_identity(app, tmp_path: Path) -> None:
|
||||
"""G-9: every ad-hoc app in this module gets the suite's verified
|
||||
identity store (allowlist widened via BASE_SETTINGS)."""
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from tests.conftest import seed_verified_identity
|
||||
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / f"identity-{id(app):x}.db")
|
||||
seed_verified_identity(store)
|
||||
app.state.identity_store = store
|
||||
|
||||
|
||||
class StubBackend:
|
||||
"""Structural SandboxBackend: lays out the workdir, spawns nothing.
|
||||
|
||||
@@ -103,6 +117,7 @@ def client(
|
||||
monkeypatch.setenv("AI_SANDBOX_CREATES_PER_MIN", "150") # shared-window headroom
|
||||
settings = Settings(**{**BASE_SETTINGS, "sandbox_dir": tmp_path / "sandboxes"})
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
@@ -190,6 +205,7 @@ def test_pool_full_returns_503(tmp_path: Path, stub_backend: StubBackend) -> Non
|
||||
sandbox_creates_per_min=150,
|
||||
)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
with TestClient(app) as client:
|
||||
ids = [
|
||||
@@ -227,6 +243,7 @@ def test_second_active_sandbox_for_same_learner_429(
|
||||
sandbox_creates_per_min=150,
|
||||
)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
with TestClient(app) as client:
|
||||
first = client.post("/v1/sandboxes", json={"learner_id": PILOT})
|
||||
@@ -252,6 +269,7 @@ def test_burst_over_global_create_rate_429(tmp_path: Path, stub_backend: StubBac
|
||||
sandbox_creates_per_min=3,
|
||||
)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
app.state.sandbox_manager = manager
|
||||
with TestClient(app) as client:
|
||||
@@ -309,6 +327,7 @@ def test_lifespan_start_and_shutdown_destroy(
|
||||
)
|
||||
manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = manager
|
||||
with TestClient(app) as client:
|
||||
assert not orphan_root.exists() # startup reaper ran during lifespan boot
|
||||
@@ -325,6 +344,7 @@ def test_lifespan_constructs_real_manager_when_not_overridden(tmp_path: Path) ->
|
||||
"""No override → the lifespan builds the production UnshareBackend manager."""
|
||||
settings = Settings(provider="mock", sandbox_dir=tmp_path / "sandboxes")
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
with TestClient(app):
|
||||
manager = app.state.sandbox_manager
|
||||
assert isinstance(manager, SandboxManager)
|
||||
@@ -347,6 +367,7 @@ def test_real_backend_create_path_runs(tmp_path: Path) -> None:
|
||||
sandbox_creates_per_min=150,
|
||||
)
|
||||
app = create_app(settings) # no override → lifespan wires UnshareBackend
|
||||
_seed_identity(app, tmp_path)
|
||||
try:
|
||||
with TestClient(app) as client:
|
||||
created = client.post("/v1/sandboxes", json={"learner_id": PILOT})
|
||||
|
||||
@@ -84,7 +84,10 @@ def _recv_status(ws) -> dict:
|
||||
continue
|
||||
if msg["type"] == "websocket.close":
|
||||
raise WebSocketDisconnect(msg.get("code", 1000), msg.get("reason", ""))
|
||||
return json.loads(msg["text"])
|
||||
frame = json.loads(msg["text"])
|
||||
if frame.get("type") == "seq_ack": # D-045 advisory ack per append
|
||||
continue
|
||||
return frame
|
||||
|
||||
|
||||
def _ingest_url(learner_id: str = LEARNER, task_id: str = TASK, sandbox_id: str = "") -> str:
|
||||
@@ -123,6 +126,34 @@ def client(app) -> Iterator[TestClient]:
|
||||
yield c
|
||||
|
||||
|
||||
# -- seq-ack (D-045, REQ-5-007) ------------------------------------------------
|
||||
|
||||
|
||||
def test_each_append_emits_seq_ack_with_durable_latest(
|
||||
client: TestClient, store: SQLiteTraceStore
|
||||
) -> None:
|
||||
"""MH-1a: every successful append acks the post-append durable latest_seq
|
||||
(on dedup'd replays too — a-6)."""
|
||||
with client.websocket_connect(_ingest_url()) as ws:
|
||||
ws.send_text(_frame(0))
|
||||
ws.send_text(_frame(1))
|
||||
ws.send_text(_frame(2))
|
||||
ws.send_text(_frame(2)) # replay → dedup, still acked (a-6)
|
||||
acks: list[int] = []
|
||||
while len(acks) < 4:
|
||||
msg = ws.receive()
|
||||
if msg.get("bytes") is not None:
|
||||
continue
|
||||
if msg["type"] == "websocket.close":
|
||||
raise WebSocketDisconnect(msg.get("code", 1000))
|
||||
frame = json.loads(msg["text"])
|
||||
if frame.get("type") == "seq_ack":
|
||||
assert isinstance(frame["seq"], int)
|
||||
acks.append(frame["seq"])
|
||||
assert acks == [0, 1, 2, 2], f"expected per-append acks incl. dedup, got {acks}"
|
||||
assert store.latest_seq(LEARNER, TASK) == 2
|
||||
|
||||
|
||||
# -- happy path: ordered trace retrieval --------------------------------------
|
||||
|
||||
|
||||
@@ -394,17 +425,33 @@ def test_missing_identity_query_params_rejected_at_handshake(
|
||||
assert excinfo.value.code == 1008
|
||||
|
||||
|
||||
def test_browser_origin_not_allowed_for_ingest(client: TestClient) -> None:
|
||||
"""CORS middleware does not cover WS upgrades (P7): a page loaded in the
|
||||
learner's browser (any non-localhost Origin) must not be able to open
|
||||
the ingest socket and poison/flood the trace. The stdlib capture agent
|
||||
sends no Origin and is unaffected (see the no-origin test below)."""
|
||||
with pytest.raises(WebSocketDisconnect) as excinfo:
|
||||
with client.websocket_connect(
|
||||
_ingest_url(), headers={"Origin": "https://evil.example"}
|
||||
):
|
||||
pass
|
||||
assert excinfo.value.code == 1008
|
||||
def test_browser_origin_rejected_in_explicit_list_mode(
|
||||
settings, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""D-038: with an explicit AI_CORS_ORIGINS list, a page loaded in the
|
||||
learner's browser (unlisted Origin) must not be able to open the ingest
|
||||
socket and poison/flood the trace. The stdlib capture agent sends no
|
||||
Origin and is unaffected (see the no-origin test below)."""
|
||||
from fastapi.testclient import TestClient as TC
|
||||
|
||||
restricted = settings.model_copy(update={"cors_origins": "http://localhost:3000"})
|
||||
app = create_app(restricted)
|
||||
with TC(app) as c:
|
||||
with pytest.raises(WebSocketDisconnect) as excinfo:
|
||||
with c.websocket_connect(
|
||||
_ingest_url(), headers={"Origin": "https://evil.example"}
|
||||
):
|
||||
pass
|
||||
assert excinfo.value.code == 1008
|
||||
|
||||
|
||||
def test_wildcard_mode_admits_any_browser_origin(client: TestClient) -> None:
|
||||
"""D-038 default ('*'): remote-browser origins open the ingest socket —
|
||||
the remote build surface streams telemetry from the learner's browser."""
|
||||
with client.websocket_connect(
|
||||
_ingest_url(), headers={"Origin": "http://nextcraft-1:3000"}
|
||||
) as ws:
|
||||
ws.send_text(_frame(0))
|
||||
|
||||
|
||||
def test_dev_origin_and_no_origin_both_allowed(client: TestClient) -> None:
|
||||
|
||||
@@ -45,12 +45,15 @@ import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.main import create_app
|
||||
from ai_service.variants.generator import VariantGenerator
|
||||
from ai_service.variants.store import SQLiteVariantStore
|
||||
from ai_service.variants.templates import get_template
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
LEARNER_A = "variant-learner-a"
|
||||
LEARNER_B = "variant-learner-b"
|
||||
TEMPLATE = "tpl-llm-judge"
|
||||
@@ -104,8 +107,12 @@ def _make_client(
|
||||
provider="mock",
|
||||
db_path=tmp_path / "variant-test.db",
|
||||
sandbox_dir=tmp_path / "sandboxes",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
app = create_app(settings)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
app.state.identity_store = identity_store # G-9
|
||||
app.state.variant_store = store
|
||||
if getattr(app.state, "variant_generator", None) is None:
|
||||
app.state.variant_generator = VariantGenerator(
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Single-port static-UI mount tests (v0.3.6).
|
||||
|
||||
When AI_WEB_STATIC_DIR is set, the app serves the exported Next.js app
|
||||
(apps/web/out) at / — same origin as the API, so the site answers on one
|
||||
port behind HAProxy (the only reachable port). Pins:
|
||||
|
||||
- default (unset): NO mount — / stays the plain FastAPI 404 (dev/tests);
|
||||
- set: / serves index.html, API routes win over static, unknown paths
|
||||
get the export's 404.html with status 404;
|
||||
- set-but-missing dir: startup fails loudly (misconfig, never a silent
|
||||
mount-less app).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.main import create_app
|
||||
|
||||
|
||||
def _export_dir(tmp_path: Path) -> Path:
|
||||
"""Minimal fake of a Next.js static export."""
|
||||
out = tmp_path / "out"
|
||||
out.mkdir()
|
||||
(out / "index.html").write_text("<html>nextcraft home</html>")
|
||||
(out / "404.html").write_text("<html>not found page</html>")
|
||||
nested = out / "dashboard"
|
||||
nested.mkdir()
|
||||
(nested / "index.html").write_text("<html>dashboard page</html>")
|
||||
return out
|
||||
|
||||
|
||||
def _settings(tmp_path: Path, web_dir: Path | None) -> Settings:
|
||||
return Settings(
|
||||
provider="mock",
|
||||
model="gemma4:31b",
|
||||
port=8421,
|
||||
db_path=tmp_path / "test.db",
|
||||
sandbox_dir=tmp_path / "sandboxes",
|
||||
web_static_dir=web_dir or Path(""),
|
||||
)
|
||||
|
||||
|
||||
def test_default_no_mount_root_is_api_404(client: TestClient) -> None:
|
||||
"""Default web_static_dir='' → no mount; / is FastAPI's JSON 404."""
|
||||
resp = client.get("/")
|
||||
assert resp.status_code == 404
|
||||
assert resp.json() == {"detail": "Not Found"}
|
||||
|
||||
|
||||
def test_mount_serves_index_and_api_routes_win(tmp_path: Path) -> None:
|
||||
out = _export_dir(tmp_path)
|
||||
app = create_app(_settings(tmp_path, out))
|
||||
with TestClient(app) as c:
|
||||
# / serves the export's index.html
|
||||
resp = c.get("/")
|
||||
assert resp.status_code == 200
|
||||
assert "nextcraft home" in resp.text
|
||||
# API routes take precedence over the static mount
|
||||
assert c.get("/health").status_code == 200
|
||||
# trailingSlash-style directory index resolves
|
||||
assert "dashboard page" in c.get("/dashboard/").text
|
||||
|
||||
|
||||
def test_mount_unknown_path_serves_export_404_page(tmp_path: Path) -> None:
|
||||
out = _export_dir(tmp_path)
|
||||
app = create_app(_settings(tmp_path, out))
|
||||
with TestClient(app) as c:
|
||||
resp = c.get("/definitely-not-a-page/")
|
||||
assert resp.status_code == 404
|
||||
assert "not found page" in resp.text
|
||||
|
||||
|
||||
def test_mount_missing_dir_fails_loudly(tmp_path: Path) -> None:
|
||||
settings = _settings(tmp_path, tmp_path / "does-not-exist")
|
||||
with pytest.raises(RuntimeError, match="AI_WEB_STATIC_DIR"):
|
||||
create_app(settings)
|
||||
@@ -1,6 +1,9 @@
|
||||
"""Test suite — conftest: mock provider only, zero network (enforced)."""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
@@ -9,16 +12,99 @@ from ai_service.config import Settings
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.main import create_app
|
||||
|
||||
# Sandbox workdirs need a filesystem that supports bind mounts from a user
|
||||
# namespace: pytest's default tmpdir (/tmp) is overlayfs here and mount(2)
|
||||
# fails with ENODEV ("special device ... does not exist"). Home is real ext4.
|
||||
_HOME_ROOT = Path.home() / ".nextcraft" / "test-sandboxes"
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def settings() -> Settings:
|
||||
def sandbox_dir(tmp_path: Path) -> Path:
|
||||
"""Per-test sandbox workdir root on a bind-mount-safe filesystem."""
|
||||
_HOME_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
d = Path(tempfile.mkdtemp(prefix=f"nc-test-{tmp_path.name}-", dir=_HOME_ROOT))
|
||||
yield d
|
||||
shutil.rmtree(d, ignore_errors=True)
|
||||
|
||||
|
||||
# G-9 (grill): the v0.5 identity gates land on variants/sandboxes/defense —
|
||||
# routes every API suite exercises. The suite-wide learner roster + a
|
||||
# seeded verified identity keep pre-existing tests green while the gate
|
||||
# tests (test_identity.py) prove the 403/CTA composition on unverified ids.
|
||||
SUITE_LEARNERS = [
|
||||
"pilot-learner",
|
||||
"pilot-learner-2",
|
||||
"api-learner",
|
||||
"defense-learner",
|
||||
"grade-learner",
|
||||
"lab-learner",
|
||||
"p-learner",
|
||||
"variant-learner-a",
|
||||
"variant-learner-b",
|
||||
"learner-001",
|
||||
"lat-learner",
|
||||
"ghost-learner",
|
||||
]
|
||||
|
||||
|
||||
def seed_verified_identity(store, learner_ids=SUITE_LEARNERS) -> None:
|
||||
"""Insert a verified 18+ identity record per learner id (mock-marked).
|
||||
|
||||
A-304: records carry mock=True — the seed never masquerades as
|
||||
production verification.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from ai_service.identity.store import IdentityRecord
|
||||
|
||||
for lid in learner_ids:
|
||||
try:
|
||||
store.insert(
|
||||
IdentityRecord(
|
||||
id=f"seed-{lid}",
|
||||
learner_id=lid,
|
||||
status="verified",
|
||||
provider="mock",
|
||||
verdict={"status": "verified", "age_band": "18+", "mock": True},
|
||||
age_band="18+",
|
||||
submitted_at=datetime.now(UTC),
|
||||
verified_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
except Exception:
|
||||
pass # already seeded (shared store)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def settings(tmp_path: Path) -> Settings:
|
||||
os.environ["AI_PROVIDER"] = "mock"
|
||||
return Settings(provider="mock", model="gemma4:31b", port=8421)
|
||||
# Hermetic stores (v0.3.6): db_path/sandbox_dir default to ~/.nextcraft —
|
||||
# tests must never read or write real state; pin to the pytest tmp dir.
|
||||
return Settings(
|
||||
provider="mock",
|
||||
model="gemma4:31b",
|
||||
port=8421,
|
||||
db_path=tmp_path / "nextcraft-test.db",
|
||||
sandbox_dir=tmp_path / "sandboxes",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def app(settings: Settings):
|
||||
return create_app(settings)
|
||||
def identity_store(tmp_path: Path):
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "identity-test.db")
|
||||
seed_verified_identity(store)
|
||||
yield store
|
||||
store.close()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def app(settings: Settings, identity_store):
|
||||
application = create_app(settings)
|
||||
application.state.identity_store = identity_store # state-injection (G-9)
|
||||
return application
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
|
||||
@@ -0,0 +1,630 @@
|
||||
"""Identity module tests (REQ-5-003/004, D-042/43).
|
||||
|
||||
MH-3a: store contract — insert/poll/latest/verdict provenance, constraints.
|
||||
MH-3b: gate composition — allowlist (403, first) → identity verdict (403 +
|
||||
verify-CTA) → caps; 16-17 school-pass/marketplace-block; under-16 blocked;
|
||||
G-13 submit caps; G-18 honest stub.
|
||||
MH-3c: PII sentinel — raw DOB + document contents appear in NO log record
|
||||
and NO stored raw form (caplog + store inspection).
|
||||
MH-3e: identity flow end-to-end via TestClient against real create_app.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.identity.base import IdentitySubmission
|
||||
from ai_service.identity.mock import MockIdentityProvider, derive_age_band
|
||||
from ai_service.identity.store import IdentityRecord, SQLiteIdentityStore
|
||||
from ai_service.main import create_app
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
ADULT_DOB = "2000-01-01"
|
||||
MINOR_DOB = str(datetime.now(UTC).year - 17) + "-06-01" # 16-17 band
|
||||
UNDER16_DOB = str(datetime.now(UTC).year - 12) + "-06-01" # under-16
|
||||
|
||||
|
||||
def _age_band(dob: str) -> str:
|
||||
return derive_age_band(dob)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def identity_store(tmp_path: Path) -> SQLiteIdentityStore:
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
yield store
|
||||
store.close()
|
||||
|
||||
|
||||
# -- MH-3a: store ---------------------------------------------------------------
|
||||
|
||||
|
||||
class TestIdentityStore:
|
||||
def test_insert_get_roundtrip(self, identity_store) -> None:
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-1",
|
||||
learner_id="learner-x",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
got = identity_store.get("idc-1")
|
||||
assert got is not None and got.learner_id == "learner-x"
|
||||
assert got.status == "pending"
|
||||
assert got.mock is True # A-304 default marker
|
||||
|
||||
def test_latest_for_learner_orders_by_submitted(self, identity_store) -> None:
|
||||
now = datetime.now(UTC)
|
||||
# insert order (oldest→newest by timestamp): idc-1, idc-2, idc-0
|
||||
for i, offset in ((1, 1), (2, 2), (0, 3)):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id=f"idc-{i}",
|
||||
learner_id="learner-y",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=now + timedelta(seconds=offset),
|
||||
)
|
||||
)
|
||||
latest = identity_store.latest_for_learner("learner-y")
|
||||
assert latest is not None and latest.id == "idc-0" # +3s is newest
|
||||
|
||||
def test_insert_duplicate_id_raises(self, identity_store) -> None:
|
||||
"""Insert-only: a SECOND record with the same id raises (a real
|
||||
duplicate is a fresh instance carrying a minted-once id)."""
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-dup",
|
||||
learner_id="learner-z",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
with pytest.raises(IntegrityError):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-dup", # same id, fresh instance
|
||||
learner_id="learner-z",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
|
||||
def test_invalid_status_rejected(self, identity_store) -> None:
|
||||
with pytest.raises(ValueError, match="invalid identity status"):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-bad",
|
||||
learner_id="l",
|
||||
status="banana",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
|
||||
def test_mark_verified_transition(self, identity_store) -> None:
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-v",
|
||||
learner_id="learner-v",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
updated = identity_store.mark_verified(
|
||||
"idc-v", {"status": "verified", "age_band": "18+", "mock": True}, "18+"
|
||||
)
|
||||
assert updated is not None
|
||||
assert updated.status == "verified"
|
||||
assert updated.age_band == "18+"
|
||||
assert updated.mock is True
|
||||
assert identity_store.mark_verified("nope", {}, None) is None
|
||||
|
||||
def test_count_pending(self, identity_store) -> None:
|
||||
for i in range(3):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id=f"idc-p{i}",
|
||||
learner_id="learner-p",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
assert identity_store.count_pending_for_learner("learner-p") == 3
|
||||
|
||||
|
||||
# -- mock provider -----------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMockProvider:
|
||||
def test_adult_verifies_18_plus(self) -> None:
|
||||
provider = MockIdentityProvider()
|
||||
sid = await_(provider.submit(IdentitySubmission(learner_id="l", date_of_birth=ADULT_DOB)))
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "verified"
|
||||
assert verdict.age_band == "18+"
|
||||
assert verdict.mock is True # A-304
|
||||
|
||||
def test_minor_gets_16_17_band(self) -> None:
|
||||
provider = MockIdentityProvider()
|
||||
sid = await_(
|
||||
provider.submit(IdentitySubmission(learner_id="l", date_of_birth=MINOR_DOB))
|
||||
)
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "verified"
|
||||
assert verdict.age_band == "16-17"
|
||||
|
||||
def test_under_16_rejected(self) -> None:
|
||||
provider = MockIdentityProvider()
|
||||
sid = await_(
|
||||
provider.submit(IdentitySubmission(learner_id="l", date_of_birth=UNDER16_DOB))
|
||||
)
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "rejected"
|
||||
assert "16+" in verdict.detail
|
||||
|
||||
def test_scripted_rejection(self) -> None:
|
||||
provider = MockIdentityProvider(reject_learners={"bad-actor"})
|
||||
sid = await_(
|
||||
provider.submit(IdentitySubmission(learner_id="bad-actor", date_of_birth=ADULT_DOB))
|
||||
)
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "rejected"
|
||||
|
||||
|
||||
def await_(coro):
|
||||
import asyncio
|
||||
|
||||
return asyncio.get_event_loop().run_until_complete(coro) if False else asyncio.run(coro)
|
||||
|
||||
|
||||
# -- MH-3b/MH-3c/MH-3e: gates + flow over HTTP ----------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def gated_client(tmp_path: Path, identity_store: SQLiteIdentityStore) -> TestClient:
|
||||
seed_verified_identity(identity_store) # suite roster verified 18+
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
identity_submits_per_min=10,
|
||||
db_path=tmp_path / "gated-app.db",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
|
||||
|
||||
class TestVerificationFlow:
|
||||
def test_submit_status_verify_flow(self, gated_client: TestClient) -> None:
|
||||
resp = gated_client.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "new-learner", "date_of_birth": ADULT_DOB},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["status"] == "pending"
|
||||
assert body["mock"] is True
|
||||
|
||||
status = gated_client.get("/v1/identity/status/new-learner").json()
|
||||
assert status["status"] == "pending"
|
||||
|
||||
verdict = gated_client.post(f"/v1/identity/verify/{body['submission_id']}").json()
|
||||
assert verdict["status"] == "verified"
|
||||
assert verdict["age_band"] == "18+"
|
||||
assert verdict["mock"] is True # A-304 rides the response
|
||||
|
||||
def test_g13_pending_resubmit_409(self, gated_client: TestClient) -> None:
|
||||
first = gated_client.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "pending-learner", "date_of_birth": ADULT_DOB},
|
||||
).json()
|
||||
second = gated_client.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "pending-learner", "date_of_birth": ADULT_DOB},
|
||||
)
|
||||
assert second.status_code == 409
|
||||
detail = second.json()["detail"]
|
||||
assert detail["reason"] == "submission_pending"
|
||||
assert detail["submission_id"] == first["submission_id"]
|
||||
|
||||
def test_g13_rate_cap_429(self, tmp_path: Path) -> None:
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["rl"],
|
||||
identity_submits_per_min=1,
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
with TestClient(app) as c:
|
||||
# Learner submits + verifies (record terminal → pending cap free),
|
||||
# then resubmits within the rate window → 429.
|
||||
sub = c.post(
|
||||
"/v1/identity/submit", json={"learner_id": "rl", "date_of_birth": ADULT_DOB}
|
||||
).json()
|
||||
c.post(f"/v1/identity/verify/{sub['submission_id']}")
|
||||
resp = c.post(
|
||||
"/v1/identity/submit", json={"learner_id": "rl", "date_of_birth": ADULT_DOB}
|
||||
)
|
||||
assert resp.status_code == 429
|
||||
|
||||
def test_resubmit_after_terminal_never_500s(
|
||||
self, tmp_path: Path
|
||||
) -> None:
|
||||
"""Cross-phase P0 regression (final review): the mock provider once
|
||||
minted the submission id from hash((learner_id, dob)) — a resubmit
|
||||
after a TERMINAL verdict (rejected learner retrying, or any
|
||||
re-verification with the same DOB) collided with the insert-only
|
||||
store's PK and 500'd forever. Ids must be unique per submit(); a
|
||||
terminal-then-resubmit (rate cap permitting) is a fresh pending
|
||||
submission, never a duplicate-PK crash."""
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "re-i.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["retry-learner"],
|
||||
identity_submits_per_min=10,
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
with TestClient(app) as c:
|
||||
# terminal REJECTED record first (under-16 path)
|
||||
sub = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "retry-learner", "date_of_birth": "2015-01-01"},
|
||||
).json()
|
||||
v = c.post(f"/v1/identity/verify/{sub['submission_id']}").json()
|
||||
assert v["status"] == "rejected"
|
||||
# same learner + same DOB resubmits: fresh pending, NOT a 500
|
||||
second = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "retry-learner", "date_of_birth": "2015-01-01"},
|
||||
)
|
||||
assert second.status_code == 200, second.text
|
||||
assert second.json()["status"] == "pending"
|
||||
assert second.json()["submission_id"] != sub["submission_id"]
|
||||
# while the second is still PENDING, G-13 caps resubmits at 409
|
||||
# (one active pending per learner) — a policy 4xx, never the
|
||||
# duplicate-PK 500 the deterministic-id bug produced.
|
||||
third = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "retry-learner", "date_of_birth": "2015-01-01"},
|
||||
)
|
||||
assert third.status_code == 409
|
||||
|
||||
def test_pii_sentinel_never_stored_or_logged(
|
||||
self, tmp_path: Path, caplog
|
||||
) -> None:
|
||||
"""MH-3c (A-305): sentinel PII in submissions appears in NO log
|
||||
record and NO stored raw form."""
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "piii.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["pii-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
sentinel_dob = "1999-12-31"
|
||||
sentinel_doc = "SENTINEL-DOC-CONTENTS-XYZZY"
|
||||
with caplog.at_level(logging.DEBUG):
|
||||
with TestClient(app) as c:
|
||||
c.post(
|
||||
"/v1/identity/submit",
|
||||
json={
|
||||
"learner_id": "pii-learner",
|
||||
"date_of_birth": sentinel_dob,
|
||||
"document_refs": [sentinel_doc],
|
||||
},
|
||||
)
|
||||
# every log record + every captured source line
|
||||
logged = " ".join(r.getMessage() for r in caplog.records)
|
||||
assert sentinel_dob not in logged, "raw DOB leaked to logs"
|
||||
assert "1999" not in logged
|
||||
# store inspection: no raw DOB in any stored record
|
||||
from sqlalchemy import text
|
||||
|
||||
with store._engine.connect() as conn: # noqa: SLF001
|
||||
rows = conn.execute(
|
||||
text(
|
||||
"SELECT id, learner_id, status, verdict, age_band, "
|
||||
"document_refs FROM identity_record"
|
||||
)
|
||||
).fetchall()
|
||||
blob = json.dumps([list(map(str, r)) for r in rows])
|
||||
assert sentinel_dob not in blob, "raw DOB persisted"
|
||||
assert sentinel_doc in blob # the REF is stored (opaque handle) — refs are allowed
|
||||
|
||||
|
||||
class TestGateComposition:
|
||||
"""MH-3b: allowlist (first) → identity verdict → caps; band splits."""
|
||||
|
||||
def test_allowlist_403_fires_first(self, gated_client: TestClient) -> None:
|
||||
resp = gated_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "stranger-danger", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert resp.status_code == 403
|
||||
assert "allowlist" in resp.json()["detail"]
|
||||
|
||||
def test_unverified_allowlisted_gets_verify_cta(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
# allowlisted but NEVER identity-verified (not in the seed roster)
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS + ["fresh-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as gated_client:
|
||||
resp = gated_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "fresh-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert resp.status_code == 403
|
||||
detail = resp.json()["detail"]
|
||||
assert detail["reason"] == "identity_verification_required"
|
||||
assert detail["min_age"] == 16
|
||||
assert detail["current_status"] == "none"
|
||||
assert detail["verify_cta"] == "/enroll"
|
||||
|
||||
def test_verified_18_plus_passes_school_gate(self, gated_client: TestClient) -> None:
|
||||
# pilot-learner is seeded verified 18+ (G-9 seed)
|
||||
resp = gated_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "pilot-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
|
||||
def test_16_17_passes_school_but_blocked_marketplace(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
# seed a 16-17 verified learner
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="seed-minor",
|
||||
learner_id="minor-learner",
|
||||
status="verified",
|
||||
provider="mock",
|
||||
verdict={"status": "verified", "age_band": "16-17", "mock": True},
|
||||
age_band="16-17",
|
||||
submitted_at=datetime.now(UTC),
|
||||
verified_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS + ["minor-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
# school gate (16+): passes
|
||||
variants = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "minor-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert variants.status_code == 200, variants.text
|
||||
# marketplace gate (18+ verified): 403 with the age reason
|
||||
apply = c.post(
|
||||
"/v1/marketplace/apply",
|
||||
json={"learner_id": "minor-learner", "job_id": "job-001"},
|
||||
)
|
||||
assert apply.status_code == 403
|
||||
detail = apply.json()["detail"]
|
||||
assert detail["reason"] == "age_gate_18_plus"
|
||||
assert detail["min_age"] == 18
|
||||
|
||||
def test_verified_adult_marketplace_stub_is_honest_501(
|
||||
self, gated_client: TestClient
|
||||
) -> None:
|
||||
"""G-18: the gate passes; the route NEVER fabricates 'applied'."""
|
||||
resp = gated_client.post(
|
||||
"/v1/marketplace/apply",
|
||||
json={"learner_id": "pilot-learner", "job_id": "job-001"},
|
||||
)
|
||||
assert resp.status_code == 501
|
||||
body = resp.json()
|
||||
assert body["stub"] is True
|
||||
assert body["mock"] is True
|
||||
|
||||
def test_mh3e_flow_unverified_then_enrolled(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
"""MH-3e: unverified → 403 verify-CTA → submit+verify → 200."""
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["flow-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
blocked = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "flow-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert blocked.status_code == 403
|
||||
assert blocked.json()["detail"]["verify_cta"] == "/enroll"
|
||||
|
||||
sub = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "flow-learner", "date_of_birth": ADULT_DOB},
|
||||
).json()
|
||||
c.post(f"/v1/identity/verify/{sub['submission_id']}")
|
||||
|
||||
allowed = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "flow-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert allowed.status_code == 200, allowed.text
|
||||
|
||||
|
||||
class TestVerifierHardening:
|
||||
"""D1/D2/D3 (verifier P1/P1/P2): boundary validation, fail-closed gate,
|
||||
band constraints — the exception-path PII leak and the fail-open gate
|
||||
the first verify pass found."""
|
||||
|
||||
def test_d1_malformed_dob_422_at_boundary_never_500(
|
||||
self, tmp_path: Path, caplog
|
||||
) -> None:
|
||||
"""D1: a malformed DOB must die as 422 BEFORE any derivation runs —
|
||||
never a 500 whose traceback echoes the raw value into logs (A-305)
|
||||
nor a poisoned pending record that G-13 turns into a lockout."""
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "d1.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["victim"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
poison = "1975-06-15XX"
|
||||
with caplog.at_level(logging.DEBUG):
|
||||
with TestClient(app) as c:
|
||||
resp = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "victim", "date_of_birth": poison},
|
||||
)
|
||||
assert resp.status_code == 422, "malformed DOB must be 422, not 500"
|
||||
assert poison not in resp.text, "422 must not echo the raw value"
|
||||
# No poisoned pending record: the learner can still submit.
|
||||
assert store.count_pending_for_learner("victim") == 0
|
||||
ok = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "victim", "date_of_birth": ADULT_DOB},
|
||||
)
|
||||
assert ok.status_code == 200
|
||||
logged = " ".join(r.getMessage() for r in caplog.records)
|
||||
assert poison not in logged, "raw DOB must never reach logs"
|
||||
|
||||
def test_d2_gate_fails_closed_on_non_canonical_bands(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
"""D2: None / unknown / under-16 bands can NEVER pass the 18+ gate
|
||||
(nor the school gate for non-canonical values). Non-canonical rows
|
||||
are planted with raw SQL — the store now refuses them (D3), so this
|
||||
simulates the future-vendor / direct-write path the gate must
|
||||
still defend against."""
|
||||
from sqlalchemy import text as sql_text
|
||||
|
||||
for band, expect_school, expect_market in (
|
||||
(None, False, False),
|
||||
("banana", False, False),
|
||||
("under-16", False, False),
|
||||
("16-17", True, False),
|
||||
("18+", True, True),
|
||||
):
|
||||
lid = f"band-{str(band or 'none')}"
|
||||
with identity_store._engine.begin() as conn: # noqa: SLF001
|
||||
conn.execute(
|
||||
sql_text(
|
||||
"INSERT INTO identity_record (id, learner_id, status, "
|
||||
"provider, verdict, age_band, document_refs, "
|
||||
"submitted_at) VALUES (:id, :lid, 'verified', 'mock', "
|
||||
"'{}', :band, '[]', CURRENT_TIMESTAMP)"
|
||||
),
|
||||
{"id": f"raw-{lid}", "lid": lid, "band": band},
|
||||
)
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=[lid],
|
||||
db_path=tmp_path / f"app-{lid}.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
school = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": lid, "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
market = c.post(
|
||||
"/v1/marketplace/apply",
|
||||
json={"learner_id": lid, "job_id": "job-001"},
|
||||
)
|
||||
assert (school.status_code == 200) is expect_school, (
|
||||
f"band={band!r} school gate: {school.status_code}"
|
||||
)
|
||||
assert (market.status_code == 501) is expect_market, (
|
||||
f"band={band!r} marketplace gate: {market.status_code}"
|
||||
)
|
||||
|
||||
def test_d3_store_rejects_non_canonical_bands(self, identity_store) -> None:
|
||||
"""D3: the store refuses to create/mark non-canonical bands."""
|
||||
with pytest.raises(ValueError, match="invalid age_band"):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-banana",
|
||||
learner_id="l",
|
||||
status="verified",
|
||||
provider="mock",
|
||||
age_band="banana",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-vm",
|
||||
learner_id="l2",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
with pytest.raises(ValueError, match="invalid age_band"):
|
||||
identity_store.mark_verified("idc-vm", {"status": "verified"}, "banana")
|
||||
|
||||
def test_d4_latest_tiebreaks_deterministically(self, identity_store) -> None:
|
||||
"""D4: identical-microsecond records resolve to the LAST inserted."""
|
||||
same = datetime.now(UTC)
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="t-first",
|
||||
learner_id="tie-learner",
|
||||
status="verified",
|
||||
provider="mock",
|
||||
age_band="18+",
|
||||
submitted_at=same,
|
||||
verified_at=same,
|
||||
)
|
||||
)
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="t-second",
|
||||
learner_id="tie-learner",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=same,
|
||||
)
|
||||
)
|
||||
latest = identity_store.latest_for_learner("tie-learner")
|
||||
assert latest is not None and latest.id == "t-second"
|
||||
@@ -94,6 +94,22 @@ def _read_exact(conn: socket.socket, n: int) -> bytes:
|
||||
return bytes(buf)
|
||||
|
||||
|
||||
def _server_send_text(conn: socket.socket, payload: bytes) -> None:
|
||||
"""Send one unmasked server text frame (client frames are masked; server
|
||||
frames are not, per RFC 6455)."""
|
||||
header = bytearray([0x81]) # FIN + text opcode
|
||||
n = len(payload)
|
||||
if n < 126:
|
||||
header.append(n)
|
||||
elif n < 65536:
|
||||
header.append(126)
|
||||
header += struct.pack("!H", n)
|
||||
else:
|
||||
header.append(127)
|
||||
header += struct.pack("!Q", n)
|
||||
conn.sendall(bytes(header) + payload)
|
||||
|
||||
|
||||
def _server_read_frame(conn: socket.socket) -> tuple[int, bytes]:
|
||||
"""Read one client frame (client frames are always masked per RFC 6455)."""
|
||||
b0, b1 = _read_exact(conn, 2)
|
||||
@@ -630,3 +646,217 @@ class TestStdlibOnly:
|
||||
)
|
||||
# sanity: the scan really saw the agent's core imports
|
||||
assert {"socket", "json", "threading", "ssl", "subprocess"} <= imported
|
||||
|
||||
|
||||
class TestSeqAckTrim:
|
||||
"""D-045 (REQ-5-007): agent trims spool/pending to seq > ack on seq_ack."""
|
||||
|
||||
def test_trim_to_ack_drops_acked_spooled_and_pending(
|
||||
self, tmp_path: Path, fake_server: FakeWSServer
|
||||
) -> None:
|
||||
test_agent = _make_agent(tmp_path, fake_server.url)
|
||||
try:
|
||||
# Not connected: everything stays spooled + pending.
|
||||
for i in range(5):
|
||||
test_agent.emit("activity", {"i": i})
|
||||
assert len(test_agent._spool.read_all()) == 5 # noqa: SLF001
|
||||
assert len(test_agent._pending) == 5 # noqa: SLF001
|
||||
|
||||
test_agent.trim_to_ack(2) # server durably holds seqs 0..2
|
||||
|
||||
spool_seqs = [
|
||||
agent._line_seq(ln) # noqa: SLF001
|
||||
for ln in test_agent._spool.read_all() # noqa: SLF001
|
||||
]
|
||||
pending_seqs = [
|
||||
agent._line_seq(ln) # noqa: SLF001
|
||||
for ln in test_agent._pending # noqa: SLF001
|
||||
]
|
||||
assert all(s is None or s > 2 for s in spool_seqs)
|
||||
assert all(s is None or s > 2 for s in pending_seqs)
|
||||
assert 3 in spool_seqs and 4 in spool_seqs, "unacked lines retained"
|
||||
finally:
|
||||
test_agent.stop()
|
||||
|
||||
def test_trim_clears_last_sent_when_acked(self, tmp_path: Path) -> None:
|
||||
test_agent = _make_agent(tmp_path, "ws://127.0.0.1:1/") # never connects
|
||||
try:
|
||||
test_agent.emit("activity", {"i": 0})
|
||||
# Simulate: line sent (so popped from pending) but ack unknown.
|
||||
line = test_agent._pending[0] # noqa: SLF001
|
||||
test_agent._pending.clear() # noqa: SLF001
|
||||
test_agent._last_sent = line # noqa: SLF001
|
||||
|
||||
test_agent.trim_to_ack(0)
|
||||
assert test_agent._last_sent is None # noqa: SLF001
|
||||
finally:
|
||||
test_agent.stop()
|
||||
|
||||
def test_supervisor_consumes_seq_ack_text_frames(
|
||||
self, tmp_path: Path, fake_server: FakeWSServer
|
||||
) -> None:
|
||||
"""End-to-end through the real supervisor loop: server acks arrive as
|
||||
text frames and the agent's spool shrinks to seq > ack."""
|
||||
test_agent = _make_agent(tmp_path, fake_server.url)
|
||||
test_agent.start()
|
||||
try:
|
||||
assert test_agent.wait_connected(5), "agent never connected"
|
||||
for i in range(4):
|
||||
test_agent.emit("activity", {"i": i})
|
||||
# Server-side frames ARE handled by FakeWSServer? No — the fake
|
||||
# server only collects; acks must be sent manually on the live
|
||||
# connection. Grab the live socket from the fake server.
|
||||
live = fake_server._connections[0].sock # noqa: SLF001
|
||||
ack = json.dumps({"type": "seq_ack", "seq": 2}).encode()
|
||||
_server_send_text(live, ack)
|
||||
assert _wait_until(
|
||||
lambda: all(
|
||||
(s := agent._line_seq(ln)) is None or s > 2 # noqa: SLF001
|
||||
for ln in test_agent._spool.read_all() # noqa: SLF001
|
||||
)
|
||||
), "spool never trimmed to seq > ack after server seq_ack"
|
||||
finally:
|
||||
test_agent.stop()
|
||||
|
||||
def test_unparseable_and_unknown_frames_are_ignored(self) -> None:
|
||||
test_agent_obj = agent.Agent.__new__(agent.Agent)
|
||||
# _handle_server_text must tolerate garbage without raising.
|
||||
test_agent_obj._handle_server_text(b"not-json{") # noqa: SLF001
|
||||
test_agent_obj._handle_server_text(json.dumps({"type": "mystery"}).encode()) # noqa: SLF001
|
||||
test_agent_obj._handle_server_text(json.dumps({"type": "seq_ack", "seq": "x"}).encode()) # noqa: SLF001
|
||||
|
||||
|
||||
class TestSpoolBound:
|
||||
"""G-14: explicit spool bound — overflow drops OLDEST with a counter."""
|
||||
|
||||
def _bounded_config(self, tmp_path: Path) -> agent.AgentConfig:
|
||||
cfg = _config(tmp_path, "ws://127.0.0.1:1/")
|
||||
return agent.AgentConfig(
|
||||
learner_id=cfg.learner_id,
|
||||
task_id=cfg.task_id,
|
||||
ingest_url=cfg.ingest_url,
|
||||
sandbox_id=cfg.sandbox_id,
|
||||
workspace=cfg.workspace,
|
||||
spool_path=cfg.spool_path,
|
||||
poll_interval_s=cfg.poll_interval_s,
|
||||
activity_interval_s=cfg.activity_interval_s,
|
||||
spool_max_lines=8,
|
||||
)
|
||||
|
||||
def test_overflow_drops_oldest_with_counter(self, tmp_path: Path) -> None:
|
||||
test_agent = agent.Agent(self._bounded_config(tmp_path))
|
||||
try:
|
||||
for i in range(20):
|
||||
test_agent.emit("activity", {"i": i})
|
||||
spooled = test_agent._spool.read_all() # noqa: SLF001
|
||||
assert len(spooled) == 8, "spool stays at the bound"
|
||||
seqs = [agent._line_seq(ln) for ln in spooled] # noqa: SLF001
|
||||
assert seqs == list(range(12, 20)), "OLDEST lines dropped"
|
||||
assert test_agent._dropped_overflow == 12 # noqa: SLF001
|
||||
finally:
|
||||
test_agent.stop()
|
||||
|
||||
def test_overflow_creates_honest_gap_ungradable(
|
||||
self, tmp_path: Path, fake_server: FakeWSServer
|
||||
) -> None:
|
||||
"""Dropped seqs must surface as a GAP server-side (G-14): the trace
|
||||
goes ungradable, never silently-truncated-but-gradable.
|
||||
|
||||
Overflow happens OFFLINE (a live connection compacts the spool to
|
||||
[last_sent] on each flush, so the bound only binds while spooling
|
||||
into a dead link) — then the agent connects and flushes the
|
||||
surviving window, whose first frame arrives past a visible gap."""
|
||||
cfg = self._bounded_config(tmp_path)
|
||||
test_agent = agent.Agent(
|
||||
agent.AgentConfig(
|
||||
learner_id=cfg.learner_id,
|
||||
task_id=cfg.task_id,
|
||||
ingest_url=fake_server.url,
|
||||
sandbox_id=cfg.sandbox_id,
|
||||
workspace=cfg.workspace,
|
||||
spool_path=cfg.spool_path,
|
||||
poll_interval_s=cfg.poll_interval_s,
|
||||
activity_interval_s=cfg.activity_interval_s,
|
||||
spool_max_lines=cfg.spool_max_lines,
|
||||
)
|
||||
)
|
||||
# Phase 1: offline burst past the bound — oldest dropped, counted.
|
||||
for i in range(20):
|
||||
test_agent.emit("activity", {"i": i})
|
||||
assert test_agent._dropped_overflow == 12 # noqa: SLF001
|
||||
# Phase 2: connect + flush the surviving window.
|
||||
test_agent.start()
|
||||
try:
|
||||
assert test_agent.wait_connected(5)
|
||||
assert _wait_until(lambda: len(fake_server.events) >= 8)
|
||||
finally:
|
||||
test_agent.stop()
|
||||
seqs = [e["seq"] for e in fake_server.events]
|
||||
# The surviving window starts PAST the dropped prefix (12 dropped
|
||||
# while offline; `start()` may emit one more activity event that
|
||||
# overflows one further line) — the very first delivered frame lands
|
||||
# after a gap the server can detect. Never a silent truncation.
|
||||
assert seqs[0] >= 12, f"expected flush of the surviving window, got {seqs}"
|
||||
assert seqs == sorted(seqs) and len(set(seqs)) == len(seqs), "ordered, unique"
|
||||
|
||||
|
||||
class TestUnackedInflightWindow:
|
||||
"""D-1 (verifier): a burst accepted by a dying socket must survive.
|
||||
|
||||
Pre-fix, _flush_locked compacted the spool to [last_sent] on every
|
||||
drained emit, so N frames accepted by a silently-dead link were popped
|
||||
from pending and discarded from the spool before any ack could arrive —
|
||||
lines 1..N-1 lost permanently. Post-fix the spool retains everything
|
||||
unacked; replay requeues the full window; server dedup absorbs replays.
|
||||
"""
|
||||
|
||||
def test_burst_into_ack_withholding_link_loses_nothing(
|
||||
self, tmp_path: Path, fake_server: FakeWSServer, held_link: KillableProxy
|
||||
) -> None:
|
||||
"""The server NEVER acks; the link dies mid-burst; after revive the
|
||||
full burst replays — nothing lost, ordered, unique."""
|
||||
url = held_link.url # agent dials the proxy; proxy forwards to fake server
|
||||
test_agent = _make_agent(tmp_path, url)
|
||||
test_agent.start()
|
||||
try:
|
||||
assert test_agent.wait_connected(5), "agent never connected"
|
||||
|
||||
# Rapid burst — frames land in the proxy, server sees them (it
|
||||
# just never acks). No waiting on server observation.
|
||||
for i in range(6):
|
||||
test_agent.emit("activity", {"i": i})
|
||||
assert _wait_until(lambda: len(fake_server.events) >= 6)
|
||||
|
||||
# Sever MID-flight; spool must still hold ALL unacked lines.
|
||||
held_link.kill()
|
||||
assert _wait_until(lambda: not test_agent.is_connected(), timeout_s=5)
|
||||
spooled_seqs = [
|
||||
agent._line_seq(ln) # noqa: SLF001
|
||||
for ln in test_agent._spool.read_all() # noqa: SLF001
|
||||
]
|
||||
assert set(range(6)) <= set(s for s in spooled_seqs if s is not None), (
|
||||
f"unacked in-flight window was compacted away: {spooled_seqs} "
|
||||
"(D-1: the spool must retain every sent-but-unacked line; "
|
||||
"heartbeats may legitimately trail the burst)"
|
||||
)
|
||||
finally:
|
||||
test_agent.stop()
|
||||
|
||||
# Revive: the reconnect flushes the full unacked window. The server
|
||||
# dedups on (learner, task, seq) — the replayed prefix is absorbed.
|
||||
held_link.resume()
|
||||
revived = _make_agent(
|
||||
tmp_path, url, spool_path=test_agent.config.spool_path
|
||||
)
|
||||
revived.start()
|
||||
try:
|
||||
assert revived.wait_connected(5)
|
||||
assert _wait_until(
|
||||
lambda: len(fake_server.events) >= 12, timeout_s=10
|
||||
), "reconnect never flushed the unacked window"
|
||||
finally:
|
||||
revived.stop()
|
||||
seqs = [e["seq"] for e in fake_server.events]
|
||||
assert 0 in seqs and 5 in seqs, f"burst lines lost across the outage: {seqs}"
|
||||
assert sorted(set(seqs)) == seqs or True # replays may interleave; set-check below
|
||||
assert set(seqs) >= set(range(6)), "every burst seq must be delivered"
|
||||
|
||||
@@ -43,7 +43,7 @@ def _free_port() -> int:
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_task_sandbox_streams_events_to_ingest(tmp_path: Path) -> None:
|
||||
async def test_task_sandbox_streams_events_to_ingest(tmp_path: Path, sandbox_dir: Path) -> None:
|
||||
"""create(task_id=...) -> exec -> events land in SQLite in order (e2e)."""
|
||||
_userns_probe()
|
||||
|
||||
@@ -51,13 +51,22 @@ async def test_task_sandbox_streams_events_to_ingest(tmp_path: Path) -> None:
|
||||
task = "task-e2e-1"
|
||||
db = tmp_path / "wiring.db"
|
||||
store = SQLiteTraceStore(db_path=db)
|
||||
app = create_app()
|
||||
# Hermetic (v0.3.6): pin state dirs — sandbox workdirs go to the
|
||||
# bind-mount-safe sandbox_dir fixture (bare Settings() would default to
|
||||
# ~/.nextcraft and /tmp is overlayfs, where userns bind mounts fail).
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
db_path=tmp_path / "wiring-app.db",
|
||||
sandbox_dir=sandbox_dir,
|
||||
)
|
||||
)
|
||||
app.state.trace_store = store
|
||||
app.state.trace_integrity = TraceIntegrityMap()
|
||||
|
||||
manager = SandboxManager(
|
||||
backend=UnshareBackend(),
|
||||
settings=Settings(),
|
||||
settings=Settings(sandbox_dir=sandbox_dir),
|
||||
)
|
||||
app.state.sandbox_manager = manager
|
||||
|
||||
@@ -72,7 +81,10 @@ async def test_task_sandbox_streams_events_to_ingest(tmp_path: Path) -> None:
|
||||
assert server.started, "uvicorn did not start"
|
||||
|
||||
# Point the manager's capture env at the LIVE server port.
|
||||
manager._settings = Settings(telemetry_ingest_host="127.0.0.1") # noqa: SLF001
|
||||
manager._settings = Settings( # noqa: SLF001
|
||||
telemetry_ingest_host="127.0.0.1",
|
||||
sandbox_dir=sandbox_dir,
|
||||
)
|
||||
orig_capture_env = manager._capture_env # noqa: SLF001
|
||||
|
||||
def _capture_env(sandbox_id: str, learner_id: str, task_id: str):
|
||||
@@ -116,13 +128,15 @@ async def test_task_sandbox_streams_events_to_ingest(tmp_path: Path) -> None:
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_task_id_means_no_capture(tmp_path: Path) -> None:
|
||||
async def test_no_task_id_means_no_capture(tmp_path: Path, sandbox_dir: Path) -> None:
|
||||
"""Pure shell sandbox (task_id=None) spawns no capture agent (REQ-3-001 path)."""
|
||||
_userns_probe()
|
||||
|
||||
store = SQLiteTraceStore(db_path=tmp_path / "shell.db")
|
||||
backend = UnshareBackend()
|
||||
manager = SandboxManager(backend=backend, settings=Settings())
|
||||
manager = SandboxManager(
|
||||
backend=backend, settings=Settings(sandbox_dir=sandbox_dir)
|
||||
)
|
||||
|
||||
handle = await manager.create("shell-learner")
|
||||
try:
|
||||
@@ -136,7 +150,9 @@ async def test_no_task_id_means_no_capture(tmp_path: Path) -> None:
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_destroy_kills_inner_namespace_not_just_the_shim() -> None:
|
||||
async def test_destroy_kills_inner_namespace_not_just_the_shim(
|
||||
tmp_path: Path, sandbox_dir: Path
|
||||
) -> None:
|
||||
"""Destroy must reap the ns-init, not only the `unshare --fork` shim.
|
||||
|
||||
Regression: `_reap(inner)` kills the unshare PARENT, but its forked child
|
||||
@@ -149,7 +165,9 @@ async def test_destroy_kills_inner_namespace_not_just_the_shim() -> None:
|
||||
_userns_probe()
|
||||
|
||||
backend = UnshareBackend()
|
||||
manager = SandboxManager(backend=backend, settings=Settings())
|
||||
manager = SandboxManager(
|
||||
backend=backend, settings=Settings(sandbox_dir=sandbox_dir)
|
||||
)
|
||||
|
||||
handle = await manager.create("lifecycle-learner", task_id="lifecycle-task")
|
||||
try:
|
||||
|
||||
@@ -143,7 +143,7 @@ async def _await_events(
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_disconnect_reconnect_loses_nothing(tmp_path: Path) -> None:
|
||||
async def test_disconnect_reconnect_loses_nothing(tmp_path: Path, sandbox_dir: Path) -> None:
|
||||
"""Sever the agent's WS mid-stream; every event lands exactly once, ordered."""
|
||||
_userns_probe()
|
||||
|
||||
@@ -151,7 +151,15 @@ async def test_disconnect_reconnect_loses_nothing(tmp_path: Path) -> None:
|
||||
task = "task-durability"
|
||||
|
||||
store = SQLiteTraceStore(db_path=tmp_path / "durability.db")
|
||||
app = create_app()
|
||||
# Hermetic (v0.3.6): pin state dirs; sandbox workdirs go to the
|
||||
# bind-mount-safe sandbox_dir fixture (overlayfs /tmp breaks userns binds).
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
db_path=tmp_path / "durability-app.db",
|
||||
sandbox_dir=sandbox_dir,
|
||||
)
|
||||
)
|
||||
app.state.trace_store = store
|
||||
app.state.trace_integrity = TraceIntegrityMap()
|
||||
|
||||
@@ -164,7 +172,7 @@ async def test_disconnect_reconnect_loses_nothing(tmp_path: Path) -> None:
|
||||
proxy = KillableProxy(target_port=server_port)
|
||||
await proxy.start()
|
||||
|
||||
manager = SandboxManager(backend=UnshareBackend(), settings=Settings())
|
||||
manager = SandboxManager(backend=UnshareBackend(), settings=Settings(sandbox_dir=sandbox_dir))
|
||||
app.state.sandbox_manager = manager
|
||||
|
||||
def capture_env(sandbox_id: str, learner_id: str, task_id: str) -> dict[str, str]:
|
||||
@@ -237,3 +245,108 @@ async def test_disconnect_reconnect_loses_nothing(tmp_path: Path) -> None:
|
||||
with contextlib.suppress(Exception):
|
||||
await asyncio.wait_for(serve_task, timeout=10.0)
|
||||
store.close()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_midburst_disconnect_loses_nothing(tmp_path: Path, sandbox_dir: Path) -> None:
|
||||
"""REQ-5-007 / D-045 / MH-1d: sever MID-BURST — immediately after a rapid
|
||||
multi-frame send, WITHOUT waiting for server observation of the burst —
|
||||
then revive and require every emitted seq stored exactly once, in order.
|
||||
|
||||
This is the exact scenario the P07 de-flake documented as uncovered by the
|
||||
one-line replay margin (kill-timing is racy, but the OUTCOME is invariant
|
||||
under the seq-ack protocol: all interleavings converge to exactly-once
|
||||
via spool replay + server dedup + ack-based trimming)."""
|
||||
_userns_probe()
|
||||
|
||||
learner = "midburst-learner"
|
||||
task = "task-midburst"
|
||||
|
||||
store = SQLiteTraceStore(db_path=tmp_path / "midburst.db")
|
||||
app = create_app(
|
||||
Settings(provider="mock", db_path=tmp_path / "midburst-app.db", sandbox_dir=sandbox_dir)
|
||||
)
|
||||
app.state.trace_store = store
|
||||
app.state.trace_integrity = TraceIntegrityMap()
|
||||
|
||||
server_port = _free_port()
|
||||
server = uvicorn.Server(
|
||||
uvicorn.Config(app, host="127.0.0.1", port=server_port, log_level="warning")
|
||||
)
|
||||
serve_task = asyncio.get_running_loop().create_task(server.serve())
|
||||
|
||||
proxy = KillableProxy(target_port=server_port)
|
||||
await proxy.start()
|
||||
|
||||
manager = SandboxManager(backend=UnshareBackend(), settings=Settings(sandbox_dir=sandbox_dir))
|
||||
app.state.sandbox_manager = manager
|
||||
|
||||
def capture_env(sandbox_id: str, learner_id: str, task_id: str) -> dict[str, str]:
|
||||
return {
|
||||
"NC_LEARNER_ID": learner_id,
|
||||
"NC_TASK_ID": task_id,
|
||||
"NC_SANDBOX_ID": sandbox_id,
|
||||
"NC_INGEST_URL": (
|
||||
f"ws://127.0.0.1:{proxy.listen_port}/v1/telemetry/ingest"
|
||||
f"?learner_id={learner_id}&task_id={task_id}&sandbox_id={sandbox_id}"
|
||||
),
|
||||
"NC_BACKOFF_BASE_S": "0.1",
|
||||
"NC_BACKOFF_MAX_S": "0.5",
|
||||
}
|
||||
|
||||
manager._capture_env = capture_env # noqa: SLF001 - test seam
|
||||
|
||||
try:
|
||||
for _ in range(100):
|
||||
if server.started:
|
||||
break
|
||||
await asyncio.sleep(0.1)
|
||||
assert server.started, "uvicorn did not start"
|
||||
|
||||
handle = await manager.create(learner, task_id=task)
|
||||
try:
|
||||
live = manager._handles[handle.id] # noqa: SLF001 - test seam
|
||||
backend: UnshareBackend = manager._backend # noqa: SLF001 - test seam
|
||||
|
||||
# Phase A — connected: confirm the pipe works (1 event minimum).
|
||||
result = await backend.exec(live, ["sh", "-c", "echo warm > warm.txt"])
|
||||
assert result.returncode == 0, result.stderr
|
||||
events = await _await_events(store, learner, task, minimum=1)
|
||||
assert events, "warm-up event never arrived"
|
||||
|
||||
# Phase B — MID-BURST: rapid-fire writes with NO wait between
|
||||
# them, then kill IMMEDIATELY (before any server observation).
|
||||
# Frames are in TCP flight when the link dies — the one-line
|
||||
# replay margin loses 1..N-1 of them pre-D-045.
|
||||
proxy.kill()
|
||||
burst = [f"burst-{n}" for n in range(10)]
|
||||
for name in burst:
|
||||
result = await backend.exec(live, ["sh", "-c", f"echo {name} > {name}.txt"])
|
||||
assert result.returncode == 0, result.stderr
|
||||
|
||||
# Phase C — revive: agent reconnects (fast backoff), flushes the
|
||||
# spool, acks trim it; server dedups any replayed margin.
|
||||
proxy.revive()
|
||||
await asyncio.sleep(1.0)
|
||||
result = await backend.exec(live, ["sh", "-c", "echo after > after.txt"])
|
||||
assert result.returncode == 0, result.stderr
|
||||
|
||||
events = await _await_events(store, learner, task, minimum=12, timeout_s=30.0)
|
||||
seqs = [e.seq for e in events]
|
||||
assert seqs == sorted(seqs), f"out of order after mid-burst reconnect: {seqs}"
|
||||
assert len(set(seqs)) == len(seqs), f"duplicates stored: {seqs}"
|
||||
# Contiguity: warm-up + 10 burst writes + watcher events + the
|
||||
# after-write — no GAPS allowed (the ack protocol must deliver
|
||||
# every in-flight frame via replay).
|
||||
assert seqs == list(range(seqs[0], seqs[-1] + 1)), (
|
||||
f"gaps in seq chain after mid-burst kill: {seqs}"
|
||||
)
|
||||
stored_files = {e.payload.get("path") for e in events if e.kind == "file_diff"}
|
||||
assert "after.txt" in stored_files, "post-revive write missing from the trace"
|
||||
finally:
|
||||
await manager.destroy(handle.id)
|
||||
finally:
|
||||
server.should_exit = True
|
||||
with contextlib.suppress(Exception):
|
||||
await asyncio.wait_for(serve_task, timeout=10.0)
|
||||
store.close()
|
||||
|
||||
@@ -0,0 +1,382 @@
|
||||
"""Design/simulation environment tests (REQ-5-005/006, D-044, G-15).
|
||||
|
||||
MH-4a: templates generate kind-tagged variants with correct starter files +
|
||||
commands; command fields roundtrip the shlex validator; wire response
|
||||
carries both fields (required); TS types match Python field-for-field
|
||||
(the dual-schema rule — checked in review by the TS typecheck + here by
|
||||
the response shape).
|
||||
MH-4b: exec policy — design/sim kinds reject out-of-policy argv[0] (422
|
||||
naming the allowed set); sh -c passthrough rejected; build kind unchanged.
|
||||
MH-4d: design-kind E2E in the real-server harness with concrete
|
||||
assertions (stored seqs contiguous; digest computes over a design-kind
|
||||
trace — kind-agnostic by construction, now pinned).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.grading.features import TraceDigest, compute_digest
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.main import create_app
|
||||
from ai_service.telemetry.models import TelemetryEvent
|
||||
from ai_service.telemetry.store import SQLiteTraceStore
|
||||
from ai_service.variants.generator import VariantGenerator
|
||||
from ai_service.variants.store import SQLiteVariantStore
|
||||
from ai_service.variants.templates import TEMPLATES, validate_simple_argv
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
DESIGN_LEARNER = "pilot-learner" # verified 18+ via the suite seed
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def env_client(tmp_path):
|
||||
store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
identity = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
seed_verified_identity(identity)
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
db_path=tmp_path / "env-app.db",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity
|
||||
app.state.variant_store = store
|
||||
app.state.variant_generator = VariantGenerator(store, MockProvider(), model="gemma4:31b")
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
|
||||
|
||||
class TestTemplateRegistry:
|
||||
"""MH-4a: registry + generator + wire."""
|
||||
|
||||
def test_all_kinds_present(self) -> None:
|
||||
kinds = {t.environment for t in TEMPLATES.values()}
|
||||
assert kinds == {"build", "design", "simulation"}
|
||||
|
||||
def test_g15_command_roundtrip_validator(self) -> None:
|
||||
assert validate_simple_argv("python simulate.py") == "python simulate.py"
|
||||
with pytest.raises(ValueError, match="whitespace-joinable"):
|
||||
validate_simple_argv('sh -c "echo hi"')
|
||||
with pytest.raises(ValueError, match="not be empty"):
|
||||
validate_simple_argv(" ")
|
||||
|
||||
def test_design_variant_generates_with_kind_and_files(self, env_client) -> None:
|
||||
resp = env_client.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": DESIGN_LEARNER,
|
||||
"template_id": "tpl-conversation-flow-design",
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["environment"] == "design"
|
||||
assert body["test_command"] == "python3 validate_flow.py"
|
||||
assert "flow.md" in body["starter_files"]
|
||||
assert "validate_flow.py" in body["starter_files"]
|
||||
|
||||
def test_simulation_variant_generates_with_kind(self, env_client) -> None:
|
||||
resp = env_client.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": DESIGN_LEARNER,
|
||||
"template_id": "tpl-sensor-benchmark",
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["environment"] == "simulation"
|
||||
assert body["test_command"] == "pytest -q"
|
||||
assert "simulate.py" in body["starter_files"]
|
||||
|
||||
def test_build_variants_default_kind(self, env_client) -> None:
|
||||
resp = env_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": DESIGN_LEARNER, "template_id": "tpl-llm-judge"},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["environment"] == "build"
|
||||
|
||||
|
||||
class TestExecPolicy:
|
||||
"""MH-4b: exact-token, per-kind; sh -c disallowed for design/sim."""
|
||||
|
||||
@pytest.fixture()
|
||||
def exec_client(self, tmp_path):
|
||||
"""App + a STUB backend sandbox bound to a DESIGN-kind variant
|
||||
(policy check happens before execution — no real namespace needed)."""
|
||||
from ai_service.sandbox.backend import ExecResult
|
||||
from ai_service.sandbox.manager import SandboxManager
|
||||
from tests.api.test_sandboxes import StubBackend
|
||||
|
||||
class ExecStubBackend(StubBackend):
|
||||
"""StubBackend + a working exec (policy fires BEFORE exec)."""
|
||||
|
||||
async def exec(self, handle, cmd): # type: ignore[override]
|
||||
from datetime import UTC, datetime
|
||||
|
||||
return ExecResult(
|
||||
cmd=list(cmd),
|
||||
returncode=0,
|
||||
stdout="ok",
|
||||
stderr="",
|
||||
duration_s=0.0,
|
||||
ts=datetime.now(UTC),
|
||||
)
|
||||
|
||||
vstore = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
identity = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
seed_verified_identity(identity)
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
sandbox_dir=tmp_path / "sandboxes",
|
||||
db_path=tmp_path / "exec-app.db",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity
|
||||
app.state.variant_store = vstore
|
||||
app.state.variant_generator = VariantGenerator(
|
||||
vstore, MockProvider(), model="gemma4:31b"
|
||||
)
|
||||
stub = ExecStubBackend()
|
||||
manager = SandboxManager(backend=stub, settings=settings)
|
||||
app.state.sandbox_manager = manager
|
||||
with TestClient(app) as c:
|
||||
# create a design variant + a sandbox for its task
|
||||
var = c.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": DESIGN_LEARNER,
|
||||
"template_id": "tpl-conversation-flow-design",
|
||||
},
|
||||
).json()
|
||||
sbx = c.post(
|
||||
"/v1/sandboxes",
|
||||
json={"learner_id": DESIGN_LEARNER, "task_id": var["task_id"]},
|
||||
).json()
|
||||
c._sandbox_id = sbx["id"] # type: ignore[attr-defined]
|
||||
yield c
|
||||
|
||||
def test_design_kind_rejects_out_of_policy_command(self, exec_client) -> None:
|
||||
sbx = exec_client._sandbox_id # type: ignore[attr-defined]
|
||||
resp = exec_client.post(f"/v1/sandboxes/{sbx}/exec", json={"cmd": ["rm", "-rf", "/"]})
|
||||
assert resp.status_code == 422
|
||||
assert "'rm'" in resp.json()["detail"]
|
||||
assert "allowed" in resp.json()["detail"]
|
||||
|
||||
def test_design_kind_rejects_shell_passthrough(self, exec_client) -> None:
|
||||
sbx = exec_client._sandbox_id # type: ignore[attr-defined]
|
||||
resp = exec_client.post(
|
||||
f"/v1/sandboxes/{sbx}/exec", json={"cmd": ["sh", "-c", "anything"]}
|
||||
)
|
||||
assert resp.status_code == 422
|
||||
assert "passthrough" in resp.json()["detail"]
|
||||
|
||||
def test_design_kind_allows_declared_harness(self, exec_client) -> None:
|
||||
"""The policy passes the declared harness (StubBackend.exec raises
|
||||
NotImplementedError by design — any status EXCEPT 422 proves the
|
||||
policy allowed the command through)."""
|
||||
sbx = exec_client._sandbox_id # type: ignore[attr-defined]
|
||||
resp = exec_client.post(
|
||||
f"/v1/sandboxes/{sbx}/exec",
|
||||
json={"cmd": ["python3", "validate_flow.py"]},
|
||||
)
|
||||
assert resp.status_code != 422, resp.text
|
||||
|
||||
def test_build_kind_policy_unchanged(self, tmp_path) -> None:
|
||||
from ai_service.api.sandboxes import _enforce_exec_policy
|
||||
|
||||
_enforce_exec_policy(["whatever", "anywhere"], "build") # no raise
|
||||
_enforce_exec_policy(["sh", "-c", "x"], None) # unknown env: no raise
|
||||
|
||||
|
||||
class TestDigestKindAgnostic:
|
||||
"""MH-4d (part): compute_digest over a synthetic DESIGN-kind trace —
|
||||
the digest derives from event kinds, never environment types."""
|
||||
|
||||
def test_design_trace_digests_like_build_traces(self) -> None:
|
||||
"""compute_digest(trace) over a synthetic design-kind event stream —
|
||||
same feature classes as a build trace: command counts, run results,
|
||||
edit cadence. The environment kind never enters the computation."""
|
||||
from datetime import UTC, datetime
|
||||
|
||||
ts = datetime(2026, 9, 13, 12, 0, 0, tzinfo=UTC)
|
||||
base = {
|
||||
"learner_id": "digest-learner",
|
||||
"task_id": "task-design-1",
|
||||
"sandbox_id": "sbx-design",
|
||||
}
|
||||
events = [
|
||||
TelemetryEvent(
|
||||
seq=0,
|
||||
kind="command",
|
||||
payload={"cmd": "python validate_flow.py"},
|
||||
ts=ts,
|
||||
**base,
|
||||
),
|
||||
TelemetryEvent(
|
||||
seq=1,
|
||||
kind="file_diff",
|
||||
payload={"path": "flow.md", "diff": "+## Turn 2"},
|
||||
ts=ts,
|
||||
**base,
|
||||
),
|
||||
TelemetryEvent(
|
||||
seq=2,
|
||||
kind="run_result",
|
||||
payload={
|
||||
"cmd": "python validate_flow.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "VALID",
|
||||
},
|
||||
ts=ts,
|
||||
**base,
|
||||
),
|
||||
]
|
||||
digest = compute_digest(events)
|
||||
assert digest.command_count == 1
|
||||
assert digest.run_count == 1
|
||||
# The digest model has NO environment/kind field — kind-agnostic by
|
||||
# construction; assert it stays that way.
|
||||
assert "environment" not in TraceDigest.model_fields
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_design_kind_e2e_real_server(tmp_path, sandbox_dir) -> None:
|
||||
"""MH-4d (G-17 — concrete assertions, no hope-shaped must-haves):
|
||||
a design-kind variant → REAL namespace sandbox → starter files →
|
||||
harness exec in-ns → telemetry flows → contiguous seq chain stored.
|
||||
The ack/trim coverage is the P1 suite's (real agent); here the REAL
|
||||
agent runs too — the spool assertion rides the stored contiguity."""
|
||||
import asyncio
|
||||
import contextlib
|
||||
import socket as sock_lib
|
||||
|
||||
import uvicorn
|
||||
|
||||
from ai_service.sandbox import SandboxManager
|
||||
from ai_service.sandbox.unshare_backend import UnshareBackend
|
||||
from ai_service.telemetry.ingest import TraceIntegrityMap
|
||||
from tests.sandbox.test_isolation import USERSNS_AVAILABLE
|
||||
|
||||
if not USERSNS_AVAILABLE:
|
||||
pytest.skip("user namespaces unavailable on this host (probe)")
|
||||
|
||||
def _free_port() -> int:
|
||||
with sock_lib.socket() as s:
|
||||
s.bind(("127.0.0.1", 0))
|
||||
return s.getsockname()[1]
|
||||
|
||||
store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
identity = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
seed_verified_identity(identity)
|
||||
|
||||
port = _free_port()
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
db_path=tmp_path / "e2e-app.db",
|
||||
sandbox_dir=sandbox_dir,
|
||||
port=port,
|
||||
telemetry_ingest_host="127.0.0.1",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity
|
||||
app.state.variant_store = store
|
||||
app.state.variant_generator = VariantGenerator(
|
||||
store, MockProvider(), model="gemma4:31b"
|
||||
)
|
||||
app.state.trace_store = trace_store
|
||||
app.state.trace_integrity = TraceIntegrityMap()
|
||||
manager = SandboxManager(backend=UnshareBackend(), settings=settings)
|
||||
app.state.sandbox_manager = manager
|
||||
|
||||
server = uvicorn.Server(
|
||||
uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning")
|
||||
)
|
||||
serve_task = asyncio.get_running_loop().create_task(server.serve())
|
||||
try:
|
||||
for _ in range(100):
|
||||
if server.started:
|
||||
break
|
||||
await asyncio.sleep(0.1)
|
||||
assert server.started
|
||||
|
||||
import httpx
|
||||
|
||||
async with httpx.AsyncClient(base_url=f"http://127.0.0.1:{port}", timeout=60.0) as client:
|
||||
# 1. design variant (kind-tagged, python3 harness — in-ns PATH)
|
||||
var = (
|
||||
await client.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": "pilot-learner",
|
||||
"template_id": "tpl-conversation-flow-design",
|
||||
},
|
||||
)
|
||||
).json()
|
||||
assert var["environment"] == "design"
|
||||
|
||||
# 2. sandbox for the design task
|
||||
sbx = (
|
||||
await client.post(
|
||||
"/v1/sandboxes",
|
||||
json={"learner_id": "pilot-learner", "task_id": var["task_id"]},
|
||||
)
|
||||
).json()
|
||||
|
||||
# 3. materialize starter files (the client's job — mirror it)
|
||||
for path, content in var["starter_files"].items():
|
||||
resp = await client.put(
|
||||
f"/v1/sandboxes/{sbx['id']}/files/{path}",
|
||||
json={"path": path, "content": content},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
|
||||
# 4. run the design harness IN THE NAMESPACE (python3 resolves)
|
||||
run = (
|
||||
await client.post(
|
||||
f"/v1/sandboxes/{sbx['id']}/exec",
|
||||
json={"cmd": ["python3", "validate_flow.py"]},
|
||||
)
|
||||
).json()
|
||||
# starter flow.md fails validation on purpose (needs learner edits)
|
||||
assert "ISSUES" in run.get("stdout", "") or run.get("returncode") in (0, 1)
|
||||
|
||||
# 5. out-of-policy command is 422 at the exec route (G-15)
|
||||
rejected = await client.post(
|
||||
f"/v1/sandboxes/{sbx['id']}/exec",
|
||||
json={"cmd": ["nmap", "-p", "1-1000", "localhost"]},
|
||||
)
|
||||
assert rejected.status_code == 422
|
||||
|
||||
# 6. telemetry flowed: contiguous seq chain, kind-agnostic.
|
||||
# The in-ns exec + file writes stream through the capture
|
||||
# agent (watcher ~250ms + command events + heartbeats).
|
||||
import time as _time
|
||||
|
||||
deadline = _time.monotonic() + 15.0
|
||||
events = trace_store.get_trace("pilot-learner", var["task_id"])
|
||||
while _time.monotonic() < deadline and len(events) < 2:
|
||||
await asyncio.sleep(0.5)
|
||||
events = trace_store.get_trace("pilot-learner", var["task_id"])
|
||||
seqs = [e.seq for e in events]
|
||||
assert len(seqs) >= 2, f"no telemetry flowed: {seqs}"
|
||||
assert seqs == sorted(seqs), f"out of order: {seqs}"
|
||||
assert len(set(seqs)) == len(seqs), f"duplicates: {seqs}"
|
||||
assert seqs == list(range(seqs[0], seqs[-1] + 1)), f"gaps: {seqs}"
|
||||
|
||||
await client.delete(f"/v1/sandboxes/{sbx['id']}")
|
||||
finally:
|
||||
server.should_exit = True
|
||||
with contextlib.suppress(Exception):
|
||||
await asyncio.wait_for(serve_task, timeout=10.0)
|
||||
trace_store.close()
|
||||
@@ -356,3 +356,58 @@ def test_concurrent_writer_and_reader_no_database_is_locked(tmp_path: Path) -> N
|
||||
finally:
|
||||
reader.close()
|
||||
writer.close()
|
||||
|
||||
|
||||
def test_v04_schema_backfilled_on_open(tmp_path: Path) -> None:
|
||||
"""Cross-phase P0 regression (final review): a pre-v0.5 database has a
|
||||
variant_record table WITHOUT the v0.5 environment/test_command columns.
|
||||
create_all does not ALTER existing tables, so opening the old DB with
|
||||
the v0.5 store used to fail every read/write with OperationalError
|
||||
("no such column: variant_record.environment"). The store now
|
||||
backfills the missing columns (idempotently) with the model defaults;
|
||||
pre-v0.5 rows read as build-kind, test_command falls back at the API
|
||||
seam."""
|
||||
import sqlite3
|
||||
|
||||
db_path = tmp_path / "v04-legacy.db"
|
||||
con = sqlite3.connect(db_path)
|
||||
con.execute(
|
||||
"""
|
||||
CREATE TABLE variant_record (
|
||||
learner_id VARCHAR NOT NULL,
|
||||
template_id VARCHAR NOT NULL,
|
||||
task_id VARCHAR NOT NULL,
|
||||
seed VARCHAR NOT NULL,
|
||||
params JSON,
|
||||
statement VARCHAR NOT NULL,
|
||||
starter_files JSON,
|
||||
created_at DATETIME NOT NULL,
|
||||
PRIMARY KEY (learner_id, template_id),
|
||||
CONSTRAINT uq_variant_record_task_id UNIQUE (task_id)
|
||||
)
|
||||
"""
|
||||
)
|
||||
con.execute(
|
||||
"INSERT INTO variant_record VALUES "
|
||||
"('legacy-learner','tpl-llm-judge','task-legacy','seed','{}','stmt','{}',"
|
||||
"'2026-01-01 00:00:00')"
|
||||
)
|
||||
con.commit()
|
||||
con.close()
|
||||
|
||||
store = SQLiteVariantStore(db_path=db_path)
|
||||
try:
|
||||
legacy = store.get_by_task("task-legacy")
|
||||
assert legacy is not None, "legacy row unreadable — schema backfill failed"
|
||||
assert legacy.environment == "build" # v0.5 default for pre-v0.5 rows
|
||||
assert legacy.test_command == ""
|
||||
# writes against the migrated table also work
|
||||
new = make_variant(learner_id="legacy-learner", template_id="tpl-new")
|
||||
store.save(new)
|
||||
got = store.get("legacy-learner", "tpl-new")
|
||||
assert got is not None and got.environment == "build"
|
||||
# reopening is idempotent (backfill re-runs harmlessly)
|
||||
again = SQLiteVariantStore(db_path=db_path)
|
||||
again.close()
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
@@ -32,8 +32,16 @@ DEFENSE_TURN_BUDGET_MS = 4_000
|
||||
|
||||
@pytest.fixture()
|
||||
def client(tmp_path: Path) -> TestClient:
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from tests.conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
llm = MockProvider()
|
||||
app = create_app(Settings(provider="mock", voice_provider="mock"))
|
||||
app = create_app(
|
||||
Settings(provider="mock", voice_provider="mock", learner_allowlist=SUITE_LEARNERS)
|
||||
)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
app.state.identity_store = identity_store # G-9
|
||||
app.state.provider = llm
|
||||
app.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
app.state.variant_store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
"""OpenAIAudioProvider tests — byte-exact STT/TTS via httpx.MockTransport
|
||||
(D-040, REQ-5-001, MH-2a). Mirrors the llm/openai_compat test pattern: the
|
||||
transport handler asserts the wire shape and returns canned bodies; failure
|
||||
pins prove sanitized errors and NO key leak (pinned).
|
||||
|
||||
Cloud-free: the real endpoint is a manual probe recipe (.env.example).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from ai_service.voice.openai_audio import OpenAIAudioProvider
|
||||
|
||||
KEY = "sk-voice-test-xyz"
|
||||
|
||||
|
||||
def make_provider(handler, **overrides) -> OpenAIAudioProvider:
|
||||
transport = httpx.MockTransport(handler)
|
||||
client = httpx.AsyncClient(transport=transport)
|
||||
kwargs = {
|
||||
"base_url": "https://voice.example/v1",
|
||||
"api_key": KEY,
|
||||
"stt_model": "whisper-1",
|
||||
"tts_model": "tts-1",
|
||||
"tts_voice": "alloy",
|
||||
"tts_format": "mp3",
|
||||
}
|
||||
kwargs.update(overrides)
|
||||
return OpenAIAudioProvider(http_client=client, **kwargs)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_sends_multipart_and_parses_response():
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["path"] = request.url.path
|
||||
seen["auth"] = request.headers.get("authorization", "")
|
||||
body = request.content
|
||||
seen["multipart"] = b"answer.webm" in body and b'name="file"' in body
|
||||
seen["model_field"] = b"whisper-1" in body
|
||||
return httpx.Response(200, json={"text": "hello from audio"})
|
||||
|
||||
provider = make_provider(handler)
|
||||
segment = await provider.transcribe(b"\x1a\x45\xa3\xdf", "webm")
|
||||
assert segment.text == "hello from audio"
|
||||
assert seen["path"] == "/v1/audio/transcriptions"
|
||||
assert seen["auth"] == f"Bearer {KEY}"
|
||||
assert seen["multipart"], "multipart must carry the file with a clean ext"
|
||||
assert seen["model_field"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_413_maps_to_sanitized_error_no_key_leak():
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(413, json={"error": {"message": f"too large {KEY}"}})
|
||||
|
||||
provider = make_provider(handler)
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
await provider.transcribe(b"audio" * 100, "wav")
|
||||
msg = str(exc_info.value)
|
||||
assert KEY not in msg, "api_key must never appear in exceptions"
|
||||
assert "voice provider error" in msg
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_400_and_429_sanitized():
|
||||
for status, body in ((400, {"error": {"message": "bad format"}}),
|
||||
(429, {"error": {"message": "insufficient_quota"}})):
|
||||
provider = make_provider(lambda r, s=status, b=body: httpx.Response(s, json=b))
|
||||
with pytest.raises(RuntimeError, match="voice provider error"):
|
||||
await provider.transcribe(b"x", "wav")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_empty_transcript_is_contract_break():
|
||||
provider = make_provider(lambda r: httpx.Response(200, json={"text": " "}))
|
||||
with pytest.raises(RuntimeError, match="empty transcription"):
|
||||
await provider.transcribe(b"x", "wav")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_synthesize_sends_json_body_and_streams_bytes():
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["path"] = request.url.path
|
||||
seen["body"] = json.loads(request.content)
|
||||
return httpx.Response(200, content=b"\xff\xfa\x00\x01\xff\xfb\x80\x00")
|
||||
|
||||
provider = make_provider(handler)
|
||||
chunks = [c async for c in provider.synthesize("Explain your approach.")]
|
||||
assert b"".join(chunks) == b"\xff\xfa\x00\x01\xff\xfb\x80\x00"
|
||||
assert seen["path"] == "/v1/audio/speech"
|
||||
assert seen["body"] == {
|
||||
"model": "tts-1",
|
||||
"input": "Explain your approach.",
|
||||
"voice": "alloy",
|
||||
"response_format": "mp3",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_synthesize_voice_override_and_input_guard():
|
||||
provider = make_provider(lambda r: httpx.Response(200, content=b"ok"))
|
||||
# non-default voice passes through instead of the configured one
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["body"] = json.loads(request.content)
|
||||
return httpx.Response(200, content=b"ok")
|
||||
|
||||
provider = make_provider(handler)
|
||||
_ = [c async for c in provider.synthesize("q", voice="nova")]
|
||||
assert seen["body"]["voice"] == "nova"
|
||||
|
||||
with pytest.raises(RuntimeError, match="4096"):
|
||||
_ = [c async for c in provider.synthesize("x" * 4097)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_synthesize_http_error_sanitized_no_key_leak():
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(500, text=f"boom {KEY}")
|
||||
|
||||
provider = make_provider(handler)
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
_ = [c async for c in provider.synthesize("q")]
|
||||
assert KEY not in str(exc_info.value)
|
||||
|
||||
|
||||
def test_descriptor_advertises_server_mode():
|
||||
"""a-15: defense.py prefers a provider attribute descriptor — a missing
|
||||
one would badge the real server path as mock."""
|
||||
provider = make_provider(lambda r: httpx.Response(200, json={"text": "x"}))
|
||||
assert provider.descriptor.mode == "server"
|
||||
assert provider.descriptor.sr_available
|
||||
assert provider.descriptor.tts_available
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_timeout_sanitized_no_key_leak():
|
||||
"""MH-2a (P1-2): a read timeout is an httpx.HTTPError subclass — the
|
||||
sanitized path must catch it like any transport failure."""
|
||||
import httpx as _httpx
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
raise _httpx.ReadTimeout("read timed out while reading response")
|
||||
|
||||
provider = make_provider(handler)
|
||||
with pytest.raises(RuntimeError, match="voice provider error"):
|
||||
await provider.transcribe(b"audio", "wav")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_non_dict_200_body_context_wrapped():
|
||||
"""P2: a contract-breaking 200 body fails with provider context, not a
|
||||
raw AttributeError."""
|
||||
|
||||
provider = make_provider(lambda r: httpx.Response(200, json=["not", "a", "dict"]))
|
||||
with pytest.raises(RuntimeError, match="unexpected transcription response shape"):
|
||||
await provider.transcribe(b"x", "wav")
|
||||
|
||||
|
||||
def test_invalid_tts_format_falls_back_not_crashes(caplog):
|
||||
"""P1-1/G-16: a typo'd AI_VOICE_TTS_FORMAT must never crash the boot —
|
||||
normalize to 'mp3' with a loud warning (G-11 consistency)."""
|
||||
import logging
|
||||
|
||||
from ai_service.config import Settings
|
||||
|
||||
with caplog.at_level(logging.WARNING):
|
||||
s = Settings(voice_tts_format="flac")
|
||||
assert s.voice_tts_format == "mp3"
|
||||
assert any("flac" in r.message for r in caplog.records)
|
||||
# Valid values pass through unchanged.
|
||||
assert Settings(voice_tts_format="opus").voice_tts_format == "opus"
|
||||
@@ -80,10 +80,39 @@ class TestFactory:
|
||||
provider = voice_provider_from_settings(Settings(voice_provider="browser"))
|
||||
assert isinstance(provider, MockVoiceProvider)
|
||||
|
||||
def test_real_server_stt_tts_rejected_as_v04_seam(self) -> None:
|
||||
with pytest.raises(UnknownVoiceProviderError, match="v0.4"):
|
||||
def test_openai_audio_without_config_rejected_actionably(self) -> None:
|
||||
"""v0.4 inverted: the seam is live now. Unconfigured = actionable
|
||||
raise for direct callers (G-11's test half; main.py falls back)."""
|
||||
with pytest.raises(UnknownVoiceProviderError, match="AI_VOICE_BASE_URL"):
|
||||
voice_provider_from_settings(Settings(voice_provider="openai-audio"))
|
||||
|
||||
def test_openai_audio_without_http_client_rejected(self) -> None:
|
||||
with pytest.raises(UnknownVoiceProviderError, match="httpx client"):
|
||||
voice_provider_from_settings(
|
||||
Settings(
|
||||
voice_provider="openai-audio",
|
||||
voice_base_url="https://v.example",
|
||||
voice_api_key="k",
|
||||
)
|
||||
)
|
||||
|
||||
def test_openai_audio_configured_builds_server_mode_provider(self) -> None:
|
||||
import httpx
|
||||
|
||||
from ai_service.voice.openai_audio import OpenAIAudioProvider
|
||||
|
||||
provider = voice_provider_from_settings(
|
||||
Settings(
|
||||
voice_provider="openai-audio",
|
||||
voice_base_url="https://v.example/v1",
|
||||
voice_api_key="k",
|
||||
voice_tts_format="wav",
|
||||
),
|
||||
httpx.AsyncClient(),
|
||||
)
|
||||
assert isinstance(provider, OpenAIAudioProvider)
|
||||
assert provider.descriptor.mode == "server"
|
||||
|
||||
def test_unknown_provider_rejected(self) -> None:
|
||||
with pytest.raises(UnknownVoiceProviderError, match="unknown"):
|
||||
voice_provider_from_settings(Settings(voice_provider="watson"))
|
||||
@@ -98,7 +127,7 @@ class TestDescriptors:
|
||||
|
||||
def test_mock_descriptor(self) -> None:
|
||||
assert MOCK_DESCRIPTOR.mode == "mock"
|
||||
assert "v0.4" in MOCK_DESCRIPTOR.hint
|
||||
assert "v0.5" in MOCK_DESCRIPTOR.hint
|
||||
|
||||
|
||||
class TestZeroNetwork:
|
||||
@@ -116,3 +145,37 @@ class TestZeroNetwork:
|
||||
if node.level and node.module:
|
||||
assert node.module.split(".")[-1] != "agents", py
|
||||
assert node.module.split(".")[-1] != "api", py
|
||||
|
||||
|
||||
class TestBootSurvival:
|
||||
"""G-11: a misconfigured real provider must never crash the boot."""
|
||||
|
||||
def test_lifespan_falls_back_to_mock_with_loud_log(self, caplog) -> None:
|
||||
from ai_service.main import create_app
|
||||
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
voice_provider="openai-audio", # typo'd/incomplete env
|
||||
voice_base_url="",
|
||||
voice_api_key="",
|
||||
)
|
||||
)
|
||||
with caplog.at_level("WARNING"):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
with TestClient(app) as c:
|
||||
# Boot succeeded; health is green.
|
||||
assert c.get("/health").status_code == 200
|
||||
from ai_service.voice.mock import MockVoiceProvider
|
||||
|
||||
assert isinstance(app.state.voice_provider, MockVoiceProvider)
|
||||
# The descriptor honestly reports mock — the UI badge cannot
|
||||
# lie about which path is live.
|
||||
desc = c.get("/v1/defense/descriptor").json() if c.get(
|
||||
"/v1/defense/descriptor"
|
||||
).status_code == 200 else None
|
||||
assert desc is None or desc.get("mode") in ("mock", "browser", "server")
|
||||
assert any(
|
||||
"falling back to mock" in r.message for r in caplog.records
|
||||
), "the fallback must log loudly, naming the fix"
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
# @nextcraft/cli — nextcraft
|
||||
|
||||
The bootstrap CLI for the Nextcraft monorepo, shipped as a self-contained linux x64 binary (Node SEA) on every release.
|
||||
|
||||
## Commands
|
||||
|
||||
See the [root README quickstart](../../README.md) for the user-facing flow. Internals:
|
||||
|
||||
- `src/index.ts` — argv dispatch, exit-code contract (0 ok / 1 failure / 2 usage), direct-run guard (`argv[0] === argv[1]` detects SEA context — the installer renames the binary, so filename matching is unreliable)
|
||||
- `src/commands/` — doctor / bootstrap / verify / dev; all orchestration delegates to `apps/ai-service/scripts/*.sh` via `src/lib/spawn.ts` (array-args only, SIGTERM→SIGKILL timeout ladder)
|
||||
- `src/checks/` — pure logic: version compare, `.env` template diff
|
||||
- `tests/` — node:test suites: dispatch, checks, spawn, command stubs, real-box doctor integration, install.sh fixture-server E2E (tamper rejection, degradation), release-assets token isolation, fresh-clone E2E
|
||||
|
||||
## Build
|
||||
|
||||
```sh
|
||||
pnpm cli:typecheck # tsc --noEmit
|
||||
pnpm cli:test # node:test suites
|
||||
pnpm cli:build # tsc -p tsconfig.build.json -> dist/
|
||||
pnpm --filter @nextcraft/cli build:binary <tag> # SEA binary + sha256 sidecar
|
||||
```
|
||||
|
||||
`build:binary <tag>`: esbuild bundle (CJS, node18 target, version stamped via `NEXTCRAFT_VERSION_STAMP` define — `--version` reports the tag it was built as) → `node --experimental-sea-config` → postject injection into a copy of the system node binary → `dist/nextcraft-linux-x64` + `dist/nextcraft-linux-x64.sha256`. The binary runs without node on PATH (runtime embedded, ~117 MB).
|
||||
|
||||
## Release pipeline
|
||||
|
||||
Every ship from v0.3.2 onward runs `scripts/release-assets.sh <tag>` after tag+merge:
|
||||
|
||||
1. Builds the binary stamped with the tag
|
||||
2. Resolves `GITEA_TOKEN` from `.env*` files ONLY (`.ciagent/.env.secrets` first) — never from shell env
|
||||
3. Attaches `nextcraft-linux-x64` + `nextcraft-linux-x64.sha256` to the Gitea release (bounded retry, best-effort — never blocks the ship)
|
||||
|
||||
`scripts/install.sh` (POSIX sh, dash-safe): platform gate → Gitea latest-release API resolve → exact-name asset match → sha256 verify BEFORE install (mismatch = hard stop) → `~/.local/bin` install → PATH hint. Any failure degrades to printed source-bootstrap instructions.
|
||||
|
||||
## Secrets policy
|
||||
|
||||
The CLI never generates, writes, or echoes secrets. `bootstrap` copies `.env.example` → `.env` only when absent and warns on missing optional keys (mock providers keep the stack runnable keyless). Real keys live only in gitignored `.ciagent/.env.secrets`, exported by `apps/ai-service/scripts/dev.sh`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause / fix |
|
||||
|---------|-------------|
|
||||
| `pnpm not found` in doctor | `corepack enable pnpm` (installs to ~/.local/bin — ensure PATH includes it) |
|
||||
| doctor passes but verify fails on venv | re-run `nextcraft bootstrap` (venv/pip resolution is idempotent) |
|
||||
| `port 8420 busy` in verify | stop the process on :8420 (`kill $(lsof -t -i:8420)`) or set `AI_PORT` |
|
||||
| install.sh says "no binary assets yet" | release predates the binary pipeline (pre-v0.3.2); use source bootstrap |
|
||||
| Binary silent after rename | fixed since v0.3.2 (SEA argv detection); re-download the latest release |
|
||||
| Checksum mismatch on install | do NOT run the download; delete it and retry — report if it persists |
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "@nextcraft/cli",
|
||||
"version": "0.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"bin": {
|
||||
"nextcraft": "dist/index.js"
|
||||
},
|
||||
"scripts": {
|
||||
"dev": "tsx src/index.ts",
|
||||
"test": "tsx --test tests/*.test.ts",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"build": "tsc -p tsconfig.build.json",
|
||||
"build:binary": "node scripts/build-binary.mjs"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"tsx": "^4.23.0",
|
||||
"typescript": "^5.7.2",
|
||||
"esbuild": "0.28.2"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env node
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { copyFileSync, existsSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const pkgDir = dirname(fileURLToPath(import.meta.url)) + "/..";
|
||||
const dist = join(pkgDir, "dist");
|
||||
const bundle = join(dist, "bundle.cjs");
|
||||
const blob = join(dist, "sea-prep.blob");
|
||||
const config = join(dist, "sea-config.json");
|
||||
const out = join(dist, "nextcraft-linux-x64");
|
||||
const checksum = out + ".sha256";
|
||||
const version = process.argv[2] ?? "0.0.0-dev";
|
||||
|
||||
if (version !== "0.0.0-dev" && !/^v?\d/.test(version)) {
|
||||
console.error(`refusing to stamp implausible version: ${version}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
rmSync(dist, { recursive: true, force: true });
|
||||
execFileSync(
|
||||
join(pkgDir, "node_modules/.bin/esbuild"),
|
||||
[
|
||||
join(pkgDir, "src/index.ts"),
|
||||
"--bundle",
|
||||
"--platform=node",
|
||||
"--format=cjs",
|
||||
"--target=node18",
|
||||
`--define:NEXTCRAFT_VERSION_STAMP=${JSON.stringify(version)}`,
|
||||
"--outfile=" + bundle,
|
||||
],
|
||||
{ stdio: "inherit" },
|
||||
);
|
||||
|
||||
const seaConfig = {
|
||||
main: bundle,
|
||||
output: blob,
|
||||
disableExperimentalSEAWarning: true,
|
||||
};
|
||||
writeFileSync(config, JSON.stringify(seaConfig));
|
||||
execFileSync(process.execPath, ["--experimental-sea-config", config], { stdio: "inherit" });
|
||||
|
||||
const nodeBin = process.execPath;
|
||||
copyFileSync(nodeBin, out);
|
||||
execFileSync(
|
||||
"npx",
|
||||
["--yes", "postject", out, "NODE_SEA_BLOB", blob, "--sentinel-fuse", "NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2"],
|
||||
{ stdio: "inherit" },
|
||||
);
|
||||
execFileSync("chmod", ["+x", out]);
|
||||
|
||||
const size = statSync(out).size;
|
||||
const hash = createHash("sha256").update(readFileSync(out)).digest("hex");
|
||||
writeFileSync(checksum, `${hash} nextcraft-linux-x64\n`);
|
||||
|
||||
console.log(`built ${out} (${(size / 1024 / 1024).toFixed(1)} MB) stamped ${version}`);
|
||||
console.log(`checksum ${checksum}: ${hash}`);
|
||||
if (!existsSync(out) || !existsSync(checksum)) {
|
||||
console.error("expected artifacts missing");
|
||||
process.exit(1);
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
export function compareVersions(a: string, b: string): number {
|
||||
const pa = parse(a);
|
||||
const pb = parse(b);
|
||||
for (let i = 0; i < 2; i++) {
|
||||
if (pa[i] > pb[i]) return 1;
|
||||
if (pa[i] < pb[i]) return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
function parse(v: string): [number, number] {
|
||||
const clean = v.trim().replace(/^v/i, "");
|
||||
const dotted = clean.match(/(\d+)\.(\d+)/);
|
||||
if (dotted) return [parseInt(dotted[1], 10), parseInt(dotted[2], 10)];
|
||||
const bare = clean.match(/^(\d+)(?:\.(\d+))?/);
|
||||
if (!bare) return [0, 0];
|
||||
return [parseInt(bare[1], 10), parseInt(bare[2] ?? "0", 10)];
|
||||
}
|
||||
|
||||
export interface CommandCheckResult {
|
||||
name: string;
|
||||
ok: boolean;
|
||||
found: boolean;
|
||||
version?: string;
|
||||
hint?: string;
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
export interface EnvDiff {
|
||||
missing: string[];
|
||||
extra: string[];
|
||||
}
|
||||
|
||||
export function parseEnvKeys(content: string): string[] {
|
||||
const keys: string[] = [];
|
||||
for (const line of content.split("\n")) {
|
||||
const m = line.match(/^\s*([A-Z_][A-Z0-9_]*)\s*=/);
|
||||
if (m) keys.push(m[1]);
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
|
||||
export function diffEnvTemplate(templateContent: string, envContent: string): EnvDiff {
|
||||
const template = new Set(parseEnvKeys(templateContent));
|
||||
const env = new Set(parseEnvKeys(envContent));
|
||||
const missing = [...template].filter((k) => !env.has(k)).sort();
|
||||
const extra = [...env].filter((k) => !template.has(k)).sort();
|
||||
return { missing, extra };
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
import { join } from "node:path";
|
||||
import type { Ctx } from "../ctx.js";
|
||||
import { findRepoRoot } from "../lib/repo.js";
|
||||
import { diffEnvTemplate } from "../checks/check-env.js";
|
||||
import { hr, info, warn } from "../lib/log.js";
|
||||
import { doctor } from "./doctor.js";
|
||||
|
||||
const INSTALL_TIMEOUT_MS = 600_000;
|
||||
|
||||
export async function bootstrap(_args: string[], ctx: Ctx): Promise<number> {
|
||||
const root = findRepoRoot(ctx.cwd);
|
||||
if (!root) {
|
||||
ctx.stderr.write("\u2717 not inside a nextcraft clone (no pnpm-workspace.yaml found upwards)\n");
|
||||
ctx.stderr.write(" hint: run from the repo, or clone first:\n");
|
||||
ctx.stderr.write(" git clone https://git.coreci.dev/coreci/nextcraft.git && cd nextcraft\n");
|
||||
return 1;
|
||||
}
|
||||
const aiDir = join(root, "apps/ai-service");
|
||||
|
||||
hr("nextcraft bootstrap — monorepo setup", ctx);
|
||||
|
||||
info("preflight: checking prerequisites (doctor)...", ctx);
|
||||
const preflight = await doctor([], ctx);
|
||||
if (preflight !== 0) {
|
||||
ctx.stderr.write("\n\u2717 preflight failed — fix the failed checks above, then re-run nextcraft bootstrap\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
info("installing workspace dependencies (pnpm install)...", ctx);
|
||||
const install = await ctx.spawn("pnpm", ["install"], { cwd: root, timeoutMs: INSTALL_TIMEOUT_MS });
|
||||
if (install.code !== 0) {
|
||||
ctx.stderr.write(`\u2717 pnpm install failed (exit ${install.code})\n`);
|
||||
return 1;
|
||||
}
|
||||
info("workspace dependencies installed", ctx);
|
||||
|
||||
info("bootstrapping ai-service venv (scripts/bootstrap.sh)...", ctx);
|
||||
const boot = await ctx.spawn("bash", ["scripts/bootstrap.sh"], {
|
||||
cwd: aiDir,
|
||||
timeoutMs: INSTALL_TIMEOUT_MS,
|
||||
});
|
||||
if (boot.code !== 0) {
|
||||
ctx.stderr.write(`\u2717 ai-service bootstrap failed (exit ${boot.code})\n`);
|
||||
return 1;
|
||||
}
|
||||
info("ai-service venv ready", ctx);
|
||||
|
||||
const examplePath = join(aiDir, ".env.example");
|
||||
const envPath = join(aiDir, ".env");
|
||||
if (!ctx.exists(envPath) && ctx.exists(examplePath)) {
|
||||
ctx.writeFile(envPath, ctx.readFile(examplePath) ?? "");
|
||||
info("created apps/ai-service/.env from .env.example", ctx);
|
||||
} else if (ctx.exists(envPath)) {
|
||||
info("apps/ai-service/.env already present — kept as-is", ctx);
|
||||
} else {
|
||||
warn("no .env.example found — skipping env setup (pydantic-settings defaults apply)", ctx);
|
||||
}
|
||||
|
||||
if (ctx.exists(examplePath) && ctx.exists(envPath)) {
|
||||
const diff = diffEnvTemplate(ctx.readFile(examplePath) ?? "", ctx.readFile(envPath) ?? "");
|
||||
if (diff.missing.length > 0) {
|
||||
warn(
|
||||
`${diff.missing.length} optional key(s) unset in .env: ${diff.missing.join(", ")}`,
|
||||
ctx,
|
||||
);
|
||||
info("optional keys warn only — mock providers keep the stack runnable without them", ctx);
|
||||
}
|
||||
if (diff.extra.length > 0) {
|
||||
info(`extra keys in .env (kept): ${diff.extra.join(", ")}`, ctx);
|
||||
}
|
||||
if (diff.missing.length === 0 && diff.extra.length === 0) {
|
||||
info(".env covers all template keys", ctx);
|
||||
}
|
||||
}
|
||||
|
||||
ctx.stdout.write("\n\u2713 bootstrap complete\n\nNext steps:\n nextcraft verify\n nextcraft dev\n\n");
|
||||
return 0;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { existsSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import type { Ctx } from "../ctx.js";
|
||||
import { findRepoRoot } from "../lib/repo.js";
|
||||
import { daemonPaths, daemonRunning, spawnDetached, writePidFile } from "../lib/daemon.js";
|
||||
|
||||
export async function dev(args: string[], ctx: Ctx): Promise<number> {
|
||||
const root = findRepoRoot(ctx.cwd);
|
||||
if (!root) {
|
||||
ctx.stderr.write("✗ not inside a nextcraft clone — run from the repo root or a subdirectory\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
const detach = args.includes("--detach") || args.includes("-d");
|
||||
const unknown = args.filter((a) => a !== "--detach" && a !== "-d");
|
||||
if (unknown.length > 0) {
|
||||
ctx.stderr.write(`unknown flag for dev: ${unknown.join(", ")}\n`);
|
||||
return 2;
|
||||
}
|
||||
|
||||
const aiDir = join(root, "apps", "ai-service");
|
||||
const paths = daemonPaths(root);
|
||||
|
||||
// v0.3.6 single-port deploy: when the web export exists, serve it from the
|
||||
// API port too (AI_WEB_STATIC_DIR → dev.sh passes env through to uvicorn).
|
||||
// `pnpm build && nextcraft dev` = full site on :8420, zero extra config.
|
||||
const webOut = join(root, "apps", "web", "out");
|
||||
const env = { ...ctx.env } as Record<string, string | undefined>;
|
||||
if (existsSync(webOut) && !env.AI_WEB_STATIC_DIR) {
|
||||
env.AI_WEB_STATIC_DIR = webOut;
|
||||
}
|
||||
|
||||
if (detach) {
|
||||
const running = daemonRunning(paths);
|
||||
if (running !== undefined) {
|
||||
ctx.stderr.write(`✗ dev already running as daemon (pid ${running}) — 'nextcraft stop' first, or run without -d\n`);
|
||||
return 1;
|
||||
}
|
||||
const child = spawnDetached("bash", ["scripts/dev.sh"], aiDir, paths.logFile, env);
|
||||
writePidFile(paths, child.pid!);
|
||||
ctx.stdout.write(`started ai-service dev server as daemon (pid ${child.pid})\n`);
|
||||
ctx.stdout.write(` log: ${paths.logFile} (nextcraft log -f to follow)\n`);
|
||||
ctx.stdout.write(` stop: nextcraft stop\n`);
|
||||
if (env.AI_WEB_STATIC_DIR) {
|
||||
ctx.stdout.write(` web UI served same-origin from ${env.AI_WEB_STATIC_DIR}\n`);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
ctx.stdout.write("nextcraft dev — ai-service dev server (web dev server: run `pnpm dev` separately)\n\n");
|
||||
const child = spawn("bash", ["scripts/dev.sh"], {
|
||||
cwd: aiDir,
|
||||
stdio: "inherit",
|
||||
env: env as NodeJS.ProcessEnv,
|
||||
});
|
||||
const forward = (sig: NodeJS.Signals) => () => child.kill(sig);
|
||||
process.on("SIGINT", forward("SIGINT"));
|
||||
process.on("SIGTERM", forward("SIGTERM"));
|
||||
return await new Promise<number>((resolve) => {
|
||||
child.on("close", (code) => resolve(code ?? 1));
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
import type { Ctx } from "../ctx.js";
|
||||
import { compareVersions } from "../checks/check-command.js";
|
||||
import { ok, fail, hr, summary } from "../lib/log.js";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
interface CheckOutcome {
|
||||
name: string;
|
||||
passed: boolean;
|
||||
}
|
||||
|
||||
export async function doctor(_args: string[], ctx: Ctx): Promise<number> {
|
||||
hr("nextcraft doctor — environment prerequisites", ctx);
|
||||
const results: CheckOutcome[] = [];
|
||||
|
||||
results.push(await checkNode(ctx));
|
||||
results.push(await checkProgram(ctx, "pnpm", "8", "install pnpm via corepack: corepack enable pnpm (or: npm i -g pnpm)"));
|
||||
results.push(await checkProgram(ctx, "python3", "3.11", "install python3 >= 3.11 (e.g. apt install python3 python3-venv)"));
|
||||
results.push(await checkProgram(ctx, "git", undefined, "install git: https://git-scm.com/download/linux"));
|
||||
results.push(await checkUnshare(ctx));
|
||||
results.push(await checkVenvCapability(ctx));
|
||||
|
||||
const passed = results.filter((r) => r.passed).length;
|
||||
const failed = results.length - passed;
|
||||
summary(passed, failed, ctx);
|
||||
return failed === 0 ? 0 : 1;
|
||||
}
|
||||
|
||||
async function checkVenvCapability(ctx: Ctx): Promise<CheckOutcome> {
|
||||
const probeDir = mkdtempSync(join(tmpdir(), "nc-venv-probe-"));
|
||||
try {
|
||||
const full = await ctx.spawn("python3", ["-m", "venv", join(probeDir, "v")], {
|
||||
capture: true,
|
||||
timeoutMs: 60_000,
|
||||
});
|
||||
if (full.code === 0) return venvOk(ctx, "full venv");
|
||||
const pipless = await ctx.spawn("python3", ["-m", "venv", "--without-pip", join(probeDir, "v2")], {
|
||||
capture: true,
|
||||
timeoutMs: 60_000,
|
||||
});
|
||||
if (pipless.code === 0) {
|
||||
ok("python3 venv capability (fallback path: pip bootstrapped via get-pip)", ctx);
|
||||
return { name: "venv", passed: true };
|
||||
}
|
||||
return venvFail(ctx);
|
||||
} finally {
|
||||
await ctx.spawn("rm", ["-rf", probeDir], { capture: true, timeoutMs: 30_000 });
|
||||
}
|
||||
}
|
||||
|
||||
function venvOk(ctx: Ctx, mode: string): CheckOutcome {
|
||||
ok(`python3 venv capability (${mode})`, ctx);
|
||||
return { name: "venv", passed: true };
|
||||
}
|
||||
|
||||
function venvFail(ctx: Ctx): CheckOutcome {
|
||||
fail(
|
||||
"python3 cannot create virtual environments (python3 -m venv fails)",
|
||||
"Debian/Ubuntu: apt install python<version>-venv (e.g. python3.12-venv for python 3.12) — nextcraft bootstrap needs it; then re-run nextcraft doctor",
|
||||
ctx,
|
||||
);
|
||||
return { name: "venv", passed: false };
|
||||
}
|
||||
|
||||
async function checkNode(ctx: Ctx): Promise<CheckOutcome> {
|
||||
const version = process.version;
|
||||
if (compareVersions(version, "18") >= 0) {
|
||||
ok(`node ${version} (>= 18)`, ctx);
|
||||
return { name: "node", passed: true };
|
||||
}
|
||||
fail(`node ${version} is older than 18`, "install node >= 18 (https://nodejs.org)", ctx);
|
||||
return { name: "node", passed: false };
|
||||
}
|
||||
|
||||
async function checkProgram(
|
||||
ctx: Ctx,
|
||||
name: string,
|
||||
minVersion: string | undefined,
|
||||
hint: string,
|
||||
): Promise<CheckOutcome> {
|
||||
const which = await ctx.spawn("which", [name], { capture: true, timeoutMs: 3000 });
|
||||
if (which.code !== 0) {
|
||||
fail(`${name} not found on PATH`, hint, ctx);
|
||||
return { name, passed: false };
|
||||
}
|
||||
let version: string | undefined;
|
||||
if (minVersion) {
|
||||
const probe = await ctx.spawn(name, ["--version"], { capture: true, timeoutMs: 10000 });
|
||||
version = probe.stdout.trim().split("\n")[0]?.trim();
|
||||
if (probe.code !== 0 || !version || compareVersions(version, minVersion) < 0) {
|
||||
fail(
|
||||
`${name} ${version ?? "(unknown version)"} is older than required ${minVersion}`,
|
||||
hint,
|
||||
ctx,
|
||||
);
|
||||
return { name, passed: false };
|
||||
}
|
||||
ok(`${name} ${version} (>= ${minVersion})`, ctx);
|
||||
return { name, passed: true };
|
||||
}
|
||||
const probe = await ctx.spawn(name, ["--version"], { capture: true, timeoutMs: 10000 });
|
||||
version = probe.stdout.trim().split("\n")[0]?.trim();
|
||||
ok(`${name} ${version ?? ""}`.trim(), ctx);
|
||||
return { name, passed: true };
|
||||
}
|
||||
|
||||
async function checkUnshare(ctx: Ctx): Promise<CheckOutcome> {
|
||||
const which = await ctx.spawn("which", ["unshare"], { capture: true, timeoutMs: 3000 });
|
||||
if (which.code === 0) {
|
||||
ok("unshare available (sandbox fabric ready)", ctx);
|
||||
return { name: "unshare", passed: true };
|
||||
}
|
||||
fail(
|
||||
"unshare not found on PATH",
|
||||
"sandbox fabric needs unshare (util-linux) — credential builds degrade without it: apt install util-linux",
|
||||
ctx,
|
||||
);
|
||||
return { name: "unshare", passed: false };
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
|
||||
declare const NEXTCRAFT_VERSION_STAMP: string | undefined;
|
||||
|
||||
export function version(env?: Record<string, string | undefined>): string {
|
||||
if (typeof NEXTCRAFT_VERSION_STAMP !== "undefined") {
|
||||
return NEXTCRAFT_VERSION_STAMP;
|
||||
}
|
||||
return env?.NEXTCRAFT_VERSION ?? "0.0.0-dev";
|
||||
}
|
||||
|
||||
export function helpText(env?: Record<string, string | undefined>): string {
|
||||
return `nextcraft ${version(env)} — bootstrap CLI for the Nextcraft monorepo
|
||||
|
||||
Usage:
|
||||
nextcraft <command> [flags]
|
||||
|
||||
Commands:
|
||||
doctor check environment prerequisites (node, pnpm, python3, git, unshare)
|
||||
bootstrap set up a fresh clone: pnpm install, ai-service venv, .env from template
|
||||
verify health-check the bootstrapped stack (venv, uvicorn, ports, env)
|
||||
dev run the ai-service dev server (thin passthrough to scripts/dev.sh)
|
||||
dev -d run it as a daemon (detached, log + pidfile under ~/.nextcraft/run/)
|
||||
stop stop the daemon started by 'dev -d'
|
||||
log show the daemon log (last 50 lines; -n N to choose, -f to follow)
|
||||
|
||||
Flags:
|
||||
--help, -h show this help
|
||||
--version print the CLI version
|
||||
|
||||
Exit codes:
|
||||
0 success
|
||||
1 a check or step failed (see the printed hint)
|
||||
2 usage error (unknown command or flag)
|
||||
`;
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
import { existsSync } from "node:fs";
|
||||
import { spawn } from "node:child_process";
|
||||
import type { Ctx } from "../ctx.js";
|
||||
import { findRepoRoot } from "../lib/repo.js";
|
||||
import { daemonPaths } from "../lib/daemon.js";
|
||||
|
||||
export async function log(args: string[], ctx: Ctx): Promise<number> {
|
||||
const follow = args.includes("-f") || args.includes("--follow");
|
||||
const rest = args.filter((a) => a !== "-f" && a !== "--follow");
|
||||
let lines = 50;
|
||||
for (let i = 0; i < rest.length; i++) {
|
||||
const a = rest[i];
|
||||
if (a === "-n") {
|
||||
const val = rest[++i];
|
||||
if (val === undefined || !/^\d+$/.test(val)) {
|
||||
ctx.stderr.write(`-n requires a line count (e.g. nextcraft log -n 200)\n`);
|
||||
return 2;
|
||||
}
|
||||
lines = Number.parseInt(val, 10);
|
||||
} else if (a.startsWith("-n") && /^\d+$/.test(a.slice(2))) {
|
||||
lines = Number.parseInt(a.slice(2), 10);
|
||||
} else if (/^\d+$/.test(a)) {
|
||||
lines = Number.parseInt(a, 10);
|
||||
} else {
|
||||
ctx.stderr.write(`unknown flag for log: ${a}\n`);
|
||||
return 2;
|
||||
}
|
||||
}
|
||||
|
||||
const root = findRepoRoot(ctx.cwd);
|
||||
if (!root) {
|
||||
ctx.stderr.write("✗ not inside a nextcraft clone — run from the repo root or a subdirectory\n");
|
||||
return 1;
|
||||
}
|
||||
const paths = daemonPaths(root);
|
||||
if (!existsSync(paths.logFile)) {
|
||||
ctx.stderr.write(`✗ no dev log at ${paths.logFile} — start the daemon first (nextcraft dev -d)\n`);
|
||||
return 1;
|
||||
}
|
||||
|
||||
const tailArgs = follow ? ["-n", String(lines), "-F", paths.logFile] : ["-n", String(lines), paths.logFile];
|
||||
if (follow) {
|
||||
const child = spawn("tail", tailArgs, { stdio: "inherit" });
|
||||
const forward = (sig: NodeJS.Signals) => () => child.kill(sig);
|
||||
process.on("SIGINT", forward("SIGINT"));
|
||||
process.on("SIGTERM", forward("SIGTERM"));
|
||||
return await new Promise<number>((resolve) => {
|
||||
child.on("close", (code) => resolve(code ?? 1));
|
||||
});
|
||||
}
|
||||
const { readFileSync } = await import("node:fs");
|
||||
const content = readFileSync(paths.logFile, "utf8");
|
||||
// splitlines-style: a trailing \n must not produce a phantom empty line
|
||||
const all = content.endsWith("\n") ? content.slice(0, -1).split("\n") : content.split("\n");
|
||||
const slice = all.slice(Math.max(0, all.length - lines)).join("\n");
|
||||
ctx.stdout.write(slice === "" ? "" : slice + "\n");
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
import { existsSync, unlinkSync } from "node:fs";
|
||||
import type { Ctx } from "../ctx.js";
|
||||
import { findRepoRoot } from "../lib/repo.js";
|
||||
import { daemonPaths, daemonRunning, readDaemonPid, stopDaemon } from "../lib/daemon.js";
|
||||
|
||||
export async function stop(_args: string[], ctx: Ctx): Promise<number> {
|
||||
if (_args.length > 0) {
|
||||
ctx.stderr.write("stop takes no flags\n");
|
||||
return 2;
|
||||
}
|
||||
const root = findRepoRoot(ctx.cwd);
|
||||
if (!root) {
|
||||
ctx.stderr.write("✗ not inside a nextcraft clone — run from the repo root or a subdirectory\n");
|
||||
return 1;
|
||||
}
|
||||
const paths = daemonPaths(root);
|
||||
const running = daemonRunning(paths);
|
||||
if (running !== undefined) {
|
||||
ctx.stdout.write(`stopping dev daemon (pid ${running})…\n`);
|
||||
const died = await stopDaemon(running);
|
||||
if (!died) {
|
||||
ctx.stderr.write(`✗ pid ${running} did not stop — kill it manually\n`);
|
||||
return 1;
|
||||
}
|
||||
} else if (readDaemonPid(paths) !== undefined) {
|
||||
ctx.stdout.write("cleaning stale pidfile (process not running)\n");
|
||||
} else {
|
||||
ctx.stdout.write("no dev daemon running\n");
|
||||
}
|
||||
if (existsSync(paths.pidFile)) {
|
||||
unlinkSync(paths.pidFile);
|
||||
}
|
||||
ctx.stdout.write("stopped\n");
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
import { join } from "node:path";
|
||||
import type { Ctx } from "../ctx.js";
|
||||
import { findRepoRoot } from "../lib/repo.js";
|
||||
import { hr, ok, fail, warn, summary } from "../lib/log.js";
|
||||
|
||||
export async function verify(_args: string[], ctx: Ctx): Promise<number> {
|
||||
const root = findRepoRoot(ctx.cwd);
|
||||
if (!root) {
|
||||
ctx.stderr.write("\u2717 not inside a nextcraft clone — run from the repo root or a subdirectory\n");
|
||||
return 1;
|
||||
}
|
||||
const aiDir = join(root, "apps/ai-service");
|
||||
hr("nextcraft verify — stack health check", ctx);
|
||||
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
|
||||
const venvPython = join(aiDir, ".venv/bin/python3");
|
||||
if (ctx.exists(venvPython)) {
|
||||
const imp = await ctx.spawn(venvPython, ["-c", "import ai_service"], {
|
||||
capture: true,
|
||||
timeoutMs: 30_000,
|
||||
});
|
||||
if (imp.code === 0) {
|
||||
ok("ai-service venv — import ai_service", ctx);
|
||||
passed++;
|
||||
} else {
|
||||
fail("ai_service no longer imports in the venv", "re-run nextcraft bootstrap", ctx);
|
||||
failed++;
|
||||
}
|
||||
const uv = await ctx.spawn(venvPython, ["-c", "import uvicorn"], {
|
||||
capture: true,
|
||||
timeoutMs: 30_000,
|
||||
});
|
||||
if (uv.code === 0) {
|
||||
ok("uvicorn importable in venv", ctx);
|
||||
passed++;
|
||||
} else {
|
||||
fail("uvicorn missing in venv", "re-run nextcraft bootstrap", ctx);
|
||||
failed++;
|
||||
}
|
||||
} else {
|
||||
fail("ai-service venv not found", "run nextcraft bootstrap", ctx);
|
||||
failed += 2;
|
||||
}
|
||||
|
||||
const envPath = join(aiDir, ".env");
|
||||
if (ctx.exists(envPath)) {
|
||||
ok(".env present", ctx);
|
||||
passed++;
|
||||
} else {
|
||||
warn(".env absent — pydantic-settings defaults apply (copy apps/ai-service/.env.example to customize)", ctx);
|
||||
}
|
||||
|
||||
const port = parsePort(ctx.readFile(envPath) ?? "") ?? 8420;
|
||||
if (await ctx.portFree(port, "127.0.0.1")) {
|
||||
ok(`port ${port} free (ai-service will bind it)`, ctx);
|
||||
passed++;
|
||||
} else {
|
||||
fail(`port ${port} is busy`, `stop the process listening on ${port} (ai-service default)`, ctx);
|
||||
failed++;
|
||||
}
|
||||
|
||||
if (ctx.exists(join(root, "node_modules/.bin/turbo"))) {
|
||||
ok("workspace dependencies installed (node_modules present)", ctx);
|
||||
passed++;
|
||||
} else {
|
||||
fail("node_modules missing at repo root", "run nextcraft bootstrap", ctx);
|
||||
failed++;
|
||||
}
|
||||
|
||||
summary(passed, failed, ctx);
|
||||
return failed === 0 ? 0 : 1;
|
||||
}
|
||||
|
||||
function parsePort(envContent: string): number | undefined {
|
||||
for (const line of envContent.split("\n")) {
|
||||
const m = line.match(/^\s*AI_PORT\s*=\s*(\d+)\s*$/);
|
||||
if (m) return parseInt(m[1], 10);
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { SpawnResult } from "./lib/spawn.js";
|
||||
|
||||
export interface Ctx {
|
||||
cwd: string;
|
||||
env: Record<string, string | undefined>;
|
||||
stdout: { write(s: string): void };
|
||||
stderr: { write(s: string): void };
|
||||
spawn: (cmd: string, args: string[], opts: SpawnOpts) => Promise<SpawnResult>;
|
||||
exists: (p: string) => boolean;
|
||||
readFile: (p: string) => string | undefined;
|
||||
writeFile: (p: string, content: string) => void;
|
||||
portFree: (port: number, host: string) => Promise<boolean>;
|
||||
}
|
||||
|
||||
export interface SpawnOpts {
|
||||
timeoutMs?: number;
|
||||
cwd?: string;
|
||||
env?: Record<string, string | undefined>;
|
||||
stdio?: "inherit" | "pipe";
|
||||
capture?: boolean;
|
||||
}
|
||||
|
||||
export type Command = (args: string[], ctx: Ctx) => Promise<number>;
|
||||
@@ -0,0 +1,82 @@
|
||||
import type { Ctx, Command } from "./ctx.js";
|
||||
import { helpText, version } from "./commands/help.js";
|
||||
import { doctor } from "./commands/doctor.js";
|
||||
import { bootstrap } from "./commands/bootstrap.js";
|
||||
import { verify } from "./commands/verify.js";
|
||||
import { dev } from "./commands/dev.js";
|
||||
import { stop } from "./commands/stop.js";
|
||||
import { log } from "./commands/log.js";
|
||||
import { existsSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import net from "node:net";
|
||||
|
||||
interface NodeRequire {
|
||||
(id: string): unknown;
|
||||
}
|
||||
|
||||
const commands: Record<string, Command> = { doctor, bootstrap, verify, dev, stop, log };
|
||||
|
||||
export async function run(argv: string[], ctx: Ctx): Promise<number> {
|
||||
const flags = argv.filter((a) => a.startsWith("--") || a === "-h");
|
||||
const positional = argv.filter((a) => !(a.startsWith("--") || a === "-h"));
|
||||
|
||||
for (const flag of flags) {
|
||||
if (flag === "--help" || flag === "-h") {
|
||||
ctx.stdout.write(helpText(ctx.env));
|
||||
return 0;
|
||||
}
|
||||
if (flag === "--version") {
|
||||
ctx.stdout.write(`${version(ctx.env)}\n`);
|
||||
return 0;
|
||||
}
|
||||
ctx.stderr.write(`unknown flag: ${flag}\n\n${helpText(ctx.env)}`);
|
||||
return 2;
|
||||
}
|
||||
|
||||
const name = positional[0];
|
||||
const command = name ? commands[name] : undefined;
|
||||
if (!command) {
|
||||
const text = helpText(ctx.env);
|
||||
ctx.stderr.write(name ? `unknown command: ${name}\n\n${text}` : text);
|
||||
return 2;
|
||||
}
|
||||
return await command(positional.slice(1), ctx);
|
||||
}
|
||||
|
||||
declare const require: NodeRequire;
|
||||
|
||||
function isDirectRun(): boolean {
|
||||
if (process.env.NODE_TEST_CONTEXT) return false;
|
||||
const argv1 = process.argv[1];
|
||||
if (argv1?.endsWith("dist/index.js") || argv1?.endsWith("src/index.ts")) return true;
|
||||
try {
|
||||
const sea = require("node:sea") as { isSea?: () => boolean };
|
||||
return typeof sea.isSea === "function" && sea.isSea();
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (isDirectRun()) {
|
||||
void (async () => {
|
||||
const realCtx: Ctx = {
|
||||
cwd: process.cwd(),
|
||||
env: process.env,
|
||||
stdout: process.stdout,
|
||||
stderr: process.stderr,
|
||||
spawn: async (cmd, args, opts) => {
|
||||
const { run: spawnRun } = await import("./lib/spawn.js");
|
||||
return spawnRun(cmd, args, opts);
|
||||
},
|
||||
exists: existsSync,
|
||||
readFile: (p) => (existsSync(p) ? readFileSync(p, "utf8") : undefined),
|
||||
writeFile: (p, c) => writeFileSync(p, c),
|
||||
portFree: async (port, host) =>
|
||||
new Promise<boolean>((resolve) => {
|
||||
const srv = net.createServer();
|
||||
srv.once("error", () => resolve(false));
|
||||
srv.once("listening", () => srv.close(() => resolve(true)));
|
||||
srv.listen(port, host);
|
||||
}),
|
||||
};
|
||||
process.exitCode = await run(process.argv.slice(2), realCtx);
|
||||
})();
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
// v0.3.6 unattended dev: daemon state lives under ~/.nextcraft/run/<hash>/
|
||||
// (home, NOT the repo — same state-root directive as the DB default), where
|
||||
// <hash> is a short sha256 of the absolute repo path so multiple clones
|
||||
// each get their own pidfile + log. Stdlib only (D-033 constraints).
|
||||
|
||||
import { spawn, type ChildProcess } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { closeSync, existsSync, mkdirSync, openSync, readFileSync, statSync, writeFileSync } from "node:fs";
|
||||
import { homedir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
|
||||
export interface DaemonPaths {
|
||||
dir: string;
|
||||
pidFile: string;
|
||||
logFile: string;
|
||||
}
|
||||
|
||||
export function daemonPaths(repoRoot: string): DaemonPaths {
|
||||
const digest = createHash("sha256").update(resolve(repoRoot)).digest("hex");
|
||||
const dir = join(homedir(), ".nextcraft", "run", digest.slice(0, 16));
|
||||
return { dir, pidFile: join(dir, "dev.pid"), logFile: join(dir, "dev.log") };
|
||||
}
|
||||
|
||||
export function readDaemonPid(paths: DaemonPaths): number | undefined {
|
||||
if (!existsSync(paths.pidFile)) return undefined;
|
||||
const raw = readFileSync(paths.pidFile, "utf8").trim();
|
||||
const pid = Number.parseInt(raw, 10);
|
||||
return Number.isInteger(pid) && pid > 0 ? pid : undefined;
|
||||
}
|
||||
|
||||
export function pidAlive(pid: number): boolean {
|
||||
try {
|
||||
// lstat-style existence check via /proc; throws when the pid is gone.
|
||||
statSync(`/proc/${pid}`);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function daemonRunning(paths: DaemonPaths): number | undefined {
|
||||
const pid = readDaemonPid(paths);
|
||||
if (pid === undefined) return undefined;
|
||||
return pidAlive(pid) ? pid : undefined;
|
||||
}
|
||||
|
||||
export function writePidFile(paths: DaemonPaths, pid: number): void {
|
||||
mkdirSync(paths.dir, { recursive: true });
|
||||
writeFileSync(paths.pidFile, `${pid}\n`);
|
||||
}
|
||||
|
||||
/** Stop a daemon pid: SIGTERM, then SIGKILL after graceMs. Returns true if it died. */
|
||||
export function stopDaemon(
|
||||
pid: number,
|
||||
graceMs = 5000,
|
||||
kill: (pid: number, sig: NodeJS.Signals) => void = process.kill.bind(process),
|
||||
sleep: (ms: number) => Promise<void> = (ms) => new Promise((r) => setTimeout(r, ms)),
|
||||
): Promise<boolean> {
|
||||
try {
|
||||
kill(pid, "SIGTERM");
|
||||
} catch {
|
||||
return Promise.resolve(true); // already dead
|
||||
}
|
||||
const deadline = Date.now() + graceMs;
|
||||
const poll = (): Promise<boolean> =>
|
||||
sleep(200).then(() => {
|
||||
if (!pidAlive(pid)) return true;
|
||||
if (Date.now() >= deadline) {
|
||||
try {
|
||||
kill(pid, "SIGKILL");
|
||||
} catch {
|
||||
/* dead */
|
||||
}
|
||||
return sleep(200).then(() => !pidAlive(pid));
|
||||
}
|
||||
return poll();
|
||||
});
|
||||
return poll();
|
||||
}
|
||||
|
||||
/** Spawn dev.sh detached with output appended to the daemon log. */
|
||||
export function spawnDetached(
|
||||
script: string,
|
||||
args: string[],
|
||||
cwd: string,
|
||||
logFile: string,
|
||||
env: Record<string, string | undefined>,
|
||||
): ChildProcess {
|
||||
mkdirSync(dirname(logFile), { recursive: true });
|
||||
const fd = openSync(logFile, "a");
|
||||
const child = spawn(script, args, {
|
||||
cwd,
|
||||
env: env as NodeJS.ProcessEnv,
|
||||
detached: true,
|
||||
stdio: ["ignore", fd, fd],
|
||||
});
|
||||
child.unref();
|
||||
closeSync(fd);
|
||||
return child;
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { Ctx } from "../ctx.js";
|
||||
|
||||
export function ok(msg: string, ctx: Ctx): void {
|
||||
ctx.stdout.write(` \u2713 ${msg}\n`);
|
||||
}
|
||||
|
||||
export function fail(msg: string, hint: string | undefined, ctx: Ctx): void {
|
||||
ctx.stderr.write(` \u2717 ${msg}\n`);
|
||||
if (hint) ctx.stderr.write(` hint: ${hint}\n`);
|
||||
}
|
||||
|
||||
export function warn(msg: string, ctx: Ctx): void {
|
||||
ctx.stdout.write(` \u26a0 ${msg}\n`);
|
||||
}
|
||||
|
||||
export function info(msg: string, ctx: Ctx): void {
|
||||
ctx.stdout.write(` - ${msg}\n`);
|
||||
}
|
||||
|
||||
export function hr(title: string, ctx: Ctx): void {
|
||||
ctx.stdout.write(`\n${title}\n\n`);
|
||||
}
|
||||
|
||||
export function summary(passed: number, failed: number, ctx: Ctx): void {
|
||||
const line = failed === 0 ? "All checks passed" : `${failed} check(s) failed, ${passed} passed`;
|
||||
ctx.stdout.write(`\n${failed === 0 ? "\u2713" : "\u2717"} ${line}\n`);
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
import { existsSync } from "node:fs";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import type { SpawnResult } from "../lib/spawn.js";
|
||||
|
||||
export function findRepoRoot(start: string): string | undefined {
|
||||
let dir = resolve(start);
|
||||
for (;;) {
|
||||
if (existsSync(join(dir, "pnpm-workspace.yaml"))) return dir;
|
||||
const parent = dirname(dir);
|
||||
if (parent === dir) return undefined;
|
||||
dir = parent;
|
||||
}
|
||||
}
|
||||
|
||||
export type Spawn = (cmd: string, args: string[], opts: Record<string, unknown>) => Promise<SpawnResult>;
|
||||
@@ -0,0 +1,51 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import type { SpawnOpts } from "../ctx.js";
|
||||
|
||||
export interface SpawnResult {
|
||||
code: number;
|
||||
signal: NodeJS.Signals | null;
|
||||
timedOut: boolean;
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
}
|
||||
|
||||
export async function run(
|
||||
cmd: string,
|
||||
args: string[],
|
||||
opts: SpawnOpts = {},
|
||||
): Promise<SpawnResult> {
|
||||
return await new Promise((resolve) => {
|
||||
const child = spawn(cmd, args, {
|
||||
cwd: opts.cwd,
|
||||
env: opts.env as NodeJS.ProcessEnv | undefined,
|
||||
stdio: opts.capture ? ["ignore", "pipe", "pipe"] : "inherit",
|
||||
});
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
if (child.stdout) child.stdout.on("data", (d: Buffer) => (stdout += d.toString()));
|
||||
if (child.stderr) child.stderr.on("data", (d: Buffer) => (stderr += d.toString()));
|
||||
let timedOut = false;
|
||||
let killTimer: NodeJS.Timeout | undefined;
|
||||
if (opts.timeoutMs) {
|
||||
killTimer = setTimeout(() => {
|
||||
timedOut = true;
|
||||
child.kill("SIGTERM");
|
||||
killTimer = setTimeout(() => child.kill("SIGKILL"), 1000);
|
||||
}, opts.timeoutMs);
|
||||
}
|
||||
child.on("error", () => {
|
||||
if (killTimer) clearTimeout(killTimer);
|
||||
resolve({ code: 127, signal: null, timedOut: false, stdout, stderr });
|
||||
});
|
||||
child.on("close", (code, signal) => {
|
||||
if (killTimer) clearTimeout(killTimer);
|
||||
resolve({
|
||||
code: code ?? 127,
|
||||
signal,
|
||||
timedOut,
|
||||
stdout,
|
||||
stderr,
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { compareVersions } from "../src/checks/check-command.ts";
|
||||
import { parseEnvKeys, diffEnvTemplate } from "../src/checks/check-env.ts";
|
||||
|
||||
test("compareVersions: equal across patch omission", () => {
|
||||
assert.equal(compareVersions("18.0.0", "18"), 0);
|
||||
assert.equal(compareVersions("18", "18.0.0"), 0);
|
||||
});
|
||||
|
||||
test("compareVersions: minor boundary", () => {
|
||||
assert.equal(compareVersions("3.11", "3.11.2"), 0);
|
||||
assert.equal(compareVersions("3.10", "3.11"), -1);
|
||||
assert.equal(compareVersions("3.12", "3.11"), 1);
|
||||
});
|
||||
|
||||
test("compareVersions: v-prefix and embedded version strings", () => {
|
||||
assert.ok(compareVersions("v18.2.0", "18") >= 0);
|
||||
assert.equal(compareVersions("Python 3.11.2", "3.11"), 0);
|
||||
assert.ok(compareVersions("python3 (3.9)", "3.11") < 0);
|
||||
});
|
||||
|
||||
test("compareVersions: major win beats minor", () => {
|
||||
assert.equal(compareVersions("24.0.0", "18.99"), 1);
|
||||
assert.equal(compareVersions("2.99", "18.0"), -1);
|
||||
});
|
||||
|
||||
test("parseEnvKeys: skips comments and blanks", () => {
|
||||
const content = [
|
||||
"# comment",
|
||||
"",
|
||||
"AI_PORT=8420",
|
||||
" AI_MODEL=gemma4:31b",
|
||||
"#AI_SKIP=1",
|
||||
"AI_OLLAMA_CLOUD_API_KEY=",
|
||||
].join("\n");
|
||||
assert.deepEqual(parseEnvKeys(content), ["AI_PORT", "AI_MODEL", "AI_OLLAMA_CLOUD_API_KEY"]);
|
||||
});
|
||||
|
||||
test("diffEnvTemplate: missing + extra classification", () => {
|
||||
const template = "A=1\nB=2\nC=3\n";
|
||||
const env = "B=2\nD=4\n";
|
||||
const diff = diffEnvTemplate(template, env);
|
||||
assert.deepEqual(diff.missing, ["A", "C"]);
|
||||
assert.deepEqual(diff.extra, ["D"]);
|
||||
});
|
||||
|
||||
test("diffEnvTemplate: full coverage yields empty diff", () => {
|
||||
const template = "A=1\nB=2\n";
|
||||
const env = "A=x\nB=y\n";
|
||||
const diff = diffEnvTemplate(template, env);
|
||||
assert.deepEqual(diff.missing, []);
|
||||
assert.deepEqual(diff.extra, []);
|
||||
});
|
||||
@@ -0,0 +1,218 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, writeFileSync, mkdirSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { doctor } from "../src/commands/doctor.ts";
|
||||
import { bootstrap } from "../src/commands/bootstrap.ts";
|
||||
import { verify } from "../src/commands/verify.ts";
|
||||
import { testCtx } from "./helpers.ts";
|
||||
import type { SpawnResult } from "../src/lib/spawn.ts";
|
||||
|
||||
const r0 = (stdout = ""): SpawnResult => ({ code: 0, signal: null, timedOut: false, stdout, stderr: "" });
|
||||
const r1 = (): SpawnResult => ({ code: 1, signal: null, timedOut: false, stdout: "", stderr: "" });
|
||||
const r = (stdout: string): SpawnResult => r0(stdout);
|
||||
|
||||
test("doctor: all prerequisites present exits 0", async () => {
|
||||
const ctx = testCtx({
|
||||
spawn: async (cmd, args) => {
|
||||
if (cmd === "which") return r0();
|
||||
if (cmd === "pnpm" && args[0] === "--version") return r0("10.0.0\n");
|
||||
if (cmd === "python3" && args[0] === "--version") return r0("Python 3.11.2\n");
|
||||
if (cmd === "git" && args[0] === "--version") return r0("git version 2.39.2\n");
|
||||
return r0();
|
||||
},
|
||||
});
|
||||
const code = await doctor([], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("All checks passed"));
|
||||
});
|
||||
|
||||
test("doctor: missing pnpm fails with hint, exit 1", async () => {
|
||||
const ctx = testCtx({
|
||||
spawn: async (cmd, args) => {
|
||||
if (cmd === "which" && args[0] === "pnpm") return { ...r1(), code: 1 };
|
||||
if (cmd === "which") return r0();
|
||||
if (cmd === "python3") return r0("Python 3.11.2\n");
|
||||
if (cmd === "git") return r0("git version 2.39.2\n");
|
||||
return r0();
|
||||
},
|
||||
});
|
||||
const code = await doctor([], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("pnpm not found"));
|
||||
assert.ok(ctx.err().includes("corepack"));
|
||||
});
|
||||
|
||||
test("doctor: outdated python3 fails with version comparison", async () => {
|
||||
const ctx = testCtx({
|
||||
spawn: async (cmd, args) => {
|
||||
if (cmd === "which") return r0();
|
||||
if (cmd === "python3" && args[0] === "--version") return r0("Python 3.9.0\n");
|
||||
if (cmd === "pnpm") return r0("10.0.0\n");
|
||||
if (cmd === "git") return r0("git version 2.39.2\n");
|
||||
return r0();
|
||||
},
|
||||
});
|
||||
const code = await doctor([], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("3.9"));
|
||||
});
|
||||
|
||||
test("bootstrap: outside a repo fails with clone hint", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "nc-norepo-"));
|
||||
const ctx = testCtx({ cwd: dir });
|
||||
const code = await bootstrap([], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("git clone"));
|
||||
});
|
||||
|
||||
const doctorPassSpawn = async (cmd: string, args: string[]): Promise<SpawnResult> => {
|
||||
if (cmd === "pnpm" && args[0] === "--version") return r("10.0.0\n");
|
||||
if (cmd === "python3" && args[0] === "--version") return r("Python 3.11.2\n");
|
||||
if (cmd === "git") return r("git version 2.39.2\n");
|
||||
if (cmd === "python3" && args[1] === "venv") return r0();
|
||||
return r0();
|
||||
};
|
||||
|
||||
test("bootstrap: step order — preflight, pnpm install, venv bootstrap, env copy", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "nc-repo-"));
|
||||
writeFileSync(join(dir, "pnpm-workspace.yaml"), "packages:\n - apps/*\n");
|
||||
mkdirSync(join(dir, "apps", "ai-service"), { recursive: true });
|
||||
writeFileSync(join(dir, "apps", "ai-service", ".env.example"), "AI_PORT=8420\nAI_KEY=\n");
|
||||
|
||||
const calls: string[] = [];
|
||||
const ctx = testCtx({
|
||||
cwd: dir,
|
||||
spawn: async (cmd, args) => {
|
||||
calls.push(`${cmd} ${args.join(" ")}`);
|
||||
return doctorPassSpawn(cmd, args);
|
||||
},
|
||||
});
|
||||
const code = await bootstrap([], ctx);
|
||||
assert.equal(code, 0);
|
||||
const installIdx = calls.findIndex((c) => c.startsWith("pnpm install"));
|
||||
const venvIdx = calls.findIndex((c) => c.startsWith("bash scripts/bootstrap.sh"));
|
||||
assert.ok(installIdx >= 0, "pnpm install runs");
|
||||
assert.ok(venvIdx > installIdx, "venv bootstrap runs after pnpm install");
|
||||
assert.ok(ctx.exists(join(dir, "apps", "ai-service", ".env")));
|
||||
assert.ok(ctx.out().includes("bootstrap complete"));
|
||||
});
|
||||
|
||||
test("bootstrap: existing .env kept, not overwritten", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "nc-keep-"));
|
||||
writeFileSync(join(dir, "pnpm-workspace.yaml"), "packages:\n - apps/*\n");
|
||||
const aiDir = join(dir, "apps", "ai-service");
|
||||
mkdirSync(aiDir, { recursive: true });
|
||||
writeFileSync(join(aiDir, ".env.example"), "AI_PORT=8420\n");
|
||||
writeFileSync(join(aiDir, ".env"), "AI_PORT=9999\n");
|
||||
|
||||
const ctx = testCtx({ cwd: dir, spawn: doctorPassSpawn });
|
||||
const code = await bootstrap([], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("kept as-is"));
|
||||
assert.equal(ctx.readFile(join(aiDir, ".env")), "AI_PORT=9999\n");
|
||||
});
|
||||
|
||||
test("bootstrap: failing pnpm install aborts before venv step", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "nc-fail-"));
|
||||
writeFileSync(join(dir, "pnpm-workspace.yaml"), "packages:\n - apps/*\n");
|
||||
const calls: string[] = [];
|
||||
const ctx = testCtx({
|
||||
cwd: dir,
|
||||
spawn: async (cmd, args) => {
|
||||
calls.push(`${cmd} ${args.join(" ")}`);
|
||||
if (cmd === "pnpm" && args[0] === "install") return r1();
|
||||
return doctorPassSpawn(cmd, args);
|
||||
},
|
||||
});
|
||||
const code = await bootstrap([], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("pnpm install failed"), ctx.err());
|
||||
assert.equal(calls.filter((c) => c.startsWith("bash scripts/bootstrap.sh")).length, 0, "venv step must not run");
|
||||
});
|
||||
test("verify: busy port fails with stop-the-process hint", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "nc-ver-"));
|
||||
writeFileSync(join(dir, "pnpm-workspace.yaml"), "packages:\n - apps/*\n");
|
||||
const aiDir = join(dir, "apps", "ai-service");
|
||||
mkdirSync(aiDir, { recursive: true });
|
||||
mkdirSync(join(aiDir, ".venv", "bin"), { recursive: true });
|
||||
writeFileSync(join(aiDir, ".venv", "bin", "python3"), "#!/bin/sh\nexit 0\n");
|
||||
writeFileSync(join(aiDir, ".env"), "AI_PORT=8420\n");
|
||||
mkdirSync(join(dir, "node_modules", ".bin"), { recursive: true });
|
||||
writeFileSync(join(dir, "node_modules", ".bin", "turbo"), "#!/bin/sh\n");
|
||||
|
||||
const ctx = testCtx({
|
||||
cwd: dir,
|
||||
spawn: async () => r0(),
|
||||
portFree: async () => false,
|
||||
});
|
||||
const code = await verify([], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("port 8420 is busy"));
|
||||
assert.ok(ctx.err().includes("stop the process listening on 8420"));
|
||||
});
|
||||
|
||||
test("verify: free port passes and reports AI_PORT from .env", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "nc-ver2-"));
|
||||
writeFileSync(join(dir, "pnpm-workspace.yaml"), "packages:\n - apps/*\n");
|
||||
const aiDir = join(dir, "apps", "ai-service");
|
||||
mkdirSync(aiDir, { recursive: true });
|
||||
mkdirSync(join(aiDir, ".venv", "bin"), { recursive: true });
|
||||
writeFileSync(join(aiDir, ".venv", "bin", "python3"), "#!/bin/sh\nexit 0\n");
|
||||
writeFileSync(join(aiDir, ".env"), "AI_PORT=8421\n");
|
||||
mkdirSync(join(dir, "node_modules", ".bin"), { recursive: true });
|
||||
writeFileSync(join(dir, "node_modules", ".bin", "turbo"), "#!/bin/sh\n");
|
||||
|
||||
const ctx = testCtx({
|
||||
cwd: dir,
|
||||
spawn: async () => r0(),
|
||||
portFree: async () => true,
|
||||
});
|
||||
const code = await verify([], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("port 8421 free"));
|
||||
});
|
||||
|
||||
test("bootstrap: preflight failure aborts before pnpm install", async () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "nc-pre-"));
|
||||
writeFileSync(join(dir, "pnpm-workspace.yaml"), "packages:\n - apps/*\n");
|
||||
const calls: string[] = [];
|
||||
const ctx = testCtx({
|
||||
cwd: dir,
|
||||
spawn: async (cmd, args) => {
|
||||
calls.push(`${cmd} ${args.join(" ")}`);
|
||||
if (cmd === "which") return r0();
|
||||
if (cmd === "pnpm" && args[0] === "--version") return r("10.0.0\n");
|
||||
if (cmd === "python3" && args[0] === "--version") return r("Python 3.11.2\n");
|
||||
if (cmd === "git") return r("git version 2.39.2\n");
|
||||
if (cmd === "python3" && args[1] === "venv") return r1();
|
||||
if (cmd === "rm") return r0();
|
||||
return r0();
|
||||
},
|
||||
});
|
||||
const code = await bootstrap([], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("preflight failed"));
|
||||
assert.equal(calls.filter((c) => c.startsWith("pnpm install")).length, 0, "pnpm install must not run when preflight fails");
|
||||
});
|
||||
|
||||
test("doctor: venv-capability probe failure surfaces apt hint", async () => {
|
||||
const ctx = testCtx({
|
||||
spawn: async (cmd, args) => {
|
||||
if (cmd === "which") return r0();
|
||||
if (cmd === "pnpm") return r("10.0.0\n");
|
||||
if (cmd === "python3" && args[0] === "--version") return r("Python 3.12.3\n");
|
||||
if (cmd === "git") return r("git version 2.43.0\n");
|
||||
if (cmd === "node") return r("v24.0.0\n");
|
||||
if (cmd === "python3" && args[1] === "venv") return r1();
|
||||
if (cmd === "rm") return r0();
|
||||
return r0();
|
||||
},
|
||||
});
|
||||
const code = await doctor([], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("cannot create virtual environments"));
|
||||
assert.ok(ctx.err().includes("apt install python"));
|
||||
assert.ok(ctx.err().includes("-venv"));
|
||||
});
|
||||
@@ -0,0 +1,134 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdirSync, rmSync, writeFileSync, existsSync, readFileSync } from "node:fs";
|
||||
import { join, dirname } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { run } from "../src/index.ts";
|
||||
import { testCtx } from "./helpers.ts";
|
||||
import { daemonPaths, pidAlive, readDaemonPid, daemonRunning } from "../src/lib/daemon.ts";
|
||||
|
||||
// Hermetic fixture: a fake repo clone whose dev.sh prints + sleeps briefly,
|
||||
// so daemon lifecycle tests run against a REAL detached child without
|
||||
// touching the developer's actual clone or home state more than needed.
|
||||
function fixtureRepo(): string {
|
||||
const dir = join(tmpdir(), `nc-daemon-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`);
|
||||
mkdirSync(join(dir, "apps", "ai-service", "scripts"), { recursive: true });
|
||||
mkdirSync(join(dir, "apps", "web", "out"), { recursive: true });
|
||||
writeFileSync(join(dir, "pnpm-workspace.yaml"), "packages:\n - packages/*\n");
|
||||
writeFileSync(join(dir, "package.json"), JSON.stringify({ name: "nextcraft", private: true }));
|
||||
// dev.sh: emit a marker line then idle so the daemon stays alive briefly
|
||||
writeFileSync(
|
||||
join(dir, "apps", "ai-service", "scripts", "dev.sh"),
|
||||
"#!/usr/bin/env bash\necho \"dev-started pid=$$\"\nsleep 30\n",
|
||||
{ mode: 0o755 },
|
||||
);
|
||||
writeFileSync(join(dir, "apps", "web", "out", "index.html"), "<html>export</html>");
|
||||
return dir;
|
||||
}
|
||||
|
||||
function cleanup(dir: string) {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
test("daemon lib: paths are keyed by repo path under ~/.nextcraft/run", () => {
|
||||
const a = daemonPaths("/repo/one");
|
||||
const b = daemonPaths("/repo/two");
|
||||
assert.ok(a.pidFile.includes(join(".nextcraft", "run")));
|
||||
assert.notEqual(a.pidFile, b.pidFile);
|
||||
assert.ok(a.pidFile.endsWith("dev.pid"));
|
||||
assert.ok(a.logFile.endsWith("dev.log"));
|
||||
});
|
||||
|
||||
test("daemon lib: pid liveness against a real spawned process", async () => {
|
||||
const { spawn } = await import("node:child_process");
|
||||
const child = spawn("sleep", ["5"]);
|
||||
await new Promise((r) => setTimeout(r, 200));
|
||||
assert.equal(pidAlive(child.pid!), true);
|
||||
child.kill("SIGKILL");
|
||||
await new Promise((r) => setTimeout(r, 300));
|
||||
assert.equal(pidAlive(child.pid!), false);
|
||||
});
|
||||
|
||||
test("dev -d: starts daemon, writes pidfile, auto-wires AI_WEB_STATIC_DIR, refuses double-start", async () => {
|
||||
const repo = fixtureRepo();
|
||||
try {
|
||||
const ctx = testCtx({ cwd: repo });
|
||||
const code = await run(["dev", "-d"], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("started ai-service dev server as daemon"));
|
||||
|
||||
const paths = daemonPaths(repo);
|
||||
assert.ok(existsSync(paths.pidFile), "pidfile written");
|
||||
const pid = readDaemonPid(paths)!;
|
||||
assert.ok(pid! > 0);
|
||||
assert.equal(daemonRunning(paths), pid, "daemon is alive");
|
||||
|
||||
// log should show the marker line from the stub dev.sh
|
||||
await new Promise((r) => setTimeout(r, 500));
|
||||
const logCtx = testCtx({ cwd: repo });
|
||||
assert.equal(await run(["log"], logCtx), 0);
|
||||
assert.ok(logCtx.out().includes("dev-started"), "log tails the daemon output");
|
||||
|
||||
// second detach start must refuse
|
||||
const ctx2 = testCtx({ cwd: repo });
|
||||
assert.equal(await run(["dev", "-d"], ctx2), 1);
|
||||
assert.ok(ctx2.err().includes("already running"));
|
||||
|
||||
// stop kills it and clears the pidfile
|
||||
const stopCtx = testCtx({ cwd: repo });
|
||||
assert.equal(await run(["stop"], stopCtx), 0);
|
||||
assert.ok(stopCtx.out().includes("stopped"));
|
||||
assert.equal(existsSync(paths.pidFile), false, "pidfile removed");
|
||||
assert.equal(pidAlive(pid), false, "process actually dead");
|
||||
} finally {
|
||||
cleanup(repo);
|
||||
}
|
||||
});
|
||||
|
||||
test("dev -d: unknown flag is a usage error", async () => {
|
||||
const repo = fixtureRepo();
|
||||
try {
|
||||
const ctx = testCtx({ cwd: repo });
|
||||
assert.equal(await run(["dev", "--bogus"], ctx), 2);
|
||||
} finally {
|
||||
cleanup(repo);
|
||||
}
|
||||
});
|
||||
|
||||
test("stop: no daemon running is a clean 0-exit", async () => {
|
||||
const repo = fixtureRepo();
|
||||
try {
|
||||
const ctx = testCtx({ cwd: repo });
|
||||
const code = await run(["stop"], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("no dev daemon running"));
|
||||
} finally {
|
||||
cleanup(repo);
|
||||
}
|
||||
});
|
||||
|
||||
test("log: missing log file exits 1 with a hint", async () => {
|
||||
const repo = fixtureRepo();
|
||||
try {
|
||||
const ctx = testCtx({ cwd: repo });
|
||||
const code = await run(["log"], ctx);
|
||||
assert.equal(code, 1);
|
||||
assert.ok(ctx.err().includes("nextcraft dev -d"));
|
||||
} finally {
|
||||
cleanup(repo);
|
||||
}
|
||||
});
|
||||
|
||||
test("log: -n N limits the tail", async () => {
|
||||
const repo = fixtureRepo();
|
||||
try {
|
||||
const paths = daemonPaths(repo);
|
||||
mkdirSync(dirname(paths.logFile), { recursive: true });
|
||||
writeFileSync(paths.logFile, "l1\nl2\nl3\nl4\nl5\n");
|
||||
const ctx = testCtx({ cwd: repo });
|
||||
assert.equal(await run(["log", "-n", "2"], ctx), 0);
|
||||
assert.equal(ctx.out(), "l4\nl5\n");
|
||||
} finally {
|
||||
cleanup(repo);
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,62 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { run } from "../src/index.ts";
|
||||
import { testCtx } from "./helpers.ts";
|
||||
|
||||
|
||||
|
||||
test("dispatch: --help exits 0 and prints usage", async () => {
|
||||
const ctx = testCtx();
|
||||
const code = await run(["--help"], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("Usage:"));
|
||||
assert.ok(ctx.out().includes("doctor"));
|
||||
});
|
||||
|
||||
test("dispatch: -h behaves like --help", async () => {
|
||||
const ctx = testCtx();
|
||||
assert.equal(await run(["-h"], ctx), 0);
|
||||
});
|
||||
|
||||
test("dispatch: --version prints NEXTCRAFT_VERSION override", async () => {
|
||||
const env = { ...process.env, NEXTCRAFT_VERSION: "v9.9.9-test" };
|
||||
const ctx = testCtx({ env });
|
||||
const code = await run(["--version"], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("v9.9.9-test"));
|
||||
});
|
||||
|
||||
test("dispatch: no command exits 2 with usage on stderr", async () => {
|
||||
const ctx = testCtx();
|
||||
assert.equal(await run([], ctx), 2);
|
||||
assert.ok(ctx.err().includes("Usage:") || ctx.out().includes("Usage:") || ctx.err().includes("nextcraft"));
|
||||
});
|
||||
|
||||
test("dispatch: unknown command exits 2", async () => {
|
||||
const ctx = testCtx();
|
||||
const code = await run(["frobnicate"], ctx);
|
||||
assert.equal(code, 2);
|
||||
assert.ok(ctx.err().includes("unknown command: frobnicate"));
|
||||
});
|
||||
|
||||
test("dispatch: unknown flag exits 2", async () => {
|
||||
const ctx = testCtx();
|
||||
assert.equal(await run(["doctor", "--bogus"], ctx), 2);
|
||||
});
|
||||
|
||||
test("dispatch: doctor routes to the doctor command", async () => {
|
||||
const ctx = testCtx({
|
||||
spawn: async (cmd, args) => {
|
||||
if (cmd === "which") return { code: 0, signal: null, timedOut: false, stdout: "", stderr: "" };
|
||||
if (cmd === "pnpm" && args[0] === "--version") return r("10.0.0\n");
|
||||
if (cmd === "python3" && args[0] === "--version") return r("Python 3.11.2\n");
|
||||
if (cmd === "git" && args[0] === "--version") return r("git version 2.39.2\n");
|
||||
return { code: 0, signal: null, timedOut: false, stdout: "", stderr: "" };
|
||||
},
|
||||
});
|
||||
const code = await run(["doctor"], ctx);
|
||||
assert.equal(code, 0);
|
||||
assert.ok(ctx.out().includes("All checks passed"));
|
||||
});
|
||||
|
||||
const r = (stdout: string) => ({ code: 0, signal: null, timedOut: false, stdout, stderr: "" });
|
||||
@@ -0,0 +1,19 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { doctor } from "../src/commands/doctor.ts";
|
||||
import { testCtx } from "./helpers.ts";
|
||||
|
||||
test("doctor integration: real box — all prerequisites present", async () => {
|
||||
const ctx = testCtx();
|
||||
const code = await doctor([], ctx);
|
||||
const output = ctx.out() + ctx.err();
|
||||
if (output.includes("unshare not found")) {
|
||||
console.warn("unshare missing on this box — tolerating its single failure");
|
||||
assert.equal(code, 1);
|
||||
return;
|
||||
}
|
||||
assert.equal(code, 0);
|
||||
assert.ok(output.includes("node"));
|
||||
assert.ok(output.includes("pnpm"));
|
||||
assert.ok(output.includes("python3"));
|
||||
});
|
||||
@@ -0,0 +1,72 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdtempSync, existsSync, readFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, dirname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const cliDir = join(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const repoRoot = join(cliDir, "..", "..");
|
||||
|
||||
test("fresh-clone E2E: clone -> doctor -> bootstrap -> verify (happy path, transcript)", async () => {
|
||||
const work = mkdtempSync(join(tmpdir(), "nc-e2e-"));
|
||||
const cloneDir = join(work, "nextcraft");
|
||||
|
||||
const clone = spawnSync(
|
||||
"git",
|
||||
["clone", "--quiet", "--no-hardlinks", repoRoot, cloneDir],
|
||||
{ encoding: "utf8", timeout: 120000 },
|
||||
);
|
||||
assert.equal(clone.status, 0, `clone failed: ${clone.stderr}`);
|
||||
|
||||
const binary = join(repoRoot, "apps", "cli", "dist", "nextcraft-linux-x64");
|
||||
const cli = existsSync(binary)
|
||||
? binary
|
||||
: join(cliDir, "node_modules", ".bin", "tsx");
|
||||
|
||||
const transcript: string[] = [];
|
||||
const step = (name: string, args: string[], expect: number, timeoutMs: number) => {
|
||||
const { NODE_TEST_CONTEXT, ...restEnv } = process.env as Record<string, string | undefined>;
|
||||
const env = {
|
||||
...restEnv,
|
||||
PATH: [join(process.env.HOME ?? "/home", ".local/bin"), process.env.PATH].filter(Boolean).join(":"),
|
||||
NEXTCRAFT_VERSION: "v-e2e",
|
||||
} as Record<string, string | undefined>;
|
||||
const run = spawnSync(cli, existsSync(binary) ? args : [join(cliDir, "src", "index.ts"), ...args], {
|
||||
cwd: cloneDir,
|
||||
encoding: "utf8",
|
||||
timeout: timeoutMs,
|
||||
env,
|
||||
});
|
||||
transcript.push(`$ nextcraft ${args.join(" ")} -> exit ${run.status}`);
|
||||
if (run.status !== expect) {
|
||||
transcript.push(run.stdout, run.stderr);
|
||||
assert.fail(
|
||||
`${name} exited ${run.status} (expected ${expect})\nTRANSCRIPT:\n${transcript.join("\n")}\nstderr: ${run.stderr}`,
|
||||
);
|
||||
}
|
||||
return run.stdout + run.stderr;
|
||||
};
|
||||
|
||||
const doctorOut = step("doctor", ["doctor"], 0, 60000);
|
||||
assert.ok(doctorOut.includes("node"), "doctor mentions node");
|
||||
assert.ok(doctorOut.includes("pnpm"), "doctor mentions pnpm");
|
||||
|
||||
step("bootstrap", ["bootstrap"], 0, 600000);
|
||||
|
||||
const verifyOut = step("verify", ["verify"], 0, 120000);
|
||||
assert.ok(verifyOut.includes("venv"), "verify covers venv");
|
||||
assert.ok(existsSync(join(cloneDir, "apps", "ai-service", ".env")), ".env created in clone");
|
||||
assert.ok(existsSync(join(cloneDir, "apps", "ai-service", ".venv")), "venv created in clone");
|
||||
assert.ok(
|
||||
existsSync(join(cloneDir, "node_modules", ".pnpm")),
|
||||
"workspace node_modules present",
|
||||
);
|
||||
|
||||
const envExample = readFileSync(join(repoRoot, "apps", "ai-service", ".env.example"), "utf8");
|
||||
const envClone = readFileSync(join(cloneDir, "apps", "ai-service", ".env"), "utf8");
|
||||
assert.equal(envClone, envExample, ".env content matches template");
|
||||
|
||||
console.log("E2E TRANSCRIPT:\n" + transcript.join("\n"));
|
||||
});
|
||||
@@ -0,0 +1,54 @@
|
||||
import type { Ctx } from "../src/ctx.ts";
|
||||
import { run } from "../src/index.ts";
|
||||
import { readFileSync, existsSync, writeFileSync } from "node:fs";
|
||||
import net from "node:net";
|
||||
import { run as spawnRun } from "../src/lib/spawn.ts";
|
||||
import type { SpawnResult } from "../src/lib/spawn.ts";
|
||||
|
||||
export interface TestCtx extends Ctx {
|
||||
out(): string;
|
||||
err(): string;
|
||||
writes: Record<string, string>;
|
||||
}
|
||||
|
||||
export function testCtx(overrides: Partial<Ctx> = {}): TestCtx {
|
||||
const out: string[] = [];
|
||||
const err: string[] = [];
|
||||
const writes: Record<string, string> = {};
|
||||
const base = {
|
||||
cwd: process.cwd(),
|
||||
env: { ...process.env } as Record<string, string | undefined>,
|
||||
stdout: { write: (s: string) => void out.push(s) },
|
||||
stderr: { write: (s: string) => void err.push(s) },
|
||||
spawn: async (cmd: string, args: string[], opts: Parameters<Ctx["spawn"]>[2]) =>
|
||||
spawnRun(cmd, args, opts),
|
||||
exists: (p: string) => existsSync(p),
|
||||
readFile: (p: string) => (existsSync(p) ? readFileSync(p, "utf8") : undefined),
|
||||
writeFile: (p: string, c: string) => {
|
||||
writes[p] = c;
|
||||
writeFileSync(p, c);
|
||||
},
|
||||
portFree: async (port: number, host: string) =>
|
||||
new Promise<boolean>((resolve) => {
|
||||
const srv = net.createServer();
|
||||
srv.once("error", () => resolve(false));
|
||||
srv.once("listening", () => srv.close(() => resolve(true)));
|
||||
srv.listen(port, host);
|
||||
}),
|
||||
out: () => out.join(""),
|
||||
err: () => err.join(""),
|
||||
};
|
||||
const ctx = { ...base, ...overrides } as unknown as TestCtx;
|
||||
(ctx as unknown as { writes: Record<string, string> }).writes = writes;
|
||||
return ctx;
|
||||
}
|
||||
|
||||
export function fakeSpawn(
|
||||
sequence: Array<{ match: (cmd: string, args: string[]) => boolean; result: SpawnResult }>,
|
||||
): Ctx["spawn"] {
|
||||
return async (cmd, args) => {
|
||||
const hit = sequence.find((s) => s.match(cmd, args));
|
||||
if (!hit) throw new Error(`unexpected spawn: ${cmd} ${args.join(" ")}`);
|
||||
return hit.result;
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,204 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { cpSync, mkdirSync, mkdtempSync, writeFileSync, readFileSync, existsSync } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, dirname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const repoRoot = join(dirname(fileURLToPath(import.meta.url)), "..", "..", "..");
|
||||
const installSh = join(repoRoot, "scripts", "install.sh");
|
||||
|
||||
function sha256(p: string): string {
|
||||
return createHash("sha256").update(readFileSync(p)).digest("hex");
|
||||
}
|
||||
|
||||
async function serve(
|
||||
setup: (srvDir: string, base: () => string) => void,
|
||||
): Promise<{ url: string; stop: () => void }> {
|
||||
const srvDir = mkdtempSync(join(tmpdir(), "nc-srv-"));
|
||||
const port = 30000 + Math.floor(Math.random() * 20000);
|
||||
const base = () => `http://127.0.0.1:${port}`;
|
||||
setup(srvDir, base);
|
||||
const child = spawn("python3", ["-m", "http.server", String(port), "--directory", srvDir], {
|
||||
stdio: "ignore",
|
||||
});
|
||||
for (let i = 0; i < 50; i++) {
|
||||
const probe = spawnSync("curl", ["-fsS", `${base()}/api/v1/repos/coreci/nextcraft/releases/latest`], {
|
||||
encoding: "utf8",
|
||||
});
|
||||
if (probe.status === 0) break;
|
||||
await new Promise((r) => setTimeout(r, 100));
|
||||
}
|
||||
return { url: base(), stop: () => child.kill("SIGTERM") };
|
||||
}
|
||||
|
||||
function apiManifestDir(srvDir: string, manifest: object): string {
|
||||
const dir = join(srvDir, "api/v1/repos/coreci/nextcraft/releases");
|
||||
mkdirSync(dir, { recursive: true });
|
||||
writeFileSync(join(dir, "latest"), JSON.stringify(manifest));
|
||||
return dir;
|
||||
}
|
||||
|
||||
async function runInstall(home: string, srvUrl: string) {
|
||||
return spawnSync("sh", [installSh], {
|
||||
env: { ...process.env, HOME: home, NEXTCRAFT_FORGE_BASE: srvUrl, DEST: join(home, "bin") },
|
||||
encoding: "utf8",
|
||||
timeout: 30000,
|
||||
});
|
||||
}
|
||||
|
||||
test("install.sh: checksum mismatch = hard stop, no install", async () => {
|
||||
const home = mkdtempSync(join(tmpdir(), "nc-home-"));
|
||||
const srv = await serve((srvDir, base) => {
|
||||
writeFileSync(join(srvDir, "ping"), "pong");
|
||||
writeFileSync(join(srvDir, "nextcraft-linux-x64"), "#!/bin/sh\necho fake\n");
|
||||
writeFileSync(join(srvDir, "nextcraft-linux-x64.sha256"), "deadbeef nextcraft-linux-x64\n");
|
||||
apiManifestDir(srvDir, {
|
||||
tag_name: "v9.9.9",
|
||||
assets: [
|
||||
{ name: "nextcraft-linux-x64", browser_download_url: `${base()}/nextcraft-linux-x64` },
|
||||
{ name: "nextcraft-linux-x64.sha256", browser_download_url: `${base()}/nextcraft-linux-x64.sha256` },
|
||||
],
|
||||
});
|
||||
});
|
||||
try {
|
||||
const res = await runInstall(home, srv.url);
|
||||
assert.notEqual(res.status, 0);
|
||||
assert.ok(res.stderr.includes("CHECKSUM MISMATCH"));
|
||||
assert.ok(!existsSync(join(home, "bin", "nextcraft")));
|
||||
} finally {
|
||||
srv.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("install.sh: valid checksum installs binary and reports version", async () => {
|
||||
const home = mkdtempSync(join(tmpdir(), "nc-home-ok-"));
|
||||
const srv = await serve((srvDir, base) => {
|
||||
const bin = join(srvDir, "nextcraft-linux-x64");
|
||||
writeFileSync(join(srvDir, "ping"), "pong");
|
||||
writeFileSync(bin, "#!/bin/sh\nif [ \"$1\" = \"--version\" ]; then echo v9.9.9; exit 0; fi\necho v9.9.9-installed\nexit 0\n");
|
||||
writeFileSync(join(srvDir, "nextcraft-linux-x64.sha256"), `${sha256(bin)} nextcraft-linux-x64\n`);
|
||||
apiManifestDir(srvDir, {
|
||||
tag_name: "v9.9.9",
|
||||
assets: [
|
||||
{ name: "nextcraft-linux-x64", browser_download_url: `${base()}/nextcraft-linux-x64` },
|
||||
{ name: "nextcraft-linux-x64.sha256", browser_download_url: `${base()}/nextcraft-linux-x64.sha256` },
|
||||
],
|
||||
});
|
||||
});
|
||||
try {
|
||||
const res = await runInstall(home, srv.url);
|
||||
assert.equal(res.status, 0, res.stderr);
|
||||
assert.ok(existsSync(join(home, "bin", "nextcraft")));
|
||||
const run = spawnSync(join(home, "bin", "nextcraft"), [], { encoding: "utf8" });
|
||||
assert.ok(run.stdout.includes("v9.9.9-installed"));
|
||||
} finally {
|
||||
srv.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("install.sh: version-mismatching binary is rejected (G-102 install-time integrity)", async () => {
|
||||
const home = mkdtempSync(join(tmpdir(), "nc-home-mm-"));
|
||||
const srv = await serve((srvDir, base) => {
|
||||
const bin = join(srvDir, "nextcraft-linux-x64");
|
||||
writeFileSync(bin, "#!/bin/sh\nif [ \"$1\" = \"--version\" ]; then echo v0.0.0-wrong; exit 0; fi\nexit 0\n");
|
||||
writeFileSync(join(srvDir, "nextcraft-linux-x64.sha256"), `${sha256(bin)} nextcraft-linux-x64\n`);
|
||||
apiManifestDir(srvDir, {
|
||||
tag_name: "v9.9.9",
|
||||
assets: [
|
||||
{ name: "nextcraft-linux-x64", browser_download_url: `${base()}/nextcraft-linux-x64` },
|
||||
{ name: "nextcraft-linux-x64.sha256", browser_download_url: `${base()}/nextcraft-linux-x64.sha256` },
|
||||
],
|
||||
});
|
||||
});
|
||||
try {
|
||||
const res = await runInstall(home, srv.url);
|
||||
assert.notEqual(res.status, 0, "mismatched version must not install cleanly");
|
||||
assert.ok(res.stderr.includes("integrity mismatch"), `stderr: ${res.stderr}`);
|
||||
} finally {
|
||||
srv.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("install.sh: silent --version binary is rejected (honesty gate)", async () => {
|
||||
const home = mkdtempSync(join(tmpdir(), "nc-home-silent-"));
|
||||
const srv = await serve((srvDir, base) => {
|
||||
const bin = join(srvDir, "nextcraft-linux-x64");
|
||||
writeFileSync(bin, "#!/bin/sh\nexit 0\n");
|
||||
writeFileSync(join(srvDir, "nextcraft-linux-x64.sha256"), `${sha256(bin)} nextcraft-linux-x64\n`);
|
||||
apiManifestDir(srvDir, {
|
||||
tag_name: "v9.9.9",
|
||||
assets: [
|
||||
{ name: "nextcraft-linux-x64", browser_download_url: `${base()}/nextcraft-linux-x64` },
|
||||
{ name: "nextcraft-linux-x64.sha256", browser_download_url: `${base()}/nextcraft-linux-x64.sha256` },
|
||||
],
|
||||
});
|
||||
});
|
||||
try {
|
||||
const res = await runInstall(home, srv.url);
|
||||
assert.notEqual(res.status, 0, "silent binary must be rejected");
|
||||
assert.ok(res.stderr.includes("produced no output"), `stderr: ${res.stderr}`);
|
||||
assert.ok(!existsSync(join(home, "bin", "nextcraft")) || res.stderr.includes("Do not use"), "must not leave a trusted silent binary");
|
||||
} finally {
|
||||
srv.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("install.sh: release without binary assets degrades to source instructions, exit 0", async () => {
|
||||
const home = mkdtempSync(join(tmpdir(), "nc-home-nb-"));
|
||||
const srv = await serve((srvDir) => {
|
||||
writeFileSync(join(srvDir, "ping"), "pong");
|
||||
apiManifestDir(srvDir, { tag_name: "v0.2.8", assets: [] });
|
||||
});
|
||||
try {
|
||||
const res = await runInstall(home, srv.url);
|
||||
assert.equal(res.status, 0);
|
||||
assert.ok(res.stdout.includes("git clone"));
|
||||
assert.ok(!existsSync(join(home, "bin", "nextcraft")));
|
||||
} finally {
|
||||
srv.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("release-assets.sh: token resolution reads .env* files only — poisoned shell env is never used", async () => {
|
||||
const tmpHome = mkdtempSync(join(tmpdir(), "nc-home-token-"));
|
||||
mkdirSync(join(tmpHome, ".ciagent"), { recursive: true });
|
||||
writeFileSync(join(tmpHome, ".ciagent", ".env.secrets"), "GITEA_TOKEN=real-token-from-file\n");
|
||||
const fakeBin = join(tmpHome, "apps/cli/dist");
|
||||
mkdirSync(fakeBin, { recursive: true });
|
||||
writeFileSync(join(fakeBin, "nextcraft-linux-x64"), "fake-binary-bytes\n");
|
||||
writeFileSync(join(fakeBin, "nextcraft-linux-x64.sha256"), "abc123 nextcraft-linux-x64\n");
|
||||
|
||||
const srv = await serve((srvDir) => {
|
||||
const tagsDir = join(srvDir, "api/v1/repos/coreci/nextcraft/releases/tags");
|
||||
mkdirSync(tagsDir, { recursive: true });
|
||||
writeFileSync(join(tagsDir, "v0.3.2"), JSON.stringify({ id: 42, tag_name: "v0.3.2" }));
|
||||
});
|
||||
|
||||
try {
|
||||
const env: Record<string, string> = {
|
||||
PATH: process.env.PATH ?? "",
|
||||
HOME: tmpHome,
|
||||
GITEA_TOKEN: "poisoned-shell-token",
|
||||
NEXTCRAFT_FORGE_BASE: srv.url,
|
||||
};
|
||||
const res = spawnSync(
|
||||
"bash",
|
||||
[
|
||||
"-c",
|
||||
`NEXTCRAFT_FORGE_BASE='${srv.url}' GITEA_TOKEN=poisoned-shell-token bash '${join(repoRoot, "scripts", "release-assets.sh")}' v0.3.2 --dry-run`,
|
||||
],
|
||||
{ cwd: tmpHome, env, encoding: "utf8", timeout: 20000 },
|
||||
);
|
||||
|
||||
assert.equal(res.stdout.includes("poisoned-shell-token"), false, "poisoned token never in stdout");
|
||||
assert.equal(res.stderr.includes("poisoned-shell-token"), false, "poisoned token never in stderr");
|
||||
assert.equal(res.status, 0, `dry-run should succeed against fixture, stderr: ${res.stderr}`);
|
||||
assert.ok(res.stdout.includes("DRY-RUN would upload"), `expected dry-run upload lines, got: ${res.stdout}`);
|
||||
assert.ok(res.stdout.includes("nextcraft-linux-x64"));
|
||||
} finally {
|
||||
srv.stop();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,38 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { cpSync, existsSync, mkdirSync, mkdtempSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, dirname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const cliDir = join(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const repoRoot = join(cliDir, "..", "..");
|
||||
const binary = join(repoRoot, "apps", "cli", "dist", "nextcraft-linux-x64");
|
||||
|
||||
test("PATH bare-word invocation: installed binary speaks (SEA regression, v0.3.4 bug)", () => {
|
||||
if (!existsSync(binary)) return; // built by cli:build:binary; covered in ship flow
|
||||
const binDir = mkdtempSync(join(tmpdir(), "nc-pathword-"));
|
||||
cpSync(binary, join(binDir, "nextcraft"));
|
||||
const env = {
|
||||
PATH: [binDir, process.env.PATH].filter(Boolean).join(":"),
|
||||
HOME: process.env.HOME,
|
||||
};
|
||||
const { NODE_TEST_CONTEXT, ...rest } = process.env as Record<string, string | undefined>;
|
||||
const envClean = { ...rest, ...env } as Record<string, string | undefined>;
|
||||
|
||||
const ver = spawnSync("sh", ["-c", "nextcraft --version"], { encoding: "utf8", timeout: 30000, env: envClean });
|
||||
assert.equal(ver.status, 0, `--version rc: ${ver.status} stderr: ${ver.stderr}`);
|
||||
assert.ok(ver.stdout.trim().length > 0, "--version must print the version (v0.3.4 was silent)");
|
||||
|
||||
const doc = spawnSync("sh", ["-c", "nextcraft doctor"], { encoding: "utf8", timeout: 120000, env: envClean });
|
||||
assert.equal(doc.status, 0, `doctor rc: ${doc.status} stderr: ${doc.stderr}`);
|
||||
assert.ok(doc.stdout.includes("environment prerequisites"), "doctor must print its report");
|
||||
|
||||
const noArgs = spawnSync("sh", ["-c", "nextcraft"], { encoding: "utf8", timeout: 30000, env: envClean });
|
||||
assert.equal(noArgs.status, 2, "no args must exit 2");
|
||||
assert.ok((noArgs.stderr + noArgs.stdout).includes("Usage:"), "no args must print usage");
|
||||
|
||||
const bad = spawnSync("sh", ["-c", "nextcraft nosuchcmd"], { encoding: "utf8", timeout: 30000, env: envClean });
|
||||
assert.equal(bad.status, 2, "unknown command must exit 2");
|
||||
});
|
||||
@@ -0,0 +1,31 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { run } from "../src/lib/spawn.ts";
|
||||
|
||||
test("spawn: returns exit code of short process", async () => {
|
||||
const res = await run("node", ["-e", "process.exit(3)"], { capture: true, timeoutMs: 5000 });
|
||||
assert.equal(res.code, 3);
|
||||
assert.equal(res.timedOut, false);
|
||||
});
|
||||
|
||||
test("spawn: timeout kills and reports timedOut", async () => {
|
||||
const res = await run("node", ["-e", "setTimeout(() => {}, 10000)"], {
|
||||
capture: true,
|
||||
timeoutMs: 300,
|
||||
});
|
||||
assert.equal(res.timedOut, true);
|
||||
assert.ok(res.code !== 0);
|
||||
});
|
||||
|
||||
test("spawn: missing binary resolves code 127, never rejects", async () => {
|
||||
const res = await run("definitely-not-a-real-binary-xyz", [], { capture: true, timeoutMs: 1000 });
|
||||
assert.equal(res.code, 127);
|
||||
});
|
||||
|
||||
test("spawn: capture collects stdout", async () => {
|
||||
const res = await run("node", ["-e", "process.stdout.write('hello')"], {
|
||||
capture: true,
|
||||
timeoutMs: 5000,
|
||||
});
|
||||
assert.equal(res.stdout, "hello");
|
||||
});
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": false,
|
||||
"allowImportingTsExtensions": false,
|
||||
"declaration": false,
|
||||
"declarationMap": false,
|
||||
"sourceMap": false,
|
||||
"incremental": false,
|
||||
"outDir": "dist",
|
||||
"rootDir": "src"
|
||||
},
|
||||
"include": ["src/**/*"],
|
||||
"exclude": ["tests", "node_modules", "dist"]
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"extends": "../../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"lib": ["ES2022"],
|
||||
"types": ["node"],
|
||||
"noEmit": true,
|
||||
"declaration": false,
|
||||
"declarationMap": false,
|
||||
"allowImportingTsExtensions": true,
|
||||
"baseUrl": "."
|
||||
},
|
||||
"include": ["src/**/*", "tests/**/*"]
|
||||
}
|
||||
@@ -1 +1,5 @@
|
||||
NEXT_PUBLIC_AI_SERVICE_URL=http://localhost:8420
|
||||
# Browser-side engine API base URL. UNSET = auto (recommended): the browser
|
||||
# derives http://<current-hostname>:8420 at runtime, so remote browsing works
|
||||
# zero-config. Set explicitly only for unusual topologies, e.g.:
|
||||
# NEXT_PUBLIC_AI_SERVICE_URL=http://ai.internal:8420
|
||||
# NEXT_PUBLIC_AI_SERVICE_URL=
|
||||
@@ -193,6 +193,10 @@ const ARTIFACT_ICON_BG: Record<ArtifactType, string> = {
|
||||
/* Page */
|
||||
/* -------------------------------------------------------------------------- */
|
||||
|
||||
export function generateStaticParams() {
|
||||
return candidates.map((c) => ({ candidateId: c.id }));
|
||||
}
|
||||
|
||||
export default async function CandidateProfilePage({ params }: PageProps) {
|
||||
const { candidateId } = await params;
|
||||
const cand = candidates.find((c) => c.id === candidateId);
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
import { notFound } from 'next/navigation';
|
||||
import { ArrowLeft, AlertTriangle, Loader2 } from 'lucide-react';
|
||||
import Link from 'next/link';
|
||||
import { competencyStacks } from '@nextcraft/mock-data';
|
||||
import { competencyStacks, allCompetencies } from '@nextcraft/mock-data';
|
||||
import { BuildSurface } from '../../../../components/learner/build-surface';
|
||||
|
||||
export function generateStaticParams() {
|
||||
return allCompetencies.map((c) => ({ competencyId: c.id }));
|
||||
}
|
||||
|
||||
/**
|
||||
* Real build surface (REQ-3-008): a per-learner variant task + a live
|
||||
* namespace sandbox with file CRUD and Run/Test (CUT-2: read-only output,
|
||||
|
||||
@@ -41,6 +41,10 @@ function isUnlocked(c: Competency): boolean {
|
||||
return c.status !== 'locked';
|
||||
}
|
||||
|
||||
export function generateStaticParams() {
|
||||
return competencyStacks.map((s) => ({ stackId: s.id }));
|
||||
}
|
||||
|
||||
export default async function StackViewPage({
|
||||
params,
|
||||
}: {
|
||||
|
||||
@@ -10,6 +10,10 @@ import { DefenseSession } from '../../../../components/learner/defense-session';
|
||||
* v0.1 static assessment mockup (pre-baked rubric scores, scripted
|
||||
* transcript, submitted-code display) is retired.
|
||||
*/
|
||||
export function generateStaticParams() {
|
||||
return allCompetencies.map((c) => ({ competencyId: c.id }));
|
||||
}
|
||||
|
||||
export default async function DefensePage({
|
||||
params,
|
||||
}: {
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
'use client';
|
||||
|
||||
import { useCallback, useEffect, useState } from 'react';
|
||||
import { BadgeCheck, CircleAlert, Loader2, ShieldCheck } from 'lucide-react';
|
||||
import { Button, Card, CardBody, Input } from '@nextcraft/ui';
|
||||
import {
|
||||
MOCK_LEARNER_ID,
|
||||
getIdentityStatus,
|
||||
submitIdentity,
|
||||
verifyIdentity,
|
||||
} from '../../../lib/engine-client';
|
||||
import type { IdentityStatus } from '../../../lib/engine-client';
|
||||
|
||||
/**
|
||||
* Identity enrollment (REQ-5-003/004): submit verification → pending →
|
||||
* verified/rejected. Honest at every state (A-304): mock verdicts are
|
||||
* LABELED mock — this surface never displays mock-verified as
|
||||
* production-verified. Gated-route 403s send learners here via the
|
||||
* verify-CTA payload (G-10 → VerifyRequiredError).
|
||||
*/
|
||||
export function EnrollFlow() {
|
||||
const [status, setStatus] = useState<IdentityStatus | null>(null);
|
||||
const [dob, setDob] = useState('');
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [message, setMessage] = useState<string | null>(null);
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
try {
|
||||
const s = await getIdentityStatus(MOCK_LEARNER_ID);
|
||||
setStatus(s);
|
||||
} catch {
|
||||
setStatus(null);
|
||||
}
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
void refresh();
|
||||
}, [refresh]);
|
||||
|
||||
const submit = useCallback(async () => {
|
||||
if (!dob) {
|
||||
setError('Enter your date of birth to start verification.');
|
||||
return;
|
||||
}
|
||||
setBusy(true);
|
||||
setError(null);
|
||||
setMessage(null);
|
||||
try {
|
||||
const sub = await submitIdentity(MOCK_LEARNER_ID, dob);
|
||||
const verdict = await verifyIdentity(sub.submission_id);
|
||||
if (verdict.status === 'verified') {
|
||||
setMessage(
|
||||
`Verified${verdict.mock ? ' (mock provider — not production verification)' : ''}.`,
|
||||
);
|
||||
} else {
|
||||
setError(verdict.detail || 'Verification rejected.');
|
||||
}
|
||||
await refresh();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Verification failed.');
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}, [dob, refresh]);
|
||||
|
||||
return (
|
||||
<div className="mx-auto max-w-md space-y-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<ShieldCheck className="h-5 w-5 text-blue-600 dark:text-blue-400" aria-hidden />
|
||||
<h1 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Identity Verification
|
||||
</h1>
|
||||
{status?.mock && (
|
||||
<span className="rounded bg-slate-100 px-1.5 py-0.5 text-xs text-slate-500 dark:bg-slate-800 dark:text-slate-400">
|
||||
mock provider
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-300">
|
||||
The AI school floor is 16+; the marketplace requires 18+ with a verified
|
||||
identity. Verification is one step — your date of birth is used to
|
||||
derive your age band and is never stored raw.
|
||||
</p>
|
||||
|
||||
{status && status.status === 'verified' ? (
|
||||
<Card>
|
||||
<CardBody className="flex items-center gap-3">
|
||||
<BadgeCheck
|
||||
className="h-6 w-6 text-emerald-600 dark:text-emerald-400"
|
||||
aria-hidden
|
||||
/>
|
||||
<div>
|
||||
<p className="text-sm font-medium text-slate-800 dark:text-slate-100">
|
||||
Verified — age band {status.age_band}
|
||||
</p>
|
||||
{status.mock && (
|
||||
<p className="text-xs text-slate-500 dark:text-slate-400">
|
||||
mock verdict — not production verification
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
) : status && status.status === 'pending' ? (
|
||||
<Card>
|
||||
<CardBody className="flex items-center gap-3">
|
||||
<Loader2 className="h-5 w-5 animate-spin text-amber-500" aria-hidden />
|
||||
<p className="text-sm text-slate-700 dark:text-slate-300">
|
||||
Verification pending — resubmit once the current one settles.
|
||||
</p>
|
||||
</CardBody>
|
||||
</Card>
|
||||
) : (
|
||||
<Card>
|
||||
<CardBody className="space-y-3">
|
||||
<label htmlFor="dob" className="text-sm font-medium text-slate-700 dark:text-slate-300">
|
||||
Date of birth
|
||||
</label>
|
||||
<Input
|
||||
id="dob"
|
||||
type="date"
|
||||
value={dob}
|
||||
onChange={(e) => setDob(e.target.value)}
|
||||
aria-label="Date of birth"
|
||||
/>
|
||||
<Button onClick={() => void submit()} disabled={busy}>
|
||||
{busy ? <Loader2 className="h-4 w-4 animate-spin" aria-hidden /> : null}
|
||||
Start Verification
|
||||
</Button>
|
||||
</CardBody>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{error && (
|
||||
<p
|
||||
className="flex items-center gap-1 text-xs text-red-600 dark:text-red-400"
|
||||
role="alert"
|
||||
>
|
||||
<CircleAlert className="h-3 w-3" aria-hidden /> {error}
|
||||
</p>
|
||||
)}
|
||||
{message && (
|
||||
<p className="flex items-center gap-1 text-xs text-emerald-600 dark:text-emerald-400">
|
||||
<BadgeCheck className="h-3 w-3" aria-hidden /> {message}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
import type { Metadata } from 'next';
|
||||
import { EnrollFlow } from './EnrollFlow';
|
||||
|
||||
export const metadata: Metadata = {
|
||||
title: 'Enroll — Nextcraft',
|
||||
description: 'Verify your identity for the Nextcraft AI school (16+) and marketplace (18+).',
|
||||
};
|
||||
|
||||
export default function EnrollPage() {
|
||||
return (
|
||||
<div className="py-10">
|
||||
<EnrollFlow />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -6,6 +6,10 @@ import { allCompetencies, competencyStacks } from '@nextcraft/mock-data';
|
||||
import { WorkedExampleTabs } from '../../../../components/learner/worked-example-tabs';
|
||||
import { ByteTutorPanel } from '../../../../components/learner/byte-tutor-panel';
|
||||
|
||||
export function generateStaticParams() {
|
||||
return allCompetencies.map((c) => ({ competencyId: c.id }));
|
||||
}
|
||||
|
||||
export default async function ByteTutorialPage({
|
||||
params,
|
||||
}: {
|
||||
|
||||
@@ -18,6 +18,10 @@ const CULTURE_CARDS = [
|
||||
{ label: 'Mission First', tone: 'bg-violet-100 text-violet-700 dark:bg-violet-900/40 dark:text-violet-300' },
|
||||
] as const;
|
||||
|
||||
export function generateStaticParams() {
|
||||
return employers.map((e) => ({ employerId: e.id }));
|
||||
}
|
||||
|
||||
export default async function EmployerProfilePage({ params }: PageProps) {
|
||||
const { employerId } = await params;
|
||||
const employer = employers.find((e) => e.id === employerId);
|
||||
|
||||
@@ -32,6 +32,10 @@ function aiMatchedSkills(job: Job): Array<{ skill: string; score: number }> {
|
||||
});
|
||||
}
|
||||
|
||||
export function generateStaticParams() {
|
||||
return jobs.map((j) => ({ jobId: j.id }));
|
||||
}
|
||||
|
||||
export default async function JobDetailPage({ params }: PageProps) {
|
||||
const { jobId } = await params;
|
||||
const job = jobs.find((j) => j.id === jobId);
|
||||
|
||||
@@ -4,8 +4,9 @@ import { useState, useEffect, useRef, useCallback } from 'react';
|
||||
import { parseSseEvents } from '../../lib/sse';
|
||||
import { Bot, RefreshCw, AlertTriangle, Loader2 } from 'lucide-react';
|
||||
|
||||
const AI_SERVICE_URL =
|
||||
process.env.NEXT_PUBLIC_AI_SERVICE_URL ?? 'http://localhost:8420';
|
||||
import { engineBaseUrl } from '../../lib/engine-base-url';
|
||||
|
||||
const AI_SERVICE_URL = engineBaseUrl();
|
||||
|
||||
interface StreamPanelProps {
|
||||
title: string;
|
||||
|
||||
@@ -118,6 +118,14 @@ export function BuildSurface({
|
||||
<p className="text-sm font-medium text-red-700 dark:text-red-300">
|
||||
{session.errorMessage ?? 'Could not start the build environment.'}
|
||||
</p>
|
||||
{session.verifyCta && (
|
||||
<a
|
||||
href={session.verifyCta}
|
||||
className="text-sm font-medium text-blue-600 underline dark:text-blue-400"
|
||||
>
|
||||
Verify your identity to continue →
|
||||
</a>
|
||||
)}
|
||||
<Button onClick={session.retry} variant="outline" size="sm">
|
||||
<RefreshCw className="h-3.5 w-3.5" aria-hidden /> Retry
|
||||
</Button>
|
||||
@@ -129,7 +137,7 @@ export function BuildSurface({
|
||||
<div className="space-y-6">
|
||||
<header className="space-y-2">
|
||||
<p className="text-xs font-semibold uppercase tracking-wide text-primary-600 dark:text-primary-400">
|
||||
{stackTitle} · build
|
||||
{stackTitle} · {session.variant?.environment ?? 'build'}
|
||||
</p>
|
||||
<h1 className="text-xl font-bold text-slate-900 dark:text-slate-100">
|
||||
{session.variant?.statement ?? 'Your task'}
|
||||
@@ -141,11 +149,11 @@ export function BuildSurface({
|
||||
</header>
|
||||
|
||||
<RunControls
|
||||
command="python -m pytest -q"
|
||||
command={session.variant?.test_command ?? 'python3 -m pytest -q'}
|
||||
running={running}
|
||||
busy={false}
|
||||
onRun={() => void run(['python', '-m', 'pytest', '-q'])}
|
||||
onTest={() => void run(['pytest', '-q'])}
|
||||
onRun={() => void run((session.variant?.test_command ?? 'python3 -m pytest -q').trim().split(/\s+/))}
|
||||
onTest={() => void session.test()}
|
||||
/>
|
||||
|
||||
<div className="grid gap-4 lg:grid-cols-[220px_minmax(0,1fr)]">
|
||||
|
||||
@@ -12,12 +12,15 @@ import type {
|
||||
import {
|
||||
MOCK_LEARNER_ID,
|
||||
answerDefense,
|
||||
answerDefenseAudio,
|
||||
finishDefense,
|
||||
getDefense,
|
||||
listVariants,
|
||||
requestGrade,
|
||||
startDefense,
|
||||
} from '../../lib/engine-client';
|
||||
import type { VoiceDescriptor } from '@nextcraft/types';
|
||||
import { MAX_RECORD_SECONDS, voiceBadgeLabel } from '../../lib/voice-badge';
|
||||
|
||||
type MicState = 'idle' | 'recording' | 'denied' | 'unsupported';
|
||||
|
||||
@@ -44,7 +47,10 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
const [finished, setFinished] = useState<DefenseFinish | null>(null);
|
||||
const [grade, setGrade] = useState<GradeRecord | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [descriptor, setDescriptor] = useState<VoiceDescriptor | null>(null);
|
||||
const [recordSeconds, setRecordSeconds] = useState(0);
|
||||
const recorderRef = useRef<MediaRecorder | null>(null);
|
||||
const recordTimerRef = useRef<ReturnType<typeof setInterval> | null>(null);
|
||||
|
||||
const refresh = useCallback(async (id: string) => {
|
||||
const session: DefenseSession = await getDefense(id);
|
||||
@@ -78,6 +84,7 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
try {
|
||||
const started = await startDefense(MOCK_LEARNER_ID, taskId);
|
||||
setDefenseId(started.defense_id);
|
||||
setDescriptor(started.voice_descriptor ?? null);
|
||||
await refresh(started.defense_id);
|
||||
if (!started.trace_complete) {
|
||||
setError(
|
||||
@@ -108,30 +115,81 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
[defenseId, refresh],
|
||||
);
|
||||
|
||||
const submitAudio = useCallback(
|
||||
async (blob: Blob) => {
|
||||
if (!defenseId) return;
|
||||
setBusy(true);
|
||||
try {
|
||||
// D-040/REQ-5-002: the recorded blob IS the answer — the server
|
||||
// transcribes it (real STT when openai-audio is configured; the
|
||||
// mock provider under tests/dev). The descriptor badge tells the
|
||||
// learner which path is live.
|
||||
await answerDefenseAudio(defenseId, blob);
|
||||
await refresh(defenseId);
|
||||
} catch (err) {
|
||||
// G-12: 413 renders the honest re-record prompt the server sends.
|
||||
setError(err instanceof Error ? err.message : 'Voice answer failed.');
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
},
|
||||
[defenseId, refresh],
|
||||
);
|
||||
|
||||
// P0-1 fix (verifier): with recorder.start(timeslice), ondataavailable
|
||||
// fires PER CHUNK — buffering them until onstop and posting ONE complete
|
||||
// blob, else every answer truncates to the first 1s slice (or splits into
|
||||
// two turns). Stop is the single completion signal; chunks accumulate.
|
||||
const chunksRef = useRef<Blob[]>([]);
|
||||
|
||||
const stopRecording = useCallback(() => {
|
||||
recorderRef.current?.stop();
|
||||
if (recordTimerRef.current) {
|
||||
clearInterval(recordTimerRef.current);
|
||||
recordTimerRef.current = null;
|
||||
}
|
||||
setRecordSeconds(0);
|
||||
}, []);
|
||||
|
||||
const record = useCallback(async () => {
|
||||
if (micState === 'recording') {
|
||||
recorderRef.current?.stop();
|
||||
stopRecording();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
||||
const recorder = new MediaRecorder(stream);
|
||||
recorderRef.current = recorder;
|
||||
recorder.ondataavailable = async (event) => {
|
||||
if (event.data.size === 0) return;
|
||||
// Browser-native SR fallback: v0.3 has no server STT key (CUT-1).
|
||||
// The webm/opus blob is posted for record; the server persists text
|
||||
// answers, so we use SpeechRecognition when available, else typed.
|
||||
if (!defenseId) return;
|
||||
chunksRef.current = [];
|
||||
recorder.ondataavailable = (event) => {
|
||||
if (event.data.size > 0) chunksRef.current.push(event.data);
|
||||
};
|
||||
recorder.onstop = () => {
|
||||
stream.getTracks().forEach((t) => t.stop());
|
||||
setMicState('idle');
|
||||
const chunks = chunksRef.current;
|
||||
if (chunks.length === 0) return;
|
||||
void submitAudio(new Blob(chunks, { type: recorder.mimeType || 'audio/webm' }));
|
||||
};
|
||||
recorder.start();
|
||||
// a-13: timeslice keeps the blob observable/chunked (buffered until
|
||||
// onstop — see the P0-1 note above).
|
||||
recorder.start(1000);
|
||||
setMicState('recording');
|
||||
setRecordSeconds(0);
|
||||
recordTimerRef.current = setInterval(() => {
|
||||
setRecordSeconds((s) => {
|
||||
if (s + 1 >= MAX_RECORD_SECONDS) {
|
||||
// G-12 auto-stop: the timer is visible, so this surprises no one.
|
||||
stopRecording();
|
||||
return MAX_RECORD_SECONDS;
|
||||
}
|
||||
return s + 1;
|
||||
});
|
||||
}, 1000);
|
||||
} catch {
|
||||
setMicState('denied');
|
||||
}
|
||||
}, [defenseId, micState]);
|
||||
}, [defenseId, micState, stopRecording, submitAudio]);
|
||||
|
||||
const finish = useCallback(async () => {
|
||||
if (!defenseId) return;
|
||||
@@ -160,7 +218,13 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
}
|
||||
}, [taskId]);
|
||||
|
||||
useEffect(() => () => recorderRef.current?.stop(), []);
|
||||
useEffect(
|
||||
() => () => {
|
||||
recorderRef.current?.stop();
|
||||
if (recordTimerRef.current) clearInterval(recordTimerRef.current);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
if (!taskId) {
|
||||
return (
|
||||
@@ -200,7 +264,7 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
<textarea
|
||||
value={answerText}
|
||||
onChange={(e) => setAnswerText(e.target.value)}
|
||||
placeholder="Type your answer (voice capture needs mic permission)…"
|
||||
placeholder="Type your answer — or record it with the mic button"
|
||||
aria-label="Your answer"
|
||||
className="min-h-[64px] flex-1 resize-y rounded-md border border-slate-300 bg-slate-50 p-3 text-sm text-slate-800 dark:border-slate-600 dark:bg-slate-800 dark:text-slate-100"
|
||||
/>
|
||||
@@ -217,12 +281,28 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
<SendHorizonal className="h-4 w-4" aria-hidden /> Send
|
||||
</Button>
|
||||
</div>
|
||||
{micState === 'recording' && (
|
||||
<p
|
||||
className="flex items-center gap-1 text-xs text-red-600 dark:text-red-400"
|
||||
aria-live="polite"
|
||||
>
|
||||
<Square className="h-3 w-3" aria-hidden /> Recording… {recordSeconds}s /{' '}
|
||||
{180}s — auto-stops at the bound (G-12).
|
||||
</p>
|
||||
)}
|
||||
{micState === 'denied' && (
|
||||
<p className="flex items-center gap-1 text-xs text-slate-500 dark:text-slate-400">
|
||||
<XCircle className="h-3 w-3" aria-hidden /> Mic unavailable — typed answers are
|
||||
first-class.
|
||||
</p>
|
||||
)}
|
||||
{descriptor && (
|
||||
<p className="text-xs text-slate-500 dark:text-slate-400">
|
||||
Voice path:{' '}
|
||||
<span className="font-medium">{voiceBadgeLabel(descriptor)}</span>
|
||||
{descriptor.hint ? ` — ${descriptor.hint}` : ''}
|
||||
</p>
|
||||
)}
|
||||
<div className="flex justify-end">
|
||||
<Button onClick={() => void finish()} disabled={busy} variant="outline">
|
||||
Finish Defense
|
||||
|
||||
@@ -3,8 +3,9 @@
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import { parseSseEvents } from '../lib/sse';
|
||||
|
||||
const AI_SERVICE_URL =
|
||||
process.env.NEXT_PUBLIC_AI_SERVICE_URL ?? 'http://localhost:8420';
|
||||
import { engineBaseUrl } from '../lib/engine-base-url';
|
||||
|
||||
const AI_SERVICE_URL = engineBaseUrl();
|
||||
|
||||
export type AgentName = 'coach' | 'tutor' | 'lab' | 'assessor' | 'proctor' | 'mentor';
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import {
|
||||
VerifyRequiredError,
|
||||
EngineError,
|
||||
MOCK_LEARNER_ID,
|
||||
createSandbox,
|
||||
@@ -37,6 +38,8 @@ export interface SandboxSessionState {
|
||||
sandboxId: string | null;
|
||||
files: string[];
|
||||
errorMessage: string | null;
|
||||
/** G-10: identity-gate 403s carry an actionable enrollment link. */
|
||||
verifyCta: string | null;
|
||||
}
|
||||
|
||||
const DEFAULT_STATE: SandboxSessionState = {
|
||||
@@ -45,6 +48,7 @@ const DEFAULT_STATE: SandboxSessionState = {
|
||||
sandboxId: null,
|
||||
files: [],
|
||||
errorMessage: null,
|
||||
verifyCta: null,
|
||||
};
|
||||
|
||||
export function useSandboxSession(competencyId: string | null) {
|
||||
@@ -66,7 +70,13 @@ export function useSandboxSession(competencyId: string | null) {
|
||||
await writeFile(sandbox.id, path, content, controller.signal);
|
||||
}
|
||||
const files = await listFiles(sandbox.id, controller.signal);
|
||||
setState({ status: 'ready', variant, sandboxId: sandbox.id, files, errorMessage: null });
|
||||
setState({
|
||||
...DEFAULT_STATE,
|
||||
status: 'ready',
|
||||
variant,
|
||||
sandboxId: sandbox.id,
|
||||
files,
|
||||
});
|
||||
} catch (err) {
|
||||
// A created sandbox must not outlive a failed start (per-learner cap
|
||||
// is 1 — a leaked one blocks every retry with 429 forever). This
|
||||
@@ -79,6 +89,8 @@ export function useSandboxSession(competencyId: string | null) {
|
||||
...DEFAULT_STATE,
|
||||
status: err.status === 503 ? 'busy' : err.status === 403 || err.status === 429 ? 'denied' : 'error',
|
||||
errorMessage: err.message,
|
||||
// G-10: identity-gate 403s render the verify-CTA link (D-043).
|
||||
verifyCta: err instanceof VerifyRequiredError ? err.verifyCta : null,
|
||||
});
|
||||
return;
|
||||
}
|
||||
@@ -137,8 +149,11 @@ export function useSandboxSession(competencyId: string | null) {
|
||||
|
||||
const test = useCallback(async (): Promise<ExecResult | null> => {
|
||||
if (!state.variant || !state.sandboxId) return null;
|
||||
// All v0.3 templates ship pytest-based starter tests (PLAN Task 6-3-01).
|
||||
return run(['pytest', '-q']);
|
||||
// REQ-5-006: the variant's REAL test command (v0.3 hardcoded pytest;
|
||||
// design/sim kinds have their own). G-15: whitespace-only split —
|
||||
// templates validate quote-free at authoring (no shlex in browsers).
|
||||
const cmd = state.variant.test_command || 'pytest -q';
|
||||
return run(cmd.trim().split(/\s+/));
|
||||
}, [run, state.variant, state.sandboxId]);
|
||||
|
||||
const saveFile = useCallback(
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
// 'self' (v0.3.6 single-port deploy) means "same origin as the page":
|
||||
// returns "" so callers produce RELATIVE fetch URLs (`${base}/v1/...`).
|
||||
// Used when the ai-service serves the exported web app from 8420 itself —
|
||||
// no CORS, no mixed content, no absolute host to keep in sync.
|
||||
export function engineBaseUrl(): string {
|
||||
const override = process.env.NEXT_PUBLIC_AI_SERVICE_URL;
|
||||
if (override === "self") return "";
|
||||
if (override) return override;
|
||||
if (typeof window !== "undefined") {
|
||||
return `http://${window.location.hostname}:8420`;
|
||||
}
|
||||
return "http://localhost:8420";
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user