Compare commits
21 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c9709e0f38 | |||
| 62bff575af | |||
| 8a43e95a4d | |||
| ee5be13c94 | |||
| 1da814e790 | |||
| cc7ec5d03f | |||
| 5c2829d9df | |||
| ec4648b37a | |||
| c5369b407b | |||
| 282f5ef150 | |||
| 3010bc4b96 | |||
| 35c4c386b5 | |||
| d8d2cebfc5 | |||
| b2a2a4023b | |||
| de431852c4 | |||
| 64e4842976 | |||
| a64733a262 | |||
| 0072689e4d | |||
| 3110be2f15 | |||
| bd5b0fee95 | |||
| c80381c4df |
@@ -106,10 +106,11 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `ai_service/sandbox/` | `backend.py` (SandboxBackend protocol), `unshare_backend.py` (userns/mount/pid/net spawner, D-024), `manager.py` (lifecycle: create/list/snapshot/destroy + concurrency guard D-032), `workdir.py` (per-sandbox fs layout) | Never imports api/ or agents/; spawns subprocesses only | config |
|
||||
| `ai_service/telemetry/` | `models.py` (TelemetryEvent, TraceSpan), `store.py` (TraceStore protocol + SQLite impl D-027), `ingest.py` (WebSocket /v1/telemetry/ingest, seq gap detection D-026) | Persistence; never imports agents/ | config |
|
||||
| `ai_service/grading/` | `features.py` (deterministic trace digest D-028), `engine.py` (rubric scoring orchestration), `store.py` (GradeStore) | LLM only via digest; never sees raw trace | llm, telemetry, prompts |
|
||||
| `ai_service/variants/` | `templates.py` (task template library), `generator.py` (seeded LLM instantiation D-029), `store.py` (VariantStore) | LLM via structured output | llm, grading |
|
||||
| `ai_service/voice/` | `base.py` (VoiceProvider protocol D-030), `browser.py` (native SR/TTS fallback descriptor), `mock.py` (deterministic; the real server STT/TTS provider is the v0.4 seam — GRILL CUT-1/G-7), `factory.py` (provider selection), `defense_store.py` (DefenseStore: transcripts + integrity signals, D-027) | Never imports agents/ or api/ | config |
|
||||
| `ai_service/telemetry/` | `models.py` (TelemetryEvent, TraceSpan), `store.py` (TraceStore protocol + SQLite impl D-027), `ingest.py` (WebSocket /v1/telemetry/ingest, seq gap detection D-026; v0.5 REQ-5-007/D-045: emits advisory `seq_ack` frames — highest-contiguous received seq per successful append; capture agent trims its spool to the ack, closing the replay-margin gap) | Persistence; never imports agents/ | config |
|
||||
| `ai_service/grading/` | `features.py` (deterministic trace digest D-028 — kind-agnostic by construction, pinned over design/sim traces in v0.5), `engine.py` (rubric scoring orchestration), `store.py` (GradeStore) | LLM only via digest; never sees raw trace | llm, telemetry, prompts |
|
||||
| `ai_service/variants/` | `templates.py` (task template library; v0.5 REQ-5-005/D-044: `environment: Literal[build,design,simulation]` registry + per-kind starter files + harness/test commands, G-15 shlex-roundtrip validation), `generator.py` (seeded LLM instantiation D-029), `store.py` (VariantStore; idempotent `_ensure_v05_columns` backfill for pre-v0.5 DBs) | LLM via structured output | llm, grading |
|
||||
| `ai_service/voice/` | `base.py` (VoiceProvider protocol D-030), `browser.py` (native SR/TTS fallback descriptor), `mock.py` (deterministic), `openai_audio.py` (v0.5 REQ-5-001, D-040: real server STT/TTS against OpenAI-compatible `/audio/transcriptions` + `/audio/speech` on the shared httpx pool — CUT-1/G-7 seam CLOSED), `factory.py` (provider selection by `AI_VOICE_PROVIDER`, G-11 boot-safe fallback to mock), `defense_store.py` (DefenseStore: transcripts + integrity signals, D-027) | Never imports agents/ or api/ | config |
|
||||
| `ai_service/identity/` | **v0.5 NEW (REQ-5-003/004, D-042/43):** `base.py` (IdentityProvider protocol: submit/poll/verify), `mock.py` (deterministic approve-on-policy mock; verdicts carry a `mock` marker, A-304), `store.py` (5th D-027 store: identity_record table — derived `age_band`, document **refs**, PII never stored raw); age-gate dependencies `require_verified_age`/`require_verified_adult` (gate composition D-043: G-5 allowlist → identity verdict → rate caps; mounted on variants/sandbox-create/defense-start + the G-18 marketplace stub), exposed via `api/identity.py` (`/v1/identity/*`) | Never imports agents/; api/ composes it via DI | config |
|
||||
| `ai_service/agents/examiner.py` | Seventh agent: oral defense examiner; streams over existing SSE, consumes process traces + emits integrity signals | reuses BaseAgent (D-018) | llm, prompts, telemetry |
|
||||
| `ai_service/data/*.db` | SQLite databases (telemetry/grades/variants/defenses) — **v0.3.6: default moved to `~/.nextcraft/data/nextcraft.db` (state out of the repo; AI_DB_PATH overrides, ~ expanded)** | outside repo (home) | — |
|
||||
| `scripts/sandbox-agent.py` | Tiny in-namespace capture process shipped into the sandbox; streams telemetry to ingest | standalone | stdlib only |
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"phase": 1,
|
||||
"stage": "execute",
|
||||
"milestone": "v0.5",
|
||||
"phase_role": "execution",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-09-13T18:35:00Z"
|
||||
}
|
||||
+1
-1
@@ -8,7 +8,7 @@ Nextcraft is an AI-native outcome school where graduates prove what they can bui
|
||||
|
||||
---
|
||||
|
||||
## Current Milestone: v0.5 — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease
|
||||
## Milestone v0.5 — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease (COMPLETE, shipped as v0.4.5)
|
||||
|
||||
**Scope (the four seams D-016 deferred out of v0.4, locked at v0.5 Phase 0 SPECIFY):**
|
||||
|
||||
|
||||
+17
-17
@@ -1,33 +1,33 @@
|
||||
# Nextcraft — REQUIREMENTS.md
|
||||
|
||||
## v0.5 Requirements (Active — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease)
|
||||
## v0.5 Requirements (Complete — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease; shipped as v0.4.5)
|
||||
|
||||
### Real Server Voice
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-001 | `openai-audio` VoiceProvider: server STT (`/audio/transcriptions`) + TTS (`/audio/speech`) against an OpenAI-compatible endpoint via the existing D-030 protocol; provider selection by `AI_VOICE_PROVIDER` (+ base URL/key from env, never committed); deterministic mock stays first-class; browser fallback unchanged | critical | 2 | pending |
|
||||
| REQ-5-002 | Voice defense real path end-to-end: examiner dialogue answers transcribed server-side (audio upload → transcript), examiner questions spoken via server TTS (audio returned to the client); transcripts + integrity signals unchanged | critical | 2 | pending |
|
||||
| REQ-5-001 | `openai-audio` VoiceProvider: server STT (`/audio/transcriptions`) + TTS (`/audio/speech`) against an OpenAI-compatible endpoint via the existing D-030 protocol; provider selection by `AI_VOICE_PROVIDER` (+ base URL/key from env, never committed); deterministic mock stays first-class; browser fallback unchanged | critical | 2 | complete |
|
||||
| REQ-5-002 | Voice defense real path end-to-end: examiner dialogue answers transcribed server-side (audio upload → transcript), examiner questions spoken via server TTS (audio returned to the client); transcripts + integrity signals unchanged | critical | 2 | complete |
|
||||
|
||||
### Identity & Age-Gating
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-003 | Identity provider protocol + mock-first backend (REQ-F-017): verify-identity flow (submit → pending → verified/rejected with document refs), provider-agnostic (mock default; a real KYC vendor drops in later), PII stored server-side only, never logged | critical | 3 | pending |
|
||||
| REQ-5-004 | Age-gating enforced by the backend: school floor 16+ verified at enrollment, marketplace 18+ with verified identity — API surfaces reject under-age/unverified callers on gated routes (replaces the v0.1 visual-only flow; G-5 allowlist evolves toward real identity, allowlist remains as pilot guard) | critical | 3 | pending |
|
||||
| REQ-5-003 | Identity provider protocol + mock-first backend (REQ-F-017): verify-identity flow (submit → pending → verified/rejected with document refs), provider-agnostic (mock default; a real KYC vendor drops in later), PII stored server-side only, never logged | critical | 3 | complete |
|
||||
| REQ-5-004 | Age-gating enforced by the backend: school floor 16+ verified at enrollment, marketplace 18+ with verified identity — API surfaces reject under-age/unverified callers on gated routes (replaces the v0.1 visual-only flow; G-5 allowlist evolves toward real identity, allowlist remains as pilot guard) | critical | 3 | complete |
|
||||
|
||||
### Sandbox Environments
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-005 | Design + simulation sandbox environment types (REQ-F-021 remainder): extend the namespace fabric with environment kinds beyond the coding IDE (design-tool: canvas/editor surfaces with file artifacts; simulation: run/benchmark harnesses) — one lifecycle, one telemetry path, per-type starter contents + allowed commands | high | 4 | pending |
|
||||
| REQ-5-006 | Environment-typed learner surface: the build/defend flow accepts environment kind, telemetry captures per-kind events, grading digest stays kind-agnostic | high | 4 | pending |
|
||||
| REQ-5-005 | Design + simulation sandbox environment types (REQ-F-021 remainder): extend the namespace fabric with environment kinds beyond the coding IDE (design-tool: canvas/editor surfaces with file artifacts; simulation: run/benchmark harnesses) — one lifecycle, one telemetry path, per-type starter contents + allowed commands | high | 4 | complete |
|
||||
| REQ-5-006 | Environment-typed learner surface: the build/defend flow accepts environment kind, telemetry captures per-kind events, grading digest stays kind-agnostic | high | 4 | complete |
|
||||
|
||||
### Telemetry Durability
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-007 | Exec-telemetry seq-lease/replay-margin fix: WS ingest acknowledges received seqs; capture agent resumes from the ack on reconnect (bounded replay margin) — closes the P6-lesson one-line ACK gap with a real-server regression test | high | 1 | pending |
|
||||
| REQ-5-007 | Exec-telemetry seq-lease/replay-margin fix: WS ingest acknowledges received seqs; capture agent resumes from the ack on reconnect (bounded replay margin) — closes the P6-lesson one-line ACK gap with a real-server regression test | high | 1 | complete |
|
||||
|
||||
## v0.4 Requirements (Complete — Distribution & Bootstrap CLI, shipped as v0.3.4)
|
||||
|
||||
@@ -193,7 +193,7 @@
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.5 | activated (REQ-5-003/004) |
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.5 | activated → complete (REQ-5-003/004) |
|
||||
| REQ-F-018 | Payment processing and subscription management | high | v0.3+ | deferred |
|
||||
| REQ-F-019 | Human tutor marketplace (third-party courses) | medium | v0.4+ | deferred |
|
||||
| REQ-F-020 | CIRR-style placement tracking and audit | medium | v0.5+ | deferred |
|
||||
@@ -216,17 +216,17 @@
|
||||
|
||||
## Traceability Matrix
|
||||
|
||||
### v0.5 (current milestone)
|
||||
### v0.5 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-5-007 | 1 | pending |
|
||||
| REQ-5-001 | 2 | pending |
|
||||
| REQ-5-002 | 2 | pending |
|
||||
| REQ-5-003 | 3 | pending |
|
||||
| REQ-5-004 | 3 | pending |
|
||||
| REQ-5-005 | 4 | pending |
|
||||
| REQ-5-006 | 4 | pending |
|
||||
| REQ-5-007 | 1 | complete |
|
||||
| REQ-5-001 | 2 | complete |
|
||||
| REQ-5-002 | 2 | complete |
|
||||
| REQ-5-003 | 3 | complete |
|
||||
| REQ-5-004 | 3 | complete |
|
||||
| REQ-5-005 | 4 | complete |
|
||||
| REQ-5-006 | 4 | complete |
|
||||
|
||||
### v0.4 (complete)
|
||||
|
||||
|
||||
+11
-7
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
**Milestone v0.5 — IN PROGRESS (started 2026-09-13).** Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease: the four seams D-016 deferred out of v0.4. Server STT/TTS for voice defense (`openai-audio` VoiceProvider), identity verification + backend-enforced age-gating (REQ-F-017), design/simulation sandbox environments (REQ-F-021 remainder), and the exec-telemetry seq-lease/replay-margin fix.
|
||||
**Milestone v0.5 — COMPLETE (shipped as v0.4.5, 2026-09-13).** Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease: the four seams D-016 deferred out of v0.4. Server STT/TTS for voice defense (`openai-audio` VoiceProvider), identity verification + backend-enforced age-gating (REQ-F-017), design/simulation sandbox environments (REQ-F-021 remainder), and the exec-telemetry seq-lease/replay-margin fix.
|
||||
|
||||
**Milestone v0.4 — COMPLETE (shipped as v0.3.4, 2026-09-13; hotfixes v0.3.5 fresh-box, v0.3.6 single-port unattended deploy).** Distribution & Bootstrap CLI: `nextcraft` CLI (doctor/bootstrap/verify/dev) shipped as a linux x64 SEA binary, one-liner install script with checksum + version integrity gates, and binaries published on **every ongoing release** (v0.3.2 onward). v0.3.6 adds the single-port same-origin deploy (static export served by the ai-service on :8420) and unattended ops (`dev -d`/`stop`/`log`), with runtime state moved to `~/.nextcraft/`.
|
||||
|
||||
@@ -22,12 +22,12 @@
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | in-progress | — | — | Specification, clarify, research, plan, grill complete; .ciagent/ files updated for v0.5 |
|
||||
| 1 | Seq-lease + replay margin | pending | 0 | REQ-5-007 | Ingest acks received seqs (`seq_ack` frame); capture agent trims spool to ack on reconnect (bounded margin); real-server mid-burst regression test closes the P07-documented gap |
|
||||
| 2 | Real server voice | pending | 0 | REQ-5-001, REQ-5-002 | `openai-audio` provider passes STT/TTS contract tests (MockTransport); voice defense runs server-side end-to-end when keys exist; mock/browser paths unchanged; suite green |
|
||||
| 3 | Identity + age-gating | pending | 0 | REQ-5-003, REQ-5-004 | Identity protocol + mock backend + verification flow API; gated routes enforce 16+/18+ (allowlist → identity → rate caps); PII hygiene pinned by caplog test |
|
||||
| 4 | Design/sim environments | pending | 0 | REQ-5-005, REQ-5-006 | Template-layer env registry (build/design/simulation); per-kind starter contents + exec policy; test_command surfaced; grading digest kind-agnostic (pinned) |
|
||||
| 5 | Final review + ship | pending | 1-4 | — | Code review clean; audit passes; milestone tagged (final v0.4.x patch); release with binary assets on Gitea |
|
||||
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan, grill complete; .ciagent/ files updated for v0.5 |
|
||||
| 1 | Seq-lease + replay margin | complete | 0 | REQ-5-007 | Ingest acks received seqs (`seq_ack` frame); capture agent trims spool to ack on reconnect (bounded margin); real-server mid-burst regression test closes the P07-documented gap |
|
||||
| 2 | Real server voice | complete | 0 | REQ-5-001, REQ-5-002 | `openai-audio` provider passes STT/TTS contract tests (MockTransport); voice defense runs server-side end-to-end when keys exist; mock/browser paths unchanged; suite green |
|
||||
| 3 | Identity + age-gating | complete | 0 | REQ-5-003, REQ-5-004 | Identity protocol + mock backend + verification flow API; gated routes enforce 16+/18+ (allowlist → identity → rate caps); PII hygiene pinned by caplog test |
|
||||
| 4 | Design/sim environments | complete | 0 | REQ-5-005, REQ-5-006 | Template-layer env registry (build/design/simulation); per-kind starter contents + exec policy; test_command surfaced; grading digest kind-agnostic (pinned) |
|
||||
| 5 | Final review + ship | complete | 1-4 | — | Code review clean; audit passes; milestone tagged (final v0.4.x patch); release with binary assets on Gitea |
|
||||
|
||||
---
|
||||
|
||||
@@ -95,6 +95,10 @@
|
||||
|
||||
---
|
||||
|
||||
## v0.5 (Complete — Shipped as v0.4.5)
|
||||
|
||||
Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease: real server STT/TTS (`openai-audio` VoiceProvider with boot-safe fallback), the identity module (5th D-027 store, provider protocol + mock, D-043 age-gate composition on variants/sandboxes/defense/marketplace), design/simulation environment kinds at the template layer with per-kind exec policy, and the seq-ack protocol closing the P07 replay-margin gap. 6 phases (P0–P5). All 7 requirements (REQ-5-001..007) complete. Tags v0.4.0–v0.4.4 per phase, milestone release v0.4.5.
|
||||
|
||||
## v0.4 (Complete — Shipped as v0.3.4)
|
||||
|
||||
Distribution & Bootstrap CLI: `nextcraft` CLI (doctor/bootstrap/verify/dev) as a self-contained linux x64 SEA binary, one-liner install with sha256 + version integrity gates, release-asset pipeline attaching binaries to every ongoing release (v0.3.2 onward), install/quickstart docs backed by a fresh-clone E2E test. 5 phases (P0–P4). All 5 requirements (REQ-4-001..005) complete. Tags v0.3.0–v0.3.3 per phase, milestone release v0.3.4.
|
||||
|
||||
@@ -80,7 +80,7 @@ Durable state (SQLite DB, sandbox workdirs, daemon pid/log) lives in `~/.nextcra
|
||||
|
||||
## Status
|
||||
|
||||
**Milestone v0.4** — Distribution & Bootstrap CLI (one-liner install, `nextcraft` binary releases on every ship)
|
||||
**Milestone v0.5** — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease (shipped as v0.4.5)
|
||||
|
||||
Prior: v0.3 Credential Engines (shipped v0.2.8) · v0.2 AI Tutor Architecture (v0.2.0) · v0.1 UI/UX Prototype (v0.1.0)
|
||||
|
||||
|
||||
@@ -42,8 +42,35 @@ AI_SANDBOX_CREATES_PER_MIN=10
|
||||
# — build it with `NEXT_PUBLIC_AI_SERVICE_URL=self pnpm build`. The
|
||||
# `nextcraft dev` command auto-sets this when apps/web/out exists.
|
||||
# AI_WEB_STATIC_DIR=../../web/out
|
||||
# --- v0.3 Voice (REQ-3-006, D-030) ---
|
||||
# 'mock' (default; no key needed — tests/dev) or 'browser' (client-native SR/TTS).
|
||||
# Real server STT/TTS ('openai-audio' + AI_VOICE_BASE_URL/AI_VOICE_API_KEY)
|
||||
# is deferred to v0.4 per GRILL CUT-1/G-7 — keys never in code or commits.
|
||||
# --- Identity (REQ-5-003, D-042) ---
|
||||
# 'mock' (default — deterministic, no vendor spend pre-pilot; verdicts carry
|
||||
# mock=True forever per A-304). A real KYC vendor drops in via the
|
||||
# IdentityProvider protocol without API changes.
|
||||
AI_IDENTITY_PROVIDER=mock
|
||||
# G-13: identity submit caps — one active pending per learner (409), and a
|
||||
# per-learner submit rate ceiling (429 over a rolling 60s window).
|
||||
AI_IDENTITY_SUBMITS_PER_MIN=3
|
||||
|
||||
# --- Voice (REQ-3-006 D-030; real server path REQ-5-001, D-040) ---
|
||||
# 'mock' (default; no key needed — tests/dev), 'browser' (client-native
|
||||
# SR/TTS), or 'openai-audio' (real server STT/TTS, live since v0.5).
|
||||
AI_VOICE_PROVIDER=mock
|
||||
# openai-audio requires BOTH (unconfigured → app boots, voice falls back to
|
||||
# mock with a loud log — G-11; the badge then honestly reports mock):
|
||||
# AI_VOICE_BASE_URL=https://your-audio-endpoint/v1
|
||||
# AI_VOICE_API_KEY=
|
||||
# Optional model/voice/format knobs (defaults shown):
|
||||
# AI_VOICE_STT_MODEL=whisper-1
|
||||
# AI_VOICE_TTS_MODEL=tts-1
|
||||
# AI_VOICE_TTS_VOICE=alloy
|
||||
# AI_VOICE_TTS_FORMAT=mp3 (enum: mp3 | wav | opus)
|
||||
# AI_VOICE_MAX_AUDIO_MB=10
|
||||
# Manual probe recipe (executable by anyone with the keys; never CI):
|
||||
# STT: curl -sS $AI_VOICE_BASE_URL/audio/transcriptions \
|
||||
# -H "Authorization: Bearer $AI_VOICE_API_KEY" \
|
||||
# -F file=@test/fixtures/answer.wav -F model=whisper-1 | jq -e '.text'
|
||||
# TTS: curl -sS $AI_VOICE_BASE_URL/audio/speech \
|
||||
# -H "Authorization: Bearer $AI_VOICE_API_KEY" \
|
||||
# -H 'Content-Type: application/json' \
|
||||
# -d '{"model":"tts-1","input":"Nextcraft","voice":"alloy"}' \
|
||||
# -o /tmp/probe.mp3 && file /tmp/probe.mp3 | grep -i audio
|
||||
|
||||
@@ -22,7 +22,7 @@ from __future__ import annotations
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, Request, UploadFile
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
@@ -40,6 +40,7 @@ from .deps import (
|
||||
get_voice_provider,
|
||||
get_voice_store,
|
||||
)
|
||||
from .identity import require_verified_age # noqa: E402 (gate dep, D-043)
|
||||
|
||||
router = APIRouter(prefix="/v1/defense", tags=["defense"])
|
||||
|
||||
@@ -90,7 +91,7 @@ def _voice_descriptor(settings) -> VoiceDescriptor:
|
||||
|
||||
Must-Have #6: browser mode returns BROWSER_FALLBACK_DESCRIPTOR so the
|
||||
web client selects native SpeechRecognition/speechSynthesis; mock mode
|
||||
returns the mock descriptor. (A v0.4 server provider would return
|
||||
returns the mock descriptor. (A real server provider returns
|
||||
mode="server" — the protocol seam.)
|
||||
"""
|
||||
if (settings.voice_provider or "mock").strip().lower() == "browser":
|
||||
@@ -103,6 +104,7 @@ def _voice_descriptor(settings) -> VoiceDescriptor:
|
||||
@router.post("/start", response_model=StartResponse)
|
||||
async def start_defense(
|
||||
body: StartRequest,
|
||||
request: Request,
|
||||
examiner: ExaminerAgent = Depends(get_examiner),
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
voice_provider=Depends(get_voice_provider),
|
||||
@@ -110,6 +112,19 @@ async def start_defense(
|
||||
variant_store=Depends(get_variant_store),
|
||||
settings=Depends(get_settings),
|
||||
) -> StartResponse:
|
||||
# v0.5 identity gate (D-043): allowlist first (G-5), then the school
|
||||
# 16+ verified verdict, before any defense machinery runs.
|
||||
if body.learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=(
|
||||
f"learner_id {body.learner_id!r} is not on the sandbox "
|
||||
"allowlist (G-5)"
|
||||
),
|
||||
)
|
||||
await require_verified_age(
|
||||
16, body.learner_id, request.app.state.identity_store
|
||||
)
|
||||
record = voice_store.start(
|
||||
DefenseRecord(
|
||||
id=f"dfn-{int(time.time() * 1000):x}-{body.learner_id[:8]}",
|
||||
@@ -161,6 +176,7 @@ async def answer_defense(
|
||||
examiner: ExaminerAgent = Depends(get_examiner),
|
||||
trace_store=Depends(get_trace_store),
|
||||
variant_store=Depends(get_variant_store),
|
||||
settings=Depends(get_settings),
|
||||
) -> AnswerResponse:
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
@@ -183,11 +199,36 @@ async def answer_defense(
|
||||
raw = await audio.read()
|
||||
if not raw:
|
||||
# Empty upload is a client error (422), not a provider crash
|
||||
# (500): validate before the provider call so every provider —
|
||||
# mock today, the v0.4 real one — sees the same contract.
|
||||
# (500): validate before the provider call so every provider
|
||||
# sees the same contract.
|
||||
raise HTTPException(status_code=422, detail="audio upload is empty")
|
||||
fmt = (audio.content_type or "audio/wav").split("/")[-1]
|
||||
segment = await voice_provider.transcribe(raw, fmt)
|
||||
max_bytes = settings.voice_max_audio_mb * 1024 * 1024
|
||||
if len(raw) > max_bytes:
|
||||
# D-041/G-12: bounded audio BEFORE the provider call — the
|
||||
# client renders this as an honest re-record prompt.
|
||||
raise HTTPException(
|
||||
status_code=413,
|
||||
detail=(
|
||||
f"audio exceeds {settings.voice_max_audio_mb}MB "
|
||||
"— re-record a shorter answer"
|
||||
),
|
||||
)
|
||||
# D-041: strip codec params — MediaRecorder sends
|
||||
# 'audio/webm;codecs=opus'; the bare extension is the provider
|
||||
# contract ('webm'), else real STT endpoints reject the multipart.
|
||||
fmt = (audio.content_type or "audio/wav").split("/")[-1].split(";")[0].strip()
|
||||
try:
|
||||
segment = await voice_provider.transcribe(raw, fmt)
|
||||
except RuntimeError as exc:
|
||||
# Provider failure is the 502 house pattern (assessment.py /
|
||||
# proctor.py), not a 500: a real endpoint outage (or the
|
||||
# default mock's unscripted queue — final-review cross-phase
|
||||
# P0) must surface as an honest upstream error. Both providers
|
||||
# raise RuntimeError with sanitized text (mock: MockVoiceFailure;
|
||||
# openai-audio: key-redacted _sanitize).
|
||||
raise HTTPException(
|
||||
status_code=502, detail=f"voice transcription failed: {exc}"
|
||||
) from exc
|
||||
stt_ms = int((time.perf_counter() - stt_started) * 1000)
|
||||
text = segment.text
|
||||
|
||||
@@ -246,6 +287,7 @@ async def defense_audio(
|
||||
turn_id: int,
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
voice_provider=Depends(get_voice_provider),
|
||||
settings=Depends(get_settings),
|
||||
):
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
@@ -258,7 +300,11 @@ async def defense_audio(
|
||||
async for chunk in voice_provider.synthesize(turn.text):
|
||||
yield chunk
|
||||
|
||||
return StreamingResponse(stream(), media_type="audio/wav")
|
||||
# G-16/D-041: the TTS format is a settings enum; the media_type maps
|
||||
# from it (was hardcoded audio/wav — wrong for every real format).
|
||||
return StreamingResponse(
|
||||
stream(), media_type=f"audio/{settings.voice_tts_format}"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{defense_id}/finish", response_model=FinishResponse)
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
"""Identity API + age-gate dependencies (REQ-5-003/004, D-042/43).
|
||||
|
||||
Flow (all under mock provider by default; A-304 mock markers ride every
|
||||
response so downstream surfaces never treat mock-verified as real):
|
||||
POST /v1/identity/submit submission → pending (G-13 caps first)
|
||||
GET /v1/identity/status/{lid} latest record + mock marker
|
||||
POST /v1/identity/verify/{sid} poll provider → terminal transition
|
||||
|
||||
Gate dependencies (D-043 binding composition order, mounted by the gated
|
||||
routes — variants/sandbox-create/defense-start for school 16+; one
|
||||
marketplace route for 18+ verified):
|
||||
allowlist (403, G-5 pilot guard) → identity verdict (403 + verify-CTA)
|
||||
→ rate caps (429, owned by the calling routes)
|
||||
|
||||
PII (A-305): raw DOB enters via the submission, is used to derive the
|
||||
band, and is NEVER stored or logged (caplog sentinel test pins it).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from ..config import Settings
|
||||
from ..identity.base import IdentityProvider, IdentitySubmission
|
||||
from ..identity.store import IdentityRecord, IdentityStore
|
||||
from .deps import get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1/identity", tags=["identity"])
|
||||
|
||||
|
||||
# -- DI ------------------------------------------------------------------------
|
||||
|
||||
|
||||
def get_identity_store(request: Request) -> IdentityStore:
|
||||
return request.app.state.identity_store
|
||||
|
||||
|
||||
def get_identity_provider(request: Request) -> IdentityProvider:
|
||||
return request.app.state.identity_provider
|
||||
|
||||
|
||||
# -- models ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class SubmitBody(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
#: ISO date. Validated AT THE BOUNDARY (D1 verifier fix): a malformed
|
||||
#: value would otherwise blow up as a 500 inside derive_age_band on the
|
||||
#: verify path — echoing the raw DOB into the traceback (A-305) and
|
||||
#: leaving a poisoned pending record that G-13 turns into a permanent
|
||||
#: learner lockout.
|
||||
date_of_birth: str = Field(description="ISO date; never stored or logged")
|
||||
document_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
@field_validator("date_of_birth")
|
||||
@classmethod
|
||||
def _validate_dob(cls, value: str) -> str:
|
||||
try:
|
||||
datetime.fromisoformat(value)
|
||||
except ValueError as exc:
|
||||
# 422 with the input scrubbed (A-305/D1 — the app-level
|
||||
# RequestValidationError handler redacts PII field inputs).
|
||||
raise ValueError("date_of_birth must be an ISO date (YYYY-MM-DD)") from exc
|
||||
return value
|
||||
|
||||
|
||||
class SubmitResponse(BaseModel):
|
||||
submission_id: str
|
||||
status: str
|
||||
#: A-304: honesty marker — a mock verdict is NEVER production-verified.
|
||||
mock: bool = True
|
||||
detail: str = ""
|
||||
|
||||
|
||||
class StatusResponse(BaseModel):
|
||||
learner_id: str
|
||||
status: str
|
||||
age_band: str | None
|
||||
mock: bool
|
||||
verified_at: str | None
|
||||
|
||||
|
||||
class VerifyResponse(SubmitResponse):
|
||||
age_band: str | None
|
||||
|
||||
|
||||
# -- G-13 submit caps ---------------------------------------------------------------
|
||||
|
||||
|
||||
class _SubmitRateLimiter:
|
||||
"""Per-learner submit rate cap (in-memory, process-local — the G-5
|
||||
creates-per-min pattern from the sandboxes route)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._window: dict[str, list[float]] = {}
|
||||
|
||||
def check(self, learner_id: str, per_min: int) -> None:
|
||||
now = time.monotonic()
|
||||
window = self._window.setdefault(learner_id, [])
|
||||
window[:] = [t for t in window if now - t < 60.0]
|
||||
if len(window) >= per_min:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail="identity submit rate exceeded — wait a minute",
|
||||
)
|
||||
window.append(now)
|
||||
|
||||
|
||||
_rate_limiter = _SubmitRateLimiter()
|
||||
|
||||
|
||||
# -- verification flow ------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/submit", response_model=SubmitResponse)
|
||||
async def submit_identity(
|
||||
body: SubmitBody,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
provider: IdentityProvider = Depends(get_identity_provider),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> SubmitResponse:
|
||||
# G-13: one active pending submission per learner — resubmit while
|
||||
# pending echoes the pending state (409), not a second submission.
|
||||
if store.count_pending_for_learner(body.learner_id) > 0:
|
||||
latest = store.latest_for_learner(body.learner_id)
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail={
|
||||
"reason": "submission_pending",
|
||||
"submission_id": latest.id if latest else None,
|
||||
"status": "pending",
|
||||
},
|
||||
)
|
||||
_rate_limiter.check(body.learner_id, settings.identity_submits_per_min)
|
||||
|
||||
submission = IdentitySubmission(
|
||||
learner_id=body.learner_id,
|
||||
date_of_birth=body.date_of_birth,
|
||||
document_refs=body.document_refs,
|
||||
)
|
||||
submission_id = await provider.submit(submission)
|
||||
|
||||
record = store.insert(
|
||||
IdentityRecord(
|
||||
id=submission_id,
|
||||
learner_id=body.learner_id,
|
||||
status="pending",
|
||||
provider="mock" if settings.identity_provider == "mock" else settings.identity_provider,
|
||||
document_refs=body.document_refs, # A-305: opaque handles, never contents
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
return SubmitResponse(
|
||||
submission_id=record.id, status=record.status, mock=record.mock
|
||||
)
|
||||
|
||||
|
||||
@router.get("/status/{learner_id}", response_model=StatusResponse)
|
||||
async def identity_status(
|
||||
learner_id: str,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
) -> StatusResponse:
|
||||
record = store.latest_for_learner(learner_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail="no identity record")
|
||||
return StatusResponse(
|
||||
learner_id=learner_id,
|
||||
status=record.status,
|
||||
age_band=record.age_band,
|
||||
mock=record.mock,
|
||||
verified_at=record.verified_at.isoformat() if record.verified_at else None,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/verify/{submission_id}", response_model=VerifyResponse)
|
||||
async def verify_identity(
|
||||
submission_id: str,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
provider: IdentityProvider = Depends(get_identity_provider),
|
||||
) -> VerifyResponse:
|
||||
record = store.get(submission_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail="no such submission")
|
||||
if record.status != "pending":
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"submission already {record.status}",
|
||||
)
|
||||
|
||||
verdict = await provider.poll(submission_id)
|
||||
# A-305: derive the band from the provider's verdict; the raw DOB never
|
||||
# entered the store and is not logged here.
|
||||
updated = store.mark_verified(
|
||||
submission_id, verdict.model_dump(), verdict.age_band
|
||||
)
|
||||
assert updated is not None # record existed a moment ago
|
||||
return VerifyResponse(
|
||||
submission_id=updated.id,
|
||||
status=updated.status,
|
||||
age_band=updated.age_band,
|
||||
mock=updated.mock,
|
||||
detail=verdict.detail,
|
||||
)
|
||||
|
||||
|
||||
# -- age-gate dependencies (D-043 composition) -------------------------------------
|
||||
|
||||
|
||||
def _verify_cta_payload(
|
||||
reason: str, min_age: int, record: IdentityRecord | None
|
||||
) -> dict:
|
||||
"""A-306/UX acceptance #2: an actionable 403 — never a bare error."""
|
||||
return {
|
||||
"reason": reason,
|
||||
"min_age": min_age,
|
||||
"current_status": record.status if record else "none",
|
||||
"verify_cta": "/enroll",
|
||||
}
|
||||
|
||||
|
||||
async def require_verified_age(
|
||||
min_age: int,
|
||||
learner_id: str,
|
||||
store: IdentityStore,
|
||||
) -> IdentityRecord:
|
||||
"""The identity half of the D-043 composition (allowlist runs FIRST in
|
||||
the calling routes; this is the second gate; caps come after).
|
||||
|
||||
School 16+ → min_age=16; marketplace 18+ verified → min_age=18.
|
||||
"""
|
||||
record = store.latest_for_learner(learner_id)
|
||||
if record is None or record.status != "verified":
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("identity_verification_required", min_age, record),
|
||||
)
|
||||
# D2 (verifier): FAIL CLOSED. Only canonical bands can pass — None,
|
||||
# unknown, or under-16 bands reject (the gate is the security boundary
|
||||
# for the future vendor and direct store writes; it never trusts a
|
||||
# band it does not recognize).
|
||||
if record.age_band not in ("16-17", "18+"):
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("identity_verification_required", min_age, record),
|
||||
)
|
||||
if min_age > 16 and record.age_band != "18+":
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("age_gate_18_plus", 18, record),
|
||||
)
|
||||
return record
|
||||
|
||||
|
||||
#: Convenience alias for the marketplace 18+ composition (D-043's
|
||||
#: `require_verified_adult` — direct calls use require_verified_age(18, ...)).
|
||||
require_verified_adult = require_verified_age
|
||||
|
||||
|
||||
# -- marketplace 18+ gated stub (G-18, REQ-5-004) ------------------------------------
|
||||
|
||||
marketplace_router = APIRouter(prefix="/v1/marketplace", tags=["marketplace"])
|
||||
|
||||
|
||||
class MarketplaceApplyBody(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
job_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
@marketplace_router.post("/apply", status_code=501)
|
||||
async def marketplace_apply_stub(
|
||||
body: MarketplaceApplyBody,
|
||||
request: Request,
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> dict:
|
||||
"""The ONE gated marketplace route (D-043): proves the 18+ verified
|
||||
composition end-to-end. G-18 honesty: after passing the gate it returns
|
||||
501 with explicit stub + mock markers — the marketplace backend does
|
||||
not exist yet; this route never fabricates an 'applied' outcome.
|
||||
"""
|
||||
if body.learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=f"learner_id {body.learner_id!r} is not on the sandbox allowlist (G-5)",
|
||||
)
|
||||
await require_verified_age(18, body.learner_id, get_identity_store(request))
|
||||
return {
|
||||
"detail": "marketplace applications are not live yet",
|
||||
"stub": True,
|
||||
"mock": True,
|
||||
}
|
||||
@@ -25,7 +25,7 @@ import time
|
||||
from collections import deque
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Response
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from ..config import Settings
|
||||
@@ -94,6 +94,10 @@ class SnapshotResponse(BaseModel):
|
||||
# -- abuse control (G-5; middleware layer, not auth) ---------------------------
|
||||
|
||||
|
||||
from ..variants.templates import get_template # noqa: E402
|
||||
from .identity import require_verified_age # noqa: E402 (gate dep, D-043)
|
||||
|
||||
|
||||
def _enforce_allowlist(learner_id: str, settings: Settings) -> None:
|
||||
if learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
@@ -139,10 +143,16 @@ def _check_global_create_rate(settings: Settings) -> None:
|
||||
@router.post("", status_code=201, response_model=SandboxResponse)
|
||||
async def create_sandbox(
|
||||
body: SandboxCreateRequest,
|
||||
request: Request,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> SandboxResponse:
|
||||
# v0.5 identity gate (D-043, REQ-5-004): allowlist (G-5) → identity
|
||||
# verdict (403 + verify-CTA) → caps (429) — the binding composition.
|
||||
_enforce_allowlist(body.learner_id, settings)
|
||||
await require_verified_age(
|
||||
16, body.learner_id, request.app.state.identity_store
|
||||
)
|
||||
_check_per_learner_cap(await manager.list(), body.learner_id, settings)
|
||||
_check_global_create_rate(settings)
|
||||
try:
|
||||
@@ -335,10 +345,54 @@ async def write_file(
|
||||
return {"path": body.path, "written": True}
|
||||
|
||||
|
||||
#: G-15 (REQ-5-005): per-kind exec command policy — EXACT argv[0] token
|
||||
#: matching, never prefix/substring (trivially bypassed via flags/-c
|
||||
#: passthrough). 'sh -c' passthrough is DISALLOWED for design/simulation
|
||||
#: kinds: the gaming vector would be faking build-style test cycles into a
|
||||
#: kind-agnostic digest. Build kinds keep v0.3 behavior (any command —
|
||||
#: the CUT-2 surface is Run/Test buttons, not a shell relay).
|
||||
#: python (bare) is deliberately absent — in-ns PATH resolves only python3
|
||||
#: (verifier P1); pip is absent (no network in the namespace).
|
||||
_GENERIC_FIRST_TOKENS = frozenset(
|
||||
{"ls", "cat", "pwd", "echo", "python3", "pytest"}
|
||||
)
|
||||
|
||||
|
||||
def _enforce_exec_policy(
|
||||
cmd: list[str], environment: str | None, allowed: set[str] | None = None
|
||||
) -> None:
|
||||
"""422 with the allowed set when a design/sim command is out of policy.
|
||||
|
||||
`allowed` defaults to the generic set; the exec route unions in the
|
||||
template's DECLARED harness argv[0] (a future non-python harness
|
||||
template must not reject its own Run command)."""
|
||||
if environment not in ("design", "simulation"):
|
||||
return # build kind: unchanged v0.3 semantics
|
||||
allowed = set(allowed) if allowed is not None else set(_GENERIC_FIRST_TOKENS)
|
||||
first = cmd[0] if cmd else ""
|
||||
if first in ("sh", "bash"):
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"shell passthrough is not allowed in a {environment} "
|
||||
f"environment; allowed commands: {sorted(allowed)}"
|
||||
),
|
||||
)
|
||||
if first not in allowed:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"command {first!r} is not allowed in a {environment} "
|
||||
f"environment; allowed commands: {sorted(allowed)}"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{sandbox_id}/exec", response_model=ExecResponse)
|
||||
async def exec_command(
|
||||
sandbox_id: str,
|
||||
body: ExecRequest,
|
||||
request: Request,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
) -> ExecResponse:
|
||||
try:
|
||||
@@ -349,5 +403,19 @@ async def exec_command(
|
||||
handle = manager._handles.get(sandbox_id) # noqa: SLF001
|
||||
if handle is None:
|
||||
raise HTTPException(status_code=404, detail=f"no live handle {sandbox_id!r}")
|
||||
# REQ-5-005 (G-15): resolve the sandbox's variant environment by its
|
||||
# task_id (manager side-table) and enforce the per-kind command policy
|
||||
# BEFORE execution.
|
||||
task_id = manager._task_ids.get(sandbox_id) # noqa: SLF001 - composition seam
|
||||
if task_id:
|
||||
variant = request.app.state.variant_store.get_by_task(task_id)
|
||||
if variant is not None:
|
||||
template = get_template(variant.template_id)
|
||||
declared = (
|
||||
{template.run_command.split()[0]} if template is not None else set()
|
||||
)
|
||||
_enforce_exec_policy(
|
||||
body.cmd, variant.environment, _GENERIC_FIRST_TOKENS | declared
|
||||
)
|
||||
result = await backend.exec(handle, body.cmd)
|
||||
return ExecResponse(**result.model_dump())
|
||||
|
||||
@@ -33,13 +33,29 @@ tests/api/test_variants.py asserts this end-to-end through the API.
|
||||
from datetime import datetime
|
||||
from typing import Self
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
from ..config import Settings
|
||||
from ..identity.store import IdentityStore
|
||||
from ..variants.generator import VariantGenerator
|
||||
from ..variants.store import VariantRecord, VariantStore
|
||||
from ..variants.templates import TaskTemplate, get_template, template_for_competency
|
||||
from .deps import get_variant_generator, get_variant_store
|
||||
from .deps import get_settings, get_variant_generator, get_variant_store
|
||||
from .identity import require_verified_age
|
||||
|
||||
|
||||
def _identity_store(request: Request) -> IdentityStore:
|
||||
return request.app.state.identity_store
|
||||
|
||||
|
||||
def _enforce_allowlist(learner_id: str, settings: Settings) -> None:
|
||||
"""G-5 pilot guard — allowlist runs FIRST in the composition (D-043)."""
|
||||
if learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=f"learner_id {learner_id!r} is not on the sandbox allowlist (G-5)",
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/v1/variants", tags=["variants"])
|
||||
|
||||
@@ -81,6 +97,9 @@ class VariantResponse(BaseModel):
|
||||
params: dict[str, str | int]
|
||||
statement: str
|
||||
starter_files: dict[str, str]
|
||||
#: REQ-5-005 (a-11): REQUIRED on the wire — always emitted.
|
||||
environment: str
|
||||
test_command: str
|
||||
created_at: datetime
|
||||
|
||||
|
||||
@@ -138,6 +157,9 @@ def _to_response(record: VariantRecord, competency_id: str) -> VariantResponse:
|
||||
params=dict(record.params),
|
||||
statement=record.statement,
|
||||
starter_files=dict(record.starter_files),
|
||||
# a-11: REQUIRED on the wire — the server always emits both (v0.5).
|
||||
environment=record.environment or "build",
|
||||
test_command=record.test_command or "pytest -q",
|
||||
created_at=record.created_at,
|
||||
)
|
||||
|
||||
@@ -148,12 +170,19 @@ def _to_response(record: VariantRecord, competency_id: str) -> VariantResponse:
|
||||
@router.post("", response_model=VariantResponse)
|
||||
async def generate_variant(
|
||||
body: VariantGenerateRequest,
|
||||
request: Request,
|
||||
generator: VariantGenerator = Depends(get_variant_generator),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> VariantResponse:
|
||||
"""The learner's variant for the resolved template — generated on the
|
||||
first request, cached (no LLM call) on every repeat: D-029 makes a
|
||||
regenerate a 200 of the SAME stored variant.
|
||||
|
||||
v0.5 identity gate (D-043, REQ-5-004): the school floor is 16+ verified.
|
||||
Composition order: G-5 allowlist (403) → identity verdict (403 + CTA).
|
||||
"""
|
||||
_enforce_allowlist(body.learner_id, settings)
|
||||
await require_verified_age(16, body.learner_id, _identity_store(request))
|
||||
template = _resolve_template(body)
|
||||
record = await generator.generate(body.learner_id, template.id)
|
||||
return _to_response(record, competency_id=template.competency_id)
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Service settings — pydantic-settings, env prefix AI_, .env support."""
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
|
||||
@@ -15,6 +16,9 @@ _SERVICE_ROOT = Path(__file__).resolve().parent.parent
|
||||
# workdirs; env overrides may use ~/ paths (expanded by the validator below).
|
||||
_STATE_ROOT = Path.home() / ".nextcraft"
|
||||
|
||||
# G-16: TTS response-format whitelist (feeds the TTS route's Content-Type).
|
||||
_TTS_FORMATS = ("mp3", "wav", "opus")
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_prefix="AI_", env_file=".env", extra="ignore")
|
||||
@@ -88,6 +92,22 @@ class Settings(BaseSettings):
|
||||
return Path(value).expanduser()
|
||||
return value
|
||||
|
||||
@field_validator("voice_tts_format")
|
||||
@classmethod
|
||||
def _validate_tts_format(cls, value: str) -> str:
|
||||
# G-16 + G-11 consistency: unknown values NEVER crash the boot —
|
||||
# fall back to the default with a loud warning (the boot-survival
|
||||
# log lives in main.py's voice fallback; this validator normalizes).
|
||||
v = value.strip().lower()
|
||||
if v not in _TTS_FORMATS:
|
||||
logging.getLogger(__name__).warning(
|
||||
"AI_VOICE_TTS_FORMAT=%r is not one of %s — falling back to 'mp3'",
|
||||
value,
|
||||
_TTS_FORMATS,
|
||||
)
|
||||
return "mp3"
|
||||
return v
|
||||
|
||||
# G-3 flood control (NOT backpressure-by-silence): max events ingested per
|
||||
# (learner_id, task_id) trace before the WS endpoint closes the connection
|
||||
# with 1008 and marks the trace INCOMPLETE_FLOODED. Drop-oldest is
|
||||
@@ -117,10 +137,38 @@ class Settings(BaseSettings):
|
||||
return ["*"]
|
||||
return [o.strip() for o in value.split(",") if o.strip()]
|
||||
|
||||
# Voice provider selection (REQ-3-006, D-030): 'mock' (default — the
|
||||
# no-key path is first-class; tests never call a real voice API) or
|
||||
# 'browser' (browser-native SpeechRecognition/speechSynthesis fallback;
|
||||
# the descriptor tells the web client). The real server STT/TTS
|
||||
# ('openai-audio') is a v0.4 seam (GRILL CUT-1 / G-7) — AI_VOICE_BASE_URL
|
||||
# and AI_VOICE_API_KEY are documented in .env.example for that future.
|
||||
# Identity provider selection (REQ-5-003, A-303): 'mock' (default —
|
||||
# deterministic, no vendor spend pre-pilot; verdicts carry mock=True
|
||||
# forever per A-304). A real KYC vendor drops in via the
|
||||
# IdentityProvider protocol without API changes.
|
||||
identity_provider: str = "mock"
|
||||
# G-13: identity submit caps — one active pending per learner (409 on
|
||||
# resubmit) and a per-learner submit rate ceiling.
|
||||
identity_submits_per_min: int = 3
|
||||
|
||||
# Voice provider selection (REQ-5-001, D-040): 'mock' (default — the
|
||||
# no-key path is first-class; tests never call a real voice API),
|
||||
# 'browser' (client-native SR/TTS; the descriptor tells the web client),
|
||||
# or 'openai-audio' (real server STT/TTS against an OpenAI-compatible
|
||||
# audio endpoint). openai-audio requires voice_base_url + voice_api_key;
|
||||
# when unconfigured the lifespan falls back to mock with a loud log
|
||||
# (G-11 — a typo'd env must never crash the unattended boot).
|
||||
voice_provider: str = "mock"
|
||||
|
||||
# Real server voice (D-040, A-301): endpoint-agnostic by config (D-014
|
||||
# pattern) — any OpenAI-compatible audio API works. Keys env-only,
|
||||
# never committed, never logged (mirrors ollama_cloud_api_key).
|
||||
voice_base_url: str = ""
|
||||
voice_api_key: str = ""
|
||||
voice_stt_model: str = "whisper-1"
|
||||
voice_tts_model: str = "tts-1"
|
||||
voice_tts_voice: str = "alloy"
|
||||
# G-16: whitelist, not free string — this feeds the TTS route's
|
||||
# Content-Type. A str + mode-after validator (NOT a pydantic Literal):
|
||||
# a Literal would raise ValidationError at Settings construction, before
|
||||
# main.py's G-11 fallback could catch it — crashing the unattended boot
|
||||
# on a typo'd env. Invalid values fall back to the default LOUDLY.
|
||||
voice_tts_format: str = "mp3"
|
||||
# A-302/D-041: upload guard before the provider call (webm/opus is
|
||||
# ~0.5-1MB/min, so 10MB tolerates very long answers).
|
||||
voice_max_audio_mb: int = 10
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
"""Identity engine package — provider protocol + store (REQ-5-003, D-042)."""
|
||||
|
||||
from .base import (
|
||||
AgeBand,
|
||||
IdentityProvider,
|
||||
IdentityStatus,
|
||||
IdentitySubmission,
|
||||
IdentityVerdict,
|
||||
)
|
||||
from .mock import MockIdentityProvider, derive_age_band
|
||||
from .store import IdentityRecord, IdentityStore, SQLiteIdentityStore
|
||||
|
||||
__all__ = [
|
||||
"AgeBand",
|
||||
"IdentityStatus",
|
||||
"IdentitySubmission",
|
||||
"IdentityVerdict",
|
||||
"IdentityProvider",
|
||||
"IdentityRecord",
|
||||
"IdentityStore",
|
||||
"SQLiteIdentityStore",
|
||||
"MockIdentityProvider",
|
||||
"derive_age_band",
|
||||
]
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Identity verification protocol (REQ-5-003, D-042, A-303).
|
||||
|
||||
Provider-agnostic like LLMProvider/VoiceProvider (D-014/D-030): a narrow
|
||||
protocol the identity API composes via DI, a deterministic mock, and a
|
||||
future real KYC vendor (Stripe Identity / Persona / Onfido class) that
|
||||
drops in without API changes. PII rules (A-305): the provider sees
|
||||
document REFERENCES, never raw documents; verdicts carry a mock marker
|
||||
(A-304) so downstream surfaces never display mock-verified as
|
||||
production-verified.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal, Protocol, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
AgeBand = Literal["16-17", "18+"]
|
||||
IdentityStatus = Literal["pending", "verified", "rejected"]
|
||||
|
||||
|
||||
class IdentitySubmission(BaseModel):
|
||||
"""What a learner submits: derived data + document refs only.
|
||||
|
||||
`date_of_birth` is a REAL date (the provider derives the age band) but
|
||||
raw DOB is NEVER persisted — only the derived band (A-305). Document
|
||||
refs are opaque handles (upload ids), never contents.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
learner_id: str = Field(min_length=1)
|
||||
date_of_birth: str = Field(description="ISO date; used to derive age_band, never stored")
|
||||
document_refs: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Opaque upload handles; raw documents are never stored",
|
||||
)
|
||||
|
||||
|
||||
class IdentityVerdict(BaseModel):
|
||||
"""Provider verdict — what gets stored + surfaced."""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
status: IdentityStatus
|
||||
age_band: AgeBand | None = None
|
||||
provider: str
|
||||
#: A-304 honesty: mock verdicts carry mock=True forever — downstream
|
||||
#: surfaces must never treat a mock verdict as production-verified.
|
||||
mock: bool = True
|
||||
detail: str = ""
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class IdentityProvider(Protocol):
|
||||
"""The KYC port: submit → (poll) → verdict. Never imports api/."""
|
||||
|
||||
async def submit(self, submission: IdentitySubmission) -> str:
|
||||
"""Start verification; returns a submission id (minted once)."""
|
||||
...
|
||||
|
||||
async def poll(self, submission_id: str) -> IdentityVerdict:
|
||||
"""Fetch the (possibly pending) verdict for a submission."""
|
||||
...
|
||||
@@ -0,0 +1,94 @@
|
||||
"""Deterministic mock identity provider (REQ-5-003, A-303).
|
||||
|
||||
Approve-on-policy: every submission verifies unless the caller scripts a
|
||||
rejection (by learner id) or the derived age band fails the floor
|
||||
(under-16 → rejected with an age detail). Verdicts are mock-marked (A-304)
|
||||
— the marker rides every verdict so no downstream surface can ever
|
||||
display mock-verified as production-verified. Never calls the network.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import itertools
|
||||
import secrets
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from .base import IdentitySubmission, IdentityVerdict
|
||||
|
||||
|
||||
def derive_age_band(date_of_birth: str, today: datetime | None = None) -> str:
|
||||
"""Derive the age band from an ISO date. Under-16 returns "under-16".
|
||||
|
||||
Pure + deterministic; used by the store test and the API alike.
|
||||
"""
|
||||
dob = datetime.fromisoformat(date_of_birth)
|
||||
now = today or datetime.now(UTC)
|
||||
age = now.year - dob.year - (
|
||||
(now.month, now.day) < (dob.month, dob.day)
|
||||
)
|
||||
if age < 16:
|
||||
return "under-16"
|
||||
if age < 18:
|
||||
return "16-17"
|
||||
return "18+"
|
||||
|
||||
|
||||
class MockIdentityProvider:
|
||||
"""Scriptable, deterministic; no network, no vendor calls.
|
||||
|
||||
Submission ids are minted UNIQUELY per submit() call (a monotonic
|
||||
counter + the per-process seed from `secrets`): the id is the PK of
|
||||
the insert-only IdentityStore, and a deterministic id derived from
|
||||
(learner_id, date_of_birth) collides on any resubmit-after-terminal
|
||||
(e.g. a rejected learner retrying with the same DOB) — the store
|
||||
surfaces IntegrityError and the API would 500 (cross-phase P0,
|
||||
final review). Uniqueness per call is the contract; determinism of
|
||||
VERDICTS (what tests actually pin) is preserved — poll() derives the
|
||||
band purely from the stored submission.
|
||||
"""
|
||||
|
||||
def __init__(self, reject_learners: set[str] | None = None) -> None:
|
||||
self._submissions: dict[str, IdentitySubmission] = {}
|
||||
self._reject_learners = reject_learners or set()
|
||||
# Per-process nonce: ids are opaque handles (A-305) — never
|
||||
# derived from PII. Counter + nonce keeps ids unique within and
|
||||
# across provider instances on one box.
|
||||
self._nonce = secrets.randbits(32)
|
||||
self._counter = itertools.count()
|
||||
|
||||
async def submit(self, submission: IdentitySubmission) -> str:
|
||||
submission_id = f"idc-{self._nonce:08x}{next(self._counter):08x}"
|
||||
self._submissions[submission_id] = submission
|
||||
return submission_id
|
||||
|
||||
async def poll(self, submission_id: str) -> IdentityVerdict:
|
||||
submission = self._submissions.get(submission_id)
|
||||
if submission is None:
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="unknown submission id",
|
||||
)
|
||||
if submission.learner_id in self._reject_learners:
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="scripted rejection (test)",
|
||||
)
|
||||
band = derive_age_band(submission.date_of_birth)
|
||||
if band == "under-16":
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="under 16 — the AI school floor is 16+ (COPPA avoidance)",
|
||||
)
|
||||
return IdentityVerdict(
|
||||
status="verified",
|
||||
age_band=band, # type: ignore[arg-type]
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="mock verdict — not production verification",
|
||||
)
|
||||
@@ -0,0 +1,214 @@
|
||||
"""IdentityStore — verification records (REQ-5-003, D-042, D-027 FIFTH store).
|
||||
|
||||
Insert-only + latest-per-learner lookup, modeled on the DefenseStore
|
||||
conventions: WAL + synchronous=NORMAL + busy_timeout + foreign_keys=ON
|
||||
pragmas at connect time, portable column types (str/datetime/JSON) for
|
||||
Postgres parity, @validates hooks for constraints sqlmodel's metaclass
|
||||
drops, tz-aware→naive→tz-aware boundary normalization.
|
||||
|
||||
PII contract (A-305): stores the DERIVED age_band (16-17 | 18+), NEVER a
|
||||
raw date of birth; document_refs are opaque handles, NEVER contents.
|
||||
Verdict provenance is audit data: every record carries provider + the
|
||||
mock marker (A-304) so downstream surfaces can label unverified state
|
||||
honestly.
|
||||
|
||||
Insert-only growth is fine at pilot scale (a-12): learner_id indexed,
|
||||
latest-per-learner lookup, no compaction pre-vendor.
|
||||
|
||||
Boundary (D-027): `identity/` never imports `agents/` / `api/`; this
|
||||
module imports config only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Protocol
|
||||
|
||||
from sqlalchemy import event, text
|
||||
from sqlalchemy.types import JSON, String
|
||||
from sqlmodel import Field, Session, SQLModel, create_engine
|
||||
|
||||
from ..config import Settings
|
||||
from .base import IdentityStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class IdentityRecord(SQLModel, table=True):
|
||||
"""One verification submission's lifecycle + verdict provenance."""
|
||||
|
||||
__tablename__ = "identity_record"
|
||||
|
||||
#: PK = the submission id minted once by the provider's submit().
|
||||
id: str = Field(primary_key=True)
|
||||
learner_id: str = Field(index=True)
|
||||
# Bare Literal annotations crash sqlmodel's column inference; explicit
|
||||
# sa_type + the validates hook below give the same contract
|
||||
# (VARCHAR column, Literal-rejected values — DefenseStore pattern).
|
||||
status: IdentityStatus = Field(default="pending", sa_type=String)
|
||||
provider: str
|
||||
#: Verdict provenance: the provider's raw verdict (mock-marked).
|
||||
verdict: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
#: DERIVED band only — raw DOB is never persisted (A-305).
|
||||
age_band: str | None = Field(default=None, sa_type=String)
|
||||
#: Opaque document handles — raw documents never stored (A-305).
|
||||
document_refs: list[str] = Field(default_factory=list, sa_type=JSON)
|
||||
submitted_at: datetime
|
||||
verified_at: datetime | None = Field(default=None)
|
||||
|
||||
@property
|
||||
def mock(self) -> bool:
|
||||
"""A-304: the mock marker rides every surface (record + API)."""
|
||||
return bool(self.verdict.get("mock", True))
|
||||
|
||||
def _validate(self) -> None:
|
||||
if not self.id or not self.learner_id:
|
||||
raise ValueError("id and learner_id must be non-empty")
|
||||
if self.status not in ("pending", "verified", "rejected"):
|
||||
raise ValueError(f"invalid identity status {self.status!r}")
|
||||
# D3 (verifier): a stored band must be canonical or None (pending).
|
||||
# The gate fails closed on anything else; the store refuses to
|
||||
# create it in the first place.
|
||||
if self.age_band is not None and self.age_band not in (
|
||||
"16-17",
|
||||
"18+",
|
||||
"under-16",
|
||||
):
|
||||
raise ValueError(f"invalid age_band {self.age_band!r}")
|
||||
|
||||
def _normalize(self) -> None:
|
||||
self.submitted_at = _as_utc(self.submitted_at)
|
||||
if self.verified_at is not None:
|
||||
self.verified_at = _as_utc(self.verified_at)
|
||||
|
||||
|
||||
def _as_utc(ts: datetime) -> datetime:
|
||||
"""SQLite stores naive; read paths re-label tz-aware UTC (D-027 pattern)."""
|
||||
if ts.tzinfo is None:
|
||||
return ts.replace(tzinfo=UTC)
|
||||
return ts
|
||||
|
||||
|
||||
def _sqlite_connect(dbapi_connection: object, _: object) -> None:
|
||||
"""Per-connection pragmas — mirrors the other D-027 stores."""
|
||||
cursor = dbapi_connection.cursor() # type: ignore[attr-defined]
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA synchronous=NORMAL")
|
||||
cursor.execute("PRAGMA busy_timeout=5000")
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close() # type: ignore[attr-defined]
|
||||
|
||||
|
||||
class IdentityStore(Protocol):
|
||||
"""Persistence contract for identity records."""
|
||||
|
||||
def insert(self, record: IdentityRecord) -> IdentityRecord:
|
||||
"""INSERT-ONLY: a duplicate id raises IntegrityError (surfaced, not
|
||||
swallowed — a submission id is minted once)."""
|
||||
...
|
||||
|
||||
def get(self, submission_id: str) -> IdentityRecord | None:
|
||||
"""Point lookup by submission id."""
|
||||
...
|
||||
|
||||
def latest_for_learner(self, learner_id: str) -> IdentityRecord | None:
|
||||
"""Newest record for the learner (or None)."""
|
||||
...
|
||||
|
||||
def count_pending_for_learner(self, learner_id: str) -> int:
|
||||
"""G-13: active pending submissions (cap = 1)."""
|
||||
...
|
||||
|
||||
def mark_verified(
|
||||
self, submission_id: str, verdict: dict[str, Any], age_band: str | None
|
||||
) -> IdentityRecord | None:
|
||||
"""Terminal transition (verified or rejected): stamp + store the
|
||||
provider verdict + derived band. Unknown id → None."""
|
||||
...
|
||||
|
||||
|
||||
class SQLiteIdentityStore:
|
||||
"""SQLite implementation of IdentityStore (D-027)."""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
self._db_path: Path = db_path if db_path is not None else Settings().db_path
|
||||
self._engine = create_engine(
|
||||
f"sqlite:///{self._db_path}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
event.listen(self._engine, "connect", _sqlite_connect)
|
||||
SQLModel.metadata.create_all(self._engine)
|
||||
|
||||
def insert(self, record: IdentityRecord) -> IdentityRecord:
|
||||
record._validate()
|
||||
record._normalize()
|
||||
with Session(self._engine) as session:
|
||||
session.add(record)
|
||||
session.commit() # IntegrityError SURFACES (insert-only, minted-once)
|
||||
session.refresh(record)
|
||||
return record
|
||||
|
||||
def get(self, submission_id: str) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
rec = session.get(IdentityRecord, submission_id)
|
||||
if rec is None:
|
||||
return None
|
||||
session.refresh(rec)
|
||||
rec.submitted_at = _as_utc(rec.submitted_at)
|
||||
if rec.verified_at is not None:
|
||||
rec.verified_at = _as_utc(rec.verified_at)
|
||||
return rec
|
||||
|
||||
def latest_for_learner(self, learner_id: str) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
# D4 (verifier): submitted_at alone can tie at microsecond
|
||||
# resolution — sqlite rowid breaks the tie deterministically
|
||||
# (last inserted wins, mirroring insert-only chronology).
|
||||
# rowid is a SQLite physical column, not a SQLModel field — it
|
||||
# rides the query as raw text.
|
||||
rec = (
|
||||
session.query(IdentityRecord)
|
||||
.filter(IdentityRecord.learner_id == learner_id)
|
||||
.order_by(
|
||||
IdentityRecord.submitted_at.desc(),
|
||||
text("rowid DESC"),
|
||||
)
|
||||
.first()
|
||||
)
|
||||
if rec is not None:
|
||||
rec.submitted_at = _as_utc(rec.submitted_at)
|
||||
if rec.verified_at is not None:
|
||||
rec.verified_at = _as_utc(rec.verified_at)
|
||||
return rec
|
||||
|
||||
def count_pending_for_learner(self, learner_id: str) -> int:
|
||||
with Session(self._engine) as session:
|
||||
return (
|
||||
session.query(IdentityRecord)
|
||||
.filter(
|
||||
IdentityRecord.learner_id == learner_id,
|
||||
IdentityRecord.status == "pending",
|
||||
)
|
||||
.count()
|
||||
)
|
||||
|
||||
def mark_verified(
|
||||
self, submission_id: str, verdict: dict[str, Any], age_band: str | None
|
||||
) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
rec = session.get(IdentityRecord, submission_id)
|
||||
if rec is None:
|
||||
return None
|
||||
rec.verdict = verdict
|
||||
rec.age_band = age_band
|
||||
rec.verified_at = datetime.now(UTC)
|
||||
rec.status = "verified" if verdict.get("status") == "verified" else "rejected"
|
||||
rec._validate() # D3: transitions validate like inserts
|
||||
session.commit()
|
||||
session.refresh(rec)
|
||||
return rec
|
||||
|
||||
def close(self) -> None:
|
||||
self._engine.dispose()
|
||||
@@ -25,6 +25,8 @@ from .api import (
|
||||
from .config import Settings
|
||||
from .grading.engine import GradingEngine
|
||||
from .grading.store import SQLiteGradeStore
|
||||
from .identity.mock import MockIdentityProvider
|
||||
from .identity.store import SQLiteIdentityStore
|
||||
from .llm import create_provider
|
||||
from .sandbox import SandboxManager, UnshareBackend
|
||||
from .telemetry.ingest import TraceIntegrityMap
|
||||
@@ -33,6 +35,7 @@ from .variants.generator import VariantGenerator
|
||||
from .variants.store import SQLiteVariantStore
|
||||
from .voice.defense_store import SQLiteDefenseStore
|
||||
from .voice.factory import voice_provider_from_settings
|
||||
from .voice.mock import MockVoiceProvider
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -116,12 +119,37 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
defense_store = SQLiteDefenseStore(db_path=settings.db_path)
|
||||
app.state.defense_store = defense_store
|
||||
if getattr(app.state, "voice_provider", None) is None:
|
||||
app.state.voice_provider = voice_provider_from_settings(settings)
|
||||
# G-11 (boot survival): a misconfigured real provider must never
|
||||
# crash the unattended deploy — fall back to mock loudly. The
|
||||
# mock provider's descriptor honestly reports mode='mock' so the
|
||||
# UI badge cannot lie about which path is live.
|
||||
try:
|
||||
app.state.voice_provider = voice_provider_from_settings(
|
||||
settings, app.state.http_client
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"voice provider %r unavailable (%s); falling back to mock "
|
||||
"— fix the AI_VOICE_* settings and restart",
|
||||
settings.voice_provider,
|
||||
exc,
|
||||
)
|
||||
app.state.voice_provider = MockVoiceProvider()
|
||||
if getattr(app.state, "examiner_agent", None) is None:
|
||||
from .agents.examiner import ExaminerAgent
|
||||
|
||||
app.state.examiner_agent = ExaminerAgent(app.state.provider, settings)
|
||||
|
||||
# Identity verification (REQ-5-003): 5th D-027 store (same SQLite
|
||||
# file) + mock-first provider (A-303). State-injection overrides
|
||||
# preserved — tests may pre-set either.
|
||||
identity_store = getattr(app.state, "identity_store", None)
|
||||
if identity_store is None:
|
||||
identity_store = SQLiteIdentityStore(db_path=settings.db_path)
|
||||
app.state.identity_store = identity_store
|
||||
if getattr(app.state, "identity_provider", None) is None:
|
||||
app.state.identity_provider = MockIdentityProvider()
|
||||
|
||||
# Grading persistence + engine (REQ-3-004): GradeStore from the same
|
||||
# SQLite file as traces (D-027), one GradingEngine singleton wired
|
||||
# through app.state — the engine receives its stores via constructor
|
||||
@@ -167,6 +195,7 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
grade_store.close()
|
||||
variant_store.close()
|
||||
defense_store.close()
|
||||
identity_store.close()
|
||||
await app.state.http_client.aclose()
|
||||
|
||||
app = FastAPI(title="Nextcraft AI Service", version="0.3.0", lifespan=lifespan)
|
||||
@@ -204,6 +233,47 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
app.include_router(variants_router)
|
||||
app.include_router(defense_router)
|
||||
|
||||
# v0.5 identity (REQ-5-003/004): verification flow + age-gate deps
|
||||
# + the one marketplace 18+ gated stub (G-18).
|
||||
from .api.identity import marketplace_router
|
||||
from .api.identity import router as identity_router
|
||||
|
||||
app.include_router(identity_router)
|
||||
app.include_router(marketplace_router)
|
||||
|
||||
# A-305/D1: PII-safe 422s — FastAPI echoes the offending `input` in
|
||||
# validation errors by default; for the identity submit body that
|
||||
# leaks the raw DOB into responses + client logs. The handler scrubs
|
||||
# PII field inputs (scoped app-wide; harmless elsewhere).
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
def _scrub_pii_validation(request, exc): # type: ignore[unused-arg]
|
||||
pii_fields = frozenset({"date_of_birth"})
|
||||
scrubbed = []
|
||||
for err in exc.errors():
|
||||
err = dict(err)
|
||||
if err.get("loc") and err["loc"][-1] in pii_fields:
|
||||
# A-305/D1: never echo the submitted value (input) or the
|
||||
# ctx payload (carries the ValueError); the msg is the
|
||||
# constraint text and is safe.
|
||||
err["input"] = "[redacted]"
|
||||
err.pop("ctx", None)
|
||||
else:
|
||||
# FastAPI's default 422s are JSON-safe EXCEPT ctx payloads
|
||||
# carrying raw ValueError objects (pydantic model_validator
|
||||
# errors); strip ctx body-wide so non-PII routes keep their
|
||||
# 422 shape (msg + loc carry the meaning).
|
||||
ctx = err.get("ctx")
|
||||
if isinstance(ctx, dict):
|
||||
err["ctx"] = {
|
||||
k: v for k, v in ctx.items() if isinstance(v, (str, int, float, bool))
|
||||
}
|
||||
scrubbed.append(err)
|
||||
return JSONResponse(status_code=422, content={"detail": scrubbed})
|
||||
|
||||
app.add_exception_handler(RequestValidationError, _scrub_pii_validation)
|
||||
|
||||
# v0.3.6 single-port deploy: serve the exported web app (apps/web/out)
|
||||
# from the SAME origin as the API when AI_WEB_STATIC_DIR is set. Mounted
|
||||
# AFTER all routers, so /v1/*, /health, /docs win; StaticFiles(html=True)
|
||||
|
||||
@@ -125,6 +125,9 @@ class SandboxManager:
|
||||
self._clock = clock or (lambda: datetime.now(UTC))
|
||||
self._handles: dict[str, SandboxHandle] = {}
|
||||
self._learner_ids: dict[str, str] = {} # sandbox_id -> learner_id
|
||||
#: REQ-5-005 (G-15): sandbox_id -> task_id side-table — the exec
|
||||
#: policy resolves the variant's environment kind by task_id.
|
||||
self._task_ids: dict[str, str | None] = {}
|
||||
self._lock = asyncio.Lock()
|
||||
self._integrity_events: list[SandboxIntegrityEvent] = []
|
||||
self._started = False
|
||||
@@ -172,6 +175,7 @@ class SandboxManager:
|
||||
handle = await self._backend.spawn(spec)
|
||||
self._handles[handle.id] = handle
|
||||
self._learner_ids[handle.id] = learner_id
|
||||
self._task_ids[handle.id] = task_id
|
||||
self._write_pid_marker(handle, learner_id)
|
||||
logger.info(
|
||||
"sandbox created: id=%s learner=%s task=%s",
|
||||
@@ -246,6 +250,7 @@ class SandboxManager:
|
||||
async with self._lock:
|
||||
handle = self._handles.pop(sandbox_id, None)
|
||||
learner_id = self._learner_ids.pop(sandbox_id, "unknown")
|
||||
self._task_ids.pop(sandbox_id, None)
|
||||
if handle is not None:
|
||||
await self._backend.destroy(handle)
|
||||
logger.info(
|
||||
|
||||
@@ -94,6 +94,8 @@ class VariantGenerator:
|
||||
params=dict(params),
|
||||
statement=statement,
|
||||
starter_files=dict(template.starter_files),
|
||||
environment=template.environment,
|
||||
test_command=template.test_command,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
self._store.save(record)
|
||||
|
||||
@@ -41,7 +41,7 @@ from pathlib import Path
|
||||
from typing import Any, Protocol
|
||||
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy import JSON, Index, UniqueConstraint
|
||||
from sqlalchemy import JSON, Index, String, UniqueConstraint
|
||||
from sqlalchemy.orm import validates
|
||||
from sqlmodel import Field, Session, SQLModel, create_engine, select
|
||||
|
||||
@@ -105,6 +105,12 @@ class VariantRecord(SQLModel, table=True):
|
||||
params: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
statement: str
|
||||
starter_files: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
#: REQ-5-005 (D-044): the environment kind rides the record to the API
|
||||
#: and TS client; defaults keep pre-v0.5 rows 'build'.
|
||||
environment: str = Field(default="build", sa_type=String)
|
||||
#: The variant's real test command (was a dead template field — v0.5
|
||||
#: surfaces it so the Run/Test buttons stop hardcoding pytest).
|
||||
test_command: str = Field(default="", sa_type=String)
|
||||
created_at: datetime
|
||||
|
||||
@validates("learner_id", "template_id", "task_id")
|
||||
@@ -215,6 +221,40 @@ class SQLiteVariantStore:
|
||||
self._engine = create_engine(f"sqlite:///{self._db_path}")
|
||||
sa.event.listen(self._engine, "connect", _sqlite_connect)
|
||||
SQLModel.metadata.create_all(self._engine)
|
||||
# v0.5 schema (D-044) added two columns to an existing table.
|
||||
# create_all does NOT ALTER existing tables: on a box with a
|
||||
# pre-v0.5 ~/.nextcraft/data/nextcraft.db, every variant read/write
|
||||
# would raise OperationalError("no such column: variant_record.
|
||||
# environment") — a silent total breakage of the variant path
|
||||
# (final-review P0, verified empirically). Backfill the missing
|
||||
# columns with the model defaults ('build' keeps pre-v0.5 rows
|
||||
# build-kind per the field contract; '' falls back to pytest at
|
||||
# the API seam, api/variants._to_response). Idempotent: the
|
||||
# PRAGMA table_info check makes re-runs no-ops.
|
||||
self._ensure_v05_columns()
|
||||
|
||||
def _ensure_v05_columns(self) -> None:
|
||||
"""Add v0.5 columns to a pre-v0.5 variant_record table (idempotent)."""
|
||||
from sqlalchemy import text
|
||||
|
||||
with self._engine.begin() as conn:
|
||||
columns = {row[1] for row in conn.execute(text("PRAGMA table_info(variant_record)"))}
|
||||
if "environment" not in columns:
|
||||
conn.execute(
|
||||
text(
|
||||
"ALTER TABLE variant_record ADD COLUMN environment "
|
||||
"VARCHAR DEFAULT 'build' NOT NULL"
|
||||
)
|
||||
)
|
||||
logger.info("variant store: backfilled 'environment' (pre-v0.5 schema)")
|
||||
if "test_command" not in columns:
|
||||
conn.execute(
|
||||
text(
|
||||
"ALTER TABLE variant_record ADD COLUMN test_command "
|
||||
"VARCHAR DEFAULT '' NOT NULL"
|
||||
)
|
||||
)
|
||||
logger.info("variant store: backfilled 'test_command' (pre-v0.5 schema)")
|
||||
|
||||
@contextmanager
|
||||
def _session(self) -> Iterator[Session]:
|
||||
|
||||
@@ -20,6 +20,31 @@ from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
#: REQ-5-005 environment kinds (D-044): one namespace fabric, typed starter
|
||||
#: contents + command policy. 'build' = the v0.3 coding IDE; 'design' =
|
||||
#: artifact editing with a validator/renderer harness; 'simulation' = a
|
||||
#: parameterized run harness (benchmark scripts + datasets).
|
||||
EnvironmentKind = Literal["build", "design", "simulation"]
|
||||
|
||||
|
||||
def validate_simple_argv(value: str) -> str:
|
||||
"""G-15: command fields must roundtrip shlex.split → join → split.
|
||||
|
||||
No quotes, no shell metachars — the TS client splits on whitespace only
|
||||
(no shlex in browsers), so anything quote-aware would split differently
|
||||
on the two sides. A violation is a template-AUTHORING bug caught here,
|
||||
at definition time, in Python where shlex exists.
|
||||
"""
|
||||
import shlex
|
||||
|
||||
parts = shlex.split(value)
|
||||
if not parts:
|
||||
raise ValueError("command must not be empty")
|
||||
joined = " ".join(parts)
|
||||
if shlex.split(joined) != parts:
|
||||
raise ValueError(f"command is not whitespace-joinable: {value!r}")
|
||||
return joined
|
||||
|
||||
SlotType = Literal["enum", "int_range", "string_set"]
|
||||
|
||||
|
||||
@@ -99,6 +124,12 @@ class TaskTemplate(BaseModel):
|
||||
rubric_anchors: RubricAnchors
|
||||
starter_files: dict[str, str] = Field(default_factory=dict) # path -> content
|
||||
test_command: str
|
||||
#: REQ-5-005 (D-044): the environment kind rides the variant through
|
||||
#: the API to the TS client; 'build' default keeps v0.3 behavior.
|
||||
environment: EnvironmentKind = "build"
|
||||
#: The kind's Run harness (design: validator/renderer; simulation:
|
||||
#: benchmark script). Defaults to the test_command for build kinds.
|
||||
harness_command: str = ""
|
||||
|
||||
@field_validator("statement_skeleton")
|
||||
@classmethod
|
||||
@@ -107,6 +138,16 @@ class TaskTemplate(BaseModel):
|
||||
raise ValueError("statement_skeleton needs at least one {slot}")
|
||||
return v
|
||||
|
||||
@field_validator("test_command", "harness_command")
|
||||
@classmethod
|
||||
def _simple_argv(cls, v: str) -> str:
|
||||
return validate_simple_argv(v) if v else v
|
||||
|
||||
@property
|
||||
def run_command(self) -> str:
|
||||
"""The Run button's command: kind harness when declared, else tests."""
|
||||
return self.harness_command or self.test_command
|
||||
|
||||
def render(self, params: dict[str, str | int]) -> str:
|
||||
"""Fill the skeleton with validated params."""
|
||||
for slot in self.slots:
|
||||
@@ -261,6 +302,134 @@ TEMPLATES: dict[str, TaskTemplate] = {
|
||||
},
|
||||
test_command="pytest -q",
|
||||
),
|
||||
# -- REQ-5-005 design environment (D-044): artifact editing with a
|
||||
# -- validator/renderer harness - same fabric, typed starter contents.
|
||||
"tpl-conversation-flow-design": TaskTemplate(
|
||||
id="tpl-conversation-flow-design",
|
||||
competency_id="stack-designer-c001",
|
||||
title="Conversational Flow Artifact",
|
||||
statement_skeleton=(
|
||||
"Design a conversational flow for a {persona} assistant helping "
|
||||
"users accomplish {goal}. Author the flow as a structured artifact "
|
||||
"with at least {turn_count} conversation turns, explicit fallback "
|
||||
"paths for misunderstandings, and an AI-transparency disclosure "
|
||||
"pattern. The flow must render validly (the harness validates "
|
||||
"structure) and read naturally end to end."
|
||||
),
|
||||
slots=[
|
||||
ParameterSlot(
|
||||
name="persona",
|
||||
type="enum",
|
||||
values=["travel-planner", "homework-tutor", "fitness-coach", "recipe-guide"],
|
||||
),
|
||||
ParameterSlot(
|
||||
name="goal",
|
||||
type="enum",
|
||||
values=["book-a-trip", "master-a-concept", "start-a-routine", "cook-a-meal"],
|
||||
),
|
||||
ParameterSlot(name="turn_count", type="int_range", lo=6, hi=12),
|
||||
],
|
||||
rubric_anchors=RubricAnchors(
|
||||
expected_edit_count_band=(3, 20),
|
||||
expected_min_test_runs=1,
|
||||
expected_error_fix_cycles_band=(0, 3),
|
||||
notes="Design kind: artifact quality + iteration cadence, not code depth.",
|
||||
),
|
||||
starter_files={
|
||||
"README.md": (
|
||||
"# Conversational Flow Design\n\n"
|
||||
"Edit flow.md - the structured flow artifact. python3 "
|
||||
"validate_flow.py checks structure (turn headings, fallback "
|
||||
"sections, a transparency disclosure) and reports issues.\n"
|
||||
),
|
||||
"flow.md": (
|
||||
"# Flow: your persona here\n\n"
|
||||
"## Turn 1\n- **AI:** (opening)\n- **User (expected):** ...\n\n"
|
||||
"## Fallback\n- (misunderstanding handling)\n\n"
|
||||
"## AI Transparency Disclosure\n- (disclosure pattern)\n"
|
||||
),
|
||||
"validate_flow.py": (
|
||||
"import re, sys\n"
|
||||
"text = open('flow.md').read()\n"
|
||||
"issues = []\n"
|
||||
"turns = len(re.findall(r'^## Turn', text, re.M))\n"
|
||||
"if turns < 3:\n"
|
||||
" issues.append(f'expected at least 3 turn sections, found {turns}')\n"
|
||||
"if not re.search(r'^## Fallback', text, re.M):\n"
|
||||
" issues.append('missing Fallback section')\n"
|
||||
"if not re.search(r'^## AI Transparency', text, re.M):\n"
|
||||
" issues.append('missing AI Transparency Disclosure')\n"
|
||||
"print('VALID' if not issues else 'ISSUES: ' + '; '.join(issues))\n"
|
||||
"sys.exit(0 if not issues else 1)\n"
|
||||
),
|
||||
},
|
||||
test_command="python3 validate_flow.py",
|
||||
environment="design",
|
||||
harness_command="python3 validate_flow.py",
|
||||
),
|
||||
# -- REQ-5-005 simulation environment (D-044): parameterized benchmark
|
||||
# -- harness with dataset generation.
|
||||
"tpl-sensor-benchmark": TaskTemplate(
|
||||
id="tpl-sensor-benchmark",
|
||||
competency_id="stack-orchestration-c011",
|
||||
title="Sensor Data Simulation Harness",
|
||||
statement_skeleton=(
|
||||
"Build a simulation harness for {sensor} readings over {duration_min} "
|
||||
"minutes at {sample_hz} Hz. Generate a synthetic dataset with a "
|
||||
"realistic noise profile, run the analysis pipeline, and print a "
|
||||
"metrics summary (mean, p95, anomaly count at {anomaly_sigma} sigma). "
|
||||
"The harness must be reproducible from the committed seed."
|
||||
),
|
||||
slots=[
|
||||
ParameterSlot(
|
||||
name="sensor",
|
||||
type="enum",
|
||||
values=["temperature", "vibration", "luminosity", "pressure"],
|
||||
),
|
||||
ParameterSlot(name="duration_min", type="int_range", lo=5, hi=60),
|
||||
ParameterSlot(name="sample_hz", type="int_range", lo=1, hi=10),
|
||||
ParameterSlot(name="anomaly_sigma", type="int_range", lo=2, hi=4),
|
||||
],
|
||||
rubric_anchors=RubricAnchors(
|
||||
expected_edit_count_band=(3, 25),
|
||||
expected_min_test_runs=2,
|
||||
expected_error_fix_cycles_band=(0, 3),
|
||||
notes="Simulation kind: pipeline correctness + reproducibility.",
|
||||
),
|
||||
starter_files={
|
||||
"README.md": (
|
||||
"# Sensor Simulation Harness\n\n"
|
||||
"Edit simulate.py - python3 simulate.py runs the full pipeline: "
|
||||
"generate, analyze, print metrics. pytest covers the analysis "
|
||||
"functions.\n"
|
||||
),
|
||||
"simulate.py": (
|
||||
"import random, statistics\n\n"
|
||||
"def generate(n=300, seed=42):\n"
|
||||
" rng = random.Random(seed)\n"
|
||||
" return [rng.gauss(20.0, 1.5) for _ in range(n)]\n\n"
|
||||
"def analyze(samples, sigma=3):\n"
|
||||
" mean = statistics.fmean(samples)\n"
|
||||
" stdev = statistics.pstdev(samples)\n"
|
||||
" anomalies = [s for s in samples if abs(s - mean) > sigma * stdev]\n"
|
||||
" p95 = sorted(samples)[int(0.95 * len(samples))]\n"
|
||||
" return {'mean': mean, 'p95': p95, 'anomalies': len(anomalies)}\n\n"
|
||||
"if __name__ == '__main__':\n"
|
||||
" print(analyze(generate()))\n"
|
||||
),
|
||||
"test_simulate.py": (
|
||||
"from simulate import generate, analyze\n\n"
|
||||
"def test_reproducible():\n"
|
||||
" assert generate() == generate()\n\n"
|
||||
"def test_metrics_shape():\n"
|
||||
" m = analyze(generate())\n"
|
||||
" assert set(m) == {'mean', 'p95', 'anomalies'}\n"
|
||||
),
|
||||
},
|
||||
test_command="pytest -q",
|
||||
environment="simulation",
|
||||
harness_command="python3 simulate.py",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ class VoiceDescriptor(BaseModel):
|
||||
"""Capability descriptor served to the web client (D-030).
|
||||
|
||||
The assessment UI reads this to decide HOW the learner speaks/hears:
|
||||
- `mode="server"` → server-side STT/TTS (v0.4 real provider seam)
|
||||
- `mode="server"` → server-side STT/TTS (openai-audio, live since v0.5)
|
||||
- `mode="browser"` → browser-native SpeechRecognition/speechSynthesis
|
||||
- `mode="mock"` → deterministic no-op path (tests / no-key dev)
|
||||
The descriptor never contains secrets — only capability hints.
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
"""Browser-native fallback descriptor (D-030, CUT-1 / G-7, REQ-3-006).
|
||||
|
||||
v0.3 has NO real server STT/TTS (deferred to v0.4 with KYC/keys — GRILL
|
||||
CUT-1). When the factory selects `browser` mode, the defense endpoints return
|
||||
this descriptor and the WEB CLIENT performs SpeechRecognition + speechSynthesis
|
||||
natively; the server persists text turns as usual.
|
||||
Browser-native SR/TTS is the no-key CLIENT-side path. When the factory
|
||||
selects `browser` mode, the defense endpoints return this descriptor and the
|
||||
WEB CLIENT performs SpeechRecognition + speechSynthesis natively; the server
|
||||
persists text turns as usual. Real server STT/TTS (openai-audio) is live
|
||||
since v0.5 — this descriptor is the no-key fallback.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -26,7 +27,7 @@ MOCK_DESCRIPTOR = VoiceDescriptor(
|
||||
sr_available=True,
|
||||
tts_available=True,
|
||||
hint=(
|
||||
"Deterministic mock voice (tests / no-key dev). Server STT/TTS "
|
||||
"endpoints serve canned responses; real server STT/TTS lands in v0.4."
|
||||
"Deterministic mock voice (tests / no-key dev). Real server "
|
||||
"STT/TTS is live since v0.5 (AI_VOICE_PROVIDER=openai-audio)."
|
||||
),
|
||||
)
|
||||
|
||||
@@ -1,24 +1,37 @@
|
||||
"""Voice provider factory (D-030, REQ-3-006).
|
||||
"""Voice provider factory (D-030; REQ-5-001 real path, D-040).
|
||||
|
||||
`AI_VOICE_PROVIDER = browser | mock` (default: mock — the no-key path is
|
||||
first-class). The real server provider (`openai-audio`) is a v0.4 seam and
|
||||
is REJECTED here with a clear error naming the deferral, so a stale env var
|
||||
can't silently pretend a real backend exists.
|
||||
`AI_VOICE_PROVIDER = mock | browser | openai-audio` (default: mock — the
|
||||
no-key path is first-class). `openai-audio` requires voice_base_url +
|
||||
voice_api_key: the factory raises `UnknownVoiceProviderError` with an
|
||||
actionable message for direct callers (tests), while the lifespan in
|
||||
main.py CATCHES it and falls back to mock with a loud log — a typo'd env
|
||||
must never crash the unattended boot (G-11), and the mock provider's
|
||||
descriptor then honestly reports mode='mock' so the UI badge cannot lie.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
|
||||
from ..config import Settings
|
||||
from .base import VoiceProvider
|
||||
from .mock import MockVoiceProvider
|
||||
from .openai_audio import OpenAIAudioProvider
|
||||
|
||||
|
||||
class UnknownVoiceProviderError(ValueError):
|
||||
"""Raised for a provider name outside the v0.3 contract."""
|
||||
"""Raised for a provider name outside the contract, or a real provider
|
||||
selected without its required configuration."""
|
||||
|
||||
|
||||
def voice_provider_from_settings(settings: Settings) -> VoiceProvider:
|
||||
"""Select the voice provider by settings (env `AI_VOICE_PROVIDER`)."""
|
||||
def voice_provider_from_settings(
|
||||
settings: Settings, http_client: httpx.AsyncClient | None = None
|
||||
) -> VoiceProvider:
|
||||
"""Select the voice provider by settings (env `AI_VOICE_PROVIDER`).
|
||||
|
||||
`http_client` is required for the `openai-audio` branch (D-017 shared
|
||||
pool); mock/browser ignore it.
|
||||
"""
|
||||
name = (settings.voice_provider or "mock").strip().lower()
|
||||
if name == "mock":
|
||||
return MockVoiceProvider()
|
||||
@@ -28,10 +41,27 @@ def voice_provider_from_settings(settings: Settings) -> VoiceProvider:
|
||||
# uses the descriptor for mic/speech). See browser.py.
|
||||
return MockVoiceProvider()
|
||||
if name in ("openai-audio", "openai", "server"):
|
||||
raise UnknownVoiceProviderError(
|
||||
"real server STT/TTS (OpenAIAudioProvider) is deferred to v0.4 "
|
||||
"(GRILL CUT-1 / G-7): set AI_VOICE_PROVIDER=mock or browser"
|
||||
if not settings.voice_base_url or not settings.voice_api_key:
|
||||
raise UnknownVoiceProviderError(
|
||||
"AI_VOICE_PROVIDER=openai-audio requires AI_VOICE_BASE_URL "
|
||||
"and AI_VOICE_API_KEY — set both, or use 'mock'/'browser'. "
|
||||
"(main.py falls back to mock when these are missing; the "
|
||||
"voice badge then honestly reports mock — G-11)"
|
||||
)
|
||||
if http_client is None:
|
||||
raise UnknownVoiceProviderError(
|
||||
"openai-audio requires the shared httpx client "
|
||||
"(voice_provider_from_settings(settings, http_client))"
|
||||
)
|
||||
return OpenAIAudioProvider(
|
||||
http_client=http_client,
|
||||
base_url=settings.voice_base_url,
|
||||
api_key=settings.voice_api_key,
|
||||
stt_model=settings.voice_stt_model,
|
||||
tts_model=settings.voice_tts_model,
|
||||
tts_voice=settings.voice_tts_voice,
|
||||
tts_format=settings.voice_tts_format,
|
||||
)
|
||||
raise UnknownVoiceProviderError(
|
||||
f"unknown AI_VOICE_PROVIDER {name!r}: use 'mock' or 'browser'"
|
||||
f"unknown AI_VOICE_PROVIDER {name!r}: use 'mock', 'browser', or 'openai-audio'"
|
||||
)
|
||||
|
||||
@@ -75,7 +75,7 @@ class MockVoiceProvider:
|
||||
|
||||
async def synthesize(self, text: str, voice: str = "default") -> AsyncIterator[bytes]: # noqa: ASYNC109 (protocol parity)
|
||||
# NOTE: protocol parity matters more than the async-generator purity
|
||||
# lint; the real provider seam (v0.4) will stream over HTTP.
|
||||
# lint; the real provider (openai_audio.py) streams over HTTP.
|
||||
self.synthesize_calls += 1
|
||||
if not text:
|
||||
raise MockVoiceFailure("cannot synthesize empty text")
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
"""OpenAI-compatible audio provider — real server STT/TTS (D-040, REQ-5-001).
|
||||
|
||||
One implementation serves any OpenAI-compatible audio endpoint (base_url is
|
||||
config; A-301 endpoint-agnostic by config, D-014 pattern). Raw httpx on the
|
||||
shared lifespan client (D-017; read=300s tolerates multi-minute clips).
|
||||
|
||||
Boundary rules (mirror llm/openai_compat.py):
|
||||
- voice/ imports nothing from agents/ or api/
|
||||
- api_key NEVER appears in exceptions, logs, or error messages
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import Literal
|
||||
|
||||
import httpx
|
||||
|
||||
from .base import TranscriptSegment, VoiceDescriptor
|
||||
|
||||
|
||||
class OpenAIAudioProvider:
|
||||
"""Server STT (`/audio/transcriptions`) + TTS (`/audio/speech`)."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
http_client: httpx.AsyncClient,
|
||||
base_url: str,
|
||||
api_key: str,
|
||||
stt_model: str = "whisper-1",
|
||||
tts_model: str = "tts-1",
|
||||
tts_voice: str = "alloy",
|
||||
tts_format: Literal["mp3", "wav", "opus"] = "mp3",
|
||||
) -> None:
|
||||
self._client = http_client
|
||||
self._base_url = base_url.rstrip("/")
|
||||
self._api_key = api_key
|
||||
self._stt_model = stt_model
|
||||
self._tts_model = tts_model
|
||||
self._tts_voice = tts_voice
|
||||
self._tts_format = tts_format
|
||||
# a-15: the descriptor is what defense.py prefers; a missing one
|
||||
# would badge the real server path as "mock".
|
||||
self.descriptor = VoiceDescriptor(
|
||||
mode="server",
|
||||
sr_available=True,
|
||||
tts_available=True,
|
||||
hint="server STT/TTS via AI_VOICE_BASE_URL",
|
||||
)
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
headers: dict[str, str] = {}
|
||||
if self._api_key:
|
||||
headers["Authorization"] = f"Bearer {self._api_key}"
|
||||
return headers
|
||||
|
||||
def _sanitize(self, exc: Exception) -> RuntimeError:
|
||||
text = str(exc)
|
||||
if self._api_key and self._api_key in text:
|
||||
text = text.replace(self._api_key, "[REDACTED]")
|
||||
return RuntimeError(f"voice provider error: {text}")
|
||||
|
||||
async def transcribe(self, audio: bytes, fmt: str) -> TranscriptSegment:
|
||||
"""STT: multipart upload (`file` + `model`) → TranscriptSegment.
|
||||
|
||||
`fmt` is a bare extension ('wav' | 'webm' | 'mp3') — the defense
|
||||
route strips codec params before this call (D-041).
|
||||
"""
|
||||
files = {"file": (f"answer.{fmt}", audio, f"audio/{fmt}")}
|
||||
data = {"model": self._stt_model, "response_format": "json"}
|
||||
try:
|
||||
resp = await self._client.post(
|
||||
f"{self._base_url}/audio/transcriptions",
|
||||
files=files,
|
||||
data=data,
|
||||
headers=self._headers(),
|
||||
)
|
||||
resp.raise_for_status()
|
||||
body = resp.json()
|
||||
except httpx.HTTPError as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
if not isinstance(body, dict):
|
||||
# P2 (verifier): a non-dict 200 body is a contract break —
|
||||
# context-wrap instead of a raw AttributeError.
|
||||
raise RuntimeError(
|
||||
"voice provider error: unexpected transcription response shape"
|
||||
)
|
||||
text = str(body.get("text", "")).strip()
|
||||
if not text:
|
||||
# 200 with an empty transcript is a provider contract break —
|
||||
# TranscriptSegment(min_length=1) would raise a bare pydantic
|
||||
# error; wrap it with provider context instead.
|
||||
raise RuntimeError("voice provider error: empty transcription")
|
||||
return TranscriptSegment(text=text)
|
||||
|
||||
def synthesize(
|
||||
self, text: str, voice: str = "default"
|
||||
) -> AsyncIterator[bytes]:
|
||||
"""TTS: JSON body → raw audio byte stream.
|
||||
|
||||
OpenAI's TTS caps `input` at 4096 chars; examiner questions are
|
||||
short, but enforce the guard so a long question fails loudly at the
|
||||
seam instead of as an opaque provider 400.
|
||||
"""
|
||||
return self._synthesize_stream(text, voice)
|
||||
|
||||
async def _synthesize_stream(
|
||||
self, text: str, voice: str
|
||||
) -> AsyncIterator[bytes]:
|
||||
if len(text) > 4096:
|
||||
raise RuntimeError(
|
||||
f"voice provider error: TTS input exceeds 4096 chars ({len(text)})"
|
||||
)
|
||||
payload = {
|
||||
"model": self._tts_model,
|
||||
"input": text,
|
||||
"voice": voice if voice != "default" else self._tts_voice,
|
||||
"response_format": self._tts_format,
|
||||
}
|
||||
try:
|
||||
async with self._client.stream(
|
||||
"POST",
|
||||
f"{self._base_url}/audio/speech",
|
||||
content=json.dumps(payload),
|
||||
headers={**self._headers(), "Content-Type": "application/json"},
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
async for chunk in resp.aiter_bytes():
|
||||
if chunk:
|
||||
yield chunk
|
||||
except httpx.HTTPError as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
@@ -48,7 +48,19 @@ class ScriptedLLM(MockProvider):
|
||||
|
||||
@pytest.fixture()
|
||||
def app(tmp_path: Path):
|
||||
application = create_app(Settings(provider="mock", voice_provider="mock"))
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
voice_provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
application = create_app(settings)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
application.state.identity_store = identity_store # G-9
|
||||
llm = ScriptedLLM()
|
||||
application.state.provider = llm
|
||||
application.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
@@ -99,7 +111,20 @@ class TestBrowserFallback:
|
||||
def test_browser_mode_serves_browser_descriptor(self, tmp_path: Path) -> None:
|
||||
"""Must-Have #6: AI_VOICE_PROVIDER=browser → start returns the
|
||||
browser-native SR/TTS fallback descriptor (D-030), not 'mock'."""
|
||||
application = create_app(Settings(provider="mock", voice_provider="browser"))
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
application = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
voice_provider="browser",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity-b.db")
|
||||
seed_verified_identity(identity_store)
|
||||
application.state.identity_store = identity_store # G-9
|
||||
application.state.provider = ScriptedLLM()
|
||||
application.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
application.state.variant_store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
@@ -232,3 +257,107 @@ class TestFinishAndGet:
|
||||
def test_unknown_defense_404_on_all(self, client) -> None:
|
||||
assert client.post("/v1/defense/dfn-nope/finish").status_code == 404
|
||||
assert client.get("/v1/defense/dfn-nope").status_code == 404
|
||||
|
||||
|
||||
class TestServerVoiceRouteFixes:
|
||||
"""MH-2c (D-041/G-16/G-12): codec-strip, size guard, format-aware TTS."""
|
||||
|
||||
def test_webm_codec_params_stripped_for_provider(self, client, app) -> None:
|
||||
"""MediaRecorder sends 'audio/webm;codecs=opus' — the provider must
|
||||
see the bare 'webm' (D-041), else a real STT endpoint 400s."""
|
||||
received_fmts: list[str] = []
|
||||
|
||||
class ProbeVoice(MockVoiceProvider):
|
||||
async def transcribe(self, audio: bytes, fmt: str):
|
||||
received_fmts.append(fmt)
|
||||
return await super().transcribe(audio, fmt)
|
||||
|
||||
app.state.voice_provider = ProbeVoice(["clean fmt seen"])
|
||||
defense_id = _start(client)["defense_id"]
|
||||
resp = client.post(
|
||||
f"/v1/defense/{defense_id}/answer",
|
||||
files={"audio": ("answer.webm", b"\x1a\x45\xa3\xdf", "audio/webm;codecs=opus")},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
assert received_fmts == ["webm"], (
|
||||
f"codec params leaked to the provider: {received_fmts}"
|
||||
)
|
||||
assert resp.json()["question"]
|
||||
|
||||
def test_oversize_audio_413_before_provider_call(self, client, app) -> None:
|
||||
"""G-12/D-041: the guard fires before any provider call — the client
|
||||
renders an honest re-record prompt."""
|
||||
called = {"n": 0}
|
||||
|
||||
class ProbeVoice(MockVoiceProvider):
|
||||
async def transcribe(self, audio: bytes, fmt: str):
|
||||
called["n"] += 1
|
||||
return await super().transcribe(audio, fmt)
|
||||
|
||||
app.state.voice_provider = ProbeVoice(["x"])
|
||||
defense_id = _start(client)["defense_id"]
|
||||
settings = Settings()
|
||||
too_big = b"\x00" * (settings.voice_max_audio_mb * 1024 * 1024 + 1)
|
||||
resp = client.post(
|
||||
f"/v1/defense/{defense_id}/answer",
|
||||
files={"audio": ("answer.webm", too_big, "audio/webm")},
|
||||
)
|
||||
assert resp.status_code == 413
|
||||
assert "re-record" in resp.json()["detail"]
|
||||
assert called["n"] == 0, "provider must not be called for oversize audio"
|
||||
|
||||
def test_tts_media_type_maps_from_settings_enum(self, tmp_path: Path) -> None:
|
||||
"""G-16: media_type follows voice_tts_format (was hardcoded wav)."""
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
application = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
voice_provider="mock",
|
||||
voice_tts_format="opus",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity-opus.db")
|
||||
seed_verified_identity(identity_store)
|
||||
application.state.identity_store = identity_store # G-9
|
||||
application.state.provider = ScriptedLLM()
|
||||
application.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
application.state.variant_store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
application.state.grade_store = SQLiteGradeStore(db_path=tmp_path / "g.db")
|
||||
application.state.trace_integrity = TraceIntegrityMap()
|
||||
application.state.defense_store = SQLiteDefenseStore(db_path=tmp_path / "d.db")
|
||||
application.state.examiner_agent = ExaminerAgent(
|
||||
application.state.provider,
|
||||
Settings(provider="mock", voice_provider="mock"),
|
||||
)
|
||||
with TestClient(application) as c:
|
||||
defense_id = _start(c)["defense_id"]
|
||||
resp = c.get(f"/v1/defense/{defense_id}/audio/0")
|
||||
assert resp.status_code == 200
|
||||
assert resp.headers["content-type"].startswith("audio/opus"), (
|
||||
resp.headers["content-type"]
|
||||
)
|
||||
|
||||
def test_stt_provider_failure_is_502_never_500(self, client, app) -> None:
|
||||
"""Cross-phase P0 regression (final review): a provider failure on
|
||||
the audio-answer path (real endpoint outage — or the DEFAULT mock
|
||||
provider's unscripted queue, which every default-configured
|
||||
deployment hits on its first audio answer) must surface as an
|
||||
honest 502 per the assessment/proctor house pattern — never an
|
||||
unhandled 500. The transcript stays unaffected (typed answers
|
||||
still work)."""
|
||||
# Unscripted mock = exactly what create_app wires on default settings.
|
||||
app.state.voice_provider = MockVoiceProvider()
|
||||
defense_id = _start(client)["defense_id"]
|
||||
resp = client.post(
|
||||
f"/v1/defense/{defense_id}/answer",
|
||||
files={"audio": ("answer.webm", b"\x1a\x45\xa3\xdf" * 64, "audio/webm")},
|
||||
)
|
||||
assert resp.status_code == 502, resp.text
|
||||
assert "transcription failed" in resp.json()["detail"]
|
||||
# the defense is still alive for typed answers (no poisoned state)
|
||||
typed = client.post(f"/v1/defense/{defense_id}/answer", data={"text": "typed"})
|
||||
assert typed.status_code == 200, typed.text
|
||||
|
||||
@@ -111,8 +111,19 @@ async def test_full_credential_flow(tmp_path: Path) -> None:
|
||||
import httpx
|
||||
|
||||
port = _free_port()
|
||||
settings = Settings(provider="mock", voice_provider="mock", port=port)
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from tests.conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
voice_provider="mock",
|
||||
port=port,
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
app = create_app(settings)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
app.state.identity_store = identity_store # G-9
|
||||
app.state.provider = FlowLLM()
|
||||
app.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
app.state.grade_store = SQLiteGradeStore(db_path=tmp_path / "g.db")
|
||||
|
||||
@@ -43,9 +43,23 @@ BASE_SETTINGS: dict = {
|
||||
"sandbox_max_concurrent": 5,
|
||||
"sandbox_max_per_learner": 1,
|
||||
"sandbox_creates_per_min": 10,
|
||||
# G-9: allowlist widened to the suite roster (identity records seeded
|
||||
# per-app below); the gate tests live in test_identity.py.
|
||||
"learner_allowlist": __import__("tests.conftest", fromlist=["SUITE_LEARNERS"]).SUITE_LEARNERS,
|
||||
}
|
||||
|
||||
|
||||
def _seed_identity(app, tmp_path: Path) -> None:
|
||||
"""G-9: every ad-hoc app in this module gets the suite's verified
|
||||
identity store (allowlist widened via BASE_SETTINGS)."""
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from tests.conftest import seed_verified_identity
|
||||
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / f"identity-{id(app):x}.db")
|
||||
seed_verified_identity(store)
|
||||
app.state.identity_store = store
|
||||
|
||||
|
||||
class StubBackend:
|
||||
"""Structural SandboxBackend: lays out the workdir, spawns nothing.
|
||||
|
||||
@@ -103,6 +117,7 @@ def client(
|
||||
monkeypatch.setenv("AI_SANDBOX_CREATES_PER_MIN", "150") # shared-window headroom
|
||||
settings = Settings(**{**BASE_SETTINGS, "sandbox_dir": tmp_path / "sandboxes"})
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
@@ -190,6 +205,7 @@ def test_pool_full_returns_503(tmp_path: Path, stub_backend: StubBackend) -> Non
|
||||
sandbox_creates_per_min=150,
|
||||
)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
with TestClient(app) as client:
|
||||
ids = [
|
||||
@@ -227,6 +243,7 @@ def test_second_active_sandbox_for_same_learner_429(
|
||||
sandbox_creates_per_min=150,
|
||||
)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
with TestClient(app) as client:
|
||||
first = client.post("/v1/sandboxes", json={"learner_id": PILOT})
|
||||
@@ -252,6 +269,7 @@ def test_burst_over_global_create_rate_429(tmp_path: Path, stub_backend: StubBac
|
||||
sandbox_creates_per_min=3,
|
||||
)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
app.state.sandbox_manager = manager
|
||||
with TestClient(app) as client:
|
||||
@@ -309,6 +327,7 @@ def test_lifespan_start_and_shutdown_destroy(
|
||||
)
|
||||
manager = SandboxManager(backend=stub_backend, settings=settings)
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
app.state.sandbox_manager = manager
|
||||
with TestClient(app) as client:
|
||||
assert not orphan_root.exists() # startup reaper ran during lifespan boot
|
||||
@@ -325,6 +344,7 @@ def test_lifespan_constructs_real_manager_when_not_overridden(tmp_path: Path) ->
|
||||
"""No override → the lifespan builds the production UnshareBackend manager."""
|
||||
settings = Settings(provider="mock", sandbox_dir=tmp_path / "sandboxes")
|
||||
app = create_app(settings)
|
||||
_seed_identity(app, tmp_path)
|
||||
with TestClient(app):
|
||||
manager = app.state.sandbox_manager
|
||||
assert isinstance(manager, SandboxManager)
|
||||
@@ -347,6 +367,7 @@ def test_real_backend_create_path_runs(tmp_path: Path) -> None:
|
||||
sandbox_creates_per_min=150,
|
||||
)
|
||||
app = create_app(settings) # no override → lifespan wires UnshareBackend
|
||||
_seed_identity(app, tmp_path)
|
||||
try:
|
||||
with TestClient(app) as client:
|
||||
created = client.post("/v1/sandboxes", json={"learner_id": PILOT})
|
||||
|
||||
@@ -45,12 +45,15 @@ import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.main import create_app
|
||||
from ai_service.variants.generator import VariantGenerator
|
||||
from ai_service.variants.store import SQLiteVariantStore
|
||||
from ai_service.variants.templates import get_template
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
LEARNER_A = "variant-learner-a"
|
||||
LEARNER_B = "variant-learner-b"
|
||||
TEMPLATE = "tpl-llm-judge"
|
||||
@@ -104,8 +107,12 @@ def _make_client(
|
||||
provider="mock",
|
||||
db_path=tmp_path / "variant-test.db",
|
||||
sandbox_dir=tmp_path / "sandboxes",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
app = create_app(settings)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
app.state.identity_store = identity_store # G-9
|
||||
app.state.variant_store = store
|
||||
if getattr(app.state, "variant_generator", None) is None:
|
||||
app.state.variant_generator = VariantGenerator(
|
||||
|
||||
@@ -27,6 +27,54 @@ def sandbox_dir(tmp_path: Path) -> Path:
|
||||
shutil.rmtree(d, ignore_errors=True)
|
||||
|
||||
|
||||
# G-9 (grill): the v0.5 identity gates land on variants/sandboxes/defense —
|
||||
# routes every API suite exercises. The suite-wide learner roster + a
|
||||
# seeded verified identity keep pre-existing tests green while the gate
|
||||
# tests (test_identity.py) prove the 403/CTA composition on unverified ids.
|
||||
SUITE_LEARNERS = [
|
||||
"pilot-learner",
|
||||
"pilot-learner-2",
|
||||
"api-learner",
|
||||
"defense-learner",
|
||||
"grade-learner",
|
||||
"lab-learner",
|
||||
"p-learner",
|
||||
"variant-learner-a",
|
||||
"variant-learner-b",
|
||||
"learner-001",
|
||||
"lat-learner",
|
||||
"ghost-learner",
|
||||
]
|
||||
|
||||
|
||||
def seed_verified_identity(store, learner_ids=SUITE_LEARNERS) -> None:
|
||||
"""Insert a verified 18+ identity record per learner id (mock-marked).
|
||||
|
||||
A-304: records carry mock=True — the seed never masquerades as
|
||||
production verification.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from ai_service.identity.store import IdentityRecord
|
||||
|
||||
for lid in learner_ids:
|
||||
try:
|
||||
store.insert(
|
||||
IdentityRecord(
|
||||
id=f"seed-{lid}",
|
||||
learner_id=lid,
|
||||
status="verified",
|
||||
provider="mock",
|
||||
verdict={"status": "verified", "age_band": "18+", "mock": True},
|
||||
age_band="18+",
|
||||
submitted_at=datetime.now(UTC),
|
||||
verified_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
except Exception:
|
||||
pass # already seeded (shared store)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def settings(tmp_path: Path) -> Settings:
|
||||
os.environ["AI_PROVIDER"] = "mock"
|
||||
@@ -38,12 +86,25 @@ def settings(tmp_path: Path) -> Settings:
|
||||
port=8421,
|
||||
db_path=tmp_path / "nextcraft-test.db",
|
||||
sandbox_dir=tmp_path / "sandboxes",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def app(settings: Settings):
|
||||
return create_app(settings)
|
||||
def identity_store(tmp_path: Path):
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "identity-test.db")
|
||||
seed_verified_identity(store)
|
||||
yield store
|
||||
store.close()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def app(settings: Settings, identity_store):
|
||||
application = create_app(settings)
|
||||
application.state.identity_store = identity_store # state-injection (G-9)
|
||||
return application
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
|
||||
@@ -0,0 +1,630 @@
|
||||
"""Identity module tests (REQ-5-003/004, D-042/43).
|
||||
|
||||
MH-3a: store contract — insert/poll/latest/verdict provenance, constraints.
|
||||
MH-3b: gate composition — allowlist (403, first) → identity verdict (403 +
|
||||
verify-CTA) → caps; 16-17 school-pass/marketplace-block; under-16 blocked;
|
||||
G-13 submit caps; G-18 honest stub.
|
||||
MH-3c: PII sentinel — raw DOB + document contents appear in NO log record
|
||||
and NO stored raw form (caplog + store inspection).
|
||||
MH-3e: identity flow end-to-end via TestClient against real create_app.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.identity.base import IdentitySubmission
|
||||
from ai_service.identity.mock import MockIdentityProvider, derive_age_band
|
||||
from ai_service.identity.store import IdentityRecord, SQLiteIdentityStore
|
||||
from ai_service.main import create_app
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
ADULT_DOB = "2000-01-01"
|
||||
MINOR_DOB = str(datetime.now(UTC).year - 17) + "-06-01" # 16-17 band
|
||||
UNDER16_DOB = str(datetime.now(UTC).year - 12) + "-06-01" # under-16
|
||||
|
||||
|
||||
def _age_band(dob: str) -> str:
|
||||
return derive_age_band(dob)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def identity_store(tmp_path: Path) -> SQLiteIdentityStore:
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
yield store
|
||||
store.close()
|
||||
|
||||
|
||||
# -- MH-3a: store ---------------------------------------------------------------
|
||||
|
||||
|
||||
class TestIdentityStore:
|
||||
def test_insert_get_roundtrip(self, identity_store) -> None:
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-1",
|
||||
learner_id="learner-x",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
got = identity_store.get("idc-1")
|
||||
assert got is not None and got.learner_id == "learner-x"
|
||||
assert got.status == "pending"
|
||||
assert got.mock is True # A-304 default marker
|
||||
|
||||
def test_latest_for_learner_orders_by_submitted(self, identity_store) -> None:
|
||||
now = datetime.now(UTC)
|
||||
# insert order (oldest→newest by timestamp): idc-1, idc-2, idc-0
|
||||
for i, offset in ((1, 1), (2, 2), (0, 3)):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id=f"idc-{i}",
|
||||
learner_id="learner-y",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=now + timedelta(seconds=offset),
|
||||
)
|
||||
)
|
||||
latest = identity_store.latest_for_learner("learner-y")
|
||||
assert latest is not None and latest.id == "idc-0" # +3s is newest
|
||||
|
||||
def test_insert_duplicate_id_raises(self, identity_store) -> None:
|
||||
"""Insert-only: a SECOND record with the same id raises (a real
|
||||
duplicate is a fresh instance carrying a minted-once id)."""
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-dup",
|
||||
learner_id="learner-z",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
with pytest.raises(IntegrityError):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-dup", # same id, fresh instance
|
||||
learner_id="learner-z",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
|
||||
def test_invalid_status_rejected(self, identity_store) -> None:
|
||||
with pytest.raises(ValueError, match="invalid identity status"):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-bad",
|
||||
learner_id="l",
|
||||
status="banana",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
|
||||
def test_mark_verified_transition(self, identity_store) -> None:
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-v",
|
||||
learner_id="learner-v",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
updated = identity_store.mark_verified(
|
||||
"idc-v", {"status": "verified", "age_band": "18+", "mock": True}, "18+"
|
||||
)
|
||||
assert updated is not None
|
||||
assert updated.status == "verified"
|
||||
assert updated.age_band == "18+"
|
||||
assert updated.mock is True
|
||||
assert identity_store.mark_verified("nope", {}, None) is None
|
||||
|
||||
def test_count_pending(self, identity_store) -> None:
|
||||
for i in range(3):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id=f"idc-p{i}",
|
||||
learner_id="learner-p",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
assert identity_store.count_pending_for_learner("learner-p") == 3
|
||||
|
||||
|
||||
# -- mock provider -----------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMockProvider:
|
||||
def test_adult_verifies_18_plus(self) -> None:
|
||||
provider = MockIdentityProvider()
|
||||
sid = await_(provider.submit(IdentitySubmission(learner_id="l", date_of_birth=ADULT_DOB)))
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "verified"
|
||||
assert verdict.age_band == "18+"
|
||||
assert verdict.mock is True # A-304
|
||||
|
||||
def test_minor_gets_16_17_band(self) -> None:
|
||||
provider = MockIdentityProvider()
|
||||
sid = await_(
|
||||
provider.submit(IdentitySubmission(learner_id="l", date_of_birth=MINOR_DOB))
|
||||
)
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "verified"
|
||||
assert verdict.age_band == "16-17"
|
||||
|
||||
def test_under_16_rejected(self) -> None:
|
||||
provider = MockIdentityProvider()
|
||||
sid = await_(
|
||||
provider.submit(IdentitySubmission(learner_id="l", date_of_birth=UNDER16_DOB))
|
||||
)
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "rejected"
|
||||
assert "16+" in verdict.detail
|
||||
|
||||
def test_scripted_rejection(self) -> None:
|
||||
provider = MockIdentityProvider(reject_learners={"bad-actor"})
|
||||
sid = await_(
|
||||
provider.submit(IdentitySubmission(learner_id="bad-actor", date_of_birth=ADULT_DOB))
|
||||
)
|
||||
verdict = await_(provider.poll(sid))
|
||||
assert verdict.status == "rejected"
|
||||
|
||||
|
||||
def await_(coro):
|
||||
import asyncio
|
||||
|
||||
return asyncio.get_event_loop().run_until_complete(coro) if False else asyncio.run(coro)
|
||||
|
||||
|
||||
# -- MH-3b/MH-3c/MH-3e: gates + flow over HTTP ----------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def gated_client(tmp_path: Path, identity_store: SQLiteIdentityStore) -> TestClient:
|
||||
seed_verified_identity(identity_store) # suite roster verified 18+
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
identity_submits_per_min=10,
|
||||
db_path=tmp_path / "gated-app.db",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
|
||||
|
||||
class TestVerificationFlow:
|
||||
def test_submit_status_verify_flow(self, gated_client: TestClient) -> None:
|
||||
resp = gated_client.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "new-learner", "date_of_birth": ADULT_DOB},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["status"] == "pending"
|
||||
assert body["mock"] is True
|
||||
|
||||
status = gated_client.get("/v1/identity/status/new-learner").json()
|
||||
assert status["status"] == "pending"
|
||||
|
||||
verdict = gated_client.post(f"/v1/identity/verify/{body['submission_id']}").json()
|
||||
assert verdict["status"] == "verified"
|
||||
assert verdict["age_band"] == "18+"
|
||||
assert verdict["mock"] is True # A-304 rides the response
|
||||
|
||||
def test_g13_pending_resubmit_409(self, gated_client: TestClient) -> None:
|
||||
first = gated_client.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "pending-learner", "date_of_birth": ADULT_DOB},
|
||||
).json()
|
||||
second = gated_client.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "pending-learner", "date_of_birth": ADULT_DOB},
|
||||
)
|
||||
assert second.status_code == 409
|
||||
detail = second.json()["detail"]
|
||||
assert detail["reason"] == "submission_pending"
|
||||
assert detail["submission_id"] == first["submission_id"]
|
||||
|
||||
def test_g13_rate_cap_429(self, tmp_path: Path) -> None:
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["rl"],
|
||||
identity_submits_per_min=1,
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
with TestClient(app) as c:
|
||||
# Learner submits + verifies (record terminal → pending cap free),
|
||||
# then resubmits within the rate window → 429.
|
||||
sub = c.post(
|
||||
"/v1/identity/submit", json={"learner_id": "rl", "date_of_birth": ADULT_DOB}
|
||||
).json()
|
||||
c.post(f"/v1/identity/verify/{sub['submission_id']}")
|
||||
resp = c.post(
|
||||
"/v1/identity/submit", json={"learner_id": "rl", "date_of_birth": ADULT_DOB}
|
||||
)
|
||||
assert resp.status_code == 429
|
||||
|
||||
def test_resubmit_after_terminal_never_500s(
|
||||
self, tmp_path: Path
|
||||
) -> None:
|
||||
"""Cross-phase P0 regression (final review): the mock provider once
|
||||
minted the submission id from hash((learner_id, dob)) — a resubmit
|
||||
after a TERMINAL verdict (rejected learner retrying, or any
|
||||
re-verification with the same DOB) collided with the insert-only
|
||||
store's PK and 500'd forever. Ids must be unique per submit(); a
|
||||
terminal-then-resubmit (rate cap permitting) is a fresh pending
|
||||
submission, never a duplicate-PK crash."""
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "re-i.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["retry-learner"],
|
||||
identity_submits_per_min=10,
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
with TestClient(app) as c:
|
||||
# terminal REJECTED record first (under-16 path)
|
||||
sub = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "retry-learner", "date_of_birth": "2015-01-01"},
|
||||
).json()
|
||||
v = c.post(f"/v1/identity/verify/{sub['submission_id']}").json()
|
||||
assert v["status"] == "rejected"
|
||||
# same learner + same DOB resubmits: fresh pending, NOT a 500
|
||||
second = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "retry-learner", "date_of_birth": "2015-01-01"},
|
||||
)
|
||||
assert second.status_code == 200, second.text
|
||||
assert second.json()["status"] == "pending"
|
||||
assert second.json()["submission_id"] != sub["submission_id"]
|
||||
# while the second is still PENDING, G-13 caps resubmits at 409
|
||||
# (one active pending per learner) — a policy 4xx, never the
|
||||
# duplicate-PK 500 the deterministic-id bug produced.
|
||||
third = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "retry-learner", "date_of_birth": "2015-01-01"},
|
||||
)
|
||||
assert third.status_code == 409
|
||||
|
||||
def test_pii_sentinel_never_stored_or_logged(
|
||||
self, tmp_path: Path, caplog
|
||||
) -> None:
|
||||
"""MH-3c (A-305): sentinel PII in submissions appears in NO log
|
||||
record and NO stored raw form."""
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "piii.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["pii-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
sentinel_dob = "1999-12-31"
|
||||
sentinel_doc = "SENTINEL-DOC-CONTENTS-XYZZY"
|
||||
with caplog.at_level(logging.DEBUG):
|
||||
with TestClient(app) as c:
|
||||
c.post(
|
||||
"/v1/identity/submit",
|
||||
json={
|
||||
"learner_id": "pii-learner",
|
||||
"date_of_birth": sentinel_dob,
|
||||
"document_refs": [sentinel_doc],
|
||||
},
|
||||
)
|
||||
# every log record + every captured source line
|
||||
logged = " ".join(r.getMessage() for r in caplog.records)
|
||||
assert sentinel_dob not in logged, "raw DOB leaked to logs"
|
||||
assert "1999" not in logged
|
||||
# store inspection: no raw DOB in any stored record
|
||||
from sqlalchemy import text
|
||||
|
||||
with store._engine.connect() as conn: # noqa: SLF001
|
||||
rows = conn.execute(
|
||||
text(
|
||||
"SELECT id, learner_id, status, verdict, age_band, "
|
||||
"document_refs FROM identity_record"
|
||||
)
|
||||
).fetchall()
|
||||
blob = json.dumps([list(map(str, r)) for r in rows])
|
||||
assert sentinel_dob not in blob, "raw DOB persisted"
|
||||
assert sentinel_doc in blob # the REF is stored (opaque handle) — refs are allowed
|
||||
|
||||
|
||||
class TestGateComposition:
|
||||
"""MH-3b: allowlist (first) → identity verdict → caps; band splits."""
|
||||
|
||||
def test_allowlist_403_fires_first(self, gated_client: TestClient) -> None:
|
||||
resp = gated_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "stranger-danger", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert resp.status_code == 403
|
||||
assert "allowlist" in resp.json()["detail"]
|
||||
|
||||
def test_unverified_allowlisted_gets_verify_cta(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
# allowlisted but NEVER identity-verified (not in the seed roster)
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS + ["fresh-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as gated_client:
|
||||
resp = gated_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "fresh-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert resp.status_code == 403
|
||||
detail = resp.json()["detail"]
|
||||
assert detail["reason"] == "identity_verification_required"
|
||||
assert detail["min_age"] == 16
|
||||
assert detail["current_status"] == "none"
|
||||
assert detail["verify_cta"] == "/enroll"
|
||||
|
||||
def test_verified_18_plus_passes_school_gate(self, gated_client: TestClient) -> None:
|
||||
# pilot-learner is seeded verified 18+ (G-9 seed)
|
||||
resp = gated_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "pilot-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
|
||||
def test_16_17_passes_school_but_blocked_marketplace(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
# seed a 16-17 verified learner
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="seed-minor",
|
||||
learner_id="minor-learner",
|
||||
status="verified",
|
||||
provider="mock",
|
||||
verdict={"status": "verified", "age_band": "16-17", "mock": True},
|
||||
age_band="16-17",
|
||||
submitted_at=datetime.now(UTC),
|
||||
verified_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS + ["minor-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
# school gate (16+): passes
|
||||
variants = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "minor-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert variants.status_code == 200, variants.text
|
||||
# marketplace gate (18+ verified): 403 with the age reason
|
||||
apply = c.post(
|
||||
"/v1/marketplace/apply",
|
||||
json={"learner_id": "minor-learner", "job_id": "job-001"},
|
||||
)
|
||||
assert apply.status_code == 403
|
||||
detail = apply.json()["detail"]
|
||||
assert detail["reason"] == "age_gate_18_plus"
|
||||
assert detail["min_age"] == 18
|
||||
|
||||
def test_verified_adult_marketplace_stub_is_honest_501(
|
||||
self, gated_client: TestClient
|
||||
) -> None:
|
||||
"""G-18: the gate passes; the route NEVER fabricates 'applied'."""
|
||||
resp = gated_client.post(
|
||||
"/v1/marketplace/apply",
|
||||
json={"learner_id": "pilot-learner", "job_id": "job-001"},
|
||||
)
|
||||
assert resp.status_code == 501
|
||||
body = resp.json()
|
||||
assert body["stub"] is True
|
||||
assert body["mock"] is True
|
||||
|
||||
def test_mh3e_flow_unverified_then_enrolled(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
"""MH-3e: unverified → 403 verify-CTA → submit+verify → 200."""
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["flow-learner"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
blocked = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "flow-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert blocked.status_code == 403
|
||||
assert blocked.json()["detail"]["verify_cta"] == "/enroll"
|
||||
|
||||
sub = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "flow-learner", "date_of_birth": ADULT_DOB},
|
||||
).json()
|
||||
c.post(f"/v1/identity/verify/{sub['submission_id']}")
|
||||
|
||||
allowed = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": "flow-learner", "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
assert allowed.status_code == 200, allowed.text
|
||||
|
||||
|
||||
class TestVerifierHardening:
|
||||
"""D1/D2/D3 (verifier P1/P1/P2): boundary validation, fail-closed gate,
|
||||
band constraints — the exception-path PII leak and the fail-open gate
|
||||
the first verify pass found."""
|
||||
|
||||
def test_d1_malformed_dob_422_at_boundary_never_500(
|
||||
self, tmp_path: Path, caplog
|
||||
) -> None:
|
||||
"""D1: a malformed DOB must die as 422 BEFORE any derivation runs —
|
||||
never a 500 whose traceback echoes the raw value into logs (A-305)
|
||||
nor a poisoned pending record that G-13 turns into a lockout."""
|
||||
store = SQLiteIdentityStore(db_path=tmp_path / "d1.db")
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=["victim"],
|
||||
db_path=tmp_path / "app.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = store
|
||||
poison = "1975-06-15XX"
|
||||
with caplog.at_level(logging.DEBUG):
|
||||
with TestClient(app) as c:
|
||||
resp = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "victim", "date_of_birth": poison},
|
||||
)
|
||||
assert resp.status_code == 422, "malformed DOB must be 422, not 500"
|
||||
assert poison not in resp.text, "422 must not echo the raw value"
|
||||
# No poisoned pending record: the learner can still submit.
|
||||
assert store.count_pending_for_learner("victim") == 0
|
||||
ok = c.post(
|
||||
"/v1/identity/submit",
|
||||
json={"learner_id": "victim", "date_of_birth": ADULT_DOB},
|
||||
)
|
||||
assert ok.status_code == 200
|
||||
logged = " ".join(r.getMessage() for r in caplog.records)
|
||||
assert poison not in logged, "raw DOB must never reach logs"
|
||||
|
||||
def test_d2_gate_fails_closed_on_non_canonical_bands(
|
||||
self, identity_store: SQLiteIdentityStore, tmp_path: Path
|
||||
) -> None:
|
||||
"""D2: None / unknown / under-16 bands can NEVER pass the 18+ gate
|
||||
(nor the school gate for non-canonical values). Non-canonical rows
|
||||
are planted with raw SQL — the store now refuses them (D3), so this
|
||||
simulates the future-vendor / direct-write path the gate must
|
||||
still defend against."""
|
||||
from sqlalchemy import text as sql_text
|
||||
|
||||
for band, expect_school, expect_market in (
|
||||
(None, False, False),
|
||||
("banana", False, False),
|
||||
("under-16", False, False),
|
||||
("16-17", True, False),
|
||||
("18+", True, True),
|
||||
):
|
||||
lid = f"band-{str(band or 'none')}"
|
||||
with identity_store._engine.begin() as conn: # noqa: SLF001
|
||||
conn.execute(
|
||||
sql_text(
|
||||
"INSERT INTO identity_record (id, learner_id, status, "
|
||||
"provider, verdict, age_band, document_refs, "
|
||||
"submitted_at) VALUES (:id, :lid, 'verified', 'mock', "
|
||||
"'{}', :band, '[]', CURRENT_TIMESTAMP)"
|
||||
),
|
||||
{"id": f"raw-{lid}", "lid": lid, "band": band},
|
||||
)
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=[lid],
|
||||
db_path=tmp_path / f"app-{lid}.db",
|
||||
)
|
||||
)
|
||||
app.state.identity_store = identity_store
|
||||
with TestClient(app) as c:
|
||||
school = c.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": lid, "competency_id": "stack-orchestration-c007"},
|
||||
)
|
||||
market = c.post(
|
||||
"/v1/marketplace/apply",
|
||||
json={"learner_id": lid, "job_id": "job-001"},
|
||||
)
|
||||
assert (school.status_code == 200) is expect_school, (
|
||||
f"band={band!r} school gate: {school.status_code}"
|
||||
)
|
||||
assert (market.status_code == 501) is expect_market, (
|
||||
f"band={band!r} marketplace gate: {market.status_code}"
|
||||
)
|
||||
|
||||
def test_d3_store_rejects_non_canonical_bands(self, identity_store) -> None:
|
||||
"""D3: the store refuses to create/mark non-canonical bands."""
|
||||
with pytest.raises(ValueError, match="invalid age_band"):
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-banana",
|
||||
learner_id="l",
|
||||
status="verified",
|
||||
provider="mock",
|
||||
age_band="banana",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="idc-vm",
|
||||
learner_id="l2",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
with pytest.raises(ValueError, match="invalid age_band"):
|
||||
identity_store.mark_verified("idc-vm", {"status": "verified"}, "banana")
|
||||
|
||||
def test_d4_latest_tiebreaks_deterministically(self, identity_store) -> None:
|
||||
"""D4: identical-microsecond records resolve to the LAST inserted."""
|
||||
same = datetime.now(UTC)
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="t-first",
|
||||
learner_id="tie-learner",
|
||||
status="verified",
|
||||
provider="mock",
|
||||
age_band="18+",
|
||||
submitted_at=same,
|
||||
verified_at=same,
|
||||
)
|
||||
)
|
||||
identity_store.insert(
|
||||
IdentityRecord(
|
||||
id="t-second",
|
||||
learner_id="tie-learner",
|
||||
status="pending",
|
||||
provider="mock",
|
||||
submitted_at=same,
|
||||
)
|
||||
)
|
||||
latest = identity_store.latest_for_learner("tie-learner")
|
||||
assert latest is not None and latest.id == "t-second"
|
||||
@@ -0,0 +1,382 @@
|
||||
"""Design/simulation environment tests (REQ-5-005/006, D-044, G-15).
|
||||
|
||||
MH-4a: templates generate kind-tagged variants with correct starter files +
|
||||
commands; command fields roundtrip the shlex validator; wire response
|
||||
carries both fields (required); TS types match Python field-for-field
|
||||
(the dual-schema rule — checked in review by the TS typecheck + here by
|
||||
the response shape).
|
||||
MH-4b: exec policy — design/sim kinds reject out-of-policy argv[0] (422
|
||||
naming the allowed set); sh -c passthrough rejected; build kind unchanged.
|
||||
MH-4d: design-kind E2E in the real-server harness with concrete
|
||||
assertions (stored seqs contiguous; digest computes over a design-kind
|
||||
trace — kind-agnostic by construction, now pinned).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.grading.features import TraceDigest, compute_digest
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.main import create_app
|
||||
from ai_service.telemetry.models import TelemetryEvent
|
||||
from ai_service.telemetry.store import SQLiteTraceStore
|
||||
from ai_service.variants.generator import VariantGenerator
|
||||
from ai_service.variants.store import SQLiteVariantStore
|
||||
from ai_service.variants.templates import TEMPLATES, validate_simple_argv
|
||||
|
||||
from ..conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
DESIGN_LEARNER = "pilot-learner" # verified 18+ via the suite seed
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def env_client(tmp_path):
|
||||
store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
identity = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
seed_verified_identity(identity)
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
db_path=tmp_path / "env-app.db",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity
|
||||
app.state.variant_store = store
|
||||
app.state.variant_generator = VariantGenerator(store, MockProvider(), model="gemma4:31b")
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
|
||||
|
||||
class TestTemplateRegistry:
|
||||
"""MH-4a: registry + generator + wire."""
|
||||
|
||||
def test_all_kinds_present(self) -> None:
|
||||
kinds = {t.environment for t in TEMPLATES.values()}
|
||||
assert kinds == {"build", "design", "simulation"}
|
||||
|
||||
def test_g15_command_roundtrip_validator(self) -> None:
|
||||
assert validate_simple_argv("python simulate.py") == "python simulate.py"
|
||||
with pytest.raises(ValueError, match="whitespace-joinable"):
|
||||
validate_simple_argv('sh -c "echo hi"')
|
||||
with pytest.raises(ValueError, match="not be empty"):
|
||||
validate_simple_argv(" ")
|
||||
|
||||
def test_design_variant_generates_with_kind_and_files(self, env_client) -> None:
|
||||
resp = env_client.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": DESIGN_LEARNER,
|
||||
"template_id": "tpl-conversation-flow-design",
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["environment"] == "design"
|
||||
assert body["test_command"] == "python3 validate_flow.py"
|
||||
assert "flow.md" in body["starter_files"]
|
||||
assert "validate_flow.py" in body["starter_files"]
|
||||
|
||||
def test_simulation_variant_generates_with_kind(self, env_client) -> None:
|
||||
resp = env_client.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": DESIGN_LEARNER,
|
||||
"template_id": "tpl-sensor-benchmark",
|
||||
},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["environment"] == "simulation"
|
||||
assert body["test_command"] == "pytest -q"
|
||||
assert "simulate.py" in body["starter_files"]
|
||||
|
||||
def test_build_variants_default_kind(self, env_client) -> None:
|
||||
resp = env_client.post(
|
||||
"/v1/variants",
|
||||
json={"learner_id": DESIGN_LEARNER, "template_id": "tpl-llm-judge"},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
body = resp.json()
|
||||
assert body["environment"] == "build"
|
||||
|
||||
|
||||
class TestExecPolicy:
|
||||
"""MH-4b: exact-token, per-kind; sh -c disallowed for design/sim."""
|
||||
|
||||
@pytest.fixture()
|
||||
def exec_client(self, tmp_path):
|
||||
"""App + a STUB backend sandbox bound to a DESIGN-kind variant
|
||||
(policy check happens before execution — no real namespace needed)."""
|
||||
from ai_service.sandbox.backend import ExecResult
|
||||
from ai_service.sandbox.manager import SandboxManager
|
||||
from tests.api.test_sandboxes import StubBackend
|
||||
|
||||
class ExecStubBackend(StubBackend):
|
||||
"""StubBackend + a working exec (policy fires BEFORE exec)."""
|
||||
|
||||
async def exec(self, handle, cmd): # type: ignore[override]
|
||||
from datetime import UTC, datetime
|
||||
|
||||
return ExecResult(
|
||||
cmd=list(cmd),
|
||||
returncode=0,
|
||||
stdout="ok",
|
||||
stderr="",
|
||||
duration_s=0.0,
|
||||
ts=datetime.now(UTC),
|
||||
)
|
||||
|
||||
vstore = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
identity = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
seed_verified_identity(identity)
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
sandbox_dir=tmp_path / "sandboxes",
|
||||
db_path=tmp_path / "exec-app.db",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity
|
||||
app.state.variant_store = vstore
|
||||
app.state.variant_generator = VariantGenerator(
|
||||
vstore, MockProvider(), model="gemma4:31b"
|
||||
)
|
||||
stub = ExecStubBackend()
|
||||
manager = SandboxManager(backend=stub, settings=settings)
|
||||
app.state.sandbox_manager = manager
|
||||
with TestClient(app) as c:
|
||||
# create a design variant + a sandbox for its task
|
||||
var = c.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": DESIGN_LEARNER,
|
||||
"template_id": "tpl-conversation-flow-design",
|
||||
},
|
||||
).json()
|
||||
sbx = c.post(
|
||||
"/v1/sandboxes",
|
||||
json={"learner_id": DESIGN_LEARNER, "task_id": var["task_id"]},
|
||||
).json()
|
||||
c._sandbox_id = sbx["id"] # type: ignore[attr-defined]
|
||||
yield c
|
||||
|
||||
def test_design_kind_rejects_out_of_policy_command(self, exec_client) -> None:
|
||||
sbx = exec_client._sandbox_id # type: ignore[attr-defined]
|
||||
resp = exec_client.post(f"/v1/sandboxes/{sbx}/exec", json={"cmd": ["rm", "-rf", "/"]})
|
||||
assert resp.status_code == 422
|
||||
assert "'rm'" in resp.json()["detail"]
|
||||
assert "allowed" in resp.json()["detail"]
|
||||
|
||||
def test_design_kind_rejects_shell_passthrough(self, exec_client) -> None:
|
||||
sbx = exec_client._sandbox_id # type: ignore[attr-defined]
|
||||
resp = exec_client.post(
|
||||
f"/v1/sandboxes/{sbx}/exec", json={"cmd": ["sh", "-c", "anything"]}
|
||||
)
|
||||
assert resp.status_code == 422
|
||||
assert "passthrough" in resp.json()["detail"]
|
||||
|
||||
def test_design_kind_allows_declared_harness(self, exec_client) -> None:
|
||||
"""The policy passes the declared harness (StubBackend.exec raises
|
||||
NotImplementedError by design — any status EXCEPT 422 proves the
|
||||
policy allowed the command through)."""
|
||||
sbx = exec_client._sandbox_id # type: ignore[attr-defined]
|
||||
resp = exec_client.post(
|
||||
f"/v1/sandboxes/{sbx}/exec",
|
||||
json={"cmd": ["python3", "validate_flow.py"]},
|
||||
)
|
||||
assert resp.status_code != 422, resp.text
|
||||
|
||||
def test_build_kind_policy_unchanged(self, tmp_path) -> None:
|
||||
from ai_service.api.sandboxes import _enforce_exec_policy
|
||||
|
||||
_enforce_exec_policy(["whatever", "anywhere"], "build") # no raise
|
||||
_enforce_exec_policy(["sh", "-c", "x"], None) # unknown env: no raise
|
||||
|
||||
|
||||
class TestDigestKindAgnostic:
|
||||
"""MH-4d (part): compute_digest over a synthetic DESIGN-kind trace —
|
||||
the digest derives from event kinds, never environment types."""
|
||||
|
||||
def test_design_trace_digests_like_build_traces(self) -> None:
|
||||
"""compute_digest(trace) over a synthetic design-kind event stream —
|
||||
same feature classes as a build trace: command counts, run results,
|
||||
edit cadence. The environment kind never enters the computation."""
|
||||
from datetime import UTC, datetime
|
||||
|
||||
ts = datetime(2026, 9, 13, 12, 0, 0, tzinfo=UTC)
|
||||
base = {
|
||||
"learner_id": "digest-learner",
|
||||
"task_id": "task-design-1",
|
||||
"sandbox_id": "sbx-design",
|
||||
}
|
||||
events = [
|
||||
TelemetryEvent(
|
||||
seq=0,
|
||||
kind="command",
|
||||
payload={"cmd": "python validate_flow.py"},
|
||||
ts=ts,
|
||||
**base,
|
||||
),
|
||||
TelemetryEvent(
|
||||
seq=1,
|
||||
kind="file_diff",
|
||||
payload={"path": "flow.md", "diff": "+## Turn 2"},
|
||||
ts=ts,
|
||||
**base,
|
||||
),
|
||||
TelemetryEvent(
|
||||
seq=2,
|
||||
kind="run_result",
|
||||
payload={
|
||||
"cmd": "python validate_flow.py",
|
||||
"exit_code": 0,
|
||||
"stdout": "VALID",
|
||||
},
|
||||
ts=ts,
|
||||
**base,
|
||||
),
|
||||
]
|
||||
digest = compute_digest(events)
|
||||
assert digest.command_count == 1
|
||||
assert digest.run_count == 1
|
||||
# The digest model has NO environment/kind field — kind-agnostic by
|
||||
# construction; assert it stays that way.
|
||||
assert "environment" not in TraceDigest.model_fields
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_design_kind_e2e_real_server(tmp_path, sandbox_dir) -> None:
|
||||
"""MH-4d (G-17 — concrete assertions, no hope-shaped must-haves):
|
||||
a design-kind variant → REAL namespace sandbox → starter files →
|
||||
harness exec in-ns → telemetry flows → contiguous seq chain stored.
|
||||
The ack/trim coverage is the P1 suite's (real agent); here the REAL
|
||||
agent runs too — the spool assertion rides the stored contiguity."""
|
||||
import asyncio
|
||||
import contextlib
|
||||
import socket as sock_lib
|
||||
|
||||
import uvicorn
|
||||
|
||||
from ai_service.sandbox import SandboxManager
|
||||
from ai_service.sandbox.unshare_backend import UnshareBackend
|
||||
from ai_service.telemetry.ingest import TraceIntegrityMap
|
||||
from tests.sandbox.test_isolation import USERSNS_AVAILABLE
|
||||
|
||||
if not USERSNS_AVAILABLE:
|
||||
pytest.skip("user namespaces unavailable on this host (probe)")
|
||||
|
||||
def _free_port() -> int:
|
||||
with sock_lib.socket() as s:
|
||||
s.bind(("127.0.0.1", 0))
|
||||
return s.getsockname()[1]
|
||||
|
||||
store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
identity = SQLiteIdentityStore(db_path=tmp_path / "i.db")
|
||||
seed_verified_identity(identity)
|
||||
|
||||
port = _free_port()
|
||||
settings = Settings(
|
||||
provider="mock",
|
||||
learner_allowlist=SUITE_LEARNERS,
|
||||
db_path=tmp_path / "e2e-app.db",
|
||||
sandbox_dir=sandbox_dir,
|
||||
port=port,
|
||||
telemetry_ingest_host="127.0.0.1",
|
||||
)
|
||||
app = create_app(settings)
|
||||
app.state.identity_store = identity
|
||||
app.state.variant_store = store
|
||||
app.state.variant_generator = VariantGenerator(
|
||||
store, MockProvider(), model="gemma4:31b"
|
||||
)
|
||||
app.state.trace_store = trace_store
|
||||
app.state.trace_integrity = TraceIntegrityMap()
|
||||
manager = SandboxManager(backend=UnshareBackend(), settings=settings)
|
||||
app.state.sandbox_manager = manager
|
||||
|
||||
server = uvicorn.Server(
|
||||
uvicorn.Config(app, host="127.0.0.1", port=port, log_level="warning")
|
||||
)
|
||||
serve_task = asyncio.get_running_loop().create_task(server.serve())
|
||||
try:
|
||||
for _ in range(100):
|
||||
if server.started:
|
||||
break
|
||||
await asyncio.sleep(0.1)
|
||||
assert server.started
|
||||
|
||||
import httpx
|
||||
|
||||
async with httpx.AsyncClient(base_url=f"http://127.0.0.1:{port}", timeout=60.0) as client:
|
||||
# 1. design variant (kind-tagged, python3 harness — in-ns PATH)
|
||||
var = (
|
||||
await client.post(
|
||||
"/v1/variants",
|
||||
json={
|
||||
"learner_id": "pilot-learner",
|
||||
"template_id": "tpl-conversation-flow-design",
|
||||
},
|
||||
)
|
||||
).json()
|
||||
assert var["environment"] == "design"
|
||||
|
||||
# 2. sandbox for the design task
|
||||
sbx = (
|
||||
await client.post(
|
||||
"/v1/sandboxes",
|
||||
json={"learner_id": "pilot-learner", "task_id": var["task_id"]},
|
||||
)
|
||||
).json()
|
||||
|
||||
# 3. materialize starter files (the client's job — mirror it)
|
||||
for path, content in var["starter_files"].items():
|
||||
resp = await client.put(
|
||||
f"/v1/sandboxes/{sbx['id']}/files/{path}",
|
||||
json={"path": path, "content": content},
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
|
||||
# 4. run the design harness IN THE NAMESPACE (python3 resolves)
|
||||
run = (
|
||||
await client.post(
|
||||
f"/v1/sandboxes/{sbx['id']}/exec",
|
||||
json={"cmd": ["python3", "validate_flow.py"]},
|
||||
)
|
||||
).json()
|
||||
# starter flow.md fails validation on purpose (needs learner edits)
|
||||
assert "ISSUES" in run.get("stdout", "") or run.get("returncode") in (0, 1)
|
||||
|
||||
# 5. out-of-policy command is 422 at the exec route (G-15)
|
||||
rejected = await client.post(
|
||||
f"/v1/sandboxes/{sbx['id']}/exec",
|
||||
json={"cmd": ["nmap", "-p", "1-1000", "localhost"]},
|
||||
)
|
||||
assert rejected.status_code == 422
|
||||
|
||||
# 6. telemetry flowed: contiguous seq chain, kind-agnostic.
|
||||
# The in-ns exec + file writes stream through the capture
|
||||
# agent (watcher ~250ms + command events + heartbeats).
|
||||
import time as _time
|
||||
|
||||
deadline = _time.monotonic() + 15.0
|
||||
events = trace_store.get_trace("pilot-learner", var["task_id"])
|
||||
while _time.monotonic() < deadline and len(events) < 2:
|
||||
await asyncio.sleep(0.5)
|
||||
events = trace_store.get_trace("pilot-learner", var["task_id"])
|
||||
seqs = [e.seq for e in events]
|
||||
assert len(seqs) >= 2, f"no telemetry flowed: {seqs}"
|
||||
assert seqs == sorted(seqs), f"out of order: {seqs}"
|
||||
assert len(set(seqs)) == len(seqs), f"duplicates: {seqs}"
|
||||
assert seqs == list(range(seqs[0], seqs[-1] + 1)), f"gaps: {seqs}"
|
||||
|
||||
await client.delete(f"/v1/sandboxes/{sbx['id']}")
|
||||
finally:
|
||||
server.should_exit = True
|
||||
with contextlib.suppress(Exception):
|
||||
await asyncio.wait_for(serve_task, timeout=10.0)
|
||||
trace_store.close()
|
||||
@@ -356,3 +356,58 @@ def test_concurrent_writer_and_reader_no_database_is_locked(tmp_path: Path) -> N
|
||||
finally:
|
||||
reader.close()
|
||||
writer.close()
|
||||
|
||||
|
||||
def test_v04_schema_backfilled_on_open(tmp_path: Path) -> None:
|
||||
"""Cross-phase P0 regression (final review): a pre-v0.5 database has a
|
||||
variant_record table WITHOUT the v0.5 environment/test_command columns.
|
||||
create_all does not ALTER existing tables, so opening the old DB with
|
||||
the v0.5 store used to fail every read/write with OperationalError
|
||||
("no such column: variant_record.environment"). The store now
|
||||
backfills the missing columns (idempotently) with the model defaults;
|
||||
pre-v0.5 rows read as build-kind, test_command falls back at the API
|
||||
seam."""
|
||||
import sqlite3
|
||||
|
||||
db_path = tmp_path / "v04-legacy.db"
|
||||
con = sqlite3.connect(db_path)
|
||||
con.execute(
|
||||
"""
|
||||
CREATE TABLE variant_record (
|
||||
learner_id VARCHAR NOT NULL,
|
||||
template_id VARCHAR NOT NULL,
|
||||
task_id VARCHAR NOT NULL,
|
||||
seed VARCHAR NOT NULL,
|
||||
params JSON,
|
||||
statement VARCHAR NOT NULL,
|
||||
starter_files JSON,
|
||||
created_at DATETIME NOT NULL,
|
||||
PRIMARY KEY (learner_id, template_id),
|
||||
CONSTRAINT uq_variant_record_task_id UNIQUE (task_id)
|
||||
)
|
||||
"""
|
||||
)
|
||||
con.execute(
|
||||
"INSERT INTO variant_record VALUES "
|
||||
"('legacy-learner','tpl-llm-judge','task-legacy','seed','{}','stmt','{}',"
|
||||
"'2026-01-01 00:00:00')"
|
||||
)
|
||||
con.commit()
|
||||
con.close()
|
||||
|
||||
store = SQLiteVariantStore(db_path=db_path)
|
||||
try:
|
||||
legacy = store.get_by_task("task-legacy")
|
||||
assert legacy is not None, "legacy row unreadable — schema backfill failed"
|
||||
assert legacy.environment == "build" # v0.5 default for pre-v0.5 rows
|
||||
assert legacy.test_command == ""
|
||||
# writes against the migrated table also work
|
||||
new = make_variant(learner_id="legacy-learner", template_id="tpl-new")
|
||||
store.save(new)
|
||||
got = store.get("legacy-learner", "tpl-new")
|
||||
assert got is not None and got.environment == "build"
|
||||
# reopening is idempotent (backfill re-runs harmlessly)
|
||||
again = SQLiteVariantStore(db_path=db_path)
|
||||
again.close()
|
||||
finally:
|
||||
store.close()
|
||||
|
||||
@@ -32,8 +32,16 @@ DEFENSE_TURN_BUDGET_MS = 4_000
|
||||
|
||||
@pytest.fixture()
|
||||
def client(tmp_path: Path) -> TestClient:
|
||||
from ai_service.identity.store import SQLiteIdentityStore
|
||||
from tests.conftest import SUITE_LEARNERS, seed_verified_identity
|
||||
|
||||
llm = MockProvider()
|
||||
app = create_app(Settings(provider="mock", voice_provider="mock"))
|
||||
app = create_app(
|
||||
Settings(provider="mock", voice_provider="mock", learner_allowlist=SUITE_LEARNERS)
|
||||
)
|
||||
identity_store = SQLiteIdentityStore(db_path=tmp_path / "identity.db")
|
||||
seed_verified_identity(identity_store)
|
||||
app.state.identity_store = identity_store # G-9
|
||||
app.state.provider = llm
|
||||
app.state.trace_store = SQLiteTraceStore(db_path=tmp_path / "t.db")
|
||||
app.state.variant_store = SQLiteVariantStore(db_path=tmp_path / "v.db")
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
"""OpenAIAudioProvider tests — byte-exact STT/TTS via httpx.MockTransport
|
||||
(D-040, REQ-5-001, MH-2a). Mirrors the llm/openai_compat test pattern: the
|
||||
transport handler asserts the wire shape and returns canned bodies; failure
|
||||
pins prove sanitized errors and NO key leak (pinned).
|
||||
|
||||
Cloud-free: the real endpoint is a manual probe recipe (.env.example).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from ai_service.voice.openai_audio import OpenAIAudioProvider
|
||||
|
||||
KEY = "sk-voice-test-xyz"
|
||||
|
||||
|
||||
def make_provider(handler, **overrides) -> OpenAIAudioProvider:
|
||||
transport = httpx.MockTransport(handler)
|
||||
client = httpx.AsyncClient(transport=transport)
|
||||
kwargs = {
|
||||
"base_url": "https://voice.example/v1",
|
||||
"api_key": KEY,
|
||||
"stt_model": "whisper-1",
|
||||
"tts_model": "tts-1",
|
||||
"tts_voice": "alloy",
|
||||
"tts_format": "mp3",
|
||||
}
|
||||
kwargs.update(overrides)
|
||||
return OpenAIAudioProvider(http_client=client, **kwargs)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_sends_multipart_and_parses_response():
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["path"] = request.url.path
|
||||
seen["auth"] = request.headers.get("authorization", "")
|
||||
body = request.content
|
||||
seen["multipart"] = b"answer.webm" in body and b'name="file"' in body
|
||||
seen["model_field"] = b"whisper-1" in body
|
||||
return httpx.Response(200, json={"text": "hello from audio"})
|
||||
|
||||
provider = make_provider(handler)
|
||||
segment = await provider.transcribe(b"\x1a\x45\xa3\xdf", "webm")
|
||||
assert segment.text == "hello from audio"
|
||||
assert seen["path"] == "/v1/audio/transcriptions"
|
||||
assert seen["auth"] == f"Bearer {KEY}"
|
||||
assert seen["multipart"], "multipart must carry the file with a clean ext"
|
||||
assert seen["model_field"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_413_maps_to_sanitized_error_no_key_leak():
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(413, json={"error": {"message": f"too large {KEY}"}})
|
||||
|
||||
provider = make_provider(handler)
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
await provider.transcribe(b"audio" * 100, "wav")
|
||||
msg = str(exc_info.value)
|
||||
assert KEY not in msg, "api_key must never appear in exceptions"
|
||||
assert "voice provider error" in msg
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_400_and_429_sanitized():
|
||||
for status, body in ((400, {"error": {"message": "bad format"}}),
|
||||
(429, {"error": {"message": "insufficient_quota"}})):
|
||||
provider = make_provider(lambda r, s=status, b=body: httpx.Response(s, json=b))
|
||||
with pytest.raises(RuntimeError, match="voice provider error"):
|
||||
await provider.transcribe(b"x", "wav")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_empty_transcript_is_contract_break():
|
||||
provider = make_provider(lambda r: httpx.Response(200, json={"text": " "}))
|
||||
with pytest.raises(RuntimeError, match="empty transcription"):
|
||||
await provider.transcribe(b"x", "wav")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_synthesize_sends_json_body_and_streams_bytes():
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["path"] = request.url.path
|
||||
seen["body"] = json.loads(request.content)
|
||||
return httpx.Response(200, content=b"\xff\xfa\x00\x01\xff\xfb\x80\x00")
|
||||
|
||||
provider = make_provider(handler)
|
||||
chunks = [c async for c in provider.synthesize("Explain your approach.")]
|
||||
assert b"".join(chunks) == b"\xff\xfa\x00\x01\xff\xfb\x80\x00"
|
||||
assert seen["path"] == "/v1/audio/speech"
|
||||
assert seen["body"] == {
|
||||
"model": "tts-1",
|
||||
"input": "Explain your approach.",
|
||||
"voice": "alloy",
|
||||
"response_format": "mp3",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_synthesize_voice_override_and_input_guard():
|
||||
provider = make_provider(lambda r: httpx.Response(200, content=b"ok"))
|
||||
# non-default voice passes through instead of the configured one
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["body"] = json.loads(request.content)
|
||||
return httpx.Response(200, content=b"ok")
|
||||
|
||||
provider = make_provider(handler)
|
||||
_ = [c async for c in provider.synthesize("q", voice="nova")]
|
||||
assert seen["body"]["voice"] == "nova"
|
||||
|
||||
with pytest.raises(RuntimeError, match="4096"):
|
||||
_ = [c async for c in provider.synthesize("x" * 4097)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_synthesize_http_error_sanitized_no_key_leak():
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(500, text=f"boom {KEY}")
|
||||
|
||||
provider = make_provider(handler)
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
_ = [c async for c in provider.synthesize("q")]
|
||||
assert KEY not in str(exc_info.value)
|
||||
|
||||
|
||||
def test_descriptor_advertises_server_mode():
|
||||
"""a-15: defense.py prefers a provider attribute descriptor — a missing
|
||||
one would badge the real server path as mock."""
|
||||
provider = make_provider(lambda r: httpx.Response(200, json={"text": "x"}))
|
||||
assert provider.descriptor.mode == "server"
|
||||
assert provider.descriptor.sr_available
|
||||
assert provider.descriptor.tts_available
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_timeout_sanitized_no_key_leak():
|
||||
"""MH-2a (P1-2): a read timeout is an httpx.HTTPError subclass — the
|
||||
sanitized path must catch it like any transport failure."""
|
||||
import httpx as _httpx
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
raise _httpx.ReadTimeout("read timed out while reading response")
|
||||
|
||||
provider = make_provider(handler)
|
||||
with pytest.raises(RuntimeError, match="voice provider error"):
|
||||
await provider.transcribe(b"audio", "wav")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transcribe_non_dict_200_body_context_wrapped():
|
||||
"""P2: a contract-breaking 200 body fails with provider context, not a
|
||||
raw AttributeError."""
|
||||
|
||||
provider = make_provider(lambda r: httpx.Response(200, json=["not", "a", "dict"]))
|
||||
with pytest.raises(RuntimeError, match="unexpected transcription response shape"):
|
||||
await provider.transcribe(b"x", "wav")
|
||||
|
||||
|
||||
def test_invalid_tts_format_falls_back_not_crashes(caplog):
|
||||
"""P1-1/G-16: a typo'd AI_VOICE_TTS_FORMAT must never crash the boot —
|
||||
normalize to 'mp3' with a loud warning (G-11 consistency)."""
|
||||
import logging
|
||||
|
||||
from ai_service.config import Settings
|
||||
|
||||
with caplog.at_level(logging.WARNING):
|
||||
s = Settings(voice_tts_format="flac")
|
||||
assert s.voice_tts_format == "mp3"
|
||||
assert any("flac" in r.message for r in caplog.records)
|
||||
# Valid values pass through unchanged.
|
||||
assert Settings(voice_tts_format="opus").voice_tts_format == "opus"
|
||||
@@ -80,10 +80,39 @@ class TestFactory:
|
||||
provider = voice_provider_from_settings(Settings(voice_provider="browser"))
|
||||
assert isinstance(provider, MockVoiceProvider)
|
||||
|
||||
def test_real_server_stt_tts_rejected_as_v04_seam(self) -> None:
|
||||
with pytest.raises(UnknownVoiceProviderError, match="v0.4"):
|
||||
def test_openai_audio_without_config_rejected_actionably(self) -> None:
|
||||
"""v0.4 inverted: the seam is live now. Unconfigured = actionable
|
||||
raise for direct callers (G-11's test half; main.py falls back)."""
|
||||
with pytest.raises(UnknownVoiceProviderError, match="AI_VOICE_BASE_URL"):
|
||||
voice_provider_from_settings(Settings(voice_provider="openai-audio"))
|
||||
|
||||
def test_openai_audio_without_http_client_rejected(self) -> None:
|
||||
with pytest.raises(UnknownVoiceProviderError, match="httpx client"):
|
||||
voice_provider_from_settings(
|
||||
Settings(
|
||||
voice_provider="openai-audio",
|
||||
voice_base_url="https://v.example",
|
||||
voice_api_key="k",
|
||||
)
|
||||
)
|
||||
|
||||
def test_openai_audio_configured_builds_server_mode_provider(self) -> None:
|
||||
import httpx
|
||||
|
||||
from ai_service.voice.openai_audio import OpenAIAudioProvider
|
||||
|
||||
provider = voice_provider_from_settings(
|
||||
Settings(
|
||||
voice_provider="openai-audio",
|
||||
voice_base_url="https://v.example/v1",
|
||||
voice_api_key="k",
|
||||
voice_tts_format="wav",
|
||||
),
|
||||
httpx.AsyncClient(),
|
||||
)
|
||||
assert isinstance(provider, OpenAIAudioProvider)
|
||||
assert provider.descriptor.mode == "server"
|
||||
|
||||
def test_unknown_provider_rejected(self) -> None:
|
||||
with pytest.raises(UnknownVoiceProviderError, match="unknown"):
|
||||
voice_provider_from_settings(Settings(voice_provider="watson"))
|
||||
@@ -98,7 +127,7 @@ class TestDescriptors:
|
||||
|
||||
def test_mock_descriptor(self) -> None:
|
||||
assert MOCK_DESCRIPTOR.mode == "mock"
|
||||
assert "v0.4" in MOCK_DESCRIPTOR.hint
|
||||
assert "v0.5" in MOCK_DESCRIPTOR.hint
|
||||
|
||||
|
||||
class TestZeroNetwork:
|
||||
@@ -116,3 +145,37 @@ class TestZeroNetwork:
|
||||
if node.level and node.module:
|
||||
assert node.module.split(".")[-1] != "agents", py
|
||||
assert node.module.split(".")[-1] != "api", py
|
||||
|
||||
|
||||
class TestBootSurvival:
|
||||
"""G-11: a misconfigured real provider must never crash the boot."""
|
||||
|
||||
def test_lifespan_falls_back_to_mock_with_loud_log(self, caplog) -> None:
|
||||
from ai_service.main import create_app
|
||||
|
||||
app = create_app(
|
||||
Settings(
|
||||
provider="mock",
|
||||
voice_provider="openai-audio", # typo'd/incomplete env
|
||||
voice_base_url="",
|
||||
voice_api_key="",
|
||||
)
|
||||
)
|
||||
with caplog.at_level("WARNING"):
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
with TestClient(app) as c:
|
||||
# Boot succeeded; health is green.
|
||||
assert c.get("/health").status_code == 200
|
||||
from ai_service.voice.mock import MockVoiceProvider
|
||||
|
||||
assert isinstance(app.state.voice_provider, MockVoiceProvider)
|
||||
# The descriptor honestly reports mock — the UI badge cannot
|
||||
# lie about which path is live.
|
||||
desc = c.get("/v1/defense/descriptor").json() if c.get(
|
||||
"/v1/defense/descriptor"
|
||||
).status_code == 200 else None
|
||||
assert desc is None or desc.get("mode") in ("mock", "browser", "server")
|
||||
assert any(
|
||||
"falling back to mock" in r.message for r in caplog.records
|
||||
), "the fallback must log loudly, naming the fix"
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
'use client';
|
||||
|
||||
import { useCallback, useEffect, useState } from 'react';
|
||||
import { BadgeCheck, CircleAlert, Loader2, ShieldCheck } from 'lucide-react';
|
||||
import { Button, Card, CardBody, Input } from '@nextcraft/ui';
|
||||
import {
|
||||
MOCK_LEARNER_ID,
|
||||
getIdentityStatus,
|
||||
submitIdentity,
|
||||
verifyIdentity,
|
||||
} from '../../../lib/engine-client';
|
||||
import type { IdentityStatus } from '../../../lib/engine-client';
|
||||
|
||||
/**
|
||||
* Identity enrollment (REQ-5-003/004): submit verification → pending →
|
||||
* verified/rejected. Honest at every state (A-304): mock verdicts are
|
||||
* LABELED mock — this surface never displays mock-verified as
|
||||
* production-verified. Gated-route 403s send learners here via the
|
||||
* verify-CTA payload (G-10 → VerifyRequiredError).
|
||||
*/
|
||||
export function EnrollFlow() {
|
||||
const [status, setStatus] = useState<IdentityStatus | null>(null);
|
||||
const [dob, setDob] = useState('');
|
||||
const [busy, setBusy] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [message, setMessage] = useState<string | null>(null);
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
try {
|
||||
const s = await getIdentityStatus(MOCK_LEARNER_ID);
|
||||
setStatus(s);
|
||||
} catch {
|
||||
setStatus(null);
|
||||
}
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
void refresh();
|
||||
}, [refresh]);
|
||||
|
||||
const submit = useCallback(async () => {
|
||||
if (!dob) {
|
||||
setError('Enter your date of birth to start verification.');
|
||||
return;
|
||||
}
|
||||
setBusy(true);
|
||||
setError(null);
|
||||
setMessage(null);
|
||||
try {
|
||||
const sub = await submitIdentity(MOCK_LEARNER_ID, dob);
|
||||
const verdict = await verifyIdentity(sub.submission_id);
|
||||
if (verdict.status === 'verified') {
|
||||
setMessage(
|
||||
`Verified${verdict.mock ? ' (mock provider — not production verification)' : ''}.`,
|
||||
);
|
||||
} else {
|
||||
setError(verdict.detail || 'Verification rejected.');
|
||||
}
|
||||
await refresh();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Verification failed.');
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}, [dob, refresh]);
|
||||
|
||||
return (
|
||||
<div className="mx-auto max-w-md space-y-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<ShieldCheck className="h-5 w-5 text-blue-600 dark:text-blue-400" aria-hidden />
|
||||
<h1 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Identity Verification
|
||||
</h1>
|
||||
{status?.mock && (
|
||||
<span className="rounded bg-slate-100 px-1.5 py-0.5 text-xs text-slate-500 dark:bg-slate-800 dark:text-slate-400">
|
||||
mock provider
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-300">
|
||||
The AI school floor is 16+; the marketplace requires 18+ with a verified
|
||||
identity. Verification is one step — your date of birth is used to
|
||||
derive your age band and is never stored raw.
|
||||
</p>
|
||||
|
||||
{status && status.status === 'verified' ? (
|
||||
<Card>
|
||||
<CardBody className="flex items-center gap-3">
|
||||
<BadgeCheck
|
||||
className="h-6 w-6 text-emerald-600 dark:text-emerald-400"
|
||||
aria-hidden
|
||||
/>
|
||||
<div>
|
||||
<p className="text-sm font-medium text-slate-800 dark:text-slate-100">
|
||||
Verified — age band {status.age_band}
|
||||
</p>
|
||||
{status.mock && (
|
||||
<p className="text-xs text-slate-500 dark:text-slate-400">
|
||||
mock verdict — not production verification
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
) : status && status.status === 'pending' ? (
|
||||
<Card>
|
||||
<CardBody className="flex items-center gap-3">
|
||||
<Loader2 className="h-5 w-5 animate-spin text-amber-500" aria-hidden />
|
||||
<p className="text-sm text-slate-700 dark:text-slate-300">
|
||||
Verification pending — resubmit once the current one settles.
|
||||
</p>
|
||||
</CardBody>
|
||||
</Card>
|
||||
) : (
|
||||
<Card>
|
||||
<CardBody className="space-y-3">
|
||||
<label htmlFor="dob" className="text-sm font-medium text-slate-700 dark:text-slate-300">
|
||||
Date of birth
|
||||
</label>
|
||||
<Input
|
||||
id="dob"
|
||||
type="date"
|
||||
value={dob}
|
||||
onChange={(e) => setDob(e.target.value)}
|
||||
aria-label="Date of birth"
|
||||
/>
|
||||
<Button onClick={() => void submit()} disabled={busy}>
|
||||
{busy ? <Loader2 className="h-4 w-4 animate-spin" aria-hidden /> : null}
|
||||
Start Verification
|
||||
</Button>
|
||||
</CardBody>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{error && (
|
||||
<p
|
||||
className="flex items-center gap-1 text-xs text-red-600 dark:text-red-400"
|
||||
role="alert"
|
||||
>
|
||||
<CircleAlert className="h-3 w-3" aria-hidden /> {error}
|
||||
</p>
|
||||
)}
|
||||
{message && (
|
||||
<p className="flex items-center gap-1 text-xs text-emerald-600 dark:text-emerald-400">
|
||||
<BadgeCheck className="h-3 w-3" aria-hidden /> {message}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
import type { Metadata } from 'next';
|
||||
import { EnrollFlow } from './EnrollFlow';
|
||||
|
||||
export const metadata: Metadata = {
|
||||
title: 'Enroll — Nextcraft',
|
||||
description: 'Verify your identity for the Nextcraft AI school (16+) and marketplace (18+).',
|
||||
};
|
||||
|
||||
export default function EnrollPage() {
|
||||
return (
|
||||
<div className="py-10">
|
||||
<EnrollFlow />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -118,6 +118,14 @@ export function BuildSurface({
|
||||
<p className="text-sm font-medium text-red-700 dark:text-red-300">
|
||||
{session.errorMessage ?? 'Could not start the build environment.'}
|
||||
</p>
|
||||
{session.verifyCta && (
|
||||
<a
|
||||
href={session.verifyCta}
|
||||
className="text-sm font-medium text-blue-600 underline dark:text-blue-400"
|
||||
>
|
||||
Verify your identity to continue →
|
||||
</a>
|
||||
)}
|
||||
<Button onClick={session.retry} variant="outline" size="sm">
|
||||
<RefreshCw className="h-3.5 w-3.5" aria-hidden /> Retry
|
||||
</Button>
|
||||
@@ -129,7 +137,7 @@ export function BuildSurface({
|
||||
<div className="space-y-6">
|
||||
<header className="space-y-2">
|
||||
<p className="text-xs font-semibold uppercase tracking-wide text-primary-600 dark:text-primary-400">
|
||||
{stackTitle} · build
|
||||
{stackTitle} · {session.variant?.environment ?? 'build'}
|
||||
</p>
|
||||
<h1 className="text-xl font-bold text-slate-900 dark:text-slate-100">
|
||||
{session.variant?.statement ?? 'Your task'}
|
||||
@@ -141,11 +149,11 @@ export function BuildSurface({
|
||||
</header>
|
||||
|
||||
<RunControls
|
||||
command="python -m pytest -q"
|
||||
command={session.variant?.test_command ?? 'python3 -m pytest -q'}
|
||||
running={running}
|
||||
busy={false}
|
||||
onRun={() => void run(['python', '-m', 'pytest', '-q'])}
|
||||
onTest={() => void run(['pytest', '-q'])}
|
||||
onRun={() => void run((session.variant?.test_command ?? 'python3 -m pytest -q').trim().split(/\s+/))}
|
||||
onTest={() => void session.test()}
|
||||
/>
|
||||
|
||||
<div className="grid gap-4 lg:grid-cols-[220px_minmax(0,1fr)]">
|
||||
|
||||
@@ -12,12 +12,15 @@ import type {
|
||||
import {
|
||||
MOCK_LEARNER_ID,
|
||||
answerDefense,
|
||||
answerDefenseAudio,
|
||||
finishDefense,
|
||||
getDefense,
|
||||
listVariants,
|
||||
requestGrade,
|
||||
startDefense,
|
||||
} from '../../lib/engine-client';
|
||||
import type { VoiceDescriptor } from '@nextcraft/types';
|
||||
import { MAX_RECORD_SECONDS, voiceBadgeLabel } from '../../lib/voice-badge';
|
||||
|
||||
type MicState = 'idle' | 'recording' | 'denied' | 'unsupported';
|
||||
|
||||
@@ -44,7 +47,10 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
const [finished, setFinished] = useState<DefenseFinish | null>(null);
|
||||
const [grade, setGrade] = useState<GradeRecord | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [descriptor, setDescriptor] = useState<VoiceDescriptor | null>(null);
|
||||
const [recordSeconds, setRecordSeconds] = useState(0);
|
||||
const recorderRef = useRef<MediaRecorder | null>(null);
|
||||
const recordTimerRef = useRef<ReturnType<typeof setInterval> | null>(null);
|
||||
|
||||
const refresh = useCallback(async (id: string) => {
|
||||
const session: DefenseSession = await getDefense(id);
|
||||
@@ -78,6 +84,7 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
try {
|
||||
const started = await startDefense(MOCK_LEARNER_ID, taskId);
|
||||
setDefenseId(started.defense_id);
|
||||
setDescriptor(started.voice_descriptor ?? null);
|
||||
await refresh(started.defense_id);
|
||||
if (!started.trace_complete) {
|
||||
setError(
|
||||
@@ -108,30 +115,81 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
[defenseId, refresh],
|
||||
);
|
||||
|
||||
const submitAudio = useCallback(
|
||||
async (blob: Blob) => {
|
||||
if (!defenseId) return;
|
||||
setBusy(true);
|
||||
try {
|
||||
// D-040/REQ-5-002: the recorded blob IS the answer — the server
|
||||
// transcribes it (real STT when openai-audio is configured; the
|
||||
// mock provider under tests/dev). The descriptor badge tells the
|
||||
// learner which path is live.
|
||||
await answerDefenseAudio(defenseId, blob);
|
||||
await refresh(defenseId);
|
||||
} catch (err) {
|
||||
// G-12: 413 renders the honest re-record prompt the server sends.
|
||||
setError(err instanceof Error ? err.message : 'Voice answer failed.');
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
},
|
||||
[defenseId, refresh],
|
||||
);
|
||||
|
||||
// P0-1 fix (verifier): with recorder.start(timeslice), ondataavailable
|
||||
// fires PER CHUNK — buffering them until onstop and posting ONE complete
|
||||
// blob, else every answer truncates to the first 1s slice (or splits into
|
||||
// two turns). Stop is the single completion signal; chunks accumulate.
|
||||
const chunksRef = useRef<Blob[]>([]);
|
||||
|
||||
const stopRecording = useCallback(() => {
|
||||
recorderRef.current?.stop();
|
||||
if (recordTimerRef.current) {
|
||||
clearInterval(recordTimerRef.current);
|
||||
recordTimerRef.current = null;
|
||||
}
|
||||
setRecordSeconds(0);
|
||||
}, []);
|
||||
|
||||
const record = useCallback(async () => {
|
||||
if (micState === 'recording') {
|
||||
recorderRef.current?.stop();
|
||||
stopRecording();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
||||
const recorder = new MediaRecorder(stream);
|
||||
recorderRef.current = recorder;
|
||||
recorder.ondataavailable = async (event) => {
|
||||
if (event.data.size === 0) return;
|
||||
// Browser-native SR fallback: v0.3 has no server STT key (CUT-1).
|
||||
// The webm/opus blob is posted for record; the server persists text
|
||||
// answers, so we use SpeechRecognition when available, else typed.
|
||||
if (!defenseId) return;
|
||||
chunksRef.current = [];
|
||||
recorder.ondataavailable = (event) => {
|
||||
if (event.data.size > 0) chunksRef.current.push(event.data);
|
||||
};
|
||||
recorder.onstop = () => {
|
||||
stream.getTracks().forEach((t) => t.stop());
|
||||
setMicState('idle');
|
||||
const chunks = chunksRef.current;
|
||||
if (chunks.length === 0) return;
|
||||
void submitAudio(new Blob(chunks, { type: recorder.mimeType || 'audio/webm' }));
|
||||
};
|
||||
recorder.start();
|
||||
// a-13: timeslice keeps the blob observable/chunked (buffered until
|
||||
// onstop — see the P0-1 note above).
|
||||
recorder.start(1000);
|
||||
setMicState('recording');
|
||||
setRecordSeconds(0);
|
||||
recordTimerRef.current = setInterval(() => {
|
||||
setRecordSeconds((s) => {
|
||||
if (s + 1 >= MAX_RECORD_SECONDS) {
|
||||
// G-12 auto-stop: the timer is visible, so this surprises no one.
|
||||
stopRecording();
|
||||
return MAX_RECORD_SECONDS;
|
||||
}
|
||||
return s + 1;
|
||||
});
|
||||
}, 1000);
|
||||
} catch {
|
||||
setMicState('denied');
|
||||
}
|
||||
}, [defenseId, micState]);
|
||||
}, [defenseId, micState, stopRecording, submitAudio]);
|
||||
|
||||
const finish = useCallback(async () => {
|
||||
if (!defenseId) return;
|
||||
@@ -160,7 +218,13 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
}
|
||||
}, [taskId]);
|
||||
|
||||
useEffect(() => () => recorderRef.current?.stop(), []);
|
||||
useEffect(
|
||||
() => () => {
|
||||
recorderRef.current?.stop();
|
||||
if (recordTimerRef.current) clearInterval(recordTimerRef.current);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
if (!taskId) {
|
||||
return (
|
||||
@@ -200,7 +264,7 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
<textarea
|
||||
value={answerText}
|
||||
onChange={(e) => setAnswerText(e.target.value)}
|
||||
placeholder="Type your answer (voice capture needs mic permission)…"
|
||||
placeholder="Type your answer — or record it with the mic button"
|
||||
aria-label="Your answer"
|
||||
className="min-h-[64px] flex-1 resize-y rounded-md border border-slate-300 bg-slate-50 p-3 text-sm text-slate-800 dark:border-slate-600 dark:bg-slate-800 dark:text-slate-100"
|
||||
/>
|
||||
@@ -217,12 +281,28 @@ export function DefenseSession({ competencyId }: { competencyId: string }) {
|
||||
<SendHorizonal className="h-4 w-4" aria-hidden /> Send
|
||||
</Button>
|
||||
</div>
|
||||
{micState === 'recording' && (
|
||||
<p
|
||||
className="flex items-center gap-1 text-xs text-red-600 dark:text-red-400"
|
||||
aria-live="polite"
|
||||
>
|
||||
<Square className="h-3 w-3" aria-hidden /> Recording… {recordSeconds}s /{' '}
|
||||
{180}s — auto-stops at the bound (G-12).
|
||||
</p>
|
||||
)}
|
||||
{micState === 'denied' && (
|
||||
<p className="flex items-center gap-1 text-xs text-slate-500 dark:text-slate-400">
|
||||
<XCircle className="h-3 w-3" aria-hidden /> Mic unavailable — typed answers are
|
||||
first-class.
|
||||
</p>
|
||||
)}
|
||||
{descriptor && (
|
||||
<p className="text-xs text-slate-500 dark:text-slate-400">
|
||||
Voice path:{' '}
|
||||
<span className="font-medium">{voiceBadgeLabel(descriptor)}</span>
|
||||
{descriptor.hint ? ` — ${descriptor.hint}` : ''}
|
||||
</p>
|
||||
)}
|
||||
<div className="flex justify-end">
|
||||
<Button onClick={() => void finish()} disabled={busy} variant="outline">
|
||||
Finish Defense
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import {
|
||||
VerifyRequiredError,
|
||||
EngineError,
|
||||
MOCK_LEARNER_ID,
|
||||
createSandbox,
|
||||
@@ -37,6 +38,8 @@ export interface SandboxSessionState {
|
||||
sandboxId: string | null;
|
||||
files: string[];
|
||||
errorMessage: string | null;
|
||||
/** G-10: identity-gate 403s carry an actionable enrollment link. */
|
||||
verifyCta: string | null;
|
||||
}
|
||||
|
||||
const DEFAULT_STATE: SandboxSessionState = {
|
||||
@@ -45,6 +48,7 @@ const DEFAULT_STATE: SandboxSessionState = {
|
||||
sandboxId: null,
|
||||
files: [],
|
||||
errorMessage: null,
|
||||
verifyCta: null,
|
||||
};
|
||||
|
||||
export function useSandboxSession(competencyId: string | null) {
|
||||
@@ -66,7 +70,13 @@ export function useSandboxSession(competencyId: string | null) {
|
||||
await writeFile(sandbox.id, path, content, controller.signal);
|
||||
}
|
||||
const files = await listFiles(sandbox.id, controller.signal);
|
||||
setState({ status: 'ready', variant, sandboxId: sandbox.id, files, errorMessage: null });
|
||||
setState({
|
||||
...DEFAULT_STATE,
|
||||
status: 'ready',
|
||||
variant,
|
||||
sandboxId: sandbox.id,
|
||||
files,
|
||||
});
|
||||
} catch (err) {
|
||||
// A created sandbox must not outlive a failed start (per-learner cap
|
||||
// is 1 — a leaked one blocks every retry with 429 forever). This
|
||||
@@ -79,6 +89,8 @@ export function useSandboxSession(competencyId: string | null) {
|
||||
...DEFAULT_STATE,
|
||||
status: err.status === 503 ? 'busy' : err.status === 403 || err.status === 429 ? 'denied' : 'error',
|
||||
errorMessage: err.message,
|
||||
// G-10: identity-gate 403s render the verify-CTA link (D-043).
|
||||
verifyCta: err instanceof VerifyRequiredError ? err.verifyCta : null,
|
||||
});
|
||||
return;
|
||||
}
|
||||
@@ -137,8 +149,11 @@ export function useSandboxSession(competencyId: string | null) {
|
||||
|
||||
const test = useCallback(async (): Promise<ExecResult | null> => {
|
||||
if (!state.variant || !state.sandboxId) return null;
|
||||
// All v0.3 templates ship pytest-based starter tests (PLAN Task 6-3-01).
|
||||
return run(['pytest', '-q']);
|
||||
// REQ-5-006: the variant's REAL test command (v0.3 hardcoded pytest;
|
||||
// design/sim kinds have their own). G-15: whitespace-only split —
|
||||
// templates validate quote-free at authoring (no shlex in browsers).
|
||||
const cmd = state.variant.test_command || 'pytest -q';
|
||||
return run(cmd.trim().split(/\s+/));
|
||||
}, [run, state.variant, state.sandboxId]);
|
||||
|
||||
const saveFile = useCallback(
|
||||
|
||||
@@ -46,6 +46,26 @@ export class EngineBusyError extends EngineError {
|
||||
}
|
||||
}
|
||||
|
||||
/** G-10 (v0.5): a 403 from the identity gate — actionable, not a dead end.
|
||||
* The verify-CTA payload tells the learner what to do (reason + minimum
|
||||
* age + where to enroll). */
|
||||
export class VerifyRequiredError extends EngineError {
|
||||
constructor(
|
||||
public readonly reason: string,
|
||||
public readonly minAge: number,
|
||||
public readonly currentStatus: string,
|
||||
public readonly verifyCta: string,
|
||||
) {
|
||||
super(
|
||||
reason === 'age_gate_18_plus'
|
||||
? 'You must be 18+ with a verified identity for the marketplace.'
|
||||
: 'Verify your identity to continue — it takes one step in enrollment.',
|
||||
403,
|
||||
);
|
||||
this.name = 'VerifyRequiredError';
|
||||
}
|
||||
}
|
||||
|
||||
export class NotAllowlistedError extends EngineError {
|
||||
constructor() {
|
||||
super('This learner is not allowlisted on this pilot.', 403);
|
||||
@@ -62,7 +82,28 @@ export class RateLimitedError extends EngineError {
|
||||
|
||||
async function parseError(resp: Response): Promise<EngineError> {
|
||||
if (resp.status === 503) return new EngineBusyError();
|
||||
if (resp.status === 403) return new NotAllowlistedError();
|
||||
if (resp.status === 403) {
|
||||
// G-10: discriminate 403 reasons — the payload shape decides.
|
||||
// verify_cta present → identity gate (actionable enrollment prompt);
|
||||
// allowlist detail → the G-5 pilot guard (unchanged message).
|
||||
let payload: unknown = null;
|
||||
try {
|
||||
payload = await resp.json();
|
||||
} catch {
|
||||
/* non-JSON 403 body */
|
||||
}
|
||||
const detail = (payload as { detail?: unknown } | null)?.detail;
|
||||
if (detail && typeof detail === 'object' && 'verify_cta' in (detail as object)) {
|
||||
const cta = detail as {
|
||||
reason: string;
|
||||
min_age: number;
|
||||
current_status: string;
|
||||
verify_cta: string;
|
||||
};
|
||||
return new VerifyRequiredError(cta.reason, cta.min_age, cta.current_status, cta.verify_cta);
|
||||
}
|
||||
return new NotAllowlistedError();
|
||||
}
|
||||
if (resp.status === 429) return new RateLimitedError();
|
||||
let detail = `${resp.status} ${resp.statusText}`;
|
||||
try {
|
||||
@@ -75,10 +116,17 @@ async function parseError(resp: Response): Promise<EngineError> {
|
||||
}
|
||||
|
||||
async function jsonFetch<T>(path: string, init?: RequestInit): Promise<T> {
|
||||
const headers = { 'Content-Type': 'application/json', ...init?.headers };
|
||||
// P0-2 (verifier): FormData must set its own Content-Type (multipart
|
||||
// boundary) — a forced application/json over multipart bytes makes every
|
||||
// audio answer die as 422 on a real server.
|
||||
if (init?.body instanceof FormData) {
|
||||
delete (headers as Record<string, unknown>)['Content-Type'];
|
||||
}
|
||||
const resp = await fetch(`${AI_SERVICE_URL}${path}`, {
|
||||
signal: init?.signal,
|
||||
...init,
|
||||
headers: { 'Content-Type': 'application/json', ...init?.headers },
|
||||
headers,
|
||||
});
|
||||
if (!resp.ok) throw await parseError(resp);
|
||||
return (await resp.json()) as T;
|
||||
@@ -208,6 +256,59 @@ export async function requestGrade(
|
||||
});
|
||||
}
|
||||
|
||||
// -- identity (REQ-5-003/004) --------------------------------------------------
|
||||
|
||||
export interface IdentitySubmitResponse {
|
||||
submission_id: string;
|
||||
status: string;
|
||||
mock: boolean;
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
export interface IdentityStatus {
|
||||
learner_id: string;
|
||||
status: string;
|
||||
age_band: string | null;
|
||||
mock: boolean;
|
||||
verified_at: string | null;
|
||||
}
|
||||
|
||||
export interface IdentityVerifyResponse extends IdentitySubmitResponse {
|
||||
age_band: string | null;
|
||||
}
|
||||
|
||||
export async function submitIdentity(
|
||||
learnerId: string,
|
||||
dateOfBirth: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<IdentitySubmitResponse> {
|
||||
return jsonFetch('/v1/identity/submit', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({
|
||||
learner_id: learnerId,
|
||||
date_of_birth: dateOfBirth,
|
||||
}),
|
||||
signal,
|
||||
});
|
||||
}
|
||||
|
||||
export async function getIdentityStatus(
|
||||
learnerId: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<IdentityStatus> {
|
||||
return jsonFetch(`/v1/identity/status/${learnerId}`, { signal });
|
||||
}
|
||||
|
||||
export async function verifyIdentity(
|
||||
submissionId: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<IdentityVerifyResponse> {
|
||||
return jsonFetch(`/v1/identity/verify/${submissionId}`, {
|
||||
method: 'POST',
|
||||
signal,
|
||||
});
|
||||
}
|
||||
|
||||
// -- oral defense (REQ-3-006) --------------------------------------------------
|
||||
|
||||
export async function startDefense(
|
||||
@@ -235,6 +336,27 @@ export async function answerDefense(
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Audio answer (D-040/REQ-5-002): POST the recorded blob as multipart —
|
||||
* the server transcribes it (real STT when openai-audio is configured,
|
||||
* mock otherwise). The 413 detail ("re-record") is surfaced verbatim so
|
||||
* the UI can render an honest retry prompt (G-12).
|
||||
*/
|
||||
export async function answerDefenseAudio(
|
||||
defenseId: string,
|
||||
blob: Blob,
|
||||
signal?: AbortSignal,
|
||||
): Promise<DefenseAnswer> {
|
||||
const form = new FormData();
|
||||
const ext = blob.type.split('/')[1]?.split(';')[0] || 'webm';
|
||||
form.append('audio', blob, `answer.${ext}`);
|
||||
return jsonFetch(`/v1/defense/${defenseId}/answer`, {
|
||||
method: 'POST',
|
||||
body: form,
|
||||
signal,
|
||||
});
|
||||
}
|
||||
|
||||
export async function finishDefense(
|
||||
defenseId: string,
|
||||
signal?: AbortSignal,
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
import type { VoiceDescriptor } from '@nextcraft/types';
|
||||
|
||||
/** G-12: bounded recording — auto-stop at this many seconds (see defense-session). */
|
||||
export const MAX_RECORD_SECONDS = 180;
|
||||
|
||||
/**
|
||||
* Badge label for the live voice path (UX acceptance #1: the badge always
|
||||
* tells the truth about which path is live — never claims server when mock
|
||||
* is wired). Pure so tests can pin it.
|
||||
*/
|
||||
export function voiceBadgeLabel(descriptor: VoiceDescriptor | null): string | null {
|
||||
if (!descriptor) return null;
|
||||
switch (descriptor.mode) {
|
||||
case 'server':
|
||||
return 'server STT/TTS (transcribed + spoken by the AI service)';
|
||||
case 'browser':
|
||||
return 'browser speech (client-side)';
|
||||
default:
|
||||
return 'mock (dev/test — no real transcription)';
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
/**
|
||||
* MH-2d (REQ-5-001/002, D-040/G-12): the audio-answer client path.
|
||||
* Fetch is stubbed so the FormData shape is asserted without a server.
|
||||
*/
|
||||
|
||||
const QS = `?t=${Date.now()}-${Math.random()}`;
|
||||
const { answerDefenseAudio } = await import(`../lib/engine-client.ts${QS}`);
|
||||
// voice-badge is pure (no env state) — a static import keeps typecheck happy.
|
||||
import { MAX_RECORD_SECONDS, voiceBadgeLabel } from '../lib/voice-badge';
|
||||
|
||||
interface Captured {
|
||||
url: string;
|
||||
init: RequestInit;
|
||||
}
|
||||
|
||||
test("answerDefenseAudio: posts multipart with a clean extension filename", async () => {
|
||||
const seen: Array<{ url: string; body?: FormData }> = [];
|
||||
const realFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async (input: unknown, init?: RequestInit) => {
|
||||
seen.push({ url: String(input), body: init?.body as FormData });
|
||||
return new Response(JSON.stringify({ question: "q", turn_latency: {} }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
}) as typeof fetch;
|
||||
try {
|
||||
const blob = new Blob(["\u001a\u0000"], { type: "audio/webm;codecs=opus" });
|
||||
await answerDefenseAudio("d-1", blob);
|
||||
assert.equal(seen.length, 1, "fetch was never called");
|
||||
const [capture] = seen;
|
||||
assert.ok(capture.url.endsWith("/v1/defense/d-1/answer"), capture.url);
|
||||
assert.ok(capture.body instanceof FormData, "body must be multipart FormData");
|
||||
const file = capture.body.get("audio") as File;
|
||||
assert.ok(file, "multipart must carry an 'audio' field");
|
||||
assert.equal(file.name, "answer.webm", "codec params must be stripped from the ext");
|
||||
} finally {
|
||||
globalThis.fetch = realFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("answerDefenseAudio: 413 detail surfaces verbatim (honest re-record prompt, G-12)", async () => {
|
||||
const realFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async () =>
|
||||
new Response(
|
||||
JSON.stringify({ detail: "audio exceeds 10MB — re-record a shorter answer" }),
|
||||
{ status: 413, headers: { "Content-Type": "application/json" } },
|
||||
)) as typeof fetch;
|
||||
try {
|
||||
await assert.rejects(
|
||||
() => answerDefenseAudio("d-1", new Blob(["x"], { type: "audio/wav" })),
|
||||
(err: Error & { status?: number }) =>
|
||||
err.status === 413 && err.message.includes("re-record"),
|
||||
);
|
||||
} finally {
|
||||
globalThis.fetch = realFetch;
|
||||
}
|
||||
});
|
||||
test("answerDefenseAudio: FormData requests must NOT force application/json (P0-2)", async () => {
|
||||
const realFetch = globalThis.fetch;
|
||||
let contentType: unknown = "unset";
|
||||
globalThis.fetch = (async (_input: unknown, init?: RequestInit) => {
|
||||
contentType = (init?.headers as Record<string, string> | undefined)?.["Content-Type"];
|
||||
return new Response(JSON.stringify({ question: "q", turn_latency: {} }), {
|
||||
status: 200,
|
||||
});
|
||||
}) as typeof fetch;
|
||||
try {
|
||||
await answerDefenseAudio("d-1", new Blob(["x"], { type: "audio/webm" }));
|
||||
assert.equal(contentType, undefined, "FormData must set its own multipart boundary");
|
||||
} finally {
|
||||
globalThis.fetch = realFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("voiceBadgeLabel: the badge always tells the truth about the live path", async () => {
|
||||
assert.equal(voiceBadgeLabel(null), null);
|
||||
assert.match(
|
||||
voiceBadgeLabel({ mode: "server", sr_available: true, tts_available: true, hint: "" })!,
|
||||
/server STT\/TTS/,
|
||||
);
|
||||
assert.match(
|
||||
voiceBadgeLabel({ mode: "browser", sr_available: true, tts_available: true, hint: "" })!,
|
||||
/browser speech/,
|
||||
);
|
||||
assert.match(
|
||||
voiceBadgeLabel({ mode: "mock", sr_available: false, tts_available: false, hint: "" })!,
|
||||
/mock/,
|
||||
);
|
||||
});
|
||||
|
||||
test("G-12 auto-stop bound: MAX_RECORD_SECONDS is 180 and finite", async () => {
|
||||
assert.equal(MAX_RECORD_SECONDS, 180);
|
||||
});
|
||||
@@ -0,0 +1,97 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
/**
|
||||
* MH-3d (REQ-5-004, G-10): 403 discrimination + identity client functions.
|
||||
* Fetch stubs pin the wire shapes without a server.
|
||||
*/
|
||||
|
||||
const QS = `?t=${Date.now()}-${Math.random()}`;
|
||||
|
||||
test("parseError: identity 403 with verify_cta → VerifyRequiredError (G-10)", async () => {
|
||||
const { VerifyRequiredError } = await import(`../lib/engine-client.ts${QS}`);
|
||||
const realFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
detail: {
|
||||
reason: "identity_verification_required",
|
||||
min_age: 16,
|
||||
current_status: "none",
|
||||
verify_cta: "/enroll",
|
||||
},
|
||||
}),
|
||||
{ status: 403, headers: { "Content-Type": "application/json" } },
|
||||
)) as typeof fetch;
|
||||
try {
|
||||
const { submitIdentity } = await import(`../lib/engine-client.ts${QS}`);
|
||||
await assert.rejects(
|
||||
() => submitIdentity("learner", "2000-01-01"),
|
||||
(err: unknown) => {
|
||||
assert.ok(err instanceof VerifyRequiredError, "must be VerifyRequiredError");
|
||||
const v = err as InstanceType<typeof VerifyRequiredError>;
|
||||
assert.equal(v.minAge, 16);
|
||||
assert.equal(v.verifyCta, "/enroll");
|
||||
assert.equal(v.currentStatus, "none");
|
||||
return true;
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
globalThis.fetch = realFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("parseError: plain allowlist 403 stays NotAllowlistedError (G-10)", async () => {
|
||||
const { NotAllowlistedError } = await import(`../lib/engine-client.ts${QS}`);
|
||||
const realFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
detail: "learner_id 'x' is not on the sandbox allowlist (G-5)",
|
||||
}),
|
||||
{ status: 403, headers: { "Content-Type": "application/json" } },
|
||||
)) as typeof fetch;
|
||||
try {
|
||||
const { submitIdentity } = await import(`../lib/engine-client.ts${QS}`);
|
||||
await assert.rejects(
|
||||
() => submitIdentity("learner", "2000-01-01"),
|
||||
(err: unknown) => {
|
||||
assert.ok(err instanceof NotAllowlistedError, "allowlist detail must stay itself");
|
||||
return true;
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
globalThis.fetch = realFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("identity client functions hit the right endpoints with the right bodies", async () => {
|
||||
const calls: Array<{ url: string; init?: RequestInit }> = [];
|
||||
const realFetch = globalThis.fetch;
|
||||
globalThis.fetch = (async (input: unknown, init?: RequestInit) => {
|
||||
calls.push({ url: String(input), init });
|
||||
return new Response(JSON.stringify({ submission_id: "idc-1", status: "pending", mock: true }), {
|
||||
status: 200,
|
||||
});
|
||||
}) as typeof fetch;
|
||||
try {
|
||||
const { submitIdentity, getIdentityStatus, verifyIdentity } = await import(
|
||||
`../lib/engine-client.ts${QS}`
|
||||
);
|
||||
await submitIdentity("learner", "2000-01-01");
|
||||
await getIdentityStatus("learner");
|
||||
await verifyIdentity("idc-1");
|
||||
// URLs are relative to the engine base (server-side default: localhost:8420)
|
||||
assert.ok(calls[0].url.endsWith("/v1/identity/submit"));
|
||||
assert.ok(calls[1].url.endsWith("/v1/identity/status/learner"));
|
||||
assert.ok(calls[2].url.endsWith("/v1/identity/verify/idc-1"));
|
||||
assert.deepEqual(
|
||||
JSON.parse(String(calls[0].init?.body)),
|
||||
{ learner_id: "learner", date_of_birth: "2000-01-01" },
|
||||
);
|
||||
assert.equal(calls[1].init?.method, undefined); // GET
|
||||
assert.equal(calls[2].init?.method, "POST");
|
||||
} finally {
|
||||
globalThis.fetch = realFetch;
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,35 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
/**
|
||||
* MH-4c (REQ-5-006): the session hook + Run/Test commands come from the
|
||||
* VARIANT (environment kind + test_command), not hardcoded pytest.
|
||||
*
|
||||
* The hook is a client React component — the pure command-derivation
|
||||
* logic is pinned directly; the wire field contract is pinned via the TS
|
||||
* type (required fields, a-11) in typecheck.
|
||||
*/
|
||||
|
||||
test("variant test_command: whitespace-split argv (G-15 — no shlex in browsers)", async () => {
|
||||
const fs = await import("node:fs");
|
||||
const src = fs.readFileSync("hooks/use-sandbox-session.ts", "utf8");
|
||||
assert.match(src, /const cmd = state\.variant\.test_command/, "test() derives from variant.test_command");
|
||||
assert.match(src, /\.trim\(\)\.split\(\/\\s\+\/\)/, "whitespace-only split (G-15)");
|
||||
// No shlex IMPORT in code (comments may mention the constraint).
|
||||
assert.doesNotMatch(src, /import[^\n]*shlex/, "no shlex import in the client hook");
|
||||
});
|
||||
|
||||
test("TaskVariant type carries REQUIRED environment + test_command (a-11)", async () => {
|
||||
const fs = await import("node:fs");
|
||||
const src = fs.readFileSync(
|
||||
"../../packages/types/variants.ts",
|
||||
"utf8",
|
||||
);
|
||||
assert.match(src, /environment:\s*'build' \| 'design' \| 'simulation';/);
|
||||
assert.match(src, /test_command:\s*string;/);
|
||||
// Required = no `?` optional marker on either field
|
||||
const envLine = src.split("\n").find((l: string) => l.includes("environment: 'build'"));
|
||||
assert.ok(envLine && !envLine.includes("?"), "environment must be required");
|
||||
const cmdLine = src.split("\n").find((l: string) => l.trim().startsWith("test_command"));
|
||||
assert.ok(cmdLine && !cmdLine.includes("?"), "test_command must be required");
|
||||
});
|
||||
@@ -89,6 +89,19 @@ rm -rf apps/ai-service/ai_service/data apps/ai-service/sandboxes
|
||||
Orphaned sandbox workdirs under the NEW `~/.nextcraft/sandboxes/` are
|
||||
reaped automatically on service startup (a-1 startup reaper).
|
||||
|
||||
## v0.5 note: pre-v0.5 databases (fresh start per the deploy directive)
|
||||
|
||||
The deployed box runs a FRESH `~/.nextcraft` state (v0.3.6 directive), so
|
||||
nothing below applies there. For any other box carrying a pre-v0.5
|
||||
`~/.nextcraft/data/nextcraft.db`: v0.5 adds two `variant_record` columns
|
||||
(`environment`, `test_command`). The variant store backfills them
|
||||
automatically on first open (idempotent `ALTER TABLE`; pre-v0.5 rows read
|
||||
as build-kind, test commands fall back to `pytest` at the API seam), so
|
||||
an in-place upgrade is safe — no schema migration step is required.
|
||||
Alternatively, follow the fresh-start convention and move the old DB
|
||||
aside (`mv ~/.nextcraft/data/nextcraft.db{,.pre-v05.bak}`); it is
|
||||
regenerated on boot.
|
||||
|
||||
## Optional: survive reboots (systemd)
|
||||
|
||||
`nextcraft dev -d` is self-managing but not boot-persistent. For pilot
|
||||
|
||||
@@ -45,6 +45,17 @@ export interface TaskVariant {
|
||||
statement: string;
|
||||
/** Workspace scaffold: filename → file content. */
|
||||
starter_files: Record<string, string>;
|
||||
/**
|
||||
* REQ-5-005 (D-044, a-11): REQUIRED on the wire — the server always emits
|
||||
* it. 'build' | 'design' | 'simulation': same namespace fabric, typed
|
||||
* starter contents + command policy.
|
||||
*/
|
||||
environment: 'build' | 'design' | 'simulation';
|
||||
/**
|
||||
* The variant's real test command (whitespace-split on the client — G-15:
|
||||
* templates validate quote-free at authoring; no shlex in browsers).
|
||||
*/
|
||||
test_command: string;
|
||||
/** ISO 8601 UTC generation timestamp. */
|
||||
created_at: string;
|
||||
}
|
||||
Reference in New Issue
Block a user