Compare commits
139 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 1d08218544 | |||
| 3901fac2b1 | |||
| 6c9198ca56 | |||
| aed709b956 | |||
| 36cd9fa71f | |||
| c2158af6d3 | |||
| 0592c9993d | |||
| f831839a90 | |||
| f0ad84fe74 | |||
| 1e6d035d6d | |||
| 1a88ab3199 | |||
| d35695e5fa | |||
| ac65f3ab92 | |||
| 1ba8a11189 | |||
| 4a6856711f | |||
| 9d120450da | |||
| 9c8837eb45 | |||
| dc00a4f8f5 | |||
| 06f049ba37 | |||
| c242808137 | |||
| 99da159866 | |||
| 85be77188f | |||
| 93502f9daf | |||
| 4e662f1d03 | |||
| 57c65d7a12 | |||
| cc968d93ff | |||
| 6adf0fd20c | |||
| 59b21b484d | |||
| d27706abeb | |||
| 02d324b896 | |||
| 9692712e63 | |||
| 660fb471cc | |||
| ae12f01998 | |||
| 63f9a0fc59 | |||
| 69811f953b | |||
| ba0afa757a | |||
| bbcb84ec6c | |||
| cda6ab1936 | |||
| efeaece465 | |||
| e2f3dbdc11 | |||
| d72cea1786 | |||
| 192fa6a655 | |||
| 4193552460 | |||
| 0324067fa7 | |||
| 75bde88e0d | |||
| e14e1fa3d2 | |||
| 843f16d737 | |||
| 3291914eaa | |||
| 4465e6325d | |||
| c9709e0f38 | |||
| 62bff575af | |||
| 8a43e95a4d | |||
| ee5be13c94 | |||
| 1da814e790 | |||
| cc7ec5d03f | |||
| 5c2829d9df | |||
| ec4648b37a | |||
| c5369b407b | |||
| 282f5ef150 | |||
| 3010bc4b96 | |||
| 35c4c386b5 | |||
| d8d2cebfc5 | |||
| b2a2a4023b | |||
| de431852c4 | |||
| 64e4842976 | |||
| a64733a262 | |||
| 0072689e4d | |||
| 3110be2f15 | |||
| bd5b0fee95 | |||
| 2e6d92dfa1 | |||
| e3c8cc7145 | |||
| 1e5ba6568a | |||
| f3f3746da7 | |||
| dd9bda27c9 | |||
| c80381c4df | |||
| 4238a06bca | |||
| 873d22069f | |||
| 5441e10a0e | |||
| 9fedeea767 | |||
| 6873ce6777 | |||
| c90bc7e618 | |||
| ff183b8fca | |||
| 08badd5ed6 | |||
| 9d8be6f466 | |||
| 6d6639b268 | |||
| 83246ed65b | |||
| 33f4c2d61d | |||
| 2c2f68e00e | |||
| 135ea21a61 | |||
| 1648467828 | |||
| 1a68d808fb | |||
| 2c68b44c1a | |||
| fb2db4ee97 | |||
| 485d86e117 | |||
| e798e1a6da | |||
| cd45097e52 | |||
| 1d03f0c8f5 | |||
| 12b2300f6f | |||
| e1460aed3c | |||
| b7a56d35bc | |||
| 16fb52d8f7 | |||
| a905eb8c67 | |||
| ed243594d2 | |||
| c760f9af2b | |||
| b4ae388f22 | |||
| 925ab096fb | |||
| 82ae839cd4 | |||
| b6c1bc9d54 | |||
| f281eeaf62 | |||
| 22d4fa212c | |||
| 007865a5a1 | |||
| 04bdccf189 | |||
| f3071e4b79 | |||
| 3d72fd28ec | |||
| 97893f2386 | |||
| e63b996361 | |||
| 0b34255855 | |||
| 6ab0ae2c0a | |||
| 9ff86f9cd0 | |||
| 4acffac71e | |||
| 82b9de382a | |||
| 430b4a727d | |||
| b52bef93e5 | |||
| e798c52edb | |||
| b303a41a45 | |||
| 3322c1dab9 | |||
| 0bde9cbf2e | |||
| 85028678c7 | |||
| 1a46606827 | |||
| 0fccb8d250 | |||
| 2474678da6 | |||
| 3f58fc3454 | |||
| edf586b03a | |||
| 4439486858 | |||
| b49b9189fe | |||
| f75352d0f0 | |||
| 26b4a5be60 | |||
| b6850fc036 | |||
| cfdceac17a |
+197
-25
@@ -6,6 +6,110 @@ Nextcraft is a TypeScript monorepo (pnpm workspaces + turborepo) with a Next.js
|
||||
|
||||
**v0.3 additions (Credential Engines):** real credential engines replace v0.2 mock inputs — a sandbox fabric (isolated per-learner coding environments via Linux user/mount/pid/net namespaces), a live build-telemetry pipeline (WebSocket ingest + SQLite-ordered event log), a process-trace grading engine, seeded per-learner variant task generation, and a voice-based oral defense (STT/TTS via a new provider-agnostic voice layer). **First real persistence introduced: SQLite** (`ai_service/telemetry/`, grading, variant, defense stores). Lab/Assessor/Proctor agents are re-grounded onto real telemetry/traces. **Identity/age-gating (KYC) deferred per founder directive** — no security engineer persona; secrets-hygiene checklist only.
|
||||
|
||||
**v0.4 additions (Distribution & Bootstrap CLI, founder directive D-016):** a new `apps/cli` package — the `nextcraft` bootstrap CLI (`doctor`/`bootstrap`/`verify`/`dev`) compiled to a self-contained linux x64 binary via **Node SEA** (probe-verified: Go/Rust absent, node v24.15.0 SEA-capable), installed by a repo-served one-liner script that resolves the latest Gitea release, downloads binary + sha256 sidecar, verifies, and installs to `~/.local/bin`. Every release from v0.4 onward attaches the binary + checksum as release assets (the "ongoing binaries" requirement). The CLI is a thin wrapper: all orchestration logic stays in `apps/ai-service/scripts/` (bootstrap.sh/dev.sh) — the CLI composes them via subprocess (A-202), duplicating nothing. Previously-planned v0.4 seams (real STT/TTS, KYC, design/sim envs, seq-lease) move to v0.5.
|
||||
|
||||
### v0.6 Research Conclusions (Phase 0 RESEARCH)
|
||||
|
||||
32. **D-048 Marketplace module layout mirrors D-031** — one new engine module `ai_service/marketplace/` inside apps/ai-service (no new apps, no new ports): `base.py` (pydantic contracts: `Posting`, `Employer`, `PostingTags`, `IngestPosting`), `store.py` (6th D-027 store family, D-049), `ingest.py` (PostingSource adapters, D-050), `tagging.py` (D-052), `embedding/` (protocol + mock + openai_compat + factory, D-051), `search.py` (hybrid facet+cosine ranking, A-405), `enrichment.py` (D-053), `resume.py` (D-054), with the HTTP surface in `ai_service/api/marketplace.py` (`/v1/marketplace/*` router) + `api/marketplace_admin.py` (`/v1/marketplace/admin/*`, D-057) composed via `deps.py` app.state DI (`get_marketplace_store`, `get_embedding_provider`). Boundary rules identical to identity/voice: `marketplace/` never imports `agents/` or `api/`; settings in `config.py` (backend-engineer territory); the G-18 stub MOVES from `api/identity.py` into the new router and becomes real (D-058) — `identity/` keeps only the `require_verified_adult` gate dependency it exports today.
|
||||
33. **D-049 Sixth store = one SQLite DB, five tables, DefenseStore conventions** — the marketplace store joins the shared `~/.nextcraft/data/nextcraft.db` (all 5 prior stores already share it; `Settings().db_path`). Tables: `posting` (id PK, source, external_id, employer_id, title, description, skills/seniority/category/location/remote/salary JSON+portable columns, status `draft|published|closed`, posted_at, unique constraint on (source, external_id) for idempotent upsert per A-402); `employer` (id PK, ingested raw fields, unique per source); `embedding` (posting_id PK, `vector` BLOB of struct-packed little-endian float32, `model`, `dim` — A-403/A-404 per-row provenance so lazy re-embed on provider change works); `application` (id PK, posting_id, learner_id, status `submitted`, created_at, unique (posting_id, learner_id) for A-408 idempotent apply); `employer_enrichment` (append-only: id PK, employer_id, field, value, source `llm|manual`, model, created_at — A-409). **[AMENDED by grill G-104: SIX tables — the `learner_profile` table (learner_id PK, profile JSON, confirmed_at, model) replaces the filesystem sidecar the plan draft proposed; the D-027 store family is the only persistence mechanism. Shipped six-table in P1.]** All tables mirror the probe-verified identity/store.py mechanics: WAL/synchronous=NORMAL/busy_timeout/foreign_keys pragmas, `sa_type=String/JSON` portable columns, `_validate()` hooks, Protocol + SQLite impl pair, surfaced IntegrityError, tz normalization. `VectorStore` is a protocol method-group on the marketplace store (pgvector-swap-ready, D-019/D-027 pattern).
|
||||
34. **D-050 PostingSource = protocol + two adapters, no scheduler** — `PostingSource` protocol: `fetch() -> Iterable[IngestPosting]` (pydantic, A-402 shape: source, external_id, employer + posting fields). Two v0.6 implementations: `seeded_corpus` (in-repo JSON seed ported ONCE from `packages/mock-data/jobs.ts` (20 jobs) + `employers.ts` (10 employers) per D-021 convention — shared IDs, mock `matchScore` DROPPED since D-051 computes real ones) and `admin_api` (the founder-authored path; the admin router calls ingest directly rather than an HTTP hop). Ingest upserts idempotent on (source, external_id); a `POST /v1/marketplace/admin/ingest` route triggers a source run explicitly (founder-invoked — no cron/crawler, A-401/A-402). The 3M+ external corpus lands later as a third adapter with zero store/route changes.
|
||||
35. **D-051 EmbeddingProvider = D-030 mirror with a corrected real path** — protocol `embed(texts: list[str]) -> list[EmbeddingVector]` (frozen pydantic: `values: list[float]`, `model`, `dim`) in `marketplace/embedding/base.py`. Deterministic mock: 256-dim feature-hash of normalized tokens (pure stdlib, similarity deterministic — tests never call the cloud). Real: `OpenAIEmbedProvider` on the shared httpx pool posting `{"model", "input": [texts]}` to `{AI_EMBED_BASE_URL}/embeddings` (OpenAI-compatible; `_sanitize` key redaction mirroring `openai_audio.py`). **Probe correction to A-404:** ollama-cloud does NOT expose `/v1/embeddings` (404 path-not-found; chat/completions auth-gates 401) — so `AI_EMBED_BASE_URL` must point at a different OpenAI-compatible endpoint (local ollama `:11434/v1` supports it, or any embeddings cloud); provider selection `AI_EMBED_PROVIDER=mock|openai-embed` (default mock, G-11 boot-safe fallback with loud log — exact `voice/factory.py` pattern, `AI_EMBED_MODEL`/`AI_EMBED_API_KEY` env-only). Storage per A-403: struct-packed `<{dim}f` float32 BLOB + model + dim per row; cosine in pure Python (sum of products / norm product, ≤10⁴ postings); candidate side embeds competency-profile text on demand, never stored. Confidence in A-404's parenthetical drops it to ~0.5 but the protocol design is unaffected (endpoint-agnostic by config is the D-014 pattern anyway).
|
||||
36. **D-052 Tagging = D-020 structured_completion, no new agent** — `marketplace/tagging.py` composes the EXISTING `structured_completion(provider, messages, model, schema, schema_hint)` (agents/structured.py, all 4 layers) with a `PostingTags` pydantic schema (skills: list[str] ≤10, seniority enum, category enum) + a versioned prompt constant in `prompts/` (or module-local, matching variants' pattern); no BaseAgent needed — tagging is a one-shot extraction like grading's rubric call. Mock-first for free: `settings.provider` defaults to `mock` and `structured_completion` runs against `MockLLMProvider.chat`, so tests stay cloud-free. Tags persisted as a JSON column on the posting row (D-049); tag facets feed the search filter sidebar (A-405 hard filters); re-tagging on posting update overwrites the tags column (single source of truth, no append-only need — unlike enrichment D-053, tags are derived data).
|
||||
37. **D-053 Enrichment = append-only provenance table + merge-on-read** — per A-409: `employer_enrichment` rows are append-only `(employer_id, field, value, source: llm|manual, model, created_at)`; read precedence **manual > llm > ingested-raw** computed at API-response time (a small `merge_employer()` over rows, raw fields last); every enriched field carries `{value, source}` provenance markers in the API response (A-304 honesty — LLM copy visibly marked). Trigger: a new employer's first published posting fires a one-shot LLM structured extraction (D-020, same layering as D-052, schema: description, social_links, logo_ref) — founder override = upsert a `manual`-source row, never a mutation of `llm` rows. No employer self-serve anywhere (A-407).
|
||||
38. **D-054 Resume parse = pypdf (probe-verified) → D-020 draft extraction** — pypdf 6.19.0 pip-reachable through the existing venv channel (probe 1; the ONLY new dependency, pinned `pypdf>=6.19,<7` in pyproject.toml). Pipeline: upload guard (PDF/TXT/MD only, ≤2 MB, A-410) → text extraction (`pypdf.PdfReader` page-text join for PDFs; raw decode for TXT/MD) → empty-extract on a PDF = honest 422 "text layer required" (no OCR) → cap ~100K chars → D-020 structured extraction into a `ParsedProfile` pydantic schema mapped to the competency-profile shape (competencies, skills, experience bands) → response returns as **DRAFT** (`status: "draft"`) — nothing persists to the learner profile until an explicit confirm call (REQ-6-005 review gate). Raw resume bytes never stored, never logged (A-305 pattern; the existing `_scrub_pii_validation` handler precedent applies). Tests use small generated PDFs or TXT fixtures — cloud-free.
|
||||
39. **D-055 Web migration = client shells + engine-client extension; remove 2 generateStaticParams** — board/detail/employer-profile become one static client shell each fetching at runtime per A-406 (the proven build/defend pattern): `marketplace/page.tsx` keeps its server shell but `JobFilters` receives fetch-backed data (it already owns all filter logic client-side — the facet contract survives unchanged); `jobs/[jobId]` and `employers/[employerId]` drop `generateStaticParams` (the ONLY two removals — learner catalog/learn/build/defend keep theirs, their data stays mock) and render a client shell that fetches then renders (or `notFound()` on 404). `engine-client.ts` gains a marketplace section: `listPostings(filters)`, `getPosting(id)`, `getEmployer(id)`, `searchPostings(query)` → typed against new `packages/types` marketplace wire shapes (data-engineer dual-schema sync); 403 discrimination (verify-CTA vs allowlist) already works. `packages/mock-data` jobs.ts/employers.ts go dormant for the marketplace path (still imported by employer/admin mockups — NOT deleted, A-406/A-407); the seed port (D-050) copies values once. No build-time DB reads — D-039 export build stays repo-only.
|
||||
40. **D-056 SEO = build-time metadata on static set + client JSON-LD on runtime set + ai-service sitemap/robots** — per A-411, split: (a) build-time-static pages (landing, catalog/stack detail, marketplace board, pricing) get `export const metadata` (probe 7: the static-export-compatible API the app already uses in layout.tsx/enroll) + OG tags + server-rendered JSON-LD (`Organization` on employer-adjacent static, `ItemList` on the board) rendered into the export; (b) runtime-data pages (job detail, employer profile) inject `JobPosting`/`Organization` JSON-LD + title client-side after fetch — accepted JS-crawler limitation, recorded; (c) `/sitemap.xml` + `/robots.txt` as plain FastAPI GET routes on the api app enumerating the static page list + all `published` postings — **probe 6 verified mount order: API routes beat the `StaticFiles("/")` mount**, so they win over the export even in single-port production; do NOT use Next `app/sitemap.ts` (it enumerates at build time and cannot see runtime postings under `output: 'export'`). robots.txt points at `/sitemap.xml`; sitemap URLs use the deployed origin (env-configured `AI_PUBLIC_BASE_URL`-style setting, backend-engineer territory).
|
||||
41. **D-057 Admin guard = AI_ADMIN_TOKEN header check, env-only, fail-closed** — all `/v1/marketplace/admin/*` routes (postings create + lifecycle transitions draft→published→closed, ingest trigger, enrichment manual-override, applications review) take a FastAPI dependency `require_admin` comparing `X-Admin-Token` (or `Authorization: Bearer`) against `Settings().admin_token` (`AI_ADMIN_TOKEN`, env-only per D-014, probe: no admin token exists in config today — greenfield). Unset token = admin routes 403-fail-closed (never open; a loud boot warning tells the founder to set it, G-11 doctrine). This is the A-401 operator surface; D-043 learner gate composition is untouched — admin routes do NOT pass the learner allowlist/identity gates (different principal class).
|
||||
42. **D-058 Apply route goes real, composition unchanged** — `POST /v1/marketplace/apply` (moved from api/identity.py into api/marketplace.py, D-048) keeps the exact D-043 order (G-5 allowlist → `require_verified_adult` → rate caps), then: posting must exist and be `published` (404/409 otherwise), insert an `application` row (status `submitted`, unique (posting_id, learner_id) → duplicate apply is idempotent 200-with-existing, A-408), return an honest confirmation payload `{application_id, posting_id, status: "submitted"}` (the `stub/mock` markers die). Founder reviews via `GET /v1/marketplace/admin/applications` (D-057-guarded); no notification infra. Body becomes `{posting_id}` + the learner identity from the verified record rather than a client-asserted `job_id`.
|
||||
43. **D-059 v0.6 wave order = ROADMAP order confirmed with two couplings** — P1 postings backend must ship WITH the web client shells in the same phase (a live store behind mock-reading pages proves nothing — REQ-6-001's success criterion is surfaces reading real data); P2 tagging before P3 search because facets drive the sidebar filters AND the posting text embedded in P3 should include the normalized tags; P4's two workstreams (enrichment REQ-6-004, resume REQ-6-005) are independent of each other — parallelizable within the phase; P5 SEO depends only on P1 (routes exist to enumerate) but stays last-before-final per ROADMAP since sitemap needs `published` postings and the board metadata should reflect final facets; P6 final review + ship. Tagging/enrichment/resume all route through the same D-020 call site — one prompt-review pass covers all three.
|
||||
|
||||
### v0.5 Research Conclusions (Phase 0 RESEARCH)
|
||||
|
||||
25. **D-040 Voice real path = `OpenAIAudioProvider` on the shared httpx pool (D-R01)** — implements the D-030 protocol: `transcribe()` = multipart `POST {AI_VOICE_BASE_URL}/audio/transcriptions` (`file` + `model`, response_format=json → `TranscriptSegment`), `synthesize()` = streaming `POST /audio/speech` (JSON body, raw byte stream, `input` ≤4096 chars). Carries `descriptor = VoiceDescriptor(mode="server", ...)` which defense.py:143 already prefers — mode selection drops in with zero API changes. Constructor takes the lifespan `httpx.AsyncClient` (D-017; read=300s already tolerates multi-minute clips). Factory signature becomes `voice_provider_from_settings(settings, http_client)`. Errors sanitized with key redaction (mirror `llm/openai_compat.py:_sanitize`).
|
||||
26. **D-041 Voice route fixes (D-R02)** — defense.py:189 fmt derivation must strip codec params (`"webm;codecs=opus"` → `webm`, else real STT 400s); ~10 MB audio guard (422/413) before provider call; TTS route media_type becomes format-aware (or force `response_format=wav`). Web: defense-session POSTs the recorded blob (FormData), engine-client gains the audio-answer variant.
|
||||
27. **D-042 Identity = 5th D-027 store + provider protocol (D-R03)** — `ai_service/identity/` (D-031): `IdentityProvider` protocol (submit/poll/verify), deterministic mock, SQLite store modeled on DefenseStore (WAL, FK on, portable columns). Stores **derived age_band** (`16-17`/`18+`, never raw DOB) + document **refs** (never raw docs); mock verdicts carry a `mock` marker so downstream never displays them as production-verified (A-304). PII hygiene pinned by caplog sentinel test. **Negative finding:** no age-gate UI or enrollment API exists today — the flow is built, not swapped (D-010's "visual flow" is vestigial: one mock field).
|
||||
28. **D-043 Gate composition (D-R04)** — api/-layer dependencies in order: G-5 allowlist (403 pilot guard, retained) → identity verdict (403 + verify-CTA payload) → rate caps (429). School 16+ gates variant generation + sandbox create + defense start; marketplace 18+ via reusable `require_verified_adult` demonstrated on one minimal gated route (marketplace has no backend today — the thin route proves the contract end-to-end). All middleware-layer, never in the manager.
|
||||
29. **D-044 Environment registry at the template/variant layer (D-R05)** — `TaskTemplate.environment: Literal["build","design","simulation"]` → carried through VariantRecord → API → TS `TaskVariant`; surfaces the existing-but-dead `test_command` field (Run/Test buttons stop hardcoding pytest). Manager/backend/workdir unchanged (D-024 untouched — A-307: an environment is a typing over starter files + command policy). Per-kind exec command policy at the api/ exec route (template-declared harness + generic file/nav commands, 422 on violation). Grading digest stays kind-agnostic by construction (features derive from event kinds — pinned with a digest test over a synthetic design-kind trace). Starter contents: design = SVG/HTML/schematic artifacts + validate/render harness (stack-designer c001/c002 already sanctioned); simulation = parameterized benchmark script + dataset files (stack-science, stack-operator).
|
||||
30. **D-045 Seq-ack protocol (D-R06/D-R07)** — new WS frame `{"type":"seq_ack","seq":N}` emitted per successful append from `IngestSession._append` (durable `latest_seq`; O(1); advisory — gap detection stays authoritative, G-3/G-4 unchanged). The capture agent's supervisor loop (which today discards all non-close frames) parses text frames and trims spool+pending to `seq > ack` under `_emit_lock` via atomic `Spool.rewrite` — closing the one-line replay-margin gap (`_flush_locked` pops N in-flight frames but `replay_margin()` requeues only `_last_sent`). An explicit spool bound is added (A-309's "spool cap exists" premise was factually wrong on disk). Regression test: mid-burst kill in the test_durability.py real-server harness (uvicorn + KillableProxy) — the exact scenario the P07 de-flake documented as uncovered.
|
||||
31. **D-046 v0.5 wave order (D-R08)** — P1 seq-lease (smallest, protocol-only, fixes transport before env phases add reconnecting producers), P2 voice, P3 identity, P4 environments, P5 final. Phase numbers renumbered accordingly (was 1=voice..4=seq-lease in the initial ROADMAP draft).
|
||||
|
||||
### v0.7 Research Conclusions (Phase 0 RESEARCH — Test Harness Review, NFR)
|
||||
|
||||
44. **Harness baseline (probe-verified, 2026-09-26)** — 785 tests all green: pytest 671 (64 files, ~15.4K LOC), web node:test 53 (7 files, 1,379 LOC), CLI node:test 61 (11 files, 1,284 LOC). Zero coverage tooling, zero framework timeouts (ad-hoc 20-30s polling deadlines + a 600s CLI bootstrap), zero markers despite a real three-tier split (fast-unit / real-uvicorn / real-userns), zero `filterwarnings` policy. Web tests are OUTSIDE the turbo pipeline (no `@nextcraft/web#test` task; no root test/web:test script). PLAN.md verification counts stale (488/22/61 vs actual 671/53/61). `packages/{types,mock-data,ui}` have zero tests (cross-package contracts pinned only via other apps' source-read tests).
|
||||
45. **Warning census (probe-verified)** — 1,716 warnings per pytest run: ~99% from 14 `session.query()` sites (SQLModel deprecation: `marketplace/store.py` ×12 lines 490-952, `identity/store.py` ×2 lines 172/189); 14 ResourceWarning sites (unclosed `UploadFile` multipart ×10 from `api/defense.py:214` + `api/marketplace.py:446`; unclosed sockets ×3 `scripts/sandbox-agent.py:559`; unclosed file ×1 `tests/grading/test_engine.py:428`); 2 upstream import-time deprecations (fastapi/starlette TestClient httpx, anyio BlockingPortal). First-party deprecated-API audit CLEAN: no `datetime.utcnow`, no distutils/pkg_resources, no pydantic-v1 idioms, no legacy Next APIs. `next build` warning-free; no `ignoreBuildErrors`/`ignoreDuringBuilds` flags; ruff passes clean; all 5 tsc projects pass.
|
||||
46. **Suppression census** — 81 Python `# noqa` ALL inert (RUF100: suppressed codes not in the enabled select-set — SLF001 ×57, ANN001 ×12, E402 ×5 stale-positions, rest scattered); 5 orphaned `eslint-disable` comments (no ESLint exists — turbo lint is a no-op for 5 of 6 packages; `next lint` deprecated); 27 Python `# type: ignore` (11 src, 16 tests) with no type checker; zero `@ts-ignore`/`any` in TS. Dead exports: `packages/mock-data/ai-scenarios.ts` module, `NavItem`/`ThemeConfig` types, `Spawn` type, unused `MicControl` primitive. `FastAPI(version="0.3.0")` hardcoded vs pyproject "0.2.0" vs shipped v0.6 — version drift.
|
||||
47. **Duplicated harness infrastructure** — `KillableProxy` ×2 (`tests/sandbox/test_sandbox_agent.py:275`, `tests/telemetry/test_durability.py:45`); `_free_port()` ×4; userns probe ×4; real-uvicorn serve-loop boilerplate ×5; full-app state-injection boilerplate ×6; `wirePosting()` ×2 (web). 18 one-off MockProvider scripted subclasses (some consolidation possible, low priority). Flakiness already documented in-tree: `test_sandbox_agent.py:542` "8s was flaky" — timeout-widening is the current mitigation strategy, replaced by D-054's bounds + deterministic waits where possible.
|
||||
48. **Low-value test census** — web: 21/53 are source-text regex pins (`readFileSync` + literal-string asserts, e.g. "openGraph exists in a file"), triaged per D-057 (keep iff documented regression lesson or invisible cross-boundary wiring; cut tautologies); `test_budget_constant_defined` asserts a literal defined in its own file; `tests/llm/test_mock.py` + `identity/test_identity.py::TestMockProvider` are mock-meta tests (defensible: the mock is the suite-wide engine — keep). Silent skips: `apps/cli/tests/path-invocation.test.ts:14` returns early (passes silently when binary missing); `doctor.integration.test.ts` accepts either exit code. Env leak: `tests/conftest.py:80` mutates `os.environ["AI_PROVIDER"]` without restore; `seed_verified_identity` swallows all exceptions (`conftest.py:74-75`). Web source-pin tests depend on CWD = `apps/web` (breaks from monorepo root).
|
||||
49. **D-052 tooling conclusions** — coverage: pytest-cov (Python) + c8 wrapped around existing `tsx --test` (no runner swap, no vitest); markers `fast`/`real_server`/`userns` under `--strict-markers`, unmarked defaults to fast, fast run = `-m "not real_server and not userns"`; pytest-timeout `signal` method, 60s global / 180s real-server+userns via collection hook (D-054); CLI fresh-clone E2E env-gated (`CLI_E2E=1`) into `cli:test:e2e` with a 900s budget (D-055); `filterwarnings = ["error", <narrow module+message-scoped upstream ignores>]` (D-056); mypy relaxed-first baseline (`check_untyped_defs`, `warn_unused_ignores`, no full strict) (D-058); recharts 2→3 verified by tsc + static build + existing tests (D-059); shared Python helpers in `tests/helpers/` package, fixtures stay in conftest (D-060); pyproject.toml is the single version source, `FastAPI(version=)` reads package metadata (D-061).
|
||||
50. **D-062 v0.7 wave order = dependency-ordered: wiring before composition before elimination before hardening** — P1 harness wiring (coverage/timeout/markers/turbo tasks/leak fixes) MUST precede P2 suite composition because the cut/consolidate decisions are evidence-based (P1's coverage data tells P2 what to cut); P3 warning elimination is independent of P1/P2 but ordered after them to avoid rebasing `session.exec()` conversions over moved test files; P4 lint hygiene (ESLint + noqa RUF100) after P3 so the eslint-disable resolutions happen under a warning-clean tree; P5 type+dependency hardening last (mypy adoption and recharts 3 are the riskiest changes — they run against a fully-stabilized harness, and recharts gets the fresh coverage/ESLint net as its safety net); P6 final review. ESLint adoption (D-048) rides P4 because the orphaned eslint-disable comments only become meaningful once a config exists.
|
||||
|
||||
### v0.9 Payment Engine Architecture (Phase 0 RESEARCH — Payment Engine, D-075..D-084 binding)
|
||||
|
||||
**Module:** `ai_service/billing/` (7th domain module, D-031 pattern — extend ai-service, no new apps). Boundary rule (family discipline): `billing/` never imports `agents/` or `api/`; it imports `config` + its own `base.py` only. Files: `base.py` (PaymentProvider protocol + pydantic contracts + normalized event names), `store.py` (7th D-027 SQLite store), `mock.py` (deterministic provider, Stripe-shaped ids/signatures per R-36/R-48), `stripe.py` (Stripe-compatible REST impl — form-encoded bodies, Basic auth, Idempotency-Key per R-33/R-49), `factory.py` (env-gated selection, UnknownPaymentProviderError → G-11 loud mock fallback), `catalog.py` (seeded catalog constant + idempotent seed).
|
||||
|
||||
**Provider protocol (D-075):** `create_checkout_session(spec) -> CheckoutSession`, `retrieve_session(id)`, `cancel_subscription(id, *, at_period_end)`, `update_subscription(id, new_price_ref, ...)` (resolves subscription_item id inside the provider — R-47 dual-charge guard), `verify_webhook(raw_body: bytes, signature_header: str, *, now=None) -> VerifiedEvent` (HMAC-SHA256 over `"{t}.{raw_body}"`, compare_digest, all-v1 iteration for secret rolls, ±300s replay window, ignore non-v1 schemes — R-38..R-40), `normalize_event(provider_event) -> CanonicalBillingEvent` (R-43 mapping).
|
||||
|
||||
**Store (D-080, 7 tables):** `packages` (one-time: posts_limit, price_cents, duration_days), `plans` (recurring: interval, price_cents), `purchases` (posts_limit/posts_used/expires_at snapshot + price snapshot — D-078), `subscriptions` (state machine: trialing/active/past_due/cancelled/expired — G-305: no `upgraded` status, upgrade is a plan_id transition within `active` recorded as a billing_event; expires_at, provider ref), `subscription_events` (provider event log, verbatim ids, UniqueConstraint event_id — dedupe), `checkout_sessions` (client_reference_id reconciliation), `billing_events` (full audit: provider events + manual admin actions with provenance). Same SQLite file (`Settings().db_path`), WAL pragmas, Protocol + SQLiteBillingStore pair, main.py lifespan wiring with state-injection override.
|
||||
|
||||
**Routes:** `api/billing.py` (`/v1/billing/catalog` public GET; `/v1/billing/webhook` public POST — raw-body-first HMAC verify, then parse, then idempotent process, 200-fast) + `api/billing_admin.py` (`/v1/billing/admin/*` — require_admin dep verbatim R-51: checkout on behalf of employer (D-076), entitlement view, manual grant/revoke, event log query) + mock-only confirm route (D-082 as amended by G-301: mounted only when provider=mock AND composed with require_admin — no unauthenticated completion route on a payment surface). Entitlement enforcement composes into `create_posting` AFTER require_admin: billing store's `consume_posting(employer_id)` → domain error → route translates to `HTTPException(402, detail={reason, posts_used, posts_limit, upgrade_cta})` (dict-detail precedent = require_verified_age).
|
||||
|
||||
**Settings (R-53):** `payment_provider: str = "mock"`, `payment_api_key: SecretStr`, `webhook_secret: str = ""` (unset → per-boot random → webhooks fail closed — correct for a payment surface). Env: `AI_PAYMENT_PROVIDER`, `AI_PAYMENT_API_KEY`, `AI_WEBHOOK_SECRET`.
|
||||
|
||||
**Web (REQ-9-002):** pricing page → server shell (metadata retained, A-411/pins) + `PricingClient` component fetching `/v1/billing/catalog` (A-406 board-client pattern, AbortController + honest loading/error/empty states); `engine-client.ts` gains `listBillingCatalog()` + `PaymentRequiredError` (402 dict-detail discrimination); types in `packages/types/billing.ts` dual-schema-pinned in pins.test.ts. Admin entitlement view: moderation surface gains per-employer entitlement state (client fetch, same pattern).
|
||||
|
||||
**Money (D-081):** integer cents end-to-end; pydantic `int ge=0`; TS `number` documented as cents; currency `usd` fixed.
|
||||
|
||||
### v0.10 B2B Employer Network Architecture (Phase 0 RESEARCH — B2B Employer Network, D-086..D-097 binding)
|
||||
|
||||
**Module:** `ai_service/employers/` (8th domain module, D-031 pattern — extend ai-service, no new apps). Boundary rule (family discipline): `employers/` never imports `agents/` or `api/`; it imports `config` + its own files only. Files: `base.py` (pydantic contracts: EmployerAccount, EmployerClaim, session/token response shapes — dual-schema-pinned against packages/types/employer.ts, D-093; PostingSummary reused from marketplace.ts, never duplicated), `store.py` (8th D-027 SQLite store, 4 tables, D-095), `passwords.py` (stdlib scrypt KDF core — the billing/webhooks.py crypto-core precedent; D-086 params N=2^14/r=8/p=1/dklen=32 as module constants pinned by test, self-describing `scrypt$16384$8$1$<salt_b64>$<hash_b64>` rows, `hmac.compare_digest` verify; per-password `secrets.token_bytes(16)` salt; zero new deps), `claim_evidence.py` (stdlib registrable-domain normalization — urlsplit hostname + suffix match; the advisory match signal, D-089). HTTP surface in `api/employers.py` (`/v1/employers/*` — session-gated) + `api/employers_admin.py` (`/v1/employers/admin/*` — require_admin imported from marketplace_admin, the billing_admin.py:27 precedent, D-094). No service.py — routes compose the store via `deps.py` app.state DI (`get_employer_store`, the "routes never import stores directly" territory rule).
|
||||
|
||||
**Store (D-087/D-089/D-095, 4 tables on the shared `Settings().db_path`):** `employer_accounts` (id PK, email UNIQUE indexed, password_hash self-describing scrypt string, employer_id UNIQUE nullable-until-claim-approval, created_at), `employer_sessions` (id PK, token_hash UNIQUE — stores ONLY sha256(token), never the token; account_id indexed, created_at, expires_at = login + SESSION_TTL_DAYS), `employer_claims` (id, account_id, employer_id indexed, status pending|approved|rejected, evidence JSON `{account_email_domain, entity_website_domain, match}`, created_at, decided_by, decided_at), `employer_audit_events` (auto id PK, account_id/employer_id nullable indexed, kind/name, detail JSON, operator, created_at — the BillingEventRow audit shape). Mechanics mirror billing/store.py exactly (WAL / synchronous=NORMAL / busy_timeout / foreign_keys pragmas, `sa_type=String/JSON` portable columns, `_validate()`/`_normalize()` hooks, `_as_utc` tz re-labeling, Protocol + SQLiteEmployerStore pair, surfaced IntegrityError, check-first unique patterns — the insert_application precedent; G-304-style single-statement writes where concurrency matters). The `employer_` prefix is mandatory — bare `employer` is marketplace's EmployerRow table. `SQLModel.metadata.create_all` is idempotent: zero migration, the 4 tables join existing DBs at next boot (the proven billing-store compat).
|
||||
|
||||
**Session flow (D-087/D-090):** register (email+password, UNVERIFIED per D-097 — no learner gates, no email verification; scrypt hash via passwords.py) → login mints `secrets.token_urlsafe(32)`; the row stores `sha256(token)` + expires_at = now + 7-day module constant (lazy TTL at read — G-306, no timer); the client persists the opaque token in sessionStorage `nc-employer-session` (the G-308 adminToken wrapper pattern) and attaches `Authorization: Bearer <token>` on every fetch (CSRF-immune by construction — no ambient credential; D-039 static export has no Next middleware). The `require_employer_session` dependency (the require_admin extract→verify→return shape): **401** on missing/expired/unknown token (credential failure), **403** reserved for cross-entity authorization; returns session + account + claimed employer_id. logout DELETEs the row + audit event (revoke = lookup miss → 401); multiple concurrent sessions per account allowed. Token namespaces cannot collide: admin routes compare against the env token, employer routes hash-lookup.
|
||||
|
||||
**Auth throttling (D-088, G-5 in-memory pattern):** (a) per-EMAIL failed-login window — Settings knob, default 5 failures / rolling 15 min → 429, successful login clears the email's window; (b) global per-min cap on login+register attempts (the `_check_global_create_rate` deque) → 429. Keyed by SUBMITTED EMAIL, never client IP (behind HAProxy the proxy owns request.client.host). Both copy the proven deque/monotonic-window mechanics (identity _SubmitRateLimiter + sandboxes global cap); test fixtures reset the module state.
|
||||
|
||||
**Claim flow (D-089):** authenticated `POST /v1/employers/claims {employer_id}` → claim row status=pending with computed evidence (stdlib domain normalization; every seeded employer website is an example.com placeholder → match=false corpus-wide — the signal is advisory, A-304) → operator reviews in the admin queue → approve sets employer_accounts.employer_id (UNIQUE — one account : one entity) + decided_by/decided_at + audit event; reject frees the entity for a new claim. One PENDING claim per entity + one APPROVED owner per entity enforced check-first in the store. Auto-approval deferred: without email verification the match is forgeable — the operator decision activates.
|
||||
|
||||
**Entitlement composition (D-091) — one shared helper, two principal gates:**
|
||||
|
||||
```
|
||||
admin arm: require_admin ──▶ consume_posting_entitlement(store, employer_id, admin-CTA) ──▶ insert
|
||||
employer arm: require_employer_session ──▶ claim check (403) ──▶ same helper(session employer, employer-CTA) ──▶ insert
|
||||
|
||||
helper: store-missing → 503 fail-closed
|
||||
NoActivePackage → 402 {reason: no_active_package, posts_used: 0, posts_limit: 0, upgrade_cta}
|
||||
PackageExhausted → 402 {reason: posting_limit_reached, posts_used: 0, posts_limit: 0, upgrade_cta}
|
||||
(posts_used/posts_limit hardcoded 0 in BOTH arms — the v0.9 wire contract, byte-identical;
|
||||
UpgradeRequiredError consumes reason + upgrade_cta unchanged; gate runs BEFORE insert)
|
||||
```
|
||||
|
||||
`consume_posting` (G-304 single conditional UPDATE) is untouched; the helper lives in api/ beside both callers (no billing/→api/ boundary violation).
|
||||
|
||||
**Employer checkout (D-091, transposes billing_admin.create_checkout):** `POST /v1/employers/billing/checkout {item_id}` — session-gated; unclaimed → 403 + claim-CTA; buyer = the session's claimed entity (supersedes D-076 operator mediation); catalog snapshot + unique-pending placeholder + CheckoutSessionSpec metadata employer_id=claimed entity, operator_ref→account provenance; success/cancel URLs from public_base_url (G-310); employer mock-complete arm mounted only under provider=mock, employer-session-gated, completing one's OWN session (403 cross-session), minting the signed event through the REAL webhook path (D-082 transposed). The v0.9 provider protocol/webhooks/store grants serve employer checkouts verbatim — only principal derivation changes.
|
||||
|
||||
**Web (D-092/D-093, engine-client employer section — no new client module):** `engine-client.ts` gains register/login/logout/me, claim, own-postings CRUD/publish/close, own-checkout, own-entitlement + a `SessionRequiredError` (401 discrimination — parseError today has no 401 arm; honest logged-out state, never a retry loop) + the `nc-employer-session` sessionStorage wrapper (the adminToken pattern; logout clears storage + calls revoke; AbortController everywhere). Conversion scope: postings page (own-postings lifecycle), overview page (real active-postings count + per-posting applicant counts from the applications table; talent-match/placement metrics stay mock-labeled), employer profile (claimed entity via the existing public GET), new /employer/login + /employer/register (greenfield — no login page exists in any route group); **talent search + candidate profile stay the v0.1 mock** (no candidate backend — D-092). Wire types in `packages/types/employer.ts` dual-schema-pinned in pins.test.ts (the billing.ts pin block template, D-093). Pricing CTAs convert from contact-path to employer-login path (supersedes D-079).
|
||||
|
||||
**Admin network overview (D-094):** the moderation page (app/(admin)/admin/moderation/page.tsx — where the v0.9 EmployerEntitlementPanel landed) gains sibling sections below it: employer accounts list (email, claimed entity, entitlement state), claim queue (evidence + domain-match signal, approve/reject), audit — fed by operator routes in api/employers_admin.py (require_admin + audit events): GET /v1/employers/admin/accounts, GET …/claims, POST …/claims/{id}/approve|reject, GET …/audit; web transport via the existing adminToken() G-308 wrapper + honest 403 locked states (the entitlement panel's locked-state pattern).
|
||||
|
||||
**Settings/wiring (D-095/R-60):** throttle knobs as Settings ints (the identity_submits_per_min shape) + .env.example entries beside AI_ADMIN_TOKEN; session TTL is a module constant (no env). main.py: employer_store lifespan wiring (the billing_store getattr-guard template, ~line 182-190) + close() in the finally block; employers + employers_admin routers mount beside the billing block (inline-import pattern); CORS `allow_headers` gains "Authorization" (main.py:341) with a tests/api/test_cors.py preflight pin — the E87 lesson applied before it ships this time. scrypt probe-verified in the shipped venv (3.11.2): ~50 ms/hash at D-086 params, default maxmem admits the 16 MiB working set.
|
||||
|
||||
### v0.8 Distribution & Release Architecture (Phase 0 RESEARCH — Ship Anywhere, D-063..D-072 binding)
|
||||
|
||||
**Runner fleet topology (CT 113, 10.0.0.113):** three org-level gitea-runner containers (official image `docker.io/gitea/runner:3` — the Docker Hub `gitea/act_runner` repo is deprecated, renamed `gitea/runner`) in **sibling-container mode**: each mounts the host `/var/run/docker.sock`, has its OWN `/data` volume holding the `.runner` registration file (identity + credentials; shared state = re-registration), registers once against `https://git.coreci.dev` (Gitea 1.26.1, probe-verified) with the founder-provided org-level registration token (chat-only, reset after fleet green). Hybrid labels via `GITEA_RUNNER_LABELS`: `node` → `docker://node:24-bookworm`, `python` → `docker://python:3.11-bookworm`, `release` → `docker://node:24-bookworm`, `ubuntu-latest` → `docker://docker.gitea.com/runner-images:ubuntu-latest` (label format `name[:schema[:args]]`; a job's `runs-on` matches label names, the `docker://` image selects the job container). Per-runner `runner.capacity: 1` (default; also caps per-job docker subnet consumption) → fleet concurrency 3. Cache: each runner's built-in cache server must be pinned to a job-reachable `cache.host` + fixed published port (dockerized runners otherwise create per-job networks the cache address is unreachable in — `actions/cache` fails ETIMEDOUT); 32 GB disk guarded by `cache.repo_size_limit`, `health_check.min_free_disk_space_mb`, and a host-side `docker system prune` cron. Workspace-path alignment inside/outside the runner container is NOT needed for our jobs (no `container.bind_workdir`; `docker build` TARS the context over the API) — documented as the known sibling-mode gotcha if a future workflow bind-mounts.
|
||||
|
||||
**Workflow lanes (`.gitea/workflows/`):** push lane on `on: push` (all branches) — node lane (label `node`): corepack pnpm 12.3.4 → `pnpm install --frozen-lockfile` → eslint + tsc via `pnpm --filter` over the 5 TS workspaces (never wholesale `turbo lint`, which pulls the venv-dependent ai-service task) → `pnpm web:test` + `pnpm cli:test`; python lane (label `python`): `scripts/bootstrap.sh` venv → `scripts/lint.sh` (ruff) → `scripts/typecheck.sh` (mypy) → `FAST=1 scripts/test.sh` (fast subset `-m "not real_server and not userns"` per D-063 — real_server/userns NEVER in job containers; docker's default seccomp blocks nested `unshare --user` there anyway); ubuntu-latest lane: workflow_dispatch hello-world only (fleet verification, REQ-8-001). Tag lane on `on: push: tags: ['v*']` is the EXCLUSIVE owner of release assets + registry push (D-064): builds from the tag checkout (jobs never have `dist/`), stamps `build:binary` with `gitea.ref_name`, attaches binaries + debs + checksums via the same REST API scripts/release-assets.sh uses (`GET /releases/tags/{tag}` → `POST /releases/{id}/assets`), never `actions/upload-artifact`. Secrets: `REGISTRY_TOKEN` + `GITEA_TOKEN` as org/repo Actions secrets (self-serve `PUT /api/v1/repos/…/actions/secrets/{name}` first, founder UI fallback); the automatic per-job `GITEA_TOKEN` CANNOT publish packages (upstream-confirmed) — the PAT secret is the only viable registry credential. pnpm/pip caches via `actions/cache@v4` (runner 3.x serves cache v2 + self-patches stock actions ≥4.2/4.4) once cache-host pinning is live; deterministic fallback = volume-mounted stores via `container.options` + `valid_volumes`.
|
||||
|
||||
**Image layers (multi-stage Dockerfile, `git.coreci.dev/coreci/nextcraft`, amd64 only per D-069):** deps/build stages — web export stage (`node:24-bookworm`, `NEXT_PUBLIC_AI_SERVICE_URL=self pnpm build` → `apps/web/out`, ~16 MB; the URL is build-time-only, inlined by engine-base-url.ts) + venv stage (`python:3.11-bookworm`, `pip install` prod deps from pyproject — no dev extras); runtime stage `python:3.11-slim-bookworm` (~400–500 MB total with the venv ~130 MB + web export + the D-066 CLI-binary embed at `/usr/local/bin/nextcraft`, ~117 MB): non-root `USER nextcraft`, `AI_WEB_STATIC_DIR=/opt/nextcraft/web` (single-port StaticFiles mount, D-039), `NC_CONTAINER=1` entrypoint marker, `HEALTHCHECK CMD` urllib `GET /health` (slim has no curl), `VOLUME /home/nextcraft/.nextcraft` (state root `~/.nextcraft` = DB `data/nextcraft.db` + `sandboxes/`, env-overridable via `AI_DB_PATH`/`AI_SANDBOX_DIR` with `~` expansion — config.py:39/77/86-93), `CMD uvicorn ai_service.main:app --host 0.0.0.0 --port 8420`. Zero AI_* env needed at build; all real-provider keys stay optional at runtime (G-11 fallbacks). Unprivileged by default; the sandbox fabric honestly degrades inside docker (below).
|
||||
|
||||
**Deb/binary asset matrix (every release, tag-workflow-built):** `nextcraft-linux-x64` + `.sha256` (existing pipeline, build-binary.mjs SEA), `nextcraft-linux-arm64` + `.sha256` (cross-build: esbuild bundle is arch-agnostic; the ONLY arch-specific step is `copyFileSync(process.execPath, out)` — replaced by a copy from the official `node-v<ver>-linux-arm64.tar.xz` base, probe-verified 200/30 MB; postject adds an ELF note, pure file surgery, never runs the target — QEMU-free per D-69, optional binfmt `--version` smoke if binfmt is registered on CT 113), `nextcraft_<ver>_amd64.deb` / `nextcraft_<ver>_arm64.deb` + `.sha256` (`dpkg-deb --root-owner-group -Zxz`, never executes payload → cross-arch safe; control fields: Package nextcraft, Version = tag minus `v` (policy-clean), Architecture per build, empty Depends, `Recommends: nodejs (>= 18), python3 (>= 3.11), git, util-linux` — soft deps, D-067), binary at `/usr/bin/nextcraft`, postinst = advisory `nextcraft doctor` (never fails dpkg) + next-steps, NO systemd unit (`deploy --service` owns unit generation) + NO proxy fragments. install.sh arch-gate becomes arch-resolve (`x86_64`→x64, `aarch64|arm64`→arm64; sha256 + version integrity gates unchanged). Image tags per D-065: semver `<tag>` + `latest` on newest release, no per-push tags.
|
||||
|
||||
**Channel upgrade semantics (D-071):** docker = `docker pull git.coreci.dev/coreci/nextcraft:<tag>` + recreate the container (state persists in the named `nc-state` volume); `deploy`/`bootstrap`/`dev` refuse in-container (D-066 — container check before the repo-root checks, pointer to the docker paths; `doctor`/`verify`/`guide` remain allowed, `verify` gains a container-mode branch checking the mounted web dir + `/health` instead of a repo clone). deb = `dpkg -i` the new deb + `nextcraft deploy --service` (idempotent v0.4.6 rail). binary = re-run install.sh + `nextcraft deploy --service`. doctor detects the channel (NC_CONTAINER/`/.dockerenv` for docker; `dpkg -s`/`/usr/bin/nextcraft` for deb) and describes its upgrade path; `nextcraft guide` serves offline-first embedded man-page text channel-appropriately (D-072 — TS constants bundled by esbuild into the SEA binary, zero runtime fetches).
|
||||
|
||||
**Sandbox degrade in containers (D-068):** capability probe promoted verbatim from tests/helpers/userns.py into `ai_service/sandbox/capability.py` (`userns_available()` = `shutil.which("unshare")` guard + real `subprocess.run(["unshare","--user","--map-root-user","true"])`) — the truth source at boot + per-create; container markers (`/.dockerenv`, `NC_CONTAINER=1`, cgroup string) only flavor remediation text. Degrade: boot notice (the main.py:404 mount-less / main.py:202 admin-token G-11 warning pattern), learner sandbox create rejects honestly with 503 + remediation pointer (a new `except SandboxUnavailableError` arm beside the existing `PoolFullError→503` in api/sandboxes.py:157-160 — today it would 500), doctor surfaces the degraded capability via a real userns attempt (upgrading the `which unshare` check, doctor.ts:138-150); chat/agents/grading untouched. Supersedes A-101's cwd-jail fallback for deployed schools (never implemented — no code removal needed).
|
||||
|
||||
**Pre-commit rails (D-070):** ONE root `.pre-commit-config.yaml`, local hooks only (ruff on Python paths; eslint/tsc via workspace-scoped invocations), no network at hook run; missing toolchain = loud failure, never silent skip; CI is the authoritative backstop.
|
||||
|
||||
### Confirmed Technology Stack (v0.2)
|
||||
|
||||
| Technology | Version | Purpose |
|
||||
@@ -26,6 +130,8 @@ Nextcraft is a TypeScript monorepo (pnpm workspaces + turborepo) with a Next.js
|
||||
| pydantic | 2.13.x | Request/response models, structured outputs |
|
||||
| pydantic-settings | 2.15.x | Settings + env-file loading (replaces python-dotenv) |
|
||||
| httpx | 0.28.x | Async LLM HTTP client (ollama-cloud + local providers) |
|
||||
| sqlmodel / sqlalchemy | 0.0.24 / 2.x | Typed SQLite persistence for the v0.3 engine stores (D-027) |
|
||||
| python-multipart | 0.0.x | Multipart audio upload for the defense answer route (REQ-3-006) |
|
||||
| sse-starlette | 3.4.x | SSE framing, ping keep-alive |
|
||||
| pytest | 9.x | Test runner |
|
||||
| pytest-asyncio | 1.4.x | Async tests (auto mode) |
|
||||
@@ -45,6 +151,16 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
7. **D-022 Monorepo integration** — zero-dependency shim `package.json` in apps/ai-service + `ai#*` turbo passthrough tasks (`cache:false, outputs:[]`) + root `ai:dev`/`ai:test` scripts + idempotent venv bootstrap.
|
||||
8. **D-023 Testing** — pytest-asyncio auto mode; TestClient `client.stream()` for SSE; httpx MockTransport for byte-exact provider parser tests; scripted mock provider incl. failure modes. Tests never call the cloud.
|
||||
|
||||
### v0.4 Architecture Decisions (from Research — Distribution & Bootstrap CLI)
|
||||
|
||||
18. **D-033 Binary toolchain = Node SEA (probe-verified)** — Go and Rust are absent from this box; node v24.15.0 ships SEA support (`--experimental-sea-config`, postject-free on linux via `cp node nextcraft && node sea-config` … blob injection with the system `dd`/`npx postject` if needed). CLI source lives in `apps/cli` (TypeScript, compiled to a single CJS bundle by esbuild, then SEA-injected into a copy of the node binary → `nextcraft-linux-x64`). Fallback if SEA breaks: python3 `zipapp` (3.11.2 available). No new toolchain deps beyond dev-scoped esbuild.
|
||||
19. **D-034 CLI = thin wrapper, orchestration stays in scripts/** — `nextcraft` composes `apps/ai-service/scripts/bootstrap.sh` and `scripts/dev.sh` equivalents via `spawn` with inherited stdio and timeout guards (A-202/A-209). doctor/bootstrap/verify implement only *checking* logic (prereqs, env template, health) — never re-implement installs. This keeps one source of truth for bootstrap semantics.
|
||||
20. **D-035 Install path = repo raw `install.sh` + Gitea latest-release API** — the one-liner `curl -fsSL <forge>/coreci/nextcraft/raw/main/scripts/install.sh | bash` resolves `GET /api/v1/repos/coreci/nextcraft/releases/latest`, downloads the `nextcraft-linux-x64` + `nextcraft-linux-x64.sha256` assets, verifies sha256 (`shasum -a 256`), installs to `~/.local/bin` (PATH hint), and degrades to printed source-bootstrap instructions when no binary asset exists or the platform mismatches (A-203/A-204/A-206).
|
||||
21. **D-036 Ongoing binaries = ship-workflow asset step** — the release pipeline (v0.3's `ShipWorkflow.createRelease` equivalent, executed as the ship step's asset stage) builds the binary + checksum and attaches both to every Gitea release from v0.4 onward (A-205). Token resolution stays `.env*`-only (D-006/D-014); binaries are linux x64 only for v0.4 (macOS arm64 deferred — unverifiable on this box).
|
||||
22. **D-037 CLI package layout** — `apps/cli` is a pnpm workspace package (`@nextcraft/cli`): `src/` (entry, commands/, checks/, lib/), `scripts/build-binary.mjs` (esbuild bundle → SEA inject), unit tests runnable via `pnpm --filter @nextcraft/cli test` (node:test, no new test framework). Root `package.json` gains `cli:*` passthrough scripts mirroring the `ai:*` pattern (D-022).
|
||||
23. **D-038 Network mode (v0.3.5)** — dev binds 0.0.0.0 (`AI_HOST`, default 0.0.0.0, revert via 127.0.0.1); CORS + WS-origin gates read `AI_CORS_ORIGINS` (default `*` — any origin, safe only because credentials are never enabled; explicit comma list restricts); the web client derives the API base URL from the browser hostname at runtime (`engine-base-url.ts`: `NEXT_PUBLIC_AI_SERVICE_URL` override → `http://${window.location.hostname}:8420` → `localhost` server-side). Hotfix also fixes: SEA direct-run detection (`require("node:sea").isSea()` — argv shape differs by invocation style), installer honesty gate (silent `--version` = hard fail), bootstrap venv recovery (poisoned partial `.venv` removal + distro-specific `apt install python3.XX-venv` hint), and doctor venv-capability probe with bootstrap preflight.
|
||||
24. **D-039 Single-port deploy + unattended dev (v0.3.6)** — only :8420 is reachable behind HAProxy, so the web app ships as a **static export** (`output: 'export'`, `NEXT_PUBLIC_AI_SERVICE_URL=self` → relative same-origin fetches) served by the ai-service itself (`AI_WEB_STATIC_DIR` StaticFiles mount, default off; `nextcraft dev` auto-wires it when `apps/web/out` exists). Daemon surface: `dev -d` (detached, `~/.nextcraft/run/<clone-hash>/dev.{pid,log}`), `stop`, `log [-n N|-f]`. Durable state (DB `~/.nextcraft/data/`, sandbox workdirs `~/.nextcraft/sandboxes/`) moves **out of the repo** (founder directive; `expanduser` validator makes `AI_DB_PATH=~/...` env overrides work). API routes beat the static mount; unknown paths serve the export's 404.html.
|
||||
|
||||
### v0.3 Architecture Decisions (from Research — Credential Engines)
|
||||
|
||||
9. **D-024 Sandbox isolation = Linux namespaces via `unshare`** — per-learner sandbox runs as a subprocess entered into fresh user+mount+pid+network namespaces (`unshare --user --map-root-user --mount --pid --fork --net`). Probe-verified on this box: in-namespace uid=0, **network fully isolated** (0 interfaces), learner writes land in a per-sandbox directory; proc-remount not permitted here but not required. Chosen because no container runtime (docker/podman/bwrap/firejail) exists on the box and there is no sudo. A `SandboxBackend` protocol abstracts the spawner so a future containerd/runc backend can replace namespace-spawning without touching callers.
|
||||
@@ -65,14 +181,14 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `ai_service/main.py` | FastAPI app factory, lifespan (httpx client pool, provider factory), CORS (localhost only), /health | App entry | config, llm, agents, api |
|
||||
| `ai_service/main.py` | FastAPI app factory, lifespan (httpx client pool, provider factory, SandboxManager + reaper loop, SQLite engine stores on app.state), CORS (localhost only, incl. PUT for file writes), /health; v0.3.6: optional StaticFiles mount of the exported web app when `AI_WEB_STATIC_DIR` is set (single-port deploy, D-039) | App entry | config, llm, agents, api, engines |
|
||||
| `ai_service/config.py` | pydantic-settings Settings (env_prefix="AI_", env_file, SecretStr key) | Configuration only | None |
|
||||
| `ai_service/api/` | Endpoints: chat.py (POST /v1/chat/stream, SSE), lab.py, assessment.py, proctor.py, mentor.py; deps.py (DI) | Composes agents + sessions; never imported by llm/ or agents/ | agents, llm |
|
||||
| `ai_service/api/` | Endpoints: chat.py (POST /v1/chat/stream, SSE), lab.py, assessment.py (POST /v1/assessment/evaluate v0.2 + POST /v1/assessment/grade v0.3), proctor.py, mentor.py, sandboxes.py (lifecycle + files/exec routes, G-5 abuse gates), telemetry.py (WS ingest + trace/gaps reads), variants.py (seeded per-learner variants), defense.py (defense loop, REQ-3-006); deps.py (DI) | Composes agents + sessions + engines; never imported by llm/ or agents/ | agents, llm, sandbox, telemetry, grading, variants, voice |
|
||||
| `ai_service/llm/` | types.py (Message; ChatDelta/ChoiceDelta removed in P3 — no consumers), base.py (LLMProvider protocol), openai_compat.py (ollama-cloud + local), mock.py (deterministic), factory.py | Never imports agents/ or api/ | config |
|
||||
| `ai_service/agents/` | base.py (BaseAgent ABC), registry.py, session.py (SessionStore), structured.py (JSON defense), coach/tutor/lab/assessor/proctor/mentor.py | Never imports api/ | llm, prompts, corpus |
|
||||
| `ai_service/agents/` | base.py (BaseAgent ABC), registry.py, session.py (SessionStore), structured.py (JSON defense), coach/tutor/lab/assessor/proctor/mentor.py + examiner.py (seventh agent, v0.3) | Never imports api/ | llm, prompts, corpus, telemetry |
|
||||
| `ai_service/prompts/` | Per-agent system prompt constants + render_context functions (str.format_map) | Data only | None |
|
||||
| `ai_service/corpus/` | Mock engine inputs: learner_context.py, telemetry.py (Lab/Proctor scenarios), artifacts.py (pre-baked artifacts, rubrics, transcripts) | Pydantic-typed; aligned with TS packages/mock-data by convention | None |
|
||||
| `scripts/` | bootstrap.sh (venv + pip install idempotent), dev.sh (exports keys from .ciagent/.env.secrets → uvicorn), test.sh (pytest), lint.sh (ruff check, G-3) | Dev entry points | pyproject.toml |
|
||||
| `ai_service/corpus/` | Mock engine inputs: learner_context.py, telemetry.py (Lab/Proctor scenarios), artifacts.py (pre-baked artifacts, rubrics, transcripts). Since v0.3 P6 these are DORMANT, test-only fixtures (dormant-header noted) — the live learner path uses real engine inputs | Pydantic-typed; aligned with TS packages/mock-data by convention | None |
|
||||
| `scripts/` | bootstrap.sh (venv + pip install idempotent), dev.sh (exports keys from .ciagent/.env.secrets → uvicorn), test.sh (pytest), lint.sh (ruff check, v0.2 G-3) | Dev entry points | pyproject.toml |
|
||||
| `tests/` | conftest.py (mock provider, settings override, TestClient), health, llm (MockTransport parser), agents (framework + per-agent), api (SSE stream tests) | Mock provider only — no cloud | all |
|
||||
|
||||
**Module boundary rules:** `llm/` never imports `agents/` or `api/`; `agents/` never imports `api/`; `api/` composes both via DI. `corpus/` is the only home of mock engine data. Prompts are code — versioned and reviewed in git.
|
||||
@@ -82,35 +198,51 @@ Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `ai_service/sandbox/` | `backend.py` (SandboxBackend protocol), `unshare_backend.py` (userns/mount/pid/net spawner, D-024), `manager.py` (lifecycle: create/list/snapshot/destroy + concurrency guard D-032), `workdir.py` (per-sandbox fs layout) | Never imports api/ or agents/; spawns subprocesses only | config |
|
||||
| `ai_service/telemetry/` | `models.py` (TelemetryEvent, TraceSpan), `store.py` (TraceStore protocol + SQLite impl D-027), `ingest.py` (WebSocket /v1/telemetry/ingest, seq gap detection D-026) | Persistence; never imports agents/ | config |
|
||||
| `ai_service/grading/` | `features.py` (deterministic trace digest D-028), `engine.py` (rubric scoring orchestration), `store.py` (GradeStore) | LLM only via digest; never sees raw trace | llm, telemetry, prompts |
|
||||
| `ai_service/variants/` | `templates.py` (task template library), `generator.py` (seeded LLM instantiation D-029), `store.py` (VariantStore) | LLM via structured output | llm, grading |
|
||||
| `ai_service/voice/` | `base.py` (VoiceProvider protocol D-030), `openai_audio.py` (STT/TTS vs compatible endpoint), `browser.py` (native SR/TTS fallback descriptor), `mock.py` (deterministic) | Never imports agents/ or api/ | config |
|
||||
| `ai_service/telemetry/` | `models.py` (TelemetryEvent, TraceSpan), `store.py` (TraceStore protocol + SQLite impl D-027), `ingest.py` (WebSocket /v1/telemetry/ingest, seq gap detection D-026; v0.5 REQ-5-007/D-045: emits advisory `seq_ack` frames — highest-contiguous received seq per successful append; capture agent trims its spool to the ack, closing the replay-margin gap) | Persistence; never imports agents/ | config |
|
||||
| `ai_service/grading/` | `features.py` (deterministic trace digest D-028 — kind-agnostic by construction, pinned over design/sim traces in v0.5), `engine.py` (rubric scoring orchestration), `store.py` (GradeStore) | LLM only via digest; never sees raw trace | llm, telemetry, prompts |
|
||||
| `ai_service/variants/` | `templates.py` (task template library; v0.5 REQ-5-005/D-044: `environment: Literal[build,design,simulation]` registry + per-kind starter files + harness/test commands, G-15 shlex-roundtrip validation), `generator.py` (seeded LLM instantiation D-029), `store.py` (VariantStore; idempotent `_ensure_v05_columns` backfill for pre-v0.5 DBs) | LLM via structured output | llm, grading |
|
||||
| `ai_service/voice/` | `base.py` (VoiceProvider protocol D-030), `browser.py` (native SR/TTS fallback descriptor), `mock.py` (deterministic), `openai_audio.py` (v0.5 REQ-5-001, D-040: real server STT/TTS against OpenAI-compatible `/audio/transcriptions` + `/audio/speech` on the shared httpx pool — CUT-1/G-7 seam CLOSED), `factory.py` (provider selection by `AI_VOICE_PROVIDER`, G-11 boot-safe fallback to mock), `defense_store.py` (DefenseStore: transcripts + integrity signals, D-027) | Never imports agents/ or api/ | config |
|
||||
| `ai_service/identity/` | **v0.5 NEW (REQ-5-003/004, D-042/43):** `base.py` (IdentityProvider protocol: submit/poll/verify), `mock.py` (deterministic approve-on-policy mock; verdicts carry a `mock` marker, A-304), `store.py` (5th D-027 store: identity_record table — derived `age_band`, document **refs**, PII never stored raw); age-gate dependencies `require_verified_age`/`require_verified_adult` (gate composition D-043: G-5 allowlist → identity verdict → rate caps; mounted on variants/sandbox-create/defense-start + the G-18 marketplace stub), exposed via `api/identity.py` (`/v1/identity/*`) | Never imports agents/; api/ composes it via DI | config |
|
||||
| `ai_service/marketplace/` | **v0.6 NEW (REQ-6-001..006, D-048..D-056):** `base.py` (pydantic contracts: Posting/Employer/PostingTags/IngestPosting), `store.py` (6th D-027 store, SIX tables per G-104: posting/employer/embedding/application/employer_enrichment/learner_profile), `ingest.py` (PostingSource protocol + seeded corpus, idempotent upserts G-102), `tagging.py` (D-020 one-shot tag extraction, overwrite semantics D-052), `embedding/` (EmbeddingProvider protocol + deterministic feature-hash mock + OpenAI-compatible real path + G-11 factory, D-051), `search.py` (hybrid facet+cosine ranking, embed-on-publish, skill_breakdown), `enrichment.py` (append-only provenance rows + merge-on-read manual>llm>raw, D-053), `resume.py` (pypdf text extraction + bounds + D-020 draft extraction, D-054); HTTP surface in `api/marketplace.py` (public + apply + resume), `api/marketplace_admin.py` (fail-closed D-057), `api/marketplace_shells.py` (G-101 catch-all), `api/sitemap.py` (dynamic sitemap/robots, D-056) | Never imports agents/ or api/; api/ composes via DI | config |
|
||||
| `ai_service/agents/examiner.py` | Seventh agent: oral defense examiner; streams over existing SSE, consumes process traces + emits integrity signals | reuses BaseAgent (D-018) | llm, prompts, telemetry |
|
||||
| `ai_service/data/*.db` | SQLite databases (telemetry/grades/variants/defenses) | gitignored | — |
|
||||
| `ai_service/data/*.db` | SQLite databases (telemetry/grades/variants/defenses) — **v0.3.6: default moved to `~/.nextcraft/data/nextcraft.db` (state out of the repo; AI_DB_PATH overrides, ~ expanded)** | outside repo (home) | — |
|
||||
| `scripts/sandbox-agent.py` | Tiny in-namespace capture process shipped into the sandbox; streams telemetry to ingest | standalone | stdlib only |
|
||||
|
||||
**Boundary additions:** `sandbox/`, `telemetry/`, `grading/`, `variants/`, `voice/` are engine modules — they never import `api/` (which composes them via DI) and never import `agents/` (agents call engines through narrow interfaces, not vice versa).
|
||||
|
||||
### apps/cli — Nextcraft Bootstrap CLI (v0.4 NEW)
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `src/index.ts` | Entry: arg parsing (no deps beyond node stdlib at runtime), command dispatch, `--help`/`--version`, exit-code contract (0 ok / 1 failure / 2 usage) | CLI surface only | commands/ |
|
||||
| `src/commands/` | `doctor.ts` (prereq checks + actionable errors), `bootstrap.ts` (pnpm install + scripts/bootstrap.sh wrapper + env template copy + key validation), `verify.ts` (health: venv imports, ports, env, build readiness), `dev.ts` (thin passthrough to scripts/dev.sh) | Compose checks/ + lib/; spawn scripts — never re-implement them | checks/, lib/ |
|
||||
| `src/checks/` | Pure check functions: `check-command.ts` (binary-on-PATH + version compare), `check-env.ts` (template diff, required/optional key classification) | Pure logic, unit-testable, no fs side effects at import | None |
|
||||
| `src/lib/` | `spawn.ts` (subprocess with timeout + inherited stdio), `log.ts` (✓/✗/warn output formatter) | Shared utilities | None |
|
||||
| `scripts/build-binary.mjs` | esbuild → CJS bundle → Node SEA injection → `dist/nextcraft-linux-x64` + sha256 sidecar | Build-time only | esbuild (dev dep) |
|
||||
| `scripts/install.sh` | The one-liner install script served from repo raw: Gitea latest-release resolve → download + checksum verify → ~/.local/bin; source-bootstrap fallback | Standalone POSIX sh | forge API |
|
||||
| `tests/` | node:test unit tests: command dispatch, check logic, env template diff, install-script shellcheck-style assertions | Fixtures only — never mutate repo state | src/ |
|
||||
|
||||
**Boundary rules:** the CLI never imports from `apps/web`, `packages/*`, or `ai_service` Python modules — it orchestrates them exclusively via subprocess/filesystem. Runtime deps: node stdlib only (no runtime npm deps; esbuild is dev-only). The binary embeds the bundle; `scripts/bootstrap.sh` remains the single source of bootstrap truth (D-034).
|
||||
|
||||
### apps/web — Next.js Application
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `app/(learner)/` | Learner surface route group: landing, catalog, competency stack, dashboard, byte viewer, sandbox mockup, assessment mockup | Learner-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/(learner)/` | Learner surface route group: landing, catalog, competency stack, dashboard, byte viewer, build surface (`/build/[competencyId]` — real in-browser build), defense surface (`/defend/[competencyId]` — live oral defense + grading) | Learner-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/(marketplace)/` | Marketplace surface route group: job board, job detail, employer profile, search/filter, pricing | Marketplace-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/(employer)/` | Employer dashboard route group: overview, talent search, candidate profile, posting management | Employer-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/(admin)/` | Admin surface route group: overview, learner management, competency graph viewer, moderation | Admin-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/layout.tsx` | Root layout: theme provider, navigation shell, responsive container | All routes | packages/ui |
|
||||
| `components/` | Surface-specific components (learner/, marketplace/, employer/, admin/) plus shared chrome (navigation-shell, header/footer, role-switcher, theme-provider, breadcrumbs, dark-mode-toggle) | App-level components | packages/ui |
|
||||
| `hooks/` | use-chat-stream.ts — SSE client hook: fetch + ReadableStream, byte buffering + frame reassembly, idempotent AbortController cleanup | Client components only | ai-service SSE |
|
||||
| `lib/` | sse.ts (shared SSE frame parser — CRLF normalization + `: ping` immunity, G-1), breadcrumbs.ts, format.ts | Pure utilities | None |
|
||||
| `components/` | Surface-specific components (learner/, marketplace/, employer/, admin/) plus shared chrome (navigation-shell, header/footer, role-switcher, theme-provider, breadcrumbs, dark-mode-toggle); v0.3 learner: build-surface, sandbox-terminal (read-only exec output), defense-session | App-level components | packages/ui |
|
||||
| `hooks/` | use-chat-stream.ts — SSE client hook: fetch + ReadableStream, byte buffering + frame reassembly, idempotent AbortController cleanup; use-sandbox-session.ts (v0.3) — sandbox lifecycle for the build session: create on task open, destroy on unmount, mid-start failure cleanup, 503/403/429 honest surfaces | Client components only | ai-service SSE / engine API |
|
||||
| `lib/` | sse.ts (shared SSE frame parser — CRLF normalization + `: ping` immunity, v0.2 G-1), breadcrumbs.ts, format.ts, engine-base-url.ts (v0.3.5: runtime API base — env override → browser hostname → localhost), engine-client.ts (v0.3: typed fetch client for /v1/sandboxes, files/exec, variants, grade, defense, traces) | Pure utilities | None |
|
||||
|
||||
### packages/ui — Shared Component Library
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `tokens/` | Design tokens as TS constants: colors, spacing, radii, shadows, breakpoints (mirrored as Tailwind v4 `@theme` tokens in apps/web globals.css) | Foundation layer — no dependencies | None |
|
||||
| `primitives/` | Button, Input, Card, Badge, Avatar — each with a Storybook story | Atomic UI components | tokens, packages/types |
|
||||
| `primitives/` | Button, Input, Card, Badge, Avatar (v0.1) + TerminalFrame, TelemetryStatus, MicControl, GradeBadge, TranscriptViewer (v0.3 build/defense surfaces) — each with a Storybook story | Atomic UI components | tokens, packages/types |
|
||||
|
||||
Composite/layout/theme components (navigation shell, tables, chat panels, graph viewer, theme provider) live in `apps/web/components/` as app-level components, not in packages/ui.
|
||||
|
||||
@@ -134,11 +266,42 @@ Composite/layout/theme components (navigation shell, tables, chat panels, graph
|
||||
| `marketplace.ts` | Job, Employer, Candidate, JobPosting, TalentMatch, SearchFilter | Marketplace types | None |
|
||||
| `user.ts` | Learner, Admin, EmployerUser, AgeGroup, Role | User types | None |
|
||||
| `ui.ts` | Component props, theme config, breakpoint definitions | UI types | None |
|
||||
| `telemetry.ts` | TelemetryEvent/ExecResult wire shapes for the live build surface (v0.3) | Engine types | None |
|
||||
| `variants.ts` | Variant/TaskTemplate shapes for per-learner task statements (v0.3) | Engine types | None |
|
||||
| `grading.ts` | GradeRecord/RubricScore shapes for live grading display (v0.3) | Engine types | None |
|
||||
| `defense.ts` | DefenseSession/transcript/integrity-signal shapes for the defense surface (v0.3) | Engine types | None |
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
### v0.3 credential flow (current)
|
||||
|
||||
```
|
||||
[learner build surface /build/*] [learner defense surface /defend/*]
|
||||
file CRUD + Run/Test (HTTP) mic MediaRecorder / typed + TTS playback
|
||||
│ │
|
||||
▼ ▼
|
||||
[api/sandboxes files/exec] ──exec──▶ [namespace sandbox] [api/defense start/answer/finish]
|
||||
│ │ capture agent │
|
||||
│ ▼ (WS telemetry) ▼
|
||||
│ [api/telemetry ingest] [DefenseStore (SQLite)]
|
||||
│ │ SQLite │ transcript + integrity signals
|
||||
│ ▼ │
|
||||
│ [TraceStore] ────▶ [GradingEngine: digest (grading/features)
|
||||
│ │ + rubric LLM (D-028)] ──▶ [GradeStore]
|
||||
│ ▼ ▼
|
||||
└──▶ Lab agent (live digest) Assessor (grade output) / Proctor (integrity)
|
||||
Examiner agent (SSE) ◀── defense sessions
|
||||
variants: [api/variants] ◀── [VariantStore (seeded, D-029)] ── per-learner task statements
|
||||
```
|
||||
|
||||
- Lab consumes the live trace digest; Assessor consumes grading-engine output; Proctor consumes telemetry + defense integrity signals (REQ-3-007) — no mock fallback in the learner path (v0.2 corpus scenarios are dormant test-only fixtures).
|
||||
- The learner's path is: variant task → in-sandbox build (telemetry streams to SQLite) → grade My Work (rubric scores from the real trace) → oral defense → verdict.
|
||||
- Flooded/gapped traces are terminal: ingest closes 1008 and marks INCOMPLETE_FLOODED (G-3); the grader returns UNGRADABLE_TRACE_INCOMPLETE (G-4) — no credential from an incomplete trace.
|
||||
|
||||
### v0.2 chat flow (complete, still live)
|
||||
|
||||
```
|
||||
[packages/mock-data + packages/types] [ai_service/corpus]
|
||||
│ (TS, web surfaces) │ (Python, agent inputs)
|
||||
@@ -153,21 +316,27 @@ Composite/layout/theme components (navigation shell, tables, chat panels, graph
|
||||
(https://ollama.com/v1)
|
||||
```
|
||||
|
||||
- Web surfaces remain server-component-first; client components (chat, filters, graph viewer, dark mode toggle) fetch directly from ai-service over SSE (A-002: no Next.js API-route proxy in v0.2).
|
||||
- The LLM provider layer is a dumb pipe — OpenAI-compatible chunks pass through byte-identical; envelope logic (meta/done/error) lives only in the API layer (D-016).
|
||||
- Lab/Assessor/Proctor read mock scenarios from `ai_service/corpus/` — real engines are v0.3+.
|
||||
- Web surfaces remain server-component-first; client components (chat, filters, graph viewer, dark mode toggle) fetch directly from ai-service over SSE (A-002: no Next.js API-route proxy).
|
||||
- The LLM provider layer is a dumb pipe — OpenAI-compatible chunks pass through byte-identically; envelope logic (meta/done/error) lives only in the API layer (D-016).
|
||||
- The v0.2 corpus scenarios (`ai_service/corpus/`) are retained as dormant, test-only fixtures (dormant-header noted); they are no longer inputs to the live learner path.
|
||||
- All automated tests use the deterministic mock provider; the cloud is for manual probes only.
|
||||
|
||||
---
|
||||
|
||||
## Build Order (v0.3)
|
||||
## Build Order (v0.4)
|
||||
|
||||
1. **Bootstrap CLI core** — apps/cli package: doctor checks (node/pnpm/python3/git/unshare), bootstrap wrapper (pnpm install + scripts/bootstrap.sh + .env template + key validation), verify health check, dev passthrough; unit tests
|
||||
2. **Binary build + release pipeline** — esbuild bundle → Node SEA binary (`nextcraft-linux-x64`) + sha256 sidecar; install.sh one-liner (Gitea latest-release resolve + checksum verify + PATH install); release-asset upload wired into the ship flow (ongoing binaries from v0.4 onward)
|
||||
3. **Install docs + fresh-clone E2E** — README quickstart (one-liner → doctor → bootstrap → dev), CLI reference, fresh-clone end-to-end test proving a clean clone reaches a running stack
|
||||
|
||||
## Build Order (v0.3 — complete)
|
||||
|
||||
1. **Sandbox fabric** — SandboxBackend protocol + unshare namespace spawner + lifecycle manager (create/list/snapshot/destroy) + concurrency guard + per-sandbox workdir; isolation + resource-limit probes
|
||||
2. **Live build telemetry** — TelemetryEvent models + SQLite TraceStore + WebSocket ingest endpoint + seq gap detection + in-sandbox capture agent
|
||||
3. **Process-trace grading engine** — deterministic feature/digest computation + rubric scoring via LLM structured output + GradeStore; calibrated against v0.2 mock corpora
|
||||
4. **Variant task generation** — template library + seeded LLM instantiation + VariantStore + difficulty normalization anchors
|
||||
5. **Oral / voice defense** — VoiceProvider protocol + STT/TTS + mock + browser fallback + Examiner agent + transcript/integrity-signal capture
|
||||
6. **Agent re-grounding + learner surface integration** — Lab/Assessor/Proctor consume real telemetry/grades/defense signals; learner sandbox mockup → real in-browser xterm.js build/run; assessment mockup → live defense + live grading
|
||||
6. **Agent re-grounding + learner surface integration** — Lab/Assessor/Proctor consume real telemetry/grades/defense signals; learner sandbox mockup → real in-browser build/run (Run/Test buttons executing in a namespace sandbox, read-only exec-output panel — no interactive shell, CUT-2/G-8); assessment mockup → live defense + live grading
|
||||
|
||||
---
|
||||
|
||||
@@ -184,16 +353,19 @@ The v0.1 build order (monorepo → types → mock data → tokens → primitives
|
||||
|
||||
---
|
||||
|
||||
## Future Architecture (Post-v0.3, for reference)
|
||||
## Future Architecture (Post-v0.4, for reference)
|
||||
|
||||
v0.3 delivers the real credential engines; later milestones fill in the remaining platform:
|
||||
v0.4 delivers distribution (CLI + binary releases); later milestones fill in the remaining platform:
|
||||
|
||||
- **In-memory sessions → PostgreSQL + Drizzle/SQLModel** — SessionStore + v0.3 TraceStore/GradeStore/VariantStore/DefenseStore protocols swap SQLite→Postgres with no API changes
|
||||
- **userns subprocess sandboxes → containerd/runc backend** — D-024 `SandboxBackend` protocol swap; same lifecycle API
|
||||
- **Coding-IDE sandbox → design tool + simulation environments** — REQ-F-021 full scope (v0.4)
|
||||
- **Coding-IDE sandbox → design tool + simulation environments** — REQ-F-021 full scope (v0.5)
|
||||
- **Mock provider → per-agent model routing** — provider factory already selects by config; per-agent `AI_<AGENT>_MODEL` overrides
|
||||
- **No auth → real KYC + sessions** — **deferred per founder directive**; REQ-F-017 identity/age-gating lands post-v0.3 (v0.4+). Age-gating remains the v0.1 visual flow mockup
|
||||
- **No auth → real KYC + sessions** — **deferred per founder directive; moved to v0.5 with D-016**; REQ-F-017 identity/age-gating lands post-v0.4. Age-gating remains the v0.1 visual flow mockup
|
||||
- **Mock voice → real server STT/TTS (openai-audio provider)** — CUT-1/G-7 seam moved to v0.5 per D-016; VoiceProvider protocol is the drop-in point
|
||||
- **linux x64 binary → macOS arm64 + auto-update** — D-036 defers non-linux targets (unverifiable on this box); `nextcraft upgrade` (self-replace from latest release) is the natural v0.5+ follow-up
|
||||
- **Exec-telemetry seq-lease / replay-margin fix** — the P6-lesson one-line ACK gap moves to v0.5 per D-016
|
||||
- **No search → Semantic vector search (pgvector)** — Filter UI replaced with vector similarity search
|
||||
- **No payments → Payment processing** — Pricing page replaced with real subscription/payment flows
|
||||
|
||||
The monorepo structure (apps/web + apps/ai-service + packages/*) accommodates further apps without restructuring.
|
||||
The monorepo structure (apps/web + apps/ai-service + apps/cli + packages/*) accommodates further apps without restructuring.
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
{
|
||||
"phase": 1,
|
||||
"stage": "complete",
|
||||
"milestone": "v0.3",
|
||||
"phase": 2,
|
||||
"stage": "ship_pending",
|
||||
"milestone": "v0.10",
|
||||
"milestone_name": "b2b-employer-network",
|
||||
"phase_role": "execution",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-09-11T18:43:22Z"
|
||||
}
|
||||
"updated_at": "2026-09-30T18:40:00Z"
|
||||
}
|
||||
@@ -1,3 +1,96 @@
|
||||
# Nextcraft v0.9 — GRILL.md (Adversarial Review Verdict)
|
||||
|
||||
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-BINDINGS (REVISE → record G-301..G-310, then SHIP) · **Confidence:** 0.85 · **Date:** 2026-09-29
|
||||
|
||||
## Summary
|
||||
|
||||
The v0.9 Payment Engine plan (5 requirement phases + final, D-074 locked) is feasible, correctly scoped, and hermetically testable — the operator-mediated checkout model is coherent end-to-end and entitlement needs no employer auth. But red-teaming against the working tree (not the plan's description of it) found 10 binding defects: a mock-complete route mounted on every default-config production box (D-082's "never on a real deployment" rationale is **false** — mock IS the default), webhook handlers that would trust forgeable event payloads (leaked secret = free entitlements), a check-then-increment race in consume_posting, event-id-only dedupe defeated by Stripe's dual invoice events, a phantom `upgraded` state with no transition rows, an un-pinned ingest bypass that would 402 the seeded corpus, a checkout route naming contradiction between REQUIREMENTS/ROADMAP and binding D-076, an admin panel requiring a browser token entry that exists nowhere in apps/web, and a stripe checkout that cannot create sessions (success_url never populated — Happy Path 5's "zero code changes" was false). Two plan citations proved wrong in-tree (the "sitemap env-gated routes" precedent does not exist; admin surface has zero admin-token plumbing). Two Gs bite inside P1's Task 1-1-02 — recording them now prevents store-schema retrofit churn after P1 ships. Nothing re-architects: every G amends an existing task, must-have, or requirement sentence; the phase/wave structure is untouched.
|
||||
|
||||
## Per-Axis Findings
|
||||
|
||||
| # | Axis | Verdict | Reason |
|
||||
|---|------|---------|--------|
|
||||
| 1 | Feasibility | PASS | Operator-mediated checkout coherent end-to-end; entitlement composes after require_admin with employer_id as body field; 6-phase shape matches the proven v0.6 cadence |
|
||||
| 2 | Scope creep | CONCERN | REQ-9-003/ROADMAP name a public POST /v1/billing/checkout contradicting D-076 (G-307); admin panel hides a browser-token requirement (G-308); "stripe switch = zero code changes" false without success/cancel URLs (G-310) |
|
||||
| 3 | Security | CONCERN | Mock-complete route unauthenticated on default deployments (G-301); webhook state resolution unspecified — payload trust = forged entitlements (G-302) |
|
||||
| 4 | Correctness | CONCERN | consume_posting check-then-increment race (G-304); dual invoice events defeat event-id dedupe (G-303); `upgraded` state hole (G-305); out-of-order events against missing targets + grace timing unspecified (G-306) |
|
||||
| 5 | Testability | PASS | Hermetic by construction: MockTransport byte-contracts, now-injection, direct-handler-call mock-complete — provided G-306 keeps grace lazy |
|
||||
| 6 | Backward compat | PASS | Fixture strategy workable (~9 test sites grant unlimited to legacy seeded employers); pricing metadata pin survives conversion; ingest bypass unpinned → G-309 |
|
||||
| 7 | Cost/benefit | PASS | D-074 stands: payments unblocks employer self-serve; no new deps (hand-rolled REST, no Stripe SDK); challenge declined |
|
||||
| 8 | Plan-vs-reality | PASS (with corrections) | All cited patterns verified in-tree; two rationale citations false (G-301/G-308); suite count 674/625-fast drift noted |
|
||||
|
||||
## Binding Decisions (G-301..G-310 — applied to PLAN/REQUIREMENTS/ROADMAP before execute)
|
||||
|
||||
**G-301 — mock-complete route: admin-gate it; D-082's "never on a real deployment" rationale is false.** The default provider is mock (D-075) and the pilot IS a real deployment — `POST /v1/billing/mock/complete` mounted "only when provider=mock" is mounted on every default-config production box. An unauthenticated completion route on a payment surface is a hole no matter how unguessable `cs_mock_…` ids are. Decision: keep the provider=mock conditional mount AND compose `require_admin` on the route — the operator completing a session after out-of-band payment is exactly the honest pilot flow D-076 describes. Amends PLAN Task 3-2-03, MH-3c; D-082's rationale corrected (ship-notes record: the sitemap "env-gated routes" precedent does not exist — this is the first factory-conditional mount in-tree). Tests: no-token → 403 under mock; route 404s under stripe; E2E green with admin-gated complete. Confidence 0.85.
|
||||
|
||||
**G-302 — webhook events are untrusted ID references; state resolves from local rows.** If checkout.completed reads session status/amount from the event payload, a leaked AI_WEBHOOK_SECRET = free entitlements (forge status=complete, payment_status=paid for any session id). Decision: handlers treat event payloads as ID references only — the session resolves from the local checkout_sessions row (creatable only via the admin route); unknown session id → recorded as an `unmatched` billing_event, NO entitlement, 200. Purchase price/name from the stored snapshot (D-078), never the event body; renewal amounts from the local plan row. Amends Task 3-2-02. Tests: signed checkout.completed with no local row → 200, zero purchases, one unmatched event; inflated payload amount → purchase price = snapshot. Confidence 0.85.
|
||||
|
||||
**G-303 — grants are idempotent per session, not per event id.** R-43/D-083 accept invoice.paid AND invoice.payment_succeeded and Stripe's async path fires distinct completed events — different event ids for the same logical completion sail past event-id dedupe and double-grant. Decision: `purchases.checkout_session_id` gets a UniqueConstraint; subscription upsert keys on provider subscription id; entitlement creation is exactly-once per session regardless of event count. Amends Task 1-1-02 (schema), Task 3-2-02; extends MH-3b. Test: two signed checkout.completed events, different event ids, same session → exactly one purchase row (IntegrityError → created=False), one subscription. Confidence 0.85.
|
||||
|
||||
**G-304 — consume_posting is a single conditional UPDATE, not check-then-increment.** insert_application survives concurrency via UniqueConstraint; posts_used has no such backstop — two concurrent create_posting calls both read posts_used=9 and both increment. Decision: consume_posting = one statement `UPDATE purchases SET posts_used = posts_used + 1 WHERE id = <newest non-expired with remaining> AND posts_used < posts_limit` (rowcount-checked; 0 rows → PackageExhausted/NoActivePackage). Amends Task 1-1-02. Test: 1 post remaining, two concurrent consumes on separate connections (WAL) → exactly one succeeds. Confidence 0.80.
|
||||
|
||||
**G-305 — `upgraded` is not a status state.** D-083's "(+ upgraded)" has no transition-matrix rows and REQ-9-005's canonical states don't include it. Decision: upgrade = plan_id transition within `active` (provider update per R-47 with items[0][id] — pinned by MH-3d), recorded as a billing_event with provenance; `upgraded` removed from the status enum — D-083's "(+ upgraded)" reinterpreted as the auditable upgrade ACTION, not a state. Amends Task 5-1-01, MH-5a. Tests: upgrade on active → status stays active, plan_id changes, billing_event exists; stored-status set contains no `upgraded`. Confidence 0.75.
|
||||
|
||||
**G-306 — out-of-order tolerance includes missing targets; grace expiry is lazy, not a timer.** (a) invoice.payment_failed/invoice.paid can arrive before checkout.completed creates the local subscription — handlers never 500 on unknown targets: record as unmatched (G-302 marker), return 200; recovery is the grant or admin manual complement. (b) past_due → expired at 7 days is computed lazily from past_due_at + 7d at read/entitlement time — no background timer, keeping tests hermetic. Amends Task 3-2-02 (tolerance rule), Task 5-1-01 (grace semantics); risks 3-4. Tests: signed invoice.paid for unknown subscription → 200 recorded no crash; entitlement check with injected now at past_due_at+6d → granted, +8d → 402/expired; no sleeps. Confidence 0.80.
|
||||
|
||||
**G-307 — align REQ-9-003/ROADMAP route naming to D-076.** REQUIREMENTS REQ-9-003 and ROADMAP phase-3 say `POST /v1/billing/checkout`; binding D-076, PLAN, and ARCHITECTURE say `/v1/billing/admin/checkout` behind require_admin. Decision: amend the wording at this gate (the requirement text must not name a public un-gated checkout route that will not exist). Test: route-inventory pin — no /v1/billing/checkout route exists; /v1/billing/admin/checkout 403s without token. Confidence 0.90.
|
||||
|
||||
**G-308 — define the browser-side admin-token entry; the admin panel is otherwise unreachable.** The admin surface is mock-data driven; no X-Admin-Token exists in apps/web. REQ-9-004's "admin surface shows per-employer entitlement state" hides a new requirement: an operator token entry in the browser — Task 4-2-01's A-406 citation is wrong (A-406 is for public routes). Decision: a small token input in the entitlement panel persisted to sessionStorage, sent by a new adminFetch wrapper in engine-client (X-Admin-Token header); 403 renders an honest locked state with the token prompt; the token is never logged. Amends Task 4-2-01. Tests: stubbed 403 → locked-state render; fetch stub asserts the header; no admin-token string in non-test web source beyond the wrapper. Confidence 0.75.
|
||||
|
||||
**G-309 — pin the ingest bypass: seed ingestion is never entitlement-gated.** Enforcement composes into the admin create_posting route ONLY, but the seeded-corpus ingest upserts ~20 postings directly through the store and must never hit the 402 path. Decision: explicit rule + test — run_ingest (and any future adapter) bypasses consume_posting by design; enforcement exists only in the admin route. Amends Task 4-1-01 (test list), risk 5 companion. Test: seeded ingest on a store with zero purchases → 20 postings upserted, zero 402s, zero consumption rows. Confidence 0.90.
|
||||
|
||||
**G-310 — default success/cancel URLs for the stripe checkout, or delete the "env-only switch" claim.** R-34: success_url is required for hosted checkout; the admin checkout body is {item_id, employer_id} and no task populates it — the stripe path cannot create sessions as planned. Decision: the checkout route defaults success_url = {public_base_url}/marketplace/pricing?checkout=success and cancel_url = …?checkout=cancelled (public_base_url exists at config.py:170); loud G-11 boot warning when stripe is selected and public_base_url is unset. Amends Task 3-2-01 (defaults) + Task 3-1-01 (byte-contract asserts both fields). Tests: MockTransport byte-contract shows success_url/cancel_url present; factory-selection test shows the loud warning when stripe + empty origin. Confidence 0.80.
|
||||
|
||||
---
|
||||
|
||||
# Nextcraft v0.6 — GRILL.md (Adversarial Review Verdict)
|
||||
|
||||
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-BINDINGS · **Confidence:** 0.72 · **Date:** 2026-09-21
|
||||
|
||||
## Summary
|
||||
|
||||
The v0.6 Marketplace Engine plan (46 tasks, 6 phases) is well-researched and tightly scoped, but its central P1 web-migration mechanism — removing `generateStaticParams` from the two marketplace dynamic routes — is **provably build-breaking**: Next 15.5.25 (installed `node_modules/next/dist/build/index.js:1244`) hard-throws error E87 for any dynamic route without `generateStaticParams` under `output: 'export'`, and even a function returning `[]` throws (`length > 0` check). The claimed "proven build/defend pattern" precedent does not exist as described — build/defend routes KEPT their `generateStaticParams`. Four quieter gaps (ingest status semantics, pagination-vs-client-filters, learner-profile sidecar persistence, sitemap origin) each ship a broken happy path or an audit flag. All fixed via binding decisions G-101..G-110 + 8 plan edits applied by the orchestrator before execute.
|
||||
|
||||
## Per-Axis Findings
|
||||
|
||||
| Axis | Verdict | Rationale |
|
||||
|------|---------|-----------|
|
||||
| Feasibility | PASS | Personas/territories match proven patterns (voice factory, identity store, D-020 layer); 46 tasks comparable to shipped v0.5. |
|
||||
| Scope discipline | PASS | Pricing/employer-auth/scheduler/crawler all absent (MH-M6, A-407/A-412, D-050); only borderline add is admin posting-edit, defensible as the re-tag trigger within A-401. |
|
||||
| Vertical-slice integrity | FAIL → fixed | P1 backend+shells coupling is right, but the shell mechanism threw E87 at build and runtime-created postings would get no exported page — fixed by G-101 (placeholder param + FastAPI catch-all shell routes). |
|
||||
| Testability | AT-RISK → fixed | MH-1e pinned a build that could not emit; MH-3d's "matchScore exists nowhere" overclaimed (mock-data retains it for employer/admin mockups) — scoped by G-107/G-110. |
|
||||
| Security posture | PASS | Fail-closed admin, D-043 gate order, upload bounds, PII sentinel tests verified against existing code; hardening added by G-105/G-108. |
|
||||
| Performance/capacity | PASS | Brute-force cosine ≤10⁴ honest; SQLite WAL/busy_timeout verified in the identity store; sitemap fine to 50k URLs. |
|
||||
| Operational readiness | AT-RISK → fixed | D-039 deploy fabric intact (mount order verified), pypdf via bootstrap channel sound; founder workflow breaks at ingest-status (G-102) and sitemap-origin (G-105) — now defined. |
|
||||
| Dependency risk | PASS | pypdf verified as the only new dep; JSON-LD/sitemap hand-rolled; embedding mock stdlib-only. A-404's stale "ollama-cloud exposes /embeddings" parenthetical annotated as superseded by D-051. |
|
||||
| Adoption/UX | AT-RISK → fixed | Happy-path steps 2-4 contradicted until ingest status defined (G-102); verify-CTA/404/409 states well-specified; sitemap URL-resolvability pinned (G-109). |
|
||||
|
||||
## Binding Decisions (applied to PLAN.md before execute)
|
||||
|
||||
- **G-101** — Do NOT remove `generateStaticParams` (E87 build break under `output:'export'`). Keep it returning one placeholder param per route; export one client shell; add FastAPI catch-all `GET /marketplace/jobs/{id}` + `/employers/{id}` (trailing-slash variants) serving the shell — same routes-beat-the-mount mechanism as sitemap. [Vertical-slice integrity]
|
||||
- **G-102** — Ingest-status contract: seed-port postings enter `published` with original `postedAt`; admin-authored start `draft`. [Adoption/UX]
|
||||
- **G-103** — Board fetch = paged fetch-all (≤10⁴ documented); API returns `total_published`; "Showing X of Y" over the fetched set with honest truncation notice. [Performance]
|
||||
- **G-104** — Confirmed learner profiles persist in the marketplace store as a 6th table `learner_profile` (learner_id PK, profile JSON, confirmed_at, model) — NOT a filesystem sidecar. D-049 amended five→six tables. [Scope discipline]
|
||||
- **G-105** — Unset `AI_PUBLIC_BASE_URL`: loud G-11 boot warning; sitemap serves empty static set + warning (never malformed/relative URLs); robots omits the Sitemap directive; boot never crashes. [Security]
|
||||
- **G-106** — Match scores appear only in search context; cards without an active query show no score; per-learner profile-driven card scores post-v0.6; REQ-6-003 "scores on job cards" satisfied conditionally — recorded. [Adoption/UX]
|
||||
- **G-107** — "Mock matchScore absent" scoped to marketplace-path API payloads + marketplace surfaces; grep excludes packages/mock-data and employer/admin route groups. [Testability]
|
||||
- **G-108** — `admin_token` as `SecretStr`; compare via `secrets.compare_digest`. [Security]
|
||||
- **G-109** — Every sitemap job/employer URL must serve 200 on the single-port app (G-101 shells); board ItemList JSON-LD enumerates static structure only — never runtime postings. [Operability]
|
||||
- **G-110** — P1 web tests pin the E87 regression class itself (shells emitted; catch-all serves arbitrary ids; `notFound()` at non-placeholder id via full page load); the honest-501 stub pin at tests/identity/test_identity.py:444 rewritten onto the marketplace router in the same phase. [Testability]
|
||||
|
||||
## Plan Edits Applied
|
||||
|
||||
1. Task 1-3-04 + MH-1e rewritten per G-101 (placeholder shells + catch-all routes; E87 test class; stub-pin rewrite)
|
||||
2. Task 1-2-01 + MH-1f ingest-status contract (G-102)
|
||||
3. Task 1-3-02 paged fetch-all + total_published (G-103)
|
||||
4. Task 4-B-03 learner_profile table replaces the JSON sidecar (G-104; D-049 → six tables)
|
||||
5. Task 5-2-01 + MH-5a/5b unset-origin behavior + URL-resolvability (G-105/G-109)
|
||||
6. Task 3-3-01 + MH-3d matchScore scope + no-query-no-score (G-106/G-107)
|
||||
7. Task 1-2-03 SecretStr + compare_digest (G-108)
|
||||
8. PROJECT.md A-404 stale parenthetical annotated as superseded by D-051 probe
|
||||
|
||||
---
|
||||
|
||||
# Nextcraft v0.3 — GRILL.md (Adversarial Review Verdict)
|
||||
|
||||
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-CHANGES · **Confidence:** 0.72
|
||||
@@ -46,3 +139,195 @@ The credential-pipeline architecture (telemetry → trace → grade → defense)
|
||||
## Outcome
|
||||
|
||||
**GO** — all six binding decisions and both scope cuts applied to PLAN.md / REQUIREMENTS.md / ROADMAP.md / PROJECT.md before Phase 1 execution. No axis requires escalation (all resolvable at confidence ≥ 0.85). The milestone no longer claims resource enforcement it cannot deliver, and the no-auth abuse vector is closed at MVP scale.
|
||||
|
||||
---
|
||||
# Nextcraft v0.4 — GRILL.md (Adversarial Review Verdict)
|
||||
|
||||
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-CHANGES · **Confidence:** 0.83
|
||||
|
||||
## Summary
|
||||
|
||||
The distribution milestone is small, founder-directed (D-016, confidence 0.99), and additive (zero changes to the running credential pipeline). The plan's central risk: **Node SEA was probe-verified as a flag, not as a working build** — the v0.3 lesson (A-101: probe the mechanism, not the existence) applies. Second gap: a binary whose `--version` lies (stale package.json) would poison the "ongoing binaries" contract. Third: sed-based JSON parsing in install.sh is a fragility + integrity risk. Fourth: "ongoing binaries" has no enforcement mechanism beyond prose. All four closed by binding decisions G-101..G-104 below. No scope cuts required — the milestone is already minimal.
|
||||
|
||||
## Per-Axis Findings
|
||||
|
||||
| Axis | Verdict | Rationale |
|
||||
|------|---------|-----------|
|
||||
| Business case | PASS | Founder directive explicit + recorded (D-016). Evidence of need: live Gitea probe shows latest release v0.2.8 with ZERO assets; bootstrap requires repo archaeology (scripts found only via package.json spelunking). |
|
||||
| Scope | PASS | 5 REQs, 3 execution phases, one focused surface (apps/cli + scripts). Smallest milestone yet. macOS arm64 already cut (D-036, unverifiable here). |
|
||||
| Feasibility | CONCERN (fixed) | SEA flag exists on node v24.15.0, but no end-to-end SEA binary was built during RESEARCH. postject availability assumed (`npx postject` — needs npm registry reachability, unproven). Zipapp fallback requires python3 on target — an honest-degradation ladder, not a silent downgrade. → G-101. |
|
||||
| Honest versioning | CONCERN (fixed) | `--version` from package.json would print a stale hardcoded version inside a per-release binary — breaks upgrade detection + the one-liner's re-run-to-upgrade promise. → G-102. |
|
||||
| Install integrity | CONCERN (fixed) | sed/grep JSON parsing is brittle; a parse failure must never fall through to installing an unverified artifact. Exact asset-name matching + hard-degrade to source instructions. → G-103. |
|
||||
| Sequencing | PASS | P1 CLI (source-runnable) → P2 binary+pipeline → P3 docs+E2E matches dependency order; each phase ships independently. |
|
||||
| Cost/quota | PASS | Zero new paid infra; binaries built on-box; Gitea releases free. Dev-only esbuild dep. |
|
||||
| Risks | CONCERN (fixed) | Top 3: SEA end-to-end (→ G-101 live probe FIRST in P2), npm registry reachability for esbuild (→ proven by P1's pnpm install must-have), Gitea asset-upload token scope (→ live-proven at the v0.3.2 ship itself). |
|
||||
| Adoption/operability | PASS | Consumer = founder + future pilots; one command replaces README archaeology. Rollback trivial (rm ~/.local/bin/nextcraft). No server changes. |
|
||||
|
||||
## Binding Decisions (applied to PLAN.md)
|
||||
|
||||
- **G-101 (BINDING) — SEA live-build probe is the FIRST P2 action.** Task 2-1-01 builds a real binary before anything depends on it; the build script encodes the fallback ladder explicitly (SEA → zipapp with "requires python3" honesty). If SEA fails on this box, zipapp becomes primary with the docs stating the requirement — no silent claim of node-less operation.
|
||||
- **G-102 (BINDING) — Version stamping at build time.** `build-binary` accepts the shipping tag and stamps it into the bundle (`NEXTCRAFT_VERSION` replace); `--version` prints it; install E2E asserts the installed binary reports the tag it was downloaded from. A binary may never report a version it was not built as.
|
||||
- **G-103 (BINDING) — Install-script integrity hard-degrade.** install.sh matches assets by EXACT name (`nextcraft-linux-x64`, `nextcraft-linux-x64.sha256`); any parse/lookup/download failure degrades to source-bootstrap instructions (exit 0) — never installs unverified or name-approximate artifacts. Checksum mismatch = hard stop, exit 1, explicit do-not-run message. dash-safe POSIX sh, no jq.
|
||||
- **G-104 (BINDING) — Ongoing-binaries enforcement.** Every ship from v0.3.2 onward MUST run `scripts/release-assets.sh <tag>` after tag+merge (best-effort, non-blocking, `release_pending` escalation on failure — but attempted + logged every release). The final-phase audit gate includes "milestone release carries both assets" as a check. This makes the founder's "ongoing binaries" directive a pipeline property, not prose.
|
||||
|
||||
## Escalations
|
||||
|
||||
None. All four concerns resolved at confidence ≥ 0.85. No axis requires founder escalation (directive already explicit).
|
||||
|
||||
## Outcome
|
||||
|
||||
**GO** — G-101..G-104 applied to PLAN.md before Phase 1 execution. The milestone claims only what its probes prove, and the ongoing-binaries contract has an enforcement mechanism.
|
||||
|
||||
---
|
||||
|
||||
# v0.5 GRILL (Phase 0, 2026-09-13) — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease
|
||||
|
||||
**Verdict: GO-WITH-CHANGES · Confidence: 0.78** — CUT-3 + G-9..G-18 binding; applied to PLAN.md before P1 execution.
|
||||
|
||||
**Evidence:** every load-bearing plan claim verified against ground-truth code and held (fmt bug defense.py:189, descriptor flow defense.py:143, supervisor frame discard sandbox-agent.py:487-496, no spool cap, dead test_command field, one-line replay margin sandbox-agent.py:416-431, factory rejection). The research layer (D-040..D-046) sold no vapor; the misses were all *interaction* surface.
|
||||
|
||||
## Binding items (all applied to PLAN.md)
|
||||
|
||||
- **CUT-3**: MH-3e identity E2E downgraded from real-app uvicorn harness to `TestClient(create_app(...))` — identity is plain JSON HTTP; the uvicorn harness stays where transport matters (P1 durability, P4 in-ns exec).
|
||||
- **G-9**: `verified_pilot` conftest fixture MUST land with the identity gates (Wave 3-2) — variants/defense are ungated today; without the fixture the existing 413-test suite 403s en masse. MH-M6 reworded: regressions zero *for verified allowlisted pilot learners*.
|
||||
- **G-10**: engine-client must discriminate 403 reasons (`verify_cta` → new `VerifyRequiredError`; allowlist detail → existing `NotAllowlistedError`) — today every 403 collapses into NotAllowlistedError, which would render a verify-CTA as an allowlist lie.
|
||||
- **G-11**: misconfigured `openai-audio` must never crash the boot — lifespan catches the factory error, logs loudly, falls back to mock (descriptor honestly reads mock). Unattended deploy (D-039) survival rule.
|
||||
- **G-12**: client recording bound — auto-stop at 180s default + visible timer + timeslice; 413 renders "re-record" honestly (never silent answer loss).
|
||||
- **G-13**: identity submit caps — one active pending per learner (409 on resubmit) + per-learner rate cap (429); each submission becomes vendor money later.
|
||||
- **G-14**: spool-bound overflow is by-design gap creation — pinned: dropped-counter > 0 → replayed trace gapped → ungradable (never silently-truncated-but-gradable); worst-case spool ≈256MB documented against the 512MB G-2 sweep.
|
||||
- **G-15**: argv contract — templates validate shlex-roundtrip at definition (no quotes/globs); TS splits whitespace-only (no shlex in browsers); exec policy matches EXACT argv tokens (never prefix); `sh -c` passthrough disallowed for design/sim kinds (the digest-gaming vector).
|
||||
- **G-16**: `voice_tts_format` is a `Literal["mp3","wav","opus"]` enum, not a free string — it feeds a Content-Type.
|
||||
- **G-17**: MH-4e "seq-ack intact" was hope-shaped; merged into MH-4d as concrete assertions (stored seqs contiguous 0..N exactly once; spool ≤ ack margin; or cite the P1 suite where a fake agent runs).
|
||||
- **G-18**: the marketplace gated stub proves the gate then returns 501 + `stub: true` + mock markers — never a fabricated "applied" outcome (A-304 honesty house rule).
|
||||
|
||||
## Advisories (recorded)
|
||||
|
||||
a-6 ack on dedup'd appends too (tight margin); a-7 MH-1d 3× runs = one-time ship validation, not per-CI; a-8 manual voice probe must be executable-by-anyone-with-keys (fixture wav + fixed phrase, falsifiable asserts); a-9 413 Content-Length fast path before buffering; a-10 ROADMAP P4 "Depends On: 4" self-typo → fix to 0; a-11 TS `TaskVariant` fields required on the wire; a-12 identity insert-only growth fine at pilot scale; a-13 `recorder.start(1000)`; a-14 do NOT build ack batching unless a flood test shows drainer stall; a-15 provider must always carry `descriptor` (missing it would badge server as mock).
|
||||
|
||||
## Escalations
|
||||
|
||||
None. All four seams remain founder-locked via D-016 / REQ-5-001..007; every finding resolved at confidence ≥ 0.65.
|
||||
|
||||
---
|
||||
|
||||
# v0.8 GRILL (Phase 0, 2026-09-27) — Ship Anywhere
|
||||
|
||||
**Verdict: GO-WITH-BINDINGS · Confidence: 0.74** — G-201..G-210 binding (3×P0, 7×P1); applied to PLAN.md before P1 execution. (Inline orchestrator grill — three delegated griller runs returned empty; logged per Step 4 error recovery at full autonomy.)
|
||||
|
||||
**Evidence:** every load-bearing plan claim re-verified against RESEARCH.md's probe-backed census (R-1..R-32), the existing v0.4 pipeline code (build-binary.mjs, release-assets.sh, install.sh), and Gitea 1.26.1 live state. The research layer held; findings are interaction-surface and lifecycle gaps.
|
||||
|
||||
## Per-Axis Findings
|
||||
|
||||
| Axis | Verdict | Rationale |
|
||||
|------|---------|-----------|
|
||||
| Goal coverage | PASS | All four founder options map to REQ-8-001..005 with falsifiable MHs; docker-per-phase-upload honored by the release workflow design. |
|
||||
| Feasibility | AT-RISK → fixed | QEMU-free arm64 SEA (R-25/26) probe-first is honest, but the plan pinned no failure exit; G-201 forces a decided fallback ladder. Runner fleet on 4GB is bounded by capacity=1 + label routing (R-4). |
|
||||
| Vertical-slice integrity | PASS | P1 ships a green fleet + live pipelines; P2 ships a pullable image; P3 ships installable assets; P4 ships channel-aware CLI; P5 ships the runbook. Each independently real. |
|
||||
| Wave ordering | AT-RISK → fixed | P2→P3 ordering claimed "forced" but both only extend P1's workflow; G-202 makes the true dependency explicit (asset-matrix ownership) and permits P2/P3 work within the same release lane without re-ordering phases. |
|
||||
| Testability | AT-RISK → fixed | MH-1d asserted workflow-built release assets but the release job runs on the tag push — which exists only at ship time; G-203 defines the within-phase proof (temporary tag on the phase branch). MH-2b smoke gate had no in-container test harness; G-204 pins the concrete assertions. |
|
||||
| Security posture | AT-RISK → fixed | Secrets chain mostly sound (R-6/R-13), but the REGISTRY_TOKEN self-serve PUT needs a value the .env token cannot mint — G-205 forces founder-side PAT creation with scoped package-write, closing a plan-time fantasy. CT root password rotation was "ask founder"; G-206 makes it a recorded P0 completion condition. |
|
||||
| Performance/capacity | PASS | Image budget ~400-500MB (R-18) asserted in smoke gate; 32GB disk bounded by prune cron + cache caps + health_check floor (R-5). |
|
||||
| Operational readiness | AT-RISK → fixed | The runbook (P5) would document a fleet whose bring-up lives only on CT 113; G-207 requires the compose/config/cron committed as fleet-as-code in-repo. Degrade remediation text must name the privileged variant precisely (G-208). |
|
||||
| Dependency risk | PASS | No new product deps; docker.io engine on release lane accepted-risk with documented future fix; corepack fallback decided (R-11). |
|
||||
| Adoption/UX | PASS | Three channels each end at doctor/verify green; guide offline-first with parity test; refusals honest. |
|
||||
|
||||
## Binding Decisions (applied to PLAN.md)
|
||||
|
||||
- **G-201 (P0)** — arm64 build has a decided fallback ladder: probe QEMU-free cross-arch SEA on the runner first; if the blob fails to run under emulation, fall back to binfmt+qemu for FULL arm64 builds (not just blob generation); if the runner can't install binfmt, ship v0.8 with x64-only debs/binaries and record the arm64 deferral — never leave the release lane red or blocked.
|
||||
- **G-202 (P1)** — dependency truth: P2 and P3 both consume P1's release workflow as their extension point; P2 precedes P3 for founder-priority (docker is the headline channel), not technical force. PLAN wave-order paragraph corrected; no phase reorder.
|
||||
- **G-203 (P1)** — release-workflow within-phase proof: MH-1d/D-3d prove via a temporary `v0.7.1-rc1`-style tag pushed to the phase branch head at verify time, workflow observed building + attaching to the (pre-release/draft or immediately-deleted) release, tag then deleted. The phase ship tag remains the real artifact.
|
||||
- **G-204 (P1)** — smoke gate pinned: container starts → /health 200 within timeout → `GET /` content-type text/html AND contains a known marker (not just non-JSON-404) → degrade boot notice present in logs → fabric create returns 503 with remediation → clean stop. All five assertions in the workflow step; failure = red release job.
|
||||
- **G-205 (P0)** — REGISTRY_TOKEN is a founder-minted PAT with package-write scope (the .env GITEA_TOKEN may lack it): P1's self-serve attempt becomes "PUT the secret IF the existing token can mint it — else escalate to founder with the exact scope needed (write:package + write:repository)". The plan must not assume the PUT succeeds; escalation path has a founder question, not a silent wait.
|
||||
- **G-206 (P0)** — CT 113 root password rotation is a P0 completion condition of Wave 1-1: after deploy-key install, the founder is asked to rotate; if not confirmed by Wave 1-2, an `escalation: ct_root_password_still_live` commit lands and the ship notes carry it. Never silently carried.
|
||||
- **G-207 (P1)** — fleet-as-code: the compose file, per-runner configs, and prune cron are committed in-repo (e.g. `deploy/ci/`) as the reproducible fleet definition; CT 113 deployment is a copy + `docker compose up -d` — the runbook's fleet recipe references the committed files, not box memory.
|
||||
- **G-208 (P1)** — degrade remediation text must be exact: name the unprivileged default, the seccomp constraint, the documented privileged/seccomp-override variant (docker run flags), and the host-side alternative (run the stack binary-side); the boot notice and 503 payload carry the same text (single remediation constant, tested).
|
||||
- **G-209 (P1)** — pre-commit hooks must not break contributor boxes lacking the toolchain silently in the other direction either: hooks fail-loud (D-070) but the README dev section documents the exact skip path (`pre-commit run --no-skip` semantics never needed; SKIP=env documented) — a contributor with no ruff gets a loud pointer, not a broken commit flow.
|
||||
- **G-210 (P1)** — the guide parity test (MH-5b) needs a defined source of truth: the runbook's channel sections are the canonical text; `nextcraft guide` renders them via an extracted shared snippet (single source file, generated or duplicated-with-test); divergence = test failure. PLAN Task 5-1-02 mechanism named.
|
||||
|
||||
## Plan Edits Applied
|
||||
|
||||
1. G-201 → Task 3-1-01 fallback ladder + MH-3a reworded (probe → binfmt-full → deferral, never red lane)
|
||||
2. G-202 → Overview wave-order paragraph corrected (P2 precedes P3 for founder priority, not technical force)
|
||||
3. G-203 → Task 1-2-02 + Task 3-2-03 verification mechanism (temporary rc tag, deleted after proof)
|
||||
4. G-204 → Task 2-2-02 five-assertion smoke gate pinned
|
||||
5. G-205 → Task 1-2-04 reworded (founder-minted PAT with package-write; escalation question not silent wait)
|
||||
6. G-206 → Task 1-1-01/1-1-03 rotation condition + ship-notes carry
|
||||
7. G-207 → new P1 Wave 1-1 task (fleet-as-code committed in deploy/ci/)
|
||||
8. G-208 → Task 2-1-02 single remediation constant, tested
|
||||
9. G-209 → Task 1-2-05 README dev-section skip path documented
|
||||
10. G-210 → Task 5-1-02 parity mechanism named (single source, divergence = test failure)
|
||||
|
||||
## Advisories (recorded, no action)
|
||||
|
||||
- a-201: docker.io rate limits on the release lane — bookworm main is fine; watch anonymous pull quotas on job-image pulls (runner-images:ubuntu-latest is docker.gitea.com, not docker.io).
|
||||
- a-202: `GITEA_TOKEN` naming collision (automatic job token vs repo secret of the same name) — Gitea precedent lowest-level-wins; keep the repo secret name `GITEA_TOKEN` per R-13 but document the collision in the workflow comments.
|
||||
- a-203: cache.host on CT 113 IP — if the LXC ever changes IP, cache breaks; fleet-as-code (G-207) documents the IP as a bring-up variable.
|
||||
- a-204: image `latest` tag vs immutable release tags — `latest` is convenience-only; runbook always pins the release tag.
|
||||
|
||||
---
|
||||
|
||||
## v0.10 Grill (Phase 0 GRILL, 2026-09-30)
|
||||
|
||||
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** SHIP-WITH-BINDINGS · **Confidence:** 0.82 · **Date:** 2026-09-30
|
||||
|
||||
**Verdict summary.** The v0.10 B2B Employer Network plan (6 phases, 15 waves, 30 tasks) is feasible and correctly sequenced — every load-bearing tree citation re-verified against the working code and held (require_admin shape, the 402 gate order + hardcoded zeros, billing_admin checkout/mock-complete mechanics, the four main.py wiring sites, the web census, both throttle precedents, the tests/billing layout, scrypt probe). The researcher's six open risks are each carried by a plan task. But red-teaming found 10 binding defects: a REQUIREMENTS-vs-PLAN gate-order contradiction (REQ-10-004 literally says sessions compose AFTER the entitlement check — backwards, the G-307 class); "web render tests" specified against a harness that does not exist in-tree (D-059 declined it — the tests as written are undeliverable without smuggling in jsdom, violating R-66); a suite-breaking throttle trap (the global 30/min deque is module state shared across every tests/employers/ file — the whole fast suite runs inside one 60s window and 429s itself around test ~16); a credential-stuffing hole hiding in an ambiguous throttle-append rule (if failed passwords don't consume global budget, D-088's stated rationale is void); an EmailStr trap (email-validator is NOT installed — the natural pydantic choice is a boot-time ImportError and the first new dependency); a TOCTOU race on one-pending-claim-per-entity with no UNIQUE backing it (insert_application's check-first is safe only because a real constraint backs it); employer postings that would inherit source="admin" provenance if the admin route is copied; a re-tag fork between admin and employer edit paths (D-052 drift); a dual-schema pin pointed at contracts no task creates (the own-postings response has no named pydantic model); and a UX criterion inviting the executor to "fix" the 402 wire zeros (risk 7 stated as acceptance criteria). All ten amend existing tasks — nothing re-architects, no scope cuts required.
|
||||
|
||||
## Per-Axis Findings
|
||||
|
||||
| # | Axis | Verdict | Reason |
|
||||
|---|------|---------|--------|
|
||||
| 1 | Feasibility | CONCERN → fixed | All citations verified in-tree; but 4 tasks assume things the tree doesn't have: a render harness (G-312), EmailStr's dependency (G-315), a backing constraint for the claim 409 (G-316), a named own-postings pydantic contract for the MH-3a pin (G-319) |
|
||||
| 2 | Scope | PASS | Big but REQ-mandated, heavy precedent reuse (the v0.6 46-task cadence); talent pages honestly left mock; no half-built surface found; no CUTs |
|
||||
| 3 | Security | CONCERN → fixed | New trust-boundary code mostly well-specced (hashed tokens, lazy TTL, email-keyed throttles, 422 scrub, own-session mock-complete); defects: throttle-append ambiguity voids the stuffing bound (G-314), throttle-vs-KDF order unspecified (G-320a), claim insert race (G-316) |
|
||||
| 4 | Honesty | CONCERN → fixed | "Render tests" would present source pins as render tests or force a harness dep (G-312); employer postings inheriting source="admin" is provenance dishonesty (G-317); UX criterion 3 contradicts the byte-stable 402 zeros (G-320b) |
|
||||
| 5 | Drift | CONCERN → fixed | D-091 pins strong (payload-parity test + unchanged v0.9 suite); but the own-postings dual-schema pin has no pydantic contract to pin against (G-319), and employer edit would fork the admin re-tag semantics (G-318) |
|
||||
| 6 | Sequencing | PASS | Wave order verified dependency-sound (P1 module → P2 claims → P3 surface → P4 writes → P5 money); pins land against base.py contracts, not pages — no hidden dependency |
|
||||
| 7 | Researcher open risks | PASS | All six (scrypt burst, CORS preflight, 422 echo, sessionStorage, mock-complete own-session, example.com evidence) carried by named tasks with pins |
|
||||
| 8 | Test coverage | CONCERN → fixed | Several must-haves unverifiable AS WORDED (render tests, G-312); the employers suite self-DOSes without a conftest-wide reset (G-313); stuffing untested (G-314) |
|
||||
| 9 | One-entity/one-account races | CONCERN → fixed | One-APPROVED-owner is structurally sound (employer_accounts.employer_id UNIQUE → IntegrityError, pinned); one-PENDING-per-entity is check-first with NO backing constraint — TOCTOU (G-316) |
|
||||
| 10 | Token entropy + storage | PASS | token_urlsafe(32) = 256-bit CSPRNG, sha256-at-rest, lazy 7-day TTL, revoke-on-logout, concurrent sessions — matches D-087 exactly; no finding |
|
||||
|
||||
## Binding Decisions (G-311..G-320 — applied to PLAN/REQUIREMENTS before execute)
|
||||
|
||||
**G-311 — REQ-10-004's gate-order wording is backwards; the session gate composes BEFORE the entitlement check.** REQ-10-004 says "employer sessions compose AFTER the existing entitlement enforcement" — literally: an unauthenticated caller hits the 402 gate before the 401, leaking entitlement state to anonymous callers and breaking the web client's SessionRequiredError-vs-UpgradeRequiredError discrimination (a 402 on an expired session renders an upgrade CTA to a logged-out visitor — a lie). The plan (Task 4-1-02, R-58) has the correct order; the requirement text contradicts it (the G-307 class: requirement prose naming a composition that must not exist). Decision: amend REQ-10-004 to "the entitlement check composes AFTER the employer session gate (401/403 precede any 402 — the R-58 admin-gate-first order preserved)". Applies to: REQUIREMENTS REQ-10-004; PLAN Task 4-1-02 (amendment note). Test: unauthenticated create → 401 (never 402); unclaimed-session create → 403 (never 402). Confidence 0.85.
|
||||
|
||||
**G-312 — no render harness exists in-tree: page/component tests are fetch-stub + source-pin, never "render tests".** Tasks 3-3-01/3-3-02 say "fetch-stub render tests" / "web render tests" — D-059 explicitly declined jsdom/happy-dom (v0.7, founder-scoped), the billing-entitlement.test.ts template's own header says "no render harness in-tree", and adding one is a new dependency violating R-66's zero-dep claim. As written, the executor either smuggles in jsdom or ships tests whose names lie about what they verify (the A-304 class). Decision: every "render test" is reworded to the D-059/E87 discipline — fetch-stub tests over engine-client employer functions (401 → SessionRequiredError, token header attached, logout revoke+clear) + source pins for component honest states (logged-out/claim-pending/loading/error branches present in source, the pins.test.ts discipline); jsdom/happy-dom are BANNED this milestone. Applies to: PLAN Tasks 3-3-01, 3-3-02 (amendment notes; 4-2-01/5-2-01/5-2-02 already say "web fetch-stub" — their loose "render" phrasing is covered by this G); MH-3b/3c/3e and MH-5c/5e read under this discipline. Confidence 0.85.
|
||||
|
||||
**G-313 — both throttle limiters reset via a SUITE-WIDE autouse fixture in tests/employers/conftest.py.** The global cap (default 30/min) is module-level process state (R-63); the entire tests/employers/ suite — register+login flows across test_auth, test_claims, test_claims_admin, test_postings_read, test_postings_write, test_checkout, test_checkout_e2e — runs in ONE process in well under 60 seconds, i.e. inside ONE window: ~90+ auth attempts against a 30-cap → guaranteed mid-suite 429s cascading through every downstream test. The plan places reset fixtures only in test_throttle.py's test list. Decision: tests/employers/conftest.py (a NEW file — the sanctioned deviation from billing's no-conftest layout, R-64) carries an autouse fixture clearing BOTH limiter module states before/after EVERY employers test; the per-file fixtures in test_throttle.py stay for its own window-math. Applies to: PLAN Task 1-2-02 (amendment note), D-096 operationalization. Test: the suite itself (it cannot pass without this — the trap is self-revealing but only after a day of flaky archaeology). Confidence 0.85.
|
||||
|
||||
**G-314 — the global cap counts every well-formed attempt, including failed passwords — or D-088's rationale is void.** The plan's "append only after ALL checks pass (a rejected attempt never consumes budget)" is copied from the sandbox create-cap, where "rejected" means server-rejected-by-validation. Read against auth, "all checks pass" plausibly includes password verification — under that reading, failed logins consume nothing and credential stuffing with rotating emails (the exact attack D-088 names: "bounding cross-email credential-stuffing volume") is UNBOUNDED, and the planned test "rejected attempts don't consume the global budget" would PIN the hole. Decision: the global deque appends for every well-formed login+register request that passes body validation, BEFORE the KDF runs — successes and failed passwords both consume budget; only 422-invalid bodies and the cap's own 429s are exempt; the per-email window keeps failures-only semantics. The test is reworded: "422-invalid attempts consume nothing; FAILED-PASSWORD attempts DO count; N distinct emails inside the window → the (cap+1)-th 429s (stuffing test, knob set low)". Applies to: PLAN Task 1-2-02 (amendment note), MH-1d. Confidence 0.75.
|
||||
|
||||
**G-315 — email fields are plain str with stdlib validation; pydantic EmailStr is BANNED.** email-validator is NOT in the venv (pyproject carries only pydantic/pydantic-settings; no in-tree email-validation precedent exists — identity keys on learner_id). Using pydantic's EmailStr in RegisterEmployerRequest/LoginEmployerRequest — the natural choice for "email + password, honest validation" — raises at model-build/validation time ("email-validator is not installed"), and fixing it means the FIRST new dependency, breaking R-66/D-086's zero-dep claim and the mypy zero-override regime. Decision: email is `str` with stdlib-only validation (non-empty, exactly one "@", non-empty domain, lowercased + stripped on normalize — mirrored in the store's _validate hook); the evidence domain extraction (R-65) already handles the parse. Applies to: PLAN Task 1-1-01 (amendment note). Test: register accepts "User@Example.COM" (normalized), rejects "not-an-email" with an honest 422 that never echoes the password (rides the G-312/1-2-03 pin). Confidence 0.85.
|
||||
|
||||
**G-316 — one-pending-claim-per-entity gets a structural backstop, not just check-first.** insert_application's check-first insert is race-safe ONLY because UniqueConstraint(posting_id, learner_id) backs it; "one PENDING claim per entity" is a CONDITIONAL uniqueness (rejected claims on the same employer_id are legal — "rejection frees the entity"), so no table-level UNIQUE exists and two concurrent claim POSTs both pass the check and both insert. Two threads reach the store (FastAPI sync endpoints run in a threadpool; the store uses check_same_thread=False per-call Sessions) — the 409 promise is TOCTOU. The one-APPROVED-owner arm IS structurally sound (employer_accounts.employer_id UNIQUE → IntegrityError, already pinned) — this G covers only the pending arm. Decision: `employer_claims.__table_args__` gains `Index("uq_pending_claim_per_entity", "employer_id", unique=True, sqlite_where=text("status = 'pending'"))` — SQLite supports partial indexes and SQLAlchemy emits them via create_all (the D-095 one-time path); insert_claim keeps its check-first 409 fast-path and SURFACES the IntegrityError as the same honest 409 under race. Applies to: PLAN Tasks 1-1-02 (schema), 2-1-02 (amendment note), MH-2c. Test: pin the partial index exists in sqlite_master; second concurrent-pending insert → 409 (store test surfaces IntegrityError, never a duplicate pending row). Confidence 0.75.
|
||||
|
||||
**G-317 — employer-created postings stamp their own provenance, never source="admin".** Task 4-1-02 mirrors the admin create route, whose Posting construction hardcodes `source="admin"`, `id=f"adm-…"`, `external_id=f"adm-…"` (marketplace_admin.py:221-243). Copied wholesale, an employer-authored posting presents as operator-authored — corpus provenance lies (A-304: the v0.6 ingest contract means source distinguishes seeded vs admin-authored; a third principal class silently inheriting the admin label makes "who wrote this" unanswerable and muddies the audit story the milestone sells). Decision: employer create stamps a distinct source (e.g., `source="employer"`) with its own id/external_id schemes (`emp-…`), keeping the (source, external_id) upsert identity unambiguous across all three authorship classes; the admin routes are untouched. Applies to: PLAN Task 4-1-02 (amendment note). Test: employer-created posting persists with the employer source stamp (pinned); admin routes still stamp "admin". Confidence 0.80.
|
||||
|
||||
**G-318 — employer edit composes the SAME re-tag path as the admin edit; no tag-semantics fork.** The v0.9 admin edit (PUT /v1/marketplace/admin/postings/{id}) calls update_posting_fields then re-tags via tag_posting — D-052: tags are DERIVED data, overwritten on edit. Task 4-1-02's employer PATCH specifies the own-drafts guard but is silent on re-tagging: an executor shipping edit-without-re-tag leaves employer-edited postings with stale tags — derived data drifting from content, a D-052 violation and a search-facet integrity hole (the exact drift class D-091 exists to prevent, on the tagging axis). Decision: the employer edit route composes the identical update_posting_fields + tag_posting path (tag failure honest — edit stands, tags stay, warning logs — the admin contract), with the own-drafts-only guard evaluated BEFORE the mutation (published/closed → 409 without side effects); PATCH-vs-PUT method naming is free choice. Applies to: PLAN Task 4-1-02 (amendment note). Test: employer edit on a mock provider re-tags (tags updated or TaggingUnavailable-swallowed-pinned); edit published → 409 AND no field/tag mutation occurred. Confidence 0.70.
|
||||
|
||||
**G-319 — the own-postings response is a NAMED pydantic contract; applicant data is COUNTS ONLY.** MH-3a/G-108 pins assert employer.ts against "ai_service/employers/base.py's pydantic contracts — both schemas", but no task's contract list includes the own-postings response: Task 1-1-01 names Account/Claim/Register/Login/Token/Me; Task 3-1-01 describes the route's payload in prose. The pin as specified either pins against nothing (a weak dict-shape grep) or blocks on a contract nobody was told to write. The route also needs a data source the plan doesn't name: applicant_count per posting, while the marketplace store's only read is list_applications(posting_id) — a per-posting N+1 across the employer's postings. Decision: (a) Task 3-1-01 adds named contracts to employers/base.py — `OwnPostingSummary` (PostingSummary reused from marketplace, PLUS applicant_count) and the facets/list response — so the MH-3a pin is schema↔schema; (b) the count rides a new in-pattern marketplace store read (e.g., `count_applications_by_posting(posting_ids) -> dict[str, int]`), never an N+1; (c) explicitly pinned: employer-facing responses expose applicant COUNTS ONLY — learner identifiers NEVER appear in any /v1/employers/* response (PII boundary, security-auditor P3/P5 review scope). Applies to: PLAN Tasks 1-1-01 (contract list), 3-1-01 (amendment note), 3-2-01, MH-3a. Confidence 0.80.
|
||||
|
||||
**G-320 — throttle-before-KDF order (a) and the 402 zeros are wire-only, never displayed truth (b).** (a) R-55's open risk (login bursts stack ~50 ms scrypt) is bounded ONLY if both throttle checks run BEFORE the derivation — a 429'd or globally-capped attempt must never burn KDF CPU. The plan never states the check order inside the login route. (b) UX acceptance criterion 3 says "limits shown are the store's truth" while the 402 payload hardcodes posts_used=0/posts_limit=0 (byte-stable per D-091/MH-4a) — as written the criterion invites the executor to "improve" the zeros (risk 7) or to render "0/0" as real limits (a lie). Decision: (a) route order is body-validation → global cap → per-email window → scrypt verify → failure-append (pinned: a 429'd attempt performs zero derivations); (b) criterion 3 reworded — the 402 payload carries reason + CTA only; real limits render exclusively from GET /v1/employers/billing/entitlement (the entitlement view is the limits truth source, Task 5-1-01). Applies to: PLAN Task 1-2-02 (a, amendment note), Task 4-2-01 + UX criterion 3 (b, amendment note). Tests: (a) a capped email's login attempt completes in ≪ KDF time / no derivation side-effect observable — pinned as "429 before KDF" via the throttle test's knob math; (b) web source pin: the 402 branch renders reason + CTA and fetches nothing from the payload's posts_used/posts_limit. Confidence 0.80.
|
||||
|
||||
## Scope Cuts
|
||||
|
||||
None. Every phase maps to a REQ-10-001..005 row; the one candidate (employer draft-edit, the least-essential P4 arm) is named by REQ-10-004 and is cheap once G-318 forces admin-parity — cutting it would unilaterally narrow a locked requirement. Talent pages staying mock is the honest deferral the plan already makes (D-092).
|
||||
|
||||
## Advisories (recorded, no plan edit)
|
||||
|
||||
- **a-311:** Employer mock-complete "own session" = checkout rows whose employer_id equals the session's claimed entity — including operator-created rows for that entity (same buyer, D-076 fallback path; not a G-301 violation). Pin the test's fixture accordingly so the 403 arm uses a DIFFERENT entity's row, not an admin-created same-entity row.
|
||||
- **a-312:** Per-email throttle inherently permits targeted lockout DoS (5 failures → victim's email 429s for 15 min). Accepted within D-088 (email-keying is locked, never re-litigated); success-clears-window is the mitigation; surface copy need not mention it.
|
||||
- **a-313:** .env.example throttle entries are documentation, not "default-off" — the Settings ints are always active with defaults 5/30 (the R-60d shape). Write the comments that way.
|
||||
- **a-314:** Logout on an already-revoked token → 401 (strict) is fine; do not "fix" it to 204 — idempotent-logout would let a stolen-tab replay confirm revocation state.
|
||||
- **a-315:** Register's 409 duplicate-email necessarily reveals account existence — standard and unavoidable; the login 401 uniformity (already specified) is the enumeration boundary that matters.
|
||||
|
||||
## Plan Edits Applied
|
||||
|
||||
1. G-311 → REQUIREMENTS REQ-10-004 wording + PLAN Task 4-1-02 amendment note
|
||||
2. G-312 → PLAN Tasks 3-3-01, 3-3-02 amendment notes
|
||||
3. G-313/G-314/G-320a → PLAN Task 1-2-02 amendment note
|
||||
4. G-315 → PLAN Task 1-1-01 amendment note
|
||||
5. G-316 → PLAN Task 2-1-02 amendment note
|
||||
6. G-317/G-318/G-311 → PLAN Task 4-1-02 amendment note (single combined note)
|
||||
7. G-319 → PLAN Task 3-1-01 amendment note
|
||||
8. G-320b → PLAN Task 4-2-01 amendment note + UX criterion 3 clause
|
||||
|
||||
+196
-141
@@ -2,19 +2,17 @@
|
||||
|
||||
## Persona Roster
|
||||
|
||||
> **v0.3 update (RESEARCH, lead-developer assessment):** backend-engineer territory extended to the new engine modules (telemetry/grading/variants persistence + APIs). New phase-relevant custom personas added: **sandbox-engineer** (Linux-namespace isolation infra) and **voice-engineer** (STT/TTS + Examiner agent audio pipeline). ai-engineer re-scoped to LLM/agents/prompts + grading/variant/voice *model-facing* logic. **security-auditor stays inactive** (KYC deferred per founder directive). frontend-engineer gains real-sandbox (xterm.js), live-telemetry, and live-defense surfaces.
|
||||
> **v0.10 update (RESEARCH, lead-developer assessment):** milestone = B2B Employer Network (Feature — ai_service/employers/ module: 8th D-027 store + scrypt auth + session/claim flows, /v1/employers routes, employer web-surface conversion + admin network overview). **Activated:** lead-developer (coordination), backend-engineer (owns ai_service/employers/* + api/employers*.py + api/employers posting/billing routes + the D-091 helper + config.py knobs + main.py wiring — REQ-10-001..005 core), data-engineer (owns the 4-table employer store schema: account/session/claim/audit rows, unique + check-first constraints, tz normalization — REQ-10-001/002), frontend-engineer (owns the employer surface conversion + engine-client employer section + SessionRequiredError + login/register pages + packages/types/employer.ts + admin overview sections + pricing CTA conversion — REQ-10-003/005), security-auditor (phase-specific: scrypt params + compare_digest, session hashing-at-rest, throttle windows, CORS Authorization, 422 password-echo scrub, claim-evidence honesty, sessionStorage exposure, PII — REQ-10-001/002), identity-engineer stays DEACTIVATED (see its note — employer auth is NOT the v0.5 learner identity module). **Deactivated:** cli-engineer, devops-engineer, sandbox-engineer, ai-engineer, voice-engineer, design-system-engineer (all unchanged surfaces this milestone).
|
||||
|
||||
### lead-developer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Coordinates task decomposition across web, AI service, engine, and sandbox territories; resolves conflicts between frontend, backend, AI, sandbox, and voice personas
|
||||
reason: Coordinates the v0.10 B2B Employer Network milestone (auth foundation → claim → surface conversion → posting management → checkout waves); owns wave order + the cross-boundary wire review (employer.ts ↔ employers/base.py dual-schema pins, the 402/401/403 discrimination contract) + the D-091 extraction's no-drift review (both principal gates compose the same helper)
|
||||
domain: coordination
|
||||
frameworks:
|
||||
- next.js
|
||||
- turborepo
|
||||
- pnpm
|
||||
- fastapi
|
||||
constraints:
|
||||
- pragmatic
|
||||
- battle-tested defaults
|
||||
@@ -22,8 +20,137 @@ constraints:
|
||||
territory:
|
||||
- "**/package.json"
|
||||
- "**/turbo.json"
|
||||
- "**/pnpm-workspace.yaml"
|
||||
- "**/tsconfig.json"
|
||||
- "apps/web/tests/pins.test.ts"
|
||||
```
|
||||
|
||||
### devops-engineer
|
||||
```yaml
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: v0.10 DEACTIVATED — no infra changes; the employer module rides the existing release lane (image + assets build automatically on tag per D-064/D-065). Roster + v0.8 territory retained.
|
||||
domain: devops
|
||||
frameworks:
|
||||
- docker
|
||||
- docker-compose
|
||||
- gitea-runner
|
||||
- github-actions-syntax (gitea-compatible subset)
|
||||
- bash
|
||||
constraints:
|
||||
- sibling-mode-socket-trust (org-scoped private repos only; accepted-risk documented)
|
||||
- secrets-via-actions-secrets-or-founder
|
||||
- job-token-cannot-publish-packages
|
||||
- cache-host-pinning
|
||||
- fast-lanes-only-in-ci (real_server/userns never in job containers, D-063)
|
||||
territory:
|
||||
- ".gitea/workflows/**"
|
||||
- "Dockerfile"
|
||||
- ".dockerignore"
|
||||
- ".pre-commit-config.yaml"
|
||||
- "deploy/runner/**"
|
||||
```
|
||||
|
||||
### cli-engineer
|
||||
```yaml
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: v0.10 DEACTIVATED — no CLI changes; the employer surface is server+web only. Roster + v0.8 territory retained.
|
||||
domain: cli
|
||||
frameworks:
|
||||
- node
|
||||
- typescript
|
||||
- node-sea
|
||||
- esbuild
|
||||
- postject
|
||||
- dpkg-deb
|
||||
- posix-sh
|
||||
constraints:
|
||||
- stdlib-only-runtime
|
||||
- thin-wrapper
|
||||
- fail-loud-exit-codes
|
||||
- postinst-never-fails-dpkg (advisory doctor, D-067)
|
||||
- deb-no-systemd-unit (deploy --service owns units, D-067)
|
||||
- guide-offline-first (zero runtime fetches, D-072)
|
||||
- deb-version-policy (tag minus leading v)
|
||||
territory:
|
||||
- "apps/cli/**"
|
||||
- "scripts/install.sh"
|
||||
- "scripts/release-assets.sh"
|
||||
- "scripts/build-deb.sh"
|
||||
```
|
||||
|
||||
### sandbox-engineer
|
||||
```yaml
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: v0.10 DEACTIVATED — no sandbox fabric changes; the D-068 degrade surface ships unchanged. Roster + v0.8 territory retained.
|
||||
domain: infra
|
||||
frameworks:
|
||||
- python
|
||||
- linux-namespaces
|
||||
- docker (degrade semantics only — no docker backend)
|
||||
- pytest
|
||||
constraints:
|
||||
- capability-probe-is-truth-source (markers only flavor remediation, D-068)
|
||||
- reject-not-fallback (unisolated learner code unacceptable in deployed schools; supersedes A-101)
|
||||
- degrade-keeps-core-features (chat/agents/grading untouched)
|
||||
- probe-semantics-verbatim (unshare --user --map-root-user true, 10s timeout)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/sandbox/**"
|
||||
- "apps/ai-service/ai_service/main.py"
|
||||
- "apps/ai-service/ai_service/api/sandboxes.py"
|
||||
- "apps/ai-service/tests/sandbox/**"
|
||||
- "apps/ai-service/tests/helpers/userns.py"
|
||||
```
|
||||
|
||||
### security-auditor
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: true
|
||||
reason: v0.10 re-activated (phase-specific) — P1 auth review: scrypt params/verify discipline (N=2^14/r=8/p=1 module constants pinned by test, hmac.compare_digest, self-describing rows so params evolve without invalidating stored hashes — D-086), session tokens hashed-at-rest (sha256(token) only, DB-leak yields nothing — D-087), throttle windows (per-email failed-login + global cap; email-keyed never IP — D-088), 401 vs 403 semantics (credential vs authorization — D-090); P2 claim review: evidence is advisory, never displayed as verification (A-304 — example.com corpus makes match=false corpus-wide), one-account-one-entity UNIQUE enforcement; P3+ the 422 password-echo fix (main.py pii_fields gains "password"+"email" — pydantic 422s echo err.input today, R-56/open-risk-3), sessionStorage exposure copy, employer mock-complete own-session check (G-301 hole class); P6 final review (no secrets/logs/releases leak credentials)
|
||||
domain: security
|
||||
frameworks:
|
||||
- pytest
|
||||
- httpx
|
||||
constraints:
|
||||
- STRIDE-classified
|
||||
- secrets-via-env-only (D-014)
|
||||
- secrets-never-logged
|
||||
- fail-closed-on-unconfigured
|
||||
- tokens-hashed-at-rest (D-087)
|
||||
- never-display-forgeable-as-verified (D-089/A-304)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/employers/**"
|
||||
- "apps/ai-service/ai_service/api/employers*.py"
|
||||
- "apps/ai-service/tests/employers/**"
|
||||
```
|
||||
|
||||
### backend-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: v0.10 ACTIVATED — owns the employers module core: ai_service/employers/base.py (pydantic contracts), passwords.py (stdlib scrypt KDF, D-086), store.py wiring via deps.py get_employer_store, claim evidence normalization (stdlib suffix match, D-089); api/employers.py (auth routes register/login/logout/me + claim + own-postings CRUD/publish/close + own-checkout + own-entitlement; require_employer_session dep per D-090) + api/employers_admin.py (accounts/claims/audit operator routes, require_admin imported from marketplace_admin — D-094); the D-091 shared consume_posting_entitlement helper + its adoption in marketplace_admin.create_posting (wire contract byte-identical); config.py throttle knobs + .env.example; main.py wiring (employer_store lifespan + close, router mounts, CORS allow_headers gains "Authorization" — R-60)
|
||||
domain: backend
|
||||
frameworks:
|
||||
- fastapi
|
||||
- pydantic-settings
|
||||
- httpx
|
||||
- bash
|
||||
- turborepo
|
||||
- pnpm
|
||||
constraints:
|
||||
- secrets-via-env-only (D-014)
|
||||
- idempotent-scripts
|
||||
- fail-closed-admin
|
||||
- never-call-cloud-in-tests (zero network by construction, D-096)
|
||||
- gate-before-insert (the 402 consumes nothing, R-58)
|
||||
- real-kdf-in-tests (no fake-hash path, D-096)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/employers/**"
|
||||
- "apps/ai-service/ai_service/config.py"
|
||||
- "apps/ai-service/ai_service/main.py"
|
||||
- "apps/ai-service/ai_service/api/**"
|
||||
- "apps/ai-service/tests/employers/**"
|
||||
- "apps/ai-service/.env.example"
|
||||
- "apps/ai-service/pyproject.toml"
|
||||
```
|
||||
|
||||
@@ -31,95 +158,30 @@ territory:
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Phase 6 real-engine learner-surface integration — xterm.js in-browser terminal, file-tree/run/test controls, live telemetry panels, live voice defense UI, live grading display. Owns all page components, layouts, surface-specific UI.
|
||||
reason: v0.10 ACTIVATED — owns the employer surface conversion (D-092): engine-client employer section (register/login/logout/me, claim, own-postings, own-checkout, own-entitlement + SessionRequiredError 401 discrimination + nc-employer-session sessionStorage wrapper — the adminToken pattern), /employer/login + /employer/register pages (greenfield), postings page own-lifecycle conversion, overview page real active-postings + applicant counts (talent metrics stay mock-labeled), employer profile via the existing public GET, packages/types/employer.ts (dual-schema pinned, D-093); admin overview sections on the moderation page (accounts list, claim queue with evidence signal + approve/reject, audit — the entitlement panel's locked-state pattern, D-094); pricing CTA conversion from contact-path to employer-login path (supersedes D-079); talent pages stay mock (D-092)
|
||||
domain: frontend
|
||||
frameworks:
|
||||
- react
|
||||
- next.js
|
||||
- tailwindcss
|
||||
- lucide-react
|
||||
- recharts
|
||||
- react-flow
|
||||
- "@xterm/xterm"
|
||||
- "@xterm/addon-fit"
|
||||
constraints:
|
||||
- component-first
|
||||
- server-components-default
|
||||
- minimal-client-js
|
||||
- sse-client-buffering (buffer bytes, split frames on \n\n, join data: lines)
|
||||
- abortcontroller-cleanup (idempotent abort in effect cleanup)
|
||||
- websocket-lifecycle (typed messages, reconnect backoff, cleanup)
|
||||
- mediarecorder-permission-ux (mic consent, graceful no-mic fallback)
|
||||
- responsive-all-breakpoints
|
||||
- dark-mode-support
|
||||
- no-build-time-db-reads (D-039 export build stays repo-only)
|
||||
- honest-states (loading/error/empty/logged-out never mocked)
|
||||
- abortcontroller-everywhere (A-406)
|
||||
- session-gated-shell (401 → honest logged-out state, never a retry loop)
|
||||
territory:
|
||||
- "apps/web/**"
|
||||
- "packages/ui/**"
|
||||
- "packages/mock-data/**"
|
||||
- "packages/types/**"
|
||||
```
|
||||
|
||||
### data-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Owns TS mock data layer schema and typed definitions + TS types for telemetry/trace/grade/variant/defense shapes the web surfaces consume. Does NOT own the Python corpus or engine stores — aligned by convention (D-021).
|
||||
domain: data
|
||||
frameworks:
|
||||
- typescript
|
||||
constraints:
|
||||
- schema-first
|
||||
- type-safe
|
||||
- migration-ready
|
||||
- mock-data-only
|
||||
territory:
|
||||
- "packages/types/**"
|
||||
- "packages/mock-data/**"
|
||||
```
|
||||
|
||||
### backend-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Owns apps/ai-service app shell, config, API endpoints (incl. WebSocket telemetry ingest), engine persistence (SQLite stores), scripts, and test harness. Extended for v0.3 engine modules.
|
||||
domain: backend
|
||||
frameworks:
|
||||
- fastapi
|
||||
- uvicorn
|
||||
- pydantic
|
||||
- pydantic-settings
|
||||
- httpx
|
||||
- pytest
|
||||
- sqlmodel
|
||||
- sqlalchemy
|
||||
- websockets
|
||||
- aiofiles
|
||||
constraints:
|
||||
- provider-agnostic-boundaries (engine modules import nothing from agents/ or api/)
|
||||
- streaming-first
|
||||
- sqlite-first-persistence (protocol-wrapped stores, Postgres-ready, D-027)
|
||||
- secrets-via-env-only
|
||||
- mock-provider-in-tests
|
||||
- websocket-contract (typed envelopes, seq gap detection, D-026)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/main.py"
|
||||
- "apps/ai-service/ai_service/config.py"
|
||||
- "apps/ai-service/ai_service/api/**"
|
||||
- "apps/ai-service/ai_service/telemetry/store.py"
|
||||
- "apps/ai-service/ai_service/telemetry/ingest.py"
|
||||
- "apps/ai-service/ai_service/grading/store.py"
|
||||
- "apps/ai-service/ai_service/variants/store.py"
|
||||
- "apps/ai-service/scripts/**"
|
||||
- "apps/ai-service/package.json"
|
||||
- "apps/ai-service/tests/api/**"
|
||||
- "turbo.json"
|
||||
- "packages/types/employer.ts"
|
||||
```
|
||||
|
||||
### ai-engineer
|
||||
```yaml
|
||||
active: true
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: Owns the LLM provider layer, agent framework, prompt library, structured outputs, and the model-facing logic of v0.3 engines — trace-digest→rubric grading prompts (grading/features.py+engine.py), variant instantiation (variants/templates.py+generator.py), and the Examiner agent. Owns the deterministic-mock corpora.
|
||||
reason: v0.10 DEACTIVATED — no LLM-facing work; employer flows need no tagging/enrichment/embedding changes (publish re-tag + enrich-on-first-publish compose unchanged behind the employer routes).
|
||||
domain: ai
|
||||
frameworks:
|
||||
- pydantic
|
||||
@@ -127,116 +189,109 @@ frameworks:
|
||||
- pytest
|
||||
constraints:
|
||||
- provider-agnostic-protocol
|
||||
- prompts-are-code
|
||||
- json-defensive-parsing
|
||||
- never-call-cloud-in-tests
|
||||
- delta-passthrough
|
||||
- llm-sees-digest-not-raw-trace (D-028)
|
||||
- seeded-variant-reproducibility (D-029)
|
||||
- mock-first
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/llm/**"
|
||||
- "apps/ai-service/ai_service/agents/**"
|
||||
- "apps/ai-service/ai_service/marketplace/**"
|
||||
- "apps/ai-service/ai_service/prompts/**"
|
||||
- "apps/ai-service/ai_service/corpus/**"
|
||||
- "apps/ai-service/ai_service/grading/features.py"
|
||||
- "apps/ai-service/ai_service/grading/engine.py"
|
||||
- "apps/ai-service/ai_service/variants/templates.py"
|
||||
- "apps/ai-service/ai_service/variants/generator.py"
|
||||
- "apps/ai-service/tests/llm/**"
|
||||
- "apps/ai-service/tests/agents/**"
|
||||
```
|
||||
|
||||
### sandbox-engineer
|
||||
### data-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: true
|
||||
reason: v0.3 custom persona (RESEARCH) — owns the sandbox fabric: SandboxBackend protocol, unshare-based Linux user/mount/pid/net namespace spawner, per-sandbox workdir, resource limits, lifecycle manager, concurrency guard, and the in-sandbox capture agent. Probe-verified isolation on this box (D-024).
|
||||
domain: infra
|
||||
phase_specific: false
|
||||
reason: v0.10 ACTIVATED — owns the 8th D-027 store schema: employers/store.py with the 4-table family (employer_accounts: email UNIQUE + scrypt hash + employer_id UNIQUE-on-approval; employer_sessions: token_hash UNIQUE, expires_at = login + 7-day constant, lazy TTL; employer_claims: pending→approved|rejected + evidence JSON + decided_by/decided_at; employer_audit_events: the BillingEventRow audit shape), the employer_ prefix (bare `employer` is marketplace's table), check-first one-pending-per-entity/one-approved-owner constraints (the insert_application precedent), _validate/_normalize + _as_utc tz discipline, SQLModel.metadata.create_all idempotent compat (D-095), consume-free read paths for accounts/claims/audit lists
|
||||
domain: data
|
||||
frameworks:
|
||||
- python
|
||||
- linux-namespaces
|
||||
- asyncio
|
||||
- sqlmodel
|
||||
- pydantic
|
||||
- typescript
|
||||
constraints:
|
||||
- schema-first
|
||||
- type-safe
|
||||
- dual-schema-sync
|
||||
- idempotency-by-constraint (UniqueConstraint + check-first)
|
||||
- tokens-never-stored-raw (sha256 only, D-087)
|
||||
- audit-append-only (provenance rows never mutated)
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/employers/store.py"
|
||||
- "apps/ai-service/tests/employers/test_store.py"
|
||||
- "packages/types/**"
|
||||
```
|
||||
|
||||
### identity-engineer
|
||||
```yaml
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: v0.10 stays DEACTIVATED — the employer auth in this milestone is NOT the v0.5 learner identity module: identity/ is the KYC/age-gate store (IdentityRecord, derived age bands, require_verified_age gates for LEARNERS per D-043); v0.10's employer principals are a THIRD principal class (D-097 — no learner gates on employer routes, the D-043 composition on learner routes untouched). The identity module, its store, and its gates ship unchanged; backend-engineer owns the new employer auth surface. Reactivation is a founder call if/when a real KYC vendor lands or employer SSO/OIDC (explicitly out of scope at pilot) is scheduled.
|
||||
domain: identity
|
||||
frameworks:
|
||||
- fastapi
|
||||
- sqlmodel
|
||||
- pytest
|
||||
constraints:
|
||||
- isolation-verified (probe must show in-ns uid=0, network isolated, writes to workdir only)
|
||||
- backend-protocol-swap (no containerd assumption; D-024)
|
||||
- resource-limits-enforced (cpu/mem/time quotas observable)
|
||||
- no-daemon (subprocess-only; no docker/containerd service)
|
||||
- capacity-guard (1-5 concurrent; 503 when full, D-032)
|
||||
- pii-never-stored-raw
|
||||
- pii-never-logged
|
||||
- gate-composition-order
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/sandbox/**"
|
||||
- "apps/ai-service/scripts/sandbox-agent.py"
|
||||
- "apps/ai-service/tests/sandbox/**"
|
||||
- "apps/ai-service/ai_service/identity/**"
|
||||
- "apps/ai-service/tests/identity/**"
|
||||
```
|
||||
|
||||
### voice-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: true
|
||||
reason: v0.3 custom persona (RESEARCH) — owns the voice layer: VoiceProvider protocol, STT/TTS against a compatible endpoint, deterministic mock (tests never call a voice API), browser-native fallback, and the media-path wiring consumed by the Examiner agent and assessment UI.
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: v0.5 territory — voice shipped complete; no v0.10 voice work.
|
||||
domain: ai-media
|
||||
frameworks:
|
||||
- pydantic
|
||||
- httpx
|
||||
- fastapi
|
||||
- pytest
|
||||
- web-mediarecorder
|
||||
- react
|
||||
constraints:
|
||||
- provider-agnostic-protocol (D-030)
|
||||
- never-call-voice-api-in-tests
|
||||
- browser-native-fallback (no-key path still functions)
|
||||
- bounded-turn-latency (conversational feel budget)
|
||||
- provider-agnostic-protocol
|
||||
- never-call-cloud-in-tests
|
||||
- key-redaction
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/voice/**"
|
||||
- "apps/ai-service/tests/voice/**"
|
||||
- "apps/web/components/learner/defense-session.tsx"
|
||||
```
|
||||
|
||||
### design-system-engineer
|
||||
```yaml
|
||||
active: true
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: Owns the shared component library, design tokens, and visual consistency. v0.3 duty: new primitives for the real build/assessment surfaces (terminal frame, telemetry status indicator, mic/record control, grade badge, defense transcript viewer).
|
||||
reason: No design-token or primitive work in v0.10 (login/register pages reuse existing Input/Button/Card primitives); roster retained.
|
||||
domain: frontend
|
||||
frameworks:
|
||||
- tailwindcss
|
||||
- storybook
|
||||
- lucide-react
|
||||
constraints:
|
||||
- design-token-driven
|
||||
- wcag-aa-contrast
|
||||
- dark-mode-required
|
||||
- consistent-across-surfaces
|
||||
territory:
|
||||
- "packages/ui/**"
|
||||
```
|
||||
|
||||
### security-auditor
|
||||
```yaml
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: Identity/age-gating (KYC) deferred beyond v0.3 per founder directive (A-110) — no real auth or PII backend lands this milestone. Security coverage remains: verifier's STRIDE layer + Phase 7 secrets-hygiene checklist (keys absent from code/logs/commits/errors, localhost-only CORS, no PII in prompts). Sandbox isolation safety is owned by sandbox-engineer's probe-verified constraint.
|
||||
domain: security
|
||||
frameworks: []
|
||||
constraints: []
|
||||
territory: []
|
||||
```
|
||||
|
||||
## Phase-Specific Personas
|
||||
|
||||
| Persona | Phases | Removed After |
|
||||
|---------|--------|---------------|
|
||||
| sandbox-engineer | 1 (primary), 2, 6 | persists while sandbox fabric exists |
|
||||
| voice-engineer | 5 (primary), 6 | persists while voice defense exists |
|
||||
| security-auditor | 1 (auth surface: scrypt + session hashing + throttles + CORS + 422-scrub fix), 2 (claim evidence honesty + UNIQUE enforcement), 5 (employer mock-complete own-session + checkout review), 6 (final review: no credentials in logs/releases) | milestone complete |
|
||||
|
||||
All other active personas span the entire milestone. data-engineer and design-system-engineer are light-touch outside their phases.
|
||||
All other personas span the milestone. Deactivated personas receive no tasks.
|
||||
|
||||
## Territory Conflict Resolution
|
||||
|
||||
| Conflict | Resolution |
|
||||
|----------|------------|
|
||||
| frontend-engineer vs data-engineer (packages/types, packages/mock-data) | data-engineer owns type definitions and mock data schema (incl. new telemetry/grade/variant/defense TS types); frontend-engineer consumes them. |
|
||||
| frontend-engineer vs design-system-engineer (packages/ui) | design-system-engineer owns design tokens and primitive components (terminal frame, mic control, grade badge); frontend-engineer owns composite components and page-level UI. |
|
||||
| ai-engineer vs backend-engineer (grading/variants) | ai-engineer owns the model-facing files (features/engine/templates/generator = LLM logic + prompts); backend-engineer owns the persistence stores + API endpoints. Boundary: stores are pure SQLite; engine logic is pure compute. |
|
||||
| sandbox-engineer vs backend-engineer (sandbox/) | sandbox-engineer owns `ai_service/sandbox/**` + capture agent; backend-engineer owns the API route that composes `sandbox/manager.py` via DI. manager.py has a narrow typed interface consumed by api/. |
|
||||
| voice-engineer vs ai-engineer (Examiner agent) | ai-engineer owns `agents/examiner.py` + its prompt; voice-engineer owns `voice/**` (audio in/out). Examiner calls `voice/` through the `VoiceProvider` protocol — never imports concrete providers. |
|
||||
| ai-engineer vs data-engineer (mock duplication) | ai-engineer owns `ai_service/corpus/` (Python); data-engineer owns `packages/mock-data` (TS). Shared IDs/shapes aligned by convention (D-021). |
|
||||
| lead-developer vs any | lead-developer coordinates only, does not directly modify code files. |
|
||||
| backend-engineer vs frontend-engineer (engine-client.ts) | backend-engineer owns the API contract shapes the client consumes; frontend-engineer owns engine-client.ts itself (the employer section, SessionRequiredError, the session wrapper) — the shared `EntitlementResponse`/`PurchaseEntitlement` interfaces in engine-client.ts (lines 730-752) are a frozen v0.9 contract neither side may re-shape; the new employer wire types land in packages/types/employer.ts (data-engineer dual-schema-syncs, lead-developer reviews the pins) |
|
||||
| backend-engineer vs security-auditor (api/employers.py) | backend-engineer implements the dep/routes/throttles; security-auditor reviews and may patch security DEFECTS directly in api/employers*.py + employers/passwords.py (its territory) — e.g. the 422 password-echo scrub fix in main.py's pii_fields; implementation vs review stays with backend |
|
||||
| backend-engineer vs data-engineer (employers/store.py) | data-engineer owns the schema + store methods; backend-engineer owns routes/deps/wiring that compose it; passwords.py (KDF core) is backend territory, the password_hash COLUMN contract is data territory |
|
||||
| backend-engineer vs frontend-engineer (main.py CORS block) | backend-engineer owns the allow_headers edit at main.py:341; security-auditor owns the tests/api/test_cors.py preflight PIN (the E87 regression lesson is a security concern — the pin lives in the security review's changeset) |
|
||||
| frontend-engineer vs security-auditor (sessionStorage) | frontend-engineer implements the nc-employer-session wrapper; security-auditor reviews the exposure copy + logout-revoke discipline and pins the no-retry-loop locked state (the G-308 pattern); neither patches the other's files — findings flow through lead-developer |
|
||||
| lead-developer vs any | lead-developer coordinates only, does not directly modify code files |
|
||||
+168
-408
@@ -1,438 +1,198 @@
|
||||
# Nextcraft v0.3 — PLAN.md
|
||||
# Nextcraft — PLAN.md (v0.10 B2B Employer Network)
|
||||
|
||||
## Overview
|
||||
|
||||
This plan covers execution phases 1-6 of milestone v0.3 (Credential Engines): the real engines that replace v0.2's mock inputs — a namespace-isolated sandbox fabric, live build telemetry over WebSocket + SQLite, a process-trace grading engine, seeded per-learner variant task generation, and an oral/voice defense with a seventh Examiner agent — plus re-grounding the Lab/Assessor/Proctor agents onto real engine inputs and wiring the v0.1 learner surfaces to the real build/defense/grading paths. Phases are strictly sequential (P1→P6); within each phase, Wave 1 tasks are parallelizable (no cross-file dependencies) and later waves depend on earlier ones.
|
||||
**Milestone:** v0.10 — B2B Employer Network (Feature, D-085): real employer auth/accounts + employer self-serve — employers become principals: registered accounts with server-side sessions (scrypt-hashed credentials, hashed-at-rest session tokens), claim/verify of their existing v0.6 `EmployerRow` entity (admin-approved at pilot, evidence advisory), the employer dashboard converting from the v0.1 mockup to real data (own-postings lifecycle, real counts, honest mock-labels only where no backend exists), self-serve posting management composing the v0.9 entitlement gate unchanged (D-091 shared helper, 402 wire contract byte-stable), and self-serve checkout over the v0.9 billing catalog (buyer = session employer, superseding D-076; admin operator routes retained as fallback + oversight, D-094 network overview). Mock-first doctrine holds: no external auth provider, tests never call a cloud, zero new dependencies (R-66).
|
||||
**Tag line:** phase tags advance the v0.9.x line (P0 → v0.9.0 … P6 → v0.9.6 = milestone release), continuing from the v0.8.6 milestone release
|
||||
**Branch:** milestone/v0.10-b2b-employer-network
|
||||
|
||||
**Environment facts (apply throughout):** Python 3.11.2 via `python3 -m venv` (no uv, no system pip); pnpm 12.3.4 via corepack; turborepo; ai-service port **8420**; default model `gemma4:31b` (config via `AI_TUTOR_MODEL`); ollama-cloud base `https://ollama.com/v1` (OpenAI-compatible, Bearer auth); keys live only in gitignored `.ciagent/.env.secrets` (exported by `scripts/dev.sh`) — never in code, commits, or logs; all automated tests use the deterministic mock LLM and mock voice providers and **never call cloud or voice APIs**. New ai-service deps this milestone: `sqlmodel`, `sqlalchemy`, `websockets`, `aiofiles` (all PyPI-verified; added in Task 1-1-02). SQLite data (`apps/ai-service/ai_service/data/`) and sandbox dirs (`apps/ai-service/sandboxes/`) are already gitignored. **KYC/age-gating is deferred per founder directive (A-110)** — no identity work; age-gating stays the v0.1 visual flow mockup. **GRILL scope decisions (this revision):** real server STT/TTS (`OpenAIAudioProvider`) deferred to v0.4 — voice defense is mock+browser-first (CUT-1/G-7); the interactive xterm.js shell relay is deferred to v0.4 — the build panel is run/test-buttons + read-only exec output (CUT-2/G-8), so `@xterm/*` is NOT a v0.3 dependency; sandbox abuse control (per-learner caps + allowlist, G-5) ships even though KYC is deferred; sandbox resource limits are partially enforced (memory/CPU/wall-clock + workdir-size sweep; per-sandbox pids and hard disk quota are accepted gaps, G-1/G-2).
|
||||
**Phase list (requirement mapping):**
|
||||
|
||||
| Phase | Name | Requirements | Waves | Personas |
|
||||
|-------|------|-------------|-------|----------|
|
||||
| 1 | Sandbox fabric | REQ-3-001, 002 | 3 | sandbox-engineer, backend-engineer, ai-engineer (W1 lint only) |
|
||||
| 2 | Live build telemetry | REQ-3-003 | 4 | sandbox-engineer, backend-engineer, data-engineer |
|
||||
| 3 | Process-trace grading engine | REQ-3-004 | 3 | ai-engineer, backend-engineer |
|
||||
| 4 | Variant task generation | REQ-3-005 | 3 | ai-engineer, backend-engineer, data-engineer |
|
||||
| 5 | Oral / voice defense | REQ-3-006 | 4 | voice-engineer, ai-engineer, backend-engineer |
|
||||
| 6 | Agent re-grounding + learner surface integration | REQ-3-007, 008 | 5 | ai-engineer, frontend-engineer, design-system-engineer, data-engineer, backend-engineer, lead-developer |
|
||||
| Phase | Requirement | Focus |
|
||||
|-------|-------------|-------|
|
||||
| P1 | REQ-10-001 | Employer auth foundation: employers/ module (8th D-027 store + stdlib scrypt KDF + hashed session rows + both throttle windows), register/login/logout/me routes, main.py wiring (store + CORS Authorization + 422 scrub) — zero web changes |
|
||||
| P2 | REQ-10-002 | Employer entity claim: claim route + advisory evidence (stdlib domain match), one account : one entity (UNIQUE + check-first), admin claim review/approve/reject operator routes + audit |
|
||||
| P3 | REQ-10-003 | Employer account surface: own-postings read route, packages/types/employer.ts + dual-schema pins, engine-client employer section (SessionRequiredError + nc-employer-session), login/register pages, session-gated shell, postings/overview conversion (talent stays mock) |
|
||||
| P4 | REQ-10-004 | Self-serve posting management: D-091 shared consume_posting_entitlement helper + admin adoption (no drift), employer create/edit/publish/close routes, PostingManager write actions |
|
||||
| P5 | REQ-10-005 | Self-serve checkout + admin overview: employer checkout + entitlement view + own-session mock-complete through the REAL webhook path, pricing CTAs → employer-login path, admin network overview sections |
|
||||
| P6 | — | Final review + audit + milestone ship (v0.9.6 = milestone release) |
|
||||
|
||||
**Wave order (binding): principals before claims before surface before management before monetization.** P1 builds the module everything else composes (store + KDF + session gate — P2's claim route authenticates through it, P3's surface gates on it, P4/P5 parameterize it). P2 precedes P3 because the dashboard's claimed-entity profile and claim-pending states need decided claims to exist. P3 precedes P4 because the postings page conversion consumes the own-postings read route and the P4 mutations wire into the converted page. P4 precedes P5 because checkout's buyer = the session's claimed, managing entity and the upgrade CTA targets the surface P4 completed. Each phase is a vertical slice: store/schema → routes → web wiring → tests green in the same wave → ship.
|
||||
|
||||
**Binding contracts:** clarify decisions D-086..D-097 (PROJECT.md) and research conclusions R-55..R-66 (RESEARCH.md) are design contracts; this plan operationalizes them, never re-litigates. Module layout is decided (ARCHITECTURE v0.10): `ai_service/employers/{__init__,base,store,passwords,claim_evidence}.py` — no service.py; `api/employers.py` + `api/employers_admin.py`; `tests/employers/`.
|
||||
|
||||
**Environment facts (probe-verified, R-55..R-66):** scrypt is stdlib in the shipped venv (3.11.2), ~47 ms/hash at N=2^14/r=8/p=1, 16 MiB working set, default maxmem admits it — never pin maxmem below ~17 MiB (R-55); require_admin extract→verify→return shape at api/marketplace_admin.py:46-85, the 422 scrubber redacts only `date_of_birth` today (main.py:398-424, pii_fields line 401 — open-risk-3) (R-56); the 8th-store recipe fully specified by billing/store.py, bare `employer` table taken by marketplace/store.py:144 (R-57); create_posting gate order + the 402 dict's hardcoded posts_used=0/posts_limit=0 in BOTH arms (marketplace_admin.py:157-251) (R-58); billing_admin.create_checkout source mechanics incl. unique-pending placeholder + G-310 URLs (billing_admin.py:77-146) + the admin mock-complete path (152-227) (R-59); the four main.py wiring sites (lifespan ~182-190, CORS allow_headers line 341, inline router mounts 366-389, Settings-int knob shape) (R-60); web census — engine-client.ts section order (employer appends beside billing at ~717), adminToken() wrapper at 759-767, the pins.test.ts billing.ts block 205-231 + "Contact Sales" pin 250, NO login page in any route group (R-61); moderation page + entitlement-panel locked-state pattern (R-62); both throttle precedents (identity _SubmitRateLimiter 97-116, sandboxes _check_global_create_rate 123-137, reset-fixture pattern tests/marketplace/conftest.py:123-131) (R-63); tests/billing/ layout + state-injection make_client fixture (test_entitlement_enforcement.py:28-59) (R-64); seeded employer websites are example.com placeholders → evidence match=false corpus-wide, stdlib urlsplit suffix-match normalization (R-65); zero new deps, zero workspace changes, two .env.example knobs (R-66).
|
||||
|
||||
**Verification commands (phase gates — the repo's own scripts, R-66):**
|
||||
|
||||
- `pnpm lint` (turbo lint — eslint across all TS workspaces) and `pnpm ai:lint` (ruff on ai-service)
|
||||
- `pnpm typecheck` (turbo typecheck — tsc workspaces) and `pnpm ai:typecheck` (mypy — zero new overrides, the D-086 passlib rejection holds)
|
||||
- `pnpm ai:test:fast` (FAST=1 scripts/test.sh — `-m "not real_server and not userns"` per D-063; tests/employers/ joins the fast lane)
|
||||
- `pnpm web:test` (node:test incl. pins.test.ts) + `pnpm cli:test` (unchanged surface, regression guard)
|
||||
- `pnpm --filter web build` (static export, D-039 unchanged) on web-touching phases (P3, P4, P5)
|
||||
|
||||
**NFR note:** this is a Feature milestone — no purity gate; `feat:` commits are expected and scoped to ai_service/employers/, api/employers*.py + the D-091 helper, config.py/main.py wiring, tests/employers/, packages/types/employer.ts, the employer web surface, and the admin overview/pricing CTA surfaces.
|
||||
|
||||
**Secrets policy (binding):** no new secrets this milestone — AI_ADMIN_TOKEN stays env-only (D-014); employer passwords are user data: stored only as self-describing scrypt hashes (D-086), never logged, never returned, never echoed in 422s (pii_fields fix, R-56); session tokens stored ONLY as sha256(token) (D-087 — a DB-file leak yields nothing usable); tests use throwaway credentials against tmp_path DBs; no credential value appears in any committed file, log line, or release asset.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Sandbox Fabric
|
||||
## Phase 1: Employer Auth Foundation (REQ-10-001)
|
||||
|
||||
**Requirements:** REQ-3-001, REQ-3-002
|
||||
**Goal:** `SandboxBackend` protocol + `unshare`-based namespace spawner (D-024) + lifecycle manager with concurrency guard (D-032) + per-sandbox workdir; isolation and resources probe-verified on this box; `/v1/sandboxes` API live; deps + gitignore landed
|
||||
**Goal:** The employers module exists and is proven: 4-table store + stdlib scrypt KDF + hashed server-side sessions + register/login/logout/me routes + both throttle windows + main.py wiring — all tested with the REAL KDF, zero web changes, zero new dependencies.
|
||||
|
||||
### Wave 1: Foundations (parallel — no shared files)
|
||||
### Wave 1-1: Module core — store + KDF + contracts (backend-engineer + data-engineer, sequential)
|
||||
|
||||
#### Task 1-1-01: SandboxBackend protocol + unshare spawner + probe test
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-001, REQ-3-002
|
||||
- **Files:** `apps/ai-service/ai_service/sandbox/__init__.py`, `apps/ai-service/ai_service/sandbox/backend.py`, `apps/ai-service/ai_service/sandbox/workdir.py`, `apps/ai-service/ai_service/sandbox/unshare_backend.py`, `apps/ai-service/tests/sandbox/__init__.py`, `apps/ai-service/tests/sandbox/test_isolation.py`
|
||||
- **Action:** `backend.py`: `SandboxBackend` protocol + `SandboxSpec` (sandbox_id, learner_id, workdir, resource limits) + `SandboxHandle` (id, pid, workdir, created_at); `spawn(spec)`, `exec(handle, cmd)`, `snapshot(handle) -> Path`, `destroy(handle)`. `workdir.py`: per-sandbox layout under `apps/ai-service/sandboxes/<id>/` (workspace/ writable, snapshot() = recursive copy to `snapshots/<ts>/`) — no symlinks as the snapshot mechanism. `unshare_backend.py`: subprocess spawner — `unshare --user --map-root-user --mount --pid --fork --net` with the per-sandbox dir bind-mounted (`--bind <dir> /work`) and `chdir /work` (D-024); pipes for stdout/stderr; async wrappers. `test_isolation.py` — **re-verify box isolation properties (runs on this box, guarded by probe skip):** (a) `id -u` inside namespace prints `0`; (b) `ip link` inside namespace shows 0 usable interfaces (loopback-only/no carrier) — network isolated; (c) file written to `/work/inside.txt` lands at `sandboxes/<id>/workspace/inside.txt` on the host; (d) attempt to write outside the mount (e.g. host tmp path via bind) does not escape the per-sandbox dir; (e) `/proc` visibility degraded (proc-remount not permitted per A-101 — assert the probe documents this, not that it fails).
|
||||
- **Verify:** `pnpm ai:test` — `tests/sandbox/test_isolation.py` green on this box (probe-gated: skips with an explicit reason if userns unavailable); `lint` clean
|
||||
- **Task 1-1-01** (backend-engineer): `ai_service/employers/__init__.py` + `base.py` — pydantic contracts (D-093's list, dual-schema-pinned later in P3): `EmployerAccount` (id, email, employer_id nullable, created_at, `email_verified: Literal[False]` — UNVERIFIED per D-097, displayed as such), `EmployerClaim` (id, account_id, employer_id, status pending|approved|rejected, evidence dict, created_at, decided_by/decided_at), `RegisterEmployerRequest`/`LoginEmployerRequest` (email + password, honest validation), `TokenResponse` (token, expires_at), `EmployerMe` (account + claimed employer_id + claim state). No service.py (R-57) — routes compose the store via deps DI. (G-315: email fields are plain `str` with stdlib-only validation — non-empty, exactly one "@", non-empty domain, stripped + lowercased on normalize; pydantic `EmailStr` is BANNED — email-validator is not in the venv and adding it would be the first new dependency, breaking R-66/D-086.)
|
||||
- **Task 1-1-02** (data-engineer): `employers/store.py` — 8th D-027 store (R-57, D-095): 4 tables, all `employer_`-prefixed (bare `employer` is marketplace's EmployerRow, marketplace/store.py:144): `employer_accounts` (id PK `acct-`-style, email UNIQUE indexed, password_hash self-describing string, employer_id UNIQUE nullable-until-claim-approval, created_at), `employer_sessions` (id PK, token_hash UNIQUE — stores ONLY sha256(token), account_id indexed, created_at, expires_at = login + `SESSION_TTL_DAYS = 7` module constant), `employer_claims` (schema per D-089, built in P2 — table lands now per D-095's one-time create_all), `employer_audit_events` (auto id PK, account_id/employer_id nullable indexed, kind/name, detail JSON, operator, created_at — the BillingEventRow audit shape, billing/store.py:337-359). Mechanics mirror billing/store.py exactly: `_sqlite_connect` pragmas (WAL / synchronous=NORMAL / busy_timeout=5000 / foreign_keys=ON), `sa_type=String/JSON` portable columns, `_validate()`/`_normalize()` hooks, `_as_utc` tz re-labeling, Protocol `EmployerStore` + `SQLiteEmployerStore` pair, surfaced IntegrityError; `SESSION_TTL_DAYS` pinned by test (the PAST_DUE_GRACE_DAYS shape, D-087/D-083). Store methods: insert_account (check-first → (row, created)), get_account_by_email, mint_session (token_hash=sha256(token), expires_at=now+TTL, `now` injectable), get_session_by_token_hash (**lazy TTL — expired row reads as absent, G-306, no timer**), revoke_session (DELETE), append_audit_event, close. Tests: `tests/employers/test_store.py` — tmp_path DB, table-existence via sqlite_master (4 employer_ tables on the shared db_path), pragma pins, UNIQUE violations surfaced (duplicate email, duplicate token_hash), **token never stored raw (row holds sha256 only)**, lazy TTL with injected now (login+6d present, +8d absent — no sleeps), revoke → lookup miss.
|
||||
- **Task 1-1-03** (backend-engineer): `employers/passwords.py` — stdlib scrypt KDF (D-086, R-55): module constants `SCRYPT_N=2**14, SCRYPT_R=8, SCRYPT_P=1, SCRYPT_DKLEN=32` (**pinned by test**); per-password salt `secrets.token_bytes(16)`; stored self-describing `scrypt$16384$8$1$<salt_b64>$<hash_b64>` (params evolve without invalidating rows); verify parses params from the stored string, derives, compares via `hmac.compare_digest`; **no maxmem argument below ~17 MiB** (R-55 probe: default 32 MiB admits the 16 MiB working set; if pinned for explicitness use 33554432); module docstring records the ~50 ms/hash budget + throttle-bounded burst risk (open-risk-1). Tests: `tests/employers/test_passwords.py` — REAL KDF, no fake-hash path (D-096): hash→verify accept, wrong password reject, two hashes of the same password differ (random salt), params-pinned constants test, self-describing round-trip (old-format row with explicit params still verifies), **tests never assert hash bytes — accept/reject behavior only**.
|
||||
|
||||
#### Task 1-1-02: v0.3 dependencies + gitignore + config additions
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-001
|
||||
- **Files:** `apps/ai-service/pyproject.toml` (update), `apps/ai-service/ai_service/config.py` (update), `apps/ai-service/.env.example` (update), `apps/ai-service/scripts/bootstrap.sh` (update if needed), root `package.json` (no change), `turbo.json` (no change)
|
||||
- **Action:** Add pinned deps: `sqlmodel`, `sqlalchemy`, `websockets`, `aiofiles` to pyproject. `config.py` additions (env_prefix `AI_`): `AI_SANDBOX_DIR` (default `apps/ai-service/sandboxes`), `AI_SANDBOX_MAX_CONCURRENT` (default 5, D-032), `AI_SANDBOX_CPU_LIMIT` (default 1 core / cpu.max), `AI_SANDBOX_MEM_LIMIT_MB` (default 512), `AI_SANDBOX_PIDS_LIMIT` (default 256), `AI_SANDBOX_TIMEOUT_S` (default 1800), `AI_DB_PATH` (default `ai_service/data/nextcraft.db`), `AI_VOICE_BASE_URL` / `AI_VOICE_API_KEY` / `AI_VOICE_STT_MODEL` / `AI_VOICE_TTS_MODEL` (all **optional**, default empty — mock-first, D-030; documented in `.env.example` and README). Confirm `.gitignore` already covers `ai_service/data/` + `sandboxes/` (it does — v0.3 block present). Re-run bootstrap idempotently to install new deps.
|
||||
- **Verify:** `pnpm ai:bootstrap` re-installs cleanly (no-op venv, new wheels land); `python -c "import sqlmodel, sqlalchemy, websockets, aiofiles"` succeeds in the venv; settings parse with new keys unset
|
||||
### Wave 1-2: Auth routes + throttles + main.py wiring (backend-engineer)
|
||||
|
||||
#### Task 1-1-03: Resource-limit probe documentation + harness ruff pass
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-001
|
||||
- **Files:** `apps/ai-service/README.md` (update: sandbox section + probe transcript), `apps/ai-service/pyproject.toml` (no change), `apps/ai-service/tests/sandbox/test_isolation.py` (no change)
|
||||
- **Action:** Record the A-101 probe transcript in README (verbatim commands + observed output from this box: `unshare --user --map-root-user --mount --pid --fork --net id -u` → `0`; `ip link` → loopback only; write containment). Document the v0.3 resource-limit mechanism choice: cgroup-v2 delegation via per-sandbox scope files is **not available** on this box without sudo → enforcement = subprocess-level (`ulimit`-equivalent via `preexec_fn`: RLIMIT_AS for memory, RLIMIT_CPU for CPU-seconds, RLIMIT_NPROC for pids) + hard wall-clock timeout kill in the manager. This is the locked v0.3 mechanism (D-024 + no-sudo constraint). Run ruff over the new tree; fix all findings.
|
||||
- **Verify:** `pnpm ai:lint` exits 0; README shows the probe transcript and the rlimit mechanism note
|
||||
- **Task 1-2-01** (backend-engineer): `api/employers.py` — `require_employer_session` dependency (D-090, R-56: the require_admin extract→verify→return shape, single-header `Authorization: Bearer <token>` via partition + scheme check; sha256(token) hash-lookup; **401** missing/expired/unknown — credential failure; **403** reserved for cross-entity authorization; returns session + account + claimed employer_id; token namespaces cannot collide with admin tokens — one class fails the other's gate) + `deps.py` `get_employer_store` (app.state DI). Routes: `POST /v1/employers/auth/register` (email+password → scrypt hash → insert_account; duplicate email → 409 honest; account UNVERIFIED per D-097 — no learner gates ever composed on employer routes), `POST /v1/employers/auth/login` (verify → mint token → `{token, expires_at}`; bad credentials → 401 honest, no user-enumeration difference in response body beyond the 401), `POST /v1/employers/auth/logout` (revoke: DELETE row + audit event; unknown/revoked token → 401), `GET /v1/employers/auth/me` (session + account + claimed employer_id + unverified email displayed as such). Tests: `tests/employers/test_auth.py` — **register→login→me→logout flow green**; duplicate register 409; wrong password 401; logout revokes (subsequent me → 401); expired token → 401 (injected now); no-token → 401; concurrent sessions per account allowed (two logins, both me-green, one logout leaves the other alive). REAL scrypt throughout (≈2 hashes ≈100 ms/test, R-64 budget).
|
||||
- **Task 1-2-02** (backend-engineer): throttles (D-088, R-63 — both copy the proven in-tree mechanics, keyed by SUBMITTED EMAIL, never client IP): (a) per-EMAIL failed-login window — module dict[email]→timestamps, `time.monotonic()` slice-trim, **default 5 failures / rolling 15 min (900 s, window seconds parameterized — identity hardcodes 60.0) → 429 with the honest "wait" detail**; failures-only append; **successful login CLEARS the email's window**; (b) global per-min cap on login+register attempts — module `deque[float]`, while-popleft older-than-60s, `len >= settings.employer_auth_attempts_per_min` → 429, **append only after ALL checks pass (a rejected attempt never consumes budget)**. Settings knobs as plain ints (the `identity_submits_per_min` shape, R-60d): `employer_login_failures_per_window: int = 5`, `employer_auth_attempts_per_min: int = 30`; `.env.example` entries beside the AI_ADMIN_TOKEN block (commented, default-off, R-66). Tests: `tests/employers/test_throttle.py` — 5 failures → 429 with wait detail; success clears the email window; global cap → 429; rejected attempts don't consume the global budget; **both limiter module states reset via fixtures (the _reset_apply_rate_window pattern, R-63)**. (G-313: an autouse fixture in a NEW tests/employers/conftest.py clears BOTH limiter module states before/after EVERY employers test — the whole suite runs inside one 60s window and self-429s otherwise; the sanctioned deviation from billing's no-conftest layout, R-64. G-314: the global deque appends every well-formed login+register attempt — FAILED PASSWORDS COUNT; only 422-invalid bodies and the cap's own 429s are exempt, or D-088's credential-stuffing bound is void; the "rejected attempts consume nothing" test is reworded to "422-invalid consumes nothing; failed-password attempts DO count; N distinct emails inside the window → the (cap+1)-th 429s — the stuffing test". G-320a: the login route order is body-validation → global cap → per-email window → scrypt verify → failure-append — a 429'd attempt performs ZERO KDF derivations, pinned.)
|
||||
- **Task 1-2-03** (backend-engineer): main.py wiring (R-60, all four sites): (a) employer_store lifespan block directly after billing_store (~line 191, same getattr-guard) + `close()` in the finally block; (b) employers router mount beside the billing block (inline-import pattern); (c) **CORS `allow_headers` gains `"Authorization"` (main.py:341) + the `tests/api/test_cors.py` preflight pin lands in THIS task** (the E87 lesson applied before it ships — open-risk-2: without it every cross-origin employer fetch dies preflight in the two-port dev + D-038 network-mode topologies); (d) **pii_fields gains `"password"` + `"email"`** (main.py:401, R-56/open-risk-3 — a pydantic 422 on register would echo the raw password into err["input"] today) + the pin: register with an invalid body → 422 whose detail never contains the submitted password (added to tests/employers/test_auth.py). Tests: test_cors.py Authorization-preflight green; 422-echo pin green.
|
||||
|
||||
### Wave 2: Lifecycle manager (depends on Wave 1)
|
||||
### Wave 1-3: P1 security review + phase gate (security-auditor + lead-developer)
|
||||
|
||||
#### Task 1-2-01: Sandbox manager + concurrency guard + snapshots
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-001, REQ-3-002
|
||||
- **Files:** `apps/ai-service/ai_service/sandbox/manager.py`, `apps/ai-service/tests/sandbox/test_manager.py`
|
||||
- **Action:** `SandboxManager`: `create(learner_id) -> SandboxHandle` (guard: active count ≥ `AI_SANDBOX_MAX_CONCURRENT` → raise `PoolFullError` → API maps to **503**, D-032; no queue); `list() -> list[SandboxHandle]`; `get(id)`; `snapshot(id) -> Path` (delegates to workdir); `destroy(id)` (kill process tree, keep or purge workdir per flag); `reap_expired()` background hook for `AI_SANDBOX_TIMEOUT_S` which also performs a **workdir-size sweep**: any sandbox whose `workdir` exceeds `AI_SANDBOX_MAX_WORKDIR_MB` (new config, default 512) is snapshotted-then-destroyed and the event logged as an integrity signal (G-2 — soft disk cap, best-effort, not kernel-enforced); the sweep runs on the same timer as the timeout reaper. Enforce rlimits per spawner (Task 1-1-03: RLIMIT_AS + RLIMIT_CPU + RLIMIT_FSIZE=50MB as a cheap single-file disk guard (a-2); RLIMIT_NPROC noted as shared-per-host-uid, not relied on (G-1)) at exec time. Handle registry persisted **in-memory** (v0.3, single process; not a store — see D-019 precedent) with a clear note that handles are process-local. Startup reaper (a-1): on lifespan boot, scan `AI_SANDBOX_DIR` for workdirs whose recorded pid is dead and reap them, logging a warning. Narrow typed interface only — manager never imports api/ (boundary rule).
|
||||
- **Verify:** `pnpm ai:test` — test_manager covers create/list/destroy/snapshot, 6th create raises PoolFullError (503 path), destroy kills the namespace process (pid gone), snapshot dir exists with workspace contents, timeout reaper removes a stale handle
|
||||
- **Task 1-3-01** (security-auditor): auth-core review — scrypt params/compare_digest discipline (constants pinned, self-describing rows), session tokens hashed-at-rest (sha256 only — DB leak yields nothing, D-087), throttle windows (email-keyed never IP, both knobs honored), 401 vs 403 semantics (D-090), CORS Authorization pin present, 422 scrub pinned, revoke-on-logout. Findings recorded; P0s fixed in-phase.
|
||||
- **Task 1-3-02** (lead-developer): P1 phase gate — `pnpm ai:lint` + `pnpm ai:typecheck` (zero new mypy overrides) + `pnpm ai:test:fast` + `pnpm web:test` + `pnpm cli:test` green (existing suites unaffected — the module is additive, marketplace/billing/identity untouched); D-097 discipline check (no require_verified_age/allowlist composed on any employer route); test counts recorded.
|
||||
|
||||
#### Task 1-2-02: Resource-limit enforcement test
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-002
|
||||
- **Files:** `apps/ai-service/tests/sandbox/test_resource_limits.py`
|
||||
- **Action:** Concrete enforcement probes (guarded like isolation tests): (a) spawn a process that allocates > `RLIMIT_AS` → assert it dies with MemoryError/killed within a bound; (b) spawn a CPU spinner past `RLIMIT_CPU` → assert SIGXCPU/kill; (c) single huge file > `RLIMIT_FSIZE` → assert write failure (a-2 partial disk guard); (d) wall-clock: spawn `sleep 9999` with a small manager timeout → reaper destroys it; (e) **disk sweep (G-2)**: write > `AI_SANDBOX_MAX_WORKDIR_MB` across many files → assert the manager sweep destroys the sandbox and logs the integrity signal. Assert limits are observable (handle reports its limit set). NOTE (G-1): per-sandbox `RLIMIT_NPROC` is shared at the host uid — the fork-bomb probe is documented as shared-budget behavior, NOT asserted as per-sandbox isolation.
|
||||
- **Verify:** `pnpm ai:test` — test_resource_limits green; limits proven enforced and observable
|
||||
## Phase 2: Employer Entity Claim (REQ-10-002)
|
||||
|
||||
### Wave 3: API exposure (depends on Wave 2)
|
||||
**Goal:** An authenticated account claims an existing EmployerRow entity; claims are pending with recorded advisory evidence; admin approve/reject activates ownership (one account : one entity); the operator queue and audit trail are real.
|
||||
|
||||
#### Task 1-3-01: Sandboxes API module
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-001, REQ-3-002
|
||||
- **Files:** `apps/ai-service/ai_service/api/sandboxes.py`, `apps/ai-service/ai_service/main.py` (update: include router + lifespan manager), `apps/ai-service/ai_service/api/deps.py` (update), `apps/ai-service/tests/api/test_sandboxes.py`
|
||||
- **Action:** DI exposes a singleton `SandboxManager`. Endpoints: `POST /v1/sandboxes {learner_id}` → 201 handle (503 when pool full); `GET /v1/sandboxes` → list; `GET /v1/sandboxes/{id}` → handle; `POST /v1/sandboxes/{id}/snapshot` → snapshot path; `DELETE /v1/sandboxes/{id}` → 204. All behind localhost CORS (A-008). Lifespan creates/destroys the manager; on shutdown destroys any live sandboxes (no orphans). **Abuse control (G-5, NOT KYC):** even in no-auth v0.3 the sandbox API enforces per-`learner_id` rate limiting (`AI_SANDBOX_MAX_PER_LEARNER`, default 1 active → 429) and a global create-rate cap (`AI_SANDBOX_CREATES_PER_MIN`, default 10 → 429); `learner_id` is validated against a server-side allowlist from config (`AI_LEARNER_ALLOWLIST`, default the single mock pilot id → unknown ids rejected 403). This ships in the no-auth milestone so a rogue local process can't exhaust shared NPROC/disk.
|
||||
- **Verify:** `pnpm ai:test` — test_sandboxes green (create→list→snapshot→delete roundtrip via TestClient; 6th create → 503; delete of unknown id → 404; abuse control (G-5): non-allowlisted learner_id → 403; >1 active sandbox for one learner → 429; burst of >10 creates/min → 429); manual probe: `curl -X POST localhost:8420/v1/sandboxes -d '{"learner_id":"l1"}'` returns a handle id
|
||||
### Wave 2-1: Claim core — evidence + store + route (backend-engineer + data-engineer)
|
||||
|
||||
### Must-Haves (Phase 1)
|
||||
- [ ] Isolation probe test green on this box: in-namespace uid=0, network isolated (0 usable interfaces), host writes confined to the per-sandbox bind dir (A-101 re-verified as an automated test, not just research notes)
|
||||
- [ ] Resource limits enforced + observable: memory (RLIMIT_AS) + CPU (RLIMIT_CPU) rlimits kill violating processes; wall-clock reaper destroys stale sandboxes; disk usage capped by a periodic workdir-size sweep in the manager (soft cap, configurable `AI_SANDBOX_MAX_WORKDIR_MB`, default 512MB — NOT kernel-enforced); per-sandbox NPROC is shared across sandboxes at the host uid — documented, not relied on for isolation (G-1, G-2 — test_resource_limits green)
|
||||
- [ ] No cross-tenant access: sandbox A cannot read sandbox B's workdir (isolation test asserts containment)
|
||||
- [ ] Lifecycle API works end-to-end: create/list/snapshot/destroy via TestClient; pool full → **503** (D-032, no queue)
|
||||
- [ ] Snapshot produces a restorable directory copy under the sandbox's own snapshots/ dir
|
||||
- [ ] `pnpm ai:test` and `pnpm ai:lint` green; new deps (`sqlmodel`, `sqlalchemy`, `websockets`, `aiofiles`) installed via idempotent bootstrap
|
||||
- [ ] Boundary rules hold: `sandbox/` imports nothing from `api/` or `agents/`; only `api/sandboxes.py` composes the manager via DI
|
||||
- [ ] No docker/podman/sudo anywhere in the spawner path (D-024); `SandboxBackend` protocol is the only coupling to the spawner (containerd swap possible later)
|
||||
- **Task 2-1-01** (backend-engineer): `employers/claim_evidence.py` — stdlib registrable-domain normalization (R-65): `urllib.parse.urlsplit(website).hostname` lowercased ("https://example.com/openai" → "example.com"), email domain after the last "@" lowercased, `match` = simple SUFFIX match (equal, or one endswith "." + other) — no PSL library, no tldextract (zero-dep discipline); module docstring records the multi-label-suffix over-match as acceptable for an advisory signal; evidence shape per D-089: `{account_email_domain, entity_website_domain, match}`. Tests: `tests/employers/test_claim_evidence.py` — normalization table (scheme/path stripped, case-folded), match true on same registrable domain, false on the example.com corpus (R-65).
|
||||
- **Task 2-1-02** (data-engineer): store claim methods (the insert_application check-first precedent, marketplace/store.py:341-347): `insert_claim` (**one PENDING claim per entity → created=False/409; one APPROVED owner per entity; rejection frees the entity for a new claim**), `decide_claim` (approve → set `employer_accounts.employer_id` — the UNIQUE one-account-one-entity constraint — + decided_by/decided_at + audit append; reject → decided_by/decided_at + audit append), `list_claims` (status filter, consume-free read path for the admin queue), `get_claim`. Tests: `tests/employers/test_claims_store.py` — second pending claim on an entity rejected; claim after rejection succeeds; approve sets employer_id + writes an audit row; second approve by another account → IntegrityError surfaced (one approved owner); decided_at/decided_at provenance pinned. (G-316: the pending-arm uniqueness is TOCTOU on check-first alone — `employer_claims.__table_args__` gains a partial UNIQUE index `uq_pending_claim_per_entity` on employer_id `sqlite_where status='pending'` (SQLite partial index, emitted by create_all — the D-095 one-time path); insert_claim keeps the check-first 409 fast-path and SURFACES the IntegrityError as the same honest 409 under race; the approved-owner arm is already structurally sound via employer_accounts.employer_id UNIQUE. Pinned: the partial index exists in sqlite_master; a raced second pending insert → 409, never a duplicate pending row.)
|
||||
- **Task 2-1-03** (backend-engineer): `POST /v1/employers/claims {employer_id}` (session-gated via require_employer_session): entity must exist in the marketplace store (404 unknown — claims target the v0.6 ingested EmployerRow data), account must not already hold an entity (409 honest — one account : one entity), evidence computed AT CLAIM TIME + stored as provenance JSON on the claim row, response = pending claim with evidence. **Auto-approval is deferred: the operator decision activates (D-089 — without email verification the match is forgeable; A-304: never present unverified as verified).** Tests: `tests/employers/test_claims.py` — claim → pending with the exact evidence dict; unknown entity 404; already-owner 409; second pending claim on the same entity 409; unauthenticated 401.
|
||||
|
||||
### Wave 2-2: Admin claim review routes + security review (backend-engineer + security-auditor)
|
||||
|
||||
- **Task 2-2-01** (backend-engineer): `api/employers_admin.py` — `require_admin` imported from marketplace_admin (the billing_admin.py:27 precedent, D-094): `GET /v1/employers/admin/accounts` (email, claimed entity, created_at, unverified marker), `GET /v1/employers/admin/claims` (queue: status, evidence, domain-match signal, decided state), `POST /v1/employers/admin/claims/{id}/approve`, `POST .../claims/{id}/reject` (set decided_by/decided_at + append employer_audit_events per D-089), `GET /v1/employers/admin/audit` (queryable trail). Fail-closed 403 on no/missing/invalid token (the R-56 matrix). Tests: `tests/employers/test_claims_admin.py` — **approve → account.employer_id set + audit row + claim approved**; reject frees the entity (a new claim from another account succeeds); decided twice → 409 honest; 403 fail-closed on all five routes; audit query returns claim decisions with operator provenance.
|
||||
- **Task 2-2-02** (security-auditor): claim-evidence review — evidence is advisory, never displayed as verification (A-304; the example.com corpus makes match=false corpus-wide, R-65/open-risk-6); one-account-one-entity UNIQUE enforced at the store (approve path surfaces IntegrityError, never silently overwrites); claim decisions audit-complete (decided_by/decided_at + events); the 409 paths don't leak account existence beyond the authenticated principal. Findings recorded; P0s fixed in-phase.
|
||||
|
||||
## Phase 3: Employer Account Surface (REQ-10-003)
|
||||
|
||||
**Goal:** The employer surface is real: own-postings read route feeding a converted dashboard (login/register pages, session-gated shell, own-postings lifecycle view, real counts), wire types dual-schema-pinned, 401 → honest logged-out state — talent pages stay mock, honestly labeled.
|
||||
|
||||
### Wave 3-1: Own-postings read route (backend-engineer)
|
||||
|
||||
- **Task 3-1-01** (backend-engineer): `GET /v1/employers/postings` (session-gated; **unclaimed account → 403 + claim-CTA detail** — the D-090 403 semantics; claimed → own postings only, resolved from the marketplace store by the session's employer_id, status facets draft|published|closed, **per-posting applicant_count from the applications table** — the D-092 overview data source). Reuses marketplace store via deps DI; no new tables. Tests: `tests/employers/test_postings_read.py` — own-entity rows only (a second claimed employer's postings never appear); unclaimed 403 with claim-CTA; status facets pinned (counts per status); applicant_count matches seeded applications; no-token 401. (G-319: the response is a NAMED pydantic contract in employers/base.py — `OwnPostingSummary` (PostingSummary reused + applicant_count) + the facets/list response — so the MH-3a dual-schema pin is schema↔schema, not a prose payload; the counts ride a new in-pattern marketplace store read `count_applications_by_posting(posting_ids) -> dict[str, int]`, never a per-posting N+1; employer-facing responses expose applicant COUNTS ONLY — learner identifiers NEVER appear in any /v1/employers/* response, PII boundary, security-auditor P3 scope.)
|
||||
|
||||
### Wave 3-2: Wire types + engine-client employer section (frontend-engineer)
|
||||
|
||||
- **Task 3-2-01** (frontend-engineer): `packages/types/employer.ts` (the billing.ts precedent, R-61/R-66) — EmployerAccount, EmployerClaim (+evidence), token/me shapes, own-postings list response (PostingSummary REUSED from marketplace.ts:99-117, never duplicated; entitlement view reuses the v0.9 shapes per D-093) + `export * from './employer'` in index.ts. Dual-schema pins in `apps/web/tests/pins.test.ts` (G-108 discipline; the billing.ts pin block 205-231 is the template): **the employer.ts pin block asserts every field field-for-field against ai_service/employers/base.py's pydantic contracts — both schemas**.
|
||||
- **Task 3-2-02** (frontend-engineer): `engine-client.ts` employer section (appended beside billing, R-61 — ONE client module, no fork): `registerEmployer`/`loginEmployer`/`logoutEmployer`/`getEmployerMe`/`claimEmployerEntity`/`listOwnEmployerPostings` + the `nc-employer-session` sessionStorage wrapper (the adminToken() 759-767 pattern: window-guarded get/set/clear; tab-scoped bounds XSS exposure — open-risk-4, documented in surface copy) + **`SessionRequiredError` — the 401 discrimination arm joining the parseError chain (no 401 arm exists today, R-61); 401 → honest logged-out state, NEVER a retry loop** + logout clears storage AND calls the revoke route + AbortController on every fetch (A-406). Tests: web fetch-stub tests (the billing-entitlement.test.ts template) — 401 → SessionRequiredError; token attached as Authorization header on every employer fetch; logout clears storage + fires revoke.
|
||||
|
||||
### Wave 3-3: Pages — login/register + dashboard conversion (frontend-engineer)
|
||||
|
||||
- **Task 3-3-01** (frontend-engineer): `/employer/login` + `/employer/register` pages (greenfield client pages, R-61 — no login page exists in any route group; existing Input/Button/Card primitives, no design-token work): honest states throughout — 401 invalid-credentials, 409 duplicate email, 429 throttle wait, 422 validation (never echoing the password — server-side pinned already); register success → logged-in dashboard; login stores the session token. Tests: fetch-stub render tests — loading/error/empty honest, 429 copy, register→login handoff. (G-312: fetch-stub + source-pin discipline — no render harness, per D-059.)
|
||||
- **Task 3-3-02** (frontend-engineer): employer dashboard conversion: postings page drops the `jobs` mock import + PostingManager's derived mock statuses/applicants (components/employer/posting-manager.tsx:29-37) → own-postings read lifecycle view (draft → published → closed visible, applicant counts real; write actions land P4); overview page drops `candidates, employers` mock imports + inline METRICS/PIPELINE constants (page.tsx:10, 18-74) → real active-postings count + per-posting applicant counts from listOwnEmployerPostings + claimed-entity profile via the existing public GET /v1/marketplace/employers/{id} (enrichment retained); **talent-match/placement metrics STAY mock-labeled (D-092 — no backend computes them); talent search + candidate profile pages keep their mock imports unchanged (no candidate backend exists — honest out-of-scope)**. Session-gated shell: no/expired session → honest logged-out state with login CTA (SessionRequiredError path), never a retry loop; claim-pending state visible (claim submitted, awaiting review). Tests: web render tests — logged-out state, claim-pending state, postings lifecycle render, real counts; **pins: postings/overview pages grep-clean of the removed mock imports; talent pages' mock imports retained**; `pnpm --filter web build` (static export) green. (G-312: "render tests" = the D-059/E87 discipline — fetch-stub tests over engine-client employer functions + SOURCE pins for component honest states (logged-out/claim-pending/loading/error branches present in source, the pins.test.ts discipline); NO jsdom/happy-dom render harness exists in-tree and none may be added this milestone.)
|
||||
|
||||
## Phase 4: Self-Serve Posting Management (REQ-10-004)
|
||||
|
||||
**Goal:** Employers manage their own postings end-to-end: the v0.9 entitlement gate extracted ONCE into a shared api/ helper, both principal gates composing it with byte-identical wire contracts; employer create/edit/publish/close routes; the converted PostingManager gains write actions with honest 402/403 states.
|
||||
|
||||
### Wave 4-1: Shared entitlement helper + employer posting routes (backend-engineer + lead-developer)
|
||||
|
||||
- **Task 4-1-01** (backend-engineer): the D-091 extraction: `consume_posting_entitlement(store, employer_id, upgrade_cta)` in api/ beside both callers (no billing/→api/ boundary violation) — extracted from marketplace_admin.create_posting lines 181-220; **canonical 402 dict `{reason, posts_used: 0, posts_limit: 0, upgrade_cta}` — posts_used/posts_limit hardcoded 0 in BOTH arms preserved EXACTLY (R-58 — the wire contract; UpgradeRequiredError consumes reason + upgrade_cta unchanged, D-080/D-084)**; store-missing → 503 fail-closed (the getattr guard); **gate runs BEFORE insert — a 402 consumes nothing, creates nothing**; `consume_posting` itself (G-304 single conditional UPDATE) UNTOUCHED. marketplace_admin.create_posting adopts the helper with the admin-checkout CTA — behavior byte-identical. Tests: **tests/billing/test_entitlement_enforcement.py green UNCHANGED (the no-drift proof)** + a new payload-parity test: admin-gate 402 vs employer-gate 402 assert equal reason/posts_used/posts_limit with only upgrade_cta differing per principal.
|
||||
- **Task 4-1-02** (backend-engineer): employer posting write routes (session gate → claim check → entitlement → insert, mirroring the "admin gate still first" composition, R-58): `POST /v1/employers/postings` (create draft — `consume_posting_entitlement(billing_store, session.employer_id, employer-CTA)`; 403 + claim-CTA unclaimed), `PATCH /v1/employers/postings/{id}` (**own drafts only — published/closed → 409 honest; another entity's posting → 403, the D-090 authorization arm; unknown → 404**), `POST .../postings/{id}/publish` (own draft → published + posted_at; the tagging/enrichment composition behind publish unchanged), `POST .../postings/{id}/close` (own posting → closed). Admin posting routes remain untouched (operator path unchanged, provenance-recorded). Tests: `tests/employers/test_postings_write.py` — create consumes from the employer's OWN package (posts_used+1 pinned); 402 both arms via direct store grants (the grant_posts precedent, test_entitlement_enforcement.py:70-85 — no package → `no_active_package`, exhausted → `posting_limit_reached`); **402 consumes nothing (gate-before-insert pinned: failed create leaves posts_used unchanged and no row)**; cross-entity 403; edit published → 409; publish/close lifecycle + applicant counts accrue; unlimited/enterprise manual grants bypass. (G-311: the session/claim gates compose BEFORE the entitlement check — unauthenticated → 401, unclaimed → 403, never a 402 (the R-58 admin-gate-first order; REQ-10-004's "AFTER" wording is corrected by this grill). G-317: employer create stamps its OWN provenance — `source="employer"` with `emp-` id/external_id schemes, never the admin route's `source="admin"` (authorship must stay answerable across seeded/admin/employer classes); the admin routes are untouched. G-318: employer edit composes the IDENTICAL update_posting_fields + tag_posting re-tag path as the admin edit (D-052 — tags are derived, overwritten on edit; tag failure honest, edit stands) with the own-drafts guard evaluated BEFORE any mutation — edit published/closed → 409 with no field/tag side effects. Pinned: employer-created posting carries the employer source stamp; employer edit re-tags on a mock provider; edit published → 409 AND nothing mutated.)
|
||||
- **Task 4-1-03** (lead-developer): D-091 no-drift review — both principal gates compose the SAME helper (source-level: no duplicated gate logic in either route); the 402 payload-parity test green; v0.9 billing/marketplace_admin suites green unmodified; wire contract (UpgradeRequiredError) unchanged.
|
||||
|
||||
### Wave 4-2: PostingManager write actions (frontend-engineer)
|
||||
|
||||
- **Task 4-2-01** (frontend-engineer): PostingManager gains the write actions over the engine-client employer section: create draft, edit own draft, publish, close — honest states: **402 → UpgradeRequiredError renders the upgrade CTA with the correct reason (the CTA routes to the employer billing surface, P5)**; 403 claim-CTA renders the claim prompt; 409/404 honest; optimistic-free, loading/error/empty per component state; AbortController everywhere. Tests: web fetch-stub — 402 CTA render with reason, publish/close dispatch, own-draft-only error copy, claim-prompt on 403. (G-320b: the 402 branch renders reason + upgrade CTA ONLY — the payload's posts_used/posts_limit are byte-stable zeros (D-091/MH-4a) and are NEVER rendered as limits; real limits render exclusively from GET /v1/employers/billing/entitlement; pinned by source pin.)
|
||||
|
||||
## Phase 5: Self-Serve Checkout + Admin Overview (REQ-10-005)
|
||||
|
||||
**Goal:** The B2B loop closes: employer-principal checkout over the v0.9 catalog through the REAL webhook path (buyer = session's claimed entity), own entitlement view, pricing CTAs → employer-login path, and the admin moderation page gains the network overview (accounts, claim queue, audit).
|
||||
|
||||
### Wave 5-1: Employer checkout + entitlement view + mock-complete (backend-engineer)
|
||||
|
||||
- **Task 5-1-01** (backend-engineer): `POST /v1/employers/billing/checkout {item_id}` (R-59 verbatim transpose of billing_admin.create_checkout — only principal derivation + metadata provenance change): session gate → **unclaimed → 403 + claim-CTA** → catalog lookup 404 → mode = subscription iff kind==plan → success/cancel URLs from `settings.public_base_url` (G-310) → `insert_checkout_session` FIRST with unique pending placeholder (`pending-{uuid4().hex[:16]}`) → CheckoutSessionSpec with metadata employer_id = claimed entity, operator_ref → the account's provenance (D-076's "operator" becomes account provenance) → provider.create_checkout_session via app.state.payment_provider → bind_provider_session_id → CheckoutResponse. Plus `GET /v1/employers/billing/entitlement` (own purchases/subscriptions: current package, posts used/limit, expiry — honest states; the v0.9 shapes reused). The webhook route + _grant_entitlement serve employer checkouts verbatim (G-303 per-session exactly-once holds). Tests: `tests/employers/test_checkout.py` — unclaimed 403 claim-CTA; 404 unknown item; 201 shape + persisted row; metadata provenance pinned (employer_id = claimed entity, account ref recorded); entitlement view honest states (none/active/expired).
|
||||
- **Task 5-1-02** (backend-engineer): employer mock-complete arm (D-082 transposed, R-59): `POST /v1/employers/billing/mock/complete` mounted ONLY under provider=mock (factory-conditional), **employer-SESSION-gated, completing one's OWN session — cross-session completion → 403 (the G-301 hole class, open-risk-5); provider_session_id resolved from LOCAL rows by the session's own checkout rows (G-302 — payloads are untrusted ID references)**; mints the signed event through the REAL webhook path (mint_event → process_webhook_event, direct handler call — the D-082 proof shape); 404 under stripe (test-pinned). Tests: `tests/employers/test_checkout_e2e.py` — **E2E: register → login → claim → admin approve → checkout → own mock-complete → webhook → purchase exists with posts available**; cross-session completion → 403; stripe-mode mount → 404; duplicate complete falls through already-complete (no double grant, G-303).
|
||||
|
||||
### Wave 5-2: Employer billing surface + pricing CTAs + admin network overview (frontend-engineer)
|
||||
|
||||
- **Task 5-2-01** (frontend-engineer): employer billing surface + pricing CTA conversion: the dashboard gains the entitlement view (current package, posts used/limit bar, expiry — honest empty/expired states) + buy flow (catalog from the public `/v1/billing/catalog`, own-checkout trigger, **in-app mock-complete confirmation honestly labeled as the pilot mock path**); **pricing-client.tsx CTAs convert from contact-path to employer-login path (superseding D-079) — the "Contact Sales" enterprise pin at pins.test.ts:250 STAYS (R-61)**; the operator-mediated copy is replaced by the self-serve truth with the admin-fallback noted honestly. Tests: web fetch-stub — entitlement render, checkout dispatch, mock-complete confirmation copy, CTA → /employer/login path, enterprise Contact Sales retained.
|
||||
- **Task 5-2-02** (frontend-engineer): admin network overview (D-094, R-62): the moderation page (app/(admin)/admin/moderation/page.tsx) gains sibling `<section>` blocks below the v0.9 EmployerEntitlementPanel: employer accounts list (email, claimed entity, entitlement state columns), claim queue (**evidence + domain-match signal rendered ADVISORY — never as verification, A-304**; approve/reject actions), audit view — fed by the P2 operator routes via engine-client admin transport over the existing `adminToken()` G-308 wrapper; **403 renders an honest locked state with the token prompt, never a retry loop (the entitlement-panel load() pattern, R-62)**. Tests: web fetch-stub — locked state on 403, claim queue renders evidence + advisory signal, approve/reject dispatch, accounts list columns.
|
||||
|
||||
### Wave 5-3: P5 security review (security-auditor)
|
||||
|
||||
- **Task 5-3-01** (security-auditor): checkout surface review — employer mock-complete own-session check (cross-session 403 pinned; mock-only mount pinned; stripe-mode 404 pinned — the G-301 class closed), unclaimed checkout 403 claim-CTA honest, entitlement view leaks nothing cross-entity, sessionStorage exposure copy present on the dashboard (open-risk-4), pricing copy honest (self-serve real, mock path labeled, enterprise pin retained). Findings recorded; P0s fixed in-phase.
|
||||
|
||||
## Phase 6: Final Review + Ship (milestone release, v0.9.x line)
|
||||
|
||||
**Goal:** Multi-persona review of all v0.10 changes; audit (reconstruction + discipline); milestone ship: merge to main, tag v0.9.6 (final v0.9.x patch = milestone release), full asset matrix + registry image via the existing release lane (no infra changes — the module rides D-064/D-065), milestone completion commits, branch cleanup.
|
||||
|
||||
### Wave 6-1: Multi-persona final review (lead-developer + security-auditor)
|
||||
|
||||
- **Task 6-1-01** (lead-developer → all active personas): full milestone-branch diff review — the D-091 no-drift verdict holds end-to-end (one helper, two gates, byte-identical 402), the cross-boundary wire review (employer.ts ↔ employers/base.py pins, the 401/402/403 discrimination contract in engine-client), D-097 discipline (learner gates untouched on learner routes; zero on employer routes), the D-092 honest-surface census (talent mock labels intact, no fake-converted surface, no convertible surface left on mock). P0s fixed in-phase, never loop back to EXECUTE.
|
||||
- **Task 6-1-02** (security-auditor): final review — no credentials in logs/releases/assets; scrypt params + self-describing rows pinned; sessions hashed-at-rest; 422 scrub covers password+email; throttle windows active; audit trail complete (claim decisions, grants, revocations all provenance-recorded); no secret ever echoed by an exception path.
|
||||
|
||||
### Wave 6-2: Audit + ship (lead-developer)
|
||||
|
||||
- **Task 6-2-01** (lead-developer → ship): audit (git log ↔ .ciagent/ reconstruction; file discipline — the R-66 addition census: ai_service/employers/, api/employers*.py, the api/ helper + adoption, config.py knobs, main.py wiring, tests/employers/, .env.example, packages/types/employer.ts, engine-client section, employer pages, admin sections, pricing CTA; branch/commit discipline — feat: scoped to the employer surface) + ship: merge milestone → main; tag v0.9.6; Gitea release with full summary + the complete asset matrix + registry image (the existing tag-triggered lane — zero workflow changes); delete milestone branches (tags preserve history); mark REQ-10-001..005 complete in REQUIREMENTS.md; ROADMAP milestone entry; checkpoint cleared.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Live Build Telemetry
|
||||
|
||||
**Requirements:** REQ-3-003
|
||||
**Goal:** First real persistence (D-027: SQLite via SQLModel) with a `TraceStore` protocol; `TelemetryEvent` model with per-(learner,task) monotonic `seq`; WebSocket ingest endpoint (D-026) with gap detection; stdlib-only in-sandbox capture agent streams real sandbox activity into ai-service; trace retrievable by learner+task
|
||||
|
||||
### Wave 1: Models + stores + TS types (parallel — no shared files)
|
||||
|
||||
#### Task 2-1-01: Telemetry event models
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/telemetry/__init__.py`, `apps/ai-service/ai_service/telemetry/models.py`, `apps/ai-service/tests/telemetry/__init__.py`, `apps/ai-service/tests/telemetry/test_models.py`
|
||||
- **Action:** Pydantic/SQLModel `TelemetryEvent`: `learner_id`, `task_id`, `seq` (int, monotonic per (learner,task)), `kind` (`command` | `file_diff` | `run_result` | `test_result` | `activity` | `stdin` | `stdout`), `payload` (JSON), `ts` (datetime, monotonic-envelope), `sandbox_id`. `TraceSpan` derived view (ordered events for one (learner,task)). Validation: seq ≥ 0, kind enum, non-empty ids.
|
||||
- **Verify:** `pnpm ai:test` — test_models green (validation rules enforced, JSON payload roundtrip)
|
||||
|
||||
#### Task 2-1-02: TraceStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/telemetry/store.py`, `apps/ai-service/tests/telemetry/test_store.py`, `apps/ai-service/ai_service/data/.gitkeep`
|
||||
- **Action:** `TraceStore` protocol (D-027, Postgres-migration-ready): `append(event) -> None` (idempotent on (learner,task,seq) — at-least-once dedup), `get_trace(learner_id, task_id) -> list[TelemetryEvent]` (ordered by seq), `gaps(learner_id, task_id) -> list[int]` (missing seqs), `latest_seq(learner_id, task_id) -> int`, `list_tasks(learner_id) -> list[str]`, `close()`. `SQLiteTraceStore(SQLModel)`: single `telemetry_event` table, composite PK ((learner_id, task_id, seq)), indexes on (learner_id, task_id). Engine creation from `AI_DB_PATH`; `SQLModel.metadata.create_all` at app lifespan. Enable `PRAGMA journal_mode=WAL` + `synchronous=NORMAL` at engine creation (a-3) so concurrent ingest (writer) and trace reads (grader) don't hit `database is locked` under concurrent sandboxes.
|
||||
- **Verify:** `pnpm ai:test` — test_store green (append/ordered-get/dedup-on-retry/gap detection/latest_seq; tmp-path SQLite per test)
|
||||
|
||||
#### Task 2-1-03: TS types for telemetry/traces
|
||||
- **Persona:** data-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `packages/types/telemetry.ts` (new), `packages/types/index.ts` (update)
|
||||
- **Action:** TS `TelemetryEvent`, `TraceSpan`, `TelemetryKind` mirroring the Python model field-for-field (cross-referencing header, same string enums). Consumed by Phase 6 web surfaces; no runtime code.
|
||||
- **Verify:** `pnpm typecheck` passes; TS type keys match Python model keys exactly
|
||||
|
||||
### Wave 2: Capture agent + ingest (depends on Wave 1)
|
||||
|
||||
#### Task 2-2-01: In-sandbox capture agent (stdlib-only)
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/scripts/sandbox-agent.py`, `apps/ai-service/tests/sandbox/test_sandbox_agent.py`
|
||||
- **Action:** Tiny standalone process (D-031, **stdlib only** — no deps shipped into the namespace): wraps a shell inside the sandbox; captures commands, file diffs (mtime/content polling of `workspace/` at 250ms), run/test results, activity; assigns per-(learner,task) `seq`; buffers to a local spool file on disconnect (at-least-once, D-026); reconnects with **exponential backoff** and flushes spool in order; small WebSocket client implemented over raw `socket` (RFC6455 client handshake + frames — stdlib only, no `websockets` in-namespace). Configured via env baked at spawn (`NC_LEARNER_ID`, `NC_TASK_ID`, `NC_INGEST_URL`).
|
||||
- **Verify:** `pnpm ai:test` — unit tests with a loopback fake WS server: ordered seq emission, spool-on-disconnect, reconnect flush preserves order (no loss, dupes deduped server-side), no third-party imports in the file (asserted by AST scan)
|
||||
|
||||
#### Task 2-2-02: WebSocket ingest endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/telemetry/ingest.py`, `apps/ai-service/ai_service/api/sandboxes.py` (update: register WS route), `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/tests/api/test_telemetry_ingest.py`
|
||||
- **Action:** `WS /v1/telemetry/ingest` (D-026): accepts connections carrying learner_id/task_id/sandbox_id; validates + appends events to `TraceStore` (idempotent — server-side dedup on (learner,task,seq)); emits **gap warnings** when seq skips (logged + surfaced in a per-connection status); ping/pong keepalive. **Backpressure / flood control (G-3 — replaces silent drop):** bounded inbound queue; on overflow OR when a per-connection cap `AI_TELEMETRY_MAX_EVENTS_PER_TASK` (default 50000) is exceeded → **reject with a 1008 policy-violation close and mark the (learner,task) trace `INCOMPLETE_FLOODED`** (an integrity signal consumed by Proctor). Silent drop-oldest is FORBIDDEN because it corrupts grading input and is indistinguishable from trace-gaming. `GET /v1/telemetry/traces/{learner_id}/{task_id}` returns the ordered trace; `GET /v1/telemetry/gaps/{learner_id}/{task_id}` returns missing seqs.
|
||||
- **Verify:** `pnpm ai:test` — test_telemetry_ingest green (TestClient websocket: connect → send 3 events → trace retrievable ordered; resend event 2 → deduped; skip seq 5 → gap reported; unknown sandbox tolerated in v0.3 no-auth mode)
|
||||
|
||||
### Wave 3: Sandbox telemetry wiring (depends on Wave 2)
|
||||
|
||||
#### Task 2-3-01: Spawn sandboxes with the capture agent
|
||||
- **Persona:** sandbox-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/ai_service/sandbox/manager.py` (update), `apps/ai-service/ai_service/sandbox/unshare_backend.py` (update), `apps/ai-service/tests/sandbox/test_telemetry_wiring.py`
|
||||
- **Action:** `create()` gains optional `task_id`; when set the spawner copies `scripts/sandbox-agent.py` into the sandbox workdir, injects `NC_*` env, and launches the agent as a child of the namespace process (agent lifecycle tied to sandbox lifecycle; destroy kills the agent). No capture when task_id absent (pure shell sandbox).
|
||||
- **Verify:** `pnpm ai:test` — end-to-end on this box: create sandbox with task_id → run 2 commands via exec → events arrive at the ingest endpoint and land in SQLite in order
|
||||
|
||||
### Wave 4: Reliability probe (depends on Wave 3)
|
||||
|
||||
#### Task 2-4-01: Dropped-connection durability probe
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-003
|
||||
- **Files:** `apps/ai-service/tests/telemetry/test_durability.py`
|
||||
- **Action:** Integration probe: run a capture agent against ingest, kill the WS connection mid-stream (simulate network failure), keep generating events, reconnect, assert the SQLite trace contains **every** event exactly once in order (spool + dedup). Document at-least-once semantics + replay path in README.
|
||||
- **Verify:** `pnpm ai:test` — test_durability green; README documents semantics
|
||||
|
||||
### Must-Haves (Phase 2)
|
||||
- [ ] Real telemetry from a live sandbox arrives at ai-service: shell commands, file diffs, run/test results appear as ordered events in SQLite (end-to-end, no mocks)
|
||||
- [ ] Per-(learner,task) monotonic `seq`; gap detection reports missing seqs; replay yields the complete ordered trace
|
||||
- [ ] At-least-once proven: transient disconnect + reconnect loses no events; duplicates deduped server-side (durability probe green)
|
||||
- [ ] Capture agent is stdlib-only (AST-verified) and its lifecycle is tied to the sandbox (destroy kills it)
|
||||
- [ ] Trace retrievable by learner+task via `GET /v1/telemetry/traces/...`; unknown trace → 404
|
||||
- [ ] `TraceStore` protocol respected: no api/ code touches SQLite directly; `telemetry/` never imports `agents/` (D-027)
|
||||
- [ ] Flood control (G-3): burst past `AI_TELEMETRY_MAX_EVENTS_PER_TASK` → connection closed 1008 + trace marked `INCOMPLETE_FLOODED`; no silent event drop on overflow
|
||||
- [ ] `pnpm ai:test` green; `packages/types` telemetry TS types compile (`pnpm typecheck`)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Process-Trace Grading Engine
|
||||
|
||||
**Requirements:** REQ-3-004
|
||||
**Goal:** Deterministic feature computation over traces (D-028) → compact digest → LLM rubric scoring via existing D-020 JSON defense → validated structured scores stored in `GradeStore`; LLM never sees the raw trace; engine calibrated against v0.2 mock corpora so process quality separates paste-and-run from iterative debugging
|
||||
|
||||
### Wave 1: Features + grades store (parallel — no shared files)
|
||||
|
||||
#### Task 3-1-01: Deterministic trace digest (features)
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/grading/__init__.py`, `apps/ai-service/ai_service/grading/features.py`, `apps/ai-service/tests/grading/__init__.py`, `apps/ai-service/tests/grading/test_features.py`
|
||||
- **Action:** Pure compute module (D-028): `compute_digest(trace: list[TelemetryEvent]) -> TraceDigest`. Deterministic features: test pass/fail counts + final status; edit count; error/fix cycle count + mean fix latency; idle gaps (>Ns, count + total); command category histogram (build/test/file/nav/debug/other); session duration; first-test-pass offset. `TraceDigest` pydantic model — compact (bounded size, no raw commands), LLM-safe.
|
||||
- **Verify:** `pnpm ai:test` — test_features green over synthetic traces: paste-and-run trace (0 error/fix cycles, single test pass at end) vs iterative trace (many cycles) produce observably different digests
|
||||
|
||||
#### Task 3-1-02: GradeStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/grading/store.py`, `apps/ai-service/tests/grading/test_store.py`
|
||||
- **Action:** `GradeStore` protocol (D-027): `save(grade)`, `get(learner_id, task_id)`, `list_for_learner(learner_id)`, `close()`. SQLModel `GradeRecord`: learner_id, task_id, variant_seed (null until P4), digest (JSON), scores (JSON), verdict, model, created_at. PK (learner_id, task_id). Postgres-migration-ready.
|
||||
- **Verify:** `pnpm ai:test` — test_store green (save/get/list roundtrip, overwrite-on-regrade documented)
|
||||
|
||||
### Wave 2: Grading engine + calibration (depends on Wave 1)
|
||||
|
||||
#### Task 3-2-01: Rubric scoring engine
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/grading/engine.py`, `apps/ai-service/ai_service/prompts/grading.py` (new), `apps/ai-service/tests/grading/test_engine.py`
|
||||
- **Action:** `GradingEngine.grade(learner_id, task_id) -> GradeRecord`: **trace-completeness gate (G-4)** — first call `TraceStore.gaps()` + check the trace `INCOMPLETE_FLOODED` flag; if gaps are non-empty OR the trace is flagged incomplete → return `verdict=UNGRADABLE_TRACE_INCOMPLETE` (a first-class verdict, not an exception) surfacing the gap list; a credential is NEVER issued from a gapped/incomplete trace. Otherwise: load trace via `TraceStore` → `compute_digest` → render rubric prompt (`prompts/grading.py`: criteria + level anchors for process quality, correctness, debugging discipline, test usage; a-4: treat high edit/command churn with no test-progress as a process-quality negative) → LLM structured output through the **existing D-020 4-layer defense** (`agents/structured.py` reused — engine composes it, never duplicates it) → validate `RubricScore` model (per-criterion 0-4 + strengths + gaps + verdict) → persist via `GradeStore`. Grading depends on `llm/` + `telemetry/` + `prompts/` only (boundary). Mock provider scripts deterministic rubric JSON for tests, including the INCOMPLETE path.
|
||||
- **Verify:** `pnpm ai:test` — test_engine green (mock provider: digest-only prompt asserted — **raw trace string absent from prompt**; validated scores returned; malformed JSON exercises D-020 retry; unknown trace → error)
|
||||
|
||||
#### Task 3-2-02: Calibration against v0.2 mock corpora
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/trace_fixtures.py` (new), `apps/ai-service/tests/grading/test_calibration.py`
|
||||
- **Action:** Synthetic trace fixtures aligned with v0.2 `corpus/telemetry.py` + `corpus/artifacts.py` scenario IDs (strong/lazy/struggling builder archetypes). Assert grading separates them: strong archetype scores ≥ lazy archetype on process-quality criterion (mock provider maps digest shape → scripted scores; test asserts the ordering contract + that fixture IDs align with existing corpus IDs, D-021).
|
||||
- **Verify:** `pnpm ai:test` — test_calibration enforces the ordering contract
|
||||
|
||||
### Wave 3: Grading endpoint (depends on Wave 2)
|
||||
|
||||
#### Task 3-3-01: Assessment grade endpoint (real traces)
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-004
|
||||
- **Files:** `apps/ai-service/ai_service/api/assessment.py` (update), `apps/ai-service/tests/api/test_grading.py`
|
||||
- **Action:** `POST /v1/assessment/grade {learner_id, task_id}` → `GradingEngine` → validated `RubricScore` JSON; `GET /v1/assessment/grade/{learner_id}/{task_id}` → stored grade; unknown trace → 404. Composed via DI (api/ owns wiring; engine knows nothing of FastAPI).
|
||||
- **Verify:** `pnpm ai:test` — test_grading green (grade roundtrip via TestClient with mock provider; 404 on unknown; GET after POST returns same scores)
|
||||
|
||||
### Must-Haves (Phase 3)
|
||||
- [ ] Engine emits structured rubric-aligned scores from a **real process trace** (not pre-baked input) — TestClient roundtrip green
|
||||
- [ ] Deterministic features computed in code (test pass/fail, edit count, error/fix cycles, idle gaps, command categories); LLM receives the **digest only** — test asserts the raw trace never reaches the prompt (D-028)
|
||||
- [ ] Scores distinguish process quality: iterative-debugging archetype out-scores paste-and-run on the process criterion (calibration contract test)
|
||||
- [ ] Grades persisted + retrievable by learner+task via GradeStore (SQLite, protocol-wrapped, D-027)
|
||||
- [ ] Boundary rules hold: `grading/` imports no `api/`/`agents/` internals except the shared D-020 structured defense; `pnpm ai:test` + `pnpm ai:lint` green
|
||||
- [ ] Incomplete-trace gate (G-4): gapped or `INCOMPLETE_FLOODED` trace → `verdict=UNGRADABLE_TRACE_INCOMPLETE` with the gap list; no credential issued from an incomplete trace (test green)
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Variant Task Generation
|
||||
|
||||
**Requirements:** REQ-3-005
|
||||
**Goal:** Template library with typed parameter slots (D-029) + seeded LLM instantiation + per-learner variant registry (SQLite) with difficulty-normalization anchors; two learners on the same competency get provably distinct, reproducible, auditable tasks
|
||||
|
||||
### Wave 1: Templates + store (parallel — no shared files)
|
||||
|
||||
#### Task 4-1-01: Task template library
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/variants/__init__.py`, `apps/ai-service/ai_service/variants/templates.py`, `apps/ai-service/tests/variants/__init__.py`, `apps/ai-service/tests/variants/test_templates.py`
|
||||
- **Action:** ≥3 initial task templates bound to existing competency IDs (D-021 alignment). `TaskTemplate`: id, competency_id, statement skeleton with `{slot}` placeholders, `ParameterSlot[]` (name, type: enum/int-range/string-set, allowed values), `rubric anchors` (difficulty normalization: expected feature envelope — e.g. expected edit-count band — used by grading context), starter-file scaffolds served to the sandbox. Seeded slot sampler is pure code (`random.Random(seed)`), fully reproducible.
|
||||
- **Verify:** `pnpm ai:test` — test_templates green (slot validation: bad value rejected; seeded sampling reproducible across runs; all templates bind to real competency IDs)
|
||||
|
||||
#### Task 4-1-02: VariantStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/variants/store.py`, `apps/ai-service/tests/variants/test_store.py`
|
||||
- **Action:** `VariantStore` protocol (D-027): `save(variant)`, `get(learner_id, template_id_or_task_id)`, `list_for_learner(learner_id)`, `list_by_template(template_id)`, `close()`. SQLModel `VariantRecord`: learner_id, task_id (the grading/telemetry task key), template_id, seed, params (JSON), statement (rendered), created_at. Unique (learner_id, template_id).
|
||||
- **Verify:** `pnpm ai:test` — test_store green (roundtrip, unique constraint, audit listing)
|
||||
|
||||
### Wave 2: Generator (depends on Wave 1)
|
||||
|
||||
#### Task 4-2-01: Seeded LLM variant generator
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/variants/generator.py`, `apps/ai-service/ai_service/prompts/variant.py` (new), `apps/ai-service/tests/variants/test_generator.py`
|
||||
- **Action:** `VariantGenerator.generate(learner_id, template_id) -> VariantRecord`: derive seed (`sha256(template_id|learner_id|milestone)` — reproducible, D-029); sample typed slots in code; render a fill prompt (statement skeleton + concrete slot values) → LLM via D-020 structured defense → unique task statement + starter files → validate → persist (seed + params + statement) via `VariantStore`. Cache: existing (learner,template) returns the stored variant (no duplicate work). Mock provider scripts deterministic statements per seed for tests.
|
||||
- **Verify:** `pnpm ai:test` — test_generator green: two different learner_ids → distinct statements for the same template; same learner twice → identical stored variant (reproducible); params JSON contains only schema-valid slot values; **fairness envelope (a-5):** two variants of one template compute digests within the template's expected feature envelope (comparable slot complexity/difficulty features) — "same bar" is testable, not asserted
|
||||
|
||||
### Wave 3: Variant endpoint + TS types (depends on Wave 2)
|
||||
|
||||
#### Task 4-3-01: Variant task endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `apps/ai-service/ai_service/api/variants.py` (new), `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/tests/api/test_variants.py`
|
||||
- **Action:** `POST /v1/variants {learner_id, template_id or competency_id}` → generated (or cached) variant: statement, starter files, task_id, seed; `GET /v1/variants/{task_id}` → stored variant; `GET /v1/variants?learner_id=` → learner's variants. DI wiring in api/ only.
|
||||
- **Verify:** `pnpm ai:test` — test_variants green (generate→get roundtrip; cache hit on regenerate; distinct learners → distinct statements asserted at the API layer)
|
||||
|
||||
#### Task 4-3-02: TS types for variants (+ grades)
|
||||
- **Persona:** data-engineer — **REQ:** REQ-3-005
|
||||
- **Files:** `packages/types/variants.ts` (new), `packages/types/grading.ts` (new), `packages/types/index.ts` (update)
|
||||
- **Action:** TS `TaskVariant`, `VariantParams`, `RubricScore`, `GradeRecord` matching Python models (cross-referencing headers; same field names). Consumed by Phase 6 surfaces.
|
||||
- **Verify:** `pnpm typecheck` passes
|
||||
|
||||
### Must-Haves (Phase 4)
|
||||
- [ ] Two learners requesting the same competency receive **provably distinct** task variants (API-level test)
|
||||
- [ ] Seed derivation reproducible: same (template, learner) → same variant, served from cache without a second LLM call (D-029)
|
||||
- [ ] Variant seed + typed params persisted + auditable (VariantStore listing; proctoring cross-check path exists)
|
||||
- [ ] Difficulty normalization anchors present per template and shipped to the grader prompt context
|
||||
- [ ] Starter-file scaffolds defined per template (P6 wires them into the sandbox workdir)
|
||||
- [ ] `pnpm ai:test` + `pnpm typecheck` green; `variants/` imports no api/ (boundary)
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Oral / Voice Defense
|
||||
|
||||
**Requirements:** REQ-3-006
|
||||
**Goal:** `VoiceProvider` protocol with mock + browser fallback (D-030) + seventh `Examiner` agent streaming over existing SSE + `DefenseStore` persisting transcript + integrity signals. **Real server STT/TTS (`OpenAIAudioProvider`) is DEFERRED to v0.4 (with KYC, when there's a real key + real users)** — voice is mock-first (D-030) and the `/audio/*` real path could never be exercised in CI, so v0.3 proves the full defense *dialogue* + integrity-signal pipeline over mock + browser-native fallback only; the protocol seam keeps the real provider a drop-in later.
|
||||
|
||||
### Wave 1: Voice provider layer (parallel — no shared files)
|
||||
|
||||
#### Task 5-1-01: VoiceProvider protocol + mock provider + browser fallback
|
||||
- **Persona:** voice-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/voice/__init__.py`, `apps/ai-service/ai_service/voice/base.py`, `apps/ai-service/ai_service/voice/mock.py`, `apps/ai-service/ai_service/voice/browser.py`, `apps/ai-service/ai_service/voice/factory.py`, `apps/ai-service/tests/voice/__init__.py`, `apps/ai-service/tests/voice/test_mock.py`, `apps/ai-service/tests/voice/test_factory.py`
|
||||
- **Action:** `VoiceProvider` protocol mirroring `LLMProvider` (D-030): `transcribe(audio: bytes, fmt) -> TranscriptSegment` + `synthesize(text, voice) -> AsyncIterator[bytes]`. `MockVoiceProvider`: deterministic canned transcript (scripted per test), canned 1kHz-tone WAV bytes, scripted failure modes. `browser.py`: fallback **descriptor** (`sr_available: true`, endpoint hints) the web client uses to select browser-native `SpeechRecognition`/`speechSynthesis` when no server provider. `factory.py`: `AI_VOICE_PROVIDER=browser | mock` (default mock when no key). **`OpenAIAudioProvider` (real server STT/TTS) intentionally NOT built in v0.3 — deferred to v0.4**; the protocol is its future seam. `voice/` never imports `agents/` or `api/`.
|
||||
- **Verify:** `pnpm ai:test` — test_mock + test_factory green (deterministic transcribe/synthesize; failure modes; factory selects mock with empty key, browser when provider=browser; zero network calls)
|
||||
|
||||
#### Task 5-1-03: DefenseStore protocol + SQLite implementation
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/voice/defense_store.py`, `apps/ai-service/tests/voice/test_defense_store.py`
|
||||
- **Action:** `DefenseStore` protocol (D-027): `start(defense)`, `append_turn(defense_id, turn)`, `finalize(defense_id, integrity_signals)`, `get(defense_id)`, `list_for_learner(learner_id)`, `close()`. SQLModel `DefenseRecord` (id, learner_id, task_id, status, created/finished_at) + `DefenseTurn` (defense_id FK, turn seq, role examiner|learner, text, ts, latency_ms) + integrity signals JSON on the record (long pauses, off-scope cadence markers — A-109).
|
||||
- **Verify:** `pnpm ai:test` — test_defense_store green (start→append turns→finalize→get roundtrip; ordered turns by seq)
|
||||
|
||||
### Wave 2: Examiner agent (depends on Wave 1)
|
||||
|
||||
#### Task 5-2-01: Examiner agent (seventh agent)
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/agents/examiner.py`, `apps/ai-service/ai_service/prompts/examiner.py` (new), `apps/ai-service/ai_service/agents/registry.py` (update: central registration, G-4 pattern), `apps/ai-service/tests/agents/test_examiner.py`
|
||||
- **Action:** `ExaminerAgent(BaseAgent)` (D-030/A-109): builds questions from learner transcript + trace digest + (P4) variant statement; probes understanding + challenges process choices ("why did you choose X at step N?"); streams questions over the existing SSE pipeline; `structured` verdict mode returns verdict + per-answer integrity signal list (long pause flags, off-scope answers) computed from turn metadata; calls voice **only through the `VoiceProvider` protocol** (PERSONAS conflict rule — never concrete providers). Session-scoped history reused from v0.2.
|
||||
- **Verify:** `pnpm ai:test` — test_examiner green (question stream references trace-digest facts; verdict structured output validates via D-020 defense; registry resolves all seven agents; mock-VoiceProvider wiring through protocol only — asserted by import scan in test)
|
||||
|
||||
### Wave 3: Defense endpoints (depends on Wave 2)
|
||||
|
||||
#### Task 5-3-01: Defense session + audio endpoints
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/ai_service/api/defense.py` (new), `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/tests/api/test_defense.py`
|
||||
- **Action:** `POST /v1/defense/start {learner_id, task_id}` → creates DefenseRecord + streams the first examiner question (SSE, agent=examiner); `POST /v1/defense/{id}/answer` (multipart audio from browser MediaRecorder, or `{text}` for typed fallback) → STT via VoiceProvider → append learner turn → stream examiner follow-up (SSE) → TTS audio chunks over `GET /v1/defense/{id}/audio/{turn_id}`; `POST /v1/defense/{id}/finish` → verdict + integrity signals persisted; `GET /v1/defense/{id}` → full transcript + signals. Browser-fallback mode: when provider=browser, start returns the fallback descriptor instead of server audio.
|
||||
- **Verify:** `pnpm ai:test` — test_defense green (full loop with mock voice + mock LLM: start → answer(text) → answer(audio bytes) → finish → transcript retrievable with per-turn latency; unknown id → 404; `GET` signals present after finish)
|
||||
|
||||
### Wave 4: Examiner latency instrumentation (depends on Wave 3)
|
||||
|
||||
#### Task 5-4-01: Per-turn latency instrumentation (mock-based)
|
||||
- **Persona:** voice-engineer — **REQ:** REQ-3-006
|
||||
- **Files:** `apps/ai-service/tests/voice/test_latency.py`, `apps/ai-service/README.md` (update: conversational-budget doc + v0.4 voice note)
|
||||
- **Action:** Instrument per-turn latency (STT ms + LLM TTFT ms + TTS ms) recorded on each DefenseTurn; deterministic test over mock providers asserts instrumentation presence + `latency_ms` populated + budget constant defined (mock runs are near-instant — wall-clock asserted in v0.4 against a real endpoint). README documents the conversational-latency target as a v0.4 acceptance criterion (real STT/TTS deferred per CUT-1/G-7).
|
||||
- **Verify:** `pnpm ai:test` — test_latency green (latency_ms fields populated on every turn; budget constant defined); README documents the deferred real-voice acceptance probe
|
||||
|
||||
### Must-Haves (Phase 5)
|
||||
- [ ] Spoken defense runs end-to-end over HTTP with mock providers: start → answer (audio + typed fallback) → examiner follow-up streams → verdict + transcript persisted (automated)
|
||||
- [ ] Examiner is the seventh registered agent; streams over the existing SSE envelope (meta agent=examiner)
|
||||
- [ ] Instrumented per-turn latency fields populated on every DefenseTurn (STT ms + LLM TTFT ms + TTS ms); conversational budget named (A-109)
|
||||
- [ ] `VoiceProvider` protocol respected: examiner + api touch voice only via the protocol; mock-first — **no task requires a real voice key to pass**
|
||||
- [ ] Browser-native SR/TTS fallback descriptor returned when no server voice provider configured (mock/browser are first-class, D-030)
|
||||
- [ ] Real server STT/TTS (`OpenAIAudioProvider`) explicitly deferred to v0.4 (with real keys/users); the defense pipeline is fully proven over mock+browser — documented in README + release note
|
||||
- [ ] Optional future voice config noted for v0.4 (`AI_VOICE_BASE_URL` / `AI_VOICE_API_KEY`) in `.env.example` + README; keys only in gitignored `.ciagent/.env.secrets`; tests never call a voice API
|
||||
- [ ] Boundary rules hold: `voice/` imports no `agents/`/`api/`; `pnpm ai:test` + `pnpm ai:lint` green
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Agent Re-grounding + Learner Surface Integration
|
||||
|
||||
**Requirements:** REQ-3-007, REQ-3-008
|
||||
**Goal:** Lab/Assessor/Proctor consume real engine inputs (live telemetry, grading output, defense signals) with **no mock fallback in the learner path**; the v0.1 sandbox + assessment mockups become real — in-browser build/run (Run/Test buttons + read-only exec output, CUT-2 — no interactive shell), live telemetry panel, browser/typed voice defense, live grading; `pnpm build` + `pnpm typecheck` + full `pnpm ai:test` green
|
||||
**Note:** E2E verification runs against the real engines over HTTP with `AI_PROVIDER=mock` + `AI_VOICE_PROVIDER=mock` permitted (G-2 precedent) — the requirement is real engine plumbing (sandbox/telemetry/grading/defense over real endpoints, no corpus mocks in the learner path); a cloud outage must not block P6.
|
||||
|
||||
### Wave 1: Agent re-grounding (parallel — no shared files)
|
||||
|
||||
#### Task 6-1-01: Lab agent on live telemetry
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/agents/lab.py` (update), `apps/ai-service/ai_service/prompts/lab.py` (update: render real trace digest), `apps/ai-service/tests/agents/test_lab_live.py` (new)
|
||||
- **Action:** Lab consumes a **live trace digest** (grading/features `compute_digest` over `TraceStore` events) instead of `corpus/telemetry.py`. build_messages renders digest facts (recent commands, failing tests, idle). Mock-provider scripts assert digest-derived content. v0.2 corpus path removed from the agent (dormant corpus retained until Task 6-1-04 check).
|
||||
- **Verify:** `pnpm ai:test` — test_lab_live green: feedback references events actually present in a seeded SQLite trace (not corpus fixtures); no `corpus.telemetry` import in `agents/lab.py` (AST-asserted)
|
||||
|
||||
#### Task 6-1-02: Assessor agent on grading output
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/agents/assessor.py` (update), `apps/ai-service/ai_service/prompts/assessor.py` (update), `apps/ai-service/tests/agents/test_assessor_live.py` (new)
|
||||
- **Action:** Assessor consumes `GradeStore` output (validated `RubricScore` + digest) for learner+task instead of pre-baked artifacts/transcripts; renders strengths/gaps/verdict with rubric-anchored coaching framing. Structured output unchanged (D-020).
|
||||
- **Verify:** `pnpm ai:test` — test_assessor_live green: given a real stored grade, Assessor output reflects its scores; corpus artifact path gone from the agent (AST-asserted)
|
||||
|
||||
#### Task 6-1-03: Proctor agent on telemetry + defense signals
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/agents/proctor.py` (update), `apps/ai-service/ai_service/prompts/proctor.py` (update), `apps/ai-service/tests/agents/test_proctor_live.py` (new)
|
||||
- **Action:** Proctor consumes real integrity inputs: idle gaps + command cadence from the trace digest + defense integrity signals from `DefenseStore` → classified signals + coaching interventions (supportive tone retained). Cross-checks variant seed params (P4) for off-template work.
|
||||
- **Verify:** `pnpm ai:test` — test_proctor_live green: signals derived from seeded real trace + defense records; corpus proctor scenarios no longer imported (AST-asserted)
|
||||
|
||||
#### Task 6-1-04: Corpus dormancy + mockup removal verification
|
||||
- **Persona:** lead-developer — **REQ:** REQ-3-007
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/telemetry.py`, `apps/ai-service/ai_service/corpus/artifacts.py`, `apps/ai-service/ai_service/corpus/` (README note), `apps/ai-service/tests/test_corpus_dormancy.py` (new)
|
||||
- **Action:** Verify no production code path imports `corpus/telemetry.py` or `corpus/artifacts.py` anymore (test scans imports across `agents/`, `api/`, engines). Retain files as Phase-3 calibration history with a header note marking them **dormant — v0.2 mocks, not used at runtime**; learner-context corpus stays (agents still need learner context). Disposes v0.2's G-5-class dead-code risk deliberately.
|
||||
- **Verify:** `pnpm ai:test` — test_corpus_dormancy green (zero runtime importers); suite otherwise unchanged
|
||||
|
||||
### Wave 2: Client plumbing + design primitives (parallel — no shared files)
|
||||
|
||||
#### Task 6-2-01: Sandbox build-panel engine client (run/test-only — no raw shell relay)
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `apps/web/hooks/use-sandbox-session.ts` (new), `apps/web/lib/engine-client.ts` (new), `apps/web/.env.example` (update)
|
||||
- **Action:** `engine-client.ts`: typed fetch client for `/v1/sandboxes` (create/destroy — learner_id from the v0.3 mock session constant, allowlisted server-side per G-5), `/v1/sandboxes/{id}/files` (workspace CRUD), `/v1/sandboxes/{id}/exec` (run/test), `/v1/variants`, `/v1/assessment/grade`, `/v1/telemetry/traces`, `/v1/defense/*`; base `NEXT_PUBLIC_AI_SERVICE_URL`. `use-sandbox-session.ts`: create sandbox+variant on task open → destroy on unmount (idempotent cleanup, AbortController pattern); 503 pool-full → user-facing "environment busy, retry" (D-032); 403/429 abuse-control surfaced honestly (G-5). **CUT-2 (G-8): NO raw interactive WS terminal relay (keystroke-level stdin/stdout) in v0.3** — the credential pipeline needs *process events* (from Run/Test + file edits), not a live shell; the interactive xterm relay is the most fragile real-time piece and is deferred to v0.4. The build panel is a **Run/Test output viewer** (exec results + telemetry pulse render), not an interactive shell. `@xterm/xterm` is therefore NOT a dependency in v0.3.
|
||||
- **Verify:** `pnpm install && pnpm typecheck` pass; hook unmount destroys the sandbox (manual probe: `curl localhost:8420/v1/sandboxes` shows count drop after navigation); RUN/TEST buttons produce streamed output + telemetry events in the trace
|
||||
|
||||
#### Task 6-2-02: New design primitives
|
||||
- **Persona:** design-system-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `packages/ui/src/primitives/terminal-frame.tsx` (new), `packages/ui/src/primitives/mic-control.tsx` (new), `packages/ui/src/primitives/grade-badge.tsx` (new), `packages/ui/src/primitives/telemetry-status.tsx` (new), `packages/ui/src/primitives/transcript-viewer.tsx` (new), `packages/ui/src/primitives/index.ts` (update), `packages/ui/src/index.ts` (update)
|
||||
- **Action:** Token-driven primitives: TerminalFrame (CUT-2: a read-only exec-output viewer chrome — streams Run/Test results, NOT an interactive shell), MicControl (record/stop with consent state + no-mic fallback state, MediaRecorder permission UX), GradeBadge (verdict rendering), TelemetryStatus (live event pulse / disconnected indicator), TranscriptViewer (examiner/learner turn list). Dark mode + WCAG AA; stories for each.
|
||||
- **Verify:** primitives import from `@nextcraft/ui`; Storybook stories render dark + light; `pnpm build` (ui package) passes
|
||||
|
||||
#### Task 6-2-03: Defense TS types
|
||||
- **Persona:** data-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `packages/types/defense.ts` (new), `packages/types/index.ts` (update)
|
||||
- **Action:** TS `DefenseSession`, `DefenseTurn`, `IntegritySignal`, `Verdict` mirroring P5 Python models (cross-referencing header).
|
||||
- **Verify:** `pnpm typecheck` passes
|
||||
|
||||
### Wave 3: Real build surface (depends on Wave 2)
|
||||
|
||||
#### Task 6-3-01: Sandbox mockup → real in-browser IDE
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `apps/web/app/(learner)/build/[competencyId]/page.tsx` (rewrite), `apps/web/components/learner/sandbox-terminal.tsx` (new), `apps/web/components/learner/file-tree.tsx` (new), `apps/web/components/learner/run-controls.tsx` (new), `apps/web/components/learner/lab-feedback-panel.tsx` (update: live trace)
|
||||
- **Action:** Replace the mockup with the real build environment (A-103, CUT-2): file tree (HTTP CRUD into the sandbox workdir via `/v1/sandboxes/{id}/files` routes added to api/sandboxes — read/write/list workspace files), syntax-highlight editor (existing), **Run**/**Test** buttons (exec in sandbox; results stream to a read-only TerminalFrame output panel — no interactive shell), starter files from the P4 variant scaffold. Lab panel posts `learner_id+task_id` → streams Lab feedback over the **live** trace (no scenario IDs). Telemetry sidebar shows live TelemetryStatus. Pool-full 503 → busy state with retry; 403/429 surfaced.
|
||||
- **Verify:** with ai-service up: open `/build/comp-01` → variant statement + starter files load → edit a file → **Run** executes the command in-sandbox and output renders in the panel → **Test** runs the test suite in-sandbox → Lab panel streams digest-derived feedback → telemetry status shows live events. Manual probe documented; `pnpm typecheck` green
|
||||
|
||||
### Wave 4: Live defense + grading surfaces (depends on Waves 2-3)
|
||||
|
||||
#### Task 6-4-01: Assessment mockup → live defense + live grading
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-3-008
|
||||
- **Files:** `apps/web/app/(learner)/defend/[competencyId]/page.tsx` (rewrite), `apps/web/components/learner/defense-session.tsx` (new), `apps/web/components/learner/assessor-results-panel.tsx` (update), `apps/web/components/learner/proctor-banner.tsx` (update), `apps/web/components/learner/oral-defense-interface.tsx` (rewrite or remove)
|
||||
- **Action:** Real assessment flow: **Start Defense** → POST `/v1/defense/start` → examiner question streams → learner answers via MicControl (MediaRecorder webm/opus → multipart POST) with typed fallback when mic denied or `provider=browser` (native `SpeechRecognition`/`speechSynthesis` path per fallback descriptor) → follow-ups stream → **Finish** → verdict + integrity signals panel (TranscriptViewer, GradeBadge) + **Grade My Work** → POST `/v1/assessment/grade` → structured rubric bars render. Proctor banner reads signals for task_id. Loading + error states throughout; no mock defense data remains in the learner path.
|
||||
- **Verify:** with ai-service up: full defense loop runs in-browser (typed fallback acceptable in CI-less manual probe; mic path exercised manually with permission granted); grade panel renders real rubric scores; `pnpm typecheck` green
|
||||
|
||||
### Wave 5: End-to-end verification (depends on Waves 3-4)
|
||||
|
||||
#### Task 6-5-01: Full learner-path E2E probe + green builds
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-3-007, REQ-3-008
|
||||
- **Files:** `apps/ai-service/tests/api/test_e2e_credential_flow.py` (new), `apps/ai-service/README.md` (update: E2E probe doc)
|
||||
- **Action:** Endpoint-level E2E (mock LLM/voice providers, real engines): create variant → create sandbox with task_id → ingest trace events via real sandbox exec → POST grade → start defense (typed answers) → finish → assert: trace persisted, grade stored + digest-linked, defense transcript + signals stored, Proctor/Assessor endpoints serve them. No corpus fixture anywhere in the flow (AST-asserted). Run `pnpm build` + `pnpm typecheck` + full `pnpm ai:test` at repo root; fix all failures before phase ship.
|
||||
- **Verify:** `pnpm ai:test` green incl. test_e2e_credential_flow; `pnpm build` + `pnpm typecheck` green; README E2E probe section documents the manual browser pass
|
||||
|
||||
### Must-Haves (Phase 6)
|
||||
- [ ] Lab/Assessor/Proctor operate on real inputs with **no mock fallback in the learner path** (AST-verified: no corpus telemetry/artifact/proctor imports in production paths)
|
||||
- [ ] Learner builds in-browser for real (CUT-2): Run/Test buttons execute in a namespace sandbox and stream output to a read-only panel; file tree CRUD works; starter files come from the learner's variant scaffold
|
||||
- [ ] Live telemetry: build activity streams to ai-service and the sidebar shows live status (TelemetryStatus); trace persisted in SQLite
|
||||
- [ ] Live defense: start → answer (mic or typed fallback) → examiner follow-ups → finish → transcript + integrity signals + verdict rendered; browser-native path works with no server voice key
|
||||
- [ ] Live grading: grade request returns structured rubric scores computed from the real trace digest; grade panel renders them
|
||||
- [ ] 503 pool-full surfaced honestly in UI; navigating away destroys the sandbox (no leaked sandboxes — `GET /v1/sandboxes` manual probe)
|
||||
- [ ] Examiner remains protocol-clean (voice only via `VoiceProvider`); module boundary rules hold across all new code
|
||||
- [ ] `pnpm build` and `pnpm typecheck` pass; full `pnpm ai:test` green; no cloud/voice calls in any automated test
|
||||
- [ ] Release-note input (for P7): v0.3 ships real engines; **identity/age-gating (KYC) remains deferred — age-gating is still a visual mockup** (A-110); sandbox scope is coding-IDE only (design tool/simulation deferred to v0.4, D-025)
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Final Review + Ship (no planned tasks)
|
||||
|
||||
Orchestrated by the SHIP stage, not this plan: multi-persona code review (correctness, testing, module boundaries, secrets hygiene — keys absent from code/logs/commits/errors, localhost-only CORS, no PII in prompts), project health audit (reconstruction, .ciagent/ discipline, branch/commit hygiene), then merge milestone → main, tag the final v0.2.x patch, create the Gitea release, mark all 8 v0.3 requirements complete.
|
||||
|
||||
**Release-note honesty:** the release note must state (a) Lab/Assessor/Proctor now run on real engine inputs (v0.2 mock-input caveat retired), (b) sandbox scope = coding IDE only — design tool + simulation environments deferred to v0.4 (D-025), (c) identity/age-gating (KYC) deferred per founder directive — age-gating remains the v0.1 visual flow mockup; abuse control (per-learner sandbox caps + server-side learner allowlist, G-5) ships in place of auth (A-110), (d) voice runs mock-first with browser-native fallback — real server STT/TTS deferred to v0.4 (CUT-1), (e) sandbox resource limits are partially enforced (memory/CPU/wall-clock kernel-enforced via rlimits; per-sandbox pids + hard disk quota are NOT — mitigated by a workdir-size sweep and per-learner caps; full enforcement requires cgroup delegation, deferred to the post-MVP containerd backend, D-024/G-1/G-2).
|
||||
|
||||
**Secrets-hygiene checklist (P7):** `.ciagent/.env.secrets` gitignored and never committed; `AI_VOICE_API_KEY` / `AI_TUTOR_API_KEY` referenced only via env; SQLite DBs + sandbox dirs gitignored; no keys in logs, error messages, or test fixtures.
|
||||
|
||||
**Disposal checks (G-5 class):** v0.2 dormant corpus files carry the dormant-header note (Task 6-1-04); any now-unused mock-data exports for the old sandbox/defense mockups (e.g. `aiTutorResponses`-class leftovers) must be removed or deprecated by review.
|
||||
|
||||
---
|
||||
## Must-Haves (per phase — concrete, falsifiable, test-linked)
|
||||
|
||||
- **P1 (REQ-10-001):** the employers module is importable + tested standalone; register/login/logout/me live with hashed sessions + both throttles; main.py wired; zero web changes. **MH-1a** register→login→me→logout flow green in tests/employers/test_auth.py; logout revokes (subsequent me → 401); concurrent sessions survive single logout. **MH-1b** REAL scrypt in every auth test (no fake-hash path; ≈2 hashes/test within the R-55 budget); params are module constants pinned by test; hash bytes never asserted. **MH-1c** employer_sessions stores ONLY sha256(token) (store-pinned); lazy 7-day TTL pinned via injected now (+6d present, +8d absent — no sleeps). **MH-1d** both throttle windows green in tests/employers/test_throttle.py — 5 failures/15 min → 429 with wait detail, success clears the email window, global cap → 429, rejected attempts consume nothing; reset fixtures isolate module state. **MH-1e** CORS preflight with `Access-Control-Request-Headers: Authorization` allowed — pinned in tests/api/test_cors.py (landed with the routes, not after). **MH-1f** a 422 on register never echoes the submitted password or email (pii_fields pin); mypy + ruff + full fast pytest green, zero new mypy overrides; no learner gate composed on any employer route (D-097).
|
||||
- **P2 (REQ-10-002):** claim → pending → admin approve → account.employer_id set, all pinned. **MH-2a** claim→approve→ownership set + audit row green in tests/employers/test_claims_admin.py. **MH-2b** evidence dict `{account_email_domain, entity_website_domain, match}` computed at claim time, stored as provenance, pinned in tests/employers/test_claims.py; displayed nowhere as verification (match=false corpus-wide per R-65). **MH-2c** one PENDING claim per entity + one APPROVED owner per entity + rejection frees the entity — all three pinned in tests/employers/test_claims_store.py. **MH-2d** all five operator routes (accounts/claims/approve/reject/audit) fail-closed 403 without a valid admin token.
|
||||
- **P3 (REQ-10-003):** the dashboard reads real data through the authenticated surface. **MH-3a** pins.test.ts employer.ts block asserts every field both schemas (ai_service/employers/base.py ↔ packages/types/employer.ts, D-093; PostingSummary reused from marketplace.ts, never duplicated). **MH-3b** 401 → SessionRequiredError → honest logged-out state with login CTA, never a retry loop (web test-pinned). **MH-3c** postings + overview pages grep-clean of removed mock imports (`jobs`, `candidates`, `employers`, inline METRICS/PIPELINE constants); own-postings lifecycle + real active count + per-posting applicant counts render from GET /v1/employers/postings; unclaimed → claim-CTA state visible; claim-pending state visible. **MH-3d** talent search + candidate profile pages keep their mock imports + mock labels (D-092 honest out-of-scope). **MH-3e** /employer/login + /employer/register render with honest 401/409/429/422 states; `pnpm --filter web build` (static export) green.
|
||||
- **P4 (REQ-10-004):** one shared gate, two principals, no drift. **MH-4a** the 402 payload is byte-identical between admin and employer gates on `{reason, posts_used: 0, posts_limit: 0}` — only upgrade_cta carries the principal's CTA — pinned by the payload-parity test; **tests/billing/test_entitlement_enforcement.py green UNCHANGED**. **MH-4b** gate-before-insert: a 402 consumes nothing and creates nothing (posts_used unchanged, no row) — pinned. **MH-4c** employer create consumes from the employer's OWN package (posts_used+1 pinned); both 402 arms (no_active_package, posting_limit_reached) proven via direct store grants; unlimited bypass holds. **MH-4d** cross-entity access → 403; own-drafts-only edit (published → 409); publish/close lifecycle green in tests/employers/test_postings_write.py; admin posting routes unchanged (existing marketplace_admin tests green unmodified).
|
||||
- **P5 (REQ-10-005):** the B2B loop closes machine-checked. **MH-5a** employer checkout E2E through the REAL webhook path green in tests/employers/test_checkout_e2e.py: register → claim → approve → checkout → own mock-complete → purchase exists with posts available (G-303 exactly-once holds — duplicate complete never double-grants). **MH-5b** cross-session mock-complete → 403; the route 404s under provider=stripe (mock-only mount) — both pinned (the G-301 class closed). **MH-5c** unclaimed checkout → 403 + claim-CTA; entitlement view renders honest none/active/expired states. **MH-5d** pricing CTAs route to /employer/login (contact-path copy gone from purchasable tiers); the enterprise "Contact Sales" pin retained (pins.test.ts:250). **MH-5e** admin network overview renders the accounts list, claim queue with advisory evidence + approve/reject actions, and the audit view; 403 → honest locked state with token prompt (web tests).
|
||||
- **P6 (ship):** **MH-6a** multi-persona review clean (P0s fixed in-phase); the D-091 no-drift + wire-pin verdicts recorded. **MH-6b** security final review clean (no credentials in logs/releases; audit trail complete). **MH-6c** reconstruction + discipline checks green; v0.9.6 tagged, merged to main, release carries the full asset matrix + registry image via the existing lane; REQ-10-001..005 marked complete; branches deleted; checkpoint cleared.
|
||||
|
||||
## Milestone Must-Haves
|
||||
|
||||
- REQ-10-001..005 all complete per their REQUIREMENTS.md rows.
|
||||
- The full B2B loop is the machine-checked E2E proof: register → login → claim → admin approve → create draft → buy package (own mock checkout → REAL webhook) → publish → posting live — every step pinned in tests/employers/.
|
||||
- Zero new dependencies (R-66), zero workspace changes, zero mypy overrides; scrypt/secrets/hmac/sha256/urllib stdlib-only end-to-end.
|
||||
- The v0.9 wire contracts unchanged: UpgradeRequiredError byte-stable, existing billing/marketplace_admin entitlement tests green unmodified.
|
||||
- Suite counts: pytest fast lane + web + cli all green, zero warnings (the v0.7 regime holds).
|
||||
- Docs: .env.example throttle knobs; REQUIREMENTS/ROADMAP updated; deploy/README.md unaffected (no admin-box changes — record explicitly); the mock-first doctrine intact (no external auth provider, tests never call a cloud).
|
||||
|
||||
## Risks
|
||||
|
||||
1. **scrypt CPU on concurrent auth** (R-55, ~50 ms/hash single-threaded; a login burst stacks) — bounded by both throttle windows; budget recorded in the passwords.py docstring; acceptable at pilot.
|
||||
2. **CORS Authorization omission kills every cross-origin employer fetch** (E87 class, R-60/open-risk-2) — the main.py edit + the test_cors.py pin land in P1 Task 1-2-03, the same wave as the routes, never after.
|
||||
3. **422 password echo** (R-56/open-risk-3) — pii_fields fix + pin land in P1 with the register route; auditor re-verifies at the P1 review.
|
||||
4. **sessionStorage token exposure** (D-092 tradeoff, open-risk-4) — tab-scoped by design, exposure documented in surface copy; tokens stored hashed server-side (DB leak yields nothing); logout revokes the row.
|
||||
5. **Employer mock-complete cross-session completion** (G-301 hole class, open-risk-5) — own-session check (403 cross-session) + mock-only mount + session gate, all pinned in P5.
|
||||
6. **example.com seed websites make claim evidence match=false corpus-wide** (R-65/open-risk-6) — the queue copy presents the signal as advisory; the operator decision activates; auto-approval stays deferred (A-304).
|
||||
7. **D-091 extraction drift** (the 402 dict's hardcoded zeros are easy to "improve") — the payload-parity test + unchanged v0.9 entitlement suite are the no-drift proof; lead-developer reviews at P4.
|
||||
8. **Throttle module-state leaks across tests** (R-63) — both limiters get reset fixtures from day one (the _reset_apply_rate_window pattern).
|
||||
|
||||
## User-Facing Surface
|
||||
|
||||
The primary user-facing surface is the **learner build + defend flow** at `http://localhost:3000`, backed by the real engines in ai-service at `http://localhost:8420`:
|
||||
1. **/employer/login + /employer/register** — greenfield auth pages with honest 401/409/429/422 states; register → logged-in dashboard; sessions tab-scoped via `nc-employer-session`.
|
||||
2. **Employer dashboard — postings page** — the employer's OWN postings lifecycle (draft → published → closed) with real applicant counts, create/edit/publish/close actions, honest 402 upgrade CTA + 403 claim prompt.
|
||||
3. **Employer dashboard — overview** — real active-postings count + per-posting applicant counts + claimed-entity profile; talent-match/placement metrics honestly mock-labeled; claim-pending state visible.
|
||||
4. **Talent search + candidate profile** — remain the v0.1 mock, honestly labeled (no candidate backend exists — D-092).
|
||||
5. **Pricing page** — CTAs convert to the employer-login path (self-serve checkout is real); enterprise tier keeps Contact Sales.
|
||||
6. **Admin moderation page — network overview** — employer accounts list, claim queue with advisory evidence + approve/reject, audit view; 403 → honest locked state.
|
||||
7. **API surfaces** — /v1/employers/* (session-gated: auth, claims, postings, billing) + /v1/employers/admin/* (operator-gated) + the existing /v1/billing/* reused verbatim.
|
||||
8. **Release notes** — the v0.9.x line ships the employer network through the existing automated lane (no admin-box changes required).
|
||||
|
||||
- `/dashboard` — AI tutor chat (Coach/Tutor, streaming) + Mentor panel (unchanged from v0.2)
|
||||
- `/learn/[competencyId]` — byte viewer with streaming Tutor explanations (unchanged)
|
||||
- `/build/[competencyId]` — **real in-browser build environment**: file tree, syntax editor, **Run/Test buttons that execute in a namespace sandbox and stream output to a read-only panel** (CUT-2 — no interactive shell), live telemetry status, live Lab feedback, per-learner variant task statement
|
||||
- `/defend/[competencyId]` — **live oral defense + live grading**: Examiner voice/typed dialogue, transcript + integrity signals, real rubric scores from the process trace
|
||||
## Happy Path (end-to-end, written BEFORE execute)
|
||||
|
||||
The marketplace, employer, and admin surfaces are unchanged from v0.1/v0.2.
|
||||
|
||||
## Happy Path
|
||||
|
||||
1. Learner opens `/build/comp-01` → a per-learner **variant statement** and starter files load; a namespace sandbox is created for the session
|
||||
2. Learner edits files in the tree and clicks **Run**/**Test** → commands execute in the sandbox and real output renders in the build panel; the telemetry sidebar pulses as events stream to ai-service and persist in SQLite
|
||||
3. The Lab panel streams feedback derived from the **live trace digest** (real commands, real failures)
|
||||
4. Learner opens `/defend/comp-01` → **Start Defense**: the Examiner streams an opening question ("Walk me through your build — why did you structure it this way?")
|
||||
5. Learner answers by voice (mic consent → MediaRecorder → STT) or typed fallback → examiner follow-ups probe the trace ("You hit three test failures before passing — what changed?"); TTS plays examiner audio (or browser speech in fallback)
|
||||
6. Learner finishes the defense → transcript + integrity signals appear; verdict renders in a GradeBadge
|
||||
7. Learner clicks **Grade My Work** → the grading engine computes the digest from the real trace, rubric-scores it, and the panel renders per-criterion bars + strengths/gaps/verdict
|
||||
8. Proctor banner shows integrity signals from the live trace + defense in coaching tone; Mentor panel on `/dashboard` can narrate the real outcome
|
||||
9. Navigating away destroys the sandbox (pool slot freed); killing ai-service shows inline error + retry states on every panel, with no crashes
|
||||
1. **Employer registers:** /employer/register → account created (email unverified, labeled as such) → logged in → dashboard shell with the claim prompt.
|
||||
2. **Employer claims their entity:** picks their EmployerRow from the marketplace corpus → claim pending, evidence recorded → dashboard shows the honest claim-pending state.
|
||||
3. **Operator approves:** admin moderation page → claim queue shows the evidence + domain-match signal (advisory) → approve → account bound to the entity (one owner) → audit row written.
|
||||
4. **Employer tries to post without a package:** create draft → 402 with upgrade CTA (reason: no_active_package) → nothing consumed, nothing created.
|
||||
5. **Employer buys a package (pilot mock):** dashboard billing surface → own checkout over the catalog → in-app mock-complete (honestly labeled) → REAL webhook path → purchase granted → posts available in the entitlement view.
|
||||
6. **Employer publishes:** create draft (posts_used+1) → publish → posting live on the public job board; applicant counts accrue on the overview; the 11th post on a 10-pack → 402 upgrade CTA again.
|
||||
7. **Operator oversight throughout:** the network overview shows the account, claimed entity, and entitlement state; the audit trail answers every claim/grant/revocation decision.
|
||||
|
||||
## UX Acceptance Criteria
|
||||
|
||||
1. Run/Test output visibly reflects the real sandbox execution (command round-trip to the sandbox, real stdout/stderr), not a replay animation
|
||||
2. The learner path contains **no mock engine data** — scenarios, canned artifacts, and scripted defense transcripts from v0.2 are gone from runtime
|
||||
3. Variant statements visibly differ between two learner sessions on the same competency
|
||||
4. Mic permission flow is graceful: consent prompt, recording indicator, no-mic/typed fallback, and browser-native speech path when no server voice key is configured
|
||||
5. Defense transcript renders turn-by-turn with latency shown; integrity signals render in coaching (supportive) tone
|
||||
6. Grade results render as structured per-criterion bars with verdict, from the real trace — not from final-output-only heuristics
|
||||
7. Pool-full (503) shows an honest "environment busy — retry" state; navigation/unmount destroys sandboxes with no leaks
|
||||
8. When ai-service is unreachable: inline error + retry on every panel — no crashes, no console errors, no blank UI
|
||||
9. All new UI uses design tokens, supports dark mode, meets WCAG AA contrast, responsive at 375px / 768px / 1280px
|
||||
10. `pnpm build` and `pnpm typecheck` pass with zero errors; `pnpm ai:test` green cloud-free and voice-key-free
|
||||
1. **Honest states everywhere:** every new surface renders real loading/error/empty states — no spinners that lie, no silent failures, no fake data.
|
||||
2. **401 → honest logged-out state:** SessionRequiredError renders the logged-out state with a login CTA — never a retry loop, never a silent redirect.
|
||||
3. **402 upgrade CTA visible with correct limits:** the payload's reason drives the copy (no package vs limit reached); the CTA routes to the employer billing surface; limits shown are the store's truth. (G-320b: the 402 payload's posts_used/posts_limit are byte-stable zeros, never rendered — real limits render from the entitlement view only.)
|
||||
4. **Claim-pending state visible to the employer:** the dashboard says "claim submitted, awaiting review" until the operator decides — never presented as verified before approval.
|
||||
5. **Admin sees the claim queue with evidence:** the domain-match signal renders as advisory (match=false is the corpus norm); approve/reject are explicit operator actions with an audit trail.
|
||||
6. **Talent pages remain mock-labeled:** the honest-surface rule cuts both ways — never fake-convert, never leave a convertible surface on mock; talent stays mock because no backend computes it.
|
||||
+237
-18
@@ -8,45 +8,259 @@ Nextcraft is an AI-native outcome school where graduates prove what they can bui
|
||||
|
||||
---
|
||||
|
||||
## Current Milestone: v0.3 — Credential Engines
|
||||
## Hotfix Round (2026-09-21, founder directive)
|
||||
|
||||
**Scope:** Replace v0.2's mock engine inputs with real credential engines. Build the sandbox fabric (sandboxed IDE / design tool / simulation), the live in-environment build-telemetry pipeline, the process-trace grading engine, per-learner variant task generation, and the oral/voice defense with AI examiner. Lab/Assessor/Proctor agents move from mock inputs to real engine inputs; the six tutor agents operate on authentic telemetry and artifacts.
|
||||
**v0.4.6 — deploy-static-404 + dev experience.** Third attempt at the site-down issue (domain root → API JSON 404). Root cause found by host probe (10.0.0.105): the deployed uvicorn ran mount-less — `apps/web/out` was never built on the host, `dev -d` auto-wiring silently skipped `AI_WEB_STATIC_DIR`, node/pnpm were invisible to non-interactive shells, and NOTHING detected the mount-less boot. Founder directive: fix the deploy experience so the one-liner install handles all of it — smooth first run, upgrades, and reboot persistence. Delivered: `nextcraft deploy` (clone/update → install → venv → export build → restart → verify, idempotent), `deploy --service` (generated systemd unit, enabled), toolchain discovery (PATH → nvm → PNPM_HOME → corepack; prefix carries node+pnpm dirs for split installs), loud mount-less boot warning, `verify` live mount probe (GET / must be HTML), install.sh next-steps rewritten. Deployed live on the host via the new command; site 200, service active+enabled.
|
||||
|
||||
**v0.4.7 — pre-public hardening (the 4 P1 flags).** WS ingest HMAC tokens (grading input no longer LAN-poisonable; manager mints into NC_INGEST_URL, endpoint verifies, per-boot secret default), nonce+counter defense-id mint (same-ms collision → permanent 500 closed), browser-mode SR/TTS client implementation (the descriptor's v0.5 contract finally honored — recognizeOnce/speakText pure module + defense-session wiring + honest unsupported state), 8 module pins. All P1 flags from the v0.5 final review are now closed; the recorded pre-public hardening backlog is empty.
|
||||
|
||||
---
|
||||
|
||||
## Milestone v0.10 — B2B Employer Network (ACTIVE, Feature, started 2026-09-30)
|
||||
|
||||
**Scope (D-085, auto-decided at SPECIFY under full autonomy D-007; founder scope confirmed 2026-09-30):** Real employer auth/accounts and employer self-serve — the milestone every prior decision sequences next: A-407 ("Real employer auth lands post-payments with the B2B employer network" — payments shipped v0.9), A-401 (employer self-serve posting management deferred pending employer auth), D-076/D-079 (operator-mediated checkout + contact-path CTAs were honest *because* no employer principals exist). Employers become principals: registered accounts with sessions, claiming their existing employer entity (the v0.6 `EmployerRow` ingested data), self-serve posting management (draft → published → closed on their own entities), self-serve checkout (the v0.9 billing catalog with employer-principal buyers instead of operator mediation), and the employer dashboard surface converting from the v0.1 mockup to real data. Rationale: the guiding outcome (≥1,000 placements) needs a self-serve employer side; the supply side (v0.6), monetization rails (v0.9), and install-anywhere distribution (v0.8) are all in place — the employer network is the missing B2B surface that unlocks CIRR tracking (REQ-F-020 needs employer-side placement data) and closes the last mock-only surface. Confidence 0.80 ≥ threshold 0.60 (D-007).
|
||||
|
||||
**Out of scope:** CIRR placement tracking (REQ-F-020, deferred), human tutor marketplace (REQ-F-019, deferred), multi-tenant SSO/OIDC enterprise login (email+password at pilot), employer team/multi-seat management, posting analytics beyond the existing enriched views.
|
||||
|
||||
**Success (milestone-level):** an employer can register an account, claim/verify their employer entity, log in, manage their own postings through the full lifecycle, purchase posting packages/talent-access plans through self-serve checkout with entitlement enforcement applying to their own posts, and see real data in the employer dashboard; admin routes retain operator oversight (impersonation-free, audit-logged); all auth is session-based with hashed credentials, rate-limited, and fail-closed on unconfigured secrets; the mock-first doctrine holds (no external auth provider, tests never call a cloud).
|
||||
|
||||
## Clarify decisions (D-086+, CLARIFY stage full autonomy, 2026-09-30)
|
||||
|
||||
| ID | Decision | Rationale | Confidence |
|
||||
|----|----------|-----------|------------|
|
||||
| D-086 | Password KDF = **stdlib `hashlib.scrypt`, zero new dependencies**: per-password 16-byte random salt (`secrets.token_bytes`), OWASP-floor params N=2^14/r=8/p=1, dklen=32, stored self-describing as `scrypt$16384$8$1$<salt_b64>$<hash_b64>` so params can evolve without invalidating stored rows; verification compares derived bytes via `hmac.compare_digest`; params are module constants pinned by test (the D-083 PAST_DUE_GRACE_DAYS shape). | passlib is unmaintained and ships no type stubs — adding it would force the FIRST `ignore_missing_imports` mypy override, breaking the v0.7 pyproject regime ("currently NONE" — pyproject.toml:143-151); bcrypt adds a compiled dep to the deb/docker/arm64 matrix (D-069) for nothing; scrypt IS a library-grade memory-hard KDF (OpenSSL-backed, probe-verified in the shipped venv) and the tree's crypto precedent is stdlib-only end-to-end (HMAC-SHA256 webhooks D-077, ingest tokens v0.4.7, `secrets` minting). pypdf was "the ONLY new v0.6 dependency" — the zero-dep discipline holds. | 0.80 |
|
||||
| D-087 | Employer sessions = **server-side rows in the new `employer_sessions` table (8th D-027 family), opaque token minted per login, stored hashed**: token = `secrets.token_urlsafe(32)` (CSPRNG — per-boot entropy inherent, the factory.py mint precedent); the row stores only `sha256(token)` and lookups re-hash the presented token, so a DB-file leak never yields usable credentials (the G-108 secrets doctrine extended to storage). TTL = fixed 7 days from login (module constant, test-pinned) evaluated LAZILY at read (G-306 — no timer); logout DELETEs the row + audit event (revoke = lookup miss → 401); multiple concurrent sessions per account (per-device rows) are allowed. The agents/ `InMemorySessionStore` is NOT reused — it is LRU-evicted chat context (D-019), a different concern; principal state is a durable D-027 row like everything else persisted. | REQ-10-001 already fixes "server-side store" (signed tokens/JWTs are off-spec); opaque random tokens need no signing key (server-side lookup is the truth source) and no per-boot secret to manage; hashed-at-rest is the standard session-store discipline at zero cost. | 0.85 |
|
||||
| D-088 | Auth throttling = **in-memory sliding windows in api/ (process-local, the G-5 precedent)**: (a) per-EMAIL failed-login window — default 5 failures / rolling 15 min → 429 (settings knob, the `identity_submits_per_min` shape); (b) a global per-min cap on login+register attempts (the `_check_global_create_rate` deque, api/sandboxes.py:123) → 429, bounding cross-email credential-stuffing volume; successful login clears the email's failure window; 429s carry the honest "wait" detail. Keyed by the SUBMITTED EMAIL, not client IP — behind HAProxy every request's `request.client.host` is the proxy (the tree has zero IP-trust machinery; D-039 single-port). | The identity `_SubmitRateLimiter` (api/identity.py:97-116) and sandbox create-cap are the proven in-tree throttle patterns — same module-state acceptability (D-019: one process, one box); IP-based keys would be fake security on this topology; brute-force economics at pilot are bounded by the two windows without any new infra. | 0.80 |
|
||||
| D-089 | Claim/verify = **claim → pending → admin decision; evidence recorded, never auto-activating at pilot**: `POST /v1/employers/claims {employer_id}` (authenticated) creates an `employer_claims` row status=pending with computed evidence `{account_email_domain, entity_website_domain, match}` (normalized registrable-domain comparison — stored as provenance); `employer_accounts.employer_id` is UNIQUE and set on approval (the REQ's "one account : one entity" UniqueConstraint); one PENDING claim per entity + one APPROVED owner per entity enforced check-first in the store (the `insert_application` precedent); rejection frees the entity for a new claim. Admin approve/reject routes (require_admin) set decided_by/decided_at + append `employer_audit_events` (the billing_events provenance shape). Auto-approval on domain match is DEFERRED: without email verification the match is forgeable, so the admin queue DISPLAYS the signal but the operator decision activates. | The seeded corpus websites are example.com placeholders — no real verification surface exists at pilot; REQ-10-002 explicitly retains "admin routes for claim review/approve/reject (operator oversight)"; an auto-approve on forgeable evidence would be a dishonest gate (the A-304 doctrine — never present unverified as verified). | 0.80 |
|
||||
| D-090 | Session transport = **`Authorization: Bearer <session-token>`** with a `require_employer_session` dependency (api/employers.py — the require_admin extract→verify→return shape): 401 on missing/expired/unknown token (the webhook-HMAC credential precedent), 403 stays the authorization failure (cross-entity access); the dep returns the session + account context (claimed employer_id included). NOT a cookie: D-039 static export has no Next middleware — the client attaches the header explicitly on every fetch (A-002); an explicit header is CSRF-immune by construction (no ambient credential, no SameSite machinery). CORS `allow_headers` gains `"Authorization"` (main.py:341 — the E87-class lesson: the v0.3 PUT preflight omission broke every cross-origin Save; tests/api/test_cors.py extended to pin it) for the two-port dev + D-038 network-mode topologies. Token namespaces cannot collide: admin routes compare against the env token, employer routes hash-lookup sessions — a token of one class fails the other's gate. Operator routes live in a separate `api/employers_admin.py` mounting `require_admin` imported from marketplace_admin (the billing_admin.py:27 import precedent). | Parallel to require_admin's dual-header acceptance but single-header for one principal class; same-origin production fetches skip CORS entirely; the dev topology needs the allow-list entry or every employer preflight dies — caught before it ships this time. | 0.85 |
|
||||
| D-091 | Entitlement reuse = **one shared api-layer helper, two principal gates composing it**: extract the 402 gate from marketplace_admin.create_posting (lines 181-220) into a shared api/ function `consume_posting_entitlement(store, employer_id, upgrade_cta)` raising the canonical 402 dict payload `{reason, posts_used, posts_limit, upgrade_cta}`; the admin route passes the admin-checkout CTA, the employer route the employer-checkout CTA — reason/posts_* identical, `UpgradeRequiredError` contract unchanged (D-080/D-084 wire stability). `consume_posting` itself (G-304 single conditional UPDATE) is untouched — no store change, no billing/→api/ boundary violation (the helper lives in api/ beside both callers, D-027/D-048 territory preserved). Employer checkout `POST /v1/employers/billing/checkout {item_id}` mirrors billing_admin.create_checkout with the buyer = the session's claimed entity (403 + claim-CTA when unclaimed; metadata operator_ref → account provenance) and an employer-scoped mock-complete arm mounted only under provider=mock, completing one's OWN session (D-082 transposed to the employer principal; success/cancel URLs from public_base_url per G-310). | REQ-10-004 demands the exact v0.9 check parameterized, not reimplemented — extraction is the only no-drift shape; the billing webhook/provider/store path is reused verbatim (REQ-10-005 "the v0.9 billing module reused"), only the principal derivation changes. | 0.85 |
|
||||
| D-092 | Web conversion = **employer section in engine-client.ts (no new client module) + sessionStorage `nc-employer-session`**: engine-client.ts is THE fetch module (identity/billing/marketplace sections already live there; pins.test.ts pins its content) — the employer section adds register/login/logout/me, claim, own-postings CRUD/publish/close, own-checkout, own-entitlement + a `SessionRequiredError` (401 → the G-308 honest logged-out state, never a retry loop; logout clears storage + calls the revoke route; every fetch aborts via AbortController — the A-406 board-client discipline). Storage = sessionStorage (the G-308 admin-token wrapper is the only in-tree credential-storage pattern; tab-scoped bounds XSS token exposure vs localStorage's persistent copy; no pilot requirement demands cross-tab persistence). Conversion scope: postings page (real own-postings lifecycle), overview page (real active-postings count + per-posting applicant counts from the applications table; talent-match/placement metrics stay mock-labeled — no backend computes them), employer profile (claimed entity via the existing public GET /v1/marketplace/employers/{id}), new /employer/login + /employer/register pages; **talent search + candidate profile pages stay the v0.1 mock** (no candidate backend exists in any milestone — v0.6 built postings/search over POSTINGS; a talent-access search backend is honestly out of scope, their mock imports remain). | One client module keeps parseError/error-class discipline in one place (a second module would fork the 402/401 discrimination); mock stays only where no real data source exists — the honest-surface rule cuts both ways (never fake-convert, never leave a convertible surface on mock). | 0.78 |
|
||||
| D-093 | Wire types = **`packages/types/employer.ts`** (EmployerAccount, EmployerClaim, EmployerPublicProfile reuse where shapes coincide — PostingSummary is reused from marketplace.ts, never duplicated, and the entitlement view reuses the v0.9 shapes) dual-schema pinned field-for-field against `ai_service/employers/base.py`'s pydantic contracts in `apps/web/tests/pins.test.ts` (the G-108 discipline; the billing.ts pin block, pins.test.ts:202-230, is the template — every field asserted both sides). | Every prior cross-boundary wire (PostingSummary, SearchResult, Package/Plan) is pinned this way (D-084); the employer surface is a new cross-boundary contract and gets the same rail from day one. | 0.85 |
|
||||
| D-094 | Admin network overview = **the admin marketplace moderation page gains the network sections** (app/(admin)/admin/moderation/page.tsx — where the v0.9 EmployerEntitlementPanel already landed, REQ-9-004's operator lane): employer accounts list (email, claimed entity, entitlement state columns), claim queue (evidence + domain-match signal, approve/reject actions), fed by new operator routes in api/employers_admin.py (require_admin + audit events): GET /v1/employers/admin/accounts, GET /v1/employers/admin/claims, POST .../claims/{id}/approve, POST .../claims/{id}/reject, GET .../audit — web transport via the existing `adminToken()` G-308 sessionStorage wrapper + honest 403 locked states. | REQ-10-005 names "the operator oversight lane"; the moderation page is the established admin marketplace surface (employer claims are the marketplace-moderation concern — REQ-025's verification queue was its v0.1 mock ancestor); one admin surface keeps the operator's one-glance workflow instead of fragmenting across pages. | 0.80 |
|
||||
| D-095 | Migration/compat = **8th D-027 store on the shared DB file, `SQLModel.metadata.create_all` idempotent — zero migration**: `SQLiteEmployerStore` mirrors billing/store.py mechanics exactly (WAL / synchronous=NORMAL / busy_timeout / foreign_keys pragmas at connect, sa_type portable columns, `_validate`/`_normalize` hooks, Protocol + SQLite impl pair, surfaced IntegrityError, tz-aware↔naive normalization); `create_all` checkfirst semantics add the 4 tables (`employer_accounts`, `employer_sessions`, `employer_claims`, `employer_audit_events`) to existing DBs at next boot — the billing store shipped the same way against the marketplace-era file (proven compat). Lifespan wiring: the state-injection override (main.py `if getattr(app.state, "employer_store", None) is None: construct from settings.db_path`) + `close()` in the finally block (the billing_store wiring, main.py:182-190/325). The `employer_` prefix is mandatory — the bare `employer` table name is taken by marketplace's EmployerRow (store.py:145). | The 7 prior stores never broke an existing DB via this exact pattern (D-027 family replication is the most-repeated store contract in the tree); REQ-10-001's table names already carry the prefix. | 0.90 |
|
||||
| D-096 | Testing doctrine = **real flows against create_app + TestClient, tmp_path DBs, real KDF, zero network by construction**: `tests/employers/` (the billing suite layout): register/login/logout/me, throttle 429 windows (both knobs), claim → admin approve → own-posting lifecycle → cross-entity 403s, entitlement 402 arms via direct store grants (the test_entitlement_enforcement.py:60 precedent), employer checkout + employer mock-complete E2E through the REAL webhook path (the D-082 proof shape). Password hashing runs the REAL scrypt in tests (N=2^14 ≈ tens of ms per hash — honest, never a fake-hash path; tests never assert hash bytes, only accept/reject behavior; salts random per hash — determinism is at the behavior level, the mock-provider verdict lesson). No MockTransport needed — auth has no external calls; the conftest zero-network guard stays satisfied; mypy green with no new overrides (stdlib only). | The suite's contract is behavior over the real app (billing/billing_admin/entitlement tests are the template); a reduced-KDF test profile would fork the code under test — the exact drift the honest-surface rule forbids. | 0.85 |
|
||||
| D-097 | Gates = **no learner gates on employer routes; email verification + self-serve password reset deferred (honest out-of-scope)**: employer routes compose ONLY the employer session gate — no `require_verified_age`, no G-5 learner allowlist (employers are a third principal class, not learners; D-043's composition on the learner routes — apply/resume/sandbox — is untouched and keeps allowlist → verified 18+ → rate caps). Registration stores email UNVERIFIED (displayed as such; that is why claim evidence is advisory per D-089); no email/notification infra exists (the A-408 out-of-band precedent) so email verification and self-serve password reset are deferred and documented honestly in the surface copy; admin retains remediation authority via the operator routes + `employer_audit_events` trail. | Employer accounts are B2B adults buying posts, not gated minors — the 18+/16+ gates exist to protect learners on learner surfaces (D-043); inventing an email-verification path without any mail transport would be security theater (A-304: never present unverified as verified). | 0.85 |
|
||||
|
||||
---
|
||||
|
||||
## Milestone v0.9 — Payment Engine (COMPLETE, Feature, 2026-09-29 → 2026-09-30, shipped as v0.8.6)
|
||||
|
||||
**Scope (D-074, auto-decided at SPECIFY under full autonomy D-007; founder scope question left open post-v0.8):** Activates **REQ-F-018 — payment processing and subscription management** as REQ-9-001..005. The pricing page (REQ-017, critical since v0.1) gains its real backend: posting packages (single, bundle, enterprise), talent-access plans for employers, and subscription tiers — with checkout, provider webhooks, entitlement enforcement on posting creation, and a subscription lifecycle (renew/cancel/upgrade/downgrade) under a provider-agnostic protocol (D-014 mock-first pattern). Rationale: the guiding outcome (≥1,000 placements via the Nextcraft marketplace) requires a monetized, self-serve employer side; the marketplace engine (v0.6) built the supply side, ship-anywhere (v0.8) made the stack installable anywhere — payments is the only high-priority requirement deferred since v0.3 that directly unblocks employer self-serve (A-407: "real employer auth lands post-payments with the B2B employer network") and CIRR-style placement tracking needs paid postings to track. Confidence 0.80 ≥ threshold 0.60 (D-007).
|
||||
|
||||
**Out of scope (landed in v0.10):** real employer auth/accounts (B2B employer network), multi-currency, tax/VAT handling beyond passthrough, invoice PDF generation, Stripe-specific UI components, CIRR tracking (REQ-F-020, deferred).
|
||||
|
||||
**Success (milestone-level):** an employer (via admin-mediated flow at pilot) can purchase a posting package or talent-access subscription through a real Stripe-compatible checkout; webhooks update entitlements automatically; posting creation enforces package limits honestly; subscription lifecycle transitions (renew/cancel/upgrade) work end-to-end; the pricing page reflects real catalog data; all provider integrations are mock-first (tests never call Stripe). — ACHIEVED, all 5 REQ complete, shipped as v0.8.6 (release with full asset matrix + registry image per D-064).
|
||||
|
||||
## Clarify decisions (D-075+, CLARIFY stage full autonomy, 2026-09-29)
|
||||
|
||||
| ID | Decision | Rationale | Confidence |
|
||||
|----|----------|-----------|------------|
|
||||
| D-075 | Both providers ship in-milestone: `mock` (deterministic, default) and `stripe` (real Stripe-compatible REST via the shared httpx pool — checkout session create/retrieve, subscription cancel/upgrade, event signature verify). Tests exercise the stripe provider ONLY through `httpx.MockTransport` (the openai-audio/embedding precedent — byte-contract pinned, cloud never called); `AI_PAYMENT_PROVIDER=stripe` is env-gated and needs `AI_PAYMENT_API_KEY` — unset key falls back to mock with a loud G-11 boot log (voice/embedding factory pattern, `billing/factory.py` + main.py lifespan catch). | The D-030 openai-audio precedent: a real provider shipped alongside mock with MockTransport byte-contract tests. A protocol without a real implementation is unproven; a real implementation without mock is untestable. Pilot runs mock; the real path is config-away. | 0.80 |
|
||||
| D-076 | Buyer model at pilot = **operator-mediated** (extends A-401): checkout sessions are created via admin-token-gated routes (`/v1/billing/admin/checkout`) on behalf of an employer ref in the request body; no employer principals exist (A-407). Every purchase records `operator_ref` (admin token identity = "operator") + `employer_id`. Webhook route is public (server-to-server) but HMAC-verified (D-077). | Employers are data, not principals (A-407); the admin token dep (`require_admin` returning Settings) is the only operator identity mechanism in the codebase. Self-serve checkout requires employer auth — explicitly post-payments scope. | 0.85 |
|
||||
| D-077 | Webhook signature = **HMAC-SHA256 over `"{timestamp}.{raw_body}"`** (Stripe's own scheme shape), verified with `hmac.compare_digest`; secret from `AI_WEBHOOK_SECRET` env, **unset → per-boot random** (the `telemetry_ingest_secret` pattern) — meaning webhooks fail closed when unconfigured, which is correct for a payment surface. Missing/bad signature → 401, event_id dedupe via UniqueConstraint + check-first (the `insert_application` at-least-once pattern). Raw body read via `await request.body()` (the resume route is the nearest raw-request precedent). | The v0.4.7 ingest-token pattern (per-boot secret, compare_digest, length-prefixed binding) is the proven in-tree HMAC precedent; Stripe's timestamp.body scheme is the industry shape and tolerates clock skew via a replay window (±5 min, pinned by test). | 0.80 |
|
||||
| D-078 | Catalog = **seeded DB tables, not a code constant**: `packages` (one-time: single $49/1 post/30d, bundle $399/10 posts/90d each, enterprise) + `plans` (recurring: starter $99/mo, pro $299/mo, enterprise-talent custom) seeded idempotently at store init from a versioned code constant (the SeededCorpusSource `seed_corpus.json` pattern). Prices = integer cents, currency `usd` fixed. Purchases **snapshot** price/name at buy time (catalog edits never rewrite history). | The pricing page and admin checkout must read ONE source (data-driven tier copy, REQ-9-002); DB tables allow future admin price edits without code changes while snapshots keep financial history immutable. Code-constant-only would make REQ-9-002's "no hardcoded tier copy drifting from backend" harder to prove. | 0.80 |
|
||||
| D-079 | Pricing page = **server shell retained** (static-export metadata stays — A-411) + new client component fetching `/v1/billing/catalog` (the A-406 board-client pattern: AbortController, honest loading/error/empty states). CTA buttons at pilot = **contact path** (mailto + "operator-mediated checkout" copy) — NOT self-serve purchase (no employer auth, D-076). The real checkout trigger lives in the admin surface (operator buys on behalf of an employer). | Self-serve CTAs without employer auth would be a dishonest button; the admin-mediated purchase is the honest pilot flow. The pricing page's job in v0.9 is truth: real catalog data, real states. | 0.80 |
|
||||
| D-080 | Entitlement model: `purchases` table (one-time packages: `posts_limit`/`posts_used`/`expires_at` snapshot) + `subscriptions` (recurring plans). Posting creation consumes 1 post from the **newest non-expired purchase with remaining posts**, else 402 with upgrade-CTA dict payload (the `require_verified_age` dict-detail precedent); enterprise = manual unlimited grant (admin route, provenance-recorded). Store tables (7th D-027 family): `packages`, `plans`, `purchases`, `subscriptions`, `subscription_events`, `checkout_sessions`, `billing_events` — `purchases` + `billing_events` added to the REQ-9-001 list (one-time packages + full audit trail need their own tables; a one-time purchase is NOT a degenerate subscription). | Mixing one-time packs into subscriptions overloads the state machine (REQ-9-005's states don't apply to a 10-post bundle); the identity/store.py + marketplace/store.py replication pattern makes an 7-table store cheap. | 0.75 |
|
||||
| D-081 | Money = **integer cents** end-to-end (pydantic `int` with ge=0, TS `number` documented as cents); no floats anywhere in billing; currency `usd` only (multi-currency out of scope). | Float money is the classic billing bug; Stripe itself is cents-native. The dual-schema pins (pins.test.ts) enforce the int wire type both sides. | 0.90 |
|
||||
| D-082 | Mock checkout completion = **a mock-provider-only confirm route** (`POST /v1/billing/mock/complete`, mounted ONLY when provider=mock): it mints a properly-signed webhook event and POSTs it through the REAL webhook processing path (fetch to self or direct handler call — same code path, proven by test). This is the E2E payment proof without cloud. The route 404s when provider=stripe. | The pipeline must be provable end-to-end in tests (webhook → dedupe → entitlement); a confirm route mounted only in mock mode is the same shape as the sitemap's env-gated routes and can never be hit on a real deployment (factory mounts it conditionally). | 0.70 |
|
||||
| D-083 | Subscription lifecycle: states `trialing → active → past_due → cancelled|expired` (+ `upgraded`), transitions driven ONLY by webhook events or admin manual actions. **Event names are OUR canonical names, normalized at the provider boundary** (R-43: Stripe has no `subscription.renewed`/`checkout.completed`): `checkout.completed` ← `checkout.session.completed`; `invoice.paid` ← `invoice.paid` (also accept `invoice.payment_succeeded`); `subscription.cancelled` ← `customer.subscription.deleted`; `subscription.renewed` ← `invoice.paid` on a renewal invoice (corroborated by `customer.subscription.updated`); provider event ids stored + deduped verbatim (never normalized). Completion predicate `status=complete AND payment_status=paid` (R-37). `past_due` grace = 7 days (module constant, test-pinned) before expiry downgrade; renewals extend `expires_at` idempotently (event_id dedupe makes double-fire safe); manual grant/revoke records `source: "manual"` + operator provenance in `billing_events`. Invalid transitions → `InvalidTransition` → 409 (store precedent). | At-least-once webhooks + idempotent state transitions is the A-309 seq-lease lesson applied to billing; manual remediation must be audit-visible (REQ-9-005's "recorded as manual events with provenance"). Research R-41..R-45 verified the state/event mapping against Stripe's actual enums. | 0.80 |
|
||||
| D-084 | Wire discipline: billing types in `packages/types/billing.ts` + pydantic contracts in `ai_service/billing/base.py`, dual-schema pinned field-for-field in `apps/web/tests/pins.test.ts` (G-108 discipline); `engine-client.ts` gains `listBillingCatalog()` + a `PaymentRequiredError` (402, dict-detail discrimination — the existing `parseError` VerifyRequiredError precedent). | Every prior cross-boundary wire (PostingSummary, SearchResult, facets) is pinned this way; the 402 CTA payload must be machine-readable for the admin surface. | 0.85 |
|
||||
|
||||
## Milestone v0.8 — Ship Anywhere (COMPLETE, Feature, 2026-09-27 → 2026-09-29)
|
||||
|
||||
**Scope (founder directive 2026-09-26, options 1–4 confirmed + scope decisions answered 2026-09-26/27):** Release and deployment guides for admins. nextcraft must be installable on any Debian-based Linux server, via every mainstream channel, with release consistency enforced by automation:
|
||||
|
||||
1. **Docker container** (Option 1): packaged image with all dependencies pre-loaded, built and uploaded to the Gitea container registry on every phase release
|
||||
2. **Release binaries** (Option 2, extends the existing v0.4 line): the CLI published as binaries in every release's downloads — x64 (existing) plus arm64, plus .deb packages
|
||||
3. **CLI full suite** (Option 3): the CLI contains the full application suite for easy onboarding — one binary takes an admin zero → running
|
||||
4. **Release consistency** (Option 4): pre-commit hooks + Gitea Actions pipelines so every release is made the same way
|
||||
|
||||
**Founder scope decisions (2026-09-26/27):** deb + arm64 both added (packaging breadth); docker container unprivileged-by-default with honest sandbox degrade (userns conflict documented, privileged variant documented); Gitea act_runner stood up by this milestone on founder-provided LXC CT 113; one milestone, 6 phases (P0 + P1–P5 + P6 final). Three org-level runners on CT 113 as docker containers (gitea/act_runner, sibling-container mode, host docker.sock, hybrid labels ubuntu-latest + node/python/release).
|
||||
|
||||
**D-073 (founder directive, 2026-09-29, post-milestone):** CT 113 root password + the org runner registration token are **RETAINED for future use — rotation declined**. This supersedes the G-206 rotation request carried in the v0.7.6 ship notes. Both are stored in the sanctioned secrets file (`.ciagent/.env.secrets`: gitignored, 0600, the only copies — verified by tree-wide scan). The live-host deploy key (`nc_ci_deploy` → root@10.0.0.105) is likewise persistent per founder approval. Security posture: the credentials never appear in git history, workflows, container layers, or release assets; access remains SSH-key-gated on both hosts.
|
||||
|
||||
**Out of scope:** Windows/macOS targets, apt repository hosting (debs ship as release assets), Helm/K8s deployment, Postgres migration, browser E2E tooling.
|
||||
|
||||
**Success (milestone-level):** `docker run` / `.deb install` / one-liner binary each take a fresh Debian server to a running verified stack; every release carries binary + checksum + deb + image tag automatically; a push to any branch runs lint/typecheck/tests on the runner fleet; admins have a single deploy guide covering all channels with upgrade + troubleshooting paths.
|
||||
|
||||
## Clarify decisions (D-063+, CLARIFY stage full autonomy, 2026-09-27)
|
||||
|
||||
| ID | Decision | Rationale | Confidence |
|
||||
|----|-----------|-----------|------------|
|
||||
| D-063 | CI push lane = lint + typecheck + **fast lanes only**: `ai:test:fast` (`-m "not real_server and not userns"`), default `web:test` + `cli:test` (E2E stays env-gated out). `real_server`/`userns` NEVER run in job containers. | `userns` is blocked by docker's default seccomp in the sibling-container runners (nested `unshare --user` fails — infeasible, not merely slow); `real_server` is feasible (plain uvicorn processes) but the locked spec wording is "pytest fast lane" and push feedback stays minutes-scale; both lanes remain local/verify-lane surfaces. The CLI E2E (900s real bootstrap) is already env-gated out of `cli:test` (D-055) and stays out | 0.85 |
|
||||
| D-064 | Tag-triggered Actions workflow is the **exclusive owner** of release assets + registry push (builds x64/arm64 binaries, checksums, debs from the tag checkout; consumes Actions secrets `GITEA_TOKEN`/`REGISTRY_TOKEN` set via the self-serve PUT mechanism). The local ship path stops calling `scripts/release-assets.sh` once the workflow is green; the script remains as the manual fallback (its .env-only token resolution unchanged). | Exclusive ownership kills the double-attach race (ship workflow + tag trigger would both upload the same asset names); the job must build from a clean tag checkout anyway (dist/ artifacts don't exist in the job), so the script's attach-what-exists logic can't serve the workflow; "release consistency enforced by automation" (founder Option 4) means the pipeline, not the founder's box, is the release maker | 0.75 |
|
||||
| D-065 | Image = `git.coreci.dev/coreci/nextcraft`, tags = the release semver (`v0.7.x`) on every phase ship + `latest` moved to the newest release. No per-push/branch tags. Admin guides pin semver; `latest` is a convenience alias only. | The Gitea registry is owner/repo-scoped (`{instance}/{owner}/{repo}`) — no other image name is addressable, so the naming question has exactly one answer; per-push tags would churn the 32 GB runner disk for zero admin value while the release-consistency scope is phase releases | 0.85 |
|
||||
| D-066 | The runtime image embeds the CLI binary (`/usr/local/bin/nextcraft`): `docker exec` gives admins in-container `doctor`/`verify`/`guide`; `bootstrap`/`deploy`/`dev` detect the container env and refuse honestly, pointing at the docker-channel paths. | REQ-8-004's "detects container env + suggests volume/privileged facts" is only truthful from inside the container (uid_map/seccomp/userns reality lives there — a host-side CLI cannot probe it without knowing the container name); ~117 MB on an image already carrying a venv + static export is acceptable; refusing beats repo-surgery-in-container (git clone + rebuild inside an immutable image is the anti-pattern the docker channel exists to remove) | 0.80 |
|
||||
| D-067 | deb = CLI binary at `/usr/bin/nextcraft` + package docs, nothing else. NO systemd unit in the deb — `deploy --service` owns service-unit generation (it bakes the real clone + binary paths; a deb unit would be stale/wrong). No proxy config fragments (HAProxy recipes are per-box, live in the deploy guide). postinst runs `nextcraft doctor` best-effort as an **advisory** summary (output printed, exit never fails the dpkg transaction) + prints next-steps (`doctor` → `deploy --service`). | The deb packages the CLI, not the stack — the unit's paths only exist after deploy chooses a clone dir; postinst failing on missing node/pnpm would brick unattended installs while advisory doctor output still tells the admin what to fix (honest-loud, never blocking — the install.sh degradation pattern) | 0.75 |
|
||||
| D-068 | Degrade detection = **capability probe**, not markers: at boot + per environment-create the backend attempts a real `unshare --user --map-root-user` (the proven `tests/helpers/userns.py` pattern promoted into the sandbox module); `/.dockerenv` + an `NC_CONTAINER=1` entrypoint var only flavor the remediation text. Degrade semantics: learner environment creation **rejects honestly** (503 + pointer to the documented privileged/seccomp variant) + loud boot notice + doctor surfaces the degraded capability; chat/agents/grading keep working. This supersedes A-101's cwd-jail fallback for deployed schools. | The probe is the truth source — `/.dockerenv` says "docker" but not "userns blocked" (a privileged container passes userns and must NOT degrade; a non-docker host with userns disabled must); unisolated learner code would run as the service user next to provider keys + network — the namespace fabric exists precisely to prevent that, so the A-101 fallback (founder-box convenience) is unacceptable for untrusted multi-learner deployments; reject-with-remediation is the honest degrade | 0.70 |
|
||||
| D-069 | arm64 = **cross-build on the x64 runner, no QEMU for the build**: the SEA pipeline is arch-agnostic file surgery (esbuild bundle + postject into an arm64 node base binary fetched as a build input); x64 + arm64 binaries and debs build in one release job in minutes. QEMU binfmt (if registered on CT 113) adds an optional `--version` smoke of the arm64 binary; `dpkg-deb` never executes its payload, so the deb needs no emulation. No multi-arch docker image in v0.8 (spec scope is binaries + debs; image stays amd64). RESEARCH verifies the arm64 node base + binfmt availability. | Emulated builds are the slow path and unnecessary: SEA injection never runs the target binary; the only execution need is a smoke, which binfmt provides opportunistically; multi-arch manifests would pull image-arm64 into a scope the spec never asked for | 0.70 |
|
||||
| D-070 | pre-commit = ONE root `.pre-commit-config.yaml` for the whole monorepo, directory-scoped (ruff on Python paths, eslint/tsc via workspace-scoped invocations), local-only, no network at hook run. On boxes without the toolchain, hooks **fail loudly** — never silently skip; `--no-verify` is the visible deliberate bypass and CI is the authoritative backstop. | A silently-skipped hook lies about release consistency (Option 4's whole point); pre-commit is inherently a repo-root single config — per-workspace configs buy nothing; CI remains the enforcement gate so a bypassed local hook cannot ship unlinted code | 0.80 |
|
||||
| D-071 | Docker-channel upgrade = `docker pull <tag>` + **recreate** the container (state persists via the `~/.nextcraft` volume) — `deploy`/`bootstrap` never run in-container (D-066 refuse). Binary channel = re-run install.sh + `deploy --service`; deb channel = `dpkg -i` the new deb + `deploy --service` (both idempotent). doctor/guide describe the detected channel's upgrade path. | The image is immutable — deploy-from-repo inside a container is meaningless by construction; the other two channels keep the v0.4.6 idempotent deploy as their upgrade rail (proven on the live box) | 0.85 |
|
||||
| D-072 | `nextcraft guide` = **offline-first embedded text** (man-page style): channel-aware sections checked into the CLI source (mirrored from deploy/README.md at build time), zero runtime fetches; forge URLs print as pointers to the living runbook. | Fresh-box and degraded installs (no network, no clone) are exactly when guidance is needed; the binary is self-contained by design (SEA) — embedding honors that; runtime fetching would make the guide fail precisely on broken boxes | 0.80 |
|
||||
|
||||
## v0.8 research conclusions (Phase 0 RESEARCH, 2026-09-27)
|
||||
|
||||
Full evidence + concrete commands live in `.ciagent/RESEARCH.md` (R-1..R-32); architecture in ARCHITECTURE.md's "v0.8 Distribution & Release Architecture" section. Brief conclusions:
|
||||
|
||||
1. **Runner image = `gitea/runner:3`** — Docker Hub's `gitea/act_runner` is deprecated/renamed; runner 3.x needs Gitea ≥1.21 (instance probe-verified at 1.26.1). Basic flavour + host docker.sock = exactly sibling mode.
|
||||
2. **3 containers, one per lane, each with its OWN `/data` volume** (`.runner` registration file — shared = lock refusal + identity loss). Registration token via env at bring-up only (founder chat-provided, reset after fleet green, never in any repo file). Org-level scope comes from the token itself.
|
||||
3. **Labels via `GITEA_RUNNER_LABELS` env** (passed to both register + daemon): `node:docker://node:24-bookworm`, `python:docker://python:3.11-bookworm`, `release:docker://node:24-bookworm`, `ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest`. Unmatched `runs-on` silently runs on a default image — always pin our labels.
|
||||
4. **Capacity 1 per runner** (config default; also bounds docker subnet consumption). Fleet concurrency = 3.
|
||||
5. **Cache requires `cache.host` + published port** on dockerized runners (per-job networks make the auto-detected cache address unreachable — actions/cache ETIMEDOUTs otherwise); 32 GB guarded by cache caps + health_check disk floor + host prune cron. `actions/cache@v4` works on runner 3.x (cache v2 API, stock action self-patched).
|
||||
6. **Sibling-mode path alignment NOT needed for our jobs** (no bind_workdir; `docker build` TARS context over the API); documented as the gotcha if a future workflow bind-mounts.
|
||||
7. **Push lane = repo's own scripts, no turbo lint wholesale** (it pulls venv-dependent ai-service tasks): node lane (corepack pnpm 12.3.4, eslint+tsc per TS workspace, web/cli node:test) + python lane (bootstrap.sh venv, ruff, mypy, FAST=1 test.sh). D-063's fast-only rule is both policy and physical (docker default seccomp blocks nested unshare). ubuntu-latest lane = dispatch verification only.
|
||||
8. **Release lane = tag-triggered exclusive owner**: builds from the tag checkout, stamps with `gitea.ref_name`, attaches assets via the same REST API scripts/release-assets.sh uses (NOT actions/upload-artifact); reuses the API in-workflow, script untouched as manual fallback (.env-only token resolution preserved).
|
||||
9. **Automatic job GITEA_TOKEN cannot publish packages (upstream-confirmed)** — registry push needs the `REGISTRY_TOKEN` PAT secret (self-serve PUT first, founder UI fallback). `docker/build-push-action` unnecessary: plain login+build+push for a single-arch amd64 image.
|
||||
10. **Dockerfile**: web-export stage (NEXT_PUBLIC_AI_SERVICE_URL=self, build-time-only) + venv stage (prod deps) + `python:3.11-slim-bookworm` runtime (~400-500 MB incl. the ~117 MB CLI embed, D-066): non-root, urllib `/health` healthcheck (no curl in slim), `VOLUME /home/nextcraft/.nextcraft` (state root is `~/.nextcraft` by config default, env-overridable with ~-expansion), zero AI_* env at build, all keys optional at runtime (G-11).
|
||||
11. **deb**: `dpkg-deb --root-owner-group -Zxz` staging build (never executes payload — cross-arch free), Version = tag minus `v`, empty Depends + `Recommends: nodejs, python3, git, util-linux` (soft deps), postinst advisory doctor (exit 0 always), no unit, no proxy fragments (D-067).
|
||||
12. **arm64 cross-build QEMU-free**: the only arch-specific step in build-binary.mjs is `copyFileSync(process.execPath, out)` — replaced by a copy from the official arm64 node tarball (probe: nodejs.org dist 200, 30 MB); esbuild bundle + postject ELF-note injection are arch-agnostic file surgery. SEA blob cross-arch needs a P3 probe (config has no useCodeCache/useSnapshot — good sign); binfmt `--version` smoke optional per D-069.
|
||||
13. **D-068 degrade**: probe promoted verbatim from tests/helpers/userns.py into `ai_service/sandbox/capability.py`; boot notice copies the main.py G-11 warning pattern; per-create rejects 503 via a new SandboxUnavailableError arm beside the existing PoolFullError→503 (today it 500s); doctor upgrades `which unshare` to a real userns attempt. Markers only flavor text.
|
||||
14. **install.sh arch-resolve**: `x86_64`→x64, `aarch64|arm64`→arm64; sha256+version integrity gates stay byte-identical per arch.
|
||||
15. **Admin-guide structure** (REQ-8-005): per-channel chapters (docker/deb/binary) each ending in the SAME verification checklist, an upgrade matrix table (D-071 semantics), symptom/cause/fix troubleshooting tables, where-state-lives table per channel, sibling-mode accepted-risk row, SEA/corepack/cache-egress probe outcomes recorded. Ecosystem scan: Ollama installer UX, Tailscale per-channel doc skeleton, Gitea/act_runner's own dual distribution, Miniflux binary+image+deb shape.
|
||||
|
||||
## Milestone v0.7 — Test Harness Review (COMPLETE, shipped as v0.6.6, 2026-09-26)
|
||||
|
||||
**Scope (founder directive post-v0.6, scope decisions answered 2026-09-26):** Once v0.6 shipped, review the entire test harness — make sure the right tests are being run, make them effective, cut tests that provide no real value, review all warnings, and address all of it within this single milestone. NFR refinement-only: zero `feat:` commits (D-001 purity gate at final phase).
|
||||
|
||||
**Baseline research (completed at phase 0, 2026-09-26):** 785 tests (671 pytest / 53 web node:test / 61 CLI node:test), all green. Coverage unmeasured anywhere. No framework-level timeouts (20-30s polling loops, 600s CLI bootstrap step). Zero pytest markers despite a real split (fast-unit vs real-uvicorn vs real-userns). Web tests outside the turbo pipeline. 1,716 warnings per pytest run (~99% from 14 `session.query()` SQLModel deprecation sites; 14 ResourceWarning sites from unclosed UploadFile/sockets/file). All 81 Python `# noqa` inert (rules not in the ruff select-set); 5 orphaned eslint-disable comments; **no ESLint exists at all** (turbo lint is a no-op for 5 of 6 packages). `recharts@2.15.4` deprecated direct dep. No Python type checker (27 inert `# type: ignore`). Stale PLAN.md counts (488/22/61 vs actual 671/53/61).
|
||||
|
||||
**Founder scope decisions:** D-048 ESLint included; D-049 recharts 2→3 included; D-050 source-pin triage (cut trivial, keep real regression pins); D-051 mypy included.
|
||||
|
||||
**Out of scope:** Gitea Actions CI YAML (the CIAgent verify hooks are the CI), Playwright/browser E2E tooling, new features of any kind.
|
||||
|
||||
**Success (milestone-level):** pytest runs green with zero first-party warnings; the fast-subset (`-m fast`) completes in seconds; coverage measured and reported in all three workspaces; `turbo lint` exercises all 6 packages; all suppressions (noqa/eslint-disable/ts-ignore/type:ignore) are enforced-and-meaningful or gone; the suite is smaller (trivial tests cut) with no loss of regression value.
|
||||
|
||||
**Clarify decisions (D-052+, CLARIFY stage full autonomy, 2026-09-26):**
|
||||
|
||||
| ID | Decision | Rationale | Confidence |
|
||||
|----|----------|-----------|------------|
|
||||
| D-052 | Coverage tooling: **pytest-cov for ai-service; c8 for web/cli** wrapping the existing `tsx --test` runner — no vitest/nyc/istanbul, no runner swap | c8 handles tsx source maps and adds one devDep per workspace; node's `--experimental-test-coverage` mis-reports through the tsx loader; vitest adoption is a framework swap, not NFR refinement | 0.85 |
|
||||
| D-053 | Marker taxonomy: registered `fast`/`real_server`/`userns` under `--strict-markers`; **unmarked tests default to fast**; canonical fast run = `-m "not real_server and not userns"` (aliased `ai:test:fast`); `real_server`/`userns` always exclude from the fast subset | Zero-churn — no need to stamp ~600 unit tests with `fast`; the exclusion markers carry the selection; strict mode catches typo'd markers that would silently deselect | 0.85 |
|
||||
| D-054 | pytest-timeout: **`signal` method**, global 60s (comfortably above the 20-30s polling loops), conftest collection hook bumps to 180s for `real_server`/`userns`-marked tests; `@pytest.mark.timeout` for known outliers | Linux-only target makes SIGALRM safe and accurate; the thread method can't cancel blocked calls; 60s bounds a hang without false-positiving real polling | 0.80 |
|
||||
| D-055 | CLI fresh-clone E2E: **keep the 600s real bootstrap as-is** (the real path is the value), give it an explicit per-test 900s budget (600 + slack), env-gate it out of the default `cli:test` run with a dedicated `cli:test:e2e` script for the verify/ship path | A 10-minute E2E on every run is wrong for feedback loops but too valuable to cut; opt-in keeps both properties | 0.85 |
|
||||
| D-056 | filterwarnings: **`error` as the base action** (new first-party warnings fail) + a narrow enumerated allowlist of upstream ignores each scoped by exact message + emitting module (fastapi/starlette TestClient httpx warning, anyio BlockingPortal); no blanket category ignores, no ignores on `ai_service.*`; first-party ResourceWarnings fixed at source, never ignored | Error-by-default makes warning hygiene self-enforcing; module-scoped ignores cannot mask first-party regressions; after REQ-7-003 cleanups the non-ignored set is empty so no dedup machinery needed | 0.85 |
|
||||
| D-057 | Source-pin triage rule (operationalizes D-050): **KEEP a pin iff** it encodes a documented regression lesson (E87-style) or verifies cross-boundary wiring invisible to typecheck/build/behavioral tests (runtime fetch wiring, env keys, gate composition, JSON-LD presence); **CUT a pin** whose assertion duplicates another test or merely asserts code says what it says; kept pins move to a single `pins.test.ts` naming their regression | Falsifiable per-assertion; expected outcome ~5-8 kept of the 21 pin sites; cuts tautologies with zero regression loss | 0.75 |
|
||||
| D-058 | mypy strictness: **relaxed-first baseline** — `check_untyped_defs`, `warn_unused_ignores` (each of the 27 ignores either types or is removed), `no_implicit_optional`, `warn_redundant_casts`, `warn_unreachable`; NOT `strict=true`; per-module tightening via `[[tool.mypy.overrides]]` available later | Full strict on a never-myped codebase turns P5 into an annotation retrofit; `warn_unused_ignores` delivers the milestone goal (suppressions meaningful or gone) at realistic cost | 0.85 |
|
||||
| D-059 | recharts 2→3 chart-surface verification = **`tsc --noEmit` green + `next build` static export green (SSR-renders the chart pages) + existing web tests green**; no jsdom/happy-dom render harness (new deps = framework addition); live mount probe on the deployed site as the operational smoke | Build+typecheck exercise the real React 19/recharts 3 integration through SSR; browser/render tooling is explicitly out of scope for v0.7 | 0.80 |
|
||||
| D-060 | Shared Python test helpers: **`tests/helpers/` package** (`KillableProxy`, `free_port`, userns probe, real-uvicorn `serve`, `wirePosting`) imported as `from tests.helpers...`; fixtures stay in conftest.py; changes are test-internal import paths only — no public/test-facing API beyond that | Helpers are direct utilities, conftest is for fixtures — explicit imports keep call-site dependency honest; `tests/` is already a package (`__init__.py` present) | 0.90 |
|
||||
| D-061 | Version drift: **pyproject.toml is the single version source** — bumped to `0.5.6` (the shipped line); `FastAPI(version=)` derives from `importlib.metadata.version("nextcraft-ai-service")` (safe fallback for un-installed runs); stale version comments removed in the same sweep | One hardcoded "0.3.0", one pyproject "0.2.0", and a shipped 0.5.x all drifted — metadata lookup makes drift structurally impossible | 0.90 |
|
||||
|
||||
## Milestone v0.6 — Marketplace Engine (COMPLETE, shipped as v0.5.6, 2026-09-26)
|
||||
|
||||
**Scope (auto-decided at SPECIFY per full autonomy — founder scope question left unanswered 2026-09-21):**
|
||||
|
||||
The marketplace has no backend today: every job board, job detail, employer profile, search, and pricing surface reads `packages/mock-data` directly; the only live route is the G-18 `POST /v1/marketplace/apply` gated stub (18+ verified, honest 501). The guiding outcome — ≥1,000 learners placed in AI-orchestration roles via the Nextcraft marketplace — runs *through* this surface, so the marketplace engine is the highest-leverage next build.
|
||||
|
||||
**Scope decision (D-047):** Activate **REQ-F-011..016** as REQ-6-001..006 — the real marketplace backend behind the existing UI:
|
||||
|
||||
1. **Job aggregation pipeline** (REQ-6-001, from REQ-F-011): job ingestion from employer postings + a seeded/synthetic corpus (3M+ external jobs is post-MVP — the pipeline ingests at pilot scale with the same adapter contract); postings store (6th D-027 store family); posting lifecycle (draft → published → closed).
|
||||
2. **AI-powered tagging** (REQ-6-002, from REQ-F-012): skills/seniority/category extraction from posting text via the existing LLM structured-output layer (D-020), provider-agnostic, mock-first.
|
||||
3. **Semantic vector search** (REQ-6-003, from REQ-F-013): embedding-backed role matching — candidate-competency vs posting-requirement similarity; SQLite-local at pilot scale (no pgvector before Postgres migration), deterministic embedding provider protocol (mock-first, real provider env-configured).
|
||||
4. **Company enrichment** (REQ-6-004, from REQ-F-014): employer profile enrichment from posting/company data (descriptions, social links, logo refs) — LLM-assisted, human-in-the-loop founder review deferred to post-pilot.
|
||||
5. **Resume/profile parsing** (REQ-6-005, from REQ-F-015): learner profile auto-fill from an uploaded resume — LLM structured extraction mapping to the competency profile shape.
|
||||
6. **SEO-optimized programmatic pages** (REQ-6-006, from REQ-F-016): server-rendered job/stack/landing pages with metadata, sitemap, structured data — built on the existing Next.js static-export single-port deploy (D-039-compatible).
|
||||
|
||||
**Deferred beyond v0.6 (unchanged):** REQ-F-018 payments/subscriptions, REQ-F-019 human tutor marketplace, REQ-F-020 CIRR placement tracking.
|
||||
|
||||
**Success (milestone-level):** the marketplace surfaces read from a real backend (postings, search, matching, employer profiles) instead of mock data; job posting + tagging + search work end-to-end for a pilot employer; the deploy stays one-command on the v0.4.x/v0.5.x fabric.
|
||||
|
||||
## Prior Milestone: v0.5 — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease (COMPLETE, shipped as v0.4.5; hotfixes v0.4.6 deploy-static-404, v0.4.7 pre-public hardening)
|
||||
|
||||
**Scope (the four seams D-016 deferred out of v0.4, locked at v0.5 Phase 0 SPECIFY):**
|
||||
|
||||
1. **Real server STT/TTS** (CUT-1/G-7 seam): implement the `openai-audio` VoiceProvider against OpenAI-compatible `/audio/transcriptions` (STT) + `/audio/speech` (TTS) — the D-030 protocol drop-in; mock stays first-class for tests; browser fallback stays for the no-key path. Voice defense (Examiner flows) gains the real server path end-to-end.
|
||||
2. **KYC/identity + age-gating backend** (REQ-F-017): real identity verification behind a provider protocol (mock-first, D-014 pattern), age-gate enforcement (school 16+, marketplace 18+ with verified identity) replacing the v0.1 visual-only flow, session/learner identity wired into the API surface (the G-5 allowlist evolves toward real identity).
|
||||
3. **Design/simulation sandbox environments** (REQ-F-021 remainder): extend the D-024/D-025 namespace fabric beyond the coding IDE — design-tool and simulation environment types alongside the existing build environment, one lifecycle, per-type telemetry.
|
||||
4. **Exec-telemetry seq-lease / replay-margin fix**: the one-line ACK gap documented in the P6 lesson (P07 review): reconnect replay margin so at-least-once ingest acknowledges received seqs and the capture agent resumes from the ack — closing the documented gap.
|
||||
|
||||
**Success (milestone-level):** voice defense runs on real server STT/TTS when keys exist; age-gating is enforced by the backend (not a mockup page); the sandbox fabric provisions design/sim environments; the reconnect-replay gap is closed with a regression test.
|
||||
|
||||
## Prior Milestone: v0.4 — Distribution & Bootstrap CLI (COMPLETE, shipped as v0.3.4; hotfixes v0.3.5 fresh-box experience, v0.3.6 single-port unattended deploy)
|
||||
|
||||
**Scope (founder directive, 2026-09-12):** Streamline installing Nextcraft. Ship a bootstrap CLI with a single-liner install script, and publish release binaries on an ongoing basis for every release going forward.
|
||||
|
||||
**Delivered:** `nextcraft` CLI — `doctor` (prerequisite checks), `bootstrap` (deps + venv + env from templates + key validation), `verify` (health check), `dev` (thin passthrough to scripts/dev.sh); one-liner install script downloading the linux x64 binary from the latest Gitea release with sha256 + version integrity gates; binary release pipeline attached to every ship from v0.3.2 onward; install/quickstart documentation backed by a fresh-clone E2E test. All 5 requirements (REQ-4-001..005) complete.
|
||||
|
||||
**Status of v0.3:** Complete and shipped (v0.2.8). Credential engines live: namespace-isolated sandbox fabric, live build telemetry, process-trace grading, seeded variants, oral defense; real learner build/defense/grading surfaces.
|
||||
|
||||
**Status of v0.2:** Complete and shipped (v0.2.0). Six AI tutor agents live over mockengine inputs (D-015).
|
||||
|
||||
**Deferred from earlier plan:** REQ-F-017 (identity verification + 16+/18+ age-gating) is explicitly deferred to a later milestone per founder directive. Age-gating remains the v0.1-style visual flow mockup; no real KYC backend is built in v0.3.
|
||||
**Deferred from earlier plan:** REQ-F-017 (identity verification + age-gating KYC), real server STT/TTS, design/simulation sandbox environments, and the exec-telemetry seq-lease are all deferred to v0.5. Age-gating remains the v0.1-style visual flow mockup.
|
||||
|
||||
**Tech stack:** v0.1 TS monorepo (pnpm/turborepo, Next.js) + v0.2 Python FastAPI ai-service + new credential-engine services (sandbox fabric orchestrator, telemetry ingest, grading engine) in Python/TypeScript as determined at RESEARCH.
|
||||
|
||||
---
|
||||
|
||||
## Requirements (Validated)
|
||||
## v0.4 Requirements (Complete)
|
||||
|
||||
The following requirements have been validated during specification and are locked for milestone v0.3 (REQ-F-007..010 and REQ-F-021 activated from the deferred pool; REQ-F-017 deferred per founder directive):
|
||||
All 5 v0.4 requirements (REQ-4-001..005) are complete and shipped as v0.3.4:
|
||||
1. Bootstrap CLI — `nextcraft` executable with `doctor` / `bootstrap` / `verify` / `dev` (REQ-4-001, REQ-4-002)
|
||||
2. One-liner install — `curl | sh` fetching the linux x64 binary from the latest Gitea release with sha256 + version integrity verification (REQ-4-003)
|
||||
3. Ongoing release binaries — every release from v0.3.2 onward ships the CLI binary + checksum as release assets (REQ-4-004)
|
||||
4. Install documentation — README quickstart + CLI reference verified by a fresh-clone E2E test (REQ-4-005)
|
||||
|
||||
1. Sandbox fabric — sandboxed IDE, design tool, and simulation environments with isolated execution and lifecycle management (REQ-F-021)
|
||||
2. Live build telemetry — in-environment capture of process events (keystrokes, commands, file diffs, run/test results) streamed to ai-service (REQ-F-010)
|
||||
3. Process-trace grading engine — grade artifacts from their process traces, not just final output (REQ-F-007); feeds the Assessor agent real inputs
|
||||
4. Variant task generation — per-learner task variants so no two learners receive identical prompts (REQ-F-008)
|
||||
5. Oral/voice defense — AI examiner conducts spoken defense of submitted work (REQ-F-009); feeds the Proctor/Mentor agents
|
||||
6. Agent re-grounding — Lab/Assessor/Proctor consume real engine inputs (telemetry, traces, defenses) instead of v0.2 mocks
|
||||
7. Learner surface integration — wire the v0.1 sandbox + assessment mockups to the real engines (build/run in-browser, live telemetry, live defense)
|
||||
## v0.3 Requirements (Complete)
|
||||
|
||||
## v0.2 Requirements (Complete)
|
||||
|
||||
All 12 v0.2 requirements (REQ-2-001..012) are complete and shipped as v0.2.0. See REQUIREMENTS.md traceability matrix.
|
||||
All 8 v0.3 requirements (REQ-3-001..008) are complete and shipped as v0.2.8. See REQUIREMENTS.md traceability matrix.
|
||||
|
||||
## v0.1 Requirements (Complete)
|
||||
|
||||
All 28 v0.1 requirements (REQ-001..028) are complete and shipped as v0.1.0. See REQUIREMENTS.md traceability matrix.
|
||||
|
||||
## Clarified Assumptions (v0.6 CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
|---|-----------|------------|-------------|
|
||||
| A-401 | Who authors postings at pilot (no employer auth exists)? | **Founder-authored; employer self-serve deferred.** Postings enter via founder/admin API routes (`/v1/marketplace/admin/postings` create + lifecycle transitions draft→published→closed), guarded by an `AI_ADMIN_TOKEN` env-header check (keys env-only, D-014; operator route — D-043 learner gate composition untouched). The employer dashboard's posting-management mockup stays mock; real employer self-serve lands with the deferred B2B employer network milestone. | 0.85 |
|
||||
| A-402 | Ingestion adapter contract shape? | **`PostingSource` protocol in `ai_service/marketplace/`** (D-031 module, D-014 mock-first): `fetch() -> Iterable[IngestPosting]` — pydantic-typed with `source`, `external_id`, employer + posting fields. Two v0.6 implementations: `seeded_corpus` (in-repo JSON seed, ported once from packages/mock-data jobs.ts/employers.ts per D-021 convention) and `admin_api` (A-401 founder path). Upserts idempotent on `(source, external_id)`. No scheduler/crawler — the 3M+ external corpus drops in later as a new adapter with zero store/route changes. | 0.80 |
|
||||
| A-403 | Where do embeddings live at SQLite scale (pip-only venv, no pgvector)? | **`embedding` table inside the marketplace SQLite store (6th D-027 family)**: posting_id → BLOB of struct-packed float32 + model + dim columns; the candidate side embeds the learner's competency profile text on demand (not stored). **Brute-force cosine in pure Python** at query time (pilot scale ≤ ~10⁴ postings — no numpy, no FAISS/chromadb, no new deps). `VectorStore` protocol-wrapped so the Postgres migration swaps to pgvector with no API changes (D-027/D-019 pattern). | 0.85 |
|
||||
| A-404 | Embedding provider + dimensionality? | **`EmbeddingProvider` protocol (mirror of D-030 VoiceProvider)**: deterministic mock (256-dim feature-hash of posting/profile tokens — similarity is deterministic; tests never call the cloud) + real OpenAI-compatible `POST {AI_EMBED_BASE_URL}/embeddings` via `AI_EMBED_MODEL`/`AI_EMBED_API_KEY` (env-only; ollama-cloud exposes /embeddings). Dimension is **not hardcoded** — persisted per row with the model name; on provider/model change, postings re-embed lazily. **[SUPERSEDED by D-051 probe: ollama-cloud does NOT expose /embeddings (404 path-not-found) — `AI_EMBED_BASE_URL` must target another OpenAI-compatible endpoint (e.g. local ollama `:11434/v1`); protocol design unaffected, default stays mock]** | 0.75 |
|
||||
| A-405 | Does semantic search replace the filter sidebar? | **Hybrid — augments, not replaces.** REQ-6-002 tag facets (skills/seniority/category + location/remote/salary) remain hard filters driving the existing sidebar; the semantic query + similarity compose as a **ranking layer over the facet-filtered set** (no facets selected → pure semantic ranking). Match score on job cards = cosine(candidate-competency vector, posting vector), replacing the mock `matchScore`; skill-breakdown panel derives from tag overlap + per-skill vector contribution. ARCHITECTURE.md's old "filter UI replaced" future-note is superseded by this hybrid. | 0.80 |
|
||||
| A-406 | What happens to packages/mock-data jobs.ts/employers.ts (runtime data vs static export)? | **Marketplace web surfaces switch from build-time mock imports to runtime client fetch** (A-002 direct-fetch + engine-client pattern, proven by build/defend surfaces): board/detail/employer-profile become client shells fetching `/v1/marketplace/*`; `generateStaticParams` on jobs/[jobId] and employers/[employerId] is removed (one static shell per route, data at runtime). jobs.ts/employers.ts are ported **once** into the Python seed fixture (A-402) and go dormant for the marketplace path (mock-data remains for admin/employer-dashboard mockups + shared types). No build-time DB reads — the D-039 deploy export build stays repo-only. | 0.80 |
|
||||
| A-407 | Employer auth model at pilot? | **None — employers are data, not principals, in v0.6.** Employer records are ingested/enriched entities keyed by `employer_id` on postings; no employer login, sessions, or accounts. The employer dashboard surface stays the v0.1 mockup; enrichment is founder-triggered/admin-reviewed (A-409), never employer-edited. Real employer auth lands post-payments with the B2B employer network. | 0.85 |
|
||||
| A-408 | Apply-flow shape beyond the 501 stub? | **The existing `POST /v1/marketplace/apply` route becomes real, D-043 composition unchanged** (G-5 allowlist → verified-18+ verdict → rate caps): body `{posting_id}` → posting must be `published` → insert an `application` row (status `submitted`, identity ref, timestamp) in the marketplace store → honest confirmation payload. Duplicate apply (same learner + posting) is idempotent. **No notification infra**: founder reviews applications via `/v1/marketplace/admin/applications` and contacts employers out-of-band. | 0.75 |
|
||||
| A-409 | Enrichment provenance rules? | **Separate `employer_enrichment` table — raw fields never overwritten** (REQ-6-004): append-only rows `(employer_id, field, value, source: llm\|manual, model, created_at)`. Read precedence **manual > llm > ingested-raw**; API responses merge and carry per-field provenance markers (A-304 honesty pattern — LLM-authored copy visibly marked). Enrich-on-post: new employer's first published posting triggers LLM structured extraction (D-020, mock-first) of description/social links/logo refs. Founder override = upsert a `manual`-source row, never a mutation of `llm` rows. | 0.85 |
|
||||
| A-410 | Resume upload format bounds? | **PDF/TXT/MD only, ≤2 MB, text-layer required.** PDF text via `pypdf` (pure-Python, pip-reachable through the A-009 venv-pip channel — the only new dep); extracted text capped ~100K chars before the D-020 structured-extraction call mapping to the competency-profile shape. Image-only/scanned PDFs → honest 422 ("text layer required") — **no OCR in v0.6**. Parsed result returns as a DRAFT: nothing persists to the learner profile until explicit learner confirm (REQ-6-005 review gate); raw resume bytes never stored, never logged (A-305 pattern). | 0.80 |
|
||||
| A-411 | Which pages get programmatic SEO treatment under static export — and how? | **Build-time-static set: landing, catalog/stack detail, marketplace board, pricing** — full server-component metadata (title/desc/OG) + JSON-LD (Organization, stack/Course, ItemList) rendered at export (D-039 build unchanged). **Runtime-data set: job detail + employer profile** — client shells (A-406) inject `JobPosting`/`Organization` JSON-LD + title client-side after fetch (best-effort per-job SEO at pilot; recorded as an accepted limitation — JS-less crawlers see the shell). **Sitemap/robots: `/sitemap.xml` + `/robots.txt` served dynamically by ai-service** (API routes beat the static mount, D-039), enumerating static pages + published postings; robots points at `/sitemap.xml`. | 0.70 |
|
||||
| A-412 | Pricing page — real or mock? | **Stays mock — zero backend.** Payments/subscriptions are REQ-F-018, explicitly deferred beyond v0.6 (D-047). The pricing page gains only build-time metadata/OG (A-411 static set); no pricing API routes, no package enforcement — posting is free at pilot (founder-mediated). | 0.95 |
|
||||
|
||||
## Clarified Assumptions (v0.5 CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
|---|-----------|------------|------------|
|
||||
| A-301 | Which STT/TTS endpoint for `openai-audio`? | **OpenAI-compatible `/audio/transcriptions` + `/audio/speech`** via `AI_VOICE_BASE_URL` (ollama-cloud does not expose audio endpoints; any OpenAI-compatible audio API works — the provider is endpoint-agnostic by config, D-014 pattern). Model via `AI_VOICE_MODEL` (default `whisper-1` STT / `tts-1` TTS class names, configurable). Keys env-only, never committed. | 0.75 |
|
||||
| A-302 | Does real voice replace browser fallback? | **No — layered.** `openai-audio` when configured, browser-native second-class fallback (CUT-1 keeps it first-class for the no-key path), deterministic mock for tests. The defense surface probes provider capability and degrades with a visible badge (mic mock vs browser vs server). | 0.85 |
|
||||
| A-303 | KYC vendor in v0.5? | **No vendor — protocol + mock-first backend.** `IdentityProvider` protocol (submit/poll/verify), deterministic mock (approve-on-policy), SQLite store. A real vendor (Stripe Identity/Persona/Onfido class) drops in later without API changes. Solo-founder economics: no vendor spend before pilot. | 0.85 |
|
||||
| A-304 | What counts as "verified 18+" for marketplace? | **Provider verdict = DOB-verified 18+; until a real vendor exists the mock verdict is explicit and marked `mock` in API responses** so downstream surfaces can label unverified state honestly (never display mock-verified as production-verified). | 0.80 |
|
||||
| A-305 | PII storage? | **Server-side only, minimal**: document refs + provider verdicts in SQLite; raw documents NEVER stored, NEVER logged (payload scrubbing pinned by test). Enrollment DOB stored as derived age-band, not raw DOB where possible. | 0.85 |
|
||||
| A-306 | Age-gate enforcement points? | **School enrollment (16+)**: identity submit → verify → age check at enrollment API. **Marketplace (18+ verified)**: gated marketplace routes check verified-identity verdict; unverified → 403 with verify-CTA payload. G-5 learner allowlist REMAINS as a pilot guard (identity doesn't replace sandbox rate caps). | 0.82 |
|
||||
| A-307 | What are "design" and "simulation" environments concretely? | **Same namespace fabric, typed starter contents + allowed commands.** `design`: canvas-style artifacts (files the learner edits: SVG/HTML/schematic text), Run = validator/renderer harness command; `simulation`: parameterized run harness (benchmark scripts + dataset files), Run = bounded harness execution. No new isolation tech — D-024 unchanged; the ENVIRONMENT is a typing over starter files + command policy. | 0.78 |
|
||||
| A-308 | Does the learner UI need new surfaces for design/sim? | **Reuse the build surface.** The existing /build flow accepts `environment` kind; the file tree, editor, Run/Test buttons, and read-only output panel (CUT-2) all carry over; only starter contents and command policy differ per kind. No new route groups; variants carry the kind. | 0.80 |
|
||||
| A-309 | Seq-lease protocol shape? | **Ingest acks highest-contiguous received seq per (learner, task) on the existing WS (JSON ack frame); the capture agent trims its spool to the ack and replays from there on reconnect.** Replay margin bounded (spool cap already exists). No new endpoint; G-3 flood semantics unchanged; acks are advisory hints, gap detection stays authoritative. | 0.80 |
|
||||
| A-310 | Where does identity live in the app? | **New `ai_service/identity/` module (D-031 pattern — extend ai-service, no new apps)**: provider protocol + mock, SQLite store (D-027 family), `/v1/identity/*` router. Web enrollment + marketplace surfaces call it via engine-client. | 0.85 |
|
||||
|
||||
## Clarified Assumptions (v0.4 CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
|---|-----------|------------|-------------|
|
||||
| A-201 | CLI language/toolchain for the binary? | **Probe-driven at RESEARCH** — Go → Rust → Node SEA → Python zipapp fallback chain; spec stays toolchain-agnostic so PLAN locks the probe-verified toolchain | 0.70 |
|
||||
| A-202 | Does bootstrap replace scripts/bootstrap.sh? | **No — reuse it.** CLI wraps existing `scripts/bootstrap.sh` + `scripts/dev.sh` via subprocess; zero orchestration logic duplicated in the CLI (thin passthrough pattern) | 0.85 |
|
||||
| A-203 | Where does the one-liner fetch the binary? | **Gitea latest-release API** (`/repos/{owner}/{repo}/releases/latest`) → download `nextcraft-linux-x64` + `.sha256` asset; repo raw serves `install.sh` as the stable URL | 0.80 |
|
||||
| A-204 | Install target + PATH? | **~/.local/bin** (XDG-style, no sudo), PATH hint printed when missing; `--dest` override flag | 0.85 |
|
||||
| A-205 | Binary "ongoing releases" scope? | **Every ship from v0.4 onward** attaches `nextcraft-linux-x64` + sha256 sidecar to the Gitea release — the ship workflow gains an asset step; retroactive binaries for old releases NOT required | 0.90 |
|
||||
| A-206 | No binary available yet / non-linux? | **Graceful degradation**: install script prints source-bootstrap instructions (git clone + scripts/bootstrap.sh) — never a hard fail | 0.88 |
|
||||
| A-207 | Checksum trust root? | **sha256 sidecar shipped as a release asset next to the binary** (same release, same channel); script verifies download against it. Signature/PKI out of scope for v0.4 (single forge, TLS transport) | 0.75 |
|
||||
| A-208 | Which prerequisites does doctor check? | node ≥18, pnpm ≥8, python3 ≥3.11, git, `unshare` availability (sandbox fabric needs it) — versions from the existing bootstrap tooling, not invented | 0.85 |
|
||||
| A-209 | Does `dev` manage multiple processes? | **No.** Thin passthrough to scripts/dev.sh only — the CLI stays bootstrap-scoped (D-016); orchestration remains in dev.sh | 0.82 |
|
||||
| A-210 | `.env.secrets` handling by bootstrap? | **Template copy only for `.env.example` → `.env`; secrets NEVER generated, NEVER committed; bootstrap validates presence of optional keys and warns (not blocks) when missing — mock-first providers keep the stack runnable** | 0.90 |
|
||||
|
||||
## Clarified Assumptions (v0.3 CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
|---|-----------|------------|-------------|
|
||||
| A-101 | Sandbox isolation technology? | **`unshare` user+mount+pid+net namespace subprocess isolation** per sandbox (probe-verified: in-ns uid=0, network fully isolated with 0 interfaces, writes land in an isolated bind-mounted workdir; proc-remount is not permitted in this context but is not required). No Docker/Podman/VMs — none present on the box; no sudo. A `SandboxBackend` protocol keeps a future containerd swap possible. Falls back further to a plain chroot-free subprocess with a cwd-jail if userns ever unavailable (tested path is userns). | 0.8 |
|
||||
| A-102 | Sandbox scope in v0.3? | **Coding IDE only** (web terminal + file tree + run/test). The "design tool" and "simulation" environments specified in REQ-F-021 are deferred to v0.4 — a single real build environment is enough to prove the credential pipeline end-to-end (telemetry → trace → grade → defense). | 0.75 |
|
||||
| A-103 | Live in-browser build UX? | **WebSocket xterm.js terminal** attached to the bwrap sandbox shell + HTTP file-tree/CRUD + run/test buttons. No full Monaco LSP in v0.3 — a code editor with syntax highlight (existing) + real shell is sufficient and far cheaper. | 0.72 |
|
||||
| A-103 | Live in-browser build UX? | **Run/Test buttons executing in the namespace sandbox + HTTP file-tree/CRUD + read-only exec-output panel** (CUT-2/G-8 — the interactive xterm.js shell relay is deferred to v0.4; `@xterm/*` is not a v0.3 dependency). No full Monaco LSP in v0.3 — a code editor with syntax highlight (existing) is sufficient and far cheaper. | 0.72 |
|
||||
| A-104 | Telemetry transport? | **WebSocket** from sandbox to a new ingestion endpoint on ai-service for live events; **SQLite-backed** ordered event log (`ai_service/telemetry/`) gives durability + at-least-once delivery + replay. Events carry monotonic `seq` per (learner,task) so gaps are detectable. | 0.8 |
|
||||
| A-105 | Where do traces live? | **SQLite** (`ai_service` data dir), introducing the first real persistence. SQLModel/SQLAlchemy for typed access. Chosen over Postgres because solo-founder + single box + low write volume; the `TraceStore` protocol is Postgres-migration-ready like SessionStore was. | 0.75 |
|
||||
| A-106 | Process-trace grading model? | **LLM-based grader**: structure the trace into a compact timeline digest (command categories, error/fix cycles, idle gaps, test passes) → Assessor-style rubric prompt → structured score via existing D-020 JSON defense. Deterministic features (test pass/fail, edit count) computed in code, not left to the LLM. | 0.7 |
|
||||
@@ -54,7 +268,7 @@ All 28 v0.1 requirements (REQ-001..028) are complete and shipped as v0.1.0. See
|
||||
| A-108 | Voice defense — STT/TTS providers? | **Provider-agnostic, mock-first like the LLM layer (D-014).** Real path: browser `MediaRecorder` → audio to ai-service → **OpenAI-compatible `/audio/transcriptions`** (Whisper STT) and **`/audio/speech`** (TTS) against ollama-cloud or a compatible endpoint; fallbacks: browser `SpeechRecognition`/`speechSynthesis` when no server keys. `VoiceProvider` protocol + deterministic mock (returns canned transcript) so tests never call a voice API. | 0.62 |
|
||||
| A-109 | Defense dialogue shape? | Reuse BaseAgent: an `Examiner` agent (seventh agent) streams examiner questions over the existing SSE pipeline; integrity signals (long pauses, off-scope answers, reading-from-notes cadence) emitted alongside the transcript to Proctor. | 0.8 |
|
||||
| A-110 | KYC / age-gating in v0.3? | **Deferred per founder directive.** No real identity backend. Age-gating stays the v0.1 visual flow mockup. Personas omit a security-engineer; security review via verifier + Phase 7 secrets-hygiene checklist. **Abuse control is NOT deferred with KYC (G-5):** v0.3 ships per-learner sandbox caps (`AI_SANDBOX_MAX_PER_LEARNER`), a global create-rate cap, and a server-side `learner_id` allowlist (`AI_LEARNER_ALLOWLIST`) so the unauthenticated surface cannot exhaust shared NPROC/disk. Documented in the release note. | 0.98 |
|
||||
| A-111 | New services vs extend ai-service? | **Extend ai-service**, don't fork new Python apps. Telemetry ingestion, trace grading, variant generation, voice, and sandbox orchestration all live as new modules in `apps/ai-service` (they share the LLM provider pool + config + session infra). Only the in-sandbox capture agent is a separate tiny Python process shipped into the bwrap environment. | 0.82 |
|
||||
| A-111 | New services vs extend ai-service? | **Extend ai-service**, don't fork new Python apps. Telemetry ingestion, trace grading, variant generation, voice, and sandbox orchestration all live as new modules in `apps/ai-service` (they share the LLM provider pool + config + session infra). Only the in-sandbox capture agent is a separate tiny Python process shipped into the namespace sandbox. | 0.82 |
|
||||
| A-112 | Sandbox on a single dev/school box — capacity? | v0.3 targets **1–5 concurrent sandboxes** (founder + pilot learners). No horizontal scaling, no queue. Concurrency guard returns 503 when full. Scaling is post-MVP. | 0.8 |
|
||||
|
||||
## Clarified Assumptions (v0.2 CLARIFY stage, full autonomy — auto-resolved)
|
||||
@@ -118,6 +332,7 @@ The following remain deferred beyond v0.3 and will be activated in subsequent mi
|
||||
|
||||
| ID | Decision | Rationale | Outcome |
|
||||
|----|----------|-----------|---------|
|
||||
| D-085 | v0.10 = B2B Employer Network (auto-decided at SPECIFY, full autonomy D-007; founder scope confirmed 2026-09-30) | Real employer auth/accounts + self-serve: registered employer accounts with sessions, employer-entity claim/verify, self-serve posting management (draft→published→closed on own entities), self-serve checkout (v0.9 catalog with employer-principal buyers, superseding D-076 operator mediation), employer dashboard real-data conversion. Rationale: A-407 sequences employer auth post-payments (payments shipped v0.9); A-401/D-076/D-079 deferred self-serve precisely pending this; the employer dashboard is the last mock-only surface; CIRR tracking (REQ-F-020) needs employer-side data. Confidence 0.80 ≥ threshold 0.60 (D-007). | Milestone scope locked at SPECIFY; tags on v0.9.x line; feature milestone; branch milestone/v0.10-b2b-employer-network |
|
||||
| D-001 | Milestone v0.1 = UI/UX prototype only, no business logic | Founder directive: validate UX before building backend. Prototype must be agreed upon before proceeding to business logic. | Scope locked to frontend surfaces with mock data |
|
||||
| D-002 | MAJOR 0 until MVP released | Founder directive: remain on v0.x until MVP is validated. Signals pre-release status. | All tags are v0.x.y until MVP agreement |
|
||||
| D-003 | TypeScript monorepo (pnpm/turborepo) + Next.js | Unified codebase for all 4 surfaces. Shared component library, types, mock data. Next.js App Router for route-based surface separation. Python AI services deferred to later milestones. | Monorepo structure with apps/web + packages/* |
|
||||
@@ -133,6 +348,10 @@ The following remain deferred beyond v0.3 and will be activated in subsequent mi
|
||||
| D-013 | v0.1 prototype founder-agreed; D-001 business-logic gate unlocked | Founder approved starting v0.2 with AI Tutor Architecture, which constitutes agreement of the v0.1 prototype per D-001. Recorded at v0.2 SPECIFY. | Business logic authorized from v0.2 onward |
|
||||
| D-014 | Provider-agnostic LLM layer; ollama-cloud as initial provider | OpenAI-compatible client abstraction with pluggable providers: ollama-cloud (https://ollama.com/v1, default), local OpenAI-compatible endpoint, deterministic mock (tests/CI). Keys in gitignored .ciagent/.env.secrets, never in code or commits. | apps/ai-service llm package with 3 providers; default=ollama-cloud |
|
||||
| D-015 | All six agents implemented as real LLM services; engines mocked | Coach/Tutor/Mentor fully real. Lab/Assessor/Proctor are real LLM logic over mock inputs (simulated telemetry, pre-baked artifacts) since sandbox fabric, assessment engine, and identity verification are v0.3+. Consistent with v0.1's mock-data approach. | REQ-F-001..006 complete in v0.2; real engines deferred to v0.3+ |
|
||||
| D-016 | v0.4 = Distribution & Bootstrap CLI (founder directive supersedes previously-named v0.4 seams) | Founder directive 2026-09-12: focus this milestone on streamlining install, a bootstrap CLI with a one-liner install script, ongoing release binaries. Real server STT/TTS, KYC, design/simulation envs, seq-lease move to v0.5. | Milestone scope locked at SPECIFY; binary = CLI-only, linux x64 |
|
||||
| D-074 | v0.9 = Payment Engine (auto-decided at SPECIFY, full autonomy; founder scope question unanswered post-v0.8) | Activates REQ-F-018 as REQ-9-001..005: posting packages (single/bundle/enterprise), talent-access plans, subscription tiers — checkout, provider webhooks, entitlement enforcement on posting creation, subscription lifecycle. Provider-agnostic protocol, mock-first (D-014 pattern). Rationale: ≥1,000 placements requires a monetized self-serve employer side; payments is the only high-priority REQ deferred since v0.3 that unblocks employer self-serve (A-407) and paid postings precede CIRR tracking. Confidence 0.80 ≥ threshold 0.60 (D-007). | Milestone scope locked at SPECIFY; tags on v0.8.x line; feature milestone; branch milestone/v0.9-payment-engine |
|
||||
| D-047 | v0.6 = Marketplace Engine (auto-decided at SPECIFY, full autonomy; founder scope question left unanswered 2026-09-21) | Activates REQ-F-011..016 as REQ-6-001..006: real marketplace backend (aggregation, tagging, vector search, enrichment, resume parsing, SEO pages). Rationale: the guiding outcome (≥1,000 placements via the marketplace) runs through the only surface with zero backend; every prior milestone fed the credential side, placement side is the bottleneck. Confidence 0.80 ≥ threshold 0.60 (D-007). External 3M+ job corpus scaled to pilot (adapter contract preserves the full-scale path). | Milestone scope locked at SPECIFY; tags on v0.5.x line; feature milestone |
|
||||
| D-039 | Single-port deploy + unattended dev (v0.3.6 hotfix, founder directive) | Only :8420 reachable behind HAProxy; site was down (domain root hit the API 404). Web app = static export served same-origin by the ai-service (`AI_WEB_STATIC_DIR`); `dev -d`/`stop`/`log` daemonize ops; durable state (DB, sandboxes, pid/log) moves to `~/.nextcraft/` out of the repo. | One port serves UI+API; unattended deploy recipe in deploy/README.md |
|
||||
|
||||
---
|
||||
|
||||
@@ -140,4 +359,4 @@ The following remain deferred beyond v0.3 and will be activated in subsequent mi
|
||||
|
||||
Nextcraft is being built by a solo founder using the CIAgent v0.7.0 autonomous pipeline. The vision document defines a 36-month, 3-phase roadmap to ≥1,000 placements. The first CIAgent milestone (v0.1) is intentionally scoped to UI/UX only — validating the product vision through interactive prototypes before any backend or business logic investment.
|
||||
|
||||
The four surfaces (Learner, Marketplace, Employer Dashboard, Admin) map directly to the four audiences in the vision: learners, employers, marketplace operators, and platform administrators. The prototype will demonstrate the complete user journey across all surfaces with realistic mock data reflecting the AI-era competency stacks, AI-orchestration job listings, and artifact+process trace+oral defense credential model.
|
||||
The four surfaces (Learner, Marketplace, Employer Dashboard, Admin) map directly to the four audiences in the vision: learners, employers, marketplace operators, and platform administrators. The prototype will demonstrate the complete user journey across all surfaces with realistic mock data reflecting the AI-era competency stacks, AI-orchestration job listings, and artifact+process trace+oral defense credential model.
|
||||
|
||||
+294
-49
@@ -1,5 +1,217 @@
|
||||
# Nextcraft — REQUIREMENTS.md
|
||||
|
||||
## v0.10 Requirements (Active — B2B Employer Network, Feature; started 2026-09-30)
|
||||
|
||||
**Milestone type: Feature** — real employer auth/accounts + employer self-serve (the B2B employer network; activated per D-085). Tags on the v0.9.x line (P0 → v0.9.0 … P6 → v0.9.6 = milestone release); branch `milestone/v0.10-b2b-employer-network`. Auth is session-based with hashed credentials (mock-first doctrine: no external auth provider, tests never call a cloud); admin operator routes are retained unchanged (D-043 composition untouched).
|
||||
|
||||
### Employer Auth Foundation
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-10-001 | Employer auth foundation: `ai_service/employers/` module (D-031 extend pattern): employer account store (8th D-027 family — `employer_accounts`, `employer_sessions`, `employer_claims`, `employer_audit_events` tables, D-095) with SQLModel typed access; password hashing via stdlib `hashlib.scrypt` (D-086 — zero new deps, per-password random salt, self-describing params); session tokens as server-side store rows with the token stored hashed, fixed 7-day TTL evaluated lazily, revoke-on-logout (D-087); register/login/logout/me routes (`/v1/employers/auth/*`, Bearer session transport per D-090); rate limiting on auth routes (D-088 — per-email failed-login window + global attempt cap, the G-5 in-memory pattern); credentials + secrets env-only (D-014); fail-closed when unconfigured | critical | 1 | complete |
|
||||
|
||||
### Employer Entity Claim
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-10-002 | Employer-entity claim/verify: an authenticated employer account claims an existing `EmployerRow` entity (the v0.6 ingested data) via a claim route; claims enter a pending state with recorded evidence (email-domain vs entity-website comparison — advisory signal, D-089) and activate via admin approval at pilot; claims recorded with provenance (`employer_claims`); one account : one entity (UniqueConstraint); admin routes for claim review/approve/reject (operator oversight retained, audit-logged per D-089); claimed-entity authorization: employers can only manage postings/purchases for their own entity | critical | 2 | active |
|
||||
|
||||
### Employer Account Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-10-003 | Employer dashboard real-data conversion: the employer surface switches from v0.1 mock imports to runtime client fetches (A-406 pattern — AbortController, honest loading/error/empty states) via the engine-client employer section + sessionStorage session state (D-092); employer auth UI (login/register pages, session-gated dashboard shell, honest logged-out states); posting management reads the employer's own postings via the authenticated surface (draft → published → closed lifecycle visible); employer profile read from the claimed entity (enrichment retained); talent search + candidate profile pages stay the v0.1 mock (no candidate backend exists — honest out-of-scope, D-092); wire types in `packages/types/employer.ts` dual-schema pinned (G-108 discipline, D-093) | critical | 3 | active |
|
||||
|
||||
### Self-Serve Posting Management
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-10-004 | Self-serve posting management: employer-principal posting routes (`/v1/employers/postings/*`) — create draft, edit own drafts, publish own drafts, close own postings, list own postings with status facets; authorization: the entitlement check composes AFTER the employer session gate (401/403 precede any 402 — the admin-gate-first order preserved, G-311) with the 402 CTA unchanged, D-080 consumption model intact, via the shared api-layer helper extracting the v0.9 gate (D-091 — no duplication); posting creation consumes from the employer's own packages (the v0.9 entitlement check parameterized by the session's employer_id instead of request body); the admin posting routes remain (operator path unchanged, provenance-recorded); 18+ gate N/A (employer surface is not learner-facing; the D-043 learner gates untouched, D-097) | high | 4 | active |
|
||||
|
||||
### Self-Serve Checkout
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-10-005 | Self-serve checkout + admin network overview: employer-principal checkout (`/v1/employers/billing/checkout`) creating provider sessions for the catalog (the v0.9 billing module reused — provider protocol, webhooks, entitlement grants unchanged; buyer = session employer instead of operator mediation, superseding D-076 while admin routes stay as the operator fallback; D-091 composes the existing checkout route logic with the session principal); entitlement view for the employer's own purchases/subscriptions (current package, posts used/limit, expiry — honest states); pricing page CTAs convert from contact-path to employer-login-path (superseding D-079); admin surface gains the network overview on the marketplace moderation page (employer accounts list, claim queue with evidence signals, per-account entity + entitlement state — the operator oversight lane, D-094) | critical | 5 | active |
|
||||
|
||||
---
|
||||
|
||||
## v0.9 Requirements (Complete — Payment Engine, Feature; shipped as v0.8.6)
|
||||
|
||||
**Milestone type: Feature** — payment processing and subscription management (REQ-F-018 activated per D-074). Shipped on the v0.8.x line (P0 → v0.8.0 … P6 → v0.8.6 = milestone release); branch `milestone/v0.9-payment-engine`. All 5 requirements complete: the billing module + 7th D-027 store + HMAC webhook core (REQ-9-001), the catalog + pricing surface (REQ-9-002), the checkout/webhook flow with per-session exactly-once grants (REQ-9-003), posting entitlement enforcement with honest 402s (REQ-9-004), the full subscription lifecycle + queryable audit log (REQ-9-005). — payment processing and subscription management (REQ-F-018 activated per D-074). Tags on the v0.8.x line (P0 → v0.8.0 … P6 → v0.8.6 = milestone release); branch `milestone/v0.9-payment-engine`. Provider-agnostic protocol, mock-first (D-014 pattern): tests never call Stripe; the real provider drops in by env config without API changes.
|
||||
|
||||
### Billing Foundation
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-9-001 | Billing foundation: `ai_service/billing/` module (D-031 pattern — extend ai-service, no new apps): `PaymentProvider` protocol (create checkout session, retrieve session, cancel subscription, upgrade/downgrade subscription, list subscription events) + deterministic mock provider (approves-on-policy, synchronous state machine); SQLite billing store (7th D-027 family — `packages`, `plans`, `subscriptions`, `subscription_events`, `checkout_sessions` tables) with SQLModel typed access; webhook signature verification (HMAC, per-boot secret default — the v0.4.7 ingest-token pattern); provider selection via `AI_PAYMENT_PROVIDER=mock\|stripe` env (keys env-only, never committed); test-only mode is the default (mock) so the stack runs with zero keys | critical | 1 | complete |
|
||||
|
||||
### Catalog + Pricing Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-9-002 | Package catalog + real pricing surface: seeded package catalog (single posting, bundle ×5, enterprise-unlimited tiers + talent-access plans: basic/pro/enterprise per REQ-017's existing pricing-page tiers) exposed via `/v1/billing/catalog`; the pricing page switches from mock imports to runtime client fetch (A-406 pattern) with honest loading/error states; package features/limits data-driven from the catalog (no hardcoded tier copy drifting from backend); build-time static metadata retained (A-411) | critical | 2 | complete |
|
||||
|
||||
### Checkout Flow
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-9-003 | Checkout + webhook flow: `POST /v1/billing/admin/checkout` (admin token-gated per D-076 — operator buys on an employer's behalf; buyer recorded as operator ref + employer ref) creates a provider checkout session for a catalog item; mock provider renders an in-app confirmation path; stripe provider redirects to Stripe Checkout (env-gated, never in tests); `POST /v1/billing/webhook` receives provider events with HMAC signature verification (reject 401 on bad signature — never process unsigned), idempotent event processing (event_id dedupe + per-session grant idempotency per G-303), entitlement granted on `checkout.completed` with state resolved from local rows only (G-302 — event payloads are untrusted ID references); webhook replay tolerance (at-least-once semantics — the A-309 seq pattern applied to billing) | critical | 3 | complete |
|
||||
|
||||
### Entitlement Enforcement
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-9-004 | Posting entitlement enforcement: posting creation (admin API route per A-401) checks the employer's active package — posting limit reached → honest 402 with upgrade-CTA payload (pointing at catalog/checkout); unlimited/enterprise bypass; expired/cancelled package degrades to the same 402 path; entitlement check composed after the admin token gate (operator route — D-043 learner gate composition untouched); admin surface shows per-employer entitlement state (active package, posts used/limit, expiry) in the existing admin marketplace moderation view | high | 4 | complete |
|
||||
|
||||
### Subscription Lifecycle
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-9-005 | Subscription lifecycle + billing admin: subscription states (trialing → active → past_due → cancelled/expired) with provider-driven transitions (webhook events `invoice.paid`, `invoice.payment_failed`, `subscription.cancelled`, `subscription.renewed`); renewals extend expiry idempotently; admin routes for manual grant/revoke/complement (operator remediation for failed webhooks — recorded as manual events with provenance); billing event log with full audit trail (every provider event + manual action stored, queryable); honest 402 + grace handling on payment failure (past_due grace window before downgrade) | high | 5 | complete |
|
||||
|
||||
## v0.8 Requirements (Complete — Ship Anywhere, Feature; shipped as v0.7.6)
|
||||
|
||||
**Milestone type: Feature** — new distribution channels (docker image, deb packages, arm64 binaries) and release automation (Gitea Actions, pre-commit). Tags on the v0.7.x line (P0 → v0.7.0 … P6 → v0.7.6 = milestone release); branch `milestone/v0.8-ship-anywhere`.
|
||||
|
||||
### Release Consistency Rails
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-8-001 | Release-consistency rails: three org-level Gitea act_runners stood up on founder LXC CT 113 (10.0.0.113, docker containers in sibling-container mode via host /var/run/docker.sock, hybrid labels ubuntu-latest/node/python/release, org-level coreci scope, registration token founder-provided and reset after fleet green), `.gitea/workflows/` lint+typecheck+test pipeline on push (v0.7 toolchain: eslint, ruff, mypy, pytest fast lane, web/cli node:test; CI test lanes = fast only per D-063: `ai:test:fast` `-m "not real_server and not userns"`, default web/cli node:test, CLI E2E env-gated out — real_server/userns never run in job containers), tag-triggered release workflow as the exclusive owner of release assets + registry push (D-064; local release-assets.sh remains manual fallback), single root `.pre-commit-config.yaml` (ruff + eslint + tsc local-only fast hooks, directory-scoped whole-monorepo, no network, secrets never in hooks or workflow env; hooks fail loudly on missing toolchain, never silently skip — D-070; CI is the authoritative backstop), runner cache-hygiene (docker prune cron + cache caps, 32 GB disk), hello-world dispatch verification across all three labels | critical | 1 | complete |
|
||||
|
||||
### Container Channel
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-8-002 | Docker image channel: multi-stage Dockerfile (deps layer: pnpm store + pip wheels pre-loaded; build layer: web static export with NEXT_PUBLIC_AI_SERVICE_URL=self; runtime: uvicorn :8420 serving API + export — single-port topology preserved; runtime image embeds the CLI binary at /usr/local/bin/nextcraft for in-container doctor/verify/guide — D-066), unprivileged default + honest degrade (capability probe at boot + per create — a real `unshare --user --map-root-user` attempt as the truth source, `/.dockerenv`/NC_CONTAINER only flavor remediation text; degraded = learner environment creation rejects 503 with privileged/seccomp remediation pointer + loud boot notice + doctor surfacing — D-068), healthcheck on /health, `~/.nextcraft` state volume, image pushed to the Gitea container registry as `git.coreci.dev/coreci/nextcraft` tagged semver release `v0.7.x` + `latest` on newest release (D-065), REGISTRY_TOKEN as Gitea Actions secret (self-serve PUT attempt first, founder UI fallback) | critical | 2 | complete |
|
||||
|
||||
### Debian-Native Channel
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-8-003 | Debian packaging: arm64 SEA binary + sha256 sidecar cross-built on the x64 runner without QEMU for the build (arch-agnostic esbuild+postject file surgery into an arm64 node base; optional binfmt `--version` smoke — D-069), .deb packages (dpkg-deb) for x64+arm64 = CLI binary at /usr/bin/nextcraft + package docs only, NO systemd unit (deploy --service owns unit generation) and no proxy fragments; postinst runs `nextcraft doctor` best-effort advisory (never fails dpkg) + prints next-steps (D-067); all attached to every release's downloads by the tag-triggered workflow (D-064); install.sh arch-gate becomes arch-resolve (x64 + arm64 binary paths); deb install + binary install both end at `nextcraft doctor` verify | high | 3 | complete |
|
||||
|
||||
### CLI Onboarding
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-8-004 | CLI full-suite onboarding: `nextcraft` alone takes an admin zero → running — doctor gains deb/docker-channel awareness (detects container env + suggests volume/privileged facts; detects deb install + knows its binary location; embedded in-container binary per D-066), `nextcraft guide` surfaces the admin runbook channel-appropriately as offline-first embedded text with forge pointers (D-072), docker-channel upgrade = docker pull + recreate (deploy/bootstrap refuse in-container — D-066/D-071), onboarding glue verified by an E2E test per channel (binary one-liner path exists from v0.4; docker + deb paths new) | high | 4 | complete |
|
||||
|
||||
### Admin Guides
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-8-005 | Admin release & deployment guides: deploy/README.md expanded into the full admin runbook — three install channels (docker / deb / binary+install.sh) first-install recipes, per-channel upgrade paths (docker pull + recreate / deb reinstall via dpkg -i + deploy --service / install.sh re-run + deploy --service, all idempotent — D-071), HAProxy + systemd recipes preserved, troubleshooting tables extended (runner, registry, docker degrade, deb), every recipe backed by a test or verified live-box evidence (house rule) | critical | 5 | complete |
|
||||
|
||||
## v0.7 Requirements (Complete — Test Harness Review, NFR; shipped as v0.6.6)
|
||||
|
||||
**Milestone type: NFR — refinement-only (D-001 purity gate: zero `feat:` commits).** Review the entire test harness: make sure the right tests are being run, make them effective, cut tests that provide no real value, eliminate all warnings, and address the findings within this single milestone. Founder scope decisions locked 2026-09-26: ESLint included (D-048), recharts 2→3 included (D-049), source-pin triage cut-trivial-keep-real (D-050), mypy included (D-051).
|
||||
|
||||
### Harness Wiring
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-7-001 | Test harness infrastructure: coverage measurement wired (pytest-cov for ai-service, c8 around the existing `tsx --test` for web/cli — D-052), pytest-timeout safety net (`signal` method, 60s global / 180s for `real_server`+`userns` via conftest collection hook — D-054; CLI fresh-clone E2E keeps its real 600s bootstrap, env-gated out of the default `cli:test` into a `cli:test:e2e` script with a 900s budget — D-055), markers `fast`/`real_server`/`userns` registered under `--strict-markers` with fast subset = `-m "not real_server and not userns"` (`ai:test:fast` script; unmarked tests default to fast — D-053), web tests inside the turbo pipeline (`web#test` task — package name is `web` + root `test`/`web:test` scripts), silent-skip fixes (path-invocation reports skip; env/global leaks fixed — `AI_PROVIDER` env mutation restored, conftest exception swallow narrowed), stale PLAN.md test counts corrected | critical | 1 | complete |
|
||||
|
||||
### Suite Composition
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-7-002 | Evidence-based suite composition: trivial/no-value tests cut per the D-057 rule (keep a source pin only if it encodes a documented regression lesson or cross-boundary wiring invisible to typecheck/build/behavioral tests — e.g. E87, runtime fetch wiring, gate composition; cut pins that merely assert the code says what it says; kept pins named after their regression), duplicated harness infrastructure consolidated (KillableProxy ×2, `_free_port` ×4, userns probe ×4, real-uvicorn serve boilerplate ×5, wirePosting ×2 → `tests/helpers/` package — D-060; fixtures stay in conftest), flake strategy replaces timeout-widening with deterministic waits where possible, leaked sandbox-droppings cleanup + `tests/sandboxes/` anchoring fixed | critical | 2 | complete |
|
||||
|
||||
### Warning Elimination
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-7-003 | Zero-warning pytest runs: `session.query()` → `session.exec()` at all 14 sites (marketplace/store.py ×12, identity/store.py ×2), resource cleanup for the 14 ResourceWarning sites (UploadFile close ×10 in api/defense.py + api/marketplace.py, socket close ×3 in scripts/sandbox-agent.py, file handle in tests/grading/test_engine.py), pytest `filterwarnings` policy (base action `error`; narrow module+message-scoped upstream ignores for the fastapi/starlette TestClient httpx warning and anyio BlockingPortal — no blanket category ignores, nothing ignored for `ai_service.*` — D-056), `fixture_loop_scope` pinned, dead `if False` branch removed | critical | 3 | complete |
|
||||
|
||||
### Lint & Suppression Hygiene
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-7-004 | Lint toolchain completion: ESLint flat config (typescript-eslint + react-hooks) for all JS/TS workspaces wired into `turbo lint` (currently a no-op for 5 of 6 packages, D-048), all 81 inert Python `# noqa` reconciled via RUF100 (dead ones removed), the 5 orphaned eslint-disable comments resolved under the real config, dead exports removed (ai-scenarios.ts, NavItem/ThemeConfig, Spawn, MicControl), stale version comments + `FastAPI(version="0.3.0")` drift fixed (pyproject.toml as single version source bumped to the shipped line; FastAPI version read from package metadata — D-061), `engines` fields added to package.jsons | high | 4 | complete |
|
||||
|
||||
### Type & Dependency Hardening
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-7-005 | Static typing for ai-service (D-051): mypy wired with `ai:typecheck` script + turbo task at a relaxed-first baseline (D-058: `check_untyped_defs`, `warn_unused_ignores`, `no_implicit_optional`, `warn_redundant_casts`, `warn_unreachable` — not full strict; per-module overrides available later), all 27 `# type: ignore` comments made meaningful or removed; recharts 2→3 upgrade (D-049) with chart-surface verification = `tsc --noEmit` + static-export `next build` + existing web tests green, live mount probe as the operational smoke (D-059 — no browser/render harness, out of scope); deprecated transitive deps resolved where toolchain allows | high | 5 | complete |
|
||||
|
||||
## v0.6 Requirements (Complete — Marketplace Engine; shipped as v0.5.6)
|
||||
|
||||
### Job Aggregation & Postings
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-6-001 | Job posting backend (from REQ-F-011, pilot-scaled): postings store (6th D-027 store family — SQLite, insert-safe, protocol-wrapped), employer posting lifecycle (draft → published → closed), ingestion adapter contract (employer-authored postings + seeded corpus at pilot scale; 3M+ external aggregation preserved by the adapter interface), marketplace API routes replacing the mock-data reads on job board/detail/employer surfaces, 18+ verified identity gate on apply/post flows (D-043 composition retained) | critical | 1 | complete |
|
||||
| REQ-6-002 | AI-powered tagging (from REQ-F-012): skills/seniority/category extraction from posting text via the LLM structured-output layer (D-020) — deterministic mock-first, provider-agnostic; tags persisted with the posting; tag facets drive the search filter sidebar | high | 2 | complete |
|
||||
|
||||
### Search & Matching
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-6-003 | Semantic vector search (from REQ-F-013): embedding provider protocol (mock-first, deterministic; real provider env-configured per D-014), vector store at pilot scale (SQLite-local — no pgvector before the Postgres migration), candidate-competency ↔ posting-requirement similarity ranking, match-score surfacing on job cards + candidate matches (replaces mock match scores) | critical | 3 | complete |
|
||||
|
||||
### Enrichment & Parsing
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-6-004 | Company enrichment (from REQ-F-014): employer profile enrichment from posting/company data (descriptions, social links, logo refs) via LLM structured output; enrich-on-post with manual founder override path; enriched data stored separately from raw (provenance marked) | medium | 4 | complete |
|
||||
| REQ-6-005 | Resume/profile parsing (from REQ-F-015): learner profile auto-fill from an uploaded resume — LLM structured extraction mapping to the competency-profile shape, learner-review-before-save (never silently overwrite), upload bounded (size/format guard, house pattern) | medium | 4 | complete |
|
||||
|
||||
### SEO & Programmatic Pages
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-6-006 | SEO-optimized programmatic pages (from REQ-F-016): server-rendered job/stack/landing pages with metadata (title/description/OG), sitemap.xml + robots.txt, JSON-LD structured data (JobPosting schema), static-export compatible (D-039 single-port deploy unchanged) | high | 5 | complete |
|
||||
|
||||
## v0.5 Requirements (Complete — Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease; shipped as v0.4.5)
|
||||
|
||||
### Real Server Voice
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-001 | `openai-audio` VoiceProvider: server STT (`/audio/transcriptions`) + TTS (`/audio/speech`) against an OpenAI-compatible endpoint via the existing D-030 protocol; provider selection by `AI_VOICE_PROVIDER` (+ base URL/key from env, never committed); deterministic mock stays first-class; browser fallback unchanged | critical | 2 | complete |
|
||||
| REQ-5-002 | Voice defense real path end-to-end: examiner dialogue answers transcribed server-side (audio upload → transcript), examiner questions spoken via server TTS (audio returned to the client); transcripts + integrity signals unchanged | critical | 2 | complete |
|
||||
|
||||
### Identity & Age-Gating
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-003 | Identity provider protocol + mock-first backend (REQ-F-017): verify-identity flow (submit → pending → verified/rejected with document refs), provider-agnostic (mock default; a real KYC vendor drops in later), PII stored server-side only, never logged | critical | 3 | complete |
|
||||
| REQ-5-004 | Age-gating enforced by the backend: school floor 16+ verified at enrollment, marketplace 18+ with verified identity — API surfaces reject under-age/unverified callers on gated routes (replaces the v0.1 visual-only flow; G-5 allowlist evolves toward real identity, allowlist remains as pilot guard) | critical | 3 | complete |
|
||||
|
||||
### Sandbox Environments
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-005 | Design + simulation sandbox environment types (REQ-F-021 remainder): extend the namespace fabric with environment kinds beyond the coding IDE (design-tool: canvas/editor surfaces with file artifacts; simulation: run/benchmark harnesses) — one lifecycle, one telemetry path, per-type starter contents + allowed commands | high | 4 | complete |
|
||||
| REQ-5-006 | Environment-typed learner surface: the build/defend flow accepts environment kind, telemetry captures per-kind events, grading digest stays kind-agnostic | high | 4 | complete |
|
||||
|
||||
### Telemetry Durability
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-5-007 | Exec-telemetry seq-lease/replay-margin fix: WS ingest acknowledges received seqs; capture agent resumes from the ack on reconnect (bounded replay margin) — closes the P6-lesson one-line ACK gap with a real-server regression test | high | 1 | complete |
|
||||
|
||||
## v0.4 Requirements (Complete — Distribution & Bootstrap CLI, shipped as v0.3.4)
|
||||
|
||||
### Bootstrap CLI
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-4-001 | `nextcraft` CLI (linux x64 binary): `doctor` command checking prerequisites (node, pnpm, python3, git, unshare) with actionable error messages | critical | 1 | complete |
|
||||
| REQ-4-002 | `bootstrap` command: pnpm install, ai-service venv + pinned deps, .env from templates, key validation, .env.secrets handling; `verify` health check (ports, imports, builds); `dev` thin passthrough to scripts/dev.sh | critical | 1 | complete |
|
||||
|
||||
### Distribution
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-4-003 | One-liner install script (`curl -fsSL <url> \| bash`): detects linux x64, resolves latest release from Gitea API, downloads binary + checksum, verifies sha256, installs to ~/.local/bin (PATH hint), degrades to source-bootstrap instructions when no binary | critical | 2 | complete |
|
||||
| REQ-4-004 | Binary release pipeline: reproducible linux x64 build script, sha256 checksum sidecar, upload as release assets on every ship from v0.4 onward (ongoing binaries requirement) | critical | 2 | complete |
|
||||
| REQ-4-005 | Install + quickstart documentation: README one-liner quickstart, CLI command reference, fresh-clone-to-running-stack end-to-end verification | high | 3 | complete |
|
||||
|
||||
## v0.3 Requirements (Credential Engines)
|
||||
|
||||
### Sandbox & Telemetry
|
||||
@@ -8,22 +220,22 @@
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-3-001 | Sandbox fabric: isolated per-learner execution environments (sandboxed IDE, design tool, simulation) with lifecycle management | critical | 1 | complete |
|
||||
| REQ-3-002 | Sandbox isolation + resource limits: per-learner isolation boundary, CPU/memory quotas (rlimits), wall-clock time quota, disk-quota via per-sandbox workdir usage sweep (best-effort, not kernel-enforced), no cross-tenant access, snapshot support. **Known gap (v0.3): per-sandbox pids and hard disk caps are NOT kernel-enforceable without cgroup delegation/sudo — documented as accepted risk** | critical | 1 | complete |
|
||||
| REQ-3-003 | Live build telemetry: in-environment capture of process events (commands, file diffs, run/test results, activity) streamed reliably to ai-service with per-learner trace persistence | critical | 2 | pending |
|
||||
| REQ-3-003 | Live build telemetry: in-environment capture of process events (commands, file diffs, run/test results, activity) streamed reliably to ai-service with per-learner trace persistence | critical | 2 | complete |
|
||||
|
||||
### Credential Engines
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-3-004 | Process-trace grading engine: grade artifacts from their full process traces; rubric-aligned structured scores; feeds Assessor real inputs | critical | 3 | pending |
|
||||
| REQ-3-005 | Variant task generation: per-learner task variants (no two learners get identical prompts); variant seed registry; difficulty normalization | high | 4 | pending |
|
||||
| REQ-3-006 | Oral/voice defense: AI examiner conducts spoken defense (STT → dialogue → TTS); transcript + integrity signals captured; feeds Proctor/Mentor | high | 5 | pending |
|
||||
| REQ-3-004 | Process-trace grading engine: grade artifacts from their full process traces; rubric-aligned structured scores; feeds Assessor real inputs | critical | 3 | complete |
|
||||
| REQ-3-005 | Variant task generation: per-learner task variants (no two learners get identical prompts); variant seed registry; difficulty normalization | high | 4 | complete |
|
||||
| REQ-3-006 | Oral/voice defense: AI examiner conducts spoken defense (STT → dialogue → TTS); transcript + integrity signals captured; feeds Proctor/Mentor | high | 5 | complete |
|
||||
|
||||
### Agent Re-grounding & Integration
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-3-007 | Agent re-grounding: Lab consumes live telemetry; Assessor consumes grading-engine output; Proctor consumes telemetry + defense integrity signals (replace v0.2 mocks) | critical | 6 | pending |
|
||||
| REQ-3-008 | Learner surface integration: sandbox mockup → real in-browser build/run with live telemetry; assessment mockup → live defense + live grading | critical | 6 | pending |
|
||||
| REQ-3-007 | Agent re-grounding: Lab consumes live telemetry; Assessor consumes grading-engine output; Proctor consumes telemetry + defense integrity signals (replace v0.2 mocks) | critical | 6 | complete |
|
||||
| REQ-3-008 | Learner surface integration: sandbox mockup → real in-browser build/run with live telemetry; assessment mockup → live defense + live grading | critical | 6 | complete |
|
||||
|
||||
---
|
||||
|
||||
@@ -65,58 +277,58 @@
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-001 | Monorepo scaffolding: pnpm workspaces, turborepo, Next.js app, TypeScript config, ESLint, Prettier | critical | 1 | complete |
|
||||
| REQ-002 | Shared component library: design tokens, Button, Input, Card, Badge, Avatar, Dialog, Navigation, Table, Tabs, Progress, Tooltip, Skeleton, Toast | critical | 1 | pending |
|
||||
| REQ-003 | Mock data layer: typed mock data for competency stacks, job listings, candidate profiles, employers, learner progress | critical | 1 | pending |
|
||||
| REQ-004 | Routing and navigation: App Router route groups for (learner), (marketplace), (employer), (admin); shared layout components; cross-surface navigation | critical | 1 | pending |
|
||||
| REQ-005 | Responsive layout system: mobile, tablet, desktop breakpoints; container components; grid system | high | 1 | pending |
|
||||
| REQ-002 | Shared component library: design tokens, Button, Input, Card, Badge, Avatar, Dialog, Navigation, Table, Tabs, Progress, Tooltip, Skeleton, Toast | critical | 1 | complete |
|
||||
| REQ-003 | Mock data layer: typed mock data for competency stacks, job listings, candidate profiles, employers, learner progress | critical | 1 | complete |
|
||||
| REQ-004 | Routing and navigation: App Router route groups for (learner), (marketplace), (employer), (admin); shared layout components; cross-surface navigation | critical | 1 | complete |
|
||||
| REQ-005 | Responsive layout system: mobile, tablet, desktop breakpoints; container components; grid system | high | 1 | complete |
|
||||
|
||||
### Learner Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-006 | Landing page: hero, value proposition, program highlights, how-it-works (Byte→Build→Demonstrate→Defend), testimonials mockup, CTA to program catalog | critical | 2 | pending |
|
||||
| REQ-007 | Program catalog: grid of competency stacks (AI Orchestration Engineer, AI Safety & Governance Lead, Human-AI Product Designer, AI-Augmented Field Operator, Computational Sciences Practitioner); stack cards with role descriptions | critical | 2 | pending |
|
||||
| REQ-008 | Competency stack view: selected stack detail with 12-18 competencies listed, progress indicators, microcredential badges, mastery status | critical | 2 | pending |
|
||||
| REQ-009 | Learner dashboard: active competencies, progress graph, recent artifacts, upcoming defenses, AI tutor chat mockup, milestone tracker | critical | 2 | pending |
|
||||
| REQ-010 | Byte tutorial viewer: 3-7 minute micro-tutorial layout with concept panel, worked example panel, code/design/simulation viewer mockup | high | 2 | pending |
|
||||
| REQ-011 | Build sandbox mockup: sandboxed IDE/design tool/simulation UI mockup with toolbar, file explorer, editor area, telemetry sidebar (process capture indicators) | high | 2 | pending |
|
||||
| REQ-012 | Assessment/defense mockup: rubric display, AI reviewer panel, oral defense interface (voice/mic mockup), process trace timeline, artifact viewer | high | 2 | pending |
|
||||
| REQ-006 | Landing page: hero, value proposition, program highlights, how-it-works (Byte→Build→Demonstrate→Defend), testimonials mockup, CTA to program catalog | critical | 2 | complete |
|
||||
| REQ-007 | Program catalog: grid of competency stacks (AI Orchestration Engineer, AI Safety & Governance Lead, Human-AI Product Designer, AI-Augmented Field Operator, Computational Sciences Practitioner); stack cards with role descriptions | critical | 2 | complete |
|
||||
| REQ-008 | Competency stack view: selected stack detail with 12-18 competencies listed, progress indicators, microcredential badges, mastery status | critical | 2 | complete |
|
||||
| REQ-009 | Learner dashboard: active competencies, progress graph, recent artifacts, upcoming defenses, AI tutor chat mockup, milestone tracker | critical | 2 | complete |
|
||||
| REQ-010 | Byte tutorial viewer: 3-7 minute micro-tutorial layout with concept panel, worked example panel, code/design/simulation viewer mockup | high | 2 | complete |
|
||||
| REQ-011 | Build sandbox mockup: sandboxed IDE/design tool/simulation UI mockup with toolbar, file explorer, editor area, telemetry sidebar (process capture indicators) | high | 2 | complete |
|
||||
| REQ-012 | Assessment/defense mockup: rubric display, AI reviewer panel, oral defense interface (voice/mic mockup), process trace timeline, artifact viewer | high | 2 | complete |
|
||||
|
||||
### Marketplace Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-013 | Job board listing: searchable grid of AI-era job listings, filter sidebar (skills, seniority, location, salary), result cards with match score | critical | 3 | pending |
|
||||
| REQ-014 | Job detail page: full job description, required competencies, employer info, application CTA, related jobs, AI-matched skills breakdown | critical | 3 | pending |
|
||||
| REQ-015 | Employer profile: company overview, logo, description, social links, open positions, company culture mockup | high | 3 | pending |
|
||||
| REQ-016 | Search/filter UI: semantic search bar, skill tags, category filters, seniority filter, remote/on-site toggle, saved searches mockup | critical | 3 | pending |
|
||||
| REQ-017 | Pricing page: job posting packages (single, bundle, enterprise), talent access plans, subscription tiers, feature comparison table | high | 3 | pending |
|
||||
| REQ-013 | Job board listing: searchable grid of AI-era job listings, filter sidebar (skills, seniority, location, salary), result cards with match score | critical | 3 | complete |
|
||||
| REQ-014 | Job detail page: full job description, required competencies, employer info, application CTA, related jobs, AI-matched skills breakdown | critical | 3 | complete |
|
||||
| REQ-015 | Employer profile: company overview, logo, description, social links, open positions, company culture mockup | high | 3 | complete |
|
||||
| REQ-016 | Search/filter UI: semantic search bar, skill tags, category filters, seniority filter, remote/on-site toggle, saved searches mockup | critical | 3 | complete |
|
||||
| REQ-017 | Pricing page: job posting packages (single, bundle, enterprise), talent access plans, subscription tiers, feature comparison table | high | 3 | complete |
|
||||
|
||||
### Employer Dashboard
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-018 | Employer dashboard overview: active postings, applicant pipeline, talent matches, analytics mockup (charts, placement stats) | critical | 4 | pending |
|
||||
| REQ-019 | Talent search: searchable candidate database with AI-matched filters, candidate cards showing competency stack, microcredentials, artifact count, defense score | critical | 4 | pending |
|
||||
| REQ-020 | Candidate profile view: full candidate profile with artifact gallery, process trace summary, oral defense transcripts, competency graph, microcredential verification | high | 4 | pending |
|
||||
| REQ-021 | Posting management: create/edit/delete job postings, posting status tracking, applicant list per posting, interview pipeline mockup | high | 4 | pending |
|
||||
| REQ-018 | Employer dashboard overview: active postings, applicant pipeline, talent matches, analytics mockup (charts, placement stats) | critical | 4 | complete |
|
||||
| REQ-019 | Talent search: searchable candidate database with AI-matched filters, candidate cards showing competency stack, microcredentials, artifact count, defense score | critical | 4 | complete |
|
||||
| REQ-020 | Candidate profile view: full candidate profile with artifact gallery, process trace summary, oral defense transcripts, competency graph, microcredential verification | high | 4 | complete |
|
||||
| REQ-021 | Posting management: create/edit/delete job postings, posting status tracking, applicant list per posting, interview pipeline mockup | high | 4 | complete |
|
||||
|
||||
### Admin Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-022 | Admin overview: platform metrics (learners, employers, placements, completion rate, NPS), recent activity feed, system health mockup | critical | 5 | pending |
|
||||
| REQ-023 | Learner management: searchable learner table, learner detail view, progress tracking, competency completion status, credential issuance log | high | 5 | pending |
|
||||
| REQ-024 | Competency graph viewer: interactive visualization of competency stacks and their relationships, node/edge graph using react-flow, stack details on node click | high | 5 | pending |
|
||||
| REQ-025 | Marketplace moderation: job posting review queue, employer verification queue, flagged content, content moderation tools mockup | high | 5 | pending |
|
||||
| REQ-022 | Admin overview: platform metrics (learners, employers, placements, completion rate, NPS), recent activity feed, system health mockup | critical | 5 | complete |
|
||||
| REQ-023 | Learner management: searchable learner table, learner detail view, progress tracking, competency completion status, credential issuance log | high | 5 | complete |
|
||||
| REQ-024 | Competency graph viewer: interactive visualization of competency stacks and their relationships, node/edge graph using react-flow, stack details on node click | high | 5 | complete |
|
||||
| REQ-025 | Marketplace moderation: job posting review queue, employer verification queue, flagged content, content moderation tools mockup | high | 5 | complete |
|
||||
|
||||
### Polish & Integration
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-026 | Cross-surface navigation: role switcher (learner/employer/admin), breadcrumbs, consistent header/footer across all surfaces | critical | 6 | pending |
|
||||
| REQ-027 | Visual consistency audit: typography scale, color palette, spacing system, dark mode toggle, accessibility baseline (WCAG AA contrast) | high | 6 | pending |
|
||||
| REQ-028 | Component library finalization: Storybook setup, component documentation, prop tables, usage examples | medium | 6 | pending |
|
||||
| REQ-026 | Cross-surface navigation: role switcher (learner/employer/admin), breadcrumbs, consistent header/footer across all surfaces | critical | 6 | complete |
|
||||
| REQ-027 | Visual consistency audit: typography scale, color palette, spacing system, dark mode toggle, accessibility baseline (WCAG AA contrast) | high | 6 | complete |
|
||||
| REQ-028 | Component library finalization: Storybook setup, component documentation, prop tables, usage examples | medium | 6 | complete |
|
||||
|
||||
---
|
||||
|
||||
@@ -136,19 +348,19 @@
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-011 | AI job aggregation pipeline (3M+ jobs from 120K companies) | high | v0.4+ | deferred |
|
||||
| REQ-F-012 | AI-powered tagging (skills, categories, seniority) | high | v0.4+ | deferred |
|
||||
| REQ-F-013 | Semantic vector search for role matching | high | v0.4+ | deferred |
|
||||
| REQ-F-014 | AI company enrichment (logos, descriptions, social links) | medium | v0.4+ | deferred |
|
||||
| REQ-F-015 | AI resume parsing (profile auto-fill) | medium | v0.4+ | deferred |
|
||||
| REQ-F-016 | SEO-optimized programmatic pages | medium | v0.4+ | deferred |
|
||||
| REQ-F-011 | AI job aggregation pipeline (3M+ jobs from 120K companies) | high | v0.6 | activated → REQ-6-001 (pilot-scaled; adapter contract preserves full scale) |
|
||||
| REQ-F-012 | AI-powered tagging (skills, categories, seniority) | high | v0.6 | activated → REQ-6-002 |
|
||||
| REQ-F-013 | Semantic vector search for role matching | high | v0.6 | activated → REQ-6-003 |
|
||||
| REQ-F-014 | AI company enrichment (logos, descriptions, social links) | medium | v0.6 | activated → REQ-6-004 |
|
||||
| REQ-F-015 | AI resume parsing (profile auto-fill) | medium | v0.6 | activated → REQ-6-005 |
|
||||
| REQ-F-016 | SEO-optimized programmatic pages | medium | v0.6 | activated → REQ-6-006 |
|
||||
|
||||
### Platform Infrastructure
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.4+ | deferred (deferred from v0.3 per founder directive) |
|
||||
| REQ-F-018 | Payment processing and subscription management | high | v0.3+ | deferred |
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.5 | activated → complete (REQ-5-003/004) |
|
||||
| REQ-F-018 | Payment processing and subscription management | high | v0.9 | activated → REQ-9-001..005 |
|
||||
| REQ-F-019 | Human tutor marketplace (third-party courses) | medium | v0.4+ | deferred |
|
||||
| REQ-F-020 | CIRR-style placement tracking and audit | medium | v0.5+ | deferred |
|
||||
|
||||
@@ -170,18 +382,51 @@
|
||||
|
||||
## Traceability Matrix
|
||||
|
||||
### v0.3 (current milestone)
|
||||
### v0.6 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-6-001 | 1 | complete |
|
||||
| REQ-6-002 | 2 | complete |
|
||||
| REQ-6-003 | 3 | complete |
|
||||
| REQ-6-004 | 4 | complete |
|
||||
| REQ-6-005 | 4 | complete |
|
||||
| REQ-6-006 | 5 | complete |
|
||||
|
||||
### v0.5 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-5-007 | 1 | complete |
|
||||
| REQ-5-001 | 2 | complete |
|
||||
| REQ-5-002 | 2 | complete |
|
||||
| REQ-5-003 | 3 | complete |
|
||||
| REQ-5-004 | 3 | complete |
|
||||
| REQ-5-005 | 4 | complete |
|
||||
| REQ-5-006 | 4 | complete |
|
||||
|
||||
### v0.4 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-4-001 | 1 | complete |
|
||||
| REQ-4-002 | 1 | complete |
|
||||
| REQ-4-003 | 2 | complete |
|
||||
| REQ-4-004 | 2 | complete |
|
||||
| REQ-4-005 | 3 | complete |
|
||||
|
||||
### v0.3 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-3-001 | 1 | complete |
|
||||
| REQ-3-002 | 1 | complete |
|
||||
| REQ-3-003 | 2 | pending |
|
||||
| REQ-3-004 | 3 | pending |
|
||||
| REQ-3-005 | 4 | pending |
|
||||
| REQ-3-006 | 5 | pending |
|
||||
| REQ-3-007 | 6 | pending |
|
||||
| REQ-3-008 | 6 | pending |
|
||||
| REQ-3-003 | 2 | complete |
|
||||
| REQ-3-004 | 3 | complete |
|
||||
| REQ-3-005 | 4 | complete |
|
||||
| REQ-3-006 | 5 | complete |
|
||||
| REQ-3-007 | 6 | complete |
|
||||
| REQ-3-008 | 6 | complete |
|
||||
|
||||
### v0.2 (complete)
|
||||
|
||||
@@ -231,4 +476,4 @@
|
||||
| REQ-025 | 5 | complete |
|
||||
| REQ-026 | 6 | complete |
|
||||
| REQ-027 | 6 | complete |
|
||||
| REQ-028 | 6 | complete |
|
||||
| REQ-028 | 6 | complete |
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+157
-115
@@ -2,30 +2,122 @@
|
||||
|
||||
## Overview
|
||||
|
||||
**Milestone v0.3** — Credential Engines: Replace v0.2's mock engine inputs with real credential engines. Build the sandbox fabric (sandboxed IDE / design tool / simulation), the live in-environment build-telemetry pipeline, the process-trace grading engine, per-learner variant task generation, and the oral/voice defense with AI examiner. Lab/Assessor/Proctor agents move from mock inputs to real engine inputs.
|
||||
**Milestone v0.10 — B2B Employer Network (ACTIVE, started 2026-09-30).** Feature milestone: real employer auth/accounts + self-serve (D-085). Employers become principals — registered accounts with sessions, employer-entity claim/verify, the employer dashboard converting from the v0.1 mockup to real data, self-serve posting management with the v0.9 entitlement enforcement intact, and self-serve checkout over the v0.9 billing catalog (superseding D-076 operator mediation; admin operator routes retained as fallback + oversight). Tags on the v0.9.x line (P0 → v0.9.0 … P6 → v0.9.6 = milestone release); branch `milestone/v0.10-b2b-employer-network`.
|
||||
|
||||
**Deferred per founder directive:** REQ-F-017 identity verification + age-gating (real KYC backend) is deferred beyond v0.3. Age-gating remains the v0.1 visual flow mockup.
|
||||
**Milestone v0.9 — Payment Engine (COMPLETE, shipped as v0.8.6, 2026-09-30).** Payment processing and subscription management (REQ-F-018 activated per D-074): the billing module (provider protocol mock+stripe, 7th D-027 store, HMAC webhooks), catalog + pricing surface, operator-mediated checkout, entitlement enforcement, full subscription lifecycle. Founder scope decision D-074; clarify D-075..D-084; grill G-301+. Tags on the v0.8.x line (v0.8.0 phase 0 → v0.8.6 final = milestone release); branch `milestone/v0.9-payment-engine`.
|
||||
|
||||
**Milestone v0.6 — Marketplace Engine (COMPLETE, shipped as v0.5.6, 2026-09-26).** The real marketplace backend behind the existing surfaces: job posting backend + aggregation adapter (REQ-6-001), AI tagging (REQ-6-002), semantic vector search + match scoring (REQ-6-003), company enrichment + resume parsing (REQ-6-004/005), SEO programmatic pages (REQ-6-006). Scope decision D-047 (auto-decided at SPECIFY, full autonomy). Tags on the v0.5.x line (v0.5.0 phase 0 → v0.5.N final = milestone release); branch `milestone/v0.6-marketplace-engine`.
|
||||
|
||||
**Hotfix v0.4.7 (2026-09-21) — pre-public hardening.** The 4 P1 flags from the v0.5 final review closed: HMAC ingest tokens on the WS grading input (LAN poisoning closed), nonce+counter defense-id mint (same-ms collision closed), browser-mode client SR/TTS per the descriptor contract (gap closed), 8 browser-voice module pins. Deployed via `nextcraft deploy --service`; site 200, service enabled.
|
||||
|
||||
**Hotfix v0.4.6 (2026-09-21) — site-down + deploy experience.** The domain root (nextcraft.coreci.dev) served the API's JSON 404 for three releases: the deployed service ran mount-less (web export never built on the host; dev -d auto-wiring silently skipped; nothing detected it). Fixed the class: `nextcraft deploy` (one-command clone/update → install → venv → export build → restart → verify), `deploy --service` (generated systemd unit, reboot-persistent), toolchain discovery (nvm/PNPM_HOME/corepack, incl. split installs — the host's exact layout), loud mount-less boot warning, verify live mount probe. Deployed live on the host via the new command; site 200, service enabled.
|
||||
|
||||
**Hotfix v0.3.5/v0.3.6 (2026-09-13) — fresh-box + single-port unattended deploy.**
|
||||
|
||||
**Milestone v0.5 — COMPLETE (shipped as v0.4.5, 2026-09-13).** Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease: the four seams D-016 deferred out of v0.4. Server STT/TTS for voice defense (`openai-audio` VoiceProvider), identity verification + backend-enforced age-gating (REQ-F-017), design/simulation sandbox environments (REQ-F-021 remainder), and the exec-telemetry seq-lease/replay-margin fix.
|
||||
|
||||
**Milestone v0.4 — COMPLETE (shipped as v0.3.4, 2026-09-13; hotfixes v0.3.5 fresh-box, v0.3.6 single-port unattended deploy).** Distribution & Bootstrap CLI: `nextcraft` CLI (doctor/bootstrap/verify/dev) shipped as a linux x64 SEA binary, one-liner install script with checksum + version integrity gates, and binaries published on **every ongoing release** (v0.3.2 onward). v0.3.6 adds the single-port same-origin deploy (static export served by the ai-service on :8420) and unattended ops (`dev -d`/`stop`/`log`), with runtime state moved to `~/.nextcraft/`.
|
||||
|
||||
**Milestone v0.3** — Credential Engines: complete, shipped as v0.2.8 (2026-09-12). Real sandbox fabric, live build telemetry, process-trace grading, per-learner variants, oral defense, real learner surfaces.
|
||||
|
||||
**Deferred per founder directive (D-016):** REQ-F-017 identity verification + age-gating (real KYC backend), real server STT/TTS, design/simulation sandbox environments, and the exec-telemetry seq-lease are deferred to v0.5. Age-gating remains the v0.1 visual flow mockup.
|
||||
|
||||
**Prior milestone:** v0.2 (ai-tutor-architecture) — complete, shipped as v0.2.0, six tutor agents live over mock engine inputs (D-015).
|
||||
|
||||
**Milestone type:** Feature (new credential-engine services + real agent inputs)
|
||||
**Tag line:** v0.2.x (patches on the v0.2 line; milestone release as the final v0.2.x patch)
|
||||
**Branch:** milestone/v0.3-credential-engines
|
||||
**Milestone type:** Feature (voice provider + identity backend + new env types + ingest protocol fix)
|
||||
**Tag line:** v0.4.x (patches on the v0.4 line; milestone release as the final v0.4.x patch)
|
||||
**Branch:** milestone/v0.5-real-voice-identity-envs
|
||||
|
||||
---
|
||||
|
||||
## v0.9 (Complete — Shipped as v0.8.6, 2026-09-30)
|
||||
|
||||
Payment processing and subscription management (REQ-F-018 activated per D-074, auto-decided at SPECIFY under full autonomy): posting packages (single/bundle/enterprise), talent-access plans, subscription tiers with real checkout, provider webhooks (HMAC-verified, idempotent), posting-creation entitlement enforcement, and a full subscription lifecycle (renew/cancel/upgrade/past_due grace) — all behind a provider-agnostic `PaymentProvider` protocol, mock-first (D-014 pattern; tests never call Stripe). The pricing page (REQ-017) gains its real backend. Out of scope: employer auth/accounts (B2B network — post-payments per A-407), multi-currency, tax/VAT beyond passthrough, invoice PDFs, CIRR tracking (REQ-F-020). Tags on the v0.8.x line (P0 → v0.8.0 … P6 → v0.8.6 = milestone release); branch `milestone/v0.9-payment-engine`.
|
||||
|
||||
## Phase List
|
||||
|
||||
### v0.10 (active)
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | in-progress | — | — | Specification, clarify, research, plan complete; .ciagent/ files updated for v0.3 |
|
||||
| 1 | Sandbox fabric | complete | 0 | REQ-3-001, REQ-3-002 | Isolated per-learner sandbox environments provisioned (IDE / design / simulation); lifecycle API (create/destroy/snapshot); resource limits enforced; no cross-tenant access |
|
||||
| 2 | Live build telemetry | pending | 1 | REQ-3-003 | In-environment capture of process events (commands, file diffs, run/test results, keystroke-level activity) streamed to ai-service; reliable transport; per-learner trace persistence |
|
||||
| 3 | Process-trace grading engine | pending | 2 | REQ-3-004 | Grades artifacts from their full process traces (not just final output); emits structured rubric-aligned scores; feeds Assessor real inputs |
|
||||
| 4 | Variant task generation | pending | 1 | REQ-3-005 | Per-learner task variants generated so no two learners receive identical prompts; variant seed recorded for grading fairness |
|
||||
| 5 | Oral / voice defense | pending | 3 | REQ-3-006 | AI examiner conducts spoken defense of submitted work; STT → dialogue → TTS; transcript + integrity signals captured; feeds Proctor/Mentor |
|
||||
| 6 | Agent re-grounding + learner surface integration | pending | 2,3,4,5 | REQ-3-007, REQ-3-008 | Lab/Assessor/Proctor consume real engine inputs; v0.1 sandbox + assessment mockups wired to real engines (in-browser build/run, live telemetry, live defense) |
|
||||
| 7 | Final review + ship | pending | 6 | — | Code review clean; audit passes; milestone tagged (v0.2.x final patch); release created on Gitea |
|
||||
| 0 | Pre-execution | complete (v0.9.0) | — | — | Specify (D-085), clarify (D-086..D-097), research (R-55..R-66), plan, grill (G-311..G-320), MVP/UX check complete (shipped as v0.9.0, release 948) |
|
||||
| 1 | Employer auth foundation | complete (v0.9.1) | 0 | REQ-10-001 | `ai_service/employers/` module: account store (8th D-027 family, 4 tables) + scrypt hashed credentials (params pinned) + hashed server-side sessions (7-day lazy TTL) + register/login/logout/me routes + dual D-088 auth throttles + CORS Authorization preflight + PII scrub; fail-closed secrets; tests never call a cloud (shipped as v0.9.1, release 949) |
|
||||
| 2 | Employer entity claim | pending | 1 | REQ-10-002 | Claim route over existing EmployerRow entities; provenance-recorded claims; one account : one entity; admin claim review/approve/reject; claimed-entity authorization enforced |
|
||||
| 3 | Employer account surface | pending | 2 | REQ-10-003 | Employer dashboard real-data conversion (mock imports → runtime fetches, A-406 pattern); login/register UI; session-gated shell; own-postings view; dual-schema wire pins |
|
||||
| 4 | Self-serve posting management | pending | 3 | REQ-10-004 | Employer-principal posting routes with own-entity authorization; entitlement enforcement parameterized by session employer (402 CTA unchanged); admin routes retained |
|
||||
| 5 | Self-serve checkout + admin overview | pending | 4 | REQ-10-005 | Employer-principal checkout over the v0.9 catalog; own entitlement view; pricing-page CTAs → employer-login path; admin network overview (accounts, claim queue, entitlement state) |
|
||||
| 6 | Final review + ship | pending | 1-5 | — | Review clean; audit pass; milestone tagged v0.9.6 (final v0.9.x patch = milestone release); release with full asset matrix + registry image; milestone merged to main |
|
||||
|
||||
---
|
||||
|
||||
## Phase List (v0.9 — complete)
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete (v0.8.0) | — | — | Specify (D-074), clarify (D-075+), research (R-33+), plan, grill (G-301+), MVP/UX check complete; shipped as v0.8.0 |
|
||||
| 1 | Billing foundation | complete (v0.8.1) | 0 | REQ-9-001 | `ai_service/billing/` module: PaymentProvider protocol + deterministic mock + SQLite store (7th D-027 family) + HMAC webhook verification + env-gated provider selection; tests never call a real provider |
|
||||
| 2 | Catalog + pricing surface | complete (v0.8.2) | 1 | REQ-9-002 | Seeded catalog (posting packages + talent-access plans) via `/v1/billing/catalog`; pricing page runtime client fetch with honest loading/error states; tier copy data-driven |
|
||||
| 3 | Checkout + webhook flow | complete (v0.8.3) | 2 | REQ-9-003 | `POST /v1/billing/admin/checkout` (admin-gated, D-076) + `POST /v1/billing/webhook` (HMAC-verified, idempotent event processing, at-least-once tolerance, local-row state resolution per G-302); entitlement granted on checkout.completed |
|
||||
| 4 | Entitlement enforcement | complete (v0.8.4) | 3 | REQ-9-004 | Posting creation enforces package limits (honest 402 + upgrade CTA); expired/cancelled degrade to 402; admin surface shows per-employer entitlement state |
|
||||
| 5 | Subscription lifecycle | complete (v0.8.5) | 4 | REQ-9-005 | Subscription state machine (trialing/active/past_due/cancelled/expired); idempotent renewals; manual admin grant/revoke with provenance; full billing event audit log; past_due grace window |
|
||||
| 6 | Final review + ship | complete (v0.8.6) | 1-5 | — | Review clean; audit pass; milestone tagged v0.8.6 (final v0.8.x patch = milestone release); release with full asset matrix + registry image; milestone merged to main |
|
||||
|
||||
---
|
||||
|
||||
## v0.8 (Complete — Shipped as v0.7.6)
|
||||
|
||||
Ship Anywhere: installable on any Debian-based Linux server via docker (Gitea registry image, honest sandbox degrade per D-068), .deb packages (amd64+arm64, advisory postinst per D-067), and arch-aware binary (x64+arm64, R-24 arch-resolve) — with release consistency enforced by pre-commit hooks + Gitea Actions (3-runner fleet on CT 113 as committed fleet-as-code, machine-made releases with the 8-asset matrix + registry image on every release), the CLI carrying the full suite (guide + channel-aware doctor, D-066/D-072), and the evidence-backed admin runbook (G-210 parity-pinned). 7 phases (P0–P6), all 5 requirements (REQ-8-001..005) complete. Tags v0.7.0–v0.7.5 per phase, milestone release v0.7.6 (release 936). Final review CLEAN (G-201..G-210 honored — the smoke gate caught 4 real image bugs pre-ship, each fixed in-phase); audit PASS (reconstruction + discipline green).
|
||||
|
||||
## Phase List
|
||||
|
||||
### v0.8 (complete)
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete (v0.7.0) | — | — | Specification, clarify (D-063..D-072), research (R-1..R-32), plan (rails→artifacts→onboarding→guides), grill (G-201..G-210), MVP/UX check complete (shipped as v0.7.0, release 913) |
|
||||
| 1 | Release-consistency rails | complete (v0.7.1) | 0 | REQ-8-001 | 3 org-level runners green on CT 113 (hybrid labels, sibling mode); ci.yaml push lanes green; hello-world × 3 labels; release.yaml builds assets on tag (v0.7.1 release machine-made); pre-commit both-direction proof; secrets audit clean |
|
||||
| 2 | Docker image channel | complete (v0.7.2) | 1 | REQ-8-002 | Multi-stage image (unprivileged, /health, state volume) pushed to the Gitea registry on every phase release; honest sandbox degrade (probe + notice + 503 remediation); in-container CLI semantics; smoke gate in workflow |
|
||||
| 3 | Debian-native channel | complete (v0.7.3) | 2 | REQ-8-003 | arm64 QEMU-free cross-build; .deb x64+arm64 (advisory postinst); install.sh arch-resolve; full asset matrix (2 binaries + 2 checksums + 2 debs) on every release |
|
||||
| 4 | CLI onboarding | complete (v0.7.4) | 3 | REQ-8-004 | Channel-aware doctor; nextcraft guide offline; honest in-container refusals; docker/deb E2Es env-gated green; guide↔runbook parity test |
|
||||
| 5 | Admin guides | complete (v0.7.5) | 4 | REQ-8-005 | deploy/README.md full runbook (3 channels × install/upgrade, upgrade matrix, troubleshooting, evidence appendix); README quickstart; every recipe evidence-backed |
|
||||
| 6 | Final review + ship | complete (v0.7.6) | 1-5 | — | Review clean; audit pass; milestone tagged v0.7.6 (final v0.7.x patch = milestone release, release 936); release with full asset matrix + registry image; milestone merged to main | Review clean; audit pass; milestone tagged v0.7.6 (final v0.7.x patch = milestone release); release with full asset matrix + image tags; milestone merged to main |
|
||||
|
||||
### v0.7 (complete)
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete | — | — | Specify, clarify (D-052..D-061), research (44-50), plan (D-062 waves), grill (G-101..G-110), MVP/UX check complete (shipped as v0.6.0) |
|
||||
| 1 | Harness wiring | complete (v0.6.1) | 0 | REQ-7-001 | Coverage in all 3 workspaces; timeouts 60s/180s probe-proven; markers fast/real_server/userns strict; web tests in turbo; leaks fixed; fast subset <60s |
|
||||
| 2 | Suite composition | complete (v0.6.2) | 1 | REQ-7-002 | 10 trivial tests cut with recorded per-assertion triage; 8 cross-boundary pins kept (CWD-independent); tests/helpers/ consolidation zero-behavior-change; droppings swept |
|
||||
| 3 | Warning elimination | complete (v0.6.3) | 2 | REQ-7-003 | session.exec ×14; ResourceWarning sites closed at source; filterwarnings=error + 2 scoped ignores; full suite ZERO warnings |
|
||||
| 4 | Lint & suppression hygiene | complete (v0.6.4) | 3 | REQ-7-004 | ESLint 6/6 workspaces; 20 violations fixed; 80 noqa removed (6 live fixed at source); dead exports gone; version single-source; engines |
|
||||
| 5 | Type & dependency hardening | complete (v0.6.5) | 4 | REQ-7-005 | mypy relaxed-first green (178 files, 27 ignores dispositioned); recharts 3.10.1 proven; transitive audit recorded |
|
||||
| 6 | Final review + ship | complete (v0.6.6) | 1-5 | — | Purity gate zero feat:; review SHIP verdict; audit PASS; milestone tagged v0.6.6; release with binary assets; milestone merged to main |
|
||||
|
||||
### v0.6 (complete)
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan, grill, MVP/UX check complete; .ciagent/ files updated for v0.6 (shipped as v0.5.0) |
|
||||
| 1 | Postings backend + aggregation adapter | complete (v0.5.1) | 0 | REQ-6-001 | Postings store + lifecycle + `/v1/marketplace/*` routes live; job board/detail/employer surfaces read real data; 18+ gates composed per D-043; seeded ingestion adapter proven |
|
||||
| 2 | AI tagging | complete (v0.5.2) | 1 | REQ-6-002 | Posting text → skills/seniority/category tags via LLM structured output (mock-first); tag facets drive search filters; tags persisted |
|
||||
| 3 | Semantic vector search | complete (v0.5.3) | 2 | REQ-6-003 | Embedding provider protocol (mock-first) + SQLite-local vectors; competency↔posting similarity ranking; match scores surface on job cards + candidate matches |
|
||||
| 4 | Enrichment + resume parsing | complete (v0.5.4) | 3 | REQ-6-004, REQ-6-005 | Employer enrichment from posting data with provenance; resume upload → profile auto-fill with learner review gate; bounded uploads |
|
||||
| 5 | SEO programmatic pages | complete (v0.5.5) | 4 | REQ-6-006 | Metadata + JSON-LD + sitemap/robots on job/stack/landing pages; static-export compatible (single-port deploy unchanged) |
|
||||
| 6 | Final review + ship | complete (v0.5.6) | 1-5 | — | Review clean; audit passes; milestone tagged (final v0.5.x patch); release with binary assets on Gitea; milestone merged to main |
|
||||
|
||||
---
|
||||
|
||||
## Phase List (v0.5 — complete)
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan, grill complete; .ciagent/ files updated for v0.5 |
|
||||
| 1 | Seq-lease + replay margin | complete | 0 | REQ-5-007 | Ingest acks received seqs (`seq_ack` frame); capture agent trims spool to ack on reconnect (bounded margin); real-server mid-burst regression test closes the P07-documented gap |
|
||||
| 2 | Real server voice | complete | 0 | REQ-5-001, REQ-5-002 | `openai-audio` provider passes STT/TTS contract tests (MockTransport); voice defense runs server-side end-to-end when keys exist; mock/browser paths unchanged; suite green |
|
||||
| 3 | Identity + age-gating | complete | 0 | REQ-5-003, REQ-5-004 | Identity protocol + mock backend + verification flow API; gated routes enforce 16+/18+ (allowlist → identity → rate caps); PII hygiene pinned by caplog test |
|
||||
| 4 | Design/sim environments | complete | 0 | REQ-5-005, REQ-5-006 | Template-layer env registry (build/design/simulation); per-kind starter contents + exec policy; test_command surfaced; grading digest kind-agnostic (pinned) |
|
||||
| 5 | Final review + ship | complete | 1-4 | — | Code review clean; audit passes; milestone tagged (final v0.4.x patch); release with binary assets on Gitea |
|
||||
|
||||
---
|
||||
|
||||
@@ -33,147 +125,97 @@
|
||||
|
||||
### Phase 0: Pre-execution
|
||||
|
||||
**Goal:** Establish v0.3 specification, clarify ambiguities, research credential-engine architecture (sandbox isolation, telemetry transport, trace grading, variant generation, voice IO), create detailed plans.
|
||||
**Goal:** Lock the v0.5 specification (four deferred seams), clarify ambiguities, research the audio endpoint contract + KYC provider landscape + env-type design + the seq-lease protocol, plan waves, grill adversarially.
|
||||
|
||||
**Stages:** SPECIFY → CLARIFY → RESEARCH → PLAN → GRILL → MVP/UX CHECK → SHIP
|
||||
|
||||
**Deliverables:**
|
||||
- Updated .ciagent/config.json, PROJECT.md, REQUIREMENTS.md, ROADMAP.md, ARCHITECTURE.md, PERSONAS.md, PLAN.md
|
||||
|
||||
**Success criteria:** All .ciagent/ files updated for v0.3; phase 0 shipped as first v0.2.x patch.
|
||||
**Success criteria:** All .ciagent/ files updated for v0.5; phase 0 shipped as v0.4.0.
|
||||
|
||||
---
|
||||
### Phase 1: Seq-Lease + Replay Margin (REQ-5-007)
|
||||
|
||||
### Phase 1: Sandbox Fabric
|
||||
|
||||
**Goal:** Provision and manage isolated per-learner execution environments.
|
||||
|
||||
**Requirements:** REQ-3-001, REQ-3-002
|
||||
**Goal:** Close the reconnect-replay ACK gap from the v0.3 P6 lesson.
|
||||
|
||||
**Key deliverables:**
|
||||
- Sandbox orchestrator service: create/list/destroy/snapshot sandbox instances (IDE, design tool, simulation)
|
||||
- Isolation boundary: per-learner containerization or VM-grade isolation; no cross-tenant filesystem/network access
|
||||
- Resource limits: CPU/memory/disk/time quotas per sandbox
|
||||
- Sandbox lifecycle API consumed by ai-service and the web learner surface
|
||||
- Ingest WS: ack frames carrying highest-contiguous received seq per (learner, task); capture agent tracks ack and resumes spool replay from acked position (bounded replay margin)
|
||||
- Real-server regression test: mid-burst kill → reconnect → no loss/dup exactly-once within margin (deterministic — waits for server-side observation like the P07 fix)
|
||||
|
||||
**Success criteria:**
|
||||
- A sandbox can be created, written to, snapshotted, and destroyed via API
|
||||
- Isolation verified: a sandbox cannot read another learner's data
|
||||
- Resource limits enforced and observable — enforcement mechanism: rlimits (memory/CPU) + wall-clock reaper + workdir-size sweep; per-sandbox pids and hard-disk-quota are accepted v0.3 gaps (no cgroup delegation/sudo on this box, G-1)
|
||||
**Success criteria:** regression test green on repeated runs; margin documented; flood/gap semantics (G-3/G-4) unchanged.
|
||||
|
||||
---
|
||||
### Phase 2: Real Server Voice (REQ-5-001, REQ-5-002)
|
||||
|
||||
### Phase 2: Live Build Telemetry
|
||||
|
||||
**Goal:** Capture in-environment process events and stream them to ai-service reliably.
|
||||
|
||||
**Requirements:** REQ-3-003
|
||||
**Goal:** The `openai-audio` VoiceProvider against OpenAI-compatible STT/TTS endpoints; voice defense real path end-to-end.
|
||||
|
||||
**Key deliverables:**
|
||||
- Telemetry capture agent (in-sandbox): commands, file diffs, run/test results, keystroke-level/activity events
|
||||
- Telemetry transport: durable, ordered, resumable stream to ai-service ingestion endpoint
|
||||
- Trace persistence: per-learner, per-task process traces stored for grading and proctoring
|
||||
- Transport hardening: retries, backpressure, exactly-once-or-at-least-once semantics documented
|
||||
- `ai_service/voice/openai_audio.py` — provider per D-030 protocol (STT: multipart upload → transcript; TTS: text → audio bytes); httpx via the app pool; timeouts; error mapping
|
||||
- `AI_VOICE_BASE_URL`/`AI_VOICE_API_KEY`/`AI_VOICE_MODEL` settings resolution (env-only, never committed); `AI_VOICE_PROVIDER=openai-audio|browser|mock` selection in factory
|
||||
- Defense answer route: real STT path (audio upload transcribed server-side when provider=openai-audio; browser/mock paths unchanged); examiner TTS question audio
|
||||
- Tests: MockTransport byte-contract tests (multipart shape, response parse, error modes); defense-flow tests stay mock-only (never call the cloud)
|
||||
|
||||
**Success criteria:**
|
||||
- Sandbox activity produces a complete ordered process trace in ai-service
|
||||
- Stream survives transient network failure without trace loss
|
||||
- Trace retrievable by learner+task ID for grading
|
||||
**Success criteria:** provider contract pinned by tests; voice defense works against a scripted audio endpoint; `pnpm ai:test` green; manual cloud probe documented.
|
||||
|
||||
---
|
||||
### Phase 3: Identity + Age-Gating (REQ-5-003, REQ-5-004)
|
||||
|
||||
### Phase 3: Process-Trace Grading Engine
|
||||
|
||||
**Goal:** Grade learner artifacts from their full process traces.
|
||||
|
||||
**Requirements:** REQ-3-004
|
||||
**Goal:** Real identity verification backend behind a provider protocol; backend-enforced age gates.
|
||||
|
||||
**Key deliverables:**
|
||||
- Trace analyzer: reconstructs build/decision timeline from a process trace
|
||||
- Grading engine: rubric-aligned scoring over the trace (process quality, not just final artifact)
|
||||
- Structured score output consumable by the Assessor agent
|
||||
- Calibration against v0.2 mock corpora to validate grading dimensions
|
||||
- `ai_service/identity/` — IdentityProvider protocol (submit/poll/verify), mock provider (deterministic), store (SQLite, D-027 family), API router (`/v1/identity/*`)
|
||||
- Age-gate enforcement: school 16+ (enrollment), marketplace 18+ verified (gated routes reject under-age/unverified with 403) — dependency-injected gate, allowlist (G-5) retained as pilot guard
|
||||
- PII hygiene: document refs stored, never raw docs in logs; secrets-hygiene checklist extended
|
||||
|
||||
**Success criteria:**
|
||||
- Engine emits structured rubric-aligned scores from a real process trace
|
||||
- Scores distinguish process quality (e.g., iterative debugging vs. paste-and-run)
|
||||
- Output feeds Assessor; replaces pre-baked artifact corpus inputs
|
||||
**Success criteria:** verification flow API green under mock; gated routes enforce ages in tests; zero PII in captured logs (pinned by test).
|
||||
|
||||
---
|
||||
### Phase 4: Design/Sim Environments (REQ-5-005, REQ-5-006)
|
||||
|
||||
### Phase 4: Variant Task Generation
|
||||
|
||||
**Goal:** Generate per-learner task variants so no two learners receive identical prompts.
|
||||
|
||||
**Requirements:** REQ-3-005
|
||||
**Goal:** Environment kinds beyond the coding IDE on the existing namespace fabric.
|
||||
|
||||
**Key deliverables:**
|
||||
- Variant generator: parameterized task templates → unique per-learner instances
|
||||
- Variant seed registry: record variant parameters for grading fairness and proctoring
|
||||
- Difficulty normalization: variants calibrated to equivalent difficulty
|
||||
- Environment-type registry in the sandbox fabric: `build` (existing), `design` (canvas/file artifacts), `simulation` (run/benchmark harness) — per-type starter contents + allowed exec commands, one lifecycle, one telemetry path
|
||||
- Learner surface + engine-client accept environment kind; telemetry events carry kind; grading digest stays kind-agnostic (D-028 unchanged)
|
||||
|
||||
**Success criteria:**
|
||||
- Two learners requesting the same competency receive distinct task variants
|
||||
- Variant parameters persisted and auditable
|
||||
- Grading engine scores variants equitably
|
||||
**Success criteria:** all three kinds provisionable + provable isolation; per-kind starter files land; kind flows through telemetry to the digest.
|
||||
|
||||
### Phase 5: Final Review + Ship
|
||||
|
||||
**Goal:** Code review, audit, milestone release with binary assets.
|
||||
|
||||
**Success criteria:** review P0s fixed in-phase; audit reconstruction clean; final v0.4.x patch tagged; Gitea release with `nextcraft-linux-x64` + `.sha256`; milestone merged to main; branches deleted.
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Oral / Voice Defense
|
||||
## Hotfixes (v0.4.x line, post-v0.5)
|
||||
|
||||
**Goal:** AI examiner conducts a spoken defense of the learner's submitted work.
|
||||
| Tag | Date | Summary |
|
||||
|-----|------|---------|
|
||||
| v0.4.7 | 2026-09-21 | Pre-public hardening (the 4 P1 flags from the v0.5 final review): HMAC ingest tokens on the WS grading input (LAN trace-poisoning closed, per-boot secret, manager mints / endpoint verifies), nonce+counter defense-id mint (same-ms PK collision closed), browser-mode client SR/TTS implemented per the descriptor contract (the v0.5 gap), 8 browser-voice module pins. 488 backend (+10), web 22/22 (+8), CLI 61/61. Deployed via `nextcraft deploy --service`; site 200, service enabled. |
|
||||
| v0.4.6 | 2026-09-21 | Site-down hotfix + one-command deploy: `nextcraft deploy [--service]` (clone/update → install → venv → export build → restart → verify), systemd unit generation (reboot persistence), toolchain discovery (nvm/PNPM_HOME/corepack + split installs), loud mount-less boot, verify live mount probe, subcommand flag dispatch. Deployed live via the command itself; nextcraft.coreci.dev → 200 HTML. |
|
||||
|
||||
**Requirements:** REQ-3-006
|
||||
## v0.7 (Complete — Shipped as v0.6.6)
|
||||
|
||||
**Key deliverables:**
|
||||
- Voice pipeline: STT → defense dialogue (LLM examiner) → TTS
|
||||
- Examiner agent: probes understanding, challenges process choices from the trace
|
||||
- Transcript + integrity signals captured for Proctor/Mentor
|
||||
- Latency budget: defense feels conversational (bounded turn latency)
|
||||
Test Harness Review (NFR refinement-only, founder directive post-v0.6): coverage measured in all three workspaces (pytest-cov 95% branch, c8 web 95.9%/cli 76.9%); pytest-timeout safety net (60s/180s dual-tier, signal method, both probe-proven); markers fast/real_server/userns under --strict-markers with a 620-test fast subset completing in ~56s; web tests inside the turbo pipeline; CLI E2E env-gated. Suite composition: 10 trivial tests cut with per-assertion recorded triage (D-057), 8 cross-boundary/regression pins kept in CWD-independent pins.test.ts; duplicated harness infra consolidated into tests/helpers/ (KillableProxy split into two semantics-distinct helpers per G-101); deterministic waits replace sleeps where hooks exist; sandbox droppings swept + session-end fixtures. Warning elimination: 1,716 warnings/run → ZERO (14 session.query→session.exec sites, 14 ResourceWarning sites closed at source, filterwarnings=error with exactly 2 scoped upstream ignores). Lint hygiene: ESLint live in all 6 workspaces (was none), 20 violations fixed, 80 inert noqa removed (6 live ones fixed at source), RUF100 enforced, dead exports gone, version single-sourced 0.6.5, engines everywhere. Type & dependency hardening: mypy 2.3.1 relaxed-first green over 178 files (215 errors found: ~140 fixed at source incl. 8+ genuine bugs, 34 scoped suppressions; all 27 original ignores dispositioned); recharts 2.15.4→3.10.1 (deprecated direct dep resolved); transitive audit recorded (4 dev-only remain, none actionable). 7 phases (P0-P6). All 5 requirements (REQ-7-001..005) complete. Tags v0.6.0-v0.6.5 per phase, milestone release v0.6.6.
|
||||
|
||||
**Success criteria:**
|
||||
- A spoken defense runs end-to-end (speak → examiner question → learner response → verdict)
|
||||
- Transcript + integrity signals persisted and consumable by Proctor
|
||||
- Turn latency within the documented budget
|
||||
## v0.6 (Complete — Shipped as v0.5.6)
|
||||
|
||||
---
|
||||
Real backend behind the marketplace surfaces: postings store + aggregation adapter + marketplace API (REQ-6-001), AI tagging (REQ-6-002), embedding-backed semantic search + match scores (REQ-6-003), company enrichment + resume parsing (REQ-6-004/005), SEO programmatic pages (REQ-6-006). 7 phases (P0–P6), all 6 requirements complete. Tags v0.5.0–v0.5.5 per phase, milestone release v0.5.6. Final review CLEAN (all G-101..G-110 honored, MH-M1..M6 met); audit PASS.
|
||||
|
||||
### Phase 6: Agent Re-grounding + Learner Surface Integration
|
||||
## v0.5 (Complete — Shipped as v0.4.5)
|
||||
|
||||
**Goal:** Move Lab/Assessor/Proctor to real engine inputs; wire learner surface to the real engines.
|
||||
Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease: real server STT/TTS (`openai-audio` VoiceProvider with boot-safe fallback), the identity module (5th D-027 store, provider protocol + mock, D-043 age-gate composition on variants/sandboxes/defense/marketplace), design/simulation environment kinds at the template layer with per-kind exec policy, and the seq-ack protocol closing the P07 replay-margin gap. 6 phases (P0–P5). All 7 requirements (REQ-5-001..007) complete. Tags v0.4.0–v0.4.4 per phase, milestone release v0.4.5.
|
||||
|
||||
**Requirements:** REQ-3-007, REQ-3-008
|
||||
## v0.4 (Complete — Shipped as v0.3.4)
|
||||
|
||||
**Key deliverables:**
|
||||
- Lab agent consumes live sandbox telemetry (replaces v0.2 mock telemetry)
|
||||
- Assessor agent consumes grading-engine output (replaces pre-baked artifacts)
|
||||
- Proctor consumes telemetry + defense integrity signals (replaces mock telemetry)
|
||||
- Learner sandbox mockup → real in-browser build/run; assessment mockup → live defense + live grading
|
||||
Distribution & Bootstrap CLI: `nextcraft` CLI (doctor/bootstrap/verify/dev) as a self-contained linux x64 SEA binary, one-liner install with sha256 + version integrity gates, release-asset pipeline attaching binaries to every ongoing release (v0.3.2 onward), install/quickstart docs backed by a fresh-clone E2E test. 5 phases (P0–P4). All 5 requirements (REQ-4-001..005) complete. Tags v0.3.0–v0.3.3 per phase, milestone release v0.3.4.
|
||||
|
||||
**Success criteria:**
|
||||
- Lab/Assessor/Proctor operate on real inputs with no mock fallback in the learner path
|
||||
- Learner can build in-browser and see live telemetry + live feedback
|
||||
- Assessment surface runs a live defense and shows live grading
|
||||
- `pnpm build` and `pnpm typecheck` pass
|
||||
## v0.3 (Complete — Shipped as v0.2.8)
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Final Review + Ship
|
||||
|
||||
**Goal:** Code review, audit, milestone release.
|
||||
|
||||
**Key deliverables:**
|
||||
- Multi-persona code review (correctness, testing, security, performance, maintainability)
|
||||
- Project health audit (reconstruction test, .ciagent/ file discipline, branch hygiene, commit discipline)
|
||||
- Milestone ship: merge milestone → main, tag final v0.2.x patch, create Gitea release
|
||||
|
||||
**Success criteria:**
|
||||
- Code review: P0 fixes applied, P1+ documented
|
||||
- Audit: all checks pass, project state reconstructable from git log
|
||||
- Ship: milestone tagged, branch merged to main, Gitea release created — release note states identity/age-gating (KYC) is deferred and age-gating remains a visual mockup
|
||||
- All v0.3 requirements marked complete
|
||||
|
||||
---
|
||||
Credential Engines: real sandbox fabric (Linux namespaces), live build telemetry
|
||||
(at-least-once/exactly-once), process-trace grading (G-4 gated), seeded per-learner
|
||||
variants (fairness anchors wired to grading), oral defense with integrity signals
|
||||
(mock-first voice, browser fallback), and real learner build/defense/grading surfaces.
|
||||
8 phases. All 8 requirements (REQ-3-001..008) complete. Tags v0.2.1–v0.2.7 per phase,
|
||||
milestone release v0.2.8.
|
||||
|
||||
## v0.2 (Complete — Shipped as v0.2.0)
|
||||
|
||||
@@ -181,4 +223,4 @@ AI Tutor Architecture: Six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor
|
||||
|
||||
## v0.1 (Complete — Shipped as v0.1.0)
|
||||
|
||||
UI/UX Prototype: High-fidelity interactive prototype of all four Nextcraft surfaces. 7 phases (P0 + P1-P6 execution + P7 final). All 28 requirements complete. Tags v0.0.1–v0.0.7, milestone release v0.1.0.
|
||||
UI/UX Prototype: High-fidelity interactive prototype of all four Nextcraft surfaces. 7 phases (P0 + P1-P6 execution + P7 final). All 28 requirements complete. Tags v0.0.1–v0.0.7, milestone release v0.1.0.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"version": "0.7.0",
|
||||
"version": "0.8.0",
|
||||
"autonomy": {
|
||||
"level": "full",
|
||||
"decision_confidence_threshold": 0.6,
|
||||
@@ -46,9 +46,9 @@
|
||||
"projects": [],
|
||||
"active_project": null,
|
||||
"milestone": {
|
||||
"version": "v0.3",
|
||||
"name": "credential-engines",
|
||||
"version": "v0.10",
|
||||
"name": "b2b-employer-network",
|
||||
"type": "feature",
|
||||
"branch": "milestone/v0.3-credential-engines"
|
||||
"branch": "milestone/v0.10-b2b-employer-network"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Build context hygiene (v0.8 P2): secrets and runtime droppings never
|
||||
# enter an image layer. The .env* exclusion is the hard rule — the v1.8
|
||||
# class of leak (tokens) must be structurally impossible here.
|
||||
.git
|
||||
.gitignore
|
||||
.dockerignore
|
||||
.env
|
||||
.env.*
|
||||
**/.env
|
||||
**/.env.*
|
||||
!.env.example
|
||||
.ciagent
|
||||
**/.ciagent
|
||||
**/.venv
|
||||
**/node_modules
|
||||
**/__pycache__
|
||||
**/*.pyc
|
||||
**/.pytest_cache
|
||||
**/.mypy_cache
|
||||
**/.ruff_cache
|
||||
**/.turbo
|
||||
**/coverage
|
||||
**/dist
|
||||
apps/web/out
|
||||
apps/web/.next
|
||||
**/*.tsbuildinfo
|
||||
.turbo
|
||||
node_modules
|
||||
*.log
|
||||
@@ -0,0 +1,130 @@
|
||||
# Push lanes — every branch, every push (REQ-8-001, D-063).
|
||||
#
|
||||
# Two jobs, two runner labels, NEVER wholesale `turbo lint`/`turbo typecheck`
|
||||
# in the node lane (RESEARCH R-8): those turbo tasks pull in
|
||||
# @nextcraft/ai-service#lint/typecheck, which hard-fail without the Python
|
||||
# venv (apps/ai-service/scripts/lint.sh:8, typecheck.sh:13) — a venv that
|
||||
# does not exist in the node:24-bookworm job container. The node lane runs
|
||||
# lint/typecheck via `pnpm --filter` over the 5 TS workspaces only; the
|
||||
# python lane invokes the three ai-service bash scripts directly (no pnpm
|
||||
# needed in that container at all).
|
||||
#
|
||||
# Concurrency: serialize per-ref so a rapid-fire branch push reuses one lane
|
||||
# instead of stacking jobs on the capacity-1 runners (R-4).
|
||||
name: CI
|
||||
|
||||
on: push
|
||||
|
||||
concurrency:
|
||||
group: ci-${{ gitea.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
node:
|
||||
# Fleet label `node` -> container docker.io/library/node:24-bookworm
|
||||
# (deploy/ci/docker-compose.yaml, R-3 label map).
|
||||
runs-on: node
|
||||
steps:
|
||||
# actions/checkout@v4 downloads from github.com (DEFAULT_ACTIONS_URL
|
||||
# default). github.com egress is verified reachable from the LAN;
|
||||
# the CT-local probe is still pending (Wave 1-1 blocked on founder
|
||||
# deploy-key install). If checkout fails at the first real run, the
|
||||
# fallback is raw git fetch steps using the automatic job token:
|
||||
# - run: |
|
||||
# git init -q .
|
||||
# git remote add origin "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY.git"
|
||||
# git fetch --depth=1 origin "$GITHUB_SHA"
|
||||
# git checkout FETCH_HEAD
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
# Optional pnpm store cache (R-12). Depends on the runner cache server
|
||||
# being reachable from job containers (R-5: cache.host = CT IP + a
|
||||
# published port — configured in deploy/ci/). If the cache server
|
||||
# ETIMEDOUTs on first real runs, drop this block (fallback: a
|
||||
# volume-mounted pnpm store per R-12, or no cache at all — install is
|
||||
# affordable on the LAN). continue-on-error keeps cache flakiness
|
||||
# from ever turning the lane red; repeated timeouts = delete.
|
||||
- name: Cache pnpm store
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.local/share/pnpm/store
|
||||
key: pnpm-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
restore-keys: pnpm-${{ runner.os }}-
|
||||
continue-on-error: true
|
||||
|
||||
# pnpm 12.3.4 is pinned by root package.json `packageManager`.
|
||||
# node:24-bookworm still bundles corepack (24.x retains it, deprecated
|
||||
# — Node >= 25 dropped it; R-11). Fallback if corepack is absent:
|
||||
# npm i -g pnpm@12.3.4
|
||||
- name: Enable pnpm
|
||||
run: corepack enable pnpm
|
||||
|
||||
- name: Install dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
# Lint: per-workspace filters over the 5 TS workspaces ONLY — never
|
||||
# `turbo lint` (see header note).
|
||||
- name: Lint (TS workspaces)
|
||||
run: >-
|
||||
pnpm --filter web --filter @nextcraft/cli --filter @nextcraft/ui
|
||||
--filter @nextcraft/types --filter @nextcraft/mock-data lint
|
||||
|
||||
# Typecheck: same filter set, per-workspace tsc --noEmit.
|
||||
- name: Typecheck (TS workspaces)
|
||||
run: >-
|
||||
pnpm --filter web --filter @nextcraft/cli --filter @nextcraft/ui
|
||||
--filter @nextcraft/types --filter @nextcraft/mock-data typecheck
|
||||
|
||||
- name: Web tests (node:test via tsx)
|
||||
run: pnpm web:test
|
||||
|
||||
# CLI default test script (tsx --test tests/*.test.ts) excludes the
|
||||
# env-gated E2E (apps/cli/package.json `test:e2e`, CLI_E2E=1) — the
|
||||
# fresh-clone E2E stays out of job containers per D-063.
|
||||
- name: CLI tests (node:test via tsx, E2E env-gated out)
|
||||
run: pnpm cli:test
|
||||
|
||||
python:
|
||||
# Fleet label `python` -> container docker.io/library/python:3.11-bookworm
|
||||
# (R-3). The three scripts below are self-contained venv-relative bash
|
||||
# (R-9) — no pnpm, no node, no turbo in this lane.
|
||||
#
|
||||
# P1 CI finding: actions/checkout@v4 is a NODE-based action — it cannot
|
||||
# exec in the python:3.11-bookworm container (exit 127: "node" not in
|
||||
# $PATH). This lane checks out via raw git steps with the automatic job
|
||||
# token instead (the R-8 fallback path, promoted to primary here).
|
||||
runs-on: python
|
||||
steps:
|
||||
- name: Checkout (raw git — node actions cannot run in this container)
|
||||
run: |
|
||||
git init -q .
|
||||
git remote add origin "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY.git"
|
||||
git fetch --depth=1 origin "$GITHUB_SHA"
|
||||
git checkout FETCH_HEAD
|
||||
|
||||
# NOTE: no actions/cache here — it is a node-based action and this
|
||||
# container has no node (exit 127; same class as the checkout finding).
|
||||
# The venv rebuild is ~25s warm — acceptable without cache; pip's own
|
||||
# HTTP cache is cold but the LAN is fast. Revisit only if the lane
|
||||
# slows materially (volume-mounted store per R-12, or a pip download
|
||||
# cache directory via container options).
|
||||
|
||||
# bootstrap.sh is idempotent: creates .venv (with the --without-pip +
|
||||
# get-pip fallback for Debian-without-ensurepip) and pip-installs
|
||||
# the project with [dev] extras (ruff, mypy, pytest).
|
||||
- name: Bootstrap ai-service venv
|
||||
run: bash apps/ai-service/scripts/bootstrap.sh
|
||||
|
||||
- name: Lint (ruff)
|
||||
run: bash apps/ai-service/scripts/lint.sh
|
||||
|
||||
- name: Typecheck (mypy, D-058 baseline)
|
||||
run: bash apps/ai-service/scripts/typecheck.sh
|
||||
|
||||
# Fast pytest lane per D-063: -m "not real_server and not userns"
|
||||
# (scripts/test.sh FAST=1). real_server/userns tiers stay local-only
|
||||
# — job containers genuinely cannot run them (docker default seccomp
|
||||
# denies the nested `unshare --user` the backend needs; R-8).
|
||||
- name: Tests (fast lane)
|
||||
run: FAST=1 bash apps/ai-service/scripts/test.sh
|
||||
@@ -0,0 +1,28 @@
|
||||
# Fleet proof (REQ-8-001, MH-1c): workflow_dispatch one-step echo job per
|
||||
# fleet label. Dispatching this workflow proves all three runners
|
||||
# register, pull their job image, and execute a job. Observed green in the
|
||||
# Actions tab = the runner fleet is verified end-to-end.
|
||||
#
|
||||
# Labels (R-3 fleet map, deploy/ci/docker-compose.yaml):
|
||||
# node -> node:24-bookworm (nc-runner-node)
|
||||
# ubuntu-latest -> runner-images:ubuntu-latest (nc-runner-node, hybrid)
|
||||
# python -> python:3.11-bookworm (nc-runner-python)
|
||||
name: Hello fleet
|
||||
|
||||
on: workflow_dispatch
|
||||
|
||||
jobs:
|
||||
node:
|
||||
runs-on: node
|
||||
steps:
|
||||
- run: echo "hello from node"
|
||||
|
||||
python:
|
||||
runs-on: python
|
||||
steps:
|
||||
- run: echo "hello from python"
|
||||
|
||||
ubuntu-latest:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- run: echo "hello from ubuntu-latest"
|
||||
@@ -0,0 +1,280 @@
|
||||
# Release lane — tag push only (REQ-8-001, D-063, R-14/R-15).
|
||||
#
|
||||
# Fires on `v*` tag pushes. Builds the x64 SEA binary stamped with the tag,
|
||||
# generates the sha256 sidecar, and attaches both to the tag's Gitea
|
||||
# release via the REST API (the same mechanics as scripts/release-assets.sh
|
||||
# — that script stays untouched as the MANUAL fallback with its
|
||||
# .env-only token resolution, per D-064/R-14).
|
||||
#
|
||||
# Within-phase proof (G-203): at verify time push a temporary
|
||||
# `v0.7.1-rc1`-style tag to the phase-branch head, observe this workflow
|
||||
# build + attach green (release immediately deleted or pre-release), then
|
||||
# delete the tag. The phase ship tag remains the real artifact.
|
||||
#
|
||||
# Token precedence (R-13): the automatic job token (auto-provided as the
|
||||
# GITHUB_TOKEN env / `${{ github.token }}` / `${{ gitea.token }}`) HAS
|
||||
# releases scope and can create releases + attach assets, but it CANNOT
|
||||
# publish packages (upstream-confirmed). The repo secret GITEA_TOKEN (a
|
||||
# founder-set PAT), when present, WINS (lowest level wins). This workflow
|
||||
# uses `${{ secrets.GITEA_TOKEN }}` when set, else the automatic token.
|
||||
#
|
||||
# Extension points (P2/P3, G-202: this workflow is the shared rail):
|
||||
# [P2] after "Build binary": docker build + login + push to
|
||||
# git.coreci.dev/coreci/nextcraft:$TAG + :latest with
|
||||
# secrets.REGISTRY_TOKEN, plus the five-assertion smoke gate (G-204).
|
||||
# [P3] arm64 cross-build (G-201 ladder) + both .deb packages (R-22/R-23)
|
||||
# join the attach step below — full asset matrix per D-064.
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
tags: ['v*']
|
||||
|
||||
jobs:
|
||||
release:
|
||||
# Fleet label `release` -> container node:24-bookworm (R-3: node +
|
||||
# dpkg-deb + git + curl; the deb toolchain ships with bookworm).
|
||||
runs-on: release
|
||||
env:
|
||||
# Token precedence per R-13: repo secret GITEA_TOKEN (PAT) wins when
|
||||
# present; else the automatic job token (releases scope is enough for
|
||||
# release-id resolution + asset upload — R-14). Never echo this.
|
||||
RELEASE_TOKEN: ${{ secrets.GITEA_TOKEN || github.token }}
|
||||
steps:
|
||||
- name: Checkout tag
|
||||
uses: actions/checkout@v4
|
||||
|
||||
# P2: the release lane builds/pushes the docker image. The job
|
||||
# container (node:24-bookworm) has no docker CLI and no daemon — the
|
||||
# host daemon is bind-mounted into this job via the runner's
|
||||
# container.options (deploy/ci/config-runner-release.yaml, sibling
|
||||
# mode); only the CLI is missing. Bookworm main ships docker.io
|
||||
# (engine+cli — the engine stays dormant; the socket is the host's).
|
||||
- name: Install docker CLI (host daemon is socket-mounted)
|
||||
run: |
|
||||
apt-get update -qq && apt-get install -y -qq docker.io >/dev/null
|
||||
docker version --format 'client {{.Client.Version}} / server {{.Server.Version}}'
|
||||
|
||||
- name: Enable pnpm
|
||||
run: corepack enable pnpm
|
||||
|
||||
- name: Install dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
# build:binary stamps NEXTCRAFT_VERSION_STAMP from the tag argument
|
||||
# (build-binary.mjs validates it; `--version` reports it — G-102).
|
||||
- name: Build x64 binary (SEA)
|
||||
run: pnpm --filter @nextcraft/cli build:binary ${{ gitea.ref_name }} --arch x64
|
||||
|
||||
# P3 (REQ-8-003, R-25/G-201): arm64 is QEMU-free — the ONLY
|
||||
# arch-specific step (the node binary copy) uses the official arm64
|
||||
# node tarball; esbuild + postject are file surgery, nothing arm64
|
||||
# executes on the x64 runner. The G-201 fallback ladder lives in
|
||||
# PLAN MH-3a: if the probe ever fails, binfmt-full then x64-only
|
||||
# deferral — never a red lane.
|
||||
- name: Build arm64 binary (SEA, QEMU-free cross-build)
|
||||
run: pnpm --filter @nextcraft/cli build:binary ${{ gitea.ref_name }} --arch arm64
|
||||
|
||||
# P3 (R-22/R-23, D-067): deb packages for both arches — binary +
|
||||
# docs, advisory postinst, Version = tag minus v.
|
||||
- name: Build debs (x64 + arm64)
|
||||
run: |
|
||||
bash scripts/build-deb.sh ${{ gitea.ref_name }} x64
|
||||
bash scripts/build-deb.sh ${{ gitea.ref_name }} arm64
|
||||
|
||||
# Deb smoke (MH-3b): the amd64 deb installs in a fresh debian
|
||||
# container and its binary speaks (the arm64 deb cannot exec on this
|
||||
# x64 runner — its checksum + structure are the release-side proof).
|
||||
# The deb enters via STDIN tar — sibling-mode bind mounts resolve on
|
||||
# the HOST, and the job workspace path does not exist there (R-7's
|
||||
# documented gotcha; docker build works because it TARs the context).
|
||||
- name: Deb smoke — fresh-container install + advisory doctor
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG="${{ gitea.ref_name }}"
|
||||
VER="${TAG#v}"
|
||||
DEB="apps/cli/dist/deb/nextcraft_${VER}_amd64.deb"
|
||||
[ -f "$DEB" ] || { echo "missing $DEB"; exit 1; }
|
||||
tar -cf - -C "$(dirname "$DEB")" "$(basename "$DEB")" \
|
||||
| docker run --rm -i --rm debian:bookworm bash -c '
|
||||
set -e
|
||||
mkdir -p /deb && tar -xf - -C /deb
|
||||
DEB_FILE="/deb/$(basename /deb/nextcraft_*.deb)"
|
||||
dpkg -i "$DEB_FILE" >/tmp/dpkg.log 2>&1 || true # advisory postinst: exit 0
|
||||
/usr/bin/nextcraft --version
|
||||
dpkg-deb --field "$DEB_FILE" Package Version Architecture
|
||||
'
|
||||
|
||||
- name: Verify artifacts
|
||||
run: |
|
||||
set -euo pipefail
|
||||
for A in x64 arm64; do
|
||||
BIN="apps/cli/dist/nextcraft-linux-$A"
|
||||
SUM="$BIN.sha256"
|
||||
[ -f "$BIN" ] || { echo "missing $BIN"; exit 1; }
|
||||
[ -f "$SUM" ] || { echo "missing $SUM"; exit 1; }
|
||||
# The sidecar names the bare file — verify from the dist dir.
|
||||
(cd "$(dirname "$BIN")" && sha256sum -c "$(basename "$SUM")")
|
||||
echo "$A size: $(stat -c%s "$BIN") bytes"
|
||||
done
|
||||
for D in apps/cli/dist/deb/*.deb; do
|
||||
[ -f "$D.sha256" ] || { echo "missing $D.sha256"; exit 1; }
|
||||
(cd "$(dirname "$D")" && sha256sum -c "$(basename "$D.sha256")")
|
||||
done
|
||||
|
||||
# Resolve the release id for the tag (the release is auto-created by
|
||||
# Gitea on tag push if the repo setting is on; if not, create it
|
||||
# first). Same call as scripts/release-assets.sh:32-49 (R-14).
|
||||
- name: Resolve release id
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG="${{ gitea.ref_name }}"
|
||||
BASE="https://git.coreci.dev/api/v1/repos/coreci/nextcraft"
|
||||
RESPONSE="$(curl -sS -w '\n%{http_code}' \
|
||||
-H "Authorization: token $RELEASE_TOKEN" \
|
||||
"$BASE/releases/tags/$TAG")"
|
||||
BODY="$(echo "$RESPONSE" | sed '$d')"
|
||||
CODE="$(echo "$RESPONSE" | tail -1)"
|
||||
if [ "$CODE" = "404" ]; then
|
||||
echo "release for $TAG not found — creating it"
|
||||
BODY="$(curl -sS -f -X POST \
|
||||
-H "Authorization: token $RELEASE_TOKEN" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"tag_name\": \"$TAG\", \"name\": \"$TAG\"}" \
|
||||
"$BASE/releases")"
|
||||
else
|
||||
echo "$CODE" | grep -q '^2' || { echo "release lookup failed: $CODE"; echo "$BODY"; exit 1; }
|
||||
fi
|
||||
RELEASE_ID="$(echo "$BODY" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')"
|
||||
echo "release id: $RELEASE_ID"
|
||||
echo "RELEASE_ID=$RELEASE_ID" >> "$GITHUB_ENV"
|
||||
|
||||
# Attach the FULL asset matrix (D-064): 2 binaries + 2 checksums +
|
||||
# 2 debs + 2 deb checksums. actions/upload-artifact is a DIFFERENT
|
||||
# mechanism (job artifacts via the cache server, not release
|
||||
# downloads) — do NOT use it here (R-14). curl multipart is the API.
|
||||
- name: Attach asset matrix to release
|
||||
run: |
|
||||
set -euo pipefail
|
||||
BASE="https://git.coreci.dev/api/v1/repos/coreci/nextcraft"
|
||||
upload() {
|
||||
local file="$1" name="$2"
|
||||
curl -sS -f -X POST \
|
||||
-H "Authorization: token $RELEASE_TOKEN" \
|
||||
-F "attachment=@$file" \
|
||||
"$BASE/releases/$RELEASE_ID/assets?name=$name" >/dev/null
|
||||
echo "attached $name"
|
||||
}
|
||||
for A in x64 arm64; do
|
||||
upload "apps/cli/dist/nextcraft-linux-$A" "nextcraft-linux-$A"
|
||||
upload "apps/cli/dist/nextcraft-linux-$A.sha256" "nextcraft-linux-$A.sha256"
|
||||
done
|
||||
TAG="${{ gitea.ref_name }}"
|
||||
VER="${TAG#v}"
|
||||
for D in amd64 arm64; do
|
||||
upload "apps/cli/dist/deb/nextcraft_${VER}_${D}.deb" "nextcraft_${VER}_${D}.deb"
|
||||
upload "apps/cli/dist/deb/nextcraft_${VER}_${D}.deb.sha256" "nextcraft_${VER}_${D}.deb.sha256"
|
||||
done
|
||||
|
||||
# -- P2 extension (REQ-8-002, R-20, D-065): build + push the docker
|
||||
# image to the Gitea registry on every phase release ----------------
|
||||
# The job token cannot publish packages (R-13, upstream-confirmed);
|
||||
# REGISTRY_TOKEN (repo Actions secret, self-served at P1: G-205) is
|
||||
# the PAT with package-write.
|
||||
- name: Build docker image
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG="${{ gitea.ref_name }}"
|
||||
docker build \
|
||||
--build-arg NEXTCRAFT_VERSION="$TAG" \
|
||||
-t "git.coreci.dev/coreci/nextcraft:$TAG" \
|
||||
-t "git.coreci.dev/coreci/nextcraft:latest" \
|
||||
.
|
||||
|
||||
- name: Push image to the Gitea registry
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "${{ secrets.REGISTRY_TOKEN }}" | docker login git.coreci.dev \
|
||||
-u coreci --password-stdin
|
||||
TAG="${{ gitea.ref_name }}"
|
||||
docker push "git.coreci.dev/coreci/nextcraft:$TAG"
|
||||
docker push "git.coreci.dev/coreci/nextcraft:latest"
|
||||
|
||||
# Smoke gate (G-204, five pinned assertions): a red image = red
|
||||
# release. Probes run via docker exec INSIDE the container — the
|
||||
# job container's 127.0.0.1 is not the host, and the smoke container
|
||||
# publishes no ports (the admin-facing port map is the admin's call,
|
||||
# not the gate's).
|
||||
- name: Smoke gate — unprivileged run proves the honest degrade
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG="${{ gitea.ref_name }}"
|
||||
CID=""
|
||||
cleanup() { [ -n "$CID" ] && docker rm -f "$CID" >/dev/null 2>&1 || true; }
|
||||
trap cleanup EXIT
|
||||
CID="$(docker run -d "git.coreci.dev/coreci/nextcraft:$TAG")"
|
||||
echo "container $CID up"
|
||||
|
||||
# 1. /health 200 within the window (in-container urllib — slim
|
||||
# has no curl; the HEALTHCHECK does the same)
|
||||
ok=""
|
||||
for i in $(seq 1 30); do
|
||||
if docker exec "$CID" python3 -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8420/health', timeout=3).status == 200 else 1)" 2>/dev/null; then ok=1; break; fi
|
||||
sleep 2
|
||||
done
|
||||
[ -n "$ok" ] || { echo "FAIL: /health never returned 200"; docker logs "$CID" 2>&1 | tail -30; exit 1; }
|
||||
echo "assert 1/5: /health 200"
|
||||
|
||||
# 2. GET / is HTML with a real document marker (not just non-JSON
|
||||
# — the v0.4.6 lesson; G-204). One-line python: checks the
|
||||
# doctype/html prefix on the FULL lowered head, not a 6-char
|
||||
# slice (rc6 finding — the slice truncated the marker).
|
||||
docker exec "$CID" python3 -c "import urllib.request,sys; body = urllib.request.urlopen('http://127.0.0.1:8420/', timeout=5).read(); head = body.lstrip()[:32].lower(); sys.exit(0 if (head.startswith(b'<!doctype') or head.startswith(b'<html')) else 1)" \
|
||||
|| { echo "FAIL: / is not HTML"; exit 1; }
|
||||
echo "assert 2/5: / serves HTML"
|
||||
|
||||
# 3. degrade boot notice present in container logs (unprivileged
|
||||
# default → seccomp blocks userns → loud notice, D-068).
|
||||
# Retry window: the /health probe can pass in the same instant
|
||||
# the boot log is still flushing (rc4 finding — 60ms race);
|
||||
# the notice check polls up to 15s, it does not trust timing.
|
||||
ok=""
|
||||
for i in $(seq 1 15); do
|
||||
if docker logs "$CID" 2>&1 | grep -q "SANDBOX FABRIC DEGRADED"; then ok=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
[ -n "$ok" ] \
|
||||
|| { echo "FAIL: degrade boot notice absent"; docker logs "$CID" 2>&1 | tail -30; exit 1; }
|
||||
echo "assert 3/5: degrade notice logged"
|
||||
|
||||
# 4. fabric env-create is refused BEFORE the fabric (503 degraded
|
||||
# or 403 gate-first — both prove the degrade path short-circuits
|
||||
# learner env creation; a 201 would prove the fabric ran, which
|
||||
# an unprivileged container cannot allow). The probe python is
|
||||
# written to a file first (a yaml block scalar cannot embed
|
||||
# unindented command substitution continuations).
|
||||
cat > /tmp/smoke_probe.py << 'PYEOF'
|
||||
import urllib.request, urllib.error, json, sys
|
||||
req = urllib.request.Request('http://127.0.0.1:8420/v1/sandboxes',
|
||||
data=json.dumps({'learner_id':'smoke','task_id':'t'}).encode(),
|
||||
headers={'Content-Type':'application/json'}, method='POST')
|
||||
try:
|
||||
print(urllib.request.urlopen(req, timeout=5).status)
|
||||
except urllib.error.HTTPError as e:
|
||||
print(e.code)
|
||||
except Exception:
|
||||
print('ERR')
|
||||
sys.stdout.flush()
|
||||
PYEOF
|
||||
docker cp /tmp/smoke_probe.py "$CID":/tmp/smoke_probe.py
|
||||
STATUS="$(docker exec "$CID" python3 /tmp/smoke_probe.py)"
|
||||
[ "$STATUS" = "503" ] || [ "$STATUS" = "403" ] \
|
||||
|| { echo "FAIL: env-create was $STATUS (expected 503 degraded or 403 gate)"; exit 1; }
|
||||
echo "assert 4/5: degraded env-create refused ($STATUS)"
|
||||
|
||||
# 5. in-container CLI speaks (embedded doctor reports the docker
|
||||
# channel — D-066) + clean stop
|
||||
docker exec "$CID" nextcraft --version \
|
||||
|| { echo "FAIL: embedded CLI does not run"; exit 1; }
|
||||
echo "assert 5/5: embedded CLI alive + clean stop"
|
||||
docker rm -f "$CID" >/dev/null && CID=""
|
||||
@@ -20,6 +20,8 @@ dist/
|
||||
.next/
|
||||
.turbo/
|
||||
*.tsbuildinfo
|
||||
# Next.js static export (v0.3.6 build output, served by the ai-service)
|
||||
apps/web/out/
|
||||
|
||||
# Storybook
|
||||
storybook-static/
|
||||
@@ -42,6 +44,7 @@ yarn-error.log*
|
||||
|
||||
# Test coverage
|
||||
coverage/
|
||||
.coverage
|
||||
.nyc_output/
|
||||
|
||||
# Python tooling caches
|
||||
@@ -55,3 +58,6 @@ apps/ai-service/ai_service/data/
|
||||
apps/ai-service/**/sandboxes/
|
||||
*.db
|
||||
*.db-journal
|
||||
|
||||
# in-sandbox capture agent runtime spool
|
||||
.nc-agent/
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
shamefully-hoist=true
|
||||
strict-peer-dependencies=false
|
||||
pnpm-approved-builds=esbuild,core-js-pure
|
||||
pnpm-approved-builds=esbuild,core-js-pure
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# Pre-commit hooks (D-070) — local hygiene mirroring the CI lanes.
|
||||
#
|
||||
# Design contract (G-209): every hook is LOCAL-ONLY (no network at hook
|
||||
# time — the mirror-style hooks below are fetched once at `pre-commit
|
||||
# install`/`pre-commit run` time from github.com, which this LAN reaches;
|
||||
# the language:system hooks run zero network). Hooks FAIL LOUD when the
|
||||
# toolchain is missing (D-070: a contributor without the toolbox gets an
|
||||
# explicit pointer, not a silent skip) — the no-toolbox escape is
|
||||
# `git commit --no-verify` (or `SKIP=...`), documented in README.md with
|
||||
# the loud note that CI still gates every push.
|
||||
#
|
||||
# Scope discipline mirrors .gitea/workflows/ci.yaml: ruff only touches
|
||||
# ai-service python; eslint/tsc only touch the TS workspaces; hygiene is
|
||||
# repo-wide.
|
||||
repos:
|
||||
# ---- basic hygiene (repo-wide, zero toolchain required)
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.6.0
|
||||
hooks:
|
||||
- id: trailing-whitespace
|
||||
- id: end-of-file-fixer
|
||||
- id: check-merge-conflict
|
||||
- id: check-added-large-files
|
||||
|
||||
# ---- ruff (ai-service python, D-063 lane parity)
|
||||
# Runs the venv-relative binary exactly like apps/ai-service/scripts/lint.sh
|
||||
# (venv is the repo's authority for the python toolchain). FAILS LOUD with
|
||||
# a bootstrap hint if the venv is missing — never a silent skip.
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: ruff
|
||||
name: ruff check (ai-service)
|
||||
entry: apps/ai-service/scripts/lint.sh
|
||||
language: system
|
||||
types: [python]
|
||||
files: ^apps/ai-service/
|
||||
pass_filenames: false
|
||||
|
||||
# ---- eslint (TS workspaces, ONE root flat config at eslint.config.mjs)
|
||||
# Each workspace's `lint` script is `eslint .` resolving that config
|
||||
# upward, so a single root invocation is exact parity with the node
|
||||
# lane's per-workspace runs. `pnpm` resolves via PATH or corepack
|
||||
# (same discovery as nextcraft doctor); FAILS LOUD if neither exists.
|
||||
- id: eslint
|
||||
name: eslint (TS workspaces)
|
||||
entry: bash -c 'PNPM="$(command -v pnpm || echo "corepack pnpm")"; $PNPM exec eslint apps/web apps/cli packages/ui packages/types packages/mock-data'
|
||||
language: system
|
||||
files: ^(apps/web|apps/cli|packages/ui|packages/types|packages/mock-data)/.*\.(ts|tsx|mjs|cjs)$
|
||||
pass_filenames: false
|
||||
|
||||
# ---- tsc --noEmit per TS workspace (node-lane parity: pnpm --filter
|
||||
# over the same five workspaces; the tsconfig graph stays per-project)
|
||||
# (the shared PCMD discovery is repeated per hook because pre-commit
|
||||
# hooks run isolated entries)
|
||||
- id: tsc-web
|
||||
name: tsc --noEmit (web)
|
||||
entry: bash -c 'PNPM="$(command -v pnpm || echo "corepack pnpm")"; $PNPM --filter web typecheck'
|
||||
language: system
|
||||
files: ^apps/web/.*\.tsx?$
|
||||
pass_filenames: false
|
||||
- id: tsc-cli
|
||||
name: tsc --noEmit (cli)
|
||||
entry: bash -c 'PNPM="$(command -v pnpm || echo "corepack pnpm")"; $PNPM --filter @nextcraft/cli typecheck'
|
||||
language: system
|
||||
files: ^apps/cli/.*\.tsx?$
|
||||
pass_filenames: false
|
||||
- id: tsc-ui
|
||||
name: tsc --noEmit (ui)
|
||||
entry: bash -c 'PNPM="$(command -v pnpm || echo "corepack pnpm")"; $PNPM --filter @nextcraft/ui typecheck'
|
||||
language: system
|
||||
files: ^packages/ui/.*\.tsx?$
|
||||
pass_filenames: false
|
||||
- id: tsc-types
|
||||
name: tsc --noEmit (types)
|
||||
entry: bash -c 'PNPM="$(command -v pnpm || echo "corepack pnpm")"; $PNPM --filter @nextcraft/types typecheck'
|
||||
language: system
|
||||
files: ^packages/types/.*\.tsx?$
|
||||
pass_filenames: false
|
||||
- id: tsc-mock-data
|
||||
name: tsc --noEmit (mock-data)
|
||||
entry: bash -c 'PNPM="$(command -v pnpm || echo "corepack pnpm")"; $PNPM --filter @nextcraft/mock-data typecheck'
|
||||
language: system
|
||||
files: ^packages/mock-data/.*\.tsx?$
|
||||
pass_filenames: false
|
||||
+111
@@ -0,0 +1,111 @@
|
||||
# Nextcraft — single-port docker image (v0.8 P2, REQ-8-002, R-16..R-19)
|
||||
#
|
||||
# Topology: uvicorn :8420 serves BOTH the API (/v1/*, /health) and the
|
||||
# exported web UI (everything else) — the same single-port production
|
||||
# shape as `nextcraft deploy` (D-039). Deps are pre-loaded in image
|
||||
# layers; runtime is unprivileged by default (honest sandbox degrade:
|
||||
# ai_service/sandbox/capability.py probes + api/sandboxes.py 503s with
|
||||
# remediation — D-068/G-208).
|
||||
#
|
||||
# Build context is the repo root. Build args:
|
||||
# NEXTCRAFT_VERSION — stamped into the embedded CLI (`--version`)
|
||||
#
|
||||
# Stages:
|
||||
# 1. web-export — node:24-bookworm: pnpm workspace install, static
|
||||
# export (NEXT_PUBLIC_AI_SERVICE_URL=self), SEA CLI binary build
|
||||
# 2. venv — python:3.11-bookworm: production deps only (no [dev])
|
||||
# 3. runtime — python:3.11-slim-bookworm: non-root, venv + package +
|
||||
# export + CLI, /health healthcheck, state volume
|
||||
|
||||
# -- stage 1: web export + CLI binary ---------------------------------------
|
||||
FROM node:24-bookworm AS web-export
|
||||
ARG NEXTCRAFT_VERSION=0.0.0-dev
|
||||
WORKDIR /repo
|
||||
|
||||
# Toolchain: corepack ships with node 24 (deprecated but present).
|
||||
RUN corepack enable pnpm
|
||||
|
||||
# Workspace manifests first (layer-cacheable install).
|
||||
COPY package.json pnpm-workspace.yaml pnpm-lock.yaml .npmrc tsconfig.json ./
|
||||
COPY apps/web/package.json apps/web/
|
||||
COPY apps/cli/package.json apps/cli/
|
||||
COPY packages/types/package.json packages/types/
|
||||
COPY packages/ui/package.json packages/ui/
|
||||
COPY packages/mock-data/package.json packages/mock-data/
|
||||
RUN pnpm install --frozen-lockfile
|
||||
|
||||
# Sources (web build needs all workspaces; CLI build needs its tree).
|
||||
COPY apps ./apps
|
||||
COPY packages ./packages
|
||||
|
||||
# Static export (same command as the deploy path): the export lands in
|
||||
# apps/web/out and is served same-origin by uvicorn.
|
||||
ENV NEXT_PUBLIC_AI_SERVICE_URL=self
|
||||
RUN pnpm --filter web build
|
||||
|
||||
# Embedded CLI (D-066): doctor/verify/guide work in-container; the daemon
|
||||
# commands refuse honestly (see apps/cli/src — NC_CONTAINER detection).
|
||||
RUN pnpm --filter @nextcraft/cli build:binary "$NEXTCRAFT_VERSION"
|
||||
|
||||
# -- stage 2: python venv ----------------------------------------------------
|
||||
FROM python:3.11-bookworm AS venv
|
||||
WORKDIR /repo
|
||||
# The full source tree installs the package (bootstrap.sh's mechanism
|
||||
# without [dev]: production deps only — the image runs the app, it never
|
||||
# lints/tests it; the runner lanes own that).
|
||||
COPY apps/ai-service /repo/apps/ai-service
|
||||
RUN python3 -m venv /opt/venv && \
|
||||
/opt/venv/bin/pip install --no-cache-dir --upgrade pip && \
|
||||
/opt/venv/bin/pip install --no-cache-dir /repo/apps/ai-service
|
||||
|
||||
# -- stage 3: runtime --------------------------------------------------------
|
||||
FROM python:3.11-slim-bookworm AS runtime
|
||||
WORKDIR /opt/nextcraft
|
||||
|
||||
# Non-root user; home carries the state volume (~/.nextcraft semantics —
|
||||
# config.py anchors db/sandboxes under $HOME/.nextcraft, R-17).
|
||||
RUN useradd --create-home --uid 10001 nextcraft
|
||||
|
||||
# venv from stage 2 (production deps only).
|
||||
COPY --from=venv /opt/venv /opt/venv
|
||||
ENV PATH=/opt/venv/bin:$PATH
|
||||
|
||||
# The installed ai_service package is IN the venv's site-packages; the
|
||||
# source tree rides along for transparency (PYTHONPATH prefers the
|
||||
# installed package).
|
||||
COPY --from=venv /repo/apps/ai-service /opt/nextcraft/ai-service-src
|
||||
RUN rm -rf /opt/nextcraft/ai-service-src/.venv
|
||||
|
||||
# Web export from stage 1.
|
||||
COPY --from=web-export /repo/apps/web/out /opt/nextcraft/web-out
|
||||
ENV AI_WEB_STATIC_DIR=/opt/nextcraft/web-out
|
||||
|
||||
# Image marker: NC_CONTAINER=1 (context for the CLI container-mode
|
||||
# branches and the degrade boot notice; the capability PROBE — not this
|
||||
# marker — decides degrade, D-068).
|
||||
ENV NC_CONTAINER=1
|
||||
|
||||
# Embedded CLI (D-066).
|
||||
COPY --from=web-export /repo/apps/cli/dist/nextcraft-linux-x64 /usr/local/bin/nextcraft
|
||||
RUN chmod 0755 /usr/local/bin/nextcraft
|
||||
|
||||
# App dir + state: db/sandboxes under /home/nextcraft/.nextcraft (R-17).
|
||||
# The state root is pre-created with correct ownership BEFORE the VOLUME
|
||||
# declaration — an anonymous volume copies the image's directory CONTENTS
|
||||
# and ownership at first use, so the app (uid 10001) can mkdir its data/
|
||||
# and sandboxes/ subdirs on a clean boot (rc5 smoke-gate finding: the
|
||||
# bare useradd home did not carry a writable .nextcraft).
|
||||
RUN mkdir -p /home/nextcraft/.nextcraft/data \
|
||||
&& chown -R nextcraft:nextcraft /opt/nextcraft /home/nextcraft/.nextcraft
|
||||
WORKDIR /opt/nextcraft/ai-service-src
|
||||
USER nextcraft
|
||||
ENV AI_HOST=0.0.0.0 AI_PORT=8420
|
||||
|
||||
EXPOSE 8420
|
||||
# slim has no curl — the healthcheck speaks python (stdlib urllib).
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
|
||||
CMD ["python3", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8420/health', timeout=4).status == 200 else 1)"]
|
||||
|
||||
VOLUME ["/home/nextcraft/.nextcraft"]
|
||||
|
||||
CMD ["python3", "-m", "uvicorn", "ai_service.main:app", "--host", "0.0.0.0", "--port", "8420"]
|
||||
@@ -2,8 +2,129 @@
|
||||
|
||||
AI-native outcome school + marketplace — graduates prove what they can build, not what they can write.
|
||||
|
||||
## Quickstart
|
||||
|
||||
nextcraft installs on any Debian-based Linux server through **three channels** (full recipes in [deploy/README.md](deploy/README.md) or `nextcraft guide`):
|
||||
|
||||
```sh
|
||||
# 1. docker — the whole stack pre-loaded (API + UI + CLI in one image):
|
||||
docker run -d --name nextcraft -p 8420:8420 \
|
||||
-v nextcraft-state:/home/nextcraft/.nextcraft \
|
||||
git.coreci.dev/coreci/nextcraft:<release-tag>
|
||||
|
||||
# 2. deb — apt-shaped (amd64 + arm64):
|
||||
dpkg -i nextcraft_<ver>_<arch>.deb && nextcraft deploy --service
|
||||
|
||||
# 3. binary — the one-liner (linux x64 + arm64):
|
||||
curl -fsSL https://git.coreci.dev/coreci/nextcraft/raw/main/scripts/install.sh | sh
|
||||
nextcraft deploy --service
|
||||
```
|
||||
|
||||
The binary download verifies sha256 + version stamps before install (mismatch = hard stop). Every release ships fresh binaries, debs, and a registry image — all machine-made by the [release workflow](.gitea/workflows/release.yaml).
|
||||
|
||||
### Server (deploy the whole stack, one command)
|
||||
|
||||
```sh
|
||||
nextcraft deploy --service
|
||||
```
|
||||
|
||||
That clones (or updates) the repo, installs deps, builds the web UI, restarts the service, verifies the site is served, and installs a systemd unit so the stack survives reboots. Idempotent — same command for first install and upgrades. Toolchain discovery included: node/pnpm behind nvm/PNPM_HOME/corepack are found even when non-interactive shells can't see them.
|
||||
|
||||
### Development (from a clone of this repo)
|
||||
|
||||
```sh
|
||||
nextcraft doctor # check prerequisites: node >= 18, pnpm >= 8, python3 >= 3.11, git, unshare
|
||||
nextcraft bootstrap # pnpm install + ai-service venv + .env from template (idempotent)
|
||||
nextcraft verify # health check: venv imports, uvicorn, ports, env, web export, live mount probe
|
||||
nextcraft dev # run the ai-service dev server on :8420 (web dev server: pnpm dev)
|
||||
```
|
||||
|
||||
The E2E test (`apps/cli/tests/fresh-clone-e2e.test.ts`) proves this exact sequence on a fresh clone.
|
||||
|
||||
#### Pre-commit hooks (optional, recommended)
|
||||
|
||||
The repo ships a `.pre-commit-config.yaml` that mirrors the CI lanes locally: ruff over the ai-service python, eslint over the five TS workspaces (one root flat config), `tsc --noEmit` per workspace, plus whitespace/merge-conflict/large-file hygiene. Hooks are local-only and fail loud.
|
||||
|
||||
```sh
|
||||
pip install pre-commit # or: pipx install pre-commit
|
||||
pre-commit install
|
||||
```
|
||||
|
||||
From then on every `git commit` runs the hooks on the staged changes. First run may fetch the hygiene hook repo (github.com, reachable from this LAN).
|
||||
|
||||
**No toolbox? The honest skip path:** if you don't have the ruff/eslint/tsc toolchain set up (e.g. no ai-service venv yet — run `nextcraft bootstrap` first), hooks will fail loud with a pointer. The simplest honest escape for a contributor without the toolbox is:
|
||||
|
||||
```sh
|
||||
git commit --no-verify
|
||||
```
|
||||
|
||||
or skip specific hooks for one commit:
|
||||
|
||||
```sh
|
||||
SKIP=ruff,eslint,tsc-web,tsc-cli,tsc-ui,tsc-types,tsc-mock-data git commit
|
||||
```
|
||||
|
||||
**Loud note: CI still gates.** Skipping locally changes nothing about the push lanes — `.gitea/workflows/ci.yaml` runs lint + typecheck + tests on every push to every branch. A `--no-verify` commit that breaks lint gets caught at push time, just later and more publicly.
|
||||
|
||||
### No binary / non-linux?
|
||||
|
||||
The installer degrades to printed source instructions. Manual equivalent:
|
||||
|
||||
```sh
|
||||
git clone https://git.coreci.dev/coreci/nextcraft.git && cd nextcraft
|
||||
pnpm install
|
||||
bash apps/ai-service/scripts/bootstrap.sh
|
||||
cp apps/ai-service/.env.example apps/ai-service/.env
|
||||
pnpm ai:dev
|
||||
```
|
||||
|
||||
## CLI reference (`nextcraft`)
|
||||
|
||||
| Command | What it does | Exit codes |
|
||||
|---------|--------------|------------|
|
||||
| `doctor` | Checks prerequisites: node >= 18, pnpm >= 8 (discovered via PATH/nvm/PNPM_HOME/corepack), python3 >= 3.11, git, unshare (sandbox fabric). Every ✗ prints a fix hint. | 0 all pass, 1 any fail |
|
||||
| `bootstrap` | Sets up the monorepo from a fresh clone: (1) locates the repo root, (2) `pnpm install` (discovered toolchain), (3) ai-service venv via `apps/ai-service/scripts/bootstrap.sh`, (4) copies `.env.example` → `.env` if absent, (5) warns on missing optional keys. Idempotent — safe to re-run. | 0 ok, 1 step failed |
|
||||
| `verify` | Health check: ai-service venv + `import ai_service`, uvicorn importable, `.env` present (warn-only), `AI_PORT` state — free, or **busy + serving the web UI** (live mount probe: `/` must return HTML, not the API's JSON 404), web export present (`apps/web/out`), workspace `node_modules`. | 0 ok, 1 failures |
|
||||
| `deploy` | One-command deploy/upgrade: clone-or-update repo (`--dir`, `--ref`), `pnpm install`, ai-service venv, `NEXT_PUBLIC_AI_SERVICE_URL=self pnpm build`, restart, verify. Fails loudly if the site isn't served. | 0 ok, 1 step failed |
|
||||
| `deploy --service` | Same, plus generates + installs a systemd unit (real paths, `Restart=always`, enabled) so the stack survives reboots; systemd becomes the supervisor. | 0 ok, 1 step failed |
|
||||
| `dev` | Thin passthrough to `apps/ai-service/scripts/dev.sh` (exports secrets from `.ciagent/.env.secrets` if present, runs uvicorn on :8420). Ctrl+C stops it. The web dev server is separate: `pnpm dev`. | child's exit code |
|
||||
| `dev -d` / `dev --detach` | Same, but as an **unattended daemon**: detached, output appended to `~/.nextcraft/run/<clone>/dev.log`, pidfile beside it. Refuses to double-start (and refuses when the systemd service is active). Auto-serves the built web UI same-origin when `apps/web/out` exists. | 0 started, 1 already running |
|
||||
| `stop` | Stops the systemd service when it supervises the stack; otherwise stops the daemon started by `dev -d` (SIGTERM, SIGKILL after 5s) and cleans the pidfile. | 0 stopped/clean, 1 kill failed |
|
||||
| `log` | Tails the daemon log: last 50 lines by default, `-n N` for more, `-f`/`--follow` to stream. | 0 ok, 1 no log yet |
|
||||
| `--help` / `-h` | Usage for the CLI or any command. | 0 |
|
||||
| `--version` | Prints the version this binary was built as (matches the release tag). | 0 |
|
||||
|
||||
Exit-code contract: `0` success, `1` check/step failure (hint printed), `2` usage error.
|
||||
|
||||
### Remote server / single-port deployment
|
||||
|
||||
`nextcraft dev` binds the API on **0.0.0.0:8420** (and `pnpm dev` serves the web app on all interfaces), so the stack works from other machines out of the box:
|
||||
|
||||
- Browse `http://<your-host>:3000` — the web app targets `http://<your-host>:8420` automatically (derived from the browser's hostname).
|
||||
- CORS admits any origin (`AI_CORS_ORIGINS=*` in `apps/ai-service/.env`). This is safe **only** because credentials are never enabled; to restrict, set an explicit list: `AI_CORS_ORIGINS=http://<your-host>:3000`.
|
||||
- To revert to loopback-only: `AI_HOST=127.0.0.1` in `apps/ai-service/.env`.
|
||||
- Security note: this is an unauthenticated dev API reachable from any network the box exposes. Mitigations that still apply: per-learner sandbox caps + global rate caps + learner allowlist (G-5), telemetry flood control (traces marked `INCOMPLETE_FLOODED` are refused by the grader). Expose only on trusted networks until identity/KYC lands (v0.5).
|
||||
|
||||
**Production behind one port (v0.3.6):** when only one port is reachable (e.g. behind HAProxy), the one-command deploy builds the web app as a static export and lets the ai-service serve it same-origin on :8420 — UI and API on one port, no CORS, no mixed content:
|
||||
|
||||
```sh
|
||||
nextcraft deploy --service # install/upgrade + web export + restart + verify + systemd unit
|
||||
```
|
||||
|
||||
Durable state (SQLite DB, sandbox workdirs, daemon pid/log) lives in `~/.nextcraft/` — never inside the repo. Full recipe incl. HAProxy timeouts and a systemd unit: [deploy/README.md](deploy/README.md).
|
||||
|
||||
## Docs
|
||||
|
||||
- [apps/cli/README.md](apps/cli/README.md) — CLI internals: build, binary pipeline, troubleshooting
|
||||
- [.ciagent/PROJECT.md](.ciagent/PROJECT.md) — product spec and milestone history
|
||||
- [.ciagent/ARCHITECTURE.md](.ciagent/ARCHITECTURE.md) — system architecture
|
||||
|
||||
## Status
|
||||
|
||||
**Milestone v0.1** — UI/UX Prototype (high-fidelity interactive, all mock data)
|
||||
**Milestone v0.8** — Ship Anywhere (in progress): installable on any Debian-based server via every mainstream channel — docker image (Gitea registry, honest sandbox degrade), .deb packages (amd64+arm64, advisory postinst), arch-aware one-liner binary — with release consistency enforced by pre-commit hooks + Gitea Actions pipelines (runner fleet on CT 113, machine-made releases), the CLI carrying the full suite (`nextcraft guide`, channel-aware doctor), and the admin runbook (deploy/README.md).
|
||||
|
||||
Initialized via CIAgent v0.7.0
|
||||
Prior: v0.7 Test Harness Review (shipped as v0.6.6) · **Milestone v0.6** — Marketplace Engine (shipped as v0.5.6): real job-postings backend behind the marketplace surfaces (6th SQLite store family + seeded ingestion adapter + admin authoring surface), AI tagging with facet-driven filters, embedding-backed semantic search with computed match scores, employer enrichment with per-field provenance, resume-to-profile auto-fill with an explicit confirm gate, and SEO programmatic pages (metadata, JSON-LD, dynamic sitemap/robots).
|
||||
|
||||
Prior: v0.5 Real Server Voice, KYC/Identity, Sandbox Environments, Seq-Lease (shipped as v0.4.5) · v0.3 Credential Engines (v0.2.8) · v0.2 AI Tutor Architecture (v0.2.0) · v0.1 UI/UX Prototype (v0.1.0)
|
||||
|
||||
Initialized via CIAgent v0.7.0
|
||||
|
||||
@@ -2,6 +2,13 @@
|
||||
# Real keys live in .ciagent/.env.secrets (gitignored) and are exported by scripts/dev.sh
|
||||
|
||||
AI_PORT=8420
|
||||
# Network mode (v0.3.5, D-038): dev server binds 0.0.0.0 so remote machines can
|
||||
# reach the stack. Set to 127.0.0.1 to revert to loopback-only.
|
||||
AI_HOST=0.0.0.0
|
||||
# CORS + WS-origin policy: '*' (default) admits any origin — safe because
|
||||
# credentials are never enabled. Restrict with a comma list, e.g.:
|
||||
# AI_CORS_ORIGINS=http://nextcraft-1:3000
|
||||
AI_CORS_ORIGINS=*
|
||||
AI_PROVIDER=ollama-cloud
|
||||
AI_MODEL=gemma4:31b
|
||||
AI_OLLAMA_CLOUD_BASE_URL=https://ollama.com/v1
|
||||
@@ -9,8 +16,10 @@ AI_OLLAMA_CLOUD_API_KEY=
|
||||
AI_LOCAL_BASE_URL=http://localhost:11434/v1
|
||||
AI_JSON_MODE=auto
|
||||
|
||||
# Sandbox fabric (v0.3)
|
||||
AI_SANDBOX_DIR=sandboxes
|
||||
# Sandbox fabric (v0.3). v0.3.6: durable state defaults to ~/.nextcraft/ —
|
||||
# OUTSIDE the repo (the old repo-relative 'sandboxes' default polluted the
|
||||
# git tree). Set an absolute path (~/ works) or a repo-relative one only for
|
||||
# throwaway dev clones.
|
||||
AI_SANDBOX_MAX_CONCURRENT=5
|
||||
AI_SANDBOX_TIMEOUT_S=900
|
||||
AI_SANDBOX_MAX_WORKDIR_MB=512
|
||||
@@ -23,5 +32,101 @@ AI_SANDBOX_MAX_PER_LEARNER=1
|
||||
# global creates per rolling 60s window (in-memory) → 429 when exceeded
|
||||
AI_SANDBOX_CREATES_PER_MIN=10
|
||||
|
||||
# Persistence (SQLite)
|
||||
AI_DB_PATH=ai_service/data/nextcraft.db
|
||||
# Persistence (SQLite). v0.3.6: default moved out of the repo to
|
||||
# ~/.nextcraft/data/nextcraft.db (~/ paths in env overrides are expanded).
|
||||
# Uncomment + edit ONLY to relocate:
|
||||
# AI_DB_PATH=~/.nextcraft/data/nextcraft.db
|
||||
|
||||
# Single-port deploy (v0.3.6): directory of the exported web app
|
||||
# (apps/web/out). When set, the UI is served from this same service on :8420
|
||||
# — build it with `NEXT_PUBLIC_AI_SERVICE_URL=self pnpm build`. The
|
||||
# `nextcraft dev` command auto-sets this when apps/web/out exists.
|
||||
# AI_WEB_STATIC_DIR=../../web/out
|
||||
# --- Identity (REQ-5-003, D-042) ---
|
||||
# 'mock' (default — deterministic, no vendor spend pre-pilot; verdicts carry
|
||||
# mock=True forever per A-304). A real KYC vendor drops in via the
|
||||
# IdentityProvider protocol without API changes.
|
||||
AI_IDENTITY_PROVIDER=mock
|
||||
# G-13: identity submit caps — one active pending per learner (409), and a
|
||||
# per-learner submit rate ceiling (429 over a rolling 60s window).
|
||||
AI_IDENTITY_SUBMITS_PER_MIN=3
|
||||
|
||||
# --- Voice (REQ-3-006 D-030; real server path REQ-5-001, D-040) ---
|
||||
# 'mock' (default; no key needed — tests/dev), 'browser' (client-native
|
||||
# SR/TTS), or 'openai-audio' (real server STT/TTS, live since v0.5).
|
||||
AI_VOICE_PROVIDER=mock
|
||||
# openai-audio requires BOTH (unconfigured → app boots, voice falls back to
|
||||
# mock with a loud log — G-11; the badge then honestly reports mock):
|
||||
# AI_VOICE_BASE_URL=https://your-audio-endpoint/v1
|
||||
# AI_VOICE_API_KEY=
|
||||
# Optional model/voice/format knobs (defaults shown):
|
||||
# AI_VOICE_STT_MODEL=whisper-1
|
||||
# AI_VOICE_TTS_MODEL=tts-1
|
||||
# AI_VOICE_TTS_VOICE=alloy
|
||||
# AI_VOICE_TTS_FORMAT=mp3 (enum: mp3 | wav | opus)
|
||||
# AI_VOICE_MAX_AUDIO_MB=10
|
||||
# Manual probe recipe (executable by anyone with the keys; never CI):
|
||||
# STT: curl -sS $AI_VOICE_BASE_URL/audio/transcriptions \
|
||||
# -H "Authorization: Bearer $AI_VOICE_API_KEY" \
|
||||
# -F file=@test/fixtures/answer.wav -F model=whisper-1 | jq -e '.text'
|
||||
# TTS: curl -sS $AI_VOICE_BASE_URL/audio/speech \
|
||||
# -H "Authorization: Bearer $AI_VOICE_API_KEY" \
|
||||
# -H 'Content-Type: application/json' \
|
||||
# -d '{"model":"tts-1","input":"Nextcraft","voice":"alloy"}' \
|
||||
# -o /tmp/probe.mp3 && file /tmp/probe.mp3 | grep -i audio
|
||||
|
||||
# --- Marketplace (REQ-6-001, D-057) ---
|
||||
# Founder/admin surface guard (/v1/marketplace/admin/*): send as the
|
||||
# 'X-Admin-Token' header OR 'Authorization: Bearer <token>'; compared
|
||||
# constant-time (G-108). UNSET = every admin route 403s (fail-closed,
|
||||
# never open) and the boot log warns loudly. Never commit real tokens.
|
||||
# AI_ADMIN_TOKEN=
|
||||
# P5 (D-056) placeholder: the deployed origin for sitemap.xml / robots.txt
|
||||
# absolute URLs (e.g. https://nextcraft.example). Unset → SEO phase
|
||||
# serves the empty-page set with a boot warning (G-105).
|
||||
# AI_PUBLIC_BASE_URL=
|
||||
|
||||
# --- Employer auth throttles (REQ-10-001, D-088) ---
|
||||
# Documentation of the ACTIVE defaults (a-313) — both knobs are always
|
||||
# on as Settings ints; these env entries override for ops tuning.
|
||||
# (a) Failed logins allowed per email per rolling 15-min window
|
||||
# (default 5 → 429 with an honest "wait" detail; a successful
|
||||
# login clears the window).
|
||||
# AI_EMPLOYER_LOGIN_FAILURES_PER_WINDOW=5
|
||||
# (b) Well-formed login+register attempts per rolling 60-s window
|
||||
# across all emails (default 30 → 429; failed passwords count —
|
||||
# the credential-stuffing bound, G-314).
|
||||
# AI_EMPLOYER_AUTH_ATTEMPTS_PER_MIN=30
|
||||
|
||||
# --- Semantic search embeddings (REQ-6-003, D-051) ---
|
||||
# 'mock' (default — deterministic 256-dim feature-hash, pure stdlib, no
|
||||
# key needed; tests never call a real embeddings API) or 'openai-embed'.
|
||||
AI_EMBED_PROVIDER=mock
|
||||
# openai-embed requires ALL THREE below; unconfigured → the app boots and
|
||||
# embeddings fall back to mock with a loud log (G-11).
|
||||
# PROBE CORRECTION (D-051): ollama-cloud does NOT expose /embeddings —
|
||||
# point AI_EMBED_BASE_URL at any OTHER OpenAI-compatible endpoint,
|
||||
# e.g. a LOCAL ollama at http://localhost:11434/v1, or any embeddings cloud.
|
||||
# AI_EMBED_BASE_URL=http://localhost:11434/v1
|
||||
# AI_EMBED_API_KEY=
|
||||
# AI_EMBED_MODEL=nomic-embed-text
|
||||
# Manual probe recipe (executable by anyone with the keys; never CI):
|
||||
# curl -sS $AI_EMBED_BASE_URL/embeddings \
|
||||
# -H "Authorization: Bearer $AI_EMBED_API_KEY" \
|
||||
# -H 'Content-Type: application/json' \
|
||||
# -d '{"model":"'"$AI_EMBED_MODEL"'","input":["Nextcraft probe"]}' | jq -e '.data[0].embedding | length'
|
||||
|
||||
# --- Billing / Payment Engine (REQ-9-001, D-075, D-077) ---
|
||||
# Payment provider: 'mock' (default — deterministic, zero-network; tests
|
||||
# NEVER call a real payment API) or 'stripe' (real Stripe-compatible REST,
|
||||
# P3+). Unknown names raise at the factory; the boot falls back to mock
|
||||
# with a loud log (G-11).
|
||||
# AI_PAYMENT_PROVIDER=mock
|
||||
# Real provider API key (Stripe-style sk_test_... placeholder — NEVER a
|
||||
# real key here; real keys live in .ciagent/.env.secrets, env-only per
|
||||
# D-014, never logged, SecretStr in Settings). 'mock' ignores it.
|
||||
# AI_PAYMENT_API_KEY=
|
||||
# Webhook HMAC secret (whsec_... style; UNSET → per-boot random → signed
|
||||
# webhooks FAIL CLOSED with 401 until configured — correct for a payment
|
||||
# surface). Set a fixed value only when a provider actually delivers.
|
||||
# AI_WEBHOOK_SECRET=
|
||||
|
||||
@@ -194,6 +194,65 @@ Unprivileged user namespaces are on the box's kernel and need neither a
|
||||
daemon, nor suid helpers, nor network access — they are the only isolation
|
||||
primitive that works here, so that's what v0.3 uses.
|
||||
|
||||
## Telemetry delivery semantics (v0.3, REQ-3-003)
|
||||
|
||||
Delivery is **at-least-once**; storage is **exactly-once** — the two compose:
|
||||
|
||||
- The in-sandbox capture agent (stdlib-only, `scripts/sandbox-agent.py`)
|
||||
spools every event to a durable JSONL file (fsync per append) BEFORE any
|
||||
send attempt, so no event can be lost to a dead socket or a SIGKILL.
|
||||
- The WS ingest endpoint (`WS /v1/telemetry/ingest?learner_id&task_id`,
|
||||
D-026) dedups server-side on the `(learner_id, task_id, seq)` primary key:
|
||||
re-sends (reconnect flushes, replay margin) are collapsed, never upserted.
|
||||
- On disconnect the agent reconnects with exponential backoff and flushes
|
||||
the spool in `seq` order; a transient outage therefore loses nothing and
|
||||
stores each event exactly once (`tests/telemetry/test_durability.py`
|
||||
proves this end-to-end against a real namespace sandbox + live server).
|
||||
- Replay/read path: `GET /v1/telemetry/traces/{learner}/{task}` returns the
|
||||
complete ordered trace; `GET /v1/telemetry/gaps/{learner}/{task}` returns
|
||||
missing seqs for gap detection.
|
||||
- Flood boundary (G-3): a connection exceeding `AI_TELEMETRY_MAX_EVENTS_PER_TASK`
|
||||
(default 50,000) is closed with WS code 1008 and its trace is marked
|
||||
`INCOMPLETE_FLOODED` — a terminal integrity flag the grader refuses to
|
||||
grade. Silent event dropping is forbidden: it would corrupt grading input.
|
||||
|
||||
## Voice defense (v0.3, REQ-3-006)
|
||||
|
||||
Voice is **mock-first** (D-030): the defense pipeline is fully proven over
|
||||
the deterministic `MockVoiceProvider` + browser-native fallback — no task
|
||||
requires a real voice key. Real server STT/TTS (`OpenAIAudioProvider` over
|
||||
OpenAI-compatible `/audio/transcriptions` + `/audio/speech`) is **deferred
|
||||
to v0.4** together with KYC (GRILL CUT-1 / G-7): it could never be exercised
|
||||
in CI, so v0.3 ships the protocol seam instead of an unverifiable claim.
|
||||
|
||||
- `AI_VOICE_PROVIDER=mock` (default) — deterministic canned STT/TTS
|
||||
- `AI_VOICE_PROVIDER=browser` — the web client uses SpeechRecognition +
|
||||
speechSynthesis; the server keeps text-turn persistence
|
||||
- Conversational budget: a defense turn should complete in **< 4s**
|
||||
(`DEFENSE_TURN_BUDGET_MS` in `tests/voice/test_latency.py`). v0.3
|
||||
asserts instrumentation (stt_ms/llm_ms/tts_ms populated per turn); the
|
||||
wall-clock acceptance probe against a real voice endpoint is a v0.4
|
||||
criterion, run manually with `AI_VOICE_PROVIDER` set to the real
|
||||
provider and keys in `.ciagent/.env.secrets` (never in code/commits).
|
||||
|
||||
## End-to-end credential flow (v0.3, REQ-3-007/008)
|
||||
|
||||
`tests/api/test_e2e_credential_flow.py` runs the full pipeline against a REAL
|
||||
uvicorn server with REAL namespace sandboxes (mock LLM/voice per G-2
|
||||
precedent): variant -> telemetry-wired sandbox -> in-sandbox exec -> trace
|
||||
persistence -> process-trace grade (variant seed stamped) -> assessor
|
||||
coaching -> oral defense -> verdict + integrity signals -> proctor. It
|
||||
asserts no corpus fixture appears anywhere in the learner path.
|
||||
|
||||
Manual browser pass (documented, not automated): `pnpm ai:dev` + `pnpm dev`,
|
||||
then open `/build/stack-orchestration-c007` — variant statement + starter
|
||||
files load, edit a file, Run/Test execute in the sandbox with output in the
|
||||
read-only panel, the telemetry status pulses, Lab streams feedback from the
|
||||
live digest; then `/defend/stack-orchestration-c007` — Start Defense, typed
|
||||
answers (mic path needs permission), Finish, Grade My Work renders the real
|
||||
rubric bars. Navigating away destroys the sandbox
|
||||
(`curl localhost:8420/v1/sandboxes` shows the count drop).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
@@ -206,4 +265,4 @@ scripts/ bootstrap.sh dev.sh test.sh lint.sh
|
||||
tests/ pytest — mock provider only
|
||||
```
|
||||
|
||||
Boundary rules: `llm/` imports nothing from `agents/` or `api/`; `agents/` imports nothing from `api/`.
|
||||
Boundary rules: `llm/` imports nothing from `agents/` or `api/`; `agents/` imports nothing from `api/`.
|
||||
|
||||
@@ -1,3 +1,38 @@
|
||||
"""Nextcraft AI tutor service — six LLM agents behind a provider-agnostic layer."""
|
||||
"""Nextcraft AI tutor service — six LLM agents behind a provider-agnostic layer.
|
||||
|
||||
__version__ = "0.2.0"
|
||||
Version single-source (D-061): pyproject.toml is the ONE place the version
|
||||
literal lives. `__version__` (and FastAPI's `version=`) resolve from, in order:
|
||||
|
||||
1. the installed distribution metadata (pip install -e builds it from
|
||||
pyproject at install time), falling back to
|
||||
2. the source tree's pyproject.toml via tomllib (un-installed source runs).
|
||||
|
||||
No fallback literal exists — drift is structurally impossible: if pyproject
|
||||
bumps, both paths follow. Path 2 only degrades when neither metadata nor a
|
||||
pyproject.toml exists (a bare directory copy), reported as "unknown".
|
||||
"""
|
||||
|
||||
import tomllib
|
||||
from importlib.metadata import PackageNotFoundError
|
||||
from importlib.metadata import version as _dist_version
|
||||
from pathlib import Path
|
||||
|
||||
_DIST_NAME = "nextcraft-ai-service"
|
||||
|
||||
|
||||
def _resolve_version() -> str:
|
||||
try:
|
||||
return _dist_version(_DIST_NAME)
|
||||
except PackageNotFoundError:
|
||||
pass
|
||||
# Un-installed source run: read the source tree's pyproject.toml
|
||||
# (ai_service/__init__.py → ../pyproject.toml).
|
||||
pyproject = Path(__file__).resolve().parent.parent / "pyproject.toml"
|
||||
try:
|
||||
data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
|
||||
return str(data["project"]["version"])
|
||||
except (OSError, KeyError, tomllib.TOMLDecodeError):
|
||||
return "unknown"
|
||||
|
||||
|
||||
__version__ = _resolve_version()
|
||||
|
||||
@@ -1,53 +1,34 @@
|
||||
"""AssessorAgent — rubric application to pre-baked artifacts (REQ-2-008).
|
||||
"""AssessorAgent — rubric coaching over REAL grading output (REQ-3-007).
|
||||
|
||||
Structured-output showcase: applies the 4-layer defense (D-020) to return
|
||||
a pydantic-validated rubric score. Mock engine inputs (corpus artifacts +
|
||||
transcripts); real process-trace grading is v0.3+.
|
||||
v0.3 re-grounding: the Assessor no longer invents scores from corpus
|
||||
artifacts — the process-trace grading engine (Phase 3) computes and
|
||||
persists the validated RubricScore. This agent now renders the STORED
|
||||
grade as rubric-anchored coaching: explains the criteria, cites strengths
|
||||
and gaps, and frames next steps. Corpus artifacts are retired from this
|
||||
path (corpus dormancy, Task 6-1-04).
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..corpus.artifacts import (
|
||||
ArtifactSubmission,
|
||||
AssessmentRubric,
|
||||
DefenseTranscript,
|
||||
render_rubric,
|
||||
render_transcript,
|
||||
)
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..grading.store import GradeRecord
|
||||
from ..prompts.assessor import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class CriterionScore(BaseModel):
|
||||
criterion_id: str
|
||||
name: str
|
||||
score: int = Field(ge=0, le=100)
|
||||
evidence: str
|
||||
class GradeCoaching(BaseModel):
|
||||
"""Rubric-anchored coaching rendered FROM the stored grade (not invented)."""
|
||||
|
||||
summary: str = Field(min_length=1)
|
||||
strengths: list[str] = Field(min_length=1, max_length=3)
|
||||
gaps: list[str] = Field(min_length=1, max_length=3)
|
||||
next_steps: list[str] = Field(min_length=1, max_length=3)
|
||||
|
||||
|
||||
class RubricScore(BaseModel):
|
||||
rubric_id: str
|
||||
artifact_id: str
|
||||
competency_id: str
|
||||
scores: list[CriterionScore]
|
||||
strengths: list[str] = Field(min_length=1, max_length=2)
|
||||
gaps: list[str] = Field(min_length=1, max_length=2)
|
||||
verdict: str # "mastered" | "developing" | "not_yet"
|
||||
|
||||
def weighted_total(self, rubric: AssessmentRubric) -> float:
|
||||
by_id = {c.criterion_id: c for c in rubric.criteria}
|
||||
total = 0.0
|
||||
for s in self.scores:
|
||||
total += s.score * by_id[s.criterion_id].weight
|
||||
return total
|
||||
|
||||
|
||||
RUBRIC_SCORE_SCHEMA_HINT = (
|
||||
'{"rubric_id": "<id>", "artifact_id": "<id>", "competency_id": "<id>", '
|
||||
'"scores": [{"criterion_id": "<id>", "name": "<name>", "score": <0-100>, '
|
||||
'"evidence": "<one sentence>"}], "strengths": ["<one sentence>"], '
|
||||
'"gaps": ["<one sentence>"], "verdict": "mastered"|"developing"|"not_yet"}'
|
||||
GRADE_COACHING_SCHEMA_HINT = (
|
||||
'{"summary": "<two sentences on the grade>", '
|
||||
'"strengths": ["<one sentence>"], "gaps": ["<one sentence>"], '
|
||||
'"next_steps": ["<one sentence>"]}'
|
||||
)
|
||||
|
||||
|
||||
@@ -58,35 +39,25 @@ class AssessorAgent(BaseAgent):
|
||||
ctx = learner_context or get_learner_context()
|
||||
return SYSTEM_PROMPT.format_map(render_context(ctx))
|
||||
|
||||
def build_evaluation_input(
|
||||
async def coach_grade(
|
||||
self,
|
||||
artifact: ArtifactSubmission,
|
||||
rubric: AssessmentRubric,
|
||||
transcript: DefenseTranscript | None,
|
||||
) -> str:
|
||||
parts = [
|
||||
f"ARTIFACT: {artifact.name} ({artifact.artifact_id})",
|
||||
f"Evidence excerpt: {artifact.evidence_excerpt}",
|
||||
"",
|
||||
render_rubric(rubric),
|
||||
]
|
||||
if transcript is not None:
|
||||
parts += ["", render_transcript(transcript)]
|
||||
return "\n".join(parts)
|
||||
|
||||
async def evaluate(
|
||||
self,
|
||||
artifact: ArtifactSubmission,
|
||||
rubric: AssessmentRubric,
|
||||
transcript: DefenseTranscript | None,
|
||||
grade: GradeRecord,
|
||||
learner_context: LearnerContext | None = None,
|
||||
) -> RubricScore:
|
||||
evaluation_input = self.build_evaluation_input(artifact, rubric, transcript)
|
||||
score: RubricScore = await self.structured_reply(
|
||||
) -> GradeCoaching:
|
||||
"""Render the STORED grade as coaching via the D-020 defense."""
|
||||
grade_json = {
|
||||
"verdict": grade.verdict,
|
||||
"scores": grade.scores,
|
||||
"digest": grade.digest,
|
||||
}
|
||||
coaching: GradeCoaching = await self.structured_reply(
|
||||
history=None,
|
||||
user_input=evaluation_input,
|
||||
user_input=(
|
||||
"The learner's process-trace grade (computed by the grading "
|
||||
f"engine) is:\n{grade_json!r}\nExplain it as coaching."
|
||||
),
|
||||
learner_context=learner_context,
|
||||
schema=RubricScore,
|
||||
schema_hint=RUBRIC_SCORE_SCHEMA_HINT,
|
||||
schema=GradeCoaching,
|
||||
schema_hint=GRADE_COACHING_SCHEMA_HINT,
|
||||
)
|
||||
return score
|
||||
return coaching
|
||||
|
||||
@@ -6,6 +6,7 @@ The default pipeline: build_messages() → provider.stream_chat()/chat().
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import TypeVar
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -15,6 +16,8 @@ from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
from .structured import structured_completion
|
||||
|
||||
T = TypeVar("T", bound=BaseModel)
|
||||
|
||||
|
||||
class BaseAgent(ABC):
|
||||
"""A tutor agent: system prompt + message assembly + provider delegation."""
|
||||
@@ -64,10 +67,15 @@ class BaseAgent(ABC):
|
||||
history: list[Message] | None = None,
|
||||
user_input: str = "",
|
||||
learner_context: LearnerContext | None = None,
|
||||
schema: type[BaseModel] | None = None,
|
||||
schema: type[T] | None = None,
|
||||
schema_hint: str = "",
|
||||
) -> BaseModel:
|
||||
"""Non-streaming completion parsed into a pydantic model (D-020 defense)."""
|
||||
) -> T:
|
||||
"""Non-streaming completion parsed into a pydantic model (D-020 defense).
|
||||
|
||||
P5: generic over `schema` — the returned model is the schema's own
|
||||
type, so callers (assessor/proctor) get concrete types, no casts.
|
||||
`schema=None` raises before T is ever produced.
|
||||
"""
|
||||
if schema is None:
|
||||
raise ValueError("structured_reply requires a schema")
|
||||
messages = self.build_messages(history, user_input, learner_context)
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
"""ExaminerAgent — the seventh agent: oral-defense examiner (REQ-3-006, A-109).
|
||||
|
||||
BOUNDARY DECISION (PERSONAS conflict rule, honored by construction): the
|
||||
examiner is a TEXT agent. It composes the LLM provider through BaseAgent and
|
||||
consumes defense transcript turns; it NEVER imports voice/ — STT/TTS belong
|
||||
to the API endpoints (they move audio bytes; the agent moves question text).
|
||||
Integrity signals (long pauses, off-scope cadence) are computed by the
|
||||
endpoint layer from turn metadata (latency_ms etc.), not by the agent.
|
||||
|
||||
Digest discipline (D-028 mirror): questions are grounded in the compact
|
||||
TraceDigest + variant statement — never the raw trace, never learner ids.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from ..grading.features import TraceDigest
|
||||
from ..llm.types import Message
|
||||
from ..prompts.examiner import SYSTEM_PROMPT, VERDICT_SCHEMA_HINT, render_digest_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class DefenseVerdict(BaseModel):
|
||||
"""D-20-validated final defense verdict (structured mode)."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
verdict: str = Field(pattern="^(mastered|developing|not_yet)$")
|
||||
understanding: str = Field(min_length=1)
|
||||
process_justification: str = Field(min_length=1)
|
||||
communication: str = Field(min_length=1)
|
||||
strengths: list[str] = Field(min_length=1, max_length=2)
|
||||
gaps: list[str] = Field(min_length=1, max_length=2)
|
||||
|
||||
|
||||
class ExaminerAgent(BaseAgent):
|
||||
"""Conducts the oral defense: next_question + final_verdict."""
|
||||
|
||||
name = "examiner"
|
||||
|
||||
def system_prompt(self, learner_context=None) -> str:
|
||||
"""Examiner is context-free (digest-anonymous, D-028 mirror)."""
|
||||
return SYSTEM_PROMPT
|
||||
|
||||
def build_defense_messages(
|
||||
self,
|
||||
trace_digest: TraceDigest | None = None,
|
||||
variant_statement: str | None = None,
|
||||
history: list[Message] | None = None,
|
||||
) -> list[Message]:
|
||||
"""System + grounding + defense transcript (no learner id — D-028)."""
|
||||
digest_json = (
|
||||
trace_digest.model_dump_json() if trace_digest is not None else "{}"
|
||||
)
|
||||
messages: list[Message] = [
|
||||
Message(role="system", content=SYSTEM_PROMPT),
|
||||
Message(role="user", content=render_digest_context(digest_json, variant_statement)),
|
||||
Message(
|
||||
role="assistant",
|
||||
content="Understood. I will question the learner about this build session.",
|
||||
),
|
||||
]
|
||||
for m in history or []:
|
||||
messages.append(m)
|
||||
return messages
|
||||
|
||||
async def next_question(
|
||||
self,
|
||||
history: list[Message],
|
||||
trace_digest: TraceDigest | None = None,
|
||||
variant_statement: str | None = None,
|
||||
) -> str:
|
||||
"""One examiner question (streamed over SSE by the endpoints)."""
|
||||
messages = self.build_defense_messages(trace_digest, variant_statement, history)
|
||||
messages.append(
|
||||
Message(role="user", content="Ask the learner your next question now.")
|
||||
)
|
||||
reply = await self.provider.chat(messages, model=self.settings.model)
|
||||
return reply
|
||||
|
||||
async def final_verdict(
|
||||
self,
|
||||
history: list[Message],
|
||||
trace_digest: TraceDigest | None = None,
|
||||
variant_statement: str | None = None,
|
||||
) -> DefenseVerdict:
|
||||
"""Structured verdict via the D-020 4-layer defense."""
|
||||
from .structured import structured_completion # module-direct (G-4)
|
||||
|
||||
messages = self.build_defense_messages(trace_digest, variant_statement, history)
|
||||
messages.append(
|
||||
Message(
|
||||
role="user",
|
||||
content="The defense is finished. Return the final verdict JSON now.",
|
||||
)
|
||||
)
|
||||
return await structured_completion(
|
||||
self.provider,
|
||||
messages,
|
||||
model=self.settings.model,
|
||||
schema=DefenseVerdict,
|
||||
schema_hint=VERDICT_SCHEMA_HINT,
|
||||
)
|
||||
@@ -1,17 +1,18 @@
|
||||
"""LabAgent — in-flow feedback over simulated sandbox telemetry (REQ-2-007).
|
||||
"""LabAgent — in-flow feedback over LIVE sandbox telemetry (REQ-3-007).
|
||||
|
||||
Scenario-driven: consumes a LabTelemetryScenario from the corpus, renders
|
||||
the event timeline into the conversation, streams concrete feedback.
|
||||
No session chat — each request is one scenario read.
|
||||
v0.3 re-grounding: consumes a TraceDigest computed from the learner's real
|
||||
trace (grading/features.compute_digest over TraceStore events) — the v0.2
|
||||
corpus scenarios are retired from this path (corpus dormancy, Task 6-1-04).
|
||||
No session chat — each request is one live-trace read.
|
||||
"""
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..corpus.telemetry import LabTelemetryScenario, summarize_scenario
|
||||
from ..grading.features import TraceDigest
|
||||
from ..llm.base import LLMProvider
|
||||
from ..prompts.lab import SYSTEM_PROMPT, render_context
|
||||
from ..prompts.lab import SYSTEM_PROMPT, render_context, render_digest_timeline
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
@@ -27,10 +28,11 @@ class LabAgent(BaseAgent):
|
||||
|
||||
async def stream_feedback(
|
||||
self,
|
||||
scenario: LabTelemetryScenario,
|
||||
digest: TraceDigest | None,
|
||||
learner_context: LearnerContext | None = None,
|
||||
) -> AsyncIterator[str]:
|
||||
timeline = summarize_scenario(scenario)
|
||||
"""Feedback grounded in the learner's live trace digest."""
|
||||
timeline = render_digest_timeline(digest)
|
||||
async for token in self.stream_reply(
|
||||
history=None, user_input=timeline, learner_context=learner_context
|
||||
):
|
||||
|
||||
@@ -1,33 +1,41 @@
|
||||
"""ProctorAgent — integrity signals + coaching interventions (REQ-2-009).
|
||||
"""ProctorAgent — integrity signals + coaching over REAL inputs (REQ-3-007).
|
||||
|
||||
Consumes a ProctorScenario from the corpus, returns pydantic-validated
|
||||
signal classifications via structured_reply (4-layer defense).
|
||||
Mock engine inputs; real identity/attention signals are v0.3+.
|
||||
v0.3 re-grounding: consumes the learner's live trace digest (idle gaps,
|
||||
command cadence), the DefenseStore integrity signals (long pauses from the
|
||||
oral defense), and the variant seed cross-check — NOT v0.2 corpus
|
||||
scenarios. The proctor COACHES: it classifies signals supportively and
|
||||
recommends one intervention; it never punishes and never accuses.
|
||||
|
||||
Integrity inputs (computed server-side, passed in by the API layer):
|
||||
- trace digest: idle_gap_count/total, command_categories histogram,
|
||||
error/fix cycles, huge-burst indicators (edit_count vs test runs)
|
||||
- defense signals: long_pauses list from the finished defense (A-109)
|
||||
- variant: seed + params when the task is variant-derived (off-template
|
||||
work is a cross-check input, not an accusation)
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..corpus.telemetry import ProctorScenario, summarize_proctor_scenario
|
||||
from ..grading.features import TraceDigest
|
||||
from ..prompts.proctor import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class IntegritySignal(BaseModel):
|
||||
signal_type: str # e.g. "context_switch" | "idle_gap" | "large_paste"
|
||||
signal_type: str # "idle_gap" | "long_pause" | "burst_edit" | "off_template"
|
||||
severity: str # "low" | "medium" | "high"
|
||||
note: str
|
||||
|
||||
|
||||
class ProctorAssessment(BaseModel):
|
||||
scenario_id: str
|
||||
signals: list[IntegritySignal] = Field(min_length=0)
|
||||
intervention: str # ONE supportive coaching recommendation
|
||||
summary: str
|
||||
|
||||
|
||||
PROCTOR_ASSESSMENT_SCHEMA_HINT = (
|
||||
'{"scenario_id": "<id>", "signals": [{"signal_type": "<type>", '
|
||||
'{"signals": [{"signal_type": "<type>", '
|
||||
'"severity": "low"|"medium"|"high", "note": "<one sentence>"}], '
|
||||
'"intervention": "<one supportive recommendation>", '
|
||||
'"summary": "<one sentence>"}'
|
||||
@@ -43,13 +51,27 @@ class ProctorAgent(BaseAgent):
|
||||
|
||||
async def assess(
|
||||
self,
|
||||
scenario: ProctorScenario,
|
||||
digest: TraceDigest | None,
|
||||
defense_signals: dict | None = None,
|
||||
variant_context: dict | None = None,
|
||||
learner_context: LearnerContext | None = None,
|
||||
) -> ProctorAssessment:
|
||||
timeline = summarize_proctor_scenario(scenario)
|
||||
"""Classify REAL integrity inputs into supportive signals + coaching."""
|
||||
parts: list[str] = []
|
||||
if digest is not None:
|
||||
parts.append(f"Build-session digest:\n{digest.model_dump_json()}")
|
||||
else:
|
||||
parts.append("No build telemetry recorded for this task yet.")
|
||||
if defense_signals:
|
||||
parts.append(f"Oral-defense integrity signals:\n{defense_signals}")
|
||||
if variant_context:
|
||||
parts.append(f"Variant audit context (seed + params):\n{variant_context}")
|
||||
assessment: ProctorAssessment = await self.structured_reply(
|
||||
history=None,
|
||||
user_input=timeline,
|
||||
user_input=(
|
||||
"Assess this learner's integrity signals supportively.\n\n"
|
||||
+ "\n\n".join(parts)
|
||||
),
|
||||
learner_context=learner_context,
|
||||
schema=ProctorAssessment,
|
||||
schema_hint=PROCTOR_ASSESSMENT_SCHEMA_HINT,
|
||||
|
||||
@@ -14,13 +14,14 @@ AgentFactory = Callable[[LLMProvider, Settings], BaseAgent]
|
||||
|
||||
|
||||
def register_builtin_agents(registry: "AgentRegistry") -> None:
|
||||
"""Central registration of all six shipped tutor agents (G-4: one pattern).
|
||||
"""Central registration of all seven shipped agents (G-4: one pattern).
|
||||
|
||||
coach, tutor, lab, assessor, proctor, mentor. New agents register here
|
||||
in their landing phase.
|
||||
coach, tutor, lab, assessor, proctor, mentor, examiner (Phase 5).
|
||||
New agents register here in their landing phase.
|
||||
"""
|
||||
from .assessor import AssessorAgent
|
||||
from .coach import CoachAgent
|
||||
from .examiner import ExaminerAgent
|
||||
from .lab import LabAgent
|
||||
from .mentor import MentorAgent
|
||||
from .proctor import ProctorAgent
|
||||
@@ -38,6 +39,9 @@ def register_builtin_agents(registry: "AgentRegistry") -> None:
|
||||
registry.register(
|
||||
"mentor", lambda provider, settings: MentorAgent(provider, settings)
|
||||
)
|
||||
registry.register(
|
||||
"examiner", lambda provider, settings: ExaminerAgent(provider, settings)
|
||||
)
|
||||
|
||||
|
||||
class UnknownAgentError(KeyError):
|
||||
|
||||
@@ -28,15 +28,19 @@ class AgentSession:
|
||||
|
||||
|
||||
class SessionStore(Protocol):
|
||||
def get(self, session_id: str) -> AgentSession | None: ...
|
||||
def create(
|
||||
# P5 (D-058): the protocol was previously declared sync while the
|
||||
# shipped implementation is async — every API call site awaited a
|
||||
# non-awaitable per the old signature. The protocol now tells the
|
||||
# truth: all five methods are async (InMemorySessionStore's shape).
|
||||
async def get(self, session_id: str) -> AgentSession | None: ...
|
||||
async def create(
|
||||
self, session_id: str, agent: str, learner_id: str = "seed-learner-1"
|
||||
) -> AgentSession: ...
|
||||
def append(self, session_id: str, message: Message) -> None: ...
|
||||
def history_window(
|
||||
async def append(self, session_id: str, message: Message) -> None: ...
|
||||
async def history_window(
|
||||
self, session_id: str, max_messages: int = DEFAULT_WINDOW
|
||||
) -> list[Message]: ...
|
||||
def delete(self, session_id: str) -> None: ...
|
||||
async def delete(self, session_id: str) -> None: ...
|
||||
|
||||
|
||||
class InMemorySessionStore:
|
||||
|
||||
@@ -5,10 +5,13 @@ Boundary rule: api/ composes agents/ and llm/; they never import api/.
|
||||
|
||||
from .assessment import router as assessment_router
|
||||
from .chat import router as chat_router
|
||||
from .defense import router as defense_router
|
||||
from .lab import router as lab_router
|
||||
from .mentor import router as mentor_router
|
||||
from .proctor import router as proctor_router
|
||||
from .sandboxes import router as sandboxes_router
|
||||
from .telemetry import router as telemetry_router
|
||||
from .variants import router as variants_router
|
||||
|
||||
__all__ = [
|
||||
"assessment_router",
|
||||
@@ -17,4 +20,7 @@ __all__ = [
|
||||
"mentor_router",
|
||||
"proctor_router",
|
||||
"sandboxes_router",
|
||||
"telemetry_router",
|
||||
"variants_router",
|
||||
"defense_router",
|
||||
]
|
||||
|
||||
@@ -1,47 +1,193 @@
|
||||
"""POST /v1/assessment/evaluate — structured rubric scores (REQ-2-008).
|
||||
"""/v1/assessment — rubric evaluation + trace grading endpoints (REQ-2-008, REQ-3-004).
|
||||
|
||||
JSON response (not SSE): a pydantic-validated RubricScore. Unknown
|
||||
artifact → 404. The Assessor's structured output IS the payload.
|
||||
Two endpoint families share this router:
|
||||
|
||||
POST /v1/assessment/evaluate (v0.2, REQ-2-008) — corpus
|
||||
artifact evaluation through
|
||||
the Assessor agent.
|
||||
POST /v1/assessment/grade (v0.3, REQ-3-004) — grade a
|
||||
REAL process trace through
|
||||
the GradingEngine.
|
||||
GET /v1/assessment/grade/{learner_id}/{task_id} — stored latest grade.
|
||||
|
||||
Grading status-code mapping (the engine's outcomes are CONTRACT, not errors):
|
||||
|
||||
GradeRecord(verdict=GRADED) → 200 — rubric scores +
|
||||
verdict (in scores.verdict)
|
||||
+ digest summary.
|
||||
GradeRecord(UNGRADABLE_TRACE_INCOMPLETE) → 200 — the ungradable
|
||||
record IS a valid result:
|
||||
the trace cannot be graded,
|
||||
and the gate surfaces WHY
|
||||
(scores.missing_seqs +
|
||||
scores.integrity_flag).
|
||||
Persisted like any grade.
|
||||
GradeRecord(UNGRADABLE_EMPTY_TRACE) → 200 — no events stored for
|
||||
the pair. This covers BOTH
|
||||
a known pair whose trace
|
||||
ended up empty AND a task
|
||||
that never had a trace at
|
||||
all: the engine cannot
|
||||
distinguish them (zero
|
||||
stored events is zero
|
||||
events), and grading an
|
||||
absent trace genuinely has
|
||||
the empty-trace outcome —
|
||||
a 404 here would erase the
|
||||
durable gate record the
|
||||
engine persists for the
|
||||
pair. PLAN's 404 applies to
|
||||
GET of a never-graded pair.
|
||||
StructuredOutputError → 502 — the trace was
|
||||
gradable but the provider
|
||||
failed the D-020 budget;
|
||||
provider failure (bad
|
||||
gateway to the model), same
|
||||
mapping as evaluate.
|
||||
GET of an unknown (never-graded) pair → 404.
|
||||
|
||||
DI (D-027/D-032 house pattern): engine + store arrive via deps.get_grading_engine
|
||||
/ get_grade_store from app.state; this module owns all FastAPI wiring — the
|
||||
engine knows nothing of HTTP. UNGRADABLE_* bodies are rendered by the same
|
||||
GradeResponse model as GRADED ones (a gate record's `scores` holds the gate
|
||||
detail instead of rubric scores), so consumers read ONE shape.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, cast
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..agents.assessor import RubricScore
|
||||
from ..agents.assessor import AssessorAgent
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..agents.structured import StructuredOutputError
|
||||
from ..config import Settings
|
||||
from ..corpus.artifacts import get_artifact_bundle, get_transcript_for_artifact
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from .deps import get_agent_registry, get_provider, get_settings
|
||||
from ..grading.engine import GradingEngine
|
||||
from ..grading.store import GradeRecord, GradeStore
|
||||
from .deps import (
|
||||
get_agent_registry,
|
||||
get_grade_store,
|
||||
get_grading_engine,
|
||||
get_provider,
|
||||
get_settings,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class AssessmentRequest(BaseModel):
|
||||
artifact_id: str = Field(min_length=1)
|
||||
learner_id: str | None = None
|
||||
# --- v0.2 artifact evaluation (REQ-2-008) --------------------------------------
|
||||
|
||||
|
||||
@router.post("/assessment/evaluate", response_model=RubricScore)
|
||||
class EvaluateRequest(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
task_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
@router.post("/assessment/evaluate")
|
||||
async def assessment_evaluate(
|
||||
body: AssessmentRequest,
|
||||
body: EvaluateRequest,
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
) -> RubricScore:
|
||||
bundle = get_artifact_bundle(body.artifact_id)
|
||||
if bundle is None:
|
||||
grade_store=Depends(get_grade_store),
|
||||
) -> dict:
|
||||
"""Assessor coaching rendered FROM the learner's stored grade (REQ-3-007).
|
||||
|
||||
The grading engine computes the scores (POST /assessment/grade); this
|
||||
endpoint explains them. No stored grade yet -> 404 (grade first).
|
||||
"""
|
||||
grade = grade_store.get(body.learner_id, body.task_id)
|
||||
if grade is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown artifact {body.artifact_id!r}"
|
||||
status_code=404,
|
||||
detail=f"no stored grade for {body.learner_id}/{body.task_id} - grade first",
|
||||
)
|
||||
artifact, rubric = bundle
|
||||
transcript = get_transcript_for_artifact(artifact.artifact_id)
|
||||
agent = registry.get(provider, settings, "assessor")
|
||||
# P5: registry.get returns BaseAgent; "assessor" is registered as
|
||||
# AssessorAgent (register_builtin_agents) — the cast states that mapping.
|
||||
agent = cast(AssessorAgent, registry.get(provider, settings, "assessor"))
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
try:
|
||||
return await agent.evaluate(artifact, rubric, transcript, learner_context)
|
||||
coaching = await agent.coach_grade(grade, learner_context)
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=502,
|
||||
detail=f"assessment evaluation failed: {exc}",
|
||||
) from exc
|
||||
return {
|
||||
"learner_id": grade.learner_id,
|
||||
"task_id": grade.task_id,
|
||||
"grade_verdict": grade.verdict,
|
||||
"grade_scores": grade.scores,
|
||||
"coaching": coaching.model_dump(),
|
||||
}
|
||||
|
||||
|
||||
# --- # --- v0.3 trace grading (REQ-3-004) ---------------------------------------------
|
||||
|
||||
|
||||
class GradeRequest(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
task_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class GradeResponse(BaseModel):
|
||||
"""GradeRecord over HTTP — one shape for GRADED and UNGRADABLE_* alike.
|
||||
|
||||
`scores` holds the validated rubric (criteria 0-4, strengths, gaps,
|
||||
rubric verdict) for a GRADED record, or the gate detail
|
||||
({integrity_flag, missing_seqs}) for an UNGRADABLE_* record — never both.
|
||||
`digest` is the compact trace summary that fed the rubric prompt (empty
|
||||
for gate records: nothing was graded).
|
||||
"""
|
||||
|
||||
learner_id: str
|
||||
task_id: str
|
||||
variant_seed: str | None
|
||||
digest: dict[str, Any]
|
||||
scores: dict[str, Any]
|
||||
verdict: str
|
||||
model: str
|
||||
created_at: datetime
|
||||
|
||||
|
||||
def _grade_response(record: GradeRecord) -> GradeResponse:
|
||||
return GradeResponse.model_validate(record, from_attributes=True)
|
||||
|
||||
|
||||
@router.post("/assessment/grade", response_model=GradeResponse)
|
||||
async def assessment_grade(
|
||||
body: GradeRequest,
|
||||
engine: GradingEngine = Depends(get_grading_engine),
|
||||
) -> GradeResponse:
|
||||
"""Run the grading engine for one (learner_id, task_id) trace.
|
||||
|
||||
Gate outcomes (UNGRADABLE_*) are 200s — they are first-class results the
|
||||
engine persists, not failures. Only a provider that exhausts the D-020
|
||||
budget turns into a 502; nothing is persisted on that path.
|
||||
"""
|
||||
try:
|
||||
record = await engine.grade(body.learner_id, body.task_id)
|
||||
except StructuredOutputError as exc:
|
||||
raise HTTPException(
|
||||
status_code=502,
|
||||
detail=f"grading failed: {exc}",
|
||||
) from exc
|
||||
return _grade_response(record)
|
||||
|
||||
|
||||
@router.get("/assessment/grade/{learner_id}/{task_id}", response_model=GradeResponse)
|
||||
async def assessment_get_grade(
|
||||
learner_id: str,
|
||||
task_id: str,
|
||||
store: GradeStore = Depends(get_grade_store),
|
||||
) -> GradeResponse:
|
||||
"""Latest stored grade for the pair; 404 when none was ever stored."""
|
||||
record = store.get(learner_id, task_id)
|
||||
if record is None:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"no stored grade for {learner_id!r}/{task_id!r}",
|
||||
)
|
||||
return _grade_response(record)
|
||||
|
||||
@@ -0,0 +1,345 @@
|
||||
"""/v1/billing — billing API (REQ-9-002/9-003, D-078/D-079/G-302).
|
||||
|
||||
P2: the public catalog read (advertising — no auth).
|
||||
P3: the public webhook receiver. Webhook discipline (risk-1/G-302):
|
||||
`await request.body()` FIRST — the exact un-parsed bytes verify the
|
||||
HMAC before ANY parsing; unsigned/bad signatures 401, never process.
|
||||
Event payloads are UNTRUSTED ID REFERENCES — session/subscription
|
||||
state resolves from LOCAL rows; unknown targets land as `unmatched`
|
||||
billing events (200, no entitlement). Provider event ids dedupe
|
||||
verbatim (at-least-once tolerance, R-45); grants are idempotent per
|
||||
SESSION (G-303 — the store's UniqueConstraints enforce exactly-once).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
from pydantic import BaseModel
|
||||
|
||||
from ..billing.base import Package, Plan, WebhookSignatureError
|
||||
from ..billing.store import BillingStore
|
||||
from .deps import get_billing_store
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/v1/billing", tags=["billing"])
|
||||
|
||||
|
||||
class CatalogResponse(BaseModel):
|
||||
"""The catalog wire shape (REQ-9-002; pinned field-for-field by
|
||||
packages/types/billing.ts via apps/web/tests/pins.test.ts — G-108
|
||||
discipline)."""
|
||||
|
||||
packages: list[Package]
|
||||
plans: list[Plan]
|
||||
|
||||
|
||||
@router.get("/catalog", response_model=CatalogResponse)
|
||||
def get_catalog(
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
) -> CatalogResponse:
|
||||
"""The seeded catalog, price-ascending (D-078 — one source of
|
||||
truth; idempotent seeding happened at app boot)."""
|
||||
packages, plans = store.list_catalog()
|
||||
return CatalogResponse(packages=packages, plans=plans)
|
||||
|
||||
|
||||
def _get_payment_provider(request: Request):
|
||||
"""The selected provider via app.state DI (main.py wires the
|
||||
factory + G-11 fallback)."""
|
||||
return request.app.state.payment_provider
|
||||
|
||||
|
||||
# -- webhook (Task 3-2-02, G-302) ----------------------------------------------
|
||||
|
||||
|
||||
async def process_webhook_event(
|
||||
*,
|
||||
store: BillingStore,
|
||||
provider: object,
|
||||
raw_body: bytes,
|
||||
signature_header: str,
|
||||
) -> dict[str, object]:
|
||||
"""The REAL webhook processing path (G-301's E2E proof runs through
|
||||
here; the HTTP route + the mock-complete route share it).
|
||||
|
||||
Order (risk-1): verify HMAC over the exact bytes FIRST, then parse,
|
||||
then dedupe, then process. 200 fast on every authenticated path —
|
||||
duplicates and unknown targets are recorded, never errors (R-45's
|
||||
at-least-once + G-306's unknown-target tolerance).
|
||||
|
||||
Crash-safety (final-review fix): the event id is recorded AFTER
|
||||
its effects commit. The old order (record id → process) made a
|
||||
crash between the two permanent: every retry deduped as
|
||||
"duplicate" while the grant never happened — the entitlement was
|
||||
lost forever. The new order keeps a bounded replay window: a
|
||||
replay re-runs processing (a no-op for grants — G-303's per-session
|
||||
UniqueConstraint is the exactly-once backstop) and re-records the
|
||||
audit rows for that one event only until its id lands."""
|
||||
from ..billing.base import PaymentProvider
|
||||
|
||||
assert isinstance(provider, PaymentProvider)
|
||||
try:
|
||||
verified = provider.verify_webhook(raw_body, signature_header)
|
||||
except WebhookSignatureError as exc:
|
||||
logger.warning("billing webhook rejected: %s", exc)
|
||||
raise HTTPException(status_code=401, detail=str(exc)) from exc
|
||||
except ValueError as exc:
|
||||
# G-306 (never 500): a correctly-SIGNED event whose type the
|
||||
# provider cannot normalize onto our canonical vocabulary
|
||||
# (Stripe's real streams deliver many event types) is honest
|
||||
# unmatched audit material, never a crashed receiver — Stripe
|
||||
# disables endpoints on error storms. The transport was
|
||||
# AUTHENTICATED (compare_digest passed) before the parse, so the
|
||||
# failure is name-level, not trust-level. WebhookSignatureError
|
||||
# is a ValueError subclass — the specific except above wins for
|
||||
# actual signature failures (401), this arm only sees
|
||||
# normalize_event's unmapped-name raise.
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
"unmapped.event",
|
||||
detail={"reason": f"unmapped provider event: {exc}"},
|
||||
)
|
||||
return {"status": "unmatched", "processed": False}
|
||||
event = verified.event
|
||||
# Dedupe VERBATIM on the provider event id (R-43/D-083) — checked
|
||||
# BEFORE the effects (fast-path a known replay), recorded AFTER
|
||||
# they commit (the crash-safety half of the final-review fix). A
|
||||
# crash between the two leaves the id UN-recorded, so the provider's
|
||||
# retry re-runs the effects — a no-op for grants (G-303's
|
||||
# per-session UniqueConstraint is the exactly-once backstop) — and
|
||||
# lands the key then. The old order (record → process) made that
|
||||
# crash permanent: every retry answered "duplicate" while the grant
|
||||
# never happened.
|
||||
if not store.has_provider_event(verified.provider_event_id):
|
||||
result = _process_canonical_event(store, event.name, dict(event.payload))
|
||||
store.insert_provider_event(
|
||||
verified.provider_event_id, event.name, dict(event.payload)
|
||||
)
|
||||
return {"status": "ok", "processed": result}
|
||||
return {"status": "duplicate", "processed": False}
|
||||
|
||||
|
||||
def _process_canonical_event(
|
||||
store: BillingStore, name: str, payload: dict[str, object]
|
||||
) -> str:
|
||||
"""G-302: the payload is UNTRUSTED — id references only. State
|
||||
resolves from LOCAL rows; unknown targets land as unmatched audit
|
||||
rows (never a 500, G-306)."""
|
||||
if name == "checkout.completed":
|
||||
session_id = str(payload.get("session_id", ""))
|
||||
local = store.get_checkout_session_by_provider_id(session_id)
|
||||
if local is None:
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
name,
|
||||
detail={"reason": "unknown session", "session_id": session_id},
|
||||
)
|
||||
return "unmatched"
|
||||
# R-37 predicate: complete AND paid — NEVER status alone (async
|
||||
# methods can leave a session complete with payment still
|
||||
# processing). Final-review fix: the payload's payment_status
|
||||
# (attacker-immutable: it is inside the HMAC) gates the flip +
|
||||
# the grant; an unpaid/processing completion is recorded, not
|
||||
# granted, and a later paid delivery grants.
|
||||
payment_status = str(payload.get("payment_status", "unpaid"))
|
||||
if payment_status not in ("paid", "no_payment_required"):
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
name,
|
||||
detail={
|
||||
"reason": f"payment_status {payment_status!r} not paid",
|
||||
"session_id": local.id,
|
||||
},
|
||||
)
|
||||
return "processing"
|
||||
# Status resolves from the LOCAL row's snapshot (the route flips
|
||||
# it on completion), never the payload's other fields.
|
||||
store.complete_checkout_session(local.id, payment_status=payment_status)
|
||||
_grant_entitlement(store, local)
|
||||
store.insert_billing_event(
|
||||
"provider",
|
||||
name,
|
||||
employer_id=local.employer_id,
|
||||
detail={"session_id": local.id},
|
||||
)
|
||||
return "granted"
|
||||
if name == "invoice.paid":
|
||||
subscription_ref = str(payload.get("subscription_id", ""))
|
||||
extended = store.extend_subscription_period(subscription_ref, days=30)
|
||||
if extended is None:
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
name,
|
||||
detail={
|
||||
"reason": "unknown subscription",
|
||||
"subscription_id": subscription_ref,
|
||||
},
|
||||
)
|
||||
return "unmatched"
|
||||
store.insert_billing_event(
|
||||
"provider", name, employer_id=extended.employer_id
|
||||
)
|
||||
return "renewed"
|
||||
if name == "invoice.payment_failed":
|
||||
subscription_ref = str(payload.get("subscription_id", ""))
|
||||
subscription = store.get_subscription_by_provider_id(subscription_ref)
|
||||
if subscription is None:
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
name,
|
||||
detail={
|
||||
"reason": "unknown subscription",
|
||||
"subscription_id": subscription_ref,
|
||||
},
|
||||
)
|
||||
return "unmatched"
|
||||
try:
|
||||
store.transition_subscription(subscription.id, "past_due")
|
||||
except Exception:
|
||||
# Already reflected (out-of-order tolerance, G-306) — record.
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
name,
|
||||
detail={"reason": "already past_due", "subscription_id": subscription_ref},
|
||||
)
|
||||
return "already"
|
||||
store.insert_billing_event(
|
||||
"provider", name, employer_id=subscription.employer_id
|
||||
)
|
||||
return "past_due"
|
||||
if name == "subscription.cancelled":
|
||||
subscription_ref = str(payload.get("subscription_id", ""))
|
||||
subscription = store.get_subscription_by_provider_id(subscription_ref)
|
||||
if subscription is None:
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
name,
|
||||
detail={
|
||||
"reason": "unknown subscription",
|
||||
"subscription_id": subscription_ref,
|
||||
},
|
||||
)
|
||||
return "unmatched"
|
||||
try:
|
||||
store.transition_subscription(subscription.id, "cancelled")
|
||||
except Exception:
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
name,
|
||||
detail={"reason": "already cancelled", "subscription_id": subscription_ref},
|
||||
)
|
||||
return "already"
|
||||
store.insert_billing_event(
|
||||
"provider", name, employer_id=subscription.employer_id
|
||||
)
|
||||
return "cancelled"
|
||||
if name == "subscription.renewed":
|
||||
# R-43: renewal = invoice.paid on a recurring subscription; the
|
||||
# corroborating customer.subscription.updated is recorded here.
|
||||
return _process_canonical_event(store, "invoice.paid", payload)
|
||||
store.insert_billing_event(
|
||||
"unmatched", name, detail={"reason": "unhandled canonical name"}
|
||||
)
|
||||
return "unhandled"
|
||||
|
||||
|
||||
def _grant_entitlement(store: BillingStore, session_row: object) -> str:
|
||||
"""Create the purchase or subscription from the LOCAL snapshot
|
||||
(D-078 prices; G-303 exactly-once per session — the store's
|
||||
UniqueConstraint refuses a second grant; the refusal is the
|
||||
already-granted answer, recorded not raised)."""
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
from ..billing.store import CheckoutSessionRow
|
||||
|
||||
assert isinstance(session_row, CheckoutSessionRow)
|
||||
if session_row.mode == "payment":
|
||||
package, _, _item = store.get_catalog_item(session_row.item_id)
|
||||
if package is not None:
|
||||
try:
|
||||
store.create_purchase_from_session(
|
||||
session_row,
|
||||
posts_limit=package.posts_limit,
|
||||
duration_days=package.duration_days,
|
||||
)
|
||||
return "granted"
|
||||
except IntegrityError:
|
||||
# G-303: a distinct event id announced the SAME
|
||||
# completion — the grant already happened; the 200 is
|
||||
# the honest answer (never a 500, R-45).
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
"checkout.completed",
|
||||
employer_id=session_row.employer_id,
|
||||
detail={
|
||||
"reason": "already granted",
|
||||
"session_id": session_row.id,
|
||||
},
|
||||
)
|
||||
return "already"
|
||||
# Final-review fix: the item vanished from the catalog AFTER the
|
||||
# checkout row snapshotted it (an operator deleted a seed id) —
|
||||
# record the anomaly; the caller's "granted"-shaped answer must
|
||||
# never silently mask a missing grant (the audit log answers
|
||||
# "why does this employer have no entitlement?").
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
"checkout.completed",
|
||||
employer_id=session_row.employer_id,
|
||||
detail={
|
||||
"reason": "catalog item missing at grant time",
|
||||
"item_id": session_row.item_id,
|
||||
"session_id": session_row.id,
|
||||
},
|
||||
)
|
||||
return "no-item"
|
||||
_, plan, _item = store.get_catalog_item(session_row.item_id)
|
||||
if plan is not None:
|
||||
# Upsert keyed on the provider ref — replay-safe by construction.
|
||||
# Final-review fix: the ref is the mock provider's DOCUMENTED
|
||||
# convention (`sub_mock_{session_id without its cs_ prefix}`,
|
||||
# mock.py confirm_completion) — the old `sub_prov_…` spelling
|
||||
# made the store row's provider_subscription_id a key the
|
||||
# provider never knew, so every provider lifecycle call
|
||||
# (cancel/update) resolved None and desynced from local state.
|
||||
store.create_subscription_from_session(
|
||||
session_row,
|
||||
provider_subscription_id=(
|
||||
f"sub_mock_{session_row.provider_session_id.removeprefix('cs_')}"
|
||||
),
|
||||
interval=plan.interval,
|
||||
)
|
||||
return "granted"
|
||||
store.insert_billing_event(
|
||||
"unmatched",
|
||||
"checkout.completed",
|
||||
employer_id=session_row.employer_id,
|
||||
detail={
|
||||
"reason": "catalog item missing at grant time",
|
||||
"item_id": session_row.item_id,
|
||||
"session_id": session_row.id,
|
||||
},
|
||||
)
|
||||
return "no-item"
|
||||
|
||||
|
||||
@router.post("/webhook")
|
||||
async def billing_webhook(
|
||||
request: Request,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
) -> Response:
|
||||
"""Public provider webhook receiver (server-to-server; CORS
|
||||
irrelevant). Risk-1 discipline: the RAW BODY is read + verified
|
||||
BEFORE any parsing — a pre-parsed body breaks the HMAC."""
|
||||
raw_body = await request.body()
|
||||
signature = request.headers.get("Stripe-Signature", "")
|
||||
await process_webhook_event(
|
||||
store=store,
|
||||
provider=_get_payment_provider(request),
|
||||
raw_body=raw_body,
|
||||
signature_header=signature,
|
||||
)
|
||||
return Response(content='{"received": true}', media_type="application/json")
|
||||
@@ -0,0 +1,536 @@
|
||||
"""/v1/billing/admin — operator billing API (REQ-9-003, D-076).
|
||||
|
||||
Operator-mediated checkout: at pilot there are no employer principals
|
||||
(A-407 — employers are data, not principals), so the operator buys on
|
||||
an employer's behalf. Every route sits behind `require_admin`
|
||||
(fail-closed, R-51 — copied verbatim from marketplace_admin.py); the
|
||||
checkout records operator_ref + employer_id provenance (D-076).
|
||||
|
||||
The mock-complete route (D-082, amended G-301) is admin-gated AND
|
||||
mounted only under provider=mock — the operator completing a session
|
||||
after out-of-band payment is the honest pilot flow.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..billing.base import (
|
||||
CheckoutSessionSpec,
|
||||
)
|
||||
from ..billing.store import BillingStore
|
||||
from ..config import Settings
|
||||
from .deps import get_billing_store
|
||||
from .marketplace_admin import require_admin
|
||||
|
||||
router = APIRouter(prefix="/v1/billing/admin", tags=["billing-admin"])
|
||||
|
||||
|
||||
# -- contracts ----------------------------------------------------------------
|
||||
|
||||
|
||||
class CheckoutCreate(BaseModel):
|
||||
"""The operator's checkout request (D-076): which catalog item, on
|
||||
behalf of which employer."""
|
||||
|
||||
item_id: str = Field(min_length=1)
|
||||
employer_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class CheckoutResponse(BaseModel):
|
||||
"""The created session handle + the local reconciliation id."""
|
||||
|
||||
session_id: str
|
||||
provider_session_id: str
|
||||
url: str
|
||||
mode: str
|
||||
|
||||
|
||||
class MockCompleteRequest(BaseModel):
|
||||
"""D-082/G-301: the operator completes a mock session after
|
||||
out-of-band payment. Admin-gated + mock-provider-only."""
|
||||
|
||||
provider_session_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class MockCompleteResponse(BaseModel):
|
||||
completed: bool
|
||||
payment_status: str
|
||||
|
||||
|
||||
class EntitlementResponse(BaseModel):
|
||||
"""The per-employer entitlement summary (REQ-9-004): active
|
||||
purchase (posts used/limit/expiry) + subscription state. The admin
|
||||
surface renders this (G-308's token-gated fetch)."""
|
||||
|
||||
employer_id: str
|
||||
purchase: dict[str, object] | None
|
||||
subscription: dict[str, object] | None
|
||||
|
||||
|
||||
# -- checkout (Task 3-2-01, G-310) ---------------------------------------------
|
||||
|
||||
|
||||
@router.post("/checkout", status_code=201, response_model=CheckoutResponse)
|
||||
async def create_checkout(
|
||||
body: CheckoutCreate,
|
||||
request: Request,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> CheckoutResponse:
|
||||
"""Create a provider checkout session for a catalog item on an
|
||||
employer's behalf (D-076). The catalog item snapshot (name +
|
||||
price_cents, D-078) rides the session and the local reconciliation
|
||||
row; success/cancel URLs default from public_base_url (G-310)."""
|
||||
package, plan, item = store.get_catalog_item(body.item_id)
|
||||
if item is None:
|
||||
raise HTTPException(status_code=404, detail="unknown catalog item")
|
||||
mode = "subscription" if item["kind"] == "plan" else "payment"
|
||||
interval: str = str(item.get("interval", "") or "")
|
||||
if settings.public_base_url:
|
||||
success_url = (
|
||||
f"{settings.public_base_url}/marketplace/pricing?checkout=success"
|
||||
)
|
||||
cancel_url = (
|
||||
f"{settings.public_base_url}/marketplace/pricing?checkout=cancelled"
|
||||
)
|
||||
else:
|
||||
# Mock pilot path: the mock provider never redirects; empty is
|
||||
# honest (stripe factory validation rejects the empty case —
|
||||
# G-310's loud boot warning covers the stripe selection).
|
||||
success_url = ""
|
||||
cancel_url = ""
|
||||
# The reconciliation key is the LOCAL row id (R-34): persist the
|
||||
# snapshot row FIRST, then mint the spec against it. The provider
|
||||
# session id is bound AFTER the provider call — until then the
|
||||
# placeholder must be UNIQUE (final-review fix: a bare "pending"
|
||||
# collided on the provider_session_id UNIQUE constraint the moment
|
||||
# two checkouts raced — the second admin POST 500'd with
|
||||
# IntegrityError; a fresh uuid per row makes the placeholder a
|
||||
# proper placeholder).
|
||||
row = store.insert_checkout_session(
|
||||
provider_session_id=f"pending-{uuid.uuid4().hex[:16]}",
|
||||
employer_id=body.employer_id,
|
||||
item_id=str(item["id"]),
|
||||
item_name=str(item["name"]),
|
||||
price_cents=int(item["price_cents"]),
|
||||
mode=mode,
|
||||
)
|
||||
spec = CheckoutSessionSpec(
|
||||
mode=mode, # type: ignore[arg-type]
|
||||
item_id=str(item["id"]),
|
||||
price_cents=int(item["price_cents"]),
|
||||
item_name=str(item["name"]),
|
||||
interval=interval,
|
||||
success_url=success_url,
|
||||
cancel_url=cancel_url,
|
||||
client_reference_id=row.id,
|
||||
metadata={
|
||||
"employer_id": body.employer_id,
|
||||
"operator_ref": "operator",
|
||||
},
|
||||
)
|
||||
provider = request.app.state.payment_provider
|
||||
session = await provider.create_checkout_session(spec)
|
||||
# Pin the provider session id onto the local row (upsert by local
|
||||
# id — the row IS the reconciliation record, G-302).
|
||||
store.bind_provider_session_id(row.id, session.id)
|
||||
return CheckoutResponse(
|
||||
session_id=row.id,
|
||||
provider_session_id=session.id,
|
||||
url=session.url,
|
||||
mode=mode,
|
||||
)
|
||||
|
||||
|
||||
# -- mock-complete (D-082, amended G-301) ---------------------------------------
|
||||
|
||||
|
||||
@router.post(
|
||||
"/mock/complete", status_code=200, response_model=MockCompleteResponse
|
||||
)
|
||||
async def mock_complete(
|
||||
body: MockCompleteRequest,
|
||||
request: Request,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> MockCompleteResponse:
|
||||
"""Complete a MOCK checkout session after out-of-band payment
|
||||
(D-082/G-301): admin-gated, mounted only under provider=mock —
|
||||
this route 404s on a stripe-configured box (the factory-conditional
|
||||
mount lives in main.py). The completion mints a properly-signed
|
||||
event through the REAL webhook processing path."""
|
||||
from ..billing.mock import MockPaymentProvider
|
||||
|
||||
provider = request.app.state.payment_provider
|
||||
if not isinstance(provider, MockPaymentProvider):
|
||||
# main.py gates the mount; this arm is defense-in-depth.
|
||||
raise HTTPException(
|
||||
status_code=404, detail="mock-complete exists only under provider=mock"
|
||||
)
|
||||
session = store.get_checkout_session_by_provider_id(body.provider_session_id)
|
||||
if session is None:
|
||||
raise HTTPException(status_code=404, detail="unknown checkout session")
|
||||
try:
|
||||
session_after = await provider.confirm_completion(
|
||||
body.provider_session_id
|
||||
)
|
||||
except ValueError as exc:
|
||||
# G-301 idempotency (the mock's documented contract: the ROUTE
|
||||
# owns it): the provider refuses a re-completion ("not open" —
|
||||
# operator double-click) or lost its in-memory state across a
|
||||
# restart ("unknown session"). The provider's CURRENT view is
|
||||
# authoritative — resolve via retrieve_session and re-run the
|
||||
# REAL webhook path (idempotent per session, G-303); NEVER a
|
||||
# 500 on the payment surface (G-306).
|
||||
recovered = await provider.retrieve_session(body.provider_session_id)
|
||||
if recovered is None:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=(
|
||||
"the provider no longer knows this session (a restart "
|
||||
"clears mock state) — create a new checkout"
|
||||
),
|
||||
) from exc
|
||||
if recovered.status != "complete":
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"session is {recovered.status!r}, not completable",
|
||||
) from exc
|
||||
# Already complete at the provider: fall through with the
|
||||
# recovered truth — the re-minted event re-runs the grant path,
|
||||
# and G-303's per-session exactly-once refuses a second grant.
|
||||
session_after = recovered
|
||||
# Mint + process the signed event through the REAL webhook path
|
||||
# (direct handler call — same code path, G-301's E2E proof).
|
||||
from .billing import process_webhook_event
|
||||
|
||||
raw_body, header, event_id = provider.mint_event(
|
||||
"checkout.completed",
|
||||
{
|
||||
"session_id": body.provider_session_id,
|
||||
"payment_status": session_after.payment_status,
|
||||
},
|
||||
)
|
||||
await process_webhook_event(
|
||||
store=store,
|
||||
provider=provider,
|
||||
raw_body=raw_body,
|
||||
signature_header=header,
|
||||
)
|
||||
return MockCompleteResponse(
|
||||
completed=session_after.status == "complete",
|
||||
payment_status=session_after.payment_status,
|
||||
)
|
||||
|
||||
|
||||
# -- entitlements (Task 4-1-02) ------------------------------------------------
|
||||
|
||||
|
||||
@router.get(
|
||||
"/entitlements/{employer_id}", status_code=200, response_model=EntitlementResponse
|
||||
)
|
||||
def get_entitlement(
|
||||
employer_id: str,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> EntitlementResponse:
|
||||
"""The employer's entitlement state: active package (posts used/
|
||||
limit/expiry — lazy past_due grace per G-306) + subscription."""
|
||||
entitlement = store.get_entitlement(employer_id)
|
||||
return EntitlementResponse(
|
||||
employer_id=entitlement["employer_id"],
|
||||
purchase=entitlement["purchase"],
|
||||
subscription=entitlement["subscription"],
|
||||
)
|
||||
|
||||
|
||||
# -- subscription lifecycle + manual grants + event log (Task 5-1-02) -------------
|
||||
|
||||
|
||||
class LifecycleResponse(BaseModel):
|
||||
"""A provider-backed or manual lifecycle transition result."""
|
||||
|
||||
subscription_id: str
|
||||
status: str
|
||||
provider_action: str | None = None
|
||||
|
||||
|
||||
class ManualGrantRequest(BaseModel):
|
||||
"""D-080's manual path: the operator grants enterprise unlimited
|
||||
posts or a plan on an employer (remediation for failed webhooks —
|
||||
D-083: recorded with provenance)."""
|
||||
|
||||
note: str = ""
|
||||
|
||||
|
||||
@router.post(
|
||||
"/subscriptions/{subscription_id}/cancel",
|
||||
status_code=200,
|
||||
response_model=LifecycleResponse,
|
||||
)
|
||||
async def cancel_subscription(
|
||||
subscription_id: str,
|
||||
request: Request,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> LifecycleResponse:
|
||||
"""Cancel a subscription (R-46): the provider call first (stripe:
|
||||
DELETE / the mock: state-only), then the local transition. Manual
|
||||
cancellation records provenance (D-083)."""
|
||||
subscription = store.get_subscription(subscription_id)
|
||||
if subscription is None:
|
||||
raise HTTPException(status_code=404, detail="unknown subscription")
|
||||
provider_action: str | None = None
|
||||
provider = request.app.state.payment_provider
|
||||
if hasattr(provider, "cancel_subscription"):
|
||||
ref = await provider.cancel_subscription(subscription.provider_subscription_id)
|
||||
provider_action = f"cancel:{ref.status}" if ref else "cancel:unknown"
|
||||
try:
|
||||
updated = store.transition_subscription(subscription_id, "cancelled")
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
assert updated is not None
|
||||
store.insert_billing_event(
|
||||
"manual",
|
||||
"cancel",
|
||||
employer_id=subscription.employer_id,
|
||||
operator="operator",
|
||||
detail={"subscription_id": subscription_id},
|
||||
)
|
||||
return LifecycleResponse(
|
||||
subscription_id=subscription_id,
|
||||
status=updated.status,
|
||||
provider_action=provider_action,
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/subscriptions/{subscription_id}/resume",
|
||||
status_code=200,
|
||||
response_model=LifecycleResponse,
|
||||
)
|
||||
def resume_subscription(
|
||||
subscription_id: str,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> LifecycleResponse:
|
||||
"""Resume past_due → active (the recovery transition; only the
|
||||
matrix's legal moves — InvalidTransition is the honest 409)."""
|
||||
subscription = store.get_subscription(subscription_id)
|
||||
if subscription is None:
|
||||
raise HTTPException(status_code=404, detail="unknown subscription")
|
||||
try:
|
||||
updated = store.transition_subscription(subscription_id, "active")
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
assert updated is not None
|
||||
store.insert_billing_event(
|
||||
"manual",
|
||||
"resume",
|
||||
employer_id=subscription.employer_id,
|
||||
operator="operator",
|
||||
detail={"subscription_id": subscription_id},
|
||||
)
|
||||
return LifecycleResponse(subscription_id=subscription_id, status=updated.status)
|
||||
|
||||
|
||||
class UpgradeRequest(BaseModel):
|
||||
plan_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/subscriptions/{subscription_id}/upgrade",
|
||||
status_code=200,
|
||||
response_model=LifecycleResponse,
|
||||
)
|
||||
async def upgrade_subscription(
|
||||
subscription_id: str,
|
||||
body: UpgradeRequest,
|
||||
request: Request,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> LifecycleResponse:
|
||||
"""G-305: upgrade is a plan_id transition WITHIN active (never a
|
||||
status move); the store records the manual billing_event with
|
||||
provenance. Stripe: the provider's update_subscription resolves
|
||||
items[0][id] inside (R-47 — the dual-charge guard)."""
|
||||
subscription = store.get_subscription(subscription_id)
|
||||
if subscription is None:
|
||||
raise HTTPException(status_code=404, detail="unknown subscription")
|
||||
_package, plan, item = store.get_catalog_item(body.plan_id)
|
||||
if plan is None or item is None:
|
||||
raise HTTPException(status_code=404, detail="unknown plan")
|
||||
provider = request.app.state.payment_provider
|
||||
if hasattr(provider, "update_subscription"):
|
||||
await provider.update_subscription(
|
||||
subscription.provider_subscription_id, int(item["price_cents"])
|
||||
)
|
||||
try:
|
||||
updated = store.upgrade_subscription(
|
||||
subscription_id,
|
||||
body.plan_id,
|
||||
str(item["name"]),
|
||||
int(item["price_cents"]),
|
||||
)
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
assert updated is not None
|
||||
return LifecycleResponse(
|
||||
subscription_id=subscription_id,
|
||||
status=updated.status,
|
||||
provider_action=f"update:{body.plan_id}",
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/employers/{employer_id}/grant",
|
||||
status_code=201,
|
||||
response_model=EntitlementResponse,
|
||||
)
|
||||
def manual_grant(
|
||||
employer_id: str,
|
||||
request: Request,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> EntitlementResponse:
|
||||
"""D-080's manual unlimited grant (the enterprise path — operator
|
||||
remediation for failed webhooks; D-083: manual provenance in the
|
||||
audit log)."""
|
||||
session_row = store.insert_checkout_session(
|
||||
# Final-review fix: a per-call unique placeholder — a second
|
||||
# grant for the same employer collided on the
|
||||
# provider_session_id UNIQUE constraint (500) where the route
|
||||
# promises idempotent re-grants.
|
||||
provider_session_id=f"cs_manual_{uuid.uuid4().hex[:16]}",
|
||||
employer_id=employer_id,
|
||||
item_id="pkg-enterprise",
|
||||
item_name="Enterprise (manual grant)",
|
||||
price_cents=0,
|
||||
mode="payment",
|
||||
)
|
||||
store.create_purchase_from_session(
|
||||
session_row, posts_limit=None, duration_days=None
|
||||
)
|
||||
store.insert_billing_event(
|
||||
"manual",
|
||||
"grant",
|
||||
employer_id=employer_id,
|
||||
operator="operator",
|
||||
detail={"grant": "unlimited"},
|
||||
)
|
||||
entitlement = store.get_entitlement(employer_id)
|
||||
return EntitlementResponse(
|
||||
employer_id=employer_id,
|
||||
purchase=entitlement["purchase"],
|
||||
subscription=entitlement["subscription"],
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/employers/{employer_id}/revoke",
|
||||
status_code=200,
|
||||
response_model=EntitlementResponse,
|
||||
)
|
||||
def manual_revoke(
|
||||
employer_id: str,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> EntitlementResponse:
|
||||
"""Revoke every purchase grant for the employer (manual remediation
|
||||
— the inverse of the grant; provenance recorded)."""
|
||||
from sqlalchemy import text as _text
|
||||
from sqlmodel import Session as _Session
|
||||
|
||||
# Final-review fix: bind a NAIVE utcnow — the same format the ORM's
|
||||
# sqlite DATETIME processor writes. The old aware bind (+00:00
|
||||
# suffix) left mixed-format TEXT in the column; comparisons stayed
|
||||
# correct in practice (ISO prefix ordering) but the column held two
|
||||
# spellings of the same field. One format, one discipline.
|
||||
with _Session(store._engine) as session: # type: ignore[attr-defined]
|
||||
session.execute(
|
||||
_text(
|
||||
"UPDATE billing_purchases SET expires_at = :past "
|
||||
"WHERE employer_id = :employer_id"
|
||||
),
|
||||
{
|
||||
"past": _utc_now_minus_day().replace(tzinfo=None),
|
||||
"employer_id": employer_id,
|
||||
},
|
||||
)
|
||||
session.commit()
|
||||
store.insert_billing_event(
|
||||
"manual",
|
||||
"revoke",
|
||||
employer_id=employer_id,
|
||||
operator="operator",
|
||||
detail={"revoke": "all purchases"},
|
||||
)
|
||||
entitlement = store.get_entitlement(employer_id)
|
||||
return EntitlementResponse(
|
||||
employer_id=employer_id,
|
||||
purchase=entitlement["purchase"],
|
||||
subscription=entitlement["subscription"],
|
||||
)
|
||||
|
||||
|
||||
def _utc_now_minus_day():
|
||||
"""Naive-UTC (matches the ORM's sqlite DATETIME storage format —
|
||||
see the revoke route's binding note)."""
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
return datetime.now(UTC).replace(tzinfo=None) - timedelta(days=1)
|
||||
|
||||
|
||||
class BillingEventRow_(BaseModel):
|
||||
"""The audit-trail wire row (REQ-9-005's queryable log)."""
|
||||
|
||||
id: int
|
||||
employer_id: str | None
|
||||
kind: str
|
||||
name: str
|
||||
detail: dict[str, object]
|
||||
operator: str | None
|
||||
created_at: str | None
|
||||
|
||||
|
||||
class BillingEventListResponse(BaseModel):
|
||||
events: list[BillingEventRow_]
|
||||
count: int
|
||||
|
||||
|
||||
@router.get(
|
||||
"/events", status_code=200, response_model=BillingEventListResponse
|
||||
)
|
||||
def list_events(
|
||||
employer_id: str | None = None,
|
||||
kind: str | None = None,
|
||||
name: str | None = None,
|
||||
limit: int = 100,
|
||||
store: BillingStore = Depends(get_billing_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> BillingEventListResponse:
|
||||
"""The billing event log (REQ-9-005): every provider event, manual
|
||||
action (with operator provenance), and unmatched delivery —
|
||||
filterable by employer/kind/name."""
|
||||
rows = store.list_billing_events(
|
||||
employer_id=employer_id, kind=kind, name=name, limit=limit
|
||||
)
|
||||
return BillingEventListResponse(
|
||||
events=[
|
||||
BillingEventRow_(
|
||||
id=r.id or 0,
|
||||
employer_id=r.employer_id,
|
||||
kind=r.kind,
|
||||
name=r.name,
|
||||
detail=dict(r.detail),
|
||||
operator=r.operator,
|
||||
created_at=r.created_at.isoformat() if r.created_at else None,
|
||||
)
|
||||
for r in rows
|
||||
],
|
||||
count=len(rows),
|
||||
)
|
||||
@@ -0,0 +1,401 @@
|
||||
"""Oral-defense endpoints (Task 5-3-01, REQ-3-006, A-109).
|
||||
|
||||
Full defense loop over HTTP with mock-first voice (D-030) and the seventh
|
||||
Examiner agent (SSE question streaming happens through the chat pipeline;
|
||||
these endpoints are the session orchestration + transcript persistence):
|
||||
|
||||
POST /v1/defense/start {learner_id, task_id}
|
||||
POST /v1/defense/{id}/answer {text} | multipart audio (STT)
|
||||
GET /v1/defense/{id}/audio/{turn_id} TTS bytes (streaming)
|
||||
POST /v1/defense/{id}/finish verdict + integrity signals
|
||||
GET /v1/defense/{id} transcript + signals
|
||||
|
||||
Integrity signals (A-109) are computed server-side from turn metadata:
|
||||
long pauses = learner turns whose latency_ms exceeds PAUSE_THRESHOLD_MS.
|
||||
The defense does NOT gate on trace completeness (the grader does, G-4);
|
||||
an incomplete trace is surfaced as `trace_complete: false` so the UI can
|
||||
disclose it before the learner defends.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import itertools
|
||||
import secrets
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, Request, UploadFile
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..agents.examiner import ExaminerAgent
|
||||
from ..grading.features import TraceDigest, compute_digest
|
||||
from ..llm.types import Message
|
||||
from ..voice.base import VoiceDescriptor
|
||||
from ..voice.browser import BROWSER_FALLBACK_DESCRIPTOR
|
||||
from ..voice.defense_store import DefenseRecord, DefenseStore, DefenseTurn
|
||||
from .deps import (
|
||||
get_examiner,
|
||||
get_settings,
|
||||
get_trace_store,
|
||||
get_variant_store,
|
||||
get_voice_provider,
|
||||
get_voice_store,
|
||||
)
|
||||
from .identity import require_verified_age
|
||||
|
||||
router = APIRouter(prefix="/v1/defense", tags=["defense"])
|
||||
|
||||
#: A-109: learner turns slower than this are flagged as long pauses (ms).
|
||||
PAUSE_THRESHOLD_MS = 15_000
|
||||
|
||||
_ROLE_EXAMINER = "examiner"
|
||||
_ROLE_LEARNER = "learner"
|
||||
|
||||
# v0.4.7 (P1, final-review flag): the old id mint `dfn-{ms:x}` collided on
|
||||
# same-millisecond starts — the insert-only store's PK then 500'd the second
|
||||
# start permanently (the identity-resubmit P0-C1 class). Nonce + counter
|
||||
# keeps ids unique within and across process restarts (the identity/mock.py
|
||||
# pattern); ids stay opaque handles, never derived from PII.
|
||||
_DEFENSE_NONCE = secrets.randbits(32)
|
||||
_DEFENSE_COUNTER = itertools.count(1)
|
||||
|
||||
|
||||
def mint_defense_id(learner_id: str) -> str:
|
||||
seq = next(_DEFENSE_COUNTER)
|
||||
return f"dfn-{_DEFENSE_NONCE:08x}{seq:08x}-{learner_id[:8]}"
|
||||
|
||||
|
||||
class StartRequest(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
task_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class StartResponse(BaseModel):
|
||||
defense_id: str
|
||||
voice_descriptor: dict
|
||||
trace_complete: bool
|
||||
first_question: str
|
||||
|
||||
|
||||
class AnswerResponse(BaseModel):
|
||||
question: str
|
||||
turn_latency: dict[str, int | None]
|
||||
|
||||
|
||||
class FinishResponse(BaseModel):
|
||||
verdict: dict
|
||||
integrity_signals: dict
|
||||
|
||||
|
||||
async def _digest_for_task(
|
||||
trace_store, learner_id: str, task_id: str
|
||||
) -> tuple[TraceDigest | None, bool]:
|
||||
"""Digest of the learner's trace for this task + completeness flag."""
|
||||
if not trace_store.list_tasks(learner_id) or task_id not in trace_store.list_tasks(
|
||||
learner_id
|
||||
):
|
||||
return None, True # no trace at all is "complete" for defense purposes
|
||||
trace = trace_store.get_trace(learner_id, task_id)
|
||||
gaps = trace_store.gaps(learner_id, task_id)
|
||||
return (compute_digest(trace) if trace else None), (len(gaps) == 0)
|
||||
|
||||
|
||||
def _voice_descriptor(settings) -> VoiceDescriptor:
|
||||
"""The capability descriptor for the configured voice mode (D-030).
|
||||
|
||||
Must-Have #6: browser mode returns BROWSER_FALLBACK_DESCRIPTOR so the
|
||||
web client selects native SpeechRecognition/speechSynthesis; mock mode
|
||||
returns the mock descriptor. (A real server provider returns
|
||||
mode="server" — the protocol seam.)
|
||||
"""
|
||||
if (settings.voice_provider or "mock").strip().lower() == "browser":
|
||||
return BROWSER_FALLBACK_DESCRIPTOR
|
||||
return VoiceDescriptor(
|
||||
mode="mock", sr_available=True, tts_available=True, hint=""
|
||||
)
|
||||
|
||||
|
||||
@router.post("/start", response_model=StartResponse)
|
||||
async def start_defense(
|
||||
body: StartRequest,
|
||||
request: Request,
|
||||
examiner: ExaminerAgent = Depends(get_examiner),
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
voice_provider=Depends(get_voice_provider),
|
||||
trace_store=Depends(get_trace_store),
|
||||
variant_store=Depends(get_variant_store),
|
||||
settings=Depends(get_settings),
|
||||
) -> StartResponse:
|
||||
# v0.5 identity gate (D-043): allowlist first (G-5), then the school
|
||||
# 16+ verified verdict, before any defense machinery runs.
|
||||
if body.learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=(
|
||||
f"learner_id {body.learner_id!r} is not on the sandbox "
|
||||
"allowlist (G-5)"
|
||||
),
|
||||
)
|
||||
await require_verified_age(
|
||||
16, body.learner_id, request.app.state.identity_store
|
||||
)
|
||||
record = voice_store.start(
|
||||
DefenseRecord(
|
||||
id=mint_defense_id(body.learner_id),
|
||||
learner_id=body.learner_id,
|
||||
task_id=body.task_id,
|
||||
status="in_progress",
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
digest, trace_complete = await _digest_for_task(trace_store, body.learner_id, body.task_id)
|
||||
variant = variant_store.get_by_task(body.task_id)
|
||||
statement = variant.statement if variant is not None else None
|
||||
|
||||
started = time.perf_counter()
|
||||
question = await examiner.next_question(
|
||||
history=[], trace_digest=digest, variant_statement=statement
|
||||
)
|
||||
llm_ms = int((time.perf_counter() - started) * 1000)
|
||||
voice_store.append_turn(
|
||||
record.id,
|
||||
DefenseTurn(
|
||||
defense_id=record.id,
|
||||
seq=0,
|
||||
role=_ROLE_EXAMINER,
|
||||
text=question,
|
||||
ts=datetime.now(UTC),
|
||||
latency_ms=llm_ms,
|
||||
created_at=datetime.now(UTC),
|
||||
),
|
||||
)
|
||||
descriptor = getattr(voice_provider, "descriptor", None) or _voice_descriptor(
|
||||
settings
|
||||
)
|
||||
return StartResponse(
|
||||
defense_id=record.id,
|
||||
voice_descriptor=descriptor.model_dump(),
|
||||
trace_complete=trace_complete,
|
||||
first_question=question,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{defense_id}/answer", response_model=AnswerResponse)
|
||||
async def answer_defense(
|
||||
defense_id: str,
|
||||
text: str | None = Form(default=None),
|
||||
audio: UploadFile | None = File(default=None),
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
voice_provider=Depends(get_voice_provider),
|
||||
examiner: ExaminerAgent = Depends(get_examiner),
|
||||
trace_store=Depends(get_trace_store),
|
||||
variant_store=Depends(get_variant_store),
|
||||
settings=Depends(get_settings),
|
||||
) -> AnswerResponse:
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail=f"no defense {defense_id!r}")
|
||||
if record.status == "finished":
|
||||
# The store owns the finished transition but does NOT police turn
|
||||
# sequencing (defense_store.py: "turns after finalize are a sequencing
|
||||
# bug for the endpoints to prevent") — this is the endpoint half of
|
||||
# that contract: a sealed transcript is append-only-no-more.
|
||||
raise HTTPException(
|
||||
status_code=409, detail="defense is finished; start a new defense"
|
||||
)
|
||||
if text is None and audio is None:
|
||||
raise HTTPException(status_code=422, detail="provide {text} or audio")
|
||||
|
||||
# STT (typed fallback bypasses the voice provider entirely).
|
||||
stt_ms: int | None = None
|
||||
if audio is not None:
|
||||
stt_started = time.perf_counter()
|
||||
raw = await audio.read()
|
||||
# P3 (Task 3-2-01): the upload is fully consumed into memory —
|
||||
# no streaming path exists (G-107), so close the spooled file
|
||||
# HERE, before any guard below can raise (422 empty / 413
|
||||
# oversize / 502 provider all operate on `raw` only). FastAPI's
|
||||
# request-end file stack would also close it — close() is
|
||||
# idempotent on the spooled file — but the route must not rely
|
||||
# on GC-timing cleanup.
|
||||
await audio.close()
|
||||
if not raw:
|
||||
# Empty upload is a client error (422), not a provider crash
|
||||
# (500): validate before the provider call so every provider
|
||||
# sees the same contract.
|
||||
raise HTTPException(status_code=422, detail="audio upload is empty")
|
||||
max_bytes = settings.voice_max_audio_mb * 1024 * 1024
|
||||
if len(raw) > max_bytes:
|
||||
# D-041/G-12: bounded audio BEFORE the provider call — the
|
||||
# client renders this as an honest re-record prompt.
|
||||
raise HTTPException(
|
||||
status_code=413,
|
||||
detail=(
|
||||
f"audio exceeds {settings.voice_max_audio_mb}MB "
|
||||
"— re-record a shorter answer"
|
||||
),
|
||||
)
|
||||
# D-041: strip codec params — MediaRecorder sends
|
||||
# 'audio/webm;codecs=opus'; the bare extension is the provider
|
||||
# contract ('webm'), else real STT endpoints reject the multipart.
|
||||
fmt = (audio.content_type or "audio/wav").split("/")[-1].split(";")[0].strip()
|
||||
try:
|
||||
segment = await voice_provider.transcribe(raw, fmt)
|
||||
except RuntimeError as exc:
|
||||
# Provider failure is the 502 house pattern (assessment.py /
|
||||
# proctor.py), not a 500: a real endpoint outage (or the
|
||||
# default mock's unscripted queue — final-review cross-phase
|
||||
# P0) must surface as an honest upstream error. Both providers
|
||||
# raise RuntimeError with sanitized text (mock: MockVoiceFailure;
|
||||
# openai-audio: key-redacted _sanitize).
|
||||
raise HTTPException(
|
||||
status_code=502, detail=f"voice transcription failed: {exc}"
|
||||
) from exc
|
||||
stt_ms = int((time.perf_counter() - stt_started) * 1000)
|
||||
text = segment.text
|
||||
|
||||
turns = record.turns if hasattr(record, "turns") else []
|
||||
history = [
|
||||
Message(role="assistant" if t.role == _ROLE_EXAMINER else "user", content=t.text)
|
||||
for t in turns
|
||||
]
|
||||
next_seq = len(turns)
|
||||
|
||||
voice_store.append_turn(
|
||||
defense_id,
|
||||
DefenseTurn(
|
||||
defense_id=defense_id,
|
||||
seq=next_seq,
|
||||
role=_ROLE_LEARNER,
|
||||
text=text or "",
|
||||
ts=datetime.now(UTC),
|
||||
latency_ms=stt_ms,
|
||||
created_at=datetime.now(UTC),
|
||||
),
|
||||
)
|
||||
|
||||
digest, _ = await _digest_for_task(trace_store, record.learner_id, record.task_id)
|
||||
variant = variant_store.get_by_task(record.task_id)
|
||||
|
||||
llm_started = time.perf_counter()
|
||||
question = await examiner.next_question(
|
||||
history=history + [Message(role="user", content=text or "")],
|
||||
trace_digest=digest,
|
||||
variant_statement=variant.statement if variant is not None else None,
|
||||
)
|
||||
llm_ms = int((time.perf_counter() - llm_started) * 1000)
|
||||
|
||||
voice_store.append_turn(
|
||||
defense_id,
|
||||
DefenseTurn(
|
||||
defense_id=defense_id,
|
||||
seq=next_seq + 1,
|
||||
role=_ROLE_EXAMINER,
|
||||
text=question,
|
||||
ts=datetime.now(UTC),
|
||||
latency_ms=llm_ms,
|
||||
created_at=datetime.now(UTC),
|
||||
),
|
||||
)
|
||||
return AnswerResponse(
|
||||
question=question,
|
||||
turn_latency={"stt_ms": stt_ms, "llm_ms": llm_ms, "tts_ms": None},
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{defense_id}/audio/{turn_id}")
|
||||
async def defense_audio(
|
||||
defense_id: str,
|
||||
turn_id: int,
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
voice_provider=Depends(get_voice_provider),
|
||||
settings=Depends(get_settings),
|
||||
):
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail=f"no defense {defense_id!r}")
|
||||
turn = next((t for t in record.turns if t.seq == turn_id), None)
|
||||
if turn is None or turn.role != _ROLE_EXAMINER:
|
||||
raise HTTPException(status_code=404, detail=f"no examiner turn {turn_id!r}")
|
||||
|
||||
async def stream():
|
||||
async for chunk in voice_provider.synthesize(turn.text):
|
||||
yield chunk
|
||||
|
||||
# G-16/D-041: the TTS format is a settings enum; the media_type maps
|
||||
# from it (was hardcoded audio/wav — wrong for every real format).
|
||||
return StreamingResponse(
|
||||
stream(), media_type=f"audio/{settings.voice_tts_format}"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{defense_id}/finish", response_model=FinishResponse)
|
||||
async def finish_defense(
|
||||
defense_id: str,
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
examiner: ExaminerAgent = Depends(get_examiner),
|
||||
trace_store=Depends(get_trace_store),
|
||||
variant_store=Depends(get_variant_store),
|
||||
) -> FinishResponse:
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail=f"no defense {defense_id!r}")
|
||||
|
||||
turns = record.turns if hasattr(record, "turns") else []
|
||||
history = [
|
||||
Message(role="assistant" if t.role == _ROLE_EXAMINER else "user", content=t.text)
|
||||
for t in turns
|
||||
]
|
||||
digest, _ = await _digest_for_task(trace_store, record.learner_id, record.task_id)
|
||||
variant = variant_store.get_by_task(record.task_id)
|
||||
verdict = await examiner.final_verdict(
|
||||
history=history,
|
||||
trace_digest=digest,
|
||||
variant_statement=variant.statement if variant is not None else None,
|
||||
)
|
||||
|
||||
signals: dict = {
|
||||
"long_pauses": [
|
||||
{"turn": t.seq, "latency_ms": t.latency_ms}
|
||||
for t in turns
|
||||
if t.role == _ROLE_LEARNER and (t.latency_ms or 0) > PAUSE_THRESHOLD_MS
|
||||
],
|
||||
"pause_threshold_ms": PAUSE_THRESHOLD_MS,
|
||||
# Must-Have #1: "verdict + transcript persisted" — the verdict is
|
||||
# stored INSIDE integrity_signals so GET /{id} after finish can
|
||||
# re-serve it (the finish response alone would lose it). Signals
|
||||
# are a JSON object dict (DefenseStore.finalize contract), so the
|
||||
# verdict nests under the "verdict" key alongside the A-109
|
||||
# markers the Proctor/Mentor feeds read.
|
||||
"verdict": verdict.model_dump(),
|
||||
}
|
||||
voice_store.finalize(defense_id, signals)
|
||||
return FinishResponse(verdict=verdict.model_dump(), integrity_signals=signals)
|
||||
|
||||
|
||||
@router.get("/{defense_id}")
|
||||
async def get_defense(
|
||||
defense_id: str,
|
||||
voice_store: DefenseStore = Depends(get_voice_store),
|
||||
):
|
||||
record = voice_store.get(defense_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail=f"no defense {defense_id!r}")
|
||||
return {
|
||||
"defense_id": record.id,
|
||||
"learner_id": record.learner_id,
|
||||
"task_id": record.task_id,
|
||||
"status": record.status,
|
||||
"turns": [
|
||||
{
|
||||
"seq": t.seq,
|
||||
"role": t.role,
|
||||
"text": t.text,
|
||||
"ts": t.ts,
|
||||
"latency_ms": t.latency_ms,
|
||||
}
|
||||
for t in record.turns
|
||||
],
|
||||
"integrity_signals": record.integrity_signals or {},
|
||||
}
|
||||
@@ -2,12 +2,25 @@
|
||||
|
||||
from fastapi import Request
|
||||
|
||||
from ..agents.examiner import ExaminerAgent
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..agents.session import SessionStore
|
||||
from ..billing.store import BillingStore
|
||||
from ..config import Settings
|
||||
from ..employers.store import EmployerStore
|
||||
from ..grading.engine import GradingEngine
|
||||
from ..grading.store import GradeStore
|
||||
from ..llm.base import LLMProvider
|
||||
from ..marketplace.embedding.base import EmbeddingProvider
|
||||
from ..marketplace.store import MarketplaceStore
|
||||
from ..sandbox.manager import SandboxManager
|
||||
from ..sandbox.workdir import SandboxDir
|
||||
from ..telemetry.ingest import TraceIntegrityMap
|
||||
from ..telemetry.store import TraceStore
|
||||
from ..variants.generator import VariantGenerator
|
||||
from ..variants.store import VariantStore
|
||||
from ..voice.base import VoiceProvider
|
||||
from ..voice.defense_store import DefenseStore
|
||||
|
||||
|
||||
def get_settings(request: Request) -> Settings:
|
||||
@@ -33,3 +46,62 @@ def get_sandbox_manager(request: Request) -> SandboxManager:
|
||||
def get_sandbox_test_layout(request: Request) -> SandboxDir | None:
|
||||
"""Optional test seam (app.state.sandbox_test_layout); always None in prod."""
|
||||
return getattr(request.app.state, "sandbox_test_layout", None)
|
||||
|
||||
|
||||
def get_trace_store(request: Request) -> TraceStore:
|
||||
return request.app.state.trace_store
|
||||
|
||||
|
||||
def get_trace_integrity(request: Request) -> TraceIntegrityMap:
|
||||
return request.app.state.trace_integrity
|
||||
|
||||
|
||||
def get_grade_store(request: Request) -> GradeStore:
|
||||
return request.app.state.grade_store
|
||||
|
||||
|
||||
def get_grading_engine(request: Request) -> GradingEngine:
|
||||
return request.app.state.grading_engine
|
||||
|
||||
|
||||
def get_variant_generator(request: Request) -> VariantGenerator:
|
||||
return request.app.state.variant_generator
|
||||
|
||||
|
||||
def get_variant_store(request: Request) -> VariantStore:
|
||||
return request.app.state.variant_store
|
||||
|
||||
def get_voice_store(request: Request) -> DefenseStore:
|
||||
return request.app.state.defense_store
|
||||
|
||||
|
||||
def get_voice_provider(request: Request) -> VoiceProvider:
|
||||
return request.app.state.voice_provider
|
||||
|
||||
|
||||
def get_examiner(request: Request) -> ExaminerAgent:
|
||||
return request.app.state.examiner_agent
|
||||
|
||||
|
||||
def get_marketplace_store(request: Request) -> MarketplaceStore:
|
||||
"""v0.6 marketplace (D-049): routes NEVER import the store directly —
|
||||
the api/ layer composes it via app.state DI (territory rule)."""
|
||||
return request.app.state.marketplace_store
|
||||
|
||||
|
||||
def get_billing_store(request: Request) -> BillingStore:
|
||||
"""v0.9 billing (D-080): the same app.state DI territory rule —
|
||||
billing routes compose the store, never import it."""
|
||||
return request.app.state.billing_store
|
||||
|
||||
|
||||
def get_employer_store(request: Request) -> EmployerStore:
|
||||
"""v0.10 employers (D-087): the same app.state DI territory rule —
|
||||
employer routes compose the store via app.state, never import it."""
|
||||
return request.app.state.employer_store
|
||||
|
||||
|
||||
def get_embedding_provider(request: Request) -> EmbeddingProvider:
|
||||
"""v0.6 P3 (D-051): the embedding port via app.state DI — the
|
||||
search route + publish hooks compose it, never import it."""
|
||||
return request.app.state.embedding_provider
|
||||
|
||||
@@ -0,0 +1,367 @@
|
||||
"""/v1/employers — employer session + claim surface (REQ-10-001/002, D-087..D-097).
|
||||
|
||||
The employer principal (D-090): `require_employer_session` is the
|
||||
require_admin extract→verify→return shape (R-56) with a SINGLE
|
||||
header — `Authorization: Bearer <token>` via partition + scheme
|
||||
check. sha256(token) is looked up against `employer_sessions`
|
||||
(only the hash is ever stored, D-087); missing/expired/unknown →
|
||||
**401** (credential failure — the token namespaces cannot collide
|
||||
with admin tokens: this gate hash-looks-up, admin compares bytes);
|
||||
**403 is reserved** for cross-entity authorization (a valid session
|
||||
touching another employer's resource — P4).
|
||||
|
||||
D-097: employer routes NEVER compose the learner gates
|
||||
(require_verified_age / allowlist) — a different principal class.
|
||||
The account's email is UNVERIFIED and /me displays it as such
|
||||
(email_verified: Literal[False], no email verification exists).
|
||||
|
||||
Claims (P2, D-089): POST /v1/employers/claims targets an EXISTING
|
||||
marketplace EmployerRow (404 unknown); one PENDING claim per entity
|
||||
(409, the G-316 partial index backstops the race); an account that
|
||||
already holds an entity → 409 honest (one account : one entity);
|
||||
evidence is computed AT CLAIM TIME (claim_evidence.py, R-65) and
|
||||
stored as provenance JSON on the claim row. Auto-approval is
|
||||
DEFERRED (D-089: without email verification the match is forgeable;
|
||||
A-304: never present unverified as verified) — the operator decision
|
||||
activates. The marketplace store composes via app.state DI
|
||||
(get_marketplace_store — the territory rule), never a direct import.
|
||||
|
||||
Tokens: `secrets.token_urlsafe(32)` — 256-bit CSPRNG, handed to the
|
||||
client EXACTLY ONCE on login; only sha256(token) persists. TTL is
|
||||
lazy (G-306): an expired row reads as absent, no timer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import secrets
|
||||
import time
|
||||
from collections import deque
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException
|
||||
|
||||
from ..config import Settings
|
||||
from ..employers.base import (
|
||||
ClaimEmployerRequest,
|
||||
EmployerAccount,
|
||||
EmployerClaim,
|
||||
EmployerMe,
|
||||
LoginEmployerRequest,
|
||||
RegisterEmployerRequest,
|
||||
TokenResponse,
|
||||
)
|
||||
from ..employers.claim_evidence import compute_claim_evidence
|
||||
from ..employers.passwords import hash_password, verify_password
|
||||
from ..employers.store import AccountRow, ClaimRow, EmployerStore, SessionRow
|
||||
from .deps import get_employer_store, get_marketplace_store, get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1/employers/auth", tags=["employers-auth"])
|
||||
|
||||
#: Claims router (P2, D-089): sibling prefix under /v1/employers —
|
||||
#: the claim route is session-gated, not auth-prefixed.
|
||||
claims_router = APIRouter(prefix="/v1/employers", tags=["employers-claims"])
|
||||
|
||||
#: Per-EMAIL failed-login window (D-088): rolling 15 min (900 s —
|
||||
#: parameterized; identity hardcodes 60.0), monotonic timestamps,
|
||||
#: failures-only append, SUCCESS CLEARS the email's window. Keyed by
|
||||
#: the SUBMITTED EMAIL, never client IP (behind HAProxy the client
|
||||
#: host is the proxy — R-63). In-memory, process-local (D-019).
|
||||
LOGIN_FAILURE_WINDOW_S = 900.0
|
||||
_LOGIN_FAILURES: dict[str, list[float]] = {}
|
||||
|
||||
#: Global per-min cap on login+register attempts (D-088): module
|
||||
#: deque, popleft older-than-60s, `len >= cap` -> 429. G-314: the
|
||||
#: deque appends EVERY well-formed attempt — FAILED PASSWORDS COUNT
|
||||
#: (only 422-invalid bodies and the cap's own 429s are exempt, or
|
||||
#: the credential-stuffing bound is void). In-memory, process-local.
|
||||
_AUTH_ATTEMPTS: deque[float] = deque()
|
||||
|
||||
|
||||
def _check_auth_attempt_rate(settings: Settings) -> None:
|
||||
"""Global cap — the _check_global_create_rate shape (R-63):
|
||||
while-popleft older-than-60s, 429 at the knob, append only on
|
||||
admission (the cap's own 429 never consumes budget)."""
|
||||
now = time.monotonic()
|
||||
while _AUTH_ATTEMPTS and now - _AUTH_ATTEMPTS[0] > 60.0:
|
||||
_AUTH_ATTEMPTS.popleft()
|
||||
if len(_AUTH_ATTEMPTS) >= settings.employer_auth_attempts_per_min:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail="employer auth attempt rate exceeded — wait a minute",
|
||||
)
|
||||
_AUTH_ATTEMPTS.append(now)
|
||||
|
||||
|
||||
def _check_login_failures(email: str, settings: Settings) -> None:
|
||||
"""Per-email failed-login window — the _SubmitRateLimiter shape
|
||||
(R-63): slice-trim on monotonic timestamps, 429 at the knob with
|
||||
the honest "wait" detail. Failures only APPEND here (the caller
|
||||
appends after a failed verify); a successful login CLEARS the
|
||||
window (never locks a healthy user out over old failures)."""
|
||||
now = time.monotonic()
|
||||
window = _LOGIN_FAILURES.setdefault(email, [])
|
||||
window[:] = [
|
||||
t for t in window if now - t < LOGIN_FAILURE_WINDOW_S
|
||||
]
|
||||
if len(window) >= settings.employer_login_failures_per_window:
|
||||
wait = int(LOGIN_FAILURE_WINDOW_S / 60) # honest wait, minutes
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail=(
|
||||
f"too many failed logins for this email — wait ~{wait} "
|
||||
"minutes before trying again"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _record_login_failure(email: str) -> None:
|
||||
"""Failure-append (G-320a order: only AFTER the verify fails)."""
|
||||
_LOGIN_FAILURES.setdefault(email, []).append(time.monotonic())
|
||||
|
||||
|
||||
def _clear_login_failures(email: str) -> None:
|
||||
"""Successful login CLEARS the email's window (D-088)."""
|
||||
_LOGIN_FAILURES.pop(email, None)
|
||||
|
||||
|
||||
# -- session gate (D-090, R-56) ---------------------------------------------------
|
||||
|
||||
|
||||
def _extract_bearer_token(
|
||||
authorization: Annotated[str | None, Header(include_in_schema=False)] = None,
|
||||
) -> str | None:
|
||||
"""Single-header extractor: `Authorization: Bearer <token>`; None when
|
||||
the header is absent or carries no bearer candidate."""
|
||||
if not authorization:
|
||||
return None
|
||||
scheme, _, token = authorization.partition(" ")
|
||||
if scheme.strip().lower() == "bearer" and token.strip():
|
||||
return token.strip()
|
||||
return None
|
||||
|
||||
|
||||
def require_employer_session(
|
||||
request_token: Annotated[str | None, Depends(_extract_bearer_token)],
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
) -> tuple[SessionRow, AccountRow]:
|
||||
"""Session gate (D-090): sha256(token) hash-lookup — 401 on
|
||||
missing/unknown/expired tokens (credential failure; TTL evaluated
|
||||
lazily at read, G-306). 403 is RESERVED for cross-entity
|
||||
authorization, never raised here. Returns (session, account)."""
|
||||
if request_token is None:
|
||||
raise HTTPException(status_code=401, detail="employer session required")
|
||||
token_hash = hashlib.sha256(request_token.encode("utf-8")).hexdigest()
|
||||
session = store.get_session_by_token_hash(token_hash)
|
||||
if session is None:
|
||||
# Unknown OR expired — one honest 401, no enumeration difference.
|
||||
raise HTTPException(status_code=401, detail="invalid or expired session")
|
||||
account = store.get_account(session.account_id)
|
||||
if account is None: # pragma: no cover - FK discipline keeps rows paired
|
||||
raise HTTPException(status_code=401, detail="invalid or expired session")
|
||||
return session, account
|
||||
|
||||
|
||||
def _register_audit(
|
||||
store: EmployerStore, name: str, account_id: str | None = None
|
||||
) -> None:
|
||||
"""Auth audit append (the BillingEventRow shape). Never fails the
|
||||
parent route — the auth event already landed."""
|
||||
store.append_audit_event(
|
||||
"auth", name, account_id=account_id, detail={"route": name}
|
||||
)
|
||||
|
||||
|
||||
# -- routes ------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/register", status_code=201, response_model=None)
|
||||
async def register_employer(
|
||||
body: RegisterEmployerRequest,
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
settings: Annotated[Settings, Depends(get_settings)],
|
||||
) -> TokenResponse:
|
||||
"""Register: global cap -> scrypt hash -> insert_account; duplicate
|
||||
email → 409 honest. The account is UNVERIFIED (D-097). Registration
|
||||
logs in the new account (a fresh session rides the 201)."""
|
||||
_check_auth_attempt_rate(settings)
|
||||
_, created = store.insert_account(
|
||||
email=body.email, password_hash=hash_password(body.password)
|
||||
)
|
||||
if not created:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="an account with this email already exists",
|
||||
)
|
||||
account = store.get_account_by_email(body.email)
|
||||
# check-first just created it (the UNIQUE backstop would surface
|
||||
# IntegrityError otherwise — never swallowed by the store).
|
||||
assert account is not None
|
||||
_register_audit(store, "register", account_id=account.id)
|
||||
token = secrets.token_urlsafe(32)
|
||||
session = store.mint_session(account_id=account.id, token=token)
|
||||
assert session.expires_at is not None # mint always sets it
|
||||
return TokenResponse(token=token, expires_at=session.expires_at)
|
||||
|
||||
|
||||
@router.post("/login", response_model=TokenResponse)
|
||||
async def login_employer(
|
||||
body: LoginEmployerRequest,
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
settings: Annotated[Settings, Depends(get_settings)],
|
||||
) -> TokenResponse:
|
||||
"""Login: bad credentials → 401 honest with no user-enumeration
|
||||
difference (one 401 body for unknown-email and wrong-password
|
||||
alike).
|
||||
|
||||
G-320a route order: body-validation (pydantic, before the handler)
|
||||
→ global cap → per-email failed-login window → scrypt verify →
|
||||
failure-append. A 429'd attempt performs ZERO KDF derivations —
|
||||
the throttle bounds the open-risk-1 burst before any CPU burns."""
|
||||
_check_auth_attempt_rate(settings)
|
||||
_check_login_failures(body.email, settings)
|
||||
account = store.get_account_by_email(body.email)
|
||||
if account is None or not verify_password(account.password_hash, body.password):
|
||||
_record_login_failure(body.email)
|
||||
# The SAME 401 for unknown-email and wrong-password — the
|
||||
# response carries no account-existence signal (D-090).
|
||||
raise HTTPException(status_code=401, detail="invalid credentials")
|
||||
_clear_login_failures(body.email)
|
||||
token = secrets.token_urlsafe(32)
|
||||
session = store.mint_session(account_id=account.id, token=token)
|
||||
assert session.expires_at is not None # mint always sets it
|
||||
return TokenResponse(token=token, expires_at=session.expires_at)
|
||||
|
||||
|
||||
@router.post("/logout", response_model=None)
|
||||
async def logout_employer(
|
||||
request_token: Annotated[str | None, Depends(_extract_bearer_token)],
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
) -> dict:
|
||||
"""Logout: DELETE the session row + an audit event. Unknown or
|
||||
already-revoked token → 401 (never a silent 200 — the caller
|
||||
learns their token is not live)."""
|
||||
if request_token is None:
|
||||
raise HTTPException(status_code=401, detail="employer session required")
|
||||
token_hash = hashlib.sha256(request_token.encode("utf-8")).hexdigest()
|
||||
session = store.get_session_by_token_hash(token_hash)
|
||||
if session is None:
|
||||
raise HTTPException(status_code=401, detail="invalid or expired session")
|
||||
store.revoke_session(token_hash)
|
||||
_register_audit(store, "logout", account_id=session.account_id)
|
||||
return {"revoked": True}
|
||||
|
||||
|
||||
@router.get("/me", response_model=EmployerMe)
|
||||
async def employer_me(
|
||||
auth: Annotated[
|
||||
tuple[SessionRow, AccountRow], Depends(require_employer_session)
|
||||
],
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
) -> EmployerMe:
|
||||
"""The session's account (email UNVERIFIED, displayed as such —
|
||||
D-097/Literal[False]) + the claimed employer_id + the claim state
|
||||
(both None until P2's claim flow)."""
|
||||
_session, account = auth
|
||||
return EmployerMe(
|
||||
account=EmployerAccount(
|
||||
id=account.id,
|
||||
email=account.email,
|
||||
employer_id=account.employer_id,
|
||||
created_at=account.created_at, # type: ignore[arg-type]
|
||||
email_verified=False,
|
||||
),
|
||||
claimed_employer_id=account.employer_id,
|
||||
claim=None, # P2's claim flow fills this (D-089)
|
||||
)
|
||||
|
||||
|
||||
# -- claims (P2, D-089 — Task 2-1-03) ---------------------------------------------
|
||||
|
||||
|
||||
def _claim_to_contract(row: ClaimRow) -> EmployerClaim:
|
||||
"""Store row → the wire contract (created_at is always set on a
|
||||
row the store returned — insert/decide stamp it)."""
|
||||
assert row.created_at is not None # the store always stamps it
|
||||
return EmployerClaim(
|
||||
id=row.id,
|
||||
account_id=row.account_id,
|
||||
employer_id=row.employer_id,
|
||||
status=row.status, # type: ignore[arg-type]
|
||||
evidence=dict(row.evidence or {}),
|
||||
created_at=row.created_at,
|
||||
decided_by=row.decided_by,
|
||||
decided_at=row.decided_at,
|
||||
)
|
||||
|
||||
|
||||
@claims_router.post(
|
||||
"/claims", status_code=201, response_model=None
|
||||
)
|
||||
async def claim_employer_entity(
|
||||
body: ClaimEmployerRequest,
|
||||
auth: Annotated[
|
||||
tuple[SessionRow, AccountRow], Depends(require_employer_session)
|
||||
],
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
marketplace_store=Depends(get_marketplace_store),
|
||||
) -> EmployerClaim:
|
||||
"""Claim an existing marketplace EmployerRow (D-089): session-gated;
|
||||
404 unknown entity (claims target the v0.6 ingested EmployerRow
|
||||
data — resolved via the marketplace store's app.state DI, never a
|
||||
direct import); 409 when the account already holds an entity (one
|
||||
account : one entity) or a claim is already PENDING on the entity
|
||||
(G-316's partial index backstops the race — the IntegrityError
|
||||
maps to the SAME honest 409). Evidence is computed AT CLAIM TIME
|
||||
from the account email + the entity website (R-65) and stored as
|
||||
provenance JSON on the claim row; the response carries the pending
|
||||
claim with its evidence. Auto-approval is DEFERRED — the operator
|
||||
decision activates (D-089; A-304: never present unverified as
|
||||
verified)."""
|
||||
_session, account = auth
|
||||
employer = marketplace_store.get_employer(body.employer_id)
|
||||
if employer is None:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"no employer with id {body.employer_id!r}",
|
||||
)
|
||||
if account.employer_id is not None:
|
||||
# One account : one entity — the honest 409 (the account
|
||||
# already holds an entity; approving a second claim would be
|
||||
# refused by the store's UNIQUE constraint anyway).
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="account already holds an employer entity",
|
||||
)
|
||||
evidence = compute_claim_evidence(account.email, employer.website)
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
try:
|
||||
row, created = store.insert_claim(
|
||||
account_id=account.id,
|
||||
employer_id=body.employer_id,
|
||||
evidence=evidence,
|
||||
)
|
||||
except IntegrityError:
|
||||
# G-316: the raced insert refused by the partial UNIQUE index
|
||||
# — the SAME honest 409 as the check-first fast path.
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="a claim is already pending for this employer",
|
||||
) from None
|
||||
if not created:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="a claim is already pending for this employer",
|
||||
)
|
||||
store.append_audit_event(
|
||||
"claim",
|
||||
"claim.submitted",
|
||||
account_id=account.id,
|
||||
employer_id=body.employer_id,
|
||||
detail={
|
||||
"claim_id": row.id,
|
||||
"evidence_match": evidence["match"],
|
||||
},
|
||||
)
|
||||
return _claim_to_contract(row)
|
||||
@@ -0,0 +1,287 @@
|
||||
"""/v1/employers/admin — employer claim operator surface (REQ-10-002, D-089/D-094).
|
||||
|
||||
Every route sits behind `require_admin` IMPORTED from
|
||||
marketplace_admin (the billing_admin.py:27 precedent, D-094 — one
|
||||
guard, never reimplemented): fail-closed 403 on no/missing/invalid
|
||||
token — the R-56 matrix; an UNSET AI_ADMIN_TOKEN 403s every route
|
||||
even with a candidate (the main.py boot warning covers the config
|
||||
class). Operator routes compose NO learner gate (D-097) and no
|
||||
employer-session gate — a different principal class, and the token
|
||||
namespaces cannot collide (this gate compares bytes; the session
|
||||
gate hash-looks-up — D-090).
|
||||
|
||||
D-089: evidence is ADVISORY — the queue returns the stored claim-time
|
||||
provenance snapshot as-is; approval is the EXPLICIT operator decision
|
||||
(auto-approval deferred: without email verification the match is
|
||||
forgeable, A-304 — never present unverified as verified). Every
|
||||
decision appends an employer_audit_events row with operator
|
||||
provenance in the SAME transaction as the claim update (decide_claim
|
||||
— all-or-nothing). The accounts wire contract carries NO password_hash
|
||||
(hash-at-rest, D-087 — never returned anywhere).
|
||||
|
||||
409 discipline (the decide arms' honest constraint 409s — details
|
||||
carry constraint names only, never account identifiers):
|
||||
- claim not pending (decided twice) → "claim is already decided"
|
||||
- the deciding account already holds an entity (the one-account-
|
||||
one-entity guard) → "the claim's account already holds an employer
|
||||
entity"
|
||||
- the entity already owned (the employer_accounts.employer_id UNIQUE
|
||||
surfacing on the approve flush) → "the employer entity is already
|
||||
owned" — surfaced, NEVER silently overwritten.
|
||||
Any OTHER IntegrityError re-raises (a 500) — never mask an unknown
|
||||
constraint violation as a decision.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
from ..config import Settings
|
||||
from ..employers.base import EmployerClaim
|
||||
from ..employers.store import ClaimRow, EmployerStore
|
||||
from .deps import get_employer_store
|
||||
from .marketplace_admin import require_admin
|
||||
|
||||
router = APIRouter(prefix="/v1/employers/admin", tags=["employers-admin"])
|
||||
|
||||
#: Operator provenance stamped on every decision + audit event (the
|
||||
#: billing_admin "operator" precedent — route-level provenance; no
|
||||
#: per-operator accounts exist at pilot, D-094).
|
||||
OPERATOR = "operator"
|
||||
|
||||
|
||||
# -- contracts ----------------------------------------------------------------
|
||||
|
||||
|
||||
class AdminEmployerAccount(BaseModel):
|
||||
"""The accounts-list wire row (D-094): email, claimed entity,
|
||||
created_at, the UNVERIFIED marker. `email_verified` is
|
||||
`Literal[False]` (D-097 — structurally incapable of True: no email
|
||||
verification exists in v0.10, so the field can never carry it).
|
||||
password_hash is NEVER on the wire (D-087)."""
|
||||
|
||||
id: str
|
||||
email: str
|
||||
employer_id: str | None
|
||||
created_at: str
|
||||
#: D-097: the account is UNVERIFIED — displayed as such, never as
|
||||
#: verified (A-304's doctrine applied to the account surface).
|
||||
email_verified: Literal[False] = False
|
||||
|
||||
|
||||
class AdminAccountsResponse(BaseModel):
|
||||
accounts: list[AdminEmployerAccount]
|
||||
count: int
|
||||
|
||||
|
||||
class AdminClaimsResponse(BaseModel):
|
||||
"""The claim queue (D-089): status, the stored claim-time evidence
|
||||
snapshot — the domain-match signal rides evidence["match"],
|
||||
ADVISORY, never a verification — and the decided state
|
||||
(decided_by/decided_at, None while pending)."""
|
||||
|
||||
claims: list[EmployerClaim]
|
||||
count: int
|
||||
|
||||
|
||||
class ClaimDecisionResponse(BaseModel):
|
||||
"""The decided claim (status approved/rejected + operator
|
||||
provenance) — either decision arm returns the same shape."""
|
||||
|
||||
claim: EmployerClaim
|
||||
|
||||
|
||||
class AdminAuditEvent(BaseModel):
|
||||
"""The audit-trail wire row (D-089's queryable operator trail —
|
||||
the BillingEventRow_ shape, billing_admin.py:488)."""
|
||||
|
||||
id: int
|
||||
account_id: str | None
|
||||
employer_id: str | None
|
||||
kind: str
|
||||
name: str
|
||||
detail: dict[str, object]
|
||||
operator: str | None
|
||||
created_at: str | None
|
||||
|
||||
|
||||
class AdminAuditResponse(BaseModel):
|
||||
events: list[AdminAuditEvent]
|
||||
count: int
|
||||
|
||||
|
||||
def _claim_to_contract(row: ClaimRow) -> EmployerClaim:
|
||||
"""Store row → wire contract (created_at is always set on a row
|
||||
the store returned — insert/decide stamp it)."""
|
||||
assert row.created_at is not None # the store always stamps it
|
||||
return EmployerClaim(
|
||||
id=row.id,
|
||||
account_id=row.account_id,
|
||||
employer_id=row.employer_id,
|
||||
status=row.status, # type: ignore[arg-type]
|
||||
evidence=dict(row.evidence or {}),
|
||||
created_at=row.created_at,
|
||||
decided_by=row.decided_by,
|
||||
decided_at=row.decided_at,
|
||||
)
|
||||
|
||||
|
||||
# -- queue reads (D-094) ---------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/accounts", response_model=AdminAccountsResponse)
|
||||
def list_accounts(
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
settings: Annotated[Settings, Depends(require_admin)],
|
||||
) -> AdminAccountsResponse:
|
||||
"""The accounts list: email, claimed entity, created_at, and the
|
||||
UNVERIFIED marker (D-097 — account emails are unverified and
|
||||
displayed as such; a claimed entity column shows the APPROVED
|
||||
ownership binding, never account verification)."""
|
||||
rows = store.list_accounts()
|
||||
return AdminAccountsResponse(
|
||||
accounts=[
|
||||
AdminEmployerAccount(
|
||||
id=account.id,
|
||||
email=account.email,
|
||||
employer_id=account.employer_id,
|
||||
created_at=account.created_at.isoformat() if account.created_at else "",
|
||||
email_verified=False, # D-097: UNVERIFIED, always False
|
||||
)
|
||||
for account in rows
|
||||
],
|
||||
count=len(rows),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/claims", response_model=AdminClaimsResponse)
|
||||
def list_claims(
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
settings: Annotated[Settings, Depends(require_admin)],
|
||||
status: str | None = None,
|
||||
) -> AdminClaimsResponse:
|
||||
"""The claim queue: status, the stored evidence snapshot (the
|
||||
domain-match signal is ADVISORY — the signal informs, the operator
|
||||
decides; match=false is the example.com corpus norm, R-65), and
|
||||
the decided state. Optional ?status= facet (the admin queue's
|
||||
consume-free read path)."""
|
||||
rows = store.list_claims(status=status)
|
||||
return AdminClaimsResponse(
|
||||
claims=[_claim_to_contract(row) for row in rows],
|
||||
count=len(rows),
|
||||
)
|
||||
|
||||
|
||||
# -- decisions (D-089 — the explicit operator action) ----------------------------
|
||||
|
||||
|
||||
def _decide(store: EmployerStore, claim_id: str, decision: str) -> EmployerClaim:
|
||||
"""The shared decide path: unknown claim → 404; the store's guard
|
||||
arms + the employer_accounts.employer_id UNIQUE surface as the
|
||||
honest 409s (constraint names only — never account identifiers);
|
||||
decide_claim appends the operator audit event in the SAME
|
||||
transaction as the decision."""
|
||||
try:
|
||||
row = store.decide_claim(claim_id, decision=decision, operator=OPERATOR)
|
||||
except IntegrityError as exc:
|
||||
message = str(getattr(exc, "orig", exc))
|
||||
if "employer_accounts.employer_id already set" in message:
|
||||
# decide_claim's account-already-owner guard — the
|
||||
# one-account-one-entity arm, hand-raised by the store.
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=("the claim's account already holds an employer entity"),
|
||||
) from None
|
||||
if "UNIQUE constraint failed: employer_accounts.employer_id" in (message):
|
||||
# The approve flush refused: a DIFFERENT account already
|
||||
# owns the entity — surfaced, never silently overwritten.
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="the employer entity is already owned",
|
||||
) from None
|
||||
if "is already" in message:
|
||||
# decide_claim's not-pending guard — deciding a decided
|
||||
# claim, the IntegrityError shape (the honest 409).
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="claim is already decided",
|
||||
) from None
|
||||
# Unmapped IntegrityError — re-raise (a 500): never mask an
|
||||
# unknown constraint violation as a decision.
|
||||
raise
|
||||
if row is None:
|
||||
raise HTTPException(status_code=404, detail="unknown claim")
|
||||
return _claim_to_contract(row)
|
||||
|
||||
|
||||
@router.post("/claims/{claim_id}/approve", response_model=ClaimDecisionResponse)
|
||||
def approve_claim(
|
||||
claim_id: str,
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
settings: Annotated[Settings, Depends(require_admin)],
|
||||
) -> ClaimDecisionResponse:
|
||||
"""Approve → the claim's account.employer_id is set (one account :
|
||||
one entity — the UNIQUE activates) + decided_by/decided_at + the
|
||||
operator audit event, all in one transaction. Evidence never
|
||||
auto-activates anything: the operator decision is the action."""
|
||||
row = _decide(store, claim_id, "approved")
|
||||
return ClaimDecisionResponse(claim=row)
|
||||
|
||||
|
||||
@router.post("/claims/{claim_id}/reject", response_model=ClaimDecisionResponse)
|
||||
def reject_claim(
|
||||
claim_id: str,
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
settings: Annotated[Settings, Depends(require_admin)],
|
||||
) -> ClaimDecisionResponse:
|
||||
"""Reject → decided_by/decided_at + the operator audit event (same
|
||||
transaction); the entity is FREED for a new claim (the partial
|
||||
pending index never touches rejected rows, G-316)."""
|
||||
row = _decide(store, claim_id, "rejected")
|
||||
return ClaimDecisionResponse(claim=row)
|
||||
|
||||
|
||||
# -- audit trail (D-089's queryable operator trail) ------------------------------
|
||||
|
||||
|
||||
@router.get("/audit", response_model=AdminAuditResponse)
|
||||
def list_audit(
|
||||
store: Annotated[EmployerStore, Depends(get_employer_store)],
|
||||
settings: Annotated[Settings, Depends(require_admin)],
|
||||
account_id: str | None = None,
|
||||
employer_id: str | None = None,
|
||||
kind: str | None = None,
|
||||
name: str | None = None,
|
||||
limit: int = 100,
|
||||
) -> AdminAuditResponse:
|
||||
"""The queryable audit trail: every auth/claim event with operator
|
||||
provenance — the claim.submitted/claim.approved/claim.rejected
|
||||
names; decided_by/decided_at ride the claim rows the decision
|
||||
events reference (detail.claim_id joins them)."""
|
||||
rows = store.list_audit_events(
|
||||
account_id=account_id,
|
||||
employer_id=employer_id,
|
||||
kind=kind,
|
||||
name=name,
|
||||
limit=limit,
|
||||
)
|
||||
return AdminAuditResponse(
|
||||
events=[
|
||||
AdminAuditEvent(
|
||||
id=event.id or 0,
|
||||
account_id=event.account_id,
|
||||
employer_id=event.employer_id,
|
||||
kind=event.kind,
|
||||
name=event.name,
|
||||
detail=dict(event.detail or {}),
|
||||
operator=event.operator,
|
||||
created_at=event.created_at.isoformat() if event.created_at else None,
|
||||
)
|
||||
for event in rows
|
||||
],
|
||||
count=len(rows),
|
||||
)
|
||||
@@ -0,0 +1,265 @@
|
||||
"""Identity API + age-gate dependencies (REQ-5-003/004, D-042/43).
|
||||
|
||||
Flow (all under mock provider by default; A-304 mock markers ride every
|
||||
response so downstream surfaces never treat mock-verified as real):
|
||||
POST /v1/identity/submit submission → pending (G-13 caps first)
|
||||
GET /v1/identity/status/{lid} latest record + mock marker
|
||||
POST /v1/identity/verify/{sid} poll provider → terminal transition
|
||||
|
||||
Gate dependencies (D-043 binding composition order, mounted by the gated
|
||||
routes — variants/sandbox-create/defense-start for school 16+; the
|
||||
marketplace 18+ apply route in api/marketplace.py):
|
||||
allowlist (403, G-5 pilot guard) → identity verdict (403 + verify-CTA)
|
||||
→ rate caps (429, owned by the calling routes)
|
||||
|
||||
PII (A-305): raw DOB enters via the submission, is used to derive the
|
||||
band, and is NEVER stored or logged (caplog sentinel test pins it).
|
||||
|
||||
v0.6 (D-048/D-058): the G-18 marketplace stub MOVED to api/marketplace.py
|
||||
and became the real apply route; identity keeps ONLY the gate exports.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from ..config import Settings
|
||||
from ..identity.base import IdentityProvider, IdentitySubmission
|
||||
from ..identity.store import IdentityRecord, IdentityStore
|
||||
from .deps import get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1/identity", tags=["identity"])
|
||||
|
||||
|
||||
# -- DI ------------------------------------------------------------------------
|
||||
|
||||
|
||||
def get_identity_store(request: Request) -> IdentityStore:
|
||||
return request.app.state.identity_store
|
||||
|
||||
|
||||
def get_identity_provider(request: Request) -> IdentityProvider:
|
||||
return request.app.state.identity_provider
|
||||
|
||||
|
||||
# -- models ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class SubmitBody(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
#: ISO date. Validated AT THE BOUNDARY (D1 verifier fix): a malformed
|
||||
#: value would otherwise blow up as a 500 inside derive_age_band on the
|
||||
#: verify path — echoing the raw DOB into the traceback (A-305) and
|
||||
#: leaving a poisoned pending record that G-13 turns into a permanent
|
||||
#: learner lockout.
|
||||
date_of_birth: str = Field(description="ISO date; never stored or logged")
|
||||
document_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
@field_validator("date_of_birth")
|
||||
@classmethod
|
||||
def _validate_dob(cls, value: str) -> str:
|
||||
try:
|
||||
datetime.fromisoformat(value)
|
||||
except ValueError as exc:
|
||||
# 422 with the input scrubbed (A-305/D1 — the app-level
|
||||
# RequestValidationError handler redacts PII field inputs).
|
||||
raise ValueError("date_of_birth must be an ISO date (YYYY-MM-DD)") from exc
|
||||
return value
|
||||
|
||||
|
||||
class SubmitResponse(BaseModel):
|
||||
submission_id: str
|
||||
status: str
|
||||
#: A-304: honesty marker — a mock verdict is NEVER production-verified.
|
||||
mock: bool = True
|
||||
detail: str = ""
|
||||
|
||||
|
||||
class StatusResponse(BaseModel):
|
||||
learner_id: str
|
||||
status: str
|
||||
age_band: str | None
|
||||
mock: bool
|
||||
verified_at: str | None
|
||||
|
||||
|
||||
class VerifyResponse(SubmitResponse):
|
||||
age_band: str | None
|
||||
|
||||
|
||||
# -- G-13 submit caps ---------------------------------------------------------------
|
||||
|
||||
|
||||
class _SubmitRateLimiter:
|
||||
"""Per-learner submit rate cap (in-memory, process-local — the G-5
|
||||
creates-per-min pattern from the sandboxes route)."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._window: dict[str, list[float]] = {}
|
||||
|
||||
def check(self, learner_id: str, per_min: int) -> None:
|
||||
now = time.monotonic()
|
||||
window = self._window.setdefault(learner_id, [])
|
||||
window[:] = [t for t in window if now - t < 60.0]
|
||||
if len(window) >= per_min:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail="identity submit rate exceeded — wait a minute",
|
||||
)
|
||||
window.append(now)
|
||||
|
||||
|
||||
_rate_limiter = _SubmitRateLimiter()
|
||||
|
||||
|
||||
# -- verification flow ------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/submit", response_model=SubmitResponse)
|
||||
async def submit_identity(
|
||||
body: SubmitBody,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
provider: IdentityProvider = Depends(get_identity_provider),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> SubmitResponse:
|
||||
# G-13: one active pending submission per learner — resubmit while
|
||||
# pending echoes the pending state (409), not a second submission.
|
||||
if store.count_pending_for_learner(body.learner_id) > 0:
|
||||
latest = store.latest_for_learner(body.learner_id)
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail={
|
||||
"reason": "submission_pending",
|
||||
"submission_id": latest.id if latest else None,
|
||||
"status": "pending",
|
||||
},
|
||||
)
|
||||
_rate_limiter.check(body.learner_id, settings.identity_submits_per_min)
|
||||
|
||||
submission = IdentitySubmission(
|
||||
learner_id=body.learner_id,
|
||||
date_of_birth=body.date_of_birth,
|
||||
document_refs=body.document_refs,
|
||||
)
|
||||
submission_id = await provider.submit(submission)
|
||||
|
||||
record = store.insert(
|
||||
IdentityRecord(
|
||||
id=submission_id,
|
||||
learner_id=body.learner_id,
|
||||
status="pending",
|
||||
provider="mock" if settings.identity_provider == "mock" else settings.identity_provider,
|
||||
document_refs=body.document_refs, # A-305: opaque handles, never contents
|
||||
submitted_at=datetime.now(UTC),
|
||||
)
|
||||
)
|
||||
return SubmitResponse(
|
||||
submission_id=record.id, status=record.status, mock=record.mock
|
||||
)
|
||||
|
||||
|
||||
@router.get("/status/{learner_id}", response_model=StatusResponse)
|
||||
async def identity_status(
|
||||
learner_id: str,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
) -> StatusResponse:
|
||||
record = store.latest_for_learner(learner_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail="no identity record")
|
||||
return StatusResponse(
|
||||
learner_id=learner_id,
|
||||
status=record.status,
|
||||
age_band=record.age_band,
|
||||
mock=record.mock,
|
||||
verified_at=record.verified_at.isoformat() if record.verified_at else None,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/verify/{submission_id}", response_model=VerifyResponse)
|
||||
async def verify_identity(
|
||||
submission_id: str,
|
||||
store: IdentityStore = Depends(get_identity_store),
|
||||
provider: IdentityProvider = Depends(get_identity_provider),
|
||||
) -> VerifyResponse:
|
||||
record = store.get(submission_id)
|
||||
if record is None:
|
||||
raise HTTPException(status_code=404, detail="no such submission")
|
||||
if record.status != "pending":
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"submission already {record.status}",
|
||||
)
|
||||
|
||||
verdict = await provider.poll(submission_id)
|
||||
# A-305: derive the band from the provider's verdict; the raw DOB never
|
||||
# entered the store and is not logged here.
|
||||
updated = store.mark_verified(
|
||||
submission_id, verdict.model_dump(), verdict.age_band
|
||||
)
|
||||
assert updated is not None # record existed a moment ago
|
||||
return VerifyResponse(
|
||||
submission_id=updated.id,
|
||||
status=updated.status,
|
||||
age_band=updated.age_band,
|
||||
mock=updated.mock,
|
||||
detail=verdict.detail,
|
||||
)
|
||||
|
||||
|
||||
# -- age-gate dependencies (D-043 composition) -------------------------------------
|
||||
|
||||
|
||||
def _verify_cta_payload(
|
||||
reason: str, min_age: int, record: IdentityRecord | None
|
||||
) -> dict:
|
||||
"""A-306/UX acceptance #2: an actionable 403 — never a bare error."""
|
||||
return {
|
||||
"reason": reason,
|
||||
"min_age": min_age,
|
||||
"current_status": record.status if record else "none",
|
||||
"verify_cta": "/enroll",
|
||||
}
|
||||
|
||||
|
||||
async def require_verified_age(
|
||||
min_age: int,
|
||||
learner_id: str,
|
||||
store: IdentityStore,
|
||||
) -> IdentityRecord:
|
||||
"""The identity half of the D-043 composition (allowlist runs FIRST in
|
||||
the calling routes; this is the second gate; caps come after).
|
||||
|
||||
School 16+ → min_age=16; marketplace 18+ verified → min_age=18.
|
||||
"""
|
||||
record = store.latest_for_learner(learner_id)
|
||||
if record is None or record.status != "verified":
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("identity_verification_required", min_age, record),
|
||||
)
|
||||
# D2 (verifier): FAIL CLOSED. Only canonical bands can pass — None,
|
||||
# unknown, or under-16 bands reject (the gate is the security boundary
|
||||
# for the future vendor and direct store writes; it never trusts a
|
||||
# band it does not recognize).
|
||||
if record.age_band not in ("16-17", "18+"):
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("identity_verification_required", min_age, record),
|
||||
)
|
||||
if min_age > 16 and record.age_band != "18+":
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=_verify_cta_payload("age_gate_18_plus", 18, record),
|
||||
)
|
||||
return record
|
||||
|
||||
|
||||
#: Convenience alias for the marketplace 18+ composition (D-043's
|
||||
#: `require_verified_adult` — direct calls use require_verified_age(18, ...)).
|
||||
#: v0.6 (D-058): the marketplace surface itself moved to api/marketplace.py;
|
||||
#: identity keeps ONLY this gate export (D-048).
|
||||
require_verified_adult = require_verified_age
|
||||
@@ -1,27 +1,38 @@
|
||||
"""POST /v1/lab/feedback — SSE stream of Lab in-flow feedback (REQ-2-007).
|
||||
"""POST /v1/lab/feedback — SSE stream of Lab in-flow feedback (REQ-3-007).
|
||||
|
||||
D-016 envelope with agent=lab. Unknown scenario → 404 before streaming.
|
||||
v0.3 re-grounding: LIVE trace digest. Request carries {learner_id, task_id};
|
||||
the digest is computed from the learner's real TraceStore events (D-028)
|
||||
and handed to the Lab agent. No corpus scenarios. Empty/unknown trace is NOT
|
||||
an error — Lab gets a "no telemetry yet" timeline and coaches the baseline.
|
||||
D-016 envelope with agent=lab.
|
||||
"""
|
||||
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import cast
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel, Field
|
||||
from sse_starlette.sse import EventSourceResponse
|
||||
|
||||
from ..agents.lab import LabAgent
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from ..corpus.telemetry import get_lab_scenario
|
||||
from .deps import get_agent_registry, get_provider, get_settings
|
||||
from ..grading.features import compute_digest
|
||||
from .deps import (
|
||||
get_agent_registry,
|
||||
get_provider,
|
||||
get_settings,
|
||||
get_trace_store,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class LabFeedbackRequest(BaseModel):
|
||||
scenario_id: str = Field(min_length=1)
|
||||
learner_id: str | None = None
|
||||
learner_id: str = Field(min_length=1)
|
||||
task_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
@router.post("/lab/feedback")
|
||||
@@ -30,25 +41,29 @@ async def lab_feedback(
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
trace_store=Depends(get_trace_store),
|
||||
) -> EventSourceResponse:
|
||||
scenario = get_lab_scenario(body.scenario_id)
|
||||
if scenario is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown scenario {body.scenario_id!r}"
|
||||
)
|
||||
agent = registry.get(provider, settings, "lab")
|
||||
# P5: registry.get returns BaseAgent; "lab" is registered as LabAgent
|
||||
# (register_builtin_agents) — the cast states that mapping.
|
||||
agent = cast(LabAgent, registry.get(provider, settings, "lab"))
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
trace = (
|
||||
trace_store.get_trace(body.learner_id, body.task_id)
|
||||
if body.task_id in trace_store.list_tasks(body.learner_id)
|
||||
else []
|
||||
)
|
||||
digest = compute_digest(trace) if trace else None
|
||||
|
||||
async def event_stream() -> AsyncIterator[dict]:
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "meta",
|
||||
"agent": "lab",
|
||||
"scenario_id": body.scenario_id,
|
||||
"task_id": body.task_id,
|
||||
"model": settings.model,
|
||||
})}
|
||||
first_byte = True
|
||||
try:
|
||||
async for token in agent.stream_feedback(scenario, learner_context):
|
||||
async for token in agent.stream_feedback(digest, learner_context):
|
||||
first_byte = False
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "delta", "content": token
|
||||
|
||||
@@ -0,0 +1,540 @@
|
||||
"""/v1/marketplace — public postings API (REQ-6-001, D-048, D-058).
|
||||
|
||||
The board's runtime data source (published-only, honest 404s, no
|
||||
matchScore anywhere — G-106/G-107) + the REAL apply route: the v0.5
|
||||
placeholder that lived in api/identity.py moved here and became this
|
||||
module's /apply (D-058) — gate composition UNCHANGED (D-043 binding
|
||||
order): G-5 allowlist (403) → require_verified_adult (403 + verify-CTA)
|
||||
→ rate caps (429), then posting exists (404) and is published (409),
|
||||
then an idempotent insert (A-408: duplicate apply → 200 with the
|
||||
EXISTING application).
|
||||
|
||||
Routes never import the store directly — deps.py app.state DI only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from collections import deque
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request
|
||||
from pydantic import BaseModel, Field, ValidationError
|
||||
|
||||
from ..config import Settings
|
||||
from ..identity.store import IdentityStore
|
||||
from ..marketplace.base import Employer, Posting
|
||||
from ..marketplace.embedding.base import EmbeddingProvider
|
||||
from ..marketplace.enrichment import merge_employer
|
||||
from ..marketplace.resume import (
|
||||
MAX_RESUME_BYTES,
|
||||
ParsedProfile,
|
||||
ResumeFormatError,
|
||||
extract_resume_text,
|
||||
parse_resume,
|
||||
)
|
||||
from ..marketplace.search import FacetFilters, rank_postings
|
||||
from .deps import get_embedding_provider, get_marketplace_store, get_provider, get_settings
|
||||
from .identity import require_verified_age
|
||||
|
||||
router = APIRouter(prefix="/v1/marketplace", tags=["marketplace"])
|
||||
|
||||
|
||||
def _identity_store(request: Request) -> IdentityStore:
|
||||
return request.app.state.identity_store
|
||||
|
||||
|
||||
def _enforce_allowlist(learner_id: str, settings: Settings) -> None:
|
||||
"""G-5 pilot guard — allowlist runs FIRST in the composition (D-043)."""
|
||||
if learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=f"learner_id {learner_id!r} is not on the sandbox allowlist (G-5)",
|
||||
)
|
||||
|
||||
|
||||
# In-memory apply-rate window (monotonic timestamps, process-local) — the
|
||||
# G-5 creates-per-min pattern from the sandboxes route (module state is
|
||||
# acceptable: one process, one box, D-019/D-027 precedent).
|
||||
_APPLY_TIMES: dict[str, deque[float]] = {}
|
||||
|
||||
#: Applies per rolling 60s window per learner (A-408 anti-spam; the
|
||||
#: idempotent duplicate makes honest retries free — only DISTINCT
|
||||
#: posting applies consume budget).
|
||||
_APPLY_PER_LEARNER_PER_MIN = 5
|
||||
|
||||
|
||||
def _check_apply_rate(learner_id: str) -> None:
|
||||
now = time.monotonic()
|
||||
window = _APPLY_TIMES.setdefault(learner_id, deque())
|
||||
while window and now - window[0] > 60.0:
|
||||
window.popleft()
|
||||
if len(window) >= _APPLY_PER_LEARNER_PER_MIN:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail="marketplace apply rate exceeded — wait a minute",
|
||||
)
|
||||
window.append(now)
|
||||
|
||||
|
||||
# -- response contracts -------------------------------------------------------
|
||||
|
||||
|
||||
class PostingResponse(BaseModel):
|
||||
"""One posting over HTTP. G-107: `matchScore` is deliberately absent
|
||||
from the schema — scores are computed search-time context (P3),
|
||||
never list/detail payload fields."""
|
||||
|
||||
id: str
|
||||
source: str
|
||||
external_id: str
|
||||
employer_id: str
|
||||
title: str
|
||||
description: str
|
||||
required_competencies: list[str]
|
||||
skills: list[str]
|
||||
seniority: str | None
|
||||
category: str | None
|
||||
location: str
|
||||
remote: bool
|
||||
salary_min: int | None
|
||||
salary_max: int | None
|
||||
status: str
|
||||
tags: dict | None
|
||||
posted_at: str | None
|
||||
|
||||
|
||||
class PostingListResponse(BaseModel):
|
||||
"""G-103: the page + the honest total_published so the board's paged
|
||||
fetch-all can render "Showing X of Y" with an honest truncation. P2
|
||||
(MH-2c): facets — value counts over stored tags of published-only
|
||||
postings, data-driving the filter sidebar."""
|
||||
|
||||
postings: list[PostingResponse]
|
||||
total_published: int
|
||||
facets: dict
|
||||
|
||||
|
||||
class ProvenanceField(BaseModel):
|
||||
"""P4-A (D-053/A-304): one merged employer field with its honest
|
||||
provenance — raw | llm | manual (+ the model that wrote llm copy)."""
|
||||
|
||||
value: object
|
||||
source: str
|
||||
model: str | None = None
|
||||
|
||||
|
||||
class EmployerResponse(BaseModel):
|
||||
"""Raw ingested fields (P1) + the P4-A `merged` section: per-field
|
||||
provenance-marked view (manual > llm > raw on read, D-053)."""
|
||||
|
||||
id: str
|
||||
source: str
|
||||
external_id: str
|
||||
name: str
|
||||
logo: str
|
||||
description: str
|
||||
industry: str
|
||||
size: str
|
||||
location: str
|
||||
website: str
|
||||
social_links: dict[str, str]
|
||||
culture: str
|
||||
merged: dict[str, ProvenanceField] | None = None
|
||||
|
||||
|
||||
class SkillBreakdownEntry(BaseModel):
|
||||
"""One skill's deterministic explanation (MH-3f): tag-overlap with
|
||||
the query (0/1) + the skill's vector contribution share."""
|
||||
|
||||
skill: str
|
||||
overlap: int
|
||||
contribution: float
|
||||
|
||||
|
||||
class SearchEntry(BaseModel):
|
||||
"""P3 search result (G-106): `match_score` exists ONLY in this
|
||||
search context — a no-query facet listing carries None (honest
|
||||
absence, never a fabricated number)."""
|
||||
|
||||
posting: PostingResponse
|
||||
match_score: float | None
|
||||
skill_breakdown: list[SkillBreakdownEntry]
|
||||
|
||||
|
||||
class SearchResponse(BaseModel):
|
||||
results: list[SearchEntry]
|
||||
query: str
|
||||
ranked: int
|
||||
|
||||
|
||||
class ApplyBody(BaseModel):
|
||||
"""D-058: the learner identity comes from the VERIFIED record, not a
|
||||
client assertion — but the pilot surface passes learner_id in the
|
||||
body like every other gated route (G-5 composition unchanged)."""
|
||||
|
||||
learner_id: str = Field(min_length=1)
|
||||
posting_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class ApplyResponse(BaseModel):
|
||||
"""Honest confirmation (D-058): exactly the application state, no
|
||||
stub/mock markers, no fabricated outcome."""
|
||||
|
||||
application_id: str
|
||||
posting_id: str
|
||||
status: str
|
||||
|
||||
|
||||
def _posting_response(posting: Posting) -> PostingResponse:
|
||||
return PostingResponse(
|
||||
id=posting.id,
|
||||
source=posting.source,
|
||||
external_id=posting.external_id,
|
||||
employer_id=posting.employer_id,
|
||||
title=posting.title,
|
||||
description=posting.description,
|
||||
required_competencies=posting.required_competencies,
|
||||
skills=posting.skills,
|
||||
seniority=posting.seniority,
|
||||
category=posting.category,
|
||||
location=posting.location,
|
||||
remote=posting.remote,
|
||||
salary_min=posting.salary_min,
|
||||
salary_max=posting.salary_max,
|
||||
status=posting.status,
|
||||
tags=dict(posting.tags) if posting.tags else None,
|
||||
posted_at=posting.posted_at.isoformat() if posting.posted_at else None,
|
||||
)
|
||||
|
||||
|
||||
def _employer_response(
|
||||
employer: Employer, merged: dict[str, ProvenanceField] | None = None
|
||||
) -> EmployerResponse:
|
||||
return EmployerResponse(
|
||||
id=employer.id,
|
||||
source=employer.source,
|
||||
external_id=employer.external_id,
|
||||
name=employer.name,
|
||||
logo=employer.logo,
|
||||
description=employer.description,
|
||||
industry=employer.industry,
|
||||
size=employer.size,
|
||||
location=employer.location,
|
||||
website=employer.website,
|
||||
social_links=dict(employer.social_links),
|
||||
culture=employer.culture,
|
||||
merged=merged,
|
||||
)
|
||||
|
||||
|
||||
# -- public endpoints ---------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/postings", response_model=PostingListResponse)
|
||||
async def list_postings(
|
||||
store=Depends(get_marketplace_store),
|
||||
limit: int = Query(default=20, ge=1, le=100),
|
||||
offset: int = Query(default=0, ge=0),
|
||||
) -> PostingListResponse:
|
||||
"""Published postings only (draft/closed never board), posted_at DESC,
|
||||
with the honest total (G-103) + facets from published-only stored
|
||||
tags (MH-2c — facets computed once per page request, identical on
|
||||
every page of the fetch-all)."""
|
||||
postings, total = store.list_postings(limit=limit, offset=offset)
|
||||
return PostingListResponse(
|
||||
postings=[_posting_response(p) for p in postings],
|
||||
total_published=total,
|
||||
facets=store.facets(),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/search", response_model=SearchResponse)
|
||||
async def search_postings(
|
||||
store=Depends(get_marketplace_store),
|
||||
embedder: EmbeddingProvider = Depends(get_embedding_provider),
|
||||
q: str = Query(default=""),
|
||||
skills: list[str] = Query(default=[]),
|
||||
seniority: str | None = Query(default=None),
|
||||
category: str | None = Query(default=None),
|
||||
location: str | None = Query(default=None),
|
||||
remote: bool | None = Query(default=None),
|
||||
salary_min: int | None = Query(default=None, ge=0),
|
||||
salary_max: int | None = Query(default=None, ge=0),
|
||||
limit: int = Query(default=20, ge=1, le=100),
|
||||
) -> SearchResponse:
|
||||
"""P3 semantic search (REQ-6-003, D-051/A-405): facet hard-filters
|
||||
SQL-side FIRST, then cosine ranking DESC over the filtered set; no
|
||||
facets → pure semantic rank. `match_score` exists ONLY here (G-106)
|
||||
— an empty `q` with facets is a facet-only listing (posted_at DESC,
|
||||
match_score null — no query → no fabricated scores)."""
|
||||
filters = FacetFilters(
|
||||
skills=skills,
|
||||
seniority=seniority,
|
||||
category=category,
|
||||
location=location,
|
||||
remote=remote,
|
||||
salary_min=salary_min,
|
||||
salary_max=salary_max,
|
||||
)
|
||||
ranked = await rank_postings(store, embedder, q, filters, limit=limit)
|
||||
has_query = bool(q.strip())
|
||||
return SearchResponse(
|
||||
results=[
|
||||
SearchEntry(
|
||||
posting=_posting_response(r.posting),
|
||||
match_score=round(r.score, 6) if has_query else None,
|
||||
# P5 scoped: search.skill_breakdown() yields raw dicts in the
|
||||
# exact entry shape (marketplace/ may not import api/ models —
|
||||
# D-048 boundary); pydantic coerces them into
|
||||
# SkillBreakdownEntry at this SearchEntry boundary.
|
||||
skill_breakdown=r.skill_breakdown if has_query else [], # type: ignore[arg-type]
|
||||
)
|
||||
for r in ranked
|
||||
],
|
||||
query=q,
|
||||
ranked=len(ranked),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/postings/{posting_id}", response_model=PostingResponse)
|
||||
async def get_posting(
|
||||
posting_id: str,
|
||||
store=Depends(get_marketplace_store),
|
||||
) -> PostingResponse:
|
||||
"""Honest 404 on any unknown id — never a fabricated job."""
|
||||
posting = store.get_posting(posting_id)
|
||||
if posting is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no posting with id {posting_id!r}"
|
||||
)
|
||||
return _posting_response(posting)
|
||||
|
||||
|
||||
@router.get("/employers/{employer_id}", response_model=EmployerResponse)
|
||||
async def get_employer(
|
||||
employer_id: str,
|
||||
store=Depends(get_marketplace_store),
|
||||
) -> EmployerResponse:
|
||||
"""Honest 404 on unknown employers. Raw fields stay as-is; the P4-A
|
||||
`merged` section carries the per-field provenance view (manual >
|
||||
llm > raw, D-053/A-304 — LLM copy is visibly marked, never
|
||||
displayed as ingested fact)."""
|
||||
employer = store.get_employer(employer_id)
|
||||
if employer is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no employer with id {employer_id!r}"
|
||||
)
|
||||
rows = store.list_enrichment(employer_id)
|
||||
merged_raw = merge_employer(employer, rows) if rows else None
|
||||
merged = (
|
||||
{
|
||||
field: ProvenanceField(
|
||||
value=entry["value"], source=entry["source"], model=entry["model"]
|
||||
)
|
||||
for field, entry in merged_raw.items()
|
||||
}
|
||||
if merged_raw
|
||||
else None
|
||||
)
|
||||
return _employer_response(employer, merged=merged)
|
||||
|
||||
|
||||
# -- the real apply route (D-058; the G-18 stub's replacement) ------------------
|
||||
|
||||
|
||||
@router.post("/apply", response_model=ApplyResponse)
|
||||
async def apply_posting(
|
||||
body: ApplyBody,
|
||||
request: Request,
|
||||
store=Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> ApplyResponse:
|
||||
"""Apply to a posting. Gate composition (D-043, unchanged order):
|
||||
G-5 allowlist → require_verified_adult (18+ verified, verify-CTA
|
||||
403) → rate caps → posting exists (404) + published (409) →
|
||||
idempotent insert (A-408: a duplicate returns the EXISTING
|
||||
application with 200)."""
|
||||
_enforce_allowlist(body.learner_id, settings)
|
||||
await require_verified_age(18, body.learner_id, _identity_store(request))
|
||||
_check_apply_rate(body.learner_id)
|
||||
|
||||
posting = store.get_posting(body.posting_id)
|
||||
if posting is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no posting with id {body.posting_id!r}"
|
||||
)
|
||||
if posting.status != "published":
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail={
|
||||
"reason": "posting_not_published",
|
||||
"status": posting.status,
|
||||
},
|
||||
)
|
||||
|
||||
application, _created = store.insert_application(
|
||||
posting_id=body.posting_id, learner_id=body.learner_id
|
||||
)
|
||||
return ApplyResponse(
|
||||
application_id=application.id,
|
||||
posting_id=application.posting_id,
|
||||
status=application.status,
|
||||
)
|
||||
|
||||
|
||||
# -- P4-B: resume upload + confirm (REQ-6-005, D-054, A-410) --------------------
|
||||
|
||||
|
||||
class ResumeDraftResponse(BaseModel):
|
||||
"""The parse result — a DRAFT. Nothing persists without the explicit
|
||||
confirm call (the REQ-6-005 review gate)."""
|
||||
|
||||
status: str = "draft"
|
||||
profile: dict
|
||||
|
||||
|
||||
class ResumeConfirmBody(BaseModel):
|
||||
"""The learner's explicit save decision: the draft profile rides the
|
||||
body VERBATIM from the parse and the server RE-VALIDATES the shape
|
||||
against ParsedProfile before persisting (verifier P1 — a bare dict
|
||||
would accept arbitrary/type-confused payloads)."""
|
||||
|
||||
learner_id: str = Field(min_length=1)
|
||||
profile: dict
|
||||
|
||||
|
||||
class ResumeConfirmResponse(BaseModel):
|
||||
status: str = "confirmed"
|
||||
learner_id: str
|
||||
confirmed_at: str
|
||||
|
||||
|
||||
async def _require_verified_marketplace_learner(
|
||||
learner_id: str,
|
||||
settings: Settings,
|
||||
request: Request,
|
||||
) -> None:
|
||||
"""Learner-gated resume surface (D-043 composition order): G-5
|
||||
allowlist → verified identity → rate caps — same order as apply."""
|
||||
_enforce_allowlist(learner_id, settings)
|
||||
await require_verified_age(18, learner_id, _identity_store(request))
|
||||
_check_apply_rate(learner_id)
|
||||
|
||||
|
||||
@router.post("/resume", response_model=ResumeDraftResponse)
|
||||
async def upload_resume(
|
||||
request: Request,
|
||||
store=Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
) -> ResumeDraftResponse:
|
||||
"""Parse an uploaded resume into a profile DRAFT (multipart: `file`
|
||||
+ `learner_id` form field). Bounds (A-410): PDF/TXT/MD only, ≤2 MB,
|
||||
text-layer required (empty PDF extract → honest 422; no OCR in
|
||||
v0.6). Raw bytes never stored, never logged (A-305). The response
|
||||
is a DRAFT — the confirm route is the only persistence path."""
|
||||
|
||||
from starlette.datastructures import UploadFile as StarletteUploadFile
|
||||
|
||||
form = await request.form()
|
||||
try:
|
||||
learner_id = form.get("learner_id")
|
||||
if not isinstance(learner_id, str) or not learner_id:
|
||||
raise HTTPException(
|
||||
status_code=422, detail="learner_id form field required"
|
||||
)
|
||||
await _require_verified_marketplace_learner(learner_id, settings, request)
|
||||
|
||||
upload = form.get("file")
|
||||
# starlette's UploadFile base (fastapi's is a subclass — the raw
|
||||
# request.form() yields the base, so isinstance must use it).
|
||||
if not isinstance(upload, StarletteUploadFile):
|
||||
raise HTTPException(status_code=422, detail="file form field required")
|
||||
data = await upload.read()
|
||||
if len(data) > MAX_RESUME_BYTES:
|
||||
raise HTTPException(
|
||||
status_code=413,
|
||||
detail=f"resume too large ({len(data)} bytes; max {MAX_RESUME_BYTES})",
|
||||
)
|
||||
if len(data) == 0:
|
||||
raise HTTPException(status_code=422, detail="empty file")
|
||||
content_type = upload.content_type or ""
|
||||
try:
|
||||
text = extract_resume_text(data, content_type)
|
||||
except ResumeFormatError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
try:
|
||||
parsed = await parse_resume(provider, model=settings.model, text=text)
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=502,
|
||||
detail="profile extraction failed — try again or edit manually",
|
||||
) from exc
|
||||
draft = ResumeDraftResponse(
|
||||
status="draft",
|
||||
profile={
|
||||
"name": parsed.name,
|
||||
"headline": parsed.headline,
|
||||
"skills": list(parsed.skills),
|
||||
"experience_bands": [
|
||||
b.model_dump() for b in parsed.experience_bands
|
||||
],
|
||||
"competencies": list(parsed.competencies),
|
||||
},
|
||||
)
|
||||
finally:
|
||||
# P3 (Task 3-2-01): this route parses the form manually (raw
|
||||
# request.form()), so FastAPI's request-end auto-close — which it
|
||||
# pushes ONLY for param-injected multipart bodies — never runs
|
||||
# here. The multipart parser has already spooled every file open
|
||||
# by the time any guard raises (learner_id 422, allowlist 403,
|
||||
# verified 403, rate 429 — the pre-read guards included), so the
|
||||
# close must span ALL paths. form.close() is starlette's
|
||||
# canonical cleanup for manually-parsed forms: it closes every
|
||||
# UploadFile the form carries. The bytes are fully in `data`
|
||||
# before any raise (G-107: no streaming path exists); the
|
||||
# response is materialized in `draft` — nothing reads the
|
||||
# spooled file after this point.
|
||||
await form.close()
|
||||
return draft
|
||||
|
||||
|
||||
@router.post("/resume/confirm", response_model=ResumeConfirmResponse)
|
||||
async def confirm_resume(
|
||||
body: ResumeConfirmBody,
|
||||
request: Request,
|
||||
store=Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> ResumeConfirmResponse:
|
||||
"""The REQ-6-005 review gate: persist the learner-CONFIRMED profile
|
||||
to the learner_profile store table (G-104 — the D-027 family is the
|
||||
only persistence mechanism). G-5 → verified 18+ → rate caps, then
|
||||
a SHAPE RE-VALIDATION against ParsedProfile (verifier P1 — the
|
||||
confirm must never persist arbitrary/type-confused payloads the
|
||||
parse never produced), then an UPSERT (re-confirm replaces; the
|
||||
draft never persisted)."""
|
||||
await _require_verified_marketplace_learner(body.learner_id, settings, request)
|
||||
try:
|
||||
validated = ParsedProfile.model_validate(body.profile)
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail={
|
||||
"reason": "invalid_profile_shape",
|
||||
"message": "the profile must match the parsed draft shape",
|
||||
"errors": [
|
||||
{"loc": list(e.get("loc", [])), "msg": e.get("msg", "")}
|
||||
for e in exc.errors()
|
||||
],
|
||||
},
|
||||
) from exc
|
||||
stored = store.save_learner_profile(
|
||||
body.learner_id, validated.model_dump(mode="json"), model=settings.model
|
||||
)
|
||||
return ResumeConfirmResponse(
|
||||
status="confirmed",
|
||||
learner_id=stored["learner_id"],
|
||||
confirmed_at=stored["confirmed_at"].isoformat()
|
||||
if stored["confirmed_at"]
|
||||
else "",
|
||||
)
|
||||
@@ -0,0 +1,479 @@
|
||||
"""/v1/marketplace/admin — founder/operator surface (REQ-6-001, D-057).
|
||||
|
||||
Guard: `require_admin` compares X-Admin-Token OR Authorization: Bearer
|
||||
against Settings().admin_token via secrets.compare_digest (G-108).
|
||||
UNSET token → every admin route 403s — fail-closed, NEVER open (a loud
|
||||
boot warning in main.py names the fix, G-11). Admin routes do NOT pass
|
||||
the learner gates (different principal class, D-057/D-043 untouched).
|
||||
|
||||
Routes: create draft posting, publish/close lifecycle (invalid
|
||||
transition → 409), seeded-corpus ingest trigger, applications review
|
||||
(filterable by posting). No cron/crawler — ingest is founder-invoked
|
||||
(A-401/A-402).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import secrets
|
||||
import uuid
|
||||
from typing import Annotated, Any
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Request
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..config import Settings
|
||||
from ..llm.base import LLMProvider
|
||||
from ..marketplace.base import Posting
|
||||
from ..marketplace.embedding.base import EmbeddingProvider
|
||||
from ..marketplace.enrichment import ENRICHABLE_FIELDS, maybe_enrich_on_publish
|
||||
from ..marketplace.ingest import SeededCorpusSource, run_ingest
|
||||
from ..marketplace.search import embed_posting
|
||||
from ..marketplace.store import InvalidTransition, MarketplaceStore
|
||||
from ..marketplace.tagging import TaggingUnavailable, tag_posting
|
||||
from .deps import (
|
||||
get_embedding_provider,
|
||||
get_marketplace_store,
|
||||
get_provider,
|
||||
get_settings,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/v1/marketplace/admin", tags=["marketplace-admin"])
|
||||
|
||||
|
||||
def _extract_admin_token(
|
||||
x_admin_token: Annotated[str | None, Header(include_in_schema=False)] = None,
|
||||
authorization: Annotated[str | None, Header(include_in_schema=False)] = None,
|
||||
) -> str | None:
|
||||
"""Bearer of the request: X-Admin-Token wins; else Authorization:
|
||||
Bearer <token>. None when neither header carries a candidate."""
|
||||
if x_admin_token:
|
||||
return x_admin_token
|
||||
if authorization:
|
||||
scheme, _, token = authorization.partition(" ")
|
||||
if scheme.strip().lower() == "bearer" and token.strip():
|
||||
return token.strip()
|
||||
return None
|
||||
|
||||
|
||||
def require_admin(
|
||||
request_token: Annotated[str | None, Depends(_extract_admin_token)],
|
||||
settings: Annotated[Settings, Depends(get_settings)],
|
||||
) -> Settings:
|
||||
"""D-057 fail-closed guard: constant-time compare (G-108) against the
|
||||
env-only AI_ADMIN_TOKEN (SecretStr — never repr'd/logged); UNSET → 403
|
||||
for EVERY request (never open). Returns the settings for the route
|
||||
handlers.
|
||||
|
||||
Bytes compare: `secrets.compare_digest` raises TypeError on
|
||||
non-ASCII str candidates (final-review F-1 — an attacker-triggerable
|
||||
unhandled 500, fail-closed in outcome but noisy); comparing the
|
||||
UTF-8 encodings accepts ANY candidate bytes honestly (403 on
|
||||
mismatch) and stays constant-time."""
|
||||
configured = settings.admin_token.get_secret_value()
|
||||
if not configured or request_token is None:
|
||||
# Fail-closed: no configured token, or no candidate at all — the
|
||||
# surface is locked, not open (D-057).
|
||||
raise HTTPException(status_code=403, detail="admin token required")
|
||||
if not secrets.compare_digest(
|
||||
request_token.encode("utf-8", "surrogatepass"),
|
||||
configured.encode("utf-8"),
|
||||
):
|
||||
raise HTTPException(status_code=403, detail="invalid admin token")
|
||||
return settings
|
||||
|
||||
|
||||
# -- contracts ----------------------------------------------------------------
|
||||
|
||||
|
||||
class AdminPostingCreate(BaseModel):
|
||||
"""Admin-authored posting body. Starts as a DRAFT (G-102: only
|
||||
seed-port postings enter published); publish is an explicit call."""
|
||||
|
||||
employer_id: str = Field(min_length=1)
|
||||
title: str = Field(min_length=1)
|
||||
description: str = ""
|
||||
required_competencies: list[str] = Field(default_factory=list)
|
||||
skills: list[str] = Field(default_factory=list)
|
||||
seniority: str | None = None
|
||||
category: str | None = None
|
||||
location: str = ""
|
||||
remote: bool = False
|
||||
salary_min: int | None = None
|
||||
salary_max: int | None = None
|
||||
|
||||
|
||||
class AdminPostingResponse(BaseModel):
|
||||
id: str
|
||||
source: str
|
||||
external_id: str
|
||||
employer_id: str
|
||||
title: str
|
||||
status: str
|
||||
|
||||
|
||||
class IngestResponse(BaseModel):
|
||||
postings_upserted: int
|
||||
employers_upserted: int
|
||||
|
||||
|
||||
class AdminPostingEdit(BaseModel):
|
||||
"""P2 (Task 2-1-03): partial content edit; fields omitted stay put.
|
||||
A successful edit RE-TAGS the posting (tags are derived data,
|
||||
D-052 — the tagger overwrites)."""
|
||||
|
||||
title: str | None = Field(default=None, min_length=1)
|
||||
description: str | None = None
|
||||
skills: list[str] | None = None
|
||||
|
||||
|
||||
class AdminEnrichmentOverride(BaseModel):
|
||||
"""P4-A (D-053): manual-override body — appends a manual-source row
|
||||
(never mutates llm rows). `value` for social_links is a JSON object
|
||||
string; the other fields are plain text."""
|
||||
|
||||
employer_id: str = Field(min_length=1)
|
||||
field: str = Field(min_length=1)
|
||||
value: str = Field(min_length=1)
|
||||
|
||||
|
||||
class AdminApplication(BaseModel):
|
||||
id: str
|
||||
posting_id: str
|
||||
learner_id: str
|
||||
status: str
|
||||
created_at: str
|
||||
|
||||
|
||||
class AdminApplicationsResponse(BaseModel):
|
||||
applications: list[AdminApplication]
|
||||
|
||||
|
||||
# -- routes --------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/postings", status_code=201, response_model=AdminPostingResponse)
|
||||
async def create_posting(
|
||||
body: AdminPostingCreate,
|
||||
request: Request,
|
||||
store: MarketplaceStore = Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> AdminPostingResponse:
|
||||
"""Create a DRAFT posting (founder authoring path, D-050 admin_api
|
||||
adapter — source=admin). Publish is a separate lifecycle call.
|
||||
|
||||
v0.9 P4 (REQ-9-004): the employer's billing entitlement composes
|
||||
AFTER the admin gate — the package limit is enforced here (one
|
||||
post per create), honest 402 + upgrade-CTA payload when the
|
||||
employer has no active package or is out of posts. G-309: the
|
||||
seeded-corpus ingest (and any future adapter) bypasses this check
|
||||
by construction — enforcement exists ONLY in this admin route.
|
||||
|
||||
Final-review fix (fail-closed): the store is fetched through the
|
||||
SAME app.state DI the billing routes use (deps.get_billing_store's
|
||||
direct attribute read). The old getattr+hasattr soft-guard made a
|
||||
wiring miss silently SKIP the entitlement check — an unbilled
|
||||
posting. A payment gate must fail closed: no store → 503 with the
|
||||
fix-pointer, never an ungated 201.
|
||||
"""
|
||||
from ..billing.store import NoActivePackage, PackageExhausted
|
||||
|
||||
billing_store = getattr(request.app.state, "billing_store", None)
|
||||
if billing_store is None:
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail=(
|
||||
"billing entitlement store is unavailable — posting "
|
||||
"creation is refused until the service is restarted "
|
||||
"(the store wires at boot)"
|
||||
),
|
||||
)
|
||||
try:
|
||||
billing_store.consume_posting(body.employer_id)
|
||||
except NoActivePackage:
|
||||
raise HTTPException(
|
||||
status_code=402,
|
||||
detail={
|
||||
"reason": "no_active_package",
|
||||
"posts_used": 0,
|
||||
"posts_limit": 0,
|
||||
"upgrade_cta": (
|
||||
"This employer has no active posting package — "
|
||||
"purchase one via /v1/billing/admin/checkout."
|
||||
),
|
||||
},
|
||||
) from None
|
||||
except PackageExhausted:
|
||||
raise HTTPException(
|
||||
status_code=402,
|
||||
detail={
|
||||
"reason": "posting_limit_reached",
|
||||
"posts_used": 0,
|
||||
"posts_limit": 0,
|
||||
"upgrade_cta": (
|
||||
"This employer's posting package is exhausted — "
|
||||
"buy more posts or upgrade via /v1/billing/admin/checkout."
|
||||
),
|
||||
},
|
||||
) from None
|
||||
posting = Posting(
|
||||
id=f"adm-{uuid.uuid4().hex[:16]}",
|
||||
source="admin",
|
||||
external_id=f"adm-{uuid.uuid4().hex[:16]}",
|
||||
employer_id=body.employer_id,
|
||||
title=body.title,
|
||||
description=body.description,
|
||||
required_competencies=body.required_competencies,
|
||||
skills=body.skills,
|
||||
# P5: admin body carries str; Posting's Literal contract — the
|
||||
# store's validates hook rejects out-of-taxonomy values.
|
||||
seniority=body.seniority, # type: ignore[arg-type]
|
||||
category=body.category,
|
||||
location=body.location,
|
||||
remote=body.remote,
|
||||
salary_min=body.salary_min,
|
||||
salary_max=body.salary_max,
|
||||
status="draft",
|
||||
)
|
||||
try:
|
||||
stored = store.insert_posting(posting)
|
||||
except Exception as exc: # surfaced IntegrityError from the store
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
return AdminPostingResponse(
|
||||
id=stored.id,
|
||||
source=stored.source,
|
||||
external_id=stored.external_id,
|
||||
employer_id=stored.employer_id,
|
||||
title=stored.title,
|
||||
status=stored.status,
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/postings/{posting_id}/publish", response_model=AdminPostingResponse
|
||||
)
|
||||
async def publish_posting(
|
||||
posting_id: str,
|
||||
store: MarketplaceStore = Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
embedder: EmbeddingProvider = Depends(get_embedding_provider),
|
||||
provider: LLMProvider = Depends(get_provider),
|
||||
) -> AdminPostingResponse:
|
||||
"""draft → published (the only valid move from draft; anything else
|
||||
409s). Stamps posted_at NOW when the row has none (G-102: seed-port
|
||||
postings keep their original postedAt).
|
||||
|
||||
P3 embed-on-publish (Task 3-2-01): the published posting embeds
|
||||
now; failure = honest warning — the posting still publishes, the
|
||||
vector stays absent (search skips it).
|
||||
|
||||
P4-A enrich-on-first-publish (D-053): the employer's FIRST publish
|
||||
fires the one-shot profile extraction (no existing llm rows → run;
|
||||
re-publishing the same employer does not re-run). Failure = honest
|
||||
warning — the publish stands, no rows written."""
|
||||
try:
|
||||
stored = store.set_posting_status(posting_id, "published")
|
||||
except InvalidTransition as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
if stored is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no posting with id {posting_id!r}"
|
||||
)
|
||||
await embed_posting(store, embedder, stored)
|
||||
try:
|
||||
await maybe_enrich_on_publish(
|
||||
store, provider, model=settings.model, employer_id=stored.employer_id
|
||||
)
|
||||
except Exception: # pragma: no cover - maybe_enrich swallows; belt+braces
|
||||
logger.exception("enrich-on-publish failed; publish stands")
|
||||
return AdminPostingResponse(
|
||||
id=stored.id,
|
||||
source=stored.source,
|
||||
external_id=stored.external_id,
|
||||
employer_id=stored.employer_id,
|
||||
title=stored.title,
|
||||
status=stored.status,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/enrichment", status_code=201)
|
||||
async def manual_enrichment_override(
|
||||
body: AdminEnrichmentOverride,
|
||||
store: MarketplaceStore = Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> dict:
|
||||
"""P4-A (D-053): founder manual override — APPENDS a manual-source
|
||||
row (read precedence manual > llm > raw picks it up); llm rows are
|
||||
NEVER mutated. Valid fields: description | social_links | logo."""
|
||||
if body.field not in ENRICHABLE_FIELDS:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=f"field must be one of {ENRICHABLE_FIELDS}",
|
||||
)
|
||||
employer = store.get_employer(body.employer_id)
|
||||
if employer is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no employer with id {body.employer_id!r}"
|
||||
)
|
||||
row = store.append_enrichment(
|
||||
body.employer_id,
|
||||
field=body.field,
|
||||
value=body.value,
|
||||
source="manual",
|
||||
)
|
||||
return {"stored": True, "row": row}
|
||||
|
||||
|
||||
@router.post(
|
||||
"/postings/{posting_id}/close", response_model=AdminPostingResponse
|
||||
)
|
||||
async def close_posting(
|
||||
posting_id: str,
|
||||
store: MarketplaceStore = Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> AdminPostingResponse:
|
||||
"""published → closed (the only valid move from published; 409 on
|
||||
closing a draft or re-closing)."""
|
||||
try:
|
||||
stored = store.set_posting_status(posting_id, "closed")
|
||||
except InvalidTransition as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
if stored is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no posting with id {posting_id!r}"
|
||||
)
|
||||
return AdminPostingResponse(
|
||||
id=stored.id,
|
||||
source=stored.source,
|
||||
external_id=stored.external_id,
|
||||
employer_id=stored.employer_id,
|
||||
title=stored.title,
|
||||
status=stored.status,
|
||||
)
|
||||
|
||||
|
||||
@router.put("/postings/{posting_id}", response_model=AdminPostingResponse)
|
||||
async def edit_posting(
|
||||
posting_id: str,
|
||||
body: AdminPostingEdit,
|
||||
store: MarketplaceStore = Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
provider: LLMProvider = Depends(get_provider),
|
||||
) -> AdminPostingResponse:
|
||||
"""P2 (Task 2-1-03): edit title/description/skills → RE-TAG the
|
||||
posting (D-052: tags are derived data — the new extraction
|
||||
OVERWRITES the old tags column; old tags gone). Tag failure is
|
||||
honest: the edit persists, tags stay as they were, a warning logs."""
|
||||
updated = store.update_posting_fields(
|
||||
posting_id,
|
||||
title=body.title,
|
||||
description=body.description,
|
||||
skills=body.skills,
|
||||
)
|
||||
if updated is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no posting with id {posting_id!r}"
|
||||
)
|
||||
try:
|
||||
tags = await tag_posting(
|
||||
provider,
|
||||
model=settings.model,
|
||||
title=updated.title,
|
||||
description=updated.description,
|
||||
skills_hint=updated.skills,
|
||||
)
|
||||
updated = store.set_posting_tags(posting_id, tags) or updated
|
||||
except TaggingUnavailable:
|
||||
# Honest: the EDIT landed; tags keep their previous value (the
|
||||
# warning already logged in tagging.py). Never 500 on tag failure.
|
||||
pass
|
||||
return AdminPostingResponse(
|
||||
id=updated.id,
|
||||
source=updated.source,
|
||||
external_id=updated.external_id,
|
||||
employer_id=updated.employer_id,
|
||||
title=updated.title,
|
||||
status=updated.status,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/ingest", response_model=IngestResponse)
|
||||
async def ingest(
|
||||
store: MarketplaceStore = Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
provider: LLMProvider = Depends(get_provider),
|
||||
embedder: EmbeddingProvider = Depends(get_embedding_provider),
|
||||
) -> IngestResponse:
|
||||
"""Run the seeded corpus source now (founder-invoked — no scheduler,
|
||||
A-401/A-402). Idempotent on (source, external_id): a double-run
|
||||
yields identical rows (MH-1b). Every posting is TAGGED after
|
||||
upsert (P2, D-052) — a tag failure leaves the posting UNTAGGED with
|
||||
a warning, never a crashed ingest. Published seed postings EMBED
|
||||
after tagging (P3 embed-on-publish path, Task 3-2-01) — an embed
|
||||
failure is the same honest warning."""
|
||||
source = SeededCorpusSource()
|
||||
result = run_ingest(source, store)
|
||||
tagged = 0
|
||||
untagged = 0
|
||||
embedded = 0
|
||||
for posting in source.fetch():
|
||||
stored = store.get_posting(posting.id)
|
||||
if stored is None: # pragma: no cover - upsert just ran
|
||||
continue
|
||||
try:
|
||||
tags = await tag_posting(
|
||||
provider,
|
||||
model=settings.model,
|
||||
title=stored.title,
|
||||
description=stored.description,
|
||||
skills_hint=stored.skills,
|
||||
)
|
||||
store.set_posting_tags(stored.id, tags)
|
||||
stored = store.get_posting(stored.id) or stored
|
||||
tagged += 1
|
||||
except TaggingUnavailable:
|
||||
untagged += 1
|
||||
if stored.status == "published":
|
||||
# Embed-on-publish for the published seed postings (G-102).
|
||||
if await embed_posting(store, embedder, stored):
|
||||
embedded += 1
|
||||
if untagged:
|
||||
logger.warning(
|
||||
"marketplace ingest tagged %s/%s postings — %s left untagged "
|
||||
"(provider failures are honest None, never fabricated)",
|
||||
tagged,
|
||||
tagged + untagged,
|
||||
untagged,
|
||||
)
|
||||
return IngestResponse(
|
||||
postings_upserted=result.postings_upserted,
|
||||
employers_upserted=result.employers_upserted,
|
||||
)
|
||||
|
||||
|
||||
@router.get(
|
||||
"/applications", response_model=AdminApplicationsResponse
|
||||
)
|
||||
async def list_applications(
|
||||
posting_id: str | None = None,
|
||||
store: MarketplaceStore = Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(require_admin),
|
||||
) -> AdminApplicationsResponse:
|
||||
"""Applications review list (newest first), filterable by posting.
|
||||
The founder contacts employers out-of-band (no notification infra,
|
||||
A-408)."""
|
||||
rows: list[Any] = store.list_applications(posting_id=posting_id)
|
||||
return AdminApplicationsResponse(
|
||||
applications=[
|
||||
AdminApplication(
|
||||
id=a.id,
|
||||
posting_id=a.posting_id,
|
||||
learner_id=a.learner_id,
|
||||
status=a.status,
|
||||
created_at=a.created_at.isoformat() if a.created_at else "",
|
||||
)
|
||||
for a in rows
|
||||
]
|
||||
)
|
||||
@@ -0,0 +1,54 @@
|
||||
"""G-101 placeholder-shell serving (v0.6).
|
||||
|
||||
Next 15's static export emits exactly one placeholder page per dynamic
|
||||
marketplace route (`out/marketplace/jobs/_/index.html`). Under the
|
||||
single-port deploy (D-039) these routes serve that shell for ANY id so
|
||||
runtime-created postings stay reachable and every sitemap URL resolves
|
||||
(G-109). The shell client-fetches the real data by path id and
|
||||
notFound()s on 404.
|
||||
|
||||
Registered before the StaticFiles mount so the routes win (the same
|
||||
routes-beat-the-mount mechanism the sitemap relies on). In dev mode
|
||||
(web_static_dir unset) the Next dev server handles routing itself —
|
||||
no-op.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, FastAPI
|
||||
from fastapi.responses import FileResponse
|
||||
|
||||
from ..config import Settings
|
||||
|
||||
_SHELL_ROUTES = (
|
||||
("/marketplace/jobs/{job_id}", "marketplace/jobs/_/index.html"),
|
||||
("/marketplace/jobs/{job_id}/", "marketplace/jobs/_/index.html"),
|
||||
("/marketplace/employers/{employer_id}", "marketplace/employers/_/index.html"),
|
||||
("/marketplace/employers/{employer_id}/", "marketplace/employers/_/index.html"),
|
||||
)
|
||||
|
||||
|
||||
def register_marketplace_shell_routes(app: FastAPI, settings: Settings) -> None:
|
||||
web_dir = Path(str(settings.web_static_dir or ""))
|
||||
if str(settings.web_static_dir) in ("", ".") or not web_dir.is_dir():
|
||||
# Dev/no-mount boots: the Next dev server or a separate web origin
|
||||
# handles these paths. Nothing to serve here.
|
||||
return
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
for path, shell in _SHELL_ROUTES:
|
||||
|
||||
def _serve(shell_file: str = shell) -> FileResponse:
|
||||
page = web_dir / shell_file
|
||||
if page.is_file():
|
||||
return FileResponse(page, media_type="text/html")
|
||||
raise RuntimeError( # pragma: no cover - deploy misconfig
|
||||
f"marketplace shell missing: {page} — rebuild the web export"
|
||||
)
|
||||
|
||||
router.get(path, include_in_schema=False)(_serve)
|
||||
|
||||
app.include_router(router)
|
||||
@@ -1,25 +1,35 @@
|
||||
"""POST /v1/proctor/signals — structured integrity signals (REQ-2-009).
|
||||
"""POST /v1/proctor/signals — integrity signals over REAL inputs (REQ-3-007).
|
||||
|
||||
JSON response (not SSE): a pydantic-validated ProctorAssessment.
|
||||
Unknown scenario → 404. Coaching-shaped interventions only.
|
||||
v0.3 re-grounding: live trace digest + DefenseStore long-pause signals +
|
||||
variant seed cross-check, no corpus scenarios. The proctor coaches:
|
||||
a pydantic-validated ProctorAssessment (JSON response, not SSE).
|
||||
"""
|
||||
|
||||
from typing import cast
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..agents.proctor import ProctorAssessment
|
||||
from ..agents.proctor import ProctorAgent, ProctorAssessment
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from ..corpus.telemetry import get_proctor_scenario
|
||||
from .deps import get_agent_registry, get_provider, get_settings
|
||||
from ..grading.features import compute_digest
|
||||
from .deps import (
|
||||
get_agent_registry,
|
||||
get_provider,
|
||||
get_settings,
|
||||
get_trace_store,
|
||||
get_variant_store,
|
||||
get_voice_store,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class ProctorRequest(BaseModel):
|
||||
scenario_id: str = Field(min_length=1)
|
||||
learner_id: str | None = None
|
||||
learner_id: str = Field(min_length=1)
|
||||
task_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
@router.post("/proctor/signals", response_model=ProctorAssessment)
|
||||
@@ -28,16 +38,42 @@ async def proctor_signals(
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
trace_store=Depends(get_trace_store),
|
||||
variant_store=Depends(get_variant_store),
|
||||
voice_store=Depends(get_voice_store),
|
||||
) -> ProctorAssessment:
|
||||
scenario = get_proctor_scenario(body.scenario_id)
|
||||
if scenario is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown scenario {body.scenario_id!r}"
|
||||
)
|
||||
agent = registry.get(provider, settings, "proctor")
|
||||
"""Real integrity inputs: live digest + defense signals + variant context."""
|
||||
# P5: registry.get returns BaseAgent; "proctor" is registered as
|
||||
# ProctorAgent (register_builtin_agents) — the cast states that mapping.
|
||||
agent = cast(ProctorAgent, registry.get(provider, settings, "proctor"))
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
|
||||
trace = (
|
||||
trace_store.get_trace(body.learner_id, body.task_id)
|
||||
if body.task_id in trace_store.list_tasks(body.learner_id)
|
||||
else []
|
||||
)
|
||||
digest = compute_digest(trace) if trace else None
|
||||
|
||||
defense_signals = None
|
||||
for record in voice_store.list_for_learner(body.learner_id):
|
||||
if record.task_id == body.task_id and record.status == "finished":
|
||||
defense_signals = record.integrity_signals or None
|
||||
break
|
||||
|
||||
variant = variant_store.get_by_task(body.task_id)
|
||||
variant_context = (
|
||||
{"template_id": variant.template_id, "seed": variant.seed, "params": variant.params}
|
||||
if variant is not None
|
||||
else None
|
||||
)
|
||||
try:
|
||||
return await agent.assess(scenario, learner_context)
|
||||
return await agent.assess(
|
||||
digest,
|
||||
defense_signals=defense_signals,
|
||||
variant_context=variant_context,
|
||||
learner_context=learner_context,
|
||||
)
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=502, detail=f"proctor assessment failed: {exc}"
|
||||
|
||||
@@ -23,11 +23,14 @@ never part of the API contract.
|
||||
|
||||
import time
|
||||
from collections import deque
|
||||
from collections.abc import Set as AbstractSet
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Response
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from ..config import Settings
|
||||
from ..sandbox.capability import DEGRADE_REMEDIATION, sandbox_fabric_degraded
|
||||
from ..sandbox.manager import (
|
||||
PoolFullError,
|
||||
SandboxHandleInfo,
|
||||
@@ -35,7 +38,9 @@ from ..sandbox.manager import (
|
||||
SandboxNotFoundError,
|
||||
)
|
||||
from ..sandbox.workdir import SandboxDir
|
||||
from ..variants.templates import get_template
|
||||
from .deps import get_sandbox_manager, get_sandbox_test_layout, get_settings
|
||||
from .identity import require_verified_age
|
||||
|
||||
router = APIRouter(prefix="/v1/sandboxes", tags=["sandboxes"])
|
||||
|
||||
@@ -50,6 +55,9 @@ _CREATE_TIMES: deque[float] = deque()
|
||||
|
||||
class SandboxCreateRequest(BaseModel):
|
||||
learner_id: str = Field(min_length=1)
|
||||
# Optional task key: when set, the sandbox is telemetry-wired (REQ-3-003)
|
||||
# — the in-sandbox capture agent streams workspace events to the ingest.
|
||||
task_id: str | None = None
|
||||
|
||||
|
||||
class SandboxResponse(BaseModel):
|
||||
@@ -135,14 +143,33 @@ def _check_global_create_rate(settings: Settings) -> None:
|
||||
@router.post("", status_code=201, response_model=SandboxResponse)
|
||||
async def create_sandbox(
|
||||
body: SandboxCreateRequest,
|
||||
request: Request,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> SandboxResponse:
|
||||
# v0.5 identity gate (D-043, REQ-5-004): allowlist (G-5) → identity
|
||||
# verdict (403 + verify-CTA) → caps (429) — the binding composition.
|
||||
_enforce_allowlist(body.learner_id, settings)
|
||||
await require_verified_age(
|
||||
16, body.learner_id, request.app.state.identity_store
|
||||
)
|
||||
_check_per_learner_cap(await manager.list(), body.learner_id, settings)
|
||||
_check_global_create_rate(settings)
|
||||
# v0.8 honest degrade (D-068/G-208): when the namespace fabric cannot
|
||||
# work on this host (unprivileged container under docker's default
|
||||
# seccomp), env creation is rejected BEFORE the manager is touched —
|
||||
# same 503 class as PoolFullError, carrying the shared remediation
|
||||
# constant (the boot notice logs the same text; they cannot drift).
|
||||
# Structural backends (test stubs — no userns involved) declare
|
||||
# `structural = True` and are exempt: the stub-backed suite must stay
|
||||
# green inside job containers (D-063) while the REAL fabric degrades
|
||||
# honestly everywhere.
|
||||
manager_backend = getattr(manager, "_backend", None)
|
||||
structural = getattr(manager_backend, "structural", False)
|
||||
if not structural and sandbox_fabric_degraded():
|
||||
raise HTTPException(status_code=503, detail=DEGRADE_REMEDIATION)
|
||||
try:
|
||||
info = await manager.create(body.learner_id)
|
||||
info = await manager.create(body.learner_id, task_id=body.task_id)
|
||||
except PoolFullError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
return _to_response(info)
|
||||
@@ -217,3 +244,194 @@ async def delete_sandbox(
|
||||
if test_layout is not None:
|
||||
result.headers["X-Workspace-Copy"] = str(test_layout.workspace)
|
||||
return result
|
||||
|
||||
|
||||
# -- workspace files + exec (Phase 6, REQ-3-008; CUT-2) -------------------------
|
||||
#
|
||||
# The build surface reads/writes/list workspace files and runs Run/Test
|
||||
# commands through the manager's backend. NO interactive shell relay (CUT-2:
|
||||
# keystroke-level stdin/stdout is v0.4) — each exec is a bounded command with
|
||||
# captured output. Paths are WORKSPACE-RELATIVE; traversal outside the
|
||||
# workspace is rejected (the workdir bind is the boundary, but the API adds
|
||||
# its own containment check — defense in depth).
|
||||
|
||||
|
||||
class FileWriteRequest(BaseModel):
|
||||
path: str = Field(min_length=1)
|
||||
content: str
|
||||
|
||||
|
||||
class ExecRequest(BaseModel):
|
||||
cmd: list[str] = Field(min_length=1)
|
||||
|
||||
|
||||
class ExecResponse(BaseModel):
|
||||
cmd: list[str]
|
||||
returncode: int
|
||||
stdout: str
|
||||
stderr: str
|
||||
duration_s: float
|
||||
|
||||
|
||||
async def _workspace_dir(manager: SandboxManager, sandbox_id: str):
|
||||
"""Resolve the sandbox workspace (tracked layout or shell layout)."""
|
||||
info = await manager.get(sandbox_id) # raises SandboxNotFoundError -> 404
|
||||
backend = manager._backend
|
||||
tracked = getattr(backend, "_tracked", {}).get(sandbox_id)
|
||||
if tracked is not None:
|
||||
return tracked.workspace, info
|
||||
return info.workdir / "workspace", info
|
||||
|
||||
|
||||
def _safe_rel_path(raw: str) -> Path:
|
||||
"""Workspace-relative path; reject absolute/traversal paths."""
|
||||
candidate = Path(raw)
|
||||
if candidate.is_absolute() or ".." in candidate.parts:
|
||||
raise HTTPException(status_code=422, detail=f"invalid workspace path {raw!r}")
|
||||
return candidate
|
||||
|
||||
|
||||
def _resolve_in_workspace(workspace: Path, rel: Path) -> Path:
|
||||
"""Resolve `rel` under `workspace`, refusing symlink escapes (P7).
|
||||
|
||||
The lexical check in `_safe_rel_path` cannot see symlinks: an exec can
|
||||
plant `ln -s /etc target` in the workspace and a follow-up read/write
|
||||
would follow it OUT of the bind. Resolve with the workspace as the
|
||||
anchor (strict: a symlink chain escaping raises) and confirm the
|
||||
normalized target still sits inside the workspace — defense in depth
|
||||
for both read_file and write_file.
|
||||
"""
|
||||
try:
|
||||
target = (workspace / rel).resolve(strict=False)
|
||||
target.relative_to(workspace.resolve(strict=False))
|
||||
except ValueError:
|
||||
raise HTTPException(
|
||||
status_code=422, detail=f"path escapes the workspace: {rel.as_posix()!r}"
|
||||
) from None
|
||||
return target
|
||||
|
||||
|
||||
@router.get("/{sandbox_id}/files")
|
||||
async def list_files(
|
||||
sandbox_id: str,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
) -> dict:
|
||||
try:
|
||||
workspace, _ = await _workspace_dir(manager, sandbox_id)
|
||||
except SandboxNotFoundError:
|
||||
raise HTTPException(status_code=404, detail=f"no sandbox {sandbox_id!r}") from None
|
||||
return {"files": sorted(p.name for p in workspace.iterdir()) if workspace.is_dir() else []}
|
||||
|
||||
|
||||
@router.get("/{sandbox_id}/files/{path:path}")
|
||||
async def read_file(
|
||||
sandbox_id: str,
|
||||
path: str,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
) -> dict:
|
||||
try:
|
||||
workspace, _ = await _workspace_dir(manager, sandbox_id)
|
||||
except SandboxNotFoundError:
|
||||
raise HTTPException(status_code=404, detail=f"no sandbox {sandbox_id!r}") from None
|
||||
rel = _safe_rel_path(path)
|
||||
target = _resolve_in_workspace(workspace, rel)
|
||||
if not target.is_file():
|
||||
raise HTTPException(status_code=404, detail=f"no file {path!r}")
|
||||
return {"path": path, "content": target.read_text(errors="replace")}
|
||||
|
||||
|
||||
@router.put("/{sandbox_id}/files/{path:path}")
|
||||
async def write_file(
|
||||
sandbox_id: str,
|
||||
path: str,
|
||||
body: FileWriteRequest,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
) -> dict:
|
||||
try:
|
||||
workspace, _ = await _workspace_dir(manager, sandbox_id)
|
||||
except SandboxNotFoundError:
|
||||
raise HTTPException(status_code=404, detail=f"no sandbox {sandbox_id!r}") from None
|
||||
rel = _safe_rel_path(body.path)
|
||||
target = _resolve_in_workspace(workspace, rel)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(body.content)
|
||||
return {"path": body.path, "written": True}
|
||||
|
||||
|
||||
#: G-15 (REQ-5-005): per-kind exec command policy — EXACT argv[0] token
|
||||
#: matching, never prefix/substring (trivially bypassed via flags/-c
|
||||
#: passthrough). 'sh -c' passthrough is DISALLOWED for design/simulation
|
||||
#: kinds: the gaming vector would be faking build-style test cycles into a
|
||||
#: kind-agnostic digest. Build kinds keep v0.3 behavior (any command —
|
||||
#: the CUT-2 surface is Run/Test buttons, not a shell relay).
|
||||
#: python (bare) is deliberately absent — in-ns PATH resolves only python3
|
||||
#: (verifier P1); pip is absent (no network in the namespace).
|
||||
_GENERIC_FIRST_TOKENS = frozenset(
|
||||
{"ls", "cat", "pwd", "echo", "python3", "pytest"}
|
||||
)
|
||||
|
||||
|
||||
def _enforce_exec_policy(
|
||||
# P5: AbstractSet[str] — the caller unions frozenset + set; membership
|
||||
# is all this function does with it.
|
||||
cmd: list[str], environment: str | None, allowed: AbstractSet[str] | None = None
|
||||
) -> None:
|
||||
"""422 with the allowed set when a design/sim command is out of policy.
|
||||
|
||||
`allowed` defaults to the generic set; the exec route unions in the
|
||||
template's DECLARED harness argv[0] (a future non-python harness
|
||||
template must not reject its own Run command)."""
|
||||
if environment not in ("design", "simulation"):
|
||||
return # build kind: unchanged v0.3 semantics
|
||||
if allowed is None:
|
||||
allowed = _GENERIC_FIRST_TOKENS
|
||||
first = cmd[0] if cmd else ""
|
||||
if first in ("sh", "bash"):
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"shell passthrough is not allowed in a {environment} "
|
||||
f"environment; allowed commands: {sorted(allowed)}"
|
||||
),
|
||||
)
|
||||
if first not in allowed:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
f"command {first!r} is not allowed in a {environment} "
|
||||
f"environment; allowed commands: {sorted(allowed)}"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{sandbox_id}/exec", response_model=ExecResponse)
|
||||
async def exec_command(
|
||||
sandbox_id: str,
|
||||
body: ExecRequest,
|
||||
request: Request,
|
||||
manager: SandboxManager = Depends(get_sandbox_manager),
|
||||
) -> ExecResponse:
|
||||
try:
|
||||
await manager.get(sandbox_id)
|
||||
except SandboxNotFoundError:
|
||||
raise HTTPException(status_code=404, detail=f"no sandbox {sandbox_id!r}") from None
|
||||
backend = manager._backend
|
||||
handle = manager._handles.get(sandbox_id)
|
||||
if handle is None:
|
||||
raise HTTPException(status_code=404, detail=f"no live handle {sandbox_id!r}")
|
||||
# REQ-5-005 (G-15): resolve the sandbox's variant environment by its
|
||||
# task_id (manager side-table) and enforce the per-kind command policy
|
||||
# BEFORE execution.
|
||||
task_id = manager._task_ids.get(sandbox_id)
|
||||
if task_id:
|
||||
variant = request.app.state.variant_store.get_by_task(task_id)
|
||||
if variant is not None:
|
||||
template = get_template(variant.template_id)
|
||||
declared = (
|
||||
{template.run_command.split()[0]} if template is not None else set()
|
||||
)
|
||||
_enforce_exec_policy(
|
||||
body.cmd, variant.environment, _GENERIC_FIRST_TOKENS | declared
|
||||
)
|
||||
result = await backend.exec(handle, body.cmd)
|
||||
return ExecResponse(**result.model_dump())
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
"""Dynamic sitemap.xml + robots.txt (REQ-6-006, D-056, A-411).
|
||||
|
||||
Next `app/sitemap.ts` cannot work under `output: 'export'` — it
|
||||
enumerates at BUILD time and cannot see runtime postings. These plain
|
||||
FastAPI routes enumerate at REQUEST time: the static page list (a code
|
||||
constant) + every PUBLISHED posting (via the G-101 catch-all shells,
|
||||
each URL resolves — G-109, the sitemap never advertises what the
|
||||
deploy 404s).
|
||||
|
||||
Mount order (probe 6, verified): API routes beat the StaticFiles("/")
|
||||
export mount, so these win in single-port production. robots.txt
|
||||
points at the sitemap; per the robots spec the Sitemap directive must
|
||||
be an ABSOLUTE URL.
|
||||
|
||||
Unset AI_PUBLIC_BASE_URL (G-105): loud G-11 boot warning (main.py);
|
||||
the sitemap serves the EMPTY static-page set with a warning comment
|
||||
(never malformed or relative URLs) and robots omits the Sitemap
|
||||
directive; fail honest at request time; boot never crashes.
|
||||
|
||||
Pilot scale bound (A-403): postings are enumerated with limit=5000 —
|
||||
the tail beyond 5000 published postings is NOT listed (no sitemap
|
||||
pagination in v0.6; a sitemap index lands with the third adapter).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from xml.sax.saxutils import escape as _xml_escape
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi.responses import Response
|
||||
|
||||
from ..config import Settings
|
||||
from .deps import get_marketplace_store, get_settings
|
||||
|
||||
router = APIRouter(include_in_schema=False)
|
||||
|
||||
#: The static page set (A-411 build-time-static pages) — code constant;
|
||||
#: runtime postings append at request time. G-109: every path is the
|
||||
#: page's real export path (the pricing page exports under
|
||||
#: /marketplace/pricing/, never /pricing/).
|
||||
STATIC_PAGES: tuple[str, ...] = (
|
||||
"/",
|
||||
"/catalog",
|
||||
"/marketplace",
|
||||
"/marketplace/pricing",
|
||||
)
|
||||
|
||||
|
||||
def _origin(settings: Settings) -> str | None:
|
||||
raw = (settings.public_base_url or "").strip().rstrip("/")
|
||||
if not raw:
|
||||
return None
|
||||
if not raw.startswith(("http://", "https://")):
|
||||
return None # malformed origin — never emit a bad absolute URL
|
||||
return raw
|
||||
|
||||
|
||||
@router.get("/sitemap.xml")
|
||||
async def sitemap(
|
||||
store=Depends(get_marketplace_store),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> Response:
|
||||
"""Published postings + the static page set, as absolute URLs from
|
||||
AI_PUBLIC_BASE_URL. Unset origin → the honest empty set (G-105)."""
|
||||
origin = _origin(settings)
|
||||
if origin is None:
|
||||
# G-105: fail honest — never relative/malformed URLs in a
|
||||
# sitemap (spec requires absolute).
|
||||
return Response(
|
||||
content="<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
|
||||
"<!-- sitemap disabled: AI_PUBLIC_BASE_URL is unset — "
|
||||
"set the deployed origin to serve this sitemap -->\n"
|
||||
"<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">\n"
|
||||
"</urlset>\n",
|
||||
media_type="application/xml",
|
||||
)
|
||||
urls: list[str] = []
|
||||
for page in STATIC_PAGES:
|
||||
urls.append(f"{origin}{page}")
|
||||
# Runtime postings — every published one resolves via the G-101
|
||||
# catch-all shells (trailingSlash: true → /marketplace/jobs/{id}/).
|
||||
# Pilot cap (A-403, ≤10⁴ postings): a single sitemap page holds
|
||||
# ≤50,000 URLs; beyond 5000 postings the tail is dropped — the
|
||||
# honest bound recorded in the module docstring (no pagination in
|
||||
# v0.6; a sitemap index lands with the 3M-corpus adapter).
|
||||
postings, _total = store.list_postings(limit=5000, offset=0)
|
||||
for posting in postings:
|
||||
urls.append(f"{origin}/marketplace/jobs/{posting.id}/")
|
||||
body = ["<?xml version=\"1.0\" encoding=\"UTF-8\"?>"]
|
||||
body.append(
|
||||
'<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">'
|
||||
)
|
||||
for url in urls:
|
||||
# XML-escape the loc text — the origin is founder-set env, but
|
||||
# defense-in-depth: metacharacters in it must never break out
|
||||
# of the <loc> element (verified: raw <script> would otherwise
|
||||
# inject into the response body).
|
||||
body.append("<url>")
|
||||
body.append(f"<loc>{_xml_escape(url)}</loc>")
|
||||
body.append("</url>")
|
||||
body.append("</urlset>")
|
||||
return Response(content="\n".join(body) + "\n", media_type="application/xml")
|
||||
|
||||
|
||||
@router.get("/robots.txt")
|
||||
async def robots(settings: Settings = Depends(get_settings)) -> Response:
|
||||
"""Crawl policy: allow all; the Sitemap directive is ABSOLUTE
|
||||
(spec) — omitted entirely when the origin is unset (G-105)."""
|
||||
lines = ["User-agent: *", "Allow: /"]
|
||||
origin = _origin(settings)
|
||||
if origin is not None:
|
||||
# The origin is founder-set env, but defense-in-depth: strip
|
||||
# control characters (newline smuggling could forge extra
|
||||
# robots directives — a newline in the origin once appended
|
||||
# "Disallow: /admin" as a directive line).
|
||||
safe = "".join(c for c in origin if ord(c) >= 32)
|
||||
lines.append(f"Sitemap: {safe}/sitemap.xml")
|
||||
return Response(content="\n".join(lines) + "\n", media_type="text/plain")
|
||||
@@ -0,0 +1,190 @@
|
||||
"""/v1/telemetry — WS ingest + trace read endpoints (REQ-3-003, D-026, G-3).
|
||||
|
||||
The router composes the telemetry engine via DI: `telemetry/ingest.py` owns
|
||||
the WS protocol (frame contract + flood control + keepalive) and this module
|
||||
only wires `app.state.trace_store` / `app.state.trace_integrity` /
|
||||
`app.state.settings` into it, plus the two HTTP read faces:
|
||||
|
||||
WS /v1/telemetry/ingest?learner_id&task_id[&sandbox_id] (D-026)
|
||||
GET /v1/telemetry/traces/{learner_id}/{task_id} ordered trace; 404 unknown
|
||||
GET /v1/telemetry/gaps/{learner_id}/{task_id} missing seqs ; 404 unknown
|
||||
|
||||
The WS route is a thin DI shell: it validates the query-param identity and
|
||||
the Origin (browser pages are gated to the localhost dev origins — CORS
|
||||
middleware does not cover WS upgrades; the stdlib capture agent sends no
|
||||
Origin and is unaffected), pulls store/integrity/settings from `app.state`,
|
||||
and calls `telemetry_ingest_endpoint(...)` — the engine stays
|
||||
FastAPI-DI-free so it's testable without a router and the api/ layer owns
|
||||
all composition.
|
||||
|
||||
Unknown-trace contract: a trace is KNOWN when it has >=1 stored event OR
|
||||
carries an integrity flag — a flooded trace with zero stored rows still 200s
|
||||
so Proctor/grader can read WHY it's unusable (G-4 consumes
|
||||
`integrity_reason`). `TraceResponse.incomplete` / `.integrity_reason` mirror
|
||||
the map so HTTP consumers never touch process internals.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, WebSocket
|
||||
from pydantic import BaseModel
|
||||
|
||||
from ..telemetry.ingest import (
|
||||
TraceIntegrityMap,
|
||||
telemetry_ingest_endpoint,
|
||||
)
|
||||
from ..telemetry.models import TelemetryEvent
|
||||
from ..telemetry.store import TraceStore
|
||||
from ..telemetry.tokens import IngestToken
|
||||
from .deps import get_trace_integrity, get_trace_store
|
||||
|
||||
router = APIRouter(prefix="/v1/telemetry", tags=["telemetry"])
|
||||
|
||||
#: Browser Origins allowed to open the ingest socket (A-008 mirror, D-038).
|
||||
#: The stdlib capture agent sends NO Origin header (it is not a browser) and
|
||||
#: stays allowed; a malicious page loaded in the learner's browser would
|
||||
#: carry an Origin and must not be able to poison/flood the trace. CORS
|
||||
#: middleware does NOT cover WebSocket upgrades, so this gate is explicit.
|
||||
#: In network mode the configured CORS list governs (default '*' — any
|
||||
#: origin, since credentials are never used); an explicit list still rejects
|
||||
#: unlisted origins with 1008.
|
||||
_LOCAL_WS_ORIGINS = frozenset(
|
||||
{"http://localhost:3000", "http://127.0.0.1:3000", "http://localhost:8420"}
|
||||
)
|
||||
|
||||
|
||||
def _allowed_ws_origins(settings: object) -> frozenset[str]:
|
||||
configured = getattr(settings, "cors_origin_list", None)
|
||||
if configured is None:
|
||||
return _LOCAL_WS_ORIGINS
|
||||
if configured == ["*"]:
|
||||
return frozenset() # empty = wildcard = every Origin passes
|
||||
return frozenset(configured) | _LOCAL_WS_ORIGINS
|
||||
|
||||
|
||||
# --- WS ingest (D-026) ---------------------------------------------------------
|
||||
|
||||
|
||||
@router.websocket("/ingest")
|
||||
async def telemetry_ingest_ws(websocket: WebSocket) -> None:
|
||||
"""DI shell: resolve app.state services, then hand the socket to the engine.
|
||||
|
||||
The engine's session + flood logic is fully typed and testable without
|
||||
FastAPI; this shim is the only place the two layers meet.
|
||||
"""
|
||||
origin = (websocket.headers.get("origin") or "").strip()
|
||||
allowed = _allowed_ws_origins(getattr(websocket.app.state, "settings", None))
|
||||
if origin and allowed and origin not in allowed:
|
||||
# Same-origin dev pages (Next.js on :3000, the service itself on
|
||||
# :8420) pass; anything else is refused pre-accept. Non-browser
|
||||
# producers (the capture agent, tests) send no Origin and pass.
|
||||
# Wildcard (empty frozenset) passes every Origin in network mode.
|
||||
await websocket.close(
|
||||
code=1008, reason=f"origin {origin!r} not allowed for telemetry ingest"
|
||||
)
|
||||
return
|
||||
query = websocket.query_params
|
||||
learner_id = query.get("learner_id", "")
|
||||
task_id = query.get("task_id", "")
|
||||
if not learner_id or not task_id:
|
||||
# Reject BEFORE accept: closing the pre-accept handshake is the
|
||||
# cheapest denial and unambiguous for the stdlib capture agent.
|
||||
await websocket.close(code=1008, reason="learner_id/task_id query params required")
|
||||
return
|
||||
# v0.4.7 (P1): HMAC ingest token. The socket is a grading input; in
|
||||
# network mode it is LAN-reachable and identity params were self-
|
||||
# asserted — anyone could poison/flood a learner's trace (grading DoS).
|
||||
# The manager bakes mint tokens into NC_INGEST_URL; verify here. The
|
||||
# token binds (learner_id, task_id, sandbox_id) to the process secret.
|
||||
token = query.get("token", "")
|
||||
verifier: IngestToken | None = getattr(websocket.app.state, "ingest_token", None)
|
||||
if verifier is None or not verifier.verify(
|
||||
token, learner_id, task_id, query.get("sandbox_id", "")
|
||||
):
|
||||
await websocket.close(
|
||||
code=1008, reason="missing or invalid ingest token"
|
||||
)
|
||||
return
|
||||
await telemetry_ingest_endpoint(
|
||||
websocket=websocket,
|
||||
learner_id=learner_id,
|
||||
task_id=task_id,
|
||||
sandbox_id=query.get("sandbox_id", ""),
|
||||
store=websocket.app.state.trace_store,
|
||||
integrity=websocket.app.state.trace_integrity,
|
||||
settings=websocket.app.state.settings,
|
||||
)
|
||||
|
||||
|
||||
# --- HTTP reads -----------------------------------------------------------------
|
||||
|
||||
|
||||
class TraceResponse(BaseModel):
|
||||
"""Ordered trace + integrity signal (G-4 reads incomplete/reason)."""
|
||||
|
||||
learner_id: str
|
||||
task_id: str
|
||||
events: list[TelemetryEvent]
|
||||
incomplete: bool
|
||||
integrity_reason: str | None
|
||||
|
||||
|
||||
class GapsResponse(BaseModel):
|
||||
"""Missing seqs + integrity signal."""
|
||||
|
||||
learner_id: str
|
||||
task_id: str
|
||||
gaps: list[int]
|
||||
incomplete: bool
|
||||
integrity_reason: str | None
|
||||
|
||||
|
||||
def _is_known_trace(
|
||||
store: TraceStore, integrity: TraceIntegrityMap, learner_id: str, task_id: str
|
||||
) -> bool:
|
||||
"""Known = has stored events OR carries an integrity flag (a flooded trace
|
||||
with zero rows must still be readable — Proctor needs the reason)."""
|
||||
return (
|
||||
store.latest_seq(learner_id, task_id) >= 0
|
||||
or integrity.is_incomplete(learner_id, task_id)
|
||||
)
|
||||
|
||||
|
||||
@router.get("/traces/{learner_id}/{task_id}", response_model=TraceResponse)
|
||||
async def get_trace(
|
||||
learner_id: str,
|
||||
task_id: str,
|
||||
store: TraceStore = Depends(get_trace_store),
|
||||
integrity: TraceIntegrityMap = Depends(get_trace_integrity),
|
||||
) -> TraceResponse:
|
||||
if not _is_known_trace(store, integrity, learner_id, task_id):
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown trace {learner_id!r}/{task_id!r}"
|
||||
)
|
||||
return TraceResponse(
|
||||
learner_id=learner_id,
|
||||
task_id=task_id,
|
||||
events=store.get_trace(learner_id, task_id),
|
||||
incomplete=integrity.is_incomplete(learner_id, task_id),
|
||||
integrity_reason=integrity.reason(learner_id, task_id),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/gaps/{learner_id}/{task_id}", response_model=GapsResponse)
|
||||
async def get_gaps(
|
||||
learner_id: str,
|
||||
task_id: str,
|
||||
store: TraceStore = Depends(get_trace_store),
|
||||
integrity: TraceIntegrityMap = Depends(get_trace_integrity),
|
||||
) -> GapsResponse:
|
||||
if not _is_known_trace(store, integrity, learner_id, task_id):
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown trace {learner_id!r}/{task_id!r}"
|
||||
)
|
||||
return GapsResponse(
|
||||
learner_id=learner_id,
|
||||
task_id=task_id,
|
||||
gaps=store.gaps(learner_id, task_id),
|
||||
incomplete=integrity.is_incomplete(learner_id, task_id),
|
||||
integrity_reason=integrity.reason(learner_id, task_id),
|
||||
)
|
||||
@@ -0,0 +1,216 @@
|
||||
"""/v1/variants — seeded per-learner task variant endpoints (REQ-3-005, D-029).
|
||||
|
||||
Three faces over the variant engine (the generator + store stay FastAPI-free;
|
||||
this module owns all HTTP wiring — the D-027/D-032 house pattern):
|
||||
|
||||
POST /v1/variants {learner_id, template_id | competency_id}
|
||||
→ 200 the learner's variant — GENERATED on the first request,
|
||||
CACHED (store read, zero LLM calls) on every repeat: D-029
|
||||
reproducibility means one (learner_id, template_id) is ONE
|
||||
variant forever, so a regenerate is always a 200 of the SAME
|
||||
variant, never a second render.
|
||||
→ 404 unknown template_id, or competency_id with no bound template.
|
||||
→ 422 neither template_id nor competency_id given.
|
||||
GET /v1/variants/{task_id}
|
||||
→ 200 the stored variant owning the task key (the grading and
|
||||
telemetry join path); 404 when no variant was ever generated
|
||||
for the task.
|
||||
GET /v1/variants?learner_id=...
|
||||
→ 200 the learner's variants, chronological; [] when none.
|
||||
|
||||
Template resolution: an explicit `template_id` wins; without it the FIRST
|
||||
template bound to `competency_id` is used (`template_for_competency`,
|
||||
D-021 corpus alignment). The response carries `competency_id` resolved
|
||||
from the template library at read time — an enrichment, not a persisted
|
||||
column (the seed re-derives the whole variant, D-029) — so the learner
|
||||
surface can bind a variant to its competency without a library round-trip.
|
||||
|
||||
Distinctness (REQ-3-005): different learners on the same template draw
|
||||
different seeded params and receive distinct statements and task_ids;
|
||||
tests/api/test_variants.py asserts this end-to-end through the API.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Self
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
from ..config import Settings
|
||||
from ..identity.store import IdentityStore
|
||||
from ..variants.generator import VariantGenerator
|
||||
from ..variants.store import VariantRecord, VariantStore
|
||||
from ..variants.templates import TaskTemplate, get_template, template_for_competency
|
||||
from .deps import get_settings, get_variant_generator, get_variant_store
|
||||
from .identity import require_verified_age
|
||||
|
||||
|
||||
def _identity_store(request: Request) -> IdentityStore:
|
||||
return request.app.state.identity_store
|
||||
|
||||
|
||||
def _enforce_allowlist(learner_id: str, settings: Settings) -> None:
|
||||
"""G-5 pilot guard — allowlist runs FIRST in the composition (D-043)."""
|
||||
if learner_id not in settings.learner_allowlist:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=f"learner_id {learner_id!r} is not on the sandbox allowlist (G-5)",
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/v1/variants", tags=["variants"])
|
||||
|
||||
|
||||
# -- contracts ------------------------------------------------------------------
|
||||
|
||||
|
||||
class VariantGenerateRequest(BaseModel):
|
||||
"""One variant identity: an explicit `template_id`, or the first
|
||||
template bound to a `competency_id` (D-021). `template_id` wins when
|
||||
both are given (explicit identity beats derived); at least one is
|
||||
required — 422 otherwise.
|
||||
"""
|
||||
|
||||
learner_id: str = Field(min_length=1)
|
||||
template_id: str | None = None
|
||||
competency_id: str | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _require_template_or_competency(self) -> Self:
|
||||
if self.template_id is None and self.competency_id is None:
|
||||
raise ValueError("template_id or competency_id is required")
|
||||
return self
|
||||
|
||||
|
||||
class VariantResponse(BaseModel):
|
||||
"""VariantRecord over HTTP, plus the `competency_id` enrichment.
|
||||
|
||||
Every field except `competency_id` mirrors `VariantRecord` exactly
|
||||
(snake_case; `created_at` is an ISO 8601 UTC datetime) — the wire shape
|
||||
typed as `TaskVariant` in packages/types/variants.ts.
|
||||
"""
|
||||
|
||||
learner_id: str
|
||||
task_id: str
|
||||
template_id: str
|
||||
competency_id: str
|
||||
seed: str
|
||||
params: dict[str, str | int]
|
||||
statement: str
|
||||
starter_files: dict[str, str]
|
||||
#: REQ-5-005 (a-11): REQUIRED on the wire — always emitted.
|
||||
environment: str
|
||||
test_command: str
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class VariantListResponse(BaseModel):
|
||||
variants: list[VariantResponse]
|
||||
|
||||
|
||||
# -- resolution + rendering -----------------------------------------------------
|
||||
|
||||
|
||||
def _resolve_template(body: VariantGenerateRequest) -> TaskTemplate:
|
||||
"""Template for the request: the explicit id, else the first template
|
||||
bound to the competency; 404 when neither resolves."""
|
||||
if body.template_id is not None:
|
||||
template = get_template(body.template_id)
|
||||
if template is None:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"no task template with id {body.template_id!r}",
|
||||
)
|
||||
return template
|
||||
# The request validator guarantees the disjunction, so reaching here
|
||||
# means a competency_id was given (never None).
|
||||
assert body.competency_id is not None
|
||||
templates = template_for_competency(body.competency_id)
|
||||
if not templates:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"no task template for competency {body.competency_id!r}",
|
||||
)
|
||||
return templates[0]
|
||||
|
||||
|
||||
def _competency_for(template_id: str) -> str:
|
||||
"""competency_id enrichment for stored records (read paths)."""
|
||||
template = get_template(template_id)
|
||||
if template is None:
|
||||
# Integrity guard: a stored variant referencing a template that is
|
||||
# no longer in the library cannot be enriched; fail loudly rather
|
||||
# than fabricate a competency binding.
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"stored variant references unknown template {template_id!r}",
|
||||
)
|
||||
return template.competency_id
|
||||
|
||||
|
||||
def _to_response(record: VariantRecord, competency_id: str) -> VariantResponse:
|
||||
return VariantResponse(
|
||||
learner_id=record.learner_id,
|
||||
task_id=record.task_id,
|
||||
template_id=record.template_id,
|
||||
competency_id=competency_id,
|
||||
seed=record.seed,
|
||||
params=dict(record.params),
|
||||
statement=record.statement,
|
||||
starter_files=dict(record.starter_files),
|
||||
# a-11: REQUIRED on the wire — the server always emits both (v0.5).
|
||||
environment=record.environment or "build",
|
||||
test_command=record.test_command or "pytest -q",
|
||||
created_at=record.created_at,
|
||||
)
|
||||
|
||||
|
||||
# -- endpoints ------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("", response_model=VariantResponse)
|
||||
async def generate_variant(
|
||||
body: VariantGenerateRequest,
|
||||
request: Request,
|
||||
generator: VariantGenerator = Depends(get_variant_generator),
|
||||
settings: Settings = Depends(get_settings),
|
||||
) -> VariantResponse:
|
||||
"""The learner's variant for the resolved template — generated on the
|
||||
first request, cached (no LLM call) on every repeat: D-029 makes a
|
||||
regenerate a 200 of the SAME stored variant.
|
||||
|
||||
v0.5 identity gate (D-043, REQ-5-004): the school floor is 16+ verified.
|
||||
Composition order: G-5 allowlist (403) → identity verdict (403 + CTA).
|
||||
"""
|
||||
_enforce_allowlist(body.learner_id, settings)
|
||||
await require_verified_age(16, body.learner_id, _identity_store(request))
|
||||
template = _resolve_template(body)
|
||||
record = await generator.generate(body.learner_id, template.id)
|
||||
return _to_response(record, competency_id=template.competency_id)
|
||||
|
||||
|
||||
@router.get("", response_model=VariantListResponse)
|
||||
async def list_variants(
|
||||
learner_id: str,
|
||||
store: VariantStore = Depends(get_variant_store),
|
||||
) -> VariantListResponse:
|
||||
"""All stored variants for the learner, chronological; [] when none."""
|
||||
variants = [
|
||||
_to_response(record, competency_id=_competency_for(record.template_id))
|
||||
for record in store.list_for_learner(learner_id)
|
||||
]
|
||||
return VariantListResponse(variants=variants)
|
||||
|
||||
|
||||
@router.get("/{task_id}", response_model=VariantResponse)
|
||||
async def get_variant(
|
||||
task_id: str,
|
||||
store: VariantStore = Depends(get_variant_store),
|
||||
) -> VariantResponse:
|
||||
"""The stored variant owning the task key — the grading and telemetry
|
||||
join path; 404 when no variant was ever generated for the task."""
|
||||
record = store.get_by_task(task_id)
|
||||
if record is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"no stored variant for task {task_id!r}"
|
||||
)
|
||||
return _to_response(record, competency_id=_competency_for(record.template_id))
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Billing engine package (REQ-9-001, D-075..D-084, v0.9 Payment Engine).
|
||||
|
||||
The 7th domain module inside apps/ai-service (no new apps, no new ports —
|
||||
the D-031 boundary): pydantic contracts + the provider-agnostic
|
||||
`PaymentProvider` protocol (`base.py`), the shared HMAC webhook
|
||||
verification core (`webhooks.py`, D-077), the deterministic mock provider
|
||||
(`mock.py`, D-075 default), and the settings-driven factory
|
||||
(`factory.py`). The D-027 store family's seventh SQLite store
|
||||
(`store.py`), the seeded catalog (`catalog.py`), and the real Stripe
|
||||
provider (`stripe.py`) land with their owning phases.
|
||||
|
||||
Boundary (D-031 family discipline): `billing/` never imports `agents/`
|
||||
or `api/`; this package imports stdlib + pydantic + `config` only.
|
||||
"""
|
||||
|
||||
from .base import (
|
||||
CANONICAL_EVENT_NAMES,
|
||||
CanonicalBillingEvent,
|
||||
CanonicalEventName,
|
||||
CheckoutMode,
|
||||
CheckoutSession,
|
||||
CheckoutSessionSpec,
|
||||
CheckoutStatus,
|
||||
Package,
|
||||
PaymentProvider,
|
||||
PaymentStatus,
|
||||
Plan,
|
||||
PlanInterval,
|
||||
SubscriptionRef,
|
||||
SubscriptionStatus,
|
||||
UnknownPaymentProviderError,
|
||||
VerifiedEvent,
|
||||
WebhookSignatureError,
|
||||
)
|
||||
from .factory import payment_provider_from_settings
|
||||
from .mock import MockPaymentProvider
|
||||
|
||||
__all__ = [
|
||||
"CANONICAL_EVENT_NAMES",
|
||||
"CanonicalBillingEvent",
|
||||
"CanonicalEventName",
|
||||
"CheckoutMode",
|
||||
"CheckoutSession",
|
||||
"CheckoutSessionSpec",
|
||||
"CheckoutStatus",
|
||||
"MockPaymentProvider",
|
||||
"Package",
|
||||
"PaymentProvider",
|
||||
"PaymentStatus",
|
||||
"Plan",
|
||||
"PlanInterval",
|
||||
"SubscriptionRef",
|
||||
"SubscriptionStatus",
|
||||
"UnknownPaymentProviderError",
|
||||
"VerifiedEvent",
|
||||
"WebhookSignatureError",
|
||||
"payment_provider_from_settings",
|
||||
]
|
||||
@@ -0,0 +1,282 @@
|
||||
"""Billing pydantic contracts + PaymentProvider protocol (REQ-9-001).
|
||||
|
||||
The wire + provider shapes for the payment engine. `Package`/`Plan` are
|
||||
the catalog item shapes (D-078: seeded DB rows; P2 seeds them); the
|
||||
`CheckoutSession*` pair is the provider session seam (R-33..R-36);
|
||||
`CanonicalBillingEvent` is OUR event vocabulary — normalized at the
|
||||
provider boundary, never taken raw from a vendor (D-083, R-43).
|
||||
|
||||
Money (D-081): integer cents end-to-end — `int` with `ge=0`, no floats
|
||||
anywhere in billing, currency `usd` fixed. Stripe itself is cents-native
|
||||
(R-48: `unit_amount: 1000` = $10.00).
|
||||
|
||||
Id honesty (the identity-mock unique-mint lesson, A-305): every id in
|
||||
this module is an opaque handle minted UNIQUELY per creation call —
|
||||
never derived from user input or a deterministic hash of it.
|
||||
|
||||
Boundary (D-031 family): `billing/` never imports `agents/` or `api/`;
|
||||
stdlib + pydantic + config only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal, Protocol, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
#: One-time posting packages vs recurring talent-access plans: one shape
|
||||
#: family, one table family (D-078/D-080 — a one-time purchase is NOT a
|
||||
#: degenerate subscription).
|
||||
PlanInterval = Literal["month", "year"]
|
||||
|
||||
#: Stripe-compatible checkout modes (R-34): payment = one-time packages,
|
||||
#: subscription = recurring plans. `setup` exists on the wire enum but no
|
||||
#: catalog item maps to it in v0.9.
|
||||
CheckoutMode = Literal["payment", "setup", "subscription"]
|
||||
|
||||
#: Session status vocabulary (R-36): open (awaiting payment) → complete
|
||||
#: (checkout finished) | expired (window elapsed). The GRANT predicate is
|
||||
#: `status == "complete" AND payment_status == "paid"` — never status
|
||||
#: alone (R-37: async methods can leave complete with payment still
|
||||
#: processing).
|
||||
CheckoutStatus = Literal["open", "complete", "expired"]
|
||||
|
||||
#: Payment status vocabulary (R-36). `no_payment_required` exists on the
|
||||
#: wire enum; v0.9 grants only on `paid`.
|
||||
PaymentStatus = Literal["paid", "unpaid", "no_payment_required"]
|
||||
|
||||
#: Subscription lifecycle states (D-083). G-305: `upgraded` is NOT a
|
||||
#: status — upgrade is a plan_id transition WITHIN `active`, recorded as
|
||||
#: a billing_event with provenance. `expired` here is the terminal past-
|
||||
#: due-grace outcome; the store owns the transition matrix (P5).
|
||||
SubscriptionStatus = Literal[
|
||||
"trialing", "active", "past_due", "cancelled", "expired"
|
||||
]
|
||||
|
||||
#: OUR canonical event names (D-083, R-43: Stripe has no
|
||||
#: `subscription.renewed`/`checkout.completed` — the provider boundary
|
||||
#: maps vendor names onto these; provider event ids are stored + deduped
|
||||
#: VERBATIM so dedupe stays truthful).
|
||||
CanonicalEventName = Literal[
|
||||
"checkout.completed",
|
||||
"invoice.paid",
|
||||
"invoice.payment_failed",
|
||||
"subscription.cancelled",
|
||||
"subscription.renewed",
|
||||
]
|
||||
|
||||
#: Tuple mirror of the Literal (docs + tests import this; keeps the enum
|
||||
#: and any exhaustive switch in one place).
|
||||
CANONICAL_EVENT_NAMES: tuple[CanonicalEventName, ...] = (
|
||||
"checkout.completed",
|
||||
"invoice.paid",
|
||||
"invoice.payment_failed",
|
||||
"subscription.cancelled",
|
||||
"subscription.renewed",
|
||||
)
|
||||
|
||||
|
||||
class WebhookSignatureError(ValueError):
|
||||
"""Webhook signature verification failed (D-077).
|
||||
|
||||
Raised with a precise reason (missing header / no v1 schemes / stale
|
||||
timestamp / signature mismatch). The webhook route maps this to 401 —
|
||||
a payment surface never processes an unverifiable event.
|
||||
"""
|
||||
|
||||
|
||||
class UnknownPaymentProviderError(ValueError):
|
||||
"""Raised for a provider name outside the contract, or a real
|
||||
provider selected without its required configuration (R-53 — the
|
||||
voice/embedding factory pattern).
|
||||
|
||||
main.py's lifespan CATCHES this and falls back to mock with a loud
|
||||
G-11 log; direct callers (tests) see the actionable raise.
|
||||
"""
|
||||
|
||||
|
||||
class Package(BaseModel):
|
||||
"""A one-time posting package catalog row (D-078/D-080).
|
||||
|
||||
`posts_limit` counts posting creations (consumed one per
|
||||
create_posting, newest-non-expired-first — D-080); `duration_days`
|
||||
bounds the purchase window from the buy time. `features` is the
|
||||
pricing-page copy list (P2 pins the wire shape both sides).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
name: str = Field(min_length=1)
|
||||
#: D-081: integer cents, ge=0 — currency usd fixed; never a float.
|
||||
price_cents: int = Field(ge=0)
|
||||
posts_limit: int = Field(ge=1)
|
||||
duration_days: int = Field(ge=1)
|
||||
features: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class Plan(BaseModel):
|
||||
"""A recurring talent-access plan catalog row (D-078/D-080).
|
||||
|
||||
`interval` distinguishes the billing cadence; entitlement derives
|
||||
from the subscription's `expires_at` extension on each renewal
|
||||
(D-083 — idempotent per event dedupe).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
name: str = Field(min_length=1)
|
||||
#: D-081: integer cents, ge=0 — currency usd fixed; never a float.
|
||||
price_cents: int = Field(ge=0)
|
||||
interval: PlanInterval
|
||||
features: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class CheckoutSessionSpec(BaseModel):
|
||||
"""What the operator asks a provider to create (R-33..R-35).
|
||||
|
||||
The checkout route resolves `item_id` against the catalog and
|
||||
carries the SNAPSHOT forward (D-078: `price_cents` + `item_name`
|
||||
copied at buy time — catalog edits never rewrite purchase
|
||||
history; the provider turns them into `line_items[0][price_data]`,
|
||||
R-35). `client_reference_id` is our reconciliation key (R-34 — the
|
||||
local checkout_sessions row id). `metadata` rides the session both
|
||||
directions (D-076: operator_ref + employer_id provenance). G-310:
|
||||
`success_url`/`cancel_url` default from `public_base_url` — a
|
||||
hosted checkout cannot be created without them (R-34).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
mode: CheckoutMode
|
||||
item_id: str = Field(min_length=1)
|
||||
#: D-078 snapshot: the catalog price at buy time (D-081 cents).
|
||||
price_cents: int = Field(ge=0)
|
||||
#: D-078 snapshot: the catalog name at buy time.
|
||||
item_name: str = Field(min_length=1)
|
||||
#: Recurring cadence for plans ("month"/"year"); empty for packages.
|
||||
interval: str = ""
|
||||
success_url: str = ""
|
||||
cancel_url: str = ""
|
||||
client_reference_id: str = Field(min_length=1, max_length=200)
|
||||
metadata: dict[str, str] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class CheckoutSession(BaseModel):
|
||||
"""Provider session state (R-36 normalized fields).
|
||||
|
||||
`url` is present only while a session is open (hosted checkout);
|
||||
`expires_at` bounds the payment window (R-34: 30m–24h on Stripe).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
url: str = ""
|
||||
status: CheckoutStatus
|
||||
payment_status: PaymentStatus
|
||||
mode: CheckoutMode
|
||||
expires_at: datetime | None = None
|
||||
|
||||
|
||||
class SubscriptionRef(BaseModel):
|
||||
"""A provider subscription handle (R-41/R-42 normalized states).
|
||||
|
||||
The store owns the lifecycle rows; this is what provider calls
|
||||
(cancel/update) and webhook payloads exchange.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
status: SubscriptionStatus
|
||||
|
||||
|
||||
class CanonicalBillingEvent(BaseModel):
|
||||
"""A provider event normalized onto OUR vocabulary (D-083, R-43).
|
||||
|
||||
`payload` is UNTRUSTED (G-302): webhook processing treats it as ID
|
||||
references only — session/subscription state resolves from LOCAL
|
||||
rows; purchase price/name come from the stored snapshot (D-078),
|
||||
never from the event body.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
name: CanonicalEventName
|
||||
payload: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class VerifiedEvent(BaseModel):
|
||||
"""What verify_webhook returns: the authenticated event.
|
||||
|
||||
Verification authenticates the TRANSPORT (HMAC over the raw body);
|
||||
`event` is the parsed body AFTER the signature gate. Providers that
|
||||
need raw-body parsing keep it inside their own verify_webhook.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
event: CanonicalBillingEvent
|
||||
#: Provider event id, stored + deduped VERBATIM (R-43/D-083 — never
|
||||
#: normalized; the UniqueConstraint is on this exact string).
|
||||
provider_event_id: str = ""
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class PaymentProvider(Protocol):
|
||||
"""The payment port (D-075): checkout + subscriptions + webhooks.
|
||||
|
||||
Two implementations ship in-milestone: deterministic mock (default —
|
||||
tests never call a real provider) and Stripe-compatible REST (P3,
|
||||
exercised ONLY through httpx.MockTransport byte-contract tests).
|
||||
Never imports agents/api; selected by `factory.py` from Settings.
|
||||
"""
|
||||
|
||||
async def create_checkout_session(
|
||||
self, spec: CheckoutSessionSpec
|
||||
) -> CheckoutSession:
|
||||
"""Create a provider checkout session for a catalog item."""
|
||||
...
|
||||
|
||||
async def retrieve_session(self, session_id: str) -> CheckoutSession | None:
|
||||
"""Fetch session state; None when the provider does not know the id."""
|
||||
...
|
||||
|
||||
async def cancel_subscription(
|
||||
self, subscription_id: str, *, at_period_end: bool = False
|
||||
) -> SubscriptionRef | None:
|
||||
"""Cancel a subscription (R-46: immediate DELETE or deferred
|
||||
cancel_at_period_end); None when unknown to the provider."""
|
||||
...
|
||||
|
||||
async def update_subscription(
|
||||
self, subscription_id: str, new_price_cents: int
|
||||
) -> SubscriptionRef | None:
|
||||
"""Move a subscription to a new price (R-47: the provider resolves
|
||||
the subscription_item id inside — items[0][id] must be in the
|
||||
request or the new price is ADDED alongside the old → dual
|
||||
charge)."""
|
||||
...
|
||||
|
||||
def verify_webhook(
|
||||
self,
|
||||
raw_body: bytes,
|
||||
signature_header: str,
|
||||
*,
|
||||
now: float | None = None,
|
||||
) -> VerifiedEvent:
|
||||
"""Authenticate + parse a webhook delivery (D-077).
|
||||
|
||||
`raw_body` must be the exact un-parsed bytes (risk-1); raises
|
||||
`WebhookSignatureError` on any verification failure — the route
|
||||
maps that to 401, never processes further.
|
||||
"""
|
||||
...
|
||||
|
||||
def normalize_event(self, provider_event: dict[str, Any]) -> CanonicalBillingEvent:
|
||||
"""Map a provider-native event payload onto our canonical names
|
||||
(R-43). Mock speaks canonical names directly → passthrough."""
|
||||
...
|
||||
@@ -0,0 +1,105 @@
|
||||
"""Seeded billing catalog — the one source of truth for pricing (D-078).
|
||||
|
||||
The catalog mirrors the pricing page's tier structure (REQ-017 /
|
||||
REQ-9-002): prices + names + feature copy live HERE and flow to the
|
||||
page via `/v1/billing/catalog` — tier copy is data-driven, never
|
||||
hardcoded on both sides (D-079's no-drift rule). Seeding is idempotent
|
||||
(`SQLiteBillingStore.seed_catalog` upserts by id), so the app lifespan
|
||||
can seed on every boot.
|
||||
|
||||
Enterprise tiers: the pricing page models them as "Custom" — the pilot
|
||||
catalog keeps that honesty by seeding them as price_cents=0 contact
|
||||
tiers (checkout never applies; an operator grants enterprise manually
|
||||
per D-080's unlimited path). The features copy records the
|
||||
"Contact Sales" semantics verbatim from the page.
|
||||
|
||||
Money (D-081): integer cents — 4900 = $49.00; currency `usd` fixed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import Package, Plan
|
||||
from .store import BillingStore
|
||||
|
||||
SEED_PACKAGES: list[Package] = [
|
||||
Package(
|
||||
id="pkg-single",
|
||||
name="Single Post",
|
||||
price_cents=4900,
|
||||
posts_limit=1,
|
||||
duration_days=30,
|
||||
features=[
|
||||
"1 job posting",
|
||||
"30 days live",
|
||||
"Basic analytics",
|
||||
"Standard placement",
|
||||
],
|
||||
),
|
||||
Package(
|
||||
id="pkg-bundle",
|
||||
name="Bundle",
|
||||
price_cents=39900,
|
||||
posts_limit=10,
|
||||
duration_days=90,
|
||||
features=[
|
||||
"10 job postings",
|
||||
"90 days each",
|
||||
"Advanced analytics",
|
||||
"Priority placement",
|
||||
],
|
||||
),
|
||||
Package(
|
||||
id="pkg-enterprise",
|
||||
name="Enterprise",
|
||||
price_cents=0,
|
||||
posts_limit=1000,
|
||||
duration_days=365,
|
||||
features=[
|
||||
"Unlimited postings",
|
||||
"Dedicated account manager",
|
||||
"API access",
|
||||
"Custom branding",
|
||||
"Custom pricing — contact sales (operator-granted at pilot)",
|
||||
],
|
||||
),
|
||||
]
|
||||
|
||||
SEED_PLANS: list[Plan] = [
|
||||
Plan(
|
||||
id="plan-starter",
|
||||
name="Starter",
|
||||
price_cents=9900,
|
||||
interval="month",
|
||||
features=["Browse talent database", "10 candidate views/mo", "Basic filters"],
|
||||
),
|
||||
Plan(
|
||||
id="plan-pro",
|
||||
name="Pro",
|
||||
price_cents=29900,
|
||||
interval="month",
|
||||
features=[
|
||||
"Unlimited views",
|
||||
"AI matching",
|
||||
"Advanced filters",
|
||||
"Saved searches",
|
||||
],
|
||||
),
|
||||
Plan(
|
||||
id="plan-enterprise",
|
||||
name="Enterprise",
|
||||
price_cents=0,
|
||||
interval="year",
|
||||
features=[
|
||||
"Everything in Pro",
|
||||
"API access",
|
||||
"Bulk export",
|
||||
"Dedicated CSM",
|
||||
"Custom pricing — contact sales (operator-granted at pilot)",
|
||||
],
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def seed_billing_catalog(store: BillingStore) -> None:
|
||||
"""Idempotent catalog seeding (D-078) — safe on every boot."""
|
||||
store.seed_catalog(SEED_PACKAGES, SEED_PLANS)
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Payment provider factory (D-075, R-53, REQ-9-001).
|
||||
|
||||
`AI_PAYMENT_PROVIDER = mock | stripe` (default: mock — the no-key path
|
||||
is first-class; tests never call a real provider). Mirrors the
|
||||
voice/embedding factory pattern: the factory raises
|
||||
`UnknownPaymentProviderError` with an actionable message for direct
|
||||
callers (tests), while the lifespan in main.py CATCHES it and falls
|
||||
back to mock with a loud log — a typo'd env must never crash the
|
||||
unattended boot (G-11).
|
||||
|
||||
P3: `stripe` is REAL (billing/stripe.py) — but needs its API key,
|
||||
webhook secret, AND `public_base_url` (G-310: hosted checkout cannot
|
||||
build success/cancel URLs without it); a missing piece raises the
|
||||
actionable error the lifespan catches → G-11 mock fallback. The
|
||||
http_client rides in from main.py's shared pool (R-33).
|
||||
|
||||
Webhook secret (D-077): the settings default is the empty string —
|
||||
UNSET → the factory substitutes a per-boot random secret (the
|
||||
telemetry_ingest_secret pattern). Restart invalidates it, so stale
|
||||
signed deliveries fail: webhooks fail closed when unconfigured, which
|
||||
is correct for a payment surface.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
|
||||
import httpx
|
||||
|
||||
from ..config import Settings
|
||||
from .base import PaymentProvider, UnknownPaymentProviderError
|
||||
from .mock import MockPaymentProvider
|
||||
|
||||
|
||||
def resolve_webhook_secret(settings: Settings) -> str:
|
||||
"""Configured `AI_WEBHOOK_SECRET` verbatim; unset → per-boot random
|
||||
(fail-closed webhooks, D-077 — the telemetry_ingest_secret shape)."""
|
||||
if settings.webhook_secret:
|
||||
return settings.webhook_secret
|
||||
return secrets.token_urlsafe(32)
|
||||
|
||||
|
||||
def payment_provider_from_settings(
|
||||
settings: Settings, http_client: httpx.AsyncClient | None = None
|
||||
) -> PaymentProvider:
|
||||
"""Select the payment provider by settings (env `AI_PAYMENT_PROVIDER`).
|
||||
|
||||
The selected provider gets the webhook secret resolved here
|
||||
(D-077). Stripe needs the shared httpx pool — main.py passes its
|
||||
app.state client; direct test callers construct their own.
|
||||
"""
|
||||
name = (settings.payment_provider or "mock").strip().lower()
|
||||
webhook_secret = resolve_webhook_secret(settings)
|
||||
if name == "mock":
|
||||
return MockPaymentProvider(webhook_secret=webhook_secret)
|
||||
if name == "stripe":
|
||||
# R-53/G-310: config errors (key/secret/base_url) surface from
|
||||
# the stripe factory's own gate — the operator-facing ones;
|
||||
# the httpx pool is a caller concern checked after config.
|
||||
from .stripe import stripe_provider_from_settings, validate_stripe_config
|
||||
|
||||
validate_stripe_config(settings)
|
||||
if http_client is None:
|
||||
raise UnknownPaymentProviderError(
|
||||
"stripe provider needs the shared httpx pool — "
|
||||
"pass http_client (main.py does; tests construct one)"
|
||||
)
|
||||
return stripe_provider_from_settings(settings, http_client)
|
||||
raise UnknownPaymentProviderError(
|
||||
f"unknown AI_PAYMENT_PROVIDER {name!r}: use 'mock' or 'stripe'"
|
||||
)
|
||||
@@ -0,0 +1,241 @@
|
||||
"""Deterministic mock payment provider (D-075, REQ-9-001).
|
||||
|
||||
Stripe-shaped without any network: sessions (`cs_mock_...` ids), the
|
||||
same HMAC webhook scheme (via webhooks.mint/verify — byte-identical to
|
||||
the real provider path), and canonical event names spoken directly
|
||||
(normalize_event is a passthrough — R-43's boundary maps VENDOR names;
|
||||
the mock has none to map). Tests never call a real provider; the
|
||||
mock-complete path (D-082, P3) mints a signed event through the REAL
|
||||
webhook processing via `mint_event`.
|
||||
|
||||
Id honesty (the identity-mock unique-mint lesson, verified there):
|
||||
session ids are minted UNIQUELY per create call — a monotonic counter
|
||||
plus per-process seed from `secrets`. A deterministic id derived from
|
||||
the spec would collide on any retry/re-entry of the same purchase
|
||||
request and the store's per-session idempotency (G-303) would silently
|
||||
conflate distinct logical checkouts. Uniqueness per call is the
|
||||
contract; determinism of VERDICTS (what tests pin) is preserved — the
|
||||
session lifecycle is scriptable, not the ids.
|
||||
|
||||
Session lifecycle: create → status open / payment unpaid; the operator
|
||||
completes out-of-band payment via `confirm_completion` (the D-076
|
||||
pilot flow — P3's admin-gated mock-complete route calls this) →
|
||||
status complete / payment_status paid (the R-37 grant predicate).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import itertools
|
||||
import json
|
||||
import secrets
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
from . import webhooks
|
||||
from .base import (
|
||||
CanonicalBillingEvent,
|
||||
CheckoutSession,
|
||||
CheckoutSessionSpec,
|
||||
SubscriptionRef,
|
||||
VerifiedEvent,
|
||||
)
|
||||
|
||||
#: Session payment window (R-34's shape; Stripe allows 30m–24h — the
|
||||
#: mock keeps a roomy 24h default so operator-mediated completion is
|
||||
#: never raced in the pilot).
|
||||
_SESSION_TTL = timedelta(hours=24)
|
||||
|
||||
#: Event-id prefix — provider event ids are stored + deduped VERBATIM
|
||||
#: (R-43/D-083); the mock mints them uniquely like session ids.
|
||||
_EVENT_ID_PREFIX = "evt_mock_"
|
||||
|
||||
|
||||
class MockPaymentProvider:
|
||||
"""In-memory, deterministic, zero-network PaymentProvider.
|
||||
|
||||
Scriptable sessions + real-scheme webhook minting — the full
|
||||
checkout→webhook loop is provable without any vendor.
|
||||
"""
|
||||
|
||||
def __init__(self, webhook_secret: str = "") -> None:
|
||||
# Per-process nonce: ids are opaque handles, never derived from
|
||||
# user input — counter + nonce keeps ids unique within and
|
||||
# across provider instances on one box.
|
||||
self._nonce = secrets.randbits(32)
|
||||
self._session_counter = itertools.count()
|
||||
self._event_counter = itertools.count()
|
||||
# Sessions by id: the provider's view of checkout state.
|
||||
self._sessions: dict[str, CheckoutSession] = {}
|
||||
# Subscriptions by id: created on subscription-mode completion
|
||||
# (P5 drives the lifecycle; the mock keeps the ref shape honest).
|
||||
self._subscriptions: dict[str, SubscriptionRef] = {}
|
||||
self._webhook_secret = webhook_secret or secrets.token_urlsafe(32)
|
||||
|
||||
# -- ids -----------------------------------------------------------------
|
||||
|
||||
def _mint_session_id(self) -> str:
|
||||
# Stripe-shaped cs_ prefix + mock marker (R-36's id shape; the
|
||||
# mock marker keeps logs/test failures self-explanatory).
|
||||
return (
|
||||
f"cs_mock_{self._nonce:08x}{next(self._session_counter):08x}"
|
||||
)
|
||||
|
||||
def _mint_event_id(self) -> str:
|
||||
return (
|
||||
f"{_EVENT_ID_PREFIX}{self._nonce:08x}{next(self._event_counter):08x}"
|
||||
)
|
||||
|
||||
# -- checkout ------------------------------------------------------------
|
||||
|
||||
async def create_checkout_session(
|
||||
self, spec: CheckoutSessionSpec
|
||||
) -> CheckoutSession:
|
||||
"""Create a session (status open, payment unpaid — R-36)."""
|
||||
session_id = self._mint_session_id()
|
||||
session = CheckoutSession(
|
||||
id=session_id,
|
||||
# Hosted-checkout stand-in: the mock has no hosted page; the
|
||||
# url is a stable pseudo-hosted pointer (never fetched).
|
||||
url=f"https://mock.local/checkout/{session_id}",
|
||||
status="open",
|
||||
payment_status="unpaid",
|
||||
mode=spec.mode,
|
||||
expires_at=datetime.now(UTC) + _SESSION_TTL,
|
||||
)
|
||||
self._sessions[session.id] = session
|
||||
return session
|
||||
|
||||
async def retrieve_session(self, session_id: str) -> CheckoutSession | None:
|
||||
return self._sessions.get(session_id)
|
||||
|
||||
async def confirm_completion(self, session_id: str) -> CheckoutSession:
|
||||
"""Flip a session to complete/paid (the R-37 grant predicate).
|
||||
|
||||
The out-of-band payment step of the pilot flow (D-076): P3's
|
||||
admin-gated mock-complete route calls this, then mints the
|
||||
signed webhook event through the real processing path. Unknown
|
||||
or already-complete sessions raise ValueError (fail loud, not
|
||||
idempotent-silent: the ROUTE owns idempotency via event dedupe).
|
||||
"""
|
||||
session = self._sessions.get(session_id)
|
||||
if session is None:
|
||||
raise ValueError(f"unknown checkout session {session_id!r}")
|
||||
if session.status != "open":
|
||||
raise ValueError(
|
||||
f"session {session_id!r} is {session.status!r}, not open"
|
||||
)
|
||||
completed = session.model_copy(
|
||||
update={"status": "complete", "payment_status": "paid"}
|
||||
)
|
||||
self._sessions[session_id] = completed
|
||||
if completed.mode == "subscription":
|
||||
# Subscription-mode completion creates the provider-side
|
||||
# subscription ref (R-36: `subscription` present in
|
||||
# subscription mode only).
|
||||
self._subscriptions.setdefault(
|
||||
f"sub_mock_{session_id.removeprefix('cs_')}",
|
||||
SubscriptionRef(
|
||||
id=f"sub_mock_{session_id.removeprefix('cs_')}",
|
||||
status="active",
|
||||
),
|
||||
)
|
||||
return completed
|
||||
|
||||
# -- subscriptions ---------------------------------------------------------
|
||||
|
||||
async def cancel_subscription(
|
||||
self, subscription_id: str, *, at_period_end: bool = False
|
||||
) -> SubscriptionRef | None:
|
||||
subscription = self._subscriptions.get(subscription_id)
|
||||
if subscription is None:
|
||||
return None
|
||||
# R-46 shape: immediate → cancelled now; deferred → stays active
|
||||
# until period end (the P5 webhook drives the final state).
|
||||
if at_period_end:
|
||||
return subscription
|
||||
cancelled = subscription.model_copy(update={"status": "cancelled"})
|
||||
self._subscriptions[subscription_id] = cancelled
|
||||
return cancelled
|
||||
|
||||
async def update_subscription(
|
||||
self, subscription_id: str, new_price_cents: int
|
||||
) -> SubscriptionRef | None:
|
||||
subscription = self._subscriptions.get(subscription_id)
|
||||
if subscription is None:
|
||||
return None
|
||||
# R-47 note: the real provider resolves items[0][id] inside; the
|
||||
# mock has no line items to dual-charge — price is recorded by
|
||||
# the store's plan snapshot (D-078), so this is a state no-op
|
||||
# that keeps the protocol shape honest.
|
||||
return subscription
|
||||
|
||||
# -- webhooks (D-077 scheme, shared core) -----------------------------------
|
||||
|
||||
def verify_webhook(
|
||||
self,
|
||||
raw_body: bytes,
|
||||
signature_header: str,
|
||||
*,
|
||||
now: float | None = None,
|
||||
) -> VerifiedEvent:
|
||||
"""Authenticate + parse a mock delivery via the shared core."""
|
||||
webhooks.verify(
|
||||
raw_body, signature_header, self._webhook_secret, now=now
|
||||
)
|
||||
body = _parse_json_body(raw_body)
|
||||
event = self.normalize_event(body)
|
||||
# Provider event id: top-level on the body, kept VERBATIM (the
|
||||
# R-43 rule — normalize the NAME, never the id).
|
||||
provider_event_id = str(body.get("id", ""))
|
||||
return VerifiedEvent(
|
||||
event=event, provider_event_id=provider_event_id
|
||||
)
|
||||
|
||||
def mint_event(
|
||||
self, name: str, payload: dict[str, Any]
|
||||
) -> tuple[bytes, str, str]:
|
||||
"""Mint a signed webhook delivery for tests + the complete path.
|
||||
|
||||
Returns `(raw_body, signature_header, provider_event_id)`. The
|
||||
body is the canonical event JSON — the exact bytes a provider
|
||||
would POST and the webhook route would read via
|
||||
`await request.body()`. Round-trip green through
|
||||
verify_webhook by construction (mint/verify share the scheme).
|
||||
"""
|
||||
provider_event_id = self._mint_event_id()
|
||||
body = json.dumps(
|
||||
{
|
||||
"id": provider_event_id,
|
||||
"name": name,
|
||||
"payload": payload,
|
||||
}
|
||||
).encode()
|
||||
header = webhooks.mint(self._webhook_secret, body)
|
||||
return body, header, provider_event_id
|
||||
|
||||
def normalize_event(
|
||||
self, provider_event: dict[str, Any]
|
||||
) -> CanonicalBillingEvent:
|
||||
"""Passthrough (R-43): the mock speaks canonical names directly."""
|
||||
name = str(provider_event.get("name", ""))
|
||||
if name not in (
|
||||
"checkout.completed",
|
||||
"invoice.paid",
|
||||
"invoice.payment_failed",
|
||||
"subscription.cancelled",
|
||||
"subscription.renewed",
|
||||
):
|
||||
raise ValueError(f"unknown event name {name!r}")
|
||||
payload = provider_event.get("payload")
|
||||
return CanonicalBillingEvent(
|
||||
name=name, # type: ignore[arg-type] # validated in-contract above
|
||||
payload=payload if isinstance(payload, dict) else {},
|
||||
)
|
||||
|
||||
|
||||
def _parse_json_body(raw_body: bytes) -> dict[str, Any]:
|
||||
"""Parse the event JSON body (post-verification, risk-1 safe)."""
|
||||
decoded = json.loads(raw_body.decode("utf-8"))
|
||||
if not isinstance(decoded, dict):
|
||||
raise ValueError("event body is not a JSON object")
|
||||
return decoded
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,343 @@
|
||||
"""StripePaymentProvider — Stripe-compatible REST via the shared httpx
|
||||
pool (REQ-9-003, D-75/R-33..R-50).
|
||||
|
||||
Form-encoded bodies with bracket nesting (R-33: `line_items[0]
|
||||
[price_data][currency]=usd` — NOT JSON), HTTP Basic auth, an
|
||||
Idempotency-Key per logical operation (R-49: a double-submit
|
||||
otherwise creates two sessions), and the webhook signature scheme
|
||||
shared with the mock (webhooks.py — D-077 is byte-for-byte Stripe's
|
||||
`t=...,v1=...` HMAC-SHA256 over `{t}.{raw_body}`).
|
||||
|
||||
Exercised ONLY through httpx.MockTransport byte-contract tests
|
||||
(D-075) — tests never call the cloud. Event normalization maps
|
||||
vendor names onto OUR canonical vocabulary (R-43); provider event ids
|
||||
are stored + deduped VERBATIM.
|
||||
|
||||
G-310: `success_url`/`cancel_url` default from `public_base_url` —
|
||||
without them a hosted checkout session cannot be created (R-34:
|
||||
success_url is required).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import uuid
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from ..config import Settings
|
||||
from .base import (
|
||||
CanonicalBillingEvent,
|
||||
CanonicalEventName,
|
||||
CheckoutSession,
|
||||
CheckoutSessionSpec,
|
||||
SubscriptionRef,
|
||||
SubscriptionStatus,
|
||||
UnknownPaymentProviderError,
|
||||
VerifiedEvent,
|
||||
WebhookSignatureError,
|
||||
)
|
||||
from .webhooks import verify as verify_webhook_signature
|
||||
|
||||
#: R-43: vendor event type → our canonical name. `subscription.renewed`
|
||||
#: has no vendor equivalent — a renewal IS an invoice.paid on a
|
||||
#: recurring subscription; the corroborating `customer.subscription.
|
||||
#: updated` maps alongside it (recorded, not conflated).
|
||||
_EVENT_NAME_MAP: dict[str, CanonicalEventName] = {
|
||||
"checkout.session.completed": "checkout.completed",
|
||||
"invoice.paid": "invoice.paid",
|
||||
"invoice.payment_succeeded": "invoice.paid",
|
||||
"invoice.payment_failed": "invoice.payment_failed",
|
||||
"customer.subscription.deleted": "subscription.cancelled",
|
||||
"customer.subscription.updated": "subscription.renewed",
|
||||
}
|
||||
|
||||
|
||||
#: R-42: vendor status spellings → OUR enum (Stripe writes `canceled`
|
||||
#: with one L; our vocabulary is `cancelled`).
|
||||
_STATUS_MAP: dict[str, SubscriptionStatus] = {
|
||||
"trialing": "trialing",
|
||||
"incomplete": "trialing",
|
||||
"active": "active",
|
||||
"past_due": "past_due",
|
||||
"canceled": "cancelled",
|
||||
"unpaid": "expired",
|
||||
"incomplete_expired": "expired",
|
||||
}
|
||||
|
||||
|
||||
def _normalize_status(raw: Any, default: SubscriptionStatus) -> SubscriptionStatus:
|
||||
return _STATUS_MAP.get(str(raw), default)
|
||||
|
||||
|
||||
class StripePaymentProvider:
|
||||
"""Stripe-compatible REST implementation of PaymentProvider."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
http_client: httpx.AsyncClient,
|
||||
api_key: str,
|
||||
webhook_secret: str,
|
||||
base_url: str = "https://api.stripe.com",
|
||||
) -> None:
|
||||
self._http = http_client
|
||||
self._api_key = api_key
|
||||
self._webhook_secret = webhook_secret
|
||||
self._base_url = base_url.rstrip("/")
|
||||
|
||||
# -- helpers ------------------------------------------------------------
|
||||
|
||||
def _auth_headers(self) -> dict[str, str]:
|
||||
# R-33: HTTP Basic with the secret key as the username and an
|
||||
# empty password (`-u sk_...:`).
|
||||
basic = base64.b64encode(f"{self._api_key}:".encode()).decode()
|
||||
return {"Authorization": f"Basic {basic}"}
|
||||
|
||||
def _raise_for_status(self, response: httpx.Response) -> None:
|
||||
if response.status_code >= 400:
|
||||
# A-305: sanitize — the api key must never appear in any
|
||||
# raised exception (the embedding-test precedent).
|
||||
detail = response.text
|
||||
if self._api_key and self._api_key in detail:
|
||||
detail = detail.replace(self._api_key, "[redacted]")
|
||||
raise httpx.HTTPStatusError(
|
||||
f"stripe {response.status_code}: {detail}",
|
||||
request=response.request,
|
||||
response=response,
|
||||
)
|
||||
|
||||
async def _form_post(self, path: str, data: dict[str, Any]) -> dict[str, Any]:
|
||||
headers = self._auth_headers()
|
||||
# R-49: an idempotency key per logical POST — a network retry of
|
||||
# session creation must not mint two sessions.
|
||||
headers["Idempotency-Key"] = str(uuid.uuid4())
|
||||
response = await self._http.post(
|
||||
f"{self._base_url}{path}",
|
||||
data=data,
|
||||
headers=headers,
|
||||
)
|
||||
self._raise_for_status(response)
|
||||
return response.json()
|
||||
|
||||
# -- checkout sessions (R-33..R-37) ---------------------------------------
|
||||
|
||||
async def create_checkout_session(
|
||||
self, spec: CheckoutSessionSpec
|
||||
) -> CheckoutSession:
|
||||
# R-33/R-35: form-encoded with bracket nesting. One-time vs
|
||||
# recurring differs only by price_data[recurring][interval]
|
||||
# (R-35) — the snapshot price rides the wire (D-078/D-081).
|
||||
data: dict[str, Any] = {
|
||||
"mode": spec.mode,
|
||||
"client_reference_id": spec.client_reference_id,
|
||||
"line_items[0][quantity]": "1",
|
||||
"line_items[0][price_data][currency]": "usd",
|
||||
"line_items[0][price_data][unit_amount]": str(spec.price_cents),
|
||||
"line_items[0][price_data][product_data][name]": spec.item_name,
|
||||
}
|
||||
if spec.interval:
|
||||
data["line_items[0][price_data][recurring][interval]"] = spec.interval
|
||||
for key, value in spec.metadata.items():
|
||||
data[f"metadata[{key}]"] = value
|
||||
# G-310: hosted checkout requires success_url (R-34) — the
|
||||
# checkout route defaults these from public_base_url.
|
||||
data["success_url"] = spec.success_url
|
||||
data["cancel_url"] = spec.cancel_url
|
||||
body = await self._form_post("/v1/checkout/sessions", data)
|
||||
return self._session_from_api(body)
|
||||
|
||||
async def retrieve_session(self, session_id: str) -> CheckoutSession | None:
|
||||
try:
|
||||
body = await self._get(f"/v1/checkout/sessions/{session_id}")
|
||||
except httpx.HTTPStatusError as exc:
|
||||
if exc.response.status_code == 404:
|
||||
return None
|
||||
raise
|
||||
return self._session_from_api(body)
|
||||
|
||||
async def _get(self, path: str) -> dict[str, Any]:
|
||||
response = await self._http.get(
|
||||
f"{self._base_url}{path}",
|
||||
headers=self._auth_headers(),
|
||||
)
|
||||
self._raise_for_status(response)
|
||||
return response.json()
|
||||
|
||||
@staticmethod
|
||||
def _session_from_api(body: dict[str, Any]) -> CheckoutSession:
|
||||
# Final-review fix: coerce + default every enum-adjacent value —
|
||||
# a vendor body with an unexpected status spelling (or a missing
|
||||
# one) must not crash the route with a pydantic ValidationError
|
||||
# (the G-306 never-500 discipline).
|
||||
status = str(body.get("status", "open"))
|
||||
if status not in ("open", "complete", "expired"):
|
||||
status = "open"
|
||||
payment_status = str(body.get("payment_status", "unpaid"))
|
||||
if payment_status not in ("paid", "unpaid", "no_payment_required"):
|
||||
payment_status = "unpaid"
|
||||
mode = str(body.get("mode", "payment"))
|
||||
if mode not in ("payment", "setup", "subscription"):
|
||||
mode = "payment"
|
||||
return CheckoutSession(
|
||||
id=str(body.get("id", "")),
|
||||
url=str(body.get("url") or ""),
|
||||
status=status, # type: ignore[arg-type]
|
||||
payment_status=payment_status, # type: ignore[arg-type]
|
||||
mode=mode, # type: ignore[arg-type]
|
||||
expires_at=None,
|
||||
)
|
||||
|
||||
# -- subscriptions (R-46/R-47) --------------------------------------------
|
||||
|
||||
async def cancel_subscription(
|
||||
self, subscription_id: str, *, at_period_end: bool = False
|
||||
) -> SubscriptionRef | None:
|
||||
try:
|
||||
if at_period_end:
|
||||
body = await self._form_post(
|
||||
f"/v1/subscriptions/{subscription_id}",
|
||||
{"cancel_at_period_end": "true"},
|
||||
)
|
||||
else:
|
||||
response = await self._http.delete(
|
||||
f"{self._base_url}/v1/subscriptions/{subscription_id}",
|
||||
headers=self._auth_headers(),
|
||||
)
|
||||
self._raise_for_status(response)
|
||||
body = response.json()
|
||||
except httpx.HTTPStatusError as exc:
|
||||
if exc.response.status_code == 404:
|
||||
return None
|
||||
raise
|
||||
return SubscriptionRef(
|
||||
id=str(body.get("id", subscription_id)),
|
||||
status=_normalize_status(body.get("status", "cancelled"), "cancelled"),
|
||||
)
|
||||
|
||||
async def update_subscription(
|
||||
self, subscription_id: str, new_price_cents: int
|
||||
) -> SubscriptionRef | None:
|
||||
"""R-47: the provider resolves the subscription_item id INSIDE —
|
||||
`items[0][id]` must ride the update or the new price is ADDED
|
||||
alongside the old (dual-charge bug, pinned by byte-contract)."""
|
||||
try:
|
||||
current = await self._get(f"/v1/subscriptions/{subscription_id}")
|
||||
items = current.get("items", {}).get("data", [])
|
||||
if not items:
|
||||
return None
|
||||
item_id = str(items[0]["id"])
|
||||
body = await self._form_post(
|
||||
f"/v1/subscriptions/{subscription_id}",
|
||||
{
|
||||
"items[0][id]": item_id,
|
||||
"items[0][price_data][currency]": "usd",
|
||||
"items[0][price_data][unit_amount]": str(new_price_cents),
|
||||
"items[0][price_data][product_data][name]": "nextcraft plan",
|
||||
"items[0][price_data][recurring][interval]": "month",
|
||||
},
|
||||
)
|
||||
except httpx.HTTPStatusError as exc:
|
||||
if exc.response.status_code == 404:
|
||||
return None
|
||||
raise
|
||||
return SubscriptionRef(
|
||||
id=str(body.get("id", subscription_id)),
|
||||
status=_normalize_status(body.get("status", "active"), "active"),
|
||||
)
|
||||
|
||||
# -- webhooks (D-077 — the shared scheme) ---------------------------------
|
||||
|
||||
def verify_webhook(
|
||||
self,
|
||||
raw_body: bytes,
|
||||
signature_header: str,
|
||||
*,
|
||||
now: float | None = None,
|
||||
) -> VerifiedEvent:
|
||||
verify_webhook_signature(
|
||||
raw_body,
|
||||
signature_header,
|
||||
self._webhook_secret,
|
||||
now=now,
|
||||
)
|
||||
payload = _parse_event_payload(raw_body)
|
||||
return VerifiedEvent(
|
||||
event=self.normalize_event(payload),
|
||||
provider_event_id=str(payload.get("id", "")),
|
||||
)
|
||||
|
||||
def normalize_event(
|
||||
self, provider_event: dict[str, Any]
|
||||
) -> CanonicalBillingEvent:
|
||||
"""R-43: map the vendor event onto OUR canonical vocabulary —
|
||||
BOTH the name and the payload keys. Final-review fix: the
|
||||
vendor object's key spellings (`id`, `subscription`) never
|
||||
become the canonical keys (`session_id`, `subscription_id`)
|
||||
the processing route reads, so under real Stripe every
|
||||
checkout/renewal resolved no local row and landed unmatched
|
||||
with NO entitlement — the mock speaks canonical keys directly
|
||||
and hid the desync."""
|
||||
vendor_type = str(provider_event.get("type", ""))
|
||||
mapped = _EVENT_NAME_MAP.get(vendor_type)
|
||||
if mapped is None:
|
||||
raise UnknownPaymentProviderError(
|
||||
f"unmapped provider event type {vendor_type!r}"
|
||||
)
|
||||
payload = dict(provider_event.get("data", {}).get("object", {}))
|
||||
canonical: dict[str, Any] = dict(payload)
|
||||
if vendor_type == "checkout.session.completed":
|
||||
canonical["session_id"] = str(payload.get("id", ""))
|
||||
else:
|
||||
# invoice.* + customer.subscription.*: Stripe carries the
|
||||
# subscription ref at data.object.subscription (and, on
|
||||
# subscription objects, at data.object.id).
|
||||
subscription_ref = payload.get("subscription") or payload.get("id")
|
||||
canonical["subscription_id"] = str(subscription_ref or "")
|
||||
return CanonicalBillingEvent(name=mapped, payload=canonical)
|
||||
|
||||
|
||||
def _parse_event_payload(raw_body: bytes) -> dict[str, Any]:
|
||||
try:
|
||||
parsed = json.loads(raw_body.decode("utf-8"))
|
||||
except (UnicodeDecodeError, ValueError) as exc:
|
||||
raise WebhookSignatureError(f"event body is not JSON: {exc}") from exc
|
||||
if not isinstance(parsed, dict):
|
||||
raise WebhookSignatureError("event body is not a JSON object")
|
||||
return parsed
|
||||
|
||||
|
||||
def validate_stripe_config(settings: Settings) -> None:
|
||||
"""R-53/G-310: the actionable config gate — main.py's lifespan
|
||||
catches and falls back to mock with the loud G-11 log."""
|
||||
api_key = settings.payment_api_key.get_secret_value()
|
||||
if not api_key:
|
||||
raise UnknownPaymentProviderError(
|
||||
"AI_PAYMENT_PROVIDER=stripe requires AI_PAYMENT_API_KEY "
|
||||
"(env-only, D-014) — set it or drop back to the mock default"
|
||||
)
|
||||
if not settings.webhook_secret:
|
||||
raise UnknownPaymentProviderError(
|
||||
"AI_PAYMENT_PROVIDER=stripe requires AI_WEBHOOK_SECRET — "
|
||||
"webhooks fail closed without it (D-077)"
|
||||
)
|
||||
if not settings.public_base_url:
|
||||
raise UnknownPaymentProviderError(
|
||||
"AI_PAYMENT_PROVIDER=stripe requires AI_PUBLIC_BASE_URL — "
|
||||
"hosted checkout cannot build success/cancel URLs without it (G-310)"
|
||||
)
|
||||
|
||||
|
||||
def stripe_provider_from_settings(
|
||||
settings: Settings, http_client: httpx.AsyncClient
|
||||
) -> StripePaymentProvider:
|
||||
"""Factory wiring (R-53): validates config, then constructs the
|
||||
provider over the shared httpx pool."""
|
||||
validate_stripe_config(settings)
|
||||
return StripePaymentProvider(
|
||||
http_client=http_client,
|
||||
api_key=settings.payment_api_key.get_secret_value(),
|
||||
webhook_secret=settings.webhook_secret,
|
||||
)
|
||||
@@ -0,0 +1,142 @@
|
||||
"""Webhook signature verification core (D-077, R-38..R-40).
|
||||
|
||||
Byte-for-byte Stripe's scheme shape, shared by every provider so the
|
||||
mock and the real Stripe path verify identically (the byte-contract
|
||||
lesson): header `Stripe-Signature: t=<unix_seconds>,v1=<hex>` on one
|
||||
line; signed payload `"{timestamp}.{raw_body}"`; HMAC-SHA256 with the
|
||||
endpoint secret; constant-time comparison; ±300 s replay window.
|
||||
|
||||
Security properties (each pinned by tests/billing/test_webhooks.py):
|
||||
- raw body: verification needs the exact UN-PARSED bytes (risk-1) — the
|
||||
caller passes `raw_body: bytes`, never a pre-parsed model;
|
||||
- multiple `v1` values (secret rolls — R-38): verify iterates ALL v1
|
||||
candidates, any match wins (R-53 risk-5);
|
||||
- non-`v1` schemes (`v0` on Stripe test events, unknown future ones):
|
||||
ignored entirely — a v0-only header has no v1 to check and fails
|
||||
(downgrade protection);
|
||||
- replay window (R-40, Stripe default 300 s): the timestamp bounds how
|
||||
long a captured delivery can be replayed; `now` is injectable so
|
||||
tests are deterministic without sleeps (risk-4);
|
||||
- fail closed (D-077): every failure raises `WebhookSignatureError`
|
||||
with a precise reason — a payment surface never processes an
|
||||
unverifiable event.
|
||||
|
||||
`mint` exists for tests + the mock provider: it produces exactly what a
|
||||
signed provider delivery looks like, so the mock-complete path (D-082)
|
||||
can post through the REAL webhook processing.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import hmac
|
||||
import time
|
||||
|
||||
from .base import WebhookSignatureError
|
||||
|
||||
#: Replay tolerance in seconds (R-40: Stripe's official default; "don't
|
||||
#: use a tolerance of 0" — clock skew between provider and box is real).
|
||||
DEFAULT_TOLERANCE_S: float = 300.0
|
||||
|
||||
|
||||
def _parse_header(header: str) -> tuple[int | None, list[str]]:
|
||||
"""Parse `t=...,v1=...,v1=...` into (timestamp, v1 candidates).
|
||||
|
||||
Elements are comma-separated `key=value` pairs (R-38). Unknown
|
||||
schemes (v0, anything else) are ignored: only `v1` is ever checked.
|
||||
Returns timestamp None when the header carries no valid `t` pair.
|
||||
"""
|
||||
timestamp: int | None = None
|
||||
v1_candidates: list[str] = []
|
||||
for element in header.split(","):
|
||||
key, sep, value = element.strip().partition("=")
|
||||
if not sep:
|
||||
continue
|
||||
if key == "t":
|
||||
try:
|
||||
timestamp = int(value)
|
||||
except ValueError:
|
||||
# Malformed t → treat as absent; verify() fails on the
|
||||
# stale/missing-timestamp branch (fail closed).
|
||||
continue
|
||||
elif key == "v1":
|
||||
v1_candidates.append(value)
|
||||
return timestamp, v1_candidates
|
||||
|
||||
|
||||
def _compute_signature(
|
||||
secret: str, timestamp: int, raw_body: bytes
|
||||
) -> str:
|
||||
"""HMAC-SHA256 over `"{timestamp}.{raw_body}"` (R-39).
|
||||
|
||||
The timestamp is attacker-immutable in transit: changing `t` without
|
||||
the secret invalidates every v1 in the header.
|
||||
"""
|
||||
signed_payload = f"{timestamp}.".encode() + raw_body
|
||||
return hmac.new(
|
||||
secret.encode("utf-8"), signed_payload, hashlib.sha256
|
||||
).hexdigest()
|
||||
|
||||
|
||||
def mint(
|
||||
secret: str,
|
||||
raw_body: bytes,
|
||||
now: float | None = None,
|
||||
) -> str:
|
||||
"""Mint a `Stripe-Signature` header for `raw_body` (D-077 scheme).
|
||||
|
||||
Test + mock-provider helper (the mint half of the telemetry
|
||||
tokens.py precedent). `now` defaults to the wall clock; inject for
|
||||
determinism. Produces exactly one `t` + one `v1` pair — what a
|
||||
provider delivery carries.
|
||||
"""
|
||||
timestamp = int(now if now is not None else time.time())
|
||||
signature = _compute_signature(secret, timestamp, raw_body)
|
||||
return f"t={timestamp},v1={signature}"
|
||||
|
||||
|
||||
def verify(
|
||||
raw_body: bytes,
|
||||
header: str | None,
|
||||
secret: str,
|
||||
*,
|
||||
now: float | None = None,
|
||||
tolerance_s: float = DEFAULT_TOLERANCE_S,
|
||||
) -> tuple[int, list[str]]:
|
||||
"""Verify a `Stripe-Signature` header against the raw body.
|
||||
|
||||
Returns `(timestamp, v1_candidates)` on success — the parsed header
|
||||
the caller may want for replay bookkeeping. Raises
|
||||
`WebhookSignatureError` with a precise reason otherwise:
|
||||
|
||||
- missing/empty header → "missing Stripe-Signature header";
|
||||
- no `v1` schemes present → "no v1 signature schemes in header"
|
||||
(a v0-only header lands here — non-v1 schemes are ignored);
|
||||
- timestamp absent or outside ±tolerance_s of now → "stale
|
||||
timestamp" (replay window, R-40);
|
||||
- no v1 candidate matches → "signature mismatch" (constant-time
|
||||
compare per candidate — any match wins across secret rolls).
|
||||
"""
|
||||
if not header:
|
||||
raise WebhookSignatureError("missing Stripe-Signature header")
|
||||
timestamp, v1_candidates = _parse_header(header)
|
||||
if not v1_candidates:
|
||||
raise WebhookSignatureError(
|
||||
f"no v1 signature schemes in header {header!r} "
|
||||
"(non-v1 schemes are ignored)"
|
||||
)
|
||||
if timestamp is None:
|
||||
raise WebhookSignatureError(
|
||||
f"missing or malformed timestamp in header {header!r}"
|
||||
)
|
||||
current = now if now is not None else time.time()
|
||||
if abs(current - timestamp) > tolerance_s:
|
||||
raise WebhookSignatureError(
|
||||
f"stale timestamp {timestamp}: outside the "
|
||||
f"{tolerance_s:g}s replay window at {current:g}"
|
||||
)
|
||||
for candidate in v1_candidates:
|
||||
expected = _compute_signature(secret, timestamp, raw_body)
|
||||
if hmac.compare_digest(expected, candidate):
|
||||
return timestamp, v1_candidates
|
||||
raise WebhookSignatureError("signature mismatch: no v1 candidate matches")
|
||||
@@ -1,14 +1,24 @@
|
||||
"""Service settings — pydantic-settings, env prefix AI_, .env support."""
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import field_validator
|
||||
from pydantic import SecretStr, field_validator
|
||||
from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict
|
||||
|
||||
# apps/ai-service/ (sandbox dir default is relative to the app, not the CWD)
|
||||
_SERVICE_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
# v0.3.6: durable runtime state lives OUTSIDE the repo (founder directive —
|
||||
# the repo is a git tree, not a data dir; old defaults under apps/ai-service/
|
||||
# polluted the working copy). ~/.nextcraft is the state root for DB + sandbox
|
||||
# workdirs; env overrides may use ~/ paths (expanded by the validator below).
|
||||
_STATE_ROOT = Path.home() / ".nextcraft"
|
||||
|
||||
# G-16: TTS response-format whitelist (feeds the TTS route's Content-Type).
|
||||
_TTS_FORMATS = ("mp3", "wav", "opus")
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_prefix="AI_", env_file=".env", extra="ignore")
|
||||
@@ -26,7 +36,7 @@ class Settings(BaseSettings):
|
||||
|
||||
# v0.3 sandbox fabric (REQ-3-001): root holding per-sandbox workdirs.
|
||||
# Relative paths resolve against the app dir (apps/ai-service/), not the CWD.
|
||||
sandbox_dir: Path = _SERVICE_ROOT / "sandboxes"
|
||||
sandbox_dir: Path = _STATE_ROOT / "sandboxes"
|
||||
|
||||
# D-032: single-box capacity, no queue — pool full → API maps to 503.
|
||||
sandbox_max_concurrent: int = 5
|
||||
@@ -63,4 +73,174 @@ class Settings(BaseSettings):
|
||||
return value
|
||||
|
||||
# D-027: SQLite path for telemetry/grades/variants/defenses stores.
|
||||
db_path: Path = _SERVICE_ROOT / "ai_service" / "data" / "nextcraft.db"
|
||||
# v0.3.6: default moved out of the repo to the home state root.
|
||||
db_path: Path = _STATE_ROOT / "data" / "nextcraft.db"
|
||||
|
||||
# v0.3.6 single-port deploy: directory of the exported Next.js app
|
||||
# (apps/web/out). When set, the app serves it at / (StaticFiles) so the
|
||||
# whole site — UI + API — answers on one port behind HAProxy; the web
|
||||
# build bakes NEXT_PUBLIC_AI_SERVICE_URL=self (relative same-origin
|
||||
# fetches). Empty default → no mount; dev/tests unchanged.
|
||||
web_static_dir: Path = Path("")
|
||||
|
||||
@field_validator("db_path", "sandbox_dir", "web_static_dir", mode="before")
|
||||
@classmethod
|
||||
def _expanduser_paths(cls, value: object) -> object:
|
||||
# Env-set paths may use ~ (e.g. AI_DB_PATH=~/.nextcraft/x.db);
|
||||
# pydantic Path does not expand it natively.
|
||||
if isinstance(value, str):
|
||||
return Path(value).expanduser()
|
||||
return value
|
||||
|
||||
@field_validator("voice_tts_format")
|
||||
@classmethod
|
||||
def _validate_tts_format(cls, value: str) -> str:
|
||||
# G-16 + G-11 consistency: unknown values NEVER crash the boot —
|
||||
# fall back to the default with a loud warning (the boot-survival
|
||||
# log lives in main.py's voice fallback; this validator normalizes).
|
||||
v = value.strip().lower()
|
||||
if v not in _TTS_FORMATS:
|
||||
logging.getLogger(__name__).warning(
|
||||
"AI_VOICE_TTS_FORMAT=%r is not one of %s — falling back to 'mp3'",
|
||||
value,
|
||||
_TTS_FORMATS,
|
||||
)
|
||||
return "mp3"
|
||||
return v
|
||||
|
||||
# G-3 flood control (NOT backpressure-by-silence): max events ingested per
|
||||
# (learner_id, task_id) trace before the WS endpoint closes the connection
|
||||
# with 1008 and marks the trace INCOMPLETE_FLOODED. Drop-oldest is
|
||||
# FORBIDDEN — it corrupts grading input (GRILL G-3).
|
||||
telemetry_max_events_per_task: int = 50000
|
||||
|
||||
# Sandbox telemetry wiring (REQ-3-003): loopback host the in-sandbox capture
|
||||
# agent dials to reach this service's WS ingest (the agent joins the sandbox
|
||||
# mount ns but NOT the net ns — exec namespaces are offline, so the agent
|
||||
# shares the host network and reaches the app over loopback). Port reuses
|
||||
# `port` (A-004); only the host is configurable — never a second port.
|
||||
telemetry_ingest_host: str = "127.0.0.1"
|
||||
|
||||
# v0.4.7 (P1, final-review flag): HMAC secret for WS ingest tokens. The
|
||||
# ingest socket is reachable on the LAN in network mode; without a token
|
||||
# check anyone could dial with arbitrary learner_id/task_id and poison or
|
||||
# flood traces (grading DoS). The manager bakes an HMAC of
|
||||
# (learner_id, task_id, sandbox_id) into NC_INGEST_URL; the WS endpoint
|
||||
# verifies it. Unset → per-boot random secret (in-process mint/verify
|
||||
# agree; restart invalidates stale URLs — sandboxes are not reboot-
|
||||
# persistent, so that is harmless). Env-set for multi-process deploys.
|
||||
telemetry_ingest_secret: str = ""
|
||||
|
||||
# v0.3.5 network mode (D-038): dev.sh binds 0.0.0.0 so remote browsers can
|
||||
# reach the stack; '*' (default) lets any origin call the API (safe ONLY
|
||||
# because credentials are never enabled — A-008). Set a comma-separated
|
||||
# origin list (e.g. 'http://nextcraft-1:3000') to restrict instead.
|
||||
# NOTE (v0.3.6): with the UI served same-origin from 8420 this is moot in
|
||||
# production (same-origin requests skip CORS); it stays for the two-port
|
||||
# dev topology.
|
||||
cors_origins: str = "*"
|
||||
|
||||
@property
|
||||
def cors_origin_list(self) -> list[str]:
|
||||
value = self.cors_origins.strip()
|
||||
if value == "*":
|
||||
return ["*"]
|
||||
return [o.strip() for o in value.split(",") if o.strip()]
|
||||
|
||||
# Identity provider selection (REQ-5-003, A-303): 'mock' (default —
|
||||
# deterministic, no vendor spend pre-pilot; verdicts carry mock=True
|
||||
# forever per A-304). A real KYC vendor drops in via the
|
||||
# IdentityProvider protocol without API changes.
|
||||
identity_provider: str = "mock"
|
||||
# G-13: identity submit caps — one active pending per learner (409 on
|
||||
# resubmit) and a per-learner submit rate ceiling.
|
||||
identity_submits_per_min: int = 3
|
||||
|
||||
# v0.6 marketplace (REQ-6-001, D-057): the founder/admin surface is
|
||||
# guarded by a shared secret — X-Admin-Token or Authorization: Bearer
|
||||
# compared via secrets.compare_digest (G-108). Env-only (D-014), never
|
||||
# logged; SecretStr keeps it out of accidental repr/logs. UNSET =
|
||||
# every /v1/marketplace/admin/* route 403s — fail closed, NEVER open
|
||||
# (a loud boot warning names the fix, G-11).
|
||||
admin_token: SecretStr = SecretStr("")
|
||||
|
||||
# v0.10 employer auth (REQ-10-001, D-088): both throttle knobs —
|
||||
# plain ints, the identity_submits_per_min shape (R-60d). Keyed by
|
||||
# SUBMITTED EMAIL, never client IP (R-63); in-memory, per-process.
|
||||
# (a) Failed logins per email per rolling 900 s window → 429 with
|
||||
# the honest "wait" detail; a successful login clears it.
|
||||
# (b) Well-formed login+register attempts per rolling 60 s window
|
||||
# across all emails → 429 (G-314: failed passwords count).
|
||||
employer_login_failures_per_window: int = 5
|
||||
employer_auth_attempts_per_min: int = 30
|
||||
|
||||
# v0.6 P5 placeholder (D-056): the deployed origin for sitemap.xml /
|
||||
# robots.txt absolute URLs (consumed by the SEO phase; documented
|
||||
# now so operators see it beside the admin token).
|
||||
public_base_url: str = ""
|
||||
|
||||
# Embedding provider selection (REQ-6-003, D-051): 'mock' (default —
|
||||
# deterministic 256-dim feature-hash, pure stdlib; tests never call
|
||||
# a real embeddings API) or 'openai-embed' (real embeddings against
|
||||
# an OpenAI-compatible endpoint). openai-embed requires
|
||||
# embed_base_url + embed_api_key + embed_model; when unconfigured the
|
||||
# lifespan falls back to mock with a loud log (G-11 — a typo'd env
|
||||
# must never crash the unattended boot).
|
||||
embed_provider: str = "mock"
|
||||
|
||||
# Real embeddings (D-051, probe-corrected): ollama-cloud does NOT
|
||||
# expose /embeddings — point AI_EMBED_BASE_URL at another
|
||||
# OpenAI-compatible endpoint (e.g. LOCAL ollama at :11434/v1, or any
|
||||
# embeddings cloud). Keys env-only, never committed, never logged
|
||||
# (SecretStr; mirrors the admin token).
|
||||
embed_base_url: str = ""
|
||||
embed_api_key: SecretStr = SecretStr("")
|
||||
embed_model: str = ""
|
||||
|
||||
# Voice provider selection (REQ-5-001, D-040): 'mock' (default — the
|
||||
# no-key path is first-class; tests never call a real voice API),
|
||||
# 'browser' (client-native SR/TTS; the descriptor tells the web client),
|
||||
# or 'openai-audio' (real server STT/TTS against an OpenAI-compatible
|
||||
# audio endpoint). openai-audio requires voice_base_url + voice_api_key;
|
||||
# when unconfigured the lifespan falls back to mock with a loud log
|
||||
# (G-11 — a typo'd env must never crash the unattended boot).
|
||||
voice_provider: str = "mock"
|
||||
|
||||
# Real server voice (D-040, A-301): endpoint-agnostic by config (D-014
|
||||
# pattern) — any OpenAI-compatible audio API works. Keys env-only,
|
||||
# never committed, never logged (mirrors ollama_cloud_api_key).
|
||||
voice_base_url: str = ""
|
||||
voice_api_key: str = ""
|
||||
voice_stt_model: str = "whisper-1"
|
||||
voice_tts_model: str = "tts-1"
|
||||
voice_tts_voice: str = "alloy"
|
||||
# G-16: whitelist, not free string — this feeds the TTS route's
|
||||
# Content-Type. A str + mode-after validator (NOT a pydantic Literal):
|
||||
# a Literal would raise ValidationError at Settings construction, before
|
||||
# main.py's G-11 fallback could catch it — crashing the unattended boot
|
||||
# on a typo'd env. Invalid values fall back to the default LOUDLY.
|
||||
voice_tts_format: str = "mp3"
|
||||
# A-302/D-041: upload guard before the provider call (webm/opus is
|
||||
# ~0.5-1MB/min, so 10MB tolerates very long answers).
|
||||
voice_max_audio_mb: int = 10
|
||||
|
||||
# v0.9 Payment Engine (REQ-9-001, D-075, R-53): payment provider
|
||||
# selection — 'mock' (default: deterministic, zero-network; tests
|
||||
# NEVER call a real payment API) or 'stripe' (real Stripe-compatible
|
||||
# REST, P3). A typo'd name raises at the factory; main.py's lifespan
|
||||
# catches it and falls back to mock with a loud log (G-11 — the
|
||||
# voice/embedding factory pattern).
|
||||
payment_provider: str = "mock"
|
||||
|
||||
# Real payment API key (Stripe sk_... class) — 'stripe' requires
|
||||
# this; 'mock' ignores it. Keys env-only (D-014), never committed,
|
||||
# never logged (SecretStr mirrors the admin token; tests use fixed
|
||||
# test secrets, never real ones).
|
||||
payment_api_key: SecretStr = SecretStr("")
|
||||
|
||||
# Webhook HMAC secret (D-077, the telemetry_ingest_secret pattern):
|
||||
# UNSET (empty default) → the factory substitutes a PER-BOOT RANDOM
|
||||
# secret — restart invalidates stale signed deliveries, so webhooks
|
||||
# FAIL CLOSED when unconfigured (401, never process unverifiable —
|
||||
# correct for a payment surface). Set for multi-process deploys.
|
||||
webhook_secret: str = ""
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
"""Pre-baked artifacts + rubrics + defense transcripts — Assessor mock inputs (REQ-2-008).
|
||||
|
||||
Counterpart: packages/mock-data/ai-scenarios.ts (artifact IDs string-identical,
|
||||
D-021). Real process-trace grading is a v0.3+ engine (assessment engine);
|
||||
these pre-baked submissions stand in for artifact + defense evaluation.
|
||||
v0.2 mock engine inputs (pre-baked artifacts/rubrics/transcripts) — DORMANT as of v0.3 re-
|
||||
grounding (Task 6-1-04): no production code path imports this module. Retained as Phase-3
|
||||
calibration history.
|
||||
|
||||
|
||||
Historical note: artifact IDs were once string-synced with a TS counterpart
|
||||
(packages/mock-data/ai-scenarios.ts, D-021) — that module was deleted in
|
||||
v0.7 P4 (dead export); this corpus is the single source. Real process-trace
|
||||
grading is a v0.3+ engine (assessment engine); these pre-baked submissions
|
||||
stand in for artifact + defense evaluation.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
"""Simulated sandbox telemetry corpus — Lab agent mock engine inputs (REQ-2-007).
|
||||
|
||||
Counterpart: packages/mock-data/ai-scenarios.ts (scenario IDs string-identical,
|
||||
D-021). Real sandbox telemetry is a v0.3+ engine (sandbox fabric); these
|
||||
scripted event streams stand in for the build-session process trace.
|
||||
v0.2 mock engine inputs (Lab/Proctor scenarios) — DORMANT as of v0.3 re-grounding (Task 6-1-04):
|
||||
no production code path imports this module. Retained as Phase-3 calibration history
|
||||
(corpus/trace_fixtures.py references it from TESTS only).
|
||||
|
||||
|
||||
Historical note: scenario IDs were once string-synced with a TS counterpart
|
||||
(packages/mock-data/ai-scenarios.ts, D-021) — that module was deleted in
|
||||
v0.7 P4 (dead export); this corpus is the single source. Real sandbox
|
||||
telemetry is a v0.3+ engine (sandbox fabric); these scripted event streams
|
||||
stand in for the build-session process trace.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
"""Synthetic trace fixtures for grading calibration (Task 3-2-02, REQ-3-004).
|
||||
|
||||
Three builder archetypes as REAL `TelemetryEvent` traces (the grading
|
||||
engine's native input — these are NOT the v0.2 corpus's simplified
|
||||
`{timestamp, kind, detail}` event shapes):
|
||||
|
||||
strong builder — iterative debugging: small edits, tests early,
|
||||
failed runs closed by targeted fixes, eventual pass.
|
||||
lazy builder — one large paste, a single late test run, pass.
|
||||
(v0.2 alignment: the `lab-scenario-flagged` paste-
|
||||
and-run archetype; D-021.)
|
||||
struggling builder — many edit/test cycles, failures never close,
|
||||
never reaches a pass.
|
||||
|
||||
ID convention (D-021 alignment, documented in each fixture):
|
||||
v0.2 corpus scenario IDs are `<domain>-scenario-<slug>` (`lab-scenario-strong`,
|
||||
`lab-scenario-struggling`, `lab-scenario-flagged`, `proctor-scenario-*` — see
|
||||
corpus/telemetry.py). Grading fixtures carry ids string-aligned to that
|
||||
convention:
|
||||
|
||||
fixture id = "<scenario id>::<archetype>-trace"
|
||||
learner id = "learner-003" (a member of the corpus learner-00N id space;
|
||||
learner-001/002 exist in learner_context.py)
|
||||
|
||||
so a fixture is greppable against its v0.2 scenario counterpart while staying
|
||||
a distinct id space (a grading trace is a real event stream, not the v0.2
|
||||
mock scenario timeline — same convention, richer event kind set).
|
||||
|
||||
Instance hygiene: fixtures store event SPECS (plain tuples) and materialize
|
||||
FRESH `TelemetryEvent` instances on every `fixture.events` access. SQLModel
|
||||
rows carry SQLAlchemy instance state once a session has flushed them —
|
||||
re-adding the SAME instance to another store is a silent no-op, which would
|
||||
poison sequential test runs (a fixture ingested by test N would vanish for
|
||||
test N+1). Materializing per access keeps every consumer independent.
|
||||
|
||||
These fixtures exist for the CALIBRATION CONTRACT (tests/grading/
|
||||
test_calibration.py): the mock provider scripts archetype-mapped scores and
|
||||
the test asserts the ORDERING the rubric must eventually enforce. They are
|
||||
NOT an LLM quality benchmark — see the test module docstring for the honest
|
||||
scope statement.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Final
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..telemetry.models import TelemetryEvent
|
||||
|
||||
_T0: Final = datetime(2026, 9, 12, 0, 0, 0, tzinfo=UTC)
|
||||
|
||||
#: Event spec: (seq, kind, payload, seconds-since-session-start).
|
||||
EventSpec = tuple[int, str, dict, float]
|
||||
|
||||
|
||||
class TraceFixture(BaseModel):
|
||||
"""One named synthetic trace + its v0.2 scenario alignment (D-021).
|
||||
|
||||
`event_specs` is the durable, session-state-free description; `events`
|
||||
materializes fresh TelemetryEvent rows from it on every access.
|
||||
"""
|
||||
|
||||
model_config = {"frozen": True}
|
||||
|
||||
fixture_id: str # "<v0.2 scenario id>::<archetype>-trace"
|
||||
archetype: str # "strong" | "lazy" | "struggling"
|
||||
aligned_scenario_id: str # the v0.2 corpus scenario this fixture mirrors
|
||||
competency_id: str # corpus competency id space (stack-orchestration-c00N)
|
||||
task_id: str # trace task id (grading operates on (learner, task))
|
||||
learner_id: str
|
||||
event_specs: tuple[EventSpec, ...] = Field(default=())
|
||||
|
||||
@property
|
||||
def events(self) -> list[TelemetryEvent]:
|
||||
"""FRESH TelemetryEvent instances — safe to ingest into any store.
|
||||
|
||||
Never cache these: an instance flushed by one SQLite session
|
||||
carries persistent identity, and re-appending it elsewhere no-ops.
|
||||
"""
|
||||
return [
|
||||
TelemetryEvent(
|
||||
learner_id=self.learner_id,
|
||||
task_id=self.task_id,
|
||||
seq=seq,
|
||||
kind=kind,
|
||||
payload=payload,
|
||||
ts=_T0 + timedelta(seconds=offset_s),
|
||||
sandbox_id="sbx-calibration",
|
||||
)
|
||||
for seq, kind, payload, offset_s in self.event_specs
|
||||
]
|
||||
|
||||
def description(self) -> str:
|
||||
return (
|
||||
f"{self.fixture_id} (archetype={self.archetype}, aligned="
|
||||
f"{self.aligned_scenario_id}, competency={self.competency_id})"
|
||||
)
|
||||
|
||||
|
||||
class _Builder:
|
||||
"""Seq-accurate event-spec builder for one (learner, task) pair."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.specs: list[EventSpec] = []
|
||||
self._seq = 0
|
||||
|
||||
def add(self, kind: str, payload: dict | None, at: float) -> None:
|
||||
self.specs.append((self._seq, kind, payload or {}, at))
|
||||
self._seq += 1
|
||||
|
||||
|
||||
def _strong_builder_specs() -> list[EventSpec]:
|
||||
"""Iterative debugging: tests early, tight edit→test loops, eventual pass.
|
||||
|
||||
Mirrors `lab-scenario-strong` (v0.2: keystrokes → file_save → run_tests →
|
||||
test_pass, c002) at full telemetry fidelity — every failed cycle is
|
||||
closed by a targeted edit followed by a re-run that passes.
|
||||
"""
|
||||
b = _Builder()
|
||||
b.add("activity", {"state": "starting"}, 0)
|
||||
b.add("file_diff", {"path": "planner.py", "added": 14}, 95) # small edit
|
||||
b.add("command", {"cmd": "pytest -q tests/test_planner.py"}, 120) # tests EARLY
|
||||
b.add("test_result", {"passed": True, "exit_code": 0}, 126)
|
||||
b.add("file_diff", {"path": "tool_node.py", "added": 22}, 180)
|
||||
b.add("command", {"cmd": "pytest -q"}, 275)
|
||||
b.add("test_result", {"passed": False, "exit_code": 1}, 281) # honest failure
|
||||
b.add("file_diff", {"path": "tool_node.py", "added": 6, "removed": 2}, 340) # targeted fix
|
||||
b.add("command", {"cmd": "pytest -q"}, 430)
|
||||
b.add("test_result", {"passed": True, "exit_code": 0}, 436) # cycle CLOSED
|
||||
b.add("command", {"cmd": "git commit -m 'tool node with retries'"}, 500)
|
||||
return b.specs
|
||||
|
||||
|
||||
def _lazy_builder_specs() -> list[EventSpec]:
|
||||
"""Paste-and-run: one large paste, a single LATE test run, instant pass.
|
||||
|
||||
Mirrors `lab-scenario-flagged` (v0.2: paste of 2,400 chars → run_tests →
|
||||
instant 6/6 pass, c003) at full telemetry fidelity — zero iteration, zero
|
||||
verification during construction, one terminal test run only.
|
||||
"""
|
||||
b = _Builder()
|
||||
b.add("activity", {"state": "starting"}, 0)
|
||||
b.add("file_diff", {"path": "eval.py", "added": 240, "removed": 0}, 30) # one bulk paste
|
||||
b.add("file_diff", {"path": "README.md", "added": 12}, 40)
|
||||
b.add("command", {"cmd": "npm run build"}, 45)
|
||||
b.add("run_result", {"exit_code": 0, "ok": True}, 60)
|
||||
b.add("command", {"cmd": "pytest -q"}, 520) # single LATE test run
|
||||
b.add("test_result", {"passed": True, "exit_code": 0}, 540) # instant pass
|
||||
return b.specs
|
||||
|
||||
|
||||
def _struggling_builder_specs() -> list[EventSpec]:
|
||||
"""Many cycles, none close: repeated failures, no eventual pass.
|
||||
|
||||
Mirrors `lab-scenario-struggling` (v0.2: repeated identical ImportErrors,
|
||||
idle gaps, no checkpoint, c002) at full telemetry fidelity — edits happen
|
||||
between failures, but the same failure recurs; no pass is ever reached.
|
||||
"""
|
||||
b = _Builder()
|
||||
b.add("activity", {"state": "starting"}, 0)
|
||||
b.add("file_diff", {"path": "main.py", "added": 40}, 20)
|
||||
b.add("command", {"cmd": "pytest -q"}, 210)
|
||||
b.add("test_result", {"passed": False, "exit_code": 1}, 215) # ImportError
|
||||
b.add("file_diff", {"path": "main.py", "added": 8, "removed": 3}, 300)
|
||||
b.add("command", {"cmd": "pytest -q"}, 520)
|
||||
b.add("test_result", {"passed": False, "exit_code": 1}, 525) # SAME error
|
||||
b.add("file_diff", {"path": "main.py", "added": 5}, 610)
|
||||
b.add("command", {"cmd": "pytest -q"}, 960)
|
||||
b.add("test_result", {"passed": False, "exit_code": 1}, 965) # STILL failing
|
||||
b.add("activity", {"state": "idle"}, 1500) # long idle
|
||||
b.add("activity", {"state": "idle"}, 2200)
|
||||
return b.specs
|
||||
|
||||
|
||||
#: Calibration learner — a member of the corpus learner-00N id space (D-021;
|
||||
#: learner-001/002 live in corpus/learner_context.py; grading fixtures use
|
||||
#: a third id so calibration traces never collide with mock-context reads).
|
||||
CALIBRATION_LEARNER_ID: Final = "learner-003"
|
||||
|
||||
|
||||
STRONG_BUILDER: Final = TraceFixture(
|
||||
fixture_id="lab-scenario-strong::strong-trace",
|
||||
archetype="strong",
|
||||
aligned_scenario_id="lab-scenario-strong",
|
||||
competency_id="stack-orchestration-c002",
|
||||
task_id="task-calibration-strong",
|
||||
learner_id=CALIBRATION_LEARNER_ID,
|
||||
event_specs=tuple(_strong_builder_specs()),
|
||||
)
|
||||
|
||||
LAZY_BUILDER: Final = TraceFixture(
|
||||
# v0.2's paste-and-run archetype is the "flagged" lab scenario (D-021):
|
||||
# large paste → instant test pass. "lazy builder" is that behavior
|
||||
# without the proctor flag; the alignment is behavioral, documented here.
|
||||
fixture_id="lab-scenario-flagged::lazy-trace",
|
||||
archetype="lazy",
|
||||
aligned_scenario_id="lab-scenario-flagged",
|
||||
competency_id="stack-orchestration-c003",
|
||||
task_id="task-calibration-lazy",
|
||||
learner_id=CALIBRATION_LEARNER_ID,
|
||||
event_specs=tuple(_lazy_builder_specs()),
|
||||
)
|
||||
|
||||
STRUGGLING_BUILDER: Final = TraceFixture(
|
||||
fixture_id="lab-scenario-struggling::struggling-trace",
|
||||
archetype="struggling",
|
||||
aligned_scenario_id="lab-scenario-struggling",
|
||||
competency_id="stack-orchestration-c002",
|
||||
task_id="task-calibration-struggling",
|
||||
learner_id=CALIBRATION_LEARNER_ID,
|
||||
event_specs=tuple(_struggling_builder_specs()),
|
||||
)
|
||||
|
||||
TRACE_FIXTURES: Final[dict[str, TraceFixture]] = {
|
||||
f.fixture_id: f
|
||||
for f in (STRONG_BUILDER, LAZY_BUILDER, STRUGGLING_BUILDER)
|
||||
}
|
||||
|
||||
|
||||
def get_trace_fixture(fixture_id: str) -> TraceFixture | None:
|
||||
return TRACE_FIXTURES.get(fixture_id)
|
||||
|
||||
|
||||
def digest_of(fixture: TraceFixture):
|
||||
"""Compute the digest for a fixture (pure compute; test-side helper)."""
|
||||
from ..grading.features import compute_digest
|
||||
|
||||
return compute_digest(fixture.events)
|
||||
@@ -0,0 +1,44 @@
|
||||
"""Employer B2B auth module (REQ-10-001, D-086..D-097, v0.10).
|
||||
|
||||
The 8th domain module inside apps/ai-service (no new apps, no new ports,
|
||||
no new dependencies — the D-031 boundary): pydantic contracts
|
||||
(`base.py`), the D-027 store family's eighth SQLite store
|
||||
(`store.py`, R-57/D-095), and the stdlib scrypt KDF (`passwords.py`,
|
||||
D-086/R-55). Claim evidence (`claim_evidence.py`) and the auth routes
|
||||
(`api/employers.py` via DI) land with their owning phases. No
|
||||
`service.py` anywhere (R-57) — routes compose the store via api/ DI.
|
||||
|
||||
G-315: email fields are plain `str` with stdlib-only validation —
|
||||
pydantic `EmailStr` is BANNED (email-validator would be the first new
|
||||
dependency, breaking R-66/D-086).
|
||||
|
||||
D-097: employer accounts are UNVERIFIED — no learner gate
|
||||
(require_verified_age/allowlist) is ever composed on an employer route.
|
||||
|
||||
Boundary (D-031 family discipline): `employers/` never imports `agents/`
|
||||
or `api/`; this package imports stdlib + pydantic + `config` only.
|
||||
"""
|
||||
|
||||
from .base import (
|
||||
CLAIM_STATUSES,
|
||||
ClaimEmployerRequest,
|
||||
ClaimStatus,
|
||||
EmployerAccount,
|
||||
EmployerClaim,
|
||||
EmployerMe,
|
||||
LoginEmployerRequest,
|
||||
RegisterEmployerRequest,
|
||||
TokenResponse,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CLAIM_STATUSES",
|
||||
"ClaimEmployerRequest",
|
||||
"ClaimStatus",
|
||||
"EmployerAccount",
|
||||
"EmployerClaim",
|
||||
"EmployerMe",
|
||||
"LoginEmployerRequest",
|
||||
"RegisterEmployerRequest",
|
||||
"TokenResponse",
|
||||
]
|
||||
@@ -0,0 +1,174 @@
|
||||
"""Employer auth pydantic contracts (REQ-10-001, D-086..D-097).
|
||||
|
||||
The wire + domain shapes for the employer module. `EmployerAccount` /
|
||||
`EmployerClaim` are the domain rows served over the wire; the request
|
||||
pair carries G-315 email discipline + D-086 password min length;
|
||||
`TokenResponse` / `EmployerMe` are the auth route responses.
|
||||
|
||||
G-315: email fields are plain `str` with stdlib-only validation —
|
||||
non-empty, exactly one "@", non-empty domain, stripped + lowercased on
|
||||
normalize. pydantic `EmailStr` is BANNED (email-validator is not in the
|
||||
venv and adding it would be the first new dependency, breaking
|
||||
R-66/D-086).
|
||||
|
||||
D-097: `EmployerAccount.email_verified` is `Literal[False]` — the
|
||||
account is UNVERIFIED, displayed as such; never presented as verified.
|
||||
|
||||
Id honesty (A-305, the family rule): every id in this module is an
|
||||
opaque handle minted UNIQUELY per creation call — never derived from
|
||||
user input or a deterministic hash of it.
|
||||
|
||||
Boundary (D-031 family): `employers/` never imports `agents/` or `api/`;
|
||||
stdlib + pydantic only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
#: Claim lifecycle states (D-089): every claim starts pending; an
|
||||
#: operator approve/reject decides it. Approved activates ownership
|
||||
#: (one account : one entity); rejection frees the entity for a new
|
||||
#: claim. Deferred to P2's claim routes — the shape lands now.
|
||||
ClaimStatus = Literal["pending", "approved", "rejected"]
|
||||
|
||||
#: Tuple mirror of the Literal (docs + tests import this; keeps the enum
|
||||
#: and any exhaustive switch in one place).
|
||||
CLAIM_STATUSES: tuple[ClaimStatus, ...] = (
|
||||
"pending",
|
||||
"approved",
|
||||
"rejected",
|
||||
)
|
||||
|
||||
#: G-315: password minimum length (D-086 honest validation floor).
|
||||
PASSWORD_MIN_LENGTH = 8
|
||||
|
||||
|
||||
def validate_email(value: str) -> str:
|
||||
"""G-315: stdlib-only email validation + normalization.
|
||||
|
||||
Non-empty (after strip), exactly one "@", non-empty local part and
|
||||
non-empty domain — normalized to stripped + lowercased. Deliberately
|
||||
NOT RFC-5322: the honest floor for an unverified pilot signup is
|
||||
structural (exactly one "@", a domain after it), never a regex
|
||||
masquerade. pydantic `EmailStr` is BANNED (it would require
|
||||
email-validator — the first new dependency, breaking R-66/D-086).
|
||||
"""
|
||||
normalized = value.strip().lower()
|
||||
if not normalized:
|
||||
raise ValueError("email must be non-empty")
|
||||
if normalized.count("@") != 1:
|
||||
raise ValueError('email must contain exactly one "@"')
|
||||
local, _, domain = normalized.partition("@")
|
||||
if not local:
|
||||
raise ValueError("email local part must be non-empty")
|
||||
if not domain:
|
||||
raise ValueError("email domain must be non-empty")
|
||||
return normalized
|
||||
|
||||
|
||||
class EmployerAccount(BaseModel):
|
||||
"""An employer-side login account (D-087).
|
||||
|
||||
`employer_id` is nullable-until-claim-approval (the UNIQUE
|
||||
one-account-one-entity constraint activates on approve, P2). The
|
||||
account is UNVERIFIED (D-097): `email_verified` is `Literal[False]`
|
||||
— no email verification exists in v0.10, so the field can never
|
||||
carry `True`, and no learner gate is ever composed on an employer
|
||||
route.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
#: G-315: plain str, stdlib-validated + normalized.
|
||||
email: str = Field(min_length=3)
|
||||
#: None until a claim is approved (P2).
|
||||
employer_id: str | None = None
|
||||
created_at: datetime
|
||||
#: D-097: UNVERIFIED — displayed as such, never as verified.
|
||||
email_verified: Literal[False] = False
|
||||
|
||||
|
||||
class EmployerClaim(BaseModel):
|
||||
"""An account's claim on an EmployerRow entity (D-089).
|
||||
|
||||
Advisory evidence is computed at claim time (P2's claim_evidence.py)
|
||||
and stored as provenance JSON; the operator decision activates
|
||||
ownership. Auto-approval is deferred (D-089: without email
|
||||
verification the match is forgeable).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
account_id: str = Field(min_length=1)
|
||||
employer_id: str = Field(min_length=1)
|
||||
status: ClaimStatus
|
||||
evidence: dict[str, Any] = Field(default_factory=dict)
|
||||
created_at: datetime
|
||||
#: Operator provenance (decided_by/decided_at) — None while pending.
|
||||
decided_by: str | None = None
|
||||
decided_at: datetime | None = None
|
||||
|
||||
|
||||
class RegisterEmployerRequest(BaseModel):
|
||||
"""POST /v1/employers/auth/register body (Task 1-2-01).
|
||||
|
||||
Honest validation (G-315 + D-086): stdlib email normalization; the
|
||||
password carries a min-length-8 floor. Neither field is ever
|
||||
echoed back by a 422 (the pii_fields scrub lands with the routes).
|
||||
"""
|
||||
|
||||
email: str = Field(min_length=3)
|
||||
password: str = Field(min_length=PASSWORD_MIN_LENGTH)
|
||||
|
||||
@field_validator("email")
|
||||
@classmethod
|
||||
def _validate_email(cls, value: str) -> str:
|
||||
return validate_email(value)
|
||||
|
||||
|
||||
class LoginEmployerRequest(RegisterEmployerRequest):
|
||||
"""POST /v1/employers/auth/login body — same honest validation
|
||||
(G-315 email + password floor); subclass so both routes share the
|
||||
contract and drift is structurally impossible."""
|
||||
|
||||
|
||||
class ClaimEmployerRequest(BaseModel):
|
||||
"""POST /v1/employers/claims body (Task 2-1-03, D-089): the
|
||||
marketplace EmployerRow id being claimed. The entity's existence
|
||||
is checked against the marketplace store at the route (404
|
||||
unknown); evidence is computed from the session account's email +
|
||||
the entity's website AT CLAIM TIME (R-65)."""
|
||||
|
||||
employer_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class TokenResponse(BaseModel):
|
||||
"""Login/register session response (D-087): the bearer token (raw,
|
||||
exactly once on this response — only sha256(token) is stored) and
|
||||
its expiry (login + SESSION_TTL_DAYS, evaluated lazily at read)."""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
token: str = Field(min_length=1)
|
||||
expires_at: datetime
|
||||
|
||||
|
||||
class EmployerMe(BaseModel):
|
||||
"""GET /v1/employers/auth/me response (Task 1-2-01): the account
|
||||
(unverified email displayed as such, D-097) + the claimed entity +
|
||||
the claim state — None until a claim exists (P2)."""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
account: EmployerAccount
|
||||
#: None until a claim is approved (P2's decide_claim sets it).
|
||||
claimed_employer_id: str | None = None
|
||||
#: None until a claim is filed; the claim state (pending/approved/
|
||||
#: rejected) once one exists (P2).
|
||||
claim: EmployerClaim | None = None
|
||||
@@ -0,0 +1,115 @@
|
||||
"""Advisory claim evidence — stdlib registrable-domain normalization (R-65, D-089).
|
||||
|
||||
The signal this module computes is ADVISORY and never displayed as
|
||||
verification (A-304): the operator decision activates a claim, the
|
||||
domain match merely informs it. Auto-approval is deferred (D-089) —
|
||||
without email verification the match is forgeable.
|
||||
|
||||
R-65 normalization is stdlib-only (zero-dep discipline, R-66): no PSL
|
||||
library, no tldextract. `urllib.parse.urlsplit(website).hostname`
|
||||
lowercases and strips scheme/path ("https://example.com/openai" →
|
||||
"example.com"); the email domain is taken after the last "@" and
|
||||
lowercased (never the local part — it rides nowhere in the evidence).
|
||||
|
||||
The comparison is a simple SUFFIX match: two domains match when equal
|
||||
or when one endswith "." + the other. KNOWN LIMIT (documented per the
|
||||
task, accepted for an advisory signal): a multi-label public suffix
|
||||
over-matches — "mail.co.uk" vs "co.uk" counts as a match even though
|
||||
"co.uk" is a public suffix, not a registrable domain, because a
|
||||
subdomain of a registrable domain is the intended match
|
||||
("shop.acme.com" vs "acme.com") and suffix equality cannot tell the
|
||||
two cases apart. That over-match is ACCEPTABLE HERE: the seeded
|
||||
corpus's websites are example.com placeholders (match=false
|
||||
corpus-wide at pilot, R-65) and the signal never gates anything — it
|
||||
is recorded provenance an operator reads. A PSL would be the first
|
||||
new dependency (R-66) — rejected.
|
||||
|
||||
Evidence shape per D-089: {account_email_domain, entity_website_domain,
|
||||
match} — computed AT CLAIM TIME (Task 2-1-03) and stored as provenance
|
||||
JSON on the claim row; it is a snapshot, never recomputed on read.
|
||||
|
||||
Boundary (D-031 family): stdlib + pydantic only; this module never
|
||||
imports agents/ or api/ — it is a pure function family over strings.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
|
||||
def normalize_email_domain(email: str) -> str:
|
||||
"""The email's domain after the last "@" — lowercased (never the
|
||||
local part). The last-@ split tolerates local parts containing "@"
|
||||
under a quoting convention; honest callers pass validated emails
|
||||
(G-315: exactly one "@" by the time an account exists), so an
|
||||
"@"-less string is an honest-absence case: ""."""
|
||||
if "@" not in email:
|
||||
return ""
|
||||
_, _, domain = email.rpartition("@")
|
||||
return domain.lower()
|
||||
|
||||
|
||||
def normalize_website_domain(website: str) -> str:
|
||||
"""The website's host, lowercased — scheme, path, port, and any
|
||||
userinfo stripped ("HTTPS://Example.COM/openai" → "example.com").
|
||||
urlsplit parses a SCHEME-qualified URL: a bare "example.com" has no
|
||||
netloc, so a scheme is prepended for the parse ("example.com" →
|
||||
"example.com"; "example.com:8443" likewise — the port drops with
|
||||
the netloc, evidence is registrable not origin-scoped). An
|
||||
unparseable/empty website yields "" (the honest absence the route
|
||||
records, never an exception)."""
|
||||
candidate = website.strip()
|
||||
if not candidate:
|
||||
return ""
|
||||
try:
|
||||
parts = urlsplit(candidate)
|
||||
if parts.scheme or parts.netloc:
|
||||
# Scheme-qualified ("https://…", "//host/…") — parse as-is.
|
||||
# Non-URL schemes ("mailto:…") land here with no netloc →
|
||||
# hostname None (honest absence).
|
||||
hostname = parts.hostname
|
||||
else:
|
||||
# Bare host (no scheme): urlsplit would treat it as a
|
||||
# path — prepending "//" makes it an authority-only URL.
|
||||
hostname = urlsplit(f"//{candidate}").hostname
|
||||
except ValueError:
|
||||
# e.g. an unterminated IPv6 literal — evidence stays advisory,
|
||||
# never a 500 on the claim route.
|
||||
return ""
|
||||
if hostname is None:
|
||||
return ""
|
||||
if any(char.isspace() for char in hostname):
|
||||
return "" # not a domain — honest absence, never junk
|
||||
return hostname
|
||||
|
||||
|
||||
def domains_match(account_email_domain: str, entity_website_domain: str) -> bool:
|
||||
"""Simple SUFFIX match (R-65): True when equal or when one endswith
|
||||
"." + the other. Honest absent input: either side empty → False
|
||||
(nothing claimed is nothing matched)."""
|
||||
if not account_email_domain or not entity_website_domain:
|
||||
return False
|
||||
if account_email_domain == entity_website_domain:
|
||||
return True
|
||||
if account_email_domain.endswith("." + entity_website_domain):
|
||||
return True
|
||||
if entity_website_domain.endswith("." + account_email_domain):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def compute_claim_evidence(email: str, website: str) -> dict:
|
||||
"""The D-089 evidence dict, computed at claim time:
|
||||
|
||||
{account_email_domain, entity_website_domain, match}
|
||||
|
||||
Pure function over the account email + the entity website — the
|
||||
claim route snapshots it onto the claim row as provenance JSON.
|
||||
"""
|
||||
account_email_domain = normalize_email_domain(email)
|
||||
entity_website_domain = normalize_website_domain(website)
|
||||
return {
|
||||
"account_email_domain": account_email_domain,
|
||||
"entity_website_domain": entity_website_domain,
|
||||
"match": domains_match(account_email_domain, entity_website_domain),
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
"""Stdlib scrypt password KDF (D-086, R-55, REQ-10-001).
|
||||
|
||||
`hashlib.scrypt` (stdlib, OpenSSL-backed — D-086: zero new
|
||||
dependencies) with per-password random salt and a SELF-DESCRIBING
|
||||
storage format, `scrypt$16384$8$1$<salt_b64>$<hash_b64>`: the KDF
|
||||
parameters ride the row, so params evolve without invalidating
|
||||
existing rows (verify parses them from the stored string, never from
|
||||
the module constants).
|
||||
|
||||
Cost budget (R-55 probe, this box's venv — Python 3.11.2): ~47-59 ms
|
||||
per hash at N=2^14/r=8/p=1/dklen=32, 128·N·r = 16 MiB working set.
|
||||
Register and login each perform exactly ONE derivation — inside
|
||||
D-096's "tens of ms, honest" budget. Burst risk (open-risk-1): a
|
||||
login burst stacks single-threaded derivations; BOTH throttle windows
|
||||
(Wave 1-2, D-088/G-320) run BEFORE this KDF so a 429'd or capped
|
||||
attempt never burns KDF CPU.
|
||||
|
||||
maxmem discipline (R-55): CPython's default 32 MiB admits the 16 MiB
|
||||
working set with NO argument; the pin below is exactly 33554432 for
|
||||
explicitness — never lower (1024 KiB raises ValueError: memory limit
|
||||
exceeded; the floor is ~17 MiB).
|
||||
|
||||
Tests run the REAL KDF (D-096: no fake-hash path) and assert
|
||||
accept/reject behavior only — never hash bytes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import secrets
|
||||
|
||||
#: KDF parameters (D-086) — pinned by test (the PAST_DUE_GRACE_DAYS
|
||||
#: shape). N=2^14, r=8, p=1, dklen=32: the R-55 probe-verified
|
||||
#: cost/memory point (~50 ms/hash, 16 MiB).
|
||||
SCRYPT_N = 2**14
|
||||
SCRYPT_R = 8
|
||||
SCRYPT_P = 1
|
||||
SCRYPT_DKLEN = 32
|
||||
|
||||
#: Per-password salt length (bytes) — `secrets.token_bytes(16)`; the
|
||||
#: random salt is what makes two hashes of one password differ.
|
||||
_SALT_BYTES = 16
|
||||
|
||||
#: R-55: pinned for explicitness — the default 32 MiB admits the
|
||||
#: 16 MiB working set; NEVER pin below ~17 MiB (1024 KiB raises).
|
||||
_SCRYPT_MAXMEM = 33554432 # 32 MiB
|
||||
|
||||
#: Storage format marker — self-describing rows start with this.
|
||||
_ALGORITHM = "scrypt"
|
||||
|
||||
|
||||
def _b64e(raw: bytes) -> str:
|
||||
return base64.b64encode(raw).decode("ascii")
|
||||
|
||||
|
||||
def _b64d(encoded: str) -> bytes:
|
||||
return base64.b64decode(encoded.encode("ascii"))
|
||||
|
||||
|
||||
def _derive(
|
||||
password: str, *, salt: bytes, n: int, r: int, p: int, dklen: int
|
||||
) -> bytes:
|
||||
"""One scrypt derivation — the ONLY derivation path (register and
|
||||
verify both route through here, so their costs stay identical)."""
|
||||
return hashlib.scrypt(
|
||||
password.encode("utf-8"),
|
||||
salt=salt,
|
||||
n=n,
|
||||
r=r,
|
||||
p=p,
|
||||
dklen=dklen,
|
||||
maxmem=_SCRYPT_MAXMEM,
|
||||
)
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
"""Hash a password with a fresh random salt → the self-describing
|
||||
`scrypt$16384$8$1$<salt_b64>$<hash_b64>` string.
|
||||
|
||||
The module constants bake in at hash time; verify re-reads them
|
||||
from the stored row, so a future param bump never invalidates
|
||||
existing rows.
|
||||
"""
|
||||
salt = secrets.token_bytes(_SALT_BYTES)
|
||||
derived = _derive(
|
||||
password,
|
||||
salt=salt,
|
||||
n=SCRYPT_N,
|
||||
r=SCRYPT_R,
|
||||
p=SCRYPT_P,
|
||||
dklen=SCRYPT_DKLEN,
|
||||
)
|
||||
return "$".join(
|
||||
(
|
||||
_ALGORITHM,
|
||||
str(SCRYPT_N),
|
||||
str(SCRYPT_R),
|
||||
str(SCRYPT_P),
|
||||
_b64e(salt),
|
||||
_b64e(derived),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def verify_password(stored: str, password: str) -> bool:
|
||||
"""Verify a candidate against a stored self-describing hash.
|
||||
|
||||
Parses the params from the stored string (never the module
|
||||
constants — a row hashed under different params still verifies),
|
||||
re-derives with the SAME pinned maxmem, compares via
|
||||
`hmac.compare_digest` (constant-time). Any malformed stored string
|
||||
is an honest False — never a raise (the auth route maps it to 401).
|
||||
"""
|
||||
parts = stored.split("$")
|
||||
if len(parts) != 6 or parts[0] != _ALGORITHM:
|
||||
return False
|
||||
try:
|
||||
n, r, p = (int(part) for part in parts[1:4])
|
||||
salt = _b64d(parts[4])
|
||||
expected = _b64d(parts[5])
|
||||
if n < 1 or r < 1 or p < 1 or not salt or not expected:
|
||||
return False
|
||||
derived = _derive(
|
||||
password, salt=salt, n=n, r=r, p=p, dklen=len(expected)
|
||||
)
|
||||
except (ValueError, TypeError):
|
||||
# Malformed stored row (bad ints, bad base64, scrypt rejection)
|
||||
# — an honest reject, never a 500 on the auth path.
|
||||
return False
|
||||
return hmac.compare_digest(derived, expected)
|
||||
@@ -0,0 +1,710 @@
|
||||
"""EmployerStore — the 8th D-027 store family (D-087/D-095, REQ-10-001).
|
||||
|
||||
Four tables on the shared `Settings().db_path` (one SQLite file with the
|
||||
seven prior stores): `employer_accounts`, `employer_sessions`,
|
||||
`employer_claims`, `employer_audit_events`. All `employer_`-prefixed —
|
||||
bare `employer` is marketplace's EmployerRow (marketplace/store.py:144).
|
||||
Mechanics mirror the probe-verified billing/store.py exactly (R-57):
|
||||
WAL / synchronous=NORMAL / busy_timeout / foreign_keys pragmas at
|
||||
connect time, `sa_type` portable columns (String/JSON for Postgres
|
||||
parity), `_validate()`/`_normalize()` hooks, Protocol + SQLite impl
|
||||
pair, surfaced IntegrityError, tz-aware → naive → tz-aware boundary
|
||||
normalization, one-time idempotent `create_schema()` (D-095 — the
|
||||
tables join an existing shared DB at next boot, zero migration).
|
||||
|
||||
Guards baked into the schema (D-087):
|
||||
- `employer_accounts.email` UNIQUE — one account per email; the
|
||||
register route's check-first 409 is the fast path, the constraint
|
||||
is the race backstop and its IntegrityError SURFACES.
|
||||
- `employer_accounts.employer_id` UNIQUE nullable — the one-account-
|
||||
one-entity constraint, activates on claim approval (P2).
|
||||
- `employer_sessions.token_hash` UNIQUE — stores ONLY
|
||||
`sha256(token)`, NEVER the raw token (D-087): a DB leak yields
|
||||
nothing usable against the Authorization header namespace.
|
||||
- G-306: session expiry is evaluated LAZILY at read time against
|
||||
`expires_at` (login + SESSION_TTL_DAYS) — an expired row reads as
|
||||
absent; no background timer, no writes on evaluation.
|
||||
- `employer_claims` landed in P1 per D-095's one-time create_all;
|
||||
its claim methods (insert_claim/decide_claim/list_claims/get_claim)
|
||||
+ the G-316 partial UNIQUE index arrive with P2 (D-089).
|
||||
|
||||
Boundary (D-027 family): `employers/` never imports `agents/`/`api/`;
|
||||
this module imports config + base only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import sqlite3
|
||||
import uuid
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any, Protocol
|
||||
|
||||
from sqlalchemy import Index, UniqueConstraint, event, text
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.types import JSON, String
|
||||
from sqlmodel import Field, Session, SQLModel, col, create_engine, select
|
||||
|
||||
from ..config import Settings
|
||||
from .base import CLAIM_STATUSES, validate_email
|
||||
|
||||
#: D-087: a login session's TTL. Module constant (not settings) —
|
||||
#: pinned by test (the PAST_DUE_GRACE_DAYS shape); evaluation is lazy
|
||||
#: at read time (G-306), never a timer.
|
||||
SESSION_TTL_DAYS = 7
|
||||
|
||||
_TABLES = (
|
||||
"employer_accounts",
|
||||
"employer_sessions",
|
||||
"employer_claims",
|
||||
"employer_audit_events",
|
||||
)
|
||||
|
||||
#: Audit event vocabulary: `auth` = the Wave 1-2 register/logout
|
||||
#: account events; `claim` = P2's claim/decide entity events; `manual`
|
||||
#: = operator action provenance (the billing kind set, employer-side).
|
||||
_AUDIT_EVENT_KINDS = ("auth", "claim", "manual")
|
||||
|
||||
|
||||
class AccountRow(SQLModel, table=True):
|
||||
"""An employer login account row (D-087).
|
||||
|
||||
Bare Literal annotations crash sqlmodel's column inference; the
|
||||
family pattern (explicit sa_type + _validate hook) is used where
|
||||
needed. `email` is normalized via base.validate_email (G-315) at
|
||||
the store boundary — routes normalize too; the store re-derives,
|
||||
never trusts."""
|
||||
|
||||
__tablename__ = "employer_accounts"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("email", name="uq_account_email"),
|
||||
UniqueConstraint("employer_id", name="uq_account_employer"),
|
||||
)
|
||||
|
||||
id: str = Field(primary_key=True, sa_type=String)
|
||||
#: G-315: plain str, stdlib-validated + stripped/lowercased.
|
||||
email: str = Field(sa_type=String)
|
||||
#: Self-describing scrypt string (passwords.py) — never a raw hash.
|
||||
password_hash: str = Field(sa_type=String)
|
||||
#: None until a claim is APPROVED (P2) — the UNIQUE nullable
|
||||
#: one-account-one-entity constraint.
|
||||
employer_id: str | None = Field(default=None, sa_type=String)
|
||||
created_at: datetime | None = Field(default=None)
|
||||
|
||||
def _validate(self) -> None:
|
||||
if not self.id:
|
||||
raise ValueError("id must be non-empty")
|
||||
if not self.password_hash:
|
||||
raise ValueError("password_hash must be non-empty")
|
||||
if self.employer_id is not None and not self.employer_id:
|
||||
raise ValueError("employer_id must be non-empty when set")
|
||||
|
||||
def _normalize(self) -> None:
|
||||
# G-315 at the store boundary: validate + strip + lowercase
|
||||
# (raises ValueError on a dishonest email — never trusts).
|
||||
self.email = validate_email(self.email)
|
||||
self.created_at = _as_utc(self.created_at) if self.created_at else None
|
||||
|
||||
|
||||
class SessionRow(SQLModel, table=True):
|
||||
"""A hashed server-side session row (D-087).
|
||||
|
||||
`token_hash` is sha256(token) — the raw token is handed to the
|
||||
client EXACTLY ONCE on login and never persisted anywhere. The
|
||||
TTL is evaluated LAZILY at read (G-306): an expired row reads as
|
||||
absent; no timer ever rewrites or deletes it."""
|
||||
|
||||
__tablename__ = "employer_sessions"
|
||||
__table_args__ = (UniqueConstraint("token_hash", name="uq_session_token_hash"),)
|
||||
|
||||
id: str = Field(primary_key=True, sa_type=String)
|
||||
token_hash: str = Field(sa_type=String)
|
||||
account_id: str = Field(index=True, sa_type=String)
|
||||
created_at: datetime | None = Field(default=None)
|
||||
expires_at: datetime | None = Field(default=None)
|
||||
|
||||
def _validate(self) -> None:
|
||||
if not self.id:
|
||||
raise ValueError("id must be non-empty")
|
||||
if not self.token_hash:
|
||||
raise ValueError("token_hash must be non-empty")
|
||||
if not self.account_id:
|
||||
raise ValueError("account_id must be non-empty")
|
||||
|
||||
def _normalize(self) -> None:
|
||||
self.created_at = _as_utc(self.created_at) if self.created_at else None
|
||||
self.expires_at = _as_utc(self.expires_at) if self.expires_at else None
|
||||
|
||||
|
||||
class ClaimRow(SQLModel, table=True):
|
||||
"""An account's claim on a marketplace EmployerRow entity (D-089).
|
||||
|
||||
The table lands now per D-095's one-time create_all; the claim
|
||||
methods (insert_claim/decide_claim/list_claims) arrive with P2.
|
||||
`evidence` is the advisory claim-time provenance JSON
|
||||
{account_email_domain, entity_website_domain, match} (R-65). The
|
||||
decided_* pair carries operator provenance and is valid only on a
|
||||
decided row (the past_due_at pairing discipline).
|
||||
|
||||
G-316: the partial UNIQUE index uq_pending_claim_per_entity on
|
||||
employer_id WHERE status='pending' makes one-PENDING-claim-per-
|
||||
entity a STRUCTURAL guard — check-first alone is TOCTOU. The
|
||||
approved-owner arm is already structural: employer_accounts.
|
||||
employer_id UNIQUE (one account : one entity). Rejection frees
|
||||
the entity — a partial index over status='pending' never touches
|
||||
rejected rows.
|
||||
"""
|
||||
|
||||
__tablename__ = "employer_claims"
|
||||
__table_args__ = (
|
||||
Index(
|
||||
"uq_pending_claim_per_entity",
|
||||
"employer_id",
|
||||
unique=True,
|
||||
sqlite_where=text("status='pending'"),
|
||||
),
|
||||
)
|
||||
|
||||
id: str = Field(primary_key=True, sa_type=String)
|
||||
account_id: str = Field(index=True, sa_type=String)
|
||||
employer_id: str = Field(index=True, sa_type=String)
|
||||
status: str = Field(default="pending", sa_type=String)
|
||||
evidence: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
created_at: datetime | None = Field(default=None)
|
||||
decided_by: str | None = Field(default=None, sa_type=String)
|
||||
decided_at: datetime | None = Field(default=None)
|
||||
|
||||
def _validate(self) -> None:
|
||||
if not self.id or not self.account_id or not self.employer_id:
|
||||
raise ValueError("id, account_id, and employer_id must be non-empty")
|
||||
if self.status not in CLAIM_STATUSES:
|
||||
raise ValueError(f"invalid claim status {self.status!r}")
|
||||
if self.status == "pending" and (
|
||||
self.decided_by is not None or self.decided_at is not None
|
||||
):
|
||||
raise ValueError("decided_by/decided_at are set only on a decided claim")
|
||||
if self.status in ("approved", "rejected") and self.decided_at is None:
|
||||
raise ValueError("decided_at must be set on a decided claim")
|
||||
|
||||
def _normalize(self) -> None:
|
||||
self.created_at = _as_utc(self.created_at) if self.created_at else None
|
||||
self.decided_at = _as_utc(self.decided_at) if self.decided_at else None
|
||||
|
||||
|
||||
class AuditEventRow(SQLModel, table=True):
|
||||
"""Employer-side audit trail — the BillingEventRow audit shape
|
||||
(billing/store.py:337-359): auth events (register/logout), claim
|
||||
events (P2), and operator actions with provenance."""
|
||||
|
||||
__tablename__ = "employer_audit_events"
|
||||
|
||||
id: int | None = Field(default=None, primary_key=True)
|
||||
account_id: str | None = Field(default=None, index=True, sa_type=String)
|
||||
employer_id: str | None = Field(default=None, index=True, sa_type=String)
|
||||
kind: str = Field(sa_type=String)
|
||||
name: str = Field(sa_type=String)
|
||||
detail: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
operator: str | None = Field(default=None, sa_type=String)
|
||||
created_at: datetime | None = Field(default=None)
|
||||
|
||||
def _validate(self) -> None:
|
||||
if self.kind not in _AUDIT_EVENT_KINDS:
|
||||
raise ValueError(f"invalid audit event kind {self.kind!r}")
|
||||
if not self.name:
|
||||
raise ValueError("name must be non-empty")
|
||||
|
||||
def _normalize(self) -> None:
|
||||
self.created_at = _as_utc(self.created_at) if self.created_at else None
|
||||
|
||||
|
||||
def _as_utc(ts: datetime) -> datetime:
|
||||
"""SQLite stores naive; read paths re-label tz-aware UTC (D-027 pattern)."""
|
||||
if ts.tzinfo is None:
|
||||
return ts.replace(tzinfo=UTC)
|
||||
return ts
|
||||
|
||||
|
||||
def _hash_token(token: str) -> str:
|
||||
"""D-087: sessions store ONLY sha256(token) — never the raw token."""
|
||||
return hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _constraint_error(message: str, params: dict[str, Any]) -> IntegrityError:
|
||||
"""Hand-raised IntegrityError (the guard arms of decide_claim) —
|
||||
the caller's honest 409, never a silent overwrite. The original
|
||||
exception is the raised guard itself."""
|
||||
return IntegrityError(message, params, ValueError(message))
|
||||
|
||||
|
||||
def _sqlite_connect(dbapi_connection: sqlite3.Connection, _: object) -> None:
|
||||
"""Per-connection pragmas — mirrors the other D-027 stores."""
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA synchronous=NORMAL")
|
||||
cursor.execute("PRAGMA busy_timeout=5000")
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close()
|
||||
|
||||
|
||||
class EmployerStore(Protocol):
|
||||
"""Persistence contract for employer auth (D-087/D-095)."""
|
||||
|
||||
def create_schema(self) -> None:
|
||||
"""Ensure all four tables exist (idempotent — shared DB file)."""
|
||||
...
|
||||
|
||||
def insert_account(self, *, email: str, password_hash: str) -> tuple[AccountRow, bool]:
|
||||
"""Check-first insert by email → (row, created); the EXISTING
|
||||
row rides a duplicate register (route → 409). The UNIQUE
|
||||
constraint is the race backstop — its IntegrityError SURFACES
|
||||
(never swallowed, never re-masked as created=False)."""
|
||||
...
|
||||
|
||||
def get_account_by_email(self, email: str) -> AccountRow | None:
|
||||
"""Point lookup by the normalized email (login path)."""
|
||||
...
|
||||
|
||||
def get_account(self, account_id: str) -> AccountRow | None:
|
||||
"""Point lookup by id (require_employer_session's account arm)."""
|
||||
...
|
||||
|
||||
def mint_session(
|
||||
self, *, account_id: str, token: str, now: datetime | None = None
|
||||
) -> SessionRow:
|
||||
"""Store sha256(token) with expires_at = now + SESSION_TTL_DAYS
|
||||
(D-087). The raw token is NEVER persisted; a duplicate
|
||||
token_hash surfaces as IntegrityError (the constraint is the
|
||||
only guard — a token is minted fresh per login)."""
|
||||
...
|
||||
|
||||
def get_session_by_token_hash(
|
||||
self, token_hash: str, *, now: datetime | None = None
|
||||
) -> SessionRow | None:
|
||||
"""Point lookup by sha256(token) with LAZY TTL (G-306): an
|
||||
expired row reads as ABSENT — no timer, no writes on
|
||||
evaluation; the row itself stays until revoked."""
|
||||
...
|
||||
|
||||
def revoke_session(self, token_hash: str) -> bool:
|
||||
"""DELETE the session row (logout); True when a row was
|
||||
removed, False when it was already gone."""
|
||||
...
|
||||
|
||||
def append_audit_event(
|
||||
self,
|
||||
kind: str,
|
||||
name: str,
|
||||
*,
|
||||
account_id: str | None = None,
|
||||
employer_id: str | None = None,
|
||||
operator: str | None = None,
|
||||
detail: dict[str, Any] | None = None,
|
||||
) -> AuditEventRow:
|
||||
"""Audit log append (auth/claim/manual)."""
|
||||
...
|
||||
|
||||
# -- claims (D-089, REQ-10-002) ---------------------------------------
|
||||
|
||||
def insert_claim(
|
||||
self,
|
||||
*,
|
||||
account_id: str,
|
||||
employer_id: str,
|
||||
evidence: dict[str, Any] | None = None,
|
||||
) -> tuple[ClaimRow, bool]:
|
||||
"""Check-first pending-claim insert (the insert_application
|
||||
precedent, marketplace/store.py:341-347): an existing PENDING
|
||||
claim on the entity returns (existing_row, False) — the route's
|
||||
honest 409 fast path. G-316: the partial UNIQUE index
|
||||
uq_pending_claim_per_entity backstops the check-first TOCTOU —
|
||||
a raced duplicate insert SURFACES its IntegrityError (never
|
||||
swallowed, never re-masked as created=False); the caller maps
|
||||
it to the SAME 409. Rejected/decided rows never block: rejection
|
||||
frees the entity for a new claim."""
|
||||
...
|
||||
|
||||
def decide_claim(
|
||||
self,
|
||||
claim_id: str,
|
||||
*,
|
||||
decision: str,
|
||||
operator: str,
|
||||
now: datetime | None = None,
|
||||
) -> ClaimRow | None:
|
||||
"""Operator decision on a PENDING claim (D-089): approve → set
|
||||
employer_accounts.employer_id + decided_by/decided_at + a
|
||||
`claim` audit event; reject → decided_by/decided_at + the audit
|
||||
event (the entity stays free). The one-account-one-entity arm
|
||||
is STRUCTURAL: approving when the account already holds an
|
||||
entity, or the entity is already owned, raises a hand-raised
|
||||
IntegrityError (the caller's honest 409 — never a silent
|
||||
overwrite). Unknown claim id → None (route 404s)."""
|
||||
...
|
||||
|
||||
def list_claims(self, status: str | None = None) -> list[ClaimRow]:
|
||||
"""Claims newest-first; optional status filter (pending/
|
||||
approved/rejected — the admin queue's consume-free read path)."""
|
||||
...
|
||||
|
||||
def list_accounts(self) -> list[AccountRow]:
|
||||
"""All accounts, created-oldest-first (the admin accounts list —
|
||||
consume-free read path; the queue is pilot-sized)."""
|
||||
...
|
||||
|
||||
def list_audit_events(
|
||||
self,
|
||||
*,
|
||||
account_id: str | None = None,
|
||||
employer_id: str | None = None,
|
||||
kind: str | None = None,
|
||||
name: str | None = None,
|
||||
limit: int = 100,
|
||||
) -> list[AuditEventRow]:
|
||||
"""Audit trail, newest-first (the list_billing_events shape,
|
||||
billing/store.py:1122-1145); optional account/employer/kind/name
|
||||
filters — the queryable operator trail (D-089)."""
|
||||
...
|
||||
|
||||
def get_claim(self, claim_id: str) -> ClaimRow | None:
|
||||
"""Point lookup by id (the admin review path)."""
|
||||
...
|
||||
|
||||
def close(self) -> None:
|
||||
"""Dispose the engine (app shutdown)."""
|
||||
...
|
||||
|
||||
|
||||
class SQLiteEmployerStore:
|
||||
"""SQLite implementation of EmployerStore (D-027/D-087)."""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
self._db_path: Path = db_path if db_path is not None else Settings().db_path
|
||||
self._engine = create_engine(
|
||||
f"sqlite:///{self._db_path}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
event.listen(self._engine, "connect", _sqlite_connect)
|
||||
self.create_schema()
|
||||
|
||||
def create_schema(self) -> None:
|
||||
"""Ensure all four tables exist (idempotent — shared DB file)."""
|
||||
SQLModel.metadata.create_all(self._engine)
|
||||
|
||||
# -- accounts -----------------------------------------------------------
|
||||
|
||||
def insert_account(self, *, email: str, password_hash: str) -> tuple[AccountRow, bool]:
|
||||
"""Check-first insert (the insert_application pattern): an
|
||||
existing email returns (existing_row, False) — the register
|
||||
route's honest 409. The UNIQUE constraint backstops the
|
||||
check-first TOCTOU; its IntegrityError is SURFACED, never
|
||||
swallowed."""
|
||||
row = AccountRow(
|
||||
id=f"acct-{uuid.uuid4().hex[:16]}",
|
||||
email=email,
|
||||
password_hash=password_hash,
|
||||
employer_id=None,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
row._validate()
|
||||
row._normalize()
|
||||
with Session(self._engine) as session:
|
||||
existing = session.exec(
|
||||
select(AccountRow).where(AccountRow.email == row.email)
|
||||
).one_or_none()
|
||||
if existing is not None:
|
||||
existing._normalize()
|
||||
return existing, False
|
||||
session.add(row)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
row._normalize()
|
||||
return row, True
|
||||
|
||||
def get_account_by_email(self, email: str) -> AccountRow | None:
|
||||
with Session(self._engine) as session:
|
||||
row = session.exec(select(AccountRow).where(AccountRow.email == email)).one_or_none()
|
||||
if row is None:
|
||||
return None
|
||||
row._normalize()
|
||||
return row
|
||||
|
||||
def get_account(self, account_id: str) -> AccountRow | None:
|
||||
with Session(self._engine) as session:
|
||||
row = session.get(AccountRow, account_id)
|
||||
if row is None:
|
||||
return None
|
||||
row._normalize()
|
||||
return row
|
||||
|
||||
# -- sessions (D-087: hashed-at-rest; G-306: lazy TTL) -------------------
|
||||
|
||||
def mint_session(
|
||||
self, *, account_id: str, token: str, now: datetime | None = None
|
||||
) -> SessionRow:
|
||||
if not token:
|
||||
raise ValueError("token must be non-empty")
|
||||
moment = _as_utc(now) if now is not None else datetime.now(UTC)
|
||||
row = SessionRow(
|
||||
id=f"ses-{uuid.uuid4().hex[:16]}",
|
||||
# D-087: ONLY the sha256 ever persists — the raw token
|
||||
# lives in the login response and nowhere else.
|
||||
token_hash=_hash_token(token),
|
||||
account_id=account_id,
|
||||
created_at=moment,
|
||||
expires_at=moment + timedelta(days=SESSION_TTL_DAYS),
|
||||
)
|
||||
row._validate()
|
||||
row._normalize()
|
||||
with Session(self._engine) as session:
|
||||
session.add(row)
|
||||
# The UNIQUE(token_hash) constraint surfaces a duplicate
|
||||
# mint as IntegrityError — the caller decides; never
|
||||
# swallowed here.
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
row._normalize()
|
||||
return row
|
||||
|
||||
def get_session_by_token_hash(
|
||||
self, token_hash: str, *, now: datetime | None = None
|
||||
) -> SessionRow | None:
|
||||
"""G-306: expired rows read as ABSENT — lazy evaluation, no
|
||||
timer, no writes on the read path."""
|
||||
moment = _as_utc(now) if now is not None else datetime.now(UTC)
|
||||
with Session(self._engine) as session:
|
||||
row = session.exec(
|
||||
select(SessionRow).where(SessionRow.token_hash == token_hash)
|
||||
).one_or_none()
|
||||
if row is None:
|
||||
return None
|
||||
row._normalize()
|
||||
assert row.expires_at is not None # mint always sets it
|
||||
if moment >= row.expires_at:
|
||||
return None # expired — reads as absent, stays on disk
|
||||
return row
|
||||
|
||||
def revoke_session(self, token_hash: str) -> bool:
|
||||
"""DELETE the row (logout). True = a row was removed; False =
|
||||
nothing to revoke (unknown/already-revoked token)."""
|
||||
with Session(self._engine) as session:
|
||||
row = session.exec(
|
||||
select(SessionRow).where(SessionRow.token_hash == token_hash)
|
||||
).one_or_none()
|
||||
if row is None:
|
||||
return False
|
||||
session.delete(row)
|
||||
session.commit()
|
||||
return True
|
||||
|
||||
# -- audit log ----------------------------------------------------------
|
||||
|
||||
def append_audit_event(
|
||||
self,
|
||||
kind: str,
|
||||
name: str,
|
||||
*,
|
||||
account_id: str | None = None,
|
||||
employer_id: str | None = None,
|
||||
operator: str | None = None,
|
||||
detail: dict[str, Any] | None = None,
|
||||
) -> AuditEventRow:
|
||||
row = AuditEventRow(
|
||||
account_id=account_id,
|
||||
employer_id=employer_id,
|
||||
kind=kind,
|
||||
name=name,
|
||||
detail=dict(detail or {}),
|
||||
operator=operator,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
row._validate()
|
||||
row._normalize()
|
||||
with Session(self._engine) as session:
|
||||
session.add(row)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
row._normalize()
|
||||
return row
|
||||
|
||||
# -- claims (D-089, REQ-10-002) ---------------------------------------
|
||||
|
||||
def insert_claim(
|
||||
self,
|
||||
*,
|
||||
account_id: str,
|
||||
employer_id: str,
|
||||
evidence: dict[str, Any] | None = None,
|
||||
) -> tuple[ClaimRow, bool]:
|
||||
"""Check-first pending-claim insert (the insert_application
|
||||
precedent): an existing PENDING claim on the entity rides
|
||||
created=False — never a second pending row. G-316: the partial
|
||||
UNIQUE index backstops the race — its IntegrityError SURFACES
|
||||
(the caller maps it to the same honest 409)."""
|
||||
row = ClaimRow(
|
||||
id=f"clm-{uuid.uuid4().hex[:16]}",
|
||||
account_id=account_id,
|
||||
employer_id=employer_id,
|
||||
status="pending",
|
||||
evidence=dict(evidence or {}),
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
row._validate()
|
||||
row._normalize()
|
||||
with Session(self._engine) as session:
|
||||
existing = session.exec(
|
||||
select(ClaimRow).where(
|
||||
ClaimRow.employer_id == employer_id,
|
||||
ClaimRow.status == "pending",
|
||||
)
|
||||
).one_or_none()
|
||||
if existing is not None:
|
||||
# Fast path: the pending claim exists — the existing
|
||||
# row rides, created=False (the route's 409).
|
||||
existing._normalize()
|
||||
return existing, False
|
||||
session.add(row)
|
||||
# G-316: a raced insert lands here — the partial UNIQUE
|
||||
# index refuses the second PENDING row and its
|
||||
# IntegrityError SURFACES, never swallowed here.
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
row._normalize()
|
||||
return row, True
|
||||
|
||||
def decide_claim(
|
||||
self,
|
||||
claim_id: str,
|
||||
*,
|
||||
decision: str,
|
||||
operator: str,
|
||||
now: datetime | None = None,
|
||||
) -> ClaimRow | None:
|
||||
"""Approve → employer_accounts.employer_id set + decided_* + a
|
||||
`claim` audit event; reject → decided_* + the audit event.
|
||||
The one-owner arms are STRUCTURAL: employer_accounts.employer_id
|
||||
UNIQUE (approve hands its IntegrityError to the caller) and a
|
||||
hand-raised guard when the claim is not pending (the caller's
|
||||
honest 409 — never a silent overwrite)."""
|
||||
if decision not in ("approved", "rejected"):
|
||||
raise ValueError(f"invalid decision {decision!r}")
|
||||
moment = _as_utc(now) if now is not None else datetime.now(UTC)
|
||||
with Session(self._engine) as session:
|
||||
row = session.get(ClaimRow, claim_id)
|
||||
if row is None:
|
||||
return None
|
||||
if row.status != "pending":
|
||||
# Hand-raised IntegrityError — the caller's honest 409
|
||||
# (deciding a decided claim, the IntegrityError shape).
|
||||
raise _constraint_error(
|
||||
f"guard: claim {claim_id!r} is already {row.status!r}",
|
||||
{"claim_id": claim_id},
|
||||
)
|
||||
row.status = decision
|
||||
row.decided_by = operator
|
||||
row.decided_at = moment
|
||||
row._validate()
|
||||
row._normalize()
|
||||
if decision == "approved":
|
||||
account = session.get(AccountRow, row.account_id)
|
||||
if account is None: # pragma: no cover - FK discipline
|
||||
raise _constraint_error(
|
||||
f"guard: claim {claim_id!r} references unknown "
|
||||
"account — refusing to approve",
|
||||
{"claim_id": claim_id},
|
||||
)
|
||||
if account.employer_id is not None:
|
||||
# One account : one entity — the account already
|
||||
# holds an entity. Hand-raised (the account's
|
||||
# employer_id UNIQUE never sees this UPDATE).
|
||||
raise _constraint_error(
|
||||
"guard: employer_accounts.employer_id already set",
|
||||
{"account_id": row.account_id},
|
||||
)
|
||||
account.employer_id = row.employer_id
|
||||
session.flush()
|
||||
# The audit row rides the SAME transaction: the decision
|
||||
# and its provenance land together (all-or-nothing).
|
||||
event = AuditEventRow(
|
||||
account_id=row.account_id,
|
||||
employer_id=row.employer_id,
|
||||
kind="claim",
|
||||
name=f"claim.{decision}",
|
||||
detail={
|
||||
"claim_id": row.id,
|
||||
"employer_id": row.employer_id,
|
||||
"operator": operator,
|
||||
},
|
||||
operator=operator,
|
||||
created_at=moment,
|
||||
)
|
||||
event._validate()
|
||||
event._normalize()
|
||||
session.add(event)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
row._normalize()
|
||||
return row
|
||||
|
||||
def list_claims(self, status: str | None = None) -> list[ClaimRow]:
|
||||
"""Newest-first; optional status filter (the admin queue)."""
|
||||
with Session(self._engine) as session:
|
||||
stmt = select(ClaimRow)
|
||||
if status is not None:
|
||||
stmt = stmt.where(ClaimRow.status == status)
|
||||
rows = session.exec(stmt.order_by(col(ClaimRow.created_at).desc(), ClaimRow.id)).all()
|
||||
for row in rows:
|
||||
row._normalize()
|
||||
return list(rows)
|
||||
|
||||
def list_accounts(self) -> list[AccountRow]:
|
||||
"""Created-oldest-first (the admin accounts list)."""
|
||||
with Session(self._engine) as session:
|
||||
rows = session.exec(
|
||||
select(AccountRow).order_by(col(AccountRow.created_at), AccountRow.id)
|
||||
).all()
|
||||
for row in rows:
|
||||
row._normalize()
|
||||
return list(rows)
|
||||
|
||||
def list_audit_events(
|
||||
self,
|
||||
*,
|
||||
account_id: str | None = None,
|
||||
employer_id: str | None = None,
|
||||
kind: str | None = None,
|
||||
name: str | None = None,
|
||||
limit: int = 100,
|
||||
) -> list[AuditEventRow]:
|
||||
"""Newest-first (the list_billing_events shape); optional
|
||||
account/employer/kind/name filters — D-089's operator trail."""
|
||||
with Session(self._engine) as session:
|
||||
stmt = select(AuditEventRow)
|
||||
if account_id is not None:
|
||||
stmt = stmt.where(AuditEventRow.account_id == account_id)
|
||||
if employer_id is not None:
|
||||
stmt = stmt.where(AuditEventRow.employer_id == employer_id)
|
||||
if kind is not None:
|
||||
stmt = stmt.where(AuditEventRow.kind == kind)
|
||||
if name is not None:
|
||||
stmt = stmt.where(AuditEventRow.name == name)
|
||||
rows = session.exec(
|
||||
stmt.order_by(
|
||||
col(AuditEventRow.created_at).desc(),
|
||||
col(AuditEventRow.id),
|
||||
).limit(limit)
|
||||
).all()
|
||||
for row in rows:
|
||||
row._normalize()
|
||||
return list(rows)
|
||||
|
||||
def get_claim(self, claim_id: str) -> ClaimRow | None:
|
||||
with Session(self._engine) as session:
|
||||
row = session.get(ClaimRow, claim_id)
|
||||
if row is None:
|
||||
return None
|
||||
row._normalize()
|
||||
return row
|
||||
|
||||
def close(self) -> None:
|
||||
self._engine.dispose()
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Process-trace grading — trace digest, rubric engine, GradeStore (REQ-3-004).
|
||||
|
||||
Boundary rule (D-027): grading/ is an engine module — it never imports
|
||||
api/; the single sanctioned agents/ dependency is the shared D-020
|
||||
structured defense (agents/structured.py), imported module-direct in
|
||||
engine.py (see its docstring for why). features.py imports telemetry/;
|
||||
store.py imports config only; engine.py composes llm/ + telemetry/ +
|
||||
prompts/ + agents.structured.
|
||||
|
||||
Wave status: features.py (TraceDigest, compute_digest) + store.py
|
||||
(GradeRecord, GradeStore, SQLiteGradeStore) landed in Wave 1 (3-1-01 +
|
||||
3-1-02); engine.py (GradingEngine, RubricScore) is Wave 2 (3-2-01).
|
||||
"""
|
||||
|
||||
from .engine import GradingEngine, RubricScore
|
||||
from .features import TraceDigest, compute_digest
|
||||
from .store import GradeRecord, GradeStore, SQLiteGradeStore
|
||||
|
||||
__all__ = [
|
||||
"GradeRecord",
|
||||
"GradeStore",
|
||||
"GradingEngine",
|
||||
"RubricScore",
|
||||
"SQLiteGradeStore",
|
||||
"TraceDigest",
|
||||
"compute_digest",
|
||||
]
|
||||
@@ -0,0 +1,313 @@
|
||||
"""GradingEngine — rubric scoring over real process traces (REQ-3-004).
|
||||
|
||||
The Wave-2 composition of the grading stack:
|
||||
|
||||
trace completeness gate (G-4, FIRST — nothing is sent to any LLM
|
||||
when the gate trips) → TraceStore.get_trace → compute_digest (D-028)
|
||||
→ prompts.grading.render_trace_digest → D-020 4-layer structured
|
||||
defense (agents/structured.py — REUSED, composed, never duplicated)
|
||||
→ RubricScore validation → GradeStore persistence → GradeRecord.
|
||||
|
||||
Gate verdicts are FIRST-CLASS RESULTS, not exceptions (G-4 is binding):
|
||||
UNGRADABLE_TRACE_INCOMPLETE — seq gaps in the store OR the trace is
|
||||
flagged by TraceIntegrityMap (INCOMPLETE_FLOODED). The gap list /
|
||||
flag reason is surfaced in `scores` for API rendering, and the
|
||||
record is PERSISTED like any grade so a learner sees why no
|
||||
credential can be issued for this trace — the gate outcome is
|
||||
durable and auditable, not a transient error string.
|
||||
UNGRADABLE_EMPTY_TRACE — no events stored for the pair.
|
||||
On either verdict `scores.criteria` is empty and the LLM is never called.
|
||||
|
||||
DI (D-027/D-032 house pattern): the engine receives trace_store,
|
||||
grade_store, integrity and provider through the constructor and knows
|
||||
NOTHING of FastAPI — api/ composes it (Task 3-3-01). `model` is injected
|
||||
alongside the provider so tests script the mock against the production
|
||||
wiring without touching Settings.
|
||||
|
||||
Boundary (D-027): grading/ imports llm/ (provider protocol + Message
|
||||
type), telemetry/ (store + integrity map), prompts/ (rubric text) and
|
||||
ONLY agents.structured — the sanctioned shared D-020 defense. We import
|
||||
the MODULE directly (`from ..agents.structured import structured_completion`)
|
||||
rather than the `agents` package, mirroring how agents/base.py consumes
|
||||
it (same direct-module import): that keeps the dependency surface to
|
||||
exactly the two names the engine needs (structured_completion,
|
||||
StructuredOutputError) and avoids executing agents/__init__ re-exports
|
||||
(BaseAgent, registry, session store) that grading has no business
|
||||
loading — a side-effect-hygiene choice that keeps this import line
|
||||
grep-auditable as "the one agents dependency". grading/ never imports api/.
|
||||
|
||||
RubricScore placement (documented decision): the validated output model
|
||||
lives HERE, not in prompts/. The pydantic model is the engine's return
|
||||
CONTRACT (the shape GradeStore.scores must hold), while prompts/grading.py
|
||||
is pure prompt text + its mirror schema HINT string — the same split as
|
||||
agents/assessor.py (model + hint) but with the model owned by the engine
|
||||
module that validates it. Prompt files hold text, per the prompts/ house
|
||||
style; engine files hold typed contracts.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import TYPE_CHECKING, Final
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
from ..agents.structured import StructuredOutputError, structured_completion
|
||||
from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
from ..prompts.grading import (
|
||||
RUBRIC_CRITERIA,
|
||||
RUBRIC_SCORE_SCHEMA_HINT,
|
||||
SYSTEM_PROMPT,
|
||||
render_trace_digest,
|
||||
)
|
||||
from ..telemetry.ingest import TraceIntegrityMap
|
||||
from ..telemetry.store import TraceStore
|
||||
from .features import compute_digest
|
||||
from .store import GradeRecord, GradeStore
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - protocol-only import for the optional
|
||||
from ..variants.store import VariantStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: First-class gate verdicts (G-4). GradeRecord.verdict values; non-empty
|
||||
#: by store contract. Rubric verdicts (mastered/developing/not_yet) ride in
|
||||
#: `scores.verdict` — `record.verdict` stays the machine-readable outcome.
|
||||
VERDICT_UNGRADABLE_INCOMPLETE: Final = "UNGRADABLE_TRACE_INCOMPLETE"
|
||||
VERDICT_UNGRADABLE_EMPTY: Final = "UNGRADABLE_EMPTY_TRACE"
|
||||
#: record.verdict for a successfully LLM-graded trace (the rubric verdict
|
||||
#: travels inside scores); keeps verdict non-empty for every persisted row.
|
||||
VERDICT_GRADED: Final = "GRADED"
|
||||
|
||||
#: Gate-detail keys surfaced in GradeRecord.scores (tests assert on these).
|
||||
_INTEGRITY_FLAG_KEY: Final = "integrity_flag"
|
||||
|
||||
|
||||
def _anchors_context(variant) -> str:
|
||||
"""Render the variant template's difficulty anchors for the grader prompt.
|
||||
|
||||
Contains only the template id + anchor numbers — no learner-identifying
|
||||
material (D-028 anonymity preserved; the digest-leak tests keep holding).
|
||||
Lazy template import: grading must not import variants/ at module load
|
||||
(variants/prompts import-cycle safety mirrors llm/ rules).
|
||||
"""
|
||||
from ..variants.templates import get_template
|
||||
|
||||
template = get_template(variant.template_id)
|
||||
if template is None:
|
||||
return f"template={variant.template_id} (anchors unavailable)"
|
||||
a = template.rubric_anchors
|
||||
return (
|
||||
f"template={template.id}; "
|
||||
f"expected_edit_count_band={list(a.expected_edit_count_band)}; "
|
||||
f"expected_min_test_runs={a.expected_min_test_runs}; "
|
||||
f"expected_error_fix_cycles_band={list(a.expected_error_fix_cycles_band)}"
|
||||
)
|
||||
_MISSING_SEQS_KEY: Final = "missing_seqs"
|
||||
|
||||
|
||||
class RubricScore(BaseModel):
|
||||
"""Validated LLM output: per-criterion 0-4 scores + strengths + gaps + verdict.
|
||||
|
||||
The D-020 schema for the grader: `structured_completion` parses the
|
||||
model reply into THIS shape (layer 3), retrying once with the
|
||||
validation error fed back (layer 4). Exact criteria set + 0-4 ranges
|
||||
are enforced here, so the scores dict persisted to GradeStore is
|
||||
always rubric-shaped no matter what the model produced.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
criteria: dict[str, int]
|
||||
strengths: list[str] = Field(min_length=1, max_length=2)
|
||||
gaps: list[str] = Field(min_length=1, max_length=2)
|
||||
verdict: str # "mastered" | "developing" | "not_yet"
|
||||
|
||||
@field_validator("criteria")
|
||||
@classmethod
|
||||
def _criteria_rubric_shaped(cls, value: dict[str, int]) -> dict[str, int]:
|
||||
"""Exact criteria keys (no extras, no omissions) and 0-4 scores."""
|
||||
expected = set(RUBRIC_CRITERIA)
|
||||
got = set(value)
|
||||
if got != expected:
|
||||
raise ValueError(
|
||||
f"criteria keys must be exactly {sorted(expected)}, got {sorted(got)}"
|
||||
)
|
||||
for key, score in value.items():
|
||||
if not 0 <= score <= 4:
|
||||
raise ValueError(f"criterion {key!r} must be within 0-4, got {score}")
|
||||
return value
|
||||
|
||||
@field_validator("verdict")
|
||||
@classmethod
|
||||
def _verdict_known(cls, value: str) -> str:
|
||||
allowed = {"mastered", "developing", "not_yet"}
|
||||
if value not in allowed:
|
||||
raise ValueError(f"verdict must be one of {sorted(allowed)}, got {value!r}")
|
||||
return value
|
||||
|
||||
|
||||
class GradingEngine:
|
||||
"""Scores a (learner_id, task_id) trace into a persisted GradeRecord."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
trace_store: TraceStore,
|
||||
grade_store: GradeStore,
|
||||
integrity: TraceIntegrityMap,
|
||||
provider: LLMProvider,
|
||||
*,
|
||||
model: str = "gemma4:31b",
|
||||
variant_store: "VariantStore | None" = None,
|
||||
) -> None:
|
||||
self._trace_store = trace_store
|
||||
self._grade_store = grade_store
|
||||
self._integrity = integrity
|
||||
self._provider = provider
|
||||
self._model = model
|
||||
# Phase 4 (MH#4): optional variant lookup — when the graded task
|
||||
# derives from a generated variant, its template's difficulty anchors
|
||||
# ship to the grader prompt (same bar for every variant of the
|
||||
# template, a-5) and the variant seed is stamped on the record.
|
||||
# Optional so engine tests stay decoupled; main.py lifespan wires it.
|
||||
self._variant_store = variant_store
|
||||
|
||||
async def grade(self, learner_id: str, task_id: str) -> GradeRecord:
|
||||
"""Grade one trace; persist latest-state (GradeStore upserts); return it.
|
||||
|
||||
Gate FIRST (G-4): the LLM is only ever reached from the fully
|
||||
guarded path — no gate state can be masked by an LLM error.
|
||||
"""
|
||||
record = await self._grade(learner_id, task_id)
|
||||
self._grade_store.save(record)
|
||||
return record
|
||||
|
||||
# ------------------------------------------------------------------ core
|
||||
|
||||
async def _grade(self, learner_id: str, task_id: str) -> GradeRecord:
|
||||
# -- G-4 gate FIRST: integrity flag OR seq gaps. Ordering matters:
|
||||
# gaps() returns [] for an EMPTY trace, so the empty check below is
|
||||
# reachable only when no rows exist at all; a gapped or flooded
|
||||
# trace can never fall through to the LLM path.
|
||||
if self._integrity.is_incomplete(learner_id, task_id):
|
||||
reason = self._integrity.reason(learner_id, task_id) or "unknown"
|
||||
gaps = self._trace_store.gaps(learner_id, task_id)
|
||||
logger.info(
|
||||
"grade gate (G-4): %s/%s integrity-flagged (%s) — ungradable",
|
||||
learner_id,
|
||||
task_id,
|
||||
reason,
|
||||
)
|
||||
return self._ungradable(
|
||||
learner_id,
|
||||
task_id,
|
||||
detail={_INTEGRITY_FLAG_KEY: reason, _MISSING_SEQS_KEY: gaps},
|
||||
verdict=VERDICT_UNGRADABLE_INCOMPLETE,
|
||||
)
|
||||
gaps = self._trace_store.gaps(learner_id, task_id)
|
||||
if gaps:
|
||||
logger.info(
|
||||
"grade gate (G-4): %s/%s seq gaps %s — ungradable",
|
||||
learner_id,
|
||||
task_id,
|
||||
gaps,
|
||||
)
|
||||
return self._ungradable(
|
||||
learner_id,
|
||||
task_id,
|
||||
detail={_INTEGRITY_FLAG_KEY: None, _MISSING_SEQS_KEY: gaps},
|
||||
verdict=VERDICT_UNGRADABLE_INCOMPLETE,
|
||||
)
|
||||
|
||||
trace = self._trace_store.get_trace(learner_id, task_id)
|
||||
if not trace:
|
||||
logger.info(
|
||||
"grade gate: %s/%s empty trace — ungradable", learner_id, task_id
|
||||
)
|
||||
return self._ungradable(
|
||||
learner_id,
|
||||
task_id,
|
||||
detail={_INTEGRITY_FLAG_KEY: None, _MISSING_SEQS_KEY: []},
|
||||
verdict=VERDICT_UNGRADABLE_EMPTY,
|
||||
)
|
||||
|
||||
# -- Guarded path: digest (D-028) → prompt → D-020 4-layer defense.
|
||||
digest = compute_digest(trace)
|
||||
variant = self._lookup_variant(task_id)
|
||||
anchors_context = (
|
||||
_anchors_context(variant) if variant is not None else None
|
||||
)
|
||||
messages = [
|
||||
Message(role="system", content=SYSTEM_PROMPT),
|
||||
Message(
|
||||
role="user",
|
||||
content=render_trace_digest(digest, anchors_context=anchors_context),
|
||||
),
|
||||
]
|
||||
try:
|
||||
rubric = await structured_completion(
|
||||
self._provider,
|
||||
messages,
|
||||
model=self._model,
|
||||
schema=RubricScore,
|
||||
schema_hint=RUBRIC_SCORE_SCHEMA_HINT,
|
||||
)
|
||||
except StructuredOutputError as exc:
|
||||
# The trace was gradable but the model failed to produce valid
|
||||
# JSON within the D-020 budget (two attempts). Raise — the API
|
||||
# layer maps this to a 502 (assessor precedent). Persisting a
|
||||
# fabricated or partial grade here would violate the no-silent-
|
||||
# fallback rule: no credential-worthy record without a validated
|
||||
# RubricScore.
|
||||
raise StructuredOutputError(f"grading LLM failed for {task_id}: {exc}") from exc
|
||||
|
||||
logger.debug(
|
||||
"graded %s/%s: %s (model=%s)",
|
||||
learner_id,
|
||||
task_id,
|
||||
rubric.verdict,
|
||||
self._model,
|
||||
)
|
||||
return GradeRecord(
|
||||
learner_id=learner_id,
|
||||
task_id=task_id,
|
||||
variant_seed=variant.seed if variant is not None else None, # D-029
|
||||
digest=digest.model_dump(),
|
||||
scores=rubric.model_dump(),
|
||||
verdict=VERDICT_GRADED,
|
||||
model=self._model,
|
||||
created_at=datetime.now(tz=UTC),
|
||||
)
|
||||
|
||||
def _lookup_variant(self, task_id: str):
|
||||
"""MH#4: resolve the graded task's variant (None when not variant-derived)."""
|
||||
if self._variant_store is None:
|
||||
return None
|
||||
return self._variant_store.get_by_task(task_id)
|
||||
|
||||
# ------------------------------------------------------------ gate record
|
||||
|
||||
@staticmethod
|
||||
def _ungradable(
|
||||
learner_id: str,
|
||||
task_id: str,
|
||||
*,
|
||||
detail: dict,
|
||||
verdict: str,
|
||||
) -> GradeRecord:
|
||||
"""Build a gate record: no digest (nothing was graded), gate detail
|
||||
surfaced in `scores` (the store allows an empty scores dict, but
|
||||
G-4 requires the gap list / flag reason surfaced — the detail IS the
|
||||
verdict's payload), model="none" (no LLM was involved; provenance
|
||||
stays honest).
|
||||
"""
|
||||
return GradeRecord(
|
||||
learner_id=learner_id,
|
||||
task_id=task_id,
|
||||
variant_seed=None,
|
||||
digest={},
|
||||
scores=detail,
|
||||
verdict=verdict,
|
||||
model="none",
|
||||
created_at=datetime.now(tz=UTC),
|
||||
)
|
||||
@@ -0,0 +1,246 @@
|
||||
"""Deterministic process-trace digest (D-028, REQ-3-004).
|
||||
|
||||
Pure compute — no LLM, no I/O. `compute_digest` reduces an ordered
|
||||
TelemetryEvent trace to a compact, bounded `TraceDigest` that is safe to
|
||||
embed in a grading prompt:
|
||||
|
||||
- FIXED fields + small histograms only; NO raw commands, NO file contents,
|
||||
NO payloads — the raw trace NEVER reaches the LLM (D-028), which also
|
||||
bounds the prompt-injection surface.
|
||||
- Tolerant to both live trace mixes: daemon-topology traces carry
|
||||
`activity` + `file_diff` kinds (workspace watcher), while REPL-driven
|
||||
traces carry `command`/`stdin`/`stdout`/`run_result`/`test_result`
|
||||
(P2 verification P1). Features derive from whatever kinds are present and
|
||||
never crash on absent kinds.
|
||||
|
||||
Feature semantics (conservative, deterministic):
|
||||
- test pass/fail counts + final status derive from `test_result` payloads
|
||||
when present, falling back to `run_result` exit codes (0 = pass).
|
||||
- an error/fix CYCLE = a failing run/test followed by >= 1 edit and then a
|
||||
later run/test (pass or fail) — the next observed result closes the cycle.
|
||||
- idle gaps = wall-clock gaps between consecutive events exceeding
|
||||
`idle_threshold_s` (default 120s): count + total seconds.
|
||||
- command category histogram classifies `command`-kind payloads: build /
|
||||
test / file / nav / debug / other.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import Counter
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..telemetry.models import TelemetryEvent
|
||||
|
||||
_IDLE_DEFAULT_S: float = 120.0
|
||||
|
||||
_TEST_HINTS = ("test", "pytest", "vitest", "jest", "mocha", "unittest", "go test", "npm test")
|
||||
_BUILD_HINTS = ("make", "npm run build", "pip install", "pnpm", "cargo build", "gcc", "tsc")
|
||||
_DEBUG_HINTS = ("gdb", "pdb", "print(", "debug", "strace", "ltrace", "curl", "ping")
|
||||
_NAV_HINTS = ("ls", "cd", "pwd", "cat ", "grep ", "find", "rg ", "tree", "head", "tail", "less")
|
||||
_FILE_HINTS = ("mv ", "cp ", "rm ", "mkdir", "touch", "chmod", "nano", "vim", "sed -i", "tee ")
|
||||
|
||||
|
||||
class TraceDigest(BaseModel):
|
||||
"""Compact, bounded, LLM-safe summary of a process trace (D-028).
|
||||
|
||||
Fixed fields + small histograms. Serializes well under 4 KB; contains no
|
||||
raw commands, file contents, or event payloads.
|
||||
"""
|
||||
|
||||
model_config = {"frozen": True}
|
||||
|
||||
event_count: int = Field(ge=0)
|
||||
session_duration_s: float = Field(ge=0.0)
|
||||
edit_count: int = Field(ge=0)
|
||||
command_count: int = Field(ge=0)
|
||||
run_count: int = Field(ge=0)
|
||||
test_pass_count: int = Field(ge=0)
|
||||
test_fail_count: int = Field(ge=0)
|
||||
final_test_status: str = Field(pattern="^(pass|fail|none)$")
|
||||
first_test_pass_offset_s: float | None = None
|
||||
error_fix_cycles: int = Field(ge=0)
|
||||
mean_fix_latency_s: float | None = None
|
||||
idle_gap_count: int = Field(ge=0)
|
||||
idle_gap_total_s: float = Field(ge=0.0)
|
||||
command_categories: dict[str, int] = Field(default_factory=dict)
|
||||
kind_histogram: dict[str, int] = Field(default_factory=dict)
|
||||
|
||||
|
||||
def _event_pass_status(event: TelemetryEvent) -> bool | None:
|
||||
"""True (pass) / False (fail) / None (not a result event) for one event."""
|
||||
payload = event.payload or {}
|
||||
if event.kind == "test_result":
|
||||
if "passed" in payload:
|
||||
return bool(payload["passed"])
|
||||
if "exit_code" in payload:
|
||||
return int(payload["exit_code"]) == 0
|
||||
if "status" in payload:
|
||||
return str(payload["status"]).lower() in ("pass", "passed", "ok", "success")
|
||||
return None
|
||||
if event.kind == "run_result":
|
||||
if "exit_code" in payload:
|
||||
return int(payload["exit_code"]) == 0
|
||||
if "ok" in payload:
|
||||
return bool(payload["ok"])
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _classify_command(text: str) -> str:
|
||||
lowered = text.lower()
|
||||
if any(h in lowered for h in _TEST_HINTS):
|
||||
return "test"
|
||||
if any(h in lowered for h in _BUILD_HINTS):
|
||||
return "build"
|
||||
if any(h in lowered for h in _DEBUG_HINTS):
|
||||
return "debug"
|
||||
if any(h in lowered for h in _NAV_HINTS):
|
||||
return "nav"
|
||||
if any(h in lowered for h in _FILE_HINTS):
|
||||
return "file"
|
||||
return "other"
|
||||
|
||||
|
||||
def _command_text(event: TelemetryEvent) -> str:
|
||||
payload = event.payload or {}
|
||||
return str(payload.get("cmd") or payload.get("command") or payload.get("line") or "")
|
||||
|
||||
|
||||
def compute_digest(
|
||||
trace: list[TelemetryEvent], *, idle_threshold_s: float = _IDLE_DEFAULT_S
|
||||
) -> TraceDigest:
|
||||
"""Reduce an ordered trace to a bounded digest. Never raises on odd input."""
|
||||
events = sorted(trace, key=lambda e: (e.seq, e.ts))
|
||||
if not events:
|
||||
return TraceDigest(
|
||||
event_count=0,
|
||||
session_duration_s=0.0,
|
||||
edit_count=0,
|
||||
command_count=0,
|
||||
run_count=0,
|
||||
test_pass_count=0,
|
||||
test_fail_count=0,
|
||||
final_test_status="none",
|
||||
first_test_pass_offset_s=None,
|
||||
error_fix_cycles=0,
|
||||
mean_fix_latency_s=None,
|
||||
idle_gap_count=0,
|
||||
idle_gap_total_s=0.0,
|
||||
command_categories={},
|
||||
kind_histogram={},
|
||||
)
|
||||
|
||||
kind_histogram = Counter(e.kind for e in events)
|
||||
start_ts = events[0].ts
|
||||
end_ts = events[-1].ts
|
||||
duration = max(0.0, (end_ts - start_ts).total_seconds())
|
||||
|
||||
edit_count = kind_histogram.get("file_diff", 0)
|
||||
command_count = kind_histogram.get("command", 0)
|
||||
run_count = kind_histogram.get("run_result", 0)
|
||||
|
||||
# Tests: prefer test_result events; fall back to run_result exit codes.
|
||||
test_statuses: list[tuple[TelemetryEvent, bool]] = []
|
||||
for e in events:
|
||||
if e.kind == "test_result":
|
||||
ok = _event_pass_status(e)
|
||||
if ok is not None:
|
||||
test_statuses.append((e, ok))
|
||||
if not test_statuses:
|
||||
for e in events:
|
||||
if e.kind == "run_result":
|
||||
ok = _event_pass_status(e)
|
||||
if ok is not None:
|
||||
test_statuses.append((e, ok))
|
||||
|
||||
test_pass_count = sum(1 for _, ok in test_statuses if ok)
|
||||
test_fail_count = len(test_statuses) - test_pass_count
|
||||
if not test_statuses:
|
||||
final_test_status = "none"
|
||||
else:
|
||||
final_test_status = "pass" if test_statuses[-1][1] else "fail"
|
||||
first_pass = next((e for e, ok in test_statuses if ok), None)
|
||||
first_pass_offset = (
|
||||
max(0.0, (first_pass.ts - start_ts).total_seconds()) if first_pass is not None else None
|
||||
)
|
||||
|
||||
# Error/fix cycles: a failing result starts a pending cycle; the NEXT
|
||||
# observed result closes it (regardless of outcome) — a fix attempt that
|
||||
# fails again is itself another iteration of debugging, so it closes the
|
||||
# previous cycle and opens a new one. Edits since the fail mark the
|
||||
# close as a genuine fix attempt; latency = first edit -> closing result.
|
||||
cycles = 0
|
||||
fix_latencies: list[float] = []
|
||||
pending_fail_ts: float | None = None # seconds since start
|
||||
edits_since_fail = 0
|
||||
first_edit_ts: float | None = None
|
||||
for e in events:
|
||||
t = max(0.0, (e.ts - start_ts).total_seconds())
|
||||
if e.kind == "file_diff":
|
||||
if pending_fail_ts is not None:
|
||||
if edits_since_fail == 0:
|
||||
first_edit_ts = t
|
||||
edits_since_fail += 1
|
||||
continue
|
||||
ok = _event_pass_status(e)
|
||||
if ok is None:
|
||||
continue
|
||||
if ok is False:
|
||||
if pending_fail_ts is not None and edits_since_fail > 0 and first_edit_ts is not None:
|
||||
# failed fix attempt: closes the previous cycle, opens a new one
|
||||
cycles += 1
|
||||
fix_latencies.append(t - first_edit_ts)
|
||||
pending_fail_ts = t
|
||||
edits_since_fail = 0
|
||||
first_edit_ts = None
|
||||
continue
|
||||
if ok is True and pending_fail_ts is not None:
|
||||
if edits_since_fail > 0 and first_edit_ts is not None:
|
||||
cycles += 1
|
||||
fix_latencies.append(t - first_edit_ts)
|
||||
pending_fail_ts = None
|
||||
edits_since_fail = 0
|
||||
first_edit_ts = None
|
||||
|
||||
mean_fix_latency = (
|
||||
sum(fix_latencies) / len(fix_latencies) if fix_latencies else None
|
||||
)
|
||||
|
||||
# Idle gaps between consecutive events.
|
||||
idle_gap_count = 0
|
||||
idle_gap_total = 0.0
|
||||
prev_ts = None
|
||||
for e in events:
|
||||
if prev_ts is not None:
|
||||
gap = (e.ts - prev_ts).total_seconds()
|
||||
if gap > idle_threshold_s:
|
||||
idle_gap_count += 1
|
||||
idle_gap_total += gap
|
||||
prev_ts = e.ts
|
||||
|
||||
# Command category histogram (command-kind events only).
|
||||
categories: Counter[str] = Counter()
|
||||
for e in events:
|
||||
if e.kind == "command":
|
||||
categories[_classify_command(_command_text(e))] += 1
|
||||
|
||||
return TraceDigest(
|
||||
event_count=len(events),
|
||||
session_duration_s=round(duration, 3),
|
||||
edit_count=edit_count,
|
||||
command_count=command_count,
|
||||
run_count=run_count,
|
||||
test_pass_count=test_pass_count,
|
||||
test_fail_count=test_fail_count,
|
||||
final_test_status=final_test_status,
|
||||
first_test_pass_offset_s=(
|
||||
round(first_pass_offset, 3) if first_pass_offset is not None else None
|
||||
),
|
||||
error_fix_cycles=cycles,
|
||||
mean_fix_latency_s=(round(mean_fix_latency, 3) if mean_fix_latency is not None else None),
|
||||
idle_gap_count=idle_gap_count,
|
||||
idle_gap_total_s=round(idle_gap_total, 3),
|
||||
command_categories=dict(sorted(categories.items())),
|
||||
kind_histogram=dict(sorted(kind_histogram.items())),
|
||||
)
|
||||
@@ -0,0 +1,239 @@
|
||||
"""GradeStore — grade persistence protocol + SQLite implementation (REQ-3-004, D-027).
|
||||
|
||||
Postgres-migration-ready (D-027): the protocol is the only surface the
|
||||
grading engine and API layers touch; swapping SQLiteGradeStore for a
|
||||
Postgres-backed implementation must not change call sites. The
|
||||
`grade_record` table uses only portable column types (str / JSON /
|
||||
datetime), so the same SQLModel schema stands up unchanged on Postgres.
|
||||
|
||||
Upsert, NOT append: (learner_id, task_id) is the grade identity — one row
|
||||
per learner per task holding the LATEST grade. `save` overwrites the whole
|
||||
row when the pair already exists, so a regrade replaces scores, verdict,
|
||||
created_at, digest, model and variant_seed wholesale. That is deliberately
|
||||
the opposite of TraceStore.append's dedup-keep-first contract: a trace is an
|
||||
append-only event log, a grade is latest-state, so the engine can re-grade
|
||||
a task idempotently as its rubric or input evolves.
|
||||
|
||||
Concurrency (a-3): the engine enables WAL + synchronous=NORMAL and a busy
|
||||
timeout at connection time, so a regrade writer and API readers do not hit
|
||||
`database is locked` on the single-box pilot.
|
||||
|
||||
`created_at` contract: callers stamp UTC (datetime.now(UTC)); SQLite stores
|
||||
it naive and the read paths re-label it tz-aware UTC (same boundary
|
||||
normalization as TelemetryEvent.ts, so the contract holds on any backend).
|
||||
|
||||
Boundary (D-027): `grading/` never imports `agents/` / `api/`; this module
|
||||
imports config only.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import sqlite3
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Protocol
|
||||
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy import JSON, Index
|
||||
from sqlalchemy.orm import validates
|
||||
from sqlmodel import Field, Session, SQLModel, col, create_engine, select
|
||||
|
||||
from ..config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class GradeRecord(SQLModel, table=True):
|
||||
"""A persisted grade; (learner_id, task_id) is the PK — latest wins.
|
||||
|
||||
Written by the grading engine (one save per grade attempt), read by the
|
||||
API layer through the GradeStore protocol. Constraint enforcement
|
||||
mirrors TelemetryEvent: sqlmodel 0.0.42's metaclass drops pydantic
|
||||
constraints on table models, so SQLAlchemy `@validates` hooks enforce
|
||||
instead and the column types stay Postgres-ready (D-027).
|
||||
|
||||
Field contract:
|
||||
learner_id — non-empty learner identifier (same id space as traces).
|
||||
task_id — non-empty task identifier; grade identity is the
|
||||
(learner_id, task_id) pair — the same pair as trace
|
||||
identity, so a grade is keyed by the exact trace it
|
||||
was computed from.
|
||||
variant_seed — task-variant seed (D-029); None when the graded task
|
||||
is not variant-derived. Since Phase 4 the engine
|
||||
stamps the graded variant's seed here (MH#4) and the
|
||||
template's difficulty anchors ship to the grader
|
||||
prompt — this column is the audit join for that.
|
||||
digest — compact deterministic trace digest (D-028) that fed
|
||||
the rubric prompt; persisted for auditability so the
|
||||
LLM's input stays reproducible.
|
||||
scores — validated rubric scores (per-criterion 0-4,
|
||||
strengths, gaps); JSON dict. An empty dict is legal
|
||||
(e.g. an UNGRADABLE_TRACE_INCOMPLETE record carries a
|
||||
verdict but no scores).
|
||||
verdict — first-class verdict string (rubric verdict or
|
||||
UNGRADABLE_TRACE_INCOMPLETE); non-empty.
|
||||
model — provider model that produced the scores (provenance).
|
||||
created_at — UTC grade timestamp; a regrade replaces it (latest
|
||||
save wins).
|
||||
"""
|
||||
|
||||
__tablename__ = "grade_record"
|
||||
# The composite PK covers (learner_id, task_id) point lookups; this
|
||||
# secondary index covers list_for_learner ordered by created_at without
|
||||
# a sort step (Postgres migration target D-027).
|
||||
__table_args__ = (
|
||||
Index("ix_grade_record_learner_created", "learner_id", "created_at"),
|
||||
)
|
||||
|
||||
learner_id: str = Field(primary_key=True)
|
||||
task_id: str = Field(primary_key=True)
|
||||
# None only for non-variant tasks (MH#4 stamps variant seeds since P4).
|
||||
variant_seed: str | None = Field(default=None)
|
||||
# JSON columns: stored as TEXT on SQLite, native JSONB on Postgres (D-027).
|
||||
digest: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
scores: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
verdict: str
|
||||
model: str
|
||||
created_at: datetime
|
||||
|
||||
@validates("learner_id", "task_id")
|
||||
def _ids_non_empty(self, key: str, value: str) -> str:
|
||||
if not value:
|
||||
raise ValueError(f"{key} must be a non-empty identifier")
|
||||
return value
|
||||
|
||||
@validates("verdict")
|
||||
def _verdict_non_empty(self, key: str, value: str) -> str:
|
||||
if not value:
|
||||
raise ValueError(f"{key} must be a non-empty verdict string")
|
||||
return value
|
||||
|
||||
|
||||
class GradeStore(Protocol):
|
||||
"""Persistence contract for latest-state grades per (learner_id, task_id).
|
||||
|
||||
Implemented by SQLiteGradeStore (v0.3, D-027); a Postgres implementation
|
||||
must satisfy the same surface.
|
||||
"""
|
||||
|
||||
def save(self, grade: GradeRecord) -> None:
|
||||
"""Persist a grade. UPSERT on (learner_id, task_id): a regrade with
|
||||
the same pair REPLACES the stored row wholesale — the latest grade
|
||||
wins. NOT append-only; contrast TraceStore.append, which is
|
||||
dedup-keep-first for at-least-once ingest.
|
||||
"""
|
||||
...
|
||||
|
||||
def get(self, learner_id: str, task_id: str) -> GradeRecord | None:
|
||||
"""Latest stored grade for the pair; None when none exists.
|
||||
|
||||
Detached from any DB session — safe to pass across layers.
|
||||
"""
|
||||
...
|
||||
|
||||
def list_for_learner(self, learner_id: str) -> list[GradeRecord]:
|
||||
"""All stored grades for the learner, ordered by created_at
|
||||
ascending (chronological). Empty list when the learner has none.
|
||||
"""
|
||||
...
|
||||
|
||||
def close(self) -> None:
|
||||
"""Release DB connections. Store must not be used after close."""
|
||||
...
|
||||
|
||||
|
||||
def _sqlite_connect(dbapi_connection: sqlite3.Connection, _: object) -> None:
|
||||
"""Per-connection pragma setup (a-3). Mirrors telemetry/store.py.
|
||||
|
||||
journal_mode=WAL — readers never block the single writer.
|
||||
synchronous=NORMAL — safe in WAL mode, avoids full fsync-per-commit.
|
||||
busy_timeout=5000 — retry briefly under contention instead of
|
||||
`OperationalError: database is locked`.
|
||||
"""
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA synchronous=NORMAL")
|
||||
cursor.execute("PRAGMA busy_timeout=5000")
|
||||
cursor.close()
|
||||
|
||||
|
||||
def _as_utc(ts: datetime) -> datetime:
|
||||
"""Timestamps travel as naive datetime on SQLite, tz-aware elsewhere.
|
||||
|
||||
SQLite (via SQLModel) drops tzinfo; Postgres TIMESTAMP WITH TIME ZONE
|
||||
keeps it. Normalizing on the read path makes the store's contract
|
||||
tz-aware UTC regardless of the backend (D-027).
|
||||
"""
|
||||
if ts.tzinfo is None:
|
||||
return ts.replace(tzinfo=UTC) # naive UTC read-side: label as UTC
|
||||
return ts.astimezone(UTC)
|
||||
|
||||
|
||||
class SQLiteGradeStore:
|
||||
"""SQLite-backed GradeStore (SQLModel). Second protocol-wrapped store
|
||||
of the D-027 family (first: SQLiteTraceStore).
|
||||
"""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
self._db_path: Path = db_path if db_path is not None else Settings().db_path
|
||||
self._engine = create_engine(f"sqlite:///{self._db_path}")
|
||||
sa.event.listen(self._engine, "connect", _sqlite_connect)
|
||||
SQLModel.metadata.create_all(self._engine)
|
||||
|
||||
@contextmanager
|
||||
def _session(self) -> Iterator[Session]:
|
||||
# expire_on_commit=False: identical session behavior to
|
||||
# SQLiteTraceStore. save() discards the merged instance and the read
|
||||
# paths never commit, but a uniform flag across the D-027 stores
|
||||
# keeps their detachment guarantees from diverging.
|
||||
with Session(self._engine, expire_on_commit=False) as session:
|
||||
yield session
|
||||
|
||||
def save(self, grade: GradeRecord) -> None:
|
||||
# `merge` = SELECT-by-PK then UPDATE or INSERT — exactly the upsert
|
||||
# contract. The trace store deliberately avoids merge (its append is
|
||||
# dedup-keep-first); here latest-wins IS the contract, so merge is
|
||||
# the right tool. The caller's object is never attached to the
|
||||
# session and stays usable (unexpired) after save.
|
||||
with self._session() as session:
|
||||
session.merge(grade)
|
||||
session.commit()
|
||||
logger.debug(
|
||||
"grade saved (regrade overwrites): %s/%s verdict=%s model=%s",
|
||||
grade.learner_id,
|
||||
grade.task_id,
|
||||
grade.verdict,
|
||||
grade.model,
|
||||
)
|
||||
|
||||
def get(self, learner_id: str, task_id: str) -> GradeRecord | None:
|
||||
with self._session() as session:
|
||||
record = session.get(GradeRecord, (learner_id, task_id))
|
||||
if record is None:
|
||||
return None
|
||||
record.created_at = _as_utc(record.created_at)
|
||||
# Detach from the session: callers must not depend on
|
||||
# open-session ORM magic (lazy loads fail once it closes).
|
||||
session.expunge(record)
|
||||
return record
|
||||
|
||||
def list_for_learner(self, learner_id: str) -> list[GradeRecord]:
|
||||
with self._session() as session:
|
||||
stmt = (
|
||||
select(GradeRecord)
|
||||
.where(GradeRecord.learner_id == learner_id)
|
||||
# Chronological; task_id is a deterministic tie-break for
|
||||
# grades stamped within the same instant.
|
||||
# P5: col() — SQLModel class attrs are Mapped[T], the
|
||||
# column reference rides col() in order_by.
|
||||
.order_by(col(GradeRecord.created_at), GradeRecord.task_id)
|
||||
)
|
||||
results = session.exec(stmt).all()
|
||||
for row in results:
|
||||
row.created_at = _as_utc(row.created_at)
|
||||
session.expunge(row)
|
||||
return list(results)
|
||||
|
||||
def close(self) -> None:
|
||||
self._engine.dispose()
|
||||
@@ -0,0 +1,24 @@
|
||||
"""Identity engine package — provider protocol + store (REQ-5-003, D-042)."""
|
||||
|
||||
from .base import (
|
||||
AgeBand,
|
||||
IdentityProvider,
|
||||
IdentityStatus,
|
||||
IdentitySubmission,
|
||||
IdentityVerdict,
|
||||
)
|
||||
from .mock import MockIdentityProvider, derive_age_band
|
||||
from .store import IdentityRecord, IdentityStore, SQLiteIdentityStore
|
||||
|
||||
__all__ = [
|
||||
"AgeBand",
|
||||
"IdentityStatus",
|
||||
"IdentitySubmission",
|
||||
"IdentityVerdict",
|
||||
"IdentityProvider",
|
||||
"IdentityRecord",
|
||||
"IdentityStore",
|
||||
"SQLiteIdentityStore",
|
||||
"MockIdentityProvider",
|
||||
"derive_age_band",
|
||||
]
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Identity verification protocol (REQ-5-003, D-042, A-303).
|
||||
|
||||
Provider-agnostic like LLMProvider/VoiceProvider (D-014/D-030): a narrow
|
||||
protocol the identity API composes via DI, a deterministic mock, and a
|
||||
future real KYC vendor (Stripe Identity / Persona / Onfido class) that
|
||||
drops in without API changes. PII rules (A-305): the provider sees
|
||||
document REFERENCES, never raw documents; verdicts carry a mock marker
|
||||
(A-304) so downstream surfaces never display mock-verified as
|
||||
production-verified.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal, Protocol, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
AgeBand = Literal["16-17", "18+"]
|
||||
IdentityStatus = Literal["pending", "verified", "rejected"]
|
||||
|
||||
|
||||
class IdentitySubmission(BaseModel):
|
||||
"""What a learner submits: derived data + document refs only.
|
||||
|
||||
`date_of_birth` is a REAL date (the provider derives the age band) but
|
||||
raw DOB is NEVER persisted — only the derived band (A-305). Document
|
||||
refs are opaque handles (upload ids), never contents.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
learner_id: str = Field(min_length=1)
|
||||
date_of_birth: str = Field(description="ISO date; used to derive age_band, never stored")
|
||||
document_refs: list[str] = Field(
|
||||
default_factory=list,
|
||||
description="Opaque upload handles; raw documents are never stored",
|
||||
)
|
||||
|
||||
|
||||
class IdentityVerdict(BaseModel):
|
||||
"""Provider verdict — what gets stored + surfaced."""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
status: IdentityStatus
|
||||
age_band: AgeBand | None = None
|
||||
provider: str
|
||||
#: A-304 honesty: mock verdicts carry mock=True forever — downstream
|
||||
#: surfaces must never treat a mock verdict as production-verified.
|
||||
mock: bool = True
|
||||
detail: str = ""
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class IdentityProvider(Protocol):
|
||||
"""The KYC port: submit → (poll) → verdict. Never imports api/."""
|
||||
|
||||
async def submit(self, submission: IdentitySubmission) -> str:
|
||||
"""Start verification; returns a submission id (minted once)."""
|
||||
...
|
||||
|
||||
async def poll(self, submission_id: str) -> IdentityVerdict:
|
||||
"""Fetch the (possibly pending) verdict for a submission."""
|
||||
...
|
||||
@@ -0,0 +1,96 @@
|
||||
"""Deterministic mock identity provider (REQ-5-003, A-303).
|
||||
|
||||
Approve-on-policy: every submission verifies unless the caller scripts a
|
||||
rejection (by learner id) or the derived age band fails the floor
|
||||
(under-16 → rejected with an age detail). Verdicts are mock-marked (A-304)
|
||||
— the marker rides every verdict so no downstream surface can ever
|
||||
display mock-verified as production-verified. Never calls the network.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import itertools
|
||||
import secrets
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from .base import IdentitySubmission, IdentityVerdict
|
||||
|
||||
|
||||
def derive_age_band(date_of_birth: str, today: datetime | None = None) -> str:
|
||||
"""Derive the age band from an ISO date. Under-16 returns "under-16".
|
||||
|
||||
Pure + deterministic; used by the store test and the API alike.
|
||||
"""
|
||||
dob = datetime.fromisoformat(date_of_birth)
|
||||
now = today or datetime.now(UTC)
|
||||
age = now.year - dob.year - (
|
||||
(now.month, now.day) < (dob.month, dob.day)
|
||||
)
|
||||
if age < 16:
|
||||
return "under-16"
|
||||
if age < 18:
|
||||
return "16-17"
|
||||
return "18+"
|
||||
|
||||
|
||||
class MockIdentityProvider:
|
||||
"""Scriptable, deterministic; no network, no vendor calls.
|
||||
|
||||
Submission ids are minted UNIQUELY per submit() call (a monotonic
|
||||
counter + the per-process seed from `secrets`): the id is the PK of
|
||||
the insert-only IdentityStore, and a deterministic id derived from
|
||||
(learner_id, date_of_birth) collides on any resubmit-after-terminal
|
||||
(e.g. a rejected learner retrying with the same DOB) — the store
|
||||
surfaces IntegrityError and the API would 500 (cross-phase P0,
|
||||
final review). Uniqueness per call is the contract; determinism of
|
||||
VERDICTS (what tests actually pin) is preserved — poll() derives the
|
||||
band purely from the stored submission.
|
||||
"""
|
||||
|
||||
def __init__(self, reject_learners: set[str] | None = None) -> None:
|
||||
self._submissions: dict[str, IdentitySubmission] = {}
|
||||
self._reject_learners = reject_learners or set()
|
||||
# Per-process nonce: ids are opaque handles (A-305) — never
|
||||
# derived from PII. Counter + nonce keeps ids unique within and
|
||||
# across provider instances on one box.
|
||||
self._nonce = secrets.randbits(32)
|
||||
self._counter = itertools.count()
|
||||
|
||||
async def submit(self, submission: IdentitySubmission) -> str:
|
||||
submission_id = f"idc-{self._nonce:08x}{next(self._counter):08x}"
|
||||
self._submissions[submission_id] = submission
|
||||
return submission_id
|
||||
|
||||
async def poll(self, submission_id: str) -> IdentityVerdict:
|
||||
submission = self._submissions.get(submission_id)
|
||||
if submission is None:
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="unknown submission id",
|
||||
)
|
||||
if submission.learner_id in self._reject_learners:
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="scripted rejection (test)",
|
||||
)
|
||||
band = derive_age_band(submission.date_of_birth)
|
||||
if band == "under-16":
|
||||
return IdentityVerdict(
|
||||
status="rejected",
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="under 16 — the AI school floor is 16+ (COPPA avoidance)",
|
||||
)
|
||||
return IdentityVerdict(
|
||||
status="verified",
|
||||
# P5: derive_age_band returns str; the verdict's Literal contract —
|
||||
# the derived bands are in-contract by construction.
|
||||
age_band=band, # type: ignore[arg-type]
|
||||
provider="mock",
|
||||
mock=True,
|
||||
detail="mock verdict — not production verification",
|
||||
)
|
||||
@@ -0,0 +1,218 @@
|
||||
"""IdentityStore — verification records (REQ-5-003, D-042, D-027 FIFTH store).
|
||||
|
||||
Insert-only + latest-per-learner lookup, modeled on the DefenseStore
|
||||
conventions: WAL + synchronous=NORMAL + busy_timeout + foreign_keys=ON
|
||||
pragmas at connect time, portable column types (str/datetime/JSON) for
|
||||
Postgres parity, @validates hooks for constraints sqlmodel's metaclass
|
||||
drops, tz-aware→naive→tz-aware boundary normalization.
|
||||
|
||||
PII contract (A-305): stores the DERIVED age_band (16-17 | 18+), NEVER a
|
||||
raw date of birth; document_refs are opaque handles, NEVER contents.
|
||||
Verdict provenance is audit data: every record carries provider + the
|
||||
mock marker (A-304) so downstream surfaces can label unverified state
|
||||
honestly.
|
||||
|
||||
Insert-only growth is fine at pilot scale (a-12): learner_id indexed,
|
||||
latest-per-learner lookup, no compaction pre-vendor.
|
||||
|
||||
Boundary (D-027): `identity/` never imports `agents/` / `api/`; this
|
||||
module imports config only.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sqlite3
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Protocol
|
||||
|
||||
from sqlalchemy import event, func, text
|
||||
from sqlalchemy.types import JSON, String
|
||||
from sqlmodel import Field, Session, SQLModel, col, create_engine, select
|
||||
|
||||
from ..config import Settings
|
||||
from .base import IdentityStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class IdentityRecord(SQLModel, table=True):
|
||||
"""One verification submission's lifecycle + verdict provenance."""
|
||||
|
||||
__tablename__ = "identity_record"
|
||||
|
||||
#: PK = the submission id minted once by the provider's submit().
|
||||
id: str = Field(primary_key=True)
|
||||
learner_id: str = Field(index=True)
|
||||
# Bare Literal annotations crash sqlmodel's column inference; explicit
|
||||
# sa_type + the validates hook below give the same contract
|
||||
# (VARCHAR column, Literal-rejected values — DefenseStore pattern).
|
||||
status: IdentityStatus = Field(default="pending", sa_type=String)
|
||||
provider: str
|
||||
#: Verdict provenance: the provider's raw verdict (mock-marked).
|
||||
verdict: dict[str, Any] = Field(default_factory=dict, sa_type=JSON)
|
||||
#: DERIVED band only — raw DOB is never persisted (A-305).
|
||||
age_band: str | None = Field(default=None, sa_type=String)
|
||||
#: Opaque document handles — raw documents never stored (A-305).
|
||||
document_refs: list[str] = Field(default_factory=list, sa_type=JSON)
|
||||
submitted_at: datetime
|
||||
verified_at: datetime | None = Field(default=None)
|
||||
|
||||
@property
|
||||
def mock(self) -> bool:
|
||||
"""A-304: the mock marker rides every surface (record + API)."""
|
||||
return bool(self.verdict.get("mock", True))
|
||||
|
||||
def _validate(self) -> None:
|
||||
if not self.id or not self.learner_id:
|
||||
raise ValueError("id and learner_id must be non-empty")
|
||||
if self.status not in ("pending", "verified", "rejected"):
|
||||
raise ValueError(f"invalid identity status {self.status!r}")
|
||||
# D3 (verifier): a stored band must be canonical or None (pending).
|
||||
# The gate fails closed on anything else; the store refuses to
|
||||
# create it in the first place.
|
||||
if self.age_band is not None and self.age_band not in (
|
||||
"16-17",
|
||||
"18+",
|
||||
"under-16",
|
||||
):
|
||||
raise ValueError(f"invalid age_band {self.age_band!r}")
|
||||
|
||||
def _normalize(self) -> None:
|
||||
self.submitted_at = _as_utc(self.submitted_at)
|
||||
if self.verified_at is not None:
|
||||
self.verified_at = _as_utc(self.verified_at)
|
||||
|
||||
|
||||
def _as_utc(ts: datetime) -> datetime:
|
||||
"""SQLite stores naive; read paths re-label tz-aware UTC (D-027 pattern)."""
|
||||
if ts.tzinfo is None:
|
||||
return ts.replace(tzinfo=UTC)
|
||||
return ts
|
||||
|
||||
|
||||
def _sqlite_connect(dbapi_connection: sqlite3.Connection, _: object) -> None:
|
||||
"""Per-connection pragmas — mirrors the other D-027 stores (P5: the
|
||||
concrete sqlite3.Connection type removes the two attr-defined ignores
|
||||
the loose `object` annotation needed)."""
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA synchronous=NORMAL")
|
||||
cursor.execute("PRAGMA busy_timeout=5000")
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close()
|
||||
|
||||
|
||||
class IdentityStore(Protocol):
|
||||
"""Persistence contract for identity records."""
|
||||
|
||||
def insert(self, record: IdentityRecord) -> IdentityRecord:
|
||||
"""INSERT-ONLY: a duplicate id raises IntegrityError (surfaced, not
|
||||
swallowed — a submission id is minted once)."""
|
||||
...
|
||||
|
||||
def get(self, submission_id: str) -> IdentityRecord | None:
|
||||
"""Point lookup by submission id."""
|
||||
...
|
||||
|
||||
def latest_for_learner(self, learner_id: str) -> IdentityRecord | None:
|
||||
"""Newest record for the learner (or None)."""
|
||||
...
|
||||
|
||||
def count_pending_for_learner(self, learner_id: str) -> int:
|
||||
"""G-13: active pending submissions (cap = 1)."""
|
||||
...
|
||||
|
||||
def mark_verified(
|
||||
self, submission_id: str, verdict: dict[str, Any], age_band: str | None
|
||||
) -> IdentityRecord | None:
|
||||
"""Terminal transition (verified or rejected): stamp + store the
|
||||
provider verdict + derived band. Unknown id → None."""
|
||||
...
|
||||
|
||||
|
||||
class SQLiteIdentityStore:
|
||||
"""SQLite implementation of IdentityStore (D-027)."""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
self._db_path: Path = db_path if db_path is not None else Settings().db_path
|
||||
self._engine = create_engine(
|
||||
f"sqlite:///{self._db_path}",
|
||||
connect_args={"check_same_thread": False},
|
||||
)
|
||||
event.listen(self._engine, "connect", _sqlite_connect)
|
||||
SQLModel.metadata.create_all(self._engine)
|
||||
|
||||
def insert(self, record: IdentityRecord) -> IdentityRecord:
|
||||
record._validate()
|
||||
record._normalize()
|
||||
with Session(self._engine) as session:
|
||||
session.add(record)
|
||||
session.commit() # IntegrityError SURFACES (insert-only, minted-once)
|
||||
session.refresh(record)
|
||||
return record
|
||||
|
||||
def get(self, submission_id: str) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
rec = session.get(IdentityRecord, submission_id)
|
||||
if rec is None:
|
||||
return None
|
||||
session.refresh(rec)
|
||||
rec.submitted_at = _as_utc(rec.submitted_at)
|
||||
if rec.verified_at is not None:
|
||||
rec.verified_at = _as_utc(rec.verified_at)
|
||||
return rec
|
||||
|
||||
def latest_for_learner(self, learner_id: str) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
# D4 (verifier): submitted_at alone can tie at microsecond
|
||||
# resolution — sqlite rowid breaks the tie deterministically
|
||||
# (last inserted wins, mirroring insert-only chronology).
|
||||
# rowid is a SQLite physical column, not a SQLModel field — it
|
||||
# rides the query as raw text.
|
||||
rec = session.exec(
|
||||
select(IdentityRecord)
|
||||
.where(IdentityRecord.learner_id == learner_id)
|
||||
.order_by(
|
||||
# P5: col() — SQLModel class attrs are Mapped[T], the
|
||||
# column operators (.desc()) ride col().
|
||||
col(IdentityRecord.submitted_at).desc(),
|
||||
text("rowid DESC"),
|
||||
)
|
||||
).first()
|
||||
if rec is not None:
|
||||
rec.submitted_at = _as_utc(rec.submitted_at)
|
||||
if rec.verified_at is not None:
|
||||
rec.verified_at = _as_utc(rec.verified_at)
|
||||
return rec
|
||||
|
||||
def count_pending_for_learner(self, learner_id: str) -> int:
|
||||
with Session(self._engine) as session:
|
||||
return session.exec(
|
||||
select(func.count())
|
||||
.select_from(IdentityRecord)
|
||||
.where(
|
||||
IdentityRecord.learner_id == learner_id,
|
||||
IdentityRecord.status == "pending",
|
||||
)
|
||||
).one()
|
||||
|
||||
def mark_verified(
|
||||
self, submission_id: str, verdict: dict[str, Any], age_band: str | None
|
||||
) -> IdentityRecord | None:
|
||||
with Session(self._engine) as session:
|
||||
rec = session.get(IdentityRecord, submission_id)
|
||||
if rec is None:
|
||||
return None
|
||||
rec.verdict = verdict
|
||||
rec.age_band = age_band
|
||||
rec.verified_at = datetime.now(UTC)
|
||||
rec.status = "verified" if verdict.get("status") == "verified" else "rejected"
|
||||
rec._validate() # D3: transitions validate like inserts
|
||||
session.commit()
|
||||
session.refresh(rec)
|
||||
return rec
|
||||
|
||||
def close(self) -> None:
|
||||
self._engine.dispose()
|
||||
@@ -12,7 +12,11 @@ from .types import Message
|
||||
|
||||
|
||||
class LLMProvider(Protocol):
|
||||
async def stream_chat(
|
||||
# P5 (D-058): plain `def` returning AsyncIterator — the honest shape for
|
||||
# async-generator implementations. `async def -> AsyncIterator` makes mypy
|
||||
# see a coroutine WRAPPING the iterator (every call site would need an
|
||||
# extra await); async generators ARE functions returning AsyncIterator.
|
||||
def stream_chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
*,
|
||||
|
||||
@@ -9,19 +9,40 @@ import httpx
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from . import __version__
|
||||
from .agents.registry import AgentRegistry, register_builtin_agents
|
||||
from .agents.session import InMemorySessionStore
|
||||
from .api import (
|
||||
assessment_router,
|
||||
chat_router,
|
||||
defense_router,
|
||||
lab_router,
|
||||
mentor_router,
|
||||
proctor_router,
|
||||
sandboxes_router,
|
||||
telemetry_router,
|
||||
variants_router,
|
||||
)
|
||||
from .config import Settings
|
||||
from .grading.engine import GradingEngine
|
||||
from .grading.store import SQLiteGradeStore
|
||||
from .identity.mock import MockIdentityProvider
|
||||
from .identity.store import SQLiteIdentityStore
|
||||
from .llm import create_provider
|
||||
from .sandbox import SandboxManager, UnshareBackend
|
||||
from .sandbox.capability import (
|
||||
DEGRADE_REMEDIATION,
|
||||
container_detected,
|
||||
sandbox_fabric_degraded,
|
||||
)
|
||||
from .telemetry.ingest import TraceIntegrityMap
|
||||
from .telemetry.store import SQLiteTraceStore
|
||||
from .telemetry.tokens import IngestToken
|
||||
from .variants.generator import VariantGenerator
|
||||
from .variants.store import SQLiteVariantStore
|
||||
from .voice.defense_store import SQLiteDefenseStore
|
||||
from .voice.factory import voice_provider_from_settings
|
||||
from .voice.mock import MockVoiceProvider
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -39,7 +60,11 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
timeout = httpx.Timeout(connect=10.0, read=300.0, write=30.0, pool=10.0)
|
||||
app.state.http_client = httpx.AsyncClient(timeout=timeout)
|
||||
app.state.settings = settings
|
||||
app.state.provider = create_provider(settings, app.state.http_client)
|
||||
# State-injection override (same pattern as the stores): tests may
|
||||
# pre-set app.state.provider with a scripted mock; only construct the
|
||||
# configured provider when none is present.
|
||||
if getattr(app.state, "provider", None) is None:
|
||||
app.state.provider = create_provider(settings, app.state.http_client)
|
||||
app.state.session_store = InMemorySessionStore()
|
||||
app.state.agent_registry = AgentRegistry()
|
||||
register_builtin_agents(app.state.agent_registry)
|
||||
@@ -48,12 +73,223 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
# app.state. Tests may pre-set app.state.sandbox_manager (dependency
|
||||
# override by state injection) to swap the backend; the lifespan then
|
||||
# adopts it instead of constructing the real UnshareBackend one.
|
||||
# v0.4.7 (P1): the ingest token is created BEFORE the manager (the
|
||||
# manager MINTS into NC_INGEST_URL; the WS endpoint VERIFIES) and
|
||||
# shared via app.state — one instance, so the per-boot random secret
|
||||
# (unset AI_TELEMETRY_INGEST_SECRET) agrees on both sides.
|
||||
if getattr(app.state, "ingest_token", None) is None:
|
||||
app.state.ingest_token = IngestToken(settings.telemetry_ingest_secret)
|
||||
manager = getattr(app.state, "sandbox_manager", None)
|
||||
if manager is None:
|
||||
manager = SandboxManager(backend=UnshareBackend(), settings=settings)
|
||||
manager = SandboxManager(
|
||||
backend=UnshareBackend(),
|
||||
settings=settings,
|
||||
ingest_token=app.state.ingest_token,
|
||||
)
|
||||
app.state.sandbox_manager = manager
|
||||
await manager.start() # a-1: reap on-disk orphans from a previous process
|
||||
|
||||
# Telemetry persistence (REQ-3-003, D-027): TraceStore wired through
|
||||
# app.state. Tests may pre-set app.state.trace_store (state-injection
|
||||
# override, same pattern as sandbox_manager) — the lifespan adopts it.
|
||||
trace_store = getattr(app.state, "trace_store", None)
|
||||
if trace_store is None:
|
||||
settings.db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
trace_store = SQLiteTraceStore(db_path=settings.db_path)
|
||||
app.state.trace_store = trace_store
|
||||
# Trace-integrity flags (G-3 INCOMPLETE_FLOODED): process-local map is
|
||||
# intentional (D-019 registry precedent); the lifespan owns it so the
|
||||
# grader and the ingest endpoint share one instance.
|
||||
if getattr(app.state, "trace_integrity", None) is None:
|
||||
app.state.trace_integrity = TraceIntegrityMap()
|
||||
|
||||
# Variant generation (REQ-3-005): VariantStore from the same
|
||||
# SQLite file as traces/grades (D-027), one VariantGenerator singleton
|
||||
# wired through app.state — the generator receives store + provider
|
||||
# via constructor DI and knows nothing of FastAPI (api/ composes it,
|
||||
# same pattern as GradingEngine). Tests may pre-set
|
||||
# app.state.variant_store / app.state.variant_generator (the same
|
||||
# state-injection override); the lifespan adopts a pre-set store but
|
||||
# NEVER rebuilds a pre-set generator (its provider binding is part
|
||||
# of the test fixture).
|
||||
# ORDER NOTE: built BEFORE the grading engine — the engine takes the
|
||||
# variant store (Phase 4 MH#4: variant anchors ship to the grader
|
||||
# prompt; variant_seed stamped on graded records).
|
||||
variant_store = getattr(app.state, "variant_store", None)
|
||||
if variant_store is None:
|
||||
settings.db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
variant_store = SQLiteVariantStore(db_path=settings.db_path)
|
||||
app.state.variant_store = variant_store
|
||||
if getattr(app.state, "variant_generator", None) is None:
|
||||
app.state.variant_generator = VariantGenerator(
|
||||
variant_store,
|
||||
app.state.provider,
|
||||
model=settings.model,
|
||||
)
|
||||
|
||||
# Oral defense (REQ-3-006): DefenseStore (same SQLite file) + the
|
||||
# mock-first voice provider (D-030) + the seventh Examiner agent.
|
||||
# Tests may pre-set app.state.defense_store / voice_provider /
|
||||
# examiner_agent (state-injection override; never rebuilt if pre-set).
|
||||
defense_store = getattr(app.state, "defense_store", None)
|
||||
if defense_store is None:
|
||||
defense_store = SQLiteDefenseStore(db_path=settings.db_path)
|
||||
app.state.defense_store = defense_store
|
||||
if getattr(app.state, "voice_provider", None) is None:
|
||||
# G-11 (boot survival): a misconfigured real provider must never
|
||||
# crash the unattended deploy — fall back to mock loudly. The
|
||||
# mock provider's descriptor honestly reports mode='mock' so the
|
||||
# UI badge cannot lie about which path is live.
|
||||
try:
|
||||
app.state.voice_provider = voice_provider_from_settings(
|
||||
settings, app.state.http_client
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"voice provider %r unavailable (%s); falling back to mock "
|
||||
"— fix the AI_VOICE_* settings and restart",
|
||||
settings.voice_provider,
|
||||
exc,
|
||||
)
|
||||
app.state.voice_provider = MockVoiceProvider()
|
||||
if getattr(app.state, "examiner_agent", None) is None:
|
||||
from .agents.examiner import ExaminerAgent
|
||||
|
||||
app.state.examiner_agent = ExaminerAgent(app.state.provider, settings)
|
||||
|
||||
# Identity verification (REQ-5-003): 5th D-027 store (same SQLite
|
||||
# file) + mock-first provider (A-303). State-injection overrides
|
||||
# preserved — tests may pre-set either.
|
||||
identity_store = getattr(app.state, "identity_store", None)
|
||||
if identity_store is None:
|
||||
identity_store = SQLiteIdentityStore(db_path=settings.db_path)
|
||||
app.state.identity_store = identity_store
|
||||
if getattr(app.state, "identity_provider", None) is None:
|
||||
app.state.identity_provider = MockIdentityProvider()
|
||||
|
||||
# Marketplace engine (v0.6, REQ-6-001): 6th D-027 store family on
|
||||
# the same SQLite file. State-injection override preserved.
|
||||
if getattr(app.state, "marketplace_store", None) is None:
|
||||
from .marketplace.store import SQLiteMarketplaceStore
|
||||
|
||||
app.state.marketplace_store = SQLiteMarketplaceStore(
|
||||
db_path=settings.db_path
|
||||
)
|
||||
|
||||
# Billing engine (v0.9, REQ-9-002): 7th D-027 store family +
|
||||
# the seeded catalog (D-078 — idempotent upsert, safe on every
|
||||
# boot). State-injection override preserved.
|
||||
if getattr(app.state, "billing_store", None) is None:
|
||||
from .billing.store import SQLiteBillingStore
|
||||
|
||||
app.state.billing_store = SQLiteBillingStore(db_path=settings.db_path)
|
||||
if getattr(app.state, "billing_seeded", None) is not True:
|
||||
from .billing.catalog import seed_billing_catalog
|
||||
|
||||
seed_billing_catalog(app.state.billing_store)
|
||||
app.state.billing_seeded = True
|
||||
|
||||
# Employer auth engine (v0.10, REQ-10-001): 8th D-027 store
|
||||
# family on the same SQLite file. State-injection override
|
||||
# preserved (the billing_store getattr-guard template).
|
||||
if getattr(app.state, "employer_store", None) is None:
|
||||
from .employers.store import SQLiteEmployerStore
|
||||
|
||||
app.state.employer_store = SQLiteEmployerStore(
|
||||
db_path=settings.db_path
|
||||
)
|
||||
|
||||
# Payment provider (v0.9 P3, REQ-9-003, D-075): mock-first
|
||||
# default; stripe over the shared httpx pool. G-11 (boot
|
||||
# survival): a misconfigured real provider must never crash the
|
||||
# unattended deploy — fall back to mock loudly (the
|
||||
# voice/embedding pattern).
|
||||
if getattr(app.state, "payment_provider", None) is None:
|
||||
from .billing.factory import payment_provider_from_settings
|
||||
from .billing.mock import MockPaymentProvider
|
||||
|
||||
try:
|
||||
app.state.payment_provider = payment_provider_from_settings(
|
||||
settings, app.state.http_client
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"payment provider %r unavailable (%s); falling back to "
|
||||
"mock — fix the AI_PAYMENT_* settings and restart",
|
||||
settings.payment_provider,
|
||||
exc,
|
||||
)
|
||||
app.state.payment_provider = MockPaymentProvider()
|
||||
|
||||
# Embedding provider (v0.6 P3, REQ-6-003, D-051): mock-first
|
||||
# feature-hash default; openai-embed against any OpenAI-compatible
|
||||
# embeddings endpoint. State-injection override preserved (the
|
||||
# voice pattern). G-11 (boot survival): a misconfigured real
|
||||
# provider must never crash the unattended deploy — fall back to
|
||||
# mock loudly.
|
||||
if getattr(app.state, "embedding_provider", None) is None:
|
||||
from .marketplace.embedding.factory import (
|
||||
embedding_provider_from_settings,
|
||||
)
|
||||
from .marketplace.embedding.mock import MockEmbedProvider
|
||||
|
||||
try:
|
||||
app.state.embedding_provider = embedding_provider_from_settings(
|
||||
settings, app.state.http_client
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"embedding provider %r unavailable (%s); falling back to "
|
||||
"mock — fix the AI_EMBED_* settings and restart",
|
||||
settings.embed_provider,
|
||||
exc,
|
||||
)
|
||||
app.state.embedding_provider = MockEmbedProvider()
|
||||
|
||||
# G-11/D-057: admin routes fail CLOSED on an unset token, and the
|
||||
# founder must be told loudly at boot (never a silent lockout).
|
||||
if not settings.admin_token.get_secret_value():
|
||||
logger.warning(
|
||||
"AI_ADMIN_TOKEN is unset — /v1/marketplace/admin/* routes "
|
||||
"are 403 (fail-closed). Set AI_ADMIN_TOKEN in .env to "
|
||||
"author postings, run ingest, and review applications."
|
||||
)
|
||||
|
||||
# G-105 (P5/D-056): an unset origin makes the sitemap serve the
|
||||
# honest empty set and robots omit the Sitemap directive — the
|
||||
# founder must be told loudly at boot (G-11 doctrine; the
|
||||
# request-time fallback never emits malformed/relative URLs).
|
||||
if not (settings.public_base_url or "").strip():
|
||||
logger.warning(
|
||||
"AI_PUBLIC_BASE_URL is unset — /sitemap.xml serves the "
|
||||
"empty static-page set and robots.txt omits the Sitemap "
|
||||
"directive (spec requires absolute URLs). Set the deployed "
|
||||
"origin (e.g. https://nextcraft.example) in .env to serve "
|
||||
"the full sitemap."
|
||||
)
|
||||
|
||||
# Grading persistence + engine (REQ-3-004): GradeStore from the same
|
||||
# SQLite file as traces (D-027), one GradingEngine singleton wired
|
||||
# through app.state — the engine receives its stores via constructor
|
||||
# DI and knows nothing of FastAPI (api/ owns composition). Tests may
|
||||
# pre-set app.state.grade_store / app.state.grading_engine (the same
|
||||
# state-injection override as sandbox_manager/trace_store) to swap
|
||||
# either; the lifespan adopts a pre-set store but NEVER rebuilds a
|
||||
# pre-set engine (its provider binding is part of the test fixture).
|
||||
grade_store = getattr(app.state, "grade_store", None)
|
||||
if grade_store is None:
|
||||
grade_store = SQLiteGradeStore(db_path=settings.db_path)
|
||||
app.state.grade_store = grade_store
|
||||
if getattr(app.state, "grading_engine", None) is None:
|
||||
app.state.grading_engine = GradingEngine(
|
||||
trace_store,
|
||||
grade_store,
|
||||
app.state.trace_integrity,
|
||||
app.state.provider,
|
||||
model=settings.model,
|
||||
variant_store=variant_store, # MH#4: anchors + seed (D-029)
|
||||
)
|
||||
|
||||
async def _reaper_loop() -> None:
|
||||
# Wall-clock timeout + G-2 workdir-size sweep, one pass per tick.
|
||||
while True:
|
||||
@@ -64,6 +300,24 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
logger.exception("sandbox reaper pass failed; retrying next tick")
|
||||
|
||||
reaper = asyncio.create_task(_reaper_loop())
|
||||
|
||||
# v0.8 honest degrade (D-068/G-208): probe once at boot; if the
|
||||
# namespace fabric cannot work on this host (unprivileged container
|
||||
# under docker's default seccomp), say so LOUDLY once with the shared
|
||||
# remediation constant — the 503 arm in api/sandboxes.py carries the
|
||||
# same text, so the surfaces cannot drift. The probe is the truth
|
||||
# source; the container marker only tailors the message.
|
||||
if sandbox_fabric_degraded():
|
||||
if container_detected():
|
||||
logger.warning(
|
||||
"SANDBOX FABRIC DEGRADED (container detected): %s",
|
||||
DEGRADE_REMEDIATION,
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"SANDBOX FABRIC DEGRADED: %s", DEGRADE_REMEDIATION
|
||||
)
|
||||
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
@@ -73,16 +327,34 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
# No orphans outlive the process (a-1, shutdown half): destroy
|
||||
# everything live; workdirs stay on disk for snapshot restore.
|
||||
await manager.destroy_all()
|
||||
trace_store.close()
|
||||
grade_store.close()
|
||||
variant_store.close()
|
||||
defense_store.close()
|
||||
identity_store.close()
|
||||
app.state.billing_store.close()
|
||||
app.state.employer_store.close()
|
||||
await app.state.http_client.aclose()
|
||||
|
||||
app = FastAPI(title="Nextcraft AI Service", version="0.3.0", lifespan=lifespan)
|
||||
app = FastAPI(title="Nextcraft AI Service", version=__version__, lifespan=lifespan)
|
||||
|
||||
# A-008: localhost-only CORS, no credentials
|
||||
# A-008 + D-038: no-credentials CORS. Default '*' admits remote-browser
|
||||
# origins in network mode (safe only because allow_credentials stays
|
||||
# False — never enable credentials with a wildcard). AI_CORS_ORIGINS
|
||||
# restricts to an explicit list. PUT is CONTRACT, not trivia: the learner
|
||||
# build surface writes workspace files with PUT (engine-client writeFile)
|
||||
# — v0.3 initially shipped without it and every cross-origin Save failed
|
||||
# preflight (caught in P7 review; tests/api/test_cors.py pins the policy).
|
||||
# v0.10 (D-090): "Authorization" joins allow_headers — the employer
|
||||
# session surface sends Bearer tokens, and without it every
|
||||
# cross-origin employer fetch dies preflight in the two-port dev +
|
||||
# D-038 network-mode topologies (the E87 lesson applied before it
|
||||
# ships; test_cors.py pins it).
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://localhost:3000", "http://127.0.0.1:3000"],
|
||||
allow_methods=["GET", "POST", "DELETE", "OPTIONS"],
|
||||
allow_headers=["Content-Type"],
|
||||
allow_origins=settings.cors_origin_list,
|
||||
allow_methods=["GET", "POST", "PUT", "DELETE", "OPTIONS"],
|
||||
allow_headers=["Content-Type", "Authorization"],
|
||||
allow_credentials=False,
|
||||
)
|
||||
|
||||
@@ -100,6 +372,158 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
app.include_router(mentor_router)
|
||||
app.include_router(proctor_router)
|
||||
app.include_router(sandboxes_router)
|
||||
app.include_router(telemetry_router)
|
||||
app.include_router(variants_router)
|
||||
app.include_router(defense_router)
|
||||
|
||||
# v0.5 identity (REQ-5-003/004): verification flow + age-gate deps.
|
||||
# v0.6: the G-18 marketplace stub moved to api/marketplace.py and is
|
||||
# REAL (D-058) — identity keeps only its own router + gate exports.
|
||||
from .api.identity import router as identity_router
|
||||
from .api.marketplace import router as marketplace_public_router
|
||||
from .api.marketplace_admin import router as marketplace_admin_router
|
||||
|
||||
app.include_router(identity_router)
|
||||
app.include_router(marketplace_public_router)
|
||||
app.include_router(marketplace_admin_router)
|
||||
|
||||
# v0.9 billing (REQ-9-002/003): the public catalog + webhook read
|
||||
# surface + the operator checkout routes (admin-gated, D-076). The
|
||||
# mock-complete route is gated by provider selection at runtime
|
||||
# (G-301 — admin-gated AND mock-only), so the router mounts whole.
|
||||
from .api.billing import router as billing_router
|
||||
from .api.billing_admin import router as billing_admin_router
|
||||
|
||||
app.include_router(billing_router)
|
||||
app.include_router(billing_admin_router)
|
||||
|
||||
# v0.10 employers (REQ-10-001): register/login/logout/me session
|
||||
# surface (D-090 Bearer transport; throttles D-088). P2 (REQ-10-002):
|
||||
# the claims route rides the sibling router (session-gated, D-089).
|
||||
from .api.employers import claims_router as employers_claims_router
|
||||
from .api.employers import router as employers_router
|
||||
from .api.employers_admin import router as employers_admin_router
|
||||
|
||||
app.include_router(employers_router)
|
||||
app.include_router(employers_claims_router)
|
||||
# P2 (REQ-10-002, D-089/D-094): the claim review/approve/reject +
|
||||
# accounts/audit operator routes — require_admin (fail-closed).
|
||||
app.include_router(employers_admin_router)
|
||||
|
||||
# v0.6 P5 (REQ-6-006, D-056): dynamic sitemap/robots — request-time
|
||||
# enumeration (build-time Next sitemap.ts cannot see runtime
|
||||
# postings under output:'export'); API routes beat the static mount.
|
||||
from .api.sitemap import router as sitemap_router
|
||||
|
||||
app.include_router(sitemap_router)
|
||||
|
||||
# A-305/D1: PII-safe 422s — FastAPI echoes the offending `input` in
|
||||
# validation errors by default; for the identity submit body that
|
||||
# leaks the raw DOB into responses + client logs. The handler scrubs
|
||||
# PII field inputs (scoped app-wide; harmless elsewhere).
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
def _scrub_pii_validation(request, exc):
|
||||
# P5: the old `# type: ignore[unused-arg]` was inert (the unused-arg
|
||||
# code is not enabled by this baseline); removed per D-058.
|
||||
# v0.10 (R-56/open-risk-3): gains "password" + "email" — a pydantic
|
||||
# 422 on an employer register/login body would echo the raw
|
||||
# password into err["input"] (the response + client logs) today.
|
||||
# Security-audit fix (Wave 1-3, task 1-3-01): the field arm alone
|
||||
# leaks on WHOLE-BODY type errors — a non-dict body (list/str/int)
|
||||
# errors at loc == ("body",) with input = the ENTIRE submitted
|
||||
# payload verbatim, which for a mistyped register attempt carries
|
||||
# the raw password. Any body-level error whose input mentions a
|
||||
# PII key (dict or payload-shaped) is scrubbed too.
|
||||
pii_fields = frozenset({"date_of_birth", "password", "email"})
|
||||
scrubbed = []
|
||||
for err in exc.errors():
|
||||
err = dict(err)
|
||||
# FastAPI's default 422s are JSON-safe EXCEPT ctx payloads
|
||||
# carrying raw ValueError objects (pydantic model_validator
|
||||
# errors); strip ctx body-wide so non-PII routes keep their
|
||||
# 422 shape (msg + loc carry the meaning).
|
||||
ctx = err.get("ctx")
|
||||
if isinstance(ctx, dict):
|
||||
err["ctx"] = {
|
||||
k: v for k, v in ctx.items() if isinstance(v, (str, int, float, bool))
|
||||
}
|
||||
if err.get("loc") and err["loc"][-1] in pii_fields:
|
||||
# A-305/D1: never echo the submitted value (input) or the
|
||||
# ctx payload (carries the ValueError); the msg is the
|
||||
# constraint text and is safe.
|
||||
err["input"] = "[redacted]"
|
||||
err.pop("ctx", None)
|
||||
elif err.get("loc") and err["loc"][-1] == "body":
|
||||
# Whole-body error: input is the raw payload — redact
|
||||
# whenever it mentions a PII key (dict with one, or any
|
||||
# container whose repr might embed one). The loc+msg still
|
||||
# carry the failure's meaning; only the echo is dropped.
|
||||
if not isinstance(err.get("input"), dict) or any(
|
||||
key in err["input"] for key in pii_fields
|
||||
):
|
||||
err["input"] = "[redacted]"
|
||||
err.pop("ctx", None)
|
||||
scrubbed.append(err)
|
||||
return JSONResponse(status_code=422, content={"detail": scrubbed})
|
||||
|
||||
app.add_exception_handler(RequestValidationError, _scrub_pii_validation)
|
||||
|
||||
# v0.6 G-101: Next 15 output:'export' hard-throws (E87) for dynamic
|
||||
# routes without generateStaticParams, so jobs/[jobId] and
|
||||
# employers/[employerId] export exactly ONE placeholder shell
|
||||
# (out/marketplace/jobs/_/index.html). These catch-all routes serve
|
||||
# that shell for ANY id — registered BEFORE the StaticFiles mount so
|
||||
# they win (the same routes-beat-the-mount mechanism the sitemap
|
||||
# relies on, G-109). The shell client-fetches the real posting by
|
||||
# the path id and notFound()s on 404 — a resolvable URL for every
|
||||
# sitemap entry.
|
||||
from .api.marketplace_shells import register_marketplace_shell_routes
|
||||
|
||||
register_marketplace_shell_routes(app, settings)
|
||||
|
||||
# v0.3.6 single-port deploy: serve the exported web app (apps/web/out)
|
||||
# from the SAME origin as the API when AI_WEB_STATIC_DIR is set. Mounted
|
||||
# AFTER all routers, so /v1/*, /health, /docs win; StaticFiles(html=True)
|
||||
# then resolves / → index.html, /dashboard/ → dashboard/index.html. A
|
||||
# 404 handler below serves the export's 404.html for unknown paths so
|
||||
# browsers see the site's not-found page instead of FastAPI's JSON.
|
||||
# Default unset → no mount, dev/tests see the plain API app.
|
||||
if str(settings.web_static_dir) not in ("", "."):
|
||||
from fastapi.responses import FileResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
|
||||
web_dir = settings.web_static_dir
|
||||
if not web_dir.is_dir():
|
||||
raise RuntimeError(
|
||||
f"AI_WEB_STATIC_DIR is set but {web_dir} does not exist — "
|
||||
"build the web app first (pnpm build) or unset the setting"
|
||||
)
|
||||
|
||||
@app.exception_handler(404)
|
||||
async def _spa_404(request, exc):
|
||||
# P5: the old `# type: ignore[unused]` was inert (the unused code
|
||||
# is not enabled by this baseline); removed per D-058.
|
||||
not_found_page = web_dir / "404.html"
|
||||
if not_found_page.is_file():
|
||||
return FileResponse(not_found_page, status_code=404)
|
||||
raise exc
|
||||
|
||||
app.mount("/", StaticFiles(directory=web_dir, html=True), name="web")
|
||||
else:
|
||||
# v0.4.6: the v0.3.6/v0.4-era "site down" root cause was a silently
|
||||
# mount-less boot — behind HAProxy, / answers this app's JSON 404
|
||||
# and nothing detects it. Unattended deploys MUST be loud (G-11
|
||||
# doctrine). Dev/tests keep the no-mount default; production boxes
|
||||
# see a warning naming the fix.
|
||||
logger.warning(
|
||||
"AI_WEB_STATIC_DIR is unset — the exported web UI is NOT served; "
|
||||
"/ answers the API's JSON 404. For single-port deploys, build the "
|
||||
"web app (NEXT_PUBLIC_AI_SERVICE_URL=self pnpm build) and run it "
|
||||
"via `nextcraft dev`/`dev -d` (auto-wires apps/web/out) or set "
|
||||
"AI_WEB_STATIC_DIR explicitly."
|
||||
)
|
||||
return app
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Marketplace engine package (REQ-6-001, D-048/D-049).
|
||||
|
||||
The 6th engine module inside apps/ai-service (no new apps, no new ports,
|
||||
D-031 boundary): pydantic contracts (`base.py`), the D-027 store family's
|
||||
sixth SQLite store (`store.py`), and PostingSource ingest adapters
|
||||
(`ingest.py`). Tagging (P2), embedding (P3), enrichment + resume (P4)
|
||||
extend this package; the HTTP surface lives in `api/marketplace*.py`
|
||||
which composes this module via DI.
|
||||
|
||||
Boundary (D-031/D-048): `marketplace/` never imports `agents/` or
|
||||
`api/`; this package imports config + stdlib only.
|
||||
"""
|
||||
@@ -0,0 +1,174 @@
|
||||
"""Marketplace pydantic contracts (REQ-6-001, D-048, A-402).
|
||||
|
||||
The wire + store shapes for the postings backend. `IngestPosting` is the
|
||||
A-402 adapter shape (what every PostingSource yields); `Posting` is the
|
||||
full row shape (status lifecycle + tags); `Employer` carries the raw
|
||||
ingested fields (enrichment + provenance merge is P4's read-time job).
|
||||
|
||||
D-051/D-053 note: the mock `matchScore` from the TS fixtures is NOT part
|
||||
of any contract — P3 computes real scores at search time; absent score =
|
||||
absent query context (G-106).
|
||||
|
||||
Boundary (D-048): mirrors D-031 — never imports `agents/` or `api/`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
PostingStatus = Literal["draft", "published", "closed"]
|
||||
|
||||
#: Lifecycle edge set (D-049/D-057): draft → published → closed. A
|
||||
#: transition not in this map is invalid (admin route → 409).
|
||||
VALID_TRANSITIONS: dict[str, str] = {
|
||||
"draft": "published",
|
||||
"published": "closed",
|
||||
}
|
||||
|
||||
Seniority = Literal["entry", "mid", "senior", "staff", "principal"]
|
||||
|
||||
#: P2 (D-052) enumerates the category taxonomy; the seed corpus carries
|
||||
#: category=null and P2's tagging pass fills it. The store accepts any
|
||||
#: short string (validation lands with the tagger); `None` = not yet
|
||||
#: tagged.
|
||||
Category = str | None
|
||||
|
||||
|
||||
class PostingTags(BaseModel):
|
||||
"""Derived tags on a posting row (D-052 schema, persisted as the
|
||||
posting's JSON `tags` column in P2; the column exists from day one).
|
||||
|
||||
`skills` is bounded at 10 by the tagger (D-052); the schema itself
|
||||
keeps the bound so the store can validate a hand-authored payload.
|
||||
`extra="forbid"`: a non-tag JSON object (e.g. the mock provider's
|
||||
canned {"summary":...}) must NOT silently validate as all-default
|
||||
tags — the untagged signal stays honest (verifier P2-3; P3
|
||||
embeddings must not treat empty-default parses as real tags).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
skills: list[str] = Field(default_factory=list, max_length=10)
|
||||
seniority: Seniority | None = None
|
||||
category: Category = None
|
||||
|
||||
|
||||
class Employer(BaseModel):
|
||||
"""Raw ingested employer fields (D-049 `employer` table row shape).
|
||||
|
||||
P4 (D-053) enriches on read with per-field `{value, source}`
|
||||
provenance; P1 surfaces the raw fields only.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
source: str = Field(min_length=1)
|
||||
external_id: str = Field(min_length=1)
|
||||
name: str = Field(min_length=1)
|
||||
logo: str = ""
|
||||
description: str = ""
|
||||
industry: str = ""
|
||||
size: str = ""
|
||||
location: str = ""
|
||||
website: str = ""
|
||||
social_links: dict[str, str] = Field(default_factory=dict)
|
||||
culture: str = ""
|
||||
|
||||
|
||||
class IngestPosting(BaseModel):
|
||||
"""The A-402 PostingSource payload: (source, external_id) identity +
|
||||
employer binding + the raw posting fields. Idempotent upsert key is
|
||||
(source, external_id) — a re-fetch overwrites the same row in place.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
source: str = Field(min_length=1)
|
||||
external_id: str = Field(min_length=1)
|
||||
employer_id: str = Field(min_length=1)
|
||||
title: str = Field(min_length=1)
|
||||
description: str = ""
|
||||
required_competencies: list[str] = Field(default_factory=list)
|
||||
skills: list[str] = Field(default_factory=list)
|
||||
seniority: Seniority | None = None
|
||||
category: Category = None
|
||||
location: str = ""
|
||||
remote: bool = False
|
||||
salary_min: int | None = None
|
||||
salary_max: int | None = None
|
||||
#: G-102: seed-port postings carry their ORIGINAL postedAt; the
|
||||
#: store applies it verbatim (status=published via the ingest
|
||||
#: contract, not the field).
|
||||
posted_at: datetime | None = None
|
||||
|
||||
|
||||
class Posting(BaseModel):
|
||||
"""Full posting row (D-049 `posting` table). Status is the lifecycle
|
||||
column (draft|published|closed, D-057); `tags` is the P2-derived
|
||||
JSON column (D-052 — empty until the tagger runs). `matchScore` is
|
||||
deliberately absent (G-106/G-107).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
source: str = Field(min_length=1)
|
||||
external_id: str = Field(min_length=1)
|
||||
employer_id: str = Field(min_length=1)
|
||||
title: str = Field(min_length=1)
|
||||
description: str = ""
|
||||
required_competencies: list[str] = Field(default_factory=list)
|
||||
skills: list[str] = Field(default_factory=list)
|
||||
seniority: Seniority | None = None
|
||||
category: Category = None
|
||||
location: str = ""
|
||||
remote: bool = False
|
||||
salary_min: int | None = None
|
||||
salary_max: int | None = None
|
||||
status: PostingStatus = "draft"
|
||||
tags: PostingTags | None = None
|
||||
posted_at: datetime | None = None
|
||||
|
||||
|
||||
class Application(BaseModel):
|
||||
"""A learner's application to a posting (D-049 `application` table).
|
||||
Unique (posting_id, learner_id) → duplicate apply is idempotent
|
||||
200-with-existing (A-408/D-058). P1 status is always `submitted`;
|
||||
richer states are post-v0.6.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
id: str = Field(min_length=1)
|
||||
posting_id: str = Field(min_length=1)
|
||||
learner_id: str = Field(min_length=1)
|
||||
status: str = "submitted"
|
||||
created_at: datetime | None = None
|
||||
|
||||
|
||||
class PostingPage(BaseModel):
|
||||
"""Published-only list response (G-103): the page + the honest
|
||||
total so the board can render "Showing X of Y" over a paged
|
||||
fetch-all without fabrication."""
|
||||
|
||||
postings: list[Posting]
|
||||
total_published: int
|
||||
|
||||
|
||||
class IngestResult(BaseModel):
|
||||
"""Honest ingest run summary (founder surface)."""
|
||||
|
||||
postings_upserted: int = 0
|
||||
employers_upserted: int = 0
|
||||
|
||||
|
||||
def posting_summary(posting: Posting) -> dict[str, Any]:
|
||||
"""Serialization helper shared by the list + detail responses — one
|
||||
shape so the G-107 pin (no `matchScore` in any marketplace-path
|
||||
payload) covers both surfaces."""
|
||||
return posting.model_dump(mode="json")
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Marketplace embedding providers (D-051, REQ-6-003).
|
||||
|
||||
Protocol + deterministic feature-hash mock + OpenAI-compatible real
|
||||
provider + factory. Mirrors the voice/ layout (base/mock/openai_compat/
|
||||
factory) — embeddings need NO descriptor: match scores are computed
|
||||
server-side in search.py, never shipped as client capability hints.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import EmbeddingProvider, EmbeddingResult
|
||||
from .factory import UnknownEmbedProviderError, embedding_provider_from_settings
|
||||
from .mock import (
|
||||
MOCK_EMBED_DIM,
|
||||
MOCK_EMBED_MODEL,
|
||||
MockEmbedProvider,
|
||||
embed_text,
|
||||
normalize_tokens,
|
||||
)
|
||||
from .openai_compat import OpenAIEmbedProvider
|
||||
|
||||
__all__ = [
|
||||
"MOCK_EMBED_DIM",
|
||||
"MOCK_EMBED_MODEL",
|
||||
"EmbeddingProvider",
|
||||
"EmbeddingResult",
|
||||
"MockEmbedProvider",
|
||||
"OpenAIEmbedProvider",
|
||||
"UnknownEmbedProviderError",
|
||||
"embed_text",
|
||||
"embedding_provider_from_settings",
|
||||
"normalize_tokens",
|
||||
]
|
||||
@@ -0,0 +1,46 @@
|
||||
"""EmbeddingProvider protocol + shared embedding contracts (D-051, REQ-6-003).
|
||||
|
||||
Mirrors the D-030/D-014 provider seam: a narrow protocol the search
|
||||
pipeline and publish hooks compose via DI, with a deterministic
|
||||
feature-hash mock (pure stdlib) and an OpenAI-compatible real provider.
|
||||
No descriptor — unlike voice, embeddings are consumed SERVER-side only
|
||||
(match scores are computed in search.py, never shipped as capability
|
||||
hints), so there is nothing for a client to badge.
|
||||
|
||||
Boundary (D-048): `marketplace/embedding/` never imports `agents/` or
|
||||
`api/`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
|
||||
class EmbeddingResult(BaseModel):
|
||||
"""One embedded text (D-051): the vector + its provenance.
|
||||
|
||||
`model`/`dim` ride every result so the store can persist per-row
|
||||
provenance (A-403/A-404: lazy re-embed on provider change works).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
values: list[float] = Field(min_length=1)
|
||||
model: str = Field(min_length=1)
|
||||
dim: int = Field(ge=1)
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class EmbeddingProvider(Protocol):
|
||||
"""The embedding port (D-051): texts in, vectors out. Never imports
|
||||
agents/api; never stores anything (storage is the store's job)."""
|
||||
|
||||
async def embed(self, texts: list[str]) -> list[EmbeddingResult]:
|
||||
"""Embed a batch of texts; result[i] corresponds to texts[i].
|
||||
|
||||
Implementations MUST be deterministic per provider config so
|
||||
stored vectors stay comparable across re-embeds.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Embedding provider factory (D-051, REQ-6-003).
|
||||
|
||||
`AI_EMBED_PROVIDER = mock | openai-embed` (default: mock — the no-key
|
||||
path is first-class; tests never call a real embeddings API).
|
||||
`openai-embed` requires embed_base_url + embed_api_key + embed_model:
|
||||
the factory raises `UnknownEmbedProviderError` with an actionable
|
||||
message for direct callers (tests), while the lifespan in main.py
|
||||
CATCHES it and falls back to mock with a loud log — a typo'd env must
|
||||
never crash the unattended boot (G-11; the exact voice/factory.py
|
||||
pattern).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
|
||||
from ...config import Settings
|
||||
from .base import EmbeddingProvider
|
||||
from .mock import MockEmbedProvider
|
||||
from .openai_compat import OpenAIEmbedProvider
|
||||
|
||||
|
||||
class UnknownEmbedProviderError(ValueError):
|
||||
"""Raised for a provider name outside the contract, or a real provider
|
||||
selected without its required configuration."""
|
||||
|
||||
|
||||
def embedding_provider_from_settings(
|
||||
settings: Settings, http_client: httpx.AsyncClient | None = None
|
||||
) -> EmbeddingProvider:
|
||||
"""Select the embedding provider by settings (env `AI_EMBED_PROVIDER`).
|
||||
|
||||
`http_client` is required for the `openai-embed` branch (D-017 shared
|
||||
pool); mock ignores it.
|
||||
"""
|
||||
name = (settings.embed_provider or "mock").strip().lower()
|
||||
if name == "mock":
|
||||
return MockEmbedProvider()
|
||||
if name in ("openai-embed", "openai", "server"):
|
||||
if (
|
||||
not settings.embed_base_url
|
||||
or not settings.embed_api_key.get_secret_value()
|
||||
or not settings.embed_model
|
||||
):
|
||||
raise UnknownEmbedProviderError(
|
||||
"AI_EMBED_PROVIDER=openai-embed requires AI_EMBED_BASE_URL, "
|
||||
"AI_EMBED_API_KEY, and AI_EMBED_MODEL — set all three, or use "
|
||||
"'mock'. NOTE (D-051 probe): ollama-cloud does NOT expose "
|
||||
"/embeddings; point AI_EMBED_BASE_URL at another "
|
||||
"OpenAI-compatible endpoint (e.g. local ollama :11434/v1). "
|
||||
"(main.py falls back to mock when these are missing — G-11)"
|
||||
)
|
||||
if http_client is None:
|
||||
raise UnknownEmbedProviderError(
|
||||
"openai-embed requires the shared httpx client "
|
||||
"(embedding_provider_from_settings(settings, http_client))"
|
||||
)
|
||||
return OpenAIEmbedProvider(
|
||||
http_client=http_client,
|
||||
base_url=settings.embed_base_url,
|
||||
api_key=settings.embed_api_key.get_secret_value(),
|
||||
model=settings.embed_model,
|
||||
)
|
||||
raise UnknownEmbedProviderError(
|
||||
f"unknown AI_EMBED_PROVIDER {name!r}: use 'mock' or 'openai-embed'"
|
||||
)
|
||||
@@ -0,0 +1,102 @@
|
||||
"""Deterministic MockEmbedProvider (D-051, REQ-6-003).
|
||||
|
||||
256-dim feature-hash of normalized tokens — pure stdlib (hashlib, math;
|
||||
NO numpy, NO python hash() — the builtin is salted per process).
|
||||
Same text → identical vector, ALWAYS (across processes, reboots,
|
||||
machines). Related texts share tokens → higher cosine than unrelated
|
||||
texts, so pinned similarity numbers stay meaningful in tests.
|
||||
|
||||
Algorithm (documented — search.py's skill_breakdown depends on it):
|
||||
1. normalize: lowercase, split on non-alphanumeric ([^a-z0-9]+)
|
||||
2. every token hashes (SHA-256 of its UTF-8 bytes) to an index
|
||||
idx = int.from_bytes(digest[:4], "little") % 256
|
||||
3. sign alternation on token-position collisions: the k-th OCCURRENCE
|
||||
of a token at the same index flips sign — (+1, -1, +1, ...) — so
|
||||
repeated tokens cannot monotonically inflate one component
|
||||
(hash("ab") == hash("ba") collisions cancel, not stack).
|
||||
occurrence = the token's own prior count at that index.
|
||||
4. accumulate counts into the 256-dim vector
|
||||
5. L2-normalize (a zero vector — no tokens — stays all-zeros so the
|
||||
cosine guard in search.py can be exercised; honest, never a fabricated
|
||||
direction).
|
||||
|
||||
`embed` embeds the WHOLE batch in one call (no state between calls —
|
||||
per-text vectors are independent), returning EmbeddingResult rows with
|
||||
model="mock-feature-hash-256".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import math
|
||||
import re
|
||||
|
||||
from .base import EmbeddingProvider, EmbeddingResult
|
||||
|
||||
#: The mock's fixed dimensionality (D-051) — also its model identity.
|
||||
MOCK_EMBED_DIM = 256
|
||||
MOCK_EMBED_MODEL = "mock-feature-hash-256"
|
||||
|
||||
_SPLIT = re.compile(r"[^a-z0-9]+")
|
||||
|
||||
|
||||
def normalize_tokens(text: str) -> list[str]:
|
||||
"""lowercase + split on non-alphanumeric; empty pieces dropped."""
|
||||
return [tok for tok in _SPLIT.split(text.lower()) if tok]
|
||||
|
||||
|
||||
def token_index(token: str) -> int:
|
||||
"""Stable, process-independent index: first 4 SHA-256 digest bytes,
|
||||
little-endian, mod 256 (never the salted builtin hash())."""
|
||||
digest = hashlib.sha256(token.encode("utf-8")).digest()
|
||||
return int.from_bytes(digest[:4], "little") % MOCK_EMBED_DIM
|
||||
|
||||
|
||||
def embed_text(text: str) -> list[float]:
|
||||
"""The deterministic feature-hash vector for one text (pure function).
|
||||
|
||||
Exposed for search.py's per-skill contribution shares — the same
|
||||
formula must produce the mock's vectors and the breakdown math.
|
||||
"""
|
||||
vec = [0.0] * MOCK_EMBED_DIM
|
||||
for token in normalize_tokens(text):
|
||||
idx = token_index(token)
|
||||
# Sign alternation per token OCCURRENCE at its index: the token's
|
||||
# own running count at idx (not its position in the text).
|
||||
vec[idx] += 1.0 if vec[idx] >= 0 else -1.0 if vec[idx] <= 0 else 0.0
|
||||
# L2-normalize; all-zero (no tokens) stays all-zero (honest zero
|
||||
# vector — the cosine guard downstream treats it as unrankable).
|
||||
norm = math.sqrt(sum(v * v for v in vec))
|
||||
if norm > 0.0:
|
||||
vec = [v / norm for v in vec]
|
||||
return vec
|
||||
|
||||
|
||||
class MockEmbedProvider:
|
||||
"""Deterministic 256-dim feature-hash embedding (D-051 mock).
|
||||
|
||||
Cloud-free by construction; identical inputs → identical outputs
|
||||
forever (hashlib-based, no PYTHONHASHSEED dependence)."""
|
||||
|
||||
model = MOCK_EMBED_MODEL
|
||||
dim = MOCK_EMBED_DIM
|
||||
|
||||
async def embed(self, texts: list[str]) -> list[EmbeddingResult]:
|
||||
return [
|
||||
EmbeddingResult(
|
||||
values=embed_text(text),
|
||||
model=MOCK_EMBED_MODEL,
|
||||
dim=MOCK_EMBED_DIM,
|
||||
)
|
||||
for text in texts
|
||||
]
|
||||
|
||||
|
||||
# Protocol-shape parity guard (mock must satisfy the D-051 port).
|
||||
|
||||
|
||||
def _assert_protocol() -> None:
|
||||
assert isinstance(MockEmbedProvider(), EmbeddingProvider)
|
||||
|
||||
|
||||
_assert_protocol()
|
||||
@@ -0,0 +1,94 @@
|
||||
"""OpenAI-compatible embedding provider (D-051, REQ-6-003).
|
||||
|
||||
One implementation serves any OpenAI-compatible embeddings endpoint:
|
||||
`POST {base_url}/embeddings` with `{"model", "input": [texts]}` →
|
||||
`{"data": [{"embedding": [...]}, ...], "model": ...}`. Raw httpx on the
|
||||
shared lifespan client (D-017). base_url is config, never code — the
|
||||
D-014 endpoint-agnostic pattern.
|
||||
|
||||
PROBE CORRECTION (D-051): ollama-cloud does NOT expose /embeddings
|
||||
(404 path-not-found) — AI_EMBED_BASE_URL must point at another
|
||||
OpenAI-compatible endpoint (e.g. LOCAL ollama at :11434/v1, which does).
|
||||
|
||||
Boundary rules (mirror voice/openai_audio.py):
|
||||
- embedding/ imports nothing from agents/ or api/
|
||||
- api_key NEVER appears in exceptions, logs, or error messages
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
|
||||
from .base import EmbeddingResult
|
||||
|
||||
|
||||
class OpenAIEmbedProvider:
|
||||
"""POST {base_url}/embeddings {"model", "input": [texts]}."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
http_client: httpx.AsyncClient,
|
||||
base_url: str,
|
||||
api_key: str,
|
||||
model: str = "",
|
||||
) -> None:
|
||||
self._client = http_client
|
||||
self._base_url = base_url.rstrip("/")
|
||||
self._api_key = api_key
|
||||
self._model = model
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
headers: dict[str, str] = {"Content-Type": "application/json"}
|
||||
if self._api_key:
|
||||
headers["Authorization"] = f"Bearer {self._api_key}"
|
||||
return headers
|
||||
|
||||
def _sanitize(self, exc: Exception) -> RuntimeError:
|
||||
text = str(exc)
|
||||
if self._api_key and self._api_key in text:
|
||||
text = text.replace(self._api_key, "[REDACTED]")
|
||||
return RuntimeError(f"embedding provider error: {text}")
|
||||
|
||||
async def embed(self, texts: list[str]) -> list[EmbeddingResult]:
|
||||
"""Batch-embed texts; result[i] corresponds to texts[i].
|
||||
|
||||
Parses the OpenAI-format response: `{"data": [{"embedding":
|
||||
[...]}], "model": ...}` — index-order preserved (the spec orders
|
||||
data by input position).
|
||||
"""
|
||||
payload = {"model": self._model, "input": list(texts)}
|
||||
try:
|
||||
resp = await self._client.post(
|
||||
f"{self._base_url}/embeddings",
|
||||
json=payload,
|
||||
headers=self._headers(),
|
||||
)
|
||||
resp.raise_for_status()
|
||||
body = resp.json()
|
||||
except httpx.HTTPError as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
if not isinstance(body, dict):
|
||||
raise RuntimeError(
|
||||
"embedding provider error: unexpected embeddings response shape"
|
||||
)
|
||||
data = body.get("data")
|
||||
if not isinstance(data, list) or len(data) != len(texts):
|
||||
raise RuntimeError(
|
||||
"embedding provider error: embeddings response/data length "
|
||||
f"mismatch (expected {len(texts)} entries)"
|
||||
)
|
||||
model = str(body.get("model") or self._model)
|
||||
results: list[EmbeddingResult] = []
|
||||
for entry in data:
|
||||
if not isinstance(entry, dict) or not isinstance(
|
||||
entry.get("embedding"), list
|
||||
):
|
||||
raise RuntimeError(
|
||||
"embedding provider error: embeddings response/data entry "
|
||||
"missing the 'embedding' array"
|
||||
)
|
||||
values = [float(v) for v in entry["embedding"]]
|
||||
results.append(
|
||||
EmbeddingResult(values=values, model=model, dim=len(values))
|
||||
)
|
||||
return results
|
||||
@@ -0,0 +1,215 @@
|
||||
"""Employer enrichment (REQ-6-004, D-053, A-409).
|
||||
|
||||
One-shot LLM extraction of employer description/social-links/logo from
|
||||
the employer's own posting data, composed through the D-020 pipeline
|
||||
(the D-052 tagging pattern) — no new agent. Enrichment rows are
|
||||
APPEND-ONLY with per-field provenance; merge-on-read precedence is
|
||||
manual > llm > ingested-raw, and every enriched field carries a
|
||||
visible {value, source} marker in API responses (A-304 honesty —
|
||||
LLM-authored copy is never displayed as ingested fact).
|
||||
|
||||
Trigger semantics (enrich-on-first-publish approximation): the publish
|
||||
route fires enrichment when the employer has NO existing llm-source
|
||||
rows — re-publishing the same employer does not re-run (the rows
|
||||
exist). Failure is honest: a warning logs, the publish still succeeds,
|
||||
no rows are written.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from ..agents.structured import structured_completion
|
||||
from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
from .base import Employer
|
||||
from .store import MarketplaceStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Prompt version — bump on any wording change (prompts are code).
|
||||
ENRICHMENT_PROMPT_VERSION = 1
|
||||
|
||||
ENRICHMENT_PROMPT = """You are an employer-profile writer for an AI-era
|
||||
roles marketplace. From the employer's own data below, write concise
|
||||
factual profile copy.
|
||||
|
||||
Rules:
|
||||
- description: 1-3 sentences summarizing what the company does, grounded
|
||||
ONLY in the provided data — never invent employers, products, or
|
||||
numbers.
|
||||
- social_links: an object of platform → URL, ONLY for platforms the
|
||||
data names (website, github, linkedin, twitter/x). Empty object if
|
||||
none.
|
||||
- logo_ref: a short reference to the logo source named in the data
|
||||
(e.g. a URL or file name); empty string if none.
|
||||
|
||||
Respond with ONLY the JSON object."""
|
||||
|
||||
_SCHEMA_HINT = (
|
||||
'{"description": "1-3 sentence string", '
|
||||
'"social_links": {"platform": "url"}, '
|
||||
'"logo_ref": "string"}'
|
||||
)
|
||||
|
||||
#: The enrichable field set (D-053) — the manual-override route validates
|
||||
#: against this and the merge walks exactly these keys.
|
||||
ENRICHABLE_FIELDS = ("description", "social_links", "logo")
|
||||
|
||||
_PRECEDENCE = ("manual", "llm")
|
||||
|
||||
|
||||
class EnrichmentUnavailable(Exception):
|
||||
"""The extractor could not produce valid enrichment (provider/parse
|
||||
failure). The publish proceeds; no rows are written (D-053 honesty)."""
|
||||
|
||||
|
||||
class EnrichmentPayload(BaseModel):
|
||||
"""The D-020 extraction schema. extra=forbid keeps the honesty
|
||||
contract (the verifier P2-3 lesson): a non-matching LLM reply must
|
||||
fail validation, not silently parse as all-defaults."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
description: str = ""
|
||||
social_links: dict[str, str] = {}
|
||||
logo_ref: str = ""
|
||||
|
||||
async def enrich_employer(
|
||||
provider: LLMProvider,
|
||||
*,
|
||||
model: str,
|
||||
employer: Employer,
|
||||
postings_hint: list[str] | None = None,
|
||||
) -> EnrichmentPayload:
|
||||
"""One-shot profile extraction via D-020 structured completion.
|
||||
|
||||
`postings_hint` (the employer's posting titles) grounds the copy in
|
||||
the employer's own data. Failures raise EnrichmentUnavailable —
|
||||
honest, never a crash, never fabricated copy."""
|
||||
hint = ""
|
||||
if postings_hint:
|
||||
hint = "\nPosting titles: " + "; ".join(postings_hint[:10])
|
||||
messages = [
|
||||
Message(role="system", content=ENRICHMENT_PROMPT),
|
||||
Message(
|
||||
role="user",
|
||||
content=(
|
||||
f"Employer name: {employer.name}\n"
|
||||
f"Industry: {employer.industry}\n"
|
||||
f"Known description: {employer.description}\n"
|
||||
f"Website: {employer.website}\n"
|
||||
f"Social links: {json.dumps(employer.social_links)}\n"
|
||||
f"Logo: {employer.logo}{hint}"
|
||||
),
|
||||
),
|
||||
]
|
||||
try:
|
||||
return await structured_completion(
|
||||
provider,
|
||||
messages,
|
||||
model=model,
|
||||
schema=EnrichmentPayload,
|
||||
schema_hint=_SCHEMA_HINT,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"marketplace enrichment failed for %r: %s — no rows written "
|
||||
"(honest absence, never fabricated copy)",
|
||||
employer.name[:60],
|
||||
exc,
|
||||
)
|
||||
raise EnrichmentUnavailable(str(exc)) from exc
|
||||
|
||||
|
||||
def merge_employer(
|
||||
employer: Employer, rows: list[dict]
|
||||
) -> dict[str, dict]:
|
||||
"""Merge-on-read with per-field provenance (D-053/A-409).
|
||||
|
||||
Precedence per FIELD: manual > llm > ingested-raw — a manual
|
||||
description + llm social_links is legal. Every field carries
|
||||
{value, source, model?}; `social_links` values are dicts (the row
|
||||
stores JSON), the others strings. Newest row per (field, source)
|
||||
wins within a source (append-only, newest-last).
|
||||
"""
|
||||
best: dict[str, dict] = {}
|
||||
for row in rows:
|
||||
field = row.get("field")
|
||||
if field not in ENRICHABLE_FIELDS:
|
||||
continue
|
||||
source = row.get("source")
|
||||
rank = _PRECEDENCE.index(source) if source in _PRECEDENCE else len(_PRECEDENCE)
|
||||
existing = best.get(field)
|
||||
if existing is None or rank < existing["_rank"]:
|
||||
best[field] = {
|
||||
"_rank": rank,
|
||||
"value": row.get("value"),
|
||||
"source": source,
|
||||
"model": row.get("model") or None,
|
||||
}
|
||||
merged: dict[str, dict] = {}
|
||||
for field in ENRICHABLE_FIELDS:
|
||||
raw_value = {
|
||||
"description": employer.description,
|
||||
"social_links": dict(employer.social_links),
|
||||
"logo": employer.logo,
|
||||
}[field]
|
||||
if field in best:
|
||||
entry = best[field]
|
||||
merged[field] = {
|
||||
"value": entry["value"],
|
||||
"source": entry["source"],
|
||||
"model": entry["model"],
|
||||
}
|
||||
else:
|
||||
# No enrichment rows for this field — the ingested raw value
|
||||
# with the honest raw marker (A-304: LLM copy never
|
||||
# masquerades as ingested fact, and vice versa).
|
||||
merged[field] = {"value": raw_value, "source": "raw", "model": None}
|
||||
return merged
|
||||
|
||||
|
||||
async def maybe_enrich_on_publish(
|
||||
store: MarketplaceStore,
|
||||
provider: LLMProvider,
|
||||
*,
|
||||
model: str,
|
||||
employer_id: str,
|
||||
) -> bool:
|
||||
"""Enrich-on-first-publish trigger: fires when the employer has NO
|
||||
existing llm-source rows (re-publishing does not re-run — the rows
|
||||
exist). Returns True when enrichment rows landed. Failures are
|
||||
honest: warning + False, the publish itself already succeeded."""
|
||||
if store.count_enrichment(employer_id, source="llm") > 0:
|
||||
return False
|
||||
employer = store.get_employer(employer_id)
|
||||
if employer is None:
|
||||
return False # dangling employer ref (the seed carries 3) — skip
|
||||
titles = store.employer_posting_titles(employer_id)
|
||||
try:
|
||||
payload = await enrich_employer(
|
||||
provider, model=model, employer=employer, postings_hint=titles
|
||||
)
|
||||
except EnrichmentUnavailable:
|
||||
return False
|
||||
stamp = {"model": model}
|
||||
rows = [
|
||||
("description", payload.description, "llm", stamp),
|
||||
(
|
||||
"social_links",
|
||||
json.dumps(payload.social_links),
|
||||
"llm",
|
||||
stamp,
|
||||
),
|
||||
("logo", payload.logo_ref, "llm", stamp),
|
||||
]
|
||||
for field, value, source, meta in rows:
|
||||
if value not in ("", "{}"):
|
||||
store.append_enrichment(
|
||||
employer_id, field=field, value=value, source=source, **meta
|
||||
)
|
||||
return True
|
||||
@@ -0,0 +1,97 @@
|
||||
"""PostingSource adapters + ingest runner (REQ-6-001, D-050, A-402).
|
||||
|
||||
`PostingSource` is the adapter port: `fetch() -> Iterable[IngestPosting]`.
|
||||
v0.6 ships ONE implementation, `SeededCorpusSource` over the in-repo
|
||||
`seed_corpus.json` (ported ONCE from packages/mock-data/jobs.ts +
|
||||
employers.ts per D-021 — shared ids, mock matchScore DROPPED: D-051
|
||||
computes real scores at search time, G-106/G-107). The 3M+ external
|
||||
corpus lands later as a third adapter with zero store/route changes.
|
||||
|
||||
Ingest-status contract (G-102): SEED-PORT postings enter as `published`
|
||||
with their ORIGINAL postedAt — otherwise the board/facets/P1's success
|
||||
criterion sit empty after ingest. ADMIN-AUTHORED postings start
|
||||
`draft` (the admin route's create path, not this runner).
|
||||
|
||||
No scheduler/crawler (A-401/A-402): ingest runs when the founder POSTs
|
||||
/v1/marketplace/admin/ingest; `run_ingest` is idempotent on
|
||||
(source, external_id) — a double-run yields identical row counts.
|
||||
|
||||
Boundary (D-048): never imports `agents/` or `api/`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
from .base import Employer, IngestPosting, IngestResult
|
||||
from .store import MarketplaceStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: The ported TS fixture corpus ships IN THE PACKAGE (importlib-free
|
||||
#: path resolution: this file's parent) — one JSON file, no codegen.
|
||||
_SEED_PATH = Path(__file__).resolve().parent / "seed_corpus.json"
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class PostingSource(Protocol):
|
||||
"""The ingest port (D-050): yields IngestPosting payloads. Employers
|
||||
ride the corpus alongside postings (the seeded source exposes both;
|
||||
a pure-posting adapter simply fetches postings and leaves employers
|
||||
to a separate source)."""
|
||||
|
||||
def fetch(self) -> Iterable[IngestPosting]:
|
||||
...
|
||||
|
||||
def fetch_employers(self) -> Iterable[Employer]:
|
||||
...
|
||||
|
||||
|
||||
class SeededCorpusSource:
|
||||
"""The seeded corpus adapter over seed_corpus.json (D-050).
|
||||
|
||||
Pure + deterministic: reads the JSON fixture once per fetch; the
|
||||
runner upserts idempotently. Raw fields map 1:1 from the TS shapes
|
||||
(camelCase → snake_case port, matchScore dropped — G-102/G-107)."""
|
||||
|
||||
def __init__(self, path: Path = _SEED_PATH) -> None:
|
||||
self._path = path
|
||||
|
||||
def _load(self) -> dict:
|
||||
with self._path.open(encoding="utf-8") as fh:
|
||||
return json.load(fh)
|
||||
|
||||
def fetch(self) -> Iterable[IngestPosting]:
|
||||
for raw in self._load().get("postings", []):
|
||||
yield IngestPosting.model_validate(raw)
|
||||
|
||||
def fetch_employers(self) -> Iterable[Employer]:
|
||||
for raw in self._load().get("employers", []):
|
||||
yield Employer.model_validate(raw)
|
||||
|
||||
|
||||
def run_ingest(source: PostingSource, store: MarketplaceStore) -> IngestResult:
|
||||
"""Idempotent upsert run: every (source, external_id) overwrites its
|
||||
row in place — a second run yields identical counts (MH-1b)."""
|
||||
|
||||
result = IngestResult()
|
||||
# Employers first: postings reference employer ids (no FK today, but
|
||||
# the reference ordering keeps the board honest mid-run).
|
||||
for employer in source.fetch_employers():
|
||||
store.upsert_employer(employer)
|
||||
result.employers_upserted += 1
|
||||
for posting in source.fetch():
|
||||
# G-102: seed-port postings enter PUBLISHED with their original
|
||||
# postedAt (the IngestPosting carries it verbatim).
|
||||
store.upsert_posting(posting, status="published")
|
||||
result.postings_upserted += 1
|
||||
logger.info(
|
||||
"marketplace ingest: %s postings, %s employers (idempotent upserts)",
|
||||
result.postings_upserted,
|
||||
result.employers_upserted,
|
||||
)
|
||||
return result
|
||||
@@ -0,0 +1,155 @@
|
||||
"""Resume → profile draft extraction (REQ-6-005, D-054, A-410).
|
||||
|
||||
Pipeline: upload guard (PDF/TXT/MD only, ≤2 MB, text-layer required —
|
||||
empty pypdf extract → honest 422, NO OCR in v0.6) → text extraction →
|
||||
~100K char cap → D-020 structured extraction into ParsedProfile →
|
||||
response carries status="draft" — NOTHING persists to the learner
|
||||
profile until the explicit confirm route (the REQ-6-005 review gate).
|
||||
|
||||
Raw resume bytes are never stored, never logged (A-305 pattern; the
|
||||
existing _scrub_pii_validation handler precedent). The only new
|
||||
dependency is pypdf (probe-verified 6.19.0, pip-reachable via the
|
||||
venv channel).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from ..agents.structured import structured_completion
|
||||
from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Prompt version — bump on any wording change (prompts are code).
|
||||
RESUME_PROMPT_VERSION = 1
|
||||
|
||||
#: Upload bounds (A-410).
|
||||
MAX_RESUME_BYTES = 2 * 1024 * 1024 # 2 MB
|
||||
ALLOWED_CONTENT_TYPES = ("pdf", "txt", "md")
|
||||
#: Extracted-text cap before the LLM call (A-410: ~100K chars).
|
||||
MAX_EXTRACTED_CHARS = 100_000
|
||||
|
||||
RESUME_PROMPT = """You are a profile parser for an AI-era skills
|
||||
platform. Extract structured profile data from the resume text.
|
||||
|
||||
Rules:
|
||||
- name: the person's name as written, or "" if absent.
|
||||
- headline: a one-line professional headline, or "".
|
||||
- skills: up to 20 concrete skills named in the resume (lowercase,
|
||||
the resume's own vocabulary preferred).
|
||||
- experience_bands: list of {role, years} objects — years is a number
|
||||
(round down; 0 if under a year); [] if none determinable.
|
||||
- competencies: up to 10 competency areas the resume evidences, each a
|
||||
short noun phrase; [] if none.
|
||||
|
||||
Respond with ONLY the JSON object."""
|
||||
|
||||
_SCHEMA_HINT = (
|
||||
'{"name": "string", "headline": "string", '
|
||||
'"skills": ["up to 20 lowercase strings"], '
|
||||
'"experience_bands": [{"role": "string", "years": 0}], '
|
||||
'"competencies": ["up to 10 short phrases"]}'
|
||||
)
|
||||
|
||||
|
||||
class ExperienceBand(BaseModel):
|
||||
"""One role/tenure band the resume evidences."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
role: str = ""
|
||||
years: float = 0
|
||||
|
||||
|
||||
class ParsedProfile(BaseModel):
|
||||
"""The D-020 extraction schema mapped to the competency-profile
|
||||
shape (D-054). extra=forbid keeps the honesty contract — a
|
||||
non-matching reply fails validation, never silently all-defaults
|
||||
(the verifier P2-3 lesson)."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
name: str = ""
|
||||
headline: str = ""
|
||||
skills: list[str] = Field(default_factory=list, max_length=20)
|
||||
experience_bands: list[ExperienceBand] = Field(default_factory=list)
|
||||
competencies: list[str] = Field(default_factory=list, max_length=10)
|
||||
|
||||
|
||||
class ResumeFormatError(Exception):
|
||||
"""Honest upload rejection (422 at the route): unsupported format,
|
||||
missing text layer (empty PDF extract — no OCR in v0.6), or empty
|
||||
file."""
|
||||
|
||||
|
||||
def extract_resume_text(data: bytes, content_type: str) -> str:
|
||||
"""Extract text from an uploaded resume (A-410 bounds enforced by
|
||||
the route BEFORE this call; this handles the decode/parse).
|
||||
|
||||
PDF via pypdf page-text join; TXT/MD raw decode. An empty PDF
|
||||
extract raises ResumeFormatError("text layer required") — the
|
||||
honest no-OCR contract. Raises ResumeFormatError on garbage text
|
||||
bytes too (the honest 422, never a 500)."""
|
||||
ctype = content_type.strip().lower()
|
||||
# Accept both bare and full content types ("application/pdf").
|
||||
if "pdf" in ctype or ctype == "pdf":
|
||||
from io import BytesIO
|
||||
|
||||
from pypdf import PdfReader
|
||||
|
||||
try:
|
||||
reader = PdfReader(BytesIO(data))
|
||||
pages = [(page.extract_text() or "") for page in reader.pages]
|
||||
except Exception as exc:
|
||||
raise ResumeFormatError(
|
||||
"could not read the PDF — is it a valid PDF file?"
|
||||
) from exc
|
||||
text = "\n".join(pages)
|
||||
if not text.strip():
|
||||
raise ResumeFormatError(
|
||||
"text layer required — image-only/scanned PDFs are not "
|
||||
"supported (no OCR in v0.6); upload a text-based PDF, "
|
||||
"TXT, or MD file"
|
||||
)
|
||||
return text
|
||||
if "markdown" in ctype or ctype in ("txt", "md", "text/plain", "text/markdown"):
|
||||
try:
|
||||
text = data.decode("utf-8")
|
||||
except UnicodeDecodeError as exc:
|
||||
raise ResumeFormatError(
|
||||
"could not decode the file as UTF-8 text"
|
||||
) from exc
|
||||
if not text.strip():
|
||||
raise ResumeFormatError("file has no text content")
|
||||
return text
|
||||
raise ResumeFormatError(
|
||||
f"unsupported resume format {content_type!r} — PDF, TXT, or MD only"
|
||||
)
|
||||
|
||||
|
||||
async def parse_resume(
|
||||
provider: LLMProvider,
|
||||
*,
|
||||
model: str,
|
||||
text: str,
|
||||
) -> ParsedProfile:
|
||||
"""Draft extraction via D-020 (never persists — the confirm route
|
||||
is the only write path, D-054/REQ-6-005 review gate)."""
|
||||
capped = text[:MAX_EXTRACTED_CHARS]
|
||||
messages = [
|
||||
Message(role="system", content=RESUME_PROMPT),
|
||||
Message(role="user", content=f"Resume text:\n{capped}"),
|
||||
]
|
||||
# Raw resume text never logs (A-305): only the failure reason rides
|
||||
# the warning below — structured_completion raises on its own path.
|
||||
return await structured_completion(
|
||||
provider,
|
||||
messages,
|
||||
model=model,
|
||||
schema=ParsedProfile,
|
||||
schema_hint=_SCHEMA_HINT,
|
||||
)
|
||||
@@ -0,0 +1,250 @@
|
||||
"""Hybrid facet+cosine search over published postings (REQ-6-003, A-405).
|
||||
|
||||
Pure-Python cosine (sum of products / norm product; ≤10⁴ postings —
|
||||
NO numpy, A-403). Hybrid ordering per D-059: facet hard-filters
|
||||
SQL-side on portable columns FIRST, then cosine ranking over the
|
||||
filtered set; no facets → pure semantic rank over all published
|
||||
postings.
|
||||
|
||||
Candidate side: the query text embeds ON DEMAND via the provider and
|
||||
is NEVER stored (A-404 — only posting vectors persist).
|
||||
|
||||
Lazy re-embed (A-403/A-404): a stored row whose model/dim ≠ the
|
||||
provider's current model/dim is re-embedded + re-stored on access, so
|
||||
a provider switch self-heals the corpus row-by-row without a
|
||||
backfill pass.
|
||||
|
||||
skill_breakdown formula (deterministic, documented for the pin):
|
||||
- overlap = does the skill string (normalized: lowercase, split on
|
||||
non-alphanumeric, rejoined) share a token with the QUERY's normalized
|
||||
token set → overlap ∈ {0, 1}
|
||||
- contribution = the share of the posting-vector dot-product carried by
|
||||
the skill's own feature-hash tokens: for the mock provider,
|
||||
contribution(skill) = Σ_components of mock.embed_text(skill) mapped
|
||||
to the same (index, sign) bucket in the mock formula — computed as
|
||||
cosine(embed_text(skill), posting_vector) when overlap=1, else 0.0.
|
||||
Both operands are mock-formula vectors, so the number is
|
||||
deterministic and provider-independent to pin (with the openai-embed
|
||||
provider the contribution uses the SAME skill-text side via the
|
||||
provider on demand; the mock pin is the MH-3f pin).
|
||||
|
||||
Boundary (D-048): never imports `agents/` or `api/`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import math
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from .base import Posting
|
||||
from .embedding.base import EmbeddingProvider, EmbeddingResult
|
||||
from .embedding.mock import embed_text as _mock_embed_text
|
||||
from .embedding.mock import normalize_tokens as _normalize
|
||||
from .store import MarketplaceStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FacetFilters:
|
||||
"""Hard filters applied SQL-side BEFORE ranking (A-405). Any facet
|
||||
may be empty — an empty filter set means pure semantic rank."""
|
||||
|
||||
skills: list[str] = field(default_factory=list)
|
||||
seniority: str | None = None
|
||||
category: str | None = None
|
||||
location: str | None = None
|
||||
remote: bool | None = None
|
||||
salary_min: int | None = None
|
||||
salary_max: int | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RankedPosting:
|
||||
"""One ranked search result. `match_score` is the computed cosine
|
||||
(G-106: computed scores exist ONLY in search context)."""
|
||||
|
||||
posting: Posting
|
||||
score: float
|
||||
skill_breakdown: list[dict]
|
||||
|
||||
|
||||
def cosine_similarity(a: list[float], b: list[float]) -> float:
|
||||
"""Pure-Python cosine: sum(a·b) / (‖a‖·‖b‖); 0.0 on a zero vector
|
||||
(guard: an unrankable vector scores an honest zero, never a
|
||||
ZeroDivisionError) or a length mismatch (defensive — the store
|
||||
codec enforces dim parity)."""
|
||||
if not a or not b or len(a) != len(b):
|
||||
return 0.0
|
||||
dot = 0.0
|
||||
norm_a = 0.0
|
||||
norm_b = 0.0
|
||||
for x, y in zip(a, b, strict=False):
|
||||
dot += x * y
|
||||
norm_a += x * x
|
||||
norm_b += y * y
|
||||
if norm_a <= 0.0 or norm_b <= 0.0:
|
||||
return 0.0
|
||||
return dot / math.sqrt(norm_a * norm_b)
|
||||
|
||||
|
||||
def posting_embedding_text(posting: Posting) -> str:
|
||||
"""D-051 posting text: title + description + tag skills joined —
|
||||
the P2 tags ride the vector so facets and semantics agree."""
|
||||
parts = [posting.title, posting.description or ""]
|
||||
if posting.tags and posting.tags.skills:
|
||||
parts.extend(posting.tags.skills)
|
||||
return "\n".join(p for p in parts if p)
|
||||
|
||||
|
||||
async def embed_posting(
|
||||
store: MarketplaceStore, provider: EmbeddingProvider, posting: Posting
|
||||
) -> bool:
|
||||
"""Embed-on-publish hook (Task 3-2-01): embed the posting text and
|
||||
store the vector. Failure is HONEST, never fatal: returns False with
|
||||
a logged warning — the posting still publishes, search skips it
|
||||
(no stored vector = honest absence, never a fabricated direction)."""
|
||||
try:
|
||||
results = await provider.embed([posting_embedding_text(posting)])
|
||||
result = results[0]
|
||||
store.put_vector(posting.id, result.values, result.model, result.dim)
|
||||
return True
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"marketplace embedding failed for posting %r: %s — posting stays "
|
||||
"published WITHOUT a vector (search skips it, honest absence)",
|
||||
posting.id,
|
||||
exc,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
async def _current_embedding(
|
||||
provider: EmbeddingProvider, text: str
|
||||
) -> EmbeddingResult:
|
||||
"""Embed one text via the provider (candidate side: on demand,
|
||||
never stored)."""
|
||||
results = await provider.embed([text])
|
||||
return results[0]
|
||||
|
||||
|
||||
async def rank_postings(
|
||||
store: MarketplaceStore,
|
||||
provider: EmbeddingProvider,
|
||||
query_text: str,
|
||||
facet_filters: FacetFilters | None = None,
|
||||
limit: int = 20,
|
||||
) -> list[RankedPosting]:
|
||||
"""Hybrid rank (A-405): facets hard-filter SQL-side first, then
|
||||
cosine ranking DESC over the filtered set.
|
||||
|
||||
- empty query → facet-only listing ordered by posted_at DESC (no
|
||||
query → no match scores — G-106 honesty; scores are 0.0 and the
|
||||
route reports match_score: null).
|
||||
- postings whose stored model/dim ≠ the provider's are re-embedded
|
||||
+ re-stored lazily (A-403/A-404).
|
||||
- postings with NO stored vector are SKIPPED (honest absence — an
|
||||
embed-on-publish failure must not fabricate a direction).
|
||||
"""
|
||||
filters = facet_filters or FacetFilters()
|
||||
candidates = store.search_candidates(
|
||||
skills=filters.skills or None,
|
||||
seniority=filters.seniority,
|
||||
category=filters.category,
|
||||
location=filters.location,
|
||||
remote=filters.remote,
|
||||
salary_min=filters.salary_min,
|
||||
salary_max=filters.salary_max,
|
||||
)
|
||||
if not query_text.strip():
|
||||
# Facet-only listing: posted_at DESC (the list_postings order),
|
||||
# no match scores — the route carries match_score: null.
|
||||
return [
|
||||
RankedPosting(posting=p, score=0.0, skill_breakdown=[])
|
||||
for p in candidates[:limit]
|
||||
]
|
||||
|
||||
query_result = await _current_embedding(provider, query_text)
|
||||
q_values = query_result.values
|
||||
q_tokens = set(_normalize(query_text))
|
||||
q_model = query_result.model
|
||||
q_dim = query_result.dim
|
||||
|
||||
# Lazy re-embed: collect stale rows (model/dim ≠ provider's) and
|
||||
# refresh them in one batch before ranking.
|
||||
stale: list[Posting] = [
|
||||
p for p in candidates if _is_stale(store, p.id, q_model, q_dim)
|
||||
]
|
||||
if stale:
|
||||
texts = [posting_embedding_text(p) for p in stale]
|
||||
try:
|
||||
refreshed = await provider.embed(texts)
|
||||
for posting, result in zip(stale, refreshed, strict=True):
|
||||
store.put_vector(posting.id, result.values, result.model, result.dim)
|
||||
except Exception as exc: # honest: stale rows keep old vectors
|
||||
logger.warning(
|
||||
"lazy re-embed of %s postings failed (%s) — ranking against "
|
||||
"existing vectors",
|
||||
len(stale),
|
||||
exc,
|
||||
)
|
||||
|
||||
ranked: list[RankedPosting] = []
|
||||
for posting in candidates:
|
||||
stored = store.get_vector(posting.id)
|
||||
if stored is None:
|
||||
continue # embed-on-publish failed → honest skip
|
||||
values, model, dim = stored
|
||||
if (model, dim) != (q_model, q_dim):
|
||||
continue # re-embed failed above → incomparable, honest skip
|
||||
score = cosine_similarity(q_values, values)
|
||||
ranked.append(
|
||||
RankedPosting(
|
||||
posting=posting,
|
||||
score=score,
|
||||
skill_breakdown=skill_breakdown(posting, q_tokens, values),
|
||||
)
|
||||
)
|
||||
ranked.sort(key=lambda r: (-r.score, r.posting.id))
|
||||
return ranked[:limit]
|
||||
|
||||
|
||||
def _is_stale(
|
||||
store: MarketplaceStore, posting_id: str, model: str, dim: int
|
||||
) -> bool:
|
||||
stored = store.get_vector(posting_id)
|
||||
if stored is None:
|
||||
return False # absent ≠ stale (no direction to refresh)
|
||||
_, row_model, row_dim = stored
|
||||
return (row_model, row_dim) != (model, dim)
|
||||
|
||||
|
||||
def skill_breakdown(
|
||||
posting: Posting, query_tokens: set[str], posting_vector: list[float]
|
||||
) -> list[dict]:
|
||||
"""Per-skill explanation (MH-3f): tag-skill overlap with the query
|
||||
+ the skill's vector contribution — both deterministic.
|
||||
|
||||
overlap: 1 when any normalized token of the skill string appears
|
||||
in the query's normalized token set, else 0.
|
||||
contribution: cosine(embed_text(skill), posting_vector) when
|
||||
overlap=1 — the mock feature-hash formula's share of the posting
|
||||
dot-product that the skill's own tokens carry; 0.0 otherwise.
|
||||
Sorted by contribution DESC then skill ASC (deterministic order).
|
||||
"""
|
||||
skills: list[str] = list(posting.tags.skills) if posting.tags else []
|
||||
breakdown: list[dict] = []
|
||||
for skill in skills:
|
||||
tokens = _normalize(skill)
|
||||
overlap = 1 if any(t in query_tokens for t in tokens) else 0
|
||||
contribution = 0.0
|
||||
if overlap:
|
||||
contribution = cosine_similarity(
|
||||
_mock_embed_text(skill), posting_vector
|
||||
)
|
||||
breakdown.append(
|
||||
{"skill": skill, "overlap": overlap, "contribution": contribution}
|
||||
)
|
||||
breakdown.sort(key=lambda e: (-e["contribution"], e["skill"]))
|
||||
return breakdown
|
||||
@@ -0,0 +1,695 @@
|
||||
{
|
||||
"employers": [
|
||||
{
|
||||
"id": "emp-openai",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-openai",
|
||||
"name": "OpenAI Labs",
|
||||
"logo": "https://placehold.co/80x80/indigo/white?text=OA",
|
||||
"description": "Mock employer building frontier AI systems. We build and deploy large language models and agentic tools used by millions of developers and consumers.",
|
||||
"industry": "Artificial Intelligence",
|
||||
"size": "2,000+",
|
||||
"location": "San Francisco, CA",
|
||||
"website": "https://example.com/openai",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/openai/x",
|
||||
"linkedin": "https://example.com/openai/li"
|
||||
},
|
||||
"culture": "Research-driven, shipping-paced, safety-conscious. We pair frontier research with product rigor."
|
||||
},
|
||||
{
|
||||
"id": "emp-anthropic",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-anthropic",
|
||||
"name": "Anthropic Research",
|
||||
"logo": "https://placehold.co/80x80/amber/white?text=AN",
|
||||
"description": "Mock employer focused on AI safety and alignment. We build reliable, interpretable, and steerable AI systems.",
|
||||
"industry": "AI Safety",
|
||||
"size": "1,000+",
|
||||
"location": "San Francisco, CA",
|
||||
"website": "https://example.com/anthropic",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/anthropic/x",
|
||||
"linkedin": "https://example.com/anthropic/li"
|
||||
},
|
||||
"culture": "Safety-first, evidence-based, calm-paced. We value thoroughness over speed when stakes are high."
|
||||
},
|
||||
{
|
||||
"id": "emp-huggingface",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-huggingface",
|
||||
"name": "Hugging Face",
|
||||
"logo": "https://placehold.co/80x80/yellow/black?text=HF",
|
||||
"description": "Mock employer building the open-source AI community. We host models, datasets, and demos for the global ML community.",
|
||||
"industry": "Open Source AI",
|
||||
"size": "500+",
|
||||
"location": "New York, NY",
|
||||
"website": "https://example.com/hf",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/hf/x",
|
||||
"linkedin": "https://example.com/hf/li"
|
||||
},
|
||||
"culture": "Community-first, open-by-default, remote-friendly. We ship in the open with thousands of contributors."
|
||||
},
|
||||
{
|
||||
"id": "emp-scaleai",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-scaleai",
|
||||
"name": "Scale AI",
|
||||
"logo": "https://placehold.co/80x80/violet/white?text=SC",
|
||||
"description": "Mock employer providing data and evaluation infrastructure for frontier AI. We power the RLHF and eval pipelines behind many model releases.",
|
||||
"industry": "AI Infrastructure",
|
||||
"size": "1,500+",
|
||||
"location": "San Francisco, CA",
|
||||
"website": "https://example.com/scale",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/scale/x",
|
||||
"linkedin": "https://example.com/scale/li"
|
||||
},
|
||||
"culture": "Infrastructure-minded, quality-obsessed, enterprise-aware. We bridge research and production data."
|
||||
},
|
||||
{
|
||||
"id": "emp-perplexity",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-perplexity",
|
||||
"name": "Perplexity",
|
||||
"logo": "https://placehold.co/80x80/teal/white?text=PX",
|
||||
"description": "Mock employer building an AI-powered answer engine. We combine retrieval, generation, and citations for trustworthy answers.",
|
||||
"industry": "AI Search",
|
||||
"size": "300+",
|
||||
"location": "San Francisco, CA",
|
||||
"website": "https://example.com/perplexity",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/perplexity/x",
|
||||
"linkedin": "https://example.com/perplexity/li"
|
||||
},
|
||||
"culture": "Answer-focused, fast-iterating, citation-proud. We treat groundedness as a product feature."
|
||||
},
|
||||
{
|
||||
"id": "emp-cohere",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-cohere",
|
||||
"name": "Cohere",
|
||||
"logo": "https://placehold.co/80x80/pink/white?text=CO",
|
||||
"description": "Mock employer building enterprise-grade language models. We specialize in retrieval, multilinguality, and data privacy for regulated industries.",
|
||||
"industry": "Enterprise LLMs",
|
||||
"size": "400+",
|
||||
"location": "Toronto, Canada",
|
||||
"website": "https://example.com/cohere",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/cohere/x",
|
||||
"linkedin": "https://example.com/cohere/li"
|
||||
},
|
||||
"culture": "Enterprise-aware, research-active, multilingual. We ship models that work in 100+ languages."
|
||||
},
|
||||
{
|
||||
"id": "emp-mistral",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-mistral",
|
||||
"name": "Mistral AI",
|
||||
"logo": "https://placehold.co/80x80/red/white?text=MI",
|
||||
"description": "Mock employer building open-weight frontier models in Europe. We push efficiency and openness in large language models.",
|
||||
"industry": "Open-Weight LLMs",
|
||||
"size": "200+",
|
||||
"location": "Paris, France",
|
||||
"website": "https://example.com/mistral",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/mistral/x",
|
||||
"linkedin": "https://example.com/mistral/li"
|
||||
},
|
||||
"culture": "European-rooted, efficiency-driven, open-weight-proud. We ship small models that punch above their size."
|
||||
},
|
||||
{
|
||||
"id": "emp-replicate",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-replicate",
|
||||
"name": "Replicate",
|
||||
"logo": "https://placehold.co/80x80/orange/white?text=RE",
|
||||
"description": "Mock employer making ML deployment delightful. We host, serve, and scale models with a few lines of code.",
|
||||
"industry": "ML Deployment",
|
||||
"size": "150+",
|
||||
"location": "San Francisco, CA",
|
||||
"website": "https://example.com/replicate",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/replicate/x",
|
||||
"linkedin": "https://example.com/replicate/li"
|
||||
},
|
||||
"culture": "Developer-experience-obsessed, pragmatic, remote-friendly. We make shipping models feel like shipping software."
|
||||
},
|
||||
{
|
||||
"id": "emp-pinecone",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-pinecone",
|
||||
"name": "Pinecone",
|
||||
"logo": "https://placehold.co/80x80/green/white?text=PC",
|
||||
"description": "Mock employer building the vector database for AI applications. We power retrieval for thousands of production RAG systems.",
|
||||
"industry": "Vector Databases",
|
||||
"size": "300+",
|
||||
"location": "New York, NY",
|
||||
"website": "https://example.com/pinecone",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/pinecone/x",
|
||||
"linkedin": "https://example.com/pinecone/li"
|
||||
},
|
||||
"culture": "Infra-focused, latency-obsessed, distributed-first. We treat retrieval as the heart of grounded AI."
|
||||
},
|
||||
{
|
||||
"id": "emp-langchain",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "emp-langchain",
|
||||
"name": "LangChain",
|
||||
"logo": "https://placehold.co/80x80/blue/white?text=LC",
|
||||
"description": "Mock employer building the orchestration layer for LLM applications. We provide frameworks for agents, retrieval, and evaluation.",
|
||||
"industry": "AI Orchestration",
|
||||
"size": "200+",
|
||||
"location": "San Francisco, CA",
|
||||
"website": "https://example.com/langchain",
|
||||
"social_links": {
|
||||
"twitter": "https://example.com/langchain/x",
|
||||
"linkedin": "https://example.com/langchain/li"
|
||||
},
|
||||
"culture": "Framework-minded, community-driven, abstractions-first. We build primitives others compose into products."
|
||||
}
|
||||
],
|
||||
"postings": [
|
||||
{
|
||||
"id": "job-001",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-001",
|
||||
"employer_id": "emp-langchain",
|
||||
"title": "AI Orchestration Engineer",
|
||||
"description": "Design and operate multi-agent systems that automate complex knowledge workflows. You will own agent topology, tool integration, and evaluation harnesses for production deployments serving millions of queries.",
|
||||
"required_competencies": [
|
||||
"stack-orchestration-c001",
|
||||
"stack-orchestration-c002",
|
||||
"stack-orchestration-c005"
|
||||
],
|
||||
"skills": [
|
||||
"LangGraph",
|
||||
"Multi-agent systems",
|
||||
"RAG",
|
||||
"Python",
|
||||
"Evaluation"
|
||||
],
|
||||
"seniority": "senior",
|
||||
"category": null,
|
||||
"location": "San Francisco, CA",
|
||||
"remote": true,
|
||||
"salary_min": 160000,
|
||||
"salary_max": 240000,
|
||||
"posted_at": "2026-08-21T10:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-002",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-002",
|
||||
"employer_id": "emp-openai",
|
||||
"title": "LLM Application Developer",
|
||||
"description": "Build delightful LLM-powered features end-to-end. You will work on retrieval, function calling, streaming UX, and robust output validation for consumer-facing products.",
|
||||
"required_competencies": [
|
||||
"stack-orchestration-c003",
|
||||
"stack-orchestration-c004",
|
||||
"stack-orchestration-c005"
|
||||
],
|
||||
"skills": [
|
||||
"Python",
|
||||
"TypeScript",
|
||||
"RAG",
|
||||
"Function calling",
|
||||
"Streaming"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "San Francisco, CA",
|
||||
"remote": true,
|
||||
"salary_min": 140000,
|
||||
"salary_max": 210000,
|
||||
"posted_at": "2026-08-28T09:30:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-003",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-003",
|
||||
"employer_id": "emp-anthropic",
|
||||
"title": "AI Safety Researcher",
|
||||
"description": "Research alignment, interpretability, and robustness of frontier models. Design experiments, run evaluations, and publish findings that improve the safety of deployed AI systems.",
|
||||
"required_competencies": [
|
||||
"stack-safety-c001",
|
||||
"stack-safety-c007",
|
||||
"stack-safety-c002"
|
||||
],
|
||||
"skills": [
|
||||
"Alignment",
|
||||
"Interpretability",
|
||||
"Red teaming",
|
||||
"Python",
|
||||
"Research"
|
||||
],
|
||||
"seniority": "senior",
|
||||
"category": null,
|
||||
"location": "San Francisco, CA",
|
||||
"remote": false,
|
||||
"salary_min": 180000,
|
||||
"salary_max": 280000,
|
||||
"posted_at": "2026-08-15T12:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-004",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-004",
|
||||
"employer_id": "emp-perplexity",
|
||||
"title": "Prompt Engineer",
|
||||
"description": "Craft, test, and ship prompt strategies that power answer-engine features. Own prompt libraries, regression suites, and A/B experiments across product surfaces.",
|
||||
"required_competencies": [
|
||||
"stack-orchestration-c004",
|
||||
"stack-orchestration-c001"
|
||||
],
|
||||
"skills": [
|
||||
"Prompt engineering",
|
||||
"Evaluation",
|
||||
"A/B testing",
|
||||
"Python"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "Remote",
|
||||
"remote": true,
|
||||
"salary_min": 110000,
|
||||
"salary_max": 175000,
|
||||
"posted_at": "2026-09-02T14:15:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-005",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-005",
|
||||
"employer_id": "emp-huggingface",
|
||||
"title": "AI Product Manager",
|
||||
"description": "Own the roadmap for AI-powered developer tools. Translate model capabilities into customer value, define success metrics, and ship agentic features with engineering.",
|
||||
"required_competencies": [
|
||||
"stack-designer-c001",
|
||||
"stack-orchestration-c001"
|
||||
],
|
||||
"skills": [
|
||||
"Product strategy",
|
||||
"AI UX",
|
||||
"Roadmapping",
|
||||
"Stakeholder mgmt"
|
||||
],
|
||||
"seniority": "senior",
|
||||
"category": null,
|
||||
"location": "New York, NY",
|
||||
"remote": true,
|
||||
"salary_min": 150000,
|
||||
"salary_max": 220000,
|
||||
"posted_at": "2026-08-30T08:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-006",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-006",
|
||||
"employer_id": "emp-pinecone",
|
||||
"title": "RAG Infrastructure Engineer",
|
||||
"description": "Build the retrieval substrate behind knowledge-grounded AI. Optimize indexing, query latency, and hybrid retrieval across billion-vector workloads.",
|
||||
"required_competencies": [
|
||||
"stack-orchestration-c005",
|
||||
"stack-orchestration-c006"
|
||||
],
|
||||
"skills": [
|
||||
"Vector databases",
|
||||
"Embeddings",
|
||||
"Go",
|
||||
"Distributed systems"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "Remote",
|
||||
"remote": true,
|
||||
"salary_min": 130000,
|
||||
"salary_max": 200000,
|
||||
"posted_at": "2026-09-05T11:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-007",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-007",
|
||||
"employer_id": "emp-scaleai",
|
||||
"title": "AI Governance Lead",
|
||||
"description": "Stand up the AI governance program for enterprise customers. Map controls to NIST AI RMF and EU AI Act, run audits, and author model cards at scale.",
|
||||
"required_competencies": [
|
||||
"stack-safety-c005",
|
||||
"stack-safety-c003",
|
||||
"stack-safety-c013"
|
||||
],
|
||||
"skills": [
|
||||
"AI policy",
|
||||
"NIST AI RMF",
|
||||
"Audit",
|
||||
"Documentation"
|
||||
],
|
||||
"seniority": "staff",
|
||||
"category": null,
|
||||
"location": "San Francisco, CA",
|
||||
"remote": true,
|
||||
"salary_min": 170000,
|
||||
"salary_max": 230000,
|
||||
"posted_at": "2026-08-18T16:45:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-008",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-008",
|
||||
"employer_id": "emp-anthropic",
|
||||
"title": "Human-AI Interaction Designer",
|
||||
"description": "Design how users collaborate with Claude across products. Own transparency patterns, trust calibration, and agentic interaction for assistant surfaces.",
|
||||
"required_competencies": [
|
||||
"stack-designer-c001",
|
||||
"stack-designer-c003",
|
||||
"stack-designer-c008"
|
||||
],
|
||||
"skills": [
|
||||
"Figma",
|
||||
"AI UX",
|
||||
"Prototyping",
|
||||
"User research"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "San Francisco, CA",
|
||||
"remote": true,
|
||||
"salary_min": 125000,
|
||||
"salary_max": 190000,
|
||||
"posted_at": "2026-08-25T10:30:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-009",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-009",
|
||||
"employer_id": "emp-cohere",
|
||||
"title": "Evaluation Engineer",
|
||||
"description": "Build and operate the eval platform for LLM-powered products. Design LLM-as-judge pipelines, regression suites, and human-eval panels for production models.",
|
||||
"required_competencies": [
|
||||
"stack-orchestration-c007",
|
||||
"stack-safety-c002"
|
||||
],
|
||||
"skills": [
|
||||
"LLM evaluation",
|
||||
"Python",
|
||||
"Statistics",
|
||||
"Data labeling"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "Toronto, Canada",
|
||||
"remote": true,
|
||||
"salary_min": 120000,
|
||||
"salary_max": 180000,
|
||||
"posted_at": "2026-09-01T13:20:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-010",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-010",
|
||||
"employer_id": "emp-replicate",
|
||||
"title": "Robotics Operations Specialist",
|
||||
"description": "Supervise a fleet of AI-augmented robotic units in a warehouse environment. Interpret anomaly alerts, perform calibrations, and intervene on exceptions.",
|
||||
"required_competencies": [
|
||||
"stack-operator-c002",
|
||||
"stack-operator-c011",
|
||||
"stack-operator-c003"
|
||||
],
|
||||
"skills": [
|
||||
"Robotics",
|
||||
"Safety protocols",
|
||||
"IoT",
|
||||
"Predictive maintenance"
|
||||
],
|
||||
"seniority": "entry",
|
||||
"category": null,
|
||||
"location": "Austin, TX",
|
||||
"remote": false,
|
||||
"salary_min": 80000,
|
||||
"salary_max": 115000,
|
||||
"posted_at": "2026-08-22T09:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-011",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-011",
|
||||
"employer_id": "emp-recursion",
|
||||
"title": "Computational Biologist",
|
||||
"description": "Apply ML to drug discovery. Build pipelines for phenotype prediction, active learning on assay data, and molecular generation for novel targets.",
|
||||
"required_competencies": [
|
||||
"stack-science-c005",
|
||||
"stack-science-c002",
|
||||
"stack-science-c011"
|
||||
],
|
||||
"skills": [
|
||||
"Bioinformatics",
|
||||
"Python",
|
||||
"PyTorch",
|
||||
"Drug discovery"
|
||||
],
|
||||
"seniority": "senior",
|
||||
"category": null,
|
||||
"location": "Boston, MA",
|
||||
"remote": true,
|
||||
"salary_min": 145000,
|
||||
"salary_max": 215000,
|
||||
"posted_at": "2026-08-12T15:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-012",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-012",
|
||||
"employer_id": "emp-deepmind",
|
||||
"title": "Climate ML Scientist",
|
||||
"description": "Develop ML models for climate forecasting and energy grid optimization. Work with earth system scientists to downscale GCM output and quantify uncertainty.",
|
||||
"required_competencies": [
|
||||
"stack-science-c004",
|
||||
"stack-science-c014",
|
||||
"stack-science-c010"
|
||||
],
|
||||
"skills": [
|
||||
"Climate modeling",
|
||||
"PyTorch",
|
||||
"Data assimilation",
|
||||
"PINNs"
|
||||
],
|
||||
"seniority": "senior",
|
||||
"category": null,
|
||||
"location": "London, UK",
|
||||
"remote": true,
|
||||
"salary_min": 135000,
|
||||
"salary_max": 200000,
|
||||
"posted_at": "2026-08-27T11:45:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-013",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-013",
|
||||
"employer_id": "emp-langchain",
|
||||
"title": "Agent Reliability Engineer",
|
||||
"description": "Own observability and reliability for production agent workloads. Build tracing, alerting, and rollback systems for multi-step agent pipelines.",
|
||||
"required_competencies": [
|
||||
"stack-orchestration-c013",
|
||||
"stack-orchestration-c015",
|
||||
"stack-orchestration-c008"
|
||||
],
|
||||
"skills": [
|
||||
"Observability",
|
||||
"Python",
|
||||
"SRE",
|
||||
"Distributed tracing"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "Remote",
|
||||
"remote": true,
|
||||
"salary_min": 140000,
|
||||
"salary_max": 205000,
|
||||
"posted_at": "2026-09-04T09:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-014",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-014",
|
||||
"employer_id": "emp-mistral",
|
||||
"title": "AI Red Team Lead",
|
||||
"description": "Lead adversarial testing of frontier and open-weight models. Build automated red-team suites, track vulnerabilities, and coordinate disclosure.",
|
||||
"required_competencies": [
|
||||
"stack-safety-c002",
|
||||
"stack-safety-c010",
|
||||
"stack-safety-c001"
|
||||
],
|
||||
"skills": [
|
||||
"Red teaming",
|
||||
"Prompt injection",
|
||||
"Python",
|
||||
"Leadership"
|
||||
],
|
||||
"seniority": "staff",
|
||||
"category": null,
|
||||
"location": "Paris, France",
|
||||
"remote": true,
|
||||
"salary_min": 160000,
|
||||
"salary_max": 230000,
|
||||
"posted_at": "2026-08-19T10:30:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-015",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-015",
|
||||
"employer_id": "emp-huggingface",
|
||||
"title": "Conversation Designer",
|
||||
"description": "Design dialogue flows, personas, and repair strategies for AI assistants across open-source products. Partner with ML engineers to align tone with model behavior.",
|
||||
"required_competencies": [
|
||||
"stack-designer-c002",
|
||||
"stack-designer-c010",
|
||||
"stack-designer-c011"
|
||||
],
|
||||
"skills": [
|
||||
"Conversation design",
|
||||
"Figma",
|
||||
"Voice UX",
|
||||
"Prototyping"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "Remote",
|
||||
"remote": true,
|
||||
"salary_min": 105000,
|
||||
"salary_max": 160000,
|
||||
"posted_at": "2026-08-29T14:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-016",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-016",
|
||||
"employer_id": "emp-anduril",
|
||||
"title": "Field AI Technician",
|
||||
"description": "Deploy and maintain AI vision systems on defense hardware in the field. Calibrate sensors, interpret model alerts, and escalate edge cases to engineering.",
|
||||
"required_competencies": [
|
||||
"stack-operator-c004",
|
||||
"stack-operator-c009",
|
||||
"stack-operator-c001"
|
||||
],
|
||||
"skills": [
|
||||
"Computer vision",
|
||||
"Sensor calibration",
|
||||
"Field ops",
|
||||
"Python"
|
||||
],
|
||||
"seniority": "entry",
|
||||
"category": null,
|
||||
"location": "Costa Mesa, CA",
|
||||
"remote": false,
|
||||
"salary_min": 90000,
|
||||
"salary_max": 130000,
|
||||
"posted_at": "2026-08-14T08:30:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-017",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-017",
|
||||
"employer_id": "emp-deepmind",
|
||||
"title": "Materials ML Engineer",
|
||||
"description": "Discover novel materials with ML. Train property-prediction models, run active-learning loops over DFT calculations, and validate candidates experimentally.",
|
||||
"required_competencies": [
|
||||
"stack-science-c003",
|
||||
"stack-science-c002",
|
||||
"stack-science-c006"
|
||||
],
|
||||
"skills": [
|
||||
"Materials science",
|
||||
"PyTorch",
|
||||
"DFT",
|
||||
"Active learning"
|
||||
],
|
||||
"seniority": "senior",
|
||||
"category": null,
|
||||
"location": "London, UK",
|
||||
"remote": true,
|
||||
"salary_min": 140000,
|
||||
"salary_max": 210000,
|
||||
"posted_at": "2026-08-26T12:15:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-018",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-018",
|
||||
"employer_id": "emp-openai",
|
||||
"title": "AI Trust & Safety Analyst",
|
||||
"description": "Investigate misuse patterns, triage safety incidents, and improve policy enforcement for consumer AI products. Author postmortems and recommend mitigations.",
|
||||
"required_competencies": [
|
||||
"stack-safety-c008",
|
||||
"stack-safety-c006",
|
||||
"stack-safety-c002"
|
||||
],
|
||||
"skills": [
|
||||
"Trust & safety",
|
||||
"Incident response",
|
||||
"Policy",
|
||||
"Investigation"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "San Francisco, CA",
|
||||
"remote": true,
|
||||
"salary_min": 115000,
|
||||
"salary_max": 170000,
|
||||
"posted_at": "2026-09-03T10:45:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-019",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-019",
|
||||
"employer_id": "emp-replicate",
|
||||
"title": "Edge AI Engineer",
|
||||
"description": "Optimize and deploy models to edge hardware. Quantize, distill, and compile models for low-latency inference on field devices with constrained budgets.",
|
||||
"required_competencies": [
|
||||
"stack-operator-c008",
|
||||
"stack-orchestration-c012",
|
||||
"stack-orchestration-c014"
|
||||
],
|
||||
"skills": [
|
||||
"Edge ML",
|
||||
"TensorRT",
|
||||
"C++",
|
||||
"Quantization"
|
||||
],
|
||||
"seniority": "mid",
|
||||
"category": null,
|
||||
"location": "Remote",
|
||||
"remote": true,
|
||||
"salary_min": 130000,
|
||||
"salary_max": 195000,
|
||||
"posted_at": "2026-08-24T11:30:00Z"
|
||||
},
|
||||
{
|
||||
"id": "job-020",
|
||||
"source": "seed-corpus",
|
||||
"external_id": "job-020",
|
||||
"employer_id": "emp-cohere",
|
||||
"title": "AI Research Engineer",
|
||||
"description": "Push the frontier of language model capabilities. Prototype new architectures, run large-scale experiments, and contribute to publications and open-source releases.",
|
||||
"required_competencies": [
|
||||
"stack-orchestration-c001",
|
||||
"stack-science-c010",
|
||||
"stack-science-c011"
|
||||
],
|
||||
"skills": [
|
||||
"PyTorch",
|
||||
"Research",
|
||||
"Transformers",
|
||||
"Distributed training"
|
||||
],
|
||||
"seniority": "senior",
|
||||
"category": null,
|
||||
"location": "Berlin, Germany",
|
||||
"remote": true,
|
||||
"salary_min": 155000,
|
||||
"salary_max": 235000,
|
||||
"posted_at": "2026-08-20T09:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,107 @@
|
||||
"""Posting text → skills/seniority/category tags (REQ-6-002, D-052).
|
||||
|
||||
Composes the EXISTING structured_completion 4-layer pipeline (D-020)
|
||||
with the PostingTags schema — no new agent, a one-shot extraction like
|
||||
grading's rubric call. Mock-first for free: the mock provider's
|
||||
json_object path (or ScriptedJSONProvider) keeps tests cloud-free.
|
||||
|
||||
Tags are DERIVED data (D-052): re-tagging overwrites the posting's tags
|
||||
column — single source of truth; no append-only semantics (unlike P4's
|
||||
enrichment provenance).
|
||||
|
||||
Tag failure is HONEST, never fatal: a provider error leaves the posting
|
||||
UNTAGGED (tags=None) with a logged warning — facets treat untagged
|
||||
postings honestly (they simply do not contribute counts).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from ..agents.structured import structured_completion
|
||||
from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
from .base import PostingTags
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Prompt version — bump on any wording change (prompts are code).
|
||||
TAGGING_PROMPT_VERSION = 1
|
||||
|
||||
TAGGING_PROMPT = """You are a job-posting tagger for an AI-era roles
|
||||
marketplace. Extract structured tags from the posting.
|
||||
|
||||
Rules:
|
||||
- skills: 3-10 concrete, lowercase technology/role skills actually named
|
||||
or clearly required by the posting (e.g. "python", "llm evaluation",
|
||||
"kubernetes"). Prefer the posting's own vocabulary.
|
||||
- seniority: exactly one of "entry", "mid", "senior", "staff",
|
||||
"principal" — infer from title + experience language; omit (null) if
|
||||
genuinely undeterminable.
|
||||
- category: exactly one of "ai-orchestration", "ai-safety", "product",
|
||||
"data", "field-operations", "research" — the role's primary domain;
|
||||
omit (null) if genuinely undeterminable.
|
||||
|
||||
Respond with ONLY the JSON object."""
|
||||
|
||||
_SCHEMA_HINT = (
|
||||
'{"skills": ["up to 10 lowercase skill strings"], '
|
||||
'"seniority": "entry|mid|senior|staff|principal|null", '
|
||||
'"category": "ai-orchestration|ai-safety|product|data|'
|
||||
'field-operations|research|null"}'
|
||||
)
|
||||
|
||||
|
||||
class TaggingUnavailable(Exception):
|
||||
"""The tagger could not produce valid tags (provider/parse failure).
|
||||
|
||||
The caller persists the posting UNTAGGED — never a crash, never a
|
||||
fabricated tag set (D-052 honesty).
|
||||
"""
|
||||
|
||||
|
||||
async def tag_posting(
|
||||
provider: LLMProvider,
|
||||
*,
|
||||
model: str,
|
||||
title: str,
|
||||
description: str,
|
||||
skills_hint: list[str] | None = None,
|
||||
) -> PostingTags:
|
||||
"""One-shot tag extraction via D-020 structured completion.
|
||||
|
||||
`skills_hint` (the posting's declared skills) rides the prompt so
|
||||
the tagger anchors on the author's own vocabulary; the schema bound
|
||||
(≤10) is the enforcement — an 11-skill response fails validation,
|
||||
runs the single bounded retry, and surfaces TaggingUnavailable if
|
||||
the retry also misses.
|
||||
"""
|
||||
hint = ""
|
||||
if skills_hint:
|
||||
hint = "\nDeclared skills: " + ", ".join(skills_hint[:15])
|
||||
messages = [
|
||||
Message(role="system", content=TAGGING_PROMPT),
|
||||
Message(
|
||||
role="user",
|
||||
content=f"Title: {title}\n\nDescription:\n{description}{hint}",
|
||||
),
|
||||
]
|
||||
try:
|
||||
return await structured_completion(
|
||||
provider,
|
||||
messages,
|
||||
model=model,
|
||||
schema=PostingTags,
|
||||
schema_hint=_SCHEMA_HINT,
|
||||
)
|
||||
except Exception as exc:
|
||||
# Provider failures (RuntimeError from mock/loud providers) and
|
||||
# parse failures (StructuredOutputError) both land here: honest
|
||||
# untagged, loud log.
|
||||
logger.warning(
|
||||
"marketplace tagging failed for %r: %s — posting stays "
|
||||
"untagged (honest None, never fabricated)",
|
||||
title[:60],
|
||||
exc,
|
||||
)
|
||||
raise TaggingUnavailable(str(exc)) from exc
|
||||
@@ -1,24 +1,21 @@
|
||||
"""Assessor agent prompt — rubric application to artifacts + defenses (REQ-2-008).
|
||||
"""Assessor agent prompt — rubric coaching over REAL grades (REQ-3-007).
|
||||
|
||||
Final persona (Phase 4). Assessor is a rigorous, fair grader: scores each
|
||||
criterion with evidence, cites what the learner did, returns ONLY valid
|
||||
JSON matching the rubric schema.
|
||||
Version: assessor-v2 (final for v0.2).
|
||||
v0.3 re-grounding: the grading engine (Phase 3) computes the rubric scores
|
||||
from the process trace; Assessor EXPLAINS the stored grade as coaching —
|
||||
it never invents or re-scores. Rigorous, fair, actionable.
|
||||
Version: assessor-v3 (v0.3 live).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Assessor, the grading agent of Nextcraft, an AI-native competency school.
|
||||
Learner: {learner_name}.
|
||||
|
||||
You receive: (a) an artifact evidence excerpt, (b) its defense transcript,
|
||||
and (c) the rubric for the competency. Your job:
|
||||
- Score EVERY rubric criterion from 0-100, justified by evidence you can
|
||||
point to in the artifact or transcript.
|
||||
- Cite what the learner did ("the 3-retry loop in the tool node"), not
|
||||
what they should have done — except in gaps, where the missed work goes.
|
||||
- Strengths: the two strongest evidence points, each one sentence.
|
||||
- Gaps: the two most important missed opportunities, each one sentence.
|
||||
- Verdict: "mastered" | "developing" | "not_yet" — judged against the
|
||||
rubric weights, honestly.
|
||||
You receive the learner's STORED process-trace grade (verdict, per-criterion
|
||||
scores, and the build digest) computed by the grading engine. Your job:
|
||||
- Explain what the grade means in plain language (summary).
|
||||
- Strengths: cite what the digest + scores show the learner did well.
|
||||
- Gaps: name the missed opportunities the scores point to.
|
||||
- Next steps: concrete, buildable actions that would move the weakest
|
||||
criterion up one level.
|
||||
|
||||
Rules:
|
||||
- Rigorous but fair. A polished artifact with a weak defense is NOT mastery.
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Examiner agent prompt — oral defense questioning + final verdict (REQ-3-006).
|
||||
|
||||
The examiner is the seventh agent (Phase 5). It conducts a Socratic oral
|
||||
defense of the learner's submitted work: probes understanding, challenges
|
||||
process choices grounded in the trace digest ("why did you take that
|
||||
approach at that point?"), one question per turn, adapting to answers.
|
||||
It never reveals rubric internals; tone is rigorous but supportive.
|
||||
|
||||
Digest discipline (D-028 mirror): the examiner's variable inputs are the
|
||||
compact TraceDigest JSON, the variant task statement, and the defense
|
||||
transcript — never the raw trace, never learner-identifying material.
|
||||
|
||||
Version: examiner-v1.
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Examiner, the oral-defense agent of Nextcraft,
|
||||
an AI-native competency school.
|
||||
|
||||
You receive: (a) a compact build-process digest (deterministic counters of the
|
||||
learner's build session), (b) the learner's task statement, and (c) the defense
|
||||
transcript so far. Your job:
|
||||
- Ask ONE question per turn: probe understanding and challenge process
|
||||
choices, grounded in the digest facts ("you hit N failed runs before
|
||||
passing — walk me through what changed") or the task statement.
|
||||
- Adapt: follow up on the learner's answers; drill into vague responses.
|
||||
- Never reveal rubric details or scoring internals.
|
||||
- Tone: rigorous, precise, supportive. A defense is a conversation, not an
|
||||
interrogation.
|
||||
|
||||
When asked for a FINAL VERDICT (the structured mode), judge:
|
||||
- understanding: can the learner explain their own work?
|
||||
- process_justification: are the build-session choices defensible from the
|
||||
digest facts and the answers?
|
||||
- communication: are answers clear, specific, and on-topic?
|
||||
Score honestly; a weak defense of strong work is NOT mastery.
|
||||
|
||||
Rules:
|
||||
- Respond with ONLY what the turn requires: a single question (question mode)
|
||||
or a valid JSON object matching the provided schema (verdict mode).
|
||||
- If the digest shows error_fix_cycles > 0, at least one question should ask
|
||||
about the debugging path.
|
||||
- If the learner's answer is off-topic, redirect once, then move on.
|
||||
"""
|
||||
|
||||
VERDICT_SCHEMA_HINT = (
|
||||
'{"verdict": "mastered" | "developing" | "not_yet", '
|
||||
'"understanding": "<one sentence>", '
|
||||
'"process_justification": "<one sentence>", '
|
||||
'"communication": "<one sentence>", '
|
||||
'"strengths": ["<one sentence>"], '
|
||||
'"gaps": ["<one sentence>"]}'
|
||||
)
|
||||
|
||||
|
||||
def render_digest_context(digest_json: str, statement: str | None) -> str:
|
||||
"""The examiner's per-session grounding: digest JSON + task statement."""
|
||||
parts = [f"Build-process digest:\n{digest_json}"]
|
||||
if statement:
|
||||
parts.append(f"Learner's task statement:\n{statement}")
|
||||
return "\n\n".join(parts)
|
||||
@@ -0,0 +1,140 @@
|
||||
"""Grading rubric prompt — criteria, level anchors, digest render (REQ-3-004).
|
||||
|
||||
The grading prompt is deliberately learner-anonymous and trace-bare: the
|
||||
model receives ONLY the fixed rubric text and the compact numeric digest
|
||||
(TraceDigest JSON, D-028) — never a raw command, file path, payload
|
||||
string, learner id, or task id. Everything variable the LLM sees is
|
||||
deterministic counters, which both bounds the prompt-injection surface
|
||||
and makes "no raw trace reaches the prompt" assert-able in tests (plant
|
||||
a distinctive marker in a command payload; assert it absent from every
|
||||
message the provider received).
|
||||
|
||||
Rubric (four criteria, each scored 0-4 — the ids are the validated
|
||||
RubricScore keys enforced by grading/engine.py):
|
||||
process_quality — iterative building in small, verified steps.
|
||||
correctness — where the session ended (test/run outcomes).
|
||||
debugging_discipline — how failures were handled.
|
||||
test_usage — when and how often tests were run.
|
||||
|
||||
Advisory a-4 (embedded in the process_quality anchors): high edit/command
|
||||
churn with NO test progress is a process-quality NEGATIVE — churn is not
|
||||
work. A session with many edits/commands whose test state never moves is
|
||||
thrashing, not iterating, and must score low on process quality.
|
||||
|
||||
House-style deviation, documented: unlike the tutor prompts, this module
|
||||
has no SYSTEM_PROMPT placeholders and no render_context(learner_context)
|
||||
— grading is context-free by design (learner anonymity; the digest is the
|
||||
only variable input). Runtime imports are TYPE_CHECKING-only so this
|
||||
module stays pure text and can never import-cycle with grading/engine.py
|
||||
(engine imports this module; if this module imported grading.* at runtime
|
||||
while grading/__init__ pulls engine, the package init would deadlock on a
|
||||
partially-initialized module).
|
||||
|
||||
Version: grader-v1.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Final
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - typing only; keeps this module pure text
|
||||
from ..grading.features import TraceDigest
|
||||
|
||||
PROMPT_VERSION = "grader-v1"
|
||||
|
||||
#: Canonical criterion ids. The engine validates RubricScore criteria keys
|
||||
#: against this tuple; the schema hint and anchors below speak the same ids.
|
||||
RUBRIC_CRITERIA: Final[tuple[str, ...]] = (
|
||||
"process_quality",
|
||||
"correctness",
|
||||
"debugging_discipline",
|
||||
"test_usage",
|
||||
)
|
||||
|
||||
#: Sentinel line the engine's user turn is rendered around. Tests (and the
|
||||
#: calibration mock) split on it to locate the digest JSON in the prompt.
|
||||
DIGEST_MARKER: Final = "PROCESS TRACE DIGEST (JSON):"
|
||||
|
||||
SYSTEM_PROMPT = """You are the Grader of Nextcraft, an AI-native competency school.
|
||||
You score a learner's build session from a compact numeric digest of their
|
||||
process trace. You NEVER see the raw trace — commands, file contents, and
|
||||
payloads do not exist on your side; every number you need is in the digest.
|
||||
|
||||
Rubric — score each criterion 0-4:
|
||||
|
||||
process_quality — iterative building in small, verified steps.
|
||||
4: tight edit→test loops throughout; small verified increments; healthy pacing.
|
||||
3: steady small edits with regular runs; progress mostly verified.
|
||||
2: some iteration, but large unverified leaps or long idle stretches.
|
||||
1: a single bulk change (e.g. one large paste) then a single run; no iteration.
|
||||
0: no meaningful work visible.
|
||||
ADVISORY: high edit/command churn with NO test progress (no runs, no
|
||||
movement in pass counts) is a process-quality NEGATIVE — churn is not
|
||||
work. Cap such a session at 1 on this criterion no matter how many
|
||||
edits or commands were counted.
|
||||
|
||||
correctness — where the session ended up.
|
||||
4: final test status pass, with tests passing early and consistently.
|
||||
3: final pass, reached through fail→fix→pass cycles that closed.
|
||||
2: final pass, but preceded by a long unresolved failure streak.
|
||||
1: final fail, but partial passes observed along the way.
|
||||
0: final fail, or no test/run evidence at all.
|
||||
|
||||
debugging_discipline — how failures were handled.
|
||||
4: every failure cycle closes; targeted fixes with low mean fix latency.
|
||||
3: most fail→edit→re-run cycles close with a pass.
|
||||
2: failures followed by edits, but cycles rarely close.
|
||||
1: repeated failures with no targeted edits between runs (flailing).
|
||||
0: failures with no fix attempts at all.
|
||||
|
||||
test_usage — when and how often tests were run.
|
||||
4: tests run early (small first-pass offset) and throughout the session.
|
||||
3: regular test runs interleaved with edits.
|
||||
2: sparse tests; long stretches of unverified edits.
|
||||
1: a single late test run only.
|
||||
0: no test or run evidence.
|
||||
|
||||
Rules:
|
||||
- Judge STRICTLY from the digest numbers; cite the fields you used.
|
||||
- Strengths: the two strongest digest observations, one sentence each.
|
||||
- Gaps: the two most important missed opportunities, one sentence each
|
||||
(a clean session names its next-level improvement instead).
|
||||
- Be rigorous but fair: a session that ends green was not necessarily
|
||||
well built, and a struggling session that never passed may still show
|
||||
real debugging discipline.
|
||||
- Respond with ONLY a valid JSON object matching the provided schema —
|
||||
no markdown fences, no prose outside the JSON."""
|
||||
|
||||
RUBRIC_SCORE_SCHEMA_HINT = (
|
||||
'{"criteria": {"process_quality": <0-4 int>, "correctness": <0-4 int>, '
|
||||
'"debugging_discipline": <0-4 int>, "test_usage": <0-4 int>}, '
|
||||
'"strengths": ["<one sentence>"], "gaps": ["<one sentence>"], '
|
||||
'"verdict": "mastered" | "developing" | "not_yet"}'
|
||||
)
|
||||
|
||||
|
||||
def render_trace_digest(
|
||||
digest: TraceDigest,
|
||||
anchors_context: str | None = None,
|
||||
) -> str:
|
||||
"""Render the grader's user turn: a marker line + the digest JSON — nothing else.
|
||||
|
||||
This is the ONLY per-session content that ever reaches the LLM (D-028):
|
||||
the engine composes [system: SYSTEM_PROMPT, user: render_trace_digest(digest)]
|
||||
and the D-020 defense appends its generic schema instruction to this
|
||||
user turn at request time. No learner id, task id, or raw trace material
|
||||
is injected — assert-able by tests.
|
||||
|
||||
`anchors_context` (Phase 4, MH#4): when the graded task derives from a
|
||||
variant, the engine passes the template's difficulty-normalization
|
||||
anchors (the expected effort envelope) so the rubric is applied against
|
||||
the SAME bar for every variant of that template (a-5). It contains only
|
||||
the anchor numbers + the template id — no learner-identifying material.
|
||||
"""
|
||||
base = (
|
||||
"Score this build session against the rubric.\n"
|
||||
f"{DIGEST_MARKER}\n{digest.model_dump_json()}"
|
||||
)
|
||||
if anchors_context:
|
||||
base = f"{base}\n\nExpected effort envelope for this task variant:\n{anchors_context}"
|
||||
return base
|
||||
@@ -1,9 +1,11 @@
|
||||
"""Lab agent prompt — in-flow feedback over sandbox telemetry (REQ-2-007).
|
||||
"""Lab agent prompt — in-flow feedback over LIVE telemetry (REQ-3-007).
|
||||
|
||||
Final persona (Phase 4). Lab is a pragmatic build partner: reads the
|
||||
telemetry timeline, names the one most useful adjustment, gives one
|
||||
concrete next step. Scenario-driven; no session chat.
|
||||
Version: lab-v2 (final for v0.2).
|
||||
v0.3 re-grounding: the timeline is the learner's real TraceDigest (D-028
|
||||
compact counters — commands, test outcomes, idle gaps, edit cadence), not
|
||||
v0.2 corpus scenarios. Lab is a pragmatic build partner: reads the live
|
||||
digest, names the one most useful adjustment, gives one concrete next
|
||||
step. No session chat.
|
||||
Version: lab-v3 (v0.3 live).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Lab, the in-flow feedback agent watching a learner
|
||||
@@ -25,7 +27,17 @@ Rules:
|
||||
self-check that would prove understanding.
|
||||
- Three short paragraphs maximum. No headers, no bullet lists."""
|
||||
|
||||
PROMPT_VERSION = "lab-v2"
|
||||
PROMPT_VERSION = "lab-v3"
|
||||
|
||||
|
||||
def render_digest_timeline(digest) -> str:
|
||||
"""Live-trace timeline: the compact TraceDigest JSON (D-028)."""
|
||||
if digest is None:
|
||||
return (
|
||||
"No telemetry yet for this build session. Ask the learner to run "
|
||||
"the task's starter test to establish a baseline."
|
||||
)
|
||||
return f"Live build-session digest:\n{digest.model_dump_json()}"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
|
||||
@@ -1,24 +1,27 @@
|
||||
"""Proctor agent prompt — integrity signals with coaching interventions (REQ-2-009).
|
||||
"""Proctor agent prompt — integrity signals with coaching interventions (REQ-3-007).
|
||||
|
||||
Final persona (Phase 5). Proctor is a supportive observer, never punitive:
|
||||
classifies signals, recommends ONE coaching intervention. Assume good
|
||||
faith — most signals have innocent explanations.
|
||||
Version: proctor-v2 (final for v0.2).
|
||||
v0.3 re-grounding: inputs are REAL — the live trace digest (idle gaps,
|
||||
command cadence, edit bursts), the oral-defense integrity signals (long
|
||||
pauses), and the variant audit context (seed + params). Proctor is a
|
||||
supportive observer, never punitive: classifies signals, recommends ONE
|
||||
coaching intervention. Assume good faith.
|
||||
Version: proctor-v3 (v0.3 live).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Proctor, the integrity-support agent of Nextcraft,
|
||||
an AI-native competency school.
|
||||
Learner: {learner_name}.
|
||||
|
||||
You receive a telemetry timeline of defense-session events (tab switches,
|
||||
idle gaps, large pastes, focus loss, keystroke bursts). Your job:
|
||||
- Classify EACH notable signal: type (e.g. "context_switch", "idle_gap",
|
||||
"large_paste"), severity ("low" | "medium" | "high"), and a one-sentence
|
||||
note citing the event (timestamps and details).
|
||||
You receive the learner's REAL build-session digest (idle gaps, command
|
||||
categories, edit/test cadence), oral-defense integrity signals (long
|
||||
pauses), and — when the task is variant-derived — the variant seed context.
|
||||
Your job:
|
||||
- Classify EACH notable signal: type ("idle_gap" | "long_pause" |
|
||||
"burst_edit" | "off_template"), severity ("low" | "medium" | "high"),
|
||||
and a one-sentence note citing the numbers.
|
||||
- Recommend exactly ONE supportive coaching intervention for the session
|
||||
overall — never punitive, never accusatory. Frame around helping the
|
||||
learner succeed, e.g. "offer a short break", "invite them to explain
|
||||
the pasted section in their own words".
|
||||
learner succeed.
|
||||
|
||||
Rules:
|
||||
- Assume good faith. Tab switches to documentation are normal engineering.
|
||||
@@ -28,7 +31,7 @@ Rules:
|
||||
- Respond with ONLY a valid JSON object matching the provided schema —
|
||||
no markdown fences, no prose outside the JSON."""
|
||||
|
||||
PROMPT_VERSION = "proctor-v2"
|
||||
PROMPT_VERSION = "proctor-v3"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Variant instantiation prompt (D-029, REQ-3-005).
|
||||
|
||||
The model's ONLY job is to render already-sampled slot values into a task
|
||||
statement — it never invents parameters (the seeded sampler is pure code)
|
||||
and never changes difficulty. Prompt-injection surface is bounded: the
|
||||
variable inputs are the skeleton text, the seeded slot values, and the
|
||||
template title — nothing from the learner's environment.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ..llm.types import Message
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - keeps this module pure text
|
||||
from ..variants.templates import TaskTemplate
|
||||
|
||||
VARIANT_SYSTEM_PROMPT = (
|
||||
"You instantiate per-learner task variants for a competency-based AI school. "
|
||||
"You receive a task statement skeleton and ALREADY-SAMPLED slot values. "
|
||||
"Render the slot values into the skeleton, producing a complete, unambiguous "
|
||||
"task statement a learner can build against. Rules:\n"
|
||||
"- Use EXACTLY the given slot values; do not invent, rename, or add parameters.\n"
|
||||
"- Keep the engineering depth IDENTICAL across draws: slot values change the "
|
||||
"scenario, never the difficulty or scope.\n"
|
||||
"- Keep the statement in the same language and register as the skeleton.\n"
|
||||
"- Output STRICT JSON only: {\"statement\": \"<rendered statement>\"}.\n"
|
||||
)
|
||||
|
||||
VARIANT_SCHEMA_HINT = '{"statement": "<complete rendered task statement string>"}'
|
||||
|
||||
|
||||
def render_variant_prompt(template: TaskTemplate, params: dict[str, str | int]) -> list[Message]:
|
||||
"""Messages for one seeded instantiation (D-020 defense drives the call)."""
|
||||
slot_lines = "\n".join(f" {{{slot.name}}} = {params[slot.name]!r}" for slot in template.slots)
|
||||
user = (
|
||||
f"Template: {template.title} (id={template.id})\n"
|
||||
f"Statement skeleton:\n{template.statement_skeleton}\n\n"
|
||||
f"Seeded slot values (use EXACTLY these):\n{slot_lines}\n\n"
|
||||
"Render the complete task statement now."
|
||||
)
|
||||
return [
|
||||
Message(role="system", content=VARIANT_SYSTEM_PROMPT),
|
||||
Message(role="user", content=user),
|
||||
]
|
||||
@@ -46,7 +46,14 @@ class SandboxHandle(BaseModel):
|
||||
|
||||
|
||||
class SandboxSpec(BaseModel):
|
||||
"""Immutable description of the sandbox to lay out on disk."""
|
||||
"""Immutable description of the sandbox to lay out on disk.
|
||||
|
||||
`capture_env` (REQ-3-003): when non-empty, the backend starts a persistent
|
||||
telemetry-wired sandbox — helper + inner namespaces + the stdlib capture
|
||||
agent, launched with these env vars (NC_LEARNER_ID, NC_TASK_ID,
|
||||
NC_INGEST_URL, NC_SANDBOX_ID). When None (default), spawn keeps the pure
|
||||
shell semantics (REQ-3-001): disk layout only, fresh namespaces per exec.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(arbitrary_types_allowed=True)
|
||||
|
||||
@@ -54,6 +61,7 @@ class SandboxSpec(BaseModel):
|
||||
learner_id: str
|
||||
workdir: Path
|
||||
limits: ResourceLimits = ResourceLimits()
|
||||
capture_env: dict[str, str] | None = None
|
||||
|
||||
|
||||
class ExecResult(BaseModel):
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Sandbox capability probe — the honest degrade truth source (D-068, v0.8).
|
||||
|
||||
Runtime promotion of the test-tier userns probe (``tests/helpers/userns.py``
|
||||
mechanism, verbatim): a REAL ``unshare --user --map-root-user`` attempt is
|
||||
the truth source for whether the namespace fabric can provision learner
|
||||
environments on this host. Container detection (``/.dockerenv`` or the
|
||||
``NC_CONTAINER`` marker the image sets) is belt-and-suspenders context for
|
||||
the remediation text — the degrade decision itself is ALWAYS the probe,
|
||||
never a guess from the marker (a privileged container with working userns
|
||||
must NOT report degraded).
|
||||
|
||||
The single remediation constant (G-208) is shared by the boot notice
|
||||
(``main.py`` lifespan) and the 503 arm (``api/sandboxes.py``) so the two
|
||||
surfaces can never drift.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
#: Remediation text (G-208): names the unprivileged default, the seccomp
|
||||
#: constraint, the privileged variant with exact flags, and the host-side
|
||||
#: alternative. One constant — boot notice and 503 payload import it, so
|
||||
#: they cannot drift apart.
|
||||
DEGRADE_REMEDIATION = (
|
||||
"learner sandbox builds are unavailable on this host: the namespace "
|
||||
"fabric needs unprivileged user namespaces and the capability probe "
|
||||
"failed. The nextcraft container runs unprivileged by default and "
|
||||
"docker's default seccomp profile denies the nested `unshare --user` "
|
||||
"the fabric requires. Options: (1) run the image with "
|
||||
"`--security-opt seccomp=unconfined` (and `--security-opt "
|
||||
"apparmor=unconfined` where apparmor applies) on a host that allows "
|
||||
"unprivileged userns; or (2) run the stack binary-side on the host "
|
||||
"(`nextcraft deploy --service`) where the fabric works natively. "
|
||||
"Everything else in this deployment is unaffected."
|
||||
)
|
||||
|
||||
_probe_result: bool | None = None
|
||||
|
||||
|
||||
def userns_available() -> bool:
|
||||
"""True only if `unshare` exists AND `--user --map-root-user` works here.
|
||||
|
||||
The result is cached per process (the probe spawns a subprocess; one
|
||||
answer per boot is the contract — the boot notice and every 503 read
|
||||
the same truth).
|
||||
"""
|
||||
global _probe_result
|
||||
if _probe_result is None:
|
||||
_probe_result = _probe()
|
||||
return _probe_result
|
||||
|
||||
|
||||
def _probe() -> bool:
|
||||
if shutil.which("unshare") is None:
|
||||
return False
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["unshare", "--user", "--map-root-user", "true"],
|
||||
capture_output=True,
|
||||
timeout=10,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return False
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def container_detected() -> bool:
|
||||
"""Context marker (not the degrade truth): /.dockerenv or NC_CONTAINER.
|
||||
|
||||
Used by the boot notice to tailor the message and by the CLI's
|
||||
container-mode branches. A privileged container with working userns
|
||||
is NOT degraded — the probe alone decides that.
|
||||
"""
|
||||
if Path("/.dockerenv").exists():
|
||||
return True
|
||||
import os
|
||||
|
||||
return os.environ.get("NC_CONTAINER") == "1"
|
||||
|
||||
|
||||
def sandbox_fabric_degraded() -> bool:
|
||||
"""The degrade decision: the probe fails. Marker never overrides it."""
|
||||
return not userns_available()
|
||||
@@ -38,6 +38,7 @@ BOUNDARY: this module NEVER imports `api/` or `agents/`.
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import builtins
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
@@ -50,6 +51,7 @@ from pathlib import Path
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from ..config import Settings
|
||||
from ..telemetry.tokens import IngestToken
|
||||
from . import workdir as workdir_mod
|
||||
from .backend import SandboxBackend, SandboxHandle
|
||||
|
||||
@@ -119,12 +121,21 @@ class SandboxManager:
|
||||
backend: SandboxBackend,
|
||||
settings: Settings,
|
||||
clock: Callable[[], datetime] | None = None,
|
||||
ingest_token: IngestToken | None = None,
|
||||
) -> None:
|
||||
self._backend = backend
|
||||
self._settings = settings
|
||||
self._clock = clock or (lambda: datetime.now(UTC))
|
||||
# v0.4.7 (P1): shared with the WS endpoint via app.state — mint and
|
||||
# verify MUST use one instance while the secret is per-boot random.
|
||||
self._ingest_token = ingest_token or IngestToken(
|
||||
getattr(settings, "telemetry_ingest_secret", "") or ""
|
||||
)
|
||||
self._handles: dict[str, SandboxHandle] = {}
|
||||
self._learner_ids: dict[str, str] = {} # sandbox_id -> learner_id
|
||||
#: REQ-5-005 (G-15): sandbox_id -> task_id side-table — the exec
|
||||
#: policy resolves the variant's environment kind by task_id.
|
||||
self._task_ids: dict[str, str | None] = {}
|
||||
self._lock = asyncio.Lock()
|
||||
self._integrity_events: list[SandboxIntegrityEvent] = []
|
||||
self._started = False
|
||||
@@ -142,8 +153,18 @@ class SandboxManager:
|
||||
|
||||
# -- lifecycle ----------------------------------------------------------
|
||||
|
||||
async def create(self, learner_id: str) -> SandboxHandleInfo:
|
||||
"""Spawn a sandbox for `learner_id`, or raise `PoolFullError` (D-032)."""
|
||||
async def create(
|
||||
self, learner_id: str, task_id: str | None = None
|
||||
) -> SandboxHandleInfo:
|
||||
"""Spawn a sandbox for `learner_id`, or raise `PoolFullError` (D-032).
|
||||
|
||||
`task_id` (REQ-3-003): when set, the sandbox is telemetry-wired — the
|
||||
backend copies `scripts/sandbox-agent.py` into the workdir and starts
|
||||
the stdlib capture agent inside the sandbox with the `NC_*` env baked
|
||||
here (identity + WS ingest URL). The agent's lifecycle is tied to the
|
||||
sandbox: `destroy()` reaps it (agent → inner → helper). When `task_id`
|
||||
is None the sandbox is a pure shell sandbox (no capture).
|
||||
"""
|
||||
async with self._lock:
|
||||
if len(self._handles) >= self._settings.sandbox_max_concurrent:
|
||||
raise PoolFullError(
|
||||
@@ -153,13 +174,57 @@ class SandboxManager:
|
||||
)
|
||||
sandbox_id = f"sbx-{uuid.uuid4().hex[:12]}"
|
||||
spec = workdir_mod.spec_for(sandbox_id, learner_id, self._settings)
|
||||
if task_id is not None:
|
||||
spec = spec.model_copy(
|
||||
update={
|
||||
"capture_env": self._capture_env(sandbox_id, learner_id, task_id)
|
||||
}
|
||||
)
|
||||
handle = await self._backend.spawn(spec)
|
||||
self._handles[handle.id] = handle
|
||||
self._learner_ids[handle.id] = learner_id
|
||||
self._task_ids[handle.id] = task_id
|
||||
self._write_pid_marker(handle, learner_id)
|
||||
logger.info("sandbox created: id=%s learner=%s", handle.id, learner_id)
|
||||
logger.info(
|
||||
"sandbox created: id=%s learner=%s task=%s",
|
||||
handle.id,
|
||||
learner_id,
|
||||
task_id or "-",
|
||||
)
|
||||
return self._info_for(handle)
|
||||
|
||||
def _capture_env(self, sandbox_id: str, learner_id: str, task_id: str) -> dict[str, str]:
|
||||
"""Env baked for the in-sandbox capture agent (REQ-3-003).
|
||||
|
||||
The agent joins the sandbox mount namespace but NOT its (offline)
|
||||
network namespace, so it reaches this service over loopback
|
||||
(`telemetry_ingest_host`, A-004 port). v0.4.7: the URL carries an
|
||||
HMAC ingest token (P1) — the WS endpoint rejects dials without a
|
||||
valid one, so LAN peers cannot poison traces with forged identity
|
||||
params.
|
||||
"""
|
||||
from urllib.parse import urlencode
|
||||
|
||||
token = self._ingest_token.mint(learner_id, task_id, sandbox_id)
|
||||
query = urlencode(
|
||||
{
|
||||
"learner_id": learner_id,
|
||||
"task_id": task_id,
|
||||
"sandbox_id": sandbox_id,
|
||||
"token": token,
|
||||
}
|
||||
)
|
||||
ingest_url = (
|
||||
f"ws://{self._settings.telemetry_ingest_host}:{self._settings.port}"
|
||||
f"/v1/telemetry/ingest?{query}"
|
||||
)
|
||||
return {
|
||||
"NC_LEARNER_ID": learner_id,
|
||||
"NC_TASK_ID": task_id,
|
||||
"NC_SANDBOX_ID": sandbox_id,
|
||||
"NC_INGEST_URL": ingest_url,
|
||||
}
|
||||
|
||||
async def list(self) -> list[SandboxHandleInfo]:
|
||||
"""All live sandboxes (idle + busy; the backend has no busy flag)."""
|
||||
async with self._lock:
|
||||
@@ -198,6 +263,7 @@ class SandboxManager:
|
||||
async with self._lock:
|
||||
handle = self._handles.pop(sandbox_id, None)
|
||||
learner_id = self._learner_ids.pop(sandbox_id, "unknown")
|
||||
self._task_ids.pop(sandbox_id, None)
|
||||
if handle is not None:
|
||||
await self._backend.destroy(handle)
|
||||
logger.info(
|
||||
@@ -216,7 +282,7 @@ class SandboxManager:
|
||||
|
||||
# -- reapers ------------------------------------------------------------
|
||||
|
||||
async def reap_expired(self) -> list[str]:
|
||||
async def reap_expired(self) -> builtins.list[str]:
|
||||
"""One reaper pass: wall-clock timeout + G-2 workdir-size sweep.
|
||||
|
||||
Destroys sandboxes older than `settings.sandbox_timeout_s`, then sweeps
|
||||
@@ -225,9 +291,14 @@ class SandboxManager:
|
||||
signal). Returns the ids destroyed this pass. Invoke on an async timer
|
||||
(the app lifespan owns the loop interval); both checks deliberately
|
||||
share one pass so the periodic work is O(live sandboxes) once.
|
||||
|
||||
P5: the return annotation is `builtins.list` because this class's
|
||||
own `list()` method shadows the `list[...]` subscript form in
|
||||
annotations below it (a mypy scoping quirk — the method name is
|
||||
public API and stays).
|
||||
"""
|
||||
now = self._clock()
|
||||
destroyed: list[str] = []
|
||||
destroyed: builtins.list[str] = []
|
||||
timeout_s = float(self._settings.sandbox_timeout_s)
|
||||
cap_bytes = int(self._settings.sandbox_max_workdir_mb) * 1024 * 1024
|
||||
|
||||
|
||||
@@ -1,24 +1,42 @@
|
||||
"""UnshareBackend — D-024 Linux-namespace sandboxing via util-linux `unshare`.
|
||||
|
||||
Each `exec` spawns:
|
||||
Two execution modes share one backend:
|
||||
|
||||
unshare --user --map-root-user --mount --pid --fork --net sh -c '<shim>'
|
||||
1. Pure shell sandbox (`task_id is None`, REQ-3-001): isolation is established
|
||||
PER-EXEC — every `exec` spawns a fresh namespace:
|
||||
|
||||
The in-namespace shim (this util-linux build, 2.38, has no `unshare --bind`,
|
||||
so the bind happens as the first mount op inside the namespace) is:
|
||||
unshare --user --map-root-user --mount --pid --fork --net sh -c '<shim>'
|
||||
|
||||
mount -t tmpfs tmpfs /tmp # private scratch, discarded on exit
|
||||
mkdir -p /tmp/work
|
||||
mount --bind <host workspace> /tmp/work
|
||||
cd /tmp/work
|
||||
ulimit -v/-t/-f … # applied AFTER the bind, so rlimits
|
||||
exec <cmd> # constrain the PAYLOAD, not unshare
|
||||
There is no persistent process; the in-namespace shim is:
|
||||
|
||||
Guarantees after this shim:
|
||||
* uid 0 inside (mapped to the unprivileged host UID outside)
|
||||
* no network: the fresh net namespace has no `lo` and no veth — zero links
|
||||
* writes under `/work` land in the per-sandbox host workspace dir
|
||||
* rlimits (RLIMIT_AS / RLIMIT_CPU / RLIMIT_FSIZE) constrain the payload only
|
||||
mount -t tmpfs tmpfs /tmp # private scratch, discarded on exit
|
||||
mkdir -p /tmp/work
|
||||
mount --bind <host workspace> /tmp/work
|
||||
cd /tmp/work
|
||||
ulimit -v/-t/-f … # applied AFTER the bind, so rlimits
|
||||
exec <cmd> # constrain the PAYLOAD, not unshare
|
||||
|
||||
2. Telemetry-wired task sandbox (REQ-3-003, `capture_env` set): a PERSISTENT,
|
||||
TRACKED topology so the stdlib capture agent can live inside the sandbox and
|
||||
still stream events to ai-service. Per exec a fresh OFFLINE namespace would
|
||||
leave the agent nowhere to run and (on this host, where a userns can't
|
||||
bring `lo` up) no loopback to reach `ws://127.0.0.1`. So spawn creates a
|
||||
long-lived helper (outer user+mount ns, ONLINE) and an inner sandbox
|
||||
(mount+pid+fork+net — OFFLINE), both rooted at a private `ns/` subtree:
|
||||
|
||||
helper : unshare --user --map-root-user --mount (mounts ns/ private)
|
||||
inner : unshare --mount --pid --fork --net (tmpfs on ns/, bind
|
||||
<workdir>/host/workspace -> <ns>/work) <- the sandbox
|
||||
exec : nsenter -t <inner sleep> -m -- sh -c … (joins inner mount ns;
|
||||
offline + pid-isolated, uid 0, writes land on the host workspace)
|
||||
agent : nsenter -t <inner sleep> -m -- python3 <agent> (joins the inner
|
||||
MOUNT ns only — NOT pid/net — so it watches the live workspace
|
||||
and stays ONLINE, reaching the app's WS ingest on loopback)
|
||||
|
||||
The agent is deliberately pid/net-exempt from the sandbox: it is OUR trusted
|
||||
capture process, and isolating its network would cut the very link it needs.
|
||||
`destroy` reaps agent → inner → helper (in that order). The handle's `pid`
|
||||
is the agent's host pid (None for a pure shell sandbox).
|
||||
|
||||
Why rlimits are applied in the shim, not Python's preexec_fn: setting
|
||||
RLIMIT_AS on the *unshare* process itself can trip the memory ceiling on the
|
||||
@@ -30,16 +48,18 @@ Containment honesty (D-024 / G-1): a user namespace is NOT a write barrier.
|
||||
Writes made OUTSIDE the bind fall through to host paths, and because inner
|
||||
uid 0 maps to the invoking host uid, a sandboxed process can write anywhere
|
||||
that host uid can write. Isolation here is: private PIDs/MNT/NET/UTS, tmpfs
|
||||
scratch at /tmp, payload rlimits, and a uid map yielding no privilege the
|
||||
host uid did not already have. A per-sandbox runtime uid (D-025) is the
|
||||
follow-up that hardens DAC.
|
||||
scratch, payload rlimits, and a uid map yielding no privilege the host uid
|
||||
did not already have. A per-sandbox runtime uid (D-025) is the follow-up that
|
||||
hardens DAC.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import shlex
|
||||
import shutil
|
||||
import signal
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
@@ -48,22 +68,27 @@ from .backend import ExecResult, ResourceLimits, SandboxHandle, SandboxSpec
|
||||
from .workdir import create_layout
|
||||
from .workdir import snapshot as workdir_snapshot
|
||||
|
||||
#: Args shared by every namespace we spawn (D-024). No `unshare --bind` on
|
||||
#: util-linux 2.38 — the bind is done from inside the namespace instead.
|
||||
#: Args shared by every PURE shell namespace we spawn (D-024). No
|
||||
#: `unshare --bind` on util-linux 2.38 — the bind is done from inside instead.
|
||||
UNSHARE_ARGS: tuple[str, ...] = (
|
||||
"--user", # new user namespace …
|
||||
"--map-root-user", # … in which we are uid 0 (mapped to host uid outside)
|
||||
"--mount", # private mount table
|
||||
"--pid", # private PID table
|
||||
"--fork", # child is PID 1 in its namespace (reaps zombies, gets signals)
|
||||
"--net", # fresh net namespace: no lo, no veth → fully offline
|
||||
"--net", # fresh net namespace: no usable route → effectively offline
|
||||
)
|
||||
|
||||
IN_NS_WORKDIR = "/tmp/work" # where the workspace is bound inside the namespace
|
||||
IN_NS_WORKDIR = "/tmp/work" # where the workspace is bound inside a pure shell ns
|
||||
|
||||
#: Sentinels the long-lived namespace supervisors print once their mounts are
|
||||
#: laid out. exec()/the manager must not run before the bind exists.
|
||||
_HELPER_READY = "NC_HELPER_READY"
|
||||
_INNER_READY = "NC_INNER_READY"
|
||||
|
||||
|
||||
class SandboxUnavailableError(RuntimeError):
|
||||
"""`unshare` missing or user namespaces blocked on this host."""
|
||||
"""`unshare`/`nsenter` missing or user namespaces blocked on this host."""
|
||||
|
||||
|
||||
def _build_shim(workspace: Path, limits: ResourceLimits, cmd: list[str]) -> str:
|
||||
@@ -90,35 +115,286 @@ def _build_shim(workspace: Path, limits: ResourceLimits, cmd: list[str]) -> str:
|
||||
)
|
||||
|
||||
|
||||
class _Tracked:
|
||||
"""The process tree + paths for one persistent (telemetry-wired) sandbox."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
helper: asyncio.subprocess.Process,
|
||||
inner: asyncio.subprocess.Process,
|
||||
agent: asyncio.subprocess.Process | None,
|
||||
inner_pid: int, # host pid of the SANDBOXED init (sleep) — ns enter target
|
||||
host_dir: Path,
|
||||
workspace: Path,
|
||||
ns_root: Path,
|
||||
ns_workdir: Path,
|
||||
) -> None:
|
||||
self.helper = helper
|
||||
self.inner = inner
|
||||
self.agent = agent
|
||||
self.inner_pid = inner_pid
|
||||
self.host_dir = host_dir
|
||||
self.workspace = workspace
|
||||
self.ns_root = ns_root
|
||||
self.ns_workdir = ns_workdir
|
||||
|
||||
|
||||
class UnshareBackend: # satisfies SandboxBackend structurally (Protocol)
|
||||
"""D-024 backend: subprocess-per-exec inside fresh Linux namespaces."""
|
||||
"""D-024 backend: namespace subprocesses; persistent tree for task sandboxes."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
unshare_path: str | None = None,
|
||||
limits: ResourceLimits | None = None, # per-spec override lands in 1-04
|
||||
nsenter_path: str | None = None,
|
||||
agent_script: Path | None = None,
|
||||
) -> None:
|
||||
self._unshare = unshare_path or shutil.which("unshare") or "unshare"
|
||||
self._nsenter = nsenter_path or shutil.which("nsenter") or "nsenter"
|
||||
self._limits = limits or ResourceLimits()
|
||||
# The stdlib-only capture agent script, copied into each tracked
|
||||
# workdir's host/ tree so nsenter can reach it inside the sandbox.
|
||||
# ai_service/sandbox/unshare_backend.py -> parents[2] = apps/ai-service.
|
||||
self._agent_script = agent_script or (
|
||||
Path(__file__).resolve().parents[2] / "scripts" / "sandbox-agent.py"
|
||||
)
|
||||
# Tracked (persistent) sandboxes by id; pure shell sandboxes are absent.
|
||||
self._tracked: dict[str, _Tracked] = {}
|
||||
|
||||
# -- spawn ------------------------------------------------------------------
|
||||
|
||||
async def spawn(self, spec: SandboxSpec) -> SandboxHandle:
|
||||
"""Lay out the workdir and return a handle.
|
||||
"""Lay out the workdir; if `spec.capture_env` is set, start the sandbox.
|
||||
|
||||
Isolation is established per-`exec` (each exec = fresh namespaces), so
|
||||
spawn only prepares on-disk state; there is no long-lived init process.
|
||||
A spec WITHOUT capture_env keeps REQ-3-001 semantics: spawn only
|
||||
prepares disk state and each exec forks a fresh (offline) namespace.
|
||||
A spec WITH capture_env starts the persistent helper/inner tree and the
|
||||
capture agent, and `handle.pid` carries the agent's host pid.
|
||||
"""
|
||||
create_layout(spec)
|
||||
if not spec.capture_env:
|
||||
return SandboxHandle(
|
||||
id=spec.sandbox_id,
|
||||
pid=None, # no persistent process; each exec forks short-lived PIDs
|
||||
workdir=spec.workdir,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
tracked = await self._spawn_tracked(spec)
|
||||
self._tracked[spec.sandbox_id] = tracked
|
||||
return SandboxHandle(
|
||||
id=spec.sandbox_id,
|
||||
pid=None, # no persistent process; each exec forks short-lived PIDs
|
||||
pid=tracked.agent.pid if tracked.agent is not None else tracked.inner_pid,
|
||||
workdir=spec.workdir,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
|
||||
async def _spawn_tracked(self, spec: SandboxSpec) -> _Tracked:
|
||||
"""Bring up helper + inner + agent for a telemetry-wired task sandbox."""
|
||||
host_dir = spec.workdir / "host"
|
||||
workspace = host_dir / "workspace"
|
||||
ns_root = host_dir / "ns"
|
||||
ns_workdir = ns_root / "work"
|
||||
for d in (workspace, ns_root):
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
# The agent script must live INSIDE the workspace: the inner ns bind
|
||||
# mounts <host_dir>/workspace -> <ns_root>/work, so only workspace
|
||||
# content is visible in-namespace at /work.
|
||||
agent_host_path = workspace / "sandbox-agent.py"
|
||||
shutil.copyfile(self._agent_script, agent_host_path)
|
||||
|
||||
helper = await self._launch_ns(
|
||||
[
|
||||
self._unshare,
|
||||
"--user",
|
||||
"--map-root-user",
|
||||
"--mount",
|
||||
"sh",
|
||||
"-c",
|
||||
(
|
||||
# Isolate ns/ so the inner tmpfs never propagates back to the
|
||||
# host mount table (make-private is best-effort on this host).
|
||||
f"mount --bind {shlex.quote(str(ns_root))} {shlex.quote(str(ns_root))}; "
|
||||
f"mount --make-private {shlex.quote(str(ns_root))} 2>/dev/null; "
|
||||
f"echo {_HELPER_READY}; exec sleep 3600"
|
||||
),
|
||||
],
|
||||
sentinel=_HELPER_READY,
|
||||
label="helper",
|
||||
)
|
||||
try:
|
||||
inner = await self._launch_ns(
|
||||
[
|
||||
*self._helper_join_argv(helper),
|
||||
self._unshare,
|
||||
"--mount",
|
||||
"--pid",
|
||||
"--fork",
|
||||
"--net",
|
||||
"sh",
|
||||
"-c",
|
||||
(
|
||||
f"mount -t tmpfs tmpfs {shlex.quote(str(ns_root))}; "
|
||||
f"mkdir -p {shlex.quote(str(ns_workdir))}; "
|
||||
f"mount --bind {shlex.quote(str(workspace))} "
|
||||
f"{shlex.quote(str(ns_workdir))}; "
|
||||
f"echo {_INNER_READY}; exec sleep 3600"
|
||||
),
|
||||
],
|
||||
sentinel=_INNER_READY,
|
||||
label="inner",
|
||||
)
|
||||
except Exception:
|
||||
await self._reap(helper)
|
||||
raise
|
||||
|
||||
await asyncio.sleep(0) # let the inner child's sleep fork settle
|
||||
inner_pid = await asyncio.to_thread(self._find_child_pid, inner.pid)
|
||||
if inner_pid is None:
|
||||
await self._reap(inner)
|
||||
await self._reap(helper)
|
||||
raise SandboxUnavailableError(
|
||||
f"could not resolve sandboxed init pid for {spec.sandbox_id}"
|
||||
)
|
||||
|
||||
tracked = _Tracked(
|
||||
helper=helper,
|
||||
inner=inner,
|
||||
agent=None,
|
||||
inner_pid=inner_pid,
|
||||
host_dir=host_dir,
|
||||
workspace=workspace,
|
||||
ns_root=ns_root,
|
||||
ns_workdir=ns_workdir,
|
||||
)
|
||||
if spec.capture_env:
|
||||
tracked.agent = await self._launch_agent(spec, tracked, agent_host_path)
|
||||
return tracked
|
||||
|
||||
# -- process launch helpers --------------------------------------------------
|
||||
|
||||
def _helper_join_argv(self, helper: asyncio.subprocess.Process) -> list[str]:
|
||||
"""nsenter argv that runs a command inside the helper's user+mount ns."""
|
||||
if helper.pid is None:
|
||||
raise SandboxUnavailableError("helper namespace process is not running")
|
||||
return [
|
||||
self._nsenter,
|
||||
"-t",
|
||||
str(helper.pid),
|
||||
"-m",
|
||||
"-U",
|
||||
"--preserve-credentials",
|
||||
"--",
|
||||
]
|
||||
|
||||
def _sandbox_join_argv(self, tracked: _Tracked) -> list[str]:
|
||||
"""nsenter argv that joins the inner sandbox MOUNT namespace (uid 0)."""
|
||||
return [self._nsenter, "-t", str(tracked.inner_pid), "-m", "--"]
|
||||
|
||||
async def _launch_agent(
|
||||
self, spec: SandboxSpec, tracked: _Tracked, agent_host_path: Path
|
||||
) -> asyncio.subprocess.Process:
|
||||
"""Launch the capture agent: joins the sandbox mount ns, NOT pid/net.
|
||||
|
||||
The nsenter chain swaps the mount table under the process, so a HOST
|
||||
cwd/relative path is invalid after the join (observed: python3
|
||||
resolved ``sandbox-agent.py`` against a stale root → ``//…`` and
|
||||
exited rc=2). The launch therefore happens through ``sh -c`` INSIDE
|
||||
the joined namespace, using only in-namespace absolute paths: the
|
||||
workspace is bind-mounted at ``<ns_root>/work``, the agent script was
|
||||
copied into the host workspace, so ``/work/sandbox-agent.py`` exists
|
||||
after the join. The agent runs ONLINE (joins mount ns only, not the
|
||||
offline net ns) so it can dial the ai-service WS ingest loopback.
|
||||
"""
|
||||
env = dict(spec.capture_env or {})
|
||||
in_ns_script = f"{tracked.ns_root / 'work' / 'sandbox-agent.py'}"
|
||||
in_ns_cwd = f"{tracked.ns_root / 'work'}"
|
||||
launch = f"cd {shlex.quote(in_ns_cwd)} && exec python3 {shlex.quote(in_ns_script)}"
|
||||
try:
|
||||
return await asyncio.create_subprocess_exec(
|
||||
*self._helper_join_argv(tracked.helper),
|
||||
*self._sandbox_join_argv(tracked),
|
||||
"sh",
|
||||
"-c",
|
||||
launch,
|
||||
stdin=asyncio.subprocess.DEVNULL,
|
||||
stdout=asyncio.subprocess.DEVNULL,
|
||||
stderr=asyncio.subprocess.DEVNULL,
|
||||
env=env,
|
||||
)
|
||||
except FileNotFoundError as exc: # pragma: no cover - env-dependent
|
||||
raise SandboxUnavailableError("python3 unavailable for capture agent") from exc
|
||||
|
||||
async def _launch_ns(
|
||||
self, argv: list[str], *, sentinel: str, label: str
|
||||
) -> asyncio.subprocess.Process:
|
||||
"""Spawn a namespace supervisor and wait for its `sentinel` line."""
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*argv,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.STDOUT,
|
||||
)
|
||||
|
||||
async def _wait_ready() -> None:
|
||||
if proc.stdout is None: # pragma: no cover (stdout is a PIPE)
|
||||
raise SandboxUnavailableError(f"{label} namespace missing stdout pipe")
|
||||
async for raw in proc.stdout:
|
||||
if raw.decode(errors="replace").strip() == sentinel:
|
||||
return
|
||||
raise SandboxUnavailableError(
|
||||
f"{label} namespace exited before signalling readiness: {argv[:3]}"
|
||||
)
|
||||
|
||||
try:
|
||||
await asyncio.wait_for(_wait_ready(), timeout=10.0)
|
||||
except TimeoutError as exc:
|
||||
await self._reap(proc)
|
||||
raise SandboxUnavailableError(
|
||||
f"{label} namespace never became ready (timeout): {argv[:3]}"
|
||||
) from exc
|
||||
except SandboxUnavailableError:
|
||||
await self._reap(proc)
|
||||
raise
|
||||
return proc
|
||||
|
||||
@staticmethod
|
||||
def _find_child_pid(parent_pid: int | None) -> int | None:
|
||||
"""First direct child of `parent_pid` (the pid-namespaced `sleep`).
|
||||
|
||||
The helper→unshare shim is inner.pid's parent chain head, but the
|
||||
SANDBOXED mount/pid namespaces belong to its forked child (the
|
||||
`sleep`). nsenter must target THAT pid to land inside the sandbox.
|
||||
Reads /proc directly — best-effort, host-local, no subprocess.
|
||||
"""
|
||||
if parent_pid is None:
|
||||
return None
|
||||
for entry in os.listdir("/proc"):
|
||||
if not entry.isdigit():
|
||||
continue
|
||||
try:
|
||||
with open(f"/proc/{entry}/stat") as fh:
|
||||
# ppid is field 4; comm (field 2) may contain spaces, so
|
||||
# parse relative to the LAST ')'.
|
||||
rest = fh.read().rsplit(") ", 1)[1].split()
|
||||
if int(rest[1]) == parent_pid: # state=rest[0], ppid=rest[1]
|
||||
return int(entry)
|
||||
except (OSError, IndexError, ValueError):
|
||||
continue
|
||||
return None
|
||||
|
||||
# -- exec --------------------------------------------------------------------
|
||||
|
||||
async def exec(self, handle: SandboxHandle, cmd: list[str]) -> ExecResult:
|
||||
"""Run `cmd` in a fresh namespace rooted at the sandbox workspace."""
|
||||
"""Run `cmd` in the sandbox workspace (cwd = the bound workspace)."""
|
||||
if not cmd:
|
||||
raise ValueError("exec requires a non-empty cmd")
|
||||
tracked = self._tracked.get(handle.id)
|
||||
if tracked is not None:
|
||||
return await self._exec_tracked(handle, tracked, cmd)
|
||||
return await self._exec_fresh(handle, cmd)
|
||||
|
||||
async def _exec_fresh(self, handle: SandboxHandle, cmd: list[str]) -> ExecResult:
|
||||
"""Pure shell sandbox: spawn one fresh offline namespace per exec."""
|
||||
workspace = handle.workdir / "workspace"
|
||||
if not workspace.is_dir():
|
||||
raise SandboxUnavailableError(f"spawn() first: no workspace at {workspace}")
|
||||
@@ -129,7 +405,6 @@ class UnshareBackend: # satisfies SandboxBackend structurally (Protocol)
|
||||
"-c",
|
||||
_build_shim(workspace, self._limits, cmd),
|
||||
]
|
||||
|
||||
started = time.monotonic()
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*argv,
|
||||
@@ -146,13 +421,128 @@ class UnshareBackend: # satisfies SandboxBackend structurally (Protocol)
|
||||
duration_s=time.monotonic() - started,
|
||||
)
|
||||
|
||||
async def _exec_tracked(
|
||||
self, handle: SandboxHandle, tracked: _Tracked, cmd: list[str]
|
||||
) -> ExecResult:
|
||||
"""Task sandbox: join the persistent inner namespace (offline, uid 0).
|
||||
|
||||
rlimits apply in the joining subshell so only the payload is limited;
|
||||
cwd is the bound workspace (`<ns>/work`).
|
||||
"""
|
||||
if tracked.inner.returncode is not None:
|
||||
raise SandboxUnavailableError(
|
||||
f"sandbox {handle.id} is not running (inner namespace exited)"
|
||||
)
|
||||
quoted_cmd = " ".join(shlex.quote(part) for part in cmd)
|
||||
rlimit_prefix = (
|
||||
f"ulimit -v {self._limits.memory_bytes // 1024}; "
|
||||
f"ulimit -t {self._limits.cpu_seconds}; "
|
||||
f"ulimit -f {self._limits.file_size_bytes // 512}; "
|
||||
)
|
||||
shell = (
|
||||
f"cd {shlex.quote(str(tracked.ns_workdir))}; "
|
||||
f"{rlimit_prefix}"
|
||||
f"exec {quoted_cmd}"
|
||||
)
|
||||
argv = [
|
||||
*self._helper_join_argv(tracked.helper),
|
||||
*self._sandbox_join_argv(tracked),
|
||||
"sh",
|
||||
"-c",
|
||||
shell,
|
||||
]
|
||||
started = time.monotonic()
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*argv,
|
||||
stdin=asyncio.subprocess.DEVNULL,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
out, err = await proc.communicate()
|
||||
return ExecResult(
|
||||
cmd=cmd,
|
||||
returncode=proc.returncode if proc.returncode is not None else -1,
|
||||
stdout=out.decode(errors="replace"),
|
||||
stderr=err.decode(errors="replace"),
|
||||
duration_s=time.monotonic() - started,
|
||||
)
|
||||
|
||||
# -- snapshot / destroy --------------------------------------------------------
|
||||
|
||||
async def snapshot(self, handle: SandboxHandle) -> Path:
|
||||
return workdir_snapshot(handle.workdir)
|
||||
workspace_root = handle.workdir
|
||||
tracked = self._tracked.get(handle.id)
|
||||
if tracked is not None:
|
||||
# Copy the tracked workspace, not the legacy <workdir>/workspace.
|
||||
dest_parent = handle.workdir / "snapshots"
|
||||
dest_parent.mkdir(parents=True, exist_ok=True)
|
||||
return workdir_snapshot_from_workspace(tracked.workspace, dest_parent)
|
||||
return workdir_snapshot(workspace_root)
|
||||
|
||||
async def destroy(self, handle: SandboxHandle) -> None:
|
||||
"""Best-effort teardown. Namespaces die with their process; nothing to kill.
|
||||
"""Best-effort teardown. Pure shell sandboxes die with their exec; for a
|
||||
tracked task sandbox reap AGENT → INNER → HELPER so no capture process
|
||||
or namespace supervisor outlives the handle (REQ-3-003 lifecycle).
|
||||
|
||||
Keeping the workdir is deliberate: snapshots must survive destroy so a
|
||||
learner's last state can be restored by the manager layer.
|
||||
"""
|
||||
tracked = self._tracked.pop(handle.id, None)
|
||||
if tracked is not None:
|
||||
# Agent first (it must not flush a "stopped" event into a dead
|
||||
# sandbox), then the namespace tree. The inner `unshare --fork`
|
||||
# shim is NOT the namespace init: killing it orphans its child
|
||||
# (the `sleep` that is PID 1 of the sandbox pid+mnt+net ns),
|
||||
# which reparents to host init and holds the tmpfs + bind for
|
||||
# a full hour (observed: ~30 leaked `sleep 3600` after a test
|
||||
# run). `--kill-child` does not reach it either (util-linux
|
||||
# 2.38 leaks the same child under this flag combo — the child
|
||||
# is reparented before unshare's signal handler runs). The
|
||||
# deterministic kill is SIGKILL on the ns-init's HOST pid,
|
||||
# which we already track as `tracked.inner_pid` (nsenter uses
|
||||
# it for exec); the kernel then tears down the namespace with
|
||||
# its init (no processes remain).
|
||||
for proc in (tracked.agent, tracked.inner, tracked.helper):
|
||||
if proc is not None:
|
||||
await self._reap(proc)
|
||||
self._kill_pid(tracked.inner_pid)
|
||||
handle.pid = None
|
||||
|
||||
@staticmethod
|
||||
def _kill_pid(pid: int | None, sig: int = signal.SIGKILL) -> None:
|
||||
"""Best-effort host-side signal; pid recycled or gone is not an error."""
|
||||
if pid is None:
|
||||
return
|
||||
try:
|
||||
os.kill(pid, sig)
|
||||
except (ProcessLookupError, PermissionError):
|
||||
pass # already dead, or not ours — nothing to do
|
||||
|
||||
@staticmethod
|
||||
async def _reap(proc: asyncio.subprocess.Process) -> None:
|
||||
"""SIGTERM then SIGKILL, tolerant of an already-dead process."""
|
||||
if proc.returncode is not None:
|
||||
return
|
||||
try:
|
||||
proc.terminate()
|
||||
except ProcessLookupError:
|
||||
return
|
||||
try:
|
||||
await asyncio.wait_for(proc.wait(), timeout=5.0)
|
||||
except TimeoutError:
|
||||
try:
|
||||
proc.kill()
|
||||
except ProcessLookupError:
|
||||
return
|
||||
try:
|
||||
await asyncio.wait_for(proc.wait(), timeout=5.0)
|
||||
except TimeoutError: # pragma: no cover - SIGKILL always wins
|
||||
pass
|
||||
|
||||
|
||||
def workdir_snapshot_from_workspace(workspace: Path, snapshots_dir: Path) -> Path:
|
||||
"""Snapshot helper for tracked sandboxes whose workspace is `<workdir>/host/workspace`
|
||||
instead of the legacy `<workdir>/workspace` layout."""
|
||||
dest = snapshots_dir / datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
|
||||
shutil.copytree(workspace, dest, symlinks=False)
|
||||
return dest
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Live build telemetry — event models and the TraceStore protocol (REQ-3-003).
|
||||
|
||||
Boundary rule (D-027): telemetry/ imports from config only — never from
|
||||
agents/ or api/ (agents call engines through narrow interfaces, never
|
||||
the reverse; api/ composes stores via DI).
|
||||
"""
|
||||
|
||||
from .ingest import IngestSession, TraceIntegrityMap, telemetry_ingest_endpoint
|
||||
from .models import EventKind, TelemetryEvent, TraceSpan
|
||||
from .store import SQLiteTraceStore, TraceStore
|
||||
|
||||
__all__ = [
|
||||
"EventKind",
|
||||
"IngestSession",
|
||||
"SQLiteTraceStore",
|
||||
"TelemetryEvent",
|
||||
"TraceIntegrityMap",
|
||||
"TraceSpan",
|
||||
"TraceStore",
|
||||
"telemetry_ingest_endpoint",
|
||||
]
|
||||
@@ -0,0 +1,458 @@
|
||||
"""WS ingest protocol for learner telemetry (REQ-3-003, D-026, G-3).
|
||||
|
||||
Frame contract — trace identity travels as QUERY PARAMS on the WS upgrade
|
||||
(`WS /v1/telemetry/ingest?learner_id=...&task_id=...&sandbox_id=...`), NOT as
|
||||
a first init frame. Rationale: the in-sandbox capture agent (Task 2-2-01) is a
|
||||
stdlib-only RFC6455 client where the URL is the cheapest thing to parametrize
|
||||
(`NC_INGEST_URL` carries the query string); identity is also visible to the
|
||||
server BEFORE accept(), so a malformed handshake can be rejected without an
|
||||
accept/close round-trip. Client messages are then ONE event per JSON text
|
||||
frame — no envelope:
|
||||
|
||||
{"seq": 0, "kind": "command", "payload": {...}, "ts": "...",
|
||||
"sandbox_id": "..."} # learner_id / task_id forbidden (URL owns them)
|
||||
|
||||
Server → client frames are typed status envelopes:
|
||||
|
||||
{"type": "ack_total", "count": N} — final flush summary, then close 1000
|
||||
{"type": "seq_ack", "seq": N} — advisory: durable latest_seq after
|
||||
each successful append (D-045; the
|
||||
agent trims its spool to seq > ack)
|
||||
{"type": "gap_warning", "missing_seqs": [...]} — seq skipped ahead
|
||||
{"type": "event_rejected", "detail": "..."} — one frame failed validation
|
||||
(seq echoed when parseable)
|
||||
{"type": "event_rejected", "seq": N, "detail": "..."} — stored-field rejected (bad kind)
|
||||
{"type": "flooded", "reason": "cap_exceeded"|"queue_overflow",
|
||||
"count": N} — sent before close(1008)
|
||||
|
||||
Keepalive: the server sends an opaque ping frame every `PING_INTERVAL_S` (the
|
||||
capture agent auto-pongs at the frame layer); a peer that is silent past
|
||||
`PONG_TIMEOUT_S` is assumed wedged, but the keepalive half only LOGS — the
|
||||
receiver half owns disconnect detection (single-box pilot: TCP EOF is
|
||||
reliable; an aggressive pong-watchdog would false-positive on loaded boxes).
|
||||
|
||||
Flood control (GRILL G-3, BINDING — silent drop-oldest is FORBIDDEN):
|
||||
* per-connection inbound queue bounded at `INBOUND_QUEUE_MAX` frames; on
|
||||
overflow → close code 1008 (policy violation) + trace marked
|
||||
INCOMPLETE_FLOODED via `TraceIntegrityMap`.
|
||||
* total events for the (learner, task) exceeding
|
||||
`Settings.telemetry_max_events_per_task` → same 1008 + INCOMPLETE_FLOODED.
|
||||
`INCOMPLETE_FLOODED` is an integrity signal Proctor/Phase-3 grader read via
|
||||
`TraceIntegrityMap.is_incomplete()` (the G-4 gate): a flooded trace can
|
||||
never yield a credential.
|
||||
|
||||
Boundary (D-027): telemetry/ never imports agents/ or api/. This module
|
||||
imports only `fastapi.WebSocket` for the socket type (a protocol surface, not
|
||||
a DI framework); the session engine below depends only on the TraceStore
|
||||
protocol + Settings, and api/telemetry.py injects both through plain
|
||||
parameters.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from typing import Any, Final
|
||||
|
||||
from fastapi import WebSocket, WebSocketDisconnect
|
||||
from pydantic import BaseModel, ConfigDict, Field, ValidationError
|
||||
|
||||
from ..config import Settings
|
||||
from .models import TelemetryEvent
|
||||
from .store import TraceStore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: WebSocket close code 1008 — policy violation (RFC 6455 §7.4.1).
|
||||
WS_CLOSE_POLICY_VIOLATION: Final = 1008
|
||||
|
||||
#: Bounded inbound queue depth per connection (G-3). Sized for burst-tolerance
|
||||
#: well above the capture agent's emission rate; overflow is a flood signal,
|
||||
#: not a backpressure knob.
|
||||
INBOUND_QUEUE_MAX: Final = 256
|
||||
|
||||
PING_INTERVAL_S: Final = 20.0
|
||||
|
||||
|
||||
class InboundEventFrame(BaseModel):
|
||||
"""Client → server event frame (one TelemetryEvent minus URL-owned ids).
|
||||
|
||||
`extra="forbid"`: learner_id/task_id arriving in the frame body is a
|
||||
contract violation — identity comes from the query params only, so a
|
||||
replayed frame can never lie about which trace it belongs to.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
seq: int = Field(ge=0)
|
||||
kind: str = Field(min_length=1)
|
||||
payload: dict[str, Any] = Field(default_factory=dict)
|
||||
ts: datetime
|
||||
sandbox_id: str = ""
|
||||
|
||||
|
||||
class TraceIntegrityMap:
|
||||
"""Integrity flags for traces that can never be graded (G-3/G-4).
|
||||
|
||||
Process-local and deliberately small: v0.3 runs ONE ai-service process per
|
||||
box, and the Phase-3 grader reads this flag through the same DI container
|
||||
— D-019-style in-memory registry precedent (the sandbox handle registry is
|
||||
the same shape). The flag is terminal within the process: a reconnect
|
||||
sending legal events does NOT clear it — the trace is already untrusted as
|
||||
grading input. Restarting ai-service resets flags; grading runs against a
|
||||
live service, and the SQLite trace rows themselves are durable.
|
||||
|
||||
All methods are sync: mutation is a dict write, reads are dict lookups —
|
||||
no await needed, so callers from any layer (API handlers, the grader)
|
||||
don't inherit an async surface for a nanosecond operation.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
# (learner_id, task_id) -> machine-readable reason (INCOMPLETE_FLOODED)
|
||||
self._flags: dict[tuple[str, str], str] = {}
|
||||
|
||||
def mark(self, learner_id: str, task_id: str, reason: str) -> None:
|
||||
"""Set an integrity flag. Presence of the flag is the signal; the
|
||||
reason is informational (last write wins)."""
|
||||
self._flags[(learner_id, task_id)] = reason
|
||||
|
||||
def clear(self, learner_id: str, task_id: str) -> None:
|
||||
"""Test seam: reset a flag (production ingest never clears)."""
|
||||
self._flags.pop((learner_id, task_id), None)
|
||||
|
||||
def is_incomplete(self, learner_id: str, task_id: str) -> bool:
|
||||
"""True when the trace carries ANY terminal integrity flag."""
|
||||
return (learner_id, task_id) in self._flags
|
||||
|
||||
def reason(self, learner_id: str, task_id: str) -> str | None:
|
||||
"""The flag's reason (INCOMPLETE_FLOODED), or None when unflagged."""
|
||||
return self._flags.get((learner_id, task_id))
|
||||
|
||||
|
||||
class IngestSession:
|
||||
"""One WebSocket ingest connection: receive → queue → drain → store.
|
||||
|
||||
Two tasks per connection:
|
||||
* `_receiver` — reads frames, validates shape, enqueues (bounded queue,
|
||||
G-3). Receives never block on SQLite.
|
||||
* `_drainer` — pops frames in arrival order, appends via TraceStore
|
||||
(idempotent on (learner,task,seq)), emits gap warnings, enforces the
|
||||
per-trace event cap.
|
||||
Either task detecting a flood closes the WS with 1008 and marks the trace
|
||||
INCOMPLETE_FLOODED. The events queue carries `None` as the client-
|
||||
disconnect sentinel.
|
||||
|
||||
- `telemetry_max_events_per_task` is consulted at connect and re-checked
|
||||
per append against the DURABLE row count (cap compares against stored
|
||||
events, so a skipped-ahead seq cannot burn budget that was never sent).
|
||||
Durable count via `TraceStore.count()` (COUNT(*)) — a single aggregate
|
||||
per append, never materializing trace rows (the pre-P7 code read
|
||||
`len(get_trace(...))` which was O(trace) per event / O(n²) per session).
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
websocket: WebSocket,
|
||||
store: TraceStore,
|
||||
integrity: TraceIntegrityMap,
|
||||
settings: Settings,
|
||||
learner_id: str,
|
||||
task_id: str,
|
||||
sandbox_id: str,
|
||||
) -> None:
|
||||
self._ws = websocket
|
||||
self._store = store
|
||||
self._integrity = integrity
|
||||
# Snapshot of the one setting ingest consults: read once at connect so
|
||||
# a hot-reloaded Settings object mid-session can't move the cap.
|
||||
self._max_events = settings.telemetry_max_events_per_task
|
||||
self.learner_id = learner_id
|
||||
self.task_id = task_id
|
||||
self.sandbox_id = sandbox_id
|
||||
|
||||
self._queue: asyncio.Queue[InboundEventFrame | None] = asyncio.Queue(
|
||||
maxsize=INBOUND_QUEUE_MAX
|
||||
)
|
||||
self._seen: set[int] = set()
|
||||
self._next_expected: int | None = None # in-connection monotonic hint
|
||||
self._received = 0
|
||||
self._stored = 0
|
||||
self._deduped = 0
|
||||
self._rejected = 0
|
||||
self._flooded = False
|
||||
self._flood_reason = ""
|
||||
|
||||
# -- receive half ----------------------------------------------------------
|
||||
|
||||
async def run(self) -> None:
|
||||
"""Accept, run receiver+drainer, close cleanly. Owns the WS lifecycle."""
|
||||
await self._ws.accept()
|
||||
logger.info(
|
||||
"telemetry ingest connected: %s/%s sandbox=%s",
|
||||
self.learner_id,
|
||||
self.task_id,
|
||||
self.sandbox_id or "(none)",
|
||||
)
|
||||
pinger = asyncio.create_task(self._keepalive())
|
||||
receiver = asyncio.create_task(self._receiver())
|
||||
drainer = asyncio.create_task(self._drainer())
|
||||
# First terminal outcome shuts the session down: client disconnect
|
||||
# (receiver ends) → drainer flushes; drainer ended (clean close after
|
||||
# flush or a 1008 flood close) → receiver must not linger.
|
||||
pending: set[asyncio.Task[None]] = {receiver, drainer}
|
||||
try:
|
||||
done, pending = await asyncio.wait(
|
||||
pending, return_when=asyncio.FIRST_COMPLETED
|
||||
)
|
||||
if receiver in done and drainer in pending:
|
||||
try:
|
||||
await drainer # final flush → sends ack_total, close 1000
|
||||
finally:
|
||||
pending.discard(drainer)
|
||||
finally:
|
||||
for task in (pinger, *pending):
|
||||
task.cancel()
|
||||
with contextlib.suppress(asyncio.CancelledError):
|
||||
await task
|
||||
|
||||
async def _receiver(self) -> None:
|
||||
"""Read frames; parse+enqueue. Overflow → flood shutdown (G-3).
|
||||
|
||||
RuntimeError from receive_text is benign here: it fires when the
|
||||
socket was closed by the drainer (1008 flood close) while this task
|
||||
was parked in receive — a terminal condition, not a bug.
|
||||
"""
|
||||
try:
|
||||
while True:
|
||||
raw = await self._ws.receive_text()
|
||||
frame = self._parse(raw)
|
||||
if frame is None:
|
||||
# Rejected frame — keep the connection open; the producer
|
||||
# gets an event_rejected status frame so a malformed batch
|
||||
# is visible (and its seq is never stored). Yield so the
|
||||
# status frame flushes before we block on the next receive.
|
||||
await self._reject_frame(raw)
|
||||
await asyncio.sleep(0)
|
||||
continue
|
||||
self._received += 1
|
||||
try:
|
||||
self._queue.put_nowait(frame)
|
||||
except asyncio.QueueFull:
|
||||
# Bounded queue — overflow is a flood, never drop-oldest.
|
||||
# _trigger_flood closes the socket; fall through to the
|
||||
# tail so the disconnect sentinel is still enqueued — the
|
||||
# drainer is never left parked on an empty queue after a
|
||||
# flood (P7 review: the pre-fix code `return`ed from the
|
||||
# QueueFull branch WITHOUT the sentinel, leaking the
|
||||
# session task set — one per flooded trace).
|
||||
await self._trigger_flood("queue_overflow")
|
||||
return
|
||||
except WebSocketDisconnect:
|
||||
pass
|
||||
except RuntimeError:
|
||||
logger.debug(
|
||||
"ingest receiver: socket already closed (flood path) %s/%s",
|
||||
self.learner_id,
|
||||
self.task_id,
|
||||
)
|
||||
# Client gone (clean close, drop, or flood close): sentinel unblocks
|
||||
# the drainer for a final flush. put_nowait can only fail under flood,
|
||||
# which already terminated the session.
|
||||
with contextlib.suppress(asyncio.QueueFull):
|
||||
self._queue.put_nowait(None)
|
||||
|
||||
def _parse(self, raw: str) -> InboundEventFrame | None:
|
||||
"""Validate one frame; None means malformed (caller rejects it)."""
|
||||
try:
|
||||
return InboundEventFrame.model_validate_json(raw)
|
||||
except ValidationError:
|
||||
return None
|
||||
|
||||
async def _reject_frame(self, raw: str) -> None:
|
||||
"""Malformed envelope: log + event_rejected status frame (never stored)."""
|
||||
self._rejected += 1
|
||||
detail = "invalid event frame"
|
||||
try:
|
||||
InboundEventFrame.model_validate_json(raw)
|
||||
except ValidationError as exc:
|
||||
detail = exc.errors()[0].get("msg", "validation error")
|
||||
logger.warning(
|
||||
"telemetry frame rejected: %s/%s: %s", self.learner_id, self.task_id, detail
|
||||
)
|
||||
seq: int | None = None
|
||||
with contextlib.suppress(Exception):
|
||||
seq = int(json.loads(raw).get("seq")) # best-effort echo for the producer
|
||||
payload: dict[str, Any] = {"type": "event_rejected", "detail": detail}
|
||||
if seq is not None:
|
||||
payload["seq"] = seq
|
||||
await self._send_json(payload)
|
||||
|
||||
# -- drain half --------------------------------------------------------------
|
||||
|
||||
async def _drainer(self) -> None:
|
||||
"""Pop queued frames, append to the store, then close 1000 + summary."""
|
||||
while True:
|
||||
frame = await self._queue.get()
|
||||
if frame is None: # disconnect sentinel → flush complete
|
||||
await self._send_json(
|
||||
{
|
||||
"type": "ack_total",
|
||||
"count": self._stored,
|
||||
"deduped": self._deduped,
|
||||
"rejected": self._rejected,
|
||||
}
|
||||
)
|
||||
with contextlib.suppress(RuntimeError, WebSocketDisconnect):
|
||||
await self._ws.close(code=1000)
|
||||
return
|
||||
await self._append(frame)
|
||||
|
||||
async def _append(self, frame: InboundEventFrame) -> None:
|
||||
# Precedence: a trace already flagged INCOMPLETE_FLOODED is terminal —
|
||||
# the connection that triggered it is being torn down, and any stray
|
||||
# queued frames must not resurrect the trace's intake.
|
||||
if self._integrity.is_incomplete(self.learner_id, self.task_id):
|
||||
await self._trigger_flood("already_flagged")
|
||||
return
|
||||
|
||||
# Per-trace cap (G-3): checked against the DURABLE row count so a
|
||||
# reconnect resumes the budget instead of resetting it, and a
|
||||
# skipped-ahead seq cannot burn budget that was never sent.
|
||||
if self._flood_breached():
|
||||
await self._trigger_flood("cap_exceeded")
|
||||
return
|
||||
|
||||
# TelemetryEvent's @validates hooks fire on CONSTRUCTION (setattr), so
|
||||
# the try must wrap building the model too — an unknown kind raises
|
||||
# before `store.append` is ever reached.
|
||||
event: TelemetryEvent
|
||||
before = self._store.latest_seq(self.learner_id, self.task_id)
|
||||
try:
|
||||
event = TelemetryEvent(
|
||||
learner_id=self.learner_id,
|
||||
task_id=self.task_id,
|
||||
seq=frame.seq,
|
||||
kind=frame.kind,
|
||||
payload=frame.payload,
|
||||
ts=frame.ts,
|
||||
sandbox_id=frame.sandbox_id or self.sandbox_id,
|
||||
)
|
||||
self._store.append(event)
|
||||
except ValueError as exc: # unknown kind / invalid field
|
||||
self._rejected += 1
|
||||
await self._send_json(
|
||||
{"type": "event_rejected", "seq": frame.seq, "detail": str(exc)}
|
||||
)
|
||||
return
|
||||
after = self._store.latest_seq(self.learner_id, self.task_id)
|
||||
|
||||
if after == before and frame.seq in self._seen:
|
||||
self._deduped += 1 # at-least-once retry; stored once (idempotent)
|
||||
else:
|
||||
self._stored += 1
|
||||
self._seen.add(frame.seq)
|
||||
# Seq-ack (D-045, REQ-5-007): advisory hint carrying the durable
|
||||
# latest_seq AFTER this append — the capture agent trims its spool to
|
||||
# seq > ack on receipt, bounding the replay margin to the in-flight
|
||||
# window. Emitted on dedup'd appends too (a-6) so a replay flush
|
||||
# tightens the margin immediately. Gap detection stays authoritative
|
||||
# (_check_gap below); G-3 flood semantics untouched.
|
||||
await self._send_json({"type": "seq_ack", "seq": after})
|
||||
await self._check_gap(frame.seq)
|
||||
# SQLite appends are sync and fast; on a burst the drainer can hold
|
||||
# the loop between receives. Yield so the WS writer flushes the close
|
||||
# and the pinger/interleave stay live under the eventlet-free portal.
|
||||
await asyncio.sleep(0)
|
||||
|
||||
def _flood_breached(self) -> bool:
|
||||
"""True when this append would exceed the per-trace event budget."""
|
||||
# Durable count (NOT latest_seq+1 — a skipped-ahead seq must not burn
|
||||
# un-sent events' budget) via COUNT(*): never materialize the trace
|
||||
# per append (P7 review — the old len(get_trace(...)) built every row
|
||||
# object per event, O(trace) per append / O(n²) per session).
|
||||
durable = self._store.count(learner_id=self.learner_id, task_id=self.task_id)
|
||||
return durable >= self._max_events
|
||||
|
||||
async def _check_gap(self, incoming_seq: int) -> None:
|
||||
"""Seq skipped ahead → log + per-connection gap_warning status frame."""
|
||||
if self._next_expected is not None and incoming_seq > self._next_expected:
|
||||
missing = list(range(self._next_expected, incoming_seq))
|
||||
logger.warning(
|
||||
"telemetry gap: %s/%s missing seqs %s (arrived seq=%d)",
|
||||
self.learner_id,
|
||||
self.task_id,
|
||||
missing,
|
||||
incoming_seq,
|
||||
)
|
||||
await self._send_json({"type": "gap_warning", "missing_seqs": missing})
|
||||
if self._next_expected is None or incoming_seq >= self._next_expected:
|
||||
self._next_expected = incoming_seq + 1
|
||||
|
||||
# -- flood + keepalive ------------------------------------------------------
|
||||
|
||||
async def _trigger_flood(self, reason: str) -> None:
|
||||
"""G-3: 1008 close + INCOMPLETE_FLOODED mark. Exactly once."""
|
||||
if self._flooded:
|
||||
return
|
||||
self._flooded = True
|
||||
self._flood_reason = reason
|
||||
self._integrity.mark(self.learner_id, self.task_id, "INCOMPLETE_FLOODED")
|
||||
logger.warning(
|
||||
"telemetry flood: %s/%s reason=%s — closing 1008, trace marked "
|
||||
"INCOMPLETE_FLOODED (G-3; Proctor/grade gate will refuse it)",
|
||||
self.learner_id,
|
||||
self.task_id,
|
||||
reason,
|
||||
)
|
||||
await self._send_json(
|
||||
{"type": "flooded", "reason": reason, "count": self._received}
|
||||
)
|
||||
with contextlib.suppress(RuntimeError, WebSocketDisconnect):
|
||||
await self._ws.close(
|
||||
code=WS_CLOSE_POLICY_VIOLATION,
|
||||
reason=f"telemetry flood control (G-3): {reason}",
|
||||
)
|
||||
|
||||
async def _keepalive(self) -> None:
|
||||
"""Protocol-level ping on an interval (agent auto-pongs at frame level).
|
||||
|
||||
A send failure means the socket is already gone — the receiver half
|
||||
independently surfaces the disconnect; we just stop pinging.
|
||||
"""
|
||||
while True:
|
||||
await asyncio.sleep(PING_INTERVAL_S)
|
||||
try:
|
||||
await self._ws.send_bytes(b"\x89ping-nextcraft")
|
||||
except (RuntimeError, WebSocketDisconnect):
|
||||
return
|
||||
|
||||
async def _send_json(self, payload: dict[str, Any]) -> None:
|
||||
"""Best-effort status frame; the socket may already be gone."""
|
||||
with contextlib.suppress(RuntimeError, WebSocketDisconnect):
|
||||
await self._ws.send_json(payload)
|
||||
|
||||
|
||||
async def telemetry_ingest_endpoint(
|
||||
websocket: WebSocket,
|
||||
learner_id: str,
|
||||
task_id: str,
|
||||
store: TraceStore,
|
||||
integrity: TraceIntegrityMap,
|
||||
settings: Settings,
|
||||
sandbox_id: str = "",
|
||||
) -> None:
|
||||
"""Engine entry: build the session and run it. api/telemetry.py calls this
|
||||
with query params + app.state services already resolved — this signature
|
||||
is deliberately Depends-free (telemetry/ never knows FastAPI DI exists).
|
||||
"""
|
||||
session = IngestSession(
|
||||
websocket=websocket,
|
||||
store=store,
|
||||
integrity=integrity,
|
||||
settings=settings,
|
||||
learner_id=learner_id,
|
||||
task_id=task_id,
|
||||
sandbox_id=sandbox_id,
|
||||
)
|
||||
await session.run()
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user