Compare commits

...

4 Commits

Author SHA1 Message Date
CIAgent 4c52d29f91 docs(P02): complete agent-framework phase
---ci---
phase: 2
milestone: v0.2
status: complete
---/ci---

REQ-2-004 complete. BaseAgent ABC, agent-scoped sessions (20-msg window,
500-cap LRU), registry, 4-layer structured output defense, 6-module
prompt library, D-021-aligned learner corpus, chat session persistence.
61/61 tests, ruff clean.
2026-09-11 15:52:00 +00:00
CIAgent dda9569b80 docs(P01): complete ai-service-scaffolding phase
---ci---
phase: 1
milestone: v0.2
status: complete
---/ci---

REQ-2-001/002/003 complete. apps/ai-service: FastAPI + provider-agnostic
LLM layer (ollama-cloud/local/mock) + D-016 SSE envelope + 20 tests
(mock-only, cloud-free guard) + ruff + turbo/pnpm integration.
2026-09-11 15:40:30 +00:00
CIAgent c7fe601481 chore(P00): checkpoint complete
---ci---
phase: 0
milestone: v0.2
status: complete
---/ci---
2026-09-11 15:16:38 +00:00
CIAgent e58b027a57 docs(P00): complete pre-execution phase
---ci---
phase: 0
milestone: v0.2
status: complete
---/ci---

Phase 0 complete: SPECIFY, CLARIFY (A-001..010), RESEARCH
(D-016..023, personas), PLAN (38 tasks, 18 waves), GRILL
(G-1..G-5 applied), MVP/UX gate passed.
2026-09-11 15:16:02 +00:00
61 changed files with 3079 additions and 531 deletions
+79 -57
View File
@@ -2,42 +2,69 @@
## Overview
Nextcraft v0.1 is a high-fidelity UI/UX prototype built as a TypeScript monorepo using pnpm workspaces and turborepo. The prototype consists of a single Next.js application with route groups for each surface (Learner, Marketplace, Employer Dashboard, Admin), backed by a shared component library, typed mock data layer, and shared types package.
Nextcraft is a TypeScript monorepo (pnpm workspaces + turborepo) with a Next.js application hosting four surfaces (Learner, Marketplace, Employer Dashboard, Admin), a shared component library, typed mock data layer, and shared types package. As of v0.2, a Python FastAPI application (`apps/ai-service`) hosts six AI tutor agents backed by a provider-agnostic LLM layer.
**No backend, no database, no authentication logic.** All data is static/mock. The architecture is designed to be replaced piece-by-piece with real backend services in future milestones.
**v0.2 additions:** real AI services (streaming chat), agent framework, mock engine inputs for Lab/Assessor/Proctor. **Still no database, no auth** — in-memory session store; real engines (sandbox fabric, assessment engine, identity) are v0.3+.
### Confirmed Technology Stack (Research Findings)
### Confirmed Technology Stack (v0.2)
| Technology | Version | Purpose |
|------------|---------|---------|
| Node.js | v24.15.0 | Runtime |
| Node.js | v24.15.0 | Runtime (web) |
| pnpm | 12.3.4 | Package manager + workspaces |
| turborepo | latest | Build orchestration |
| Next.js | latest (App Router) | Web application framework |
| turborepo | 2.3.3 | Build orchestration |
| Next.js | 15 (App Router) | Web application framework |
| React | 19+ | UI library |
| TypeScript | 5.x | Type system |
| Tailwind CSS | v4 | Utility-first CSS framework |
| lucide-react | latest | Icon system |
| recharts | latest | Charts for employer/admin dashboards |
| @xyflow/react (react-flow) | latest | Competency graph viewer in admin surface |
| Inter font | via next/font | Typography |
| ESLint | via Next.js | Linting |
| Prettier | latest | Code formatting |
| Tailwind CSS | v4 | Utility-first CSS |
| lucide-react | latest | Icons |
| recharts | latest | Charts |
| @xyflow/react | latest | Competency graph viewer |
| Python | 3.11.2 | Runtime (ai-service) |
| FastAPI | 0.141.x | AI service framework |
| uvicorn | 0.52.x | ASGI server |
| pydantic | 2.13.x | Request/response models, structured outputs |
| pydantic-settings | 2.15.x | Settings + env-file loading (replaces python-dotenv) |
| httpx | 0.28.x | Async LLM HTTP client (ollama-cloud + local providers) |
| sse-starlette | 3.4.x | SSE framing, ping keep-alive |
| pytest | 9.x | Test runner |
| pytest-asyncio | 1.4.x | Async tests (auto mode) |
| ruff | latest | Python lint (check-only, no formatter) — `pnpm ai:lint` |
| ollama-cloud | https://ollama.com/v1 | Default LLM provider (OpenAI-compatible, Bearer auth) |
### Architecture Decisions from Research
Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the port; raw httpx keeps delta passthrough and ollama-cloud quirk tolerance), python-dotenv (pydantic-settings reads .env natively), respx (httpx MockTransport is built in).
1. **Next.js App Router with route groups**`(learner)`, `(marketplace)`, `(employer)`, `(admin)` provide clean URL separation without affecting paths
2. **Tailwind CSS v4** — Configured via `@theme` in CSS, no `tailwind.config.js` needed (v4 paradigm shift). Dark mode via `class` strategy.
3. **Server components by default** — All pages are server components. Client components only for interactive elements (filters, search, chat, graph viewer, dark mode toggle)
4. **Mock data as ES modules** — Typed TS files exported from `packages/mock-data`. No JSON files — all mock data is programmatically generated for richer structure.
5. **react-flow (@xyflow/react)** — Confirmed for competency graph viewer. Provides interactive node/edge rendering with built-in controls.
6. **recharts** — Confirmed for analytics dashboards. Responsive, composable, integrates well with React server components.
7. **pnpm workspaces**`apps/web` + `packages/ui` + `packages/mock-data` + `packages/types`. Shared deps hoisted.
### v0.2 Architecture Decisions (from Research)
1. **D-016 SSE envelope**`meta` event (agent/session/model, flushed before first token) → raw OpenAI-compatible chunk passthrough → optional `done``error` event before `[DONE]` on mid-stream failure. Pre-first-byte failures use proper HTTP status. Headers: `Cache-Control: no-cache`, `X-Accel-Buffering: no`.
2. **D-017 httpx direct client** — lifespan-managed `httpx.AsyncClient` (10s connect / 300s read), shared by ollama-cloud and local providers; no SDK.
3. **D-018 Agent framework**`BaseAgent` ABC (system_prompt/build_messages/stream_reply/structured_reply) + explicit registry; prompts are versioned code in `prompts/`.
4. **D-019 Session store**`SessionStore` protocol + `InMemorySessionStore` (asyncio.Lock, 20-message window, 500-cap LRU, agent-scoped sessions). DB-migration-ready.
5. **D-020 Structured outputs** — 4-layer defense: `response_format` (auto-degrade) → prompt-embedded schema → fence-strip/first-balanced-object parse → single bounded retry.
6. **D-021 Mock corpus in Python**`ai_service/corpus/` pydantic-typed, convention-aligned with TS `packages/mock-data` (shared IDs, cross-referencing headers); no codegen in v0.2.
7. **D-022 Monorepo integration** — zero-dependency shim `package.json` in apps/ai-service + `ai#*` turbo passthrough tasks (`cache:false, outputs:[]`) + root `ai:dev`/`ai:test` scripts + idempotent venv bootstrap.
8. **D-023 Testing** — pytest-asyncio auto mode; TestClient `client.stream()` for SSE; httpx MockTransport for byte-exact provider parser tests; scripted mock provider incl. failure modes. Tests never call the cloud.
---
## Components
### apps/ai-service — AI Tutor Service (v0.2 NEW)
| Component | Description | Boundaries | Depends On |
|-----------|-------------|------------|------------|
| `ai_service/main.py` | FastAPI app factory, lifespan (httpx client pool, provider factory), CORS (localhost only), /health | App entry | config, llm, agents, api |
| `ai_service/config.py` | pydantic-settings Settings (env_prefix="AI_", env_file, SecretStr key) | Configuration only | None |
| `ai_service/api/` | Endpoints: chat.py (POST /v1/chat/stream, SSE), lab.py, assessment.py, proctor.py, mentor.py; deps.py (DI) | Composes agents + sessions; never imported by llm/ or agents/ | agents, llm |
| `ai_service/llm/` | types.py (Message, ChatDelta), base.py (LLMProvider protocol), openai_compat.py (ollama-cloud + local), mock.py (deterministic), factory.py | Never imports agents/ or api/ | config |
| `ai_service/agents/` | base.py (BaseAgent ABC), registry.py, session.py (SessionStore), structured.py (JSON defense), coach/tutor/lab/assessor/proctor/mentor.py | Never imports api/ | llm, prompts, corpus |
| `ai_service/prompts/` | Per-agent system prompt constants + render_context functions (str.format_map) | Data only | None |
| `ai_service/corpus/` | Mock engine inputs: learner_context.py, telemetry.py (Lab/Proctor scenarios), artifacts.py (pre-baked artifacts, rubrics, transcripts) | Pydantic-typed; aligned with TS packages/mock-data by convention | None |
| `scripts/` | bootstrap.sh (venv + pip install idempotent), dev.sh (exports keys from .ciagent/.env.secrets → uvicorn), test.sh (pytest) | Dev entry points | pyproject.toml |
| `tests/` | conftest.py (mock provider, settings override, TestClient), health, llm (MockTransport parser), agents (framework + per-agent), api (SSE stream tests) | Mock provider only — no cloud | all |
**Module boundary rules:** `llm/` never imports `agents/` or `api/`; `agents/` never imports `api/`; `api/` composes both via DI. `corpus/` is the only home of mock engine data. Prompts are code — versioned and reviewed in git.
### apps/web — Next.js Application
| Component | Description | Boundaries | Depends On |
@@ -84,53 +111,48 @@ Nextcraft v0.1 is a high-fidelity UI/UX prototype built as a TypeScript monorepo
## Data Flow
```
[Mock Data Layer] ──typed──> [Shared Types] <──typed──> [UI Components]
│ │
▼ ▼
[Next.js Route Groups] [Surface Components]
(learner)/ (marketplace)/ (employer)/ (admin)/
└──────────────────┴─────────────────┴───────────────┘
[Root Layout + Theme Provider]
[Responsive Navigation Shell]
[packages/mock-data + packages/types] [ai_service/corpus]
(TS, web surfaces) │ (Python, agent inputs)
▼ ▼
[Next.js Route Groups] [ai-service agents]
(learner)/ (marketplace)/ coach tutor lab assessor proctor mentor
(employer)/ (admin)/ │
│ SSE (fetch + ReadableStream) [LLMProvider]
└────── client components ◄───────────────────┤
http://localhost:8420 ollama-cloud / local / mock
(https://ollama.com/v1)
```
All data flows from the mock data layer through typed imports into Next.js route handlers / server components, which pass data as props to UI components. No client-side data fetching, no API routes, no server actions in v0.1.
- Web surfaces remain server-component-first; client components (chat, filters, graph viewer, dark mode toggle) fetch directly from ai-service over SSE (A-002: no Next.js API-route proxy in v0.2).
- The LLM provider layer is a dumb pipe — OpenAI-compatible chunks pass through byte-identical; envelope logic (meta/done/error) lives only in the API layer (D-016).
- Lab/Assessor/Proctor read mock scenarios from `ai_service/corpus/` — real engines are v0.3+.
- All automated tests use the deterministic mock provider; the cloud is for manual probes only.
---
## Build Order
## Build Order (v0.2)
1. **Monorepo scaffolding**pnpm-workspace.yaml, turbo.json, tsconfig.json, package.json, Next.js app initialization
2. **Shared types** — packages/types with all domain, marketplace, user, and UI type definitions
3. **Mock data layer** — packages/mock-data with typed mock data for all surfaces
4. **Design tokens** — packages/ui/design-tokens with CSS custom properties
5. **UI primitives** — Button, Input, Card, Badge, Avatar, Dialog, Tabs, Progress, Tooltip, Skeleton, Toast
6. **Root layout + navigation shell** — Root layout with theme provider, responsive navigation, role switcher
7. **UI composites** — Navigation, Table, SearchBar, FilterPanel, ChatInterface, GraphViewer, ArtifactCard, CompetencyBadge, JobCard, CandidateCard, MetricCard
8. **Learner surface routes** — Landing, catalog, competency stack, dashboard, byte viewer, sandbox mockup, assessment mockup
9. **Marketplace surface routes** — Job board, job detail, employer profile, search/filter, pricing
10. **Employer dashboard routes** — Overview, talent search, candidate profile, posting management
11. **Admin surface routes** — Overview, learner management, competency graph viewer, moderation
12. **Polish + integration** — Cross-surface navigation, responsive QA, visual consistency, Storybook
1. **AI service scaffolding**apps/ai-service: FastAPI app, config, provider layer (ollama-cloud/local/mock), SSE chat endpoint, pytest harness, turbo integration
2. **Agent framework** — BaseAgent, registry, session store, structured output, prompts scaffolding, learner-context corpus
3. **Coach + Tutor agents** — full implementations, chat endpoint agent routing
4. **Lab + Assessor agents** — telemetry scenarios + pre-baked artifacts corpus, /v1/lab/feedback + /v1/assessment/evaluate
5. **Proctor + Mentor agents** — proctor scenarios, /v1/proctor/signals + /v1/mentor/narrative
6. **Learner surface integration** — useChatStream hook, agent switcher, streaming/error/loading states, agent output panels across the four learner surfaces
The v0.1 build order (monorepo → types → mock data → tokens → primitives → layout → composites → surfaces → polish) is complete and preserved in git history (tags v0.0.1v0.1.0).
---
## Future Architecture (Post-v0.1, for reference)
## Future Architecture (Post-v0.2, for reference)
The v0.1 prototype is designed to be replaced piece-by-piece with real backend services:
v0.2 delivers the ai-service skeleton that later milestones fill in:
- **Mock data → PostgreSQL + Drizzle ORM** — mock data files replaced with database queries via repository layer
- **Static routes → Fastify API + Next.js SSR** — API routes replaced with Fastify backend services
- **Mock AI tutor → Python FastAPI AI services** — Chat interface mockup replaced with real AI agent microservices
- **Mock assessment → Assessment engine** — Assessment mockup replaced with process-trace grading + oral defense engine
- **No auth → Identity verification + age-gating** — Registration flow mockup replaced with real KYC and age verification
- **Mock corpus → real engines** — Lab consumes real sandbox telemetry (v0.3 sandbox fabric); Assessor grades real process traces (v0.3 assessment engine); Proctor consumes real identity/attention signals (v0.3 identity verification)
- **In-memory sessions → PostgreSQL + Drizzle ORM** — SessionStore protocol swap, no API changes
- **Mock provider → per-agent model routing** — provider factory already selects by config; per-agent `AI_<AGENT>_MODEL` overrides
- **No auth → real KYC + sessions** — A-008 dropped in v0.3 when identity verification lands
- **No search → Semantic vector search (pgvector)** — Filter UI replaced with vector similarity search
- **No payments → Payment processing** — Pricing page replaced with real subscription/payment flows
The monorepo structure (apps/web + packages/*) is designed to accommodate additional apps (e.g., apps/api, apps/ai-service) in future milestones without restructuring.
The monorepo structure (apps/web + apps/ai-service + packages/*) accommodates further apps without restructuring.
+5 -5
View File
@@ -1,8 +1,8 @@
{
"phase": 7,
"stage": "complete",
"milestone": "v0.1",
"phase_role": "final",
"phase": 2,
"stage": "verify",
"milestone": "v0.2",
"phase_role": "execution",
"attempts": 0,
"updated_at": "2026-09-10T22:48:00Z"
"updated_at": "2026-09-11T17:35:00Z"
}
+36
View File
@@ -0,0 +1,36 @@
# Nextcraft v0.2 — GRILL.md (Adversarial Review Verdict)
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-CHANGES · **Confidence:** 0.82
## Per-Axis Findings
| Axis | Verdict | Rationale |
|------|---------|-----------|
| Feasibility | PASS | Environment claims verified (venv works, all 8 pinned packages on PyPI for py3.11, port 8420 free, secrets gitignored). Stall points pre-mitigated (idempotent bootstrap, 300s read timeout, idempotent abort; reactStrictMode confirmed in next.config). |
| Over-scoping | CONCERN (mild) | 4-layer JSON defense justified; 500-cap LRU is over-spec but cheap — keep, extend no further. Real gaps: no Python lint (fixed via G-3), cache dirs in gitignore (fixed), script path resolution (advisory c). |
| Architecture risk | CONCERN | sse-starlette `: ping` frames invisible to short TestClient streams — parser gap fixed via G-1. Agent registration had two contradictory patterns — fixed via G-4. Cloud outage blast radius contained by mock-only tests. |
| Phase sequencing | PASS | Integration-last correct: P1 freezes the SSE contract before client code exists; A-002 eliminates dev-server buffering trap. P4 heaviest but mechanical. |
| Verification honesty | CONCERN | Persona distinctness circular against self-authored mocks (disclosed; cloud probe optional). P6 must-haves manual-only. D-021 ID alignment unmechanized (advisory b). Disclosed honestly. |
| Cost/quota | PASS | Cloud burn bounded (~<100K tokens milestone-wide, manual probes only). Mock-only rule structurally enforced; mechanical guard via advisory (a). |
| Milestone honesty | PASS (conditional) | Mock-engine caveats present everywhere that matters. Release note content now bound by G-5; dead `aiTutorResponses` disposal bound by G-5. |
## Binding Decisions (applied to PLAN.md/ROADMAP.md)
- **G-1 (BINDING):** SSE frame parser in Task 6-1-01 must ignore frames with no `data:` lines (sse-starlette ping keep-alive). Added to Action + Phase 6 must-have.
- **G-2 (BINDING):** P6 end-to-end verification may run with `AI_PROVIDER=mock` fallback — real ai-service over HTTP is the requirement; provider choice is service-internal. Prevents cloud outage blocking P6.
- **G-3 (BINDING):** ruff (check-only) added: Task 1-1-04, pyproject dev extra, scripts/lint.sh, root `ai:lint`, turbo `ai#lint`, Phase 1 must-have.
- **G-4 (BINDING):** Mentor/Proctor registered centrally in `registry.py` (single registration pattern), matching P3/P4.
- **G-5 (BINDING):** v0.2.0 release note must state Lab/Assessor/Proctor run on mock engine inputs (v0.3+ for real); dead `aiTutorResponses` export disposed of in P7.
## Advisory (non-binding; applied where cheap)
- (a) conftest asserts provider is MockProvider — **applied in P1 implementation**
- (b) pytest reads packages/mock-data as text, asserts corpus IDs appear — **applied in P4 implementation**
- (c) scripts resolve repo root via script-relative dirname — **folded into Task 1-1-04/1-1-03**
- (d) `.pytest_cache/` + `.ruff_cache/` in .gitignore — **applied**
- (e) keep LRU as-is; no further session-store sophistication in v0.2
- (f) Task 6-2-02 REQ tag fixed to REQ-2-011 (routing) — **applied**
## Outcome
GO — all five binding decisions applied to PLAN.md/ROADMAP.md/.gitignore/ARCHITECTURE.md before Phase 1 execution. No axis requires escalation.
+66 -21
View File
@@ -6,12 +6,13 @@
```yaml
active: true
phase_specific: false
reason: Coordinates task decomposition across surfaces, resolves conflicts between frontend and data personas
reason: Coordinates task decomposition across web, AI service, and data territories; resolves conflicts between frontend, backend, and AI personas
domain: coordination
frameworks:
- next.js
- turborepo
- pnpm
- fastapi
constraints:
- pragmatic
- battle-tested defaults
@@ -21,13 +22,14 @@ territory:
- "**/turbo.json"
- "**/pnpm-workspace.yaml"
- "**/tsconfig.json"
- "apps/ai-service/pyproject.toml"
```
### frontend-engineer
```yaml
active: true
phase_specific: false
reason: Primary persona for v0.1 — all 28 REQ-IDs are UI/frontend work. Owns all page components, layouts, and surface-specific UI.
reason: Phase 6 learner-surface integration — useChatStream hook, agent switcher, streaming/error/loading states, Lab/Assessor/Proctor output panels. Owns all page components, layouts, and surface-specific UI.
domain: frontend
frameworks:
- react
@@ -40,7 +42,8 @@ constraints:
- component-first
- server-components-default
- minimal-client-js
- mock-data-only
- sse-client-buffering (buffer bytes, split frames on \n\n, join data: lines)
- abortcontroller-cleanup (idempotent abort in effect cleanup)
- responsive-all-breakpoints
- dark-mode-support
territory:
@@ -54,7 +57,7 @@ territory:
```yaml
active: true
phase_specific: false
reason: Owns mock data layer schema and typed definitions. No real database in v0.1, but data structures must be well-typed for future migration.
reason: Owns TS mock data layer schema and typed definitions. Does NOT own the Python corpus (that is ai-engineer territory) — the two are aligned by documented convention (D-021).
domain: data
frameworks:
- typescript
@@ -70,33 +73,62 @@ territory:
### backend-engineer
```yaml
active: false
active: true
phase_specific: false
reason: No backend in v0.1. All data is mock/static. Backend persona deactivated until v0.2+ when API services are needed.
reason: Reactivated for v0.2 — owns apps/ai-service infrastructure: FastAPI app, settings, SSE plumbing, endpoints, monorepo/turbo integration, test harness.
domain: backend
frameworks: []
constraints: []
territory: []
frameworks:
- fastapi
- uvicorn
- pydantic
- httpx
- pytest
constraints:
- provider-agnostic-boundaries (llm/ imports nothing from agents/ or api/)
- streaming-first
- no-database-v0.2
- secrets-via-env-only
- mock-provider-in-tests
territory:
- "apps/ai-service/ai_service/main.py"
- "apps/ai-service/ai_service/config.py"
- "apps/ai-service/ai_service/api/**"
- "apps/ai-service/scripts/**"
- "apps/ai-service/package.json"
- "apps/ai-service/tests/api/**"
- "turbo.json"
```
### security-auditor
### ai-engineer
```yaml
active: false
active: true
phase_specific: false
reason: No auth, no API, no real data in v0.1. Security review handled by verifier's STRIDE analysis layer. No dedicated security persona needed for UI prototype.
domain: security
frameworks: []
constraints: []
territory: []
reason: Custom persona for v0.2 — owns the LLM provider layer, agent framework, prompt library, structured outputs, and mock corpora for the six tutor agents.
domain: ai
frameworks:
- pydantic
- httpx
- pytest
constraints:
- provider-agnostic-protocol
- prompts-are-code
- json-defensive-parsing
- never-call-cloud-in-tests
- delta-passthrough
territory:
- "apps/ai-service/ai_service/llm/**"
- "apps/ai-service/ai_service/agents/**"
- "apps/ai-service/ai_service/prompts/**"
- "apps/ai-service/ai_service/corpus/**"
- "apps/ai-service/tests/llm/**"
- "apps/ai-service/tests/agents/**"
```
## Custom Personas
### design-system-engineer
```yaml
active: true
phase_specific: false
reason: Custom persona for v0.1 — owns the shared component library, design tokens, and visual consistency. Combines frontend expertise with design system ownership.
reason: Owns the shared component library, design tokens, and visual consistency. Light duty in v0.2: Phase 6 may need new primitives (agent-switcher control, stream-status indicator, error toast variant).
domain: frontend
frameworks:
- tailwindcss
@@ -111,14 +143,27 @@ territory:
- "packages/ui/**"
```
### security-auditor
```yaml
active: false
phase_specific: false
reason: No auth in v0.2 (A-008); CORS is localhost-only; no real user data. Security review handled by verifier's STRIDE analysis layer plus a Phase 7 checklist item: secrets hygiene (key absent from code/logs/commits/errors), localhost-only CORS, no PII in prompts.
domain: security
frameworks: []
constraints: []
territory: []
```
## Phase-Specific Personas
None for v0.1. All active personas span the entire milestone.
None for v0.2. All active personas span the entire milestone. data-engineer and design-system-engineer are light-touch after Phase 1.
## Territory Conflict Resolution
| Conflict | Resolution |
|----------|-----------|
|----------|------------|
| frontend-engineer vs data-engineer (packages/types, packages/mock-data) | data-engineer owns type definitions and mock data schema; frontend-engineer consumes them. If changes needed, data-engineer updates types first. |
| frontend-engineer vs design-system-engineer (packages/ui) | design-system-engineer owns design tokens and primitive components; frontend-engineer owns composite components and page-level UI. |
| ai-engineer vs data-engineer (mock data duplication) | ai-engineer owns `ai_service/corpus/` (Python); data-engineer owns `packages/mock-data` (TS). Shared entity IDs and shapes kept aligned by documented convention (D-021): cross-referencing file headers, identical `comp-*` ID strings. |
| backend-engineer vs ai-engineer (apps/ai-service) | backend-engineer owns app shell, config, API endpoints, scripts, and test harness; ai-engineer owns llm/, agents/, prompts/, corpus/. Boundary: `ai_service/api/` (backend) composes `ai_service/agents/` (AI) via DI — agents never import api/. |
| lead-developer vs any | lead-developer coordinates only, does not directly modify code files. |
+333 -309
View File
@@ -1,398 +1,422 @@
# Nextcraft v0.1 — PLAN.md
# Nextcraft v0.2 — PLAN.md
## Overview
This plan covers execution phases 1-6 of milestone v0.1 (UI/UX Prototype). Each phase is a vertical slice that produces a demoable milestone. Phases are ordered by dependency — later phases build on earlier ones.
This plan covers execution phases 1-6 of milestone v0.2 (AI Tutor Architecture): the six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface with streaming responses. Phases are strictly sequential (P1→P6); within each phase, Wave 1 tasks are parallelizable (no cross-file dependencies) and later waves depend on earlier ones.
**Environment facts (apply throughout):** Python via `python3 -m venv` (no system pip, no uv); pnpm 12.3.4 via corepack; ai-service port **8420**; default model `gemma4:31b` (config via `AI_TUTOR_MODEL`); ollama-cloud base `https://ollama.com/v1` (OpenAI-compatible, Bearer auth); API keys live only in gitignored `.ciagent/.env.secrets`, exported by `scripts/dev.sh` — never in code, commits, or logs; all automated tests use the deterministic mock provider and **never call the cloud**.
| Phase | Name | Requirements | Waves | Personas |
|-------|------|-------------|-------|----------|
| 1 | AI service scaffolding | REQ-2-001, 002, 003 | 3 | backend-engineer, ai-engineer |
| 2 | Agent framework | REQ-2-004 | 2 | ai-engineer, backend-engineer |
| 3 | Coach + Tutor agents | REQ-2-005, 006 | 3 | ai-engineer, backend-engineer |
| 4 | Lab + Assessor agents | REQ-2-007, 008 | 4 | ai-engineer, backend-engineer, data-engineer |
| 5 | Proctor + Mentor agents | REQ-2-009, 010 | 3 | ai-engineer, backend-engineer |
| 6 | Learner surface integration | REQ-2-011, 012 | 3 | frontend-engineer, design-system-engineer |
---
## Phase 1: Project Scaffolding
## Phase 1: AI Service Scaffolding
**Requirements:** REQ-001, REQ-002, REQ-003, REQ-004, REQ-005
**Persona:** design-system-engineer (Wave 1), data-engineer (Wave 1), frontend-engineer (Wave 2)
**Goal:** Monorepo builds, dev server starts, component library has all primitives, mock data typed, routing works
**Requirements:** REQ-2-001, REQ-2-002, REQ-2-003
**Goal:** apps/ai-service runs under uvicorn, /health responds, provider-agnostic LLM layer with ollama-cloud/local/mock providers, SSE chat streaming verified, pytest suite green with mock provider, turbo integration wired
### Wave 1: Foundation (parallel — no shared file conflicts)
### Wave 1: Service shell + LLM core (parallel — no shared files)
#### Task 1-1-01: Monorepo scaffolding
- **Persona:** design-system-engineer
- **Files:** `package.json`, `pnpm-workspace.yaml`, `turbo.json`, `tsconfig.json`, `.npmrc`
- **Action:** Create root monorepo config. pnpm workspaces pointing to `apps/*` and `packages/*`. Turborepo with build/dev/lint/typecheck pipelines. Root tsconfig with path aliases.
- **Verify:** `pnpm install` succeeds; workspace packages detected
- **Done:** `pnpm list -r` shows all workspace packages
#### Task 1-1-01: FastAPI app scaffolding
- **Persona:** backend-engineer — **REQ:** REQ-2-001
- **Files:** `apps/ai-service/pyproject.toml`, `apps/ai-service/ai_service/main.py`, `apps/ai-service/ai_service/config.py`, `apps/ai-service/ai_service/__init__.py`, `apps/ai-service/tests/conftest.py`, `apps/ai-service/tests/test_health.py`, `apps/ai-service/.env.example`, `apps/ai-service/README.md`
- **Action:** pydantic-settings `Settings` (env_prefix `AI_`, env_file, `SecretStr` key, port 8420, provider select, `AI_TUTOR_MODEL` default `gemma4:31b`). FastAPI app factory in `main.py` with lifespan stub (httpx client pool comes in Wave 2), CORS localhost-only (A-008), `GET /health`. pyproject with pinned deps (fastapi, uvicorn, pydantic, pydantic-settings, httpx, sse-starlette, pytest, pytest-asyncio) and dev extra. conftest: settings override + TestClient fixture. `.env.example` documents all `AI_*` vars; README documents venv setup and dev workflow.
- **Verify:** `scripts/bootstrap.sh && scripts/test.sh` — test_health passes; `curl localhost:8420/health` returns 200
#### Task 1-1-02: Shared types package
- **Persona:** data-engineer
- **Files:** `packages/types/package.json`, `packages/types/tsconfig.json`, `packages/types/domain.ts`, `packages/types/marketplace.ts`, `packages/types/user.ts`, `packages/types/ui.ts`, `packages/types/index.ts`
- **Action:** Create all shared TypeScript type definitions: Competency, CompetencyStack, Microcredential, Artifact, ProcessTrace, OralDefense, AssessmentRubric, Job, Employer, Candidate, JobPosting, TalentMatch, SearchFilter, Learner, Admin, EmployerUser, AgeGroup, Role, plus UI types (ComponentProps, ThemeConfig, Breakpoint).
- **Verify:** `pnpm typecheck` passes in packages/types
- **Done:** All types exported from index.ts; no type errors
#### Task 1-1-02: LLM types, protocol, mock provider
- **Persona:** ai-engineer**REQ:** REQ-2-002
- **Files:** `apps/ai-service/ai_service/llm/types.py`, `apps/ai-service/ai_service/llm/base.py`, `apps/ai-service/ai_service/llm/mock.py`, `apps/ai-service/ai_service/llm/__init__.py`
- **Action:** `types.py`: pydantic `Message` (role/content), `ChatDelta` (OpenAI-compatible chunk shape). `base.py`: `LLMProvider` protocol — async `stream_chat(messages, model, response_format=None) -> AsyncIterator[ChatDelta]`; the provider is a dumb pipe, no envelope logic (D-016 keeps envelope in API layer). `mock.py`: deterministic scripted provider (hash-seeded token streams, scripted failure modes: connect error, mid-stream error, malformed JSON) for tests and CI.
- **Verify:** mock provider importable and deterministic; two identical calls yield identical streams
#### Task 1-1-03: Mock data package
- **Persona:** data-engineer
- **Files:** `packages/mock-data/package.json`, `packages/mock-data/tsconfig.json`, `packages/mock-data/competency-stacks.ts`, `packages/mock-data/jobs.ts`, `packages/mock-data/candidates.ts`, `packages/mock-data/employers.ts`, `packages/mock-data/learner-progress.ts`, `packages/mock-data/ai-tutor-responses.ts`, `packages/mock-data/index.ts`
- **Action:** Create typed mock data: 5 competency stacks (AI Orchestration Engineer with 15 competencies, AI Safety & Governance with 14, Human-AI Product Designer with 13, AI-Augmented Field Operator with 12, Computational Sciences with 16). 20+ mock jobs. 15+ mock candidates. 10+ mock employers. Mock learner progress. Pre-scripted AI tutor responses.
- **Verify:** All mock data matches types from packages/types; `pnpm typecheck` passes
- **Done:** All mock data exported from index.ts; types match
#### Task 1-1-03: Monorepo integration (shim + turbo + scripts)
- **Persona:** backend-engineer**REQ:** REQ-2-001
- **Files:** `apps/ai-service/package.json`, `apps/ai-service/scripts/bootstrap.sh`, `apps/ai-service/scripts/dev.sh`, `apps/ai-service/scripts/test.sh`, `turbo.json` (update), `package.json` (root, update)
- **Action:** zero-dependency shim `package.json` in apps/ai-service with `dev`/`test`/`bootstrap` script entries. Turbo passthrough tasks `ai#dev`, `ai#test`, `ai#bootstrap` (`cache: false`, `outputs: []`). Root scripts `ai:dev`, `ai:test`, `ai:bootstrap`. `bootstrap.sh`: idempotent `python3 -m venv .venv` + pip install. `dev.sh`: exports keys from `.ciagent/.env.secrets` → uvicorn on 8420. `test.sh`: pytest via venv.
- **Verify:** `corepack pnpm install && pnpm ai:bootstrap && pnpm ai:test` runs pytest through turbo; re-running bootstrap is a no-op
### Wave 2: UI Foundation (depends on Wave 1)
#### Task 1-1-04: Python lint (ruff, check-only)
- **Persona:** backend-engineer — **REQ:** REQ-2-001
- **Files:** `apps/ai-service/pyproject.toml` (update: `[tool.ruff]` config + `ruff` in dev extra), `apps/ai-service/scripts/lint.sh`, `package.json` (root, update), `turbo.json` (update)
- **Action:** Add `ruff` (check-only, no formatter) to the dev extra; `[tool.ruff]` with line-length 100, target py311. `scripts/lint.sh`: `.venv/bin/ruff check .` with repo-root path resolution via script-relative dirname (not CWD). Root script `ai:lint`, turbo passthrough `ai#lint` (`cache: false`, `outputs: []`). Run over the entire ai-service tree; fix all findings before phase ship (G-3).
- **Verify:** `pnpm ai:lint` exits 0 on the Phase 1 codebase
#### Task 1-2-01: Next.js app initialization
- **Persona:** frontend-engineer
- **Files:** `apps/web/package.json`, `apps/web/tsconfig.json`, `apps/web/next.config.ts`, `apps/web/app/layout.tsx`, `apps/web/app/page.tsx`, `apps/web/app/globals.css`
- **Action:** Initialize Next.js app with App Router, TypeScript, Tailwind CSS v4. Configure Inter font via next/font. Set up path aliases to packages. Create root layout with theme provider. Create placeholder home page.
- **Verify:** `pnpm dev` starts; `pnpm build` succeeds; page renders at localhost:3000
- **Done:** Next.js app runs with Tailwind CSS and Inter font
### Wave 2: Real providers + SSE endpoint (depends on Wave 1)
#### Task 1-2-02: Design tokens and Tailwind config
- **Persona:** design-system-engineer
- **Files:** `packages/ui/package.json`, `packages/ui/tsconfig.json`, `packages/ui/src/index.ts`, `packages/ui/src/tokens/index.ts`, `apps/web/app/globals.css` (update)
- **Action:** Create packages/ui package. Define design tokens as TypeScript constants and CSS custom properties: color palette (indigo/violet primary, emerald accent, slate neutral, dark mode variants), typography scale (12px-48px), spacing system (4px base), breakpoints (375px, 768px, 1280px), shadows, radii. Configure Tailwind v4 `@theme` with custom colors.
- **Verify:** Design tokens importable from packages/ui; Tailwind classes work with custom colors
- **Done:** `import { tokens } from '@nextcraft/ui'` works; `bg-primary-500` class works
#### Task 1-2-01: OpenAI-compatible provider + factory
- **Persona:** ai-engineer — **REQ:** REQ-2-002
- **Files:** `apps/ai-service/ai_service/llm/openai_compat.py`, `apps/ai-service/ai_service/llm/factory.py`
- **Action:** `openai_compat.py`: single `OpenAICompatProvider` for ollama-cloud (`https://ollama.com/v1`, Bearer) and local endpoints (base URL from settings); raw httpx against `/v1/chat/completions` with `stream: true`, byte-identical delta passthrough, tolerant of ollama-cloud quirks. Uses the lifespan-managed `httpx.AsyncClient` (10s connect / 300s read, D-017) — no openai SDK. `factory.py`: select provider from settings (`ollama-cloud` | `local` | `mock`).
- **Verify:** provider constructs from settings for all 3 names; manual probe against ollama-cloud streams tokens (documented in README, not a test)
#### Task 1-2-03: UI primitives
- **Persona:** design-system-engineer
- **Files:** `packages/ui/src/primitives/button.tsx`, `packages/ui/src/primitives/input.tsx`, `packages/ui/src/primitives/card.tsx`, `packages/ui/src/primitives/badge.tsx`, `packages/ui/src/primitives/avatar.tsx`, `packages/ui/src/primitives/dialog.tsx`, `packages/ui/src/primitives/tabs.tsx`, `packages/ui/src/primitives/progress.tsx`, `packages/ui/src/primitives/tooltip.tsx`, `packages/ui/src/primitives/skeleton.tsx`, `packages/ui/src/primitives/toast.tsx`, `packages/ui/src/primitives/index.ts`
- **Action:** Create all 12 primitive components. Each is a React component with TypeScript props, Tailwind styling, forwardRef, and size variants. Button (primary, secondary, ghost, destructive). Input (text, search, with label). Card (with header, body, footer slots). Badge (default, success, warning, error). Avatar (with image, fallback, sizes). Dialog (modal, with overlay). Tabs (horizontal, with active indicator). Progress (linear, circular). Tooltip (on hover). Skeleton (shimmer). Toast (with variants).
- **Verify:** All primitives importable; each renders without error; variants work
- **Done:** 12 primitives exported from packages/ui
#### Task 1-2-02: Lifespan wiring + SSE chat endpoint
- **Persona:** backend-engineer — **REQ:** REQ-2-003
- **Files:** `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/ai_service/api/deps.py`, `apps/ai-service/ai_service/api/chat.py`, `apps/ai-service/ai_service/api/__init__.py`
- **Action:** Lifespan creates the shared `httpx.AsyncClient` and provider factory; deps.py provides provider via DI. `POST /v1/chat/stream` in chat.py implements the D-016 envelope: `meta` event (agent/session/model) flushed before first token → raw OpenAI chunks passed through as `data: {json}``done` event → `error` event before `[DONE]` on mid-stream failure; pre-first-byte failures return proper HTTP status codes. Headers `Cache-Control: no-cache`, `X-Accel-Buffering: no`; sse-starlette ping keep-alive.
- **Verify:** `curl -N -X POST localhost:8420/v1/chat/stream` with mock provider shows meta event, token deltas, done, `[DONE]`
#### Task 1-2-04: Layout components
- **Persona:** design-system-engineer
- **Files:** `packages/ui/src/layouts/container.tsx`, `packages/ui/src/layouts/grid.tsx`, `packages/ui/src/layouts/sidebar.tsx`, `packages/ui/src/layouts/split-panel.tsx`, `packages/ui/src/layouts/dashboard-layout.tsx`, `packages/ui/src/layouts/index.ts`
- **Action:** Create layout components. Container (max-width variants: sm, md, lg, xl, full). Grid (responsive cols prop). Sidebar (collapsible, with nav items). SplitPanel (resizable divider). DashboardLayout (sidebar + main content area).
- **Verify:** Layout components render; responsive breakpoints work
- **Done:** 5 layout components exported
### Wave 3: Provider + endpoint test suites (depends on Wave 2)
#### Task 1-2-05: Root layout + navigation shell
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/layout.tsx` (update), `apps/web/components/navigation-shell.tsx`, `apps/web/components/role-switcher.tsx`, `apps/web/components/header.tsx`, `apps/web/components/footer.tsx`
- **Action:** Update root layout with theme provider (dark mode toggle), Inter font, responsive navigation shell. Create role switcher (learner/employer/admin toggle in header). Create header with logo, nav links, role switcher, dark mode toggle. Create footer with links.
- **Verify:** Dark mode toggle works; role switcher navigates between surfaces; responsive at 375px/768px/1280px
- **Done:** Navigation shell renders on all pages; role switcher functional
#### Task 1-3-01: LLM provider tests (byte-exact, no cloud)
- **Persona:** ai-engineer**REQ:** REQ-2-002
- **Files:** `apps/ai-service/tests/llm/test_openai_compat.py`, `apps/ai-service/tests/llm/test_mock.py`, `apps/ai-service/tests/llm/__init__.py`
- **Action:** httpx `MockTransport` tests parsing byte-exact fixture streams (happy path, empty delta, `[DONE]`, malformed line, mid-stream disconnect). Mock provider tests: determinism, scripted failure modes, response_format echo.
- **Verify:** `pnpm ai:test` — llm suite green; zero network calls in tests
#### Task 1-2-06: Route groups with placeholder pages
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/page.tsx`, `apps/web/app/(learner)/layout.tsx`, `apps/web/app/(marketplace)/page.tsx`, `apps/web/app/(marketplace)/layout.tsx`, `apps/web/app/(employer)/page.tsx`, `apps/web/app/(employer)/layout.tsx`, `apps/web/app/(admin)/page.tsx`, `apps/web/app/(admin)/layout.tsx`
- **Action:** Create route groups with layouts for each surface. Each layout has surface-specific navigation. Placeholder pages with surface name and brief description.
- **Verify:** Navigation to / (learner), /marketplace, /employer, /admin works
- **Done:** 4 route groups with layouts and placeholder pages
#### Task 1-3-02: SSE stream endpoint tests
- **Persona:** backend-engineer**REQ:** REQ-2-003
- **Files:** `apps/ai-service/tests/api/test_chat_stream.py`, `apps/ai-service/tests/api/__init__.py`
- **Action:** TestClient `client.stream()` tests: meta-first ordering, delta passthrough, done + `[DONE]` sentinel, mid-stream error event, pre-first-byte failure → HTTP status, required headers. pytest-asyncio auto mode (D-023).
- **Verify:** `pnpm ai:test` — api suite green
### Must-Haves (Phase 1)
- [ ] `pnpm install` succeeds
- [ ] `pnpm dev` starts Next.js dev server
- [ ] `pnpm build` succeeds
- [ ] `pnpm typecheck` passes
- [ ] 12 UI primitives importable from `@nextcraft/ui`
- [ ] Mock data typed and importable from `@nextcraft/mock-data`
- [ ] Types importable from `@nextcraft/types`
- [ ] 4 route groups with layouts
- [ ] Dark mode toggle works
- [ ] Role switcher navigates between surfaces
- [ ] Responsive at 375px, 768px, 1280px
- [ ] `scripts/bootstrap.sh` is idempotent; creates venv + installs deps without system pip
- [ ] `pnpm ai:dev` starts uvicorn; `curl localhost:8420/health` returns 200
- [ ] `pnpm ai:lint` exits 0 (ruff check over the ai-service tree) (G-3)
- [ ] `pnpm ai:test` runs the full pytest suite via turbo and passes (mock provider only — no network)
- [ ] SSE stream delivers tokens: meta event, incremental deltas, done, `[DONE]` observed via `curl -N`
- [ ] Mid-stream failure emits `error` event before `[DONE]`; pre-first-byte failure returns HTTP error status
- [ ] Provider factory resolves ollama-cloud / local / mock from settings; manual ollama-cloud probe documented in README
- [ ] `llm/` imports nothing from `agents/` or `api/` (boundary rule holds)
---
## Phase 2: Learner Surface UI
## Phase 2: Agent Framework
**Requirements:** REQ-006, REQ-007, REQ-008, REQ-009, REQ-010, REQ-011, REQ-012
**Persona:** frontend-engineer
**Goal:** All 7 learner pages render with mock data; navigation works; responsive
**Requirements:** REQ-2-004
**Goal:** Shared framework all six agents use: BaseAgent contract, session store, prompt library, registry, structured outputs — all tested against the mock provider
### Wave 1: Core learner pages
### Wave 1: Framework primitives (parallel — no shared files)
#### Task 2-1-01: Landing page
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/page.tsx`, `apps/web/components/learner/hero.tsx`, `apps/web/components/learner/how-it-works.tsx`, `apps/web/components/learner/program-highlights.tsx`, `apps/web/components/learner/testimonials.tsx`
- **Action:** Build landing page with hero section (headline, subheadline, CTA buttons), how-it-works section (Byte→Build→Demonstrate→Defend visual flow), program highlights (3-4 featured competency stacks), testimonials mockup (3 cards with avatar, quote, name, role), CTA to program catalog.
- **Verify:** Landing page renders with all sections; CTA links to /catalog
- **Done:** Landing page complete
#### Task 2-1-01: BaseAgent ABC
- **Persona:** ai-engineer**REQ:** REQ-2-004
- **Files:** `apps/ai-service/ai_service/agents/base.py`, `apps/ai-service/tests/agents/test_base.py`, `apps/ai-service/ai_service/agents/__init__.py`, `apps/ai-service/tests/agents/__init__.py`
- **Action:** `BaseAgent` ABC (D-018): `name`, `system_prompt`, `build_messages(history, learner_context)`, `stream_reply(...) -> AsyncIterator[ChatDelta]` (delegates to provider), `structured_reply(...)` (delegates to structured module, landed Wave 2). Subclass contract tested with a stub agent + mock provider.
- **Verify:** `pnpm ai:test` — test_base green
#### Task 2-1-02: Program catalog
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/catalog/page.tsx`, `apps/web/components/learner/stack-card.tsx`
- **Action:** Build program catalog page. Grid of 5 competency stack cards. Each card: stack name, role description, competency count, estimated duration, microcredential count, "Explore" button linking to stack detail.
- **Verify:** 5 stack cards render with mock data; clicking "Explore" navigates to stack detail
- **Done:** Catalog page complete
#### Task 2-1-02: SessionStore protocol + in-memory implementation
- **Persona:** ai-engineer**REQ:** REQ-2-004
- **Files:** `apps/ai-service/ai_service/agents/session.py`, `apps/ai-service/tests/test_session.py`
- **Action:** `SessionStore` protocol + `InMemorySessionStore` (D-019): asyncio.Lock-guarded dict, agent-scoped session keys, 20-message rolling window, 500-cap LRU eviction. Protocol shape is DB-migration-ready (A-003).
- **Verify:** test_session covers create/append/window-trim/LRU-eviction/agent scoping
#### Task 2-1-03: Competency stack view
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/catalog/[stackId]/page.tsx`, `apps/web/components/learner/competency-list.tsx`, `apps/web/components/learner/competency-item.tsx`, `apps/web/components/learner/microcredential-badge.tsx`
- **Action:** Build competency stack detail page. Header with stack name, role, description. List of 12-18 competencies. Each competency: name, description, status (locked/available/in-progress/mastered), microcredential badge, progress bar. Clicking a competency navigates to byte tutorial viewer.
- **Verify:** Competencies render with correct status indicators; microcredential badges display
- **Done:** Stack detail page complete
#### Task 2-1-03: Prompt library scaffolding
- **Persona:** ai-engineer**REQ:** REQ-2-004
- **Files:** `apps/ai-service/ai_service/prompts/coach.py`, `apps/ai-service/ai_service/prompts/tutor.py`, `apps/ai-service/ai_service/prompts/lab.py`, `apps/ai-service/ai_service/prompts/assessor.py`, `apps/ai-service/ai_service/prompts/proctor.py`, `apps/ai-service/ai_service/prompts/mentor.py`, `apps/ai-service/ai_service/prompts/__init__.py`
- **Action:** Per-agent module with a versioned `SYSTEM_PROMPT` constant + `render_context(learner_context) -> dict` using `str.format_map` for learner-context injection (D-018: prompts are code, versioned in git). Initial drafts for all six; final personas land in Phases 3-5.
- **Verify:** all six prompt modules import; render_context fills placeholders without KeyError
#### Task 2-1-04: Learner dashboard
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/dashboard/page.tsx`, `apps/web/components/learner/dashboard/active-competencies.tsx`, `apps/web/components/learner/dashboard/progress-graph.tsx`, `apps/web/components/learner/dashboard/recent-artifacts.tsx`, `apps/web/components/learner/dashboard/upcoming-defenses.tsx`, `apps/web/components/learner/dashboard/ai-tutor-chat.tsx`, `apps/web/components/learner/dashboard/milestone-tracker.tsx`
- **Action:** Build learner dashboard. Active competencies panel (current stack, progress). Progress graph (recharts area chart showing mastery over time). Recent artifacts (cards with artifact name, type, date). Upcoming defenses (list with competency name, date, status). AI tutor chat mockup (chat interface with pre-scripted responses, message input). Milestone tracker (progress through stack).
- **Verify:** Dashboard renders all panels; AI tutor chat displays scripted responses on "send"
- **Done:** Dashboard complete
#### Task 2-1-04: Learner context corpus
- **Persona:** ai-engineer**REQ:** REQ-2-004
- **Files:** `apps/ai-service/ai_service/corpus/learner_context.py`, `apps/ai-service/ai_service/corpus/__init__.py`
- **Action:** Pydantic-typed learner context (active stack, competencies, progress, recent artifacts) mirroring TS `packages/mock-data` IDs per D-021 convention (cross-referencing header comment, identical `comp-*`/`stack-*` ID strings).
- **Verify:** context renders into prompt placeholders; IDs match packages/mock-data strings
### Wave 2: Detail views
### Wave 2: Composition layers (depends on Wave 1)
#### Task 2-2-01: Byte tutorial viewer
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/learn/[competencyId]/page.tsx`, `apps/web/components/learner/byte-viewer/concept-panel.tsx`, `apps/web/components/learner/byte-viewer/worked-example.tsx`, `apps/web/components/learner/byte-viewer/viewer-tabs.tsx`
- **Action:** Build byte tutorial viewer. Layout: left panel (concept text, 3-7 minute read), right panel (worked example with code/design/simulation viewer mockup). Tabbed viewer (code, design, simulation). Navigation to build sandbox.
- **Verify:** Tutorial viewer renders with concept and example panels; tabs switch
- **Done:** Byte tutorial viewer complete
#### Task 2-2-01: Structured output defense
- **Persona:** ai-engineer**REQ:** REQ-2-004
- **Files:** `apps/ai-service/ai_service/agents/structured.py`, `apps/ai-service/tests/test_structured.py`
- **Action:** 4-layer defense (D-020): (1) `response_format` request with auto-degrade on provider 400; (2) prompt-embedded JSON schema; (3) parse: strip code fences → first balanced JSON object; (4) single bounded retry with validation-error feedback. Returns pydantic-validated model or raises `StructuredOutputError`.
- **Verify:** test_structured covers fenced/unfenced/invalid JSON, retry path, degrade path — all against mock provider
#### Task 2-2-02: Build sandbox mockup
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/build/[competencyId]/page.tsx`, `apps/web/components/learner/sandbox/toolbar.tsx`, `apps/web/components/learner/sandbox/file-explorer.tsx`, `apps/web/components/learner/sandbox/editor-area.tsx`, `apps/web/components/learner/sandbox/telemetry-sidebar.tsx`
- **Action:** Build sandbox mockup IDE UI. Layout: top toolbar (run, save, submit buttons), left sidebar (file explorer tree), center (code editor area with syntax-highlighted mock code), right sidebar (telemetry — process capture indicators showing commits, keystrokes, time spent). No real code execution.
- **Verify:** Sandbox UI renders with all panels; toolbar buttons are interactive (non-functional)
- **Done:** Build sandbox mockup complete
#### Task 2-2-02: Agent registry
- **Persona:** ai-engineer**REQ:** REQ-2-004
- **Files:** `apps/ai-service/ai_service/agents/registry.py`, `apps/ai-service/tests/test_registry.py`
- **Action:** Explicit registry: name → agent factory map with `register(name, factory)` / `get(name)`; raises on unknown agent. Agents are registered in their own phases (P3-P5).
- **Verify:** test_registry: register/get round-trip, unknown-agent error, duplicate registration error
#### Task 2-2-03: Assessment/defense mockup
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(learner)/defend/[competencyId]/page.tsx`, `apps/web/components/learner/assessment/rubric-display.tsx`, `apps/web/components/learner/assessment/ai-reviewer-panel.tsx`, `apps/web/components/learner/assessment/oral-defense.tsx`, `apps/web/components/learner/assessment/process-trace.tsx`, `apps/web/components/learner/assessment/artifact-viewer.tsx`
- **Action:** Build assessment/defense mockup. Layout: top (artifact viewer — submitted work preview), left (rubric display — competency criteria with checkmarks), right (AI reviewer panel — assessment results), bottom (oral defense interface — mic button mockup, waveform animation, transcript area). Process trace timeline (showing build steps with timestamps).
- **Verify:** Assessment mockup renders all panels; mic button has hover state; timeline displays mock events
- **Done:** Assessment/defense mockup complete
#### Task 2-2-03: Session + agent DI wiring into API layer
- **Persona:** backend-engineer**REQ:** REQ-2-004
- **Files:** `apps/ai-service/ai_service/api/deps.py` (update), `apps/ai-service/ai_service/api/chat.py` (update)
- **Action:** deps.py exposes SessionStore and provider singletons via DI. chat.py persists turn history through the session store (agent-scoped) and includes session ID in the meta event. API composes agents via DI — agents never import api/.
- **Verify:** chat request appends to and replays windowed history; `pnpm ai:test` green
### Must-Haves (Phase 2)
- [ ] Landing page renders with hero, how-it-works, highlights, testimonials, CTA
- [ ] Program catalog shows 5 competency stack cards
- [ ] Competency stack view shows 12-18 competencies with status and badges
- [ ] Learner dashboard shows all 6 panels including AI tutor chat mockup
- [ ] Byte tutorial viewer renders with concept and worked example panels
- [ ] Build sandbox mockup renders with toolbar, file explorer, editor, telemetry
- [ ] Assessment mockup renders with rubric, AI reviewer, oral defense, process trace
- [ ] All pages responsive at 375px, 768px, 1280px
- [ ] Navigation between all learner pages works
- [ ] BaseAgent unit tests pass (stub agent streams via mock provider)
- [ ] Session store tested: create/append, 20-message window trim, 500-cap LRU eviction, agent-scoped keys
- [ ] Structured output parsing tested against mock provider: fence-strip, first-balanced-object, invalid JSON, one bounded retry, response_format auto-degrade
- [ ] Registry tested: register/get/unknown/duplicate
- [ ] Six prompt modules render learner context without errors
- [ ] Module boundaries hold: `agents/` never imports `api/`; `llm/` never imports `agents/` or `api/`
---
## Phase 3: Marketplace Surface UI
## Phase 3: Coach + Tutor Agents
**Requirements:** REQ-013, REQ-014, REQ-015, REQ-016, REQ-017
**Persona:** frontend-engineer
**Goal:** All 5 marketplace pages render with mock data; search/filter interactive
**Requirements:** REQ-2-005, REQ-2-006
**Goal:** Both learner-facing conversational agents fully implemented with distinct personas, registered, routed through the chat streaming endpoint
### Wave 1: Job board + search
### Wave 1: Agent implementations (parallel — no shared files)
#### Task 3-1-01: Job board listing
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(marketplace)/page.tsx`, `apps/web/components/marketplace/job-card.tsx`, `apps/web/components/marketplace/job-listing-grid.tsx`
- **Action:** Build job board listing page. Layout: left sidebar (filters), main area (grid of job cards). Each job card: job title, company name, logo, location, remote badge, salary range, match score (percentage badge), required skills (tag chips), posted date. 20+ mock job listings.
- **Verify:** 20+ job cards render; match scores display; skill tags show
- **Done:** Job board listing complete
#### Task 3-1-01: Coach agent
- **Persona:** ai-engineer**REQ:** REQ-2-005
- **Files:** `apps/ai-service/ai_service/prompts/coach.py` (finalize), `apps/ai-service/ai_service/agents/coach.py`, `apps/ai-service/tests/test_coach.py`
- **Action:** Final Coach persona: pacing guidance, motivation, retrieval practice prompts; system prompt injects learner context (active stack, progress). `CoachAgent(BaseAgent)` streams replies. Mock provider scripts a distinct coach-voice response for tests.
- **Verify:** test_coach: build_messages includes system prompt + windowed history; stream_reply yields deltas; on-persona content asserted against mock script
#### Task 3-1-02: Search/filter UI
- **Persona:** frontend-engineer
- **Files:** `apps/web/components/marketplace/filter-sidebar.tsx`, `apps/web/components/marketplace/search-bar.tsx`, `apps/web/components/marketplace/saved-searches.tsx`
- **Action:** Build filter sidebar. Semantic search bar at top (with icon, placeholder text). Filters: skills (multi-select chips), seniority (dropdown), location (text input), remote toggle, salary range (min/max inputs). Saved searches mockup section. Client-side filtering of mock jobs using useState.
- **Verify:** Typing in search filters jobs; selecting skill filters jobs; toggling remote filters jobs
- **Done:** Search/filter UI functional with client-side filtering
#### Task 3-1-02: Tutor agent
- **Persona:** ai-engineer**REQ:** REQ-2-006
- **Files:** `apps/ai-service/ai_service/prompts/tutor.py` (finalize), `apps/ai-service/ai_service/agents/tutor.py`, `apps/ai-service/tests/test_tutor.py`
- **Action:** Final Tutor persona: concept delivery, Socratic questioning, worked examples. `TutorAgent(BaseAgent)` streams replies; mock scripts a distinct tutor-voice response.
- **Verify:** test_tutor mirrors test_coach; Coach and Tutor mock outputs are observably distinct
### Wave 2: Detail pages
### Wave 2: Registration + persona verification (depends on Wave 1)
#### Task 3-2-01: Job detail page
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(marketplace)/jobs/[jobId]/page.tsx`, `apps/web/components/marketplace/job-detail.tsx`, `apps/web/components/marketplace/skills-breakdown.tsx`, `apps/web/components/marketplace/related-jobs.tsx`
- **Action:** Build job detail page. Layout: header (job title, company, location, salary, apply button), body (full description, required competencies list, AI-matched skills breakdown with match percentages), sidebar (employer info card, related jobs list).
- **Verify:** Job detail renders with all sections; apply button is interactive (non-functional)
- **Done:** Job detail page complete
#### Task 3-2-01: Register Coach + Tutor; document cloud probe
- **Persona:** ai-engineer**REQ:** REQ-2-005, REQ-2-006
- **Files:** `apps/ai-service/ai_service/agents/registry.py` (update), `apps/ai-service/tests/test_registry.py` (update), `apps/ai-service/README.md` (update)
- **Action:** Register both agents in the explicit registry. Extend test_registry to assert both resolve. Document the manual ollama-cloud persona probe in README (curl commands with `AI_PROVIDER=ollama-cloud`): Coach and Tutor produce distinct on-persona responses; tests remain cloud-free.
- **Verify:** `pnpm ai:test` green; manual probe against ollama-cloud shows distinct personas (documented, not automated)
#### Task 3-2-02: Employer profile
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(marketplace)/employers/[employerId]/page.tsx`, `apps/web/components/marketplace/employer-header.tsx`, `apps/web/components/marketplace/employer-openings.tsx`, `apps/web/components/marketplace/employer-culture.tsx`
- **Action:** Build employer profile page. Header: logo, company name, tagline, industry, size, location, social links. Body: about section, culture section (mock photos grid), open positions list (job cards).
- **Verify:** Employer profile renders with all sections; social links are clickable (mock URLs)
- **Done:** Employer profile complete
### Wave 3: Chat endpoint agent routing (depends on Wave 2)
#### Task 3-2-03: Pricing page
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(marketplace)/pricing/page.tsx`, `apps/web/components/marketplace/pricing-card.tsx`, `apps/web/components/marketplace/feature-comparison.tsx`
- **Action:** Build pricing page. Three pricing cards: Job Posting (single — $49, bundle — $399 for 10, enterprise — custom). Talent Access plans (starter, pro, enterprise). Feature comparison table. CTA buttons on each plan.
- **Verify:** Pricing page renders with 3 tiers and comparison table; responsive
- **Done:** Pricing page complete
#### Task 3-3-01: Agent routing on /v1/chat/stream
- **Persona:** backend-engineer**REQ:** REQ-2-005, REQ-2-006
- **Files:** `apps/ai-service/ai_service/api/chat.py` (update), `apps/ai-service/ai_service/api/deps.py` (update), `apps/ai-service/tests/api/test_chat_stream.py` (update)
- **Action:** Chat request gains `agent` field (validated against the registry; unknown agent → 422). Endpoint resolves the agent via DI, persists to the agent-scoped session, meta event carries the agent name. No autonomous routing in v0.2 (A-007).
- **Verify:** TestClient tests: `agent=coach` and `agent=tutor` route correctly, session scoped per agent, unknown agent rejected
### Must-Haves (Phase 3)
- [ ] Job board shows 20+ mock job listings with match scores
- [ ] Search bar filters jobs client-side
- [ ] Filter sidebar filters by skills, seniority, remote, salary
- [ ] Job detail page shows full description, competencies, skills breakdown
- [ ] Employer profile shows company info, culture, open positions
- [ ] Pricing page shows 3 tiers with feature comparison
- [ ] All pages responsive at 375px, 768px, 1280px
- [ ] Both agents produce distinct, on-persona responses (mock-asserted; manual ollama-cloud probe documented in README)
- [ ] Agent routing tested: coach/tutor resolve via registry; unknown agent returns 422
- [ ] Both agents exposed end-to-end via `POST /v1/chat/stream` with agent-scoped session history
- [ ] `pnpm ai:test` green; no cloud calls in tests
---
## Phase 4: Employer Dashboard UI
## Phase 4: Lab + Assessor Agents
**Requirements:** REQ-018, REQ-019, REQ-020, REQ-021
**Persona:** frontend-engineer
**Goal:** All 4 employer dashboard pages render with mock data; charts display
**Requirements:** REQ-2-007, REQ-2-008
**Goal:** Lab consumes simulated sandbox telemetry and streams in-flow feedback; Assessor applies rubrics to pre-baked artifacts and returns structured scores — both over mock engine inputs
### Wave 1: Dashboard + talent search
### Wave 1: Mock engine inputs (parallel — no shared files)
#### Task 4-1-01: Employer dashboard overview
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(employer)/page.tsx`, `apps/web/components/employer/overview/metric-cards.tsx`, `apps/web/components/employer/overview/applicant-pipeline.tsx`, `apps/web/components/employer/overview/talent-matches.tsx`, `apps/web/components/employer/overview/analytics-charts.tsx`
- **Action:** Build employer dashboard overview. Top row: 4 metric cards (active postings, total applicants, talent matches, placement rate). Middle: applicant pipeline (kanban-style columns: applied, screening, interview, offer, hired). Right: talent matches (list of matched candidates with match score). Bottom: analytics charts (recharts — bar chart for postings over time, donut chart for applicant sources, line chart for placement trends).
- **Verify:** Dashboard renders with metric cards, pipeline, matches, and 3 chart types
- **Done:** Employer dashboard overview complete
#### Task 4-1-01: Simulated sandbox telemetry corpus
- **Persona:** ai-engineer**REQ:** REQ-2-007
- **Files:** `apps/ai-service/ai_service/corpus/telemetry.py`
- **Action:** Pydantic-typed Lab telemetry scenarios: scripted build-session event streams (keystrokes, commits, test runs, errors, idle gaps) keyed by scenario ID, aligned with packages/mock-data IDs (D-021).
- **Verify:** scenarios import, validate, and are addressable by ID
#### Task 4-1-02: Talent search
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(employer)/talent/page.tsx`, `apps/web/components/employer/talent/candidate-card.tsx`, `apps/web/components/employer/talent/talent-filters.tsx`
- **Action:** Build talent search page. Layout: top (search bar + AI-matched filters), main (grid of candidate cards). Each candidate card: avatar, name, headline (competency stack + level), microcredential badges (top 3), artifact count, defense score, match percentage, "View Profile" button. 15+ mock candidates. Client-side filtering.
- **Verify:** 15+ candidate cards render; filters work client-side; clicking "View Profile" navigates to candidate profile
- **Done:** Talent search complete
#### Task 4-1-02: Pre-baked artifacts + rubrics corpus
- **Persona:** ai-engineer**REQ:** REQ-2-008
- **Files:** `apps/ai-service/ai_service/corpus/artifacts.py`
- **Action:** Pre-baked artifacts (code, design, simulation), assessment rubrics (criteria, levels, weights), and defense transcripts keyed by ID — the Assessor's mock inputs (real engines are v0.3+).
- **Verify:** rubric/artifact/transcript fixtures validate; IDs align with TS mock data
### Wave 2: Detail + management
#### Task 4-1-03: TS mock-data alignment for engine inputs
- **Persona:** data-engineer — **REQ:** REQ-2-012
- **Files:** `packages/mock-data/ai-scenarios.ts` (new), `packages/mock-data/index.ts` (update)
- **Action:** Export scenario/artifact ID constants + display metadata used by the Phase 6 learner panels, mirroring `ai_service/corpus/` IDs exactly (D-021). Data-engineer owns the TS side; headers cross-reference the Python corpus.
- **Verify:** `pnpm typecheck` passes; IDs string-equal to corpus IDs
#### Task 4-2-01: Candidate profile view
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(employer)/talent/[candidateId]/page.tsx`, `apps/web/components/employer/talent-profile/artifact-gallery.tsx`, `apps/web/components/employer/talent-profile/process-trace-summary.tsx`, `apps/web/components/employer/talent-profile/defense-transcripts.tsx`, `apps/web/components/employer/talent-profile/competency-mini-graph.tsx`, `apps/web/components/employer/talent-profile/microcredential-verification.tsx`
- **Action:** Build candidate profile view. Header: avatar, name, headline, match score, contact button. Body sections: artifact gallery (grid of project artifacts with thumbnails), process trace summary (timeline of build steps), oral defense transcripts (accordion of defense sessions with Q&A), competency mini-graph (react-flow visualization of candidate's competencies), microcredential verification (list of earned credentials with verification badges).
- **Verify:** All profile sections render; defense transcripts accordion expands; mini-graph renders
- **Done:** Candidate profile complete
### Wave 2: Agent implementations (depends on Wave 1)
#### Task 4-2-02: Posting management
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(employer)/postings/page.tsx`, `apps/web/components/employer/postings/posting-list.tsx`, `apps/web/components/employer/postings/posting-form.tsx`, `apps/web/components/employer/postings/applicant-list.tsx`
- **Action:** Build posting management page. Layout: left (list of job postings with status badges: active, draft, expired), right (selected posting detail or create/edit form). Form fields: title, description, required competencies (multi-select), seniority, salary range, location, remote toggle. Applicant list per posting (table with name, applied date, status, match score). "Create New Posting" button opens form.
- **Verify:** Posting list renders; form is interactive (inputs work, non-functional submit); applicant list displays
- **Done:** Posting management complete
#### Task 4-2-01: Lab agent
- **Persona:** ai-engineer**REQ:** REQ-2-007
- **Files:** `apps/ai-service/ai_service/prompts/lab.py` (finalize), `apps/ai-service/ai_service/agents/lab.py`, `apps/ai-service/tests/test_lab.py`
- **Action:** `LabAgent(BaseAgent)` consumes a telemetry scenario, builds messages summarizing the event stream, streams concrete in-flow feedback (what happened, what to adjust, next step). No session chat — scenario-driven.
- **Verify:** test_lab: given a mock scenario, feedback references scenario events (mock-scripted assertions)
#### Task 4-2-02: Assessor agent
- **Persona:** ai-engineer — **REQ:** REQ-2-008
- **Files:** `apps/ai-service/ai_service/prompts/assessor.py` (finalize), `apps/ai-service/ai_service/agents/assessor.py`, `apps/ai-service/tests/test_assessor.py`
- **Action:** `AssessorAgent(BaseAgent)` applies a rubric to an artifact + defense transcript via `structured_reply`, returning a pydantic-validated rubric score model (per-criterion scores, strengths, gaps, verdict).
- **Verify:** test_assessor: structured output validates against the rubric model; failure modes exercise the 4-layer defense
### Wave 3: Registration (depends on Wave 2)
#### Task 4-3-01: Register Lab + Assessor
- **Persona:** ai-engineer — **REQ:** REQ-2-007, REQ-2-008
- **Files:** `apps/ai-service/ai_service/agents/registry.py` (update), `apps/ai-service/tests/test_registry.py` (update)
- **Action:** Register both agents; extend registry tests.
- **Verify:** registry resolves coach/tutor/lab/assessor; `pnpm ai:test` green
### Wave 4: Endpoints (depends on Wave 3)
#### Task 4-4-01: Lab feedback endpoint
- **Persona:** backend-engineer — **REQ:** REQ-2-007
- **Files:** `apps/ai-service/ai_service/api/lab.py`, `apps/ai-service/tests/api/test_lab.py`
- **Action:** `POST /v1/lab/feedback` with scenario ID → resolves corpus scenario + Lab agent → SSE stream using the D-016 envelope (meta names agent=lab). Unknown scenario → 404.
- **Verify:** TestClient streams meta + deltas + done + `[DONE]`; unknown scenario 404
#### Task 4-4-02: Assessment evaluate endpoint
- **Persona:** backend-engineer — **REQ:** REQ-2-008
- **Files:** `apps/ai-service/ai_service/api/assessment.py`, `apps/ai-service/tests/api/test_assessment.py`
- **Action:** `POST /v1/assessment/evaluate` with artifact ID → resolves corpus artifact/rubric/transcript + Assessor agent → JSON response (validated rubric score model). Unknown artifact → 404.
- **Verify:** TestClient returns validated rubric JSON; unknown artifact 404; `pnpm ai:test` green
### Must-Haves (Phase 4)
- [ ] Dashboard overview shows 4 metric cards, applicant pipeline, 3 charts
- [ ] Talent search shows 15+ candidate cards with filters
- [ ] Candidate profile shows artifact gallery, process trace, defense transcripts, mini-graph, credentials
- [ ] Posting management shows posting list, create/edit form, applicant list
- [ ] All pages responsive at 375px, 768px, 1280px
- [ ] Lab produces scenario-relevant in-flow feedback for mock telemetry scenarios (mock provider, tested)
- [ ] Assessor returns structured rubric scores (pydantic-validated JSON) for pre-baked artifacts/transcripts
- [ ] `POST /v1/lab/feedback` streams (meta → deltas → done → `[DONE]`); `POST /v1/assessment/evaluate` returns validated JSON
- [ ] Unknown scenario/artifact IDs return 404
- [ ] Corpus IDs align with packages/mock-data (D-021); `pnpm ai:test` and `pnpm typecheck` green
---
## Phase 5: Admin Surface UI
## Phase 5: Proctor + Mentor Agents
**Requirements:** REQ-022, REQ-023, REQ-024, REQ-025
**Persona:** frontend-engineer
**Goal:** All 4 admin pages render; competency graph viewer interactive
**Requirements:** REQ-2-009, REQ-2-010
**Goal:** Proctor classifies integrity signals with coaching interventions from mock telemetry; Mentor generates long-horizon career narrative; both exposed via endpoints
### Wave 1: Overview + learner management
### Wave 1: Proctor scenarios + Mentor agent (parallel — no shared files)
#### Task 5-1-01: Admin overview
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(admin)/page.tsx`, `apps/web/components/admin/overview/platform-metrics.tsx`, `apps/web/components/admin/overview/activity-feed.tsx`, `apps/web/components/admin/overview/system-health.tsx`
- **Action:** Build admin overview. Top: platform metric cards (total learners, total employers, placements, completion rate, NPS). Middle: activity feed (timeline of recent platform events — new registrations, completions, job postings, placements). Right: system health mockup (status indicators for services, uptime bars, error rates).
- **Verify:** Admin overview renders with metrics, activity feed, system health
- **Done:** Admin overview complete
#### Task 5-1-01: Proctor telemetry scenarios
- **Persona:** ai-engineer**REQ:** REQ-2-009
- **Files:** `apps/ai-service/ai_service/corpus/telemetry.py` (update)
- **Action:** Add proctor scenarios: tab switches, idle time, paste events, focus loss — scripted integrity-relevant event sets keyed by scenario ID.
- **Verify:** proctor scenarios validate; distinguishable from lab scenarios by type
#### Task 5-1-02: Learner management
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(admin)/learners/page.tsx`, `apps/web/components/admin/learners/learner-table.tsx`, `apps/web/components/admin/learners/learner-detail.tsx`
- **Action:** Build learner management page. Layout: searchable learner table (columns: name, email, stack, progress %, status, joined date). Search and filter (by stack, status). Clicking a learner opens detail panel (progress tracking, competency completion, credential issuance log, recent activity).
- **Verify:** Learner table renders with mock data; search filters; detail panel opens on click
- **Done:** Learner management complete
#### Task 5-1-02: Mentor agent (+ registration)
- **Persona:** ai-engineer**REQ:** REQ-2-010
- **Files:** `apps/ai-service/ai_service/prompts/mentor.py` (finalize), `apps/ai-service/ai_service/agents/mentor.py`, `apps/ai-service/tests/test_mentor.py`, `apps/ai-service/ai_service/agents/registry.py` (update)
- **Action:** `MentorAgent(BaseAgent)`: long-horizon career narrative — trajectory story, competency-stack progression guidance, market positioning — streaming, session-backed. Registered centrally in `registry.py` (single registration pattern, G-4).
- **Verify:** test_mentor: narrative references learner context (mock-scripted); registry resolves mentor
### Wave 2: Graph viewer + moderation
### Wave 2: Proctor agent + Mentor endpoint (depends on Wave 1)
#### Task 5-2-01: Competency graph viewer
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(admin)/graph/page.tsx`, `apps/web/components/admin/graph/competency-graph.tsx`, `apps/web/components/admin/graph/node-detail.tsx`
- **Action:** Build competency graph viewer using @xyflow/react. Interactive node/edge graph showing competency stacks and their relationships. Nodes: competency stacks (colored by category) and individual competencies. Edges: prerequisite relationships. Node click opens detail panel (competency name, description, stack, prerequisites, learners mastering it). Controls: zoom, pan, fit-to-screen. Mock graph data.
- **Verify:** Graph renders with nodes and edges; nodes are clickable; detail panel opens; zoom/pan works
- **Done:** Competency graph viewer complete
#### Task 5-2-01: Proctor agent (+ registration)
- **Persona:** ai-engineer**REQ:** REQ-2-009
- **Files:** `apps/ai-service/ai_service/prompts/proctor.py` (finalize), `apps/ai-service/ai_service/agents/proctor.py`, `apps/ai-service/tests/test_proctor.py`, `apps/ai-service/ai_service/agents/registry.py` (update)
- **Action:** `ProctorAgent(BaseAgent)`: consumes proctor scenario → `structured_reply` returns pydantic-validated signal classification (severity, signal type) + recommended coaching intervention (supportive, not punitive). Registered centrally in `registry.py` (single registration pattern, G-4).
- **Verify:** test_proctor: classified signals + interventions validate for each mock scenario; registry resolves all six agents
#### Task 5-2-02: Marketplace moderation
- **Persona:** frontend-engineer
- **Files:** `apps/web/app/(admin)/moderation/page.tsx`, `apps/web/components/admin/moderation/review-queue.tsx`, `apps/web/components/admin/moderation/verification-queue.tsx`, `apps/web/components/admin/moderation/flagged-content.tsx`
- **Action:** Build marketplace moderation page. Three tabs: Job Posting Review (queue of pending job postings with approve/reject buttons), Employer Verification (queue of employers awaiting identity verification), Flagged Content (list of reported content with reason, reporter, actions). All buttons non-functional but interactive (hover states, click feedback).
- **Verify:** Moderation page renders with 3 tabs; queues display mock items; tab switching works
- **Done:** Marketplace moderation complete
#### Task 5-2-02: Mentor narrative endpoint
- **Persona:** backend-engineer**REQ:** REQ-2-010
- **Files:** `apps/ai-service/ai_service/api/mentor.py`, `apps/ai-service/tests/api/test_mentor.py`
- **Action:** `POST /v1/mentor/narrative` → Mentor agent → SSE stream with D-016 envelope, session-backed.
- **Verify:** TestClient streams meta (agent=mentor) → deltas → done → `[DONE]`
### Wave 3: Proctor endpoint (depends on Wave 2)
#### Task 5-3-01: Proctor signals endpoint
- **Persona:** backend-engineer — **REQ:** REQ-2-009
- **Files:** `apps/ai-service/ai_service/api/proctor.py`, `apps/ai-service/tests/api/test_proctor.py`
- **Action:** `POST /v1/proctor/signals` with scenario ID → resolves corpus scenario + Proctor agent → JSON response (validated signals + interventions). Unknown scenario → 404.
- **Verify:** TestClient returns classified signals JSON; `pnpm ai:test` green — full suite (all six agents registered)
### Must-Haves (Phase 5)
- [ ] Admin overview shows platform metrics, activity feed, system health
- [ ] Learner management shows searchable table with detail panel
- [ ] Competency graph viewer renders interactive graph with clickable nodes
- [ ] Marketplace moderation shows 3 tabbed queues
- [ ] All pages responsive at 375px, 768px, 1280px
- [ ] Proctor produces classified signals with recommended coaching interventions for each mock scenario (structured JSON, validated)
- [ ] Mentor produces coherent long-horizon career narrative (streaming, session-backed)
- [ ] `POST /v1/proctor/signals` returns validated JSON; `POST /v1/mentor/narrative` streams
- [ ] Registry resolves all six agents; full ai-service test suite green, cloud-free
---
## Phase 6: Polish + Integration
## Phase 6: Learner Surface Integration
**Requirements:** REQ-026, REQ-027, REQ-028
**Persona:** frontend-engineer, design-system-engineer
**Goal:** Cross-surface consistency, dark mode everywhere, Storybook
**Requirements:** REQ-2-011, REQ-2-012
**Goal:** v0.1 learner surfaces wired to the real ai-service: streaming chat with agent switcher, Lab/Assessor/Proctor/Mentor outputs surfaced, error/loading states, build + typecheck green
**Note (G-2):** End-to-end verification may run with `AI_PROVIDER=mock` as a fallback — the requirement is the real ai-service over HTTP (not canned client-side responses); provider choice is service-internal. This prevents an ollama-cloud outage from blocking P6 verification. Cloud persona probes remain separate (Task 3-2-01).
### Wave 1: Navigation + consistency
### Wave 1: Client plumbing + primitives (parallel — no shared files)
#### Task 6-1-01: Cross-surface navigation polish
- **Persona:** frontend-engineer
- **Files:** `apps/web/components/navigation-shell.tsx` (update), `apps/web/components/role-switcher.tsx` (update), `apps/web/components/breadcrumbs.tsx`
- **Action:** Polish cross-surface navigation. Role switcher dropdown with surface-specific sub-navigation. Breadcrumbs on all pages showing current location. Consistent header/footer across all 4 surfaces. Active nav link highlighting.
- **Verify:** Breadcrumbs show on all pages; role switcher dropdown works; active links highlighted
- **Done:** Cross-surface navigation polished
#### Task 6-1-01: useChatStream hook
- **Persona:** frontend-engineer**REQ:** REQ-2-011
- **Files:** `apps/web/hooks/use-chat-stream.ts`, `apps/web/.env.example` (update)
- **Action:** `useChatStream(agent)` hook: `fetch` POST to `${NEXT_PUBLIC_AI_SERVICE_URL}/v1/chat/stream` (default `http://localhost:8420`, A-002 — no API-route proxy); consumes `ReadableStream` with byte buffering, frame split on `\n\n`, joined `data:` lines; **ignores frames containing no `data:` lines (sse-starlette `: ping` keep-alive comment frames)** — TestClient streams are too short to surface pings, but real cloud delta gaps emit them (G-1); handles meta / delta / done / error events and `[DONE]` sentinel; idempotent `AbortController.abort()` in effect cleanup; exposes `{messages, isStreaming, error, send, retry, abort}`. `.env.example` gains `NEXT_PUBLIC_AI_SERVICE_URL`.
- **Verify:** hook unit-tested or exercised via the chat UI; unmount mid-stream aborts cleanly (no state updates after unmount)
#### Task 6-1-02: Visual consistency audit
- **Persona:** design-system-engineer
- **Files:** `apps/web/app/globals.css` (update), `packages/ui/src/tokens/index.ts` (update if needed)
- **Action:** Audit all surfaces for visual consistency. Check: typography scale (all text uses design tokens), color palette (no hardcoded colors), spacing system (all margins/padding use design tokens), dark mode (all pages support dark mode toggle), WCAG AA contrast (all text meets 4.5:1 ratio). Fix any inconsistencies found.
- **Verify:** No hardcoded colors; dark mode works on all pages; contrast check passes
- **Done:** Visual consistency audit complete
#### Task 6-1-02: Agent switcher + streaming primitives
- **Persona:** design-system-engineer**REQ:** REQ-2-011
- **Files:** `packages/ui/src/primitives/agent-switcher.tsx`, `packages/ui/src/primitives/stream-status.tsx`, `packages/ui/src/primitives/toast.tsx`, `packages/ui/src/primitives/index.ts` (update), `packages/ui/src/index.ts` (update)
- **Action:** Token-driven primitives: AgentSwitcher (segmented coach/tutor control with active state), StreamStatus (idle/streaming/error indicator), Toast with error variant. Dark mode + WCAG AA contrast; exported from `@nextcraft/ui`.
- **Verify:** primitives import from `@nextcraft/ui`; storybook stories render (dark + light)
### Wave 2: Storybook
### Wave 2: Chat rewrite + Byte viewer panel (depends on Wave 1)
#### Task 6-2-01: Storybook setup
- **Persona:** design-system-engineer
- **Files:** `apps/web/.storybook/main.ts`, `apps/web/.storybook/preview.ts`, `packages/ui/src/primitives/*.stories.tsx` (one per primitive)
- **Action:** Set up Storybook for the component library. Configure for Next.js + Tailwind. Create stories for all 12 primitives and key composites (JobCard, CandidateCard, CompetencyBadge, MetricCard). Each story: default, variants, sizes, dark mode.
- **Verify:** `pnpm storybook` starts; all primitive stories render; dark mode toggle in Storybook
- **Done:** Storybook documents all primitives + key composites
#### Task 6-2-01: Real streaming learner chat
- **Persona:** frontend-engineer — **REQ:** REQ-2-011
- **Files:** `apps/web/components/learner/ai-tutor-chat.tsx` (rewrite), `apps/web/components/learner/agent-switcher.tsx` (composition wrapper)
- **Action:** Replace the canned `aiTutorResponses` behavior with `useChatStream`: agent switcher (Coach/Tutor per A-007), token-by-token rendering, streaming cursor + loading state, error state with retry button when ai-service is down (A-010), suggested-action chips from the meta event. Seed welcome message stays static.
- **Verify:** with ai-service running, messages stream visibly token-by-token; with ai-service stopped, error state + retry appears (no crash, no console errors)
#### Task 6-2-02: Byte viewer Tutor panel
- **Persona:** frontend-engineer — **REQ:** REQ-2-011 (agent routing: byte viewer always uses Tutor, A-007)
- **Files:** `apps/web/app/(learner)/learn/[competencyId]/page.tsx` (update), `apps/web/components/learner/byte-tutor-panel.tsx` (new)
- **Action:** Byte viewer gains a Tutor explanation panel (byte viewer always uses Tutor, A-007): "Explain this byte" streams a Socratic concept walkthrough for the current competency via useChatStream (agent fixed to tutor).
- **Verify:** on a byte page, the panel streams a Tutor explanation; error state when service down
### Wave 3: Lab / Assessment / Mentor panels (depends on Wave 1 hook; parallel — no shared files)
#### Task 6-3-01: Sandbox Lab feedback panel
- **Persona:** frontend-engineer — **REQ:** REQ-2-012
- **Files:** `apps/web/app/(learner)/build/[competencyId]/page.tsx` (update), `apps/web/components/learner/lab-feedback-panel.tsx` (new)
- **Action:** Sandbox telemetry sidebar gains a Lab feedback panel: posts the scenario ID (from `packages/mock-data/ai-scenarios`) to `/v1/lab/feedback`, streams in-flow feedback into the panel; loading + error states.
- **Verify:** sandbox page streams Lab feedback for the mock scenario; error state when service down
#### Task 6-3-02: Assessment Assessor + Proctor surfaces
- **Persona:** frontend-engineer — **REQ:** REQ-2-012
- **Files:** `apps/web/app/(learner)/defend/[competencyId]/page.tsx` (update), `apps/web/components/learner/assessor-results-panel.tsx` (new), `apps/web/components/learner/proctor-banner.tsx` (new)
- **Action:** Assessment mockup: AI reviewer panel calls `/v1/assessment/evaluate` with the artifact ID and renders the structured rubric scores (per-criterion bars, strengths, gaps, verdict); a Proctor integrity banner surfaces `/v1/proctor/signals` classifications with coaching tone. Loading skeletons + error states.
- **Verify:** defend page renders real Assessor rubric output + Proctor banner; error states when service down
#### Task 6-3-03: Dashboard Mentor panel
- **Persona:** frontend-engineer — **REQ:** REQ-2-012
- **Files:** `apps/web/app/(learner)/dashboard/page.tsx` (update), `apps/web/components/learner/mentor-panel.tsx` (new)
- **Action:** Learner dashboard gains a Mentor panel: streams career narrative from `/v1/mentor/narrative` (learner progress context), with regenerate button, loading + error states. Sits alongside the existing AI tutor chat.
- **Verify:** dashboard shows streaming Mentor narrative; error state when service down
### Must-Haves (Phase 6)
- [ ] Breadcrumbs on all pages
- [ ] Role switcher dropdown with surface-specific nav
- [ ] Active nav link highlighting
- [ ] No hardcoded colors — all from design tokens
- [ ] Dark mode works on all 4 surfaces
- [ ] WCAG AA contrast passes
- [ ] Storybook runs with all primitive stories
- [ ] With ai-service running: learner chat at http://localhost:3000/dashboard streams real responses token-by-token (mock provider fallback allowed per G-2 — service over HTTP is the requirement)
- [ ] Agent switcher flips Coach ↔ Tutor and the response persona changes accordingly
- [ ] Hook tolerates keep-alive comment frames (`: ping`, no data lines) during live streams (G-1)
- [ ] With ai-service stopped: all chat/panels show error states with retry — no crashes, no unhandled promise rejections, no console errors
- [ ] Byte viewer, sandbox, and assessment mockups surface Tutor/Lab/Assessor/Proctor outputs; dashboard shows Mentor narrative
- [ ] Unmounting/navigating mid-stream aborts cleanly (no post-unmount state updates)
- [ ] `pnpm build` and `pnpm typecheck` pass; `pnpm ai:test` still green
---
## MVP/UX Sections
## Phase 7: Final Review + Ship (no planned tasks)
### User-Facing Surface
The primary user-facing surface is the **Nextcraft web application** at `localhost:3000` (dev) with 4 route groups:
- `/` — Learner surface (landing, catalog, dashboard, learn, build, defend)
- `/marketplace` — Marketplace surface (jobs, employers, pricing)
- `/employer` — Employer dashboard (overview, talent, postings)
- `/admin` — Admin surface (overview, learners, graph, moderation)
Orchestrated by the SHIP stage, not this plan: multi-persona code review (correctness, testing, secrets hygiene — key absent from code/logs/commits/errors, localhost-only CORS, no PII in prompts), project health audit (reconstruction test, .ciagent/ discipline, branch/commit hygiene), then merge milestone → main, tag v0.2.0, create Gitea release, mark all 12 v0.2 requirements complete.
### Happy Path
1. User visits landing page → sees hero, how-it-works (Byte→Build→Demonstrate→Defend), program highlights
2. User clicks "Explore Programs" → program catalog shows 5 competency stacks
3. User clicks "AI Orchestration Engineer" → competency stack view shows 15 competencies
4. User clicks a competency → byte tutorial viewer shows concept + worked example
5. User clicks "Start Building" → build sandbox mockup shows IDE UI with telemetry
6. User clicks "Submit for Assessment" → assessment/defense mockup shows rubric, AI reviewer, oral defense interface
7. User navigates to marketplace → job board shows AI-era job listings with match scores
8. User searches/filters jobs → results update client-side
9. User clicks a job → job detail page shows full description, required competencies, employer info
10. User switches to employer dashboard → overview shows metrics, pipeline, charts
11. User searches talent → candidate cards with microcredentials, defense scores
12. User clicks a candidate → profile shows artifact gallery, process trace, defense transcripts
13. User switches to admin → overview shows platform metrics, activity feed
14. User opens competency graph viewer → interactive graph with clickable nodes
**Release-note honesty (G-5):** the v0.2.0 release note must explicitly state that Lab/Assessor/Proctor operate on mock engine inputs (real engines are v0.3+), per D-015.
### UX Acceptance Criteria
1. All 4 surfaces are accessible via the role switcher in the header
2. Navigation is consistent across all surfaces (same header, footer, breadcrumb pattern)
3. All pages are responsive at 375px (mobile), 768px (tablet), 1280px (desktop)
4. Dark mode toggle works on all surfaces
5. All interactive elements have hover states and click feedback
6. Form inputs are non-functional but visually complete (placeholders, labels, validation states)
7. Mock data is realistic and typed — no "lorem ipsum" or placeholder text
8. All charts render correctly (recharts) with mock data
9. Competency graph viewer is interactive (zoom, pan, click nodes)
10. AI tutor chat displays pre-scripted responses on message "send"
11. No console errors on any page
12. `pnpm build` succeeds without warnings
**Dead-code disposal (G-5):** review must dispose of `aiTutorResponses` (packages/mock-data/ai-tutor-responses.ts) — its only consumer is rewritten in Task 6-2-01; remove the export or mark it deprecated.
---
## User-Facing Surface
The primary user-facing surface is the **learner dashboard and learning flow** at `http://localhost:3000`, backed by the real ai-service at `http://localhost:8420`:
- `/dashboard` — AI tutor chat (Coach/Tutor switcher, streaming) + Mentor career-narrative panel
- `/learn/[competencyId]` — byte viewer with streaming Tutor explanations
- `/build/[competencyId]` — sandbox with Lab in-flow feedback panel (mock telemetry)
- `/defend/[competencyId]` — assessment with live Assessor rubric scores + Proctor integrity banner
The marketplace, employer, and admin surfaces are unchanged from v0.1.
## Happy Path
1. Learner opens `/dashboard` → chat shows welcome message; meta event confirms coach/model in the stream
2. Learner types "I'm stuck on multi-agent communication" → reply streams token-by-token with pacing guidance + a retrieval-practice prompt
3. Learner switches to **Tutor** → asks the same question → gets a Socratic concept walkthrough instead
4. Learner opens a byte tutorial → Tutor panel streams an explanation of the current competency
5. Learner opens the build sandbox → Lab panel streams feedback on the simulated telemetry scenario
6. Learner opens the defense mockup → Assessor panel shows structured rubric scores; Proctor banner shows integrity signals in coaching tone
7. Back on `/dashboard`, the Mentor panel streams a career narrative tied to the learner's progress
8. Learner kills ai-service (or it crashes) → next message shows an inline error state with **Retry**; restarting the service and retrying resumes streaming
## UX Acceptance Criteria
1. Streaming is visibly incremental — tokens appear as they arrive, not as one blob
2. Agent switcher shows the active agent (Coach/Tutor) and the response persona visibly changes
3. Loading state during connection (streaming cursor / skeleton) before first token
4. When ai-service is unreachable: inline error state + retry action on every chat/panel — no crashes, no console errors, no blank UI
5. `[DONE]` reliably ends the stream (input re-enables, no stuck "typing" state)
6. Navigating away mid-stream aborts cleanly — no leaked requests or post-unmount updates
7. All new UI uses design tokens, supports dark mode, meets WCAG AA contrast
8. Responsive at 375px, 768px, 1280px
9. No hardcoded model names or URLs in UI code — all via `NEXT_PUBLIC_AI_SERVICE_URL` and server meta events
10. `pnpm build` and `pnpm typecheck` pass with zero errors
+43 -19
View File
@@ -8,38 +8,59 @@ Nextcraft is an AI-native outcome school where graduates prove what they can bui
---
## Current Milestone: v0.1UI/UX Prototype
## Current Milestone: v0.2AI Tutor Architecture
**Scope:** High-fidelity interactive prototype of all four Nextcraft surfaces with realistic mock data, navigation flows, responsive layouts, and a shared component library. No backend, no business logic, no database — everything static/mocked.
**Scope:** The six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface chat UI with streaming responses. Provider-agnostic LLM layer (ollama-cloud default, local endpoint + deterministic mock for tests). Lab/Assessor/Proctor operate on mock engine inputs (simulated telemetry, pre-baked artifacts) — their real engines (sandbox fabric, assessment engine, identity verification) are v0.3+.
**Constraint:** MAJOR 0 until MVP is released. No business logic until the first milestone prototype is agreed upon by the founder.
**Status of v0.1:** Complete and shipped (v0.1.0). Founder agreement recorded (D-013).
**Tech stack:** TypeScript monorepo (pnpm/turborepo) with Next.js for all web surfaces. Python FastAPI AI services planned for later milestones (not in v0.1).
**Tech stack:** v0.1 TS monorepo (pnpm/turborepo, Next.js) + new Python FastAPI service (`apps/ai-service`) with pydantic, SSE streaming, and an OpenAI-compatible provider client.
---
## Requirements (Validated)
The following requirements have been validated during specification and are locked for milestone v0.1:
The following requirements have been validated during specification and are locked for milestone v0.2 (REQ-F-001..006 activated from the deferred pool):
1. Learner surface UI — landing page, program catalog, competency stack view, learner dashboard, byte tutorial viewer, build sandbox mockup, assessment/defense mockup
2. Marketplace surface UI — job board listing, job detail page, employer profile, search/filter UI, job posting packages pricing page
3. Employer dashboard UI — overview, talent search, candidate profile view, posting management
4. Admin surface UI — overview, learner management, competency graph viewer, marketplace moderation
5. Shared component library — design system, reusable UI primitives, surface-specific theming
6. Responsive layout system — mobile, tablet, desktop breakpoints
7. Mock data layer — typed, realistic data reflecting the vision (competency stacks, job listings, candidate profiles)
8. Navigation/routing — cross-surface navigation, role-based route groups
1. AI tutor service infrastructure — `apps/ai-service` FastAPI application, provider-agnostic LLM client, SSE streaming, session/state handling
2. Agent framework — base agent contracts, prompt management, streaming pipeline, structured outputs
3. Coach agent — pacing, motivation, retrieval practice (REQ-F-001)
4. Tutor agent — concept delivery, Socratic questioning (REQ-F-002)
5. Lab agent — in-flow feedback over simulated sandbox telemetry (REQ-F-003, mock inputs)
6. Assessor agent — rubric application to pre-baked artifacts and defenses (REQ-F-004, mock inputs)
7. Proctor agent — integrity signals from mock telemetry, coaching interventions (REQ-F-005, mock inputs)
8. Mentor agent — long-horizon career narrative (REQ-F-006)
9. Learner surface integration — streaming chat UI wired to the real service, error/loading states
## v0.1 Requirements (Complete)
All 28 v0.1 requirements (REQ-001..028) are complete and shipped as v0.1.0. See REQUIREMENTS.md traceability matrix.
## Clarified Assumptions (CLARIFY stage, full autonomy — auto-resolved)
| # | Ambiguity | Resolution | Confidence |
|---|-----------|------------|-------------|
| A-001 | Where does the ai-service live in the monorepo? | `apps/ai-service` — pnpm-workspace ignores Python; it integrates via root package.json scripts (`ai:dev`, `ai:test`), not as a pnpm package. Turbo gets passthrough tasks. | 0.95 |
| A-002 | How does the Next.js client talk to ai-service? | Direct fetch to `http://localhost:8420` (configurable via `NEXT_PUBLIC_AI_SERVICE_URL`) with SSE parsing. No Next.js API-route proxy in v0.2 — client components talk straight to the service. | 0.85 |
| A-003 | Session persistence? | In-memory dict keyed by session ID (v0.2 has no DB). Sessions lost on restart — acceptable for this milestone; store interface is DB-migration-ready. | 0.9 |
| A-004 | Port for ai-service? | 8420 (avoids common dev-port collisions with 3000/8000; documented in .env.example). | 0.8 |
| A-005 | Which ollama-cloud model? | Default `gemma4:31b` (probe-verified); configurable via `AI_TUTOR_MODEL` env. Model choice is a config, not code. | 0.85 |
| A-006 | Streaming format? | SSE with `data:` JSON lines (OpenAI-compatible delta objects), terminated by `data: [DONE]`. Matches the provider contract, so the provider layer passes deltas through unchanged. | 0.9 |
| A-007 | Agent routing in the chat UI? | Explicit agent switcher (Coach/Tutor) in the learner chat; Byte viewer always uses Tutor; sandbox uses Lab; assessment uses Assessor+Proctor; dashboard Mentor panel. No autonomous routing in v0.2. | 0.9 |
| A-008 | Auth between web and ai-service? | None in v0.2 (local dev surface). CORS limited to localhost origins. Real auth is v0.3+ with identity work. | 0.85 |
| A-009 | Python tooling? | `python3 -m venv` + pip (venv is the only available mechanism in this environment; no uv). Pydantic v2, FastAPI, uvicorn, pytest — all PyPI-reachable (verified). | 0.9 |
| A-010 | What happens when the LLM provider is unreachable? | Streaming endpoints return an error event; the UI shows error states with retry. Mock provider guarantees tests never call the cloud. | 0.9 |
## Requirements (Active — Future Milestones)
The following are deferred beyond v0.1 and will be activated in subsequent milestones:
The following remain deferred beyond v0.2 and will be activated in subsequent milestones:
The following remain deferred beyond v0.2 and will be activated in subsequent milestones:
- AI tutor agent architecture (Coach, Tutor, Lab, Assessor, Proctor, Mentor)
- Competency graph engine and adaptive pathways
- Assessment engine (process-trace grading, oral defense, per-learner variant tasks)
- Sandbox fabric (sandboxed IDE, design tool, simulation)
- Identity verification and age-gating logic (16+/18+)
- Assessment engine (process-trace grading, oral defense, per-learner variant tasks) — v0.3+
- Sandbox fabric (sandboxed IDE, design tool, simulation) — v0.3+
- Identity verification and age-gating logic (16+/18+) — the real KYC backend (v0.3+; visual flow already exists in v0.1)
- Marketplace job aggregation pipeline (3M+ jobs from 120K companies)
- AI-powered tagging, semantic vector search, company enrichment
- AI resume parsing and job matching
@@ -91,7 +112,10 @@ The following are deferred beyond v0.1 and will be activated in subsequent miles
| D-009 | AI tutor UI as chat interface mockup with pre-scripted responses | The learner surface includes an AI tutor chat UI mockup. No real AI backend — pre-scripted responses simulate the Coach and Tutor agents. | Mockup only in v0.1, real agents in future milestone |
| D-010 | Age-gating represented as visual registration flow mockup | 16+/18+ age-gating shown as a UI flow with age verification step. No actual verification logic. | Visual mockup only |
| D-011 | Competency graph viewer as interactive static visualization | Admin surface includes a competency graph viewer using react-flow or similar. Mock competency nodes and edges. No real graph data. | Static graph with mock data |
| D-012 | Tech stack: TS monorepo + Python AI services (future) | v0.1 uses TS only. Python FastAPI microservices planned for AI tutor agents and assessment engine in later milestones. | v0.1: TS only. Future: TS + Python |
| D-012 | Tech stack: TS monorepo + Python AI services (future) | v0.1 uses TS only. Python FastAPI microservices planned for AI tutor agents and assessment engine in later milestones. | v0.1: TS only. v0.2: TS + Python (apps/ai-service) |
| D-013 | v0.1 prototype founder-agreed; D-001 business-logic gate unlocked | Founder approved starting v0.2 with AI Tutor Architecture, which constitutes agreement of the v0.1 prototype per D-001. Recorded at v0.2 SPECIFY. | Business logic authorized from v0.2 onward |
| D-014 | Provider-agnostic LLM layer; ollama-cloud as initial provider | OpenAI-compatible client abstraction with pluggable providers: ollama-cloud (https://ollama.com/v1, default), local OpenAI-compatible endpoint, deterministic mock (tests/CI). Keys in gitignored .ciagent/.env.secrets, never in code or commits. | apps/ai-service llm package with 3 providers; default=ollama-cloud |
| D-015 | All six agents implemented as real LLM services; engines mocked | Coach/Tutor/Mentor fully real. Lab/Assessor/Proctor are real LLM logic over mock inputs (simulated telemetry, pre-baked artifacts) since sandbox fabric, assessment engine, and identity verification are v0.3+. Consistent with v0.1's mock-data approach. | REQ-F-001..006 complete in v0.2; real engines deferred to v0.3+ |
---
+54 -14
View File
@@ -1,12 +1,43 @@
# Nextcraft — REQUIREMENTS.md
## v0.1 Requirements (UI/UX Prototype)
## v0.2 Requirements (AI Tutor Architecture)
### AI Service Infrastructure
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-2-001 | apps/ai-service scaffolding: Python FastAPI app, pydantic settings, uvicorn, health endpoint, CORS, pytest setup, pnpm/turbo integration scripts | critical | 1 | pending |
| REQ-2-002 | Provider-agnostic LLM client: OpenAI-compatible provider interface with ollama-cloud (default), local-endpoint, and deterministic mock providers; key resolution from env files | critical | 1 | pending |
| REQ-2-003 | SSE streaming endpoint plumbing: chat completion streaming from provider through FastAPI to the Next.js client | critical | 1 | pending |
| REQ-2-004 | Agent framework: base agent contracts, session/state store, prompt management, streaming pipeline, structured output support | critical | 2 | pending |
### AI Tutor Agents
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-2-005 | Coach agent (REQ-F-001): pacing, motivation, retrieval practice — full LLM implementation | critical | 3 | pending |
| REQ-2-006 | Tutor agent (REQ-F-002): concept delivery, Socratic questioning — full LLM implementation | critical | 3 | pending |
| REQ-2-007 | Lab agent (REQ-F-003): in-flow feedback over simulated sandbox telemetry (mock inputs) | high | 4 | pending |
| REQ-2-008 | Assessor agent (REQ-F-004): rubric application to pre-baked artifacts and defense transcripts (mock inputs) | high | 4 | pending |
| REQ-2-009 | Proctor agent (REQ-F-005): integrity signals from mock telemetry, coaching interventions | high | 5 | pending |
| REQ-2-010 | Mentor agent (REQ-F-006): long-horizon career narrative | high | 5 | pending |
### Learner Surface Integration
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-2-011 | Learner chat UI wired to real service: streaming responses, agent routing, error and loading states | critical | 6 | pending |
| REQ-2-012 | Byte tutorial viewer, build sandbox, and assessment mockups surface Lab/Assessor/Proctor outputs (mock engine inputs) | high | 6 | pending |
---
## v0.1 Requirements (Complete)
### Infrastructure
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-001 | Monorepo scaffolding: pnpm workspaces, turborepo, Next.js app, TypeScript config, ESLint, Prettier | critical | 1 | pending |
| REQ-001 | Monorepo scaffolding: pnpm workspaces, turborepo, Next.js app, TypeScript config, ESLint, Prettier | critical | 1 | complete |
| REQ-002 | Shared component library: design tokens, Button, Input, Card, Badge, Avatar, Dialog, Navigation, Table, Tabs, Progress, Tooltip, Skeleton, Toast | critical | 1 | pending |
| REQ-003 | Mock data layer: typed mock data for competency stacks, job listings, candidate profiles, employers, learner progress | critical | 1 | pending |
| REQ-004 | Routing and navigation: App Router route groups for (learner), (marketplace), (employer), (admin); shared layout components; cross-surface navigation | critical | 1 | pending |
@@ -64,17 +95,6 @@
## v2 Requirements (Future Milestones — Deferred)
### AI Tutor Architecture
| ID | Description | Priority | Milestone | Status |
|----|-------------|----------|-----------|--------|
| REQ-F-001 | Coach agent: pacing, motivation, retrieval practice | high | v0.2+ | deferred |
| REQ-F-002 | Tutor agent: concept delivery, Socratic questioning | high | v0.2+ | deferred |
| REQ-F-003 | Lab agent: sandbox execution, in-flow feedback | high | v0.2+ | deferred |
| REQ-F-004 | Assessor agent: rubric application to artifacts and defenses | high | v0.2+ | deferred |
| REQ-F-005 | Proctor agent: identity, attention, integrity signals | high | v0.2+ | deferred |
| REQ-F-006 | Mentor agent: long-horizon career narrative | medium | v0.2+ | deferred |
### Assessment Engine
| ID | Description | Priority | Milestone | Status |
@@ -83,6 +103,7 @@
| REQ-F-008 | Per-learner variant task generation | high | v0.3+ | deferred |
| REQ-F-009 | Oral/voice defense with AI examiner | high | v0.3+ | deferred |
| REQ-F-010 | Live in-environment build with telemetry | high | v0.3+ | deferred |
| REQ-F-021 | Sandbox fabric: sandboxed IDE, design tool, simulation | high | v0.3+ | deferred |
### Marketplace Engine
@@ -99,7 +120,7 @@
| ID | Description | Priority | Milestone | Status |
|----|-------------|----------|-----------|--------|
| REQ-F-017 | Identity verification and age-gating (16+/18+) | high | v0.2+ | deferred |
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.3+ | deferred |
| REQ-F-018 | Payment processing and subscription management | high | v0.3+ | deferred |
| REQ-F-019 | Human tutor marketplace (third-party courses) | medium | v0.4+ | deferred |
| REQ-F-020 | CIRR-style placement tracking and audit | medium | v0.5+ | deferred |
@@ -122,6 +143,25 @@
## Traceability Matrix
### v0.2 (current milestone)
| Requirement | Phase | Status |
|-------------|-------|--------|
| REQ-2-001 | 1 | pending |
| REQ-2-002 | 1 | pending |
| REQ-2-003 | 1 | pending |
| REQ-2-004 | 2 | pending |
| REQ-2-005 | 3 | pending |
| REQ-2-006 | 3 | pending |
| REQ-2-007 | 4 | pending |
| REQ-2-008 | 4 | pending |
| REQ-2-009 | 5 | pending |
| REQ-2-010 | 5 | pending |
| REQ-2-011 | 6 | pending |
| REQ-2-012 | 6 | pending |
### v0.1 (complete)
| Requirement | Phase | Status |
|-------------|-------|--------|
| REQ-001 | 1 | complete |
+89 -101
View File
@@ -2,11 +2,13 @@
## Overview
**Milestone v0.1**UI/UX Prototype: High-fidelity interactive prototype of all four Nextcraft surfaces (Learner, Marketplace, Employer Dashboard, Admin) with realistic mock data, shared component library, responsive layouts, and navigation flows. No backend, no business logic.
**Milestone v0.2**AI Tutor Architecture: The six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface with streaming responses. Provider-agnostic LLM layer (ollama-cloud default). Lab/Assessor/Proctor operate on mock engine inputs — their real engines are v0.3+.
**Milestone type:** Feature (includes new UI features)
**Tag line:** v0.0.x (patches on the v0.0 line; milestone release as v0.1.0)
**Branch:** milestone/v0.1-nextcraft-ui-prototype
**Prior milestone:** v0.1 (nextcraft-ui-prototype) — complete, shipped as v0.1.0, founder-agreed (D-013).
**Milestone type:** Feature (new AI service + real agent capabilities)
**Tag line:** v0.1.x (patches on the v0.1 line; milestone release as v0.2.0)
**Branch:** milestone/v0.2-ai-tutor-architecture
---
@@ -14,14 +16,14 @@
| # | Name | Status | Depends On | Requirements | Success Criteria |
|---|------|--------|------------|--------------|------------------|
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan complete; .ciagent/ files created |
| 1 | Project scaffolding | complete | 0 | REQ-001, REQ-002, REQ-003, REQ-004, REQ-005 | Monorepo builds; dev server starts; component library has all primitives; mock data typed; routing works between route groups |
| 2 | Learner surface UI | complete | 1 | REQ-006, REQ-007, REQ-008, REQ-009, REQ-010, REQ-011, REQ-012 | All 7 learner pages render with mock data; navigation between pages works; responsive at mobile/tablet/desktop |
| 3 | Marketplace surface UI | complete | 1 | REQ-013, REQ-014, REQ-015, REQ-016, REQ-017 | All 5 marketplace pages render with mock data; search/filter UI interactive (client-side); job board listing displays mock jobs |
| 4 | Employer dashboard UI | complete | 1 | REQ-018, REQ-019, REQ-020, REQ-021 | All 4 employer pages render with mock data; talent search displays mock candidates; candidate profile shows artifact gallery + process trace |
| 5 | Admin surface UI | complete | 1 | REQ-022, REQ-023, REQ-024, REQ-025 | All 4 admin pages render with mock data; competency graph viewer renders interactive graph; moderation queue displays mock flagged content |
| 6 | Polish + integration | complete | 2, 3, 4, 5 | REQ-026, REQ-027, REQ-028 | Cross-surface navigation works; visual consistency audit passes; dark mode toggle functional; Storybook documents all components |
| 7 | Final review + ship | complete | 6 | — | Code review clean; audit passes; milestone tagged v0.1.0; release created on Gitea |
| 0 | Pre-execution | in_progress | — | — | Specification, clarify, research, plan complete; .ciagent/ files updated for v0.2 |
| 1 | AI service scaffolding | pending | 0 | REQ-2-001, REQ-2-002, REQ-2-003 | apps/ai-service runs (uvicorn), health endpoint responds, provider-agnostic LLM client with 3 providers (ollama-cloud/local/mock), SSE streaming verified, pytest suite passes with mock provider, turbo scripts wired |
| 2 | Agent framework | pending | 1 | REQ-2-004 | Base agent contract, session/state store, prompt templates, streaming pipeline, structured outputs; all tested |
| 3 | Coach + Tutor agents | pending | 2 | REQ-2-005, REQ-2-006 | Coach (pacing/motivation/retrieval practice) and Tutor (concept delivery/Socratic questioning) fully implemented with system prompts, tested against mock provider, wired to chat endpoint |
| 4 | Lab + Assessor agents | pending | 2 | REQ-2-007, REQ-2-008 | Lab consumes simulated sandbox telemetry (mock); Assessor applies rubrics to pre-baked artifacts/defense transcripts (mock); both tested |
| 5 | Proctor + Mentor agents | pending | 2 | REQ-2-009, REQ-2-010 | Proctor produces integrity signals + coaching interventions from mock telemetry; Mentor generates long-horizon career narrative; both tested |
| 6 | Learner surface integration | pending | 3, 4, 5 | REQ-2-011, REQ-2-012 | Learner chat streams real responses; agent routing works; byte viewer/sandbox/assessment mockups surface agent outputs; error/loading states; pnpm build + typecheck pass |
| 7 | Final review + ship | pending | 6 | — | Code review clean; audit passes; milestone tagged v0.2.0; release created on Gitea |
---
@@ -29,150 +31,130 @@
### Phase 0: Pre-execution
**Goal:** Establish project specification, clarify ambiguities, research tech stack, create detailed plans.
**Goal:** Establish v0.2 specification, clarify ambiguities, research AI service architecture, create detailed plans.
**Stages:** SPECIFY → CLARIFY → RESEARCH → PLAN → (GRILL optional) → SHIP
**Stages:** SPECIFY → CLARIFY → RESEARCH → PLAN → GRILL → SHIP
**Deliverables:**
- .ciagent/config.json
- .ciagent/PROJECT.md
- .ciagent/REQUIREMENTS.md
- .ciagent/ARCHITECTURE.md
- .ciagent/ROADMAP.md
- .ciagent/CHECKPOINT.json
- Updated .ciagent/config.json, PROJECT.md, REQUIREMENTS.md, ROADMAP.md, ARCHITECTURE.md, PERSONAS.md, PLAN.md
**Success criteria:** All .ciagent/ files created; initial commit with ---ci--- block; phase 0 shipped as v0.0.1.
**Success criteria:** All .ciagent/ files updated for v0.2; phase 0 shipped as v0.1.1.
---
### Phase 1: Project Scaffolding
### Phase 1: AI Service Scaffolding
**Goal:** Set up the monorepo, component library, mock data layer, routing, and responsive system.
**Goal:** Stand up apps/ai-service with the provider-agnostic LLM layer and SSE streaming.
**Requirements:** REQ-001, REQ-002, REQ-003, REQ-004, REQ-005
**Requirements:** REQ-2-001, REQ-2-002, REQ-2-003
**Key deliverables:**
- pnpm-workspace.yaml + turbo.json + root package.json
- apps/web Next.js app with App Router
- packages/ui with design tokens + all primitives (Button, Input, Card, Badge, Avatar, Dialog, Tabs, Progress, Tooltip, Skeleton, Toast)
- packages/mock-data with typed mock data for all domains
- packages/types with shared TypeScript type definitions
- Route groups: (learner), (marketplace), (employer), (admin)
- Responsive layout system with breakpoints
- Root layout with theme provider
- apps/ai-service: FastAPI app, pydantic-settings, uvicorn, /health, CORS for localhost
- llm package: provider interface + ollama-cloud/local/mock providers; key resolution from .ciagent/.env.secrets via env
- SSE streaming: /v1/chat/stream endpoint streaming provider deltas
- pytest suite with mock provider; root scripts: ai:dev, ai:test; turbo integration
**Success criteria:**
- `pnpm dev` starts the Next.js dev server
- `pnpm build` succeeds without errors
- `pnpm typecheck` passes
- All primitive components exist and are importable
- Mock data is typed and importable
- Navigation between route groups works (even if pages are placeholders)
- `python -m uvicorn` starts the service; /health returns 200
- Provider unit tests pass (mock); ollama-cloud integration probe works (manual)
- SSE stream delivers tokens to an HTTP client
---
### Phase 2: Learner Surface UI
### Phase 2: Agent Framework
**Goal:** Build all 7 learner surface pages with realistic mock data and interactive elements.
**Goal:** Build the shared framework all six agents use.
**Requirements:** REQ-006, REQ-007, REQ-008, REQ-009, REQ-010, REQ-011, REQ-012
**Requirements:** REQ-2-004
**Key deliverables:**
- Landing page: hero, value proposition, Byte→Build→Demonstrate→Defend flow, testimonials, CTA
- Program catalog: 5 competency stack cards with role descriptions
- Competency stack view: selected stack with 12-18 competencies, progress indicators, microcredential badges
- Learner dashboard: active competencies, progress graph, recent artifacts, AI tutor chat mockup, milestones
- Byte tutorial viewer: concept panel, worked example, code/design viewer mockup
- Build sandbox mockup: IDE UI with toolbar, file explorer, editor area, telemetry sidebar
- Assessment/defense mockup: rubric display, AI reviewer panel, oral defense interface, process trace timeline
- BaseAgent contract: system prompt, message history, streaming completion, structured output
- Session/state store: in-memory per-learner session with message history
- Prompt management: per-agent system prompt templates with learner context injection
- Streaming pipeline: agent → provider → SSE with agent identification
- Structured outputs: JSON-schema outputs for Assessor rubric scores, Proctor signals
**Success criteria:**
- All 7 pages render with mock data
- Navigation between pages works
- AI tutor chat mockup displays pre-scripted responses
- Responsive at mobile (375px), tablet (768px), desktop (1280px)
- Hover states and interactive elements functional
- BaseAgent unit tests pass
- Session store tested (create/append/persist in-memory)
- Structured output parsing tested against mock provider
---
### Phase 3: Marketplace Surface UI
### Phase 3: Coach + Tutor Agents
**Goal:** Build all 5 marketplace surface pages with job listings, search, and pricing.
**Goal:** Implement the two learner-facing conversational agents.
**Requirements:** REQ-013, REQ-014, REQ-015, REQ-016, REQ-017
**Requirements:** REQ-2-005, REQ-2-006
**Key deliverables:**
- Job board listing: searchable grid of mock AI-era jobs, filter sidebar, match score cards
- Job detail page: full description, required competencies, employer info, AI-matched skills
- Employer profile: company overview, logo, open positions, culture mockup
- Search/filter UI: semantic search bar, skill tags, filters (seniority, remote, salary), saved searches
- Pricing page: job posting packages, talent access plans, feature comparison table
- Coach agent: pacing guidance, motivation, retrieval practice prompts; distinct persona
- Tutor agent: concept delivery, Socratic questioning, worked examples
- Agent registry: route chat messages to the correct agent by context/selection
- Per-agent system prompts with competency-stack context injection from packages/mock-data
**Success criteria:**
- All 5 pages render with mock data
- Filter sidebar interactive (client-side filtering of mock jobs)
- Job cards display match scores and skill tags
- Pricing table is responsive and readable
- Both agents produce distinct, on-persona responses (verified against mock + ollama-cloud)
- Agent routing tested
- Both agents exposed via the chat streaming endpoint
---
### Phase 4: Employer Dashboard UI
### Phase 4: Lab + Assessor Agents
**Goal:** Build all 4 employer dashboard pages for talent search and posting management.
**Goal:** Implement the two build/assessment agents over mock engine inputs.
**Requirements:** REQ-018, REQ-019, REQ-020, REQ-021
**Requirements:** REQ-2-007, REQ-2-008
**Key deliverables:**
- Dashboard overview: active postings, applicant pipeline, talent matches, analytics charts
- Talent search: searchable candidate database with AI-matched filters, candidate cards
- Candidate profile: artifact gallery, process trace summary, defense transcripts, competency graph, microcredentials
- Posting management: create/edit/delete job postings, status tracking, applicant list, interview pipeline
- Lab agent: consumes simulated sandbox telemetry (mock event streams), produces in-flow feedback
- Assessor agent: applies rubrics to pre-baked artifacts and defense transcripts, returns structured scores + feedback
- Mock engine inputs: simulated telemetry generator, pre-baked artifact corpus in packages/mock-data
- Endpoints: /v1/lab/feedback, /v1/assessment/evaluate
**Success criteria:**
- All 4 pages render with mock data
- Analytics charts display mock metrics (bar/line/donut charts)
- Candidate cards show competency stacks, microcredentials, defense scores
- Posting management form inputs are interactive (non-functional submit)
- Lab produces relevant feedback for mock telemetry scenarios
- Assessor returns structured rubric scores (JSON) for pre-baked artifacts
- Both tested against mock provider
---
### Phase 5: Admin Surface UI
### Phase 5: Proctor + Mentor Agents
**Goal:** Build all 4 admin surface pages including the interactive competency graph viewer.
**Goal:** Implement the integrity and narrative agents.
**Requirements:** REQ-022, REQ-023, REQ-024, REQ-025
**Requirements:** REQ-2-009, REQ-2-010
**Key deliverables:**
- Admin overview: platform metrics (learners, employers, placements, completion rate, NPS), activity feed, system health
- Learner management: searchable learner table, detail view, progress tracking, credential log
- Competency graph viewer: interactive node/edge graph using react-flow, competency stack nodes, dependency edges
- Marketplace moderation: job posting review queue, employer verification queue, flagged content, moderation tools
- Proctor agent: integrity signals from mock telemetry (tab switches, idle time, paste events), coaching interventions
- Mentor agent: long-horizon career narrative, competency-stack progression guidance
- Endpoints: /v1/proctor/signals, /v1/mentor/narrative
**Success criteria:**
- All 4 pages render with mock data
- Competency graph viewer renders an interactive graph with clickable nodes
- Admin table supports sorting and filtering (client-side, mock data)
- Moderation queue displays mock flagged items with approve/reject buttons (non-functional)
- Proctor produces classified signals with recommended interventions for mock scenarios
- Mentor produces coherent career-narrative responses
- Both tested against mock provider
---
### Phase 6: Polish + Integration
### Phase 6: Learner Surface Integration
**Goal:** Ensure cross-surface consistency, responsive quality, and component library documentation.
**Goal:** Wire the v0.1 learner surface to the real AI service.
**Requirements:** REQ-026, REQ-027, REQ-028
**Requirements:** REQ-2-011, REQ-2-012
**Key deliverables:**
- Cross-surface navigation: role switcher (learner/employer/admin), breadcrumbs, consistent header/footer
- Visual consistency: typography scale, color palette, spacing system, dark mode toggle, WCAG AA contrast
- Storybook: component documentation, prop tables, usage examples for all primitives and composites
- Learner dashboard chat: real streaming via SSE, agent switcher (Coach/Tutor), error/loading states
- Byte tutorial viewer: Tutor concept explanations
- Build sandbox: Lab feedback panel fed by mock telemetry + Lab agent
- Assessment mockup: Assessor rubric output display, Proctor integrity banner
- Mentor panel on learner dashboard
**Success criteria:**
- Role switcher navigates between surfaces
- Dark mode toggle works across all surfaces
- All pages pass WCAG AA contrast checks
- Storybook runs and documents all components
- No visual inconsistencies between surfaces
- Streaming chat works end-to-end with ai-service running
- All four learner surfaces surface agent outputs
- Graceful degradation when ai-service is down (error states, not crashes)
- `pnpm build` and `pnpm typecheck` pass
---
@@ -183,10 +165,16 @@
**Key deliverables:**
- Multi-persona code review (correctness, testing, security, performance, maintainability)
- Project health audit (reconstruction test, .ciagent/ file discipline, branch hygiene, commit discipline)
- Milestone ship: merge milestone → main, tag v0.1.0, create Gitea release
- Milestone ship: merge milestone → main, tag v0.2.0, create Gitea release
**Success criteria:**
- Code review: P0 fixes applied, P1+ documented
- Audit: all checks pass, project state reconstructable from git log
- Ship: v0.1.0 tagged, milestone branch merged to main, Gitea release created
- All 28 requirements marked complete
- Ship: v0.2.0 tagged, milestone branch merged to main, Gitea release created**release note explicitly states Lab/Assessor/Proctor operate on mock engine inputs (real engines v0.3+)** (G-5); dead `aiTutorResponses` export disposed of (G-5)
- All 12 v0.2 requirements marked complete
---
## v0.1 (Complete — Shipped as v0.1.0)
UI/UX Prototype: High-fidelity interactive prototype of all four Nextcraft surfaces. 7 phases (P0 + P1-P6 execution + P7 final). All 28 requirements complete. Tags v0.0.1v0.0.7, milestone release v0.1.0.
+3 -3
View File
@@ -46,9 +46,9 @@
"projects": [],
"active_project": null,
"milestone": {
"version": "v0.1",
"name": "nextcraft-ui-prototype",
"version": "v0.2",
"name": "ai-tutor-architecture",
"type": "feature",
"branch": "milestone/v0.1-nextcraft-ui-prototype"
"branch": "milestone/v0.2-ai-tutor-architecture"
}
}
+6 -1
View File
@@ -42,4 +42,9 @@ yarn-error.log*
# Test coverage
coverage/
.nyc_output/
.nyc_output/
# Python tooling caches
.pytest_cache/
.ruff_cache/
*.egg-info/
+10
View File
@@ -0,0 +1,10 @@
# Nextcraft AI Service — environment template (copy values, never commit real keys)
# Real keys live in .ciagent/.env.secrets (gitignored) and are exported by scripts/dev.sh
AI_PORT=8420
AI_PROVIDER=ollama-cloud
AI_MODEL=gemma4:31b
AI_OLLAMA_CLOUD_BASE_URL=https://ollama.com/v1
AI_OLLAMA_CLOUD_API_KEY=
AI_LOCAL_BASE_URL=http://localhost:11434/v1
AI_JSON_MODE=auto
+81
View File
@@ -0,0 +1,81 @@
# Nextcraft AI Service (`apps/ai-service`)
Python FastAPI service hosting the six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) behind a provider-agnostic LLM layer. Port **8420**.
## Quickstart
```bash
# 1. Bootstrap (idempotent): venv + deps
bash scripts/bootstrap.sh
# 2. Run tests (mock provider only — zero network calls)
bash scripts/test.sh
# 3. Lint
bash scripts/lint.sh
# 4. Dev server (exports keys from .ciagent/.env.secrets if present)
bash scripts/dev.sh
```
Or via the monorepo root (`corepack pnpm install` first):
```bash
pnpm ai:bootstrap
pnpm ai:test
pnpm ai:lint
pnpm ai:dev
```
## Configuration
All settings use the `AI_` env prefix (pydantic-settings; see `.env.example`).
| Var | Default | Purpose |
|-----|---------|---------|
| `AI_PORT` | 8420 | Listen port |
| `AI_PROVIDER` | mock | `ollama-cloud` \| `local` \| `mock` |
| `AI_MODEL` | gemma4:31b | Model for all agents |
| `AI_OLLAMA_CLOUD_BASE_URL` | https://ollama.com/v1 | Cloud base URL |
| `AI_OLLAMA_CLOUD_API_KEY` | (empty) | Bearer key — **never commit** |
| `AI_JSON_MODE` | auto | `auto` sends response_format, degrades on 400; `off` never sends |
Tests run with `AI_PROVIDER=mock` (enforced in `tests/conftest.py` by an instance assertion) — the suite never calls the cloud.
## Endpoints
- `GET /health` — status, configured provider, model (no cloud call)
- `POST /v1/chat/stream` — SSE chat stream. Body: `{"agent": "coach", "session_id": "...", "messages": [{"role":"user","content":"..."}]}`
SSE envelope (D-016): `meta` event first (agent/session/model), then `delta` events (incremental content), then `done`; on mid-stream failure an `error` event precedes the terminal `[DONE]` sentinel. sse-starlette emits `: ping` keep-alive comment lines on idle connections — clients must ignore frames without `data:`.
## Manual ollama-cloud probe (not automated)
```bash
# Streaming probe against ollama-cloud with the real provider:
AI_PROVIDER=ollama-cloud .venv/bin/uvicorn ai_service.main:app --port 8420
curl -sN -X POST localhost:8420/v1/chat/stream \
-H 'Content-Type: application/json' \
-d '{"agent":"tutor","session_id":"probe","messages":[{"role":"user","content":"Explain retrieval practice in one sentence."}]}'
# Direct provider probe:
source ../../.ciagent/.env.secrets # never in a committed script
curl -s https://ollama.com/v1/chat/completions \
-H "Authorization: Bearer $OLLAMA_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"model":"gemma4:31b","messages":[{"role":"user","content":"hi"}],"max_tokens":10}'
```
## Layout
```
ai_service/
main.py app factory, lifespan (httpx pool), CORS, /health
config.py pydantic-settings
api/ endpoints (SSE envelope lives here, D-016)
llm/ provider layer — dumb pipe, no envelope logic
scripts/ bootstrap.sh dev.sh test.sh lint.sh
tests/ pytest — mock provider only
```
Boundary rules: `llm/` imports nothing from `agents/` or `api/`; `agents/` imports nothing from `api/`.
+3
View File
@@ -0,0 +1,3 @@
"""Nextcraft AI tutor service — six LLM agents behind a provider-agnostic layer."""
__version__ = "0.2.0"
@@ -0,0 +1,18 @@
"""Agent framework — BaseAgent ABC (D-018), registry, sessions, structured outputs.
Boundary rule: agents/ imports from llm/, prompts/, corpus/ — never from api/.
"""
from .base import BaseAgent
from .registry import AgentRegistry
from .session import InMemorySessionStore, SessionStore
from .structured import StructuredOutputError, extract_json_object
__all__ = [
"AgentRegistry",
"BaseAgent",
"InMemorySessionStore",
"SessionStore",
"StructuredOutputError",
"extract_json_object",
]
+77
View File
@@ -0,0 +1,77 @@
"""BaseAgent ABC — the contract all six tutor agents implement (D-018).
Subclasses set `name`, override `system_prompt()`, and rarely `stream_reply()`.
The default pipeline: build_messages() → provider.stream_chat()/chat().
"""
from abc import ABC, abstractmethod
from collections.abc import AsyncIterator
from pydantic import BaseModel
from ..config import Settings
from ..corpus.learner_context import LearnerContext
from ..llm.base import LLMProvider
from ..llm.types import Message
from .structured import structured_completion
class BaseAgent(ABC):
"""A tutor agent: system prompt + message assembly + provider delegation."""
name: str = "base"
def __init__(self, provider: LLMProvider, settings: Settings) -> None:
self.provider = provider
self.settings = settings
@abstractmethod
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
"""Return the agent's system prompt, learner-context-aware."""
def build_messages(
self,
history: list[Message] | None = None,
user_input: str = "",
learner_context: LearnerContext | None = None,
) -> list[Message]:
"""Compose the full message list: system prompt + history + user turn."""
messages: list[Message] = [
Message(role="system", content=self.system_prompt(learner_context))
]
for m in history or []:
messages.append(m)
if user_input:
messages.append(Message(role="user", content=user_input))
return messages
async def stream_reply(
self,
history: list[Message] | None = None,
user_input: str = "",
learner_context: LearnerContext | None = None,
response_format: dict | None = None,
) -> AsyncIterator[str]:
"""Stream incremental content deltas for a conversational reply."""
messages = self.build_messages(history, user_input, learner_context)
async for token in self.provider.stream_chat(
messages, model=self.settings.model, response_format=response_format
):
yield token
async def structured_reply(
self,
history: list[Message] | None = None,
user_input: str = "",
learner_context: LearnerContext | None = None,
schema: type[BaseModel] | None = None,
schema_hint: str = "",
) -> BaseModel:
"""Non-streaming completion parsed into a pydantic model (D-020 defense)."""
if schema is None:
raise ValueError("structured_reply requires a schema")
messages = self.build_messages(history, user_input, learner_context)
return await structured_completion(
self.provider, messages, model=self.settings.model,
schema=schema, schema_hint=schema_hint,
)
@@ -0,0 +1,43 @@
"""Agent registry — explicit name → agent factory map (D-018, G-4).
Agents are registered centrally in their own phases (P3-P5) via
`registry.register(name, factory)`. One registration pattern, one registry.
"""
from collections.abc import Callable
from ..config import Settings
from ..llm.base import LLMProvider
from .base import BaseAgent
AgentFactory = Callable[[LLMProvider, Settings], BaseAgent]
class UnknownAgentError(KeyError):
"""Raised when resolving an agent name that was never registered."""
class DuplicateAgentError(ValueError):
"""Raised when registering an agent name that already exists."""
class AgentRegistry:
def __init__(self) -> None:
self._factories: dict[str, AgentFactory] = {}
def register(self, name: str, factory: AgentFactory) -> None:
if name in self._factories:
raise DuplicateAgentError(f"agent {name!r} already registered")
self._factories[name] = factory
def names(self) -> list[str]:
return sorted(self._factories)
def get(self, provider: LLMProvider, settings: Settings, name: str) -> BaseAgent:
try:
factory = self._factories[name]
except KeyError:
raise UnknownAgentError(
f"unknown agent {name!r}; registered: {self.names()}"
) from None
return factory(provider, settings)
@@ -0,0 +1,94 @@
"""SessionStore — protocol + in-memory implementation (D-019).
Protocol is DB-migration-ready (A-003): swap InMemorySessionStore for a
Redis/PG-backed implementation without touching the API layer.
Sessions are agent-scoped: switching agents starts a new session ID (avoids
persona bleed, A-007). History windowing happens here (last N messages),
controlling token growth per session.
"""
import asyncio
from collections import OrderedDict
from dataclasses import dataclass, field
from typing import Protocol
from ..llm.types import Message
DEFAULT_WINDOW = 20
DEFAULT_MAX_SESSIONS = 500
@dataclass
class AgentSession:
session_id: str
agent: str
learner_id: str = "seed-learner-1"
messages: list[Message] = field(default_factory=list)
class SessionStore(Protocol):
def get(self, session_id: str) -> AgentSession | None: ...
def create(
self, session_id: str, agent: str, learner_id: str = "seed-learner-1"
) -> AgentSession: ...
def append(self, session_id: str, message: Message) -> None: ...
def history_window(
self, session_id: str, max_messages: int = DEFAULT_WINDOW
) -> list[Message]: ...
def delete(self, session_id: str) -> None: ...
class InMemorySessionStore:
"""asyncio.Lock-guarded dict with 20-message windows and 500-cap LRU eviction."""
def __init__(
self,
window: int = DEFAULT_WINDOW,
max_sessions: int = DEFAULT_MAX_SESSIONS,
) -> None:
self._sessions: OrderedDict[str, AgentSession] = OrderedDict()
self._lock = asyncio.Lock()
self._window = window
self._max_sessions = max_sessions
async def get(self, session_id: str) -> AgentSession | None:
async with self._lock:
session = self._sessions.get(session_id)
if session is not None:
self._sessions.move_to_end(session_id) # LRU touch
return session
async def create(
self, session_id: str, agent: str, learner_id: str = "seed-learner-1"
) -> AgentSession:
async with self._lock:
session = AgentSession(session_id=session_id, agent=agent, learner_id=learner_id)
self._sessions[session_id] = session
self._evict_locked()
return session
async def append(self, session_id: str, message: Message) -> None:
async with self._lock:
session = self._sessions.get(session_id)
if session is None:
raise KeyError(f"unknown session {session_id!r}")
session.messages.append(message)
self._sessions.move_to_end(session_id)
async def history_window(
self, session_id: str, max_messages: int = DEFAULT_WINDOW
) -> list[Message]:
async with self._lock:
session = self._sessions.get(session_id)
if session is None:
raise KeyError(f"unknown session {session_id!r}")
return list(session.messages[-max_messages:])
async def delete(self, session_id: str) -> None:
async with self._lock:
self._sessions.pop(session_id, None)
def _evict_locked(self) -> None:
while len(self._sessions) > self._max_sessions:
self._sessions.popitem(last=False) # evict least-recently-used
@@ -0,0 +1,117 @@
"""Structured output defense — 4 layers (D-020).
Layer 1: response_format={"type":"json_object"} request (auto-degrades on 400
inside the provider).
Layer 2: prompt-embedded schema hint ("Respond with ONLY valid JSON...").
Layer 3: defensive parse — strip markdown fences, extract first balanced
JSON object, pydantic model_validate.
Layer 4: single bounded retry with the validation error fed back.
"""
from typing import TypeVar
from pydantic import BaseModel, ValidationError
from ..llm.base import LLMProvider
from ..llm.types import Message
T = TypeVar("T", bound=BaseModel)
class StructuredOutputError(Exception):
"""Raised when the model output cannot be validated after one retry."""
def extract_json_object(text: str) -> str:
"""Strip fences and return the first balanced {...} block from text."""
stripped = text.strip()
if stripped.startswith("```"):
first_newline = stripped.find("\n")
if first_newline != -1:
stripped = stripped[first_newline + 1:]
if stripped.rstrip().endswith("```"):
stripped = stripped.rstrip()[:-3]
stripped = stripped.strip()
start = stripped.find("{")
if start == -1:
raise StructuredOutputError("no JSON object found in model output")
depth = 0
in_string = False
escape = False
for i, ch in enumerate(stripped[start:], start=start):
if escape:
escape = False
continue
if ch == "\\":
escape = True
continue
if ch == '"' and not escape:
in_string = not in_string
continue
if in_string:
continue
if ch == "{":
depth += 1
elif ch == "}":
depth -= 1
if depth == 0:
return stripped[start:i + 1]
raise StructuredOutputError("unbalanced JSON object in model output")
def parse_structured(text: str, schema: type[T]) -> T:
"""Layer 3: fence-strip + first-balanced-object + pydantic validation."""
candidate = extract_json_object(text)
try:
return schema.model_validate_json(candidate)
except ValidationError as exc:
raise StructuredOutputError(f"schema validation failed: {exc}") from exc
def schema_instruction(schema_hint: str) -> str:
"""Layer 2: prompt-side schema text."""
return (
"Respond with ONLY a valid JSON object matching this schema — "
"no markdown fences, no prose outside the JSON. "
f"Schema: {schema_hint}"
)
async def structured_completion(
provider: LLMProvider,
messages: list[Message],
*,
model: str,
schema: type[T],
schema_hint: str,
retry_feedback: str | None = None,
) -> T:
"""Full 4-layer pipeline. One bounded retry (layer 4), then raise."""
# Build request: append schema instruction to the last user message (layer 2).
request = list(messages)
last_user = next((m for m in reversed(request) if m.role == "user"), None)
if last_user is not None:
request = [
Message(role=m.role, content=(m.content + "\n\n" + schema_instruction(schema_hint)))
if m is last_user else m
for m in request
]
response_format = {"type": "json_object"}
raw = await provider.chat(request, model=model, response_format=response_format)
try:
return parse_structured(raw, schema) # layers 1+2+3
except StructuredOutputError as exc:
# Layer 4: single bounded retry with error feedback
retry_prompt = (
f"Your previous response was invalid: {exc}. "
f"Return ONLY the corrected JSON matching: {schema_hint}"
)
request2 = list(messages)
request2.append(Message(role="user", content=retry_prompt))
raw2 = await provider.chat(request2, model=model, response_format=response_format)
try:
return parse_structured(raw2, schema)
except StructuredOutputError as exc2:
raise StructuredOutputError(
f"structured output failed after retry: {exc2}"
) from exc2
@@ -0,0 +1,8 @@
"""API package — composes providers, sessions, and agents via DI.
Boundary rule: api/ composes agents/ and llm/; they never import api/.
"""
from .chat import router as chat_router
__all__ = ["chat_router"]
+96
View File
@@ -0,0 +1,96 @@
"""POST /v1/chat/stream — SSE chat with the D-016 envelope.
Envelope: meta event first (flushed before first token), then raw content
deltas, then done; error event before [DONE] on mid-stream failure.
Pre-first-byte failures become proper HTTP error statuses.
"""
import json
from collections.abc import AsyncIterator
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field
from sse_starlette.sse import EventSourceResponse
from ..agents.session import SessionStore
from ..config import Settings
from ..llm.base import LLMProvider
from ..llm.types import Message
from .deps import get_provider, get_session_store, get_settings
router = APIRouter(prefix="/v1")
class ChatStreamRequest(BaseModel):
agent: str = Field(min_length=1)
session_id: str = Field(min_length=1)
messages: list[Message] = Field(min_length=1)
@router.post("/chat/stream")
async def chat_stream(
body: ChatStreamRequest,
provider: LLMProvider = Depends(get_provider),
settings: Settings = Depends(get_settings),
sessions: SessionStore = Depends(get_session_store),
) -> EventSourceResponse:
if not body.messages:
raise HTTPException(status_code=422, detail="messages must not be empty")
# Agent-scoped session (A-007/G-4): persisted turn history, windowed replay.
session = await sessions.get(body.session_id)
if session is None:
session = await sessions.create(body.session_id, agent=body.agent)
history = await sessions.history_window(body.session_id)
# Persist this turn's user message before streaming.
await sessions.append(body.session_id, body.messages[-1])
async def event_stream() -> AsyncIterator[dict]:
yield {"event": "message", "data": json.dumps({
"type": "meta",
"agent": body.agent,
"session_id": body.session_id,
"model": settings.model,
})}
first_byte = True
reply_parts: list[str] = []
try:
async for token in provider.stream_chat(
body.messages if not history else history + body.messages,
model=settings.model,
):
first_byte = False
reply_parts.append(token)
yield {"event": "message", "data": json.dumps({
"type": "delta", "content": token
})}
full_reply = "".join(reply_parts)
if full_reply:
await sessions.append(
body.session_id, Message(role="assistant", content=full_reply)
)
yield {"event": "message", "data": json.dumps({
"type": "done", "finish_reason": "stop"
})}
except Exception as exc: # CancelledError is BaseException — passes through
message = str(exc)
if first_byte:
# Pre-first-byte failure: we already flushed meta + 200 headers;
# surface as in-band error (status change is impossible post-flush).
yield {"event": "message", "data": json.dumps({
"type": "error", "code": "provider_unavailable", "message": message
})}
else:
yield {"event": "message", "data": json.dumps({
"type": "error", "code": "provider_error", "message": message
})}
finally:
yield {"event": "message", "data": "[DONE]"}
return EventSourceResponse(
event_stream(),
headers={
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no",
},
)
+24
View File
@@ -0,0 +1,24 @@
"""FastAPI dependencies — provider, settings, sessions, agents via app.state (DI)."""
from fastapi import Request
from ..agents.registry import AgentRegistry
from ..agents.session import SessionStore
from ..config import Settings
from ..llm.base import LLMProvider
def get_settings(request: Request) -> Settings:
return request.app.state.settings
def get_provider(request: Request) -> LLMProvider:
return request.app.state.provider
def get_session_store(request: Request) -> SessionStore:
return request.app.state.session_store
def get_agent_registry(request: Request) -> AgentRegistry:
return request.app.state.agent_registry
+18
View File
@@ -0,0 +1,18 @@
"""Service settings — pydantic-settings, env prefix AI_, .env support."""
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_prefix="AI_", env_file=".env", extra="ignore")
port: int = 8420
provider: str = "mock"
model: str = "gemma4:31b"
ollama_cloud_base_url: str = "https://ollama.com/v1"
ollama_cloud_api_key: str = "" # SecretStr adds friction here; never logged, never echoed
local_base_url: str = "http://localhost:11434/v1"
# "auto" sends response_format and degrades on 400; "off" never sends it
json_mode: str = "auto"
@@ -0,0 +1,11 @@
"""Mock engine inputs — pydantic-typed corpus (D-021).
Convention-aligned with the TS `packages/mock-data` layer: identical ID
strings (stack-*, comp-*, learner-*, art-*, mc-*), cross-referenced by the
counterpart files. No codegen in v0.2 — alignment is by documented
convention; revisit codegen only if drift bites (v0.3).
"""
from .learner_context import LEARNER_CONTEXTS, LearnerContext, get_learner_context
__all__ = ["LEARNER_CONTEXTS", "LearnerContext", "get_learner_context"]
@@ -0,0 +1,108 @@
"""Learner context corpus — pydantic mirror of TS learner-progress.ts (D-021).
Counterpart: packages/mock-data/src/learner-progress.ts (or learner-progress.ts
at package root). IDs are string-identical: learner-001, stack-orchestration,
stack-safety, stack-orchestration-c00N, art-*, mc-*.
"""
from pydantic import BaseModel
class CompetencyProgress(BaseModel):
competency_id: str
title: str
status: str # "mastered" | "in_progress" | "not_started"
class StackProgress(BaseModel):
stack_id: str
title: str
percent: int
class LearnerContext(BaseModel):
learner_id: str
name: str
active_stacks: list[StackProgress]
active_competencies: list[CompetencyProgress]
microcredential_count: int
recent_artifacts: list[str] # artifact names
_STACK_ORCHESTRATION = StackProgress(
stack_id="stack-orchestration", title="AI Orchestration Engineer", percent=62
)
_STACK_SAFETY = StackProgress(
stack_id="stack-safety", title="AI Safety & Governance Lead", percent=41
)
_LEARNER_1 = LearnerContext(
learner_id="learner-001",
name="Alex Rivera",
active_stacks=[_STACK_ORCHESTRATION, _STACK_SAFETY],
active_competencies=[
CompetencyProgress(
competency_id="stack-orchestration-c001",
title="Agent architecture fundamentals",
status="mastered",
),
CompetencyProgress(
competency_id="stack-orchestration-c002",
title="Multi-agent communication patterns",
status="in_progress",
),
CompetencyProgress(
competency_id="stack-orchestration-c003",
title="Tool use and function calling",
status="in_progress",
),
CompetencyProgress(
competency_id="stack-safety-c021",
title="Red-team basics for agent systems",
status="in_progress",
),
],
microcredential_count=4,
recent_artifacts=[
"Multi-agent research assistant",
"RAG retrieval quality dashboard",
],
)
_LEARNER_2 = LearnerContext(
learner_id="learner-002",
name="Priya Chen",
active_stacks=[
StackProgress(
stack_id="stack-designer", title="Human-AI Product Designer", percent=55
),
],
active_competencies=[
CompetencyProgress(
competency_id="stack-designer-c001",
title="Prompt-to-prototype workflows",
status="mastered",
),
CompetencyProgress(
competency_id="stack-designer-c002",
title="Evaluating AI UX patterns",
status="in_progress",
),
],
microcredential_count=2,
recent_artifacts=["AI onboarding flow concept test"],
)
LEARNER_CONTEXTS: dict[str, LearnerContext] = {
_LEARNER_1.learner_id: _LEARNER_1,
_LEARNER_2.learner_id: _LEARNER_2,
}
DEFAULT_LEARNER_ID = "learner-001"
def get_learner_context(learner_id: str | None = None) -> LearnerContext:
"""Resolve a learner context by ID, falling back to the default seed."""
if learner_id is None:
return LEARNER_CONTEXTS[DEFAULT_LEARNER_ID]
return LEARNER_CONTEXTS.get(learner_id, LEARNER_CONTEXTS[DEFAULT_LEARNER_ID])
@@ -0,0 +1,16 @@
"""LLM package — provider-agnostic layer (D-017)."""
from .base import LLMProvider
from .factory import create_provider
from .mock import MockProvider
from .openai_compat import OpenAICompatProvider
from .types import ChatDelta, Message
__all__ = [
"ChatDelta",
"LLMProvider",
"Message",
"MockProvider",
"OpenAICompatProvider",
"create_provider",
]
+35
View File
@@ -0,0 +1,35 @@
"""LLMProvider protocol — the port all agents depend on (D-017).
Implementations: openai_compat.OpenAICompatProvider (ollama-cloud + local),
mock.MockProvider (deterministic, tests/CI). Providers are dumb pipes:
no envelope logic here — the API layer owns meta/done/error events (D-016).
"""
from collections.abc import AsyncIterator
from typing import Protocol
from .types import Message
class LLMProvider(Protocol):
async def stream_chat(
self,
messages: list[Message],
*,
model: str,
temperature: float = 0.7,
response_format: dict | None = None,
) -> AsyncIterator[str]:
"""Yield incremental content deltas (plain text chunks)."""
...
async def chat(
self,
messages: list[Message],
*,
model: str,
temperature: float = 0.7,
response_format: dict | None = None,
) -> str:
"""Non-streaming completion — returns the full reply text."""
...
+34
View File
@@ -0,0 +1,34 @@
"""Provider factory — selects the LLM provider from settings (D-014)."""
import httpx
from ..config import Settings
from .mock import MockProvider
from .openai_compat import OpenAICompatProvider
PROVIDER_NAMES = ("ollama-cloud", "local", "mock")
def create_provider(settings: Settings, http_client: httpx.AsyncClient):
"""Return the provider instance for settings.provider.
Raises ValueError for unknown provider names.
"""
if settings.provider == "ollama-cloud":
return OpenAICompatProvider(
http_client=http_client,
base_url=settings.ollama_cloud_base_url,
api_key=settings.ollama_cloud_api_key,
json_mode=settings.json_mode,
)
if settings.provider == "local":
return OpenAICompatProvider(
http_client=http_client,
base_url=settings.local_base_url,
json_mode=settings.json_mode,
)
if settings.provider == "mock":
return MockProvider()
raise ValueError(
f"unknown provider {settings.provider!r}; expected one of {PROVIDER_NAMES}"
)
+94
View File
@@ -0,0 +1,94 @@
"""Deterministic mock provider — tests and CI. NEVER calls the network.
Determinism: the reply text is seeded from the message content hash, so
identical inputs always produce identical outputs. Supports scripted
failure modes for error-path coverage (D-023, A-010).
"""
import hashlib
import json
from collections.abc import AsyncIterator
from .types import Message
_REPLIES = [
"Great question — let's break this down step by step and see where it leads.",
"Here is the key idea: small, verified moves compound into mastery over time.",
"Think about it this way: what would the simplest working version look like?",
"You are closer than you think. Try restating the goal in one sentence first.",
"Let me offer a different angle before we move to the next step.",
]
_JSON_REPLY = '{"summary": "mock structured reply", "confidence": 0.87}'
class MockProvider:
"""Scripted provider: deterministic streams, no network, failure injection."""
def __init__(self) -> None:
self.fail_before_first_token: bool = False
self.fail_mid_stream_at_index: int | None = None
self.abort_recorded: bool = False # set in stream finally-block (cancellation test)
def _reply_for(self, messages: list[Message], response_format: dict | None) -> str:
seed_src = "|".join(f"{m.role}:{m.content}" for m in messages)
if response_format is not None and response_format.get("type") == "json_object":
return _JSON_REPLY
digest = hashlib.sha256(seed_src.encode()).hexdigest()
base = _REPLIES[int(digest[:2], 16) % len(_REPLIES)]
# Deterministic seed tag guarantees distinct inputs → distinct replies
return f"{base} [#{digest[:8]}]"
def _tokenize(self, text: str) -> list[str]:
words = text.split(" ")
tokens: list[str] = []
for i, word in enumerate(words):
suffix = " " if i < len(words) - 1 else ""
tokens.append(word + suffix)
return tokens
async def stream_chat(
self,
messages: list[Message],
*,
model: str,
temperature: float = 0.7,
response_format: dict | None = None,
) -> AsyncIterator[str]:
if self.fail_before_first_token:
raise RuntimeError("mock provider: scripted failure before first token")
reply = self._reply_for(messages, response_format)
tokens = self._tokenize(reply)
try:
for i, token in enumerate(tokens):
if self.fail_mid_stream_at_index is not None and i == self.fail_mid_stream_at_index:
raise RuntimeError("mock provider: scripted mid-stream failure")
yield token
finally:
# Cancellation (GeneratorExit/CancelledError) lands here — tests assert this.
self.abort_recorded = True
async def chat(
self,
messages: list[Message],
*,
model: str,
temperature: float = 0.7,
response_format: dict | None = None,
) -> str:
if self.fail_before_first_token:
raise RuntimeError("mock provider: scripted failure before completion")
return self._reply_for(messages, response_format)
class ScriptedJSONProvider(MockProvider):
"""Mock variant returning a fixed JSON payload for structured tests."""
def __init__(self, payload: dict) -> None:
super().__init__()
self.payload = payload
def _reply_for(self, messages: list[Message], response_format: dict | None) -> str:
if response_format is not None and response_format.get("type") == "json_object":
return json.dumps(self.payload)
return super()._reply_for(messages, response_format)
@@ -0,0 +1,125 @@
"""OpenAI-compatible provider — one implementation serves ollama-cloud AND local
endpoints (they differ only in base_url/key). Raw httpx, no SDK (D-017).
Boundary rules:
- llm/ imports nothing from agents/ or api/
- api_key NEVER appears in exceptions, logs, or error messages
"""
import json
from collections.abc import AsyncIterator
import httpx
from .types import Message
class OpenAICompatProvider:
def __init__(
self,
http_client: httpx.AsyncClient,
base_url: str,
api_key: str = "",
json_mode: str = "auto",
) -> None:
self._client = http_client
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._json_mode = json_mode
def _headers(self) -> dict[str, str]:
headers = {"Content-Type": "application/json"}
if self._api_key:
headers["Authorization"] = f"Bearer {self._api_key}"
return headers
def _payload(
self,
messages: list[Message],
model: str,
temperature: float,
response_format: dict | None,
stream: bool,
) -> dict:
payload: dict = {
"model": model,
"messages": [{"role": m.role, "content": m.content} for m in messages],
"temperature": temperature,
}
if stream:
payload["stream"] = True
else:
payload["stream"] = False
# json_mode="auto": send response_format and degrade on 400; "off": never send
if response_format is not None and self._json_mode == "auto":
payload["response_format"] = response_format
return payload
def _sanitize(self, exc: Exception) -> RuntimeError:
text = str(exc)
if self._api_key and self._api_key in text:
text = text.replace(self._api_key, "[REDACTED]")
return RuntimeError(f"llm provider error: {text}")
async def stream_chat(
self,
messages: list[Message],
*,
model: str,
temperature: float = 0.7,
response_format: dict | None = None,
) -> AsyncIterator[str]:
payload = self._payload(messages, model, temperature, response_format, stream=True)
try:
async with self._client.stream(
"POST", f"{self._base_url}/chat/completions",
json=payload, headers=self._headers(),
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
if not line or not line.startswith("data:"):
continue # keep-alive comments (": ping"), empty lines
data = line.removeprefix("data:").strip()
if data == "[DONE]":
return
try:
chunk = json.loads(data)
except json.JSONDecodeError:
continue # malformed line — tolerate (ollama-cloud quirks)
choices = chunk.get("choices") or []
if not choices:
continue
content = (choices[0].get("delta") or {}).get("content")
if content:
yield content
except httpx.HTTPError as exc:
raise self._sanitize(exc) from exc
async def chat(
self,
messages: list[Message],
*,
model: str,
temperature: float = 0.7,
response_format: dict | None = None,
) -> str:
payload = self._payload(messages, model, temperature, response_format, stream=False)
try:
response = await self._client.post(
f"{self._base_url}/chat/completions",
json=payload, headers=self._headers(),
)
if response.status_code == 400 and "response_format" in payload:
# json_mode auto-degrade (D-020 layer 1): retry once without it
payload.pop("response_format")
response = await self._client.post(
f"{self._base_url}/chat/completions",
json=payload, headers=self._headers(),
)
response.raise_for_status()
data = response.json()
return (data["choices"][0]["message"]["content"]) or ""
except httpx.HTTPError as exc:
raise self._sanitize(exc) from exc
except (KeyError, ValueError) as exc:
raise self._sanitize(exc) from exc
+42
View File
@@ -0,0 +1,42 @@
"""LLM layer types — messages and OpenAI-compatible stream chunks.
Boundary rule: nothing in llm/ imports from agents/ or api/.
"""
from pydantic import BaseModel
class Message(BaseModel):
role: str
content: str
class ChoiceDelta(BaseModel):
index: int = 0
delta_content: str = ""
finish_reason: str | None = None
class ChatDelta(BaseModel):
"""Mirrors the OpenAI-compatible streaming chunk shape (D-016 passthrough)."""
id: str = ""
object: str = "chat.completion.chunk"
created: int = 0
model: str = ""
index: int = 0
delta_content: str = ""
finish_reason: str | None = None
def payload(self) -> dict:
"""OpenAI wire shape — the API layer emits this verbatim."""
return {
"id": self.id,
"object": self.object,
"created": self.created,
"model": self.model,
"choices": [
{"index": self.index, "delta": {"content": self.delta_content},
"finish_reason": self.finish_reason}
],
}
+54
View File
@@ -0,0 +1,54 @@
"""FastAPI app factory — lifespan, CORS, health, routers."""
from contextlib import asynccontextmanager
import httpx
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from .agents.registry import AgentRegistry
from .agents.session import InMemorySessionStore
from .api import chat_router
from .config import Settings
from .llm import create_provider
def create_app(settings: Settings | None = None) -> FastAPI:
settings = settings or Settings()
@asynccontextmanager
async def lifespan(app: FastAPI):
# Shared HTTP client pool (D-017): 10s connect / 300s read for cloud TTFT
timeout = httpx.Timeout(connect=10.0, read=300.0, write=30.0, pool=10.0)
app.state.http_client = httpx.AsyncClient(timeout=timeout)
app.state.settings = settings
app.state.provider = create_provider(settings, app.state.http_client)
app.state.session_store = InMemorySessionStore()
app.state.agent_registry = AgentRegistry()
yield
await app.state.http_client.aclose()
app = FastAPI(title="Nextcraft AI Service", version="0.2.0", lifespan=lifespan)
# A-008: localhost-only CORS, no credentials
app.add_middleware(
CORSMiddleware,
allow_origins=["http://localhost:3000", "http://127.0.0.1:3000"],
allow_methods=["GET", "POST", "OPTIONS"],
allow_headers=["Content-Type"],
allow_credentials=False,
)
@app.get("/health")
async def health() -> dict:
return {
"status": "ok",
"provider": settings.provider,
"model": settings.model,
}
app.include_router(chat_router)
return app
app = create_app()
@@ -0,0 +1,22 @@
"""Prompt library — prompts are code: versioned in git, reviewed like code (D-018).
Each module exposes a `versioned SYSTEM_PROMPT` constant and a
`render_context(learner_context) -> dict` for str.format_map injection.
Final personas land in Phases 3-5; these are the initial drafts.
"""
from .coach import SYSTEM_PROMPT as COACH_PROMPT
from .coach import render_context as render_coach
from .mentor import SYSTEM_PROMPT as MENTOR_PROMPT
from .mentor import render_context as render_mentor
from .tutor import SYSTEM_PROMPT as TUTOR_PROMPT
from .tutor import render_context as render_tutor
__all__ = [
"COACH_PROMPT",
"MENTOR_PROMPT",
"TUTOR_PROMPT",
"render_coach",
"render_mentor",
"render_tutor",
]
@@ -0,0 +1,19 @@
"""Assessor agent prompt — rubric application to artifacts and defenses (REQ-2-008).
Persona: rigorous, fair grader. Applies the rubric to the artifact and defense
transcript, returns structured JSON scores. Versioned: v1 draft (Phase 2);
final persona + rubric models in Phase 4.
"""
SYSTEM_PROMPT = """You are Assessor, the grading agent of an AI-native competency school.
Learner: {learner_name}.
You receive an artifact, its defense transcript, and a rubric. Your job:
score each rubric criterion with evidence from the artifact and transcript.
Be rigorous but fair — cite what the learner did, not what they should have
done. Respond with ONLY valid JSON matching the provided rubric schema."""
PROMPT_VERSION = "assessor-v1-draft"
def render_context(learner_context) -> dict:
return {"learner_name": learner_context.name}
@@ -0,0 +1,29 @@
"""Coach agent prompt — pacing, motivation, retrieval practice (REQ-2-005).
Persona: warm, action-oriented, accountability partner. Asks for commitments,
uses retrieval practice, keeps momentum. Versioned: v1 draft (Phase 2);
final persona in Phase 3.
"""
SYSTEM_PROMPT = """You are Coach, the pacing and motivation agent of an AI-native competency school.
Learner: {learner_name}. Active stack: {stacks}. Progress: {progress}.
Your job: keep the learner moving. Pace their next step, motivate without fluff,
and weave in retrieval practice — ask them to recall or apply something they
already covered before introducing new material. Be concise, warm, and direct.
End with exactly one clear next action."""
PROMPT_VERSION = "coach-v1-draft"
def render_context(learner_context) -> dict:
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
progress = (
f"{learner_context.active_competencies[0].title} in progress"
if learner_context.active_competencies
else "no active competencies"
)
return {
"learner_name": learner_context.name,
"stacks": stacks or "none yet",
"progress": progress,
}
+23
View File
@@ -0,0 +1,23 @@
"""Lab agent prompt — in-flow feedback over sandbox telemetry (REQ-2-007).
Persona: pragmatic build partner. Reads the telemetry timeline and gives
concrete in-flow feedback: what happened, what to adjust, next step.
Versioned: v1 draft (Phase 2); final persona + scenario serialization in Phase 4.
"""
SYSTEM_PROMPT = """You are Lab, the in-flow feedback agent watching a learner build in the sandbox.
Learner: {learner_name}. Active stack: {stacks}.
You receive a telemetry timeline of the learner's build session. Your job:
describe what the telemetry shows, name the single most useful adjustment,
and give one concrete next step. Be specific to the events you see —
no generic advice. Three short paragraphs maximum."""
PROMPT_VERSION = "lab-v1-draft"
def render_context(learner_context) -> dict:
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
return {
"learner_name": learner_context.name,
"stacks": stacks or "none yet",
}
@@ -0,0 +1,30 @@
"""Mentor agent prompt — long-horizon career narrative (REQ-2-010).
Persona: wise career guide. Connects today's competencies to a long-horizon
trajectory in AI-era roles. Versioned: v1 draft (Phase 2); final in Phase 5.
"""
SYSTEM_PROMPT = """You are Mentor, the long-horizon career agent of an AI-native competency school.
Learner: {learner_name}. Active stack: {stacks}. Progress: {progress}.
Microcredentials earned: {microcredentials}. Recent artifacts: {artifacts}.
Your job: narrate the learner's trajectory — where they are now, what their
competency progress unlocks next, and how their artifacts position them in
the AI-era labor market. Two to three paragraphs, forward-looking, concrete."""
PROMPT_VERSION = "mentor-v1-draft"
def render_context(learner_context) -> dict:
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
progress = (
f"{learner_context.active_competencies[0].title} in progress"
if learner_context.active_competencies
else "no active competencies"
)
return {
"learner_name": learner_context.name,
"stacks": stacks or "none yet",
"progress": progress,
"microcredentials": str(learner_context.microcredential_count),
"artifacts": ", ".join(learner_context.recent_artifacts) or "none yet",
}
@@ -0,0 +1,20 @@
"""Proctor agent prompt — integrity signals with coaching interventions (REQ-2-009).
Persona: supportive observer, not punitive. Classifies integrity signals from
telemetry and recommends coaching interventions. Versioned: v1 draft (Phase 2);
final persona + signal models in Phase 5.
"""
SYSTEM_PROMPT = """You are Proctor, the integrity-support agent of an AI-native competency school.
Learner: {learner_name}.
You receive a telemetry timeline of session events (focus, tab switches,
paste events, idle time). Your job: classify each signal by type and severity,
then recommend ONE supportive coaching intervention — never punitive, never
accusatory. Assume good faith; most signals have innocent explanations.
Respond with ONLY valid JSON matching the provided signals schema."""
PROMPT_VERSION = "proctor-v1-draft"
def render_context(learner_context) -> dict:
return {"learner_name": learner_context.name}
@@ -0,0 +1,28 @@
"""Tutor agent prompt — concept delivery, Socratic questioning (REQ-2-006).
Persona: patient expert teacher. Delivers one concept at a time, checks
understanding with Socratic questions, uses worked examples.
Versioned: v1 draft (Phase 2); final persona in Phase 3.
"""
SYSTEM_PROMPT = """You are Tutor, the concept-delivery agent of an AI-native competency school.
Learner: {learner_name}. Active stack: {stacks}. Progress: {progress}.
Your job: teach concepts clearly, one at a time. Prefer Socratic questioning —
guide the learner to the insight with a worked example, then ask one question
that checks understanding before moving on. Never dump long walls of text."""
PROMPT_VERSION = "tutor-v1-draft"
def render_context(learner_context) -> dict:
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
progress = (
f"{learner_context.active_competencies[0].title} in progress"
if learner_context.active_competencies
else "no active competencies"
)
return {
"learner_name": learner_context.name,
"stacks": stacks or "none yet",
"progress": progress,
}
+11
View File
@@ -0,0 +1,11 @@
{
"name": "@nextcraft/ai-service",
"private": true,
"version": "0.2.0",
"scripts": {
"dev": "bash scripts/dev.sh",
"test": "bash scripts/test.sh",
"bootstrap": "bash scripts/bootstrap.sh",
"lint": "bash scripts/lint.sh"
}
}
+40
View File
@@ -0,0 +1,40 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "nextcraft-ai-service"
version = "0.2.0"
description = "Nextcraft AI tutor service — six LLM agents behind a provider-agnostic layer"
requires-python = ">=3.11"
dependencies = [
"fastapi>=0.141,<0.142",
"uvicorn>=0.52,<0.53",
"pydantic>=2.13,<2.14",
"pydantic-settings>=2.15,<2.16",
"httpx>=0.28,<0.29",
"sse-starlette>=3.4,<3.5",
]
[project.optional-dependencies]
dev = [
"pytest>=9.1,<10",
"pytest-asyncio>=1.4,<2",
"ruff>=0.14",
]
[tool.setuptools.packages.find]
include = ["ai_service*"]
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
[tool.ruff]
line-length = 100
target-version = "py311"
[tool.ruff.lint]
select = ["E", "F", "W", "I", "UP", "B"]
# B008: Depends() in argument defaults is the idiomatic FastAPI DI pattern
ignore = ["B008"]
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Idempotent bootstrap: create venv + install deps.
# Handles Debian systems without python3-venv/ensurepip via --without-pip + get-pip.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
VENV="$APP_DIR/.venv"
mkdir -p "$HOME/.cache/ciagent"
if [ ! -x "$VENV/bin/python3" ]; then
if python3 -m venv "$VENV" 2>/dev/null; then
:
else
# No ensurepip available — create bare venv and bootstrap pip separately.
python3 -m venv --without-pip "$VENV"
fi
fi
if [ ! -x "$VENV/bin/pip" ]; then
GET_PIP="$HOME/.cache/ciagent/get-pip.py"
if [ ! -f "$GET_PIP" ]; then
curl -sSf --max-time 60 https://bootstrap.pypa.io/get-pip.py -o "$GET_PIP"
fi
"$VENV/bin/python3" "$GET_PIP" --quiet
fi
"$VENV/bin/pip" install --quiet --upgrade pip
"$VENV/bin/pip" install --quiet -e "$APP_DIR[dev]"
echo "bootstrap complete: $VENV"
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
# Dev server: export secrets (if present) then run uvicorn on :8420.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
REPO_ROOT="$(cd "$APP_DIR/../.." && pwd)"
VENV="$APP_DIR/.venv"
if [ ! -x "$VENV/bin/uvicorn" ]; then
echo "venv missing — run scripts/bootstrap.sh first" >&2
exit 1
fi
SECRETS="$REPO_ROOT/.ciagent/.env.secrets"
if [ -f "$SECRETS" ]; then
while IFS='=' read -r key value; do
case "$key" in
OLLAMA_API_KEY) export AI_OLLAMA_CLOUD_API_KEY="$value" ;;
OLLAMA_BASE_URL) export AI_OLLAMA_CLOUD_BASE_URL="$value" ;;
AI_TUTOR_MODEL) export AI_MODEL="$value" ;;
esac
done < "$SECRETS"
fi
cd "$APP_DIR"
exec "$VENV/bin/uvicorn" ai_service.main:app --reload --port 8420
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Lint: ruff check over the ai-service tree.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
VENV="$APP_DIR/.venv"
if [ ! -x "$VENV/bin/ruff" ]; then
echo "venv missing — run scripts/bootstrap.sh first" >&2
exit 1
fi
cd "$APP_DIR"
exec "$VENV/bin/ruff" check .
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Test runner: pytest via venv — mock provider only, zero network calls.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
VENV="$APP_DIR/.venv"
if [ ! -x "$VENV/bin/pytest" ]; then
echo "venv missing — run scripts/bootstrap.sh first" >&2
exit 1
fi
cd "$APP_DIR"
exec "$VENV/bin/pytest" -q "$@"
View File
+55
View File
@@ -0,0 +1,55 @@
"""BaseAgent contract tests — stub agent + mock provider."""
from ai_service.agents.base import BaseAgent
from ai_service.config import Settings
from ai_service.corpus.learner_context import get_learner_context
from ai_service.llm.mock import MockProvider
from ai_service.llm.types import Message
class StubAgent(BaseAgent):
name = "stub"
def system_prompt(self, learner_context=None) -> str:
return "You are Stub. Answer briefly."
def make_agent() -> StubAgent:
return StubAgent(MockProvider(), Settings(provider="mock"))
async def test_build_messages_composition():
agent = make_agent()
history = [Message(role="user", content="earlier"), Message(role="assistant", content="reply")]
messages = agent.build_messages(history, "new question")
assert messages[0].role == "system"
assert messages[0].content == "You are Stub. Answer briefly."
assert [m.content for m in messages[1:]] == ["earlier", "reply", "new question"]
async def test_stream_reply_yields_deltas():
agent = make_agent()
tokens = [t async for t in agent.stream_reply(user_input="hello")]
assert len(tokens) >= 1
assert all(isinstance(t, str) for t in tokens)
async def test_stream_reply_with_history_and_context():
agent = make_agent()
ctx = get_learner_context()
history = [Message(role="user", content="earlier"), Message(role="assistant", content="ok")]
tokens = [t async for t in agent.stream_reply(history, "next", ctx)]
assert tokens
async def test_structured_reply_requires_schema():
import pytest
agent = make_agent()
with pytest.raises(ValueError):
await agent.structured_reply(user_input="x", schema=None)
async def test_name_defaults():
assert make_agent().name == "stub"
@@ -0,0 +1,71 @@
"""Learner context corpus tests — D-021 ID alignment + prompt rendering."""
from ai_service.corpus.learner_context import (
LEARNER_CONTEXTS,
get_learner_context,
)
from ai_service.prompts.assessor import render_context as render_assessor
from ai_service.prompts.coach import SYSTEM_PROMPT as COACH_PROMPT
from ai_service.prompts.coach import render_context as render_coach
from ai_service.prompts.lab import SYSTEM_PROMPT as LAB_PROMPT
from ai_service.prompts.lab import render_context as render_lab
from ai_service.prompts.mentor import SYSTEM_PROMPT as MENTOR_PROMPT
from ai_service.prompts.mentor import render_context as render_mentor
from ai_service.prompts.proctor import SYSTEM_PROMPT as PROCTOR_PROMPT
from ai_service.prompts.proctor import render_context as render_proctor
from ai_service.prompts.tutor import SYSTEM_PROMPT as TUTOR_PROMPT
from ai_service.prompts.tutor import render_context as render_tutor
def test_default_learner_resolves():
ctx = get_learner_context()
assert ctx.learner_id == "learner-001"
assert ctx.name == "Alex Rivera"
def test_unknown_learner_falls_back_to_default():
assert get_learner_context("nobody").learner_id == "learner-001"
def test_ids_align_with_ts_mock_data():
# D-021: identical ID strings to packages/mock-data (learner-progress.ts)
ctx = get_learner_context("learner-001")
stack_ids = {s.stack_id for s in ctx.active_stacks}
assert {"stack-orchestration", "stack-safety"} <= stack_ids
competency_ids = {c.competency_id for c in ctx.active_competencies}
assert "stack-orchestration-c001" in competency_ids
def test_all_prompt_modules_render_without_keyerror():
ctx = get_learner_context()
for render in (render_coach, render_tutor, render_mentor, render_assessor):
values = render(ctx)
assert isinstance(values, dict)
assert "learner_name" in values
assert values["learner_name"] == "Alex Rivera"
def test_prompts_format_map_with_rendered_context():
ctx = get_learner_context()
for prompt, render in (
(COACH_PROMPT, render_coach),
(TUTOR_PROMPT, render_tutor),
(MENTOR_PROMPT, render_mentor),
):
rendered = prompt.format_map(render(ctx))
assert "Alex Rivera" in rendered
assert "{" not in rendered # all placeholders filled
def test_lab_and_proctor_prompts_render():
ctx = get_learner_context()
# Each module renders through its OWN render_context (its own placeholders).
assert "Alex Rivera" in LAB_PROMPT.format_map(render_lab(ctx))
assert "Alex Rivera" in PROCTOR_PROMPT.format_map(render_proctor(ctx))
for prompt, render in ((LAB_PROMPT, render_lab), (PROCTOR_PROMPT, render_proctor)):
rendered = prompt.format_map(render(ctx))
assert "{" not in rendered # all placeholders filled
def test_two_seed_learners_exist():
assert set(LEARNER_CONTEXTS) == {"learner-001", "learner-002"}
@@ -0,0 +1,53 @@
"""Agent registry tests — register/get round-trip, error paths (G-4)."""
import pytest
from ai_service.agents.base import BaseAgent
from ai_service.agents.registry import (
AgentRegistry,
DuplicateAgentError,
UnknownAgentError,
)
from ai_service.config import Settings
from ai_service.llm.mock import MockProvider
class DummyAgent(BaseAgent):
name = "dummy"
def system_prompt(self, learner_context=None) -> str:
return "dummy"
def make_factory():
def factory(provider, settings):
return DummyAgent(provider, settings)
return factory
def test_register_and_get():
registry = AgentRegistry()
registry.register("dummy", make_factory())
agent = registry.get(MockProvider(), Settings(provider="mock"), "dummy")
assert isinstance(agent, DummyAgent)
assert agent.name == "dummy"
def test_unknown_agent_raises():
registry = AgentRegistry()
with pytest.raises(UnknownAgentError):
registry.get(MockProvider(), Settings(provider="mock"), "ghost")
def test_duplicate_registration_raises():
registry = AgentRegistry()
registry.register("dummy", make_factory())
with pytest.raises(DuplicateAgentError):
registry.register("dummy", make_factory())
def test_names_sorted():
registry = AgentRegistry()
registry.register("zeta", make_factory())
registry.register("alpha", make_factory())
assert registry.names() == ["alpha", "zeta"]
@@ -0,0 +1,86 @@
"""SessionStore tests — create/append/window/LRU/agent scoping (D-019)."""
import pytest
from ai_service.agents.session import InMemorySessionStore
from ai_service.llm.types import Message
def msg(n: int) -> Message:
return Message(role="user", content=f"m{n}")
async def test_create_and_get():
store = InMemorySessionStore()
session = await store.create("s1", agent="coach")
assert session.agent == "coach"
assert (await store.get("s1")).session_id == "s1"
assert await store.get("missing") is None
async def test_append_and_history_window():
store = InMemorySessionStore()
await store.create("s1", agent="coach")
for i in range(30):
await store.append("s1", msg(i))
window = await store.history_window("s1", max_messages=20)
assert len(window) == 20
assert window[0].content == "m10" # last 20 of m0..m29
assert window[-1].content == "m29"
async def test_default_window_uses_20():
store = InMemorySessionStore()
await store.create("s1", agent="coach")
for i in range(25):
await store.append("s1", msg(i))
window = await store.history_window("s1")
assert len(window) == 20
assert window[0].content == "m5"
async def test_lru_eviction_at_cap():
store = InMemorySessionStore(window=20, max_sessions=3)
for i in range(3):
await store.create(f"s{i}", agent="coach")
# touch s0 so s1 becomes least-recently-used
await store.get("s0")
await store.create("s3", agent="coach") # evicts s1
assert await store.get("s1") is None
assert await store.get("s0") is not None
assert await store.get("s2") is not None
assert await store.get("s3") is not None
async def test_lru_eviction_at_default_500_cap():
store = InMemorySessionStore() # defaults: window=20, max_sessions=500
for i in range(500):
await store.create(f"s{i}", agent="coach")
await store.get("s0") # touch the oldest → s1 becomes least-recently-used
await store.create("s500", agent="coach") # evicts s1
assert await store.get("s1") is None
assert await store.get("s0") is not None
assert await store.get("s499") is not None
assert await store.get("s500") is not None
async def test_append_unknown_session_raises():
store = InMemorySessionStore()
with pytest.raises(KeyError):
await store.append("nope", msg(0))
async def test_delete():
store = InMemorySessionStore()
await store.create("s1", agent="coach")
await store.delete("s1")
assert await store.get("s1") is None
async def test_sessions_are_agent_scoped():
store = InMemorySessionStore()
a = await store.create("coach-session", agent="coach")
b = await store.create("tutor-session", agent="tutor")
assert a.agent == "coach"
assert b.agent == "tutor"
assert a.session_id != b.session_id
@@ -0,0 +1,139 @@
"""Structured output defense tests — 4 layers (D-020), against mock providers."""
import pytest
from pydantic import BaseModel
from ai_service.agents.structured import (
StructuredOutputError,
extract_json_object,
parse_structured,
structured_completion,
)
from ai_service.llm.mock import MockProvider, ScriptedJSONProvider
from ai_service.llm.types import Message
class Score(BaseModel):
score: int
verdict: str
HINT = '{"score": <int 0-100>, "verdict": "<short verdict>"}'
def test_extract_json_plain():
assert extract_json_object('{"a": 1}') == '{"a": 1}'
def test_extract_json_fenced():
text = '```json\n{"a": 1}\n```'
assert extract_json_object(text) == '{"a": 1}'
def test_extract_json_with_prose_around():
text = 'Sure! Here is my answer: {"a": {"b": "x } y"}, "c": 2} hope that helps'
assert extract_json_object(text) == '{"a": {"b": "x } y"}, "c": 2}'
def test_extract_json_no_object_raises():
with pytest.raises(StructuredOutputError):
extract_json_object("no json here")
def test_extract_json_unbalanced_raises():
with pytest.raises(StructuredOutputError):
extract_json_object('{"a": 1')
def test_parse_structured_valid():
result = parse_structured('{"score": 88, "verdict": "solid"}', Score)
assert result.score == 88
def test_parse_structured_invalid_schema_raises():
with pytest.raises(StructuredOutputError):
parse_structured('{"wrong": "shape"}', Score)
async def test_structured_completion_happy_path():
provider = ScriptedJSONProvider({"score": 91, "verdict": "excellent work"})
messages = [Message(role="user", content="grade my artifact")]
result = await structured_completion(
provider, messages, model="m", schema=Score, schema_hint=HINT
)
assert result.score == 91
assert result.verdict == "excellent work"
async def test_structured_completion_retries_then_raises():
# Plain MockProvider returns non-schema JSON for json_object requests →
# both attempts fail validation → StructuredOutputError after ONE retry.
provider = MockProvider()
provider.received_calls = []
messages = [Message(role="user", content="grade me")]
with pytest.raises(StructuredOutputError):
await structured_completion(
provider, messages, model="m", schema=Score, schema_hint=HINT
)
async def test_structured_completion_retry_succeeds_after_invalid_first_response():
# Layer 4 recovery: first reply is wrong-schema fenced JSON, retry is valid.
class FlakyProvider(MockProvider):
def __init__(self) -> None:
super().__init__()
self.n = 0
self.retry_request: list[Message] = []
async def chat(self, messages, *, model, temperature=0.7, response_format=None):
self.n += 1
if self.n == 1:
return '```json\n{"summary": "wrong shape"}\n```'
self.retry_request = list(messages)
return '{"score": 75, "verdict": "recovered"}'
provider = FlakyProvider()
messages = [Message(role="user", content="grade me")]
result = await structured_completion(
provider, messages, model="m", schema=Score, schema_hint=HINT
)
assert result.score == 75
assert result.verdict == "recovered"
assert provider.n == 2
# The retry must feed the validation error back to the model.
retry_contents = " ".join(m.content for m in provider.retry_request)
assert "previous response was invalid" in retry_contents
assert HINT in retry_contents
async def test_structured_completion_is_bounded_to_one_retry():
# Permanently-invalid provider: exactly two provider calls, then raise.
class CountingProvider(MockProvider):
def __init__(self) -> None:
super().__init__()
self.calls = 0
async def chat(self, messages, *, model, temperature=0.7, response_format=None):
self.calls += 1
return await super().chat(
messages, model=model, response_format=response_format
)
provider = CountingProvider()
messages = [Message(role="user", content="grade me")]
with pytest.raises(StructuredOutputError, match="after retry"):
await structured_completion(
provider, messages, model="m", schema=Score, schema_hint=HINT
)
assert provider.calls == 2
async def test_structured_completion_schema_instruction_appended():
provider = ScriptedJSONProvider({"score": 70, "verdict": "passing"})
messages = [Message(role="user", content="grade")]
await structured_completion(provider, messages, model="m", schema=Score, schema_hint=HINT)
# Determinism check: same request yields same reply
again = await structured_completion(
provider, messages, model="m", schema=Score, schema_hint=HINT
)
assert again.score == 70
@@ -0,0 +1,155 @@
"""Chat endpoint session integration tests — history persistence + windowed replay."""
import json
def stream_events(client, payload) -> list[dict]:
with client.stream("POST", "/v1/chat/stream", json=payload) as response:
assert response.status_code == 200
events = []
for line in response.iter_lines():
if line.startswith("data:"):
d = line.removeprefix("data:").strip()
if d == "[DONE]":
events.append({"type": "[DONE]"})
else:
events.append(json.loads(d))
return events
def test_first_turn_creates_session_and_persists(client):
payload = {
"agent": "coach",
"session_id": "sess-1",
"messages": [{"role": "user", "content": "first turn"}],
}
events = stream_events(client, payload)
assert events[0]["type"] == "meta"
assert events[0]["session_id"] == "sess-1"
store = client.app.state.session_store
import asyncio
async def check():
return await store.history_window("sess-1")
contents = [m.content for m in asyncio.run(check())]
assert "first turn" in contents
assert any("Think" in c or "[" in c for c in contents) # mock reply persisted
def test_second_turn_replays_windowed_history(client):
payload1 = {
"agent": "coach",
"session_id": "sess-2",
"messages": [{"role": "user", "content": "turn one"}],
}
stream_events(client, payload1)
# Second turn: the API passes history + new message to the provider.
# With the mock provider we cannot observe provider inputs directly,
# but the session store must now hold both turns.
store = client.app.state.session_store
# The store is async; use the app's internals through a short event loop
import asyncio
result = {}
async def check():
result["window"] = await store.history_window("sess-2")
asyncio.run(check())
contents = [m.content for m in result["window"]]
assert "turn one" in contents
assert any(m.role == "assistant" for m in result["window"])
def test_second_turn_replays_history_to_provider(client):
# Observable provider input: a recording provider wrapper captures what
# the endpoint sends. Turn 2 must include turn 1's persisted messages.
from ai_service.llm.types import Message
provider = client.app.state.provider
original = provider.stream_chat
captured: list[list[Message]] = []
async def recording_stream(messages, *, model, temperature=0.7, response_format=None):
captured.append(list(messages))
async for t in original(
messages, model=model, temperature=temperature, response_format=response_format
):
yield t
provider.stream_chat = recording_stream
try:
stream_events(client, {
"agent": "coach", "session_id": "sess-replay",
"messages": [{"role": "user", "content": "turn one"}],
})
stream_events(client, {
"agent": "coach", "session_id": "sess-replay",
"messages": [{"role": "user", "content": "turn two"}],
})
finally:
provider.stream_chat = original
assert len(captured) == 2
turn1, turn2 = captured
assert [m.content for m in turn1] == ["turn one"]
turn2_contents = [m.content for m in turn2]
assert "turn one" in turn2_contents
assert "turn two" in turn2_contents
assert any(m.role == "assistant" for m in turn2) # persisted reply replayed
assert "turn two" == turn2_contents[-1] # new user turn last
def test_history_replay_is_windowed(client):
# Windowing: only the last 20 stored messages are replayed to the provider.
from ai_service.llm.types import Message
provider = client.app.state.provider
original = provider.stream_chat
captured: list[list[Message]] = []
async def recording_stream(messages, *, model, temperature=0.7, response_format=None):
captured.append(list(messages))
async for t in original(
messages, model=model, temperature=temperature, response_format=response_format
):
yield t
provider.stream_chat = recording_stream
try:
# 15 turns → 30 persisted messages (user + assistant per turn) > 20 window.
for i in range(15):
stream_events(client, {
"agent": "coach", "session_id": "sess-window",
"messages": [{"role": "user", "content": f"turn {i}"}],
})
finally:
provider.stream_chat = original
last_input = captured[-1]
contents = [m.content for m in last_input]
assert len(last_input) == 20 + 1 # window(20) + the new user message
assert "turn 0" not in contents # oldest messages trimmed out of replay
assert "turn 14" in contents
assert contents[-1] == "turn 14"
def test_session_agent_scoped(client):
payload = {
"agent": "tutor",
"session_id": "sess-3",
"messages": [{"role": "user", "content": "hello"}],
}
stream_events(client, payload)
import asyncio
store = client.app.state.session_store
async def check():
return await store.get("sess-3")
session = asyncio.run(check())
assert session.agent == "tutor"
@@ -0,0 +1,78 @@
"""SSE chat stream endpoint tests — envelope ordering, errors, headers."""
import json
def stream_lines(client, payload: dict) -> list[str]:
with client.stream("POST", "/v1/chat/stream", json=payload) as response:
assert response.status_code == 200
assert "no-cache" in response.headers.get("cache-control", "")
return [line for line in response.iter_lines() if line.strip()]
def parse_events(raw_lines: list[str]) -> list[dict]:
"""Parse SSE lines into event dicts; strips event:/data: prefixes."""
events = []
for line in raw_lines:
if line.startswith("data:"):
data = line.removeprefix("data:").strip()
if data == "[DONE]":
events.append({"type": "[DONE]"})
else:
events.append(json.loads(data))
return events
PAYLOAD = {
"agent": "tutor",
"session_id": "s1",
"messages": [{"role": "user", "content": "hello"}],
}
def test_meta_first_then_deltas_done_done_sentinel(client):
events = parse_events(stream_lines(client, PAYLOAD))
assert events[0]["type"] == "meta"
assert events[0]["agent"] == "tutor"
assert events[0]["session_id"] == "s1"
assert events[-1]["type"] == "[DONE]"
middle = events[1:-1]
deltas = [e for e in middle if e["type"] == "delta"]
dones = [e for e in middle if e["type"] == "done"]
assert len(deltas) >= 1
assert len(dones) == 1
assert dones[0]["finish_reason"] == "stop"
# delta events after meta, done before [DONE]
assert events.index(dones[0]) > events.index(deltas[0])
def test_empty_messages_rejected(client):
response = client.post(
"/v1/chat/stream",
json={"agent": "tutor", "session_id": "s", "messages": []},
)
assert response.status_code == 422
def test_mid_stream_failure_yields_error_then_done(client):
provider = client.app.state.provider
provider.fail_mid_stream_at_index = 1
events = parse_events(stream_lines(client, PAYLOAD))
provider.fail_mid_stream_at_index = None
error_events = [e for e in events if e["type"] == "error"]
assert len(error_events) == 1
assert error_events[0]["code"] == "provider_error"
assert events[-1]["type"] == "[DONE]"
# error must precede the terminal sentinel
assert events.index(error_events[0]) < events.index(events[-1])
def test_pre_first_byte_failure_yields_provider_unavailable(client):
provider = client.app.state.provider
provider.fail_before_first_token = True
events = parse_events(stream_lines(client, PAYLOAD))
provider.fail_before_first_token = False
error_events = [e for e in events if e["type"] == "error"]
assert len(error_events) == 1
assert error_events[0]["code"] == "provider_unavailable"
assert events[-1]["type"] == "[DONE]"
+32
View File
@@ -0,0 +1,32 @@
"""Test suite — conftest: mock provider only, zero network (enforced)."""
import os
import pytest
from fastapi.testclient import TestClient
from ai_service.config import Settings
from ai_service.llm.mock import MockProvider
from ai_service.main import create_app
@pytest.fixture()
def settings() -> Settings:
os.environ["AI_PROVIDER"] = "mock"
return Settings(provider="mock", model="gemma4:31b", port=8421)
@pytest.fixture()
def app(settings: Settings):
return create_app(settings)
@pytest.fixture()
def client(app):
with TestClient(app) as c:
# Mechanical cloud-free guard (GRILL advisory a): the app under test
# MUST be wired to the deterministic mock provider.
assert isinstance(app.state.provider, MockProvider), (
f"tests must run against MockProvider, got {type(app.state.provider).__name__}"
)
yield c
+73
View File
@@ -0,0 +1,73 @@
"""MockProvider tests — determinism, JSON mode, failure modes."""
import pytest
from ai_service.llm.mock import MockProvider
from ai_service.llm.types import Message
MSGS = [Message(role="user", content="hello there")]
async def test_stream_is_deterministic():
p1, p2 = MockProvider(), MockProvider()
out1 = [t async for t in p1.stream_chat(MSGS, model="m")]
out2 = [t async for t in p2.stream_chat(MSGS, model="m")]
assert "".join(out1) == "".join(out2)
assert out1 == out2
async def test_stream_content_differs_for_different_input():
p = MockProvider()
a = "".join([t async for t in p.stream_chat(MSGS, model="m")])
b = "".join(
[t async for t in p.stream_chat([Message(role="user", content="other")], model="m")]
)
assert a != b
async def test_json_object_response_format():
import json
p = MockProvider()
out = "".join(
[
t
async for t in p.stream_chat(
MSGS, model="m", response_format={"type": "json_object"}
)
]
)
assert json.loads(out) == {"summary": "mock structured reply", "confidence": 0.87}
async def test_fail_before_first_token():
p = MockProvider()
p.fail_before_first_token = True
with pytest.raises(RuntimeError):
async for _ in p.stream_chat(MSGS, model="m"):
pass
async def test_fail_mid_stream():
p = MockProvider()
p.fail_mid_stream_at_index = 2
tokens = []
with pytest.raises(RuntimeError):
async for t in p.stream_chat(MSGS, model="m"):
tokens.append(t)
assert len(tokens) == 2
async def test_cancellation_records_abort():
p = MockProvider()
gen = p.stream_chat(MSGS, model="m")
await gen.__anext__()
await gen.aclose()
assert p.abort_recorded is True
async def test_chat_returns_full_reply():
p = MockProvider()
reply = await p.chat(MSGS, model="m")
streamed = "".join([t async for t in p.stream_chat(MSGS, model="m")])
assert reply == streamed
@@ -0,0 +1,177 @@
"""OpenAICompatProvider tests — byte-exact SSE parsing via httpx.MockTransport.
Covers: multi-delta happy path, keep-alive comment lines, [DONE] sentinel,
malformed line tolerance, missing optional fields, non-streaming chat(),
response_format auto-degrade on 400, api_key never leaking into exceptions.
"""
import json
import httpx
import pytest
from ai_service.llm.openai_compat import OpenAICompatProvider
from ai_service.llm.types import Message
MSGS = [Message(role="user", content="hi")]
KEY = "sk-test-abc123"
def make_client(handler) -> httpx.AsyncClient:
transport = httpx.MockTransport(handler)
return httpx.AsyncClient(transport=transport)
def sse_body(deltas: list[str], with_comments: bool = True) -> bytes:
lines = []
if with_comments:
lines.append(": ping")
for d in deltas:
lines.append("data: " + json.dumps({
"id": "chatcmpl-1", "object": "chat.completion.chunk",
"created": 1, "model": "gemma4:31b",
"choices": [{"index": 0, "delta": {"content": d}, "finish_reason": None}],
}))
if with_comments:
lines.append(": ping")
lines.append("data: [DONE]")
return ("\n".join(lines) + "\n").encode()
async def test_stream_happy_path_with_comments_and_done():
def handler(request: httpx.Request) -> httpx.Response:
assert request.url.path == "/v1/chat/completions"
return httpx.Response(200, content=sse_body(["Hel", "lo", " world"]))
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1")
tokens = [t async for t in provider.stream_chat(MSGS, model="gemma4:31b")]
assert tokens == ["Hel", "lo", " world"]
async def test_stream_tolerates_malformed_lines():
body = (
"data: not-json\n"
"data: "
+ json.dumps({
"id": "x", "object": "chat.completion.chunk", "created": 1, "model": "m",
"choices": [{"index": 0, "delta": {"content": "ok"}, "finish_reason": None}],
})
+ "\ndata: [DONE]\n"
)
def handler(request):
return httpx.Response(200, content=body.encode())
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1")
tokens = [t async for t in provider.stream_chat(MSGS, model="m")]
assert tokens == ["ok"]
async def test_stream_skips_empty_content_and_empty_choices():
chunk_empty_delta = json.dumps(
{"id": "x", "model": "m",
"choices": [{"index": 0, "delta": {}, "finish_reason": None}]}
)
chunk_empty_choices = json.dumps({"id": "x", "model": "m", "choices": []})
chunk_yes = json.dumps(
{"id": "x", "model": "m",
"choices": [{"index": 0, "delta": {"content": "yes"}, "finish_reason": None}]}
)
body = (
f"data: {chunk_empty_delta}\n"
f"data: {chunk_empty_choices}\n"
f"data: {chunk_yes}\n"
"data: [DONE]\n"
)
def handler(request):
return httpx.Response(200, content=body.encode())
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1")
tokens = [t async for t in provider.stream_chat(MSGS, model="m")]
assert tokens == ["yes"]
async def test_chat_non_streaming():
def handler(request):
payload = json.loads(request.content)
assert payload["stream"] is False
body = {"id": "1", "object": "chat.completion", "created": 1, "model": "m",
"choices": [{"index": 0, "message": {"role": "assistant",
"content": "full reply"},
"finish_reason": "stop"}]}
return httpx.Response(200, json=body)
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1")
assert await provider.chat(MSGS, model="m") == "full reply"
async def test_response_format_auto_degrades_on_400():
calls = []
def handler(request):
payload = json.loads(request.content)
calls.append(payload)
if "response_format" in payload:
return httpx.Response(400, json={"error": "response_format unsupported"})
body = {"id": "1", "object": "chat.completion", "created": 1, "model": "m",
"choices": [{"index": 0, "message": {"role": "assistant",
"content": "json"},
"finish_reason": "stop"}]}
return httpx.Response(200, json=body)
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1", json_mode="auto")
reply = await provider.chat(MSGS, model="m", response_format={"type": "json_object"})
assert reply == "json"
assert len(calls) == 2
assert "response_format" in calls[0]
assert "response_format" not in calls[1]
async def test_response_format_off_never_sends():
calls = []
def handler(request):
payload = json.loads(request.content)
calls.append(payload)
body = {"id": "1", "object": "chat.completion", "created": 1, "model": "m",
"choices": [{"index": 0, "message": {"role": "assistant",
"content": "x"},
"finish_reason": "stop"}]}
return httpx.Response(200, json=body)
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1", json_mode="off")
await provider.chat(MSGS, model="m", response_format={"type": "json_object"})
assert len(calls) == 1
assert "response_format" not in calls[0]
async def test_api_key_never_in_exception():
def handler(request):
raise httpx.ConnectError("connection refused while using sk-test-abc123")
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1", api_key=KEY)
with pytest.raises(RuntimeError) as exc_info:
async for _ in provider.stream_chat(MSGS, model="m"):
pass
assert "sk-test-abc123" not in str(exc_info.value)
async def test_bearer_header_sent():
seen = {}
def handler(request):
seen["auth"] = request.headers.get("Authorization")
return httpx.Response(200, content=sse_body(["x"]))
async with make_client(handler) as client:
provider = OpenAICompatProvider(client, "https://fake/v1", api_key=KEY)
_ = [t async for t in provider.stream_chat(MSGS, model="m")]
assert seen["auth"] == f"Bearer {KEY}"
+10
View File
@@ -0,0 +1,10 @@
"""Health endpoint tests."""
def test_health_returns_ok(client):
response = client.get("/health")
assert response.status_code == 200
data = response.json()
assert data["status"] == "ok"
assert data["provider"] == "mock"
assert data["model"] == "gemma4:31b"
+5 -1
View File
@@ -9,7 +9,11 @@
"build": "turbo build",
"lint": "turbo lint",
"typecheck": "turbo typecheck",
"clean": "turbo clean && rm -rf node_modules"
"clean": "turbo clean && rm -rf node_modules",
"ai:dev": "turbo run dev --filter=@nextcraft/ai-service",
"ai:test": "turbo run test --filter=@nextcraft/ai-service",
"ai:bootstrap": "turbo run bootstrap --filter=@nextcraft/ai-service",
"ai:lint": "turbo run lint --filter=@nextcraft/ai-service"
},
"devDependencies": {
"turbo": "^2.3.3",
+17
View File
@@ -17,6 +17,23 @@
},
"clean": {
"cache": false
},
"@nextcraft/ai-service#dev": {
"cache": false,
"persistent": true,
"outputs": []
},
"@nextcraft/ai-service#test": {
"cache": false,
"outputs": []
},
"@nextcraft/ai-service#bootstrap": {
"cache": false,
"outputs": []
},
"@nextcraft/ai-service#lint": {
"cache": false,
"outputs": []
}
}
}