Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 CIAgent
|
88a1dab810 | ||
|
CIAgent
|
71728e35c8 | ||
|
CIAgent
|
daa0463a97 | ||
|
CIAgent
|
810485ecc8 |
@@ -0,0 +1,160 @@
|
||||
# Nextcraft — ARCHITECTURE.md
|
||||
|
||||
## Overview
|
||||
|
||||
Nextcraft is a TypeScript monorepo (pnpm workspaces + turborepo) with a Next.js application hosting four surfaces (Learner, Marketplace, Employer Dashboard, Admin), a shared component library, typed mock data layer, and shared types package. As of v0.2, a Python FastAPI application (`apps/ai-service`) hosts six AI tutor agents backed by a provider-agnostic LLM layer.
|
||||
|
||||
**v0.2 additions:** real AI services (streaming chat), agent framework, mock engine inputs for Lab/Assessor/Proctor. **Still no database, no auth** — in-memory session store; real engines (sandbox fabric, assessment engine, identity) are v0.3+.
|
||||
|
||||
### Confirmed Technology Stack (v0.2)
|
||||
|
||||
| Technology | Version | Purpose |
|
||||
|------------|---------|---------|
|
||||
| Node.js | v24.15.0 | Runtime (web) |
|
||||
| pnpm | 12.3.4 | Package manager + workspaces |
|
||||
| turborepo | 2.3.3 | Build orchestration |
|
||||
| Next.js | 15 (App Router) | Web application framework |
|
||||
| React | 19+ | UI library |
|
||||
| TypeScript | 5.x | Type system |
|
||||
| Tailwind CSS | v4 | Utility-first CSS |
|
||||
| lucide-react | latest | Icons |
|
||||
| recharts | latest | Charts |
|
||||
| @xyflow/react | latest | Competency graph viewer |
|
||||
| Python | 3.11.2 | Runtime (ai-service) |
|
||||
| FastAPI | 0.141.x | AI service framework |
|
||||
| uvicorn | 0.52.x | ASGI server |
|
||||
| pydantic | 2.13.x | Request/response models, structured outputs |
|
||||
| pydantic-settings | 2.15.x | Settings + env-file loading (replaces python-dotenv) |
|
||||
| httpx | 0.28.x | Async LLM HTTP client (ollama-cloud + local providers) |
|
||||
| sse-starlette | 3.4.x | SSE framing, ping keep-alive |
|
||||
| pytest | 9.x | Test runner |
|
||||
| pytest-asyncio | 1.4.x | Async tests (auto mode) |
|
||||
| ruff | latest | Python lint (check-only, no formatter) — `pnpm ai:lint` |
|
||||
| ollama-cloud | https://ollama.com/v1 | Default LLM provider (OpenAI-compatible, Bearer auth) |
|
||||
|
||||
Deliberately **not** used: openai-python SDK (the `LLMProvider` protocol is the port; raw httpx keeps delta passthrough and ollama-cloud quirk tolerance), python-dotenv (pydantic-settings reads .env natively), respx (httpx MockTransport is built in).
|
||||
|
||||
### v0.2 Architecture Decisions (from Research)
|
||||
|
||||
1. **D-016 SSE envelope** — `meta` event (agent/session/model, flushed before first token) → raw OpenAI-compatible chunk passthrough → optional `done` → `error` event before `[DONE]` on mid-stream failure. Pre-first-byte failures use proper HTTP status. Headers: `Cache-Control: no-cache`, `X-Accel-Buffering: no`.
|
||||
2. **D-017 httpx direct client** — lifespan-managed `httpx.AsyncClient` (10s connect / 300s read), shared by ollama-cloud and local providers; no SDK.
|
||||
3. **D-018 Agent framework** — `BaseAgent` ABC (system_prompt/build_messages/stream_reply/structured_reply) + explicit registry; prompts are versioned code in `prompts/`.
|
||||
4. **D-019 Session store** — `SessionStore` protocol + `InMemorySessionStore` (asyncio.Lock, 20-message window, 500-cap LRU, agent-scoped sessions). DB-migration-ready.
|
||||
5. **D-020 Structured outputs** — 4-layer defense: `response_format` (auto-degrade) → prompt-embedded schema → fence-strip/first-balanced-object parse → single bounded retry.
|
||||
6. **D-021 Mock corpus in Python** — `ai_service/corpus/` pydantic-typed, convention-aligned with TS `packages/mock-data` (shared IDs, cross-referencing headers); no codegen in v0.2.
|
||||
7. **D-022 Monorepo integration** — zero-dependency shim `package.json` in apps/ai-service + `ai#*` turbo passthrough tasks (`cache:false, outputs:[]`) + root `ai:dev`/`ai:test` scripts + idempotent venv bootstrap.
|
||||
8. **D-023 Testing** — pytest-asyncio auto mode; TestClient `client.stream()` for SSE; httpx MockTransport for byte-exact provider parser tests; scripted mock provider incl. failure modes. Tests never call the cloud.
|
||||
|
||||
---
|
||||
|
||||
## Components
|
||||
|
||||
### apps/ai-service — AI Tutor Service (v0.2 NEW)
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `ai_service/main.py` | FastAPI app factory, lifespan (httpx client pool, provider factory), CORS (localhost only), /health | App entry | config, llm, agents, api |
|
||||
| `ai_service/config.py` | pydantic-settings Settings (env_prefix="AI_", env_file, SecretStr key) | Configuration only | None |
|
||||
| `ai_service/api/` | Endpoints: chat.py (POST /v1/chat/stream, SSE), lab.py, assessment.py, proctor.py, mentor.py; deps.py (DI) | Composes agents + sessions; never imported by llm/ or agents/ | agents, llm |
|
||||
| `ai_service/llm/` | types.py (Message; ChatDelta/ChoiceDelta removed in P3 — no consumers), base.py (LLMProvider protocol), openai_compat.py (ollama-cloud + local), mock.py (deterministic), factory.py | Never imports agents/ or api/ | config |
|
||||
| `ai_service/agents/` | base.py (BaseAgent ABC), registry.py, session.py (SessionStore), structured.py (JSON defense), coach/tutor/lab/assessor/proctor/mentor.py | Never imports api/ | llm, prompts, corpus |
|
||||
| `ai_service/prompts/` | Per-agent system prompt constants + render_context functions (str.format_map) | Data only | None |
|
||||
| `ai_service/corpus/` | Mock engine inputs: learner_context.py, telemetry.py (Lab/Proctor scenarios), artifacts.py (pre-baked artifacts, rubrics, transcripts) | Pydantic-typed; aligned with TS packages/mock-data by convention | None |
|
||||
| `scripts/` | bootstrap.sh (venv + pip install idempotent), dev.sh (exports keys from .ciagent/.env.secrets → uvicorn), test.sh (pytest), lint.sh (ruff check, G-3) | Dev entry points | pyproject.toml |
|
||||
| `tests/` | conftest.py (mock provider, settings override, TestClient), health, llm (MockTransport parser), agents (framework + per-agent), api (SSE stream tests) | Mock provider only — no cloud | all |
|
||||
|
||||
**Module boundary rules:** `llm/` never imports `agents/` or `api/`; `agents/` never imports `api/`; `api/` composes both via DI. `corpus/` is the only home of mock engine data. Prompts are code — versioned and reviewed in git.
|
||||
|
||||
### apps/web — Next.js Application
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `app/(learner)/` | Learner surface route group: landing, catalog, competency stack, dashboard, byte viewer, sandbox mockup, assessment mockup | Learner-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/(marketplace)/` | Marketplace surface route group: job board, job detail, employer profile, search/filter, pricing | Marketplace-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/(employer)/` | Employer dashboard route group: overview, talent search, candidate profile, posting management | Employer-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/(admin)/` | Admin surface route group: overview, learner management, competency graph viewer, moderation | Admin-only routes and layouts | packages/ui, packages/mock-data, packages/types |
|
||||
| `app/layout.tsx` | Root layout: theme provider, navigation shell, responsive container | All routes | packages/ui |
|
||||
| `components/` | Surface-specific components (learner/, marketplace/, employer/, admin/) plus shared chrome (navigation-shell, header/footer, role-switcher, theme-provider, breadcrumbs, dark-mode-toggle) | App-level components | packages/ui |
|
||||
| `hooks/` | use-chat-stream.ts — SSE client hook: fetch + ReadableStream, byte buffering + frame reassembly, idempotent AbortController cleanup | Client components only | ai-service SSE |
|
||||
| `lib/` | sse.ts (shared SSE frame parser — CRLF normalization + `: ping` immunity, G-1), breadcrumbs.ts, format.ts | Pure utilities | None |
|
||||
|
||||
### packages/ui — Shared Component Library
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `tokens/` | Design tokens as TS constants: colors, spacing, radii, shadows, breakpoints (mirrored as Tailwind v4 `@theme` tokens in apps/web globals.css) | Foundation layer — no dependencies | None |
|
||||
| `primitives/` | Button, Input, Card, Badge, Avatar — each with a Storybook story | Atomic UI components | tokens, packages/types |
|
||||
|
||||
Composite/layout/theme components (navigation shell, tables, chat panels, graph viewer, theme provider) live in `apps/web/components/` as app-level components, not in packages/ui.
|
||||
|
||||
### packages/mock-data — Mock Data Layer
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `competency-stacks.ts` | 5 competency stacks (AI Orchestration Engineer, AI Safety & Governance, Human-AI Product Designer, AI-Augmented Field Operator, Computational Sciences), each with 12-18 competencies | Typed mock data | packages/types |
|
||||
| `jobs.ts` | 20+ mock AI-era job listings with skills, seniority, salary, match scores | Typed mock data | packages/types |
|
||||
| `candidates.ts` | 15+ mock candidate profiles with artifacts, process traces, defense scores, microcredentials | Typed mock data | packages/types |
|
||||
| `employers.ts` | 10+ mock employer profiles with logos, descriptions, open positions | Typed mock data | packages/types |
|
||||
| `learner-progress.ts` | Mock learner progress data: active competencies, completion percentages, recent artifacts | Typed mock data | packages/types |
|
||||
| `admin.ts` | Admin surface mock data: platform metrics, activity feed, system health, learner roster (admin view), moderation queues | Typed mock data | packages/types |
|
||||
| `ai-scenarios.ts` | AI engine-input scenario IDs + display metadata for the learner agent panels; IDs string-identical to `ai_service/corpus/` (D-021) | Typed mock data | packages/types |
|
||||
|
||||
### packages/types — Shared Types
|
||||
|
||||
| Component | Description | Boundaries | Depends On |
|
||||
|-----------|-------------|------------|------------|
|
||||
| `domain.ts` | Competency, CompetencyStack, Microcredential, Artifact, ProcessTrace, OralDefense, AssessmentRubric | Domain types | None |
|
||||
| `marketplace.ts` | Job, Employer, Candidate, JobPosting, TalentMatch, SearchFilter | Marketplace types | None |
|
||||
| `user.ts` | Learner, Admin, EmployerUser, AgeGroup, Role | User types | None |
|
||||
| `ui.ts` | Component props, theme config, breakpoint definitions | UI types | None |
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
[packages/mock-data + packages/types] [ai_service/corpus]
|
||||
│ (TS, web surfaces) │ (Python, agent inputs)
|
||||
▼ ▼
|
||||
[Next.js Route Groups] [ai-service agents]
|
||||
(learner)/ (marketplace)/ coach tutor lab assessor proctor mentor
|
||||
(employer)/ (admin)/ │
|
||||
│ ▼
|
||||
│ SSE (fetch + ReadableStream) [LLMProvider]
|
||||
└────── client components ◄───────────────────┤
|
||||
http://localhost:8420 ollama-cloud / local / mock
|
||||
(https://ollama.com/v1)
|
||||
```
|
||||
|
||||
- Web surfaces remain server-component-first; client components (chat, filters, graph viewer, dark mode toggle) fetch directly from ai-service over SSE (A-002: no Next.js API-route proxy in v0.2).
|
||||
- The LLM provider layer is a dumb pipe — OpenAI-compatible chunks pass through byte-identical; envelope logic (meta/done/error) lives only in the API layer (D-016).
|
||||
- Lab/Assessor/Proctor read mock scenarios from `ai_service/corpus/` — real engines are v0.3+.
|
||||
- All automated tests use the deterministic mock provider; the cloud is for manual probes only.
|
||||
|
||||
---
|
||||
|
||||
## Build Order (v0.2)
|
||||
|
||||
1. **AI service scaffolding** — apps/ai-service: FastAPI app, config, provider layer (ollama-cloud/local/mock), SSE chat endpoint, pytest harness, turbo integration
|
||||
2. **Agent framework** — BaseAgent, registry, session store, structured output, prompts scaffolding, learner-context corpus
|
||||
3. **Coach + Tutor agents** — full implementations, chat endpoint agent routing
|
||||
4. **Lab + Assessor agents** — telemetry scenarios + pre-baked artifacts corpus, /v1/lab/feedback + /v1/assessment/evaluate
|
||||
5. **Proctor + Mentor agents** — proctor scenarios, /v1/proctor/signals + /v1/mentor/narrative
|
||||
6. **Learner surface integration** — useChatStream hook, agent switcher, streaming/error/loading states, agent output panels across the four learner surfaces
|
||||
|
||||
The v0.1 build order (monorepo → types → mock data → tokens → primitives → layout → composites → surfaces → polish) is complete and preserved in git history (tags v0.0.1–v0.1.0).
|
||||
|
||||
---
|
||||
|
||||
## Future Architecture (Post-v0.2, for reference)
|
||||
|
||||
v0.2 delivers the ai-service skeleton that later milestones fill in:
|
||||
|
||||
- **Mock corpus → real engines** — Lab consumes real sandbox telemetry (v0.3 sandbox fabric); Assessor grades real process traces (v0.3 assessment engine); Proctor consumes real identity/attention signals (v0.3 identity verification)
|
||||
- **In-memory sessions → PostgreSQL + Drizzle ORM** — SessionStore protocol swap, no API changes
|
||||
- **Mock provider → per-agent model routing** — provider factory already selects by config; per-agent `AI_<AGENT>_MODEL` overrides
|
||||
- **No auth → real KYC + sessions** — A-008 dropped in v0.3 when identity verification lands
|
||||
- **No search → Semantic vector search (pgvector)** — Filter UI replaced with vector similarity search
|
||||
- **No payments → Payment processing** — Pricing page replaced with real subscription/payment flows
|
||||
|
||||
The monorepo structure (apps/web + apps/ai-service + packages/*) accommodates further apps without restructuring.
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"phase": 7,
|
||||
"stage": "execute",
|
||||
"milestone": "v0.2",
|
||||
"phase_role": "final",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-09-11T23:15:00Z"
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
# Nextcraft v0.2 — GRILL.md (Adversarial Review Verdict)
|
||||
|
||||
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-CHANGES · **Confidence:** 0.82
|
||||
|
||||
## Per-Axis Findings
|
||||
|
||||
| Axis | Verdict | Rationale |
|
||||
|------|---------|-----------|
|
||||
| Feasibility | PASS | Environment claims verified (venv works, all 8 pinned packages on PyPI for py3.11, port 8420 free, secrets gitignored). Stall points pre-mitigated (idempotent bootstrap, 300s read timeout, idempotent abort; reactStrictMode confirmed in next.config). |
|
||||
| Over-scoping | CONCERN (mild) | 4-layer JSON defense justified; 500-cap LRU is over-spec but cheap — keep, extend no further. Real gaps: no Python lint (fixed via G-3), cache dirs in gitignore (fixed), script path resolution (advisory c). |
|
||||
| Architecture risk | CONCERN | sse-starlette `: ping` frames invisible to short TestClient streams — parser gap fixed via G-1. Agent registration had two contradictory patterns — fixed via G-4. Cloud outage blast radius contained by mock-only tests. |
|
||||
| Phase sequencing | PASS | Integration-last correct: P1 freezes the SSE contract before client code exists; A-002 eliminates dev-server buffering trap. P4 heaviest but mechanical. |
|
||||
| Verification honesty | CONCERN | Persona distinctness circular against self-authored mocks (disclosed; cloud probe optional). P6 must-haves manual-only. D-021 ID alignment unmechanized (advisory b). Disclosed honestly. |
|
||||
| Cost/quota | PASS | Cloud burn bounded (~<100K tokens milestone-wide, manual probes only). Mock-only rule structurally enforced; mechanical guard via advisory (a). |
|
||||
| Milestone honesty | PASS (conditional) | Mock-engine caveats present everywhere that matters. Release note content now bound by G-5; dead `aiTutorResponses` disposal bound by G-5. |
|
||||
|
||||
## Binding Decisions (applied to PLAN.md/ROADMAP.md)
|
||||
|
||||
- **G-1 (BINDING):** SSE frame parser in Task 6-1-01 must ignore frames with no `data:` lines (sse-starlette ping keep-alive). Added to Action + Phase 6 must-have.
|
||||
- **G-2 (BINDING):** P6 end-to-end verification may run with `AI_PROVIDER=mock` fallback — real ai-service over HTTP is the requirement; provider choice is service-internal. Prevents cloud outage blocking P6.
|
||||
- **G-3 (BINDING):** ruff (check-only) added: Task 1-1-04, pyproject dev extra, scripts/lint.sh, root `ai:lint`, turbo `ai#lint`, Phase 1 must-have.
|
||||
- **G-4 (BINDING):** Mentor/Proctor registered centrally in `registry.py` (single registration pattern), matching P3/P4.
|
||||
- **G-5 (BINDING):** v0.2.0 release note must state Lab/Assessor/Proctor run on mock engine inputs (v0.3+ for real); dead `aiTutorResponses` export disposed of in P7.
|
||||
|
||||
## Advisory (non-binding; applied where cheap)
|
||||
|
||||
- (a) conftest asserts provider is MockProvider — **applied in P1 implementation**
|
||||
- (b) pytest reads packages/mock-data as text, asserts corpus IDs appear — **applied in P4 implementation**
|
||||
- (c) scripts resolve repo root via script-relative dirname — **folded into Task 1-1-04/1-1-03**
|
||||
- (d) `.pytest_cache/` + `.ruff_cache/` in .gitignore — **applied**
|
||||
- (e) keep LRU as-is; no further session-store sophistication in v0.2
|
||||
- (f) Task 6-2-02 REQ tag fixed to REQ-2-011 (routing) — **applied**
|
||||
|
||||
## Outcome
|
||||
|
||||
GO — all five binding decisions applied to PLAN.md/ROADMAP.md/.gitignore/ARCHITECTURE.md before Phase 1 execution. No axis requires escalation.
|
||||
@@ -0,0 +1,169 @@
|
||||
# Nextcraft — PERSONAS.md
|
||||
|
||||
## Persona Roster
|
||||
|
||||
### lead-developer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Coordinates task decomposition across web, AI service, and data territories; resolves conflicts between frontend, backend, and AI personas
|
||||
domain: coordination
|
||||
frameworks:
|
||||
- next.js
|
||||
- turborepo
|
||||
- pnpm
|
||||
- fastapi
|
||||
constraints:
|
||||
- pragmatic
|
||||
- battle-tested defaults
|
||||
- monorepo-architecture
|
||||
territory:
|
||||
- "**/package.json"
|
||||
- "**/turbo.json"
|
||||
- "**/pnpm-workspace.yaml"
|
||||
- "**/tsconfig.json"
|
||||
- "apps/ai-service/pyproject.toml"
|
||||
```
|
||||
|
||||
### frontend-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Phase 6 learner-surface integration — useChatStream hook, agent switcher, streaming/error/loading states, Lab/Assessor/Proctor output panels. Owns all page components, layouts, and surface-specific UI.
|
||||
domain: frontend
|
||||
frameworks:
|
||||
- react
|
||||
- next.js
|
||||
- tailwindcss
|
||||
- lucide-react
|
||||
- recharts
|
||||
- react-flow
|
||||
constraints:
|
||||
- component-first
|
||||
- server-components-default
|
||||
- minimal-client-js
|
||||
- sse-client-buffering (buffer bytes, split frames on \n\n, join data: lines)
|
||||
- abortcontroller-cleanup (idempotent abort in effect cleanup)
|
||||
- responsive-all-breakpoints
|
||||
- dark-mode-support
|
||||
territory:
|
||||
- "apps/web/**"
|
||||
- "packages/ui/**"
|
||||
- "packages/mock-data/**"
|
||||
- "packages/types/**"
|
||||
```
|
||||
|
||||
### data-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Owns TS mock data layer schema and typed definitions. Does NOT own the Python corpus (that is ai-engineer territory) — the two are aligned by documented convention (D-021).
|
||||
domain: data
|
||||
frameworks:
|
||||
- typescript
|
||||
constraints:
|
||||
- schema-first
|
||||
- type-safe
|
||||
- migration-ready
|
||||
- mock-data-only
|
||||
territory:
|
||||
- "packages/types/**"
|
||||
- "packages/mock-data/**"
|
||||
```
|
||||
|
||||
### backend-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Reactivated for v0.2 — owns apps/ai-service infrastructure: FastAPI app, settings, SSE plumbing, endpoints, monorepo/turbo integration, test harness.
|
||||
domain: backend
|
||||
frameworks:
|
||||
- fastapi
|
||||
- uvicorn
|
||||
- pydantic
|
||||
- httpx
|
||||
- pytest
|
||||
constraints:
|
||||
- provider-agnostic-boundaries (llm/ imports nothing from agents/ or api/)
|
||||
- streaming-first
|
||||
- no-database-v0.2
|
||||
- secrets-via-env-only
|
||||
- mock-provider-in-tests
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/main.py"
|
||||
- "apps/ai-service/ai_service/config.py"
|
||||
- "apps/ai-service/ai_service/api/**"
|
||||
- "apps/ai-service/scripts/**"
|
||||
- "apps/ai-service/package.json"
|
||||
- "apps/ai-service/tests/api/**"
|
||||
- "turbo.json"
|
||||
```
|
||||
|
||||
### ai-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Custom persona for v0.2 — owns the LLM provider layer, agent framework, prompt library, structured outputs, and mock corpora for the six tutor agents.
|
||||
domain: ai
|
||||
frameworks:
|
||||
- pydantic
|
||||
- httpx
|
||||
- pytest
|
||||
constraints:
|
||||
- provider-agnostic-protocol
|
||||
- prompts-are-code
|
||||
- json-defensive-parsing
|
||||
- never-call-cloud-in-tests
|
||||
- delta-passthrough
|
||||
territory:
|
||||
- "apps/ai-service/ai_service/llm/**"
|
||||
- "apps/ai-service/ai_service/agents/**"
|
||||
- "apps/ai-service/ai_service/prompts/**"
|
||||
- "apps/ai-service/ai_service/corpus/**"
|
||||
- "apps/ai-service/tests/llm/**"
|
||||
- "apps/ai-service/tests/agents/**"
|
||||
```
|
||||
|
||||
### design-system-engineer
|
||||
```yaml
|
||||
active: true
|
||||
phase_specific: false
|
||||
reason: Owns the shared component library, design tokens, and visual consistency. Light duty in v0.2: Phase 6 may need new primitives (agent-switcher control, stream-status indicator, error toast variant).
|
||||
domain: frontend
|
||||
frameworks:
|
||||
- tailwindcss
|
||||
- storybook
|
||||
- lucide-react
|
||||
constraints:
|
||||
- design-token-driven
|
||||
- wcag-aa-contrast
|
||||
- dark-mode-required
|
||||
- consistent-across-surfaces
|
||||
territory:
|
||||
- "packages/ui/**"
|
||||
```
|
||||
|
||||
### security-auditor
|
||||
```yaml
|
||||
active: false
|
||||
phase_specific: false
|
||||
reason: No auth in v0.2 (A-008); CORS is localhost-only; no real user data. Security review handled by verifier's STRIDE analysis layer plus a Phase 7 checklist item: secrets hygiene (key absent from code/logs/commits/errors), localhost-only CORS, no PII in prompts.
|
||||
domain: security
|
||||
frameworks: []
|
||||
constraints: []
|
||||
territory: []
|
||||
```
|
||||
|
||||
## Phase-Specific Personas
|
||||
|
||||
None for v0.2. All active personas span the entire milestone. data-engineer and design-system-engineer are light-touch after Phase 1.
|
||||
|
||||
## Territory Conflict Resolution
|
||||
|
||||
| Conflict | Resolution |
|
||||
|----------|------------|
|
||||
| frontend-engineer vs data-engineer (packages/types, packages/mock-data) | data-engineer owns type definitions and mock data schema; frontend-engineer consumes them. If changes needed, data-engineer updates types first. |
|
||||
| frontend-engineer vs design-system-engineer (packages/ui) | design-system-engineer owns design tokens and primitive components; frontend-engineer owns composite components and page-level UI. |
|
||||
| ai-engineer vs data-engineer (mock data duplication) | ai-engineer owns `ai_service/corpus/` (Python); data-engineer owns `packages/mock-data` (TS). Shared entity IDs and shapes kept aligned by documented convention (D-021): cross-referencing file headers, identical `comp-*` ID strings. |
|
||||
| backend-engineer vs ai-engineer (apps/ai-service) | backend-engineer owns app shell, config, API endpoints, scripts, and test harness; ai-engineer owns llm/, agents/, prompts/, corpus/. Boundary: `ai_service/api/` (backend) composes `ai_service/agents/` (AI) via DI — agents never import api/. |
|
||||
| lead-developer vs any | lead-developer coordinates only, does not directly modify code files. |
|
||||
@@ -0,0 +1,422 @@
|
||||
# Nextcraft v0.2 — PLAN.md
|
||||
|
||||
## Overview
|
||||
|
||||
This plan covers execution phases 1-6 of milestone v0.2 (AI Tutor Architecture): the six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface with streaming responses. Phases are strictly sequential (P1→P6); within each phase, Wave 1 tasks are parallelizable (no cross-file dependencies) and later waves depend on earlier ones.
|
||||
|
||||
**Environment facts (apply throughout):** Python via `python3 -m venv` (no system pip, no uv); pnpm 12.3.4 via corepack; ai-service port **8420**; default model `gemma4:31b` (config via `AI_TUTOR_MODEL`); ollama-cloud base `https://ollama.com/v1` (OpenAI-compatible, Bearer auth); API keys live only in gitignored `.ciagent/.env.secrets`, exported by `scripts/dev.sh` — never in code, commits, or logs; all automated tests use the deterministic mock provider and **never call the cloud**.
|
||||
|
||||
| Phase | Name | Requirements | Waves | Personas |
|
||||
|-------|------|-------------|-------|----------|
|
||||
| 1 | AI service scaffolding | REQ-2-001, 002, 003 | 3 | backend-engineer, ai-engineer |
|
||||
| 2 | Agent framework | REQ-2-004 | 2 | ai-engineer, backend-engineer |
|
||||
| 3 | Coach + Tutor agents | REQ-2-005, 006 | 3 | ai-engineer, backend-engineer |
|
||||
| 4 | Lab + Assessor agents | REQ-2-007, 008 | 4 | ai-engineer, backend-engineer, data-engineer |
|
||||
| 5 | Proctor + Mentor agents | REQ-2-009, 010 | 3 | ai-engineer, backend-engineer |
|
||||
| 6 | Learner surface integration | REQ-2-011, 012 | 3 | frontend-engineer, design-system-engineer |
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: AI Service Scaffolding
|
||||
|
||||
**Requirements:** REQ-2-001, REQ-2-002, REQ-2-003
|
||||
**Goal:** apps/ai-service runs under uvicorn, /health responds, provider-agnostic LLM layer with ollama-cloud/local/mock providers, SSE chat streaming verified, pytest suite green with mock provider, turbo integration wired
|
||||
|
||||
### Wave 1: Service shell + LLM core (parallel — no shared files)
|
||||
|
||||
#### Task 1-1-01: FastAPI app scaffolding
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-001
|
||||
- **Files:** `apps/ai-service/pyproject.toml`, `apps/ai-service/ai_service/main.py`, `apps/ai-service/ai_service/config.py`, `apps/ai-service/ai_service/__init__.py`, `apps/ai-service/tests/conftest.py`, `apps/ai-service/tests/test_health.py`, `apps/ai-service/.env.example`, `apps/ai-service/README.md`
|
||||
- **Action:** pydantic-settings `Settings` (env_prefix `AI_`, env_file, `SecretStr` key, port 8420, provider select, `AI_TUTOR_MODEL` default `gemma4:31b`). FastAPI app factory in `main.py` with lifespan stub (httpx client pool comes in Wave 2), CORS localhost-only (A-008), `GET /health`. pyproject with pinned deps (fastapi, uvicorn, pydantic, pydantic-settings, httpx, sse-starlette, pytest, pytest-asyncio) and dev extra. conftest: settings override + TestClient fixture. `.env.example` documents all `AI_*` vars; README documents venv setup and dev workflow.
|
||||
- **Verify:** `scripts/bootstrap.sh && scripts/test.sh` — test_health passes; `curl localhost:8420/health` returns 200
|
||||
|
||||
#### Task 1-1-02: LLM types, protocol, mock provider
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-002
|
||||
- **Files:** `apps/ai-service/ai_service/llm/types.py`, `apps/ai-service/ai_service/llm/base.py`, `apps/ai-service/ai_service/llm/mock.py`, `apps/ai-service/ai_service/llm/__init__.py`
|
||||
- **Action:** `types.py`: pydantic `Message` (role/content), `ChatDelta` (OpenAI-compatible chunk shape). `base.py`: `LLMProvider` protocol — async `stream_chat(messages, model, response_format=None) -> AsyncIterator[ChatDelta]`; the provider is a dumb pipe, no envelope logic (D-016 keeps envelope in API layer). `mock.py`: deterministic scripted provider (hash-seeded token streams, scripted failure modes: connect error, mid-stream error, malformed JSON) for tests and CI.
|
||||
- **Verify:** mock provider importable and deterministic; two identical calls yield identical streams
|
||||
|
||||
#### Task 1-1-03: Monorepo integration (shim + turbo + scripts)
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-001
|
||||
- **Files:** `apps/ai-service/package.json`, `apps/ai-service/scripts/bootstrap.sh`, `apps/ai-service/scripts/dev.sh`, `apps/ai-service/scripts/test.sh`, `turbo.json` (update), `package.json` (root, update)
|
||||
- **Action:** zero-dependency shim `package.json` in apps/ai-service with `dev`/`test`/`bootstrap` script entries. Turbo passthrough tasks `ai#dev`, `ai#test`, `ai#bootstrap` (`cache: false`, `outputs: []`). Root scripts `ai:dev`, `ai:test`, `ai:bootstrap`. `bootstrap.sh`: idempotent `python3 -m venv .venv` + pip install. `dev.sh`: exports keys from `.ciagent/.env.secrets` → uvicorn on 8420. `test.sh`: pytest via venv.
|
||||
- **Verify:** `corepack pnpm install && pnpm ai:bootstrap && pnpm ai:test` runs pytest through turbo; re-running bootstrap is a no-op
|
||||
|
||||
#### Task 1-1-04: Python lint (ruff, check-only)
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-001
|
||||
- **Files:** `apps/ai-service/pyproject.toml` (update: `[tool.ruff]` config + `ruff` in dev extra), `apps/ai-service/scripts/lint.sh`, `package.json` (root, update), `turbo.json` (update)
|
||||
- **Action:** Add `ruff` (check-only, no formatter) to the dev extra; `[tool.ruff]` with line-length 100, target py311. `scripts/lint.sh`: `.venv/bin/ruff check .` with repo-root path resolution via script-relative dirname (not CWD). Root script `ai:lint`, turbo passthrough `ai#lint` (`cache: false`, `outputs: []`). Run over the entire ai-service tree; fix all findings before phase ship (G-3).
|
||||
- **Verify:** `pnpm ai:lint` exits 0 on the Phase 1 codebase
|
||||
|
||||
### Wave 2: Real providers + SSE endpoint (depends on Wave 1)
|
||||
|
||||
#### Task 1-2-01: OpenAI-compatible provider + factory
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-002
|
||||
- **Files:** `apps/ai-service/ai_service/llm/openai_compat.py`, `apps/ai-service/ai_service/llm/factory.py`
|
||||
- **Action:** `openai_compat.py`: single `OpenAICompatProvider` for ollama-cloud (`https://ollama.com/v1`, Bearer) and local endpoints (base URL from settings); raw httpx against `/v1/chat/completions` with `stream: true`, byte-identical delta passthrough, tolerant of ollama-cloud quirks. Uses the lifespan-managed `httpx.AsyncClient` (10s connect / 300s read, D-017) — no openai SDK. `factory.py`: select provider from settings (`ollama-cloud` | `local` | `mock`).
|
||||
- **Verify:** provider constructs from settings for all 3 names; manual probe against ollama-cloud streams tokens (documented in README, not a test)
|
||||
|
||||
#### Task 1-2-02: Lifespan wiring + SSE chat endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-003
|
||||
- **Files:** `apps/ai-service/ai_service/main.py` (update), `apps/ai-service/ai_service/api/deps.py`, `apps/ai-service/ai_service/api/chat.py`, `apps/ai-service/ai_service/api/__init__.py`
|
||||
- **Action:** Lifespan creates the shared `httpx.AsyncClient` and provider factory; deps.py provides provider via DI. `POST /v1/chat/stream` in chat.py implements the D-016 envelope: `meta` event (agent/session/model) flushed before first token → raw OpenAI chunks passed through as `data: {json}` → `done` event → `error` event before `[DONE]` on mid-stream failure; pre-first-byte failures return proper HTTP status codes. Headers `Cache-Control: no-cache`, `X-Accel-Buffering: no`; sse-starlette ping keep-alive.
|
||||
- **Verify:** `curl -N -X POST localhost:8420/v1/chat/stream` with mock provider shows meta event, token deltas, done, `[DONE]`
|
||||
|
||||
### Wave 3: Provider + endpoint test suites (depends on Wave 2)
|
||||
|
||||
#### Task 1-3-01: LLM provider tests (byte-exact, no cloud)
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-002
|
||||
- **Files:** `apps/ai-service/tests/llm/test_openai_compat.py`, `apps/ai-service/tests/llm/test_mock.py`, `apps/ai-service/tests/llm/__init__.py`
|
||||
- **Action:** httpx `MockTransport` tests parsing byte-exact fixture streams (happy path, empty delta, `[DONE]`, malformed line, mid-stream disconnect). Mock provider tests: determinism, scripted failure modes, response_format echo.
|
||||
- **Verify:** `pnpm ai:test` — llm suite green; zero network calls in tests
|
||||
|
||||
#### Task 1-3-02: SSE stream endpoint tests
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-003
|
||||
- **Files:** `apps/ai-service/tests/api/test_chat_stream.py`, `apps/ai-service/tests/api/__init__.py`
|
||||
- **Action:** TestClient `client.stream()` tests: meta-first ordering, delta passthrough, done + `[DONE]` sentinel, mid-stream error event, pre-first-byte failure → HTTP status, required headers. pytest-asyncio auto mode (D-023).
|
||||
- **Verify:** `pnpm ai:test` — api suite green
|
||||
|
||||
### Must-Haves (Phase 1)
|
||||
- [ ] `scripts/bootstrap.sh` is idempotent; creates venv + installs deps without system pip
|
||||
- [ ] `pnpm ai:dev` starts uvicorn; `curl localhost:8420/health` returns 200
|
||||
- [ ] `pnpm ai:lint` exits 0 (ruff check over the ai-service tree) (G-3)
|
||||
- [ ] `pnpm ai:test` runs the full pytest suite via turbo and passes (mock provider only — no network)
|
||||
- [ ] SSE stream delivers tokens: meta event, incremental deltas, done, `[DONE]` observed via `curl -N`
|
||||
- [ ] Mid-stream failure emits `error` event before `[DONE]`; pre-first-byte failure returns HTTP error status
|
||||
- [ ] Provider factory resolves ollama-cloud / local / mock from settings; manual ollama-cloud probe documented in README
|
||||
- [ ] `llm/` imports nothing from `agents/` or `api/` (boundary rule holds)
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Agent Framework
|
||||
|
||||
**Requirements:** REQ-2-004
|
||||
**Goal:** Shared framework all six agents use: BaseAgent contract, session store, prompt library, registry, structured outputs — all tested against the mock provider
|
||||
|
||||
### Wave 1: Framework primitives (parallel — no shared files)
|
||||
|
||||
#### Task 2-1-01: BaseAgent ABC
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-004
|
||||
- **Files:** `apps/ai-service/ai_service/agents/base.py`, `apps/ai-service/tests/agents/test_base.py`, `apps/ai-service/ai_service/agents/__init__.py`, `apps/ai-service/tests/agents/__init__.py`
|
||||
- **Action:** `BaseAgent` ABC (D-018): `name`, `system_prompt`, `build_messages(history, learner_context)`, `stream_reply(...) -> AsyncIterator[ChatDelta]` (delegates to provider), `structured_reply(...)` (delegates to structured module, landed Wave 2). Subclass contract tested with a stub agent + mock provider.
|
||||
- **Verify:** `pnpm ai:test` — test_base green
|
||||
|
||||
#### Task 2-1-02: SessionStore protocol + in-memory implementation
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-004
|
||||
- **Files:** `apps/ai-service/ai_service/agents/session.py`, `apps/ai-service/tests/test_session.py`
|
||||
- **Action:** `SessionStore` protocol + `InMemorySessionStore` (D-019): asyncio.Lock-guarded dict, agent-scoped session keys, 20-message rolling window, 500-cap LRU eviction. Protocol shape is DB-migration-ready (A-003).
|
||||
- **Verify:** test_session covers create/append/window-trim/LRU-eviction/agent scoping
|
||||
|
||||
#### Task 2-1-03: Prompt library scaffolding
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-004
|
||||
- **Files:** `apps/ai-service/ai_service/prompts/coach.py`, `apps/ai-service/ai_service/prompts/tutor.py`, `apps/ai-service/ai_service/prompts/lab.py`, `apps/ai-service/ai_service/prompts/assessor.py`, `apps/ai-service/ai_service/prompts/proctor.py`, `apps/ai-service/ai_service/prompts/mentor.py`, `apps/ai-service/ai_service/prompts/__init__.py`
|
||||
- **Action:** Per-agent module with a versioned `SYSTEM_PROMPT` constant + `render_context(learner_context) -> dict` using `str.format_map` for learner-context injection (D-018: prompts are code, versioned in git). Initial drafts for all six; final personas land in Phases 3-5.
|
||||
- **Verify:** all six prompt modules import; render_context fills placeholders without KeyError
|
||||
|
||||
#### Task 2-1-04: Learner context corpus
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-004
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/learner_context.py`, `apps/ai-service/ai_service/corpus/__init__.py`
|
||||
- **Action:** Pydantic-typed learner context (active stack, competencies, progress, recent artifacts) mirroring TS `packages/mock-data` IDs per D-021 convention (cross-referencing header comment, identical `comp-*`/`stack-*` ID strings).
|
||||
- **Verify:** context renders into prompt placeholders; IDs match packages/mock-data strings
|
||||
|
||||
### Wave 2: Composition layers (depends on Wave 1)
|
||||
|
||||
#### Task 2-2-01: Structured output defense
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-004
|
||||
- **Files:** `apps/ai-service/ai_service/agents/structured.py`, `apps/ai-service/tests/test_structured.py`
|
||||
- **Action:** 4-layer defense (D-020): (1) `response_format` request with auto-degrade on provider 400; (2) prompt-embedded JSON schema; (3) parse: strip code fences → first balanced JSON object; (4) single bounded retry with validation-error feedback. Returns pydantic-validated model or raises `StructuredOutputError`.
|
||||
- **Verify:** test_structured covers fenced/unfenced/invalid JSON, retry path, degrade path — all against mock provider
|
||||
|
||||
#### Task 2-2-02: Agent registry
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-004
|
||||
- **Files:** `apps/ai-service/ai_service/agents/registry.py`, `apps/ai-service/tests/test_registry.py`
|
||||
- **Action:** Explicit registry: name → agent factory map with `register(name, factory)` / `get(name)`; raises on unknown agent. Agents are registered in their own phases (P3-P5).
|
||||
- **Verify:** test_registry: register/get round-trip, unknown-agent error, duplicate registration error
|
||||
|
||||
#### Task 2-2-03: Session + agent DI wiring into API layer
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-004
|
||||
- **Files:** `apps/ai-service/ai_service/api/deps.py` (update), `apps/ai-service/ai_service/api/chat.py` (update)
|
||||
- **Action:** deps.py exposes SessionStore and provider singletons via DI. chat.py persists turn history through the session store (agent-scoped) and includes session ID in the meta event. API composes agents via DI — agents never import api/.
|
||||
- **Verify:** chat request appends to and replays windowed history; `pnpm ai:test` green
|
||||
|
||||
### Must-Haves (Phase 2)
|
||||
- [ ] BaseAgent unit tests pass (stub agent streams via mock provider)
|
||||
- [ ] Session store tested: create/append, 20-message window trim, 500-cap LRU eviction, agent-scoped keys
|
||||
- [ ] Structured output parsing tested against mock provider: fence-strip, first-balanced-object, invalid JSON, one bounded retry, response_format auto-degrade
|
||||
- [ ] Registry tested: register/get/unknown/duplicate
|
||||
- [ ] Six prompt modules render learner context without errors
|
||||
- [ ] Module boundaries hold: `agents/` never imports `api/`; `llm/` never imports `agents/` or `api/`
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Coach + Tutor Agents
|
||||
|
||||
**Requirements:** REQ-2-005, REQ-2-006
|
||||
**Goal:** Both learner-facing conversational agents fully implemented with distinct personas, registered, routed through the chat streaming endpoint
|
||||
|
||||
### Wave 1: Agent implementations (parallel — no shared files)
|
||||
|
||||
#### Task 3-1-01: Coach agent
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-005
|
||||
- **Files:** `apps/ai-service/ai_service/prompts/coach.py` (finalize), `apps/ai-service/ai_service/agents/coach.py`, `apps/ai-service/tests/test_coach.py`
|
||||
- **Action:** Final Coach persona: pacing guidance, motivation, retrieval practice prompts; system prompt injects learner context (active stack, progress). `CoachAgent(BaseAgent)` streams replies. Mock provider scripts a distinct coach-voice response for tests.
|
||||
- **Verify:** test_coach: build_messages includes system prompt + windowed history; stream_reply yields deltas; on-persona content asserted against mock script
|
||||
|
||||
#### Task 3-1-02: Tutor agent
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-006
|
||||
- **Files:** `apps/ai-service/ai_service/prompts/tutor.py` (finalize), `apps/ai-service/ai_service/agents/tutor.py`, `apps/ai-service/tests/test_tutor.py`
|
||||
- **Action:** Final Tutor persona: concept delivery, Socratic questioning, worked examples. `TutorAgent(BaseAgent)` streams replies; mock scripts a distinct tutor-voice response.
|
||||
- **Verify:** test_tutor mirrors test_coach; Coach and Tutor mock outputs are observably distinct
|
||||
|
||||
### Wave 2: Registration + persona verification (depends on Wave 1)
|
||||
|
||||
#### Task 3-2-01: Register Coach + Tutor; document cloud probe
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-005, REQ-2-006
|
||||
- **Files:** `apps/ai-service/ai_service/agents/registry.py` (update), `apps/ai-service/tests/test_registry.py` (update), `apps/ai-service/README.md` (update)
|
||||
- **Action:** Register both agents in the explicit registry. Extend test_registry to assert both resolve. Document the manual ollama-cloud persona probe in README (curl commands with `AI_PROVIDER=ollama-cloud`): Coach and Tutor produce distinct on-persona responses; tests remain cloud-free.
|
||||
- **Verify:** `pnpm ai:test` green; manual probe against ollama-cloud shows distinct personas (documented, not automated)
|
||||
|
||||
### Wave 3: Chat endpoint agent routing (depends on Wave 2)
|
||||
|
||||
#### Task 3-3-01: Agent routing on /v1/chat/stream
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-005, REQ-2-006
|
||||
- **Files:** `apps/ai-service/ai_service/api/chat.py` (update), `apps/ai-service/ai_service/api/deps.py` (update), `apps/ai-service/tests/api/test_chat_stream.py` (update)
|
||||
- **Action:** Chat request gains `agent` field (validated against the registry; unknown agent → 422). Endpoint resolves the agent via DI, persists to the agent-scoped session, meta event carries the agent name. No autonomous routing in v0.2 (A-007).
|
||||
- **Verify:** TestClient tests: `agent=coach` and `agent=tutor` route correctly, session scoped per agent, unknown agent rejected
|
||||
|
||||
### Must-Haves (Phase 3)
|
||||
- [ ] Both agents produce distinct, on-persona responses (mock-asserted; manual ollama-cloud probe documented in README)
|
||||
- [ ] Agent routing tested: coach/tutor resolve via registry; unknown agent returns 422
|
||||
- [ ] Both agents exposed end-to-end via `POST /v1/chat/stream` with agent-scoped session history
|
||||
- [ ] `pnpm ai:test` green; no cloud calls in tests
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Lab + Assessor Agents
|
||||
|
||||
**Requirements:** REQ-2-007, REQ-2-008
|
||||
**Goal:** Lab consumes simulated sandbox telemetry and streams in-flow feedback; Assessor applies rubrics to pre-baked artifacts and returns structured scores — both over mock engine inputs
|
||||
|
||||
### Wave 1: Mock engine inputs (parallel — no shared files)
|
||||
|
||||
#### Task 4-1-01: Simulated sandbox telemetry corpus
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-007
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/telemetry.py`
|
||||
- **Action:** Pydantic-typed Lab telemetry scenarios: scripted build-session event streams (keystrokes, commits, test runs, errors, idle gaps) keyed by scenario ID, aligned with packages/mock-data IDs (D-021).
|
||||
- **Verify:** scenarios import, validate, and are addressable by ID
|
||||
|
||||
#### Task 4-1-02: Pre-baked artifacts + rubrics corpus
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-008
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/artifacts.py`
|
||||
- **Action:** Pre-baked artifacts (code, design, simulation), assessment rubrics (criteria, levels, weights), and defense transcripts keyed by ID — the Assessor's mock inputs (real engines are v0.3+).
|
||||
- **Verify:** rubric/artifact/transcript fixtures validate; IDs align with TS mock data
|
||||
|
||||
#### Task 4-1-03: TS mock-data alignment for engine inputs
|
||||
- **Persona:** data-engineer — **REQ:** REQ-2-012
|
||||
- **Files:** `packages/mock-data/ai-scenarios.ts` (new), `packages/mock-data/index.ts` (update)
|
||||
- **Action:** Export scenario/artifact ID constants + display metadata used by the Phase 6 learner panels, mirroring `ai_service/corpus/` IDs exactly (D-021). Data-engineer owns the TS side; headers cross-reference the Python corpus.
|
||||
- **Verify:** `pnpm typecheck` passes; IDs string-equal to corpus IDs
|
||||
|
||||
### Wave 2: Agent implementations (depends on Wave 1)
|
||||
|
||||
#### Task 4-2-01: Lab agent
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-007
|
||||
- **Files:** `apps/ai-service/ai_service/prompts/lab.py` (finalize), `apps/ai-service/ai_service/agents/lab.py`, `apps/ai-service/tests/test_lab.py`
|
||||
- **Action:** `LabAgent(BaseAgent)` consumes a telemetry scenario, builds messages summarizing the event stream, streams concrete in-flow feedback (what happened, what to adjust, next step). No session chat — scenario-driven.
|
||||
- **Verify:** test_lab: given a mock scenario, feedback references scenario events (mock-scripted assertions)
|
||||
|
||||
#### Task 4-2-02: Assessor agent
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-008
|
||||
- **Files:** `apps/ai-service/ai_service/prompts/assessor.py` (finalize), `apps/ai-service/ai_service/agents/assessor.py`, `apps/ai-service/tests/test_assessor.py`
|
||||
- **Action:** `AssessorAgent(BaseAgent)` applies a rubric to an artifact + defense transcript via `structured_reply`, returning a pydantic-validated rubric score model (per-criterion scores, strengths, gaps, verdict).
|
||||
- **Verify:** test_assessor: structured output validates against the rubric model; failure modes exercise the 4-layer defense
|
||||
|
||||
### Wave 3: Registration (depends on Wave 2)
|
||||
|
||||
#### Task 4-3-01: Register Lab + Assessor
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-007, REQ-2-008
|
||||
- **Files:** `apps/ai-service/ai_service/agents/registry.py` (update), `apps/ai-service/tests/test_registry.py` (update)
|
||||
- **Action:** Register both agents; extend registry tests.
|
||||
- **Verify:** registry resolves coach/tutor/lab/assessor; `pnpm ai:test` green
|
||||
|
||||
### Wave 4: Endpoints (depends on Wave 3)
|
||||
|
||||
#### Task 4-4-01: Lab feedback endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-007
|
||||
- **Files:** `apps/ai-service/ai_service/api/lab.py`, `apps/ai-service/tests/api/test_lab.py`
|
||||
- **Action:** `POST /v1/lab/feedback` with scenario ID → resolves corpus scenario + Lab agent → SSE stream using the D-016 envelope (meta names agent=lab). Unknown scenario → 404.
|
||||
- **Verify:** TestClient streams meta + deltas + done + `[DONE]`; unknown scenario 404
|
||||
|
||||
#### Task 4-4-02: Assessment evaluate endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-008
|
||||
- **Files:** `apps/ai-service/ai_service/api/assessment.py`, `apps/ai-service/tests/api/test_assessment.py`
|
||||
- **Action:** `POST /v1/assessment/evaluate` with artifact ID → resolves corpus artifact/rubric/transcript + Assessor agent → JSON response (validated rubric score model). Unknown artifact → 404.
|
||||
- **Verify:** TestClient returns validated rubric JSON; unknown artifact 404; `pnpm ai:test` green
|
||||
|
||||
### Must-Haves (Phase 4)
|
||||
- [ ] Lab produces scenario-relevant in-flow feedback for mock telemetry scenarios (mock provider, tested)
|
||||
- [ ] Assessor returns structured rubric scores (pydantic-validated JSON) for pre-baked artifacts/transcripts
|
||||
- [ ] `POST /v1/lab/feedback` streams (meta → deltas → done → `[DONE]`); `POST /v1/assessment/evaluate` returns validated JSON
|
||||
- [ ] Unknown scenario/artifact IDs return 404
|
||||
- [ ] Corpus IDs align with packages/mock-data (D-021); `pnpm ai:test` and `pnpm typecheck` green
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Proctor + Mentor Agents
|
||||
|
||||
**Requirements:** REQ-2-009, REQ-2-010
|
||||
**Goal:** Proctor classifies integrity signals with coaching interventions from mock telemetry; Mentor generates long-horizon career narrative; both exposed via endpoints
|
||||
|
||||
### Wave 1: Proctor scenarios + Mentor agent (parallel — no shared files)
|
||||
|
||||
#### Task 5-1-01: Proctor telemetry scenarios
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-009
|
||||
- **Files:** `apps/ai-service/ai_service/corpus/telemetry.py` (update)
|
||||
- **Action:** Add proctor scenarios: tab switches, idle time, paste events, focus loss — scripted integrity-relevant event sets keyed by scenario ID.
|
||||
- **Verify:** proctor scenarios validate; distinguishable from lab scenarios by type
|
||||
|
||||
#### Task 5-1-02: Mentor agent (+ registration)
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-010
|
||||
- **Files:** `apps/ai-service/ai_service/prompts/mentor.py` (finalize), `apps/ai-service/ai_service/agents/mentor.py`, `apps/ai-service/tests/test_mentor.py`, `apps/ai-service/ai_service/agents/registry.py` (update)
|
||||
- **Action:** `MentorAgent(BaseAgent)`: long-horizon career narrative — trajectory story, competency-stack progression guidance, market positioning — streaming, session-backed. Registered centrally in `registry.py` (single registration pattern, G-4).
|
||||
- **Verify:** test_mentor: narrative references learner context (mock-scripted); registry resolves mentor
|
||||
|
||||
### Wave 2: Proctor agent + Mentor endpoint (depends on Wave 1)
|
||||
|
||||
#### Task 5-2-01: Proctor agent (+ registration)
|
||||
- **Persona:** ai-engineer — **REQ:** REQ-2-009
|
||||
- **Files:** `apps/ai-service/ai_service/prompts/proctor.py` (finalize), `apps/ai-service/ai_service/agents/proctor.py`, `apps/ai-service/tests/test_proctor.py`, `apps/ai-service/ai_service/agents/registry.py` (update)
|
||||
- **Action:** `ProctorAgent(BaseAgent)`: consumes proctor scenario → `structured_reply` returns pydantic-validated signal classification (severity, signal type) + recommended coaching intervention (supportive, not punitive). Registered centrally in `registry.py` (single registration pattern, G-4).
|
||||
- **Verify:** test_proctor: classified signals + interventions validate for each mock scenario; registry resolves all six agents
|
||||
|
||||
#### Task 5-2-02: Mentor narrative endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-010
|
||||
- **Files:** `apps/ai-service/ai_service/api/mentor.py`, `apps/ai-service/tests/api/test_mentor.py`
|
||||
- **Action:** `POST /v1/mentor/narrative` → Mentor agent → SSE stream with D-016 envelope, session-backed.
|
||||
- **Verify:** TestClient streams meta (agent=mentor) → deltas → done → `[DONE]`
|
||||
|
||||
### Wave 3: Proctor endpoint (depends on Wave 2)
|
||||
|
||||
#### Task 5-3-01: Proctor signals endpoint
|
||||
- **Persona:** backend-engineer — **REQ:** REQ-2-009
|
||||
- **Files:** `apps/ai-service/ai_service/api/proctor.py`, `apps/ai-service/tests/api/test_proctor.py`
|
||||
- **Action:** `POST /v1/proctor/signals` with scenario ID → resolves corpus scenario + Proctor agent → JSON response (validated signals + interventions). Unknown scenario → 404.
|
||||
- **Verify:** TestClient returns classified signals JSON; `pnpm ai:test` green — full suite (all six agents registered)
|
||||
|
||||
### Must-Haves (Phase 5)
|
||||
- [ ] Proctor produces classified signals with recommended coaching interventions for each mock scenario (structured JSON, validated)
|
||||
- [ ] Mentor produces coherent long-horizon career narrative (streaming, session-backed)
|
||||
- [ ] `POST /v1/proctor/signals` returns validated JSON; `POST /v1/mentor/narrative` streams
|
||||
- [ ] Registry resolves all six agents; full ai-service test suite green, cloud-free
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Learner Surface Integration
|
||||
|
||||
**Requirements:** REQ-2-011, REQ-2-012
|
||||
**Goal:** v0.1 learner surfaces wired to the real ai-service: streaming chat with agent switcher, Lab/Assessor/Proctor/Mentor outputs surfaced, error/loading states, build + typecheck green
|
||||
**Note (G-2):** End-to-end verification may run with `AI_PROVIDER=mock` as a fallback — the requirement is the real ai-service over HTTP (not canned client-side responses); provider choice is service-internal. This prevents an ollama-cloud outage from blocking P6 verification. Cloud persona probes remain separate (Task 3-2-01).
|
||||
|
||||
### Wave 1: Client plumbing + primitives (parallel — no shared files)
|
||||
|
||||
#### Task 6-1-01: useChatStream hook
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-2-011
|
||||
- **Files:** `apps/web/hooks/use-chat-stream.ts`, `apps/web/.env.example` (update)
|
||||
- **Action:** `useChatStream(agent)` hook: `fetch` POST to `${NEXT_PUBLIC_AI_SERVICE_URL}/v1/chat/stream` (default `http://localhost:8420`, A-002 — no API-route proxy); consumes `ReadableStream` with byte buffering, frame split on `\n\n`, joined `data:` lines; **ignores frames containing no `data:` lines (sse-starlette `: ping` keep-alive comment frames)** — TestClient streams are too short to surface pings, but real cloud delta gaps emit them (G-1); handles meta / delta / done / error events and `[DONE]` sentinel; idempotent `AbortController.abort()` in effect cleanup; exposes `{messages, isStreaming, error, send, retry, abort}`. `.env.example` gains `NEXT_PUBLIC_AI_SERVICE_URL`.
|
||||
- **Verify:** hook unit-tested or exercised via the chat UI; unmount mid-stream aborts cleanly (no state updates after unmount)
|
||||
|
||||
#### Task 6-1-02: Agent switcher + streaming primitives
|
||||
- **Persona:** design-system-engineer — **REQ:** REQ-2-011
|
||||
- **Files:** `packages/ui/src/primitives/agent-switcher.tsx`, `packages/ui/src/primitives/stream-status.tsx`, `packages/ui/src/primitives/toast.tsx`, `packages/ui/src/primitives/index.ts` (update), `packages/ui/src/index.ts` (update)
|
||||
- **Action:** Token-driven primitives: AgentSwitcher (segmented coach/tutor control with active state), StreamStatus (idle/streaming/error indicator), Toast with error variant. Dark mode + WCAG AA contrast; exported from `@nextcraft/ui`.
|
||||
- **Verify:** primitives import from `@nextcraft/ui`; storybook stories render (dark + light)
|
||||
|
||||
### Wave 2: Chat rewrite + Byte viewer panel (depends on Wave 1)
|
||||
|
||||
#### Task 6-2-01: Real streaming learner chat
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-2-011
|
||||
- **Files:** `apps/web/components/learner/ai-tutor-chat.tsx` (rewrite), `apps/web/components/learner/agent-switcher.tsx` (composition wrapper)
|
||||
- **Action:** Replace the canned `aiTutorResponses` behavior with `useChatStream`: agent switcher (Coach/Tutor per A-007), token-by-token rendering, streaming cursor + loading state, error state with retry button when ai-service is down (A-010), suggested-action chips from the meta event. Seed welcome message stays static.
|
||||
- **Verify:** with ai-service running, messages stream visibly token-by-token; with ai-service stopped, error state + retry appears (no crash, no console errors)
|
||||
|
||||
#### Task 6-2-02: Byte viewer Tutor panel
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-2-011 (agent routing: byte viewer always uses Tutor, A-007)
|
||||
- **Files:** `apps/web/app/(learner)/learn/[competencyId]/page.tsx` (update), `apps/web/components/learner/byte-tutor-panel.tsx` (new)
|
||||
- **Action:** Byte viewer gains a Tutor explanation panel (byte viewer always uses Tutor, A-007): "Explain this byte" streams a Socratic concept walkthrough for the current competency via useChatStream (agent fixed to tutor).
|
||||
- **Verify:** on a byte page, the panel streams a Tutor explanation; error state when service down
|
||||
|
||||
### Wave 3: Lab / Assessment / Mentor panels (depends on Wave 1 hook; parallel — no shared files)
|
||||
|
||||
#### Task 6-3-01: Sandbox Lab feedback panel
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-2-012
|
||||
- **Files:** `apps/web/app/(learner)/build/[competencyId]/page.tsx` (update), `apps/web/components/learner/lab-feedback-panel.tsx` (new)
|
||||
- **Action:** Sandbox telemetry sidebar gains a Lab feedback panel: posts the scenario ID (from `packages/mock-data/ai-scenarios`) to `/v1/lab/feedback`, streams in-flow feedback into the panel; loading + error states.
|
||||
- **Verify:** sandbox page streams Lab feedback for the mock scenario; error state when service down
|
||||
|
||||
#### Task 6-3-02: Assessment Assessor + Proctor surfaces
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-2-012
|
||||
- **Files:** `apps/web/app/(learner)/defend/[competencyId]/page.tsx` (update), `apps/web/components/learner/assessor-results-panel.tsx` (new), `apps/web/components/learner/proctor-banner.tsx` (new)
|
||||
- **Action:** Assessment mockup: AI reviewer panel calls `/v1/assessment/evaluate` with the artifact ID and renders the structured rubric scores (per-criterion bars, strengths, gaps, verdict); a Proctor integrity banner surfaces `/v1/proctor/signals` classifications with coaching tone. Loading skeletons + error states.
|
||||
- **Verify:** defend page renders real Assessor rubric output + Proctor banner; error states when service down
|
||||
|
||||
#### Task 6-3-03: Dashboard Mentor panel
|
||||
- **Persona:** frontend-engineer — **REQ:** REQ-2-012
|
||||
- **Files:** `apps/web/app/(learner)/dashboard/page.tsx` (update), `apps/web/components/learner/mentor-panel.tsx` (new)
|
||||
- **Action:** Learner dashboard gains a Mentor panel: streams career narrative from `/v1/mentor/narrative` (learner progress context), with regenerate button, loading + error states. Sits alongside the existing AI tutor chat.
|
||||
- **Verify:** dashboard shows streaming Mentor narrative; error state when service down
|
||||
|
||||
### Must-Haves (Phase 6)
|
||||
- [ ] With ai-service running: learner chat at http://localhost:3000/dashboard streams real responses token-by-token (mock provider fallback allowed per G-2 — service over HTTP is the requirement)
|
||||
- [ ] Agent switcher flips Coach ↔ Tutor and the response persona changes accordingly
|
||||
- [ ] Hook tolerates keep-alive comment frames (`: ping`, no data lines) during live streams (G-1)
|
||||
- [ ] With ai-service stopped: all chat/panels show error states with retry — no crashes, no unhandled promise rejections, no console errors
|
||||
- [ ] Byte viewer, sandbox, and assessment mockups surface Tutor/Lab/Assessor/Proctor outputs; dashboard shows Mentor narrative
|
||||
- [ ] Unmounting/navigating mid-stream aborts cleanly (no post-unmount state updates)
|
||||
- [ ] `pnpm build` and `pnpm typecheck` pass; `pnpm ai:test` still green
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Final Review + Ship (no planned tasks)
|
||||
|
||||
Orchestrated by the SHIP stage, not this plan: multi-persona code review (correctness, testing, secrets hygiene — key absent from code/logs/commits/errors, localhost-only CORS, no PII in prompts), project health audit (reconstruction test, .ciagent/ discipline, branch/commit hygiene), then merge milestone → main, tag v0.2.0, create Gitea release, mark all 12 v0.2 requirements complete.
|
||||
|
||||
**Release-note honesty (G-5):** the v0.2.0 release note must explicitly state that Lab/Assessor/Proctor operate on mock engine inputs (real engines are v0.3+), per D-015.
|
||||
|
||||
**Dead-code disposal (G-5):** review must dispose of `aiTutorResponses` (packages/mock-data/ai-tutor-responses.ts) — its only consumer is rewritten in Task 6-2-01; remove the export or mark it deprecated.
|
||||
|
||||
---
|
||||
|
||||
## User-Facing Surface
|
||||
|
||||
The primary user-facing surface is the **learner dashboard and learning flow** at `http://localhost:3000`, backed by the real ai-service at `http://localhost:8420`:
|
||||
|
||||
- `/dashboard` — AI tutor chat (Coach/Tutor switcher, streaming) + Mentor career-narrative panel
|
||||
- `/learn/[competencyId]` — byte viewer with streaming Tutor explanations
|
||||
- `/build/[competencyId]` — sandbox with Lab in-flow feedback panel (mock telemetry)
|
||||
- `/defend/[competencyId]` — assessment with live Assessor rubric scores + Proctor integrity banner
|
||||
|
||||
The marketplace, employer, and admin surfaces are unchanged from v0.1.
|
||||
|
||||
## Happy Path
|
||||
|
||||
1. Learner opens `/dashboard` → chat shows welcome message; meta event confirms coach/model in the stream
|
||||
2. Learner types "I'm stuck on multi-agent communication" → reply streams token-by-token with pacing guidance + a retrieval-practice prompt
|
||||
3. Learner switches to **Tutor** → asks the same question → gets a Socratic concept walkthrough instead
|
||||
4. Learner opens a byte tutorial → Tutor panel streams an explanation of the current competency
|
||||
5. Learner opens the build sandbox → Lab panel streams feedback on the simulated telemetry scenario
|
||||
6. Learner opens the defense mockup → Assessor panel shows structured rubric scores; Proctor banner shows integrity signals in coaching tone
|
||||
7. Back on `/dashboard`, the Mentor panel streams a career narrative tied to the learner's progress
|
||||
8. Learner kills ai-service (or it crashes) → next message shows an inline error state with **Retry**; restarting the service and retrying resumes streaming
|
||||
|
||||
## UX Acceptance Criteria
|
||||
|
||||
1. Streaming is visibly incremental — tokens appear as they arrive, not as one blob
|
||||
2. Agent switcher shows the active agent (Coach/Tutor) and the response persona visibly changes
|
||||
3. Loading state during connection (streaming cursor / skeleton) before first token
|
||||
4. When ai-service is unreachable: inline error state + retry action on every chat/panel — no crashes, no console errors, no blank UI
|
||||
5. `[DONE]` reliably ends the stream (input re-enables, no stuck "typing" state)
|
||||
6. Navigating away mid-stream aborts cleanly — no leaked requests or post-unmount updates
|
||||
7. All new UI uses design tokens, supports dark mode, meets WCAG AA contrast
|
||||
8. Responsive at 375px, 768px, 1280px
|
||||
9. No hardcoded model names or URLs in UI code — all via `NEXT_PUBLIC_AI_SERVICE_URL` and server meta events
|
||||
10. `pnpm build` and `pnpm typecheck` pass with zero errors
|
||||
@@ -0,0 +1,124 @@
|
||||
# Nextcraft — PROJECT.md
|
||||
|
||||
## What This Is
|
||||
|
||||
Nextcraft is an AI-native outcome school where graduates prove what they can build — not what they can write. It credentials verifiable skill for a post-AI labor market, rejects legacy degree structures, and trains the workforce of tomorrow through hands-on competency stacks assessed entirely by AI tutors. The marketplace is the symbiotic second surface: employers meet AI-credentialed talent through an automated, algorithmically-matched job board that runs without human headcount.
|
||||
|
||||
**Single guiding outcome:** ≥1,000 learners placed in AI-orchestration roles via the Nextcraft marketplace within 36 months of launch, verified by CIRR-style third-party audit.
|
||||
|
||||
---
|
||||
|
||||
## Current Milestone: v0.2 — AI Tutor Architecture
|
||||
|
||||
**Scope:** The six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface chat UI with streaming responses. Provider-agnostic LLM layer (ollama-cloud default, local endpoint + deterministic mock for tests). Lab/Assessor/Proctor operate on mock engine inputs (simulated telemetry, pre-baked artifacts) — their real engines (sandbox fabric, assessment engine, identity verification) are v0.3+.
|
||||
|
||||
**Status of v0.1:** Complete and shipped (v0.1.0). Founder agreement recorded (D-013).
|
||||
|
||||
**Tech stack:** v0.1 TS monorepo (pnpm/turborepo, Next.js) + new Python FastAPI service (`apps/ai-service`) with pydantic, SSE streaming, and an OpenAI-compatible provider client.
|
||||
|
||||
---
|
||||
|
||||
## Requirements (Validated)
|
||||
|
||||
The following requirements have been validated during specification and are locked for milestone v0.2 (REQ-F-001..006 activated from the deferred pool):
|
||||
|
||||
1. AI tutor service infrastructure — `apps/ai-service` FastAPI application, provider-agnostic LLM client, SSE streaming, session/state handling
|
||||
2. Agent framework — base agent contracts, prompt management, streaming pipeline, structured outputs
|
||||
3. Coach agent — pacing, motivation, retrieval practice (REQ-F-001)
|
||||
4. Tutor agent — concept delivery, Socratic questioning (REQ-F-002)
|
||||
5. Lab agent — in-flow feedback over simulated sandbox telemetry (REQ-F-003, mock inputs)
|
||||
6. Assessor agent — rubric application to pre-baked artifacts and defenses (REQ-F-004, mock inputs)
|
||||
7. Proctor agent — integrity signals from mock telemetry, coaching interventions (REQ-F-005, mock inputs)
|
||||
8. Mentor agent — long-horizon career narrative (REQ-F-006)
|
||||
9. Learner surface integration — streaming chat UI wired to the real service, error/loading states
|
||||
|
||||
## v0.1 Requirements (Complete)
|
||||
|
||||
All 28 v0.1 requirements (REQ-001..028) are complete and shipped as v0.1.0. See REQUIREMENTS.md traceability matrix.
|
||||
|
||||
## Clarified Assumptions (CLARIFY stage, full autonomy — auto-resolved)
|
||||
|
||||
| # | Ambiguity | Resolution | Confidence |
|
||||
|---|-----------|------------|-------------|
|
||||
| A-001 | Where does the ai-service live in the monorepo? | `apps/ai-service` — pnpm-workspace ignores Python; it integrates via root package.json scripts (`ai:dev`, `ai:test`), not as a pnpm package. Turbo gets passthrough tasks. | 0.95 |
|
||||
| A-002 | How does the Next.js client talk to ai-service? | Direct fetch to `http://localhost:8420` (configurable via `NEXT_PUBLIC_AI_SERVICE_URL`) with SSE parsing. No Next.js API-route proxy in v0.2 — client components talk straight to the service. | 0.85 |
|
||||
| A-003 | Session persistence? | In-memory dict keyed by session ID (v0.2 has no DB). Sessions lost on restart — acceptable for this milestone; store interface is DB-migration-ready. | 0.9 |
|
||||
| A-004 | Port for ai-service? | 8420 (avoids common dev-port collisions with 3000/8000; documented in .env.example). | 0.8 |
|
||||
| A-005 | Which ollama-cloud model? | Default `gemma4:31b` (probe-verified); configurable via `AI_TUTOR_MODEL` env. Model choice is a config, not code. | 0.85 |
|
||||
| A-006 | Streaming format? | SSE with `data:` JSON lines (OpenAI-compatible delta objects), terminated by `data: [DONE]`. Matches the provider contract, so the provider layer passes deltas through unchanged. | 0.9 |
|
||||
| A-007 | Agent routing in the chat UI? | Explicit agent switcher (Coach/Tutor) in the learner chat; Byte viewer always uses Tutor; sandbox uses Lab; assessment uses Assessor+Proctor; dashboard Mentor panel. No autonomous routing in v0.2. | 0.9 |
|
||||
| A-008 | Auth between web and ai-service? | None in v0.2 (local dev surface). CORS limited to localhost origins. Real auth is v0.3+ with identity work. | 0.85 |
|
||||
| A-009 | Python tooling? | `python3 -m venv` + pip (venv is the only available mechanism in this environment; no uv). Pydantic v2, FastAPI, uvicorn, pytest — all PyPI-reachable (verified). | 0.9 |
|
||||
| A-010 | What happens when the LLM provider is unreachable? | Streaming endpoints return an error event; the UI shows error states with retry. Mock provider guarantees tests never call the cloud. | 0.9 |
|
||||
|
||||
## Requirements (Active — Future Milestones)
|
||||
|
||||
The following remain deferred beyond v0.2 and will be activated in subsequent milestones:
|
||||
|
||||
- Competency graph engine and adaptive pathways
|
||||
- Assessment engine (process-trace grading, oral defense, per-learner variant tasks) — v0.3+
|
||||
- Sandbox fabric (sandboxed IDE, design tool, simulation) — v0.3+
|
||||
- Identity verification and age-gating logic (16+/18+) — the real KYC backend (v0.3+; visual flow already exists in v0.1)
|
||||
- Marketplace job aggregation pipeline (3M+ jobs from 120K companies)
|
||||
- AI-powered tagging, semantic vector search, company enrichment
|
||||
- AI resume parsing and job matching
|
||||
- SEO-optimized programmatic pages
|
||||
- Payment processing and subscription management
|
||||
- Human tutor marketplace (third-party course creation)
|
||||
- B2B employer network functionality
|
||||
- CIRR-style placement tracking and audit
|
||||
|
||||
## Requirements (Out of Scope — Per Vision Doctrine)
|
||||
|
||||
- Traditional accreditation — not sought, not pursued, not revisited
|
||||
- Under-16 learners — excluded; AI school floor is 16+
|
||||
- Marketplace for under-18 — excluded; marketplace is 18+ with verified identity
|
||||
- Human tutors in the core school — excluded; humans exist only in the open marketplace
|
||||
- Graded written exams without process trace — excluded
|
||||
- Legacy job titles in curriculum — excluded
|
||||
- Junk advertising — excluded
|
||||
|
||||
---
|
||||
|
||||
## Constraints
|
||||
|
||||
1. **MAJOR 0 until MVP** — all versions remain v0.x until the MVP is released and agreed upon by the founder
|
||||
2. **No business logic in v0.1** — pure UI/UX prototype with mock data only
|
||||
3. **High-fidelity interactive** — clickable navigation, realistic content, hover states, form inputs (non-functional), responsive breakpoints, loading states (mock)
|
||||
4. **All data mocked** — no real API calls, no database, no authentication logic
|
||||
5. **Shared design system** — all 4 surfaces use a unified component library with surface-specific theming via CSS variables
|
||||
6. **TypeScript monorepo** — pnpm workspaces + turborepo for build orchestration
|
||||
7. **Next.js App Router** — route groups for each surface: (learner), (marketplace), (employer), (admin)
|
||||
8. **Solo founder constraint** — Phase 1 is build-only with no headcount; the prototype must be producible by a solo developer with AI assistance
|
||||
9. **No traditional accreditation** — proprietary credential replaces degrees
|
||||
10. **AI-first architecture** — AI tutors are the primary human-facing layer (in future milestones)
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
| ID | Decision | Rationale | Outcome |
|
||||
|----|----------|-----------|---------|
|
||||
| D-001 | Milestone v0.1 = UI/UX prototype only, no business logic | Founder directive: validate UX before building backend. Prototype must be agreed upon before proceeding to business logic. | Scope locked to frontend surfaces with mock data |
|
||||
| D-002 | MAJOR 0 until MVP released | Founder directive: remain on v0.x until MVP is validated. Signals pre-release status. | All tags are v0.x.y until MVP agreement |
|
||||
| D-003 | TypeScript monorepo (pnpm/turborepo) + Next.js | Unified codebase for all 4 surfaces. Shared component library, types, mock data. Next.js App Router for route-based surface separation. Python AI services deferred to later milestones. | Monorepo structure with apps/web + packages/* |
|
||||
| D-004 | All 4 surfaces in v0.1 (Learner, Marketplace, Employer, Admin) | Founder selected all 4 surfaces for the prototype. Complete product visualization before any backend work. | 24 REQ-IDs covering all surfaces + shared infrastructure |
|
||||
| D-005 | High-fidelity interactive prototype | Founder selected high-fidelity over wireframes. Realistic mock data, navigation flows, responsive layouts, component library. No backend calls. | Clickable prototype with realistic content |
|
||||
| D-006 | Release forge = Gitea @ git.cloudinit.dev, owner=coreci, repo=nextcraft | Founder-provided Gitea instance for release management. Token stored in .ciagent/.env.secrets. | Ship workflow creates tags + releases on Gitea |
|
||||
| D-007 | Full autonomy for CIAgent pipeline | Founder selected full autonomy. No HITL after clarify. Auto-decide above confidence 0.60. Escalation hooks: deploy, delete_data, merge_to_main. | Rapid autonomous building with kill criteria |
|
||||
| D-008 | Shared component library in packages/ui/ | All surfaces share a unified design system with surface-specific theming via CSS variables. Promotes consistency and reduces duplication. | packages/ui, packages/mock-data, packages/types |
|
||||
| D-009 | AI tutor UI as chat interface mockup with pre-scripted responses | The learner surface includes an AI tutor chat UI mockup. No real AI backend — pre-scripted responses simulate the Coach and Tutor agents. | Mockup only in v0.1, real agents in future milestone |
|
||||
| D-010 | Age-gating represented as visual registration flow mockup | 16+/18+ age-gating shown as a UI flow with age verification step. No actual verification logic. | Visual mockup only |
|
||||
| D-011 | Competency graph viewer as interactive static visualization | Admin surface includes a competency graph viewer using react-flow or similar. Mock competency nodes and edges. No real graph data. | Static graph with mock data |
|
||||
| D-012 | Tech stack: TS monorepo + Python AI services (future) | v0.1 uses TS only. Python FastAPI microservices planned for AI tutor agents and assessment engine in later milestones. | v0.1: TS only. v0.2: TS + Python (apps/ai-service) |
|
||||
| D-013 | v0.1 prototype founder-agreed; D-001 business-logic gate unlocked | Founder approved starting v0.2 with AI Tutor Architecture, which constitutes agreement of the v0.1 prototype per D-001. Recorded at v0.2 SPECIFY. | Business logic authorized from v0.2 onward |
|
||||
| D-014 | Provider-agnostic LLM layer; ollama-cloud as initial provider | OpenAI-compatible client abstraction with pluggable providers: ollama-cloud (https://ollama.com/v1, default), local OpenAI-compatible endpoint, deterministic mock (tests/CI). Keys in gitignored .ciagent/.env.secrets, never in code or commits. | apps/ai-service llm package with 3 providers; default=ollama-cloud |
|
||||
| D-015 | All six agents implemented as real LLM services; engines mocked | Coach/Tutor/Mentor fully real. Lab/Assessor/Proctor are real LLM logic over mock inputs (simulated telemetry, pre-baked artifacts) since sandbox fabric, assessment engine, and identity verification are v0.3+. Consistent with v0.1's mock-data approach. | REQ-F-001..006 complete in v0.2; real engines deferred to v0.3+ |
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
Nextcraft is being built by a solo founder using the CIAgent v0.7.0 autonomous pipeline. The vision document defines a 36-month, 3-phase roadmap to ≥1,000 placements. The first CIAgent milestone (v0.1) is intentionally scoped to UI/UX only — validating the product vision through interactive prototypes before any backend or business logic investment.
|
||||
|
||||
The four surfaces (Learner, Marketplace, Employer Dashboard, Admin) map directly to the four audiences in the vision: learners, employers, marketplace operators, and platform administrators. The prototype will demonstrate the complete user journey across all surfaces with realistic mock data reflecting the AI-era competency stacks, AI-orchestration job listings, and artifact+process trace+oral defense credential model.
|
||||
@@ -0,0 +1,194 @@
|
||||
# Nextcraft — REQUIREMENTS.md
|
||||
|
||||
## v0.2 Requirements (AI Tutor Architecture)
|
||||
|
||||
### AI Service Infrastructure
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-2-001 | apps/ai-service scaffolding: Python FastAPI app, pydantic settings, uvicorn, health endpoint, CORS, pytest setup, pnpm/turbo integration scripts | critical | 1 | complete |
|
||||
| REQ-2-002 | Provider-agnostic LLM client: OpenAI-compatible provider interface with ollama-cloud (default), local-endpoint, and deterministic mock providers; key resolution from env files | critical | 1 | complete |
|
||||
| REQ-2-003 | SSE streaming endpoint plumbing: chat completion streaming from provider through FastAPI to the Next.js client | critical | 1 | complete |
|
||||
| REQ-2-004 | Agent framework: base agent contracts, session/state store, prompt management, streaming pipeline, structured output support | critical | 2 | complete |
|
||||
|
||||
### AI Tutor Agents
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-2-005 | Coach agent (REQ-F-001): pacing, motivation, retrieval practice — full LLM implementation | critical | 3 | complete |
|
||||
| REQ-2-006 | Tutor agent (REQ-F-002): concept delivery, Socratic questioning — full LLM implementation | critical | 3 | complete |
|
||||
| REQ-2-007 | Lab agent (REQ-F-003): in-flow feedback over simulated sandbox telemetry (mock inputs) | high | 4 | complete |
|
||||
| REQ-2-008 | Assessor agent (REQ-F-004): rubric application to pre-baked artifacts and defense transcripts (mock inputs) | high | 4 | complete |
|
||||
| REQ-2-009 | Proctor agent (REQ-F-005): integrity signals from mock telemetry, coaching interventions | high | 5 | complete |
|
||||
| REQ-2-010 | Mentor agent (REQ-F-006): long-horizon career narrative | high | 5 | complete |
|
||||
|
||||
### Learner Surface Integration
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-2-011 | Learner chat UI wired to real service: streaming responses, agent routing, error and loading states | critical | 6 | complete |
|
||||
| REQ-2-012 | Byte tutorial viewer, build sandbox, and assessment mockups surface Lab/Assessor/Proctor outputs (mock engine inputs) | high | 6 | complete |
|
||||
|
||||
---
|
||||
|
||||
## v0.1 Requirements (Complete)
|
||||
|
||||
### Infrastructure
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-001 | Monorepo scaffolding: pnpm workspaces, turborepo, Next.js app, TypeScript config, ESLint, Prettier | critical | 1 | complete |
|
||||
| REQ-002 | Shared component library: design tokens, Button, Input, Card, Badge, Avatar, Dialog, Navigation, Table, Tabs, Progress, Tooltip, Skeleton, Toast | critical | 1 | pending |
|
||||
| REQ-003 | Mock data layer: typed mock data for competency stacks, job listings, candidate profiles, employers, learner progress | critical | 1 | pending |
|
||||
| REQ-004 | Routing and navigation: App Router route groups for (learner), (marketplace), (employer), (admin); shared layout components; cross-surface navigation | critical | 1 | pending |
|
||||
| REQ-005 | Responsive layout system: mobile, tablet, desktop breakpoints; container components; grid system | high | 1 | pending |
|
||||
|
||||
### Learner Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-006 | Landing page: hero, value proposition, program highlights, how-it-works (Byte→Build→Demonstrate→Defend), testimonials mockup, CTA to program catalog | critical | 2 | pending |
|
||||
| REQ-007 | Program catalog: grid of competency stacks (AI Orchestration Engineer, AI Safety & Governance Lead, Human-AI Product Designer, AI-Augmented Field Operator, Computational Sciences Practitioner); stack cards with role descriptions | critical | 2 | pending |
|
||||
| REQ-008 | Competency stack view: selected stack detail with 12-18 competencies listed, progress indicators, microcredential badges, mastery status | critical | 2 | pending |
|
||||
| REQ-009 | Learner dashboard: active competencies, progress graph, recent artifacts, upcoming defenses, AI tutor chat mockup, milestone tracker | critical | 2 | pending |
|
||||
| REQ-010 | Byte tutorial viewer: 3-7 minute micro-tutorial layout with concept panel, worked example panel, code/design/simulation viewer mockup | high | 2 | pending |
|
||||
| REQ-011 | Build sandbox mockup: sandboxed IDE/design tool/simulation UI mockup with toolbar, file explorer, editor area, telemetry sidebar (process capture indicators) | high | 2 | pending |
|
||||
| REQ-012 | Assessment/defense mockup: rubric display, AI reviewer panel, oral defense interface (voice/mic mockup), process trace timeline, artifact viewer | high | 2 | pending |
|
||||
|
||||
### Marketplace Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-013 | Job board listing: searchable grid of AI-era job listings, filter sidebar (skills, seniority, location, salary), result cards with match score | critical | 3 | pending |
|
||||
| REQ-014 | Job detail page: full job description, required competencies, employer info, application CTA, related jobs, AI-matched skills breakdown | critical | 3 | pending |
|
||||
| REQ-015 | Employer profile: company overview, logo, description, social links, open positions, company culture mockup | high | 3 | pending |
|
||||
| REQ-016 | Search/filter UI: semantic search bar, skill tags, category filters, seniority filter, remote/on-site toggle, saved searches mockup | critical | 3 | pending |
|
||||
| REQ-017 | Pricing page: job posting packages (single, bundle, enterprise), talent access plans, subscription tiers, feature comparison table | high | 3 | pending |
|
||||
|
||||
### Employer Dashboard
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-018 | Employer dashboard overview: active postings, applicant pipeline, talent matches, analytics mockup (charts, placement stats) | critical | 4 | pending |
|
||||
| REQ-019 | Talent search: searchable candidate database with AI-matched filters, candidate cards showing competency stack, microcredentials, artifact count, defense score | critical | 4 | pending |
|
||||
| REQ-020 | Candidate profile view: full candidate profile with artifact gallery, process trace summary, oral defense transcripts, competency graph, microcredential verification | high | 4 | pending |
|
||||
| REQ-021 | Posting management: create/edit/delete job postings, posting status tracking, applicant list per posting, interview pipeline mockup | high | 4 | pending |
|
||||
|
||||
### Admin Surface
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-022 | Admin overview: platform metrics (learners, employers, placements, completion rate, NPS), recent activity feed, system health mockup | critical | 5 | pending |
|
||||
| REQ-023 | Learner management: searchable learner table, learner detail view, progress tracking, competency completion status, credential issuance log | high | 5 | pending |
|
||||
| REQ-024 | Competency graph viewer: interactive visualization of competency stacks and their relationships, node/edge graph using react-flow, stack details on node click | high | 5 | pending |
|
||||
| REQ-025 | Marketplace moderation: job posting review queue, employer verification queue, flagged content, content moderation tools mockup | high | 5 | pending |
|
||||
|
||||
### Polish & Integration
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-026 | Cross-surface navigation: role switcher (learner/employer/admin), breadcrumbs, consistent header/footer across all surfaces | critical | 6 | pending |
|
||||
| REQ-027 | Visual consistency audit: typography scale, color palette, spacing system, dark mode toggle, accessibility baseline (WCAG AA contrast) | high | 6 | pending |
|
||||
| REQ-028 | Component library finalization: Storybook setup, component documentation, prop tables, usage examples | medium | 6 | pending |
|
||||
|
||||
---
|
||||
|
||||
## v2 Requirements (Future Milestones — Deferred)
|
||||
|
||||
### Assessment Engine
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-007 | Process-trace grading engine | high | v0.3+ | deferred |
|
||||
| REQ-F-008 | Per-learner variant task generation | high | v0.3+ | deferred |
|
||||
| REQ-F-009 | Oral/voice defense with AI examiner | high | v0.3+ | deferred |
|
||||
| REQ-F-010 | Live in-environment build with telemetry | high | v0.3+ | deferred |
|
||||
| REQ-F-021 | Sandbox fabric: sandboxed IDE, design tool, simulation | high | v0.3+ | deferred |
|
||||
|
||||
### Marketplace Engine
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-011 | AI job aggregation pipeline (3M+ jobs from 120K companies) | high | v0.4+ | deferred |
|
||||
| REQ-F-012 | AI-powered tagging (skills, categories, seniority) | high | v0.4+ | deferred |
|
||||
| REQ-F-013 | Semantic vector search for role matching | high | v0.4+ | deferred |
|
||||
| REQ-F-014 | AI company enrichment (logos, descriptions, social links) | medium | v0.4+ | deferred |
|
||||
| REQ-F-015 | AI resume parsing (profile auto-fill) | medium | v0.4+ | deferred |
|
||||
| REQ-F-016 | SEO-optimized programmatic pages | medium | v0.4+ | deferred |
|
||||
|
||||
### Platform Infrastructure
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.3+ | deferred |
|
||||
| REQ-F-018 | Payment processing and subscription management | high | v0.3+ | deferred |
|
||||
| REQ-F-019 | Human tutor marketplace (third-party courses) | medium | v0.4+ | deferred |
|
||||
| REQ-F-020 | CIRR-style placement tracking and audit | medium | v0.5+ | deferred |
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope (Per Vision Doctrine)
|
||||
|
||||
| Feature | Reason |
|
||||
|---------|--------|
|
||||
| Traditional accreditation | Not sought, not pursued, not revisited (Principle 4) |
|
||||
| Under-16 learners | COPPA avoidance; AI school floor is 16+ |
|
||||
| Marketplace for under-18 | Marketplace restricted to 18+ with verified identity |
|
||||
| Human tutors in core school | Humans exist only in open marketplace as third-party sellers |
|
||||
| Graded written exams without process trace | Trivially solvable by frontier AI; process-trace + oral defense more valid |
|
||||
| Legacy job titles in curriculum | Programs train for AI-era roles, not obsolete ones |
|
||||
| Junk advertising | Ad quality enforced algorithmically; relevant ads only |
|
||||
|
||||
---
|
||||
|
||||
## Traceability Matrix
|
||||
|
||||
### v0.2 (current milestone)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-2-001 | 1 | complete |
|
||||
| REQ-2-002 | complete |
|
||||
| REQ-2-003 | 1 | complete |
|
||||
| REQ-2-004 | complete |
|
||||
| REQ-2-005 | complete |
|
||||
| REQ-2-006 | complete |
|
||||
| REQ-2-007 | complete |
|
||||
| REQ-2-008 | complete |
|
||||
| REQ-2-009 | complete |
|
||||
| REQ-2-010 | complete |
|
||||
| REQ-2-011 | complete |
|
||||
| REQ-2-012 | complete |
|
||||
|
||||
### v0.1 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-001 | 1 | complete |
|
||||
| REQ-002 | 1 | complete |
|
||||
| REQ-003 | 1 | complete |
|
||||
| REQ-004 | 1 | complete |
|
||||
| REQ-005 | 1 | complete |
|
||||
| REQ-006 | 2 | complete |
|
||||
| REQ-007 | 2 | complete |
|
||||
| REQ-008 | 2 | complete |
|
||||
| REQ-009 | 2 | complete |
|
||||
| REQ-010 | 2 | complete |
|
||||
| REQ-011 | 2 | complete |
|
||||
| REQ-012 | 2 | complete |
|
||||
| REQ-013 | 3 | complete |
|
||||
| REQ-014 | 3 | complete |
|
||||
| REQ-015 | 3 | complete |
|
||||
| REQ-016 | 3 | complete |
|
||||
| REQ-017 | 3 | complete |
|
||||
| REQ-018 | 4 | complete |
|
||||
| REQ-019 | 4 | complete |
|
||||
| REQ-020 | 4 | complete |
|
||||
| REQ-021 | 4 | complete |
|
||||
| REQ-022 | 5 | complete |
|
||||
| REQ-023 | 5 | complete |
|
||||
| REQ-024 | 5 | complete |
|
||||
| REQ-025 | 5 | complete |
|
||||
| REQ-026 | 6 | complete |
|
||||
| REQ-027 | 6 | complete |
|
||||
| REQ-028 | 6 | complete |
|
||||
@@ -0,0 +1,180 @@
|
||||
# Nextcraft — ROADMAP.md
|
||||
|
||||
## Overview
|
||||
|
||||
**Milestone v0.2** — AI Tutor Architecture: The six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface with streaming responses. Provider-agnostic LLM layer (ollama-cloud default). Lab/Assessor/Proctor operate on mock engine inputs — their real engines are v0.3+.
|
||||
|
||||
**Prior milestone:** v0.1 (nextcraft-ui-prototype) — complete, shipped as v0.1.0, founder-agreed (D-013).
|
||||
|
||||
**Milestone type:** Feature (new AI service + real agent capabilities)
|
||||
**Tag line:** v0.1.x (patches on the v0.1 line; milestone release as v0.2.0)
|
||||
**Branch:** milestone/v0.2-ai-tutor-architecture
|
||||
|
||||
---
|
||||
|
||||
## Phase List
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan complete; .ciagent/ files updated for v0.2 |
|
||||
| 1 | AI service scaffolding | complete | 0 | REQ-2-001, REQ-2-002, REQ-2-003 | apps/ai-service runs (uvicorn), health endpoint responds, provider-agnostic LLM client with 3 providers (ollama-cloud/local/mock), SSE streaming verified, pytest suite passes with mock provider, turbo scripts wired |
|
||||
| 2 | Agent framework | complete | 1 | REQ-2-004 | Base agent contract, session/state store, prompt templates, streaming pipeline, structured outputs; all tested |
|
||||
| 3 | Coach + Tutor agents | complete | 2 | REQ-2-005, REQ-2-006 | Coach (pacing/motivation/retrieval practice) and Tutor (concept delivery/Socratic questioning) fully implemented with system prompts, tested against mock provider, wired to chat endpoint |
|
||||
| 4 | Lab + Assessor agents | complete | 2 | REQ-2-007, REQ-2-008 | Lab consumes simulated sandbox telemetry (mock); Assessor applies rubrics to pre-baked artifacts/defense transcripts (mock); both tested |
|
||||
| 5 | Proctor + Mentor agents | complete | 2 | REQ-2-009, REQ-2-010 | Proctor produces integrity signals + coaching interventions from mock telemetry; Mentor generates long-horizon career narrative; both tested |
|
||||
| 6 | Learner surface integration | complete | 3, 4, 5 | REQ-2-011, REQ-2-012 | Learner chat streams real responses; agent routing works; byte viewer/sandbox/assessment mockups surface agent outputs; error/loading states; pnpm build + typecheck pass |
|
||||
| 7 | Final review + ship | complete | 6 | — | Code review clean; audit passes; milestone tagged v0.2.0; release created on Gitea |
|
||||
|
||||
---
|
||||
|
||||
## Phase Details
|
||||
|
||||
### Phase 0: Pre-execution
|
||||
|
||||
**Goal:** Establish v0.2 specification, clarify ambiguities, research AI service architecture, create detailed plans.
|
||||
|
||||
**Stages:** SPECIFY → CLARIFY → RESEARCH → PLAN → GRILL → SHIP
|
||||
|
||||
**Deliverables:**
|
||||
- Updated .ciagent/config.json, PROJECT.md, REQUIREMENTS.md, ROADMAP.md, ARCHITECTURE.md, PERSONAS.md, PLAN.md
|
||||
|
||||
**Success criteria:** All .ciagent/ files updated for v0.2; phase 0 shipped as v0.1.1.
|
||||
|
||||
---
|
||||
|
||||
### Phase 1: AI Service Scaffolding
|
||||
|
||||
**Goal:** Stand up apps/ai-service with the provider-agnostic LLM layer and SSE streaming.
|
||||
|
||||
**Requirements:** REQ-2-001, REQ-2-002, REQ-2-003
|
||||
|
||||
**Key deliverables:**
|
||||
- apps/ai-service: FastAPI app, pydantic-settings, uvicorn, /health, CORS for localhost
|
||||
- llm package: provider interface + ollama-cloud/local/mock providers; key resolution from .ciagent/.env.secrets via env
|
||||
- SSE streaming: /v1/chat/stream endpoint streaming provider deltas
|
||||
- pytest suite with mock provider; root scripts: ai:dev, ai:test; turbo integration
|
||||
|
||||
**Success criteria:**
|
||||
- `python -m uvicorn` starts the service; /health returns 200
|
||||
- Provider unit tests pass (mock); ollama-cloud integration probe works (manual)
|
||||
- SSE stream delivers tokens to an HTTP client
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Agent Framework
|
||||
|
||||
**Goal:** Build the shared framework all six agents use.
|
||||
|
||||
**Requirements:** REQ-2-004
|
||||
|
||||
**Key deliverables:**
|
||||
- BaseAgent contract: system prompt, message history, streaming completion, structured output
|
||||
- Session/state store: in-memory per-learner session with message history
|
||||
- Prompt management: per-agent system prompt templates with learner context injection
|
||||
- Streaming pipeline: agent → provider → SSE with agent identification
|
||||
- Structured outputs: JSON-schema outputs for Assessor rubric scores, Proctor signals
|
||||
|
||||
**Success criteria:**
|
||||
- BaseAgent unit tests pass
|
||||
- Session store tested (create/append/persist in-memory)
|
||||
- Structured output parsing tested against mock provider
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Coach + Tutor Agents
|
||||
|
||||
**Goal:** Implement the two learner-facing conversational agents.
|
||||
|
||||
**Requirements:** REQ-2-005, REQ-2-006
|
||||
|
||||
**Key deliverables:**
|
||||
- Coach agent: pacing guidance, motivation, retrieval practice prompts; distinct persona
|
||||
- Tutor agent: concept delivery, Socratic questioning, worked examples
|
||||
- Agent registry: route chat messages to the correct agent by context/selection
|
||||
- Per-agent system prompts with competency-stack context injection from packages/mock-data
|
||||
|
||||
**Success criteria:**
|
||||
- Both agents produce distinct, on-persona responses (verified against mock + ollama-cloud)
|
||||
- Agent routing tested
|
||||
- Both agents exposed via the chat streaming endpoint
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Lab + Assessor Agents
|
||||
|
||||
**Goal:** Implement the two build/assessment agents over mock engine inputs.
|
||||
|
||||
**Requirements:** REQ-2-007, REQ-2-008
|
||||
|
||||
**Key deliverables:**
|
||||
- Lab agent: consumes simulated sandbox telemetry (mock event streams), produces in-flow feedback
|
||||
- Assessor agent: applies rubrics to pre-baked artifacts and defense transcripts, returns structured scores + feedback
|
||||
- Mock engine inputs: simulated telemetry generator, pre-baked artifact corpus in packages/mock-data
|
||||
- Endpoints: /v1/lab/feedback, /v1/assessment/evaluate
|
||||
|
||||
**Success criteria:**
|
||||
- Lab produces relevant feedback for mock telemetry scenarios
|
||||
- Assessor returns structured rubric scores (JSON) for pre-baked artifacts
|
||||
- Both tested against mock provider
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Proctor + Mentor Agents
|
||||
|
||||
**Goal:** Implement the integrity and narrative agents.
|
||||
|
||||
**Requirements:** REQ-2-009, REQ-2-010
|
||||
|
||||
**Key deliverables:**
|
||||
- Proctor agent: integrity signals from mock telemetry (tab switches, idle time, paste events), coaching interventions
|
||||
- Mentor agent: long-horizon career narrative, competency-stack progression guidance
|
||||
- Endpoints: /v1/proctor/signals, /v1/mentor/narrative
|
||||
|
||||
**Success criteria:**
|
||||
- Proctor produces classified signals with recommended interventions for mock scenarios
|
||||
- Mentor produces coherent career-narrative responses
|
||||
- Both tested against mock provider
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Learner Surface Integration
|
||||
|
||||
**Goal:** Wire the v0.1 learner surface to the real AI service.
|
||||
|
||||
**Requirements:** REQ-2-011, REQ-2-012
|
||||
|
||||
**Key deliverables:**
|
||||
- Learner dashboard chat: real streaming via SSE, agent switcher (Coach/Tutor), error/loading states
|
||||
- Byte tutorial viewer: Tutor concept explanations
|
||||
- Build sandbox: Lab feedback panel fed by mock telemetry + Lab agent
|
||||
- Assessment mockup: Assessor rubric output display, Proctor integrity banner
|
||||
- Mentor panel on learner dashboard
|
||||
|
||||
**Success criteria:**
|
||||
- Streaming chat works end-to-end with ai-service running
|
||||
- All four learner surfaces surface agent outputs
|
||||
- Graceful degradation when ai-service is down (error states, not crashes)
|
||||
- `pnpm build` and `pnpm typecheck` pass
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Final Review + Ship
|
||||
|
||||
**Goal:** Code review, audit, milestone release.
|
||||
|
||||
**Key deliverables:**
|
||||
- Multi-persona code review (correctness, testing, security, performance, maintainability)
|
||||
- Project health audit (reconstruction test, .ciagent/ file discipline, branch hygiene, commit discipline)
|
||||
- Milestone ship: merge milestone → main, tag v0.2.0, create Gitea release
|
||||
|
||||
**Success criteria:**
|
||||
- Code review: P0 fixes applied, P1+ documented
|
||||
- Audit: all checks pass, project state reconstructable from git log
|
||||
- Ship: v0.2.0 tagged, milestone branch merged to main, Gitea release created — **release note explicitly states Lab/Assessor/Proctor operate on mock engine inputs (real engines v0.3+)** (G-5); dead `aiTutorResponses` export disposed of (G-5)
|
||||
- All 12 v0.2 requirements marked complete
|
||||
|
||||
---
|
||||
|
||||
## v0.1 (Complete — Shipped as v0.1.0)
|
||||
|
||||
UI/UX Prototype: High-fidelity interactive prototype of all four Nextcraft surfaces. 7 phases (P0 + P1-P6 execution + P7 final). All 28 requirements complete. Tags v0.0.1–v0.0.7, milestone release v0.1.0.
|
||||
@@ -0,0 +1,54 @@
|
||||
{
|
||||
"version": "0.7.0",
|
||||
"autonomy": {
|
||||
"level": "full",
|
||||
"decision_confidence_threshold": 0.6,
|
||||
"escalation_hooks": ["deploy", "delete_data", "merge_to_main"],
|
||||
"max_revision_iterations": 3,
|
||||
"clarify_budget": 10
|
||||
},
|
||||
"release": {
|
||||
"forge": "gitea",
|
||||
"base_url": "https://git.cloudinit.dev",
|
||||
"owner": "coreci",
|
||||
"repo": "nextcraft"
|
||||
},
|
||||
"secrets": {
|
||||
"scopes": ["gitea"],
|
||||
"env_files": [".env", ".env.secrets", ".env.*"]
|
||||
},
|
||||
"ship": {
|
||||
"per_phase": true,
|
||||
"allow_skip": false,
|
||||
"release_blocking": false,
|
||||
"max_release_retries": 3
|
||||
},
|
||||
"verification": {
|
||||
"bdd_default": false
|
||||
},
|
||||
"personas": {
|
||||
"enabled": true,
|
||||
"territory_enforcement": "warn"
|
||||
},
|
||||
"parallelization": {
|
||||
"enabled": false,
|
||||
"max_concurrent_agents": 1
|
||||
},
|
||||
"ideation": {
|
||||
"max_ideas": 20,
|
||||
"categories": ["security", "quality", "architecture", "coverage", "improvement", "spec", "chaos", "bdd"]
|
||||
},
|
||||
"security": {
|
||||
"bash_allowlist": {
|
||||
"blocked_env_vars": ["GITEA_TOKEN", "GITHUB_TOKEN", "GITLAB_TOKEN"]
|
||||
}
|
||||
},
|
||||
"projects": [],
|
||||
"active_project": null,
|
||||
"milestone": {
|
||||
"version": "v0.2",
|
||||
"name": "ai-tutor-architecture",
|
||||
"type": "feature",
|
||||
"branch": "milestone/v0.2-ai-tutor-architecture"
|
||||
}
|
||||
}
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Environment files — never commit credentials
|
||||
.env
|
||||
.env.secrets
|
||||
.env.*
|
||||
!.env.example
|
||||
|
||||
# CIAgent secrets
|
||||
.ciagent/.env.secrets
|
||||
|
||||
# Dependencies
|
||||
node_modules/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.venv/
|
||||
venv/
|
||||
|
||||
# Build artifacts
|
||||
dist/
|
||||
/build/
|
||||
.next/
|
||||
.turbo/
|
||||
*.tsbuildinfo
|
||||
|
||||
# Storybook
|
||||
storybook-static/
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
|
||||
# Test coverage
|
||||
coverage/
|
||||
.nyc_output/
|
||||
|
||||
# Python tooling caches
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
*.egg-info/
|
||||
@@ -0,0 +1,3 @@
|
||||
shamefully-hoist=true
|
||||
strict-peer-dependencies=false
|
||||
pnpm-approved-builds=esbuild,core-js-pure
|
||||
@@ -0,0 +1,10 @@
|
||||
# Nextcraft AI Service — environment template (copy values, never commit real keys)
|
||||
# Real keys live in .ciagent/.env.secrets (gitignored) and are exported by scripts/dev.sh
|
||||
|
||||
AI_PORT=8420
|
||||
AI_PROVIDER=ollama-cloud
|
||||
AI_MODEL=gemma4:31b
|
||||
AI_OLLAMA_CLOUD_BASE_URL=https://ollama.com/v1
|
||||
AI_OLLAMA_CLOUD_API_KEY=
|
||||
AI_LOCAL_BASE_URL=http://localhost:11434/v1
|
||||
AI_JSON_MODE=auto
|
||||
@@ -0,0 +1,90 @@
|
||||
# Nextcraft AI Service (`apps/ai-service`)
|
||||
|
||||
Python FastAPI service hosting the six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) behind a provider-agnostic LLM layer. Port **8420**.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
# 1. Bootstrap (idempotent): venv + deps
|
||||
bash scripts/bootstrap.sh
|
||||
|
||||
# 2. Run tests (mock provider only — zero network calls)
|
||||
bash scripts/test.sh
|
||||
|
||||
# 3. Lint
|
||||
bash scripts/lint.sh
|
||||
|
||||
# 4. Dev server (exports keys from .ciagent/.env.secrets if present)
|
||||
bash scripts/dev.sh
|
||||
```
|
||||
|
||||
Or via the monorepo root (`corepack pnpm install` first):
|
||||
|
||||
```bash
|
||||
pnpm ai:bootstrap
|
||||
pnpm ai:test
|
||||
pnpm ai:lint
|
||||
pnpm ai:dev
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings use the `AI_` env prefix (pydantic-settings; see `.env.example`).
|
||||
|
||||
| Var | Default | Purpose |
|
||||
|-----|---------|---------|
|
||||
| `AI_PORT` | 8420 | Listen port |
|
||||
| `AI_PROVIDER` | mock | `ollama-cloud` \| `local` \| `mock` |
|
||||
| `AI_MODEL` | gemma4:31b | Model for all agents |
|
||||
| `AI_OLLAMA_CLOUD_BASE_URL` | https://ollama.com/v1 | Cloud base URL |
|
||||
| `AI_OLLAMA_CLOUD_API_KEY` | (empty) | Bearer key — **never commit** |
|
||||
| `AI_JSON_MODE` | auto | `auto` sends response_format, degrades on 400; `off` never sends |
|
||||
|
||||
Tests run with `AI_PROVIDER=mock` (enforced in `tests/conftest.py` by an instance assertion) — the suite never calls the cloud.
|
||||
|
||||
## Endpoints
|
||||
|
||||
- `GET /health` — status, configured provider, model (no cloud call)
|
||||
- `POST /v1/chat/stream` — SSE chat stream. Body: `{"agent": "coach"|"tutor", "session_id": "...", "messages": [{"role":"user","content":"..."}]}`. Unknown agents are rejected with 422.
|
||||
|
||||
SSE envelope (D-016): `meta` event first (agent/session/model), then `delta` events (incremental content), then `done`; on mid-stream failure an `error` event precedes the terminal `[DONE]` sentinel. sse-starlette emits `: ping` keep-alive comment lines on idle connections — clients must ignore frames without `data:`.
|
||||
|
||||
## Manual ollama-cloud persona probe (Phase 3, documented — not automated)
|
||||
|
||||
With the real provider, Coach and Tutor must produce distinct on-persona
|
||||
responses to the same prompt:
|
||||
|
||||
```bash
|
||||
# start with the cloud provider (keys exported from .ciagent/.env.secrets)
|
||||
AI_PROVIDER=ollama-cloud .venv/bin/uvicorn ai_service.main:app --port 8420
|
||||
|
||||
# Coach: expect pacing + one concrete next action + a retrieval-practice question
|
||||
curl -sN -X POST localhost:8420/v1/chat/stream -H 'Content-Type: application/json' \
|
||||
-d '{"agent":"coach","session_id":"probe-coach","messages":[{"role":"user","content":"I am stuck on multi-agent communication patterns"}]}' \
|
||||
| grep '^data:'
|
||||
|
||||
# Tutor: expect ONE concept + a worked example + a Socratic check question
|
||||
curl -sN -X POST localhost:8420/v1/chat/stream -H 'Content-Type: application/json' \
|
||||
-d '{"agent":"tutor","session_id":"probe-tutor","messages":[{"role":"user","content":"I am stuck on multi-agent communication patterns"}]}' \
|
||||
| grep '^data:'
|
||||
```
|
||||
|
||||
Verify: the two responses have visibly different voice/structure (Coach:
|
||||
action + accountability; Tutor: concept + example + question). The
|
||||
automated suite never calls the cloud — distinctness is enforced against
|
||||
the deterministic mock (distinct system prompts → distinct hash-seeded
|
||||
outputs).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
ai_service/
|
||||
main.py app factory, lifespan (httpx pool), CORS, /health
|
||||
config.py pydantic-settings
|
||||
api/ endpoints (SSE envelope lives here, D-016)
|
||||
llm/ provider layer — dumb pipe, no envelope logic
|
||||
scripts/ bootstrap.sh dev.sh test.sh lint.sh
|
||||
tests/ pytest — mock provider only
|
||||
```
|
||||
|
||||
Boundary rules: `llm/` imports nothing from `agents/` or `api/`; `agents/` imports nothing from `api/`.
|
||||
@@ -0,0 +1,3 @@
|
||||
"""Nextcraft AI tutor service — six LLM agents behind a provider-agnostic layer."""
|
||||
|
||||
__version__ = "0.2.0"
|
||||
@@ -0,0 +1,18 @@
|
||||
"""Agent framework — BaseAgent ABC (D-018), registry, sessions, structured outputs.
|
||||
|
||||
Boundary rule: agents/ imports from llm/, prompts/, corpus/ — never from api/.
|
||||
"""
|
||||
|
||||
from .base import BaseAgent
|
||||
from .registry import AgentRegistry
|
||||
from .session import InMemorySessionStore, SessionStore
|
||||
from .structured import StructuredOutputError, extract_json_object
|
||||
|
||||
__all__ = [
|
||||
"AgentRegistry",
|
||||
"BaseAgent",
|
||||
"InMemorySessionStore",
|
||||
"SessionStore",
|
||||
"StructuredOutputError",
|
||||
"extract_json_object",
|
||||
]
|
||||
@@ -0,0 +1,92 @@
|
||||
"""AssessorAgent — rubric application to pre-baked artifacts (REQ-2-008).
|
||||
|
||||
Structured-output showcase: applies the 4-layer defense (D-020) to return
|
||||
a pydantic-validated rubric score. Mock engine inputs (corpus artifacts +
|
||||
transcripts); real process-trace grading is v0.3+.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..corpus.artifacts import (
|
||||
ArtifactSubmission,
|
||||
AssessmentRubric,
|
||||
DefenseTranscript,
|
||||
render_rubric,
|
||||
render_transcript,
|
||||
)
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..prompts.assessor import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class CriterionScore(BaseModel):
|
||||
criterion_id: str
|
||||
name: str
|
||||
score: int = Field(ge=0, le=100)
|
||||
evidence: str
|
||||
|
||||
|
||||
class RubricScore(BaseModel):
|
||||
rubric_id: str
|
||||
artifact_id: str
|
||||
competency_id: str
|
||||
scores: list[CriterionScore]
|
||||
strengths: list[str] = Field(min_length=1, max_length=2)
|
||||
gaps: list[str] = Field(min_length=1, max_length=2)
|
||||
verdict: str # "mastered" | "developing" | "not_yet"
|
||||
|
||||
def weighted_total(self, rubric: AssessmentRubric) -> float:
|
||||
by_id = {c.criterion_id: c for c in rubric.criteria}
|
||||
total = 0.0
|
||||
for s in self.scores:
|
||||
total += s.score * by_id[s.criterion_id].weight
|
||||
return total
|
||||
|
||||
|
||||
RUBRIC_SCORE_SCHEMA_HINT = (
|
||||
'{"rubric_id": "<id>", "artifact_id": "<id>", "competency_id": "<id>", '
|
||||
'"scores": [{"criterion_id": "<id>", "name": "<name>", "score": <0-100>, '
|
||||
'"evidence": "<one sentence>"}], "strengths": ["<one sentence>"], '
|
||||
'"gaps": ["<one sentence>"], "verdict": "mastered"|"developing"|"not_yet"}'
|
||||
)
|
||||
|
||||
|
||||
class AssessorAgent(BaseAgent):
|
||||
name = "assessor"
|
||||
|
||||
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
|
||||
ctx = learner_context or get_learner_context()
|
||||
return SYSTEM_PROMPT.format_map(render_context(ctx))
|
||||
|
||||
def build_evaluation_input(
|
||||
self,
|
||||
artifact: ArtifactSubmission,
|
||||
rubric: AssessmentRubric,
|
||||
transcript: DefenseTranscript | None,
|
||||
) -> str:
|
||||
parts = [
|
||||
f"ARTIFACT: {artifact.name} ({artifact.artifact_id})",
|
||||
f"Evidence excerpt: {artifact.evidence_excerpt}",
|
||||
"",
|
||||
render_rubric(rubric),
|
||||
]
|
||||
if transcript is not None:
|
||||
parts += ["", render_transcript(transcript)]
|
||||
return "\n".join(parts)
|
||||
|
||||
async def evaluate(
|
||||
self,
|
||||
artifact: ArtifactSubmission,
|
||||
rubric: AssessmentRubric,
|
||||
transcript: DefenseTranscript | None,
|
||||
learner_context: LearnerContext | None = None,
|
||||
) -> RubricScore:
|
||||
evaluation_input = self.build_evaluation_input(artifact, rubric, transcript)
|
||||
score: RubricScore = await self.structured_reply(
|
||||
history=None,
|
||||
user_input=evaluation_input,
|
||||
learner_context=learner_context,
|
||||
schema=RubricScore,
|
||||
schema_hint=RUBRIC_SCORE_SCHEMA_HINT,
|
||||
)
|
||||
return score
|
||||
@@ -0,0 +1,77 @@
|
||||
"""BaseAgent ABC — the contract all six tutor agents implement (D-018).
|
||||
|
||||
Subclasses set `name`, override `system_prompt()`, and rarely `stream_reply()`.
|
||||
The default pipeline: build_messages() → provider.stream_chat()/chat().
|
||||
"""
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import LearnerContext
|
||||
from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
from .structured import structured_completion
|
||||
|
||||
|
||||
class BaseAgent(ABC):
|
||||
"""A tutor agent: system prompt + message assembly + provider delegation."""
|
||||
|
||||
name: str = "base"
|
||||
|
||||
def __init__(self, provider: LLMProvider, settings: Settings) -> None:
|
||||
self.provider = provider
|
||||
self.settings = settings
|
||||
|
||||
@abstractmethod
|
||||
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
|
||||
"""Return the agent's system prompt, learner-context-aware."""
|
||||
|
||||
def build_messages(
|
||||
self,
|
||||
history: list[Message] | None = None,
|
||||
user_input: str = "",
|
||||
learner_context: LearnerContext | None = None,
|
||||
) -> list[Message]:
|
||||
"""Compose the full message list: system prompt + history + user turn."""
|
||||
messages: list[Message] = [
|
||||
Message(role="system", content=self.system_prompt(learner_context))
|
||||
]
|
||||
for m in history or []:
|
||||
messages.append(m)
|
||||
if user_input:
|
||||
messages.append(Message(role="user", content=user_input))
|
||||
return messages
|
||||
|
||||
async def stream_reply(
|
||||
self,
|
||||
history: list[Message] | None = None,
|
||||
user_input: str = "",
|
||||
learner_context: LearnerContext | None = None,
|
||||
response_format: dict | None = None,
|
||||
) -> AsyncIterator[str]:
|
||||
"""Stream incremental content deltas for a conversational reply."""
|
||||
messages = self.build_messages(history, user_input, learner_context)
|
||||
async for token in self.provider.stream_chat(
|
||||
messages, model=self.settings.model, response_format=response_format
|
||||
):
|
||||
yield token
|
||||
|
||||
async def structured_reply(
|
||||
self,
|
||||
history: list[Message] | None = None,
|
||||
user_input: str = "",
|
||||
learner_context: LearnerContext | None = None,
|
||||
schema: type[BaseModel] | None = None,
|
||||
schema_hint: str = "",
|
||||
) -> BaseModel:
|
||||
"""Non-streaming completion parsed into a pydantic model (D-020 defense)."""
|
||||
if schema is None:
|
||||
raise ValueError("structured_reply requires a schema")
|
||||
messages = self.build_messages(history, user_input, learner_context)
|
||||
return await structured_completion(
|
||||
self.provider, messages, model=self.settings.model,
|
||||
schema=schema, schema_hint=schema_hint,
|
||||
)
|
||||
@@ -0,0 +1,13 @@
|
||||
"""CoachAgent — pacing, motivation, retrieval practice (REQ-2-005)."""
|
||||
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..prompts.coach import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class CoachAgent(BaseAgent):
|
||||
name = "coach"
|
||||
|
||||
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
|
||||
ctx = learner_context or get_learner_context()
|
||||
return SYSTEM_PROMPT.format_map(render_context(ctx))
|
||||
@@ -0,0 +1,37 @@
|
||||
"""LabAgent — in-flow feedback over simulated sandbox telemetry (REQ-2-007).
|
||||
|
||||
Scenario-driven: consumes a LabTelemetryScenario from the corpus, renders
|
||||
the event timeline into the conversation, streams concrete feedback.
|
||||
No session chat — each request is one scenario read.
|
||||
"""
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..corpus.telemetry import LabTelemetryScenario, summarize_scenario
|
||||
from ..llm.base import LLMProvider
|
||||
from ..prompts.lab import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class LabAgent(BaseAgent):
|
||||
name = "lab"
|
||||
|
||||
def __init__(self, provider: LLMProvider, settings: Settings) -> None:
|
||||
super().__init__(provider, settings)
|
||||
|
||||
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
|
||||
ctx = learner_context or get_learner_context()
|
||||
return SYSTEM_PROMPT.format_map(render_context(ctx))
|
||||
|
||||
async def stream_feedback(
|
||||
self,
|
||||
scenario: LabTelemetryScenario,
|
||||
learner_context: LearnerContext | None = None,
|
||||
) -> AsyncIterator[str]:
|
||||
timeline = summarize_scenario(scenario)
|
||||
async for token in self.stream_reply(
|
||||
history=None, user_input=timeline, learner_context=learner_context
|
||||
):
|
||||
yield token
|
||||
@@ -0,0 +1,17 @@
|
||||
"""MentorAgent — long-horizon career narrative (REQ-2-010).
|
||||
|
||||
Streaming, session-backed conversational agent: the learner can ask
|
||||
follow-up questions about their trajectory and the Mentor keeps context.
|
||||
"""
|
||||
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..prompts.mentor import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class MentorAgent(BaseAgent):
|
||||
name = "mentor"
|
||||
|
||||
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
|
||||
ctx = learner_context or get_learner_context()
|
||||
return SYSTEM_PROMPT.format_map(render_context(ctx))
|
||||
@@ -0,0 +1,57 @@
|
||||
"""ProctorAgent — integrity signals + coaching interventions (REQ-2-009).
|
||||
|
||||
Consumes a ProctorScenario from the corpus, returns pydantic-validated
|
||||
signal classifications via structured_reply (4-layer defense).
|
||||
Mock engine inputs; real identity/attention signals are v0.3+.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..corpus.telemetry import ProctorScenario, summarize_proctor_scenario
|
||||
from ..prompts.proctor import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class IntegritySignal(BaseModel):
|
||||
signal_type: str # e.g. "context_switch" | "idle_gap" | "large_paste"
|
||||
severity: str # "low" | "medium" | "high"
|
||||
note: str
|
||||
|
||||
|
||||
class ProctorAssessment(BaseModel):
|
||||
scenario_id: str
|
||||
signals: list[IntegritySignal] = Field(min_length=0)
|
||||
intervention: str # ONE supportive coaching recommendation
|
||||
summary: str
|
||||
|
||||
|
||||
PROCTOR_ASSESSMENT_SCHEMA_HINT = (
|
||||
'{"scenario_id": "<id>", "signals": [{"signal_type": "<type>", '
|
||||
'"severity": "low"|"medium"|"high", "note": "<one sentence>"}], '
|
||||
'"intervention": "<one supportive recommendation>", '
|
||||
'"summary": "<one sentence>"}'
|
||||
)
|
||||
|
||||
|
||||
class ProctorAgent(BaseAgent):
|
||||
name = "proctor"
|
||||
|
||||
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
|
||||
ctx = learner_context or get_learner_context()
|
||||
return SYSTEM_PROMPT.format_map(render_context(ctx))
|
||||
|
||||
async def assess(
|
||||
self,
|
||||
scenario: ProctorScenario,
|
||||
learner_context: LearnerContext | None = None,
|
||||
) -> ProctorAssessment:
|
||||
timeline = summarize_proctor_scenario(scenario)
|
||||
assessment: ProctorAssessment = await self.structured_reply(
|
||||
history=None,
|
||||
user_input=timeline,
|
||||
learner_context=learner_context,
|
||||
schema=ProctorAssessment,
|
||||
schema_hint=PROCTOR_ASSESSMENT_SCHEMA_HINT,
|
||||
)
|
||||
return assessment
|
||||
@@ -0,0 +1,70 @@
|
||||
"""Agent registry — explicit name → agent factory map (D-018, G-4).
|
||||
|
||||
Agents are registered centrally in their own phases (P3-P5) via
|
||||
`registry.register(name, factory)`. One registration pattern, one registry.
|
||||
"""
|
||||
|
||||
from collections.abc import Callable
|
||||
|
||||
from ..config import Settings
|
||||
from ..llm.base import LLMProvider
|
||||
from .base import BaseAgent
|
||||
|
||||
AgentFactory = Callable[[LLMProvider, Settings], BaseAgent]
|
||||
|
||||
|
||||
def register_builtin_agents(registry: "AgentRegistry") -> None:
|
||||
"""Central registration of all six shipped tutor agents (G-4: one pattern).
|
||||
|
||||
coach, tutor, lab, assessor, proctor, mentor. New agents register here
|
||||
in their landing phase.
|
||||
"""
|
||||
from .assessor import AssessorAgent
|
||||
from .coach import CoachAgent
|
||||
from .lab import LabAgent
|
||||
from .mentor import MentorAgent
|
||||
from .proctor import ProctorAgent
|
||||
from .tutor import TutorAgent
|
||||
|
||||
registry.register("coach", lambda provider, settings: CoachAgent(provider, settings))
|
||||
registry.register("tutor", lambda provider, settings: TutorAgent(provider, settings))
|
||||
registry.register("lab", lambda provider, settings: LabAgent(provider, settings))
|
||||
registry.register(
|
||||
"assessor", lambda provider, settings: AssessorAgent(provider, settings)
|
||||
)
|
||||
registry.register(
|
||||
"proctor", lambda provider, settings: ProctorAgent(provider, settings)
|
||||
)
|
||||
registry.register(
|
||||
"mentor", lambda provider, settings: MentorAgent(provider, settings)
|
||||
)
|
||||
|
||||
|
||||
class UnknownAgentError(KeyError):
|
||||
"""Raised when resolving an agent name that was never registered."""
|
||||
|
||||
|
||||
class DuplicateAgentError(ValueError):
|
||||
"""Raised when registering an agent name that already exists."""
|
||||
|
||||
|
||||
class AgentRegistry:
|
||||
def __init__(self) -> None:
|
||||
self._factories: dict[str, AgentFactory] = {}
|
||||
|
||||
def register(self, name: str, factory: AgentFactory) -> None:
|
||||
if name in self._factories:
|
||||
raise DuplicateAgentError(f"agent {name!r} already registered")
|
||||
self._factories[name] = factory
|
||||
|
||||
def names(self) -> list[str]:
|
||||
return sorted(self._factories)
|
||||
|
||||
def get(self, provider: LLMProvider, settings: Settings, name: str) -> BaseAgent:
|
||||
try:
|
||||
factory = self._factories[name]
|
||||
except KeyError:
|
||||
raise UnknownAgentError(
|
||||
f"unknown agent {name!r}; registered: {self.names()}"
|
||||
) from None
|
||||
return factory(provider, settings)
|
||||
@@ -0,0 +1,98 @@
|
||||
"""SessionStore — protocol + in-memory implementation (D-019).
|
||||
|
||||
Protocol is DB-migration-ready (A-003): swap InMemorySessionStore for a
|
||||
Redis/PG-backed implementation without touching the API layer.
|
||||
|
||||
Sessions are agent-scoped: switching agents starts a new session ID (avoids
|
||||
persona bleed, A-007). History windowing happens here (last N messages),
|
||||
controlling token growth per session.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from collections import OrderedDict
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Protocol
|
||||
|
||||
from ..llm.types import Message
|
||||
|
||||
DEFAULT_WINDOW = 20
|
||||
DEFAULT_MAX_SESSIONS = 500
|
||||
|
||||
|
||||
@dataclass
|
||||
class AgentSession:
|
||||
session_id: str
|
||||
agent: str
|
||||
learner_id: str = "seed-learner-1"
|
||||
messages: list[Message] = field(default_factory=list)
|
||||
|
||||
|
||||
class SessionStore(Protocol):
|
||||
def get(self, session_id: str) -> AgentSession | None: ...
|
||||
def create(
|
||||
self, session_id: str, agent: str, learner_id: str = "seed-learner-1"
|
||||
) -> AgentSession: ...
|
||||
def append(self, session_id: str, message: Message) -> None: ...
|
||||
def history_window(
|
||||
self, session_id: str, max_messages: int = DEFAULT_WINDOW
|
||||
) -> list[Message]: ...
|
||||
def delete(self, session_id: str) -> None: ...
|
||||
|
||||
|
||||
class InMemorySessionStore:
|
||||
"""asyncio.Lock-guarded dict with 20-message windows and 500-cap LRU eviction."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
window: int = DEFAULT_WINDOW,
|
||||
max_sessions: int = DEFAULT_MAX_SESSIONS,
|
||||
) -> None:
|
||||
self._sessions: OrderedDict[str, AgentSession] = OrderedDict()
|
||||
self._lock = asyncio.Lock()
|
||||
self._window = window
|
||||
self._max_sessions = max_sessions
|
||||
|
||||
async def get(self, session_id: str) -> AgentSession | None:
|
||||
async with self._lock:
|
||||
session = self._sessions.get(session_id)
|
||||
if session is not None:
|
||||
self._sessions.move_to_end(session_id) # LRU touch
|
||||
return session
|
||||
|
||||
async def create(
|
||||
self, session_id: str, agent: str, learner_id: str = "seed-learner-1"
|
||||
) -> AgentSession:
|
||||
async with self._lock:
|
||||
session = AgentSession(session_id=session_id, agent=agent, learner_id=learner_id)
|
||||
self._sessions[session_id] = session
|
||||
self._evict_locked()
|
||||
return session
|
||||
|
||||
async def append(self, session_id: str, message: Message) -> None:
|
||||
async with self._lock:
|
||||
session = self._sessions.get(session_id)
|
||||
if session is None:
|
||||
raise KeyError(f"unknown session {session_id!r}")
|
||||
session.messages.append(message)
|
||||
# Bound stored history too (window bounds replay, not storage):
|
||||
# keep at most 2x window so retries/recent context survive.
|
||||
if len(session.messages) > self._window * 2:
|
||||
del session.messages[: len(session.messages) - self._window * 2]
|
||||
self._sessions.move_to_end(session_id)
|
||||
|
||||
async def history_window(
|
||||
self, session_id: str, max_messages: int = DEFAULT_WINDOW
|
||||
) -> list[Message]:
|
||||
async with self._lock:
|
||||
session = self._sessions.get(session_id)
|
||||
if session is None:
|
||||
raise KeyError(f"unknown session {session_id!r}")
|
||||
return list(session.messages[-max_messages:])
|
||||
|
||||
async def delete(self, session_id: str) -> None:
|
||||
async with self._lock:
|
||||
self._sessions.pop(session_id, None)
|
||||
|
||||
def _evict_locked(self) -> None:
|
||||
while len(self._sessions) > self._max_sessions:
|
||||
self._sessions.popitem(last=False) # evict least-recently-used
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Structured output defense — 4 layers (D-020).
|
||||
|
||||
Layer 1: response_format={"type":"json_object"} request (auto-degrades on 400
|
||||
inside the provider).
|
||||
Layer 2: prompt-embedded schema hint ("Respond with ONLY valid JSON...").
|
||||
Layer 3: defensive parse — strip markdown fences, extract first balanced
|
||||
JSON object, pydantic model_validate.
|
||||
Layer 4: single bounded retry with the validation error fed back.
|
||||
"""
|
||||
|
||||
from typing import TypeVar
|
||||
|
||||
from pydantic import BaseModel, ValidationError
|
||||
|
||||
from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
|
||||
T = TypeVar("T", bound=BaseModel)
|
||||
|
||||
|
||||
class StructuredOutputError(Exception):
|
||||
"""Raised when the model output cannot be validated after one retry."""
|
||||
|
||||
|
||||
def extract_json_object(text: str) -> str:
|
||||
"""Strip fences and return the first balanced {...} block from text."""
|
||||
stripped = text.strip()
|
||||
if stripped.startswith("```"):
|
||||
first_newline = stripped.find("\n")
|
||||
if first_newline != -1:
|
||||
stripped = stripped[first_newline + 1:]
|
||||
if stripped.rstrip().endswith("```"):
|
||||
stripped = stripped.rstrip()[:-3]
|
||||
stripped = stripped.strip()
|
||||
start = stripped.find("{")
|
||||
if start == -1:
|
||||
raise StructuredOutputError("no JSON object found in model output")
|
||||
depth = 0
|
||||
in_string = False
|
||||
escape = False
|
||||
for i, ch in enumerate(stripped[start:], start=start):
|
||||
if escape:
|
||||
escape = False
|
||||
continue
|
||||
if ch == "\\":
|
||||
escape = True
|
||||
continue
|
||||
if ch == '"' and not escape:
|
||||
in_string = not in_string
|
||||
continue
|
||||
if in_string:
|
||||
continue
|
||||
if ch == "{":
|
||||
depth += 1
|
||||
elif ch == "}":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return stripped[start:i + 1]
|
||||
raise StructuredOutputError("unbalanced JSON object in model output")
|
||||
|
||||
|
||||
def parse_structured(text: str, schema: type[T]) -> T:
|
||||
"""Layer 3: fence-strip + first-balanced-object + pydantic validation."""
|
||||
candidate = extract_json_object(text)
|
||||
try:
|
||||
return schema.model_validate_json(candidate)
|
||||
except ValidationError as exc:
|
||||
raise StructuredOutputError(f"schema validation failed: {exc}") from exc
|
||||
|
||||
|
||||
def schema_instruction(schema_hint: str) -> str:
|
||||
"""Layer 2: prompt-side schema text."""
|
||||
return (
|
||||
"Respond with ONLY a valid JSON object matching this schema — "
|
||||
"no markdown fences, no prose outside the JSON. "
|
||||
f"Schema: {schema_hint}"
|
||||
)
|
||||
|
||||
|
||||
async def structured_completion(
|
||||
provider: LLMProvider,
|
||||
messages: list[Message],
|
||||
*,
|
||||
model: str,
|
||||
schema: type[T],
|
||||
schema_hint: str,
|
||||
retry_feedback: str | None = None,
|
||||
) -> T:
|
||||
"""Full 4-layer pipeline. One bounded retry (layer 4), then raise."""
|
||||
# Build request: append schema instruction to the last user message (layer 2).
|
||||
request = list(messages)
|
||||
last_user = next((m for m in reversed(request) if m.role == "user"), None)
|
||||
if last_user is not None:
|
||||
request = [
|
||||
Message(role=m.role, content=(m.content + "\n\n" + schema_instruction(schema_hint)))
|
||||
if m is last_user else m
|
||||
for m in request
|
||||
]
|
||||
response_format = {"type": "json_object"}
|
||||
raw = await provider.chat(request, model=model, response_format=response_format)
|
||||
try:
|
||||
return parse_structured(raw, schema) # layers 1+2+3
|
||||
except StructuredOutputError as exc:
|
||||
# Layer 4: single bounded retry with error feedback
|
||||
retry_prompt = (
|
||||
f"Your previous response was invalid: {exc}. "
|
||||
f"Return ONLY the corrected JSON matching: {schema_hint}"
|
||||
)
|
||||
request2 = list(messages)
|
||||
request2.append(Message(role="user", content=retry_prompt))
|
||||
raw2 = await provider.chat(request2, model=model, response_format=response_format)
|
||||
try:
|
||||
return parse_structured(raw2, schema)
|
||||
except StructuredOutputError as exc2:
|
||||
raise StructuredOutputError(
|
||||
f"structured output failed after retry: {exc2}"
|
||||
) from exc2
|
||||
@@ -0,0 +1,13 @@
|
||||
"""TutorAgent — concept delivery, Socratic questioning (REQ-2-006)."""
|
||||
|
||||
from ..corpus.learner_context import LearnerContext, get_learner_context
|
||||
from ..prompts.tutor import SYSTEM_PROMPT, render_context
|
||||
from .base import BaseAgent
|
||||
|
||||
|
||||
class TutorAgent(BaseAgent):
|
||||
name = "tutor"
|
||||
|
||||
def system_prompt(self, learner_context: LearnerContext | None = None) -> str:
|
||||
ctx = learner_context or get_learner_context()
|
||||
return SYSTEM_PROMPT.format_map(render_context(ctx))
|
||||
@@ -0,0 +1,18 @@
|
||||
"""API package — composes providers, sessions, and agents via DI.
|
||||
|
||||
Boundary rule: api/ composes agents/ and llm/; they never import api/.
|
||||
"""
|
||||
|
||||
from .assessment import router as assessment_router
|
||||
from .chat import router as chat_router
|
||||
from .lab import router as lab_router
|
||||
from .mentor import router as mentor_router
|
||||
from .proctor import router as proctor_router
|
||||
|
||||
__all__ = [
|
||||
"assessment_router",
|
||||
"chat_router",
|
||||
"lab_router",
|
||||
"mentor_router",
|
||||
"proctor_router",
|
||||
]
|
||||
@@ -0,0 +1,47 @@
|
||||
"""POST /v1/assessment/evaluate — structured rubric scores (REQ-2-008).
|
||||
|
||||
JSON response (not SSE): a pydantic-validated RubricScore. Unknown
|
||||
artifact → 404. The Assessor's structured output IS the payload.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..agents.assessor import RubricScore
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..config import Settings
|
||||
from ..corpus.artifacts import get_artifact_bundle, get_transcript_for_artifact
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from .deps import get_agent_registry, get_provider, get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class AssessmentRequest(BaseModel):
|
||||
artifact_id: str = Field(min_length=1)
|
||||
learner_id: str | None = None
|
||||
|
||||
|
||||
@router.post("/assessment/evaluate", response_model=RubricScore)
|
||||
async def assessment_evaluate(
|
||||
body: AssessmentRequest,
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
) -> RubricScore:
|
||||
bundle = get_artifact_bundle(body.artifact_id)
|
||||
if bundle is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown artifact {body.artifact_id!r}"
|
||||
)
|
||||
artifact, rubric = bundle
|
||||
transcript = get_transcript_for_artifact(artifact.artifact_id)
|
||||
agent = registry.get(provider, settings, "assessor")
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
try:
|
||||
return await agent.evaluate(artifact, rubric, transcript, learner_context)
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=502,
|
||||
detail=f"assessment evaluation failed: {exc}",
|
||||
) from exc
|
||||
@@ -0,0 +1,128 @@
|
||||
"""POST /v1/chat/stream — SSE chat with the D-016 envelope + agent routing.
|
||||
|
||||
Envelope: meta event first (flushed before first token), then raw content
|
||||
deltas, then done; error event before [DONE] on mid-stream failure.
|
||||
Pre-first-byte provider failures surface as in-band `provider_unavailable`
|
||||
error events (SSE 200 headers are already committed once meta flushes).
|
||||
|
||||
Agent routing (A-007): the request names its agent; unknown agents are
|
||||
rejected with 422. No autonomous routing in v0.2.
|
||||
"""
|
||||
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
from sse_starlette.sse import EventSourceResponse
|
||||
|
||||
from ..agents.registry import AgentRegistry, UnknownAgentError
|
||||
from ..agents.session import SessionStore
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from ..llm.base import LLMProvider
|
||||
from ..llm.types import Message
|
||||
from .deps import get_agent_registry, get_provider, get_session_store, get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class ChatStreamRequest(BaseModel):
|
||||
agent: str = Field(min_length=1)
|
||||
session_id: str = Field(min_length=1)
|
||||
learner_id: str | None = None
|
||||
messages: list[Message] = Field(min_length=1)
|
||||
|
||||
|
||||
@router.post("/chat/stream")
|
||||
async def chat_stream(
|
||||
body: ChatStreamRequest,
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
sessions: SessionStore = Depends(get_session_store),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider: LLMProvider = Depends(get_provider),
|
||||
) -> EventSourceResponse:
|
||||
# Route to the named agent (A-007); unknown → 422 before any streaming.
|
||||
try:
|
||||
agent = registry.get(provider, settings, body.agent)
|
||||
except UnknownAgentError as exc:
|
||||
raise HTTPException(
|
||||
status_code=422, detail=str(exc)
|
||||
) from None
|
||||
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
|
||||
# Agent-scoped session (A-007/G-4): persisted turn history, windowed replay.
|
||||
session = await sessions.get(body.session_id)
|
||||
if session is None:
|
||||
session = await sessions.create(
|
||||
body.session_id, agent=body.agent, learner_id=body.learner_id or "learner-001"
|
||||
)
|
||||
# The new user turn is the last message of the request.
|
||||
user_turn = body.messages[-1]
|
||||
history = await sessions.history_window(body.session_id)
|
||||
# Retry dedupe (P1 from final review): a client retry resends the same
|
||||
# turn after a provider failure — don't double-append it to history.
|
||||
last_stored = history[-1] if history else None
|
||||
is_retry = (
|
||||
last_stored is not None
|
||||
and last_stored.role == "user"
|
||||
and last_stored.content == user_turn.content
|
||||
)
|
||||
if not is_retry:
|
||||
await sessions.append(body.session_id, user_turn)
|
||||
else:
|
||||
# On retry the history replay should exclude the stored duplicate.
|
||||
history = history[:-1]
|
||||
|
||||
async def event_stream() -> AsyncIterator[dict]:
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "meta",
|
||||
"agent": body.agent,
|
||||
"session_id": body.session_id,
|
||||
"model": settings.model,
|
||||
})}
|
||||
first_byte = True
|
||||
reply_parts: list[str] = []
|
||||
try:
|
||||
async for token in agent.stream_reply(
|
||||
history=history,
|
||||
user_input=user_turn.content,
|
||||
learner_context=learner_context,
|
||||
):
|
||||
first_byte = False
|
||||
reply_parts.append(token)
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "delta", "content": token
|
||||
})}
|
||||
full_reply = "".join(reply_parts)
|
||||
if full_reply:
|
||||
await sessions.append(
|
||||
body.session_id, Message(role="assistant", content=full_reply)
|
||||
)
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "done", "finish_reason": "stop"
|
||||
})}
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
except Exception as exc: # CancelledError is BaseException — passes through
|
||||
message = str(exc)
|
||||
if first_byte:
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "error", "code": "provider_unavailable", "message": message
|
||||
})}
|
||||
else:
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "error", "code": "provider_error", "message": message
|
||||
})}
|
||||
# [DONE] is yielded from the except branch, NEVER from finally:
|
||||
# a yield inside finally would re-raise after GeneratorExit when the
|
||||
# client disconnects ("async generator ignored GeneratorExit").
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
|
||||
return EventSourceResponse(
|
||||
event_stream(),
|
||||
headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"X-Accel-Buffering": "no",
|
||||
},
|
||||
)
|
||||
@@ -0,0 +1,24 @@
|
||||
"""FastAPI dependencies — provider, settings, sessions, agents via app.state (DI)."""
|
||||
|
||||
from fastapi import Request
|
||||
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..agents.session import SessionStore
|
||||
from ..config import Settings
|
||||
from ..llm.base import LLMProvider
|
||||
|
||||
|
||||
def get_settings(request: Request) -> Settings:
|
||||
return request.app.state.settings
|
||||
|
||||
|
||||
def get_provider(request: Request) -> LLMProvider:
|
||||
return request.app.state.provider
|
||||
|
||||
|
||||
def get_session_store(request: Request) -> SessionStore:
|
||||
return request.app.state.session_store
|
||||
|
||||
|
||||
def get_agent_registry(request: Request) -> AgentRegistry:
|
||||
return request.app.state.agent_registry
|
||||
@@ -0,0 +1,72 @@
|
||||
"""POST /v1/lab/feedback — SSE stream of Lab in-flow feedback (REQ-2-007).
|
||||
|
||||
D-016 envelope with agent=lab. Unknown scenario → 404 before streaming.
|
||||
"""
|
||||
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
from sse_starlette.sse import EventSourceResponse
|
||||
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from ..corpus.telemetry import get_lab_scenario
|
||||
from .deps import get_agent_registry, get_provider, get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class LabFeedbackRequest(BaseModel):
|
||||
scenario_id: str = Field(min_length=1)
|
||||
learner_id: str | None = None
|
||||
|
||||
|
||||
@router.post("/lab/feedback")
|
||||
async def lab_feedback(
|
||||
body: LabFeedbackRequest,
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
) -> EventSourceResponse:
|
||||
scenario = get_lab_scenario(body.scenario_id)
|
||||
if scenario is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown scenario {body.scenario_id!r}"
|
||||
)
|
||||
agent = registry.get(provider, settings, "lab")
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
|
||||
async def event_stream() -> AsyncIterator[dict]:
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "meta",
|
||||
"agent": "lab",
|
||||
"scenario_id": body.scenario_id,
|
||||
"model": settings.model,
|
||||
})}
|
||||
first_byte = True
|
||||
try:
|
||||
async for token in agent.stream_feedback(scenario, learner_context):
|
||||
first_byte = False
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "delta", "content": token
|
||||
})}
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "done", "finish_reason": "stop"
|
||||
})}
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
except Exception as exc:
|
||||
code = "provider_unavailable" if first_byte else "provider_error"
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "error", "code": code, "message": str(exc)
|
||||
})}
|
||||
# [DONE] from except, not finally — a yield in finally would
|
||||
# re-raise after GeneratorExit on client disconnect.
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
|
||||
return EventSourceResponse(
|
||||
event_stream(),
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
@@ -0,0 +1,95 @@
|
||||
"""POST /v1/mentor/narrative — SSE career narrative stream (REQ-2-010).
|
||||
|
||||
D-016 envelope with agent=mentor. Session-backed: the client supplies a
|
||||
session_id; the Mentor keeps conversation context across follow-ups.
|
||||
"""
|
||||
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
from sse_starlette.sse import EventSourceResponse
|
||||
|
||||
from ..agents.registry import AgentRegistry, UnknownAgentError
|
||||
from ..agents.session import SessionStore
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from ..llm.types import Message
|
||||
from .deps import get_agent_registry, get_provider, get_session_store, get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class MentorNarrativeRequest(BaseModel):
|
||||
session_id: str = Field(min_length=1)
|
||||
prompt: str = Field(default="Narrate my trajectory.")
|
||||
learner_id: str | None = None
|
||||
|
||||
|
||||
@router.post("/mentor/narrative")
|
||||
async def mentor_narrative(
|
||||
body: MentorNarrativeRequest,
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
sessions: SessionStore = Depends(get_session_store),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
) -> EventSourceResponse:
|
||||
try:
|
||||
agent = registry.get(provider, settings, "mentor")
|
||||
except UnknownAgentError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from None
|
||||
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
|
||||
session = await sessions.get(body.session_id)
|
||||
if session is None:
|
||||
session = await sessions.create(
|
||||
body.session_id, agent="mentor", learner_id=body.learner_id or "learner-001"
|
||||
)
|
||||
history = await sessions.history_window(body.session_id)
|
||||
user_message = Message(role="user", content=body.prompt)
|
||||
await sessions.append(body.session_id, user_message)
|
||||
|
||||
async def event_stream() -> AsyncIterator[dict]:
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "meta",
|
||||
"agent": "mentor",
|
||||
"session_id": body.session_id,
|
||||
"model": settings.model,
|
||||
})}
|
||||
first_byte = True
|
||||
reply_parts: list[str] = []
|
||||
try:
|
||||
async for token in agent.stream_reply(
|
||||
history=history,
|
||||
user_input=body.prompt,
|
||||
learner_context=learner_context,
|
||||
):
|
||||
first_byte = False
|
||||
reply_parts.append(token)
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "delta", "content": token
|
||||
})}
|
||||
full_reply = "".join(reply_parts)
|
||||
if full_reply:
|
||||
await sessions.append(
|
||||
body.session_id, Message(role="assistant", content=full_reply)
|
||||
)
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "done", "finish_reason": "stop"
|
||||
})}
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
except Exception as exc:
|
||||
code = "provider_unavailable" if first_byte else "provider_error"
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "error", "code": code, "message": str(exc)
|
||||
})}
|
||||
# [DONE] from except, not finally — a yield in finally would
|
||||
# re-raise after GeneratorExit on client disconnect.
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
|
||||
return EventSourceResponse(
|
||||
event_stream(),
|
||||
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
|
||||
)
|
||||
@@ -0,0 +1,44 @@
|
||||
"""POST /v1/proctor/signals — structured integrity signals (REQ-2-009).
|
||||
|
||||
JSON response (not SSE): a pydantic-validated ProctorAssessment.
|
||||
Unknown scenario → 404. Coaching-shaped interventions only.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..agents.proctor import ProctorAssessment
|
||||
from ..agents.registry import AgentRegistry
|
||||
from ..config import Settings
|
||||
from ..corpus.learner_context import get_learner_context
|
||||
from ..corpus.telemetry import get_proctor_scenario
|
||||
from .deps import get_agent_registry, get_provider, get_settings
|
||||
|
||||
router = APIRouter(prefix="/v1")
|
||||
|
||||
|
||||
class ProctorRequest(BaseModel):
|
||||
scenario_id: str = Field(min_length=1)
|
||||
learner_id: str | None = None
|
||||
|
||||
|
||||
@router.post("/proctor/signals", response_model=ProctorAssessment)
|
||||
async def proctor_signals(
|
||||
body: ProctorRequest,
|
||||
registry: AgentRegistry = Depends(get_agent_registry),
|
||||
settings: Settings = Depends(get_settings),
|
||||
provider=Depends(get_provider),
|
||||
) -> ProctorAssessment:
|
||||
scenario = get_proctor_scenario(body.scenario_id)
|
||||
if scenario is None:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"unknown scenario {body.scenario_id!r}"
|
||||
)
|
||||
agent = registry.get(provider, settings, "proctor")
|
||||
learner_context = get_learner_context(body.learner_id)
|
||||
try:
|
||||
return await agent.assess(scenario, learner_context)
|
||||
except Exception as exc:
|
||||
raise HTTPException(
|
||||
status_code=502, detail=f"proctor assessment failed: {exc}"
|
||||
) from exc
|
||||
@@ -0,0 +1,18 @@
|
||||
"""Service settings — pydantic-settings, env prefix AI_, .env support."""
|
||||
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_prefix="AI_", env_file=".env", extra="ignore")
|
||||
|
||||
port: int = 8420
|
||||
provider: str = "mock"
|
||||
model: str = "gemma4:31b"
|
||||
|
||||
ollama_cloud_base_url: str = "https://ollama.com/v1"
|
||||
ollama_cloud_api_key: str = "" # SecretStr adds friction here; never logged, never echoed
|
||||
local_base_url: str = "http://localhost:11434/v1"
|
||||
|
||||
# "auto" sends response_format and degrades on 400; "off" never sends it
|
||||
json_mode: str = "auto"
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Mock engine inputs — pydantic-typed corpus (D-021).
|
||||
|
||||
Convention-aligned with the TS `packages/mock-data` layer: identical ID
|
||||
strings (stack-*, comp-*, learner-*, art-*, mc-*), cross-referenced by the
|
||||
counterpart files. No codegen in v0.2 — alignment is by documented
|
||||
convention; revisit codegen only if drift bites (v0.3).
|
||||
"""
|
||||
|
||||
from .learner_context import LEARNER_CONTEXTS, LearnerContext, get_learner_context
|
||||
|
||||
__all__ = [
|
||||
"LEARNER_CONTEXTS",
|
||||
"LearnerContext",
|
||||
"get_learner_context",
|
||||
]
|
||||
@@ -0,0 +1,209 @@
|
||||
"""Pre-baked artifacts + rubrics + defense transcripts — Assessor mock inputs (REQ-2-008).
|
||||
|
||||
Counterpart: packages/mock-data/ai-scenarios.ts (artifact IDs string-identical,
|
||||
D-021). Real process-trace grading is a v0.3+ engine (assessment engine);
|
||||
these pre-baked submissions stand in for artifact + defense evaluation.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class RubricCriterion(BaseModel):
|
||||
criterion_id: str
|
||||
name: str
|
||||
weight: float
|
||||
description: str
|
||||
|
||||
|
||||
class AssessmentRubric(BaseModel):
|
||||
rubric_id: str
|
||||
competency_id: str
|
||||
criteria: list[RubricCriterion]
|
||||
|
||||
|
||||
class ArtifactSubmission(BaseModel):
|
||||
artifact_id: str
|
||||
name: str
|
||||
artifact_type: str # "code" | "design" | "simulation"
|
||||
competency_id: str
|
||||
description: str
|
||||
evidence_excerpt: str # what the grader sees of the artifact itself
|
||||
|
||||
|
||||
class DefenseTranscript(BaseModel):
|
||||
transcript_id: str
|
||||
artifact_id: str
|
||||
turns: list[dict] # {"speaker": "examiner"|"learner", "text": "..."}
|
||||
|
||||
|
||||
_RUBRIC_ORCHESTRATION = AssessmentRubric(
|
||||
rubric_id="rubric-orchestration-c002",
|
||||
competency_id="stack-orchestration-c002",
|
||||
criteria=[
|
||||
RubricCriterion(
|
||||
criterion_id="rc-architecture",
|
||||
name="Agent architecture soundness",
|
||||
weight=0.3,
|
||||
description="State boundaries and responsibilities are clearly separated",
|
||||
),
|
||||
RubricCriterion(
|
||||
criterion_id="rc-communication",
|
||||
name="Inter-agent communication design",
|
||||
weight=0.3,
|
||||
description="Message contracts are explicit, typed, and failure-aware",
|
||||
),
|
||||
RubricCriterion(
|
||||
criterion_id="rc-reliability",
|
||||
name="Reliability engineering",
|
||||
weight=0.25,
|
||||
description="Retries, timeouts, and degradation paths handled",
|
||||
),
|
||||
RubricCriterion(
|
||||
criterion_id="rc-process",
|
||||
name="Process trace quality",
|
||||
weight=0.15,
|
||||
description="Telemetry shows iterative building with real checkpoints",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
_RUBRIC_TOOL_USE = AssessmentRubric(
|
||||
rubric_id="rubric-orchestration-c003",
|
||||
competency_id="stack-orchestration-c003",
|
||||
criteria=[
|
||||
RubricCriterion(
|
||||
criterion_id="rc-eval-design",
|
||||
name="Evaluation design rigor",
|
||||
weight=0.35,
|
||||
description="Hypotheses, controls, and metrics are explicit and defensible",
|
||||
),
|
||||
RubricCriterion(
|
||||
criterion_id="rc-eval-robustness",
|
||||
name="Harness robustness",
|
||||
weight=0.35,
|
||||
description="Error handling, variance awareness, and reproducibility",
|
||||
),
|
||||
RubricCriterion(
|
||||
criterion_id="rc-eval-insight",
|
||||
name="Insight extraction",
|
||||
weight=0.3,
|
||||
description="Results are interpreted into concrete engineering decisions",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
_ARTIFACT_RESEARCH_ASSISTANT = ArtifactSubmission(
|
||||
artifact_id="art-eval-research-assistant",
|
||||
name="Multi-agent research assistant (eval build)",
|
||||
artifact_type="code",
|
||||
competency_id="stack-orchestration-c002",
|
||||
description=(
|
||||
"LangGraph-based assistant planning, retrieving, drafting cited reviews."
|
||||
),
|
||||
evidence_excerpt=(
|
||||
"planner.py defines state schema with explicit fields "
|
||||
"(plan, findings, draft); tool_node.py wraps retrieval with a "
|
||||
"3-retry loop and typed ToolMessage responses; tests cover "
|
||||
"planner->tool->writer handoffs; README shows graph diagram"
|
||||
),
|
||||
)
|
||||
|
||||
_TRANSCRIPT_RESEARCH_ASSISTANT = DefenseTranscript(
|
||||
transcript_id="defense-art-eval-research-assistant",
|
||||
artifact_id="art-eval-research-assistant",
|
||||
turns=[
|
||||
{"speaker": "examiner",
|
||||
"text": "Why did you give the planner sole write access to the plan field?"},
|
||||
{"speaker": "learner",
|
||||
"text": "So worker nodes can't mutate each other's inputs — "
|
||||
"the state stays predictable and the graph is debuggable"},
|
||||
{"speaker": "examiner",
|
||||
"text": "What happens when the retrieval tool times out three times?"},
|
||||
{"speaker": "learner",
|
||||
"text": "The tool node degrades to a no-op ToolMessage with a "
|
||||
"retry flag so the writer can fall back to existing findings"},
|
||||
{"speaker": "examiner", "text": "How would you extend this to a third agent?"},
|
||||
{"speaker": "learner",
|
||||
"text": "Add a reviewer node with its own typed messages, same pattern"},
|
||||
],
|
||||
)
|
||||
|
||||
_ARTIFACT_RAG_DASHBOARD = ArtifactSubmission(
|
||||
artifact_id="art-eval-rag-dashboard",
|
||||
name="RAG retrieval quality dashboard (eval build)",
|
||||
artifact_type="code",
|
||||
competency_id="stack-orchestration-c003",
|
||||
description="Dashboard comparing chunking strategies/rerankers across 800 queries.",
|
||||
evidence_excerpt=(
|
||||
"eval harness sweeps 4 chunk sizes x 3 rerankers; results table auto-generated; "
|
||||
"no error handling on the query loader; tests only cover the happy path"
|
||||
),
|
||||
)
|
||||
|
||||
_TRANSCRIPT_RAG_DASHBOARD = DefenseTranscript(
|
||||
transcript_id="defense-art-eval-rag-dashboard",
|
||||
artifact_id="art-eval-rag-dashboard",
|
||||
turns=[
|
||||
{"speaker": "examiner", "text": "How did you control for query difficulty across runs?"},
|
||||
{"speaker": "learner", "text": "I, um, used the same query set each time"},
|
||||
{"speaker": "examiner", "text": "What happens if the query loader hits a malformed row?"},
|
||||
{"speaker": "learner", "text": "I didn't handle that. It would probably crash."},
|
||||
{"speaker": "examiner", "text": "What would you improve first?"},
|
||||
{"speaker": "learner",
|
||||
"text": "Probably add the error handling, then look at variance between runs"},
|
||||
],
|
||||
)
|
||||
|
||||
RUBRICS: dict[str, AssessmentRubric] = {
|
||||
_RUBRIC_ORCHESTRATION.rubric_id: _RUBRIC_ORCHESTRATION,
|
||||
_RUBRIC_TOOL_USE.rubric_id: _RUBRIC_TOOL_USE,
|
||||
}
|
||||
|
||||
ARTIFACTS: dict[str, ArtifactSubmission] = {
|
||||
a.artifact_id: a
|
||||
for a in (_ARTIFACT_RESEARCH_ASSISTANT, _ARTIFACT_RAG_DASHBOARD)
|
||||
}
|
||||
|
||||
TRANSCRIPTS: dict[str, DefenseTranscript] = {
|
||||
t.transcript_id: t
|
||||
for t in (_TRANSCRIPT_RESEARCH_ASSISTANT, _TRANSCRIPT_RAG_DASHBOARD)
|
||||
}
|
||||
|
||||
|
||||
def rubric_for_competency(competency_id: str) -> AssessmentRubric | None:
|
||||
for rubric in RUBRICS.values():
|
||||
if rubric.competency_id == competency_id:
|
||||
return rubric
|
||||
return None
|
||||
|
||||
|
||||
def get_artifact_bundle(artifact_id: str) -> tuple[ArtifactSubmission, AssessmentRubric] | None:
|
||||
"""Resolve (artifact, rubric) for an artifact ID; None if unknown."""
|
||||
artifact = ARTIFACTS.get(artifact_id)
|
||||
if artifact is None:
|
||||
return None
|
||||
rubric = rubric_for_competency(artifact.competency_id)
|
||||
if rubric is None:
|
||||
return None
|
||||
return artifact, rubric
|
||||
|
||||
|
||||
def get_transcript_for_artifact(artifact_id: str) -> DefenseTranscript | None:
|
||||
for transcript in TRANSCRIPTS.values():
|
||||
if transcript.artifact_id == artifact_id:
|
||||
return transcript
|
||||
return None
|
||||
|
||||
|
||||
def render_rubric(rubric: AssessmentRubric) -> str:
|
||||
lines = [f"Rubric: {rubric.rubric_id} (competency {rubric.competency_id})"]
|
||||
for c in rubric.criteria:
|
||||
lines.append(f"- {c.criterion_id} ({c.weight:.2f}): {c.name} — {c.description}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_transcript(transcript: DefenseTranscript) -> str:
|
||||
lines = [f"Defense transcript: {transcript.transcript_id}"]
|
||||
for turn in transcript.turns:
|
||||
lines.append(f"{turn['speaker']}: {turn['text']}")
|
||||
return "\n".join(lines)
|
||||
@@ -0,0 +1,108 @@
|
||||
"""Learner context corpus — pydantic mirror of TS learner-progress.ts (D-021).
|
||||
|
||||
Counterpart: packages/mock-data/src/learner-progress.ts (or learner-progress.ts
|
||||
at package root). IDs are string-identical: learner-001, stack-orchestration,
|
||||
stack-safety, stack-orchestration-c00N, art-*, mc-*.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class CompetencyProgress(BaseModel):
|
||||
competency_id: str
|
||||
title: str
|
||||
status: str # "mastered" | "in_progress" | "not_started"
|
||||
|
||||
|
||||
class StackProgress(BaseModel):
|
||||
stack_id: str
|
||||
title: str
|
||||
percent: int
|
||||
|
||||
|
||||
class LearnerContext(BaseModel):
|
||||
learner_id: str
|
||||
name: str
|
||||
active_stacks: list[StackProgress]
|
||||
active_competencies: list[CompetencyProgress]
|
||||
microcredential_count: int
|
||||
recent_artifacts: list[str] # artifact names
|
||||
|
||||
|
||||
_STACK_ORCHESTRATION = StackProgress(
|
||||
stack_id="stack-orchestration", title="AI Orchestration Engineer", percent=62
|
||||
)
|
||||
_STACK_SAFETY = StackProgress(
|
||||
stack_id="stack-safety", title="AI Safety & Governance Lead", percent=41
|
||||
)
|
||||
|
||||
_LEARNER_1 = LearnerContext(
|
||||
learner_id="learner-001",
|
||||
name="Alex Rivera",
|
||||
active_stacks=[_STACK_ORCHESTRATION, _STACK_SAFETY],
|
||||
active_competencies=[
|
||||
CompetencyProgress(
|
||||
competency_id="stack-orchestration-c001",
|
||||
title="Agent architecture fundamentals",
|
||||
status="mastered",
|
||||
),
|
||||
CompetencyProgress(
|
||||
competency_id="stack-orchestration-c002",
|
||||
title="Multi-agent communication patterns",
|
||||
status="in_progress",
|
||||
),
|
||||
CompetencyProgress(
|
||||
competency_id="stack-orchestration-c003",
|
||||
title="Tool use and function calling",
|
||||
status="in_progress",
|
||||
),
|
||||
CompetencyProgress(
|
||||
competency_id="stack-safety-c021",
|
||||
title="Red-team basics for agent systems",
|
||||
status="in_progress",
|
||||
),
|
||||
],
|
||||
microcredential_count=4,
|
||||
recent_artifacts=[
|
||||
"Multi-agent research assistant",
|
||||
"RAG retrieval quality dashboard",
|
||||
],
|
||||
)
|
||||
|
||||
_LEARNER_2 = LearnerContext(
|
||||
learner_id="learner-002",
|
||||
name="Priya Chen",
|
||||
active_stacks=[
|
||||
StackProgress(
|
||||
stack_id="stack-designer", title="Human-AI Product Designer", percent=55
|
||||
),
|
||||
],
|
||||
active_competencies=[
|
||||
CompetencyProgress(
|
||||
competency_id="stack-designer-c001",
|
||||
title="Prompt-to-prototype workflows",
|
||||
status="mastered",
|
||||
),
|
||||
CompetencyProgress(
|
||||
competency_id="stack-designer-c002",
|
||||
title="Evaluating AI UX patterns",
|
||||
status="in_progress",
|
||||
),
|
||||
],
|
||||
microcredential_count=2,
|
||||
recent_artifacts=["AI onboarding flow concept test"],
|
||||
)
|
||||
|
||||
LEARNER_CONTEXTS: dict[str, LearnerContext] = {
|
||||
_LEARNER_1.learner_id: _LEARNER_1,
|
||||
_LEARNER_2.learner_id: _LEARNER_2,
|
||||
}
|
||||
|
||||
DEFAULT_LEARNER_ID = "learner-001"
|
||||
|
||||
|
||||
def get_learner_context(learner_id: str | None = None) -> LearnerContext:
|
||||
"""Resolve a learner context by ID, falling back to the default seed."""
|
||||
if learner_id is None:
|
||||
return LEARNER_CONTEXTS[DEFAULT_LEARNER_ID]
|
||||
return LEARNER_CONTEXTS.get(learner_id, LEARNER_CONTEXTS[DEFAULT_LEARNER_ID])
|
||||
@@ -0,0 +1,178 @@
|
||||
"""Simulated sandbox telemetry corpus — Lab agent mock engine inputs (REQ-2-007).
|
||||
|
||||
Counterpart: packages/mock-data/ai-scenarios.ts (scenario IDs string-identical,
|
||||
D-021). Real sandbox telemetry is a v0.3+ engine (sandbox fabric); these
|
||||
scripted event streams stand in for the build-session process trace.
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class TelemetryEvent(BaseModel):
|
||||
timestamp: int # seconds since session start
|
||||
kind: str # "keystroke_burst" | "file_save" | "run_tests" | "test_pass"
|
||||
# | "test_fail" | "console_error" | "idle" | "paste" | "commit"
|
||||
|
||||
|
||||
detail: str = ""
|
||||
|
||||
|
||||
class LabTelemetryScenario(BaseModel):
|
||||
scenario_id: str
|
||||
title: str
|
||||
competency_id: str
|
||||
events: list[TelemetryEvent]
|
||||
|
||||
|
||||
class ProctorEvent(BaseModel):
|
||||
timestamp: int # seconds since session start
|
||||
kind: str # "tab_switch" | "idle" | "paste_large" | "focus_lost" | "keystroke_burst"
|
||||
detail: str = ""
|
||||
|
||||
|
||||
class ProctorScenario(BaseModel):
|
||||
scenario_id: str
|
||||
title: str
|
||||
competency_id: str
|
||||
events: list[ProctorEvent]
|
||||
|
||||
|
||||
_PROCTOR_SCENARIO_HEALTHY = ProctorScenario(
|
||||
scenario_id="proctor-scenario-healthy",
|
||||
title="Healthy defense session — focused throughout",
|
||||
competency_id="stack-orchestration-c002",
|
||||
events=[
|
||||
ProctorEvent(timestamp=0, kind="keystroke_burst", detail="session begins"),
|
||||
ProctorEvent(timestamp=310, kind="keystroke_burst", detail="long answer in progress"),
|
||||
ProctorEvent(timestamp=640, kind="keystroke_burst", detail="revision pass"),
|
||||
ProctorEvent(timestamp=900, kind="keystroke_burst", detail="final answer"),
|
||||
],
|
||||
)
|
||||
|
||||
_PROCTOR_SCENARIO_DISTRACTED = ProctorScenario(
|
||||
scenario_id="proctor-scenario-distracted",
|
||||
title="Distracted defense session — tab switches and idle gaps",
|
||||
competency_id="stack-orchestration-c002",
|
||||
events=[
|
||||
ProctorEvent(timestamp=0, kind="keystroke_burst", detail="session begins"),
|
||||
ProctorEvent(timestamp=120, kind="tab_switch", detail="to docs.nextjs.org"),
|
||||
ProctorEvent(timestamp=125, kind="focus_lost", detail="window blur 40s"),
|
||||
ProctorEvent(timestamp=300, kind="idle", detail="no activity for 5 minutes"),
|
||||
ProctorEvent(timestamp=600, kind="tab_switch", detail="to github.com"),
|
||||
ProctorEvent(timestamp=605, kind="focus_lost", detail="window blur 2m"),
|
||||
ProctorEvent(timestamp=720, kind="keystroke_burst", detail="resumes typing"),
|
||||
],
|
||||
)
|
||||
|
||||
_PROCTOR_SCENARIO_FLAGGED = ProctorScenario(
|
||||
scenario_id="proctor-scenario-flagged",
|
||||
title="Flagged defense session — large paste during exam",
|
||||
competency_id="stack-orchestration-c003",
|
||||
events=[
|
||||
ProctorEvent(timestamp=0, kind="keystroke_burst", detail="short intro typed"),
|
||||
ProctorEvent(timestamp=85, kind="paste_large", detail="3,100 chars pasted in 2s"),
|
||||
ProctorEvent(timestamp=90, kind="idle", detail="no activity for 4 minutes"),
|
||||
ProctorEvent(timestamp=330, kind="paste_large", detail="2,800 chars pasted in 2s"),
|
||||
],
|
||||
)
|
||||
|
||||
PROCTOR_SCENARIOS: dict[str, ProctorScenario] = {
|
||||
s.scenario_id: s
|
||||
for s in (
|
||||
_PROCTOR_SCENARIO_HEALTHY,
|
||||
_PROCTOR_SCENARIO_DISTRACTED,
|
||||
_PROCTOR_SCENARIO_FLAGGED,
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
def get_proctor_scenario(scenario_id: str) -> ProctorScenario | None:
|
||||
return PROCTOR_SCENARIOS.get(scenario_id)
|
||||
|
||||
|
||||
def summarize_proctor_scenario(scenario: ProctorScenario) -> str:
|
||||
"""Render the proctor event timeline as compact text for prompt injection."""
|
||||
lines = [f"Defense session: {scenario.title} (competency {scenario.competency_id})"]
|
||||
for event in scenario.events:
|
||||
lines.append(f"t+{event.timestamp}s {event.kind}: {event.detail}".rstrip(": "))
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
_LAB_SCENARIO_STRONG = LabTelemetryScenario(
|
||||
scenario_id="lab-scenario-strong",
|
||||
title="Strong build session — multi-agent research assistant",
|
||||
competency_id="stack-orchestration-c002",
|
||||
events=[
|
||||
TelemetryEvent(timestamp=0, kind="keystroke_burst", detail="planner.py"),
|
||||
TelemetryEvent(timestamp=95, kind="file_save", detail="planner.py"),
|
||||
TelemetryEvent(timestamp=120, kind="run_tests", detail="3 tests"),
|
||||
TelemetryEvent(timestamp=126, kind="test_pass",
|
||||
detail="3/3 passed"),
|
||||
TelemetryEvent(timestamp=180, kind="keystroke_burst", detail="tool_node.py"),
|
||||
TelemetryEvent(timestamp=260, kind="file_save", detail="tool_node.py"),
|
||||
TelemetryEvent(timestamp=275, kind="run_tests", detail="4 tests"),
|
||||
TelemetryEvent(timestamp=281, kind="test_pass",
|
||||
detail="4/4 passed"),
|
||||
TelemetryEvent(timestamp=340, kind="commit",
|
||||
detail="add tool node with retries"),
|
||||
],
|
||||
)
|
||||
|
||||
_LAB_SCENARIO_STRUGGLING = LabTelemetryScenario(
|
||||
scenario_id="lab-scenario-struggling",
|
||||
title="Struggling build session — repeated failures, no checkpoints",
|
||||
competency_id="stack-orchestration-c002",
|
||||
events=[
|
||||
TelemetryEvent(timestamp=0, kind="keystroke_burst", detail="main.py"),
|
||||
TelemetryEvent(timestamp=210, kind="run_tests", detail="2 tests"),
|
||||
TelemetryEvent(timestamp=215, kind="test_fail",
|
||||
detail="ImportError: no module named 'tools'"),
|
||||
TelemetryEvent(timestamp=216, kind="console_error", detail="traceback dumped"),
|
||||
TelemetryEvent(timestamp=300, kind="keystroke_burst",
|
||||
detail="main.py"),
|
||||
TelemetryEvent(timestamp=520, kind="run_tests",
|
||||
detail="2 tests"),
|
||||
TelemetryEvent(timestamp=525, kind="test_fail",
|
||||
detail="ImportError: no module named 'tools'"),
|
||||
TelemetryEvent(timestamp=526, kind="console_error",
|
||||
detail="same traceback as before"),
|
||||
TelemetryEvent(timestamp=600, kind="idle",
|
||||
detail="no activity for 6 minutes"),
|
||||
TelemetryEvent(timestamp=960, kind="idle",
|
||||
detail="no activity for 14 minutes"),
|
||||
],
|
||||
)
|
||||
|
||||
_LAB_SCENARIO_FLAGGED = LabTelemetryScenario(
|
||||
scenario_id="lab-scenario-flagged",
|
||||
title="Flagged build session — large paste, instant pass",
|
||||
competency_id="stack-orchestration-c003",
|
||||
events=[
|
||||
TelemetryEvent(timestamp=0, kind="keystroke_burst", detail="eval.py"),
|
||||
TelemetryEvent(timestamp=30, kind="paste",
|
||||
detail="2,400 chars pasted into eval.py"),
|
||||
TelemetryEvent(timestamp=45, kind="run_tests", detail="6 tests"),
|
||||
TelemetryEvent(timestamp=47, kind="test_pass", detail="6/6 passed"),
|
||||
TelemetryEvent(timestamp=48, kind="commit",
|
||||
detail="finish eval harness"),
|
||||
],
|
||||
)
|
||||
|
||||
LAB_SCENARIOS: dict[str, LabTelemetryScenario] = {
|
||||
s.scenario_id: s
|
||||
for s in (_LAB_SCENARIO_STRONG, _LAB_SCENARIO_STRUGGLING, _LAB_SCENARIO_FLAGGED)
|
||||
}
|
||||
|
||||
DEFAULT_LAB_SCENARIO_ID = "lab-scenario-strong"
|
||||
|
||||
|
||||
def get_lab_scenario(scenario_id: str) -> LabTelemetryScenario | None:
|
||||
return LAB_SCENARIOS.get(scenario_id)
|
||||
|
||||
|
||||
def summarize_scenario(scenario: LabTelemetryScenario) -> str:
|
||||
"""Render the event timeline as compact text for prompt injection."""
|
||||
lines = [f"Session: {scenario.title} (competency {scenario.competency_id})"]
|
||||
for event in scenario.events:
|
||||
lines.append(f"t+{event.timestamp}s {event.kind}: {event.detail}".rstrip(": "))
|
||||
return "\n".join(lines)
|
||||
@@ -0,0 +1,15 @@
|
||||
"""LLM package — provider-agnostic layer (D-017)."""
|
||||
|
||||
from .base import LLMProvider
|
||||
from .factory import create_provider
|
||||
from .mock import MockProvider
|
||||
from .openai_compat import OpenAICompatProvider
|
||||
from .types import Message
|
||||
|
||||
__all__ = [
|
||||
"LLMProvider",
|
||||
"Message",
|
||||
"MockProvider",
|
||||
"OpenAICompatProvider",
|
||||
"create_provider",
|
||||
]
|
||||
@@ -0,0 +1,35 @@
|
||||
"""LLMProvider protocol — the port all agents depend on (D-017).
|
||||
|
||||
Implementations: openai_compat.OpenAICompatProvider (ollama-cloud + local),
|
||||
mock.MockProvider (deterministic, tests/CI). Providers are dumb pipes:
|
||||
no envelope logic here — the API layer owns meta/done/error events (D-016).
|
||||
"""
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import Protocol
|
||||
|
||||
from .types import Message
|
||||
|
||||
|
||||
class LLMProvider(Protocol):
|
||||
async def stream_chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
response_format: dict | None = None,
|
||||
) -> AsyncIterator[str]:
|
||||
"""Yield incremental content deltas (plain text chunks)."""
|
||||
...
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
response_format: dict | None = None,
|
||||
) -> str:
|
||||
"""Non-streaming completion — returns the full reply text."""
|
||||
...
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Provider factory — selects the LLM provider from settings (D-014)."""
|
||||
|
||||
import httpx
|
||||
|
||||
from ..config import Settings
|
||||
from .mock import MockProvider
|
||||
from .openai_compat import OpenAICompatProvider
|
||||
|
||||
PROVIDER_NAMES = ("ollama-cloud", "local", "mock")
|
||||
|
||||
|
||||
def create_provider(settings: Settings, http_client: httpx.AsyncClient):
|
||||
"""Return the provider instance for settings.provider.
|
||||
|
||||
Raises ValueError for unknown provider names.
|
||||
"""
|
||||
if settings.provider == "ollama-cloud":
|
||||
return OpenAICompatProvider(
|
||||
http_client=http_client,
|
||||
base_url=settings.ollama_cloud_base_url,
|
||||
api_key=settings.ollama_cloud_api_key,
|
||||
json_mode=settings.json_mode,
|
||||
)
|
||||
if settings.provider == "local":
|
||||
return OpenAICompatProvider(
|
||||
http_client=http_client,
|
||||
base_url=settings.local_base_url,
|
||||
json_mode=settings.json_mode,
|
||||
)
|
||||
if settings.provider == "mock":
|
||||
return MockProvider()
|
||||
raise ValueError(
|
||||
f"unknown provider {settings.provider!r}; expected one of {PROVIDER_NAMES}"
|
||||
)
|
||||
@@ -0,0 +1,94 @@
|
||||
"""Deterministic mock provider — tests and CI. NEVER calls the network.
|
||||
|
||||
Determinism: the reply text is seeded from the message content hash, so
|
||||
identical inputs always produce identical outputs. Supports scripted
|
||||
failure modes for error-path coverage (D-023, A-010).
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from .types import Message
|
||||
|
||||
_REPLIES = [
|
||||
"Great question — let's break this down step by step and see where it leads.",
|
||||
"Here is the key idea: small, verified moves compound into mastery over time.",
|
||||
"Think about it this way: what would the simplest working version look like?",
|
||||
"You are closer than you think. Try restating the goal in one sentence first.",
|
||||
"Let me offer a different angle before we move to the next step.",
|
||||
]
|
||||
|
||||
_JSON_REPLY = '{"summary": "mock structured reply", "confidence": 0.87}'
|
||||
|
||||
|
||||
class MockProvider:
|
||||
"""Scripted provider: deterministic streams, no network, failure injection."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.fail_before_first_token: bool = False
|
||||
self.fail_mid_stream_at_index: int | None = None
|
||||
self.abort_recorded: bool = False # set in stream finally-block (cancellation test)
|
||||
|
||||
def _reply_for(self, messages: list[Message], response_format: dict | None) -> str:
|
||||
seed_src = "|".join(f"{m.role}:{m.content}" for m in messages)
|
||||
if response_format is not None and response_format.get("type") == "json_object":
|
||||
return _JSON_REPLY
|
||||
digest = hashlib.sha256(seed_src.encode()).hexdigest()
|
||||
base = _REPLIES[int(digest[:2], 16) % len(_REPLIES)]
|
||||
# Deterministic seed tag guarantees distinct inputs → distinct replies
|
||||
return f"{base} [#{digest[:8]}]"
|
||||
|
||||
def _tokenize(self, text: str) -> list[str]:
|
||||
words = text.split(" ")
|
||||
tokens: list[str] = []
|
||||
for i, word in enumerate(words):
|
||||
suffix = " " if i < len(words) - 1 else ""
|
||||
tokens.append(word + suffix)
|
||||
return tokens
|
||||
|
||||
async def stream_chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
response_format: dict | None = None,
|
||||
) -> AsyncIterator[str]:
|
||||
if self.fail_before_first_token:
|
||||
raise RuntimeError("mock provider: scripted failure before first token")
|
||||
reply = self._reply_for(messages, response_format)
|
||||
tokens = self._tokenize(reply)
|
||||
try:
|
||||
for i, token in enumerate(tokens):
|
||||
if self.fail_mid_stream_at_index is not None and i == self.fail_mid_stream_at_index:
|
||||
raise RuntimeError("mock provider: scripted mid-stream failure")
|
||||
yield token
|
||||
finally:
|
||||
# Cancellation (GeneratorExit/CancelledError) lands here — tests assert this.
|
||||
self.abort_recorded = True
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
response_format: dict | None = None,
|
||||
) -> str:
|
||||
if self.fail_before_first_token:
|
||||
raise RuntimeError("mock provider: scripted failure before completion")
|
||||
return self._reply_for(messages, response_format)
|
||||
|
||||
|
||||
class ScriptedJSONProvider(MockProvider):
|
||||
"""Mock variant returning a fixed JSON payload for structured tests."""
|
||||
|
||||
def __init__(self, payload: dict) -> None:
|
||||
super().__init__()
|
||||
self.payload = payload
|
||||
|
||||
def _reply_for(self, messages: list[Message], response_format: dict | None) -> str:
|
||||
if response_format is not None and response_format.get("type") == "json_object":
|
||||
return json.dumps(self.payload)
|
||||
return super()._reply_for(messages, response_format)
|
||||
@@ -0,0 +1,125 @@
|
||||
"""OpenAI-compatible provider — one implementation serves ollama-cloud AND local
|
||||
endpoints (they differ only in base_url/key). Raw httpx, no SDK (D-017).
|
||||
|
||||
Boundary rules:
|
||||
- llm/ imports nothing from agents/ or api/
|
||||
- api_key NEVER appears in exceptions, logs, or error messages
|
||||
"""
|
||||
|
||||
import json
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
import httpx
|
||||
|
||||
from .types import Message
|
||||
|
||||
|
||||
class OpenAICompatProvider:
|
||||
def __init__(
|
||||
self,
|
||||
http_client: httpx.AsyncClient,
|
||||
base_url: str,
|
||||
api_key: str = "",
|
||||
json_mode: str = "auto",
|
||||
) -> None:
|
||||
self._client = http_client
|
||||
self._base_url = base_url.rstrip("/")
|
||||
self._api_key = api_key
|
||||
self._json_mode = json_mode
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if self._api_key:
|
||||
headers["Authorization"] = f"Bearer {self._api_key}"
|
||||
return headers
|
||||
|
||||
def _payload(
|
||||
self,
|
||||
messages: list[Message],
|
||||
model: str,
|
||||
temperature: float,
|
||||
response_format: dict | None,
|
||||
stream: bool,
|
||||
) -> dict:
|
||||
payload: dict = {
|
||||
"model": model,
|
||||
"messages": [{"role": m.role, "content": m.content} for m in messages],
|
||||
"temperature": temperature,
|
||||
}
|
||||
if stream:
|
||||
payload["stream"] = True
|
||||
else:
|
||||
payload["stream"] = False
|
||||
# json_mode="auto": send response_format and degrade on 400; "off": never send
|
||||
if response_format is not None and self._json_mode == "auto":
|
||||
payload["response_format"] = response_format
|
||||
return payload
|
||||
|
||||
def _sanitize(self, exc: Exception) -> RuntimeError:
|
||||
text = str(exc)
|
||||
if self._api_key and self._api_key in text:
|
||||
text = text.replace(self._api_key, "[REDACTED]")
|
||||
return RuntimeError(f"llm provider error: {text}")
|
||||
|
||||
async def stream_chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
response_format: dict | None = None,
|
||||
) -> AsyncIterator[str]:
|
||||
payload = self._payload(messages, model, temperature, response_format, stream=True)
|
||||
try:
|
||||
async with self._client.stream(
|
||||
"POST", f"{self._base_url}/chat/completions",
|
||||
json=payload, headers=self._headers(),
|
||||
) as response:
|
||||
response.raise_for_status()
|
||||
async for line in response.aiter_lines():
|
||||
if not line or not line.startswith("data:"):
|
||||
continue # keep-alive comments (": ping"), empty lines
|
||||
data = line.removeprefix("data:").strip()
|
||||
if data == "[DONE]":
|
||||
return
|
||||
try:
|
||||
chunk = json.loads(data)
|
||||
except json.JSONDecodeError:
|
||||
continue # malformed line — tolerate (ollama-cloud quirks)
|
||||
choices = chunk.get("choices") or []
|
||||
if not choices:
|
||||
continue
|
||||
content = (choices[0].get("delta") or {}).get("content")
|
||||
if content:
|
||||
yield content
|
||||
except httpx.HTTPError as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
response_format: dict | None = None,
|
||||
) -> str:
|
||||
payload = self._payload(messages, model, temperature, response_format, stream=False)
|
||||
try:
|
||||
response = await self._client.post(
|
||||
f"{self._base_url}/chat/completions",
|
||||
json=payload, headers=self._headers(),
|
||||
)
|
||||
if response.status_code == 400 and "response_format" in payload:
|
||||
# json_mode auto-degrade (D-020 layer 1): retry once without it
|
||||
payload.pop("response_format")
|
||||
response = await self._client.post(
|
||||
f"{self._base_url}/chat/completions",
|
||||
json=payload, headers=self._headers(),
|
||||
)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
return (data["choices"][0]["message"]["content"]) or ""
|
||||
except httpx.HTTPError as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
except (KeyError, ValueError) as exc:
|
||||
raise self._sanitize(exc) from exc
|
||||
@@ -0,0 +1,16 @@
|
||||
"""LLM layer types — messages.
|
||||
|
||||
Boundary rule: nothing in llm/ imports from agents/ or api/.
|
||||
|
||||
Providers yield plain str deltas (providers-as-pipes, D-016/D-017);
|
||||
the OpenAI chunk shape lives only at the wire level inside
|
||||
openai_compat.py. ChatDelta/ChoiceDelta were removed in Phase 3 after
|
||||
two verification cycles confirmed no consumers (P2-a finding).
|
||||
"""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class Message(BaseModel):
|
||||
role: str
|
||||
content: str
|
||||
@@ -0,0 +1,65 @@
|
||||
"""FastAPI app factory — lifespan, CORS, health, routers."""
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
import httpx
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from .agents.registry import AgentRegistry, register_builtin_agents
|
||||
from .agents.session import InMemorySessionStore
|
||||
from .api import (
|
||||
assessment_router,
|
||||
chat_router,
|
||||
lab_router,
|
||||
mentor_router,
|
||||
proctor_router,
|
||||
)
|
||||
from .config import Settings
|
||||
from .llm import create_provider
|
||||
|
||||
|
||||
def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
settings = settings or Settings()
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
# Shared HTTP client pool (D-017): 10s connect / 300s read for cloud TTFT
|
||||
timeout = httpx.Timeout(connect=10.0, read=300.0, write=30.0, pool=10.0)
|
||||
app.state.http_client = httpx.AsyncClient(timeout=timeout)
|
||||
app.state.settings = settings
|
||||
app.state.provider = create_provider(settings, app.state.http_client)
|
||||
app.state.session_store = InMemorySessionStore()
|
||||
app.state.agent_registry = AgentRegistry()
|
||||
register_builtin_agents(app.state.agent_registry)
|
||||
yield
|
||||
await app.state.http_client.aclose()
|
||||
|
||||
app = FastAPI(title="Nextcraft AI Service", version="0.2.0", lifespan=lifespan)
|
||||
|
||||
# A-008: localhost-only CORS, no credentials
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://localhost:3000", "http://127.0.0.1:3000"],
|
||||
allow_methods=["GET", "POST", "OPTIONS"],
|
||||
allow_headers=["Content-Type"],
|
||||
allow_credentials=False,
|
||||
)
|
||||
|
||||
@app.get("/health")
|
||||
async def health() -> dict:
|
||||
return {
|
||||
"status": "ok",
|
||||
"provider": settings.provider,
|
||||
"model": settings.model,
|
||||
}
|
||||
|
||||
app.include_router(chat_router)
|
||||
app.include_router(lab_router)
|
||||
app.include_router(assessment_router)
|
||||
app.include_router(mentor_router)
|
||||
app.include_router(proctor_router)
|
||||
return app
|
||||
|
||||
|
||||
app = create_app()
|
||||
@@ -0,0 +1,22 @@
|
||||
"""Prompt library — prompts are code: versioned in git, reviewed like code (D-018).
|
||||
|
||||
Each module exposes a `versioned SYSTEM_PROMPT` constant and a
|
||||
`render_context(learner_context) -> dict` for str.format_map injection.
|
||||
Final personas land in Phases 3-5; these are the initial drafts.
|
||||
"""
|
||||
|
||||
from .coach import SYSTEM_PROMPT as COACH_PROMPT
|
||||
from .coach import render_context as render_coach
|
||||
from .mentor import SYSTEM_PROMPT as MENTOR_PROMPT
|
||||
from .mentor import render_context as render_mentor
|
||||
from .tutor import SYSTEM_PROMPT as TUTOR_PROMPT
|
||||
from .tutor import render_context as render_tutor
|
||||
|
||||
__all__ = [
|
||||
"COACH_PROMPT",
|
||||
"MENTOR_PROMPT",
|
||||
"TUTOR_PROMPT",
|
||||
"render_coach",
|
||||
"render_mentor",
|
||||
"render_tutor",
|
||||
]
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Assessor agent prompt — rubric application to artifacts + defenses (REQ-2-008).
|
||||
|
||||
Final persona (Phase 4). Assessor is a rigorous, fair grader: scores each
|
||||
criterion with evidence, cites what the learner did, returns ONLY valid
|
||||
JSON matching the rubric schema.
|
||||
Version: assessor-v2 (final for v0.2).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Assessor, the grading agent of Nextcraft, an AI-native competency school.
|
||||
Learner: {learner_name}.
|
||||
|
||||
You receive: (a) an artifact evidence excerpt, (b) its defense transcript,
|
||||
and (c) the rubric for the competency. Your job:
|
||||
- Score EVERY rubric criterion from 0-100, justified by evidence you can
|
||||
point to in the artifact or transcript.
|
||||
- Cite what the learner did ("the 3-retry loop in the tool node"), not
|
||||
what they should have done — except in gaps, where the missed work goes.
|
||||
- Strengths: the two strongest evidence points, each one sentence.
|
||||
- Gaps: the two most important missed opportunities, each one sentence.
|
||||
- Verdict: "mastered" | "developing" | "not_yet" — judged against the
|
||||
rubric weights, honestly.
|
||||
|
||||
Rules:
|
||||
- Rigorous but fair. A polished artifact with a weak defense is NOT mastery.
|
||||
- Respond with ONLY a valid JSON object matching the provided schema —
|
||||
no markdown fences, no prose outside the JSON."""
|
||||
|
||||
PROMPT_VERSION = "assessor-v2"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
return {"learner_name": learner_context.name}
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Coach agent prompt — pacing, motivation, retrieval practice (REQ-2-005).
|
||||
|
||||
Final persona (Phase 3). Coach is an accountability partner: warm,
|
||||
action-oriented, allergic to fluff. Always ends with exactly one next action
|
||||
and weaves retrieval practice into every reply.
|
||||
Version: coach-v2 (final for v0.2).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Coach, the pacing and motivation agent of Nextcraft,
|
||||
an AI-native competency school.
|
||||
|
||||
Learner: {learner_name}
|
||||
Active stack: {stacks}
|
||||
Current focus: {progress}
|
||||
|
||||
Your style:
|
||||
- Warm, direct, allergic to fluff. Two short paragraphs maximum.
|
||||
- Pacing: name the learner's next concrete step in their current competency.
|
||||
- Motivation: tie effort to their trajectory — what this unlocks, specifically.
|
||||
- Retrieval practice: before introducing anything new, ask the learner to
|
||||
recall or apply something they already covered (one pointed question).
|
||||
|
||||
Rules:
|
||||
- End with exactly ONE clear next action phrased as a command ("Post your
|
||||
plan for the orchestrator retry loop before starting").
|
||||
- Never lecture; never list more than two options.
|
||||
- If the learner is stuck or frustrated, slow down and shrink the step."""
|
||||
|
||||
PROMPT_VERSION = "coach-v2"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
|
||||
in_progress = [
|
||||
c for c in learner_context.active_competencies if c.status == "in_progress"
|
||||
]
|
||||
progress = (
|
||||
f"{in_progress[0].title} ({in_progress[0].competency_id})"
|
||||
if in_progress
|
||||
else "no competency currently in progress"
|
||||
)
|
||||
return {
|
||||
"learner_name": learner_context.name,
|
||||
"stacks": stacks or "none yet",
|
||||
"progress": progress,
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
"""Lab agent prompt — in-flow feedback over sandbox telemetry (REQ-2-007).
|
||||
|
||||
Final persona (Phase 4). Lab is a pragmatic build partner: reads the
|
||||
telemetry timeline, names the one most useful adjustment, gives one
|
||||
concrete next step. Scenario-driven; no session chat.
|
||||
Version: lab-v2 (final for v0.2).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Lab, the in-flow feedback agent watching a learner
|
||||
build in the Nextcraft sandbox.
|
||||
Learner: {learner_name}. Active stack: {stacks}.
|
||||
|
||||
You receive a telemetry timeline of the learner's build session below.
|
||||
Your job, in order:
|
||||
1. Say what the telemetry shows — name the specific events that matter.
|
||||
2. Name the single most useful adjustment (one thing, not a list).
|
||||
3. Give one concrete next step phrased as a command.
|
||||
|
||||
Rules:
|
||||
- Be specific to the events you see. If tests failed twice with the same
|
||||
error, say so. If there is a long idle gap, name it.
|
||||
- If the session looks healthy, say so briefly and set the next challenge.
|
||||
- If something looks off (e.g., a huge paste followed by instant success),
|
||||
treat it as a coaching moment, not an accusation — suggest a quick
|
||||
self-check that would prove understanding.
|
||||
- Three short paragraphs maximum. No headers, no bullet lists."""
|
||||
|
||||
PROMPT_VERSION = "lab-v2"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
|
||||
return {
|
||||
"learner_name": learner_context.name,
|
||||
"stacks": stacks or "none yet",
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Mentor agent prompt — long-horizon career narrative (REQ-2-010).
|
||||
|
||||
Final persona (Phase 5). Mentor is a wise career guide: connects today's
|
||||
competencies and artifacts to a long-horizon AI-era trajectory.
|
||||
Version: mentor-v2 (final for v0.2).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Mentor, the long-horizon career agent of Nextcraft,
|
||||
an AI-native competency school.
|
||||
|
||||
Learner: {learner_name}
|
||||
Active stacks: {stacks}
|
||||
Current focus: {progress}
|
||||
Microcredentials earned: {microcredentials}
|
||||
Recent artifacts: {artifacts}
|
||||
|
||||
Your job: narrate the learner's trajectory in two to three paragraphs:
|
||||
1. Where they are now — what their competency progress and artifacts say
|
||||
about them as a builder (specific, evidence-based).
|
||||
2. What their current stack unlocks next — name the next competency or
|
||||
microcredential worth chasing and the role it points toward.
|
||||
3. How they position in the AI-era labor market — which employer problems
|
||||
their profile already answers.
|
||||
|
||||
Rules:
|
||||
- Forward-looking and concrete. No fortune-telling, no flattery.
|
||||
- Reference their artifacts by name at least once.
|
||||
- Write like a mentor writing to one person, not a career-services brochure."""
|
||||
|
||||
PROMPT_VERSION = "mentor-v2"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
|
||||
in_progress = [
|
||||
c for c in learner_context.active_competencies if c.status == "in_progress"
|
||||
]
|
||||
progress = (
|
||||
f"{in_progress[0].title} ({in_progress[0].competency_id})"
|
||||
if in_progress
|
||||
else "no competency currently in progress"
|
||||
)
|
||||
return {
|
||||
"learner_name": learner_context.name,
|
||||
"stacks": stacks or "none yet",
|
||||
"progress": progress,
|
||||
"microcredentials": str(learner_context.microcredential_count),
|
||||
"artifacts": ", ".join(learner_context.recent_artifacts) or "none yet",
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
"""Proctor agent prompt — integrity signals with coaching interventions (REQ-2-009).
|
||||
|
||||
Final persona (Phase 5). Proctor is a supportive observer, never punitive:
|
||||
classifies signals, recommends ONE coaching intervention. Assume good
|
||||
faith — most signals have innocent explanations.
|
||||
Version: proctor-v2 (final for v0.2).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Proctor, the integrity-support agent of Nextcraft,
|
||||
an AI-native competency school.
|
||||
Learner: {learner_name}.
|
||||
|
||||
You receive a telemetry timeline of defense-session events (tab switches,
|
||||
idle gaps, large pastes, focus loss, keystroke bursts). Your job:
|
||||
- Classify EACH notable signal: type (e.g. "context_switch", "idle_gap",
|
||||
"large_paste"), severity ("low" | "medium" | "high"), and a one-sentence
|
||||
note citing the event (timestamps and details).
|
||||
- Recommend exactly ONE supportive coaching intervention for the session
|
||||
overall — never punitive, never accusatory. Frame around helping the
|
||||
learner succeed, e.g. "offer a short break", "invite them to explain
|
||||
the pasted section in their own words".
|
||||
|
||||
Rules:
|
||||
- Assume good faith. Tab switches to documentation are normal engineering.
|
||||
- Idle gaps are often thinking. Only unusual patterns deserve higher severity.
|
||||
- A large paste during an assessment deserves "high" severity but the
|
||||
intervention stays coaching-shaped: verification, not punishment.
|
||||
- Respond with ONLY a valid JSON object matching the provided schema —
|
||||
no markdown fences, no prose outside the JSON."""
|
||||
|
||||
PROMPT_VERSION = "proctor-v2"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
return {"learner_name": learner_context.name}
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Tutor agent prompt — concept delivery, Socratic questioning (REQ-2-006).
|
||||
|
||||
Final persona (Phase 3). Tutor is a patient expert teacher: one concept at
|
||||
a time, worked example first, Socratic check before moving on.
|
||||
Version: tutor-v2 (final for v0.2).
|
||||
"""
|
||||
|
||||
SYSTEM_PROMPT = """You are Tutor, the concept-delivery agent of Nextcraft,
|
||||
an AI-native competency school.
|
||||
|
||||
Learner: {learner_name}
|
||||
Active stack: {stacks}
|
||||
Current focus: {progress}
|
||||
|
||||
Your style:
|
||||
- Teach exactly ONE concept per reply. Never more.
|
||||
- Structure: (1) name the concept in one sentence, (2) give a short worked
|
||||
example (5-8 lines) the learner can trace, (3) ask ONE Socratic question
|
||||
that checks whether they can apply it to a slightly different case.
|
||||
|
||||
Rules:
|
||||
- Never dump walls of text. If the concept needs more than ~150 words, teach
|
||||
only its first slice and promise the rest after the learner answers.
|
||||
- If the learner's last message reveals a misconception, correct it gently
|
||||
before teaching.
|
||||
- If the learner answers your question, evaluate the answer explicitly
|
||||
(right / partly right / not yet) before the next concept."""
|
||||
|
||||
PROMPT_VERSION = "tutor-v2"
|
||||
|
||||
|
||||
def render_context(learner_context) -> dict:
|
||||
stacks = ", ".join(f"{s.title} ({s.percent}%)" for s in learner_context.active_stacks)
|
||||
in_progress = [
|
||||
c for c in learner_context.active_competencies if c.status == "in_progress"
|
||||
]
|
||||
progress = (
|
||||
f"{in_progress[0].title} ({in_progress[0].competency_id})"
|
||||
if in_progress
|
||||
else "no competency currently in progress"
|
||||
)
|
||||
return {
|
||||
"learner_name": learner_context.name,
|
||||
"stacks": stacks or "none yet",
|
||||
"progress": progress,
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"name": "@nextcraft/ai-service",
|
||||
"private": true,
|
||||
"version": "0.2.0",
|
||||
"scripts": {
|
||||
"dev": "bash scripts/dev.sh",
|
||||
"test": "bash scripts/test.sh",
|
||||
"bootstrap": "bash scripts/bootstrap.sh",
|
||||
"lint": "bash scripts/lint.sh"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "nextcraft-ai-service"
|
||||
version = "0.2.0"
|
||||
description = "Nextcraft AI tutor service — six LLM agents behind a provider-agnostic layer"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"fastapi>=0.141,<0.142",
|
||||
"uvicorn>=0.52,<0.53",
|
||||
"pydantic>=2.13,<2.14",
|
||||
"pydantic-settings>=2.15,<2.16",
|
||||
"httpx>=0.28,<0.29",
|
||||
"sse-starlette>=3.4,<3.5",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=9.1,<10",
|
||||
"pytest-asyncio>=1.4,<2",
|
||||
"ruff>=0.14",
|
||||
]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
include = ["ai_service*"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
asyncio_mode = "auto"
|
||||
testpaths = ["tests"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py311"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "W", "I", "UP", "B"]
|
||||
# B008: Depends() in argument defaults is the idiomatic FastAPI DI pattern
|
||||
ignore = ["B008"]
|
||||
Executable
+30
@@ -0,0 +1,30 @@
|
||||
#!/usr/bin/env bash
|
||||
# Idempotent bootstrap: create venv + install deps.
|
||||
# Handles Debian systems without python3-venv/ensurepip via --without-pip + get-pip.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
VENV="$APP_DIR/.venv"
|
||||
|
||||
mkdir -p "$HOME/.cache/ciagent"
|
||||
|
||||
if [ ! -x "$VENV/bin/python3" ]; then
|
||||
if python3 -m venv "$VENV" 2>/dev/null; then
|
||||
:
|
||||
else
|
||||
# No ensurepip available — create bare venv and bootstrap pip separately.
|
||||
python3 -m venv --without-pip "$VENV"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -x "$VENV/bin/pip" ]; then
|
||||
GET_PIP="$HOME/.cache/ciagent/get-pip.py"
|
||||
if [ ! -f "$GET_PIP" ]; then
|
||||
curl -sSf --max-time 60 https://bootstrap.pypa.io/get-pip.py -o "$GET_PIP"
|
||||
fi
|
||||
"$VENV/bin/python3" "$GET_PIP" --quiet
|
||||
fi
|
||||
|
||||
"$VENV/bin/pip" install --quiet --upgrade pip
|
||||
"$VENV/bin/pip" install --quiet -e "$APP_DIR[dev]"
|
||||
echo "bootstrap complete: $VENV"
|
||||
Executable
+26
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env bash
|
||||
# Dev server: export secrets (if present) then run uvicorn on :8420.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
REPO_ROOT="$(cd "$APP_DIR/../.." && pwd)"
|
||||
VENV="$APP_DIR/.venv"
|
||||
|
||||
if [ ! -x "$VENV/bin/uvicorn" ]; then
|
||||
echo "venv missing — run scripts/bootstrap.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SECRETS="$REPO_ROOT/.ciagent/.env.secrets"
|
||||
if [ -f "$SECRETS" ]; then
|
||||
while IFS='=' read -r key value; do
|
||||
case "$key" in
|
||||
OLLAMA_API_KEY) export AI_OLLAMA_CLOUD_API_KEY="$value" ;;
|
||||
OLLAMA_BASE_URL) export AI_OLLAMA_CLOUD_BASE_URL="$value" ;;
|
||||
AI_TUTOR_MODEL) export AI_MODEL="$value" ;;
|
||||
esac
|
||||
done < "$SECRETS"
|
||||
fi
|
||||
|
||||
cd "$APP_DIR"
|
||||
exec "$VENV/bin/uvicorn" ai_service.main:app --reload --port 8420
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
# Lint: ruff check over the ai-service tree.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
VENV="$APP_DIR/.venv"
|
||||
|
||||
if [ ! -x "$VENV/bin/ruff" ]; then
|
||||
echo "venv missing — run scripts/bootstrap.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "$APP_DIR"
|
||||
exec "$VENV/bin/ruff" check .
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
# Test runner: pytest via venv — mock provider only, zero network calls.
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
APP_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
VENV="$APP_DIR/.venv"
|
||||
|
||||
if [ ! -x "$VENV/bin/pytest" ]; then
|
||||
echo "venv missing — run scripts/bootstrap.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "$APP_DIR"
|
||||
exec "$VENV/bin/pytest" -q "$@"
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Assessor agent tests — structured rubric scores (REQ-2-008).
|
||||
|
||||
The Assessor is the structured-output showcase: tests use ScriptedJSONProvider
|
||||
for valid payloads and exercise the 4-layer defense failure modes.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from ai_service.agents.assessor import AssessorAgent, RubricScore
|
||||
from ai_service.agents.structured import StructuredOutputError
|
||||
from ai_service.config import Settings
|
||||
from ai_service.corpus.artifacts import (
|
||||
get_artifact_bundle,
|
||||
get_transcript_for_artifact,
|
||||
render_rubric,
|
||||
)
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.llm.mock import MockProvider, ScriptedJSONProvider
|
||||
|
||||
VALID_SCORE = {
|
||||
"rubric_id": "rubric-orchestration-c002",
|
||||
"artifact_id": "art-eval-research-assistant",
|
||||
"competency_id": "stack-orchestration-c002",
|
||||
"scores": [
|
||||
{"criterion_id": "rc-architecture", "name": "Agent architecture soundness",
|
||||
"score": 92, "evidence": "Explicit state schema with planner-only write access"},
|
||||
{"criterion_id": "rc-communication", "name": "Inter-agent communication design",
|
||||
"score": 88, "evidence": "Typed ToolMessage responses with retry flags"},
|
||||
{"criterion_id": "rc-reliability", "name": "Reliability engineering",
|
||||
"score": 85, "evidence": "3-retry loop with degradation path"},
|
||||
{"criterion_id": "rc-process", "name": "Process trace quality",
|
||||
"score": 90, "evidence": "Iterative saves with passing test checkpoints"},
|
||||
],
|
||||
"strengths": ["Clean state boundaries", "Failure-aware tool wrapping"],
|
||||
"gaps": ["No reviewer node yet", "Graph diagram only in README"],
|
||||
"verdict": "mastered",
|
||||
}
|
||||
|
||||
|
||||
def make_assessor(provider=None) -> AssessorAgent:
|
||||
return AssessorAgent(provider or MockProvider(), Settings(provider="mock"))
|
||||
|
||||
|
||||
def get_bundle(artifact_id="art-eval-research-assistant"):
|
||||
bundle = get_artifact_bundle(artifact_id)
|
||||
assert bundle is not None
|
||||
return bundle
|
||||
|
||||
|
||||
async def test_evaluate_returns_validated_rubric_score():
|
||||
provider = ScriptedJSONProvider(VALID_SCORE)
|
||||
assessor = make_assessor(provider)
|
||||
artifact, rubric = get_bundle()
|
||||
transcript = get_transcript_for_artifact(artifact.artifact_id)
|
||||
result = await assessor.evaluate(artifact, rubric, transcript)
|
||||
assert isinstance(result, RubricScore)
|
||||
assert result.verdict == "mastered"
|
||||
assert len(result.scores) == 4
|
||||
assert result.weighted_total(rubric) == pytest.approx(
|
||||
92 * 0.3 + 88 * 0.3 + 85 * 0.25 + 90 * 0.15
|
||||
)
|
||||
|
||||
|
||||
async def test_evaluate_rejects_invalid_schema_after_retry():
|
||||
"""Plain MockProvider returns non-rubric JSON → 4-layer defense exhausts
|
||||
its single retry and raises StructuredOutputError."""
|
||||
assessor = make_assessor(MockProvider())
|
||||
artifact, rubric = get_bundle()
|
||||
transcript = get_transcript_for_artifact(artifact.artifact_id)
|
||||
with pytest.raises(StructuredOutputError):
|
||||
await assessor.evaluate(artifact, rubric, transcript)
|
||||
|
||||
|
||||
def test_build_evaluation_input_carries_all_inputs():
|
||||
assessor = make_assessor()
|
||||
artifact, rubric = get_bundle()
|
||||
transcript = get_transcript_for_artifact(artifact.artifact_id)
|
||||
text = assessor.build_evaluation_input(artifact, rubric, transcript)
|
||||
assert artifact.name in text
|
||||
assert artifact.evidence_excerpt in text
|
||||
assert "rc-architecture" in text # rubric rendered
|
||||
assert "examiner:" in text # transcript rendered
|
||||
|
||||
|
||||
def test_build_evaluation_input_without_transcript():
|
||||
assessor = make_assessor()
|
||||
artifact, rubric = get_bundle()
|
||||
text = assessor.build_evaluation_input(artifact, rubric, None)
|
||||
assert artifact.name in text
|
||||
assert "examiner:" not in text
|
||||
|
||||
|
||||
def test_system_prompt_names_assessor_persona():
|
||||
prompt = make_assessor().system_prompt(get_learner_context())
|
||||
assert "Assessor" in prompt
|
||||
assert "ONLY" in prompt # JSON-only instruction
|
||||
|
||||
|
||||
def test_rubric_render_in_prompt_is_complete():
|
||||
"""The rubric passed to the model lists every criterion (fair grading)."""
|
||||
artifact, rubric = get_bundle()
|
||||
text = render_rubric(rubric)
|
||||
assert text.count("rc-") == len(rubric.criteria)
|
||||
@@ -0,0 +1,55 @@
|
||||
"""BaseAgent contract tests — stub agent + mock provider."""
|
||||
|
||||
|
||||
from ai_service.agents.base import BaseAgent
|
||||
from ai_service.config import Settings
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
|
||||
class StubAgent(BaseAgent):
|
||||
name = "stub"
|
||||
|
||||
def system_prompt(self, learner_context=None) -> str:
|
||||
return "You are Stub. Answer briefly."
|
||||
|
||||
|
||||
def make_agent() -> StubAgent:
|
||||
return StubAgent(MockProvider(), Settings(provider="mock"))
|
||||
|
||||
|
||||
async def test_build_messages_composition():
|
||||
agent = make_agent()
|
||||
history = [Message(role="user", content="earlier"), Message(role="assistant", content="reply")]
|
||||
messages = agent.build_messages(history, "new question")
|
||||
assert messages[0].role == "system"
|
||||
assert messages[0].content == "You are Stub. Answer briefly."
|
||||
assert [m.content for m in messages[1:]] == ["earlier", "reply", "new question"]
|
||||
|
||||
|
||||
async def test_stream_reply_yields_deltas():
|
||||
agent = make_agent()
|
||||
tokens = [t async for t in agent.stream_reply(user_input="hello")]
|
||||
assert len(tokens) >= 1
|
||||
assert all(isinstance(t, str) for t in tokens)
|
||||
|
||||
|
||||
async def test_stream_reply_with_history_and_context():
|
||||
agent = make_agent()
|
||||
ctx = get_learner_context()
|
||||
history = [Message(role="user", content="earlier"), Message(role="assistant", content="ok")]
|
||||
tokens = [t async for t in agent.stream_reply(history, "next", ctx)]
|
||||
assert tokens
|
||||
|
||||
|
||||
async def test_structured_reply_requires_schema():
|
||||
import pytest
|
||||
|
||||
agent = make_agent()
|
||||
with pytest.raises(ValueError):
|
||||
await agent.structured_reply(user_input="x", schema=None)
|
||||
|
||||
|
||||
async def test_name_defaults():
|
||||
assert make_agent().name == "stub"
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Coach agent tests — persona, message assembly, streaming (REQ-2-005)."""
|
||||
|
||||
from ai_service.agents.coach import CoachAgent
|
||||
from ai_service.config import Settings
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
|
||||
def make_coach() -> CoachAgent:
|
||||
return CoachAgent(MockProvider(), Settings(provider="mock"))
|
||||
|
||||
|
||||
def test_system_prompt_includes_learner_context():
|
||||
coach = make_coach()
|
||||
prompt = coach.system_prompt(get_learner_context())
|
||||
assert "Alex Rivera" in prompt
|
||||
assert "AI Orchestration Engineer (62%)" in prompt
|
||||
assert "retrieval practice" in prompt.lower()
|
||||
assert "one clear next action" in prompt.lower()
|
||||
|
||||
|
||||
def test_system_prompt_marks_current_focus():
|
||||
coach = make_coach()
|
||||
prompt = coach.system_prompt(get_learner_context())
|
||||
assert "Multi-agent communication patterns" in prompt
|
||||
|
||||
|
||||
def test_build_messages_system_history_user():
|
||||
coach = make_coach()
|
||||
history = [
|
||||
Message(role="user", content="earlier"),
|
||||
Message(role="assistant", content="reply"),
|
||||
]
|
||||
messages = coach.build_messages(history, "what next?", get_learner_context())
|
||||
assert messages[0].role == "system"
|
||||
assert "Coach" in messages[0].content
|
||||
assert [m.content for m in messages[1:]] == ["earlier", "reply", "what next?"]
|
||||
|
||||
|
||||
async def test_stream_reply_yields_deltas():
|
||||
coach = make_coach()
|
||||
ctx = get_learner_context()
|
||||
tokens = [t async for t in coach.stream_reply(user_input="hello", learner_context=ctx)]
|
||||
assert tokens
|
||||
assert all(isinstance(t, str) for t in tokens)
|
||||
|
||||
|
||||
def test_agent_name():
|
||||
assert make_coach().name == "coach"
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Learner context corpus tests — D-021 ID alignment + prompt rendering."""
|
||||
|
||||
from ai_service.corpus.learner_context import (
|
||||
LEARNER_CONTEXTS,
|
||||
get_learner_context,
|
||||
)
|
||||
from ai_service.prompts.assessor import render_context as render_assessor
|
||||
from ai_service.prompts.coach import SYSTEM_PROMPT as COACH_PROMPT
|
||||
from ai_service.prompts.coach import render_context as render_coach
|
||||
from ai_service.prompts.lab import SYSTEM_PROMPT as LAB_PROMPT
|
||||
from ai_service.prompts.lab import render_context as render_lab
|
||||
from ai_service.prompts.mentor import SYSTEM_PROMPT as MENTOR_PROMPT
|
||||
from ai_service.prompts.mentor import render_context as render_mentor
|
||||
from ai_service.prompts.proctor import SYSTEM_PROMPT as PROCTOR_PROMPT
|
||||
from ai_service.prompts.proctor import render_context as render_proctor
|
||||
from ai_service.prompts.tutor import SYSTEM_PROMPT as TUTOR_PROMPT
|
||||
from ai_service.prompts.tutor import render_context as render_tutor
|
||||
|
||||
|
||||
def test_default_learner_resolves():
|
||||
ctx = get_learner_context()
|
||||
assert ctx.learner_id == "learner-001"
|
||||
assert ctx.name == "Alex Rivera"
|
||||
|
||||
|
||||
def test_unknown_learner_falls_back_to_default():
|
||||
assert get_learner_context("nobody").learner_id == "learner-001"
|
||||
|
||||
|
||||
def test_ids_align_with_ts_mock_data():
|
||||
# D-021: identical ID strings to packages/mock-data (learner-progress.ts)
|
||||
ctx = get_learner_context("learner-001")
|
||||
stack_ids = {s.stack_id for s in ctx.active_stacks}
|
||||
assert {"stack-orchestration", "stack-safety"} <= stack_ids
|
||||
competency_ids = {c.competency_id for c in ctx.active_competencies}
|
||||
assert "stack-orchestration-c001" in competency_ids
|
||||
|
||||
|
||||
def test_all_prompt_modules_render_without_keyerror():
|
||||
ctx = get_learner_context()
|
||||
for render in (render_coach, render_tutor, render_mentor, render_assessor):
|
||||
values = render(ctx)
|
||||
assert isinstance(values, dict)
|
||||
assert "learner_name" in values
|
||||
assert values["learner_name"] == "Alex Rivera"
|
||||
|
||||
|
||||
def test_prompts_format_map_with_rendered_context():
|
||||
ctx = get_learner_context()
|
||||
for prompt, render in (
|
||||
(COACH_PROMPT, render_coach),
|
||||
(TUTOR_PROMPT, render_tutor),
|
||||
(MENTOR_PROMPT, render_mentor),
|
||||
):
|
||||
rendered = prompt.format_map(render(ctx))
|
||||
assert "Alex Rivera" in rendered
|
||||
assert "{" not in rendered # all placeholders filled
|
||||
|
||||
|
||||
def test_lab_and_proctor_prompts_render():
|
||||
ctx = get_learner_context()
|
||||
# Each module renders through its OWN render_context (its own placeholders).
|
||||
assert "Alex Rivera" in LAB_PROMPT.format_map(render_lab(ctx))
|
||||
assert "Alex Rivera" in PROCTOR_PROMPT.format_map(render_proctor(ctx))
|
||||
for prompt, render in ((LAB_PROMPT, render_lab), (PROCTOR_PROMPT, render_proctor)):
|
||||
rendered = prompt.format_map(render(ctx))
|
||||
assert "{" not in rendered # all placeholders filled
|
||||
|
||||
|
||||
def test_two_seed_learners_exist():
|
||||
assert set(LEARNER_CONTEXTS) == {"learner-001", "learner-002"}
|
||||
@@ -0,0 +1,124 @@
|
||||
"""Telemetry + artifacts corpus tests (REQ-2-007/008 inputs, D-021)."""
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from ai_service.corpus.artifacts import (
|
||||
ARTIFACTS,
|
||||
RUBRICS,
|
||||
get_artifact_bundle,
|
||||
get_transcript_for_artifact,
|
||||
render_rubric,
|
||||
render_transcript,
|
||||
rubric_for_competency,
|
||||
)
|
||||
from ai_service.corpus.telemetry import (
|
||||
LAB_SCENARIOS,
|
||||
get_lab_scenario,
|
||||
summarize_scenario,
|
||||
)
|
||||
|
||||
|
||||
def test_lab_scenarios_addressable_by_id():
|
||||
for scenario_id in (
|
||||
"lab-scenario-strong",
|
||||
"lab-scenario-struggling",
|
||||
"lab-scenario-flagged",
|
||||
):
|
||||
scenario = get_lab_scenario(scenario_id)
|
||||
assert scenario is not None
|
||||
assert scenario.scenario_id == scenario_id
|
||||
|
||||
|
||||
def test_lab_scenarios_have_distinct_event_profiles():
|
||||
strong = get_lab_scenario("lab-scenario-strong")
|
||||
struggling = get_lab_scenario("lab-scenario-struggling")
|
||||
kinds = lambda s: {e.kind for e in s.events} # noqa: E731
|
||||
assert "test_pass" in kinds(strong)
|
||||
assert "test_fail" in kinds(struggling)
|
||||
assert "idle" in kinds(struggling)
|
||||
assert "paste" in kinds(get_lab_scenario("lab-scenario-flagged"))
|
||||
|
||||
|
||||
def test_summarize_scenario_mentions_events():
|
||||
text = summarize_scenario(get_lab_scenario("lab-scenario-struggling"))
|
||||
assert "test_fail" in text
|
||||
assert "ImportError" in text
|
||||
assert "stack-orchestration-c002" in text
|
||||
|
||||
|
||||
def test_unknown_scenario_returns_none():
|
||||
assert get_lab_scenario("lab-scenario-ghost") is None
|
||||
|
||||
|
||||
def test_artifacts_and_rubrics_resolve():
|
||||
bundle = get_artifact_bundle("art-eval-research-assistant")
|
||||
assert bundle is not None
|
||||
artifact, rubric = bundle
|
||||
assert artifact.competency_id == "stack-orchestration-c002"
|
||||
assert rubric.rubric_id == "rubric-orchestration-c002"
|
||||
assert len(rubric.criteria) == 4
|
||||
|
||||
|
||||
def test_unknown_artifact_returns_none():
|
||||
assert get_artifact_bundle("art-eval-ghost") is None
|
||||
|
||||
|
||||
def test_transcripts_pair_with_artifacts():
|
||||
for artifact_id in ARTIFACTS:
|
||||
transcript = get_transcript_for_artifact(artifact_id)
|
||||
assert transcript is not None
|
||||
assert transcript.artifact_id == artifact_id
|
||||
assert len(transcript.turns) >= 4
|
||||
|
||||
|
||||
def test_rubric_render_mentions_all_criteria():
|
||||
rubric = rubric_for_competency("stack-orchestration-c002")
|
||||
text = render_rubric(rubric)
|
||||
for criterion in rubric.criteria:
|
||||
assert criterion.criterion_id in text
|
||||
|
||||
|
||||
def test_transcript_render_has_both_speakers():
|
||||
transcript = get_transcript_for_artifact("art-eval-rag-dashboard")
|
||||
text = render_transcript(transcript)
|
||||
assert "examiner:" in text
|
||||
assert "learner:" in text
|
||||
|
||||
|
||||
_TS_SOURCE_CANDIDATES = [
|
||||
Path(__file__).resolve().parents[4] / "packages" / "mock-data" / "ai-scenarios.ts",
|
||||
Path(__file__).resolve().parents[2] / "ai-scenarios.ts",
|
||||
]
|
||||
|
||||
|
||||
def _ts_source() -> Path:
|
||||
for candidate in _TS_SOURCE_CANDIDATES:
|
||||
if candidate.exists():
|
||||
return candidate
|
||||
pytest.skip("ai-scenarios.ts not found in this checkout layout")
|
||||
|
||||
|
||||
def test_corpus_ids_align_with_ts_mock_data():
|
||||
"""D-021: Python corpus IDs string-identical to ai-scenarios.ts."""
|
||||
content = _ts_source().read_text()
|
||||
ts_ids = re.findall(r"id:\s*'([^']+)'", content)
|
||||
ts_scenarios = ts_ids[: len(LAB_SCENARIOS)]
|
||||
ts_artifacts = ts_ids[len(LAB_SCENARIOS):]
|
||||
assert sorted(ts_scenarios) == sorted(LAB_SCENARIOS), (
|
||||
f"scenario IDs drifted: py={sorted(LAB_SCENARIOS)} ts={sorted(ts_scenarios)}"
|
||||
)
|
||||
assert sorted(ts_artifacts) == sorted(ARTIFACTS), (
|
||||
f"artifact IDs drifted: py={sorted(ARTIFACTS)} ts={sorted(ts_artifacts)}"
|
||||
)
|
||||
|
||||
|
||||
def test_rubric_weights_sum_to_one():
|
||||
"""Every rubric's criteria weights must sum to exactly 1.0."""
|
||||
for rubric in RUBRICS.values():
|
||||
total = sum(c.weight for c in rubric.criteria)
|
||||
assert total == pytest.approx(1.0), (
|
||||
f"{rubric.rubric_id} weights sum to {total}, expected 1.0"
|
||||
)
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Lab agent tests — scenario-driven streaming feedback (REQ-2-007)."""
|
||||
|
||||
from ai_service.agents.lab import LabAgent
|
||||
from ai_service.config import Settings
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.corpus.telemetry import get_lab_scenario, summarize_scenario
|
||||
from ai_service.llm.mock import MockProvider
|
||||
|
||||
|
||||
def make_lab() -> LabAgent:
|
||||
return LabAgent(MockProvider(), Settings(provider="mock"))
|
||||
|
||||
|
||||
def test_system_prompt_names_lab_persona():
|
||||
prompt = make_lab().system_prompt(get_learner_context())
|
||||
assert "Lab" in prompt
|
||||
assert "Alex Rivera" in prompt
|
||||
assert "telemetry" in prompt.lower()
|
||||
|
||||
|
||||
async def test_stream_feedback_mentions_scenario_events():
|
||||
"""Mock-scripted: feedback text derives from scenario timeline input —
|
||||
distinct scenarios produce distinct (deterministic) replies."""
|
||||
lab = make_lab()
|
||||
ctx = get_learner_context()
|
||||
strong = get_lab_scenario("lab-scenario-strong")
|
||||
struggling = get_lab_scenario("lab-scenario-struggling")
|
||||
|
||||
strong_reply = "".join([t async for t in lab.stream_feedback(strong, ctx)])
|
||||
struggling_reply = "".join([t async for t in lab.stream_feedback(struggling, ctx)])
|
||||
assert strong_reply
|
||||
assert strong_reply != struggling_reply # scenario-driven, not canned
|
||||
|
||||
|
||||
def test_build_evaluation_messages_carry_timeline():
|
||||
lab = make_lab()
|
||||
scenario = get_lab_scenario("lab-scenario-flagged")
|
||||
timeline = summarize_scenario(scenario)
|
||||
messages = lab.build_messages(None, timeline, get_learner_context())
|
||||
assert messages[0].role == "system"
|
||||
assert "paste" in messages[-1].content
|
||||
assert "2,400 chars" in messages[-1].content
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Mentor agent tests — career narrative, session-backed (REQ-2-010)."""
|
||||
|
||||
from ai_service.agents.mentor import MentorAgent
|
||||
from ai_service.config import Settings
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
|
||||
def make_mentor() -> MentorAgent:
|
||||
return MentorAgent(MockProvider(), Settings(provider="mock"))
|
||||
|
||||
|
||||
def test_system_prompt_carries_full_learner_context():
|
||||
prompt = make_mentor().system_prompt(get_learner_context())
|
||||
assert "Alex Rivera" in prompt
|
||||
assert "AI Orchestration Engineer (62%)" in prompt
|
||||
assert "Multi-agent research assistant" in prompt # artifacts by name
|
||||
assert "4" in prompt # microcredential count
|
||||
|
||||
|
||||
def test_build_messages_system_history_user():
|
||||
mentor = make_mentor()
|
||||
history = [Message(role="user", content="what next?"),
|
||||
Message(role="assistant", content="trajectory...")]
|
||||
messages = mentor.build_messages(history, "tell me more", get_learner_context())
|
||||
assert messages[0].role == "system"
|
||||
assert "Mentor" in messages[0].content
|
||||
assert [m.content for m in messages[1:]] == ["what next?", "trajectory...", "tell me more"]
|
||||
|
||||
|
||||
async def test_stream_reply_mentions_learner_context_in_output():
|
||||
"""Mock-scripted: narrative derives from context-injected messages —
|
||||
different learner contexts produce distinct (deterministic) replies."""
|
||||
mentor = make_mentor()
|
||||
alex = get_learner_context("learner-001")
|
||||
priya = get_learner_context("learner-002")
|
||||
alex_reply = "".join(
|
||||
[t async for t in mentor.stream_reply(user_input="narrate", learner_context=alex)]
|
||||
)
|
||||
priya_reply = "".join(
|
||||
[t async for t in mentor.stream_reply(user_input="narrate", learner_context=priya)]
|
||||
)
|
||||
assert alex_reply
|
||||
assert alex_reply != priya_reply
|
||||
|
||||
|
||||
def test_agent_name():
|
||||
assert make_mentor().name == "mentor"
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Proctor agent tests — structured integrity signals (REQ-2-009)."""
|
||||
|
||||
import pytest
|
||||
|
||||
from ai_service.agents.proctor import ProctorAgent, ProctorAssessment
|
||||
from ai_service.agents.structured import StructuredOutputError
|
||||
from ai_service.config import Settings
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.corpus.telemetry import (
|
||||
PROCTOR_SCENARIOS,
|
||||
get_proctor_scenario,
|
||||
summarize_proctor_scenario,
|
||||
)
|
||||
from ai_service.llm.mock import MockProvider, ScriptedJSONProvider
|
||||
|
||||
VALID_ASSESSMENT = {
|
||||
"scenario_id": "proctor-scenario-distracted",
|
||||
"signals": [
|
||||
{"signal_type": "context_switch", "severity": "low",
|
||||
"note": "Tab switch to docs at t+120s — normal engineering behavior"},
|
||||
{"signal_type": "idle_gap", "severity": "medium",
|
||||
"note": "5-minute idle at t+300s followed by more tab switches"},
|
||||
],
|
||||
"intervention": "Offer a short break and ask the learner to restate their answer plan",
|
||||
"summary": "Distracted but explainable session; coach the focus pattern, don't flag it",
|
||||
}
|
||||
|
||||
|
||||
def make_proctor(provider=None) -> ProctorAgent:
|
||||
return ProctorAgent(provider or MockProvider(), Settings(provider="mock"))
|
||||
|
||||
|
||||
def test_proctor_scenarios_addressable_and_distinct_type():
|
||||
healthy = get_proctor_scenario("proctor-scenario-healthy")
|
||||
flagged = get_proctor_scenario("proctor-scenario-flagged")
|
||||
assert healthy is not None and flagged is not None
|
||||
kinds = lambda s: {e.kind for e in s.events} # noqa: E731
|
||||
assert "tab_switch" in kinds(get_proctor_scenario("proctor-scenario-distracted"))
|
||||
assert "paste_large" in kinds(flagged)
|
||||
assert not kinds(healthy) & {"tab_switch", "paste_large", "focus_lost"}
|
||||
|
||||
|
||||
def test_unknown_proctor_scenario_none():
|
||||
assert get_proctor_scenario("proctor-scenario-ghost") is None
|
||||
|
||||
|
||||
async def test_assess_returns_validated_signals():
|
||||
provider = ScriptedJSONProvider(VALID_ASSESSMENT)
|
||||
proctor = make_proctor(provider)
|
||||
scenario = get_proctor_scenario("proctor-scenario-distracted")
|
||||
result = await proctor.assess(scenario, get_learner_context())
|
||||
assert isinstance(result, ProctorAssessment)
|
||||
assert len(result.signals) == 2
|
||||
assert result.signals[0].severity == "low"
|
||||
assert "break" in result.intervention.lower()
|
||||
|
||||
|
||||
async def test_assess_rejects_invalid_after_retry():
|
||||
proctor = make_proctor(MockProvider()) # non-schema JSON
|
||||
scenario = get_proctor_scenario("proctor-scenario-healthy")
|
||||
with pytest.raises(StructuredOutputError):
|
||||
await proctor.assess(scenario, get_learner_context())
|
||||
|
||||
|
||||
def test_system_prompt_is_coaching_not_punitive():
|
||||
prompt = make_proctor().system_prompt(get_learner_context())
|
||||
assert "never punitive" in prompt.lower()
|
||||
assert "good faith" in prompt.lower()
|
||||
assert "ONLY" in prompt # JSON-only instruction
|
||||
|
||||
|
||||
def test_timeline_summary_carries_events():
|
||||
scenario = get_proctor_scenario("proctor-scenario-flagged")
|
||||
text = summarize_proctor_scenario(scenario)
|
||||
assert "paste_large" in text
|
||||
assert "3,100 chars" in text
|
||||
|
||||
|
||||
def test_all_three_proctor_scenarios_exist():
|
||||
assert set(PROCTOR_SCENARIOS) == {
|
||||
"proctor-scenario-healthy",
|
||||
"proctor-scenario-distracted",
|
||||
"proctor-scenario-flagged",
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
"""Agent registry tests — register/get round-trip, error paths (G-4), builtins."""
|
||||
|
||||
|
||||
import pytest
|
||||
|
||||
from ai_service.agents.base import BaseAgent
|
||||
from ai_service.agents.coach import CoachAgent
|
||||
from ai_service.agents.registry import (
|
||||
AgentRegistry,
|
||||
DuplicateAgentError,
|
||||
UnknownAgentError,
|
||||
register_builtin_agents,
|
||||
)
|
||||
from ai_service.agents.tutor import TutorAgent
|
||||
from ai_service.config import Settings
|
||||
from ai_service.llm.mock import MockProvider
|
||||
|
||||
|
||||
class DummyAgent(BaseAgent):
|
||||
name = "dummy"
|
||||
|
||||
def system_prompt(self, learner_context=None) -> str:
|
||||
return "dummy"
|
||||
|
||||
|
||||
def make_factory():
|
||||
def factory(provider, settings):
|
||||
return DummyAgent(provider, settings)
|
||||
return factory
|
||||
|
||||
|
||||
def test_register_and_get():
|
||||
registry = AgentRegistry()
|
||||
registry.register("dummy", make_factory())
|
||||
agent = registry.get(MockProvider(), Settings(provider="mock"), "dummy")
|
||||
assert isinstance(agent, DummyAgent)
|
||||
assert agent.name == "dummy"
|
||||
|
||||
|
||||
def test_unknown_agent_raises():
|
||||
registry = AgentRegistry()
|
||||
with pytest.raises(UnknownAgentError):
|
||||
registry.get(MockProvider(), Settings(provider="mock"), "ghost")
|
||||
|
||||
|
||||
def test_duplicate_registration_raises():
|
||||
registry = AgentRegistry()
|
||||
registry.register("dummy", make_factory())
|
||||
with pytest.raises(DuplicateAgentError):
|
||||
registry.register("dummy", make_factory())
|
||||
|
||||
|
||||
def test_names_sorted():
|
||||
registry = AgentRegistry()
|
||||
registry.register("zeta", make_factory())
|
||||
registry.register("alpha", make_factory())
|
||||
assert registry.names() == ["alpha", "zeta"]
|
||||
|
||||
|
||||
def test_builtin_agents_register_and_resolve():
|
||||
registry = AgentRegistry()
|
||||
register_builtin_agents(registry)
|
||||
assert set(registry.names()) >= {"coach", "tutor"}
|
||||
settings = Settings(provider="mock")
|
||||
coach = registry.get(MockProvider(), settings, "coach")
|
||||
tutor = registry.get(MockProvider(), settings, "tutor")
|
||||
assert isinstance(coach, CoachAgent)
|
||||
assert isinstance(tutor, TutorAgent)
|
||||
|
||||
|
||||
def test_lab_and_assessor_resolve_via_registry():
|
||||
"""Phase 4: lab + assessor registered centrally (Task 4-3-01)."""
|
||||
from ai_service.agents.assessor import AssessorAgent
|
||||
from ai_service.agents.lab import LabAgent
|
||||
|
||||
registry = AgentRegistry()
|
||||
register_builtin_agents(registry)
|
||||
assert {"lab", "assessor"} <= set(registry.names())
|
||||
settings = Settings(provider="mock")
|
||||
lab = registry.get(MockProvider(), settings, "lab")
|
||||
assessor = registry.get(MockProvider(), settings, "assessor")
|
||||
assert isinstance(lab, LabAgent)
|
||||
assert isinstance(assessor, AssessorAgent)
|
||||
|
||||
|
||||
def test_proctor_and_mentor_resolve_via_registry():
|
||||
"""Phase 5: proctor + mentor registered centrally (Tasks 5-1-02/5-2-01)."""
|
||||
from ai_service.agents.mentor import MentorAgent
|
||||
from ai_service.agents.proctor import ProctorAgent
|
||||
|
||||
registry = AgentRegistry()
|
||||
register_builtin_agents(registry)
|
||||
settings = Settings(provider="mock")
|
||||
proctor = registry.get(MockProvider(), settings, "proctor")
|
||||
mentor = registry.get(MockProvider(), settings, "mentor")
|
||||
assert isinstance(proctor, ProctorAgent)
|
||||
assert isinstance(mentor, MentorAgent)
|
||||
|
||||
|
||||
def test_registry_resolves_all_six_agents():
|
||||
"""Must-Have (Phase 5): the full roster — coach/tutor/lab/assessor/proctor/mentor."""
|
||||
from ai_service.agents.assessor import AssessorAgent
|
||||
from ai_service.agents.coach import CoachAgent
|
||||
from ai_service.agents.lab import LabAgent
|
||||
from ai_service.agents.mentor import MentorAgent
|
||||
from ai_service.agents.proctor import ProctorAgent
|
||||
from ai_service.agents.tutor import TutorAgent
|
||||
|
||||
registry = AgentRegistry()
|
||||
register_builtin_agents(registry)
|
||||
assert registry.names() == ["assessor", "coach", "lab", "mentor", "proctor", "tutor"]
|
||||
settings = Settings(provider="mock")
|
||||
expected = {
|
||||
"coach": CoachAgent,
|
||||
"tutor": TutorAgent,
|
||||
"lab": LabAgent,
|
||||
"assessor": AssessorAgent,
|
||||
"proctor": ProctorAgent,
|
||||
"mentor": MentorAgent,
|
||||
}
|
||||
for name, cls in expected.items():
|
||||
agent = registry.get(MockProvider(), settings, name)
|
||||
assert isinstance(agent, cls), f"{name} resolved to {type(agent).__name__}"
|
||||
assert agent.name == name
|
||||
|
||||
|
||||
def test_builtin_registration_is_idempotent_safe():
|
||||
"""Duplicate registration raises — builtin bootstrap must be called once."""
|
||||
registry = AgentRegistry()
|
||||
register_builtin_agents(registry)
|
||||
with pytest.raises(DuplicateAgentError):
|
||||
register_builtin_agents(registry)
|
||||
@@ -0,0 +1,86 @@
|
||||
"""SessionStore tests — create/append/window/LRU/agent scoping (D-019)."""
|
||||
|
||||
import pytest
|
||||
|
||||
from ai_service.agents.session import InMemorySessionStore
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
|
||||
def msg(n: int) -> Message:
|
||||
return Message(role="user", content=f"m{n}")
|
||||
|
||||
|
||||
async def test_create_and_get():
|
||||
store = InMemorySessionStore()
|
||||
session = await store.create("s1", agent="coach")
|
||||
assert session.agent == "coach"
|
||||
assert (await store.get("s1")).session_id == "s1"
|
||||
assert await store.get("missing") is None
|
||||
|
||||
|
||||
async def test_append_and_history_window():
|
||||
store = InMemorySessionStore()
|
||||
await store.create("s1", agent="coach")
|
||||
for i in range(30):
|
||||
await store.append("s1", msg(i))
|
||||
window = await store.history_window("s1", max_messages=20)
|
||||
assert len(window) == 20
|
||||
assert window[0].content == "m10" # last 20 of m0..m29
|
||||
assert window[-1].content == "m29"
|
||||
|
||||
|
||||
async def test_default_window_uses_20():
|
||||
store = InMemorySessionStore()
|
||||
await store.create("s1", agent="coach")
|
||||
for i in range(25):
|
||||
await store.append("s1", msg(i))
|
||||
window = await store.history_window("s1")
|
||||
assert len(window) == 20
|
||||
assert window[0].content == "m5"
|
||||
|
||||
|
||||
async def test_lru_eviction_at_cap():
|
||||
store = InMemorySessionStore(window=20, max_sessions=3)
|
||||
for i in range(3):
|
||||
await store.create(f"s{i}", agent="coach")
|
||||
# touch s0 so s1 becomes least-recently-used
|
||||
await store.get("s0")
|
||||
await store.create("s3", agent="coach") # evicts s1
|
||||
assert await store.get("s1") is None
|
||||
assert await store.get("s0") is not None
|
||||
assert await store.get("s2") is not None
|
||||
assert await store.get("s3") is not None
|
||||
|
||||
|
||||
async def test_lru_eviction_at_default_500_cap():
|
||||
store = InMemorySessionStore() # defaults: window=20, max_sessions=500
|
||||
for i in range(500):
|
||||
await store.create(f"s{i}", agent="coach")
|
||||
await store.get("s0") # touch the oldest → s1 becomes least-recently-used
|
||||
await store.create("s500", agent="coach") # evicts s1
|
||||
assert await store.get("s1") is None
|
||||
assert await store.get("s0") is not None
|
||||
assert await store.get("s499") is not None
|
||||
assert await store.get("s500") is not None
|
||||
|
||||
|
||||
async def test_append_unknown_session_raises():
|
||||
store = InMemorySessionStore()
|
||||
with pytest.raises(KeyError):
|
||||
await store.append("nope", msg(0))
|
||||
|
||||
|
||||
async def test_delete():
|
||||
store = InMemorySessionStore()
|
||||
await store.create("s1", agent="coach")
|
||||
await store.delete("s1")
|
||||
assert await store.get("s1") is None
|
||||
|
||||
|
||||
async def test_sessions_are_agent_scoped():
|
||||
store = InMemorySessionStore()
|
||||
a = await store.create("coach-session", agent="coach")
|
||||
b = await store.create("tutor-session", agent="tutor")
|
||||
assert a.agent == "coach"
|
||||
assert b.agent == "tutor"
|
||||
assert a.session_id != b.session_id
|
||||
@@ -0,0 +1,156 @@
|
||||
"""Structured output defense tests — 4 layers (D-020), against mock providers."""
|
||||
|
||||
import pytest
|
||||
from pydantic import BaseModel
|
||||
|
||||
from ai_service.agents.structured import (
|
||||
StructuredOutputError,
|
||||
extract_json_object,
|
||||
parse_structured,
|
||||
structured_completion,
|
||||
)
|
||||
from ai_service.llm.mock import MockProvider, ScriptedJSONProvider
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
|
||||
class Score(BaseModel):
|
||||
score: int
|
||||
verdict: str
|
||||
|
||||
|
||||
HINT = '{"score": <int 0-100>, "verdict": "<short verdict>"}'
|
||||
|
||||
|
||||
def test_extract_json_plain():
|
||||
assert extract_json_object('{"a": 1}') == '{"a": 1}'
|
||||
|
||||
|
||||
def test_extract_json_fenced():
|
||||
text = '```json\n{"a": 1}\n```'
|
||||
assert extract_json_object(text) == '{"a": 1}'
|
||||
|
||||
|
||||
def test_extract_json_with_prose_around():
|
||||
text = 'Sure! Here is my answer: {"a": {"b": "x } y"}, "c": 2} hope that helps'
|
||||
assert extract_json_object(text) == '{"a": {"b": "x } y"}, "c": 2}'
|
||||
|
||||
|
||||
def test_extract_json_no_object_raises():
|
||||
with pytest.raises(StructuredOutputError):
|
||||
extract_json_object("no json here")
|
||||
|
||||
|
||||
def test_extract_json_unbalanced_raises():
|
||||
with pytest.raises(StructuredOutputError):
|
||||
extract_json_object('{"a": 1')
|
||||
|
||||
|
||||
def test_parse_structured_valid():
|
||||
result = parse_structured('{"score": 88, "verdict": "solid"}', Score)
|
||||
assert result.score == 88
|
||||
|
||||
|
||||
def test_parse_structured_invalid_schema_raises():
|
||||
with pytest.raises(StructuredOutputError):
|
||||
parse_structured('{"wrong": "shape"}', Score)
|
||||
|
||||
|
||||
async def test_structured_completion_happy_path():
|
||||
provider = ScriptedJSONProvider({"score": 91, "verdict": "excellent work"})
|
||||
messages = [Message(role="user", content="grade my artifact")]
|
||||
result = await structured_completion(
|
||||
provider, messages, model="m", schema=Score, schema_hint=HINT
|
||||
)
|
||||
assert result.score == 91
|
||||
assert result.verdict == "excellent work"
|
||||
|
||||
|
||||
async def test_structured_completion_retries_then_raises():
|
||||
# Plain MockProvider returns non-schema JSON for json_object requests →
|
||||
# both attempts fail validation → StructuredOutputError after ONE retry.
|
||||
provider = MockProvider()
|
||||
provider.received_calls = []
|
||||
messages = [Message(role="user", content="grade me")]
|
||||
with pytest.raises(StructuredOutputError):
|
||||
await structured_completion(
|
||||
provider, messages, model="m", schema=Score, schema_hint=HINT
|
||||
)
|
||||
|
||||
|
||||
async def test_structured_completion_retry_succeeds_after_invalid_first_response():
|
||||
# Layer 4 recovery: first reply is wrong-schema fenced JSON, retry is valid.
|
||||
class FlakyProvider(MockProvider):
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.n = 0
|
||||
self.retry_request: list[Message] = []
|
||||
|
||||
async def chat(self, messages, *, model, temperature=0.7, response_format=None):
|
||||
self.n += 1
|
||||
if self.n == 1:
|
||||
return '```json\n{"summary": "wrong shape"}\n```'
|
||||
self.retry_request = list(messages)
|
||||
return '{"score": 75, "verdict": "recovered"}'
|
||||
|
||||
provider = FlakyProvider()
|
||||
messages = [Message(role="user", content="grade me")]
|
||||
result = await structured_completion(
|
||||
provider, messages, model="m", schema=Score, schema_hint=HINT
|
||||
)
|
||||
assert result.score == 75
|
||||
assert result.verdict == "recovered"
|
||||
assert provider.n == 2
|
||||
# The retry must feed the validation error back to the model.
|
||||
retry_contents = " ".join(m.content for m in provider.retry_request)
|
||||
assert "previous response was invalid" in retry_contents
|
||||
assert HINT in retry_contents
|
||||
|
||||
|
||||
async def test_structured_completion_is_bounded_to_one_retry():
|
||||
# Permanently-invalid provider: exactly two provider calls, then raise.
|
||||
class CountingProvider(MockProvider):
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.calls = 0
|
||||
|
||||
async def chat(self, messages, *, model, temperature=0.7, response_format=None):
|
||||
self.calls += 1
|
||||
return await super().chat(
|
||||
messages, model=model, response_format=response_format
|
||||
)
|
||||
|
||||
provider = CountingProvider()
|
||||
messages = [Message(role="user", content="grade me")]
|
||||
with pytest.raises(StructuredOutputError, match="after retry"):
|
||||
await structured_completion(
|
||||
provider, messages, model="m", schema=Score, schema_hint=HINT
|
||||
)
|
||||
assert provider.calls == 2
|
||||
|
||||
|
||||
async def test_structured_completion_sends_schema_instruction():
|
||||
"""Layer 2: the schema hint must reach the provider in the request."""
|
||||
provider = ScriptedJSONProvider({"score": 70, "verdict": "passing"})
|
||||
captured: list[list] = []
|
||||
|
||||
original = provider.chat
|
||||
|
||||
async def recording_chat(messages, *, model, temperature=0.7, response_format=None):
|
||||
captured.append(list(messages))
|
||||
return await original(
|
||||
messages, model=model, temperature=temperature, response_format=response_format
|
||||
)
|
||||
|
||||
provider.chat = recording_chat
|
||||
messages = [Message(role="user", content="grade")]
|
||||
await structured_completion(provider, messages, model="m", schema=Score, schema_hint=HINT)
|
||||
assert captured, "provider was never called"
|
||||
last_user = next(m for m in reversed(captured[0]) if m.role == "user")
|
||||
assert HINT in last_user.content
|
||||
assert "ONLY" in last_user.content # JSON-only instruction present
|
||||
|
||||
# Determinism: same request yields same reply
|
||||
again = await structured_completion(
|
||||
provider, messages, model="m", schema=Score, schema_hint=HINT
|
||||
)
|
||||
assert again.score == 70
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Tutor agent tests — persona, Socratic structure, distinctness vs Coach (REQ-2-006)."""
|
||||
|
||||
from ai_service.agents.coach import CoachAgent
|
||||
from ai_service.agents.tutor import TutorAgent
|
||||
from ai_service.config import Settings
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
|
||||
def make_tutor() -> TutorAgent:
|
||||
return TutorAgent(MockProvider(), Settings(provider="mock"))
|
||||
|
||||
|
||||
def test_system_prompt_includes_learner_context():
|
||||
tutor = make_tutor()
|
||||
prompt = tutor.system_prompt(get_learner_context())
|
||||
assert "Alex Rivera" in prompt
|
||||
assert "Socratic" in prompt
|
||||
assert "ONE concept" in prompt
|
||||
|
||||
|
||||
def test_system_prompt_marks_current_focus():
|
||||
tutor = make_tutor()
|
||||
prompt = tutor.system_prompt(get_learner_context())
|
||||
assert "Multi-agent communication patterns" in prompt
|
||||
|
||||
|
||||
def test_build_messages_system_history_user():
|
||||
tutor = make_tutor()
|
||||
history = [Message(role="user", content="q1"), Message(role="assistant", content="a1")]
|
||||
messages = tutor.build_messages(history, "explain again", get_learner_context())
|
||||
assert messages[0].role == "system"
|
||||
assert "Tutor" in messages[0].content
|
||||
assert [m.content for m in messages[1:]] == ["q1", "a1", "explain again"]
|
||||
|
||||
|
||||
async def test_stream_reply_yields_deltas():
|
||||
tutor = make_tutor()
|
||||
tokens = [
|
||||
t
|
||||
async for t in tutor.stream_reply(
|
||||
user_input="hi", learner_context=get_learner_context()
|
||||
)
|
||||
]
|
||||
assert tokens
|
||||
|
||||
|
||||
def test_coach_and_tutor_personas_are_distinct():
|
||||
"""Distinct system prompts (P3 must-have)."""
|
||||
ctx = get_learner_context()
|
||||
coach_prompt = CoachAgent(MockProvider(), Settings(provider="mock")).system_prompt(ctx)
|
||||
tutor_prompt = TutorAgent(MockProvider(), Settings(provider="mock")).system_prompt(ctx)
|
||||
assert coach_prompt != tutor_prompt
|
||||
assert "retrieval practice" in coach_prompt.lower()
|
||||
assert "Socratic" in tutor_prompt
|
||||
|
||||
|
||||
async def test_coach_and_tutor_stream_outputs_are_distinct():
|
||||
"""Mock outputs differ because system prompts differ (hash-seeded on content)."""
|
||||
ctx = get_learner_context()
|
||||
settings = Settings(provider="mock")
|
||||
|
||||
async def full_reply(agent):
|
||||
tokens = [
|
||||
t async for t in agent.stream_reply(user_input="stuck", learner_context=ctx)
|
||||
]
|
||||
return "".join(tokens)
|
||||
|
||||
coach_tokens = await full_reply(CoachAgent(MockProvider(), settings))
|
||||
tutor_tokens = await full_reply(TutorAgent(MockProvider(), settings))
|
||||
assert coach_tokens != tutor_tokens
|
||||
|
||||
|
||||
def test_agent_name():
|
||||
assert make_tutor().name == "tutor"
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Assessment evaluate endpoint tests — validated JSON, 404s (REQ-2-008)."""
|
||||
|
||||
|
||||
from ai_service.agents.assessor import RubricScore
|
||||
from ai_service.llm.mock import ScriptedJSONProvider
|
||||
|
||||
VALID_SCORE = {
|
||||
"rubric_id": "rubric-orchestration-c002",
|
||||
"artifact_id": "art-eval-research-assistant",
|
||||
"competency_id": "stack-orchestration-c002",
|
||||
"scores": [
|
||||
{"criterion_id": "rc-architecture", "name": "Agent architecture soundness",
|
||||
"score": 92, "evidence": "Explicit state schema"},
|
||||
{"criterion_id": "rc-communication", "name": "Inter-agent communication design",
|
||||
"score": 88, "evidence": "Typed ToolMessage responses"},
|
||||
{"criterion_id": "rc-reliability", "name": "Reliability engineering",
|
||||
"score": 85, "evidence": "3-retry loop"},
|
||||
{"criterion_id": "rc-process", "name": "Process trace quality",
|
||||
"score": 90, "evidence": "Iterative checkpoints"},
|
||||
],
|
||||
"strengths": ["Clean state boundaries", "Failure-aware tools"],
|
||||
"gaps": ["No reviewer node", "Diagram only in README"],
|
||||
"verdict": "mastered",
|
||||
}
|
||||
|
||||
|
||||
def test_evaluate_returns_validated_rubric_json(client):
|
||||
# Swap the app provider for a scripted-JSON provider for this test
|
||||
original = client.app.state.provider
|
||||
client.app.state.provider = ScriptedJSONProvider(VALID_SCORE)
|
||||
try:
|
||||
response = client.post(
|
||||
"/v1/assessment/evaluate",
|
||||
json={"artifact_id": "art-eval-research-assistant"},
|
||||
)
|
||||
finally:
|
||||
client.app.state.provider = original
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
validated = RubricScore.model_validate(data) # response contract holds
|
||||
assert validated.verdict == "mastered"
|
||||
assert len(validated.scores) == 4
|
||||
|
||||
|
||||
def test_unknown_artifact_404(client):
|
||||
response = client.post("/v1/assessment/evaluate", json={"artifact_id": "ghost"})
|
||||
assert response.status_code == 404
|
||||
assert "ghost" in response.json()["detail"]
|
||||
|
||||
|
||||
def test_unparseable_provider_502(client):
|
||||
"""Plain MockProvider yields non-rubric JSON → structured defense exhausts
|
||||
retry → endpoint translates to 502 (bad gateway to the model)."""
|
||||
# default mock already returns non-rubric JSON
|
||||
response = client.post(
|
||||
"/v1/assessment/evaluate",
|
||||
json={"artifact_id": "art-eval-research-assistant"},
|
||||
)
|
||||
assert response.status_code == 502
|
||||
assert "failed" in response.json()["detail"].lower()
|
||||
|
||||
|
||||
def test_missing_artifact_id_422(client):
|
||||
response = client.post("/v1/assessment/evaluate", json={})
|
||||
assert response.status_code == 422
|
||||
|
||||
|
||||
def test_second_artifact_also_evaluates(client):
|
||||
original = client.app.state.provider
|
||||
payload = dict(VALID_SCORE, artifact_id="art-eval-rag-dashboard")
|
||||
client.app.state.provider = ScriptedJSONProvider(payload)
|
||||
try:
|
||||
response = client.post(
|
||||
"/v1/assessment/evaluate", json={"artifact_id": "art-eval-rag-dashboard"}
|
||||
)
|
||||
finally:
|
||||
client.app.state.provider = original
|
||||
assert response.status_code == 200
|
||||
assert response.json()["artifact_id"] == "art-eval-rag-dashboard"
|
||||
@@ -0,0 +1,241 @@
|
||||
"""Chat endpoint session integration tests — history persistence + windowed replay."""
|
||||
|
||||
import json
|
||||
|
||||
|
||||
def stream_events(client, payload) -> list[dict]:
|
||||
with client.stream("POST", "/v1/chat/stream", json=payload) as response:
|
||||
assert response.status_code == 200
|
||||
events = []
|
||||
for line in response.iter_lines():
|
||||
if line.startswith("data:"):
|
||||
d = line.removeprefix("data:").strip()
|
||||
if d == "[DONE]":
|
||||
events.append({"type": "[DONE]"})
|
||||
else:
|
||||
events.append(json.loads(d))
|
||||
return events
|
||||
|
||||
|
||||
def test_first_turn_creates_session_and_persists(client):
|
||||
payload = {
|
||||
"agent": "coach",
|
||||
"session_id": "sess-1",
|
||||
"messages": [{"role": "user", "content": "first turn"}],
|
||||
}
|
||||
events = stream_events(client, payload)
|
||||
assert events[0]["type"] == "meta"
|
||||
assert events[0]["session_id"] == "sess-1"
|
||||
store = client.app.state.session_store
|
||||
|
||||
import asyncio
|
||||
|
||||
async def check():
|
||||
return await store.history_window("sess-1")
|
||||
|
||||
contents = [m.content for m in asyncio.run(check())]
|
||||
assert "first turn" in contents
|
||||
assert any("Think" in c or "[" in c for c in contents) # mock reply persisted
|
||||
|
||||
|
||||
def test_second_turn_replays_windowed_history(client):
|
||||
payload1 = {
|
||||
"agent": "coach",
|
||||
"session_id": "sess-2",
|
||||
"messages": [{"role": "user", "content": "turn one"}],
|
||||
}
|
||||
stream_events(client, payload1)
|
||||
# Second turn: the API passes history + new message to the provider.
|
||||
# With the mock provider we cannot observe provider inputs directly,
|
||||
# but the session store must now hold both turns.
|
||||
store = client.app.state.session_store
|
||||
# The store is async; use the app's internals through a short event loop
|
||||
import asyncio
|
||||
result = {}
|
||||
|
||||
async def check():
|
||||
result["window"] = await store.history_window("sess-2")
|
||||
|
||||
asyncio.run(check())
|
||||
contents = [m.content for m in result["window"]]
|
||||
assert "turn one" in contents
|
||||
assert any(m.role == "assistant" for m in result["window"])
|
||||
|
||||
|
||||
def test_second_turn_replays_history_to_provider(client):
|
||||
# Observable provider input: a recording provider wrapper captures what
|
||||
# the endpoint sends. Turn 2 must include turn 1's persisted messages.
|
||||
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
provider = client.app.state.provider
|
||||
original = provider.stream_chat
|
||||
captured: list[list[Message]] = []
|
||||
|
||||
async def recording_stream(messages, *, model, temperature=0.7, response_format=None):
|
||||
captured.append(list(messages))
|
||||
async for t in original(
|
||||
messages, model=model, temperature=temperature, response_format=response_format
|
||||
):
|
||||
yield t
|
||||
|
||||
provider.stream_chat = recording_stream
|
||||
try:
|
||||
stream_events(client, {
|
||||
"agent": "coach", "session_id": "sess-replay",
|
||||
"messages": [{"role": "user", "content": "turn one"}],
|
||||
})
|
||||
stream_events(client, {
|
||||
"agent": "coach", "session_id": "sess-replay",
|
||||
"messages": [{"role": "user", "content": "turn two"}],
|
||||
})
|
||||
finally:
|
||||
provider.stream_chat = original
|
||||
|
||||
assert len(captured) == 2
|
||||
turn1, turn2 = captured
|
||||
# Agent routing (P3): provider now receives [system, ...history, user turn]
|
||||
assert turn1[0].role == "system"
|
||||
assert turn1[-1].content == "turn one"
|
||||
assert len(turn1) == 2 # system + first user turn
|
||||
turn2_contents = [m.content for m in turn2]
|
||||
assert "turn one" in turn2_contents
|
||||
assert "turn two" in turn2_contents
|
||||
assert any(m.role == "assistant" for m in turn2) # persisted reply replayed
|
||||
assert "turn two" == turn2_contents[-1] # new user turn last
|
||||
assert turn2[0].role == "system" # every routed call starts with the persona
|
||||
|
||||
|
||||
def test_history_replay_is_windowed(client):
|
||||
# Windowing: only the last 20 stored messages are replayed to the provider.
|
||||
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
provider = client.app.state.provider
|
||||
original = provider.stream_chat
|
||||
captured: list[list[Message]] = []
|
||||
|
||||
async def recording_stream(messages, *, model, temperature=0.7, response_format=None):
|
||||
captured.append(list(messages))
|
||||
async for t in original(
|
||||
messages, model=model, temperature=temperature, response_format=response_format
|
||||
):
|
||||
yield t
|
||||
|
||||
provider.stream_chat = recording_stream
|
||||
try:
|
||||
# 15 turns → 30 persisted messages (user + assistant per turn) > 20 window.
|
||||
for i in range(15):
|
||||
stream_events(client, {
|
||||
"agent": "coach", "session_id": "sess-window",
|
||||
"messages": [{"role": "user", "content": f"turn {i}"}],
|
||||
})
|
||||
finally:
|
||||
provider.stream_chat = original
|
||||
|
||||
last_input = captured[-1]
|
||||
contents = [m.content for m in last_input]
|
||||
# system prompt + window(20) + the new user message
|
||||
assert last_input[0].role == "system"
|
||||
assert len(last_input) == 1 + 20 + 1
|
||||
assert "turn 0" not in contents # oldest messages trimmed out of replay
|
||||
assert "turn 14" in contents
|
||||
assert contents[-1] == "turn 14"
|
||||
|
||||
|
||||
def test_replayed_system_prompt_carries_routed_persona(client):
|
||||
"""History replay puts the routed agent's persona at position 0 (P3).
|
||||
|
||||
The system prompt on every turn — including replays — must match the
|
||||
routed agent: Coach calls get the Coach persona, Tutor calls the Tutor
|
||||
persona, and the two are observably distinct.
|
||||
"""
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
def capture_two_turns(agent_name: str, session: str) -> list[list[Message]]:
|
||||
provider = client.app.state.provider
|
||||
original = provider.stream_chat
|
||||
captured: list[list[Message]] = []
|
||||
|
||||
async def recording_stream(messages, *, model, temperature=0.7, response_format=None):
|
||||
captured.append(list(messages))
|
||||
async for t in original(
|
||||
messages, model=model, temperature=temperature, response_format=response_format
|
||||
):
|
||||
yield t
|
||||
|
||||
provider.stream_chat = recording_stream
|
||||
try:
|
||||
for content in ("turn one", "turn two"):
|
||||
stream_events(client, {
|
||||
"agent": agent_name, "session_id": session,
|
||||
"messages": [{"role": "user", "content": content}],
|
||||
})
|
||||
finally:
|
||||
provider.stream_chat = original
|
||||
return captured
|
||||
|
||||
coach_calls = capture_two_turns("coach", "persona-coach")
|
||||
tutor_calls = capture_two_turns("tutor", "persona-tutor")
|
||||
|
||||
coach_replay_system = coach_calls[1][0]
|
||||
tutor_replay_system = tutor_calls[1][0]
|
||||
assert coach_replay_system.role == "system"
|
||||
assert tutor_replay_system.role == "system"
|
||||
assert "Coach" in coach_replay_system.content
|
||||
assert "Tutor" in tutor_replay_system.content
|
||||
assert coach_replay_system.content != tutor_replay_system.content
|
||||
# persona is stable across turns within one session
|
||||
assert coach_calls[0][0].content == coach_replay_system.content
|
||||
|
||||
|
||||
def test_session_agent_scoped(client):
|
||||
payload = {
|
||||
"agent": "tutor",
|
||||
"session_id": "sess-3",
|
||||
"messages": [{"role": "user", "content": "hello"}],
|
||||
}
|
||||
stream_events(client, payload)
|
||||
import asyncio
|
||||
|
||||
store = client.app.state.session_store
|
||||
|
||||
async def check():
|
||||
return await store.get("sess-3")
|
||||
|
||||
session = asyncio.run(check())
|
||||
assert session.agent == "tutor"
|
||||
|
||||
|
||||
def test_client_retry_does_not_duplicate_user_turn(client):
|
||||
"""P1 fix (final review): resending the same user turn after a failure
|
||||
must not double-append it to session history."""
|
||||
import asyncio
|
||||
|
||||
payload = {
|
||||
"agent": "coach",
|
||||
"session_id": "sess-retry",
|
||||
"messages": [{"role": "user", "content": "same question"}],
|
||||
}
|
||||
# First attempt fails mid-stream (turn was already appended before streaming)
|
||||
provider = client.app.state.provider
|
||||
provider.fail_mid_stream_at_index = 0
|
||||
stream_events_retry(client, payload)
|
||||
provider.fail_mid_stream_at_index = None
|
||||
# Client retry: identical payload
|
||||
stream_events_retry(client, payload)
|
||||
|
||||
store = client.app.state.session_store
|
||||
|
||||
async def check():
|
||||
return await store.history_window("sess-retry")
|
||||
|
||||
contents = [m.content for m in asyncio.run(check())]
|
||||
user_turns = [c for c in contents if c == "same question"]
|
||||
assert len(user_turns) == 1, f"expected exactly 1 stored user turn, got {len(user_turns)}"
|
||||
|
||||
|
||||
def stream_events_retry(client, payload):
|
||||
with client.stream("POST", "/v1/chat/stream", json=payload) as response:
|
||||
for _ in response.iter_lines():
|
||||
pass
|
||||
@@ -0,0 +1,131 @@
|
||||
"""SSE chat stream endpoint tests — envelope ordering, errors, headers."""
|
||||
|
||||
import json
|
||||
|
||||
|
||||
def stream_lines(client, payload: dict) -> list[str]:
|
||||
with client.stream("POST", "/v1/chat/stream", json=payload) as response:
|
||||
assert response.status_code == 200
|
||||
assert "no-cache" in response.headers.get("cache-control", "")
|
||||
return [line for line in response.iter_lines() if line.strip()]
|
||||
|
||||
|
||||
def parse_events(raw_lines: list[str]) -> list[dict]:
|
||||
"""Parse SSE lines into event dicts; strips event:/data: prefixes."""
|
||||
events = []
|
||||
for line in raw_lines:
|
||||
if line.startswith("data:"):
|
||||
data = line.removeprefix("data:").strip()
|
||||
if data == "[DONE]":
|
||||
events.append({"type": "[DONE]"})
|
||||
else:
|
||||
events.append(json.loads(data))
|
||||
return events
|
||||
|
||||
|
||||
PAYLOAD = {
|
||||
"agent": "tutor",
|
||||
"session_id": "s1",
|
||||
"messages": [{"role": "user", "content": "hello"}],
|
||||
}
|
||||
|
||||
|
||||
def test_meta_first_then_deltas_done_done_sentinel(client):
|
||||
events = parse_events(stream_lines(client, PAYLOAD))
|
||||
assert events[0]["type"] == "meta"
|
||||
assert events[0]["agent"] == "tutor"
|
||||
assert events[0]["session_id"] == "s1"
|
||||
assert events[-1]["type"] == "[DONE]"
|
||||
middle = events[1:-1]
|
||||
deltas = [e for e in middle if e["type"] == "delta"]
|
||||
dones = [e for e in middle if e["type"] == "done"]
|
||||
assert len(deltas) >= 1
|
||||
assert len(dones) == 1
|
||||
assert dones[0]["finish_reason"] == "stop"
|
||||
# delta events after meta, done before [DONE]
|
||||
assert events.index(dones[0]) > events.index(deltas[0])
|
||||
|
||||
|
||||
def test_empty_messages_rejected(client):
|
||||
response = client.post(
|
||||
"/v1/chat/stream",
|
||||
json={"agent": "tutor", "session_id": "s", "messages": []},
|
||||
)
|
||||
assert response.status_code == 422
|
||||
|
||||
|
||||
def test_mid_stream_failure_yields_error_then_done(client):
|
||||
provider = client.app.state.provider
|
||||
provider.fail_mid_stream_at_index = 1
|
||||
events = parse_events(stream_lines(client, PAYLOAD))
|
||||
provider.fail_mid_stream_at_index = None
|
||||
error_events = [e for e in events if e["type"] == "error"]
|
||||
assert len(error_events) == 1
|
||||
assert error_events[0]["code"] == "provider_error"
|
||||
assert events[-1]["type"] == "[DONE]"
|
||||
# error must precede the terminal sentinel
|
||||
assert events.index(error_events[0]) < events.index(events[-1])
|
||||
|
||||
|
||||
def test_pre_first_byte_failure_yields_provider_unavailable(client):
|
||||
provider = client.app.state.provider
|
||||
provider.fail_before_first_token = True
|
||||
events = parse_events(stream_lines(client, PAYLOAD))
|
||||
provider.fail_before_first_token = False
|
||||
error_events = [e for e in events if e["type"] == "error"]
|
||||
assert len(error_events) == 1
|
||||
assert error_events[0]["code"] == "provider_unavailable"
|
||||
|
||||
|
||||
def test_unknown_agent_rejected_422(client):
|
||||
response = client.post(
|
||||
"/v1/chat/stream",
|
||||
json={
|
||||
"agent": "oracle",
|
||||
"session_id": "s",
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
},
|
||||
)
|
||||
assert response.status_code == 422
|
||||
assert "oracle" in response.json()["detail"]
|
||||
|
||||
|
||||
def test_coach_routes_and_meta_names_agent(client):
|
||||
payload = {
|
||||
"agent": "coach",
|
||||
"session_id": "route-coach",
|
||||
"messages": [{"role": "user", "content": "pace me"}],
|
||||
}
|
||||
events = parse_events(stream_lines(client, payload))
|
||||
assert events[0]["type"] == "meta"
|
||||
assert events[0]["agent"] == "coach"
|
||||
assert events[-1]["type"] == "[DONE]"
|
||||
|
||||
|
||||
def test_tutor_routes_and_meta_names_agent(client):
|
||||
payload = {
|
||||
"agent": "tutor",
|
||||
"session_id": "route-tutor",
|
||||
"messages": [{"role": "user", "content": "teach me"}],
|
||||
}
|
||||
events = parse_events(stream_lines(client, payload))
|
||||
assert events[0]["type"] == "meta"
|
||||
assert events[0]["agent"] == "tutor"
|
||||
assert events[-1]["type"] == "[DONE]"
|
||||
|
||||
|
||||
def test_coach_and_tutor_streams_are_distinct(client):
|
||||
"""Agent routing selects the right agent: distinct system prompts →
|
||||
distinct hash-seeded mock outputs for the same user input."""
|
||||
same_message = [{"role": "user", "content": "same question"}]
|
||||
coach = parse_events(stream_lines(client, {
|
||||
"agent": "coach", "session_id": "d1", "messages": same_message,
|
||||
}))
|
||||
tutor = parse_events(stream_lines(client, {
|
||||
"agent": "tutor", "session_id": "d2", "messages": same_message,
|
||||
}))
|
||||
coach_text = "".join(e["content"] for e in coach if e["type"] == "delta")
|
||||
tutor_text = "".join(e["content"] for e in tutor if e["type"] == "delta")
|
||||
assert coach_text and tutor_text
|
||||
assert coach_text != tutor_text
|
||||
assert coach[-1]["type"] == "[DONE]"
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Lab feedback endpoint tests — SSE envelope with agent=lab (REQ-2-007)."""
|
||||
|
||||
import json
|
||||
|
||||
|
||||
def stream_events(client, payload) -> list[dict]:
|
||||
with client.stream("POST", "/v1/lab/feedback", json=payload) as response:
|
||||
assert response.status_code == 200
|
||||
events = []
|
||||
for line in response.iter_lines():
|
||||
if line.startswith("data:"):
|
||||
d = line.removeprefix("data:").strip()
|
||||
if d == "[DONE]":
|
||||
events.append({"type": "[DONE]"})
|
||||
else:
|
||||
events.append(json.loads(d))
|
||||
return events
|
||||
|
||||
|
||||
def test_lab_feedback_streams_full_envelope(client):
|
||||
events = stream_events(client, {"scenario_id": "lab-scenario-strong"})
|
||||
assert events[0]["type"] == "meta"
|
||||
assert events[0]["agent"] == "lab"
|
||||
assert events[0]["scenario_id"] == "lab-scenario-strong"
|
||||
deltas = [e for e in events if e["type"] == "delta"]
|
||||
assert len(deltas) >= 1
|
||||
assert any(e["type"] == "done" for e in events)
|
||||
assert events[-1]["type"] == "[DONE]"
|
||||
|
||||
|
||||
def test_unknown_scenario_404(client):
|
||||
response = client.post("/v1/lab/feedback", json={"scenario_id": "nope"})
|
||||
assert response.status_code == 404
|
||||
assert "nope" in response.json()["detail"]
|
||||
|
||||
|
||||
def test_distinct_scenarios_distinct_replies(client):
|
||||
strong = stream_events(client, {"scenario_id": "lab-scenario-strong"})
|
||||
struggling = stream_events(client, {"scenario_id": "lab-scenario-struggling"})
|
||||
strong_text = "".join(e["content"] for e in strong if e["type"] == "delta")
|
||||
struggling_text = "".join(e["content"] for e in struggling if e["type"] == "delta")
|
||||
assert strong_text != struggling_text
|
||||
|
||||
|
||||
def test_missing_scenario_id_422(client):
|
||||
response = client.post("/v1/lab/feedback", json={})
|
||||
assert response.status_code == 422
|
||||
@@ -0,0 +1,56 @@
|
||||
"""Mentor narrative endpoint tests — SSE, session-backed (REQ-2-010)."""
|
||||
|
||||
import json
|
||||
|
||||
|
||||
def stream_events(client, payload) -> list[dict]:
|
||||
with client.stream("POST", "/v1/mentor/narrative", json=payload) as response:
|
||||
assert response.status_code == 200
|
||||
events = []
|
||||
for line in response.iter_lines():
|
||||
if line.startswith("data:"):
|
||||
d = line.removeprefix("data:").strip()
|
||||
if d == "[DONE]":
|
||||
events.append({"type": "[DONE]"})
|
||||
else:
|
||||
events.append(json.loads(d))
|
||||
return events
|
||||
|
||||
|
||||
def test_narrative_streams_full_envelope(client):
|
||||
events = stream_events(client, {"session_id": "mentor-1"})
|
||||
assert events[0]["type"] == "meta"
|
||||
assert events[0]["agent"] == "mentor"
|
||||
deltas = [e for e in events if e["type"] == "delta"]
|
||||
assert len(deltas) >= 1
|
||||
assert any(e["type"] == "done" for e in events)
|
||||
assert events[-1]["type"] == "[DONE]"
|
||||
|
||||
|
||||
def test_narrative_is_session_backed(client):
|
||||
"""Second call replays history: provider input grows; distinct mock output."""
|
||||
first = stream_events(client, {"session_id": "mentor-2", "prompt": "narrate my path"})
|
||||
second = stream_events(client, {"session_id": "mentor-2", "prompt": "what next?"})
|
||||
first_text = "".join(e["content"] for e in first if e["type"] == "delta")
|
||||
second_text = "".join(e["content"] for e in second if e["type"] == "delta")
|
||||
assert first_text != second_text
|
||||
|
||||
|
||||
def test_narrative_persists_turns(client):
|
||||
import asyncio
|
||||
|
||||
store = client.app.state.session_store
|
||||
|
||||
stream_events(client, {"session_id": "mentor-3", "prompt": "hello trajectory"})
|
||||
|
||||
async def check():
|
||||
return await store.history_window("mentor-3")
|
||||
|
||||
contents = [m.content for m in asyncio.run(check())]
|
||||
assert "hello trajectory" in contents
|
||||
assert len(contents) >= 2 # user + assistant persisted
|
||||
|
||||
|
||||
def test_missing_session_id_422(client):
|
||||
response = client.post("/v1/mentor/narrative", json={"prompt": "hi"})
|
||||
assert response.status_code == 422
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Proctor signals endpoint tests — validated JSON, 404s (REQ-2-009)."""
|
||||
|
||||
from ai_service.agents.proctor import ProctorAssessment
|
||||
from ai_service.llm.mock import ScriptedJSONProvider
|
||||
|
||||
VALID = {
|
||||
"scenario_id": "proctor-scenario-distracted",
|
||||
"signals": [
|
||||
{"signal_type": "context_switch", "severity": "low",
|
||||
"note": "Docs tab at t+120s is normal"},
|
||||
{"signal_type": "idle_gap", "severity": "medium",
|
||||
"note": "5-minute idle at t+300s"},
|
||||
],
|
||||
"intervention": "Offer a short break, then restate the plan",
|
||||
"summary": "Coaching-shaped session note",
|
||||
}
|
||||
|
||||
|
||||
def test_signals_returns_validated_json(client):
|
||||
original = client.app.state.provider
|
||||
client.app.state.provider = ScriptedJSONProvider(VALID)
|
||||
try:
|
||||
response = client.post(
|
||||
"/v1/proctor/signals", json={"scenario_id": "proctor-scenario-distracted"}
|
||||
)
|
||||
finally:
|
||||
client.app.state.provider = original
|
||||
assert response.status_code == 200
|
||||
validated = ProctorAssessment.model_validate(response.json())
|
||||
assert validated.scenario_id == "proctor-scenario-distracted"
|
||||
assert validated.intervention
|
||||
|
||||
|
||||
def test_unknown_scenario_404(client):
|
||||
response = client.post("/v1/proctor/signals", json={"scenario_id": "ghost"})
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
def test_unparseable_provider_502(client):
|
||||
response = client.post(
|
||||
"/v1/proctor/signals", json={"scenario_id": "proctor-scenario-healthy"}
|
||||
)
|
||||
assert response.status_code == 502
|
||||
assert "failed" in response.json()["detail"].lower()
|
||||
|
||||
|
||||
def test_missing_scenario_id_422(client):
|
||||
response = client.post("/v1/proctor/signals", json={})
|
||||
assert response.status_code == 422
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Client-disconnect regression tests — SSE generators must tolerate aclose().
|
||||
|
||||
A `yield` inside `finally` re-raises "async generator ignored GeneratorExit"
|
||||
when sse-starlette closes the iterator on client disconnect (P0 finding,
|
||||
final review). These tests reproduce the close path directly against each
|
||||
endpoint's event_stream generator shape.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
from ai_service.corpus.learner_context import get_learner_context
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
|
||||
async def _chat_event_stream(client, session_id="close-chat", content="hi"):
|
||||
"""Rebuild the chat endpoint's event_stream generator exactly as
|
||||
chat.py builds it (same code shape, same session flow)."""
|
||||
app = client.app
|
||||
settings = app.state.settings
|
||||
provider = app.state.provider
|
||||
registry = app.state.agent_registry
|
||||
sessions = app.state.session_store
|
||||
agent = registry.get(provider, settings, "tutor")
|
||||
learner_context = get_learner_context(None)
|
||||
|
||||
if await sessions.get(session_id) is None:
|
||||
await sessions.create(session_id, agent="tutor", learner_id="learner-001")
|
||||
user_turn = Message(role="user", content=content)
|
||||
history = await sessions.history_window(session_id)
|
||||
await sessions.append(session_id, user_turn)
|
||||
|
||||
async def event_stream():
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "meta", "agent": "tutor", "session_id": session_id,
|
||||
"model": settings.model,
|
||||
})}
|
||||
first_byte = True
|
||||
reply_parts: list[str] = []
|
||||
try:
|
||||
async for token in agent.stream_reply(
|
||||
history=history, user_input=user_turn.content,
|
||||
learner_context=learner_context,
|
||||
):
|
||||
first_byte = False
|
||||
reply_parts.append(token)
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "delta", "content": token
|
||||
})}
|
||||
full_reply = "".join(reply_parts)
|
||||
if full_reply:
|
||||
await sessions.append(
|
||||
session_id, Message(role="assistant", content=full_reply)
|
||||
)
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "done", "finish_reason": "stop"
|
||||
})}
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
except Exception as exc:
|
||||
code = "provider_unavailable" if first_byte else "provider_error"
|
||||
yield {"event": "message", "data": json.dumps({
|
||||
"type": "error", "code": code, "message": str(exc)
|
||||
})}
|
||||
yield {"event": "message", "data": "[DONE]"}
|
||||
|
||||
return event_stream()
|
||||
|
||||
|
||||
async def test_chat_stream_generator_survives_aclose(client):
|
||||
"""Partial consumption then aclose() must not raise
|
||||
'async generator ignored GeneratorExit' (yield-in-finally regression)."""
|
||||
gen = await _chat_event_stream(client, session_id="close-chat")
|
||||
meta = await gen.__anext__()
|
||||
assert json.loads(meta["data"])["type"] == "meta"
|
||||
delta = await gen.__anext__()
|
||||
assert json.loads(delta["data"])["type"] == "delta"
|
||||
# The critical assertion: closing mid-stream must be clean (no raise).
|
||||
await gen.aclose()
|
||||
|
||||
|
||||
async def test_chat_stream_survives_close_at_different_points(client):
|
||||
"""Close right after meta, and right after done — all must be clean."""
|
||||
gen = await _chat_event_stream(client, session_id="close-early")
|
||||
await gen.__anext__() # meta only
|
||||
await gen.aclose()
|
||||
|
||||
gen2 = await _chat_event_stream(client, session_id="close-late")
|
||||
events = []
|
||||
async for ev in gen2:
|
||||
events.append(json.loads(ev["data"]))
|
||||
if len(events) == 2:
|
||||
break
|
||||
await gen2.aclose()
|
||||
assert events[0]["type"] == "meta"
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Test suite — conftest: mock provider only, zero network (enforced)."""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from ai_service.config import Settings
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.main import create_app
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def settings() -> Settings:
|
||||
os.environ["AI_PROVIDER"] = "mock"
|
||||
return Settings(provider="mock", model="gemma4:31b", port=8421)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def app(settings: Settings):
|
||||
return create_app(settings)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(app):
|
||||
with TestClient(app) as c:
|
||||
# Mechanical cloud-free guard (GRILL advisory a): the app under test
|
||||
# MUST be wired to the deterministic mock provider.
|
||||
assert isinstance(app.state.provider, MockProvider), (
|
||||
f"tests must run against MockProvider, got {type(app.state.provider).__name__}"
|
||||
)
|
||||
yield c
|
||||
@@ -0,0 +1,73 @@
|
||||
"""MockProvider tests — determinism, JSON mode, failure modes."""
|
||||
|
||||
import pytest
|
||||
|
||||
from ai_service.llm.mock import MockProvider
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
MSGS = [Message(role="user", content="hello there")]
|
||||
|
||||
|
||||
async def test_stream_is_deterministic():
|
||||
p1, p2 = MockProvider(), MockProvider()
|
||||
out1 = [t async for t in p1.stream_chat(MSGS, model="m")]
|
||||
out2 = [t async for t in p2.stream_chat(MSGS, model="m")]
|
||||
assert "".join(out1) == "".join(out2)
|
||||
assert out1 == out2
|
||||
|
||||
|
||||
async def test_stream_content_differs_for_different_input():
|
||||
p = MockProvider()
|
||||
a = "".join([t async for t in p.stream_chat(MSGS, model="m")])
|
||||
b = "".join(
|
||||
[t async for t in p.stream_chat([Message(role="user", content="other")], model="m")]
|
||||
)
|
||||
assert a != b
|
||||
|
||||
|
||||
async def test_json_object_response_format():
|
||||
import json
|
||||
|
||||
p = MockProvider()
|
||||
out = "".join(
|
||||
[
|
||||
t
|
||||
async for t in p.stream_chat(
|
||||
MSGS, model="m", response_format={"type": "json_object"}
|
||||
)
|
||||
]
|
||||
)
|
||||
assert json.loads(out) == {"summary": "mock structured reply", "confidence": 0.87}
|
||||
|
||||
|
||||
async def test_fail_before_first_token():
|
||||
p = MockProvider()
|
||||
p.fail_before_first_token = True
|
||||
with pytest.raises(RuntimeError):
|
||||
async for _ in p.stream_chat(MSGS, model="m"):
|
||||
pass
|
||||
|
||||
|
||||
async def test_fail_mid_stream():
|
||||
p = MockProvider()
|
||||
p.fail_mid_stream_at_index = 2
|
||||
tokens = []
|
||||
with pytest.raises(RuntimeError):
|
||||
async for t in p.stream_chat(MSGS, model="m"):
|
||||
tokens.append(t)
|
||||
assert len(tokens) == 2
|
||||
|
||||
|
||||
async def test_cancellation_records_abort():
|
||||
p = MockProvider()
|
||||
gen = p.stream_chat(MSGS, model="m")
|
||||
await gen.__anext__()
|
||||
await gen.aclose()
|
||||
assert p.abort_recorded is True
|
||||
|
||||
|
||||
async def test_chat_returns_full_reply():
|
||||
p = MockProvider()
|
||||
reply = await p.chat(MSGS, model="m")
|
||||
streamed = "".join([t async for t in p.stream_chat(MSGS, model="m")])
|
||||
assert reply == streamed
|
||||
@@ -0,0 +1,177 @@
|
||||
"""OpenAICompatProvider tests — byte-exact SSE parsing via httpx.MockTransport.
|
||||
|
||||
Covers: multi-delta happy path, keep-alive comment lines, [DONE] sentinel,
|
||||
malformed line tolerance, missing optional fields, non-streaming chat(),
|
||||
response_format auto-degrade on 400, api_key never leaking into exceptions.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from ai_service.llm.openai_compat import OpenAICompatProvider
|
||||
from ai_service.llm.types import Message
|
||||
|
||||
MSGS = [Message(role="user", content="hi")]
|
||||
KEY = "sk-test-abc123"
|
||||
|
||||
|
||||
def make_client(handler) -> httpx.AsyncClient:
|
||||
transport = httpx.MockTransport(handler)
|
||||
return httpx.AsyncClient(transport=transport)
|
||||
|
||||
|
||||
def sse_body(deltas: list[str], with_comments: bool = True) -> bytes:
|
||||
lines = []
|
||||
if with_comments:
|
||||
lines.append(": ping")
|
||||
for d in deltas:
|
||||
lines.append("data: " + json.dumps({
|
||||
"id": "chatcmpl-1", "object": "chat.completion.chunk",
|
||||
"created": 1, "model": "gemma4:31b",
|
||||
"choices": [{"index": 0, "delta": {"content": d}, "finish_reason": None}],
|
||||
}))
|
||||
if with_comments:
|
||||
lines.append(": ping")
|
||||
lines.append("data: [DONE]")
|
||||
return ("\n".join(lines) + "\n").encode()
|
||||
|
||||
|
||||
async def test_stream_happy_path_with_comments_and_done():
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
assert request.url.path == "/v1/chat/completions"
|
||||
return httpx.Response(200, content=sse_body(["Hel", "lo", " world"]))
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1")
|
||||
tokens = [t async for t in provider.stream_chat(MSGS, model="gemma4:31b")]
|
||||
assert tokens == ["Hel", "lo", " world"]
|
||||
|
||||
|
||||
async def test_stream_tolerates_malformed_lines():
|
||||
body = (
|
||||
"data: not-json\n"
|
||||
"data: "
|
||||
+ json.dumps({
|
||||
"id": "x", "object": "chat.completion.chunk", "created": 1, "model": "m",
|
||||
"choices": [{"index": 0, "delta": {"content": "ok"}, "finish_reason": None}],
|
||||
})
|
||||
+ "\ndata: [DONE]\n"
|
||||
)
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(200, content=body.encode())
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1")
|
||||
tokens = [t async for t in provider.stream_chat(MSGS, model="m")]
|
||||
assert tokens == ["ok"]
|
||||
|
||||
|
||||
async def test_stream_skips_empty_content_and_empty_choices():
|
||||
chunk_empty_delta = json.dumps(
|
||||
{"id": "x", "model": "m",
|
||||
"choices": [{"index": 0, "delta": {}, "finish_reason": None}]}
|
||||
)
|
||||
chunk_empty_choices = json.dumps({"id": "x", "model": "m", "choices": []})
|
||||
chunk_yes = json.dumps(
|
||||
{"id": "x", "model": "m",
|
||||
"choices": [{"index": 0, "delta": {"content": "yes"}, "finish_reason": None}]}
|
||||
)
|
||||
body = (
|
||||
f"data: {chunk_empty_delta}\n"
|
||||
f"data: {chunk_empty_choices}\n"
|
||||
f"data: {chunk_yes}\n"
|
||||
"data: [DONE]\n"
|
||||
)
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(200, content=body.encode())
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1")
|
||||
tokens = [t async for t in provider.stream_chat(MSGS, model="m")]
|
||||
assert tokens == ["yes"]
|
||||
|
||||
|
||||
async def test_chat_non_streaming():
|
||||
def handler(request):
|
||||
payload = json.loads(request.content)
|
||||
assert payload["stream"] is False
|
||||
body = {"id": "1", "object": "chat.completion", "created": 1, "model": "m",
|
||||
"choices": [{"index": 0, "message": {"role": "assistant",
|
||||
"content": "full reply"},
|
||||
"finish_reason": "stop"}]}
|
||||
return httpx.Response(200, json=body)
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1")
|
||||
assert await provider.chat(MSGS, model="m") == "full reply"
|
||||
|
||||
|
||||
async def test_response_format_auto_degrades_on_400():
|
||||
calls = []
|
||||
|
||||
def handler(request):
|
||||
payload = json.loads(request.content)
|
||||
calls.append(payload)
|
||||
if "response_format" in payload:
|
||||
return httpx.Response(400, json={"error": "response_format unsupported"})
|
||||
body = {"id": "1", "object": "chat.completion", "created": 1, "model": "m",
|
||||
"choices": [{"index": 0, "message": {"role": "assistant",
|
||||
"content": "json"},
|
||||
"finish_reason": "stop"}]}
|
||||
return httpx.Response(200, json=body)
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1", json_mode="auto")
|
||||
reply = await provider.chat(MSGS, model="m", response_format={"type": "json_object"})
|
||||
assert reply == "json"
|
||||
assert len(calls) == 2
|
||||
assert "response_format" in calls[0]
|
||||
assert "response_format" not in calls[1]
|
||||
|
||||
|
||||
async def test_response_format_off_never_sends():
|
||||
calls = []
|
||||
|
||||
def handler(request):
|
||||
payload = json.loads(request.content)
|
||||
calls.append(payload)
|
||||
body = {"id": "1", "object": "chat.completion", "created": 1, "model": "m",
|
||||
"choices": [{"index": 0, "message": {"role": "assistant",
|
||||
"content": "x"},
|
||||
"finish_reason": "stop"}]}
|
||||
return httpx.Response(200, json=body)
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1", json_mode="off")
|
||||
await provider.chat(MSGS, model="m", response_format={"type": "json_object"})
|
||||
assert len(calls) == 1
|
||||
assert "response_format" not in calls[0]
|
||||
|
||||
|
||||
async def test_api_key_never_in_exception():
|
||||
def handler(request):
|
||||
raise httpx.ConnectError("connection refused while using sk-test-abc123")
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1", api_key=KEY)
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
async for _ in provider.stream_chat(MSGS, model="m"):
|
||||
pass
|
||||
assert "sk-test-abc123" not in str(exc_info.value)
|
||||
|
||||
|
||||
async def test_bearer_header_sent():
|
||||
seen = {}
|
||||
|
||||
def handler(request):
|
||||
seen["auth"] = request.headers.get("Authorization")
|
||||
return httpx.Response(200, content=sse_body(["x"]))
|
||||
|
||||
async with make_client(handler) as client:
|
||||
provider = OpenAICompatProvider(client, "https://fake/v1", api_key=KEY)
|
||||
_ = [t async for t in provider.stream_chat(MSGS, model="m")]
|
||||
assert seen["auth"] == f"Bearer {KEY}"
|
||||
@@ -0,0 +1,10 @@
|
||||
"""Health endpoint tests."""
|
||||
|
||||
|
||||
def test_health_returns_ok(client):
|
||||
response = client.get("/health")
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "ok"
|
||||
assert data["provider"] == "mock"
|
||||
assert data["model"] == "gemma4:31b"
|
||||
@@ -0,0 +1,2 @@
|
||||
# AI service (v0.2) — learner chat/panels stream from this FastAPI service
|
||||
NEXT_PUBLIC_AI_SERVICE_URL=http://localhost:8420
|
||||
@@ -0,0 +1,28 @@
|
||||
import type { StorybookConfig } from '@storybook/nextjs';
|
||||
|
||||
/**
|
||||
* Storybook config for the Nextcraft monorepo.
|
||||
*
|
||||
* Stories live alongside the UI primitives in `packages/ui` (referenced
|
||||
* through the `@nextcraft/ui` workspace symlink so Storybook's webpack
|
||||
* resolver finds them inside the app's module graph) and next to composite
|
||||
* components in `apps/web/components`. The Next.js framework preset bundles
|
||||
* the essential addons (controls, docs, actions, viewport, backgrounds,
|
||||
* toolbars) so we don't add `@storybook/addon-essentials` separately.
|
||||
*/
|
||||
const config: StorybookConfig = {
|
||||
stories: [
|
||||
'../node_modules/@nextcraft/ui/src/**/*.stories.@(ts|tsx|mdx)',
|
||||
'../components/**/*.stories.@(ts|tsx|mdx)',
|
||||
],
|
||||
addons: [],
|
||||
framework: {
|
||||
name: '@storybook/nextjs',
|
||||
options: {},
|
||||
},
|
||||
docs: {
|
||||
autodocs: true,
|
||||
},
|
||||
};
|
||||
|
||||
export default config;
|
||||
@@ -0,0 +1,38 @@
|
||||
import type { Preview } from '@storybook/react';
|
||||
import '../app/globals.css';
|
||||
|
||||
/**
|
||||
* Storybook preview — imports the app's Tailwind stylesheet so stories
|
||||
* render with the full design-token system (colors, fonts, dark mode).
|
||||
*/
|
||||
const preview: Preview = {
|
||||
parameters: {
|
||||
controls: {
|
||||
matchers: {
|
||||
color: /(background|color)$/i,
|
||||
date: /Date$/i,
|
||||
},
|
||||
},
|
||||
backgrounds: {
|
||||
default: 'light',
|
||||
values: [
|
||||
{ name: 'light', value: '#ffffff' },
|
||||
{ name: 'dark', value: '#020617' },
|
||||
],
|
||||
},
|
||||
layout: 'padded',
|
||||
},
|
||||
// Apply the `dark` class to the story root when the dark background is
|
||||
// active so class-based dark: variants resolve.
|
||||
decorators: [
|
||||
(Story, context) => {
|
||||
const isDark = context.globals?.backgrounds?.value === '#020617';
|
||||
if (typeof document !== 'undefined') {
|
||||
document.documentElement.classList.toggle('dark', isDark);
|
||||
}
|
||||
return <Story />;
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
export default preview;
|
||||
@@ -0,0 +1,17 @@
|
||||
import { CompetencyGraph } from '../../../../components/admin/competency-graph';
|
||||
|
||||
export default function CompetencyGraphPage() {
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
<header className="flex flex-col gap-1">
|
||||
<h1 className="text-2xl font-bold text-slate-900 dark:text-slate-100">
|
||||
Competency Graph
|
||||
</h1>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">
|
||||
Visualize stack relationships and dependencies
|
||||
</p>
|
||||
</header>
|
||||
<CompetencyGraph />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
import { LearnerTable } from '../../../../components/admin/learner-table';
|
||||
import { adminLearners } from '@nextcraft/mock-data';
|
||||
|
||||
export default function LearnerManagementPage() {
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
<header className="flex flex-col gap-1">
|
||||
<h1 className="text-2xl font-bold text-slate-900 dark:text-slate-100">
|
||||
Learner Management
|
||||
</h1>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">
|
||||
Track and manage learner progress
|
||||
</p>
|
||||
</header>
|
||||
<LearnerTable learners={adminLearners} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
import { ModerationTabs } from '../../../../components/admin/moderation-tabs';
|
||||
|
||||
export default function MarketplaceModerationPage() {
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
<header className="flex flex-col gap-1">
|
||||
<h1 className="text-2xl font-bold text-slate-900 dark:text-slate-100">
|
||||
Marketplace Moderation
|
||||
</h1>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">
|
||||
Review and manage marketplace content
|
||||
</p>
|
||||
</header>
|
||||
<ModerationTabs />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,244 @@
|
||||
import {
|
||||
Activity,
|
||||
AlertTriangle,
|
||||
Briefcase,
|
||||
Building2,
|
||||
CheckCircle,
|
||||
Clock,
|
||||
Cpu,
|
||||
Database,
|
||||
GraduationCap,
|
||||
Server,
|
||||
ThumbsUp,
|
||||
Users,
|
||||
} from 'lucide-react';
|
||||
import type { LucideIcon } from 'lucide-react';
|
||||
import { Card, CardBody, CardHeader } from '@nextcraft/ui';
|
||||
import {
|
||||
activityFeed,
|
||||
platformMetrics,
|
||||
serviceHealth,
|
||||
systemStats,
|
||||
uptimeBars,
|
||||
} from '@nextcraft/mock-data';
|
||||
|
||||
const ICONS: Record<string, LucideIcon> = {
|
||||
Users,
|
||||
Building2,
|
||||
Briefcase,
|
||||
GraduationCap,
|
||||
ThumbsUp,
|
||||
Activity,
|
||||
Server,
|
||||
Database,
|
||||
Cpu,
|
||||
};
|
||||
|
||||
const TONE_CLASSES: Record<string, string> = {
|
||||
indigo: 'bg-indigo-100 text-indigo-700 dark:bg-indigo-900/40 dark:text-indigo-300',
|
||||
emerald: 'bg-emerald-100 text-emerald-700 dark:bg-emerald-900/40 dark:text-emerald-300',
|
||||
amber: 'bg-amber-100 text-amber-700 dark:bg-amber-900/40 dark:text-amber-300',
|
||||
rose: 'bg-rose-100 text-rose-700 dark:bg-rose-900/40 dark:text-rose-300',
|
||||
cyan: 'bg-cyan-100 text-cyan-700 dark:bg-cyan-900/40 dark:text-cyan-300',
|
||||
violet: 'bg-violet-100 text-violet-700 dark:bg-violet-900/40 dark:text-violet-300',
|
||||
};
|
||||
|
||||
function statusDot(status: string): string {
|
||||
if (status === 'operational') return 'bg-emerald-500';
|
||||
if (status === 'degraded') return 'bg-amber-500';
|
||||
return 'bg-rose-500';
|
||||
}
|
||||
|
||||
function statusLabel(status: string): string {
|
||||
if (status === 'operational') return 'Operational';
|
||||
if (status === 'degraded') return 'Degraded';
|
||||
return 'Down';
|
||||
}
|
||||
|
||||
export default function AdminOverviewPage() {
|
||||
return (
|
||||
<div className="flex flex-col gap-8">
|
||||
{/* Page header */}
|
||||
<header className="flex flex-col gap-1">
|
||||
<h1 className="text-2xl font-bold text-slate-900 dark:text-slate-100">
|
||||
Admin Dashboard
|
||||
</h1>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">
|
||||
Platform overview
|
||||
</p>
|
||||
</header>
|
||||
|
||||
{/* Metric cards */}
|
||||
<section
|
||||
aria-label="Platform metrics"
|
||||
className="grid grid-cols-2 gap-4 md:grid-cols-3 xl:grid-cols-5"
|
||||
>
|
||||
{platformMetrics.map((m) => {
|
||||
const Icon = ICONS[m.icon] ?? Activity;
|
||||
const positive = m.trendPct >= 0;
|
||||
return (
|
||||
<Card key={m.id}>
|
||||
<CardBody className="flex flex-col gap-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span
|
||||
className={`inline-flex h-9 w-9 items-center justify-center rounded-full ${TONE_CLASSES[m.tone]}`}
|
||||
aria-hidden
|
||||
>
|
||||
<Icon className="h-4 w-4" />
|
||||
</span>
|
||||
<span
|
||||
className={`text-xs font-medium ${positive ? 'text-emerald-600 dark:text-emerald-400' : 'text-rose-600 dark:text-rose-400'}`}
|
||||
>
|
||||
{positive ? '+' : ''}
|
||||
{m.trendPct}%
|
||||
</span>
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-medium uppercase tracking-wide text-slate-500 dark:text-slate-400">
|
||||
{m.label}
|
||||
</p>
|
||||
<p className="mt-0.5 text-2xl font-bold text-slate-900 dark:text-slate-100">
|
||||
{m.value.toLocaleString()}
|
||||
{m.suffix}
|
||||
</p>
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
);
|
||||
})}
|
||||
</section>
|
||||
|
||||
{/* Activity + System health */}
|
||||
<section className="grid grid-cols-1 gap-6 lg:grid-cols-3">
|
||||
{/* Activity feed — 2/3 width */}
|
||||
<div className="lg:col-span-2">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<div className="flex items-center gap-2">
|
||||
<Activity className="h-4 w-4 text-primary-600" aria-hidden />
|
||||
<h2 className="text-base font-semibold text-slate-900 dark:text-slate-100">
|
||||
Activity Feed
|
||||
</h2>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardBody className="p-0">
|
||||
<ol className="flex flex-col">
|
||||
{activityFeed.map((evt, i) => {
|
||||
const Icon = ICONS[evt.icon] ?? Activity;
|
||||
const last = i === activityFeed.length - 1;
|
||||
return (
|
||||
<li
|
||||
key={evt.id}
|
||||
className={`flex items-start gap-3 px-6 py-3 ${last ? '' : 'border-b border-slate-100 dark:border-slate-800'}`}
|
||||
>
|
||||
<span
|
||||
className={`mt-0.5 inline-flex h-8 w-8 shrink-0 items-center justify-center rounded-full ${TONE_CLASSES[evt.tone]}`}
|
||||
aria-hidden
|
||||
>
|
||||
<Icon className="h-4 w-4" />
|
||||
</span>
|
||||
<div className="flex flex-1 flex-col gap-0.5">
|
||||
<p className="text-sm text-slate-700 dark:text-slate-200">
|
||||
{evt.message}
|
||||
</p>
|
||||
<span className="flex items-center gap-1 text-xs text-slate-500 dark:text-slate-400">
|
||||
<Clock className="h-3 w-3" aria-hidden />
|
||||
{evt.relativeTime}
|
||||
</span>
|
||||
</div>
|
||||
</li>
|
||||
);
|
||||
})}
|
||||
</ol>
|
||||
</CardBody>
|
||||
</Card>
|
||||
</div>
|
||||
|
||||
{/* System health — 1/3 width */}
|
||||
<div>
|
||||
<Card className="h-full">
|
||||
<CardHeader>
|
||||
<div className="flex items-center gap-2">
|
||||
<Server className="h-4 w-4 text-primary-600" aria-hidden />
|
||||
<h2 className="text-base font-semibold text-slate-900 dark:text-slate-100">
|
||||
System Health
|
||||
</h2>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardBody className="flex flex-col gap-5">
|
||||
<ul className="flex flex-col gap-2.5">
|
||||
{serviceHealth.map((svc) => (
|
||||
<li
|
||||
key={svc.id}
|
||||
className="flex items-center justify-between gap-2"
|
||||
>
|
||||
<span className="text-sm text-slate-700 dark:text-slate-200">
|
||||
{svc.name}
|
||||
</span>
|
||||
<span className="flex items-center gap-2">
|
||||
<span
|
||||
className={`h-2.5 w-2.5 rounded-full ${statusDot(svc.status)}`}
|
||||
aria-hidden
|
||||
/>
|
||||
<span
|
||||
className={`text-xs font-medium ${svc.status === 'operational' ? 'text-emerald-600 dark:text-emerald-400' : svc.status === 'degraded' ? 'text-amber-600 dark:text-amber-400' : 'text-rose-600 dark:text-rose-400'}`}
|
||||
>
|
||||
{statusLabel(svc.status)}
|
||||
</span>
|
||||
</span>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
|
||||
<div className="flex flex-col gap-1.5">
|
||||
<div className="flex items-center justify-between">
|
||||
<h3 className="text-xs font-semibold uppercase tracking-wide text-slate-500 dark:text-slate-400">
|
||||
Uptime (30d)
|
||||
</h3>
|
||||
<span className="text-xs font-medium text-slate-600 dark:text-slate-300">
|
||||
99.96%
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex h-8 items-end gap-0.5" aria-hidden>
|
||||
{uptimeBars.map((u, idx) => {
|
||||
const h = Math.max(20, Math.round((u - 99.8) * 100 * 4));
|
||||
const degraded = u < 99.95;
|
||||
return (
|
||||
<span
|
||||
key={idx}
|
||||
className={`flex-1 rounded-sm ${degraded ? 'bg-amber-400' : 'bg-emerald-400'}`}
|
||||
style={{ height: `${Math.min(100, h)}%` }}
|
||||
title={`${u.toFixed(2)}%`}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-2 gap-3 border-t border-slate-200 pt-4 dark:border-slate-800">
|
||||
<div className="flex flex-col gap-0.5">
|
||||
<span className="text-xs text-slate-500 dark:text-slate-400">
|
||||
Error Rate
|
||||
</span>
|
||||
<span className="flex items-center gap-1 text-sm font-semibold text-slate-900 dark:text-slate-100">
|
||||
<AlertTriangle className="h-3.5 w-3.5 text-amber-500" aria-hidden />
|
||||
{systemStats.errorRate}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex flex-col gap-0.5">
|
||||
<span className="text-xs text-slate-500 dark:text-slate-400">
|
||||
Avg Response Time
|
||||
</span>
|
||||
<span className="flex items-center gap-1 text-sm font-semibold text-slate-900 dark:text-slate-100">
|
||||
<CheckCircle className="h-3.5 w-3.5 text-emerald-500" aria-hidden />
|
||||
{systemStats.avgResponseMs}ms
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
'use client';
|
||||
|
||||
import type { ReactNode } from 'react';
|
||||
import Link from 'next/link';
|
||||
import { usePathname } from 'next/navigation';
|
||||
import { Shield } from 'lucide-react';
|
||||
|
||||
const LINKS = [
|
||||
{ label: 'Overview', href: '/admin' },
|
||||
{ label: 'Learners', href: '/admin/learners' },
|
||||
{ label: 'Competency Graph', href: '/admin/graph' },
|
||||
{ label: 'Moderation', href: '/admin/moderation' },
|
||||
];
|
||||
|
||||
export default function AdminLayout({ children }: { children: ReactNode }) {
|
||||
const pathname = usePathname() ?? '';
|
||||
|
||||
return (
|
||||
<div className="mx-auto flex max-w-7xl gap-6 px-4 py-8 sm:px-6">
|
||||
<aside className="hidden w-56 shrink-0 md:block">
|
||||
<div className="mb-4 flex items-center gap-2 text-sm font-semibold text-slate-500 dark:text-slate-400">
|
||||
<Shield className="h-4 w-4 text-rose-600" />
|
||||
Admin surface
|
||||
</div>
|
||||
<nav className="flex flex-col gap-1">
|
||||
{LINKS.map((l) => {
|
||||
const isActive = pathname === l.href || pathname.startsWith(l.href + '/');
|
||||
return (
|
||||
<Link
|
||||
key={l.href}
|
||||
href={l.href}
|
||||
aria-current={isActive ? 'page' : undefined}
|
||||
className={`rounded-md px-3 py-2 text-sm font-medium transition-colors ${
|
||||
isActive
|
||||
? 'bg-rose-50 text-rose-700 dark:bg-rose-900/30 dark:text-rose-200'
|
||||
: 'text-slate-700 hover:bg-slate-100 dark:text-slate-200 dark:hover:bg-slate-800'
|
||||
}`}
|
||||
>
|
||||
{l.label}
|
||||
</Link>
|
||||
);
|
||||
})}
|
||||
</nav>
|
||||
</aside>
|
||||
<div className="flex-1">{children}</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
import Link from 'next/link';
|
||||
import {
|
||||
Briefcase,
|
||||
Users,
|
||||
TrendingUp,
|
||||
Target,
|
||||
ArrowRight,
|
||||
} from 'lucide-react';
|
||||
import { Badge, Card, CardBody, Avatar } from '@nextcraft/ui';
|
||||
import { candidates, employers } from '@nextcraft/mock-data';
|
||||
import { AnalyticsCharts } from '../../../components/employer/analytics-charts';
|
||||
import { matchBadgeClasses, initials } from '../../../lib/format';
|
||||
|
||||
/* -------------------------------------------------------------------------- */
|
||||
/* Mock pipeline + metric data */
|
||||
/* -------------------------------------------------------------------------- */
|
||||
|
||||
const METRICS = [
|
||||
{ label: 'Active Postings', value: 7, trend: '+12%', icon: Briefcase, tone: 'primary' as const },
|
||||
{ label: 'Total Applicants', value: 142, trend: '+24%', icon: Users, tone: 'accent' as const },
|
||||
{ label: 'Talent Matches', value: 28, trend: '+8%', icon: Target, tone: 'violet' as const },
|
||||
{ label: 'Placement Rate', value: '18%', trend: '+3%', icon: TrendingUp, tone: 'amber' as const },
|
||||
];
|
||||
|
||||
const TONE_CLASSES: Record<string, string> = {
|
||||
primary: 'bg-primary-100 text-primary-700 dark:bg-primary-900/40 dark:text-primary-300',
|
||||
accent: 'bg-emerald-100 text-emerald-700 dark:bg-emerald-900/40 dark:text-emerald-300',
|
||||
violet: 'bg-violet-100 text-violet-700 dark:bg-violet-900/40 dark:text-violet-300',
|
||||
amber: 'bg-amber-100 text-amber-700 dark:bg-amber-900/40 dark:text-amber-300',
|
||||
};
|
||||
|
||||
interface PipelineStage {
|
||||
name: string;
|
||||
count: number;
|
||||
cards: Array<{ name: string; position: string; score: number }>;
|
||||
}
|
||||
|
||||
const PIPELINE: PipelineStage[] = [
|
||||
{
|
||||
name: 'Applied',
|
||||
count: 8,
|
||||
cards: [
|
||||
{ name: 'Maya Okonkwo', position: 'AI Orchestration Engineer', score: 96 },
|
||||
{ name: 'Devon Park', position: 'AI Safety Researcher', score: 91 },
|
||||
{ name: 'Priya Iyer', position: 'Human-AI Designer', score: 89 },
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'Screening',
|
||||
count: 5,
|
||||
cards: [
|
||||
{ name: 'Tomás Vega', position: 'LLM App Developer', score: 93 },
|
||||
{ name: 'Sofia Marchetti', position: 'AI Governance Lead', score: 85 },
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'Interview',
|
||||
count: 3,
|
||||
cards: [
|
||||
{ name: 'Liam Chen', position: 'Agent Reliability Eng', score: 90 },
|
||||
{ name: 'Yuki Tanaka', position: 'Evaluation Engineer', score: 86 },
|
||||
],
|
||||
},
|
||||
{
|
||||
name: 'Offer',
|
||||
count: 2,
|
||||
cards: [{ name: 'Ingrid Solberg', position: 'AI Red Team Lead', score: 88 }],
|
||||
},
|
||||
{
|
||||
name: 'Hired',
|
||||
count: 1,
|
||||
cards: [{ name: 'Hana Lindqvist', position: 'Comp Biologist', score: 88 }],
|
||||
},
|
||||
];
|
||||
|
||||
/* -------------------------------------------------------------------------- */
|
||||
/* Page */
|
||||
/* -------------------------------------------------------------------------- */
|
||||
|
||||
export default function EmployerDashboardPage() {
|
||||
const employer = employers[0];
|
||||
const topMatches = [...candidates].sort((a, b) => b.matchScore - a.matchScore).slice(0, 5);
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-8">
|
||||
{/* Header */}
|
||||
<header className="flex flex-col gap-2">
|
||||
<Badge variant="info">Employer Dashboard</Badge>
|
||||
<h1 className="text-3xl font-bold tracking-tight text-slate-900 dark:text-slate-100">
|
||||
Employer Dashboard
|
||||
</h1>
|
||||
<p className="text-slate-600 dark:text-slate-400">
|
||||
Welcome back, <span className="font-semibold text-slate-800 dark:text-slate-200">{employer.name}</span>.
|
||||
Here is your talent pipeline at a glance.
|
||||
</p>
|
||||
</header>
|
||||
|
||||
{/* Metric cards */}
|
||||
<section className="grid grid-cols-2 gap-4 lg:grid-cols-4">
|
||||
{METRICS.map((m) => {
|
||||
const Icon = m.icon;
|
||||
return (
|
||||
<Card key={m.label}>
|
||||
<CardBody className="flex flex-col gap-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span
|
||||
className={`inline-flex h-10 w-10 items-center justify-center rounded-full ${TONE_CLASSES[m.tone]}`}
|
||||
>
|
||||
<Icon className="h-5 w-5" />
|
||||
</span>
|
||||
<span className="text-xs font-medium text-emerald-600 dark:text-emerald-400">
|
||||
{m.trend}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex flex-col">
|
||||
<span className="text-2xl font-bold text-slate-900 dark:text-slate-100">
|
||||
{m.value}
|
||||
</span>
|
||||
<span className="text-sm text-slate-500 dark:text-slate-400">{m.label}</span>
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
);
|
||||
})}
|
||||
</section>
|
||||
|
||||
{/* Middle row — pipeline + talent matches */}
|
||||
<section className="grid grid-cols-1 gap-6 lg:grid-cols-3">
|
||||
{/* Applicant pipeline — 2/3 width */}
|
||||
<div className="lg:col-span-2">
|
||||
<div className="mb-3 flex items-center justify-between">
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Applicant Pipeline
|
||||
</h2>
|
||||
<span className="text-sm text-slate-500 dark:text-slate-400">
|
||||
{PIPELINE.reduce((s, c) => s + c.count, 0)} candidates in flight
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex gap-3 overflow-x-auto pb-2">
|
||||
{PIPELINE.map((stage) => (
|
||||
<div
|
||||
key={stage.name}
|
||||
className="flex w-56 shrink-0 flex-col gap-2 rounded-lg border border-slate-200 bg-slate-50 p-3 dark:border-slate-800 dark:bg-slate-900/60"
|
||||
>
|
||||
<div className="flex items-center justify-between">
|
||||
<h3 className="text-sm font-semibold text-slate-800 dark:text-slate-200">
|
||||
{stage.name}
|
||||
</h3>
|
||||
<span className="inline-flex h-6 min-w-6 items-center justify-center rounded-full bg-slate-200 px-1.5 text-xs font-medium text-slate-700 dark:bg-slate-700 dark:text-slate-200">
|
||||
{stage.count}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex flex-col gap-2">
|
||||
{stage.cards.map((c) => (
|
||||
<div
|
||||
key={c.name}
|
||||
className="flex items-center gap-2 rounded-md border border-slate-200 bg-white p-2 dark:border-slate-800 dark:bg-slate-900"
|
||||
>
|
||||
<Avatar name={c.name} size="sm" />
|
||||
<div className="flex min-w-0 flex-1 flex-col">
|
||||
<span className="truncate text-xs font-medium text-slate-800 dark:text-slate-100">
|
||||
{c.name}
|
||||
</span>
|
||||
<span className="truncate text-xs text-slate-500 dark:text-slate-400">
|
||||
{c.position}
|
||||
</span>
|
||||
</div>
|
||||
<span
|
||||
className={`inline-flex h-7 w-9 shrink-0 items-center justify-center rounded-full text-xs font-bold ring-2 ${matchBadgeClasses(
|
||||
c.score,
|
||||
)}`}
|
||||
>
|
||||
{c.score}
|
||||
</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Talent matches — 1/3 width */}
|
||||
<div>
|
||||
<div className="mb-3 flex items-center justify-between">
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Talent Matches
|
||||
</h2>
|
||||
<Link
|
||||
href="/employer/talent"
|
||||
className="inline-flex items-center gap-1 text-sm font-medium text-primary-700 hover:underline dark:text-primary-300"
|
||||
>
|
||||
See all <ArrowRight className="h-3.5 w-3.5" />
|
||||
</Link>
|
||||
</div>
|
||||
<Card>
|
||||
<CardBody className="flex flex-col gap-2 p-3">
|
||||
{topMatches.map((cand) => (
|
||||
<Link
|
||||
key={cand.id}
|
||||
href={`/employer/talent/${cand.id}`}
|
||||
className="flex items-center gap-3 rounded-md p-2 transition-colors hover:bg-slate-50 dark:hover:bg-slate-800"
|
||||
>
|
||||
<Avatar name={cand.name} src={cand.avatar} size="md" />
|
||||
<div className="flex min-w-0 flex-1 flex-col">
|
||||
<span className="truncate text-sm font-medium text-slate-800 dark:text-slate-100">
|
||||
{cand.name}
|
||||
</span>
|
||||
<span className="truncate text-xs text-slate-500 dark:text-slate-400">
|
||||
{cand.headline}
|
||||
</span>
|
||||
</div>
|
||||
<span
|
||||
className={`inline-flex h-8 w-12 shrink-0 items-center justify-center rounded-full text-xs font-bold ring-2 ${matchBadgeClasses(
|
||||
cand.matchScore,
|
||||
)}`}
|
||||
>
|
||||
{cand.matchScore}%
|
||||
</span>
|
||||
</Link>
|
||||
))}
|
||||
</CardBody>
|
||||
</Card>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Analytics charts */}
|
||||
<section>
|
||||
<h2 className="mb-3 text-lg font-semibold text-slate-900 dark:text-slate-100">Analytics</h2>
|
||||
<AnalyticsCharts />
|
||||
</section>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
import { Badge } from '@nextcraft/ui';
|
||||
import { jobs } from '@nextcraft/mock-data';
|
||||
import { PostingManager } from '../../../../components/employer/posting-manager';
|
||||
|
||||
export default function PostingsPage() {
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
<header className="flex flex-col gap-2">
|
||||
<Badge variant="info">Posting Management</Badge>
|
||||
<h1 className="text-3xl font-bold tracking-tight text-slate-900 dark:text-slate-100">
|
||||
Posting Management
|
||||
</h1>
|
||||
<p className="max-w-2xl text-slate-600 dark:text-slate-400">
|
||||
Create, edit, and track your job postings. View the applicant pipeline for each posting
|
||||
and manage interview stages.
|
||||
</p>
|
||||
</header>
|
||||
|
||||
<PostingManager jobs={jobs} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,443 @@
|
||||
import Link from 'next/link';
|
||||
import { notFound } from 'next/navigation';
|
||||
import {
|
||||
ArrowLeft,
|
||||
CheckCircle,
|
||||
Clock,
|
||||
FileCode,
|
||||
Palette,
|
||||
Cpu,
|
||||
Mail,
|
||||
ShieldCheck,
|
||||
} from 'lucide-react';
|
||||
import { Avatar, Badge, Button, Card, CardBody } from '@nextcraft/ui';
|
||||
import { candidates, competencyStacks } from '@nextcraft/mock-data';
|
||||
import type { Artifact, ArtifactType } from '@nextcraft/types';
|
||||
import { matchBadgeClasses, initials } from '../../../../../lib/format';
|
||||
import { DefenseAccordion } from '../../../../../components/employer/defense-accordion';
|
||||
|
||||
interface PageProps {
|
||||
params: Promise<{ candidateId: string }>;
|
||||
}
|
||||
|
||||
/* -------------------------------------------------------------------------- */
|
||||
/* Deterministic mock-data generators (per-candidate) */
|
||||
/* -------------------------------------------------------------------------- */
|
||||
|
||||
function hashId(id: string): number {
|
||||
let h = 0;
|
||||
for (let i = 0; i < id.length; i++) h = (h * 31 + id.charCodeAt(i)) >>> 0;
|
||||
return h;
|
||||
}
|
||||
|
||||
function generateArtifacts(cand: { id: string; artifactCount: number; competencyStackId: string }): Artifact[] {
|
||||
const namesByStack: Record<string, Array<{ name: string; type: ArtifactType; desc: string }>> = {
|
||||
'stack-orchestration': [
|
||||
{ name: 'Multi-agent research assistant', type: 'code', desc: 'LangGraph assistant with eval harness for cited literature reviews.' },
|
||||
{ name: 'RAG retrieval quality dashboard', type: 'code', desc: 'Streamlit dashboard comparing chunking strategies across 800 eval queries.' },
|
||||
{ name: 'Agent topology diagram', type: 'design', desc: 'Plan-and-execute agent with reflection and tool-retrieval sub-graphs.' },
|
||||
{ name: 'Prompt regression suite', type: 'code', desc: 'Pytest suite of 320 prompt assertions with LLM-as-judge scoring.' },
|
||||
{ name: 'Token-cost alerting system', type: 'code', desc: 'Real-time cost monitoring across agent calls with Slack escalation.' },
|
||||
{ name: 'Function calling sandbox', type: 'simulation', desc: 'Sandboxed tool execution environment with retry + validation.' },
|
||||
],
|
||||
'stack-safety': [
|
||||
{ name: 'Red-team probe suite', type: 'code', desc: '1,200+ adversarial probes across three model families with scoring.' },
|
||||
{ name: 'Model card — internal Q&A agent', type: 'document', desc: 'Capabilities, limitations, intended use, and red-team findings.' },
|
||||
{ name: 'Bias audit dashboard', type: 'code', desc: 'Disparate-impact testing across demographics with visual report.' },
|
||||
{ name: 'Risk register template', type: 'document', desc: 'Risk taxonomy with severity and likelihood for 30+ deployed systems.' },
|
||||
{ name: 'Jailbreak defense prototype', type: 'code', desc: 'Instruction-hierarchy enforcement with indirect injection mitigation.' },
|
||||
],
|
||||
'stack-designer': [
|
||||
{ name: 'Transparency pattern library', type: 'design', desc: 'Confidence indicators, source attribution, and model limitation disclosure.' },
|
||||
{ name: 'Agentic interaction prototype', type: 'design', desc: 'Figma prototype for delegating, interrupting, and reviewing autonomous agents.' },
|
||||
{ name: 'Repair flow spec', type: 'document', desc: 'Multi-turn dialogue repair flows reducing escalations by 35%.' },
|
||||
{ name: 'Persona consistency framework', type: 'design', desc: 'Character design for AI assistants with contextual tone adaptation.' },
|
||||
{ name: 'Trust calibration study', type: 'document', desc: 'User mental model alignment research with over-trust mitigations.' },
|
||||
],
|
||||
'stack-operator': [
|
||||
{ name: 'Anomaly triage dashboard', type: 'code', desc: 'ML-based anomaly score visualization cutting false positives by 40%.' },
|
||||
{ name: 'Sensor calibration log', type: 'document', desc: 'Calibration schedule and drift-correction workflow for fleet sensors.' },
|
||||
{ name: 'Vision inspection tuning', type: 'simulation', desc: 'Threshold tuning for vision-based quality control station.' },
|
||||
{ name: 'Field data collection template', type: 'document', desc: 'Structured annotation workflow for high-quality dataset capture.' },
|
||||
],
|
||||
'stack-science': [
|
||||
{ name: 'Active-learning DFT toolkit', type: 'code', desc: 'Open-source toolkit for active-learning loops over DFT calculations.' },
|
||||
{ name: 'Phenotype prediction pipeline', type: 'code', desc: 'ML pipeline for drug discovery with reproducible Snakemake workflows.' },
|
||||
{ name: 'Wind forecast downscaling', type: 'simulation', desc: 'Physics-informed model improving 72-hour wind forecasts by 18%.' },
|
||||
{ name: 'Materials screening notebook', type: 'code', desc: 'Jupyter notebook for ML property prediction with active learning.' },
|
||||
{ name: 'Reproducible research container', type: 'document', desc: 'Containerized workflow with Nextflow and DOI assignment.' },
|
||||
],
|
||||
};
|
||||
|
||||
const pool = namesByStack[cand.competencyStackId] ?? namesByStack['stack-orchestration'];
|
||||
const offset = hashId(cand.id) % pool.length;
|
||||
const count = Math.min(cand.artifactCount, pool.length);
|
||||
const artifacts: Artifact[] = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
const item = pool[(offset + i) % pool.length];
|
||||
artifacts.push({
|
||||
id: `art-${cand.id}-${i}`,
|
||||
name: item.name,
|
||||
type: item.type,
|
||||
url: `https://example.com/artifacts/${cand.id}/${i}`,
|
||||
description: item.desc,
|
||||
createdAt: new Date(Date.UTC(2026, 7 - i, 20 - i * 3)).toISOString(),
|
||||
});
|
||||
}
|
||||
return artifacts;
|
||||
}
|
||||
|
||||
function generateProcessTrace(cand: { id: string }) {
|
||||
const offset = hashId(cand.id);
|
||||
const baseEvents = [
|
||||
{ action: 'Repository initialized', detail: 'Created project scaffold with README and license.' },
|
||||
{ action: 'First commit pushed', detail: 'Initial proof-of-concept with placeholder data.' },
|
||||
{ action: 'Evaluation harness added', detail: 'Wired up 50-question regression suite with LLM-as-judge.' },
|
||||
{ action: 'Peer review feedback', detail: 'Two reviewers flagged edge cases in retrieval fallback path.' },
|
||||
{ action: 'Iteration — fallback hardened', detail: 'Added retry + validation; eval score improved 12 points.' },
|
||||
{ action: 'Final submission', detail: 'Artifact submitted for oral defense scheduling.' },
|
||||
];
|
||||
return baseEvents.map((e, i) => ({
|
||||
timestamp: new Date(Date.UTC(2026, 6 + Math.floor(i / 2), (offset % 20) + i * 2)).toISOString(),
|
||||
...e,
|
||||
}));
|
||||
}
|
||||
|
||||
function generateDefenseSessions(cand: { id: string; competencyStackId: string; microcredentials: number }) {
|
||||
const stack = competencyStacks.find((s) => s.id === cand.competencyStackId);
|
||||
const mastered = (stack?.competencies ?? []).filter((c) => c.status === 'mastered' || c.status === 'in_progress');
|
||||
const offset = hashId(cand.id);
|
||||
const count = Math.min(cand.microcredentials, mastered.length, 3);
|
||||
const sessions = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
const comp = mastered[(offset + i) % mastered.length];
|
||||
const score = 82 + ((offset + i) % 12);
|
||||
sessions.push({
|
||||
id: `def-${cand.id}-${i}`,
|
||||
competencyName: comp.name,
|
||||
score,
|
||||
date: new Date(Date.UTC(2026, 5 + i, (offset % 24) + 1)).toISOString(),
|
||||
transcript: [
|
||||
{
|
||||
question: 'Walk us through the architecture of your artifact. Why did you choose this approach?',
|
||||
answer:
|
||||
'I chose a plan-and-execute topology because the task required multi-step retrieval with reflection. The plan node decomposes the query, sub-agents retrieve and draft in parallel, and a reflection node scores and routes for a second pass when below threshold.',
|
||||
},
|
||||
{
|
||||
question: 'What evaluation did you run, and what were the headline numbers?',
|
||||
answer:
|
||||
'I ran a 50-query regression suite scored by LLM-as-judge calibrated against a human panel (0.86 agreement). Baseline scored 71%; the reflection pass lifted it to 88% with a 14% latency cost, which stayed within budget.',
|
||||
},
|
||||
{
|
||||
question: 'Describe a failure mode you found and how you mitigated it.',
|
||||
answer:
|
||||
'Retrieval fallback returned stale context on schema changes. I added a freshness check + retry with a smaller context window, which reduced stale-grounded answers from 9% to under 2%.',
|
||||
},
|
||||
],
|
||||
});
|
||||
}
|
||||
return sessions;
|
||||
}
|
||||
|
||||
function generateMicrocredentials(cand: { id: string; competencyStackId: string; microcredentials: number }) {
|
||||
const stack = competencyStacks.find((s) => s.id === cand.competencyStackId);
|
||||
const mastered = (stack?.competencies ?? []).filter((c) => c.status === 'mastered' || c.status === 'in_progress');
|
||||
const offset = hashId(cand.id);
|
||||
const count = Math.min(cand.microcredentials, mastered.length, 5);
|
||||
const items = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
const comp = mastered[(offset + i) % mastered.length];
|
||||
const score = 85 + ((offset + i) % 11);
|
||||
items.push({
|
||||
id: `mc-${cand.id}-${i}`,
|
||||
competencyName: comp.name,
|
||||
issuedAt: new Date(Date.UTC(2026, 5 + i, (offset % 24) + 1)).toISOString(),
|
||||
score,
|
||||
});
|
||||
}
|
||||
return items;
|
||||
}
|
||||
|
||||
function generateCompetencies(cand: { id: string; competencyStackId: string }) {
|
||||
const stack = competencyStacks.find((s) => s.id === cand.competencyStackId);
|
||||
const comps = stack?.competencies ?? [];
|
||||
const offset = hashId(cand.id);
|
||||
return comps.slice(0, 8).map((c, i) => {
|
||||
// Deterministically assign status biased toward mastered/in_progress for high-score candidates.
|
||||
const r = (offset + i) % 10;
|
||||
const status = r < 5 ? 'mastered' : r < 8 ? 'in_progress' : 'available';
|
||||
return { id: c.id, name: c.name, status: status as 'mastered' | 'in_progress' | 'available' };
|
||||
});
|
||||
}
|
||||
|
||||
/* -------------------------------------------------------------------------- */
|
||||
/* Icon helpers */
|
||||
/* -------------------------------------------------------------------------- */
|
||||
|
||||
function ArtifactTypeIcon({ type }: { type: ArtifactType }) {
|
||||
const cls = 'h-5 w-5';
|
||||
if (type === 'code') return <FileCode className={cls} />;
|
||||
if (type === 'design') return <Palette className={cls} />;
|
||||
if (type === 'simulation') return <Cpu className={cls} />;
|
||||
return <FileCode className={cls} />;
|
||||
}
|
||||
|
||||
const ARTIFACT_ICON_BG: Record<ArtifactType, string> = {
|
||||
code: 'bg-primary-100 text-primary-700 dark:bg-primary-900/40 dark:text-primary-300',
|
||||
design: 'bg-violet-100 text-violet-700 dark:bg-violet-900/40 dark:text-violet-300',
|
||||
simulation: 'bg-emerald-100 text-emerald-700 dark:bg-emerald-900/40 dark:text-emerald-300',
|
||||
document: 'bg-amber-100 text-amber-700 dark:bg-amber-900/40 dark:text-amber-300',
|
||||
};
|
||||
|
||||
/* -------------------------------------------------------------------------- */
|
||||
/* Page */
|
||||
/* -------------------------------------------------------------------------- */
|
||||
|
||||
export default async function CandidateProfilePage({ params }: PageProps) {
|
||||
const { candidateId } = await params;
|
||||
const cand = candidates.find((c) => c.id === candidateId);
|
||||
if (!cand) notFound();
|
||||
|
||||
const stack = competencyStacks.find((s) => s.id === cand.competencyStackId);
|
||||
const artifacts = generateArtifacts(cand);
|
||||
const processTrace = generateProcessTrace(cand);
|
||||
const defenseSessions = generateDefenseSessions(cand);
|
||||
const microcredentials = generateMicrocredentials(cand);
|
||||
const competencies = generateCompetencies(cand);
|
||||
|
||||
const masteredCount = competencies.filter((c) => c.status === 'mastered').length;
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
{/* Back link */}
|
||||
<div>
|
||||
<Link
|
||||
href="/employer/talent"
|
||||
className="inline-flex items-center gap-1.5 text-sm font-medium text-slate-600 hover:text-slate-900 dark:text-slate-300 dark:hover:text-slate-100"
|
||||
>
|
||||
<ArrowLeft className="h-4 w-4" />
|
||||
Back to talent search
|
||||
</Link>
|
||||
</div>
|
||||
|
||||
{/* Header */}
|
||||
<Card>
|
||||
<CardBody className="flex flex-col gap-4">
|
||||
<div className="flex flex-col gap-4 md:flex-row md:items-start md:justify-between">
|
||||
<div className="flex items-start gap-4">
|
||||
<Avatar name={cand.name} src={cand.avatar} size="lg" />
|
||||
<div className="flex flex-col gap-1">
|
||||
<h1 className="text-2xl font-bold text-slate-900 dark:text-slate-100">
|
||||
{cand.name}
|
||||
</h1>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">{cand.headline}</p>
|
||||
<div className="mt-1 flex flex-wrap items-center gap-2">
|
||||
<Badge variant="info">{stack?.name ?? 'AI'}</Badge>
|
||||
<Badge variant="success">Verified</Badge>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center gap-3">
|
||||
<div className="flex flex-col items-end">
|
||||
<span className="text-xs font-medium uppercase tracking-wide text-slate-500 dark:text-slate-400">
|
||||
Match Score
|
||||
</span>
|
||||
<span
|
||||
className={`inline-flex h-12 w-14 items-center justify-center rounded-full text-base font-bold ring-2 ${matchBadgeClasses(
|
||||
cand.matchScore,
|
||||
)}`}
|
||||
>
|
||||
{cand.matchScore}%
|
||||
</span>
|
||||
</div>
|
||||
<Button variant="primary" size="md" icon={<Mail className="h-4 w-4" />} type="button">
|
||||
Contact
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-sm leading-relaxed text-slate-600 dark:text-slate-400">{cand.bio}</p>
|
||||
</CardBody>
|
||||
</Card>
|
||||
|
||||
{/* Summary stats bar */}
|
||||
<div className="grid grid-cols-2 gap-4 md:grid-cols-4">
|
||||
<StatCard label="Artifacts" value={cand.artifactCount} icon={<FileCode className="h-4 w-4" />} />
|
||||
<StatCard label="Defense Score (avg)" value={cand.defenseScore} icon={<ShieldCheck className="h-4 w-4" />} />
|
||||
<StatCard label="Microcredentials" value={cand.microcredentials} icon={<CheckCircle className="h-4 w-4" />} />
|
||||
<StatCard label="Competencies Mastered" value={masteredCount} icon={<Cpu className="h-4 w-4" />} />
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 gap-6 lg:grid-cols-2">
|
||||
{/* Artifact gallery */}
|
||||
<Card>
|
||||
<CardBody className="flex flex-col gap-4">
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Artifact Gallery
|
||||
</h2>
|
||||
<div className="grid grid-cols-1 gap-3 sm:grid-cols-2">
|
||||
{artifacts.map((art) => (
|
||||
<a
|
||||
key={art.id}
|
||||
href={art.url}
|
||||
className="flex flex-col gap-2 rounded-lg border border-slate-200 p-3 transition-colors hover:bg-slate-50 dark:border-slate-800 dark:hover:bg-slate-800"
|
||||
>
|
||||
<div className="flex items-center gap-2">
|
||||
<span
|
||||
className={`inline-flex h-8 w-8 items-center justify-center rounded-full ${ARTIFACT_ICON_BG[art.type]}`}
|
||||
>
|
||||
<ArtifactTypeIcon type={art.type} />
|
||||
</span>
|
||||
<span className="text-sm font-semibold text-slate-900 dark:text-slate-100">
|
||||
{art.name}
|
||||
</span>
|
||||
</div>
|
||||
<p className="text-xs leading-relaxed text-slate-600 dark:text-slate-400">
|
||||
{art.description}
|
||||
</p>
|
||||
<span className="text-xs text-slate-400">
|
||||
{new Date(art.createdAt).toLocaleDateString('en-US', {
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
})}
|
||||
</span>
|
||||
</a>
|
||||
))}
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
|
||||
{/* Process trace summary */}
|
||||
<Card>
|
||||
<CardBody className="flex flex-col gap-4">
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Process Trace Summary
|
||||
</h2>
|
||||
<ol className="relative flex flex-col gap-4 border-l border-slate-200 pl-6 dark:border-slate-800">
|
||||
{processTrace.map((step, i) => (
|
||||
<li key={i} className="relative">
|
||||
<span className="absolute -left-[1.6rem] flex h-5 w-5 items-center justify-center rounded-full bg-primary-100 ring-4 ring-white dark:bg-primary-900/40 dark:ring-slate-900">
|
||||
<span className="h-1.5 w-1.5 rounded-full bg-primary-600" />
|
||||
</span>
|
||||
<div className="flex flex-col gap-0.5">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-sm font-medium text-slate-900 dark:text-slate-100">
|
||||
{step.action}
|
||||
</span>
|
||||
<span className="inline-flex items-center gap-1 text-xs text-slate-400">
|
||||
<Clock className="h-3 w-3" />
|
||||
{new Date(step.timestamp).toLocaleDateString('en-US', {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
<p className="text-xs text-slate-600 dark:text-slate-400">{step.detail}</p>
|
||||
</div>
|
||||
</li>
|
||||
))}
|
||||
</ol>
|
||||
</CardBody>
|
||||
</Card>
|
||||
|
||||
{/* Oral defense transcripts */}
|
||||
<Card className="lg:col-span-2">
|
||||
<CardBody className="flex flex-col gap-4">
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Oral Defense Transcripts
|
||||
</h2>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">
|
||||
Recorded Q&A from each verified oral defense session. Expand a session to read the
|
||||
transcript.
|
||||
</p>
|
||||
<DefenseAccordion sessions={defenseSessions} />
|
||||
</CardBody>
|
||||
</Card>
|
||||
|
||||
{/* Competency mini-graph */}
|
||||
<Card>
|
||||
<CardBody className="flex flex-col gap-4">
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Competency Progress
|
||||
</h2>
|
||||
<div className="flex flex-col gap-2">
|
||||
{competencies.map((c) => (
|
||||
<div
|
||||
key={c.id}
|
||||
className="flex items-center justify-between rounded-md border border-slate-200 p-2.5 dark:border-slate-800"
|
||||
>
|
||||
<span className="text-sm text-slate-800 dark:text-slate-200">{c.name}</span>
|
||||
{c.status === 'mastered' ? (
|
||||
<Badge variant="success">
|
||||
<CheckCircle className="h-3 w-3" /> Mastered
|
||||
</Badge>
|
||||
) : c.status === 'in_progress' ? (
|
||||
<Badge variant="warning">
|
||||
<Clock className="h-3 w-3" /> In Progress
|
||||
</Badge>
|
||||
) : (
|
||||
<Badge variant="default">Available</Badge>
|
||||
)}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
|
||||
{/* Microcredential verification */}
|
||||
<Card>
|
||||
<CardBody className="flex flex-col gap-4">
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
Microcredential Verification
|
||||
</h2>
|
||||
<div className="flex flex-col gap-2">
|
||||
{microcredentials.map((mc) => (
|
||||
<div
|
||||
key={mc.id}
|
||||
className="flex items-center justify-between rounded-md border border-slate-200 p-3 dark:border-slate-800"
|
||||
>
|
||||
<div className="flex items-center gap-2">
|
||||
<CheckCircle className="h-4 w-4 shrink-0 text-emerald-600 dark:text-emerald-400" />
|
||||
<div className="flex flex-col">
|
||||
<span className="text-sm font-medium text-slate-900 dark:text-slate-100">
|
||||
{mc.competencyName}
|
||||
</span>
|
||||
<span className="text-xs text-slate-500 dark:text-slate-400">
|
||||
Issued{' '}
|
||||
{new Date(mc.issuedAt).toLocaleDateString('en-US', {
|
||||
year: 'numeric',
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-sm font-semibold text-slate-700 dark:text-slate-200">
|
||||
{mc.score}
|
||||
</span>
|
||||
<Badge variant="success">Verified</Badge>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function StatCard({ label, value, icon }: { label: string; value: number | string; icon: React.ReactNode }) {
|
||||
return (
|
||||
<div className="flex items-center gap-3 rounded-lg border border-slate-200 bg-white p-4 dark:border-slate-800 dark:bg-slate-900">
|
||||
<span className="inline-flex h-9 w-9 items-center justify-center rounded-full bg-slate-100 text-slate-600 dark:bg-slate-800 dark:text-slate-300">
|
||||
{icon}
|
||||
</span>
|
||||
<div className="flex flex-col">
|
||||
<span className="text-xl font-bold text-slate-900 dark:text-slate-100">{value}</span>
|
||||
<span className="text-xs text-slate-500 dark:text-slate-400">{label}</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
import { Badge } from '@nextcraft/ui';
|
||||
import { candidates } from '@nextcraft/mock-data';
|
||||
import { TalentSearch } from '../../../../components/employer/talent-search';
|
||||
|
||||
export default function TalentSearchPage() {
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
<header className="flex flex-col gap-2">
|
||||
<Badge variant="info">Talent Search</Badge>
|
||||
<h1 className="text-3xl font-bold tracking-tight text-slate-900 dark:text-slate-100">
|
||||
Talent Search
|
||||
</h1>
|
||||
<p className="max-w-2xl text-slate-600 dark:text-slate-400">
|
||||
AI-credentialed candidates with verified microcredentials, evidence portfolios, and oral
|
||||
defense scores. Filter by competency stack, defense score, and artifact count.
|
||||
</p>
|
||||
</header>
|
||||
|
||||
<TalentSearch candidates={candidates} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
'use client';
|
||||
|
||||
import type { ReactNode } from 'react';
|
||||
import Link from 'next/link';
|
||||
import { usePathname } from 'next/navigation';
|
||||
import { Building2 } from 'lucide-react';
|
||||
|
||||
const LINKS = [
|
||||
{ label: 'Overview', href: '/employer' },
|
||||
{ label: 'Talent search', href: '/employer/talent' },
|
||||
{ label: 'Postings', href: '/employer/postings' },
|
||||
];
|
||||
|
||||
export default function EmployerLayout({ children }: { children: ReactNode }) {
|
||||
const pathname = usePathname() ?? '';
|
||||
|
||||
return (
|
||||
<div className="mx-auto flex max-w-7xl gap-6 px-4 py-8 sm:px-6">
|
||||
<aside className="hidden w-56 shrink-0 md:block">
|
||||
<div className="mb-4 flex items-center gap-2 text-sm font-semibold text-slate-500 dark:text-slate-400">
|
||||
<Building2 className="h-4 w-4 text-primary-600" />
|
||||
Employer surface
|
||||
</div>
|
||||
<nav className="flex flex-col gap-1">
|
||||
{LINKS.map((l) => {
|
||||
const isActive = pathname === l.href || pathname.startsWith(l.href + '/');
|
||||
return (
|
||||
<Link
|
||||
key={l.href}
|
||||
href={l.href}
|
||||
aria-current={isActive ? 'page' : undefined}
|
||||
className={`rounded-md px-3 py-2 text-sm font-medium transition-colors ${
|
||||
isActive
|
||||
? 'bg-primary-50 text-primary-700 dark:bg-primary-900/30 dark:text-primary-200'
|
||||
: 'text-slate-700 hover:bg-slate-100 dark:text-slate-200 dark:hover:bg-slate-800'
|
||||
}`}
|
||||
>
|
||||
{l.label}
|
||||
</Link>
|
||||
);
|
||||
})}
|
||||
</nav>
|
||||
</aside>
|
||||
<div className="flex-1">{children}</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,300 @@
|
||||
import Link from 'next/link';
|
||||
import { notFound } from 'next/navigation';
|
||||
import {
|
||||
ArrowLeft,
|
||||
ArrowRight,
|
||||
Play,
|
||||
Save,
|
||||
Upload,
|
||||
Folder,
|
||||
FileText,
|
||||
Activity,
|
||||
} from 'lucide-react';
|
||||
import { Button } from '@nextcraft/ui';
|
||||
import { allCompetencies, competencyStacks, aiLabScenarios } from '@nextcraft/mock-data';
|
||||
import { LabFeedbackPanel } from '../../../../components/learner/lab-feedback-panel';
|
||||
|
||||
interface FileEntry {
|
||||
label: string;
|
||||
children?: { label: string }[];
|
||||
}
|
||||
|
||||
const FILE_TREE: FileEntry[] = [
|
||||
{
|
||||
label: 'src/',
|
||||
children: [
|
||||
{ label: 'main.ts' },
|
||||
{ label: 'agent.ts' },
|
||||
{ label: 'tools.ts' },
|
||||
],
|
||||
},
|
||||
{
|
||||
label: 'tests/',
|
||||
children: [{ label: 'agent.test.ts' }],
|
||||
},
|
||||
{ label: 'README.md' },
|
||||
];
|
||||
|
||||
const EDITOR_LINES = [
|
||||
{ n: 1, content: "import { ChatOpenAI } from '@langchain/openai';" },
|
||||
{ n: 2, content: "import { tool } from '@langchain/core/tools';" },
|
||||
{ n: 3, content: "import { z } from 'zod';" },
|
||||
{ n: 4, content: '' },
|
||||
{ n: 5, content: '// Define a typed tool for fetching the current weather' },
|
||||
{ n: 6, content: "const getWeather = tool(" },
|
||||
{ n: 7, content: ' async ({ city, units }) => {' },
|
||||
{ n: 8, content: ' const res = await fetch(`/api/weather?city=${city}&units=${units}`);' },
|
||||
{ n: 9, content: ' return res.json();' },
|
||||
{ n: 10, content: ' },' },
|
||||
{ n: 11, content: ' {' },
|
||||
{ n: 12, content: " name: 'get_weather'," },
|
||||
{ n: 13, content: " description: 'Fetch the current weather for a city'," },
|
||||
{ n: 14, content: ' schema: z.object({' },
|
||||
{ n: 15, content: " city: z.string().describe('City to fetch weather for')," },
|
||||
{ n: 16, content: " units: z.enum(['celsius', 'fahrenheit']).default('celsius')," },
|
||||
{ n: 17, content: ' }),' },
|
||||
{ n: 18, content: ' },' },
|
||||
{ n: 19, content: ');' },
|
||||
{ n: 20, content: '' },
|
||||
{ n: 21, content: 'export async function main(query: string) {' },
|
||||
{ n: 22, content: ' const model = new ChatOpenAI({ model: "gpt-4o-mini" });' },
|
||||
{ n: 23, content: ' const modelWithTools = model.bindTools([getWeather]);' },
|
||||
{ n: 24, content: ' const response = await modelWithTools.invoke(query);' },
|
||||
{ n: 25, content: ' return response.tool_calls;' },
|
||||
{ n: 26, content: '}' },
|
||||
];
|
||||
|
||||
const TELEMETRY_METRICS = [
|
||||
{ label: 'Commits', value: '7' },
|
||||
{ label: 'Keystrokes', value: '1,247' },
|
||||
{ label: 'Time spent', value: '23 min' },
|
||||
{ label: 'Build attempts', value: '3' },
|
||||
];
|
||||
|
||||
const TELEMETRY_EVENTS = [
|
||||
{ time: '14:02:11', action: 'File created: src/main.ts' },
|
||||
{ time: '14:09:48', action: 'First build attempt (failed)' },
|
||||
{ time: '14:12:30', action: 'Test suite passed (2/2)' },
|
||||
{ time: '14:18:05', action: 'Commit: scaffold agent entrypoint' },
|
||||
{ time: '14:21:42', action: 'Tool schema validated' },
|
||||
{ time: '14:25:17', action: 'Build attempt 2 (success)' },
|
||||
{ time: '14:28:03', action: 'Commit: implement tool calling' },
|
||||
];
|
||||
|
||||
function highlight(line: string): { text: string; cls: string }[] {
|
||||
// Very small token highlighter for the mock editor
|
||||
const tokens: { text: string; cls: string }[] = [];
|
||||
let i = 0;
|
||||
while (i < line.length) {
|
||||
// comment
|
||||
if (line.slice(i).startsWith('//')) {
|
||||
tokens.push({ text: line.slice(i), cls: 'text-slate-500' });
|
||||
break;
|
||||
}
|
||||
// string with backtick or single/double quote
|
||||
const ch = line[i];
|
||||
if (ch === '`' || ch === "'" || ch === '"') {
|
||||
const end = line.indexOf(ch, i + 1);
|
||||
if (end !== -1) {
|
||||
tokens.push({ text: line.slice(i, end + 1), cls: 'text-emerald-300' });
|
||||
i = end + 1;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
// keyword
|
||||
const rest = line.slice(i);
|
||||
const kwMatch = rest.match(/^(import|from|export|async|function|const|return|await|new)/);
|
||||
if (kwMatch) {
|
||||
tokens.push({ text: kwMatch[0], cls: 'text-primary-300' });
|
||||
i += kwMatch[0].length;
|
||||
continue;
|
||||
}
|
||||
// default: consume one char
|
||||
tokens.push({ text: ch, cls: 'text-slate-200' });
|
||||
i += 1;
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
export default async function BuildSandboxPage({
|
||||
params,
|
||||
}: {
|
||||
params: Promise<{ competencyId: string }>;
|
||||
}) {
|
||||
const { competencyId } = await params;
|
||||
const competency = allCompetencies.find((c) => c.id === competencyId);
|
||||
if (!competency) notFound();
|
||||
const stack = competencyStacks.find((s) => s.id === competency.stackId);
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-4">
|
||||
<Link
|
||||
href={`/learn/${competency.id}`}
|
||||
className="inline-flex w-fit items-center gap-1 text-sm text-slate-500 hover:text-slate-800 dark:hover:text-slate-200"
|
||||
>
|
||||
<ArrowLeft className="h-3.5 w-3.5" />
|
||||
Back to Byte
|
||||
</Link>
|
||||
|
||||
{/* Toolbar */}
|
||||
<div className="flex flex-wrap items-center justify-between gap-3 rounded-lg border border-slate-200 bg-white px-4 py-3 dark:border-slate-800 dark:bg-slate-900">
|
||||
<div className="flex items-center gap-3">
|
||||
<span className="rounded-md bg-primary-100 px-2 py-0.5 text-xs font-semibold text-primary-700 dark:bg-primary-900/40 dark:text-primary-300">
|
||||
Sandbox
|
||||
</span>
|
||||
<span className="text-sm font-medium text-slate-800 dark:text-slate-100">
|
||||
{competency.name}
|
||||
</span>
|
||||
<span className="hidden text-xs text-slate-500 sm:inline dark:text-slate-400">
|
||||
{stack?.name}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<button
|
||||
type="button"
|
||||
className="inline-flex h-8 items-center gap-1.5 rounded-md bg-emerald-600 px-3 text-xs font-medium text-white transition-colors hover:bg-emerald-700"
|
||||
>
|
||||
<Play className="h-3.5 w-3.5" />
|
||||
Run
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="inline-flex h-8 items-center gap-1.5 rounded-md border border-slate-300 px-3 text-xs font-medium text-slate-700 transition-colors hover:bg-slate-100 dark:border-slate-700 dark:text-slate-200 dark:hover:bg-slate-800"
|
||||
>
|
||||
<Save className="h-3.5 w-3.5" />
|
||||
Save
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="inline-flex h-8 items-center gap-1.5 rounded-md border border-slate-300 px-3 text-xs font-medium text-slate-700 transition-colors hover:bg-slate-100 dark:border-slate-700 dark:text-slate-200 dark:hover:bg-slate-800"
|
||||
>
|
||||
<Upload className="h-3.5 w-3.5" />
|
||||
Submit
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* IDE layout */}
|
||||
<div className="grid grid-cols-1 gap-4 lg:grid-cols-[14rem_1fr_16rem]">
|
||||
{/* File explorer */}
|
||||
<aside className="rounded-lg border border-slate-200 bg-slate-50 p-3 dark:border-slate-800 dark:bg-slate-900">
|
||||
<h3 className="mb-2 text-xs font-semibold uppercase tracking-wide text-slate-500 dark:text-slate-400">
|
||||
Explorer
|
||||
</h3>
|
||||
<ul className="space-y-1 text-sm">
|
||||
{FILE_TREE.map((node) =>
|
||||
node.children ? (
|
||||
<li key={node.label}>
|
||||
<div className="flex items-center gap-1.5 text-slate-700 dark:text-slate-200">
|
||||
<Folder className="h-3.5 w-3.5 text-amber-500" />
|
||||
{node.label}
|
||||
</div>
|
||||
<ul className="ml-4 mt-1 space-y-1">
|
||||
{node.children.map((child) => (
|
||||
<li
|
||||
key={child.label}
|
||||
className="flex items-center gap-1.5 text-slate-600 dark:text-slate-300"
|
||||
>
|
||||
<FileText className="h-3.5 w-3.5 text-slate-400" />
|
||||
{child.label}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</li>
|
||||
) : (
|
||||
<li
|
||||
key={node.label}
|
||||
className="flex items-center gap-1.5 text-slate-600 dark:text-slate-300"
|
||||
>
|
||||
<FileText className="h-3.5 w-3.5 text-slate-400" />
|
||||
{node.label}
|
||||
</li>
|
||||
),
|
||||
)}
|
||||
</ul>
|
||||
</aside>
|
||||
|
||||
{/* Editor */}
|
||||
<section className="overflow-hidden rounded-lg border border-slate-200 bg-slate-950 dark:border-slate-800">
|
||||
<div className="flex items-center gap-2 border-b border-slate-800 px-3 py-2 text-xs text-slate-400">
|
||||
<FileText className="h-3.5 w-3.5" />
|
||||
src/main.ts
|
||||
</div>
|
||||
<pre className="overflow-auto p-3 font-mono text-xs leading-relaxed">
|
||||
<code>
|
||||
{EDITOR_LINES.map((line) => (
|
||||
<div key={line.n} className="flex">
|
||||
<span className="mr-4 inline-block w-8 select-none text-right text-slate-600">
|
||||
{line.n}
|
||||
</span>
|
||||
<span className="flex-1 whitespace-pre">
|
||||
{line.content === '' ? (
|
||||
<span> </span>
|
||||
) : (
|
||||
highlight(line.content).map((t, idx) => (
|
||||
<span key={idx} className={t.cls}>
|
||||
{t.text}
|
||||
</span>
|
||||
))
|
||||
)}
|
||||
</span>
|
||||
</div>
|
||||
))}
|
||||
</code>
|
||||
</pre>
|
||||
</section>
|
||||
|
||||
{/* Telemetry */}
|
||||
<aside className="flex flex-col gap-3 rounded-lg border border-slate-200 bg-white p-3 dark:border-slate-800 dark:bg-slate-900">
|
||||
<div className="flex items-center gap-2">
|
||||
<Activity className="h-4 w-4 text-primary-600" />
|
||||
<h3 className="text-sm font-semibold text-slate-800 dark:text-slate-100">
|
||||
Process Capture
|
||||
</h3>
|
||||
</div>
|
||||
<dl className="grid grid-cols-2 gap-2">
|
||||
{TELEMETRY_METRICS.map((m) => (
|
||||
<div
|
||||
key={m.label}
|
||||
className="rounded-md border border-slate-200 bg-slate-50 p-2 text-center dark:border-slate-800 dark:bg-slate-900/50"
|
||||
>
|
||||
<dd className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
{m.value}
|
||||
</dd>
|
||||
<dt className="text-[10px] uppercase tracking-wide text-slate-500 dark:text-slate-400">
|
||||
{m.label}
|
||||
</dt>
|
||||
</div>
|
||||
))}
|
||||
</dl>
|
||||
<div>
|
||||
<h4 className="mb-1 text-xs font-semibold uppercase tracking-wide text-slate-500 dark:text-slate-400">
|
||||
Recent events
|
||||
</h4>
|
||||
<ul className="space-y-1.5 text-xs">
|
||||
{TELEMETRY_EVENTS.map((e, i) => (
|
||||
<li key={i} className="flex gap-2">
|
||||
<span className="font-mono text-slate-400">{e.time}</span>
|
||||
<span className="text-slate-700 dark:text-slate-300">{e.action}</span>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
{/* Lab in-flow feedback — mock telemetry scenario (real engine v0.3+) */}
|
||||
<div className="border-t border-slate-200 pt-3 dark:border-slate-800">
|
||||
<LabFeedbackPanel scenarioId={aiLabScenarios[0].id} />
|
||||
</div>
|
||||
</aside>
|
||||
</div>
|
||||
|
||||
{/* Submit */}
|
||||
<div className="flex justify-end">
|
||||
<Link href={`/defend/${competency.id}`}>
|
||||
<Button iconRight={<ArrowRight className="h-4 w-4" />}>
|
||||
Submit for Assessment
|
||||
</Button>
|
||||
</Link>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,164 @@
|
||||
import Link from 'next/link';
|
||||
import { notFound } from 'next/navigation';
|
||||
import {
|
||||
ArrowLeft,
|
||||
ArrowRight,
|
||||
Lock,
|
||||
Circle,
|
||||
Loader,
|
||||
CheckCircle,
|
||||
Award,
|
||||
Layers,
|
||||
} from 'lucide-react';
|
||||
import { Button, Card, CardBody, Badge } from '@nextcraft/ui';
|
||||
import { competencyStacks, learnerMicrocredentials } from '@nextcraft/mock-data';
|
||||
import type { Competency, CompetencyStatus } from '@nextcraft/types';
|
||||
|
||||
const STATUS_META: Record<
|
||||
CompetencyStatus,
|
||||
{ label: string; icon: typeof Lock; badgeClass: string }
|
||||
> = {
|
||||
locked: { label: 'Locked', icon: Lock, badgeClass: 'bg-slate-100 text-slate-500 dark:bg-slate-800 dark:text-slate-400' },
|
||||
available: { label: 'Available', icon: Circle, badgeClass: 'bg-blue-100 text-blue-700 dark:bg-blue-900/40 dark:text-blue-300' },
|
||||
in_progress: { label: 'In progress', icon: Loader, badgeClass: 'bg-amber-100 text-amber-700 dark:bg-amber-900/40 dark:text-amber-300' },
|
||||
mastered: { label: 'Mastered', icon: CheckCircle, badgeClass: 'bg-emerald-100 text-emerald-700 dark:bg-emerald-900/40 dark:text-emerald-300' },
|
||||
};
|
||||
|
||||
const PROGRESS_BY_ID: Record<string, number> = {
|
||||
'stack-orchestration-c003': 45,
|
||||
'stack-orchestration-c008': 38,
|
||||
'stack-safety-c019': 52,
|
||||
'stack-safety-c023': 30,
|
||||
'stack-designer-c034': 60,
|
||||
'stack-designer-c037': 41,
|
||||
'stack-operator-c048': 28,
|
||||
'stack-operator-c053': 35,
|
||||
'stack-science-c063': 47,
|
||||
'stack-science-c066': 33,
|
||||
};
|
||||
|
||||
function isUnlocked(c: Competency): boolean {
|
||||
return c.status !== 'locked';
|
||||
}
|
||||
|
||||
export default async function StackViewPage({
|
||||
params,
|
||||
}: {
|
||||
params: Promise<{ stackId: string }>;
|
||||
}) {
|
||||
const { stackId } = await params;
|
||||
const stack = competencyStacks.find((s) => s.id === stackId);
|
||||
if (!stack) notFound();
|
||||
|
||||
const masteredCount = stack.competencies.filter((c) => c.status === 'mastered').length;
|
||||
const inProgressCount = stack.competencies.filter((c) => c.status === 'in_progress').length;
|
||||
const mcByComp = new Map(learnerMicrocredentials.map((m) => [m.competencyId, m]));
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
{/* Back link */}
|
||||
<Link
|
||||
href="/catalog"
|
||||
className="inline-flex w-fit items-center gap-1 text-sm text-slate-500 hover:text-slate-800 dark:hover:text-slate-200"
|
||||
>
|
||||
<ArrowLeft className="h-3.5 w-3.5" />
|
||||
Back to catalog
|
||||
</Link>
|
||||
|
||||
{/* Header */}
|
||||
<header className="flex flex-col gap-3 border-l-4 border-primary-500 pl-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<Layers className="h-5 w-5 text-primary-600" />
|
||||
<h1 className="text-3xl font-bold tracking-tight text-slate-900 dark:text-slate-100">
|
||||
{stack.name}
|
||||
</h1>
|
||||
</div>
|
||||
<p className="max-w-2xl text-slate-600 dark:text-slate-400">{stack.description}</p>
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<Badge variant="info">{stack.competencies.length} competencies</Badge>
|
||||
<Badge variant="success">{masteredCount} mastered</Badge>
|
||||
<Badge variant="warning">{inProgressCount} in progress</Badge>
|
||||
<span className="text-xs text-slate-500 dark:text-slate-400">
|
||||
Target roles: {stack.targetRoles}
|
||||
</span>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
{/* Competency list */}
|
||||
<div className="flex flex-col gap-3">
|
||||
{stack.competencies.map((c) => {
|
||||
const meta = STATUS_META[c.status];
|
||||
const StatusIcon = meta.icon;
|
||||
const mc = mcByComp.get(c.id);
|
||||
const progress = PROGRESS_BY_ID[c.id];
|
||||
|
||||
return (
|
||||
<Card key={c.id}>
|
||||
<CardBody className="flex flex-col gap-3">
|
||||
<div className="flex flex-col gap-3 sm:flex-row sm:items-start sm:justify-between">
|
||||
<div className="flex flex-1 flex-col gap-1">
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<h3 className="text-base font-semibold text-slate-900 dark:text-slate-100">
|
||||
{c.name}
|
||||
</h3>
|
||||
<span
|
||||
className={`inline-flex items-center gap-1 rounded-full px-2 py-0.5 text-xs font-medium ${meta.badgeClass}`}
|
||||
>
|
||||
<StatusIcon className="h-3 w-3" />
|
||||
{meta.label}
|
||||
</span>
|
||||
{mc && mc.verified && (
|
||||
<span className="inline-flex items-center gap-1 rounded-full bg-accent-100 px-2 py-0.5 text-xs font-medium text-accent-700 dark:bg-accent-900/40 dark:text-accent-300">
|
||||
<Award className="h-3 w-3" />
|
||||
Microcredential · {mc.score}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">
|
||||
{c.description}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="flex shrink-0 flex-col items-end gap-2">
|
||||
{isUnlocked(c) ? (
|
||||
<Link href={`/learn/${c.id}`}>
|
||||
<Button size="sm" iconRight={<ArrowRight className="h-3.5 w-3.5" />}>
|
||||
Start Byte
|
||||
</Button>
|
||||
</Link>
|
||||
) : (
|
||||
<Button size="sm" variant="ghost" disabled icon={<Lock className="h-3.5 w-3.5" />}>
|
||||
Locked
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Progress / mastery bar */}
|
||||
{c.status === 'in_progress' && typeof progress === 'number' && (
|
||||
<div className="flex items-center gap-2">
|
||||
<div className="h-2 flex-1 overflow-hidden rounded-full bg-slate-100 dark:bg-slate-800">
|
||||
<div
|
||||
className="h-full rounded-full bg-amber-500"
|
||||
style={{ width: `${progress}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="text-xs text-slate-500 dark:text-slate-400">
|
||||
{progress}%
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{c.status === 'mastered' && (
|
||||
<div className="flex items-center gap-2 text-xs text-emerald-700 dark:text-emerald-300">
|
||||
<Award className="h-3.5 w-3.5" />
|
||||
Microcredential earned
|
||||
</div>
|
||||
)}
|
||||
</CardBody>
|
||||
</Card>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
import Link from 'next/link';
|
||||
import { ArrowRight, Layers } from 'lucide-react';
|
||||
import { Button, Card, CardBody, Badge } from '@nextcraft/ui';
|
||||
import { competencyStacks } from '@nextcraft/mock-data';
|
||||
|
||||
export default function CatalogPage() {
|
||||
return (
|
||||
<div className="flex flex-col gap-6">
|
||||
{/* Header */}
|
||||
<header className="flex flex-col gap-2">
|
||||
<Badge variant="info">Learner · Catalog</Badge>
|
||||
<h1 className="text-3xl font-bold tracking-tight text-slate-900 dark:text-slate-100">
|
||||
Program Catalog
|
||||
</h1>
|
||||
<p className="max-w-2xl text-slate-600 dark:text-slate-400">
|
||||
Five competency stacks covering orchestration, safety, design, field operations, and
|
||||
computational sciences. Each stack is a sequence of micro-tutorials, sandbox builds,
|
||||
artifacts, and oral defenses.
|
||||
</p>
|
||||
</header>
|
||||
|
||||
{/* Grid */}
|
||||
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2 lg:grid-cols-3">
|
||||
{competencyStacks.map((stack) => (
|
||||
<Card key={stack.id} className="flex h-full flex-col">
|
||||
<CardBody className="flex flex-1 flex-col gap-3">
|
||||
<div className="flex items-start justify-between gap-2">
|
||||
<span className="flex h-10 w-10 items-center justify-center rounded-lg bg-primary-100 text-primary-700 dark:bg-primary-900/40 dark:text-primary-300">
|
||||
<Layers className="h-5 w-5" />
|
||||
</span>
|
||||
<Badge variant="default">{stack.competencies.length} competencies</Badge>
|
||||
</div>
|
||||
<h2 className="text-lg font-semibold text-slate-900 dark:text-slate-100">
|
||||
{stack.name}
|
||||
</h2>
|
||||
<p className="text-sm text-slate-600 dark:text-slate-400">
|
||||
{stack.description}
|
||||
</p>
|
||||
<p className="text-xs text-slate-500 dark:text-slate-500">
|
||||
<span className="font-medium text-slate-600 dark:text-slate-400">
|
||||
Target roles:
|
||||
</span>{' '}
|
||||
{stack.targetRoles}
|
||||
</p>
|
||||
<div className="mt-auto pt-2">
|
||||
<Link href={`/catalog/${stack.id}`}>
|
||||
<Button variant="outline" size="sm" iconRight={<ArrowRight className="h-3.5 w-3.5" />}>
|
||||
Explore Stack
|
||||
</Button>
|
||||
</Link>
|
||||
</div>
|
||||
</CardBody>
|
||||
</Card>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user