docs(init): validate specification — milestone v0.3 credential-engines
- config.json: milestone v0.3 credential-engines, branch milestone/v0.3-credential-engines - PROJECT.md: v0.3 scope sandbox fabric, telemetry, trace grading, variant gen, voice defense; REQ-F-017 (identity/age-gating) explicitly deferred per founder directive - REQUIREMENTS.md: activate REQ-F-007/008/009/010/021 as REQ-3-001..008; REQ-F-017 re-deferred to v0.4+ - ROADMAP.md: v0.3 phase structure P0-P7 - .gitignore: ignore .ciagent/bin/ (local credential helper) - Clear stale v0.2 CHECKPOINT.json (milestone complete; new run starts at SPECIFY) ---ci--- phase: 0 milestone: v0.3 status: specify ---/ci---
This commit is contained in:
CIAgent
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"phase": 7,
|
||||
"stage": "complete",
|
||||
"milestone": "v0.2",
|
||||
"phase_role": "final",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-09-12T00:20:00Z"
|
||||
}
|
||||
+20
-18
@@ -8,29 +8,33 @@ Nextcraft is an AI-native outcome school where graduates prove what they can bui
|
||||
|
||||
---
|
||||
|
||||
## Current Milestone: v0.2 — AI Tutor Architecture
|
||||
## Current Milestone: v0.3 — Credential Engines
|
||||
|
||||
**Scope:** The six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface chat UI with streaming responses. Provider-agnostic LLM layer (ollama-cloud default, local endpoint + deterministic mock for tests). Lab/Assessor/Proctor operate on mock engine inputs (simulated telemetry, pre-baked artifacts) — their real engines (sandbox fabric, assessment engine, identity verification) are v0.3+.
|
||||
**Scope:** Replace v0.2's mock engine inputs with real credential engines. Build the sandbox fabric (sandboxed IDE / design tool / simulation), the live in-environment build-telemetry pipeline, the process-trace grading engine, per-learner variant task generation, and the oral/voice defense with AI examiner. Lab/Assessor/Proctor agents move from mock inputs to real engine inputs; the six tutor agents operate on authentic telemetry and artifacts.
|
||||
|
||||
**Status of v0.1:** Complete and shipped (v0.1.0). Founder agreement recorded (D-013).
|
||||
**Status of v0.2:** Complete and shipped (v0.2.0). Six AI tutor agents live over mockengine inputs (D-015).
|
||||
|
||||
**Tech stack:** v0.1 TS monorepo (pnpm/turborepo, Next.js) + new Python FastAPI service (`apps/ai-service`) with pydantic, SSE streaming, and an OpenAI-compatible provider client.
|
||||
**Deferred from earlier plan:** REQ-F-017 (identity verification + 16+/18+ age-gating) is explicitly deferred to a later milestone per founder directive. Age-gating remains the v0.1-style visual flow mockup; no real KYC backend is built in v0.3.
|
||||
|
||||
**Tech stack:** v0.1 TS monorepo (pnpm/turborepo, Next.js) + v0.2 Python FastAPI ai-service + new credential-engine services (sandbox fabric orchestrator, telemetry ingest, grading engine) in Python/TypeScript as determined at RESEARCH.
|
||||
|
||||
---
|
||||
|
||||
## Requirements (Validated)
|
||||
|
||||
The following requirements have been validated during specification and are locked for milestone v0.2 (REQ-F-001..006 activated from the deferred pool):
|
||||
The following requirements have been validated during specification and are locked for milestone v0.3 (REQ-F-007..010 and REQ-F-021 activated from the deferred pool; REQ-F-017 deferred per founder directive):
|
||||
|
||||
1. AI tutor service infrastructure — `apps/ai-service` FastAPI application, provider-agnostic LLM client, SSE streaming, session/state handling
|
||||
2. Agent framework — base agent contracts, prompt management, streaming pipeline, structured outputs
|
||||
3. Coach agent — pacing, motivation, retrieval practice (REQ-F-001)
|
||||
4. Tutor agent — concept delivery, Socratic questioning (REQ-F-002)
|
||||
5. Lab agent — in-flow feedback over simulated sandbox telemetry (REQ-F-003, mock inputs)
|
||||
6. Assessor agent — rubric application to pre-baked artifacts and defenses (REQ-F-004, mock inputs)
|
||||
7. Proctor agent — integrity signals from mock telemetry, coaching interventions (REQ-F-005, mock inputs)
|
||||
8. Mentor agent — long-horizon career narrative (REQ-F-006)
|
||||
9. Learner surface integration — streaming chat UI wired to the real service, error/loading states
|
||||
1. Sandbox fabric — sandboxed IDE, design tool, and simulation environments with isolated execution and lifecycle management (REQ-F-021)
|
||||
2. Live build telemetry — in-environment capture of process events (keystrokes, commands, file diffs, run/test results) streamed to ai-service (REQ-F-010)
|
||||
3. Process-trace grading engine — grade artifacts from their process traces, not just final output (REQ-F-007); feeds the Assessor agent real inputs
|
||||
4. Variant task generation — per-learner task variants so no two learners receive identical prompts (REQ-F-008)
|
||||
5. Oral/voice defense — AI examiner conducts spoken defense of submitted work (REQ-F-009); feeds the Proctor/Mentor agents
|
||||
6. Agent re-grounding — Lab/Assessor/Proctor consume real engine inputs (telemetry, traces, defenses) instead of v0.2 mocks
|
||||
7. Learner surface integration — wire the v0.1 sandbox + assessment mockups to the real engines (build/run in-browser, live telemetry, live defense)
|
||||
|
||||
## v0.2 Requirements (Complete)
|
||||
|
||||
All 12 v0.2 requirements (REQ-2-001..012) are complete and shipped as v0.2.0. See REQUIREMENTS.md traceability matrix.
|
||||
|
||||
## v0.1 Requirements (Complete)
|
||||
|
||||
@@ -53,12 +57,10 @@ All 28 v0.1 requirements (REQ-001..028) are complete and shipped as v0.1.0. See
|
||||
|
||||
## Requirements (Active — Future Milestones)
|
||||
|
||||
The following remain deferred beyond v0.2 and will be activated in subsequent milestones:
|
||||
The following remain deferred beyond v0.3 and will be activated in subsequent milestones:
|
||||
|
||||
- Identity verification and age-gating logic (16+/18+) — the real KYC backend (**deferred from v0.3 per founder directive**; visual flow already exists in v0.1)
|
||||
- Competency graph engine and adaptive pathways
|
||||
- Assessment engine (process-trace grading, oral defense, per-learner variant tasks) — v0.3+
|
||||
- Sandbox fabric (sandboxed IDE, design tool, simulation) — v0.3+
|
||||
- Identity verification and age-gating logic (16+/18+) — the real KYC backend (v0.3+; visual flow already exists in v0.1)
|
||||
- Marketplace job aggregation pipeline (3M+ jobs from 120K companies)
|
||||
- AI-powered tagging, semantic vector search, company enrichment
|
||||
- AI resume parsing and job matching
|
||||
|
||||
+58
-18
@@ -1,6 +1,33 @@
|
||||
# Nextcraft — REQUIREMENTS.md
|
||||
|
||||
## v0.2 Requirements (AI Tutor Architecture)
|
||||
## v0.3 Requirements (Credential Engines)
|
||||
|
||||
### Sandbox & Telemetry
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-3-001 | Sandbox fabric: isolated per-learner execution environments (sandboxed IDE, design tool, simulation) with lifecycle management | critical | 1 | pending |
|
||||
| REQ-3-002 | Sandbox isolation + resource limits: per-learner isolation boundary, CPU/memory/disk/time quotas, no cross-tenant access, snapshot support | critical | 1 | pending |
|
||||
| REQ-3-003 | Live build telemetry: in-environment capture of process events (commands, file diffs, run/test results, activity) streamed reliably to ai-service with per-learner trace persistence | critical | 2 | pending |
|
||||
|
||||
### Credential Engines
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-3-004 | Process-trace grading engine: grade artifacts from their full process traces; rubric-aligned structured scores; feeds Assessor real inputs | critical | 3 | pending |
|
||||
| REQ-3-005 | Variant task generation: per-learner task variants (no two learners get identical prompts); variant seed registry; difficulty normalization | high | 4 | pending |
|
||||
| REQ-3-006 | Oral/voice defense: AI examiner conducts spoken defense (STT → dialogue → TTS); transcript + integrity signals captured; feeds Proctor/Mentor | high | 5 | pending |
|
||||
|
||||
### Agent Re-grounding & Integration
|
||||
|
||||
| ID | Description | Priority | Phase | Status |
|
||||
|----|-------------|----------|-------|--------|
|
||||
| REQ-3-007 | Agent re-grounding: Lab consumes live telemetry; Assessor consumes grading-engine output; Proctor consumes telemetry + defense integrity signals (replace v0.2 mocks) | critical | 6 | pending |
|
||||
| REQ-3-008 | Learner surface integration: sandbox mockup → real in-browser build/run with live telemetry; assessment mockup → live defense + live grading | critical | 6 | pending |
|
||||
|
||||
---
|
||||
|
||||
## v0.2 Requirements (Complete — AI Tutor Architecture)
|
||||
|
||||
### AI Service Infrastructure
|
||||
|
||||
@@ -99,11 +126,11 @@
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-007 | Process-trace grading engine | high | v0.3+ | deferred |
|
||||
| REQ-F-008 | Per-learner variant task generation | high | v0.3+ | deferred |
|
||||
| REQ-F-009 | Oral/voice defense with AI examiner | high | v0.3+ | deferred |
|
||||
| REQ-F-010 | Live in-environment build with telemetry | high | v0.3+ | deferred |
|
||||
| REQ-F-021 | Sandbox fabric: sandboxed IDE, design tool, simulation | high | v0.3+ | deferred |
|
||||
| REQ-F-007 | Process-trace grading engine → activated as REQ-3-004 | high | v0.3 | activated |
|
||||
| REQ-F-008 | Per-learner variant task generation → activated as REQ-3-005 | high | v0.3 | activated |
|
||||
| REQ-F-009 | Oral/voice defense with AI examiner → activated as REQ-3-006 | high | v0.3 | activated |
|
||||
| REQ-F-010 | Live in-environment build with telemetry → activated as REQ-3-003 | high | v0.3 | activated |
|
||||
| REQ-F-021 | Sandbox fabric: sandboxed IDE, design tool, simulation → activated as REQ-3-001/002 | high | v0.3 | activated |
|
||||
|
||||
### Marketplace Engine
|
||||
|
||||
@@ -120,7 +147,7 @@
|
||||
|
||||
| ID | Description | Priority | Milestone | Status |
|
||||
|----|-------------|----------|-----------|--------|
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.3+ | deferred |
|
||||
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.4+ | deferred (deferred from v0.3 per founder directive) |
|
||||
| REQ-F-018 | Payment processing and subscription management | high | v0.3+ | deferred |
|
||||
| REQ-F-019 | Human tutor marketplace (third-party courses) | medium | v0.4+ | deferred |
|
||||
| REQ-F-020 | CIRR-style placement tracking and audit | medium | v0.5+ | deferred |
|
||||
@@ -143,22 +170,35 @@
|
||||
|
||||
## Traceability Matrix
|
||||
|
||||
### v0.2 (current milestone)
|
||||
### v0.3 (current milestone)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-3-001 | 1 | pending |
|
||||
| REQ-3-002 | 1 | pending |
|
||||
| REQ-3-003 | 2 | pending |
|
||||
| REQ-3-004 | 3 | pending |
|
||||
| REQ-3-005 | 4 | pending |
|
||||
| REQ-3-006 | 5 | pending |
|
||||
| REQ-3-007 | 6 | pending |
|
||||
| REQ-3-008 | 6 | pending |
|
||||
|
||||
### v0.2 (complete)
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| REQ-2-001 | 1 | complete |
|
||||
| REQ-2-002 | complete |
|
||||
| REQ-2-002 | 1 | complete |
|
||||
| REQ-2-003 | 1 | complete |
|
||||
| REQ-2-004 | complete |
|
||||
| REQ-2-005 | complete |
|
||||
| REQ-2-006 | complete |
|
||||
| REQ-2-007 | complete |
|
||||
| REQ-2-008 | complete |
|
||||
| REQ-2-009 | complete |
|
||||
| REQ-2-010 | complete |
|
||||
| REQ-2-011 | complete |
|
||||
| REQ-2-012 | complete |
|
||||
| REQ-2-004 | 2 | complete |
|
||||
| REQ-2-005 | 3 | complete |
|
||||
| REQ-2-006 | 3 | complete |
|
||||
| REQ-2-007 | 4 | complete |
|
||||
| REQ-2-008 | 4 | complete |
|
||||
| REQ-2-009 | 5 | complete |
|
||||
| REQ-2-010 | 5 | complete |
|
||||
| REQ-2-011 | 6 | complete |
|
||||
| REQ-2-012 | 6 | complete |
|
||||
|
||||
### v0.1 (complete)
|
||||
|
||||
|
||||
+84
-80
@@ -2,13 +2,15 @@
|
||||
|
||||
## Overview
|
||||
|
||||
**Milestone v0.2** — AI Tutor Architecture: The six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services in a new `apps/ai-service` Python FastAPI application, wired into the existing v0.1 learner surface with streaming responses. Provider-agnostic LLM layer (ollama-cloud default). Lab/Assessor/Proctor operate on mock engine inputs — their real engines are v0.3+.
|
||||
**Milestone v0.3** — Credential Engines: Replace v0.2's mock engine inputs with real credential engines. Build the sandbox fabric (sandboxed IDE / design tool / simulation), the live in-environment build-telemetry pipeline, the process-trace grading engine, per-learner variant task generation, and the oral/voice defense with AI examiner. Lab/Assessor/Proctor agents move from mock inputs to real engine inputs.
|
||||
|
||||
**Prior milestone:** v0.1 (nextcraft-ui-prototype) — complete, shipped as v0.1.0, founder-agreed (D-013).
|
||||
**Deferred per founder directive:** REQ-F-017 identity verification + age-gating (real KYC backend) is deferred beyond v0.3. Age-gating remains the v0.1 visual flow mockup.
|
||||
|
||||
**Milestone type:** Feature (new AI service + real agent capabilities)
|
||||
**Tag line:** v0.1.x (patches on the v0.1 line; milestone release as v0.2.0)
|
||||
**Branch:** milestone/v0.2-ai-tutor-architecture
|
||||
**Prior milestone:** v0.2 (ai-tutor-architecture) — complete, shipped as v0.2.0, six tutor agents live over mock engine inputs (D-015).
|
||||
|
||||
**Milestone type:** Feature (new credential-engine services + real agent inputs)
|
||||
**Tag line:** v0.2.x (patches on the v0.2 line; milestone release as the final v0.2.x patch)
|
||||
**Branch:** milestone/v0.3-credential-engines
|
||||
|
||||
---
|
||||
|
||||
@@ -16,14 +18,14 @@
|
||||
|
||||
| # | Name | Status | Depends On | Requirements | Success Criteria |
|
||||
|---|------|--------|------------|--------------|------------------|
|
||||
| 0 | Pre-execution | complete | — | — | Specification, clarify, research, plan complete; .ciagent/ files updated for v0.2 |
|
||||
| 1 | AI service scaffolding | complete | 0 | REQ-2-001, REQ-2-002, REQ-2-003 | apps/ai-service runs (uvicorn), health endpoint responds, provider-agnostic LLM client with 3 providers (ollama-cloud/local/mock), SSE streaming verified, pytest suite passes with mock provider, turbo scripts wired |
|
||||
| 2 | Agent framework | complete | 1 | REQ-2-004 | Base agent contract, session/state store, prompt templates, streaming pipeline, structured outputs; all tested |
|
||||
| 3 | Coach + Tutor agents | complete | 2 | REQ-2-005, REQ-2-006 | Coach (pacing/motivation/retrieval practice) and Tutor (concept delivery/Socratic questioning) fully implemented with system prompts, tested against mock provider, wired to chat endpoint |
|
||||
| 4 | Lab + Assessor agents | complete | 2 | REQ-2-007, REQ-2-008 | Lab consumes simulated sandbox telemetry (mock); Assessor applies rubrics to pre-baked artifacts/defense transcripts (mock); both tested |
|
||||
| 5 | Proctor + Mentor agents | complete | 2 | REQ-2-009, REQ-2-010 | Proctor produces integrity signals + coaching interventions from mock telemetry; Mentor generates long-horizon career narrative; both tested |
|
||||
| 6 | Learner surface integration | complete | 3, 4, 5 | REQ-2-011, REQ-2-012 | Learner chat streams real responses; agent routing works; byte viewer/sandbox/assessment mockups surface agent outputs; error/loading states; pnpm build + typecheck pass |
|
||||
| 7 | Final review + ship | complete | 6 | — | Code review clean; audit passes; milestone tagged v0.2.0; release created on Gitea |
|
||||
| 0 | Pre-execution | in-progress | — | — | Specification, clarify, research, plan complete; .ciagent/ files updated for v0.3 |
|
||||
| 1 | Sandbox fabric | pending | 0 | REQ-3-001, REQ-3-002 | Isolated per-learner sandbox environments provisioned (IDE / design / simulation); lifecycle API (create/destroy/snapshot); resource limits enforced; no cross-tenant access |
|
||||
| 2 | Live build telemetry | pending | 1 | REQ-3-003 | In-environment capture of process events (commands, file diffs, run/test results, keystroke-level activity) streamed to ai-service; reliable transport; per-learner trace persistence |
|
||||
| 3 | Process-trace grading engine | pending | 2 | REQ-3-004 | Grades artifacts from their full process traces (not just final output); emits structured rubric-aligned scores; feeds Assessor real inputs |
|
||||
| 4 | Variant task generation | pending | 1 | REQ-3-005 | Per-learner task variants generated so no two learners receive identical prompts; variant seed recorded for grading fairness |
|
||||
| 5 | Oral / voice defense | pending | 3 | REQ-3-006 | AI examiner conducts spoken defense of submitted work; STT → dialogue → TTS; transcript + integrity signals captured; feeds Proctor/Mentor |
|
||||
| 6 | Agent re-grounding + learner surface integration | pending | 2,3,4,5 | REQ-3-007, REQ-3-008 | Lab/Assessor/Proctor consume real engine inputs; v0.1 sandbox + assessment mockups wired to real engines (in-browser build/run, live telemetry, live defense) |
|
||||
| 7 | Final review + ship | pending | 6 | — | Code review clean; audit passes; milestone tagged (v0.2.x final patch); release created on Gitea |
|
||||
|
||||
---
|
||||
|
||||
@@ -31,129 +33,127 @@
|
||||
|
||||
### Phase 0: Pre-execution
|
||||
|
||||
**Goal:** Establish v0.2 specification, clarify ambiguities, research AI service architecture, create detailed plans.
|
||||
**Goal:** Establish v0.3 specification, clarify ambiguities, research credential-engine architecture (sandbox isolation, telemetry transport, trace grading, variant generation, voice IO), create detailed plans.
|
||||
|
||||
**Stages:** SPECIFY → CLARIFY → RESEARCH → PLAN → GRILL → SHIP
|
||||
**Stages:** SPECIFY → CLARIFY → RESEARCH → PLAN → GRILL → MVP/UX CHECK → SHIP
|
||||
|
||||
**Deliverables:**
|
||||
- Updated .ciagent/config.json, PROJECT.md, REQUIREMENTS.md, ROADMAP.md, ARCHITECTURE.md, PERSONAS.md, PLAN.md
|
||||
|
||||
**Success criteria:** All .ciagent/ files updated for v0.2; phase 0 shipped as v0.1.1.
|
||||
**Success criteria:** All .ciagent/ files updated for v0.3; phase 0 shipped as first v0.2.x patch.
|
||||
|
||||
---
|
||||
|
||||
### Phase 1: AI Service Scaffolding
|
||||
### Phase 1: Sandbox Fabric
|
||||
|
||||
**Goal:** Stand up apps/ai-service with the provider-agnostic LLM layer and SSE streaming.
|
||||
**Goal:** Provision and manage isolated per-learner execution environments.
|
||||
|
||||
**Requirements:** REQ-2-001, REQ-2-002, REQ-2-003
|
||||
**Requirements:** REQ-3-001, REQ-3-002
|
||||
|
||||
**Key deliverables:**
|
||||
- apps/ai-service: FastAPI app, pydantic-settings, uvicorn, /health, CORS for localhost
|
||||
- llm package: provider interface + ollama-cloud/local/mock providers; key resolution from .ciagent/.env.secrets via env
|
||||
- SSE streaming: /v1/chat/stream endpoint streaming provider deltas
|
||||
- pytest suite with mock provider; root scripts: ai:dev, ai:test; turbo integration
|
||||
- Sandbox orchestrator service: create/list/destroy/snapshot sandbox instances (IDE, design tool, simulation)
|
||||
- Isolation boundary: per-learner containerization or VM-grade isolation; no cross-tenant filesystem/network access
|
||||
- Resource limits: CPU/memory/disk/time quotas per sandbox
|
||||
- Sandbox lifecycle API consumed by ai-service and the web learner surface
|
||||
|
||||
**Success criteria:**
|
||||
- `python -m uvicorn` starts the service; /health returns 200
|
||||
- Provider unit tests pass (mock); ollama-cloud integration probe works (manual)
|
||||
- SSE stream delivers tokens to an HTTP client
|
||||
- A sandbox can be created, written to, snapshotted, and destroyed via API
|
||||
- Isolation verified: a sandbox cannot read another learner's data
|
||||
- Resource limits enforced and observable
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Agent Framework
|
||||
### Phase 2: Live Build Telemetry
|
||||
|
||||
**Goal:** Build the shared framework all six agents use.
|
||||
**Goal:** Capture in-environment process events and stream them to ai-service reliably.
|
||||
|
||||
**Requirements:** REQ-2-004
|
||||
**Requirements:** REQ-3-003
|
||||
|
||||
**Key deliverables:**
|
||||
- BaseAgent contract: system prompt, message history, streaming completion, structured output
|
||||
- Session/state store: in-memory per-learner session with message history
|
||||
- Prompt management: per-agent system prompt templates with learner context injection
|
||||
- Streaming pipeline: agent → provider → SSE with agent identification
|
||||
- Structured outputs: JSON-schema outputs for Assessor rubric scores, Proctor signals
|
||||
- Telemetry capture agent (in-sandbox): commands, file diffs, run/test results, keystroke-level/activity events
|
||||
- Telemetry transport: durable, ordered, resumable stream to ai-service ingestion endpoint
|
||||
- Trace persistence: per-learner, per-task process traces stored for grading and proctoring
|
||||
- Transport hardening: retries, backpressure, exactly-once-or-at-least-once semantics documented
|
||||
|
||||
**Success criteria:**
|
||||
- BaseAgent unit tests pass
|
||||
- Session store tested (create/append/persist in-memory)
|
||||
- Structured output parsing tested against mock provider
|
||||
- Sandbox activity produces a complete ordered process trace in ai-service
|
||||
- Stream survives transient network failure without trace loss
|
||||
- Trace retrievable by learner+task ID for grading
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Coach + Tutor Agents
|
||||
### Phase 3: Process-Trace Grading Engine
|
||||
|
||||
**Goal:** Implement the two learner-facing conversational agents.
|
||||
**Goal:** Grade learner artifacts from their full process traces.
|
||||
|
||||
**Requirements:** REQ-2-005, REQ-2-006
|
||||
**Requirements:** REQ-3-004
|
||||
|
||||
**Key deliverables:**
|
||||
- Coach agent: pacing guidance, motivation, retrieval practice prompts; distinct persona
|
||||
- Tutor agent: concept delivery, Socratic questioning, worked examples
|
||||
- Agent registry: route chat messages to the correct agent by context/selection
|
||||
- Per-agent system prompts with competency-stack context injection from packages/mock-data
|
||||
- Trace analyzer: reconstructs build/decision timeline from a process trace
|
||||
- Grading engine: rubric-aligned scoring over the trace (process quality, not just final artifact)
|
||||
- Structured score output consumable by the Assessor agent
|
||||
- Calibration against v0.2 mock corpora to validate grading dimensions
|
||||
|
||||
**Success criteria:**
|
||||
- Both agents produce distinct, on-persona responses (verified against mock + ollama-cloud)
|
||||
- Agent routing tested
|
||||
- Both agents exposed via the chat streaming endpoint
|
||||
- Engine emits structured rubric-aligned scores from a real process trace
|
||||
- Scores distinguish process quality (e.g., iterative debugging vs. paste-and-run)
|
||||
- Output feeds Assessor; replaces pre-baked artifact corpus inputs
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Lab + Assessor Agents
|
||||
### Phase 4: Variant Task Generation
|
||||
|
||||
**Goal:** Implement the two build/assessment agents over mock engine inputs.
|
||||
**Goal:** Generate per-learner task variants so no two learners receive identical prompts.
|
||||
|
||||
**Requirements:** REQ-2-007, REQ-2-008
|
||||
**Requirements:** REQ-3-005
|
||||
|
||||
**Key deliverables:**
|
||||
- Lab agent: consumes simulated sandbox telemetry (mock event streams), produces in-flow feedback
|
||||
- Assessor agent: applies rubrics to pre-baked artifacts and defense transcripts, returns structured scores + feedback
|
||||
- Mock engine inputs: simulated telemetry generator, pre-baked artifact corpus in packages/mock-data
|
||||
- Endpoints: /v1/lab/feedback, /v1/assessment/evaluate
|
||||
- Variant generator: parameterized task templates → unique per-learner instances
|
||||
- Variant seed registry: record variant parameters for grading fairness and proctoring
|
||||
- Difficulty normalization: variants calibrated to equivalent difficulty
|
||||
|
||||
**Success criteria:**
|
||||
- Lab produces relevant feedback for mock telemetry scenarios
|
||||
- Assessor returns structured rubric scores (JSON) for pre-baked artifacts
|
||||
- Both tested against mock provider
|
||||
- Two learners requesting the same competency receive distinct task variants
|
||||
- Variant parameters persisted and auditable
|
||||
- Grading engine scores variants equitably
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Proctor + Mentor Agents
|
||||
### Phase 5: Oral / Voice Defense
|
||||
|
||||
**Goal:** Implement the integrity and narrative agents.
|
||||
**Goal:** AI examiner conducts a spoken defense of the learner's submitted work.
|
||||
|
||||
**Requirements:** REQ-2-009, REQ-2-010
|
||||
**Requirements:** REQ-3-006
|
||||
|
||||
**Key deliverables:**
|
||||
- Proctor agent: integrity signals from mock telemetry (tab switches, idle time, paste events), coaching interventions
|
||||
- Mentor agent: long-horizon career narrative, competency-stack progression guidance
|
||||
- Endpoints: /v1/proctor/signals, /v1/mentor/narrative
|
||||
- Voice pipeline: STT → defense dialogue (LLM examiner) → TTS
|
||||
- Examiner agent: probes understanding, challenges process choices from the trace
|
||||
- Transcript + integrity signals captured for Proctor/Mentor
|
||||
- Latency budget: defense feels conversational (bounded turn latency)
|
||||
|
||||
**Success criteria:**
|
||||
- Proctor produces classified signals with recommended interventions for mock scenarios
|
||||
- Mentor produces coherent career-narrative responses
|
||||
- Both tested against mock provider
|
||||
- A spoken defense runs end-to-end (speak → examiner question → learner response → verdict)
|
||||
- Transcript + integrity signals persisted and consumable by Proctor
|
||||
- Turn latency within the documented budget
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Learner Surface Integration
|
||||
### Phase 6: Agent Re-grounding + Learner Surface Integration
|
||||
|
||||
**Goal:** Wire the v0.1 learner surface to the real AI service.
|
||||
**Goal:** Move Lab/Assessor/Proctor to real engine inputs; wire learner surface to the real engines.
|
||||
|
||||
**Requirements:** REQ-2-011, REQ-2-012
|
||||
**Requirements:** REQ-3-007, REQ-3-008
|
||||
|
||||
**Key deliverables:**
|
||||
- Learner dashboard chat: real streaming via SSE, agent switcher (Coach/Tutor), error/loading states
|
||||
- Byte tutorial viewer: Tutor concept explanations
|
||||
- Build sandbox: Lab feedback panel fed by mock telemetry + Lab agent
|
||||
- Assessment mockup: Assessor rubric output display, Proctor integrity banner
|
||||
- Mentor panel on learner dashboard
|
||||
- Lab agent consumes live sandbox telemetry (replaces v0.2 mock telemetry)
|
||||
- Assessor agent consumes grading-engine output (replaces pre-baked artifacts)
|
||||
- Proctor consumes telemetry + defense integrity signals (replaces mock telemetry)
|
||||
- Learner sandbox mockup → real in-browser build/run; assessment mockup → live defense + live grading
|
||||
|
||||
**Success criteria:**
|
||||
- Streaming chat works end-to-end with ai-service running
|
||||
- All four learner surfaces surface agent outputs
|
||||
- Graceful degradation when ai-service is down (error states, not crashes)
|
||||
- Lab/Assessor/Proctor operate on real inputs with no mock fallback in the learner path
|
||||
- Learner can build in-browser and see live telemetry + live feedback
|
||||
- Assessment surface runs a live defense and shows live grading
|
||||
- `pnpm build` and `pnpm typecheck` pass
|
||||
|
||||
---
|
||||
@@ -165,16 +165,20 @@
|
||||
**Key deliverables:**
|
||||
- Multi-persona code review (correctness, testing, security, performance, maintainability)
|
||||
- Project health audit (reconstruction test, .ciagent/ file discipline, branch hygiene, commit discipline)
|
||||
- Milestone ship: merge milestone → main, tag v0.2.0, create Gitea release
|
||||
- Milestone ship: merge milestone → main, tag final v0.2.x patch, create Gitea release
|
||||
|
||||
**Success criteria:**
|
||||
- Code review: P0 fixes applied, P1+ documented
|
||||
- Audit: all checks pass, project state reconstructable from git log
|
||||
- Ship: v0.2.0 tagged, milestone branch merged to main, Gitea release created — **release note explicitly states Lab/Assessor/Proctor operate on mock engine inputs (real engines v0.3+)** (G-5); dead `aiTutorResponses` export disposed of (G-5)
|
||||
- All 12 v0.2 requirements marked complete
|
||||
- Ship: milestone tagged, branch merged to main, Gitea release created — release note states identity/age-gating (KYC) is deferred and age-gating remains a visual mockup
|
||||
- All v0.3 requirements marked complete
|
||||
|
||||
---
|
||||
|
||||
## v0.2 (Complete — Shipped as v0.2.0)
|
||||
|
||||
AI Tutor Architecture: Six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) as real LLM-backed services over mock engine inputs, wired into the learner surface with streaming. 7 phases. All 12 requirements complete. Milestone release v0.2.0.
|
||||
|
||||
## v0.1 (Complete — Shipped as v0.1.0)
|
||||
|
||||
UI/UX Prototype: High-fidelity interactive prototype of all four Nextcraft surfaces. 7 phases (P0 + P1-P6 execution + P7 final). All 28 requirements complete. Tags v0.0.1–v0.0.7, milestone release v0.1.0.
|
||||
@@ -46,9 +46,9 @@
|
||||
"projects": [],
|
||||
"active_project": null,
|
||||
"milestone": {
|
||||
"version": "v0.2",
|
||||
"name": "ai-tutor-architecture",
|
||||
"version": "v0.3",
|
||||
"name": "credential-engines",
|
||||
"type": "feature",
|
||||
"branch": "milestone/v0.2-ai-tutor-architecture"
|
||||
"branch": "milestone/v0.3-credential-engines"
|
||||
}
|
||||
}
|
||||
@@ -48,3 +48,4 @@ coverage/
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
*.egg-info/
|
||||
.ciagent/bin/
|
||||
|
||||
Reference in New Issue
Block a user