nextcraft/.ciagent/ARCHITECTURE.md

Nextcraft — ARCHITECTURE.md

Overview

Nextcraft is a TypeScript monorepo (pnpm workspaces + turborepo) with a Next.js application hosting four surfaces (Learner, Marketplace, Employer Dashboard, Admin), a shared component library, typed mock data layer, and shared types package. As of v0.2, a Python FastAPI application (apps/ai-service) hosts six AI tutor agents backed by a provider-agnostic LLM layer.

v0.2 additions: real AI services (streaming chat), agent framework, mock engine inputs for Lab/Assessor/Proctor. Still no database, no auth — in-memory session store; real engines (sandbox fabric, assessment engine, identity) are v0.3+.

Confirmed Technology Stack (v0.2)

Technology	Version	Purpose
Node.js	v24.15.0	Runtime (web)
pnpm	12.3.4	Package manager + workspaces
turborepo	2.3.3	Build orchestration
Next.js	15 (App Router)	Web application framework
React	19+	UI library
TypeScript	5.x	Type system
Tailwind CSS	v4	Utility-first CSS
lucide-react	latest	Icons
recharts	latest	Charts
@xyflow/react	latest	Competency graph viewer
Python	3.11.2	Runtime (ai-service)
FastAPI	0.141.x	AI service framework
uvicorn	0.52.x	ASGI server
pydantic	2.13.x	Request/response models, structured outputs
pydantic-settings	2.15.x	Settings + env-file loading (replaces python-dotenv)
httpx	0.28.x	Async LLM HTTP client (ollama-cloud + local providers)
sse-starlette	3.4.x	SSE framing, ping keep-alive
pytest	9.x	Test runner
pytest-asyncio	1.4.x	Async tests (auto mode)
ruff	latest	Python lint (check-only, no formatter) — pnpm ai:lint
ollama-cloud	https://ollama.com/v1	Default LLM provider (OpenAI-compatible, Bearer auth)

Deliberately not used: openai-python SDK (the LLMProvider protocol is the port; raw httpx keeps delta passthrough and ollama-cloud quirk tolerance), python-dotenv (pydantic-settings reads .env natively), respx (httpx MockTransport is built in).

v0.2 Architecture Decisions (from Research)

1. D-016 SSE envelope — meta event (agent/session/model, flushed before first token) → raw OpenAI-compatible chunk passthrough → optional done → error event before [DONE] on mid-stream failure. Pre-first-byte failures use proper HTTP status. Headers: Cache-Control: no-cache, X-Accel-Buffering: no.
2. D-017 httpx direct client — lifespan-managed httpx.AsyncClient (10s connect / 300s read), shared by ollama-cloud and local providers; no SDK.
3. D-018 Agent framework — BaseAgent ABC (system_prompt/build_messages/stream_reply/structured_reply) + explicit registry; prompts are versioned code in prompts/.
4. D-019 Session store — SessionStore protocol + InMemorySessionStore (asyncio.Lock, 20-message window, 500-cap LRU, agent-scoped sessions). DB-migration-ready.
5. D-020 Structured outputs — 4-layer defense: response_format (auto-degrade) → prompt-embedded schema → fence-strip/first-balanced-object parse → single bounded retry.
6. D-021 Mock corpus in Python — ai_service/corpus/ pydantic-typed, convention-aligned with TS packages/mock-data (shared IDs, cross-referencing headers); no codegen in v0.2.
7. D-022 Monorepo integration — zero-dependency shim package.json in apps/ai-service + ai#* turbo passthrough tasks (cache:false, outputs:[]) + root ai:dev/ai:test scripts + idempotent venv bootstrap.
8. D-023 Testing — pytest-asyncio auto mode; TestClient client.stream() for SSE; httpx MockTransport for byte-exact provider parser tests; scripted mock provider incl. failure modes. Tests never call the cloud.

---

Components

apps/ai-service — AI Tutor Service (v0.2 NEW)

Component	Description	Boundaries	Depends On
ai_service/main.py	FastAPI app factory, lifespan (httpx client pool, provider factory), CORS (localhost only), /health	App entry	config, llm, agents, api
ai_service/config.py	pydantic-settings Settings (env_prefix="AI_", env_file, SecretStr key)	Configuration only	None
ai_service/api/	Endpoints: chat.py (POST /v1/chat/stream, SSE), lab.py, assessment.py, proctor.py, mentor.py; deps.py (DI)	Composes agents + sessions; never imported by llm/ or agents/	agents, llm
ai_service/llm/	types.py (Message; ChatDelta/ChoiceDelta removed in P3 — no consumers), base.py (LLMProvider protocol), openai_compat.py (ollama-cloud + local), mock.py (deterministic), factory.py	Never imports agents/ or api/	config
ai_service/agents/	base.py (BaseAgent ABC), registry.py, session.py (SessionStore), structured.py (JSON defense), coach/tutor/lab/assessor/proctor/mentor.py	Never imports api/	llm, prompts, corpus
ai_service/prompts/	Per-agent system prompt constants + render_context functions (str.format_map)	Data only	None
ai_service/corpus/	Mock engine inputs: learner_context.py, telemetry.py (Lab/Proctor scenarios), artifacts.py (pre-baked artifacts, rubrics, transcripts)	Pydantic-typed; aligned with TS packages/mock-data by convention	None
scripts/	bootstrap.sh (venv + pip install idempotent), dev.sh (exports keys from .ciagent/.env.secrets → uvicorn), test.sh (pytest), lint.sh (ruff check, G-3)	Dev entry points	pyproject.toml
tests/	conftest.py (mock provider, settings override, TestClient), health, llm (MockTransport parser), agents (framework + per-agent), api (SSE stream tests)	Mock provider only — no cloud	all

Module boundary rules: llm/ never imports agents/ or api/; agents/ never imports api/; api/ composes both via DI. corpus/ is the only home of mock engine data. Prompts are code — versioned and reviewed in git.

apps/web — Next.js Application

Component	Description	Boundaries	Depends On
app/(learner)/	Learner surface route group: landing, catalog, competency stack, dashboard, byte viewer, sandbox mockup, assessment mockup	Learner-only routes and layouts	packages/ui, packages/mock-data, packages/types
app/(marketplace)/	Marketplace surface route group: job board, job detail, employer profile, search/filter, pricing	Marketplace-only routes and layouts	packages/ui, packages/mock-data, packages/types
app/(employer)/	Employer dashboard route group: overview, talent search, candidate profile, posting management	Employer-only routes and layouts	packages/ui, packages/mock-data, packages/types
app/(admin)/	Admin surface route group: overview, learner management, competency graph viewer, moderation	Admin-only routes and layouts	packages/ui, packages/mock-data, packages/types
app/layout.tsx	Root layout: theme provider, navigation shell, responsive container	All routes	packages/ui
components/	Surface-specific components (learner/, marketplace/, employer/, admin/) plus shared chrome (navigation-shell, header/footer, role-switcher, theme-provider, breadcrumbs, dark-mode-toggle)	App-level components	packages/ui
hooks/	use-chat-stream.ts — SSE client hook: fetch + ReadableStream, byte buffering + frame reassembly, idempotent AbortController cleanup	Client components only	ai-service SSE
lib/	sse.ts (shared SSE frame parser — CRLF normalization + : ping immunity, G-1), breadcrumbs.ts, format.ts	Pure utilities	None

packages/ui — Shared Component Library

Component	Description	Boundaries	Depends On
tokens/	Design tokens as TS constants: colors, spacing, radii, shadows, breakpoints (mirrored as Tailwind v4 @theme tokens in apps/web globals.css)	Foundation layer — no dependencies	None
primitives/	Button, Input, Card, Badge, Avatar — each with a Storybook story	Atomic UI components	tokens, packages/types

Composite/layout/theme components (navigation shell, tables, chat panels, graph viewer, theme provider) live in apps/web/components/ as app-level components, not in packages/ui.

packages/mock-data — Mock Data Layer

Component	Description	Boundaries	Depends On
competency-stacks.ts	5 competency stacks (AI Orchestration Engineer, AI Safety & Governance, Human-AI Product Designer, AI-Augmented Field Operator, Computational Sciences), each with 12-18 competencies	Typed mock data	packages/types
jobs.ts	20+ mock AI-era job listings with skills, seniority, salary, match scores	Typed mock data	packages/types
candidates.ts	15+ mock candidate profiles with artifacts, process traces, defense scores, microcredentials	Typed mock data	packages/types
employers.ts	10+ mock employer profiles with logos, descriptions, open positions	Typed mock data	packages/types
learner-progress.ts	Mock learner progress data: active competencies, completion percentages, recent artifacts	Typed mock data	packages/types
admin.ts	Admin surface mock data: platform metrics, activity feed, system health, learner roster (admin view), moderation queues	Typed mock data	packages/types
ai-scenarios.ts	AI engine-input scenario IDs + display metadata for the learner agent panels; IDs string-identical to ai_service/corpus/ (D-021)	Typed mock data	packages/types

packages/types — Shared Types

Component	Description	Boundaries	Depends On
domain.ts	Competency, CompetencyStack, Microcredential, Artifact, ProcessTrace, OralDefense, AssessmentRubric	Domain types	None
marketplace.ts	Job, Employer, Candidate, JobPosting, TalentMatch, SearchFilter	Marketplace types	None
user.ts	Learner, Admin, EmployerUser, AgeGroup, Role	User types	None
ui.ts	Component props, theme config, breakpoint definitions	UI types	None

---

Data Flow

[packages/mock-data + packages/types]      [ai_service/corpus]
        │ (TS, web surfaces)                      │ (Python, agent inputs)
        ▼                                         ▼
[Next.js Route Groups]                    [ai-service agents]
  (learner)/ (marketplace)/                 coach tutor lab assessor proctor mentor
  (employer)/ (admin)/                                │
        │                                             ▼
        │      SSE (fetch + ReadableStream)     [LLMProvider]
        └────── client components ◄───────────────────┤
               http://localhost:8420          ollama-cloud / local / mock
                                               (https://ollama.com/v1)

• Web surfaces remain server-component-first; client components (chat, filters, graph viewer, dark mode toggle) fetch directly from ai-service over SSE (A-002: no Next.js API-route proxy in v0.2).
• The LLM provider layer is a dumb pipe — OpenAI-compatible chunks pass through byte-identical; envelope logic (meta/done/error) lives only in the API layer (D-016).
• Lab/Assessor/Proctor read mock scenarios from ai_service/corpus/ — real engines are v0.3+.
• All automated tests use the deterministic mock provider; the cloud is for manual probes only.

---

Build Order (v0.2)

1. AI service scaffolding — apps/ai-service: FastAPI app, config, provider layer (ollama-cloud/local/mock), SSE chat endpoint, pytest harness, turbo integration
2. Agent framework — BaseAgent, registry, session store, structured output, prompts scaffolding, learner-context corpus
3. Coach + Tutor agents — full implementations, chat endpoint agent routing
4. Lab + Assessor agents — telemetry scenarios + pre-baked artifacts corpus, /v1/lab/feedback + /v1/assessment/evaluate
5. Proctor + Mentor agents — proctor scenarios, /v1/proctor/signals + /v1/mentor/narrative
6. Learner surface integration — useChatStream hook, agent switcher, streaming/error/loading states, agent output panels across the four learner surfaces

The v0.1 build order (monorepo → types → mock data → tokens → primitives → layout → composites → surfaces → polish) is complete and preserved in git history (tags v0.0.1–v0.1.0).

---

Future Architecture (Post-v0.2, for reference)

v0.2 delivers the ai-service skeleton that later milestones fill in:

• Mock corpus → real engines — Lab consumes real sandbox telemetry (v0.3 sandbox fabric); Assessor grades real process traces (v0.3 assessment engine); Proctor consumes real identity/attention signals (v0.3 identity verification)
• In-memory sessions → PostgreSQL + Drizzle ORM — SessionStore protocol swap, no API changes
• Mock provider → per-agent model routing — provider factory already selects by config; per-agent AI_<AGENT>_MODEL overrides
• No auth → real KYC + sessions — A-008 dropped in v0.3 when identity verification lands
• No search → Semantic vector search (pgvector) — Filter UI replaced with vector similarity search
• No payments → Payment processing — Pricing page replaced with real subscription/payment flows

The monorepo structure (apps/web + apps/ai-service + packages/*) accommodates further apps without restructuring.