nextcraft/apps/ai-service

Nextcraft AI Service (apps/ai-service)

Python FastAPI service hosting the six AI tutor agents (Coach, Tutor, Lab, Assessor, Proctor, Mentor) behind a provider-agnostic LLM layer. Port 8420.

Quickstart

# 1. Bootstrap (idempotent): venv + deps
bash scripts/bootstrap.sh

# 2. Run tests (mock provider only — zero network calls)
bash scripts/test.sh

# 3. Lint
bash scripts/lint.sh

# 4. Dev server (exports keys from .ciagent/.env.secrets if present)
bash scripts/dev.sh

Or via the monorepo root (corepack pnpm install first):

pnpm ai:bootstrap
pnpm ai:test
pnpm ai:lint
pnpm ai:dev

Configuration

All settings use the AI_ env prefix (pydantic-settings; see .env.example).

Var	Default	Purpose
AI_PORT	8420	Listen port
AI_PROVIDER	mock	ollama-cloud | local | mock
AI_MODEL	gemma4:31b	Model for all agents
AI_OLLAMA_CLOUD_BASE_URL	https://ollama.com/v1	Cloud base URL
AI_OLLAMA_CLOUD_API_KEY	(empty)	Bearer key — never commit
AI_JSON_MODE	auto	auto sends response_format, degrades on 400; off never sends

Tests run with AI_PROVIDER=mock (enforced in tests/conftest.py by an instance assertion) — the suite never calls the cloud.

Endpoints

• GET /health — status, configured provider, model (no cloud call)
• POST /v1/chat/stream — SSE chat stream. Body: {"agent": "coach"|"tutor", "session_id": "...", "messages": [{"role":"user","content":"..."}]}. Unknown agents are rejected with 422.

SSE envelope (D-016): meta event first (agent/session/model), then delta events (incremental content), then done; on mid-stream failure an error event precedes the terminal [DONE] sentinel. sse-starlette emits : ping keep-alive comment lines on idle connections — clients must ignore frames without data:.

Manual ollama-cloud persona probe (Phase 3, documented — not automated)

With the real provider, Coach and Tutor must produce distinct on-persona responses to the same prompt:

# start with the cloud provider (keys exported from .ciagent/.env.secrets)
AI_PROVIDER=ollama-cloud .venv/bin/uvicorn ai_service.main:app --port 8420

# Coach: expect pacing + one concrete next action + a retrieval-practice question
curl -sN -X POST localhost:8420/v1/chat/stream -H 'Content-Type: application/json' \
  -d '{"agent":"coach","session_id":"probe-coach","messages":[{"role":"user","content":"I am stuck on multi-agent communication patterns"}]}' \
  | grep '^data:'

# Tutor: expect ONE concept + a worked example + a Socratic check question
curl -sN -X POST localhost:8420/v1/chat/stream -H 'Content-Type: application/json' \
  -d '{"agent":"tutor","session_id":"probe-tutor","messages":[{"role":"user","content":"I am stuck on multi-agent communication patterns"}]}' \
  | grep '^data:'

Verify: the two responses have visibly different voice/structure (Coach: action + accountability; Tutor: concept + example + question). The automated suite never calls the cloud — distinctness is enforced against the deterministic mock (distinct system prompts → distinct hash-seeded outputs).

Sandbox isolation (v0.3)

The v0.3 code-execution sandbox runs learner/agent code in a Linux user namespace (unshare --user --map-root-user --mount --pid --fork --net): the child is uid 0 inside the userns (mapped to the unprivileged host uid), gets a private mount + PID + network namespace, and uses RLIMIT_* for resource caps. No containers, no sudo — see "Why not containers" below.

A-101 / D-024 isolation probe transcript

Verbatim output captured on the CI box (Linux, uid 1001 opencode, no docker/podman/bwrap; iproute2 absent so interface state is read from kernel sockets + /proc/net/dev).

1. Root-in-userns, uid 0, isolated namespaces:

$ unshare --user --map-root-user --mount --pid --fork --net id -u
0

2. Fresh netns has exactly one interface: lo only (no eth0, no route out). Host baseline for contrast:

$ unshare --user --map-root-user --mount --pid --fork --net \
    python3 -c "import socket; print(socket.if_nameindex())"
[(1, 'lo')]

$ unshare --user --map-root-user --mount --pid --fork --net \
    awk 'NR>2{print $1}' /proc/net/dev
lo:

$ python3 -c "import socket; print(socket.if_nameindex())"   # host
[(1, 'lo'), (2, 'eth0')]

(Note: /sys/class/net shows host interfaces even inside the netns because sysfs here is not netns-aware — the socket-level view above is the authoritative kernel evidence: 1 interface, loopback only, zero rx bytes, no carrier to any external link.)

3. Write containment — writes inside the sandbox workdir are visible on the host under the sandbox dir, owned by the real (unprivileged) host uid:

$ unshare --user --map-root-user --mount --pid --fork --net bash -c "
    mkdir -p /tmp/demo-work && cd /tmp/demo-work
    echo 'hello-from-inside-sandbox (uid=0 in-ns)' > contained.txt
    id -u"
0

$ cat /tmp/demo-work/contained.txt                          # host
hello-from-inside-sandbox (uid=0 in-ns)
$ ls -la /tmp/demo-work/contained.txt                       # host
-rw-r--r-- 1 opencode opencode 40 ... /tmp/demo-work/contained.txt

The in-userns "root" writes land on the host filesystem as uid 1001 (opencode) — the uid-mapping is doing the confinement; nothing escapes the sandbox workdir as any other identity.

4. /proc remount is NOT permitted in this context — probe + exact error:

$ unshare --user --map-root-user --mount --pid --fork --net \
    bash -c "mount -t proc proc /proc"
mount: /proc: permission denied.
       dmesg(1) may have more information after failed mount system call.
(exit 32)

mount -t proc fails even with in-ns "root" because /proc is owned by a userns that does not contain our uid mapping (the box's / is itself owned by nobody:nogroup — we're already inside a container). This is acceptable for v0.3: the sandbox does not depend on a custom /proc view; the child sees the host /proc read-only-ish view which is already filtered by the pid namespace (only in-ns pids are visible). The pidns itself is what provides process isolation, not the proc remount.

Locked resource-limit mechanism (G-1 / G-2)

Resource enforcement is settled for v0.3 — this is the locked decision:

Resource	Mechanism	Notes
Memory	RLIMIT_AS (address space)	setrlimit in the child pre-exec; deterministic, no cgroup needed
CPU	RLIMIT_CPU	kernel SIGKILL at the cpu-seconds ceiling
Single-file size	RLIMIT_FSIZE	catches runaway single-file writes
Wall clock	manager reaper kill (parent watchdog)	RLIMIT_CPU doesn't cover sleeping/idle children; the manager kills the sandbox on wall-clock timeout
Per-sandbox process count	RLIMIT_NPROC	⚠️ SHARED at the host uid, not per-sandbox — the counter is per-real-uid across all of that uid's process trees, so two concurrent sandboxes share the same NPROC budget. Accepted v0.3 gap: without cgroup delegation there's no per-sandbox pid cap; mitigations are (a) the manager serializes sandbox runs and (b) NPROC is still a hard fork-bomb ceiling.
Hard disk quota	NOT kernel-enforceable	⚠️ without cgroup delegation or sudo (quotactl, project quotas) there is no kernel-enforced per-sandbox disk cap. Accepted v0.3 gap. Mitigation: a manager-side workdir-size sweep — after each run (and on a periodic reaper pass) the manager walks the sandbox workdir and enforces AI_SANDBOX_MAX_WORKDIR_MB (default 512 MB); oversized dirs are reaped. Combined with RLIMIT_FSIZE this bounds disk growth between sweeps.

Both accepted gaps (shared NPROC, no kernel disk quota) are documented here as v0.3 scope boundaries; closing them requires cgroup v2 delegation or sudo, neither of which is available in the target environment.

Why not containers

Container runtimes / privileged wrapper tools are probed-and-absent on the box, and we have no sudo:

$ for cmd in docker podman bwrap firejail; do
    printf '%-8s: ' "$cmd"; command -v "$cmd" || echo MISSING
  done; printf '%-8s: ' sudo; command -v sudo || echo MISSING
docker  : MISSING
podman  : MISSING
bwrap   : MISSING
firejail: MISSING
sudo    : MISSING
$ id -u
1001

Unprivileged user namespaces are on the box's kernel and need neither a daemon, nor suid helpers, nor network access — they are the only isolation primitive that works here, so that's what v0.3 uses.

Telemetry delivery semantics (v0.3, REQ-3-003)

Delivery is at-least-once; storage is exactly-once — the two compose:

• The in-sandbox capture agent (stdlib-only, scripts/sandbox-agent.py) spools every event to a durable JSONL file (fsync per append) BEFORE any send attempt, so no event can be lost to a dead socket or a SIGKILL.
• The WS ingest endpoint (WS /v1/telemetry/ingest?learner_id&task_id, D-026) dedups server-side on the (learner_id, task_id, seq) primary key: re-sends (reconnect flushes, replay margin) are collapsed, never upserted.
• On disconnect the agent reconnects with exponential backoff and flushes the spool in seq order; a transient outage therefore loses nothing and stores each event exactly once (tests/telemetry/test_durability.py proves this end-to-end against a real namespace sandbox + live server).
• Replay/read path: GET /v1/telemetry/traces/{learner}/{task} returns the complete ordered trace; GET /v1/telemetry/gaps/{learner}/{task} returns missing seqs for gap detection.
• Flood boundary (G-3): a connection exceeding AI_TELEMETRY_MAX_EVENTS_PER_TASK (default 50,000) is closed with WS code 1008 and its trace is marked INCOMPLETE_FLOODED — a terminal integrity flag the grader refuses to grade. Silent event dropping is forbidden: it would corrupt grading input.

Voice defense (v0.3, REQ-3-006)

Voice is mock-first (D-030): the defense pipeline is fully proven over the deterministic MockVoiceProvider + browser-native fallback — no task requires a real voice key. Real server STT/TTS (OpenAIAudioProvider over OpenAI-compatible /audio/transcriptions + /audio/speech) is deferred to v0.4 together with KYC (GRILL CUT-1 / G-7): it could never be exercised in CI, so v0.3 ships the protocol seam instead of an unverifiable claim.

• AI_VOICE_PROVIDER=mock (default) — deterministic canned STT/TTS
• AI_VOICE_PROVIDER=browser — the web client uses SpeechRecognition + speechSynthesis; the server keeps text-turn persistence
• Conversational budget: a defense turn should complete in < 4s (DEFENSE_TURN_BUDGET_MS in tests/voice/test_latency.py). v0.3 asserts instrumentation (stt_ms/llm_ms/tts_ms populated per turn); the wall-clock acceptance probe against a real voice endpoint is a v0.4 criterion, run manually with AI_VOICE_PROVIDER set to the real provider and keys in .ciagent/.env.secrets (never in code/commits).

End-to-end credential flow (v0.3, REQ-3-007/008)

tests/api/test_e2e_credential_flow.py runs the full pipeline against a REAL uvicorn server with REAL namespace sandboxes (mock LLM/voice per G-2 precedent): variant -> telemetry-wired sandbox -> in-sandbox exec -> trace persistence -> process-trace grade (variant seed stamped) -> assessor coaching -> oral defense -> verdict + integrity signals -> proctor. It asserts no corpus fixture appears anywhere in the learner path.

Manual browser pass (documented, not automated): pnpm ai:dev + pnpm dev, then open /build/stack-orchestration-c007 — variant statement + starter files load, edit a file, Run/Test execute in the sandbox with output in the read-only panel, the telemetry status pulses, Lab streams feedback from the live digest; then /defend/stack-orchestration-c007 — Start Defense, typed answers (mic path needs permission), Finish, Grade My Work renders the real rubric bars. Navigating away destroys the sandbox (curl localhost:8420/v1/sandboxes shows the count drop).

Layout

ai_service/
  main.py       app factory, lifespan (httpx pool), CORS, /health
  config.py     pydantic-settings
  api/          endpoints (SSE envelope lives here, D-016)
  llm/          provider layer — dumb pipe, no envelope logic
scripts/        bootstrap.sh dev.sh test.sh lint.sh
tests/          pytest — mock provider only

Boundary rules: llm/ imports nothing from agents/ or api/; agents/ imports nothing from api/.