nextcraft/.ciagent/GRILL.md

# Nextcraft v0.3 — GRILL.md (Adversarial Review Verdict)

**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-CHANGES · **Confidence:** 0.72

## Summary

The credential-pipeline architecture (telemetry → trace → grade → defense) is sound and correctly sequenced. Three plan claims did NOT survive contact with this box and were correct before execution. The central problem: the plan **overstated sandbox resource-limit enforcement** and deferred KYC **without closing the resulting no-auth local abuse vector**. Fixed via binding decisions G-1..G-6 + scope cuts CUT-1/CUT-2 — no redesign required.

## Per-Axis Findings

| Axis | Verdict | Rationale |
|------|---------|-----------|
| Feasibility | CONCERN | Core `unshare` userns/mount/pid/net isolation probe-verified (uid=0 in-ns, network isolated, writes contained). But rlimit enforcement is partial: `RLIMIT_NPROC` scopes to the real host uid (5 sandboxes share one pids budget) and no disk-quota tool exists on the box. |
| Over-scoping | CONCERN | 39 tasks across 5 new subsystems + learner-surface rewrite for a solo founder. Voice-real-path and the interactive terminal relay are separable from the pipeline proof → cut (CUT-1, CUT-2). |
| Architecture risk | CONCERN | SandboxBackend/TraceStore protocols are the right seams. Overclaimed "limits enforced" + WS ingest "drop-oldest on unbounded growth" contradicted the at-least-once grading guarantee. |
| Phase sequencing | PASS | P1 sandbox → P2 telemetry → P3 grading → P4 variants → P5 voice → P6 integration is a correct dependency DAG. |
| Verification honesty | FAIL (fixed) | Must-Haves asserted "resource limits enforced + observable" (CPU/memory/disk/time quotas) the named mechanism cannot satisfy; probe tests would pass while the guarantee was false. Corrected by G-1. |
| Cost/quota | CONCERN | LLM/voice mock-gated (good). No-auth sandbox creation + unbounded disk + shared NPROC let one learner starve others at zero cost → closed by G-5. |
| Milestone honesty | CONCERN (fixed) | Release note disclosed KYC deferral + IDE-only but was silent on partial resource-limit enforcement → G-6. |
| Security | FAIL (fixed) | `POST /v1/sandboxes {learner_id}` client-supplied over localhost CORS let any local process mint sandboxes/flood/exhaust shared resources → G-5 abuse control ships despite KYC deferral. |
| Operability | CONCERN (advisory) | In-memory sandbox registry loses handles on restart (orphaned namespaces) → a-1 startup reaper. |

## Binding Decisions (applied to PLAN.md/ARCHITECTURE-adjacent docs/REQUIREMENTS.md/ROADMAP.md/PROJECT.md)

- **G-1 (BINDING) — Resource-limit claims match the deliverable mechanism.** Memory (RLIMIT_AS) + CPU (RLIMIT_CPU) + single-file (RLIMIT_FSIZE) + wall-clock reaper are kernel-enforced; per-sandbox pids and hard disk quota are NOT kernel-enforceable without cgroup delegation/sudo → documented as accepted v0.3 risk. Applied to REQ-3-002, PLAN P1 Must-Haves + Task 1-2-02, ROADMAP P1 criteria.
- **G-2 (BINDING) — Disk cap via manager workdir-size sweep.** `AI_SANDBOX_MAX_WORKDIR_MB` (default 512MB); sweep snapshots+destroys over-cap sandboxes and logs an integrity signal; closes the unbounded-`dd` hole. Applied to PLAN Task 1-2-01 + 1-2-02(e) + P1 Must-Haves.
- **G-3 (BINDING) — Telemetry flood control WITHOUT silent drop.** Bounded queue; on overflow or >`AI_TELEMETRY_MAX_EVENTS_PER_TASK` (default 50k) → WS close 1008 + trace marked `INCOMPLETE_FLOODED` (Proctor signal). Silent drop-oldest forbidden (corrupts grading). Applied to PLAN Task 2-2-02 + P2 Must-Haves.
- **G-4 (BINDING) — Grader refuses incomplete/gapped traces.** `grade()` gates on `TraceStore.gaps()` + `INCOMPLETE_FLOODED` → returns `verdict=UNGRADABLE_TRACE_INCOMPLETE`; no credential from a gapped trace. Applied to PLAN Task 3-2-01 + P3 Must-Haves.
- **G-5 (BINDING) — No-auth abuse control at MVP scale.** Per-learner sandbox cap + global create-rate cap (429) + server-side `learner_id` allowlist (403) so the unauthenticated surface can't exhaust shared NPROC/disk. Ships WITH the milestone even though KYC is deferred. Applied to PLAN Task 1-3-01 + PROJECT A-110.
- **G-6 (BINDING) — Disclose partial enforcement in the P7 release note.** Item (e): which limits are kernel-enforced vs best-effort, and that full enforcement is deferred to the post-MVP containerd backend. Applied to PLAN P7 release-note honesty block.

## Scope Cuts (accepted — preserve the end-to-end credential pipeline)

- **CUT-1 (G-7) — Real server STT/TTS (`OpenAIAudioProvider`) deferred to v0.4.** Voice is mock-first (D-030); the `/audio/*` real path can never run in CI and was the least-verifiable surface. v0.3 proves the full defense *dialogue* + integrity-signal pipeline over mock + browser-native fallback; the `VoiceProvider` protocol is the future drop-in seam. Applied to PLAN Phase 5 Goal + Task 5-1-01/5-4-01 + P5 Must-Haves.
- **CUT-2 (G-8) — Interactive xterm.js shell relay deferred to v0.4.** The credential pipeline needs *process events* (Run/Test + file edits), not a live keystroke-level shell — the most fragile real-time piece, unverifiable without a real terminal. The build panel becomes Run/Test buttons + read-only exec output render; `@xterm/*` is NOT a v0.3 dependency. Applied to PLAN Env-facts, P6 Goal, Task 6-2-01/6-2-02/6-3-01, P6 Must-Haves, MVP/UX sections.
- Variants (Phase 4) and the Examiner agent dialogue **KEPT** — both are on the credential critical path (anti-collusion + the defense dialogue).

## Advisory (applied)

- **a-1** Startup reaper: on lifespan boot, scan `AI_SANDBOX_DIR`, reap workdirs whose recorded pid is dead, log a warning. → Task 1-2-01.
- **a-2** `RLIMIT_FSIZE` (~50MB) as a cheap partial single-file disk guard in the spawner's `preexec_fn`. → Task 1-2-01 + 1-2-02(c).
- **a-3** SQLite `PRAGMA journal_mode=WAL` + `synchronous=NORMAL` at engine creation (avoids `database is locked` under concurrent ingest + grader reads). → Task 2-1-02.
- **a-4** Grading prompt note: treat high edit/command churn with no test-progress as a process-quality negative (softens digest-gaming naivety). → Task 3-2-01.
- **a-5** Variant fairness envelope: two variants of one template must compute digests within the template's expected feature envelope ("same bar" is testable). → Task 4-2-01 Verify.

## Outcome

**GO** — all six binding decisions and both scope cuts applied to PLAN.md / REQUIREMENTS.md / ROADMAP.md / PROJECT.md before Phase 1 execution. No axis requires escalation (all resolvable at confidence ≥ 0.85). The milestone no longer claims resource enforcement it cannot deliver, and the no-auth abuse vector is closed at MVP scale.

---
# Nextcraft v0.4 — GRILL.md (Adversarial Review Verdict)

**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-CHANGES · **Confidence:** 0.83

## Summary

The distribution milestone is small, founder-directed (D-016, confidence 0.99), and additive (zero changes to the running credential pipeline). The plan's central risk: **Node SEA was probe-verified as a flag, not as a working build** — the v0.3 lesson (A-101: probe the mechanism, not the existence) applies. Second gap: a binary whose `--version` lies (stale package.json) would poison the "ongoing binaries" contract. Third: sed-based JSON parsing in install.sh is a fragility + integrity risk. Fourth: "ongoing binaries" has no enforcement mechanism beyond prose. All four closed by binding decisions G-101..G-104 below. No scope cuts required — the milestone is already minimal.

## Per-Axis Findings

| Axis | Verdict | Rationale |
|------|---------|-----------|
| Business case | PASS | Founder directive explicit + recorded (D-016). Evidence of need: live Gitea probe shows latest release v0.2.8 with ZERO assets; bootstrap requires repo archaeology (scripts found only via package.json spelunking). |
| Scope | PASS | 5 REQs, 3 execution phases, one focused surface (apps/cli + scripts). Smallest milestone yet. macOS arm64 already cut (D-036, unverifiable here). |
| Feasibility | CONCERN (fixed) | SEA flag exists on node v24.15.0, but no end-to-end SEA binary was built during RESEARCH. postject availability assumed (`npx postject` — needs npm registry reachability, unproven). Zipapp fallback requires python3 on target — an honest-degradation ladder, not a silent downgrade. → G-101. |
| Honest versioning | CONCERN (fixed) | `--version` from package.json would print a stale hardcoded version inside a per-release binary — breaks upgrade detection + the one-liner's re-run-to-upgrade promise. → G-102. |
| Install integrity | CONCERN (fixed) | sed/grep JSON parsing is brittle; a parse failure must never fall through to installing an unverified artifact. Exact asset-name matching + hard-degrade to source instructions. → G-103. |
| Sequencing | PASS | P1 CLI (source-runnable) → P2 binary+pipeline → P3 docs+E2E matches dependency order; each phase ships independently. |
| Cost/quota | PASS | Zero new paid infra; binaries built on-box; Gitea releases free. Dev-only esbuild dep. |
| Risks | CONCERN (fixed) | Top 3: SEA end-to-end (→ G-101 live probe FIRST in P2), npm registry reachability for esbuild (→ proven by P1's pnpm install must-have), Gitea asset-upload token scope (→ live-proven at the v0.3.2 ship itself). |
| Adoption/operability | PASS | Consumer = founder + future pilots; one command replaces README archaeology. Rollback trivial (rm ~/.local/bin/nextcraft). No server changes. |

## Binding Decisions (applied to PLAN.md)

- **G-101 (BINDING) — SEA live-build probe is the FIRST P2 action.** Task 2-1-01 builds a real binary before anything depends on it; the build script encodes the fallback ladder explicitly (SEA → zipapp with "requires python3" honesty). If SEA fails on this box, zipapp becomes primary with the docs stating the requirement — no silent claim of node-less operation.
- **G-102 (BINDING) — Version stamping at build time.** `build-binary` accepts the shipping tag and stamps it into the bundle (`NEXTCRAFT_VERSION` replace); `--version` prints it; install E2E asserts the installed binary reports the tag it was downloaded from. A binary may never report a version it was not built as.
- **G-103 (BINDING) — Install-script integrity hard-degrade.** install.sh matches assets by EXACT name (`nextcraft-linux-x64`, `nextcraft-linux-x64.sha256`); any parse/lookup/download failure degrades to source-bootstrap instructions (exit 0) — never installs unverified or name-approximate artifacts. Checksum mismatch = hard stop, exit 1, explicit do-not-run message. dash-safe POSIX sh, no jq.
- **G-104 (BINDING) — Ongoing-binaries enforcement.** Every ship from v0.3.2 onward MUST run `scripts/release-assets.sh <tag>` after tag+merge (best-effort, non-blocking, `release_pending` escalation on failure — but attempted + logged every release). The final-phase audit gate includes "milestone release carries both assets" as a check. This makes the founder's "ongoing binaries" directive a pipeline property, not prose.

## Escalations

None. All four concerns resolved at confidence ≥ 0.85. No axis requires founder escalation (directive already explicit).

## Outcome

**GO** — G-101..G-104 applied to PLAN.md before Phase 1 execution. The milestone claims only what its probes prove, and the ongoing-binaries contract has an enforcement mechanism.