nextcraft/.ciagent/PLAN.md

Nextcraft v0.4 — PLAN.md

Overview

This plan covers execution phases 1–3 of milestone v0.4 (Distribution & Bootstrap CLI) plus the final phase (P4 review+ship). The milestone delivers the founder directive (D-016): a streamlined install for Nextcraft — a nextcraft bootstrap CLI shipped as a linux x64 binary, installed via a one-liner script, with binaries published on every ongoing release from v0.4 onward. Phases are strictly sequential (P1→P3); within each phase, Wave 1 tasks are parallelizable (no cross-file dependencies) and later waves depend on earlier ones.

Environment facts (probe-verified, apply throughout): Go MISSING, Rust MISSING, gcc 12.2 present, node v24.15.0 x64 linux (SEA-capable), python3 3.11.2, shasum 6.02, pnpm 12.3.4 via corepack, turborepo 2.3.3, tsx 4.23 in root devDeps path. Gitea API verified live at https://git.coreci.dev/api/v1 (latest release v0.2.8, zero assets — the gap this milestone closes). Existing orchestration: apps/ai-service/scripts/bootstrap.sh (idempotent venv+pip incl. the no-ensurepip get-pip path), apps/ai-service/scripts/dev.sh (secrets export → uvicorn :8420), apps/ai-service/.env.example (full AI_* template). Root scripts: ai:dev/ai:test/ai:bootstrap/ai:lint turbo passthroughs (D-022 pattern to mirror as cli:*). Secrets live only in gitignored .ciagent/.env.secrets (GITEA_TOKEN, OLLAMA_API_KEY, OLLAMA_BASE_URL) — never in code, commits, or logs; tests never call the cloud or the forge (mocks/fixtures only).

Milestone type: feature. Tags: phase 0 → v0.3.0, P1 → v0.3.1, P2 → v0.3.2, P3 → v0.3.3, final phase P4 → v0.3.4 = milestone release. GRILL binding decisions (this revision): G-101 — SEA live-build probe is the FIRST P2 action (mechanism, not flag, must be proven); fallback ladder encoded honestly (zipapp requires python3 on target). G-102 — binary --version stamped from the shipping tag at build time (never a stale package.json version); install E2E asserts the installed binary reports its release tag. G-103 — install.sh matches assets by exact name; any parse/download failure degrades to source-bootstrap instructions (exit 0), never installs unverified artifacts; checksum mismatch = hard stop exit 1. G-104 — every ship from v0.3.2 onward runs scripts/release-assets.sh <tag> (best-effort, logged, non-blocking); the P4 audit gate checks the milestone release carries both assets.

Phase	Name	Requirements	Waves	Personas
1	Bootstrap CLI core	REQ-4-001, REQ-4-002	3	cli-engineer, backend-engineer, security-auditor (W3 review)
2	Binary build + release pipeline	REQ-4-003, REQ-4-004	3	cli-engineer, backend-engineer, security-auditor
3	Install docs + fresh-clone E2E	REQ-4-005	2	cli-engineer, backend-engineer
4	Final review + ship	—	1	all reviewers

User-Facing Surface

1. One-liner install (README quickstart): curl -fsSL https://git.coreci.dev/coreci/nextcraft/raw/main/scripts/install.sh | bash — downloads the latest release's nextcraft-linux-x64 binary, verifies its sha256, installs to ~/.local/bin, prints a PATH hint if needed.
2. CLI commands: nextcraft doctor (prereq checks), nextcraft bootstrap (fresh clone → runnable stack), nextcraft verify (health check), nextcraft dev (dev server passthrough), plus --help/--version.
3. Release surface: every Gitea release from v0.3.2 onward carries nextcraft-linux-x64 + nextcraft-linux-x64.sha256 assets.

Happy Path

Before execution, the end-to-end scenario this milestone must make true:

1. A consumer on a linux x64 box runs the one-liner; nextcraft lands in ~/.local/bin.
2. They clone the repo (or the CLI detects the repo root), run nextcraft doctor — all prerequisites report ✓ with actionable messages for any gap.
3. nextcraft bootstrap — pnpm install, ai-service venv via the existing bootstrap.sh, .env created from .env.example, optional-key warnings (not blockers), mock providers keep the stack runnable keyless.
4. nextcraft verify — venv imports, ports, env presence, build readiness all ✓.
5. nextcraft dev — the dev stack runs; Ctrl+C stops it (passthrough semantics).
6. On every ship, the Gitea release shows the binary + checksum assets; re-running the one-liner upgrades to the latest binary.

UX Acceptance Criteria

• doctor output lists every prerequisite with ✓/✗ and a fix hint on every ✗; exit code 1 if any ✗, 0 otherwise.
• bootstrap is idempotent — running twice produces the same end state, second run fast (no reinstalls where avoidable).
• bootstrap never writes secrets, never blocks on missing optional keys — warns with the exact key names and where to set them.
• verify gives a single-glance green/red summary; every red item names the failing command it ran.
• Every command supports --help; unknown command/flag exits 2 with usage.
• The one-liner never hard-fails silently: any error path (no release, no binary asset, checksum mismatch, platform mismatch) prints a specific message + the source-bootstrap alternative.
• Checksum mismatch = hard stop + explicit "do not run this binary" message.
• PATH hint: if ~/.local/bin is not on PATH, the installer prints the exact export line to add.
• Binary runs standalone on a box with node NOT installed (SEA self-containment) — ./nextcraft-linux-x64 --version works.

---

Phase 1: Bootstrap CLI Core

Requirements: REQ-4-001, REQ-4-002 Goal: apps/cli package with doctor/bootstrap/verify/dev fully working from source (node dist + pnpm bin), unit-tested, wired into the monorepo (turbo + root scripts), composing — not duplicating — the existing scripts.

Wave 1: Package foundation (parallel)

Task 1-1-01: CLI package scaffold + entry + dispatch

• Persona: cli-engineer — REQ: REQ-4-001
• Files: apps/cli/package.json, apps/cli/tsconfig.json, apps/cli/src/index.ts, apps/cli/src/commands/help.ts (usage text), apps/cli/tests/dispatch.test.ts
• Action: pnpm workspace package @nextcraft/cli (private, "bin": {"nextcraft": "dist/index.js"}). Entry: parse argv (hand-rolled, no runtime deps), dispatch to commands, --help/-h, --version (from package.json version), unknown → exit 2 with usage. Exit-code contract: 0 ok / 1 failure / 2 usage. shebang #!/usr/bin/env node on the built entry (esbuild banner in P2; for P1 tsx runs in dev via package script "dev": "tsx src/index.ts").
• Verify: pnpm --filter @nextcraft/cli test green (dispatch: routes doctor/bootstrap/verify/dev; unknown exits 2; --help exits 0; --version prints package version); pnpm typecheck green.

Task 1-1-02: Checks library (pure logic)

• Persona: cli-engineer — REQ: REQ-4-001, REQ-4-002
• Files: apps/cli/src/checks/check-command.ts, apps/cli/src/checks/check-env.ts, apps/cli/src/lib/log.ts, apps/cli/tests/checks.test.ts
• Action: check-command: given a name + optional --version probe + a min-version parser, resolve binary on PATH (which), semver-ish compare (major.minor tolerant), return CheckResult {name, ok, found, version, hint}. check-env: diff .env.example template keys vs an existing .env (missing keys → warn-classified; required-vs-optional classification table from the template's own comments + a static required list of zero keys — all optional per A-210), return per-key results. log.ts: ok(msg), fail(msg, hint), warn(msg), info(msg) formatters with symbols and consistent alignment. Pure functions — no side effects at import; fs access injected as parameters for testability.
• Verify: unit tests green: version compare (>= boundaries), missing binary → ok:false + hint, env diff missing/new/extra keys, required-optional classification.

Task 1-1-03: Root + turbo wiring

• Persona: backend-engineer — REQ: REQ-4-002
• Files: root package.json (update), turbo.json (update), pnpm-workspace.yaml (verify apps/* already covered — no change expected)
• Action: Add cli:dev, cli:test, cli:build, cli:typecheck, cli:lint root scripts mirroring the ai:* passthrough pattern (D-022/D-037). Turbo tasks for the CLI package: build (dependsOn ^build, outputs dist/**), test, typecheck, lint (cache:false, outputs:[] for test — same shape as ai-service). No changes to existing ai:* tasks.
• Verify: pnpm cli:test + pnpm cli:typecheck green from repo root; pnpm build still green for web+ai-service (turbo graph unaffected); pnpm ai:test still green.

Wave 2: Commands (depends on Wave 1)

Task 1-2-01: doctor command

• Persona: cli-engineer — REQ: REQ-4-001
• Files: apps/cli/src/commands/doctor.ts, apps/cli/tests/doctor.test.ts
• Action: Checks (each with actionable hint): node ≥18 (process.version), pnpm ≥8 on PATH (pnpm --version), python3 ≥3.11 (python3 --version parse), git (git --version), corepack available-or-pnpm-present nuance folded into pnpm check, unshare binary on PATH (which unshare — sandbox fabric needs it; hint explains what breaks without it). Sequential execution with per-check timeout; summary line; exit 1 if any ✗. Runs from any cwd (no repo required — pure environment check).
• Verify: unit tests with injected spawn results: all-pass → exit 0 + summary; missing pnpm → ✗ + hint + exit 1; missing unshare → ✗ with sandbox-specific hint.

Task 1-2-02: bootstrap command

• Persona: cli-engineer — REQ: REQ-4-002
• Files: apps/cli/src/commands/bootstrap.ts, apps/cli/src/lib/spawn.ts, apps/cli/tests/bootstrap.test.ts
• Action: spawn.ts: run(cmd, args, {timeoutMs, cwd, env}) — promisified child_process.spawn, inherited stdio, timeout kill (SIGTERM→SIGKILL escalation), returns {code}; throws never (codes always returned). bootstrap.ts steps (each logged before/after): (1) locate repo root (walk up for pnpm-workspace.yaml; error with hint if not in a clone); (2) pnpm install at root; (3) delegate ai-service venv to apps/ai-service/scripts/bootstrap.sh via spawn with generous timeout (10 min) — zero pip/venv logic in the CLI (A-202); (4) copy .env.example → .env if absent (preserve existing; report created vs kept); (5) validate optional keys in .env vs template — warn-only (A-210); never touch .ciagent/.env.secrets; (6) print next-steps (nextcraft verify, nextcraft dev). Idempotent: every step safe to re-run.
• Verify: unit tests with stub spawn: step order, env copy semantics (absent → create, present → keep), timeout path returns failure code, secrets file never written; pnpm cli:test green.

Task 1-2-03: verify + dev commands

• Persona: cli-engineer — REQ: REQ-4-002
• Files: apps/cli/src/commands/verify.ts, apps/cli/src/commands/dev.ts, apps/cli/tests/verify.test.ts
• Action: verify.ts health checks (each runnable + reported): ai-service venv python imports (import ai_service via venv python), uvicorn present in venv, ports 3000/8420 free (net stat via node), .env exists with AI_PORT parseable, pnpm build dry readiness (turbo graph parses — run turbo build --dry=json cheap check or typecheck-only default; choose the cheap one). Summary + exit code. dev.ts: locate repo root, exec passthrough to apps/ai-service/scripts/dev.sh with inherited stdio and signals (Ctrl+C semantics), no timeout (long-running); document that web dev server runs via pnpm dev separately (dev.sh owns ai-service only).
• Verify: unit tests: verify aggregates check results → exit codes; dev spawns dev.sh with signal passthrough assertions (mock spawn).

Wave 3: Integration review (depends on Wave 2)

Task 1-3-01: CLI security + integration review pass

• Persona: security-auditor — REQ: REQ-4-001, REQ-4-002
• Files: apps/cli/src/lib/spawn.ts (review; patch if defect), apps/cli/src/commands/bootstrap.ts (review), apps/cli/tests/** (add regression if defect found)
• Action: STRIDE pass on the CLI surface: spawn injection (args never through shell string — array form only), timeout enforcement, secrets never logged, env template copy doesn't overwrite user edits, no shell=true anywhere, PATH resolution honest errors. Findings → P0 patches now with regression tests; P1+ noted for final-phase review.
• Verify: pnpm cli:test green incl. any added regressions; grep -rn "shell: *true" apps/cli/src returns nothing.

Must-Haves (Phase 1)

• pnpm --filter @nextcraft/cli test green; pnpm typecheck green; pnpm build green
• doctor: every prerequisite reported with ✓/✗ + actionable hint; exit 1 on any ✗; runs outside a repo clone
• bootstrap: composes scripts/bootstrap.sh (no pip/venv logic in CLI); idempotent; .env created from template only when absent; optional-key warnings, never blocks; never writes secrets
• verify: venv import + uvicorn + ports + env checks with single-glance summary and named failing commands
• dev: passthrough with signal inheritance (Ctrl+C stops the stack)
• Exit-code contract: 0/1/2; --help everywhere; unknown command → 2
• No runtime npm dependencies in apps/cli (dev deps only)
• Root cli:* scripts work from repo root; ai:* scripts unaffected

---

Phase 2: Binary Build + Release Pipeline

Requirements: REQ-4-003, REQ-4-004 Goal: nextcraft-linux-x64 SEA binary + sha256 sidecar built reproducibly from the CLI package; one-liner install.sh verified end-to-end against a real release; release-asset upload wired so every ship from now on carries binaries.

Wave 1: Binary build (parallel)

Task 2-1-01: SEA binary build script (G-101: live-build probe FIRST — mechanism must be proven before the pipeline depends on it)

• Persona: cli-engineer — REQ: REQ-4-004
• Files: apps/cli/scripts/build-binary.mjs, apps/cli/package.json (add build:binary script), apps/cli/.sea-config.json (or generated in-script)
• Action: First action of this task: build one real SEA binary end-to-end and run it (--version + doctor smoke) before writing the polished script.** Pipeline: esbuild bundle src/index.ts → dist/bundle.cjs (platform node, target node18, banner shebang, SEA config: {main: "dist/bundle.cjs", output: "dist/sea-prep.blob", disableExperimentalSEAWarning: true}) → node --experimental-sea-config → copy system node binary → inject blob (npx postject with sentinel NODE_SEA_BLOB_FUSE fuse, or dd fallback) → chmod +x → dist/nextcraft-linux-x64 → stamp version from the shipping tag argument (NEXTCRAFT_VERSION injected via esbuild define, G-102 — --version prints it; absent arg → dev stamp 0.0.0-dev) → shasum -a 256 → dist/nextcraft-linux-x64.sha256. Fallback (documented, scripted, honest): if SEA injection fails, python3 zipapp builds nextcraft-linux-x64.pyz (requires python3 on target — install.sh handles both asset shapes and the docs say so; NO silent claim of node-less operation, G-101).
• Verify: pnpm --filter @nextcraft/cli build:binary produces the binary; ./dist/nextcraft-linux-x64 --version runs with node absent from PATH (test via env -i /bin/sh -c 'PATH=/usr/bin:/bin ...' sandbox or by temporarily stripping PATH in a subprocess test); sha256 file matches shasum -c.

Task 2-1-02: Release-asset upload helper

• Persona: cli-engineer — REQ: REQ-4-004
• Files: scripts/release-assets.sh, apps/cli/tests/release-assets.test.ts (fixture-level)
• Action: Given a tag: build binary (Task 2-1-01), resolve GITEA_TOKEN from .env/.env.secrets/.env.* via the secrets loader only (never shell env — v1.8 root cause), create/locate the Gitea release via API, upload both assets (POST /api/v1/repos/{owner}/{repo}/releases/{id}/assets?name=... multipart). Bounded retry (3) per config.ship.max_release_retries; token never echoed; failure = non-blocking escalation message (release_pending semantics) — tag+merge already complete the ship.
• Verify: fixture test: token resolution order (.env.secrets wins over .env; shell env NEVER consulted — assert with a poisoned env var fixture); dry-run mode prints the exact curl-multipart it would send (no net in tests).

Task 2-1-03: install.sh one-liner

• Persona: cli-engineer — REQ: REQ-4-003
• Files: scripts/install.sh, apps/cli/tests/install-script.test.ts
• Action: POSIX sh (no bashisms — dash-safe): set -eu; platform check (uname linux + x86_64; else print source-bootstrap path + exit 0 — a graceful no-op, not an error); resolve latest release via Gitea API (curl -fsSL .../releases/latest, parse tag_name + asset browser_download_urls with sed/grep — no jq dependency); match assets by EXACT name (nextcraft-linux-x64, nextcraft-linux-x64.sha256 — any parse/lookup miss = degrade to source-bootstrap instructions, exit 0, G-103 — never a name-approximate install); handle the zipapp asset shape (nextcraft-linux-x64.pyz + sidecar) when the binary is absent, printing the python3 requirement honestly; download both assets to mktemp -d (trap cleanup EXIT); verify sha256 before anything else (shasum -a 256 -c or sha256sum); on mismatch → hard stop, explicit "do not run" message, exit 1; install to ~/.local/bin (mkdir -p; --dest override); PATH hint when missing (print exact export line); print the binary's own --version output (G-102: must equal the resolved release tag — mismatch = install-time integrity stop) + nextcraft doctor next-step. No-binary-asset path: print the git-clone + scripts/bootstrap.sh instructions + exit 0. Zero secrets required (public release assets).
• Verify: unit tests over the script's pure helpers extracted where feasible; live E2E in Task 2-3-01. sh -n scripts/install.sh syntax-clean; dash scripts/install.sh --help safe if dash present.

Wave 2: Ship-flow integration (depends on Wave 1)

Task 2-2-01: Wire binaries into every ship (G-104: enforcement, not prose)

• Persona: backend-engineer — REQ: REQ-4-004
• Files: .ciagent/config.json (no schema change needed — release section already configured), this repo's ship procedure notes (update .ciagent/ARCHITECTURE.md Build Order note if needed), scripts/release-assets.sh (finalize from 2-1-02)
• Action: Establish the ship-time contract going forward: after every phase ship (tag + merge complete = ship gate per config.ship), run scripts/release-assets.sh <tag> to attach binary + checksum to the freshly created release. G-104: this run is MANDATORY-ATTEMPTED on every release from v0.3.2 onward — best-effort/non-blocking like release creation (release_pending escalation on exhaustion), logged in the ship commit, and the P4 final audit gate includes "milestone release carries both assets" as an explicit check. This makes "ongoing binaries" a property of the pipeline, not a one-off.
• Verify: The P2 ship itself executes the step against tag v0.3.2 (live validation — see Ship).

Wave 3: End-to-end validation (depends on Wave 2)

Task 2-3-01: Install E2E against the live release

• Persona: security-auditor — REQ: REQ-4-003
• Files: apps/cli/tests/install-e2e.test.ts (marked slow/e2e), apps/cli/README.md (install internals section)
• Action: Live E2E after the v0.3.2 release exists (run post-ship, documented as the verify gate for this phase's asset path): fresh HOME tmpdir → run install.sh → assert binary at $HOME/.local/bin/nextcraft, --version output equals the release tag (G-102 integrity assertion), checksum verified path taken (tamper test: flip a byte in a local fixture download → script refuses + exits 1). Record the transcript in the phase verify commit. If the live release isn't reachable at verify time, run the full local equivalent (serve assets from a fixture dir via python3 -m http.server + FORGE_BASE override) and mark live re-check as a P1 follow-up.
• Verify: E2E green locally (fixture server path mandatory in tests — no test depends on the live forge); tamper-rejection proven; transcript recorded.

Must-Haves (Phase 2)

• G-101: a real SEA binary built + smoke-run BEFORE the pipeline depends on it; if SEA fails, zipapp is primary and docs state the python3 requirement
• G-102: binary --version reports the shipping tag (stamped at build); install E2E asserts version == release tag
• pnpm --filter @nextcraft/cli build:binary produces nextcraft-linux-x64 + .sha256; binary runs without node on PATH (--version, doctor smoke)
• G-103: sh -n scripts/install.sh clean; dash-safe; exact-name asset matching; platform mismatch → graceful source-bootstrap path (exit 0)
• Checksum verified before install; tamper → hard stop with explicit warning (E2E-proven)
• install.sh resolves latest release + assets from the Gitea API with zero secrets and no jq
• release-assets.sh resolves GITEA_TOKEN from .env* files only (never shell env — tested with poisoned env)
• G-104: v0.3.2 release carries both assets (live validation at ship); upload failure is non-blocking escalation, attempted + logged every release
• pnpm build, pnpm typecheck, pnpm cli:test all green

---

Phase 3: Install Docs + Fresh-Clone E2E

Requirements: REQ-4-005 Goal: README quickstart + CLI reference matching the tested reality exactly, plus a fresh-clone E2E test proving the happy path end-to-end.

Wave 1: Fresh-clone E2E (drives doc accuracy)

Task 3-1-01: Fresh-clone bootstrap E2E

• Persona: cli-engineer — REQ: REQ-4-005
• Files: apps/cli/tests/fresh-clone-e2e.test.ts (slow/e2e-marked)
• Action: In a mktemp -d sandbox: git clone the repo locally (file:// clone of HEAD — no network), run pnpm --filter @nextcraft/cli dev -- doctor (or the built binary from P2) → then bootstrap → then verify, asserting each step's exit codes and key output markers. Skips gracefully when network-dependent steps are unavailable (CI marker). Documents the exact happy path the README will state.
• Verify: E2E green locally (clone of the working tree); output transcript matches README claims (cross-checked in 3-2-01).

Wave 2: Documentation (depends on Wave 1 transcript)

Task 3-2-01: README quickstart + CLI reference

• Persona: backend-engineer — REQ: REQ-4-005
• Files: root README.md (update quickstart section), apps/cli/README.md (CLI reference)
• Action: Root README quickstart: the one-liner (exact tested URL), then doctor → bootstrap → verify → dev sequence with expected outputs; source-bootstrap alternative documented (clone + scripts). apps/cli README: every command, flags, exit codes, the env-template copy semantics, optional-key warning semantics, secrets policy (never generated/committed; .ciagent/.env.secrets location), binary install internals, troubleshooting table keyed to actual failure modes observed in E2E.
• Verify: Every command line in both READMEs is copy-paste runnable — verified against the 3-1-01 transcript; doc drift check: no references to commands/flags that don't exist in --help output.

Must-Haves (Phase 3)

• Fresh-clone E2E green: doctor → bootstrap → verify sequence from a clean clone
• README quickstart matches the E2E transcript exactly (no aspirational docs)
• CLI reference covers all 4 commands + --help/--version + exit codes
• pnpm build, pnpm typecheck, pnpm test (all suites) green

---

Phase 4: Final Review + Ship (milestone release v0.3.4)

1. Branch gate → phase/04-final-review-ship.
2. Multi-persona review across the milestone (correctness, testing, security, performance, maintainability, adversarial) — P0 auto-fixed, P1+ fixed in this phase.
3. Audit: reconstruction test (.ciagent files ↔ git log), file discipline, branch hygiene, commit discipline, P0-review flags resolved, G-104 gate: milestone release v0.3.4 carries nextcraft-linux-x64 + .sha256 assets.
4. Milestone ship: merge phase/04 → milestone/v0.4-distribution; merge milestone → main; tag v0.3.4 (= milestone release); attach binary + checksum assets (the ongoing-binaries contract); release notes with full milestone summary (all phases, all REQ-4-001..005, the "ongoing binaries from now on" statement, v0.5 deferral list per D-016); delete all milestone/phase branches.
5. Complete: REQUIREMENTS.md REQ-4-001..005 → complete; ROADMAP.md v0.4 → complete; checkpoint cleared.

Must-Haves (Milestone)

• One-liner installs a working binary from the live Gitea release (E2E-proven, tamper-tested)
• Fresh clone → doctor → bootstrap → verify → dev: the full happy path green from a clean environment
• Every release from v0.3.2 onward carries nextcraft-linux-x64 + .sha256 assets
• Zero runtime npm deps in the CLI; secrets only ever from .env* files; never in code/logs/commits
• All suites green: pnpm build, pnpm typecheck, pnpm ai:test, pnpm cli:test