orca/.ciagent/ROADMAP.md

# Roadmap: Orca

## Milestone v0.1: Foundation — **COMPLETE**

- [x] Phase 0: Project Initialization & Specification
- [x] Phase 1: Core CLI Skeleton & Command Parsing
- [x] Phase 2: Basic Node Management (Join/Leave)
- [x] Phase 3: Simple Task Execution Engine
- [x] Phase 4: Local State Persistence
- [x] Phase 5: Basic Health Checking
- [x] Phase 6: CoreCI Full Release Flow

**Tagged `v0.2.0`** (next-minor per feature-milestone promotion rule).

## Deferred to v0.2 (out of scope for v0.1)

- Multi-node scheduling (D-004 decision: single-node only in v0.1)
- mTLS for inter-node communication (REQ-011, REQ-023)
- `gosec` + `govulncheck` in CI pipeline (REQ-014)
- `iter.Seq` streaming job lists (REQ-022)
- Frontend / devops personas (no web UI; CoreCI handles release)

## Milestone v0.2: Networking, Observability, Security Hardening — **COMPLETE (merged to main via v0.3)**

Scope: extend v0.1 with secure cross-node transport, multi-node scheduling,
richer CI security scanning, and streaming I/O.

- [x] Phase 8: mTLS handshake + internal CA with CSR join (Wave 1) — shipped v0.2.1
- [x] Phase 9: Multi-node scheduling & job dispatch (Wave 1) — shipped v0.2.2
- [x] Phase 10: `gosec` + `govulncheck` + gitleaks in CI (Wave 2) — shipped v0.2.3
- [x] Phase 11: `iter.Seq` streaming job/node lists (Wave 2) — **completed in v0.3 P01** (shipped v0.3.1)

**Milestone tag**: `v0.4.0` (shipped — v0.2 work merged to main via v0.3 milestone).

Per-phase tags: `v0.2.1` (P01), `v0.2.2` (P02), `v0.2.3` (P03) — all shipped.

## Milestone v0.3: Scheduling & Streaming Completion — **COMPLETE**

Scope: complete the two work items deferred from v0.2 that were not
already shipped in P08-P10. A re-init SPECIFY codebase audit confirmed
that REQ-014/027/028/029/031/037/039/040 all shipped in P08-P10 despite
stale REQUIREMENTS.md marking them Pending. The remaining work is lean:

- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — shipped v0.3.0
- [x] Phase 1: `iter.Seq` streaming for `--watch` flags (REQ-022, REQ-030) — shipped v0.3.1
- [x] Phase 2: `orca doctor` network + db full implementation (REQ-032 completion) — shipped v0.3.2
- [x] Phase 3: Final review + ship + audit (milestone release) — shipped v0.3.3

**Milestone tag**: `v0.4.0` (next-minor per feature-milestone promotion rule).

Per-phase tags: `v0.3.0` (P0), `v0.3.1` (P01), `v0.3.2` (P02), `v0.3.3` (P03 final = milestone release).
Per `.ciagent/RELEASE_POLICY.md`, every phase tag produces a Gitea release.

### Per-phase REQ coverage

- **P01 — `iter.Seq` streaming for `--watch` flags**
  - REQ-022 (`iter.Seq` for streaming job lists, Go 1.25+)
  - REQ-030 (`--watch` output format mode: table default vs streaming JSON per event)
  - Applies to both `orca job list --watch` and `orca node list --watch`
    (D-024, per ARCHITECTURE.md CLI layer + D-017)

- **P02 — `orca doctor` network + db full implementation**
  - REQ-032 (completion: network reachability via mTLS `/healthz` probe,
    db integrity via `PRAGMA integrity_check` + migration version)
  - Replaces `NetworkStub` and `DBStub` from v0.2 P01

### v0.3 is a completion milestone, not a direction change

The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.3 closes out the v0.2 deferrals and merges
the accumulated v0.2 work to main.

## Milestone v0.5: Distribution — **COMPLETE**

Scope: make Orca installable, distributable, and containerized. The
engine functionality from v0.1–v0.3 is unchanged; this milestone is
purely about delivery surface.

- [x] Phase 0: Pre-execution (specify → clarify → research → plan) — shipped `v0.4.1` (+ repo public)
- [x] Phase 1: Namespace unification (`ORCA_HOME` + `--system`) (REQ-041, REQ-042) — shipped `v0.4.2`
- [x] Phase 2: `install.sh` + in-place update + README quickstart (REQ-043, REQ-044) — shipped `v0.4.3`
- [x] Phase 3: Docker release (Dockerfile + Gitea container registry) (REQ-046) — shipped `v0.4.4`
- [x] Phase 4: Final review + ship + audit (milestone release) — shipped `v0.4.5`

**Operational prerequisite (P0 ship)**: repo + org visibility flipped to
public (REQ-045) — unauth releases API + asset download + docker pull all
verified HTTP 200.

**Milestone tag**: `v0.4.5` (final phase patch = milestone release per
feature-milestone promotion rule). Per-phase tags: `v0.4.1`…`v0.4.5`.

## Milestone v0.6: Node Bootstrap & Proxmox

## Milestone v0.6: Node Bootstrap & Proxmox — **COMPLETE**

Scope: make `orca init` produce a fully working single-node cluster
(CA + server cert + DB + localhost node registered with auto-detected
OS), and add Proxmox 8 & 9 as a first-class remote node type joined
over SSH with least-privilege role delegation.

- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag `v0.5.0`
- [x] Phase 1: `orca init` full bootstrap + localhost node + schema 0006 (REQ-047, REQ-048, REQ-049) — tag `v0.5.1`
- [x] Phase 2: Proxmox SSH join + OrcaOperator role + sudoers allowlist (REQ-050, REQ-051) — tag `v0.5.2`
- [x] Phase 3: `doctor os` + `doctor proxmox` SSH probe + audit logging (REQ-052) — tag `v0.5.3`
- [x] Phase 4: Final review + ship + audit (milestone release) — tag `v0.5.4`

**Milestone type**: feature (P1/P2/P3 ship `feat` phases).
**Milestone tag**: `v0.5.4` (final phase patch = milestone release per
feature-milestone promotion rule). Per-phase tags: `v0.5.0`…`v0.5.4`.

Tags run on the previous minor's patch line (v0.5.x) per
branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.6-node-bootstrap-proxmox`); no separate minor
tag is created.

## Milestone v0.7: Hardening & Completion — **COMPLETE**

Scope: NFR milestone closing gaps surfaced by the v0.7 IDEATE stage —
an unreachable command tree, a missing config file layer, low test
coverage in core packages, and the long-deferred pprof endpoint.

- [x] Phase 0: Pre-execution (specify → clarify → research → ideate → plan) — tag `v0.6.0` (shipped)
- [x] Phase 1: Register `orca cert` command tree + cert_repo tests (REQ-053) — tag `v0.6.1` (shipped)
- [x] Phase 2: HCL config file parsing — `internal/config` package (REQ-054) — tag `v0.6.2` (shipped)
- [x] Phase 3: Test coverage uplift — engine/transport/proxmox/audit ≥ 50% (REQ-055) — tag `v0.6.3` (shipped)
- [x] Phase 4: `--pprof` opt-in on `orca daemon` (REQ-056) — tag `v0.6.4` (shipped)
- [x] Phase 5: Final review + ship + audit (milestone release) — tag `v0.6.5` (shipped)

**Milestone type**: NFR (all phases are fix/test/chore; no `feat` phases).
**Milestone tag**: `v0.6.5` (final phase patch = milestone release per
NFR-milestone progressive-patch rule). Per-phase tags: `v0.6.0`…`v0.6.5`.
Tags run on the previous minor's patch line (v0.6.x) per
branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.7-hardening-completion`); no separate minor tag.

## Milestone v0.8: Coverage & Trust Hardening — **COMPLETE**

Scope: continue the v0.7 hardening theme. v0.7 P03's ≥ 50% floor left
six packages still under 50% (engine 8.3%, proxmox 5.1%, cli 27.6%,
transport 26.3%, store 46.7%, jobspec 47.6%) and three packages with
no tests at all (`internal/audit`, `internal/certpaths`, `cmd/orca`).
v0.8 also closes the two SSH-trust "future enhancement" hooks deferred
in v0.6 (D-035 `--host-key-fingerprint` pre-pin, RESEARCH_v0.6 §80
`orca node key-reset`) and adds a requirements-hygiene gate to prevent
the stale-REQ-status drift seen after v0.7 ship.

- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag `v0.7.0` (shipped)
- [x] Phase 1: Test coverage uplift round 2 — 6 packages to ≥ 70%, 3 zero-test packages to first tests (REQ-057) — tag `v0.7.1` (shipped)
- [x] Phase 2: SSH trust hardening — `--host-key-fingerprint` pre-pin + `orca node key-reset` + TOFU bugfix + `HostKeyFingerprint` population (REQ-058, REQ-059) — tag `v0.7.2` (shipped)
- [x] Phase 3: Requirements-hygiene gate — `make verify-reqs` + verify assertion (REQ-060) — tag `v0.7.3` (shipped)
- [x] Phase 4: Final review + ship + audit (milestone release) — tag `v0.7.4` (shipped)

**Milestone type**: NFR (P01 test, P02 chore on trust surface per
D-043, P03 chore, P04 docs/review). Final phase patch IS the milestone
release per NFR-milestone progressive-patch rule. Per-phase tags:
`v0.7.0`…`v0.7.4`. Tags run on the previous minor's patch line (v0.7.x)
per branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.8-coverage-trust-hardening`); no separate minor
tag.

### Per-phase REQ coverage

- **P01 — Coverage uplift round 2**
  - REQ-057 (raise `internal/engine`, `internal/proxmox`,
    `internal/cli`, `internal/transport`, `internal/store`,
    `internal/jobspec` to ≥ 70%; add first tests for `internal/audit`,
    `internal/certpaths`, `cmd/orca`)

- **P02 — SSH trust hardening**
  - REQ-058 (`--host-key-fingerprint <sha256>` pre-pin flag on
    `orca node join --type proxmox`; fail fast on mismatch; supersedes
    TOFU for pre-pinned deployments)
  - REQ-059 (`orca node key-reset <node>` clears persisted SSH host
    key so next `doctor proxmox`/dispatch re-pins via TOFU or
    `--host-key-fingerprint`)

- **P03 — Requirements-hygiene gate**
  - REQ-060 (`make verify-reqs` target + verify-stage assertion:
    every REQ `Complete` in ROADMAP.md has matching `Complete` row in
    REQUIREMENTS.md; enforced in CI `validate` pipeline)

### v0.8 is a continuation milestone, not a direction change

The vision ("minimalist, offline-first, CLI-first orchestration
engine") is unchanged. v0.8 closes the coverage debt left by v0.7's
50% floor and the trust-surface gaps explicitly deferred in v0.6.

## Milestone v0.9: Re-architecture Foundation & Workloads — **COMPLETE**

**Scope**: This milestone SUPERSPEDES the shipped v0.1–v0.8 architecture per
the adopted PRD (`.ciagent/PRD_v0.9.md`). The re-architecture is justified on
six grounds recorded in the PROJECT.md Supersession Table: (1) the v0.8 daemon
model is operationally failing, (2) step-ca is externally mandated, (3)
multi-tenancy is a hard product requirement, (4) WASM is a hard workload
requirement, (5) SSH-push is the only viable deployment target, (6) vision
correction. The 16 load-bearing rules (R-001…R-016) are invariants. The
ci-griller reviewed the re-architecture adversarially; the user overrode the
Re-architecture Justification REPLAN with the six-part evidence basis; the
19 binding conditions (C-01..C-19) and 10 phase challenges (PC-01..PC-10)
from `GRILL_v0.9.md` are adopted as execution gates. 30 net-new requirements
(REQ-061..REQ-090) derive from `IDEATION_v0.9.md`.

**Milestone type**: feature (P01..P10 ship `feat` phases; P00/P0X are
chore/docs).

- [ ] Phase 0: Pre-execution (specify → clarify → research → ideate → plan → grill) — tag `v0.8.0` (shipped; this is the phase you are reading)
- [x] Phase P00: Deprecation sweep + bash tooling gate + render contract + doc banners (REQ-068,072,088,089,090; gates C-03,C-05,C-06,C-15..C-18) — tag `v0.8.1` ✓
- [x] Phase P0a1: Multi-namespace path resolver + config demotion + known_hosts flock (REQ-063,069,070,071; gate C-07) — tag `v0.8.2` ✓
- [x] Phase P0a2: Namespace CRUD + inheritance engine (REQ-082) — tag `v0.8.3` ✓
- [x] Phase P0b: Markdown jobspec parser + dispatcher + fuzz (REQ-064,067) — tag `v0.8.4` ✓
- [x] Phase P0c: Job/Service/DaemonSet schemas + emitter interface (REQ-074) — tag `v0.8.5` ✓
- [x] Phase P01: SSH-push transport (REQ-073) — tag `v0.8.6` ✓
- [x] Phase P02: Service block + Traefik emitter (REQ-077; gate C-10) — tag `v0.8.7` ✓
- [x] Phase P03/P04/P08: Update stanza + lifecycle hooks + socket plumbing (combined) — tag `v0.8.8` ✓
- [x] Phase P05: CLI-side scheduler + CEL constraints (REQ-083) — tag `v0.8.9` ✓
- [x] Phase P06: Task groups (multi-process services) — tag `v0.8.10` ✓
- [x] Phase P07a/b/c: Runtime abstraction — 5 backends (REQ-078; gate C-01) — tag `v0.8.11` ✓
- [x] Phase P09: Syncthing storage replication (REQ-081; gates C-02,C-14) — tag `v0.8.12` ✓
- [x] Phase P10: Lead rules + step-ca (REQ-076) — tag `v0.8.13` ✓
- [x] Phase P0X: Ship + audit (REQ-062,068) — tag `v0.8.14` ✓

**Milestone tag**: `v0.8.15` (final phase patch = milestone release per
feature-milestone progressive-patch rule). Per-phase tags: `v0.8.1`…`v0.8.14`.
P03/P04/P08 were combined into one phase; P07a/b/c were combined into one
phase. Actual execution: 14 tagged phases. Tags run on the previous minor's
patch line (v0.8.x) per branch-strategy.md. The milestone branch label uses
the milestone number (`milestone/v0.9-rearchitecture`); no separate minor tag.

### Per-phase REQ coverage (v0.9)

- **P00** — Deprecation/migration/test-infra/persona/docs foundation (REQ-072, REQ-085, REQ-088, REQ-089, REQ-090)
- **P0a1** — Path resolver + config demotion + known_hosts flock (REQ-063, REQ-069, REQ-070, REQ-071)
- **P0a2** — Namespace inheritance resolver (REQ-082)
- **P0b** — Markdown parser + adapter + fuzz (REQ-064, REQ-067)
- **P0c** — Schemas + emitter interface (REQ-074)
- **P01** — SSH-push transport (REQ-073)
- **P02** — Service + Traefik emitter (REQ-077)
- **P05** — CLI-side scheduler (REQ-083)
- **P07a/b/c** — Runtime abstraction (REQ-078) + step-ca integration (REQ-076)
- **P09** — Syncthing replication (REQ-081)
- **P0X** — Coverage gate (REQ-062) + deprecation warnings (REQ-068)

### v0.9 is a DIRECTION CHANGE — first in the project's history

Every prior milestone (v0.1–v0.8) explicitly said "the vision is unchanged;
this milestone is not a direction change." v0.9 is the first milestone that
reverses the vision's anti-patterns (daemon-on-every-node, internal CA,
HCL-canonical, single-namespace, no-container-runtime, no-SPIFFE). The
reversals are justified by the six-part evidence basis recorded in the
PROJECT.md Supersession Table.

## Milestone v0.10: Docs & Install Hardening — **COMPLETE**

**Scope**: close the documentation gap left by the v0.9 re-architecture
and fix the release/install pipeline bug that caused `install.sh` to
resolve to v0.4.5 instead of the latest release. The v0.9
re-architecture shipped a complete CLI surface (markdown jobspec,
`orca ns`, `orca node capacity`, CLI-side scheduler, emitters, Traefik
ingress) but no operator-facing reference documentation. This milestone
ships that documentation plus a worked full-stack example with ingress
configured, and hardens the release pipeline so every Gitea release
carries a Linux binary asset.

**Milestone type**: feature (P1 ships `fix` phases; P2/P3/P4 ship `docs`
phases; at least one non-docs phase makes this a feature milestone per
the versioning logic).

- [x] Phase 0: Pre-execution (specify → clarify → research → ideate → plan → grill) — tag `v0.9.0`
- [x] Phase P1: release.sh + install.sh fix (REQ-097, REQ-098) — tag `v0.9.1`
- [x] Phase P2: docs/cli.md + docs/jobspec.md + docs/ingress.md (REQ-091, REQ-092, REQ-093) — tag `v0.9.2`
- [x] Phase P3: examples/full-stack/ (REQ-094) — tag `v0.9.3`
- [x] Phase P4: README.md + docs/namespace.md refresh (REQ-095, REQ-096) — tag `v0.9.4`
- [x] Phase P5: Final review + ship + audit (milestone release) — tag `v0.9.5` = v0.10.0 milestone release

**Milestone tag**: `v0.9.5` (final phase patch = milestone release per
feature-milestone progressive-patch rule). Per-phase tags: `v0.9.0`…`v0.9.5`.
Tags run on the previous minor's patch line (v0.9.x) per
branch-strategy.md. The milestone branch label uses the milestone
number (`milestone/v0.10-docs-cli-examples`); no separate minor tag.

### Per-phase REQ coverage (v0.10 docs milestone)

- **P1** — release.sh cross-build + asset verification (REQ-097); install.sh fallback walk (REQ-098)
- **P2** — CLI reference (REQ-091); jobspec reference (REQ-092); ingress guide (REQ-093)
- **P3** — full-stack examples (REQ-094)
- **P4** — README refresh (REQ-095); namespace.md v0.9 layout (REQ-096)

### Root cause of the v0.4.5 install (documented in RESEARCH_v0.10.md)

The v0.8.x releases (v0.8.0–v0.8.15) shipped with zero binary assets
attached to their Gitea releases. `install.sh` resolves "latest" →
v0.8.15, looks for `orca-v0.8.15-linux-amd64.tar.gz`, finds nothing, and
errors out. The v0.4.5 install came from an earlier run or a pinned
`--version`. The fix is forward: release.sh cross-builds amd64 and
verifies the asset post-create; install.sh walks backward through
releases if the latest lacks the asset.

## Milestone v0.11: Production Hardening — **COMPLETE**

**Scope**: ship a cluster that operators can run. Builds on the v0.9
re-architecture foundation with the production-grade subsystems:
secrets, transactions, ACL/SPIFFE, backup/restore, drain, recovery, and
the v0.8→v1.0 migration. **Phase 0 adopts 4 new load-bearing rules
(R-017…R-020) and 23 new decisions (D-215…D-237) from 5 research docs
covering ingress hardening, drift detection, platform-engineer
positioning, strategic framing, and the systemd Path unit
implementation.** No new phases added; scope is folded into existing
phases per operator decisions Q2=C (add 5 CLI commands), Q3=A (fold
ingress into P15.5).

**Milestone type**: feature (multiple `feat` phases).

- [x] Phase 0: Pre-execution (specify → clarify → research → plan → grill) — tag `v0.10.0`
- [x] Phase P00: CLI cache layer (REQ-062 cache floor; R-008) — tag `v0.10.1`
- [x] Phase P01: Metrics endpoint (hand-rolled text exposition) — tag `v0.10.2`
- [x] Phase P01.5: SPIFFE SVID minting spike (REQ-076; **gate C-08** — if spike fails, fall back to mTLS identity) — tag `v0.10.3`
- [x] Phase P02: ACL (SPIFFE + token identities) — tag `v0.10.4`
- [x] Phase P03: Secrets subsystem (REQ-080; **gate C-19** threat model) — tag `v0.10.5`
- [x] Phase P04: Backup/restore (tar + signed) — tag `v0.10.6`
- [x] Phase P05: Drain + daemon drain-and-stop (REQ-061) + **`orca job migrate` (REQ-116)** — tag `v0.10.7`
- [x] Phase P06: Alloc history (CLI-side SQLite retention; REQ-071 cache DB) + **`orca logs --all-nodes --since` (REQ-117)** — tag `v0.10.8`
- [x] Phase P07: Recovery (`orca restore`) — tag `v0.10.9`
- [x] Phase P08: Integration tests — expand hermetic harness (REQ-087) + **drift-detection integration tests (auto-remediation, NFS fallback, cooldown, secret exclusion)** — tag `v0.10.10`
- [x] Phase P09: Collector + aggregator (opt-in; **gates C-11, C-12, C-14**) + **drift-event aggregation extension (REQ-107, D-237)** — tag `v0.10.11`
- [x] Phase P10a: Transactional plane (REQ-075, REQ-079; **gate C-09**; **gate C-23** cluster-wide vs ns-scoped txn distinction) — tag `v0.10.12`
- [x] Phase P10b: Drift detection (R-018/R-019/R-020; REQ-103..REQ-113; `orca drift` CLI, systemd Path unit emitter, `orca-drift-notify.sh`, `orca-remediate.sh`, cadence config, `--force`+per-ns gate, `orca` system user, NFS detection) — depends on P10a — tag `v0.10.13`
- [x] Phase P11: `orca job lint` (REQ-084) — tag `v0.10.14`
- [x] Phase P12: `orca job verify` (dry-run txn through lead) — tag `v0.10.15`
- [x] Phase P13: `orca ns` subcommands (full surface) + deprecation warnings (REQ-068) — tag `v0.10.16`
- [x] Phase P14a: v0.8→v1.0 data migration (REQ-066; **gate C-07**; **gate C-25** post-cutover verification + rollback; **gate C-27** orca user creation) + **`orca upgrade --to-vX` (REQ-115, thin wrapper, handles R-017 binding cutover)** — tag `v0.10.17`
- [x] Phase P14b: Daemon cutover + running-allocation adoption + **`orca cluster rotate-lead` (REQ-114)** — tag `v0.10.18`
- [x] Phase P14c: Mixed-version tolerance + no-orca-on-server enforcement (REQ-065, REQ-086; implements C-13) — tag `v0.10.19`
- [x] Phase P15: README quickstart (REQ-089; **Nomad-inspired framing per Q5=A, honest-trade-offs table from research doc 3**) — tag `v0.10.20`
- [x] Phase P15.5: Threat model + security review (**gate C-19**; **gate C-28** two sub-waves) + **ingress hybrid (R-017; nft emitter REQ-099, Traefik binding REQ-100, `orca doctor nft` REQ-101, `orca nft` CLI REQ-102) + `orca doctor mTLS` (REQ-118)** — tag `v0.10.21`
- [x] Phase P16: Final review + ship + audit — **v0.11.0 milestone release** — tag `v0.10.22` (v1.0.0 cut separately after UAT sign-off)

**Milestone tag**: `v0.11.0` (the v0.11 milestone release tag; v1.0.0 is
UAT-gated and cut separately after v0.11 completion per operator decision —
the v1.0.0 tag marks production-ready sign-off, not a separate milestone).
Per-phase patches run on the v0.10.x line per branch-strategy.md. Per-phase
tags: `v0.10.0`…`v0.10.21`.

### Per-phase REQ coverage (v0.11)

- **P00** — CLI cache (R-008)
- **P01.5** — SPIFFE spike (REQ-076; C-08)
- **P03** — Secrets (REQ-080; C-19)
- **P05** — Drain + daemon stop (REQ-061) + `orca job migrate` (REQ-116)
- **P06** — Alloc history (REQ-071 cache DB) + `orca logs --all-nodes --since` (REQ-117)
- **P08** — Integration tests (REQ-087) + drift-detection integration tests
- **P09** — Collector + aggregator (C-11, C-12, C-14) + drift-event aggregation (REQ-107, D-237)
- **P10a** — Transactional plane (REQ-075, REQ-079; C-09; C-23)
- **P10b** — Drift detection (R-018/R-019/R-020; REQ-103..REQ-113)
- **P11** — Job lint (REQ-084)
- **P13** — ns subcommands + deprecation warnings (REQ-068)
- **P14a/b/c** — Migration (REQ-066, REQ-065, REQ-086; C-07, C-13) + `orca upgrade` (REQ-115) + `orca cluster rotate-lead` (REQ-114)
- **P15** — README (REQ-089; Q5=A framing)
- **P15.5** — Threat model (C-19) + ingress hybrid (R-017; REQ-099..REQ-102) + `orca doctor mTLS` (REQ-118)

### New load-bearing rules adopted in Phase 0

- **R-017** — Ingress hybrid: nft DNAT → Traefik on `127.0.0.1:8443`; opt-out via `--public-binding`; `service { ingress: native }` per-workload opt-in
- **R-018** — Drift cadence: default 60s; critical 5s + systemd Path units; standard 30s
- **R-019** — Drift detector is a BACKSTOP; primary = systemd/Traefik/step-ca/Syncthing
- **R-020** — Hard gate: applier refuses txns on pre-flight drift; `--force` + per-ns scoping override

### Risk register (from grill + research, for ongoing monitoring)

- **step-ca single-instance SPOF** (mitigation: C-12 doc; v1.x HA via systemd failover)
- **master.key passphrase-less 0600** (mitigation: C-19 threat model; consider OS keyring in v1.x)
- **wasmtime CGO breaks cross-compile** (mitigation: C-01 spike; fallback to podman/process primary)
- **bash control plane drift** (mitigation: C-15..C-18 render-format contract + bats gate)
- **daemon cutover orphans running allocs** (mitigation: P14b split; test adoption)
- **27→35+ phase scope** (mitigation: C-04 resolved — operator accepted 40 phases; v0.11 grows to 24 phases per grill C-24 split of P10→P10a/P10b; scope folded in, no other new phases)
- **R-020 deadlock** (mitigation: `--force` flag + per-namespace scoping per Q4=A; drifted peer in ns-A doesn't block ns-B)
- **P10 sizing** (mitigation: P10 is the largest phase — drift detection + txn plane; grill may split into P10a/P10b if vertical slice is too large)
- **Ingress default migration** (mitigation: `orca upgrade` [REQ-115] handles Traefik binding cutover from `:443` to `127.0.0.1:8443` for existing v0.9/v0.10 clusters)
- **`orca` system user on peers** (mitigation: net-new operational requirement; peer-setup emits `useradd -r orca` idempotently; documented in P10)

## Deferred to v1.x (out of scope for v0.11)

- `sqlite-wal-shared` state backend (R-009 abstractions ship in v1.0; backend in v1.x)
- `git` state backend
- `file+flock` state backend
- `orca cluster setup-shared` UX
- HA `step-ca` (active/passive via systemd)
- Journald log shipping (optional centralized audit)
- Network policy (`nftables` snippets)
- GPU / TPU constraints

## Deferred to v2.x (out of scope for v1.x)

- Full Nomad-HCL parser with no conversion round-trip
- Nomad-API subset for migrating existing Nomad fleets
- Nomad driver bridge
- Helm-equivalent templating (probably never)
- Service mesh beyond Traefik
- CRDs / Operators / Plugin model
- Leader-elected Raft coordinator
- External CA / Let's Encrypt / cert transparency
- Online-only features (HSTS, OCSP stapling, telemetry)

## Milestone v0.12: Security Hardening (Zero-Trust Identity) — **COMPLETE**

**Scope**: comprehensive security hardening across the entire attack
surface, **including the operating system itself**, plus adoption of a
zero-trust identity model. The v0.12 threat-model review (Phase 0
RESEARCH) surfaced 25 distinct findings (F1..F25) spanning injection,
traversal, ACL, audit, crypto, OS scripts, emitters, sudoers, system
users, file modes, daemon auth, backup, SQLite, install.sh, and
migration. v0.12 closes all of them and adopts **R-021** (no Orca
credentials) as the load-bearing architectural change: human identity is
exclusively external (OIDC), machine identity is exclusively
mTLS/SPIFFE, and no passwords/Orca-issued-tokens/CA-key-passphrases
exist anywhere in the system.

The operator locked two architectural decisions: **(1) bundled Dex by
default + BYO external IdP override** (D-239), and **(2) master key
seal-to-OIDC + Shamir 3-of-5 recovery** (D-241). A third decision added
**WebAuthn (passkeys) as the bundled password-free authenticator** for
Dex (D-240) -- passkeys are public-key credentials (private key never
leaves the authenticator), directly satisfying R-021.

**Milestone type**: feature (P04 OIDC+Dex and P05 WebAuthn ship `feat`
phases; the rest are `fix`/`chore`/`test`/`docs`/`refactor`).

- [x] Phase 0: Pre-execution (specify -> clarify -> research -> ideate -> plan -> grill) -- tag `v0.11.0`
- [x] Phase P0[0-9]: Command injection fix (podman/wasm shellQuote) (REQ-119, F3) -- tag `v0.11.1`
- [x] Phase P0[0-9]: Namespace path traversal fix (REQ-120, F4) -- tag `v0.11.2`
- [x] Phase P0[0-9]: Txn apply path allowlist (REQ-121, F5) -- tag `v0.11.3`
- [x] Phase P0[0-9]: OIDC client + bundled Dex (REQ-144; BYO-IdP override) -- tag `v0.11.4`
- [x] Phase P0[0-9]: WebAuthn connector for Dex (REQ-148; passkeys, browser auth+register) -- tag `v0.11.5`
- [x] Phase P0[0-9]: ACL rewrite to OIDC claims + enforcement (REQ-145, REQ-122, F1) -- tag `v0.11.6`
- [x] Phase P0[0-9]: Remove all password/token paths (breaking; REQ-146, R-021, C-34) -- tag `v0.11.7`
- [x] Phase P0[0-9]: Master key seal-to-OIDC + Shamir 3-of-5 (REQ-147, C-35) -- tag `v0.11.8`
- [x] Phase P0[0-9]: Daemon auth hardening (REQ-123, REQ-124, F6, F24) -- tag `v0.11.9`
- [x] Phase P0+: Audit log tamper-evidence (REQ-125, F2) -- tag `v0.11.10`
- [x] Phase P0+: SVID chain validation (REQ-126, F9) -- tag `v0.11.11`
- [x] Phase P0+: Backup symlink validation (REQ-127, F7) -- tag `v0.11.12`
- [x] Phase P0+: step-ca /tmp hardening (REQ-128, F10) -- tag `v0.11.13`
- [x] Phase P0+: Master key rotation (re-seal to OIDC; REQ-129, F12, C-30) -- tag `v0.11.14`
- [x] Phase P0+: File-mode audit expansion (REQ-130, F13) -- tag `v0.11.15`
- [x] Phase P0+: aggregate.sh JSON injection + drift-gate parse fix (REQ-131, F11, F18) -- tag `v0.11.16`
- [x] Phase P0+: install.sh checksum+GPG verification (REQ-132, F14) -- tag `v0.11.17`
- [x] Phase P0+: nftables ruleset hardening (REQ-133, F21) -- tag `v0.11.18`
- [x] Phase P0+: sudoers hardening (REQ-134, F22) -- tag `v0.11.19`
- [x] Phase P0+: System user consistency (REQ-135, F23) -- tag `v0.11.20`
- [x] Phase P0+: SQLite file-mode + at-rest encryption (REQ-136, F8, C-31) -- tag `v0.11.21`
- [x] Phase P0+: Migration safety + identity migration (REQ-137, F19, C-34) -- tag `v0.11.22`
- [x] Phase P0+: Legacy CA/mTLS/daemon + step-ca password-provisioner deletion (REQ-138, F16; **gate C-29: P06/P08/P09/P11**) -- tag `v0.11.23`
- [x] Phase P0+: known_hosts tightening + transport hardening (REQ-139, F15, F25) -- tag `v0.11.24`
- [x] Phase P0+: Drift event authentication (REQ-140, F18) -- tag `v0.11.25`
- [x] Phase P0+: Security integration test suite (REQ-141, C-33) -- tag `v0.11.26`
- [x] Phase P0+: Zero-trust + OIDC + WebAuthn + threat-model docs (REQ-142) -- tag `v0.11.27`
- [x] Phase P0+: Final review + ship + audit (milestone release) -- tag `v0.11.28` = **v0.12 milestone release**

**Milestone tag**: `v0.11.28` (final phase patch = milestone release per
feature-milestone progressive-patch rule; no separate `v0.12.0` tag).
Per-phase tags: `v0.11.0`..`v0.11.28` (29 tags). Tags run on the
previous minor's patch line (v0.11.x) per branch-strategy.md. The
milestone branch label uses the milestone number
(`milestone/v0.12-security-hardening`); no separate minor tag.

The v1.0.0 production-ready tag stays deferred for post-v0.12 UAT
(per v0.11 PRD; v0.12 is a minor feature milestone, not the v1.0 cut).

### Per-phase REQ coverage (v0.12)

- **P01** -- Command injection (REQ-119, F3)
- **P02** -- Namespace path traversal (REQ-120, F4)
- **P03** -- Txn apply path allowlist (REQ-121, F5)
- **P04** -- OIDC client + bundled Dex (REQ-144; D-239, D-242, D-246)
- **P05** -- WebAuthn connector (REQ-148; D-240, D-243, D-244, C-38)
- **P06** -- ACL rewrite + enforcement (REQ-145, REQ-122, F1)
- **P07** -- Remove password/token paths (REQ-146, R-021, C-34)
- **P08** -- Master key seal-to-OIDC + Shamir (REQ-147, D-241, C-35)
- **P09** -- Daemon auth (REQ-123, REQ-124, F6, F24)
- **P10** -- Audit tamper-evidence (REQ-125, F2)
- **P11** -- SVID chain validation (REQ-126, F9)
- **P12** -- Backup symlink validation (REQ-127, F7)
- **P13** -- step-ca /tmp hardening (REQ-128, F10)
- **P14** -- Master key rotation (REQ-129, F12, C-30)
- **P15** -- File-mode audit expansion (REQ-130, F13)
- **P16** -- aggregate.sh JSON injection + drift-gate (REQ-131, F11, F18)
- **P17** -- install.sh checksum+GPG (REQ-132, F14)
- **P18** -- nftables ruleset hardening (REQ-133, F21)
- **P19** -- sudoers hardening (REQ-134, F22)
- **P20** -- System user consistency (REQ-135, F23)
- **P21** -- SQLite file-mode + encryption (REQ-136, F8, C-31)
- **P22** -- Migration safety + identity migration (REQ-137, F19, C-34)
- **P23** -- Dual-write closure (REQ-138, F16; **gate C-29**)
- **P24** -- known_hosts + transport hardening (REQ-139, F15, F25)
- **P25** -- Drift event authentication (REQ-140, F18)
- **P26** -- Security integration test suite (REQ-141, C-33)
- **P27** -- Zero-trust + OIDC + WebAuthn + threat-model docs (REQ-142)
- **P28** -- Final review + ship + audit (REQ-143)

### New load-bearing rule adopted in Phase 0

- **R-021** -- Orca never issues, stores, or accepts human-identity
  credentials. Human identity is exclusively external (OIDC). Machine
  identity is exclusively mTLS/SPIFFE. No passwords, no Orca-issued
  tokens, no CA-key passphrases.

### Binding conditions (for GRILL ratification; C-29..C-38)

- **C-29**: P23 (dual-write closure) gated on P06/P08/P09/P11 all shipped.
- **C-30**: P14 (master key rotation) reversible; `--dry-run` mandatory; auto-rollback to old sealed key on any ns failure.
- **C-31**: P21 (SQLite encryption): CGO-free fallback to file-mode 0600 + documented threat if SQLCipher needs CGO. No CGO.
- **C-32**: **Human-gate**: leaked GITEA_TOKEN (F17) rotated + `.env` re-seeded before P28 ships. History-scrub best-effort, non-blocking. Escalation hook in `---ci---`.
- **C-33**: P26 (security integration tests) in `.coreci.yml` `validate`, gates merges -- not opt-in.
- **C-34**: P07 (password/token removal) breaking. `orca upgrade` (P22) refuses v0.11 clusters using `--password`/bare-tokens without `--accept-identity-migration`. No silent breakage.
- **C-35**: P08 (Shamir recovery): 3-of-5 shards printed at seal time, operator stores offline. If IdP lost AND quorum unavailable -> cluster unrecoverable by design (documented residual risk). No backdoor.
- **C-36**: OIDC client secret (confidential clients) at `ClusterDir()/oidc-client-secret` (0600), rotatable via `orca auth rotate-client-secret`, never committed. Public PKCE clients avoid even this.
- **C-37**: P04 (bundled Dex): if WebAuthn proves infeasible, bundled Dex ships mTLS-client-cert-only; password-based upstreams require BYO external IdP. The "no Orca credentials" invariant holds regardless. *(Largely moot -- WebAuthn solves it.)*
- **C-38**: P05 (WebAuthn): RP ID must match the cluster's Traefik-served domain; `orca auth init-idp` configures it. HTTPS secure context via Traefik (step-ca cert). P26 integration tests use the WebAuthn virtual-authenticator API -- no hardware key required in CI.

### Risk register (from grill + research, for ongoing monitoring)

- **P07 breaking change** (mitigation: C-34 migration gate)
- **P08 master key seal is riskiest** (mitigation: `--dry-run`, atomic, auto-rollback, C-35 Shamir recovery)
- **P21 SQLite encryption may need CGO** (mitigation: C-31 fallback to file-mode 0600)
- **P23 dual-write closure high-impact** (mitigation: gate C-29; full test coverage before deletion)
- **P05 WebAuthn connector is new ground** (mitigation: C-37 mTLS-client-cert fallback; virtual-authenticator tests in P26)
- **Bundled Dex is a new systemd unit + Traefik route** (mitigation: `orca doctor oidc` health check)
- **C-32 human gate could stall final ship** (mitigation: ship as `v0.11.28-rc1` if rotation pending)
- **29 phases is large** (mitigation: grill may split/merge; operator accepted "more than 20 if warranted")

### Deferred to v1.x (out of scope for v0.12)

- HA step-ca (active/passive via systemd)
- `sqlite-wal-shared` / `git` / `file+flock` state backends
- OS keyring integration for master key (v0.12 uses OIDC seal instead)
- Full cluster-rolling-upgrade orchestrator (v0.12 ships the thin `orca upgrade` wrapper only)
- Live-migrate with storage replication (v0.12 ships drain+reschedule only)
- Journald log shipping (optional centralized audit)
- Network policy (`nftables` snippets beyond the ingress ruleset)
- GPU / TPU constraints

### Deferred to v2.x (out of scope for v1.x)

- Full Nomad-HCL parser with no conversion round-trip
- Nomad-API subset for migrating existing Nomad fleets
- Nomad driver bridge
- Helm-equivalent templating (probably never)
- Service mesh beyond Traefik
- CRDs / Operators / Plugin model
- Leader-elected Raft coordinator
- External CA / Let's Encrypt / cert transparency
- Online-only features (HSTS, OCSP stapling, telemetry)

## Milestone v0.13: Production Hardening Round 2 + UAT Plan — **COMPLETE**

**Scope**: final production hardening round before the v1.0.0
production-ready tag. Three deep codebase sweeps (security, reliability,
feature/doc claims) surfaced ~60 gaps beyond v0.12 — the most critical
being that `orca job run` runs locally via `exec.CommandContext` and
never invokes the scheduler/emitter/SSH-push path (the documented
deployment model is non-functional), jobspec `schedule:`/`timeout:` are
silently dropped by the markdown parser (DaemonSet is fundamentally
broken), `acl.Check` is called zero times in the codebase (v0.12's
headline zero-trust feature is library-complete but not wired), and
several command-injection vectors remain (`orca logs --job` backtick
RCE via `%q`, tar-slip in restore, sudoers injection, etc.). v0.13
closes all critical/high/medium findings and delivers the UAT plan +
signoff script that gates the v1.0.0 cut.

**Load-bearing architectural changes**:
- **R-022** — `orca job run` deploys to remote nodes via the scheduler
  → emitter → SSH-push pipeline. The local `exec.CommandContext` path
  is removed. Constraints/capacity/affinity are enforced. This makes
  the documented deployment model functional and is the prerequisite
  for the UAT plan.
- **R-023** — Zero-trust enforcement is operationally wired:
  `acl.Check` is invoked on every daemon handler + sshpush + txn apply
  path; `acl.json` is 0600; audit `actor` carries OIDC sub/SVID;
  WebAuthn registration requires auth; `cluster seal`/`unseal` +
  `doctor audit`/`doctor modes` CLI commands exist.

### Phases (14 total: P0 + P01..P12 + P13 final)

- [x] Phase P0: Pre-execution (SPECIFY→CLARIFY→RESEARCH→IDEATE→PLAN→GRILL) — tag `v0.12.0`
- [x] Phase P01: Toolchain & dependency vulns (REQ-149) — tag `v0.12.1`
- [x] Phase P02: Input validation & injection hardening (REQ-150) — tag `v0.12.2`
- [x] Phase P03: Scheduler/deployment wiring + jobspec parser (REQ-151, REQ-152) — tag `v0.12.3`
- [x] Phase P04: ACL enforcement + WebAuthn registration auth (REQ-153) — tag `v0.12.4`
- [x] Phase P05: Seal/audit CLI + chain race + key zeroing (REQ-154) — tag `v0.12.5`
- [x] Phase P06: auth init-idp real + auth register (REQ-155) — tag `v0.12.6`
- [x] Phase P07: Concurrency safety (REQ-156) — tag `v0.12.7`
- [x] Phase P08: Transport & SSH safety (REQ-157) — tag `v0.12.8`
- [x] Phase P09: Migration & operational safety (REQ-158) — tag `v0.12.9`
- [x] Phase P10: Observability & metrics (REQ-159) — tag `v0.12.10`
- [x] Phase P11: Doc drift round 2 (REQ-160) — tag `v0.12.11`
- [x] Phase P12: `--type linux` + UAT plan + signoff script (REQ-161, REQ-162, REQ-163) — tag `v0.12.12`
- [x] Phase P13: Final review + ship + audit (milestone release) — tag `v0.12.13` = **v0.13 milestone release**

**Milestone tag**: `v0.12.13` (final phase patch = milestone release per
feature-milestone rule; no separate `v0.13.0` tag). Per-phase tags:
`v0.12.0`..`v0.12.13` (14 tags). Tags run on the previous minor's patch
line (v0.12.x). The milestone branch label uses the milestone number
(`milestone/v0.13-production-hardening-2`); no separate minor tag.

The v1.0.0 production-ready tag stays deferred for post-v0.13 UAT
signoff (operator runs `scripts/uat-signoff.sh`, pastes output back;
CI agent verifies and cuts v1.0.0).

### Per-phase REQ coverage (v0.13)

- **P01** — Toolchain bump (REQ-149)
- **P02** — Injection hardening (REQ-150)
- **P03** — Scheduler wiring + jobspec parser (REQ-151, REQ-152)
- **P04** — ACL enforcement + WebAuthn reg auth (REQ-153)
- **P05** — Seal/audit CLI + chain race + key zeroing (REQ-154)
- **P06** — auth init-idp real + auth register (REQ-155)
- **P07** — Concurrency safety (REQ-156)
- **P08** — Transport & SSH safety (REQ-157)
- **P09** — Migration & operational safety (REQ-158)
- **P10** — Observability & metrics (REQ-159)
- **P11** — Doc drift round 2 (REQ-160)
- **P12** — `--type linux` + UAT plan + signoff (REQ-161, REQ-162, REQ-163)
- **P13** — Final review + ship + audit

### New load-bearing rules adopted in Phase 0

- **R-022** — `orca job run` deploys to remote nodes via scheduler →
  emitter → SSH-push. Local exec path removed. Constraints/capacity/
  affinity enforced.
- **R-023** — Zero-trust enforcement is operationally wired:
  `acl.Check` on every request path; `acl.json` 0600; audit actor =
  OIDC sub/SVID; WebAuthn registration requires auth.

### Binding conditions (for GRILL ratification — C-39..C-49)

- **C-39**: P03 (scheduler wiring) is the riskiest phase — changes the
  core `job run` path. Must not break existing `job run` (local
  fallback if no remote nodes registered). Full test coverage before
  P04 ships.
- **C-40**: P04 (ACL enforcement) is deny-by-default — must not lock
  out the operator. Bootstrap ACL grants `cluster-admin` to the init
  cert's SPIFFE SVID. Staged rollout: log-only mode for first run,
  enforce after bootstrap ACL verified.
- **C-41**: P05 (seal) — C-35 residual risk still applies (IdP lost +
  Shamir quorum unavailable → cluster unrecoverable). No backdoor.
- **C-42**: P12 (UAT plan + signoff) is the v1.0 gate artifact. If
  P01..P11 slip, P12 still ships (honest signal via failing
  assertions). The signoff script is idempotent and read-only.
- **C-43**: `verify-reqs` bold-format regex must be fixed in P11 so
- **C-44**: P03 MUST fail-closed when scheduler selects a node but SSH-push fails. Local fallback only when `len(registeredNodes)==0`. Test case mandatory.
- **C-45**: P04 MUST implement log-only/dry-run mode as default for first invocation after ACL wiring. Enforce mode after bootstrap ACL verified.
- **C-46**: P12 dependency table MUST include P05 (seal) and P06 (auth init-idp) in addition to P03 and P04.
- **C-47**: P12 `uat-signoff.sh` MUST include explicit assertions for: (a) job deployed to remote node, (b) ACL deny-by-default, (c) seal/unseal round-trip, (d) OIDC health check.
- **C-48**: P12 `docs/uat.md` MUST document hardware prerequisites (Proxmox VE 8/9 host required). Alternative UAT path (3x Ubuntu, Proxmox claims skipped) MUST be documented.
- **C-49**: Plan narrative MUST soften "last hardening round" to "last hardening round before UAT validation." UAT will likely surface 3-7 issues requiring patch release.
  the consistency gate works for v0.12 AND v0.13.

### Risk register (for grill + research, for ongoing monitoring)

- **P03 scheduler wiring is riskiest** (mitigation: C-39 local fallback)
- **P04 ACL deny-by-default could lock out operator** (mitigation: C-40 bootstrap ACL + staged rollout)
- **P05 seal residual risk** (mitigation: C-41 documented, no backdoor)
- **P02 injection hardening is high-count** (11 sub-fixes; mitigation: each is small and independently testable)
- **14 phases is large** (mitigation: operator accepted "no limit on phases"; many phases are small fix bundles)
- **UAT plan depends on P03 (scheduler) being functional** (mitigation: P12 ships regardless; failing assertions are honest signal)

### Deferred to v1.x (out of scope for v0.13) — unchanged from v0.12

- HA step-ca (active/passive via systemd)
- `sqlite-wal-shared` / `git` / `file+flock` state backends
- OS keyring integration for master key
- Full cluster-rolling-upgrade orchestrator (v0.13 ships the thin `orca upgrade` wrapper only)
- Live-migrate with storage replication (v0.13 ships drain+reschedule only)
- Journald log shipping (optional centralized audit)
- Network policy (`nftables` snippets beyond the ingress ruleset)
- GPU / TPU constraints
- jobspec `health` prober (v0.13 adds lint warning; enforcement deferred)
- jobspec `update` rolling/canary controller (v0.13 adds lint warning; enforcement deferred)
- jobspec `schedule.cron` scheduler loop (v0.13 adds lint warning; enforcement deferred)