docs(P00): grill v0.11 — 6 binding conditions (C-23..C-28), P10 split into P10a/P10b

GRILL stage: adversarial review across 6 forcing questions. Verdict: PROCEED-WITH-CONDITIONS.
- C-23: orca-pull.sh distinguishes cluster-wide vs ns-scoped txns (gate P10a)
- C-24: split P10 into P10a (txn plane) + P10b (drift detection); phase count 23->24; tags shift by 1
- C-25: orca upgrade post-cutover verification + rollback (gate P14a)
- C-26: per-phase LoC soft ceiling ~800 (no gate, recorded)
- C-27: orca upgrade creates orca system user on existing peers (gate P14a)
- C-28: P15.5 two sub-waves (ingress+doctor nft, then threat model+doctor mTLS) (gate P15.5)
ROADMAP updated: 24 phases, tags v0.10.0..v0.10.22.

---ci---
project: orca
phase: 0
milestone: v0.11
status: grill
---/ci---
This commit is contained in:
Jon Chery
2026-08-07 03:40:55 +00:00
parent 5b99bbd2e8
commit e45611b416
3 changed files with 191 additions and 15 deletions
+6 -3
View File
@@ -1,13 +1,16 @@
{ {
"phase": 0, "phase": 0,
"stage": "plan", "stage": "grill",
"milestone": "v0.11", "milestone": "v0.11",
"milestone_slug": "production-hardening", "milestone_slug": "production-hardening",
"phase_role": "pre_execution", "phase_role": "pre_execution",
"attempts": 0, "attempts": 0,
"updated_at": "2026-08-07T00:03:00Z", "updated_at": "2026-08-07T00:04:00Z",
"milestone_complete": false, "milestone_complete": false,
"previous_milestone": "v0.10", "previous_milestone": "v0.10",
"research_docs_ingested": 5, "research_docs_ingested": 5,
"locked_decisions": {"Q1": "A", "Q2": "C", "Q3": "A", "Q4": "A", "Q5": "A"} "locked_decisions": {"Q1": "A", "Q2": "C", "Q3": "A", "Q4": "A", "Q5": "A"},
"grill_verdict": "PROCEED-WITH-CONDITIONS",
"binding_conditions": ["C-23", "C-24", "C-25", "C-26", "C-27", "C-28"],
"phase_count": 24
} }
+171
View File
@@ -0,0 +1,171 @@
# Grill: v0.11 Production Hardening — Phase 0 Adversarial Review
**Status**: PROCEED-WITH-CONDITIONS. The v0.11 plan is sound; 6 binding
conditions (C-23…C-28) gate specific phases. The plan adopts R-017…R-020
and D-215…D-237 from 5 research docs with operator decisions Q1=A, Q2=C,
Q3=A, Q4=A, Q5=A. The grill reviewed the plan adversarially across the
same 9 axes as GRILL_v0.9 (vision, feasibility, scope, risk, security,
operational, cost, competitive, exit).
## Forcing questions + verdicts
### FQ1 — R-020 deadlock with `--force` + per-ns scoping
**Question**: With `--force` + per-namespace scoping (Q4=A), can a single
drifted peer still block a *cluster-wide* txn (e.g., namespace creation)?
If yes, is the `--force` escape hatch documented in C-09's failure
contract?
**Verdict**: PARTIAL-BLOCK remains for cluster-wide txns. A namespace
*creation* txn touches all peers (the new namespace dir is created on
every peer). If one peer is drifted, the pre-flight gate refuses the
txn cluster-wide. `--force` overrides this, but `--force` on a
namespace-creation txn is risky (it forces the new namespace onto a
drifted peer without reconciling the drift first).
**Binding condition C-23**: `orca-pull.sh` (C-09) must distinguish
*cluster-wide* txns from *namespace-scoped* txns. Cluster-wide txns
require `--force` with an explicit `--i-understand-the-risk` confirmation
(or `--yes` for non-interactive). Namespace-scoped txns use per-ns
scoping (drifted peer in ns-A doesn't block ns-B). **Gate**: P10.
**Confidence**: 0.88
### FQ2 — P10 sizing (txn plane + drift detection in one phase)
**Question**: P10 now absorbs drift detection (~500 LoC Go + 150 LoC
bash + systemd units), the largest single phase. Is this a vertical
slice that can ship atomically, or does it need splitting (P10a txn
plane, P10b drift)?
**Verdict**: SPLIT RECOMMENDED. P10 has 13 tasks spanning two distinct
subsystems: (1) the transactional plane (T1-T2: txn bundle render, SCP,
apply, C-09 failure contract) and (2) drift detection (T3-T13: `internal/drift/`,
Path unit emitter, notify/remediate scripts, cadence config, pre-flight
gate, `orca` user, NFS detection, job restart). The txn plane is a
prerequisite for drift detection (T3's `Aggregate` reads applied txn
manifests), so the split is clean: P10a (txn plane, T1-T2) ships first,
P10b (drift detection, T3-T13) ships after P10a.
**Binding condition C-24**: Split P10 into P10a (transactional plane,
REQ-075/079, C-09) and P10b (drift detection, R-018/R-019/R-020,
REQ-103..113). P10a ships first; P10b depends on P10a. Tags: P10a
`v0.10.12`, P10b `v0.10.13`. All subsequent phase tags shift by 1
(P11→`v0.10.14`, …, P16→`v0.10.22`). **Phase count: 23 → 24.**
**Confidence**: 0.92
### FQ3 — Ingress default migration path (C1)
**Question**: Existing v0.9/v0.10 clusters run Traefik on `:443`. R-017
makes `127.0.0.1:8443` + nft the default. What's the upgrade path? Does
`orca upgrade` (Q2=C) handle the binding cutover, or is it a manual
operator step?
**Verdict**: UPGRADE HANDLES IT, but with a safety check. `orca upgrade`
(REQ-115, P14a) is the thin wrapper (C2=a) that handles the Traefik
binding cutover. The cutover is: (1) emit new Traefik static config with
`127.0.0.1:8443`, (2) emit `/etc/nftables.d/orca.nft` with DNAT, (3)
`systemctl reload traefik` + `nft -f`, (4) verify `curl :443` still
routes. If step 4 fails, rollback to `:443` + remove nft rules.
**Binding condition C-25**: `orca upgrade` (REQ-115) must include a
post-cutover verification step (`curl -k https://localhost:443/` returns
200 from Traefik) with automatic rollback on failure. Document the
rollback procedure in `docs/ingress.md`. **Gate**: P14a.
**Confidence**: 0.90
### FQ4 — Scope ceiling (LoC vs phase count)
**Question**: v0.11 stays at 23 phases (now 24 with C-24), but P09/P10
(now P10a/P10b)/P15.5 grow substantially. Is the *phase count* the right
ceiling, or should there be a *LoC/effort* ceiling per phase?
**Verdict**: LOOSE LoC CEILING. Phase count is a proxy for effort, but
P10b (drift detection) is ~650 LoC across Go + bash + systemd — at the
upper end of what a single-phase vertical slice can handle. The grill
recommends a soft LoC ceiling of ~800 LoC per phase (Go + bash + config),
with splitting required above ~1200 LoC.
**Binding condition C-26**: Per-phase LoC soft ceiling: ~800 LoC (Go +
bash + config). Split required above ~1200 LoC. P10b (~650 LoC) is within
the soft ceiling; P15.5 (~400 LoC: nft emitter 200 + doctor mTLS 100 +
threat model doc) is within. No action required for v0.11; recorded for
future milestones. **No gate.**
**Confidence**: 0.85
### FQ5 — `orca` system user on peers (operational impact)
**Question**: Creating a system user on every peer is a new operational
requirement. Does this break any existing v0.9/v0.10 deployment that
runs as root or as an existing service account?
**Verdict**: NO BREAK for existing deployments; NEW requirement for drift
detection. The `orca` system user (REQ-111) is created at peer setup
(`orca node join` / peer-setup script). Existing v0.9/v0.10 peers don't
have the `orca` user, so drift detection's systemd Path units (which run
as `User=orca`) won't start until the user is created. `orca upgrade`
(REQ-115) must create the `orca` user on existing peers as part of the
v0.11 migration.
**Binding condition C-27**: `orca upgrade` (REQ-115, P14a) must create
the `orca` system user on existing peers (`useradd -r orca` idempotent)
before P10b's drift detection can function. Document this as a
migration prerequisite. **Gate**: P14a.
**Confidence**: 0.91
### FQ6 — P15.5 is now a mega-phase (threat model + ingress + doctor mTLS)
**Question**: P15.5 was originally "threat model + security review" (C-19).
It now absorbs ingress hybrid (R-017; REQ-099..102, ~400 LoC) + `orca
doctor mTLS` (REQ-118). Is this too much for one phase?
**Verdict**: MANAGEABLE but at the ceiling. P15.5 is now ~500 LoC (nft
emitter 200 + doctor mTLS 100 + threat model doc + tests). The ingress
hybrid and threat model are related (both are security-hardening), so
keeping them together is defensible. The `orca doctor mTLS` (REQ-118)
is small and reuses P01/P01.5 infrastructure. The grill recommends
keeping P15.5 as one phase but splitting the *work* into two sub-waves
within the phase: (1) ingress hybrid + doctor nft, (2) threat model +
doctor mTLS.
**Binding condition C-28**: P15.5 commits in two sub-waves: (1) ingress
hybrid (REQ-099..102) + `orca doctor nft` (REQ-101), (2) threat model
(C-19) + `orca doctor mTLS` (REQ-118). Both ship under the same phase
tag (`v0.10.20`). **No new phase; internal ordering only.**
**Confidence**: 0.89
## Binding conditions summary
| ID | Condition | Gate | Verification |
|----|-----------|------|--------------|
| C-23 | `orca-pull.sh` distinguishes cluster-wide vs namespace-scoped txns; cluster-wide requires `--force` + `--i-understand-the-risk` (or `--yes`) | P10a | Test: cluster-wide txn refused without `--force`; ns-scoped txn blocks only the drifted ns |
| C-24 | Split P10 into P10a (txn plane, REQ-075/079, C-09) + P10b (drift detection, R-018/R-019/R-020, REQ-103..113); P10b depends on P10a; tags shift by 1 | P10a→P10b | Plan shows P10a + P10b as separate phases; P10b tasks reference P10a txn manifests |
| C-25 | `orca upgrade` (REQ-115) includes post-cutover verification (`curl -k https://localhost:443/` returns 200) with automatic rollback on failure; rollback documented in `docs/ingress.md` | P14a | Test: cutover succeeds → 200; cutover fails → rollback to `:443` |
| C-26 | Per-phase LoC soft ceiling: ~800 LoC (Go + bash + config); split required above ~1200 LoC | (no gate) | Recorded for future milestones |
| C-27 | `orca upgrade` (REQ-115) creates `orca` system user on existing peers before P10b drift detection can function | P14a | Test: existing peer without `orca` user → `orca upgrade` creates it → drift detection starts |
| C-28 | P15.5 commits in two sub-waves: (1) ingress hybrid + doctor nft, (2) threat model + doctor mTLS; same phase tag | P15.5 | Commits show two sub-waves; both under `v0.10.20` |
## Phase challenge summary
| PC | Phase | Challenge | Resolution |
|----|-------|-----------|------------|
| PC-11 | P10a/P10b | Txn plane + drift detection too large for one phase | Split per C-24; P10a ships first, P10b depends on it |
| PC-12 | P15.5 | Mega-phase (threat model + ingress + doctor mTLS) | Keep as one phase; two sub-waves per C-28 |
| PC-13 | P14a | `orca upgrade` handles 3 migrations (data + binding + orca user) | All three land in P14a per C-25, C-27; thin wrapper (C2=a) |
| PC-14 | P09 | Aggregator extension depends on P10b drift detection | P09 in Wave 6 (after Wave 5 P10b); aggregator extension (REQ-107) only works once drift events exist |
## Overall verdict
**PROCEED-WITH-CONDITIONS**. The v0.11 plan is sound. 6 binding conditions
(C-23…C-28) gate specific phases. The plan grows from 23 → 24 phases
(C-24 splits P10 into P10a/P10b). All other phases are unchanged in
count; their scope expands per the research folding (Q2=C, Q3=A).
The grill's confidence in the v0.11 plan is high (avg 0.89 across FQs).
The primary risks (P10 sizing, R-020 deadlock, ingress migration) are
all gated with verifiable conditions.
+14 -12
View File
@@ -322,16 +322,17 @@ ingress into P15.5).
- [ ] Phase P07: Recovery (`orca restore`) — tag `v0.10.9` - [ ] Phase P07: Recovery (`orca restore`) — tag `v0.10.9`
- [ ] Phase P08: Integration tests — expand hermetic harness (REQ-087) + **drift-detection integration tests (auto-remediation, NFS fallback, cooldown, secret exclusion)** — tag `v0.10.10` - [ ] Phase P08: Integration tests — expand hermetic harness (REQ-087) + **drift-detection integration tests (auto-remediation, NFS fallback, cooldown, secret exclusion)** — tag `v0.10.10`
- [ ] Phase P09: Collector + aggregator (opt-in; **gates C-11, C-12, C-14**) + **drift-event aggregation extension (REQ-107, D-237)** — tag `v0.10.11` - [ ] Phase P09: Collector + aggregator (opt-in; **gates C-11, C-12, C-14**) + **drift-event aggregation extension (REQ-107, D-237)** — tag `v0.10.11`
- [ ] Phase P10: Transactional plane (REQ-075, REQ-079; **gate C-09**) + **drift detection (R-018/R-019/R-020, REQ-103..REQ-113; `orca drift` CLI, systemd Path unit emitter, `orca-drift-notify.sh`, `orca-remediate.sh`, cadence config, `--force`+per-ns gate, `orca` system user, NFS detection)** — tag `v0.10.12` - [ ] Phase P10a: Transactional plane (REQ-075, REQ-079; **gate C-09**; **gate C-23** cluster-wide vs ns-scoped txn distinction) — tag `v0.10.12`
- [ ] Phase P11: `orca job lint` (REQ-084) — tag `v0.10.13` - [ ] Phase P10b: Drift detection (R-018/R-019/R-020; REQ-103..REQ-113; `orca drift` CLI, systemd Path unit emitter, `orca-drift-notify.sh`, `orca-remediate.sh`, cadence config, `--force`+per-ns gate, `orca` system user, NFS detection) — depends on P10a — tag `v0.10.13`
- [ ] Phase P12: `orca job verify` (dry-run txn through lead) — tag `v0.10.14` - [ ] Phase P11: `orca job lint` (REQ-084) — tag `v0.10.14`
- [ ] Phase P13: `orca ns` subcommands (full surface) + deprecation warnings (REQ-068) — tag `v0.10.15` - [ ] Phase P12: `orca job verify` (dry-run txn through lead) — tag `v0.10.15`
- [ ] Phase P14a: v0.8→v1.0 data migration (REQ-066; **gate C-07**) + **`orca upgrade --to-vX` (REQ-115, thin wrapper, handles R-017 binding cutover)** — tag `v0.10.16` - [ ] Phase P13: `orca ns` subcommands (full surface) + deprecation warnings (REQ-068) — tag `v0.10.16`
- [ ] Phase P14b: Daemon cutover + running-allocation adoption + **`orca cluster rotate-lead` (REQ-114)** — tag `v0.10.17` - [ ] Phase P14a: v0.8→v1.0 data migration (REQ-066; **gate C-07**; **gate C-25** post-cutover verification + rollback; **gate C-27** orca user creation) + **`orca upgrade --to-vX` (REQ-115, thin wrapper, handles R-017 binding cutover)** — tag `v0.10.17`
- [ ] Phase P14c: Mixed-version tolerance + no-orca-on-server enforcement (REQ-065, REQ-086; implements C-13) — tag `v0.10.18` - [ ] Phase P14b: Daemon cutover + running-allocation adoption + **`orca cluster rotate-lead` (REQ-114)** — tag `v0.10.18`
- [ ] Phase P15: README quickstart (REQ-089; **Nomad-inspired framing per Q5=A, honest-trade-offs table from research doc 3**) — tag `v0.10.19` - [ ] Phase P14c: Mixed-version tolerance + no-orca-on-server enforcement (REQ-065, REQ-086; implements C-13) — tag `v0.10.19`
- [ ] Phase P15.5: Threat model + security review (**gate C-19**) + **ingress hybrid (R-017; nft emitter REQ-099, Traefik binding REQ-100, `orca doctor nft` REQ-101, `orca nft` CLI REQ-102) + `orca doctor mTLS` (REQ-118)** — tag `v0.10.20` - [ ] Phase P15: README quickstart (REQ-089; **Nomad-inspired framing per Q5=A, honest-trade-offs table from research doc 3**) — tag `v0.10.20`
- [ ] Phase P16: Final review + ship + audit — **v0.11.0 milestone release** — tag `v0.10.21` (v1.0.0 cut separately after UAT sign-off) - [ ] Phase P15.5: Threat model + security review (**gate C-19**; **gate C-28** two sub-waves) + **ingress hybrid (R-017; nft emitter REQ-099, Traefik binding REQ-100, `orca doctor nft` REQ-101, `orca nft` CLI REQ-102) + `orca doctor mTLS` (REQ-118)** — tag `v0.10.21`
- [ ] Phase P16: Final review + ship + audit — **v0.11.0 milestone release** — tag `v0.10.22` (v1.0.0 cut separately after UAT sign-off)
**Milestone tag**: `v0.11.0` (the v0.11 milestone release tag; v1.0.0 is **Milestone tag**: `v0.11.0` (the v0.11 milestone release tag; v1.0.0 is
UAT-gated and cut separately after v0.11 completion per operator decision — UAT-gated and cut separately after v0.11 completion per operator decision —
@@ -348,7 +349,8 @@ tags: `v0.10.0`…`v0.10.21`.
- **P06** — Alloc history (REQ-071 cache DB) + `orca logs --all-nodes --since` (REQ-117) - **P06** — Alloc history (REQ-071 cache DB) + `orca logs --all-nodes --since` (REQ-117)
- **P08** — Integration tests (REQ-087) + drift-detection integration tests - **P08** — Integration tests (REQ-087) + drift-detection integration tests
- **P09** — Collector + aggregator (C-11, C-12, C-14) + drift-event aggregation (REQ-107, D-237) - **P09** — Collector + aggregator (C-11, C-12, C-14) + drift-event aggregation (REQ-107, D-237)
- **P10** — Transactional plane (REQ-075, REQ-079; C-09) + drift detection (R-018/R-019/R-020; REQ-103..REQ-113) - **P10a** — Transactional plane (REQ-075, REQ-079; C-09; C-23)
- **P10b** — Drift detection (R-018/R-019/R-020; REQ-103..REQ-113)
- **P11** — Job lint (REQ-084) - **P11** — Job lint (REQ-084)
- **P13** — ns subcommands + deprecation warnings (REQ-068) - **P13** — ns subcommands + deprecation warnings (REQ-068)
- **P14a/b/c** — Migration (REQ-066, REQ-065, REQ-086; C-07, C-13) + `orca upgrade` (REQ-115) + `orca cluster rotate-lead` (REQ-114) - **P14a/b/c** — Migration (REQ-066, REQ-065, REQ-086; C-07, C-13) + `orca upgrade` (REQ-115) + `orca cluster rotate-lead` (REQ-114)
@@ -369,7 +371,7 @@ tags: `v0.10.0`…`v0.10.21`.
- **wasmtime CGO breaks cross-compile** (mitigation: C-01 spike; fallback to podman/process primary) - **wasmtime CGO breaks cross-compile** (mitigation: C-01 spike; fallback to podman/process primary)
- **bash control plane drift** (mitigation: C-15..C-18 render-format contract + bats gate) - **bash control plane drift** (mitigation: C-15..C-18 render-format contract + bats gate)
- **daemon cutover orphans running allocs** (mitigation: P14b split; test adoption) - **daemon cutover orphans running allocs** (mitigation: P14b split; test adoption)
- **27→35+ phase scope** (mitigation: C-04 resolved — operator accepted 40 phases; v0.11 stays at 23 phases with scope folded in, no new phases) - **27→35+ phase scope** (mitigation: C-04 resolved — operator accepted 40 phases; v0.11 grows to 24 phases per grill C-24 split of P10→P10a/P10b; scope folded in, no other new phases)
- **R-020 deadlock** (mitigation: `--force` flag + per-namespace scoping per Q4=A; drifted peer in ns-A doesn't block ns-B) - **R-020 deadlock** (mitigation: `--force` flag + per-namespace scoping per Q4=A; drifted peer in ns-A doesn't block ns-B)
- **P10 sizing** (mitigation: P10 is the largest phase — drift detection + txn plane; grill may split into P10a/P10b if vertical slice is too large) - **P10 sizing** (mitigation: P10 is the largest phase — drift detection + txn plane; grill may split into P10a/P10b if vertical slice is too large)
- **Ingress default migration** (mitigation: `orca upgrade` [REQ-115] handles Traefik binding cutover from `:443` to `127.0.0.1:8443` for existing v0.9/v0.10 clusters) - **Ingress default migration** (mitigation: `orca upgrade` [REQ-115] handles Traefik binding cutover from `:443` to `127.0.0.1:8443` for existing v0.9/v0.10 clusters)