docs(P00): grill v0.11 — 6 binding conditions (C-23..C-28), P10 split into P10a/P10b
GRILL stage: adversarial review across 6 forcing questions. Verdict: PROCEED-WITH-CONDITIONS. - C-23: orca-pull.sh distinguishes cluster-wide vs ns-scoped txns (gate P10a) - C-24: split P10 into P10a (txn plane) + P10b (drift detection); phase count 23->24; tags shift by 1 - C-25: orca upgrade post-cutover verification + rollback (gate P14a) - C-26: per-phase LoC soft ceiling ~800 (no gate, recorded) - C-27: orca upgrade creates orca system user on existing peers (gate P14a) - C-28: P15.5 two sub-waves (ingress+doctor nft, then threat model+doctor mTLS) (gate P15.5) ROADMAP updated: 24 phases, tags v0.10.0..v0.10.22. ---ci--- project: orca phase: 0 milestone: v0.11 status: grill ---/ci---
This commit is contained in:
Jon Chery
@@ -1,13 +1,16 @@
|
||||
{
|
||||
"phase": 0,
|
||||
"stage": "plan",
|
||||
"stage": "grill",
|
||||
"milestone": "v0.11",
|
||||
"milestone_slug": "production-hardening",
|
||||
"phase_role": "pre_execution",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-08-07T00:03:00Z",
|
||||
"updated_at": "2026-08-07T00:04:00Z",
|
||||
"milestone_complete": false,
|
||||
"previous_milestone": "v0.10",
|
||||
"research_docs_ingested": 5,
|
||||
"locked_decisions": {"Q1": "A", "Q2": "C", "Q3": "A", "Q4": "A", "Q5": "A"}
|
||||
"locked_decisions": {"Q1": "A", "Q2": "C", "Q3": "A", "Q4": "A", "Q5": "A"},
|
||||
"grill_verdict": "PROCEED-WITH-CONDITIONS",
|
||||
"binding_conditions": ["C-23", "C-24", "C-25", "C-26", "C-27", "C-28"],
|
||||
"phase_count": 24
|
||||
}
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
# Grill: v0.11 Production Hardening — Phase 0 Adversarial Review
|
||||
|
||||
**Status**: PROCEED-WITH-CONDITIONS. The v0.11 plan is sound; 6 binding
|
||||
conditions (C-23…C-28) gate specific phases. The plan adopts R-017…R-020
|
||||
and D-215…D-237 from 5 research docs with operator decisions Q1=A, Q2=C,
|
||||
Q3=A, Q4=A, Q5=A. The grill reviewed the plan adversarially across the
|
||||
same 9 axes as GRILL_v0.9 (vision, feasibility, scope, risk, security,
|
||||
operational, cost, competitive, exit).
|
||||
|
||||
## Forcing questions + verdicts
|
||||
|
||||
### FQ1 — R-020 deadlock with `--force` + per-ns scoping
|
||||
|
||||
**Question**: With `--force` + per-namespace scoping (Q4=A), can a single
|
||||
drifted peer still block a *cluster-wide* txn (e.g., namespace creation)?
|
||||
If yes, is the `--force` escape hatch documented in C-09's failure
|
||||
contract?
|
||||
|
||||
**Verdict**: PARTIAL-BLOCK remains for cluster-wide txns. A namespace
|
||||
*creation* txn touches all peers (the new namespace dir is created on
|
||||
every peer). If one peer is drifted, the pre-flight gate refuses the
|
||||
txn cluster-wide. `--force` overrides this, but `--force` on a
|
||||
namespace-creation txn is risky (it forces the new namespace onto a
|
||||
drifted peer without reconciling the drift first).
|
||||
|
||||
**Binding condition C-23**: `orca-pull.sh` (C-09) must distinguish
|
||||
*cluster-wide* txns from *namespace-scoped* txns. Cluster-wide txns
|
||||
require `--force` with an explicit `--i-understand-the-risk` confirmation
|
||||
(or `--yes` for non-interactive). Namespace-scoped txns use per-ns
|
||||
scoping (drifted peer in ns-A doesn't block ns-B). **Gate**: P10.
|
||||
|
||||
**Confidence**: 0.88
|
||||
|
||||
### FQ2 — P10 sizing (txn plane + drift detection in one phase)
|
||||
|
||||
**Question**: P10 now absorbs drift detection (~500 LoC Go + 150 LoC
|
||||
bash + systemd units), the largest single phase. Is this a vertical
|
||||
slice that can ship atomically, or does it need splitting (P10a txn
|
||||
plane, P10b drift)?
|
||||
|
||||
**Verdict**: SPLIT RECOMMENDED. P10 has 13 tasks spanning two distinct
|
||||
subsystems: (1) the transactional plane (T1-T2: txn bundle render, SCP,
|
||||
apply, C-09 failure contract) and (2) drift detection (T3-T13: `internal/drift/`,
|
||||
Path unit emitter, notify/remediate scripts, cadence config, pre-flight
|
||||
gate, `orca` user, NFS detection, job restart). The txn plane is a
|
||||
prerequisite for drift detection (T3's `Aggregate` reads applied txn
|
||||
manifests), so the split is clean: P10a (txn plane, T1-T2) ships first,
|
||||
P10b (drift detection, T3-T13) ships after P10a.
|
||||
|
||||
**Binding condition C-24**: Split P10 into P10a (transactional plane,
|
||||
REQ-075/079, C-09) and P10b (drift detection, R-018/R-019/R-020,
|
||||
REQ-103..113). P10a ships first; P10b depends on P10a. Tags: P10a
|
||||
`v0.10.12`, P10b `v0.10.13`. All subsequent phase tags shift by 1
|
||||
(P11→`v0.10.14`, …, P16→`v0.10.22`). **Phase count: 23 → 24.**
|
||||
|
||||
**Confidence**: 0.92
|
||||
|
||||
### FQ3 — Ingress default migration path (C1)
|
||||
|
||||
**Question**: Existing v0.9/v0.10 clusters run Traefik on `:443`. R-017
|
||||
makes `127.0.0.1:8443` + nft the default. What's the upgrade path? Does
|
||||
`orca upgrade` (Q2=C) handle the binding cutover, or is it a manual
|
||||
operator step?
|
||||
|
||||
**Verdict**: UPGRADE HANDLES IT, but with a safety check. `orca upgrade`
|
||||
(REQ-115, P14a) is the thin wrapper (C2=a) that handles the Traefik
|
||||
binding cutover. The cutover is: (1) emit new Traefik static config with
|
||||
`127.0.0.1:8443`, (2) emit `/etc/nftables.d/orca.nft` with DNAT, (3)
|
||||
`systemctl reload traefik` + `nft -f`, (4) verify `curl :443` still
|
||||
routes. If step 4 fails, rollback to `:443` + remove nft rules.
|
||||
|
||||
**Binding condition C-25**: `orca upgrade` (REQ-115) must include a
|
||||
post-cutover verification step (`curl -k https://localhost:443/` returns
|
||||
200 from Traefik) with automatic rollback on failure. Document the
|
||||
rollback procedure in `docs/ingress.md`. **Gate**: P14a.
|
||||
|
||||
**Confidence**: 0.90
|
||||
|
||||
### FQ4 — Scope ceiling (LoC vs phase count)
|
||||
|
||||
**Question**: v0.11 stays at 23 phases (now 24 with C-24), but P09/P10
|
||||
(now P10a/P10b)/P15.5 grow substantially. Is the *phase count* the right
|
||||
ceiling, or should there be a *LoC/effort* ceiling per phase?
|
||||
|
||||
**Verdict**: LOOSE LoC CEILING. Phase count is a proxy for effort, but
|
||||
P10b (drift detection) is ~650 LoC across Go + bash + systemd — at the
|
||||
upper end of what a single-phase vertical slice can handle. The grill
|
||||
recommends a soft LoC ceiling of ~800 LoC per phase (Go + bash + config),
|
||||
with splitting required above ~1200 LoC.
|
||||
|
||||
**Binding condition C-26**: Per-phase LoC soft ceiling: ~800 LoC (Go +
|
||||
bash + config). Split required above ~1200 LoC. P10b (~650 LoC) is within
|
||||
the soft ceiling; P15.5 (~400 LoC: nft emitter 200 + doctor mTLS 100 +
|
||||
threat model doc) is within. No action required for v0.11; recorded for
|
||||
future milestones. **No gate.**
|
||||
|
||||
**Confidence**: 0.85
|
||||
|
||||
### FQ5 — `orca` system user on peers (operational impact)
|
||||
|
||||
**Question**: Creating a system user on every peer is a new operational
|
||||
requirement. Does this break any existing v0.9/v0.10 deployment that
|
||||
runs as root or as an existing service account?
|
||||
|
||||
**Verdict**: NO BREAK for existing deployments; NEW requirement for drift
|
||||
detection. The `orca` system user (REQ-111) is created at peer setup
|
||||
(`orca node join` / peer-setup script). Existing v0.9/v0.10 peers don't
|
||||
have the `orca` user, so drift detection's systemd Path units (which run
|
||||
as `User=orca`) won't start until the user is created. `orca upgrade`
|
||||
(REQ-115) must create the `orca` user on existing peers as part of the
|
||||
v0.11 migration.
|
||||
|
||||
**Binding condition C-27**: `orca upgrade` (REQ-115, P14a) must create
|
||||
the `orca` system user on existing peers (`useradd -r orca` idempotent)
|
||||
before P10b's drift detection can function. Document this as a
|
||||
migration prerequisite. **Gate**: P14a.
|
||||
|
||||
**Confidence**: 0.91
|
||||
|
||||
### FQ6 — P15.5 is now a mega-phase (threat model + ingress + doctor mTLS)
|
||||
|
||||
**Question**: P15.5 was originally "threat model + security review" (C-19).
|
||||
It now absorbs ingress hybrid (R-017; REQ-099..102, ~400 LoC) + `orca
|
||||
doctor mTLS` (REQ-118). Is this too much for one phase?
|
||||
|
||||
**Verdict**: MANAGEABLE but at the ceiling. P15.5 is now ~500 LoC (nft
|
||||
emitter 200 + doctor mTLS 100 + threat model doc + tests). The ingress
|
||||
hybrid and threat model are related (both are security-hardening), so
|
||||
keeping them together is defensible. The `orca doctor mTLS` (REQ-118)
|
||||
is small and reuses P01/P01.5 infrastructure. The grill recommends
|
||||
keeping P15.5 as one phase but splitting the *work* into two sub-waves
|
||||
within the phase: (1) ingress hybrid + doctor nft, (2) threat model +
|
||||
doctor mTLS.
|
||||
|
||||
**Binding condition C-28**: P15.5 commits in two sub-waves: (1) ingress
|
||||
hybrid (REQ-099..102) + `orca doctor nft` (REQ-101), (2) threat model
|
||||
(C-19) + `orca doctor mTLS` (REQ-118). Both ship under the same phase
|
||||
tag (`v0.10.20`). **No new phase; internal ordering only.**
|
||||
|
||||
**Confidence**: 0.89
|
||||
|
||||
## Binding conditions summary
|
||||
|
||||
| ID | Condition | Gate | Verification |
|
||||
|----|-----------|------|--------------|
|
||||
| C-23 | `orca-pull.sh` distinguishes cluster-wide vs namespace-scoped txns; cluster-wide requires `--force` + `--i-understand-the-risk` (or `--yes`) | P10a | Test: cluster-wide txn refused without `--force`; ns-scoped txn blocks only the drifted ns |
|
||||
| C-24 | Split P10 into P10a (txn plane, REQ-075/079, C-09) + P10b (drift detection, R-018/R-019/R-020, REQ-103..113); P10b depends on P10a; tags shift by 1 | P10a→P10b | Plan shows P10a + P10b as separate phases; P10b tasks reference P10a txn manifests |
|
||||
| C-25 | `orca upgrade` (REQ-115) includes post-cutover verification (`curl -k https://localhost:443/` returns 200) with automatic rollback on failure; rollback documented in `docs/ingress.md` | P14a | Test: cutover succeeds → 200; cutover fails → rollback to `:443` |
|
||||
| C-26 | Per-phase LoC soft ceiling: ~800 LoC (Go + bash + config); split required above ~1200 LoC | (no gate) | Recorded for future milestones |
|
||||
| C-27 | `orca upgrade` (REQ-115) creates `orca` system user on existing peers before P10b drift detection can function | P14a | Test: existing peer without `orca` user → `orca upgrade` creates it → drift detection starts |
|
||||
| C-28 | P15.5 commits in two sub-waves: (1) ingress hybrid + doctor nft, (2) threat model + doctor mTLS; same phase tag | P15.5 | Commits show two sub-waves; both under `v0.10.20` |
|
||||
|
||||
## Phase challenge summary
|
||||
|
||||
| PC | Phase | Challenge | Resolution |
|
||||
|----|-------|-----------|------------|
|
||||
| PC-11 | P10a/P10b | Txn plane + drift detection too large for one phase | Split per C-24; P10a ships first, P10b depends on it |
|
||||
| PC-12 | P15.5 | Mega-phase (threat model + ingress + doctor mTLS) | Keep as one phase; two sub-waves per C-28 |
|
||||
| PC-13 | P14a | `orca upgrade` handles 3 migrations (data + binding + orca user) | All three land in P14a per C-25, C-27; thin wrapper (C2=a) |
|
||||
| PC-14 | P09 | Aggregator extension depends on P10b drift detection | P09 in Wave 6 (after Wave 5 P10b); aggregator extension (REQ-107) only works once drift events exist |
|
||||
|
||||
## Overall verdict
|
||||
|
||||
**PROCEED-WITH-CONDITIONS**. The v0.11 plan is sound. 6 binding conditions
|
||||
(C-23…C-28) gate specific phases. The plan grows from 23 → 24 phases
|
||||
(C-24 splits P10 into P10a/P10b). All other phases are unchanged in
|
||||
count; their scope expands per the research folding (Q2=C, Q3=A).
|
||||
|
||||
The grill's confidence in the v0.11 plan is high (avg 0.89 across FQs).
|
||||
The primary risks (P10 sizing, R-020 deadlock, ingress migration) are
|
||||
all gated with verifiable conditions.
|
||||
+14
-12
@@ -322,16 +322,17 @@ ingress into P15.5).
|
||||
- [ ] Phase P07: Recovery (`orca restore`) — tag `v0.10.9`
|
||||
- [ ] Phase P08: Integration tests — expand hermetic harness (REQ-087) + **drift-detection integration tests (auto-remediation, NFS fallback, cooldown, secret exclusion)** — tag `v0.10.10`
|
||||
- [ ] Phase P09: Collector + aggregator (opt-in; **gates C-11, C-12, C-14**) + **drift-event aggregation extension (REQ-107, D-237)** — tag `v0.10.11`
|
||||
- [ ] Phase P10: Transactional plane (REQ-075, REQ-079; **gate C-09**) + **drift detection (R-018/R-019/R-020, REQ-103..REQ-113; `orca drift` CLI, systemd Path unit emitter, `orca-drift-notify.sh`, `orca-remediate.sh`, cadence config, `--force`+per-ns gate, `orca` system user, NFS detection)** — tag `v0.10.12`
|
||||
- [ ] Phase P11: `orca job lint` (REQ-084) — tag `v0.10.13`
|
||||
- [ ] Phase P12: `orca job verify` (dry-run txn through lead) — tag `v0.10.14`
|
||||
- [ ] Phase P13: `orca ns` subcommands (full surface) + deprecation warnings (REQ-068) — tag `v0.10.15`
|
||||
- [ ] Phase P14a: v0.8→v1.0 data migration (REQ-066; **gate C-07**) + **`orca upgrade --to-vX` (REQ-115, thin wrapper, handles R-017 binding cutover)** — tag `v0.10.16`
|
||||
- [ ] Phase P14b: Daemon cutover + running-allocation adoption + **`orca cluster rotate-lead` (REQ-114)** — tag `v0.10.17`
|
||||
- [ ] Phase P14c: Mixed-version tolerance + no-orca-on-server enforcement (REQ-065, REQ-086; implements C-13) — tag `v0.10.18`
|
||||
- [ ] Phase P15: README quickstart (REQ-089; **Nomad-inspired framing per Q5=A, honest-trade-offs table from research doc 3**) — tag `v0.10.19`
|
||||
- [ ] Phase P15.5: Threat model + security review (**gate C-19**) + **ingress hybrid (R-017; nft emitter REQ-099, Traefik binding REQ-100, `orca doctor nft` REQ-101, `orca nft` CLI REQ-102) + `orca doctor mTLS` (REQ-118)** — tag `v0.10.20`
|
||||
- [ ] Phase P16: Final review + ship + audit — **v0.11.0 milestone release** — tag `v0.10.21` (v1.0.0 cut separately after UAT sign-off)
|
||||
- [ ] Phase P10a: Transactional plane (REQ-075, REQ-079; **gate C-09**; **gate C-23** cluster-wide vs ns-scoped txn distinction) — tag `v0.10.12`
|
||||
- [ ] Phase P10b: Drift detection (R-018/R-019/R-020; REQ-103..REQ-113; `orca drift` CLI, systemd Path unit emitter, `orca-drift-notify.sh`, `orca-remediate.sh`, cadence config, `--force`+per-ns gate, `orca` system user, NFS detection) — depends on P10a — tag `v0.10.13`
|
||||
- [ ] Phase P11: `orca job lint` (REQ-084) — tag `v0.10.14`
|
||||
- [ ] Phase P12: `orca job verify` (dry-run txn through lead) — tag `v0.10.15`
|
||||
- [ ] Phase P13: `orca ns` subcommands (full surface) + deprecation warnings (REQ-068) — tag `v0.10.16`
|
||||
- [ ] Phase P14a: v0.8→v1.0 data migration (REQ-066; **gate C-07**; **gate C-25** post-cutover verification + rollback; **gate C-27** orca user creation) + **`orca upgrade --to-vX` (REQ-115, thin wrapper, handles R-017 binding cutover)** — tag `v0.10.17`
|
||||
- [ ] Phase P14b: Daemon cutover + running-allocation adoption + **`orca cluster rotate-lead` (REQ-114)** — tag `v0.10.18`
|
||||
- [ ] Phase P14c: Mixed-version tolerance + no-orca-on-server enforcement (REQ-065, REQ-086; implements C-13) — tag `v0.10.19`
|
||||
- [ ] Phase P15: README quickstart (REQ-089; **Nomad-inspired framing per Q5=A, honest-trade-offs table from research doc 3**) — tag `v0.10.20`
|
||||
- [ ] Phase P15.5: Threat model + security review (**gate C-19**; **gate C-28** two sub-waves) + **ingress hybrid (R-017; nft emitter REQ-099, Traefik binding REQ-100, `orca doctor nft` REQ-101, `orca nft` CLI REQ-102) + `orca doctor mTLS` (REQ-118)** — tag `v0.10.21`
|
||||
- [ ] Phase P16: Final review + ship + audit — **v0.11.0 milestone release** — tag `v0.10.22` (v1.0.0 cut separately after UAT sign-off)
|
||||
|
||||
**Milestone tag**: `v0.11.0` (the v0.11 milestone release tag; v1.0.0 is
|
||||
UAT-gated and cut separately after v0.11 completion per operator decision —
|
||||
@@ -348,7 +349,8 @@ tags: `v0.10.0`…`v0.10.21`.
|
||||
- **P06** — Alloc history (REQ-071 cache DB) + `orca logs --all-nodes --since` (REQ-117)
|
||||
- **P08** — Integration tests (REQ-087) + drift-detection integration tests
|
||||
- **P09** — Collector + aggregator (C-11, C-12, C-14) + drift-event aggregation (REQ-107, D-237)
|
||||
- **P10** — Transactional plane (REQ-075, REQ-079; C-09) + drift detection (R-018/R-019/R-020; REQ-103..REQ-113)
|
||||
- **P10a** — Transactional plane (REQ-075, REQ-079; C-09; C-23)
|
||||
- **P10b** — Drift detection (R-018/R-019/R-020; REQ-103..REQ-113)
|
||||
- **P11** — Job lint (REQ-084)
|
||||
- **P13** — ns subcommands + deprecation warnings (REQ-068)
|
||||
- **P14a/b/c** — Migration (REQ-066, REQ-065, REQ-086; C-07, C-13) + `orca upgrade` (REQ-115) + `orca cluster rotate-lead` (REQ-114)
|
||||
@@ -369,7 +371,7 @@ tags: `v0.10.0`…`v0.10.21`.
|
||||
- **wasmtime CGO breaks cross-compile** (mitigation: C-01 spike; fallback to podman/process primary)
|
||||
- **bash control plane drift** (mitigation: C-15..C-18 render-format contract + bats gate)
|
||||
- **daemon cutover orphans running allocs** (mitigation: P14b split; test adoption)
|
||||
- **27→35+ phase scope** (mitigation: C-04 resolved — operator accepted 40 phases; v0.11 stays at 23 phases with scope folded in, no new phases)
|
||||
- **27→35+ phase scope** (mitigation: C-04 resolved — operator accepted 40 phases; v0.11 grows to 24 phases per grill C-24 split of P10→P10a/P10b; scope folded in, no other new phases)
|
||||
- **R-020 deadlock** (mitigation: `--force` flag + per-namespace scoping per Q4=A; drifted peer in ns-A doesn't block ns-B)
|
||||
- **P10 sizing** (mitigation: P10 is the largest phase — drift detection + txn plane; grill may split into P10a/P10b if vertical slice is too large)
|
||||
- **Ingress default migration** (mitigation: `orca upgrade` [REQ-115] handles Traefik binding cutover from `:443` to `127.0.0.1:8443` for existing v0.9/v0.10 clusters)
|
||||
|
||||
Reference in New Issue
Block a user