Files

T

Jon Chery 9897df04b2 fix(P52): add regression-class VERIFY (D-091) — catches capability decay

The prior VERIFY stage was diff-scoped: it checked the phase diff only
and never re-ran underlying platform capability. This structural defect
(D-091) let 8 NFR-patch phases (v1.9.1-v1.9.8, deck rework) pass VERIFY
while the platform they described decayed underneath.

Phase 52 remediation:
- core/regression_verify.py: regression-class VERIFY with 10 seeded
  local-tier capability checks (CAP-001..CAP-010). Tags each
  Verified/Decayed/Broken; fails closed on any non-Verified.
- scripts/run_regression.sh: shell wrapper; writes
  .ciagent/REGRESSION_REPORT.{md,json}; exits non-zero on decay.
- tests/test_verify_regression_mode.py: 11 tests (8 fast + 3 slow).
  Confirms the gate catches decay (fails closed) and that regression
  mode is additive (diff-scoped VERIFY behavior preserved).
- pyproject.toml: slow marker registered; run_ci.sh excludes slow
  tests to avoid recursion.

Verified: 502 fast tests pass (was 493 at v1.9; +9 new). 3 slow
integration tests pass. run_regression.sh reports all 10 seeded
local-tier capabilities Verified against current code. The
decay-surfacing test injects a broken cloud-backed check and confirms
the run tags it Broken and fails closed.

Cloud-backed capability re-verification (live ECS, DynamoDB writes,
Lambda invocation) lands in Phase 54 (D-093).

---ci---
project: acdl
phase: 52
milestone: v1.10
status: verify
requirements:
  covered: [REQ-112]
  partial: []
decisions: [D-091]
regression:
  - { capability: CAP-001, status: Verified }
  - { capability: CAP-002, status: Verified }
  - { capability: CAP-003, status: Verified }
  - { capability: CAP-004, status: Verified }
  - { capability: CAP-005, status: Verified }
  - { capability: CAP-006, status: Verified }
  - { capability: CAP-007, status: Verified }
  - { capability: CAP-008, status: Verified }
  - { capability: CAP-009, status: Verified }
  - { capability: CAP-010, status: Verified }
---/ci---

2026-07-27 17:29:52 +00:00

2.9 KiB

Raw Blame History

Phase 52 — Verify (v1.10) — Pipeline Regression-VERIFY Fix

Structural

core/regression_verify.py — new module implementing regression-class VERIFY (D-091). 10 seeded local-tier capability checks (CAP-001..CAP-010).
scripts/run_regression.sh — shell wrapper invoking the module; writes .ciagent/REGRESSION_REPORT.md + .json; exits non-zero on any non-Verified capability (fails closed).
tests/test_verify_regression_mode.py — 11 tests (8 fast + 3 slow).
pyproject.toml — slow marker registered; run_ci.sh excludes slow tests to avoid recursion.
Existing diff-scoped VERIFY artifacts (run_ci.sh, run_platform.sh, .ciagent/VERIFY.md per-phase record) preserved unchanged in behavior. PASS.

Behavioral

pytest tests/ -m "not slow": 502 passed, 3 deselected (was 493 at v1.9; +9 new fast tests). No regressions.
pytest tests/test_verify_regression_mode.py -m slow: 3 passed (integration: seeded registry runs honestly; regression mode is additive; decay-surfacing confirms the gate fails closed).
bash scripts/run_regression.sh: all 10 seeded local-tier capabilities Verified against current code; gate passes; report written to .ciagent/REGRESSION_REPORT.{md,json}.
Decay-surfacing test (test_regression_surfaces_decay_when_seeded_with_broken_check) injects a deliberately-broken cloud-backed check and confirms the run tags it Broken and fails closed. PASS.

Security

No new credentials, network calls, or cloud mutations introduced.
The regression module runs subprocess checks in the local shell only; the live-AWS tier is deferred to Phase 54 (D-093).
No secrets logged; subprocess output is truncated to 200/300 chars in report detail fields. PASS.

Quality

test_verify_regression_mode.py covers: all-Verified passes; one-Decayed blocks; one-Broken blocks; check-raising is Broken; report serialization; md+json output; broken-subprocess is Broken; missing-executable is Broken; seeded registry runs honestly; regression mode is additive (diff-scoped behavior preserved); decay-surfacing (gate fails closed on injected Broken). PASS.

Verdict

VERIFY PASS — regression-class VERIFY (D-091) implemented and tested. The gate catches decay (fails closed). Existing diff-scoped VERIFY behavior preserved. 502 offline tests pass; no AWS required for Phase 52.

Diff-scoped VERIFY defect (recorded for traceability)

The prior VERIFY stage was diff-scoped: it checked the phase diff only and never re-runs underlying platform capability. This let 8 NFR-patch phases (v1.9.1→v1.9.8, deck rework) pass VERIFY while the platform they described decayed underneath. The defect is recorded as D-091 and remediated by core/regression_verify.py. The regression run is now a milestone-completion gate (D-091). Cloud-backed capability re-verification (live ECS, DynamoDB writes, Lambda invocation) lands in Phase 54 (D-093).

2.9 KiB Raw Blame History