verify(P04): gaps_found — P0 docstring fixes; anchor->grader shipment is a P1 gap

Trivial P0 fixes (this commit): two docstrings claimed things the wiring
does not do. variants/__init__.py said the package 'never imports agents/'
— false since Wave 2: generator.py holds the module-direct
agents.structured import (the sanctioned D-020 shared defense, same
exception as grading/engine.py); docstring now states the real boundary.
templates.py (header + RubricAnchors) claimed rubric anchors are 'used by
grading context' / 'shipped to the grader' — false in the current wiring:
GradingEngine is variant-blind (variant_seed=None; no variant lookup; no
anchor consumption; render_trace_digest takes only the digest). Docstrings
now tell the truth and name the follow-up.

Verification (four layers, evidence in the phase report):
- Structural: AST boundary audit clean — variants/ has zero api/fastapi
  imports; only sanctioned agents.structured + llm/prompts/config/store.
- Behavioral: variants slice 41/41 green; full suite 324/324 green;
  pnpm typecheck 7/7 green (forced, no cache). Live app probes confirmed
  distinct learners -> distinct statements/seeds/task_ids at the API
  level, cache hit = zero LLM calls, deterministic fallback (calls==2),
  a-5 fairness envelope test green, sha256(template|learner|milestone)
  seed derivation verified byte-exact against the spec formula.
- Security: secrets scan over the P04 diff (b52bef9..4acffac, 17 files
  +2100/-1) clean — no key/token/password assignments, no URLs, no key
  shapes. Prompt-injection surface bounded: the variant prompt carries
  only template skeleton/title/id + seeded slot values — no learner id
  or user-controlled content reaches the LLM. Empty learner_id -> 422.
- Quality: ruff clean; AI_MODEL env override verified live; tests are
  mock-only (MockProvider family, zero network imports).

Must-Haves: 5 of 6 SATISFIED. NOT satisfied: MH#4 half — anchors are
present per template and a-5-testable, but NOT shipped to the grader
prompt context (non-trivial cross-module wiring: engine + prompt
signature + lifespan ordering — grading engine is built before the
variant store exists; reported as P1, not fixed here).

---ci---
phase: 4
milestone: v0.3
status: verify
requirements:
  covered: [REQ-3-005]
  partial: [REQ-3-005]
lessons:
  - Docstrings that describe a must-have's target state ('shipped to the
    grader prompt') read as done in review — verify wiring, not words:
    grep the consumer side (grading/) before believing the producer side.
  - main.py builds GradingEngine before variant_store exists; any P4/P6
    anchor-shipment fix must reorder lifespan construction or inject the
    variant store into the engine after the fact.
  - Params distinctness is parametric (280-64908 combos per template) —
    two learners CAN draw identical params (~15% at 10 learners on the
    tightest template); distinctness is proven via seeds/task_ids/statement
    embeddings, which is what the must-have actually requires.
---/ci---
This commit is contained in:
CIAgent
2026-09-12 03:46:30 +00:00
parent 4acffac71e
commit 9ff86f9cd0
2 changed files with 18 additions and 10 deletions
@@ -1,8 +1,10 @@
"""Per-learner variant task generation — templates, generator, VariantStore (REQ-3-005).
Boundary rule (D-027): variants/ is an engine module — it never imports
api/ or agents/ (api/ composes the generator and store via DI; store.py
imports config only).
api/; its ONLY agents/ dependency is the module-direct
agents.structured import in generator.py (the sanctioned shared D-020
structured defense, same exception as grading/engine.py). api/ composes
the generator and store via DI; store.py imports config only.
CO-ORDINATION NOTE (ADD, don't REMOVE — same convention as grading/):
This __init__.py is a minimal placeholder created by the VariantStore
@@ -2,9 +2,10 @@
A `TaskTemplate` binds a competency (D-021-aligned corpus ID), a statement
skeleton with `{slot}` placeholders, typed `ParameterSlot`s, difficulty-
normalization rubric anchors (the expected feature envelope the grader
prompt receives as context — a-5 makes "same bar" testable), and starter-
file scaffolds served into the sandbox workdir (wired in P6).
normalization rubric anchors (the expected feature envelope that bounds
variant fairness in the a-5 envelope test — grader-prompt shipment is the
tracked P4 follow-up; grading is variant-blind today), and starter-file
scaffolds served into the sandbox workdir (wired in P6).
Slot sampling is PURE CODE: `random.Random(seed)` over typed slots — fully
reproducible for a given seed, independent of the LLM. The LLM only renders
@@ -64,12 +65,17 @@ class ParameterSlot(BaseModel):
class RubricAnchors(BaseModel):
"""Difficulty-normalization anchors shipped to the grader as context.
"""Difficulty-normalization anchors for the grader (a-5).
Expected FEATURE ENVELOPE (digest-space): the grading prompt receives
these so two variants of one template are held to the same bar — the
anchors bound what "comparable effort" looks like for this template
regardless of which slot values a learner drew (a-5).
Expected FEATURE ENVELOPE (digest-space): the expected effort band
for this template, so two variants of one template are held to the
same bar regardless of which slot values a learner drew. The a-5
envelope test (tests/variants/test_generator.py) binds variants to
these bands in code. Shipping them into the grader prompt context is
the P4 must-have follow-up tracked for final review: grading is
variant-blind in the current wiring (engine.py stamps
variant_seed=None), so today the anchors gate variant fairness in
tests only — not yet in the LLM prompt.
"""
model_config = ConfigDict(frozen=True)