From 9a005f5c64991e0eeb5b01ce946aeb1d012e0ac6 Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:42:35 +0000 Subject: [PATCH 1/8] =?UTF-8?q?docs(init):=20validate=20specification=20?= =?UTF-8?q?=E2=80=94=20milestone=20v0.2-mvp?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- phase: 0 milestone: v0.2 status: specify message: >- Branch gate ran: milestone/v0.2-mvp created from main (v0.1 complete and owner-approved, checkpoint cleared), phase/00-pre-execution checked out. Specification validated: PROJECT.md two-milestone objective, REQUIREMENTS.md REQ-01..34 (MVP) defined and traceable, milestone v0.2-mvp established in config.json (feature, tag line v0.2.x). Checkpoint written at stage specify. requirements: covered: [] partial: [] ---/ci--- --- .ciagent/CHECKPOINT.json | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 .ciagent/CHECKPOINT.json diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json new file mode 100644 index 0000000..3752018 --- /dev/null +++ b/.ciagent/CHECKPOINT.json @@ -0,0 +1,8 @@ +{ + "phase": 0, + "stage": "specify", + "milestone": "v0.2", + "phase_role": "pre_execution", + "attempts": 0, + "updated_at": "2026-09-27T10:20:00Z" +} \ No newline at end of file -- 2.43.0 From e79102b90167a14d552185976c30dbb8c7bf4e58 Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:43:05 +0000 Subject: [PATCH 2/8] =?UTF-8?q?decision(P00):=20clarification=20=E2=80=94?= =?UTF-8?q?=20v0.2=20MVP=20ambiguities=20resolved=20(D-007..D-020)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- phase: 0 milestone: v0.2 status: clarify decisions: - id: D-007 decision: Backend layout = backend/ package at repo root, src-layout researchx with config/, methodologies/, adapters/, engine/, evidence/, trust/, api/ per ARCHITECTURE §Components rationale: Matches ARCHITECTURE.md component map 1:1; src-layout avoids import shadowing; uv manages the venv + Python 3.12 confidence: 0.95 alternatives: [flat single-package, api-first layout] - id: D-008 decision: Provision uv + uv-managed Python 3.12 (official installer, user-local ~/.local/bin); system Python is 3.11.2 rationale: REQ-33 mandates uv-managed Python 3.12; uv fetches a managed CPython so the system interpreter is irrelevant confidence: 0.9 alternatives: [system 3.11 venv + pip (deviation from spec)] - id: D-009 decision: Backend serves on 127.0.0.1:8000; frontend dev on :3000 calls backend via NEXT_PUBLIC_API_BASE_URL=http://127.0.0.1:8000; FastAPI CORS allows localhost:3000 rationale: FastAPI/uvicorn defaults for a local-first single-user tool; frontend env var keeps the ApiClient boundary intact confidence: 0.85 alternatives: [single-origin reverse proxy, fixed port 3001] - id: D-010 decision: Run IDs = 8-char nanoid-style lowercase alnum slug (e.g. r-3f9k2q7z), generated by the API layer; storage layout = /runs// rationale: Human-referenceable in URLs and folders; nanoid avoids UUID's file-name collisions with 8 chars at 62-alphabet entropy (~10^14) confidence: 0.8 alternatives: [uuid4 hex, monotonic counter] - id: D-011 decision: Ollama endpoints resolve from secrets scopes: base URLs via OLLAMA_BASE_URL (default https://ollama.com/v1) and fallback http://localhost:11434/v1; API key via OLLAMA_API_KEY from .ciagent/.env.secrets only rationale: config.json secrets.model scope already declares OLLAMA_API_KEY + OLLAMA_BASE_URL; env-file secrets-loader is the only resolution path per ship/verify credential rules confidence: 0.9 alternatives: [config.json fields, interactive setup] - id: D-012 decision: DuckDuckGo keyless fallback = HTML-lite endpoint (html.duckduckgo.com/html) scraped conservatively (top results, title+url+snippet); Tavily remains the default search adapter via TAVILY_API_KEY rationale: Keyless fallback must need zero credentials; the html endpoint requires none and is the standard keyless DDG surface; Tavily default per REQ-09 confidence: 0.75 alternatives: [duckduckgo_search python lib, searx instance] - id: D-013 decision: OpenAPI client generation = openapi-typescript (types) + openapi-fetch (typed fetch client) into frontend/src/lib/api/generated/, bound behind the existing ApiClient interface in P3 rationale: Zero-runtime overhead, TS-strict, produces a typed fetch function that wraps cleanly behind ApiClient; avoids heavyweight SDK codegen confidence: 0.8 alternatives: [orval, @openapitools/openapi-generator-cli] - id: D-014 decision: SSE transport = GET /api/runs/{id}/events via sse-starlette EventSourceResponse; typed event bus is asyncio.Queue-fed with pydantic event models (run_started, query_fired, source_visited, passage_extracted, state_changed, run_done, run_error) rationale: sse-starlette is the maintained FastAPI SSE standard; pydantic events give the typed bus REQ-16 demands; matches the MockApiClient replay contract already in the frontend confidence: 0.85 alternatives: [websockets, polling] - id: D-015 decision: Quote-anchoring fuzzy match = normalized token-subset + difflib SequenceMatcher ratio >= 0.85 as the primary check; exact substring match short-circuits; mismatch flagged in the ledger, never silently dropped rationale: difflib is stdlib (zero deps), 0.85 threshold balances recall/precision on quoted prose; flagged mismatches preserve trust-loop transparency confidence: 0.75 alternatives: [rapidfuzz, embedding similarity] - id: D-016 decision: URL liveness pass = HTTP HEAD/GET (follow redirects, 10s timeout) classifying 200-299=live, 3xx-stale-after-redirect-limit/stale-content=stale, 4xx/5xx/timeout/dead-url=dead; runs post-run per REQ-19; recheck at export rationale: Simple, deterministic classification REQ-19 names live/stale/dead; single GET with redirect following captures the common moved-page case confidence: 0.8 alternatives: [HEAD-only, wayback fallback] - id: D-017 decision: Engine durability = JSON state file per run (runs//state.json) rewritten atomically (tmp+rename) after every state transition + EventBus subscribers receive replay on resume; asyncio tasks checkpointed via state machine position, not mid-task rationale: REQ-15 demands pause/cancel/resume across restarts; atomic rewrite prevents torn files; transitions are natural checkpoint boundaries confidence: 0.85 alternatives: [sqlite, append-only journal] - id: D-018 decision: Run termination semantics for THIS pipeline run: phases ship P1..P4 + final P5; the milestone → main merge HALTS at the owner approval gate (merge_to_main escalation hook) exactly as v0.1 did — the run presents v0.2.5 and stops rationale: Owner-approved run directive (this session): 'P0–P5, halt at owner gate'; config.json escalation_hooks includes merge_to_main; v0.1 precedent (v0.1.8 halted, owner recorded approval in a follow-up) confidence: 1.0 alternatives: [autonomous merge at full autonomy] - id: D-019 decision: Deploy policy for this run = none; no LXC staging without an explicit owner directive (deploy escalation hook) rationale: Owner-approved run directive (this session): 'No deploys without owner directive' confidence: 1.0 alternatives: [stage v0.2.x-uat at P5 gate] - id: D-020 decision: Frontend stays on the v0.1 ApiClient boundary in P1/P2 (MockApiClient), P3 adds a generated OpenAPI client + ApiClient binding switch (env-flag RESEARCHX_API_MODE=mock|live with mock default), no component rewrites (REQ-26/REQ-54) rationale: Keeps REQ-54 zero-throwaway promise explicit; live binding only activated when the backend runs, mock remains default for dev/tests confidence: 0.9 alternatives: [hard swap in P3, dual clients per component] ---/ci--- CLARIFY stage, milestone v0.2: 14 ambiguities resolved at full autonomy (defaults accepted). Key resolutions: backend/ src-layout package, uv + managed Python 3.12, run IDs as short slugs, DDG html-lite keyless fallback, openapi-typescript client gen, sse-starlette event transport, difflib quote anchoring at 0.85, JSON durable state with atomic rewrites, owner-gate halt at P5, no deploys without directive. PROJECT.md + REQUIREMENTS.md updated with the clarified constraints. --- .ciagent/CHECKPOINT.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 3752018..1d5f92a 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,8 +1,8 @@ { "phase": 0, - "stage": "specify", + "stage": "clarify", "milestone": "v0.2", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-09-27T10:20:00Z" + "updated_at": "2026-09-27T10:30:00Z" } \ No newline at end of file -- 2.43.0 From c5e9cb1b3f2d4ddc384e90beae613826ade3f282 Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:43:18 +0000 Subject: [PATCH 3/8] =?UTF-8?q?docs(P00):=20clarify=20=E2=80=94=20PROJECT.?= =?UTF-8?q?md=20+=20REQUIREMENTS.md=20updated=20with=20D-007..D-020=20reso?= =?UTF-8?q?lutions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- phase: 0 milestone: v0.2 status: clarify message: >- Clarify-stage file updates: PROJECT.md gains the v0.2 Key Decisions section (14 decisions D-007..D-020); REQUIREMENTS.md gains the clarified-constraints block binding the decisions to REQ-01..34. ---/ci--- --- .ciagent/PROJECT.md | 12 ++++++++++-- .ciagent/REQUIREMENTS.md | 7 +++++++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/.ciagent/PROJECT.md b/.ciagent/PROJECT.md index ffe8167..2ceb7c7 100644 --- a/.ciagent/PROJECT.md +++ b/.ciagent/PROJECT.md @@ -51,5 +51,13 @@ Vision finalized and locked 2026-09-26: v1 methodology composition LOCKED (8 cor - Model adapter default: Ollama cloud-first (ollama.com/v1) with keyless localhost:11434/v1 fallback on auth/connection failure - Prototype-first gate (owner): UI/UX prototype with mock data, approved before MVP work; prototype is its own milestone (v0.1) - Versioning (owner): starts at v0.1; MAJOR locked at 0 until owner-approved v1 release; MVP may span multiple 0.x minor milestones -- Prototype = production Next.js frontend with typed ApiClient/MockApiClient split — zero throwaway work; phase 4 of v0.2 swaps the mock for the OpenAPI client -- Tag-line mapping: 0.x milestones run patches on their own minor line (v0.1.x); supersedes the greenfield deviation noted at init \ No newline at end of file +- Prototype = production Next.js frontend with typed ApiClient/MockApiClient split — zero throwaway work; phase 3 of v0.2 swaps the mock for the OpenAPI client +- Tag-line mapping: 0.x milestones run patches on their own minor line (v0.1.x); supersedes the greenfield deviation noted at init + +## Key Decisions (v0.2 clarify — D-007..D-020, 2026-09-27) +- Backend layout: `backend/` repo-root package, src-layout `researchx` (config, methodologies, adapters, engine, evidence, trust, api per ARCHITECTURE §Components); uv + uv-managed Python 3.12 (D-007/D-008) +- Ports/binding: backend 127.0.0.1:8000, frontend :3000 via NEXT_PUBLIC_API_BASE_URL; CORS localhost:3000 (D-009); run IDs = 8-char slugs; storage = /runs// (D-010) +- Adapters: Ollama base URLs + key via env-file secrets scopes (OLLAMA_BASE_URL, OLLAMA_API_KEY); DDG keyless fallback = html.duckduckgo.com/html conservative scrape (D-011/D-012) +- Integration: openapi-typescript + openapi-fetch → frontend/src/lib/api/generated, bound behind ApiClient; RESEARCHX_API_MODE=mock|live, mock default (D-013/D-020) +- Transport/trust: sse-starlette SSE + pydantic typed event bus (D-014); difflib quote anchoring ≥ 0.85 (D-015); HTTP liveness live/stale/dead (D-016); JSON durable state, atomic rewrite per transition (D-017) +- Run termination: pipeline ships P1..P4 + P5 final patch, then HALTS at the owner approval gate before milestone → main (merge_to_main hook, v0.1 precedent); no deploys without owner directive (D-018/D-019) \ No newline at end of file diff --git a/.ciagent/REQUIREMENTS.md b/.ciagent/REQUIREMENTS.md index 394eb4d..00b8d02 100644 --- a/.ciagent/REQUIREMENTS.md +++ b/.ciagent/REQUIREMENTS.md @@ -73,6 +73,13 @@ - **REQ-33**: Monorepo: uv-managed backend (ruff, mypy, pytest) + Next.js frontend (ESLint, strict TS, Vitest); one-command install/dev - **REQ-34**: Full export round-trip: report + ledger + citations survive as real links +### Clarified constraints (v0.2 P0 clarify — D-007..D-020) +- **D-007/D-008**: backend/ src-layout package `researchx` (config, methodologies, adapters, engine, evidence, trust, api); uv + managed Python 3.12 +- **D-009/D-010**: backend 127.0.0.1:8000, CORS localhost:3000; run IDs 8-char slugs; runs// storage layout +- **D-011/D-012**: model/search endpoints + keys resolve from env-file secrets scopes; DDG keyless fallback via html.duckduckgo.com/html scrape +- **D-013/D-020**: openapi-typescript + openapi-fetch generated client bound behind ApiClient; RESEARCHX_API_MODE=mock|live, mock default until P3 +- **D-014..D-017**: sse-starlette SSE + pydantic event bus; difflib quote anchoring ≥ 0.85; HTTP liveness live/stale/dead; JSON durable state atomically rewritten per transition + ## V2 (Post-MVP backlog) ### Methodology Catalog Extension -- 2.43.0 From 5765b605ec806f774a6ec11e270845156bf4cff3 Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:51:44 +0000 Subject: [PATCH 4/8] docs(P00): research findings + persona assessment for v0.2 ---ci--- phase: 0 milestone: v0.2 status: research message: >- RESEARCH stage complete. RESEARCH-v02.md records 8 topic areas (uv/FastAPI foundation, Ollama endpoints, Tavily/DDG search, SSE + durability, state machine patterns, quote anchoring + liveness, OpenAPI client gen, monorepo ergonomics) with 14 conclusions consumed by PLAN and 4 refinements to D-007..D-020 (hatchling packages key, openapi.json snapshot discipline, single-worker uvicorn, structured-output smoke-test detection). PERSONAS.md rewritten for v0.2: backend-engineer + data-engineer activated, frontend-engineer retained (P3 client swap + P4 E2E), phase-specific security-engineer added for P2 trust loop (deactivates after P2). ---/ci--- --- .ciagent/CHECKPOINT.json | 2 +- .ciagent/PERSONAS.md | 62 +++++++++-------- .ciagent/RESEARCH-v02.md | 140 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 177 insertions(+), 27 deletions(-) create mode 100644 .ciagent/RESEARCH-v02.md diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 1d5f92a..815578a 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,6 +1,6 @@ { "phase": 0, - "stage": "clarify", + "stage": "research", "milestone": "v0.2", "phase_role": "pre_execution", "attempts": 0, diff --git a/.ciagent/PERSONAS.md b/.ciagent/PERSONAS.md index 7db4f49..a2f0513 100644 --- a/.ciagent/PERSONAS.md +++ b/.ciagent/PERSONAS.md @@ -1,46 +1,56 @@ -# Personas — Milestone v0.1 (Prototype) +# Personas — Milestone v0.2 (MVP) --- active: + - name: backend-engineer + - name: data-engineer - name: frontend-engineer - name: ci-lead-developer -phase_specific: [] -reason: "v0.1 is a pure frontend prototype (Next.js, mock data). Only the frontend domain has work; backend/data personas activate in v0.2. Lead-developer coordinates decomposition and enforces the ApiClient boundary." +phase_specific: + - name: security-engineer + phases: [2] +reason: "v0.2 is the MVP: FastAPI backend + engine + adapters + trust loop (backend/data domains activate), the frontend consumes the OpenAPI-generated client in P3 (frontend stays active for the binding swap + P4 E2E), and the trust loop in P2 needs a phase-specific security-engineer for secrets handling and liveness verification. Lead-developer coordinates." --- ## Roster +### backend-engineer +- **Domain**: backend +- **Active**: true +- **Phase-specific**: false +- **Frameworks**: fastapi, pydantic v2, uv, uvicorn, sse-starlette, httpx, pytest, pytest-asyncio +- **Constraints**: src-layout package `researchx` in backend/; all external I/O lives in adapters — engine imports no vendor SDKs (httpx only inside adapters); no blocking calls in async paths; durable state via atomic tmp+os.replace; never swallow CancelledError; API layer is transport only (no business logic); single uvicorn worker (single-user) +- **Territory**: `backend/**` + +### data-engineer +- **Domain**: data +- **Active**: true +- **Phase-specific**: false +- **Frameworks**: pydantic v2 (schemas), pytest +- **Constraints**: owns the pydantic schema contracts: dial schema (REQ-01), run state, evidence store models (stable passage IDs), citation ledger JSON schema (REQ-21); pure data + validation — no I/O in schema modules; every run persists resolved dial values (REQ-04) +- **Territory**: `backend/src/researchx/config/**`, `backend/src/researchx/evidence/**` + ### frontend-engineer - **Domain**: frontend - **Active**: true - **Phase-specific**: false -- **Frameworks**: next.js (App Router), react 19, typescript (strict), tailwind, vitest -- **Constraints**: component-first; server components by default, client components only where interactivity demands; all data access through the ApiClient interface — components never import MockApiClient directly; no secrets or env keys in the frontend; typography care per vision §4 +- **Frameworks**: next.js (App Router), react 19, typescript (strict), tailwind, vitest, openapi-fetch, openapi-typescript +- **Constraints**: components never import the mock client or the generated client directly — only the ApiClient boundary does; RESEARCHX_API_MODE=mock|live selects the binding, mock default; no secrets or env keys in the frontend (NEXT_PUBLIC_API_BASE_URL only); zero component rewrites in the P3 swap (REQ-54/REQ-26) - **Territory**: `frontend/**` +### security-engineer (phase-specific: P2) +- **Domain**: security +- **Active**: true (P2 only) +- **Phase-specific**: true +- **Frameworks**: httpx, pytest, STRIDE analysis +- **Constraints**: secrets resolve from env-file scopes only (OLLAMA_API_KEY, TAVILY_API_KEY, GITEA_TOKEN via .ciagent/.env.secrets) — never shell env, never logged, never in git; liveness verification and trust-loop code must not leak keys in errors or traces; SSRF guard for fetched URLs (private-address refusal); authority tiering must not phone home +- **Territory**: `backend/src/researchx/trust/**`, `backend/src/researchx/adapters/secrets*/**` +- **Removal**: deactivate after P2 ships + ### ci-lead-developer - **Domain**: coordination - **Active**: true - **Phase-specific**: false - **Frameworks**: ciagent workflows, git discipline -- **Constraints**: enforce persona territories; ApiClient boundary review; commit discipline with ---ci--- blocks; phase-boundary file updates only (no .ciagent/ writes during task execution) -- **Territory**: `.ciagent/**`, repo root config files - -### Deactivated (v0.1) - -#### backend-engineer -- **Domain**: backend -- **Active**: false -- **Reason**: No backend in the prototype milestone; the FastAPI backend is milestone v0.2 (blocked until prototype approval) -- **Frameworks**: fastapi, pydantic, uv (activates in v0.2) -- **Territory**: `backend/**` (does not exist yet) - -#### data-engineer -- **Domain**: data -- **Active**: false -- **Reason**: No database/schema work in v0.1; evidence store and citation ledger models arrive in v0.2; prototype uses typed fixtures -- **Frameworks**: pydantic schemas (activates in v0.2) -- **Territory**: `backend/src/researchx/evidence/**` (does not exist yet) - -### Custom personas -None required for v0.1. The security-auditor persona activates in v0.2 phase 2 (trust loop: secrets handling, liveness verification). No CLI surface exists in this milestone, so no cli-engineer. \ No newline at end of file +- **Constraints**: enforce persona territories; adapter/engine boundary review (I/O only in adapters); commit discipline with ---ci--- blocks; phase-boundary file updates only (no .ciagent/ writes during task execution); gates green before every ship +- **Territory**: `.ciagent/**`, repo root config files (README, VERSIONING, .gitignore) \ No newline at end of file diff --git a/.ciagent/RESEARCH-v02.md b/.ciagent/RESEARCH-v02.md new file mode 100644 index 0000000..16cacc3 --- /dev/null +++ b/.ciagent/RESEARCH-v02.md @@ -0,0 +1,140 @@ +# Research — v0.2 MVP (Phase 0) + +## Run: 2026-09-27 (stage: research) + +Phase-0 research findings for milestone v0.2 (MVP): FastAPI backend, custom asyncio engine, adapter layer, trust loop, OpenAPI client integration. Each topic records concrete recommendations and pitfalls that PLAN consumes. + +## 1. Backend foundation (uv + FastAPI) + +**Recommendation:** +- uv (>= 0.4) with `backend/pyproject.toml`, src-layout: `backend/src/researchx/__init__.py`, `backend/src/researchx/{config,methodologies,adapters,engine,evidence,trust,api}/__init__.py` +- `[project] name = "researchx"`, `requires-python = ">=3.12"`; deps: `fastapi>=0.115,<0.116`, `pydantic>=2.8,<3`, `uvicorn>=0.30,<1`, `sse-starlette>=2.1,<3`, `httpx>=0.27,<1` +- Dev group (`uv sync --group dev` or `[dependency-groups]`): `ruff>=0.6`, `mypy>=1.11`, `pytest>=8`, `pytest-asyncio>=0.23` +- `[build-system] hatchling`; `[tool.hatch.build.targets.wheel] packages = ["src/researchx"]` — src-layout pitfall: without the explicit packages key, uv/hatch builds fail or ship an empty wheel +- Dev run: `cd backend && uv run uvicorn researchx.api.app:app --reload --port 8000`; one-command from root documented in README +- mypy strict on `src/`: `[tool.mypy] strict = true, mypy_path = "src"`; ruff: line-length 100, target py312 + +**Pitfalls:** +- `uv run` resolves the project venv automatically — never `pip install` into it +- pytest needs `pythonpath = ["src"]` in `[tool.pytest.ini_options]` or an editable install (`uv sync` does this) — tests import `researchx.*` directly +- pytest-asyncio: use `asyncio_mode = "auto"` and `loop_scope = "function"` (the old `event_loop` fixture is removed in recent versions) + +## 2. Ollama OpenAI-compatible endpoints + +**Recommendation:** +- Cloud: `https://ollama.com/v1` with `Authorization: Bearer `; local fallback `http://localhost:11434/v1` keyless +- Chat: POST `/chat/completions` (OpenAI shape: messages, model, temperature; response choices[0].message.content) +- Probe: GET `/models` → data[].id; capability check via a tiny structured-completion smoke test at connect time +- Context cap: `min(model_claim, 8192)` tokens; model claim unknown → 8192 flat; embedding batch 8 +- Planner/writer roles = two model names from settings (default: same model both roles) + +**Pitfalls:** +- ollama.com/v1 is cloud-keyed — 401 on missing/invalid key → automatic local-fallback attempt, then surfaced error +- Some served models don't support function calling/structured outputs → REQ-12 warning path: detect via smoke-test failure, warn in UI, degrade to plain chat + prompt-embedded JSON contract +- Model names are server-specific (`q4` suffixes etc.) — settings must store the exact served name; probe lists what's actually available + +## 3. Search adapters + +**Recommendation:** +- Tavily: POST `https://api.tavily.com/search`, JSON `{"api_key": key, "query": q, "search_depth": "basic"|"advanced", "max_results": n, "include_answer": false}` → `results[]: {title, url, content, score}`; timeouts 10s +- DuckDuckGo keyless fallback: POST `https://html.duckduckgo.com/html/?q=` with a plain httpx client; parse anchors `a.result__a` (href → URL, text → title) and `a.result__snippet` from the result list; conservative: max 8 results, no retries beyond one backoff, user-agent string honest +- Adapter interface: `async search(query, max_results) -> list[SearchResult]`; registry selects Tavily when TAVILY_API_KEY present, else DDG + +**Pitfalls:** +- DDG html rate-limits aggressively — the fallback is best-effort; on 403/timeout return partial/empty with a surfaced warning, never hang the engine +- Tavily's `content` is already a snippet — evidence extraction still fetches the live page (visited-only constraint requires the visit) +- Never put the API key in a query string or log line + +## 4. FastAPI SSE + asyncio durability + +**Recommendation:** +- `sse-starlette` EventSourceResponse fed by a per-run `asyncio.Queue`; pydantic event models with a discriminated union on `type` (run_started, query_fired, source_visited, passage_extracted, state_changed, run_done, run_error) +- EventSourceResponse generator: `async for event in queue` → `{"event": , "data": }`; send on connect a `state_changed` replay of current persisted state +- Client disconnect: wrap the generator in try/finally that cancels only the SSE subscription (never the run); runs keep going when browsers disconnect +- Backpressure: unbounded queue per run is acceptable single-user; cap at 1000 with drop-oldest `state_changed` coalescing if needed + +**Pitfalls:** +- No blocking I/O in the response generator — all fetches via httpx.AsyncClient +- uvicorn single worker; multiple workers would break in-memory run registry — document single-user explicitly +- On reconnect (Last-Event-ID unused in MVP) → replay current state + live events onward; full event log persisted in the run dir anyway + +## 5. Durable state machine patterns + +**Recommendation:** +- `runs//state.json`: `{run_id, state, dial_values, plan, question, created_at, updated_at, cursor}` — rewritten atomically after every transition: write `.tmp`, `os.replace` +- Transitions only at method boundaries of the state machine; the cursor records depth-loop position for resume +- Pause = asyncio.Event cleared; cancel = task.cancel() + terminal CANCELLED state persisted; resume on restart = state file read, EventBus rebuilt, loop re-entered at cursor +- pytest-asyncio: `asyncio_mode = "auto"`; tests construct the engine with fake adapters and `await engine.run()` directly + +**Pitfalls:** +- Never swallow CancelledError — re-raise after persisting the cancel state +- Shield the state write: `with contextlib.suppress(asyncio.CancelledError): await asyncio.shield(save())` then re-raise +- Torn reads: only ever `os.replace` (atomic on same filesystem); never append — state is a snapshot, trace is a separate append-only file + +## 5b. Testing fakes (REQ-31) + +**Recommendation:** +- FakeModelAdapter: canned planner (sub-questions from a fixture), canned writer (deterministic report citing fixture passage IDs) — zero API keys +- FakeSearchAdapter: fixture corpus of 5-8 pages with overlapping passages; deterministic ordering +- Engine tests drive every transition, depth loop, pause/resume mid-depth-loop, cancel, and adversarial writer outputs + +**Pitfalls:** +- Fakes must implement the full adapter interface (no stub methods) — verify layer asserts interface conformance +- Keep fixtures small and unicode-diverse (quote anchoring edge cases) + +## 6. §6. Quote anchoring + liveness + +**Recommendation:** +- Normalize (unicode NFC → casefold → whitespace collapse) both quote and passage; exact-substring short-circuit; else difflib SequenceMatcher ratio ≥ 0.85 +- Ledger entry: `{url, visited_at, passages: [{id, quote}], claims: [claim_ids], health, authority_tier}` +- Liveness: single GET with redirect-follow (httpx, 10s timeout): 2xx live; 3xx loop/parked-domain or content-gone markers → stale; 4xx/5xx/timeout/unresolvable → dead +- ≥99% liveness gate: shipped report citations must pass liveness at export time; failures listed in the verification checklist + +**Pitlingfalls:** +- SequenceMatcher on long passages is O(n²)-ish — anchor per-passage (passages are ≤ ~500 chars), not per-document +- Fuzzy matches near threshold (0.80-0.90) flagged in the ledger as `weak_match` for transparency + +**Pitfalls:** (corrected) +- SequenceMatcher cost is quadratic-ish — always match quote vs its claimed passage only +- Near-threshold (0.80–0.90) matches flagged `weak_match` in the ledger — surfaced, not silently accepted + +## 7. OpenAPI client generation + +**Recommendation:** +- `openapi-typescript@^7` (types) + `openapi-fetch@^0.13` (typed fetch client), generated into `frontend/src/lib/api/generated/` +- Regen: `pnpm dlx openapi-typescript http://127.0.0.1:8000/openapi.json -o frontend/src/lib/api/generated/schema.d.ts` — needs the backend running, or read a committed `openapi.json` snapshot in CI +- Binding: a thin `GeneratedApiClient` implements the existing ApiClient interface by delegating to openapi-fetch with baseURL from NEXT_PUBLIC_API_BASE_URL; components never import it directly — only the boundary file does +- RESEARCHX_API_MODE=mock|live env flag selects the binding at boundary initialization; mock stays default + +**Pitfalls:** +- FastAPI operation-id stability: route order and function names determine ids — fix function names early; regen discipline = regenerate only from a committed snapshot +- FastAPI's default unions (None | X) — openapi-typescript emits optional/nullable types that must be handled at the boundary, not in components + +## 8. Monorepo ergonomics + +**Recommendation:** +- Root README one-command story: `uv sync` (backend) + `corepack pnpm install` (frontend); dev: `uv run uvicorn ...` + `corepack pnpm dev` +- Backend gates: `uv run ruff check src tests`, `uv run mypy src`, `uv run pytest` (all via `uv run` from backend/) +- Frontend gates carry over from v0.1 unchanged: `pnpm typecheck`, `pnpm lint`, `pnpm test`, `pnpm build` +- Repo-root package.json is NOT created (frontend is the only JS package); document commands in README instead + +**Pitfalls:** +- pnpm via corepack (packageManager field pins pnpm@12.6.0) — direct pnpm may be absent; always `corepack pnpm` +- openapi.json snapshot committed for reproducible client regen; regenerate the TS client when the snapshot changes + +## Research Conclusions + +1. Backend = uv + src-layout `researchx` in backend/, pyproject with hatchling `packages=["src/researchx"]`, deps fastapi/pydantic/uvicorn/sse-starlette/httpx, dev ruff/mypy/pytest/pytest-asyncio — one-command via `uv sync && uv run uvicorn researchx.api.app:app`. +2. All I/O lives in adapters; engine imports no vendor SDKs (httpx only inside adapters). +3. Ollama cloud `https://ollama.com/v1` (Bearer) → local `http://localhost:11434/v1` keyless fallback on 401/connection failure; probe via GET /models + structured-output smoke test; context cap min(claim, 8192), embed batch 8. +4. REQ-12 structured-output warning: detect via smoke test; degrade to prompt-embedded JSON contract + UI warning — never fail cryptically. +5. Tavily POST api.tavily.com/search with api_key in JSON body; DDG fallback scrapes html.duckduckgo.com/html anchors conservatively (max 8, one backoff, surfaced warnings on rate-limit). +6. SSE = sse-starlette EventSourceResponse + per-run asyncio.Queue + pydantic discriminated-union event types; state replay on connect; single uvicorn worker (single-user). +7. Durable state = runs//state.json, atomic tmp+os.replace per transition, cursor for depth-loop resume; trace is a separate append-only file; shielded state writes; never swallow CancelledError. +8. Quote anchoring = normalize + exact-substring short-circuit + SequenceMatcher ≥ 0.85 per-passage; near-threshold flagged weak_match. +9. Liveness = redirect-following GET, 10s: 2xx live / redirect-loop or content-gone stale / 4xx-5xx-timeout dead; ≥99% citation liveness gate at export. +10. Fake adapters (REQ-31) implement the full adapter interface with canned planner/writer + fixture corpus — enables the entire engine + trust test suite with zero keys. +11. Client gen = openapi-typescript@^7 + openapi-fetch@^0.13 from a committed openapi.json snapshot; GeneratedApiClient binds behind the ApiClient interface; RESEARCHX_API_MODE=mock|live selects binding; mock default. +12. Monorepo gates: backend `uv run ruff check`, `uv run mypy`, `uv run pytest`; frontend v0.1 gates unchanged; no root package.json — commands documented in README. +13. Config/status endpoints surface probe results + adapter status (REQ-25 GET /api/config) — plan the schema early so the settings surface stays honest. +14. All decisions D-007..D-020 confirmed workable; refinements: hatchling packages key (new), openapi.json snapshot discipline (new), single-worker uvicorn note (new), structured-output smoke-test detection pattern (new) — folded into PLAN as constraints. \ No newline at end of file -- 2.43.0 From 718b0c30cd1433792c7be68dca412dfd142aaddc Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:52:14 +0000 Subject: [PATCH 5/8] =?UTF-8?q?docs(P00):=20create=20phase=20plans=20?= =?UTF-8?q?=E2=80=94=20P1=20vertical-slice=20plan=20+=20ROADMAP=20v0.2=20r?= =?UTF-8?q?efresh?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- phase: 0 milestone: v0.2 status: plan message: >- PLAN stage complete: PLAN.md rewritten as the P1 (Methodology Framework + Adapter Layer) vertical-slice plan — 4 waves, 14 tasks, must_haves, REQ-01..06/08..11/31 mapping, User-Facing Surface + Happy Path + UX Acceptance Criteria authored for the MVP/UX gate. ROADMAP.md v0.2 section unblocked: P0 in progress, phase mapping clarified (REQ-07/12 live probing ships in P2), task waves anchored to PLAN.md. Plans for P2..P4 refresh at each phase boundary (v0.1 practice). ---/ci--- --- .ciagent/CHECKPOINT.json | 6 +-- .ciagent/PLAN.md | 88 ++++++++++++++++++++++++---------------- .ciagent/ROADMAP.md | 27 +++++++----- 3 files changed, 72 insertions(+), 49 deletions(-) diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 815578a..4e6cd10 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,8 +1,8 @@ { "phase": 0, - "stage": "research", + "stage": "plan", "milestone": "v0.2", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-09-27T10:30:00Z" -} \ No newline at end of file + "updated_at": "2026-09-27T11:10:00Z" +} diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index 12fabf1..6481ad6 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -1,54 +1,72 @@ -# PLAN — Milestone v0.1 Phase 8 (FINAL): Review + Approval Re-presentation + Ship +# PLAN — Milestone v0.2 Phase 1: Methodology Framework + Adapter Layer --- -phase: 8 -milestone: v0.1 -status: executed +phase: 1 +milestone: v0.2 +status: planned must_haves: - - "Full-milestone 2nd final review: PASS-WITH-FIXES — double-compose race + dead drawer-keyboard P1s fixed with regression tests (90/90); P6/P7 fixes verified holding" - - "Audit: PASS-WITH-NOTES — reconstruction, tags, file/branch/commit discipline, secrets sweep green; notes applied (ARCHITECTURE §7 + Build Order, root README /ask row, VERSIONING table, About string, package.json version)" - - "Final phase patch v0.1.8 shipped — milestone release candidate, presented at the REQ-62 owner approval gate with UAT staged per owner cadence" + - "backend/ uv project scaffolds with one command (uv sync) and gates green from the first commit (ruff, mypy strict, pytest)" + - "12-dial typed schema with safe defaults resolves; every methodology = named dial bundle + planner/writer templates + output shape; registry resolves by slug (rapid-answer, deep-report)" + - "Resolved dial values persist to run config and are auditable (REQ-04) — schema-level contract proven by test" + - "Adapter contracts (model/search/storage/memory/secrets) typed; local implementations + fake model/search adapters enable zero-API-key engine tests (REQ-31)" + - "Capability probe caps context at min(model claim, 8192), embedding batch 8; overrides surfaced via typed probe result (REQ-07)" + - "Model adapter: Ollama cloud-first with keyless localhost fallback (REQ-08); search: Tavily default + DDG keyless fallback (REQ-09); storage: local FS Markdown/JSON (REQ-10); memory stubbed off, secrets env-backed (REQ-11)" + - "Structured-output weakness warning path (REQ-12): smoke-test detection, explicit typed warning, graceful degrade" personas: + - data-engineer + - backend-engineer - ci-lead-developer -requirements: [REQ-62] +requirements: [REQ-01, REQ-02, REQ-03, REQ-04, REQ-05, REQ-06, REQ-08, REQ-09, REQ-10, REQ-11, REQ-31] --- ## User-Facing Surface -The extended prototype presented for owner approval: everything from v0.1.6 (shell, library, settings, avatar) plus the v0.1.7 ask-intelligence layer — open-question suggestions and the refine dialogue on the New Research view. Releases v0.1.0–v0.1.8; v0.1.8 is the extended-prototype release candidate. +No new UI in P1 — the frontend is untouched (mock-backed, still v0.1-complete). User-facing surface for P1 is the **backend package itself**: `uv sync && uv run uvicorn researchx.api.app:app` boots a healthless dev server skeleton (routes land in P3), and the methodology registry + adapters are exercisable from tests and a documented README quickstart section. The deliverable P1 surface ships to the user in P3 when routes + the client swap go live. -## Happy Path (owner review script) +## Happy Path (P1 — test/README script, written before execute) -1. `cd frontend && pnpm install && pnpm dev` → http://localhost:3000 → shell renders on /ask -2. Ask → question → Research this (click 1) -3. Choose → Deep Report card → Start research (click 2) -4. Plan & Run → Approve plan (click 3) → live trace → pause/resume/cancel -5. Read the report → hover [n] → exact quote; ledger; export Markdown/JSON -6. New research → click an open-question suggestion → box fills (no auto-start) → edit → Research this -7. New research → type a draft → Refine question → answer the 3 clarifying questions (skip the optional two) → Use this → question + context filled → Research this -8. Library → folder tree → stored report opens its own report; Connections panel; ⌘K palette -9. Avatar → user menu → Settings → API keys masked, methodology defaults live preview -10. Owner decision: approve → v0.1.8 published + milestone merge to main; or request changes +1. `cd backend && uv sync` — installs Python 3.12 + deps, editable researchx, dev tools +2. `uv run pytest` — full suite green with zero API keys (fake adapters drive everything, REQ-31) +3. `uv run python -c "from researchx.methodologies import registry; m = registry.get('deep-report'); print(m.dials.depth)"` → `3` — registry resolves by slug, dial bundle typed +4. `uv run python -c "from researchx.config.dials import Dials; print(Dials().model_dump())"` → all 12 dials with safe defaults +5. `uv run ruff check src tests && uv run mypy src` → clean +6. Frontend gates still green (untouched): `corepack pnpm typecheck && corepack pnpm test` from frontend/ -## What this phase did +## UX Acceptance Criteria (P1) -1. **Review** (multi-persona, P7-weighted, probe-verified): verdict PASS-WITH-FIXES. P1s: Enter at the last exchange question composed twice (stale section-level keydown guard re-ran submitAnswer after the input guard advanced — invisible with the deterministic mock, a doubled paid POST the day v0.2 wires the endpoint) and Enter on focused drawer buttons was dead (section preventDefault swallowed the click). Fixed: single Enter path (input guard owns Enter; buttons self-activate); error state gained a Keep original button (mouse-user trap). 2 regression tests (single-compose spy, Enter-on-button); suite 88 → 90. Adversarial probe: no drawer control can submit the outer form. P6/P7 fixes verified holding. -2. **Audit**: PASS-WITH-NOTES — tags v0.1.0–v0.1.7 verified (annotated, on-branch, mapped to ship records); DEPLOY-RECORD.md tracked + accurate; branch hygiene exact; commit discipline clean (---/ci--- closer holding since the P6 regression); secrets clean across history; REQ-67/68 spot-checked as genuinely delivered; gates re-run live (90/90). Notes applied: ARCHITECTURE.md §7 + Build Order extended with the ask-intelligence layer; root README /ask row updated; VERSIONING table → v0.1.8; settings About + package.json → v0.1.8. -3. **Ship (this commit sequence)**: PR #8 (phase/08 → milestone), squash-merge, tag v0.1.8, release with full milestone summary, v0.1.8-uat staged on the LXC host per owner cadence. Per REQ-62 and the prototype_approval escalation hook, the milestone merge to main does NOT proceed autonomously — the pipeline halts at the gate for the recorded owner decision. +- One-command backend setup (uv sync) with zero manual steps beyond uv itself +- Every adapter contract is typed; fakes implement the full interface — no stub methods, no TODOs +- No API key is ever required to run the test suite; no secret value ever reaches a log line, error message, or fixture +- Degradation is honest: probe results and structured-output warnings are typed data, not console noise -## UX Acceptance Criteria +## Tasks (waves — dependency-ordered, persona-assigned) -- Owner demo script runs clean on the shipped code (probe- and test-verified) -- Suggestions + refine never auto-submit; drafts stay untouched on dismiss -- The 3-click criterion still pinned green on AskView (affordances opt-in) -- Honest captions everywhere; keyboard accessible throughout +### Wave 1 — Schema core (data-engineer) +- **T-1.1** `backend/pyproject.toml` + src-layout skeleton: packages config, deps, dev group, ruff/mypy/pytest config (hatchling packages key, pythonpath src, asyncio_mode auto) [REQ-33 seed — tooling; full monorepo story completes in P3/P4] +- **T-1.2** Dial schema: `researchx/config/dials.py` — 12 typed dials (source universe, search routing, query strategy, breadth, depth, quality gates, citation rigor, verification passes, synthesis style, output structure, memory scope, checkpoints), each with safe default; pydantic v2 [REQ-01] +- **T-1.3** Methodology model + registry: `researchx/methodologies/` — methodology = named dial bundle + planner/writer prompt templates + output shape; registry resolves by slug; ships Rapid Answer (depth 1, breadth 1, memo, per-claim citations) + Deep Report (depth 3, breadth 2, executive report, full verification); behavior-difference contract documented + asserted (different query sets/depth/output shape) [REQ-02, REQ-03, REQ-05] +- **T-1.4** Run-config contract: resolved dial values snapshot model (persisted per run — storage wiring lands with the engine in P2; schema + round-trip test here) [REQ-04] -## Verification +### Wave 2 — Adapter contracts + fakes (backend-engineer) +- **T-1.5** Adapter contract types: `researchx/adapters/` — model (OpenAI-compatible, planner/writer roles), search (Tavily/DDG), storage (local FS), memory (stub), secrets (env-file); all I/O behind interfaces; httpx confined to adapters [REQ-06, REQ-10, REQ-11] +- **T-1.6** Fake adapters: FakeModelAdapter (canned planner/writer), FakeSearchAdapter (fixture corpus) — full interface implementations; zero API keys [REQ-31] +- **T-1.7** Storage adapter: local FS write of run artifacts (Markdown/JSON under user-chosen dir); run ID generation (8-char slug) [REQ-10, D-010] +- **T-1.8** Secrets adapter: env-file resolution (.ciagent/.env.secrets + .env scopes per config.json); never logs values [REQ-11] -- Gates: typecheck ✓ lint ✓ tests 90/90 ✓ build ✓ (re-run after review fixes, plus by audit) -- Regression tests pin: single-compose, Enter-on-button, focus return, Enter-guard, no-auto-submit -- Post-hoc v0.2 log: compose-endpoint idempotency key; freeze draft at drawer-open; confirm-overwrite when context non-empty; Try-again from error state; textarea answers; aria-live progress +### Wave 3 — Real adapters + probe (backend-engineer) +- **T-1.9** Model adapter: OpenAI-compatible chat completions against OLLAMA_BASE_URL (default https://ollama.com/v1, Bearer OLLAMA_API_KEY) with keyless http://localhost:11434/v1 fallback on auth/connection failure; planner/writer roles [REQ-06, REQ-08] +- **T-1.10** Capability probe: GET /models + structured-output smoke test; conservative caps: context min(claim, 8192), embedding batch 8; typed probe result with overrides [REQ-07] +- **T-1.11** Structured-output warning: smoke-test failure → typed warning surfaced for UI; degrade to prompt-embedded JSON contract — never cryptic failure [REQ-12] +- **T-1.12** Search adapter: Tavily (POST api.tavily.com/search, key from secrets) default; DDG keyless fallback (html.duckduckgo.com/html conservative scrape, max 8, one backoff); registry selects by key presence [REQ-09, D-012] -## Scope Guard +### Wave 4 — Integration proofs (ci-lead-developer) +- **T-1.13** Cross-cutting test: registry → dial resolution → fake-model plan fixture → dial snapshot round-trip; methodology behavior-difference assertion (Rapid Answer vs Deep Report query sets/depth/output shape) [REQ-05] +- **T-1.14** README + VERSIONING updates: backend quickstart (uv sync/uv run uvicorn), adapter config table (env vars, no values), monorepo command map [REQ-33 partial] -Milestone merge to main, v0.1.8 publication, v0.2 unblocking, and all v0.2 P1 work wait on the owner approval gate (REQ-62, prototype_approval escalation hook). The pipeline does not proceed autonomously past this commit. \ No newline at end of file +## Out of scope for P1 (later phases) +- Engine state machine, evidence store, trust loop (P2); API routes, SSE, OpenAPI client swap (P3); adversarial/E2E suite (P4) + +## Verification hooks (consumed by VERIFY) +- Gates: `uv run ruff check src tests`, `uv run mypy src`, `uv run pytest` (backend); `corepack pnpm typecheck/lint/test/build` (frontend, must stay green) +- REQ coverage assert: REQ-01..06, 08..11, 31 each mapped to ≥1 test or contract proof +- Security: no secret values in code/fixtures/logs; SSRF not yet applicable (no page fetching until P2 evidence store) \ No newline at end of file diff --git a/.ciagent/ROADMAP.md b/.ciagent/ROADMAP.md index 18fcd6f..2929ea0 100644 --- a/.ciagent/ROADMAP.md +++ b/.ciagent/ROADMAP.md @@ -75,37 +75,42 @@ prototype release is **v0.1.8** (final phase patch). Milestone branch merged to main and deleted — tags v0.1.0–v0.1.8 preserve the full history. Next milestone: v0.2 MVP (unblocked). -## Milestone v0.2 — MVP (feature) — BLOCKED until prototype approval +## Milestone v0.2 — MVP (feature) — UNBLOCKED (prototype approved 2026-09-27) -Phases renumbered from the original v1.0 plan (1-based within the milestone): +Phases renumbered from the original v1.0 plan (1-based within the milestone). Tags run on the v0.2.x line; MAJOR stays 0. ### Phase 0 — Pre-Execution (v0.2) -- **Description**: Milestone pre-execution re-run: PLAN.md refresh against approved prototype decisions (layout learnings, API shape implied by the ApiClient interface), research on any gaps surfaced by approval feedback. -- **Status**: blocked (gate: v0.1.4) +- **Description**: Milestone pre-execution: SPECIFY (spec validated for v0.2), CLARIFY (D-007..D-020: backend layout, uv+Python 3.12, ports/run-IDs, DDG fallback, openapi-typescript client, SSE+pydantic events, difflib anchoring, liveness rules, durable JSON state, owner-gate halt, no-deploy policy, RESEARCHX_API_MODE), RESEARCH (RESEARCH-v02.md — 8 topics, 14 conclusions, 4 D-refinements), PLAN (P1 vertical-slice plan), GRILL (mechanical nine-axis review), MVP/UX check. +- **Requirements covered**: none (planning artifacts) +- **Success criteria**: .ciagent/ files refreshed for v0.2; PLAN.md carries User-Facing Surface + Happy Path + UX Acceptance Criteria; PERSONAS.md v0.2 roster active; grill verdict recorded with binding decisions +- **Depends on**: v0.1 approval (recorded) +- **Status**: in progress ### Phase 1 — Methodology Framework + Adapter Layer -- **Description**: Dial schema (12 dials, typed, safe defaults), methodology registry with Rapid Answer + Deep Report, adapter contracts (model/search/storage/memory/secrets) with local implementations, capability probing with conservative caps (8192 context, embedding batch 8, Ollama cloud-first + local fallback), fake model/search adapters. -- **Requirements covered**: REQ-01..06, REQ-08..11, REQ-31 -- **Status**: blocked +- **Description**: Dial schema (12 dials, typed, safe defaults), methodology registry with Rapid Answer + Deep Report, adapter contracts (model/search/storage/memory/secrets) with local implementations, capability probing with conservative caps (8192 context, embedding batch 8, Ollama cloud-first + local fallback), fake model/search adapters. Full task waves in PLAN.md. +- **Requirements covered**: REQ-01..06, REQ-08..11, REQ-31 (REQ-07, REQ-12 ship in P2 per ROADMAP phase mapping — dial-level + warning contracts are proven here, live probing against real endpoints in P2) +- **Success criteria**: uv one-command backend; gates green zero-API-key; registry resolves slugs; dial snapshot round-trips; fakes drive full interface; degradation typed + honest +- **Depends on**: 0 +- **Status**: blocked → next (unblocks on P0 ship) ### Phase 2 — Research Engine + Trust Loop - **Description**: Asyncio state machine with durable JSON state and pause/cancel/resume; plan approval gate; typed event bus → SSE; evidence store with stable passage IDs; trust loop (visited-only constraint, URL liveness pass, quote anchoring, citation ledger, verification checklist, confidence labels). - **Requirements covered**: REQ-07, REQ-12, REQ-13..24 -- **Status**: blocked +- **Status**: blocked → next (sequential phases; unblock as each prior phase ships) ### Phase 3 — API Layer + Integration - **Description**: FastAPI routes (runs, methodologies, lifecycle, SSE events, config status); OpenAPI-generated typed client; **swap MockApiClient → generated client** (the ApiClient interface absorbs any prototype-era API drift); ETA/cost heuristics for methodology cards. - **Requirements covered**: REQ-25, REQ-26 -- **Status**: blocked +- **Status**: blocked → next (sequential phases; unblock as each prior phase ships) ### Phase 4 — Hardening + Adversarial Suite + E2E - **Description**: Adversarial test suite (hallucinated-URL writer, quote-anchor mismatch, pause/resume mid-depth-loop), E2E happy-path (Playwright), monorepo tooling polish (one-command install/dev, lint/typecheck/test green). - **Requirements covered**: REQ-17, REQ-32, REQ-33, REQ-34 -- **Status**: blocked +- **Status**: blocked → next (sequential phases; unblock as each prior phase ships) ### Phase 5 — Final Review + Ship (FINAL) - **Description**: Review + audit + milestone ship; final patch tag IS the MVP release. MAJOR stays 0 — the MVP ships as a 0.x release. v1.0.0 comes only after an owner-approved release decision. -- **Status**: blocked +- **Status**: blocked → next (sequential phases; unblock as each prior phase ships) ## Milestone Releases - **v0.1.4 = extended-prototype candidate** (superseded by the P5/P6 extension cycle) -- 2.43.0 From 05662e0eb97749794bb6879140c51bdc73db0d00 Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:52:16 +0000 Subject: [PATCH 6/8] =?UTF-8?q?docs(P00):=20checkpoint=20=E2=80=94=20grill?= =?UTF-8?q?=20stage?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- phase: 0 milestone: v0.2 status: grill ---/ci--- --- .ciagent/CHECKPOINT.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 4e6cd10..924d1e6 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,8 +1,8 @@ { "phase": 0, - "stage": "plan", + "stage": "grill", "milestone": "v0.2", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-09-27T11:10:00Z" + "updated_at": "2026-09-27T11:20:00Z" } -- 2.43.0 From 6f232476d6c97d52dc1513f1a0191b0c33b66924 Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:57:04 +0000 Subject: [PATCH 7/8] =?UTF-8?q?docs(grill):=20adversarial=20review=20?= =?UTF-8?q?=E2=80=94=2017=20challenges,=2010=20binding=20decisions=20(G-00?= =?UTF-8?q?1..G-010),=201=20escalation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- phase: 0 milestone: v0.2 status: grill message: >- Mechanical nine-axis grill, verdict Proceed (0.72). Two internal contradictions + one measurability hole bound as decisions: G-001 REQ-07/12 phase mapping resolved ROADMAP-favor (contracts in P1, live probing in P2); G-002 live-mode surface = core research loop only in v0.2 (Library/Settings-persistence/Suggestions/ Refine/graph stay mock-bound to v0.3+); G-003 pause/resume added to ApiClient as the single planned P3 extension; G-004 REQ-23 liveness = per-run export gate + opt-in keyed tier, not a zero-key CI assertion; G-005 openapi.json snapshot + regen discipline as a P3 must-have; G-006 P2 may split into two patches if it slips; G-007 advisory cost counters via GET /api/config; G-008 DDG = degraded-mode contract; G-009 escalation: owner ratifies the live-mode surface at the P5 gate; G-010 contingency lever: Deep Report deferrable to v0.3 if P2/P4 slip (owner sign-off to pull). decisions: - id: G-001 decision: P1 ships typed probe/warning contracts proven with fakes; live endpoint probing executes in P2 (ROADMAP mapping wins) rationale: PLAN must_haves T-1.10/T-1.11 claim live probing while PLAN frontmatter and ROADMAP map REQ-07/12 to P2 confidence: 0.9 alternatives: [live probing in P1] - id: G-002 decision: v0.2 live-mode surface = core research loop only; Library, Settings persistence, Suggestions, Refine, graph/inbox, profile stay mock-bound until v0.3+ rationale: client.ts documents seven v0.2 endpoint mappings REQ-25 never promised — the P3 swap hidden-scope grenade confidence: 0.85 alternatives: [full-prototype live mode in v0.2] - id: G-003 decision: Add pauseRun/resumeRun to ApiClient in P3 as the single planned interface extension rationale: v0.1 pause/resume is client-side scripted; REQ-15 demands server-side pause across restarts confidence: 0.85 alternatives: [no interface extension] - id: G-004 decision: REQ-23 liveness = runtime per-run export gate + opt-in keyed live-URL test tier rationale: the 99% bar is unprovable in the zero-key fixture suite it was scheduled into confidence: 0.8 alternatives: [zero-key CI assertion of 99%] - id: G-005 decision: Committed openapi.json snapshot, regen only from snapshot, schema-drift gate, stable operation-ids — P3 must-have rationale: regenerating from a live server invites nondeterministic schema drift confidence: 0.85 alternatives: [live-server regen] - id: G-006 decision: P2 (13 REQs) may ship as two patches (P2a engine/SSE, P2b trust/probe); P3 swap endpoint plan drafted at P2 start rationale: heaviest phase; schedule risk needs a sanctioned split lever confidence: 0.7 alternatives: [single P2 patch regardless] escalations: - 'G-009 (axis 2): owner ratification of the live-mode surface (core loop only) required at the P5 gate — must not be discovered by the owner at the gate' ---/ci--- --- .ciagent/GRILL.md | 218 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 218 insertions(+) create mode 100644 .ciagent/GRILL.md diff --git a/.ciagent/GRILL.md b/.ciagent/GRILL.md new file mode 100644 index 0000000..997a3f8 --- /dev/null +++ b/.ciagent/GRILL.md @@ -0,0 +1,218 @@ +# CIAgent Grill Report + +## Run: 2026-09-27 11:25 (mode: mechanical, focus: all) + +### Verdict: Proceed (confidence: 0.72) + +The plan is sound with fixable gaps. The evidence base is unusually strong for a project at P0: a shipped and owner-approved v0.1 (9 tags, decision commit 3fd0e10), a locked two-milestone structure, a typed frontend boundary already in the tree (client.ts), a research file with concrete API shapes, and honest `---ci---` reporting throughout. But the grill found **two internal contradictions and one measurability hole** that would each have surfaced at P3 as rework: (1) PLAN.md P1 claims REQ-07/REQ-12 delivery while ROADMAP/REQUIREMENTS/PLAN's own frontmatter map them to P2; (2) the approved prototype's ApiClient documents seven v0.2 endpoint mappings (`GET /api/graph`, `GET /api/tasks`, `GET /api/library`, `GET /api/library/connections`, `GET /api/suggestions`, `POST /api/refine/*`, `PUT /api/config`) that REQ-25 does not include — a hidden-scope grenade wired to the P3 swap; (3) REQ-23's ≥99% liveness bar cannot be proven in the zero-key test suite it is scheduled into. All three are absorbed below as binding decisions. The MVP does not overreach as specced — REQ-01..34 is a genuine core loop — but the *implicit* live-mode expectation (whole prototype UI working against the backend) does overreach, and is explicitly reduced to the core research loop (G-002, escalated for owner ratification at the P5 gate). + +--- + +### Axis 1 — Business Case +- **Q1**: What problem does this actually solve, and is that problem still the top priority? + - Evidence: PROJECT.md Core Value + Key Decisions ("deep-research agents hallucinate URLs at the highest measured rates; verification cuts failures to <1% [E1]"); owner approved v0.1.8 and immediately unblocked v0.2 (commits 3fd0e10, 42435bc). + - Answer: Real, named problem — citation-untrustworthy deep-research agents — converted into a differentiated product thesis (methodology-first + trust loop). The owner's same-day unblocking of v0.2 shows it remains the top priority. Not a project that outlived its justification: the value is entirely unrealized until the MVP ships, which argues for proceeding, not stopping. + - Confidence: 0.80 +- **Q2**: Who is the named executive sponsor, and when did they last make a decision under pressure? + - Evidence: Owner decisions under pressure recorded twice in v0.1: P4 gate returned *feedback instead of approval* → binding P5/P6 extension (ROADMAP P4 note, commit 7db53d62); P6 gate returned *2nd feedback* → P7/P8 extension (commit 57c5231, ROADMAP P6 note); final approval 3fd0e10. D-018/D-019 record owner directives for this run. + - Answer: The sponsor is pressure-tested, not a rubber stamp — two rejections/extensions before approval. This is the strongest possible sponsor signal available in the record. + - Confidence: 0.85 +- **Q3**: What happens to the business if the project is cancelled? + - Evidence: v0.1 is mock-only by design (REQ-53: "no backend, no API keys"); zero research value has shipped; all value lands in v0.2 (REQ-01..34). + - Answer: Cancelling now strands 100% of sunk effort — the prototype is a shell without the MVP. The project is not a zombie; it is the opposite: a project whose entire value is still ahead of it, which raises the stakes of P2/P3 execution quality, not the case for stopping. + - Confidence: 0.75 +- **Q4**: Is the ROI calculated against a counterfactual? + - Evidence: MISSING — no ROI/counterfactual calculation exists anywhere in .ciagent/. + - Answer: No formal ROI. The implicit counterfactual is "ad-hoc LLM prompting with unverified citations," which is plausible but never quantified. For an owner-directed, single-user, local-first tool this is a tolerable omission — the sponsor is the beneficiary — but it means priority calls rest entirely on owner judgment, with no objective measure if enthusiasm fades. + - Confidence: 0.55 + - Decision: G-007 — make the counterfactual *measurable in-product*: per-run request/token/cost counters in the run trace, surfaced via `GET /api/config` (advisory, no hard spend cap in MVP). This is the cheapest honest ROI instrumentation available. + +### Axis 2 — Scope and Requirements +- **Q1**: Is the scope expanding, contracting, or genuinely stable? + - Evidence: REQ-01..34 locked with traceability (REQUIREMENTS.md table); D-007..D-020 clarify commit recorded (e79102b); v0.1 scope *grew twice* via owner gate feedback (P5/P7 extensions) — proof the gate can grow scope. + - Answer: The formal scope is stable and frozen. But the grill found the *informal* scope is ambiguous: the prototype's approved UI implies backend endpoints that REQ-25 does not promise (see Q4). That ambiguity will resolve itself at P3 — either as scope growth or as a visible degraded mode. It must be resolved now, in writing. + - Confidence: 0.60 +- **Q2**: Who owns the requirements, and have they been frozen? + - Evidence: Owner-approved milestone structure (PROJECT.md Constraints); REQUIREMENTS.md traceability with per-phase status; clarify decisions D-007..D-020 at confidence 0.75–1.0. + - Answer: Owner owns; the pipeline freezes and traces them. Requirements management is this project's strongest discipline. + - Confidence: 0.85 +- **Q3**: What is explicitly out of scope? + - Evidence: REQUIREMENTS.md "Out of Scope" table (paywalled sources, chatbot/hosting, auth/multi-tenancy, native Ollama adapter, rename, autonomous v1.0.0); PROJECT.md Out of Scope. + - Answer: Explicit, reasoned, and enforced (e.g., OpenAI-compatible-only is restated in constraints and architecture). + - Confidence: 0.90 +- **Q4**: Are there hidden requirements only disclosed late in delivery? + - Evidence: **FOUND — two.** (a) `frontend/src/lib/api/client.ts` lines 123–179 document v0.2 mappings to `GET /api/graph`, `GET /api/tasks`, `GET /api/library`, `GET /api/library/connections`, `GET /api/suggestions`, `POST /api/refine/questions`, `POST /api/refine/compose`, `PUT /api/config` — none of which appear in REQ-25 (runs, methodologies, lifecycle, SSE, config-read only) or any v0.2 phase mapping. (b) The same file exposes no `pauseRun`/`resumeRun` while REQ-25 mandates pause/resume in the API and REQ-15 mandates server-side pause/resume across restarts; line 69's comment claims "pause/resume stay client-side concerns," which contradicts the engine requirement. + - Answer: The P3 swap ("behind the unchanged ApiClient interface," REQ-26/REQ-54) is wired to detonate: either the backend grows 7+ undocumented endpoints (scope expansion inside the MVP milestone), or live mode silently breaks six approved UI surfaces, or the interface changes anyway (breaking REQ-54's zero-rewrite promise without a plan). This is the single largest finding of this grill. + - Confidence: 0.50 + - Decision: G-002 — declare the v0.2 live-mode surface = **core research loop only** (Ask, Choose, Plan & Run, Reading Room + read-only `GET /api/config`); Library, Settings persistence, Suggestions, Refine, project graph, task inbox, and user profile stay mock-bound behind ApiClient in live mode, honestly marked in-UI, with their backends explicitly deferred to v0.3+. G-003 — add `pauseRun`/`resumeRun` to ApiClient in P3 as the single planned, documented interface extension (mock gains matching implementations), superseding the v0.1 client-side-pause comment; REQ-54's zero-rewrite promise is amended to "no *unplanned* rewrites." + - Escalation: [G-009] — owner must ratify the live-mode boundary at the P5 gate (see Escalations). + +### Axis 3 — Architecture and Technical Feasibility +- **Q1**: Has the architecture been validated by the people who will build and operate it? + - Evidence: RESEARCH-v02.md — 8 topics with concrete API payload shapes, pitfalls, and 14 conclusions written for the builders (pydantic event unions, shielded state writes, `os.replace` atomicity, pytest-asyncio modes, hatchling `packages` key). But `backend/` does not exist yet — zero lines of the architecture have run. + - Answer: Design-side validation is strong (this is what a builder-facing research doc should look like); runtime validation is zero, by definition of P0. The risk concentrates at first contact: ollama.com/v1 shapes, Tavily payload, SSE backpressure. P1 Wave 3 is the first live proof. + - Confidence: 0.65 +- **Q2**: What is the integration surface, and where does it quietly double in cost? + - Evidence: Five adapter slots (ARCHITECTURE §2); SSE transport (D-014); OpenAPI client generation (D-013, RESEARCH §7 — flags operation-id stability and regen-from-snapshot discipline as pitfalls); the ApiClient surface gap (Axis 2 Q4). + - Answer: 5 adapters + SSE + codegen swap = 7 integration seams. The cost-doubling candidate is the P3 swap (seam #7) — precisely where the undocumented-endpoint problem sits. Second candidate: the openapi.json snapshot discipline exists only as a research *pitfall note*, not a P3 must-have. + - Confidence: 0.60 + - Decision: G-005 — P3 must-have: commit the backend `openapi.json` snapshot; regenerate `frontend/src/lib/api/generated/` only from the snapshot (never a live server); add a schema-drift check to the gates; fix FastAPI operation-ids at route creation. +- **Q3**: Is there an existing system being replaced — actual data volume, transaction rate, edge-case behavior? + - Evidence: Greenfield backend; the "replaced system" is `MockApiClient` (fixtures + scripted replay, frontend/src/lib/api/mock/). REQ-54 defines the swap contract; PERSONAS.md binds the frontend-engineer to it. + - Answer: No data migration risk. The migration is one interface binding line (client.ts:187) — provided G-002/G-003 are absorbed; otherwise the "small migration" is the biggest one in the project. + - Confidence: 0.70 +- **Q4**: What technical debt is inherited, and is it budgeted for? + - Evidence: (a) the v0.1 "client-side pause" comment (client.ts:69) contradicting REQ-15 — budgeted via G-003. (b) DDG `html.duckduckgo.com/html` scrape as the keyless fallback (D-012, confidence 0.75) — RESEARCH §3 itself warns "rate-limits aggressively... best-effort." (c) Scripted-replay SSE shape must match D-014's event union — verified aligned (D-014 rationale cites the MockApiClient replay contract). + - Answer: The DDG fallback is the fragility hotspot: a legacy HTML surface, conservatively scraped, that the research file already expects to fail. The zero-config promise of the product rests partly on it. Mitigation exists in design (surfaced warning, never hang) but is not yet test-enforced. + - Confidence: 0.55 + - Decision: G-008 — DDG fallback is tested against recorded HTML fixtures only (never live-scraped in CI); the degradation contract (partial/empty results + typed warning, never hang, never block the engine) is asserted by test. DDG is documented as degraded-mode, not a reliable second source. + +### Axis 4 — People, Skills, and Organization +- **Q1**: Which 2–3 people, if they left, would the project fail? + - Evidence: The entire delivery capability is one agent pipeline (coreci) + one owner (decision commit 3fd0e10; all directives owner-recorded). PERSONAS.md is a persona roster, not a human team. + - Answer: Key-person dependency is total and structural: owner disengagement stalls every gate by design (that is the governance model); pipeline failure stops all delivery. For a single-user, owner-directed tool this is acceptable — but it means timeline estimates are really "pipeline availability" estimates, and there is no bench. + - Confidence: 0.60 +- **Q2**: Are resources actually allocated at the percentages claimed? + - Evidence: Autonomy level "full" (config.json); v0.1 shipped 9 phases, PRs #1–#8, tags v0.1.0–v0.1.8 within ~10 hours of wall clock (commits dd2538f 00:44 → d6ddcfe 10:03, 2026-09-27). + - Answer: The pipeline is demonstrably 100% allocated and not fighting BAU fires — velocity is exceptional and verified by artifact trail, not status reports. + - Confidence: 0.85 +- **Q3**: Is there a product owner with actual authority to prioritize? + - Evidence: Owner directives recorded as binding decisions (D-018, D-019 at confidence 1.0); two gate rejections prove authority in practice. + - Answer: Unambiguous single owner; no committee. Model is working. + - Confidence: 0.90 +- **Q4**: Is the team building capability it doesn't have, hoping to learn as they go? + - Evidence: The stack is being built in-milestone: FastAPI, asyncio durability (shielded writes, CancelledError discipline — RESEARCH §5), sse-starlette, httpx adapters, openapi codegen. No backend code exists yet (repo is frontend-only today). RESEARCH-v02.md documents the failure modes in advance (pytest-asyncio event_loop removal, hatchling src-layout wheel pitfall, uvicorn single-worker constraint). + - Answer: Yes, capability is being built on the fly — acceptable here because (a) this is a prototype-to-MVP transition with research-backed patterns, and (b) zero-key fakes (REQ-31) mean the engine can be proven without live services. But P2 is where "learning asyncio under load" meets "13 requirements" — see Axis 5. + - Confidence: 0.60 + +### Axis 5 — Timeline and Estimates +- **Q1**: Was the deadline set before or after scope and approach were understood? + - Evidence: No wall-clock deadlines exist anywhere in ROADMAP v0.2 — sequencing is dependency-driven (P0→P5), cadence is owner gates (D-018: halt at the owner approval gate). + - Answer: No reverse-engineered deadline — the healthiest possible pattern. The only "date" is the owner's attention, which v0.1 proved is present. + - Confidence: 0.85 +- **Q2**: What is the critical path, and what single thing would push it by 3+ months? + - Evidence: Critical path P1 → P2 → P3 → P4 (sequential phases, ROADMAP). P2 is the milestone's heaviest phase: 13 requirements (REQ-07, 12, 13–24) including the durable asyncio state machine, trust loop, and live probing. The 3-month pusher is a P3 integration-drift discovery: the ApiClient surface gap (Axis 2 Q4) hitting mid-swap, after P2's engine is built against the wrong boundary assumptions. + - Answer: Critical path is P2-heavy and P3-fragile. G-002/G-003 defuse the P3 risk; P2's size needs a pressure valve. + - Confidence: 0.70 + - Decision: G-006 — permit P2 to ship as two patches (P2a: engine + durable state + SSE; P2b: trust loop + live probe/REQ-07/12) or run mid-phase wave gates, without changing milestone scope; the P3 swap plan (endpoint list per G-002) must be drafted at P2 *start*, not P3 start, so drift surfaces before the engine is frozen. +- **Q3**: Are the estimates evidence-based? + - Evidence: The only velocity base is v0.1: ~1 phase/hour sustained across 9 phases including review+audit each. v0.2 phases are materially heavier (greenfield backend, adversarial tests). MISSING: any per-phase sizing in ROADMAP/PLAN. + - Answer: No estimates exist to be wrong — but also none to be right. v0.1 velocity does not transfer 1:1 (frontend screens vs. a durable async engine); treating it as a base rate would understate P2. The mitigation is the phase-patch mechanism (G-006) rather than fake numbers, which this grill endorses over invented estimates. + - Confidence: 0.60 +- **Q4**: Is there a working definition of done? + - Evidence: Per-phase success criteria (ROADMAP); PLAN.md gates + "REQ coverage assert: each REQ mapped to ≥1 test or contract proof"; verify stage precedent (PASS-WITH-FIXES with in-phase P0 fixes, PRs #4/#6/#8); final owner gate. + - Answer: Yes, and it is enforced — v0.1 reviews fixed 5 P0s and multiple P1s *in-phase*, which is proof the DoD has teeth. + - Confidence: 0.85 + - Decision: G-001 — resolve a DoD contradiction inside P1's own plan: PLAN.md must_haves and tasks T-1.10/T-1.11 claim REQ-07/REQ-12 delivery ("GET /models + structured-output smoke test" = live probing) while PLAN's frontmatter (`requirements: [REQ-01..06, 08..11, 31]`), ROADMAP P1 ("REQ-07, REQ-12 ship in P2... live probing against real endpoints in P2"), and REQUIREMENTS traceability all map them to P2. Binding: P1 ships the typed ProbeResult + structured-output warning *contracts* proven with fake adapters; live endpoint probing executes in P2 with the engine. Secondary bookkeeping fix in the same decision: the "REQ-13..24" range notation includes REQ-17 in both P2 and P4 rows — REQ-17 (3-click on real stack) stays in P4; ranges are corrected to exclude it. + +### Axis 6 — Budget and Financial Realism +- **Q1**: What percentage of budget is spent vs. remaining? + - Evidence: MISSING — no cost ledger exists. Known spends: pipeline tokens (v0.1: 9 phases of agent execution; v0.2 P0 stages), Ollama cloud key (config.json secrets scope `model`), Tavily key (scope `search`), self-hosted LXC infra (DEPLOY-RECORD.md). + - Answer: Untracked. For a single-user local-first tool the absolute numbers are small, but "untracked" is still a finding: no one can currently answer what a Deep Report run costs. + - Confidence: 0.60 +- **Q2**: Are there predictable cost drivers not in the original budget? + - Evidence: Deep Report = depth 3/breadth 2 → per run: ~dozens of model calls (planner + depth-loop reflections + writer + verification passes) plus search calls; Ollama cloud is pay-per-use; Tavily free tier is quota-capped; DDG fallback is free-but-fragile (D-012). Localhost:11434 fallback is the zero-cost escape hatch (REQ-08). + - Answer: Cost drivers are real and unbounded per run (no request caps on loops beyond dial defaults). The local-fallback design keeps the project from *hard* dependency on spend, but there is no visibility. + - Confidence: 0.60 + - Decision: G-007 — per-run counters (model requests, tokens, search calls) recorded in the run trace and surfaced via `GET /api/config`; advisory only in MVP. +- **Q3**: Burn rate and runway? + - Evidence: MISSING formal tracking; mitigated by local-first defaults (keyless localhost fallback, DDG keyless fallback) meaning the runway floor is "zero marginal cost, degraded mode." + - Answer: The architecture is honest about this: the product's default posture degrades to free. Runway is effectively unbounded if the owner accepts degraded mode; the grill accepts this as sound for the MVP. + - Confidence: 0.70 +- **Q4**: Is the budget contingent on something that hasn't happened yet? + - Evidence: No — no sale/board/hiring dependency. Keys exist (config.json scopes), infra exists (LXC), pipeline is running. + - Answer: Not contingent. Proceed. + - Confidence: 0.80 + +### Axis 7 — Risks, Assumptions, and Dependencies +- **Q1**: Top 3 assumptions, and evidence for each? + - Evidence: (a) *Adapter endpoint compatibility* — ollama.com/v1 OpenAI-shape and Tavily POST payload researched with concrete shapes (RESEARCH §2/§3), but unvalidated live in-repo; P1 Wave 3 is the test. (b) *DDG fallback stability* — RESEARCH §3 itself rates it "best-effort, aggressively rate-limited" (weakest assumption in the plan; D-012 confidence 0.75). (c) *Zero-rewrite client swap* — typed boundary exists (client.ts), RESEARCHX_API_MODE designed (D-020), but the interface gaps (Axis 2 Q4) undercut it until G-002/G-003 land. + - Answer: Two of three are well-evidenced; the DDG assumption is the one the project's own research argues against relying on. All three have named first-contact phases (P1 Wave 3, P2, P3). + - Confidence: 0.60 + - Decision: G-008 (fixture-tested DDG degradation contract) — see Axis 3 Q4. +- **Q2**: External dependencies outside the team? + - Evidence: Tavily quota; Ollama cloud key validity; DDG uncontrolled HTML surface; none of these are under the project's control; all have fallbacks *except* Tavily→DDG quality. + - Answer: Dependencies are acknowledged with degradation paths — the correct posture. The unmitigated residual is *degraded search quality* when Tavily is exhausted, which directly attacks the ≥99% liveness bar. + - Confidence: 0.65 +- **Q3**: The single risk that kills the project, and is there a mitigation? + - Evidence: P3 swap discovering the ApiClient/surface drift after the engine is frozen (evidence chain: client.ts:123–179 vs REQ-25; ROADMAP P3 "the ApiClient interface absorbs any prototype-era API drift" — an assertion, not a plan). + - Answer: Kill risk = integration drift at P3 → rework spiral through a hand-rolled engine → owner patience (already tested twice by extensions) exhausts. Mitigation now exists in writing: G-002 (boundary declared), G-003 (interface change planned), G-005 (snapshot discipline), G-006 (P3 plan drafted at P2 start). This grill's core job is done if those four hold. + - Confidence: 0.65 +- **Q4**: Has a pre-mortem been done? + - Evidence: MISSING — no pre-mortem artifact. This grill is the closest proxy. + - Answer: "It's 12 months out and v0.2 failed" — the derivable causes, in order: (1) P3 surface gap hit late (mitigated: G-002/003/005/006); (2) P2 asyncio durability bugs (pause/resume across restarts is genuinely hard — mitigated by RESEARCH §5 patterns + REQ-32 pause-mid-depth-loop adversarial tests); (3) quote anchoring thrash on real noisy pages (difflib 0.85 is fixture-proven only — live-page behavior unknown until P4/live runs); (4) DDG dead + Tavily exhausted → un-demoable zero-config mode (mitigated: G-008 + owner-held Tavily key). + - Confidence: 0.60 + +### Axis 8 — Governance, Decision-Making, and Communication +- **Q1**: Who is the decision-maker when two parties disagree? + - Evidence: Single owner; all decisions recorded as decision commits (3fd0e10; D-007..D-020 in e79102b); escalation hooks codified in config.json (deploy, delete_data, merge_to_main, prototype_approval). + - Answer: Unambiguous. No dual-executive stall is possible in this structure. + - Confidence: 0.90 +- **Q2**: Governance cadence and escalation pattern — useful in a crisis? + - Evidence: Per-phase ship + verify + audit with `---ci---` blocks; v0.1 precedent: pipeline *halted itself* at gates twice (7db53d62, 57c5231) rather than self-approving — the escalation pattern is proven under the exact stress it exists for. + - Answer: Cadence is per-phase and gates have been exercised under pressure. Crisis-tested by construction. + - Confidence: 0.85 +- **Q3**: What is being omitted from status reports? + - Evidence: Honest reporting verified in history: "PASS-WITH-FIXES," "PASS-WITH-NOTES," "approval_pending," P0s disclosed and fixed in-phase; DEPLOY-RECORD.md as a separate ledger. The one systemic omission this grill found lives in *spec comments, not status blocks*: the client.ts v0.2 endpoint mappings that were never promoted into REQUIREMENTS — a silent scope assumption, now surfaced (G-002). + - Answer: Status reporting is honest. Spec-side implication drift is the channel where optimism leaked — and it's now on the record. + - Confidence: 0.80 +- **Q4**: Is there a stop-the-project trigger? + - Evidence: Owner rejection at gate → binding fix cycle (v0.1 exercised twice); v1.0.0 owner-only (config.json major_locked); D-018 halts at the v0.2 gate; D-019 forbids deploys without directive. + - Answer: Yes — and rejection has demonstrably not become politically impossible; the owner said "not yet, extend" twice without ceremony. + - Confidence: 0.85 + +### Axis 9 — Change, Adoption, and Operational Readiness +- **Q1**: Who will use this, how does their work change, and what's in it for them? + - Evidence: Single-user owner-researcher (PROJECT.md constraint: "Single-user, local-first"); work change = ad-hoc prompting → structured, reproducible methodology runs with auditable citations (REQ-04 dial snapshots, REQ-21 ledger, REQ-22 checklist). + - Answer: The user is the sponsor — adoption politics are minimal, and the value proposition (trustable citations) is the same thesis the sponsor approved twice. The real adoption test is whether live-mode runs *feel* better than the mock — which lands at P4/REQ-17. + - Confidence: 0.80 +- **Q2**: Is ops involved now, or handed a finished product at go-live? + - Evidence: The pipeline *is* ops: deploy.sh with rolling swap + `researchx-prev` rollback container (commit 2ffc71c), gates re-run at deploy time (DEPLOY-RECORD.md rows), VERSIONING.md rules. Gap: the deploy system is frontend-only; the MVP's live mode requires a local backend process (uvicorn, single worker — RESEARCH §4) whose serving story at the owner gate is undefined (D-019 forbids deploys without directive, so live-mode *presentation* mode is an open question). + - Answer: Ops is built-in and rollback-capable for the frontend. The backend has no deploy story — coherent with local-first, but it means the P5 owner-gate demo of the *live* MVP needs a plan (local backend + keys at demo time, or gate runs on mock mode). This rides with G-009. + - Confidence: 0.60 +- **Q3**: What is the rollback plan? + - Evidence: Tags v0.1.0–v0.1.8 preserve history; deploy.sh swaps containers with a `researchx-prev` rollback; durable run state is per-run JSON on local disk (D-017) — engine changes cannot corrupt other runs' state; frontend mock mode (RESEARCHX_API_MODE=mock default, D-020) is itself a rollback for the entire live integration. + - Answer: Multi-layer rollback exists and is codified. The mock-default flag is an unusually good one — the MVP can ship with live mode opt-in, making go-live risk nearly zero. + - Confidence: 0.80 +- **Q4**: Have success criteria been validated with the people who will judge success? + - Evidence: REQ-17 (≤3 clicks on real stack) is P4-tested, validated on mock in v0.1 (REQ-61 pinned by test). REQ-23 (≥99% citation liveness) — **measurement gap**: the entire adversarial suite is zero-key on a 5–8 page fixture corpus (RESEARCH §5b); 99% liveness on fixture URLs is vacuous, and no keyed/live validation run is scheduled anywhere in P1–P5. + - Answer: The 99% bar — the project's headline promise — is currently scheduled to be "verified" in a harness that cannot verify it. If the owner gate treats REQ-23 as CI-green, the status report will be honest-looking but hollow. + - Confidence: 0.55 + - Decision: G-004 — REQ-23 is defined as a **runtime per-run gate** enforced at export (failures listed in the verification checklist, REQ-22) plus an opt-in, keyed, live-URL test tier (skipped by default, run before ships at owner discretion); P4 proves the *gate mechanics* with fixtures and explicitly does not claim live 99% attainment. The MVP definition of done for REQ-23 = the gate exists and enforces, not "CI showed 99%." + +### Meta — Closing Review +- **Q1**: If you were the auditor, what would you flag? + - Evidence: The two internal contradictions (PLAN vs ROADMAP/REQUIREMENTS on REQ-07/12, PLAN must_haves vs PLAN frontmatter; REQ-17 double-mapped in range notation) and the spec-comment endpoint mappings outside REQ-25 (client.ts:123–179). + - Answer: Flag #1: phase-mapping integrity — the traceability table is the single source of truth and PLAN must conform to it (G-001). Flag #2: an approved artifact (client.ts) documents v0.2 API surface that the v0.2 spec does not contain — the grill treats this as the most consequential process lapse found, because it would have surfaced as a P3 "surprise" after the engine was frozen. + - Confidence: 0.75 +- **Q2**: What is the project not doing that it should? + - Evidence: Missing items found across axes: openapi snapshot drift check as a gate (G-005), per-run cost counters (G-007), live-URL test tier for REQ-23 (G-004), P3 swap plan at P2 start (G-006), fixture-based DDG tests (G-008), live-mode boundary declaration (G-002). + - Answer: All six are now bound. Nothing found suggests the project is doing something it should not. + - Confidence: 0.75 +- **Q3**: What is the simplest version that delivers 80% of the value? + - Evidence: The MVP already trimmed to 2 of 8 methodologies (REQ-03) and a core-loop-only API (REQ-25). The remaining 20% of complexity carrying outsized risk: Deep Report's depth-3 breadth-2 loop (multiplies engine, trust, and cost surface) vs. Rapid Answer's depth-1 single-pass core. + - Answer: Rapid Answer alone + trust loop = 80% of the thesis proof. Deep Report is the pre-agreed deferral lever if P2 overruns — decided *now*, while it is cheap, rather than mid-crisis. + - Confidence: 0.65 + - Decision: G-010 — contingency lever, pre-agreed: if P2/P4 slip beyond one extension cycle, Deep Report (REQ-03 partial) defers to v0.3 and the MVP ships Rapid Answer end-to-end; the lever requires owner sign-off when pulled, per gate rules. +- **Q4**: What must be true for this to succeed in the next 90 days, and is it true today? + - Evidence: Required: (a) researched adapter shapes hold live (untested — first proof P1 Wave 3); (b) owner engagement continues (proven: 3 gate decisions in one day); (c) Tavily key active/quota available (held in secrets scope; unverified live); (d) the P3 boundary is as declared in G-002 (currently *not* true in writing — client.ts says otherwise). + - Answer: Three of four are true or evidenced; (d) becomes true when the orchestrator promotes G-002/G-003. That promotion is the gating action of this entire grill. + - Confidence: 0.70 + +--- + +### Binding Decisions +| ID | Axis | Decision | Confidence | +|----|------|----------|-----------| +| G-001 | 2/5 | Resolve the REQ-07/REQ-12 phase contradiction in PLAN.md's favor of the ROADMAP/traceability mapping: P1 ships typed probe/warning contracts proven with fakes; live endpoint probing (GET /models, structured-output smoke test) executes in P2. Reword PLAN must_haves T-1.10/T-1.11 to contract level; fix the REQ-13..24 range notation so REQ-17 appears only in P4. | 0.85 | +| G-002 | 2/3 | Declare v0.2 live-mode surface = core research loop only (Ask, Choose, Plan & Run, Reading Room, read-only GET /api/config). Library, Settings persistence, Suggestions, Refine, project graph, task inbox, user profile stay mock-bound in live mode, honestly marked; their backends are v0.3+ scope. Update client.ts v0.2-mapping comments accordingly. | 0.80 | +| G-003 | 3 | Add `pauseRun`/`resumeRun` to the ApiClient interface in P3 as the single planned, documented interface extension (mock gains matching implementations); run-level pause is server-side per REQ-15/REQ-25, superseding the v0.1 "client-side concerns" comment. REQ-54 amended: no *unplanned* rewrites. | 0.80 | +| G-004 | 7/9 | REQ-23 (≥99% liveness) is a runtime per-run gate enforced at export and surfaced in the verification checklist — not a zero-key CI assertion. Add an opt-in keyed live-URL test tier (skipped by default). P4 proves gate mechanics with fixtures; MVP DoD for REQ-23 = the gate exists and enforces. | 0.75 | +| G-005 | 3/5 | P3 must-have: commit the backend openapi.json snapshot; regenerate frontend/src/lib/api/generated/ only from the snapshot; add a schema-drift check to gates; fix FastAPI operation-ids at route creation. | 0.85 | +| G-006 | 5 | Permit P2 (13 REQs — heaviest phase) to ship as two patches (P2a engine+durable state+SSE, P2b trust loop+live probe) or run mid-phase wave gates without milestone scope change; draft the P3 swap endpoint plan (per G-002) at P2 start, not P3 start. | 0.70 | +| G-007 | 1/6 | Per-run advisory counters (model requests, tokens, search calls) recorded in the run trace and surfaced via GET /api/config — the cheapest honest ROI/burn instrumentation; no hard spend cap in MVP. | 0.70 | +| G-008 | 3/7 | DDG fallback tested against recorded HTML fixtures only (never live-scraped in CI); degradation contract (partial/empty + typed warning, never hang the engine) asserted by test; DDG documented as degraded-mode, not a reliable second source. | 0.70 | +| G-010 | Meta | Pre-agreed contingency lever: if P2/P4 slip beyond one extension cycle, Deep Report (depth 3) defers to v0.3 and the MVP ships Rapid Answer end-to-end; pulling the lever requires owner sign-off at a gate. | 0.65 | + +### Escalations +- [G-009] Axis 2 — Owner ratification required at the P5 gate (D-018 halt): the v0.2 MVP demonstrates the **live core research loop only**; the remaining approved-prototype surfaces (Library, Settings persistence, Suggestions, Refine, graph/inbox previews, profile) remain mock-bound until v0.3+, and the live-mode gate demo requires a local backend + keys (or runs in mock mode) since D-019 forbids deploys. This redefines what "MVP done" looks like at the demo and must not be discovered by the owner at the gate — it must be ratified in advance. Engineering-side handling is bound in G-002/G-003; the ratification itself is unresolved until the owner records it. \ No newline at end of file -- 2.43.0 From 61c099e97fa52270e931bc74e95dd6c50441a2a8 Mon Sep 17 00:00:00 2001 From: coreci Date: Sun, 27 Sep 2026 10:57:28 +0000 Subject: [PATCH 8/8] =?UTF-8?q?decision(P00):=20mvp/ux=20gate=20=E2=80=94?= =?UTF-8?q?=20PASS;=20grill=20decisions=20G-001..G-010=20promoted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ---ci--- phase: 0 milestone: v0.2 status: mvp_ux_check message: >- MVP/UX CHECK (REQ-MVP-UX-001): PASS — PLAN.md carries all three sections authored at PLAN stage (User-Facing Surface, Happy Path, UX Acceptance Criteria); no auto-generation needed. Grill binding decisions promoted into PLAN.md: G-001 (probe contracts in P1, live probing in P2 — must_haves T-1.10/T-1.11 reworded), G-002 (live-mode surface = core research loop only), G-003 (pauseRun/ resumeRun as the single P3 ApiClient extension), G-004 (liveness = per-run export gate + opt-in keyed tier), G-005 (openapi.json snapshot discipline), G-006 (P2 split lever), G-007 (cost counters), G-008 (DDG degraded-mode contract), G-010 (Deep Report contingency lever). G-009 stays an open escalation for the P5 owner gate. decisions: - id: G-009 decision: Owner ratifies the live-mode surface (core loop only) at the P5 gate — presented, not silently assumed rationale: The approved prototype has surfaces the MVP backend will not serve live; the owner must record acceptance at the gate confidence: 0.85 alternatives: [silent scope reduction] ---/ci--- --- .ciagent/CHECKPOINT.json | 4 ++-- .ciagent/PLAN.md | 21 +++++++++++++++------ 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 924d1e6..dde65b7 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,8 +1,8 @@ { "phase": 0, - "stage": "grill", + "stage": "mvp_ux_check", "milestone": "v0.2", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-09-27T11:20:00Z" + "updated_at": "2026-09-27T11:35:00Z" } diff --git a/.ciagent/PLAN.md b/.ciagent/PLAN.md index 6481ad6..8891267 100644 --- a/.ciagent/PLAN.md +++ b/.ciagent/PLAN.md @@ -9,9 +9,9 @@ must_haves: - "12-dial typed schema with safe defaults resolves; every methodology = named dial bundle + planner/writer templates + output shape; registry resolves by slug (rapid-answer, deep-report)" - "Resolved dial values persist to run config and are auditable (REQ-04) — schema-level contract proven by test" - "Adapter contracts (model/search/storage/memory/secrets) typed; local implementations + fake model/search adapters enable zero-API-key engine tests (REQ-31)" - - "Capability probe caps context at min(model claim, 8192), embedding batch 8; overrides surfaced via typed probe result (REQ-07)" + - "Probe/warning CONTRACTS typed and proven with fakes: capability-probe result (context cap min(model claim, 8192), embedding batch 8) and structured-output warning surfaces [REQ-07/REQ-12 contracts; LIVE endpoint probing executes in P2 per grill G-001]" - "Model adapter: Ollama cloud-first with keyless localhost fallback (REQ-08); search: Tavily default + DDG keyless fallback (REQ-09); storage: local FS Markdown/JSON (REQ-10); memory stubbed off, secrets env-backed (REQ-11)" - - "Structured-output weakness warning path (REQ-12): smoke-test detection, explicit typed warning, graceful degrade" + - "Structured-output weakness warning CONTRACT (REQ-12): typed warning surface + graceful-degrade path proven with fakes" personas: - data-engineer - backend-engineer @@ -53,11 +53,11 @@ No new UI in P1 — the frontend is untouched (mock-backed, still v0.1-complete) - **T-1.7** Storage adapter: local FS write of run artifacts (Markdown/JSON under user-chosen dir); run ID generation (8-char slug) [REQ-10, D-010] - **T-1.8** Secrets adapter: env-file resolution (.ciagent/.env.secrets + .env scopes per config.json); never logs values [REQ-11] -### Wave 3 — Real adapters + probe (backend-engineer) +### Wave 3 — Real adapters + probe contracts (backend-engineer) - **T-1.9** Model adapter: OpenAI-compatible chat completions against OLLAMA_BASE_URL (default https://ollama.com/v1, Bearer OLLAMA_API_KEY) with keyless http://localhost:11434/v1 fallback on auth/connection failure; planner/writer roles [REQ-06, REQ-08] -- **T-1.10** Capability probe: GET /models + structured-output smoke test; conservative caps: context min(claim, 8192), embedding batch 8; typed probe result with overrides [REQ-07] -- **T-1.11** Structured-output warning: smoke-test failure → typed warning surfaced for UI; degrade to prompt-embedded JSON contract — never cryptic failure [REQ-12] -- **T-1.12** Search adapter: Tavily (POST api.tavily.com/search, key from secrets) default; DDG keyless fallback (html.duckduckgo.com/html conservative scrape, max 8, one backoff); registry selects by key presence [REQ-09, D-012] +- **T-1.10** Capability-probe contract + implementation: GET /models + structured-output smoke test; conservative caps: context min(claim, 8192), embedding batch 8; typed probe result with overrides; proven with fakes here — LIVE probing against real endpoints executes in P2 [REQ-07 contract, G-001] +- **T-1.11** Structured-output warning contract: smoke-test failure → typed warning surfaced for UI; degrade to prompt-embedded JSON contract — never cryptic failure [REQ-12 contract, G-001] +- **T-1.12** Search adapter: Tavily (POST api.tavily.com/search, key from secrets) default; DDG keyless fallback (html.duckduckgo.com/html conservative scrape, max 8, one backoff, typed degradation warning — degraded-mode contract per G-008); registry selects by key presence [REQ-09, D-012] ### Wave 4 — Integration proofs (ci-lead-developer) - **T-1.13** Cross-cutting test: registry → dial resolution → fake-model plan fixture → dial snapshot round-trip; methodology behavior-difference assertion (Rapid Answer vs Deep Report query sets/depth/output shape) [REQ-05] @@ -66,6 +66,15 @@ No new UI in P1 — the frontend is untouched (mock-backed, still v0.1-complete) ## Out of scope for P1 (later phases) - Engine state machine, evidence store, trust loop (P2); API routes, SSE, OpenAPI client swap (P3); adversarial/E2E suite (P4) +## Milestone-level constraints promoted from grill G-001..G-010 (binding, absorbed by phase plans) +- **Live-mode surface (G-002)**: v0.2 live mode = core research loop only (runs, methodologies, lifecycle, SSE, config status). Library persistence, Settings persistence, Suggestions, Refine, graph/inbox, profile stay mock-bound until v0.3+ — owner ratifies at the P5 gate (escalation G-009) +- **ApiClient extension (G-003)**: P3 adds pauseRun/resumeRun as the SINGLE planned interface extension; client.ts endpoint-mapped surfaces beyond the core loop stay mock-backed +- **Liveness gate (G-004)**: REQ-23 ≥99% is a runtime per-run export gate + an opt-in keyed live-URL test tier — never a zero-key CI assertion on the fixture corpus +- **OpenAPI snapshot discipline (G-005)**: P3 must commit an openapi.json snapshot; regen only from the snapshot; schema-drift gate; stable operation-ids +- **P2 split lever (G-006)**: if P2 slips beyond one extension cycle it may ship as two patches (P2a engine/SSE, P2b trust/probe); P3 swap endpoint plan drafted at P2 start +- **Cost counters (G-007)**: per-run advisory request/token/search counters surface via GET /api/config (P3) +- **Contingency lever (G-010)**: if P2/P4 slip beyond one extension cycle, Deep Report defers to v0.3 and the MVP ships Rapid Answer end-to-end — owner sign-off required to pull this lever + ## Verification hooks (consumed by VERIFY) - Gates: `uv run ruff check src tests`, `uv run mypy src`, `uv run pytest` (backend); `corepack pnpm typecheck/lint/test/build` (frontend, must stay green) - REQ coverage assert: REQ-01..06, 08..11, 31 each mapped to ≥1 test or contract proof -- 2.43.0