14 Commits

Author SHA1 Message Date
jonathan 57528d7473 Merge pull request 'fix: count a run completed at the DONE transition, not after it (researchx#28)' (#29) from po/fix-cost-counter-race into main
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (push) Successful in 32s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (push) Successful in 1m37s
Reviewed-on: #29
2026-10-27 22:36:42 +00:00
jonathan f790435023 Merge branch 'main' into po/fix-cost-counter-race
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (push) Successful in 32s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (push) Successful in 1m36s
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (pull_request) Successful in 32s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (pull_request) Successful in 1m35s
2026-10-27 22:35:50 +00:00
jonathan b29105adf3 Merge pull request 'ci: quote the two job names so the gate file parses (researchx#24)' (#27) from po/phase4-ci-trigger2 into main
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (push) Successful in 31s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (push) Successful in 1m36s
Reviewed-on: #27
2026-10-27 22:35:39 +00:00
coreci-researchx-po 70243eeb8b fix: count a run completed at the DONE transition, not after it (researchx#28)
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (push) Successful in 31s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (push) Successful in 1m35s
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (pull_request) Successful in 31s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (pull_request) Successful in 1m35s
The cost counter was incremented by a watcher that awaited the run task, so it
lagged the status flip: the run read as done while /api/config still counted
zero completed runs. The increment now happens in the same step as the
transition to DONE, so any client that observes the run as done also observes
the counter.

Refs researchx#28
2026-10-27 05:18:51 +00:00
coreci-researchx-po c7ce780edd ci: quote the two job names - the file could not parse
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (pull_request) Failing after 31s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (pull_request) Successful in 1m39s
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (push) Successful in 32s
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (push) Successful in 1m38s
The gate file has never parsed: the two job names were plain scalars
containing ': ' (YAML 1.2 plain-scalar rule - a colon followed by a space
ends the scalar and starts a mapping, so the file fails to load). Gitea
cannot build a run from a file it cannot parse, and the tell is a
dispatch POST answered HTTP 500 with an empty body.

Measured at this seat, own PAT, own repo: dispatch on main -> HTTP 500,
dispatch on this branch -> HTTP 500, 0 run records, 0 queued jobs.
Quoting the two names is the whole change; gates, labels, toolchain and
the trigger shape (workflow_dispatch: {} -> workflow_dispatch:) are
unchanged. Evidence and the follow-up dispatch live on researchx#24.
2026-10-27 04:34:06 +00:00
coreci-researchx-po 95681c4b97 ci: workflow_dispatch as a null trigger, not an empty map (researchx#24 - the fleet control) 2026-10-27 03:41:30 +00:00
jonathan 5d9b6fca23 Merge pull request 'ci: fire the pipeline - the trigger shape this instance actually runs (researchx#24)' (#26) from po/phase4-ci-trigger into main
Reviewed-on: #26
2026-10-27 03:34:53 +00:00
coreci-researchx-po ae2584c7f7 ci: pin a runner label that exists (ubuntu-latest) and install uv+pnpm in-file (researchx#24) 2026-10-27 03:29:06 +00:00
coreci-researchx-po de6c4e096c ci: fire the pipeline - bare push trigger and a non-null workflow_dispatch (researchx#24) 2026-10-27 00:59:08 +00:00
jonathan b9108ae873 Merge pull request 'ci: register the phase-4 gate - backend + frontend jobs, one dispatched green run on main opens the code phase (researchx#24)' (#25) from po/phase4-ci-gate into main
Reviewed-on: #25
2026-10-26 23:18:24 +00:00
coreci-researchx-po 01e2d2702c ci: register the phase-4 gating workflow (researchx#24) - backend (uv/Python 3.12: ruff, mypy, pytest) + frontend (Node 22/pnpm 12.6.0: typecheck, lint, test, build); one dispatched run on main with both jobs green opens the code phase 2026-10-26 12:55:48 +00:00
coreci-hap ab1cf5f3f8 Merge pull request 'comms-etiquette v1.0 pass: TL;DR prologues for README + frontend/README (researchx#20)' (#21) from docs/comms-etiquette-tldr-prologues into main 2026-10-19 19:51:57 +00:00
coreci-researchx-po 9ace9d8a02 docs(comms): TL;DR prologue for frontend/README.md (comms-etiquette v1.0; researchx#20) 2026-10-19 19:43:57 +00:00
coreci-researchx-po c1e8e53e5b docs(comms): TL;DR prologue for README.md (comms-etiquette v1.0; researchx#20) 2026-10-19 19:43:51 +00:00
15 changed files with 268 additions and 1182 deletions
+31 -42
View File
@@ -1,7 +1,7 @@
# Architecture
## Overview
Three-milestone build (owner-directed). Milestone v0.1 (COMPLETE) built the UI/UX prototype as the production Next.js frontend behind a typed `ApiClient`/`MockApiClient` boundary. Milestone v0.2 (COMPLETE) added the FastAPI backend, custom async research engine, adapter layer, and trust loop, then swapped MockApiClient for the OpenAPI-generated client. Milestone v0.3 — **Hosted** — converts the product to a hosted multi-user service: auth + server-side sessions, per-user encrypted model keys, user-scoped storage and surfaces, and a two-service deployment behind an SSE-safe reverse proxy. The engine keeps its hand-rolled asyncio state machine; every run persists resolved dials, full trace, and structured report/evidence artifacts for reproducibility and auditability.
Two-milestone build (owner-directed 2026-09-27). Milestone v0.1 builds the UI/UX prototype as the production Next.js frontend with all data access behind a typed `ApiClient` interface backed by `MockApiClient` — no backend, zero throwaway work. Milestone v0.2 (gated on prototype approval) adds the FastAPI backend, the custom async research engine, and the adapter layer, then swaps `MockApiClient` for the OpenAPI-generated client. The engine is a hand-rolled asyncio state machine; every run persists its resolved 12-dial configuration and full event trace for reproducibility and auditability.
## Components
@@ -13,72 +13,61 @@ Typed pydantic schema for the 12 dials (source universe, search routing, query s
### 2. Adapter Layer (`src/researchx/adapters/`)
Five pluggable slots with versioned contracts:
- **Model**: any OpenAI-compatible endpoint (covers Ollama cloud/local and vLLM); separate planner/writer roles; capability probe on connect (context window estimate, conservative request cap = min(model claim, 8192), embedding batch 8) surfaced as UI overrides; structured-output capability warning. **v0.3:** adapters are constructed **per run** by the Model Adapter Factory (`model_factory.py`) — no shared instance; the per-instance fallback (cloud → localhost) preserves REQ-08 semantics without cross-run mutation; optional `api_key` constructor param carries the resolved per-user credential.
- **Model**: any OpenAI-compatible endpoint (covers Ollama cloud/local and vLLM); separate planner/writer roles; capability probe on connect (context window estimate, conservative request cap = min(model claim, 8192), embedding batch 8) surfaced as UI overrides; structured-output capability warning. Default: ollama.com/v1 cloud-first with localhost:11434/v1 fallback.
- **Search**: Tavily default; DuckDuckGo keyless fallback; interface ready for Exa/Perplexity later.
- **Storage**: local filesystem adapter; reports as plain Markdown/JSON under a user-chosen local directory; run state as JSON after each transition. **v0.3:** owner-scoped `LocalStorageAdapter(base_dir, owner_id)` instances — run layout `runs/<user_id>/<run_id>/` (supersedes D-010); data root via `RESEARCHX_DATA_DIR` (`researchx.config.paths.data_root()`); protocol gains `append_run_artifact`.
- **Memory**: off — interface stubbed for semantic caching.
- **Secrets**: env-file provider — interface stubbed for OpenBao. **v0.3:** the scopes map gains a `'keys'` scope (`RESEARCHX_KEK`) consumed by the per-user key store.
- **Storage**: local filesystem adapter; reports as plain Markdown/JSON under user-chosen directory; run state as JSON after each transition.
- **Memory**: off in v1.0 — interface stubbed for semantic caching.
- **Secrets**: env-file provider in v1.0 — interface stubbed for OpenBao.
- **Boundaries:** all external I/O lives here; engine never imports vendor SDKs directly.
- **DependsOn:** config (dial schemas only)
### 3. Identity & Sessions (`src/researchx/auth/`) — NEW v0.3
User model (8-char slug `user_id`, email login, argon2id password hashing via argon2-cffi, all hashing off the event loop), server-side session store (opaque `token_urlsafe(32)` httpOnly SameSite=Lax cookies; in-process dict + durable `sessions.json` at the data root; 14-day sliding expiry; immediate revocation on logout/deactivation; token rotation at login), auth guard as a router-level dependency (`Depends(require_user)`) covering all `/api/*` except login/register, CSRF check (custom header on non-GET), first-user-is-admin + admin-managed accounts (`/api/admin/users`), user store `users.json` at the data root.
- **Boundaries:** owns user/session data + password hashing; no run knowledge.
- **DependsOn:** adapters (secrets for the KEK scope), storage (data-root persistence)
### 4. Per-User Key Store (`src/researchx/auth/key_store.py`) — NEW v0.3
AES-256-GCM envelope keyed by `RESEARCHX_KEK` (env-file 'keys' scope, 32 bytes b64/hex), `user_id` as AAD (ciphertext is bound to its owner — swapped files fail `InvalidTag`), `os.urandom(12)` nonce per write, `keys/<user_id>.json` (v1/entries format, atomic rewrite, 0600). API surface is masked-only (`configured` + last-4). Without a configured KEK the store is disabled with honest status; the resolution chain falls to server/keyless.
- **Boundaries:** owns key encryption/decryption; never returns key material.
- **DependsOn:** Identity, adapters/secrets
### 5. Model Adapter Factory (`src/researchx/adapters/model_factory.py`) — NEW v0.3
Per-run credential resolution (REQ-74): user's own key → server env-file key → keyless localhost:11434; constructs one model adapter per run and returns the resolved tier (`user|server|keyless`); the capability probe runs against the resolved tier; `GET /api/config` resolves the factory for the current user without creating a run (fingerprint-cached probe). Fixes the v0.2 shared-adapter fallback-mutation bug by construction.
- **Boundaries:** resolves credentials and constructs adapters; no engine knowledge.
- **DependsOn:** Identity (user), Key Store, adapters (model, secrets)
### 6. Research Engine (`src/researchx/engine/`)
Custom asyncio state machine: PLANNING → AWAITING_APPROVAL → SEARCHING → EXTRACTING → REFLECTING → (depth loop) → SYNTHESIZING → VERIFYING → DONE. One RunActor per run; durable JSON state persisted to storage after each transition (pause/cancel/resume); typed event bus feeding SSE live trace; plan (sub-questions + source universe + outline) halts at AWAITING_APPROVAL until approved/edited. **v0.3:** `RunState` carries `owner_id`; at DONE the engine persists two additional artifacts — `report.json` (structured sections + anchored citations with quotes) and `evidence.json` (passage texts) — written in the verify phase before the evidence store is discarded; citation markers are normalized `[p-N]` → `[^pN]` at the report.json boundary (report.md keeps the trust-loop form); startup sweep marks in-flight runs CANCELLED on process restart (rehydration out of v0.3 scope).
### 3. Research Engine (`src/researchx/engine/`)
Custom asyncio state machine: PLANNING → AWAITING_APPROVAL → SEARCHING → EXTRACTING → REFLECTING → (depth loop) → SYNTHESIZING → VERIFYING → DONE. One RunActor per run; durable JSON state persisted to storage after each transition (pause/cancel/resume); typed event bus feeding SSE live trace; plan (sub-questions + source universe + outline) halts at AWAITING_APPROVAL until approved/edited.
- **Boundaries:** orchestrates; contains no HTTP handling and no vendor I/O.
- **DependsOn:** adapters, methodologies, trust, evidence
### 7. Evidence Store (`src/researchx/evidence/`)
Visited-page store with stable passage IDs; passage extraction from fetched pages. The only source of writer context — the visited-only citation constraint is enforced by construction: the writer sees nothing else. **v0.3:** serialized to `evidence.json` at DONE (passage texts survive the actor).
### 4. Evidence Store (`src/researchx/evidence/`)
Visited-page store with stable passage IDs; passage extraction from fetched pages. The only source of writer context — the visited-only citation constraint is enforced by construction: the writer sees nothing else.
- **Boundaries:** owns evidence data model.
- **DependsOn:** adapters (search/storage only)
### 8. Trust Loop (`src/researchx/trust/`)
### 5. Trust Loop (`src/researchx/trust/`)
Runs inside every methodology: generation-time visited-only constraint [E1]; post-run URL liveness pass with live/stale/dead classification [E1]; claim-to-passage quote anchoring (fuzzy match against stored passages) [E1b]; citation ledger JSON per run (URL, visited_at, passages, claims supported, health, authority tier); verification checklist export.
- **Boundaries:** read side of evidence store; verification reports feed the ledger.
- **DependsOn:** evidence, adapters (model for verification passes)
### 9. API Layer (`src/researchx/api/`)
FastAPI routes — **v0.3 inventory**: `/api/auth/{register,login,logout,me}` (public router; register open only while the user store is empty), `/api/admin/users*` (admin-guarded), `POST /api/runs`, `GET /api/methodologies`, `GET /api/runs/{id}`, approve/pause/cancel/resume, `GET /api/runs/{id}/events` (SSE), `GET /api/runs/{id}/report` (enriched: markdown + structured report + evidence), `GET /api/runs/{id}/ledger`, `GET /api/config` (read-only, user-contextualized: resolved tier, key-store status, per-user cost counters, probe), `GET/PUT /api/config/settings` (per-user settings), `GET /api/library`, `GET /api/library/connections`, `GET /api/graph`, `GET /api/tasks`, `GET /api/suggestions`, `POST /api/refine/{questions,compose}`, `GET/PUT /api/profile`, and `/healthz` outside `/api` (unauthenticated, for proxy health checks). Read-model routes (library/connections/graph/tasks/suggestions/profile/settings) read disk artifacts only — never the RunRegistry; the registry is owner-keyed `(owner_id, run_id)` and pruned of terminal runs. OpenAPI schema generates the typed frontend client.
### 6. API Layer (`src/researchx/api/`)
FastAPI routes: POST /api/runs, GET /api/methodologies, GET /api/runs/{id}, approve/pause/cancel/resume, GET /api/runs/{id}/events (SSE), GET /api/config. OpenAPI schema generates the typed frontend client.
- **Boundaries:** transport only — no business logic.
- **DependsOn:** engine, methodologies, config, auth, key store, model factory
- **DependsOn:** engine, methodologies, config
### 10. Frontend (Next.js App Router, TypeScript strict)
App shell (persistent sidebar + top bar with ⌘K slot + avatar). **v0.3 additions:** login screen; session-aware shell (avatar menu → real profile, logout); live binding of ALL surfaces (the G-002 mock-delegation boundary is lifted — `GeneratedApiClient` becomes a standalone `ApiClient` implementation with no `MockApiClient` inheritance); auth + CSRF fetch wrapper (`credentials: "include"`, `X-ResearchX-Request` on mutations); EventSource `withCredentials`; two build profiles (mock dev default; hosted build inlines `NEXT_PUBLIC_RESEARCHX_API_MODE=live` + empty same-origin base URL). Screens: Ask (question + context; suggestions below the form; opt-in refine dialogue — now real planner calls), Choose (methodology cards + dials drawer), Plan & Run (approve/edit, live SSE trace, pause/cancel/resume), Reading Room (structured report with anchored hover citations from report.json/evidence.json, ledger panel, exports), Library (folder structure, report cards, connections), Settings (per-user: masked model key, methodology defaults, model names, display-only storage), profile.
### 7. Frontend (Next.js App Router, TypeScript strict)
App shell (root layout, P5): persistent sidebar (primary nav — New research / Library / Settings; adapter connection-status dots as a v2 preview) + top bar (⌘K navigation palette — screens + stored reports; avatar button → user menu with the local single-user profile). Screens: Ask (question + optional context; open-question suggestions below the form — follow-ups from unanswered questions of past runs, click-fills-the-box, REQ-67; opt-in Refine dialogue — inline drawer, scripted clarifying exchange, composed refined question + suggested context, REQ-68), Choose (methodology cards with ETA/cost/why-recommended; dials behind "advanced"), Plan & Run (approve/edit plan, live SSE trace, pause/cancel), Reading Room v0 (rendered report, hover-a-citation → exact quoted passage, ledger panel, Markdown/JSON export), Library (P5: folder-structure report storage — auto-categorized folders, recency/title sorting, report cards into the Reading Room; cross-report Connections panel — shared-sources ⇄ / related-topic → edges; project graph + task inbox as v2-preview panels), Settings (P5: masked API-key status, storage directory, default methodology + resolved 12-dial display, appearance preview, about/version).
- **Boundaries:** consumes the typed `ApiClient` interface only — components never import the mock or generated client directly; the boundary selects the binding by build profile.
- **DependsOn:** ApiClient contract
- **Boundaries:** consumes the typed `ApiClient` interface only — components never import the mock client directly. In v0.1 the interface is bound to `MockApiClient` (fixture data + scripted event replay); in v0.2 phase 3 the binding swaps to the OpenAPI-generated client with no component changes.
- **DependsOn:** ApiClient contract (v0.1: mock; v0.2: API layer)
## Data Flow
v0.3 (hosted): browser → HAProxy (TLS, `/api` + `/healthz` → backend :8000, SSE-safe: no response buffering, no API-backend compression, 3600s timeouts) → FastAPI (session guard resolves user → per-user settings + key store → model factory resolves credential tier user→server→keyless → owner-scoped engine run) → durable artifacts under `runs/<user_id>/<run_id>/` → read-models scan disk → Library/Graph/Tasks/Suggestions render → Reading Room renders report.json with anchored citations (hover quotes from evidence.json) → exports round-trip real data. Engine internals (state machine, evidence, trust loop) unchanged from v0.2.
v0.1 (prototype): Ask (question + context) → Choose (methodology slug + dial overrides) → MockApiClient returns scripted plan → Plan & Run (scripted live trace replay, pause/cancel) → Reading Room (mock report with anchored citations, ledger, real file exports). All flows exercise the exact component paths the MVP will use.
## Deployment Topology (NEW v0.3)
Two containers on the LXC host behind the existing HAProxy/TLS front (research.coreci.dev): `researchx` (Next.js frontend, :3000, live build args) and `researchx-api` (FastAPI backend, :8000 host-published, non-root uid 10001, `uvicorn --workers 1` — mandatory: in-memory registry + session store), `researchx-data` volume at `/var/lib/researchx` (RESEARCHX_DATA_DIR), runtime secrets via `--env-file` only. Rollback pairs `researchx-prev` / `researchx-api-prev`. deploy.sh builds both images at the tag, health-checks `/healthz` (backend) + `/ask` (frontend), and asserts live mode in the smoke test.
v0.2 (MVP): Ask (question + context) → Choose (methodology slug + dial overrides) → POST /api/runs → engine PLANNING (planner model via model adapter, dial bundle resolved) → AWAITING_APPROVAL (plan persisted; SSE events; user approves/edits) → SEARCHING (search adapter, routing per dials) → EXTRACTING (fetch pages → evidence store, stable passage IDs) → REFLECTING (planner model; depth loop per dials) → SYNTHESIZING (writer model; context = evidence passages only, per-claim citation markers) → VERIFYING (liveness pass, quote anchoring, confidence labels) → DONE (report + ledger + checklist persisted to storage; Reading Room renders; export).
## Build Order
Milestone v0.1 (prototype — COMPLETE): frontend scaffold → Ask/Choose → Plan & Run/Reading Room → mockups/3-click → owner-gate extensions (app shell, library, settings, avatar; suggestions, refine) → approval.
Milestone v0.2 (MVP — COMPLETE): methodology framework → adapters → evidence → trust → engine → API → client swap → adversarial + E2E → approval.
Milestone v0.3 (hosted): 1. Auth + sessions + user scoping (storage partitioning, registry owner-keying, auth suite) → 2. Per-user settings + encrypted key store + model factory (credential chain, probe-per-tier) → 3. Live surfaces (read-models on disk artifacts, report.json/evidence.json + Reading Room fidelity, refine/suggestions real calls, frontend live swaps, mock-delegation ends) → 4. Hosting (backend Dockerfile + production entrypoint, two-service deploy.sh, SSE-safe HAProxy, live-default hosted build) → 5. Final review + owner gate.
Milestone v0.1 (prototype): 1. Frontend scaffold + ApiClient/MockApiClient boundary → 2. Ask + Choose → 3. Plan & Run + Reading Room v0 → 4. v2 mockups + polish + 3-click validation → 5. review + owner approval gate (owner returned gate feedback → extension cycle: 5. app shell + research library + settings + avatar → 6. final review + approval re-presentation (2nd owner gate feedback: 7. ask suggestions + refine dialogue → 8. final re-review + ship)).
Build order is dependency-driven: auth precedes key store (identity); key store precedes the factory (credential chain); read-models precede hosting (disk-backed reads must exist before redeploys); hosting precedes the gate (the owner reviews the deployed thing).
Milestone v0.2 (MVP, gated on v0.1 approval):
1. Methodology framework (dial schema + 2 seed methodologies + registry)
2. Adapter layer (model/search/storage contracts + local implementations; fake adapters for tests)
3. Evidence store (visited pages, passage extraction)
4. Trust loop (liveness, anchoring, ledger)
5. Engine (state machine, event bus, durable state, pause/resume)
6. API layer (routes + SSE)
7. Integration (swap MockApiClient → OpenAPI-generated client)
8. Adversarial test suite (hallucinated-URL detection, quote-mismatch detection, pause/resume mid-depth-loop) + E2E happy path
Build order is dependency-driven: trust verification requires evidence; engine requires adapters + trust; API requires engine; integration requires API; frontend components carry over unchanged from v0.1.
+5 -5
View File
@@ -1,8 +1,8 @@
{
"phase": 0,
"phase": 5,
"stage": "complete",
"milestone": "v0.3",
"phase_role": "pre_execution",
"milestone": "v0.2",
"phase_role": "final",
"attempts": 0,
"updated_at": "2026-09-29T17:25:00Z"
}
"updated_at": "2026-09-27T16:30:00Z"
}
+1 -214
View File
@@ -216,217 +216,4 @@ The plan is sound with fixable gaps. The evidence base is unusually strong for a
### Escalations
- [G-009] Axis 2 — Owner ratification required at the P5 gate (D-018 halt): the v0.2 MVP demonstrates the **live core research loop only**; the remaining approved-prototype surfaces (Library, Settings persistence, Suggestions, Refine, graph/inbox previews, profile) remain mock-bound until v0.3+, and the live-mode gate demo requires a local backend + keys (or runs in mock mode) since D-019 forbids deploys. This redefines what "MVP done" looks like at the demo and must not be discovered by the owner at the gate — it must be ratified in advance. Engineering-side handling is bound in G-002/G-003; the ratification itself is unresolved until the owner records it.
- **RESOLVED 2026-09-27: RATIFIED.** Owner approval recorded via /ci-run 'APPROVED' at the v0.2.5 gate (decision commit 4721c2a), following the owner-directed UAT staging at https://research.coreci.dev. Live-mode scope = core research loop only is the accepted MVP boundary; mock-bound surfaces move to v0.3+.
# CIAgent Grill Report
## Run: 2026-09-29 17:05 (mode: mechanical, focus: all)
### Verdict: Proceed (confidence: 0.68)
The plan survives adversarial re-verification better than any artifact this project has produced: the grill independently re-checked the research file's load-bearing claims against the tree and found them exact — the shared-adapter fallback mutation (`ollama.py:73-79`), the `config_status` private-registry read (`app.py:174-177`), the CORS-credentials gap (`app.py:143-148`), the fake-serving module global (`app.py:359`), the missing `append_run_artifact` protocol method, the `[p-N]`/`[^pN]` marker split (`actor.py:337` vs `report-body.ts:18`), the export fabrication (`live-client.ts:296-304/335/376`), and the 8-test-file partitioning blast radius. The auth supersession is traceable end to end (a4afccc constraint rewrite + D-101..D-115 @ 33f4fa0 + ROADMAP session note — matching the owner's "one milestone / admin-managed / server-key-fallback" session picks). Engine-contract preservation is a real design discipline, not a slogan. But the grill found **one "verified" research claim that is false** (the restart sweep's CANCELLED-from-any-state legality — `states.py` forbids PLANNING→CANCELLED), **three phase/bookkeeping contradictions** (REQ-75 triple-mapped across three documents; two stale out-of-scope rows contradicting the same-file supersession; C3-19's pitfall-phase mapping vs PLAN), **one unaddressed prototype-fidelity gap** (the task-inbox checkbox cannot survive the derived live read-model), and **an honestly-unbudgeted public attack surface** (no password policy, no login rate limit on a TLS-fronted public URL). All are absorbed as binding decisions G-101..G-110; one item is genuinely owner-required (G-111: P4's definition-of-done depends on a staged host deployment that D-019 forbids without a directive). The milestone is heavier than v0.2 — the P3 pressure valve absent from the plan is bound here (G-108) as the G-006 precedent. Proceed.
### Axis 1 — Business Case
- **Q1**: What problem does this actually solve, and is that problem still the top priority?
- Evidence: v0.2.5 approved and merged 2026-09-27 (8db45e4, b11b202); v0.3 scope owner-directed the next day, 2026-09-29 (a4afccc; ROADMAP "owner session 2026-09-29"); DEPLOY-RECORD.md itself records "live mode additionally needs the backend API on the host, which is v0.3+ deployment work" — the deployment gap is acknowledged in the ship record, not invented by the pipeline.
- Answer: Real and owner-named: convert the ratified mock-bound surfaces + single-user local loop into a hosted multi-user product with real inference. The same-day sequencing after v0.2.5 approval is the strongest available priority signal.
- Confidence: 0.80
- **Q2**: Who is the named executive sponsor, and when did they last make a decision under pressure?
- Evidence: Owner pressure-tested through v0.1 (two gate rejections → binding extensions, commits 57c5231, 3fd0e10), G-009 ratified at the v0.2 gate (4721c2a); the v0.3 direction itself is a decision under pressure — superseding a recorded constraint ("single-user, local-first") the same day the MVP shipped.
- Answer: Sponsor engaged and decisive; the supersession is a recorded constraint change, not scope drift.
- Confidence: 0.85
- **Q3**: What happens to the business if the project is cancelled?
- Evidence: v0.2.5 live mode = core research loop only, mock-default frontend, no hosting (G-009 ratification, ROADMAP v0.2 complete note); the owner-stated v0.3 purpose is the hosted product.
- Answer: Cancelling strands the G-009 promise ("mock-bound surfaces move to v0.3+") and leaves the product undeployable in its owner-intended form. Not a zombie — the payoff milestone.
- Confidence: 0.75
- **Q4**: Is the ROI calculated against a counterfactual?
- Evidence: MISSING — no formal ROI anywhere; G-007 counters carried advisory-only, owner-keyed in P2 (T-2.6, P-13). v0.3 adds real costs v0.2 did not carry: public infrastructure exposure and the server-tier key serving any admin-created user's runs.
- Answer: Still owner-judgment — tolerable for a sponsor-beneficiary product with an admin-bounded user set, but the server-key cost posture is now a stated-nowhere exposure; bound as G-110.
- Confidence: 0.55
- Decision: G-110 — document the cost posture: server-tier runs bill the owner's key; the bound is admin-managed accounts; per-user counters surface server-tier usage; hard caps deferred v0.4+.
### Axis 2 — Scope and Requirements
- **Q1**: Is the scope expanding, contracting, or genuinely stable?
- Evidence: Scope grew by owner directive (ROADMAP 2026-09-29 note); REQ-69..85 frozen with per-phase traceability; PLAN frontmatter carries all 17. But the G-001-class phase contradictions persisting: REQ-75 is mapped P3 by REQUIREMENTS traceability, P4-only by ROADMAP P3/P4 lists, and split by PLAN (delegation-end in P3, hosted in P4). C3-19 likewise maps P-1/P-2/P-13 to P1 while PLAN closes them in P2.
- Answer: Formal scope stable and owner-frozen; the traceability layer is where the drift lives — three documents disagree about one requirement. Bound in G-103.
- Confidence: 0.60
- Decision: G-103 — bind the REQ-75 split explicitly; fix the traceability row notation; declare PLAN's pitfall-closure mapping authoritative over C3-19's compressed prose.
- **Q2**: Who owns the requirements, and have they been frozen?
- Evidence: Owner-directed supersession traceable (a4afccc strikethrough + rationale; D-101..D-115 clarify commit 33f4fa0 at confidence 0.75–0.95).
- Answer: Owner owns; pipeline freezes and traces. The supersession discipline is exemplary — the constraint change is visible, dated, and strikethrough-preserved rather than silently rewritten.
- Confidence: 0.85
- **Q3**: What is explicitly out of scope?
- Evidence: **CONTRADICTION FOUND** — REQUIREMENTS.md Out-of-Scope table still carries "Auth / multi-tenancy | Single-user local-first MVP"; PROJECT.md Out of Scope still lists "multi-tenant/auth" — while PROJECT.md:34 records the supersession in the same file and REQ-69..71 define the now-in-scope auth. The honest deferrals (OAuth v0.4+, 6 methodologies v0.4+, KEK rotation, account deletion, rehydration) are all recorded.
- Answer: Mostly honest, with two stale rows that directly contradict a recorded decision — exactly the spec-comment drift class the v0.2 grill flagged. Bound in G-102.
- Confidence: 0.85
- Decision: G-102 — update both stale out-of-scope rows to match the supersession.
- **Q4**: Are there hidden requirements only disclosed late in delivery?
- Evidence: **FOUND — three.** (a) REQ-78 promises "checkable follow-ups"; `library-view.tsx:430-471` ships a working checkbox; the live read-model (D-114) derives status answered-iff-DONE over a GET-only route — a live user's check silently reverts on refetch; PLAN never addresses it. (b) No password minimum length exists anywhere (grep-verified). (c) No login rate limiting anywhere (grep-verified) — argon2 verify is ~50-100ms CPU, a public flood is a DoS amplifier on a single worker.
- Answer: The v0.2 G-002 lesson recurs once (the toggle); the exposure items are new-hosting hidden requirements. All three bound (G-109, G-104).
- Confidence: 0.75
### Axis 3 — Architecture and Technical Feasibility
- **Q1**: Has the architecture been validated by the people who will build and operate it?
- Evidence: RESEARCH-v03 claims re-verified by this grill: 11 of 12 sampled claims exact. **One claim is FALSE**: RESEARCH §7/C3-11 "rewrite every non-terminal state.json as CANCELLED (legal from every non-terminal state, verified states.py)" — `states.py` `_LEGAL_TRANSITIONS[PLANNING] = {AWAITING_APPROVAL, FAILED}`: no CANCELLED. A crash mid-PLANNING leaves a persisted PLANNING state the sweep cannot legally cancel.
- Answer: Design-side validation is the strongest yet — but the single false "verified" claim is the precise failure mode mechanical research invites: verification asserted, not re-run. Bound in G-101.
- Confidence: 0.85
- Decision: G-101 — startup sweep must handle PLANNING-state runs: add PLANNING→CANCELLED to `_LEGAL_TRANSITIONS` or document the sweep as an explicit transition-validation bypass; pinned by test in T-1.8.
- **Q2**: What is the integration surface, and where does it quietly double in cost?
- Evidence: Five seams: auth dependency threading (P1), per-run factory + probe-per-tier (P2), disk read-models (P3), snapshot regen ×3 (G-005 wired into P1/P2/P3), HAProxy front extension (P4). The cost-doubler candidate is SSE-through-proxy: the only seam whose live configuration is deferred to deploy time.
- Answer: Residual risk is bounded, not eliminated: the P4 smoke asserts a full unbuffered SSE run through the proxy path with the `-prev` rollback pair retained — worst case is one deploy iteration, not milestone rework. Accepted with the G-111 governance condition.
- Confidence: 0.65
- **Q3**: Is there an existing system being replaced?
- Evidence: Greenfield hosting; pre-hosted artifacts explicitly disposable (D-106, owner-recorded); the "replaced system" is the mock delegation layer (live-client.ts:61) — deletion is D-112/T-3.8 with component tests proving zero rewrites (T-3.12). Volume model: admin-managed accounts (REQ-70) bound the user set; single worker mandatory.
- Answer: No migration risk; the swap is designed as a deletion with regression pins. The one fidelity trap (task-inbox toggle) is found and bound (G-109).
- Confidence: 0.75
- **Q4**: What is the technical debt being inherited, and is it budgeted for?
- Evidence: The 14-item pitfall ledger is budgeted per-phase (PLAN cross-phase notes) — with C3-19's compressed phase mapping contradicting PLAN (folded into G-103). Marker normalization: one-directional at the report.json boundary only; report.md raw form pinned by test; trust-loop regexes untouched.
- Answer: If a future writer emits `[^pN]` natively, `_cited_urls` matches nothing → ledger empties → the G-004 export gate fails visibly at runtime, and the raw-form pin catches it in CI first. Silent corruption is not reachable.
- Confidence: 0.70
### Axis 4 — People, Skills, and Organization
- **Q1**: Which 2–3 people, if they left, would the project fail?
- Evidence: One agent pipeline + one owner; gates owner-halt by design (D-018). The security-engineer persona is phase-specific P1/P2/P4 with an explicit removal note (PERSONAS.md).
- Answer: Total key-person dependency remains structural and accepted; the P5 full multi-persona review covers the security persona's absence.
- Confidence: 0.60
- **Q2**: Are resources actually allocated at the percentages claimed?
- Evidence: v0.1 shipped ~9 phases in ~1 day; v0.2 shipped P0..P5 in one day; v0.3 P0 commits same-day (a4afccc→dca86a7).
- Answer: Velocity verified by artifact trail. v0.3's four phases are each multi-domain — the evidence supports capacity, with the G-108 valve for the heaviest phase.
- Confidence: 0.80
- **Q3**: Is there a product owner with actual authority to prioritize?
- Evidence: Owner-directed supersession recorded same-day as MVP ship (a4afccc); G-009 ratified at gate (4721c2a); D-018/D-019 carry.
- Answer: Unambiguous single owner, exercised.
- Confidence: 0.90
- **Q4**: Is the team building capability it doesn't have, hoping to learn as they go?
- Evidence: argon2-cffi, AESGCM envelope encryption, server-side sessions, SSE-through-proxy are all new to the codebase; mitigations are research-verified concrete shapes + adversarial suites specced + a dedicated security-engineer persona on exactly the phases that carry the new capability.
- Answer: Yes — learning-while-doing on a security-critical public surface, the highest-stakes version of this risk. The mitigation structure is proportionate. Acceptable; the G-104 hardening items close the gaps the research did not self-identify.
- Confidence: 0.60
### Axis 5 — Timeline and Estimates
- **Q1**: Was the deadline set before or after scope and approach were understood?
- Evidence: No wall-clock deadlines anywhere; dependency-driven phases; owner gates as the only cadence.
- Answer: Healthiest pattern, unchanged from v0.2.
- Confidence: 0.85
- **Q2**: What is the critical path, and what single thing would push it by 3+ months?
- Evidence: P1→P2→P3→P4 sequential. P3 is the heaviest (7 REQs + standalone client + every surface swap + Reading Room fidelity + delegation-end); P1 is the widest (auth + partitioning across the 10-touchpoint list). The v0.2 pusher-class risk (integration drift at the swap) is pre-defused by G-005 wired into all three route-touching phases + the C3-18 exhaustive interface list.
- Answer: The pusher is P3 overrun with no pre-agreed valve — v0.2 had G-006; v0.3's plan had none. Bound in G-108.
- Confidence: 0.70
- Decision: G-108 — P3 may ship as two patches (P3a backend; P3b frontend) or run mid-phase wave gates, without scope change; tags extend on the v0.3.x line.
- **Q3**: Are the estimates evidence-based?
- Evidence: v0.1/v0.2 velocity does not transfer 1:1 (auth/crypto/hosting are new domains); no per-phase sizing exists. The honest mitigation remains the phase-patch mechanism — now with the P3 seam named in advance.
- Answer: No estimates to be wrong; the valve replaces fake numbers. Endorsed, per the v0.2 precedent.
- Confidence: 0.60
- **Q4**: Is there a working definition of done?
- Evidence: Per-phase must-haves are concretely testable (401 pins, two-user isolation, tamper pins, sweep test, drift gate, four smoke assertions); P5 owner gate. **One DoD term is undefined**: T-4.7/T-4.9 run the smoke "against a staged deploy" — "staged" is never defined, and the host runs one container today.
- Answer: DoD has teeth everywhere except the P4 smoke's target. Bound in G-107; its owner-contingency half escalates (G-111).
- Confidence: 0.75
- Decision: G-107 — "staged deploy" defined: candidate pair built + run on the LXC host, health green, the four smoke assertions executed host-locally against the candidates; public TLS-front cutover remains a discrete owner-directive act (D-019) with post-cutover public assertions run with the -prev pair armed.
### Axis 6 — Budget and Financial Realism
- **Q1**: What percentage of budget is spent vs. remaining?
- Evidence: MISSING — no cost ledger (v0.2 finding, carried). New v0.3 spends: second container family, argon2-cffi + cryptography (OSS), HAProxy front extension (existing instance), owner's Ollama key serving server-tier users.
- Answer: Untracked, small in absolute terms, tolerable for sponsor-beneficiary scope — but the server-tier exposure is a stated-nowhere cost posture (G-110).
- Confidence: 0.60
- **Q2**: Are there predictable cost drivers not in the original budget?
- Evidence: (a) Server-key tier: any admin-created user's keyless runs bill the owner (REQ-74 chain; G-007 counters advisory-only). (b) Host disk: deploy.sh (212 lines, verified) builds one image today with no free-space assert and no image prune; P4 doubles image families and runs 4 containers.
- Answer: Both are real; (a) is a policy posture to state (G-110), (b) is deploy hygiene to add (G-106).
- Confidence: 0.70
- Decision: G-106 — deploy.sh gains a pre-build free-disk assertion and a post-swap dangling-image prune; the two-image/rollback-pair footprint is documented as the host budget.
- **Q3**: Burn rate and runway?
- Evidence: Keyless localhost + DDG fallback keep the zero-marginal-cost degraded posture (G-008 carries); per-user keys shift burn to users; refine endpoints ride the same chain with honest 503.
- Answer: Runway effectively unbounded in degraded mode; hosting costs are fixed and small. Sound.
- Confidence: 0.70
- **Q4**: Is the budget contingent on something that hasn't happened yet?
- Evidence: Infra exists (LXC + HAProxy already TLS-fronting research.coreci.dev); keys exist in scopes. One new prerequisite: RESEARCHX_KEK does not exist yet (provisioned into `.ciagent/.env.secrets` at P2 — never git).
- Answer: Not contingent in any structural sense; the KEK is a trivial owner action — which is precisely why its loss posture must be stated before it becomes load-bearing (G-105).
- Confidence: 0.75
- Decision: G-105 — KEK-loss posture stated honestly: loss = per-user encrypted keys unrecoverable, chain degrades to server/keyless honestly, users re-enter keys under a re-provisioned KEK; the owner MUST back up the KEK off-repo; rotation tooling stays out of v0.3 (recorded, not silently dropped).
### Axis 7 — Risks, Assumptions, and Dependencies
- **Q1**: Top 3 assumptions, and evidence for each?
- Evidence: (a) *HAProxy live-host config accepts the SSE-safe front extension* — version and TLS front evidenced by two prior deploys; global/default sections unread until P4. Medium evidence. (b) *Single-worker in-memory registry/sessions suffice* — bounded by admin-managed accounts; restart sweep + pruning + disk read-models remove restart-fragility. Good evidence. (c) *Zero-component-rewrite surface swaps hold* — evidenced by the v0.2 P3 precedent + G-005 + T-3.12 regression suite; the task-toggle is the one discovered exception (G-109).
- Answer: Two well-evidenced; (a) is the residual, bounded by the P4 smoke + rollback pair.
- Confidence: 0.60
- **Q2**: External dependencies outside the team?
- Evidence: HAProxy host state (owner-held, D-019-gated), Ollama cloud/local + Tavily/DDG (carried v0.2, G-008 carries), Docker host disk (G-106 binds the missing hygiene).
- Answer: All acknowledged with degradation or rollback paths; the proxy is the only one without a code-level fallback, and it has a process-level one.
- Confidence: 0.65
- **Q3**: The single risk that kills the project, and is there a mitigation?
- Evidence: Authorization-class auth bugs on a public URL (cross-user leakage) — mitigated by construction: owner-keyed registry 404s + partitioned storage + per-request owner-scoped adapters + the two-user isolation test pinned in P1/P3 + the P4 exposure audit + admin-managed accounts bounding the victim set. The sweep-legality bug (G-101) is the found instance of the adjacent class.
- Answer: Kill risk is real but the mitigation stack is the plan's strongest suit; the found bug proves the stack needs the grill, and now it has the fix.
- Confidence: 0.70
- **Q4**: Has a pre-mortem been done?
- Evidence: MISSING as a formal artifact; this grill is the proxy.
- Answer: "12 months out, v0.3 failed" — derivable causes: (1) the sweep bug corrupting restarts at first redeploy (bound: G-101); (2) P3 fidelity parity slips — one found (toggle, G-109), the rest pinned by parity tests; (3) proxy/SSE iteration thrash post-cutover (bounded: smoke + rollback); (4) KEK lost with no documented posture (bound: G-105); (5) public brute-force/DoS on a fresh auth surface (bound: G-104). All five now bound or bounded-by-design.
- Confidence: 0.65
### Axis 8 — Governance, Decision-Making, and Communication
- **Q1**: Who is the decision-maker when two parties disagree?
- Evidence: Single owner; D-018/D-019 carry; escalation hooks codified (config.json).
- Answer: Unambiguous; no dual-executive stall possible.
- Confidence: 0.90
- **Q2**: Governance cadence and escalation pattern — useful in a crisis?
- Evidence: Per-phase ship + gates; the pipeline halted itself at the v0.2 gate correctly; deploy is owner-directive-only. **New v0.3 wrinkle**: P4's phase-ship DoD requires a host deployment that D-019 forbids without a directive — the first time a phase's success criteria are contingent on a mid-milestone owner action. T-4.9 acknowledges it but does not resolve the contingency.
- Answer: The escalation pattern is crisis-proven, but this dependency is new and unresolved — if the owner declines the directive at P4, REQ-83..85 ship unproven-on-host and the milestone gate presents hosted requirements on faith. Escalated as G-111.
- Confidence: 0.70
- **Q3**: What is being omitted from status reports?
- Evidence: `---ci---` honesty verified historically. The channel where optimism leaked this cycle is the research file: C3-11 asserts "verified states.py" for a claim that is false — asserted verification that was not re-run. This grill re-ran it.
- Answer: Status reporting remains honest; asserted-but-unre-run verification is the leak channel, and the fix is cultural (G-101's test pin makes the claim checkable).
- Confidence: 0.80
- **Q4**: Is there a stop-the-project trigger?
- Evidence: Owner rejection precedent (twice in v0.1); D-019 is the hosting-scope-specific stop trigger — no public cutover without directive; merge halts at the gate (D-018).
- Answer: Yes, and the hosting design adds a second one: the staged/public split (G-107) means exposure is a discrete owner decision, not a gradual slide.
- Confidence: 0.85
### Axis 9 — Change, Adoption, and Operational Readiness
- **Q1**: Who will use this, how does their work change, and what's in it for them?
- Evidence: Admin-managed accounts (REQ-70) — the owner creates the user set; work change = login + hosted real inference with per-user keys vs. local setup; UX Acceptance Criteria preserve the ≤3-click path with login as a once-per-session step.
- Answer: Small, known, sponsor-adjacent user set; adoption politics minimal, value concrete.
- Confidence: 0.70
- **Q2**: Is ops involved now, or handed a finished product at go-live?
- Evidence: The pipeline is ops; deploy.sh extends with paired rollback for both services (T-4.4); the startup sweep is the redeploy safety net (T-1.8); volume ownership handled (chown 10001, T-4.4). Gaps found: no disk assert/prune (G-106), undefined "staged" (G-107), KEK operational posture undocumented (G-105).
- Answer: Ops is built-in; three hygiene gaps bound.
- Confidence: 0.65
- **Q3**: What is the rollback plan?
- Evidence: Paired `-prev` containers for BOTH services (T-4.4); mock-dev default preserved at the code level (REQ-75/D-112); pre-hosted artifacts disposable (D-106); sessions durable across restarts (D-101).
- Answer: Multi-layer and codified — the strongest single dimension of this plan.
- Confidence: 0.80
- **Q4**: Have success criteria been validated with the people who will judge success?
- Evidence: ROADMAP v0.3 success criteria are concrete (401 curl, SSE unbuffered, two-user isolation, live-mode default on the staged instance); owner gate precedent enforces criteria. **Found**: REQ-78's own text promises "checkable follow-ups" while D-114/T-3.5 derive answered-iff-DONE over a GET-only route — the approved prototype's working checkbox (`library-view.tsx:463-471`) cannot hold live.
- Answer: One criterion-vs-implementation mismatch found; bound as honest degradation (G-109) rather than silent reversion.
- Confidence: 0.65
- Decision: G-109 — live-mode task status is derived (open unless run DONE); the inbox checkbox becomes display-honest in live mode (disabled/annotated; mock keeps the free toggle for dev); persisted per-question check state is v0.4+ scope (REQ-45 backlog); pinned by a live-mode component test.
### Meta — Closing Review
- **Q1**: If you were the auditor, what would you flag?
- Evidence: The false "verified" claim (C3-11 sweep legality vs `states.py` PLANNING); the REQ-75 triple mapping; the two stale out-of-scope rows contradicting the same-file supersession; C3-19 vs PLAN pitfall-phase mapping.
- Answer: Flag #1 is the sweep bug — a restart-path correctness claim that would have detonated at the first hosted redeploy, the exact moment the milestone's value proposition is under evaluation. Flags #2-4 are the recurring phase-mapping/bookkeeping class (G-001's relatives), now bound (G-102/G-103).
- Confidence: 0.80
- **Q2**: What is the project not doing that it should?
- Evidence: Found across axes: exposure hardening (password policy, login rate limit — G-104); KEK-loss posture (G-105); deploy disk hygiene (G-106); staged-deploy definition (G-107); P3 pressure valve (G-108); task-toggle honesty (G-109); cost posture statement (G-110).
- Answer: Seven omissions, all bound. Nothing found suggests the project is doing something it should not — engine-contract preservation holds everywhere checked.
- Confidence: 0.75
- **Q3**: What is the simplest version that delivers 80% of the value?
- Evidence: The owner explicitly picked "one milestone"; deferring live surfaces would re-create the G-009 mock-bound state the owner just moved past — the lever is not deferral.
- Answer: The pre-agreed lever is the P3 split (G-108), decided now while it is cheap — not surface deferral, which would contradict the owner's directive.
- Confidence: 0.65
- **Q4**: What must be true for this to succeed in the next 90 days, and is it true today?
- Evidence: (a) D-101..D-115 workable against the tree — re-verified (11/12 sampled exact, 1 false and bound). (b) Owner engagement — evidenced same-day. (c) The KEK provisioned by P2 — trivial owner action, posture bound (G-105). (d) The P4 staged host deployment authorized — currently NOT true; D-019 requires a directive the plan does not pre-arrange (G-111).
- Answer: (a)(b)(c) true or evidenced; (d) becomes true when the owner records the pre-authorization. That escalation is the gating action of this grill.
- Confidence: 0.70
### Binding Decisions
| ID | Axis | Decision | Confidence |
|----|------|----------|-----------|
| G-101 | 3/7 | Startup sweep must handle PLANNING-state runs: RESEARCH §7/C3-11's "CANCELLED legal from every non-terminal state" is false (`states.py`: PLANNING → {AWAITING_APPROVAL, FAILED}). Add PLANNING→CANCELLED to `_LEGAL_TRANSITIONS` or document the sweep as an explicit transition-validation bypass; pin with a PLANNING-fixture→CANCELLED sweep test in T-1.8. | 0.90 |
| G-102 | 2 | Fix the stale out-of-scope rows contradicting the recorded supersession: REQUIREMENTS.md Out-of-Scope ("Auth / multi-tenancy") and PROJECT.md Out of Scope ("multi-tenant/auth") — replaced with the honest v0.3 posture (auth/multi-user in scope; open self-serve registration and OAuth out; admin-managed accounts only, OAuth v0.4+). | 0.95 |
| G-103 | 2/5 | Phase-mapping integrity (G-001 precedent): bind REQ-75's split — delegation-end half closes in P3 (pinned by the standalone-client/no-mock-delegation tests), hosted-live-default half in P4 (pinned by the live-mode smoke marker); fix the traceability row (REQ-75..82 → REQ-76..82 + REQ-75 split note); declare PLAN's pitfall-closure mapping authoritative over C3-19's compressed phase prose. | 0.85 |
| G-104 | 2/3 | Public-exposure hardening, budgeted honestly: (a) minimum password length ≥12 enforced at register + admin-create (P1 T-1.4/T-1.5, uniform error messages preserved); (b) per-IP login rate limit at the HAProxy front (stick-table on /api/auth/*, P4 T-4.5) bounding argon2 CPU burn on the single worker; (c) the T-4.8 exposure audit asserts the throttle and confirms uniform 401s (no user enumeration). | 0.80 |
| G-105 | 6/7 | KEK-loss posture stated honestly (PLAN T-2.2 + owner docs): RESEARCHX_KEK's only copy is the env-file (never git); loss = per-user encrypted keys unrecoverable (AES-256-GCM + AAD), chain degrades to server/keyless honestly, users re-enter keys under a re-provisioned KEK; the owner MUST back up the KEK off-repo; rotation/re-wrap tooling stays out of v0.3 (recorded, not silently dropped). | 0.85 |
| G-106 | 6 | deploy.sh two-image hygiene: pre-build free-disk assertion, post-swap dangling-image prune, and a documented host budget (2 image families × live+prev, 4 containers) — the current script has neither, and P4 doubles the footprint on a single LXC host. | 0.75 |
| G-107 | 5/9 | "Staged deploy" defined: candidate pair (researchx + researchx-api) built + run on the LXC host, health checks green, the four smoke assertions executed host-locally against the candidates; public TLS-front cutover remains a discrete owner-directive act (D-019) with post-cutover public assertions (public 401, SSE through proxy) run with the -prev pair armed. | 0.75 |
| G-108 | 5 | G-006 precedent applied to P3: may ship as two patches (P3a backend artifacts + read-models + routes + read-model suite; P3b frontend standalone client + surface swaps + Reading Room fidelity + delegation-end) or run mid-phase wave gates, without scope change; tags extend on the v0.3.x line; the milestone release remains the final phase's patch. | 0.70 |
| G-109 | 9 | Task-inbox honesty (G-002 lesson): live-mode status is derived (answered-iff run DONE, D-114); the inbox checkbox (`library-view.tsx:463-471`) becomes display-honest in live mode — disabled/annotated, never a silent reversion; mock keeps the free toggle for dev; persisted per-question check state recorded as v0.4+ (REQ-45 backlog); pinned by a live-mode component test in T-3.12. | 0.75 |
| G-110 | 1/6 | Cost posture stated in PLAN/PROJECT: server-tier runs bill the owner's OLLAMA_API_KEY; the bound is the admin-managed account set (registration closed, REQ-70); per-user counters (P-13, T-2.6) surface server-tier usage per user for owner audit; no hard spend cap in v0.3 — hard caps are v0.4+ scope if the user set grows; keyless localhost remains the zero-cost path. | 0.70 |
### Escalations
- [G-111] Axis 5/8 — Owner pre-authorization required before P4 executes: P4's own definition of done ("staged smoke suite passes all four assertions", ROADMAP/PLAN T-4.7/T-4.9) requires deploying the candidate container pair to the LXC host — an action D-019 forbids without an owner directive. Without pre-authorization, P4 ships with REQ-83..85 unproven on the real host (the HAProxy 2.8 live config is unread until then) and the milestone gate would present hosting requirements on faith. The owner must either (a) pre-authorize staged host deployments for P4 DoD purposes (candidate pair + host-local smoke, NO public cutover — cutover stays a separate owner directive per G-107), or (b) accept that P4's phase ship carries hosting proven only on a local docker pair, with the host proofs deferred to the gate. This is the first phase in the project whose success criteria are contingent on a mid-milestone owner action; it must not be discovered at P4 ship time. Engineering-side handling is bound in G-107; the authorization itself is unresolved until the owner records it.
- **RESOLVED 2026-09-27: RATIFIED.** Owner approval recorded via /ci-run 'APPROVED' at the v0.2.5 gate (decision commit 4721c2a), following the owner-directed UAT staging at https://research.coreci.dev. Live-mode scope = core research loop only is the accepted MVP boundary; mock-bound surfaces move to v0.3+.
+17 -17
View File
@@ -1,4 +1,4 @@
# Personas — Milestone v0.3 (Hosted)
# Personas — Milestone v0.2 (MVP)
---
active:
@@ -8,8 +8,8 @@ active:
- name: ci-lead-developer
phase_specific:
- name: security-engineer
phases: [1, 2, 4]
reason: "v0.3 is the hosted multi-user milestone: P1 lands auth/sessions/CSRF/user-scoping, P2 lands the encrypted key store + credential chain, and P4 exposes the whole surface through a public proxy — security review is needed at all three, but not at P3 (live surfaces are read-model + binding work with no new secret handling) or P5 (the review phase runs the full multi-persona pass anyway)."
phases: [2]
reason: "v0.2 is the MVP: FastAPI backend + engine + adapters + trust loop (backend/data domains activate), the frontend consumes the OpenAPI-generated client in P3 (frontend stays active for the binding swap + P4 E2E), and the trust loop in P2 needs a phase-specific security-engineer for secrets handling and liveness verification. Lead-developer coordinates."
---
## Roster
@@ -18,8 +18,8 @@ reason: "v0.3 is the hosted multi-user milestone: P1 lands auth/sessions/CSRF/us
- **Domain**: backend
- **Active**: true
- **Phase-specific**: false
- **Frameworks**: fastapi, pydantic v2, argon2-cffi, cryptography, uv, uvicorn, sse-starlette, httpx, pytest, pytest-asyncio
- **Constraints**: single uvicorn worker mandatory (in-memory registry + session store); all external I/O lives in adapters — engine imports no vendor SDKs; no blocking calls in async paths (argon2/AES via `asyncio.to_thread`); durable JSON via atomic tmp+os.replace; auth guard is router-dependency-based (`Depends(require_user)`), never BaseHTTPMiddleware (SSE safety); RunRegistry owner-keyed `(owner_id, run_id)` + pruned; engine contracts unchanged (per-run adapter injection); `create_app(auth_enabled=True)` default with parameter-only dev override — never an env var readable in production images
- **Frameworks**: fastapi, pydantic v2, uv, uvicorn, sse-starlette, httpx, pytest, pytest-asyncio
- **Constraints**: src-layout package `researchx` in backend/; all external I/O lives in adapters — engine imports no vendor SDKs (httpx only inside adapters); no blocking calls in async paths; durable state via atomic tmp+os.replace; never swallow CancelledError; API layer is transport only (no business logic); single uvicorn worker (single-user)
- **Territory**: `backend/**`
### data-engineer
@@ -27,30 +27,30 @@ reason: "v0.3 is the hosted multi-user milestone: P1 lands auth/sessions/CSRF/us
- **Active**: true
- **Phase-specific**: false
- **Frameworks**: pydantic v2 (schemas), pytest
- **Constraints**: owns the new pydantic schema contracts: User/Session, per-user Settings, key-store envelope file, `RunState.owner_id` (serialization-compatible), report.json/evidence.json (marker normalization `p-N`→`pN`, frontend Report/Citation compatible), read-model payloads (library/tasks/suggestions/graph/profile); pure data + validation — no I/O in schema modules; every run persists resolved dial values (REQ-04 carries)
- **Territory**: `backend/src/researchx/config/**`, `backend/src/researchx/evidence/**` (+ schema review on `backend/src/researchx/auth/**` models)
- **Constraints**: owns the pydantic schema contracts: dial schema (REQ-01), run state, evidence store models (stable passage IDs), citation ledger JSON schema (REQ-21); pure data + validation — no I/O in schema modules; every run persists resolved dial values (REQ-04)
- **Territory**: `backend/src/researchx/config/**`, `backend/src/researchx/evidence/**`
### frontend-engineer
- **Domain**: frontend
- **Active**: true
- **Phase-specific**: false
- **Frameworks**: next.js (App Router), react 19, typescript (strict), tailwind, vitest, openapi-fetch, eventsource
- **Constraints**: components never import the mock or generated client directly — only the ApiClient boundary does; PLANNED interface additions (login/logout/getSession, admin hooks) documented before P1; fetch `credentials:"include"` + `X-ResearchX-Request` on all mutations; EventSource `withCredentials: true`; build-time NEXT_PUBLIC_* inlining profiles (mock dev default / live hosted with empty same-origin base); zero component rewrites in live swaps (REQ-54); no secrets or env keys in the frontend
- **Frameworks**: next.js (App Router), react 19, typescript (strict), tailwind, vitest, openapi-fetch, openapi-typescript
- **Constraints**: components never import the mock client or the generated client directly — only the ApiClient boundary does; RESEARCHX_API_MODE=mock|live selects the binding, mock default; no secrets or env keys in the frontend (NEXT_PUBLIC_API_BASE_URL only); zero component rewrites in the P3 swap (REQ-54/REQ-26)
- **Territory**: `frontend/**`
### security-engineer (phase-specific: P1, P2, P4)
### security-engineer (phase-specific: P2)
- **Domain**: security
- **Active**: true while P1/P2/P4 execute; deactivated between and after P4
- **Active**: false (P2 shipped — removed per phase-specific policy)
- **Phase-specific**: true
- **Frameworks**: argon2-cffi, cryptography (AESGCM), httpx, pytest, STRIDE analysis
- **Constraints**: KEK from env-file 'keys' scope only — never shell env, never logged, never in git, never in API responses (masked status only); AAD binds ciphertext to user_id; opaque tokens 32-byte urlsafe; SameSite=Lax + custom-header CSRF verified end-to-end; cross-user 404s provable by test; SSRF guard carries to any new fetch path; hosted exposure audit (401 on public /api, /healthz the only unauthenticated route); no `auth_enabled` env-var backdoor in production images
- **Territory**: `backend/src/researchx/auth/**` (P1/P2), `.ciagent/release/**` + hosting config review (P4)
- **Removal**: deactivate after P4 ships
- **Frameworks**: httpx, pytest, STRIDE analysis
- **Constraints**: secrets resolve from env-file scopes only (OLLAMA_API_KEY, TAVILY_API_KEY, GITEA_TOKEN via .ciagent/.env.secrets) — never shell env, never logged, never in git; liveness verification and trust-loop code must not leak keys in errors or traces; SSRF guard for fetched URLs (private-address refusal); authority tiering must not phone home
- **Territory**: `backend/src/researchx/trust/**`, `backend/src/researchx/adapters/secrets*/**`
- **Removal**: deactivate after P2 ships
### ci-lead-developer
- **Domain**: coordination
- **Active**: true
- **Phase-specific**: false
- **Frameworks**: ciagent workflows, git discipline
- **Constraints**: enforce persona territories; adapter/engine boundary review (I/O only in adapters); commit discipline with ---ci--- blocks; phase-boundary file updates only (no .ciagent/ writes during task execution); gates green before every ship; deploy.sh two-service extension (build both images, paired rollback containers, health on /healthz + /ask, live-mode build args asserted in the smoke test); secrets only from .ciagent/.env.secrets; no deploys without owner directive (D-019) and merge-to-main halts at the owner gate (D-018)
- **Territory**: `.ciagent/**`, repo root config files (README, VERSIONING, .gitignore, Dockerfiles/deploy scripts)
- **Constraints**: enforce persona territories; adapter/engine boundary review (I/O only in adapters); commit discipline with ---ci--- blocks; phase-boundary file updates only (no .ciagent/ writes during task execution); gates green before every ship
- **Territory**: `.ciagent/**`, repo root config files (README, VERSIONING, .gitignore)
+25 -244
View File
@@ -1,263 +1,44 @@
# PLAN — Milestone v0.3 (Hosted): Working Plan for Phases 1–4
# PLAN — Milestone v0.2 Phase 5 (FINAL): Review + Audit + Ship
---
phase: 0
milestone: v0.3
status: planned
phase: 5
milestone: v0.2
status: executed
must_haves:
- "Engine contracts unchanged: all v0.3 work lands outside the engine — per-run adapter injection, owner-scoped storage constructors, owner_id stamping via make_run_config; no methodology/engine protocol rewrites"
- "No secret values in git, logs, traces, or API responses: model keys masked (last 4 chars) at the API boundary and encrypted at rest (AES-256-GCM, KEK from the env-file 'keys' scope), passwords argon2id-hashed and never emitted"
- "User scoping provable end to end: runs/<user_id>/<run_id>/ partitioning, owner-keyed RunRegistry, disk read-models — cross-user access 404s, pinned by test in every phase that touches the data flow"
- "Every phase ships a patch (v0.3.1..v0.3.4; P5 tags v0.3.5 = the milestone release); gates green before every ship; pipeline halts at the merge_to_main owner approval gate (D-018); no deploys without owner directive (D-019)"
- "openapi.json snapshot committed and the generated frontend client regenerated from the snapshot (never a live server) in every phase that touches routes; schema-drift check wired into the gates (G-005)"
- "Zero-key test discipline: every CI suite runs on REQ-31 fakes with auth via create_app(auth_enabled=False) or dependency_overrides; keyed/live tiers are opt-in, skipped by default"
- "Single uvicorn worker in every serving mode — the in-memory RunRegistry and session store are single-process by design"
- "Mock demoted to dev/test fixture: the code default of RESEARCHX_API_MODE stays mock; after P3 no live-mode surface delegates to MockApiClient; hosted builds inline NEXT_PUBLIC_RESEARCHX_API_MODE=live (REQ-75)"
- "G-101: startup sweep handles PLANNING-state runs — add PLANNING→CANCELLED to _LEGAL_TRANSITIONS (or document the sweep as an explicit validation bypass); pinned by a PLANNING-fixture→CANCELLED sweep test in T-1.8"
- "G-104: public-exposure hardening — password minimum length ≥12 at register + admin-create (T-1.4/T-1.5); per-IP login rate limit at the HAProxy front (stick-table on /api/auth/*, T-4.5); exposure audit asserts the throttle + uniform 401s (T-4.8)"
- "G-105: KEK-loss posture documented (T-2.2 + owner docs): loss = per-user keys unrecoverable, honest degradation to server/keyless, re-entry under a re-provisioned KEK; owner backs up the KEK off-repo; rotation tooling out of v0.3 (recorded)"
- "G-106: deploy.sh hygiene — pre-build free-disk assertion + post-swap dangling-image prune + documented host budget (2 image families × live+prev)"
- "G-107: 'staged deploy' = candidate pair built + run on the LXC host, health green, four smoke assertions host-locally; public TLS-front cutover stays a discrete owner-directive act with -prev armed"
- "G-108: P3 pressure valve (G-006 precedent) — may ship as P3a (backend artifacts + read-models + routes + suite) and P3b (standalone client + swaps + fidelity + delegation-end), tags extending on the v0.3.x line, without scope change"
- "G-109: task-inbox honesty — live-mode status is derived (answered-iff run DONE); the checkbox is display-honest in live mode (disabled/annotated; mock keeps the free toggle); persisted check state is v0.4+ (REQ-45 backlog); pinned by a live-mode component test in T-3.12"
- "G-110: cost posture stated — server-tier runs bill the owner's OLLAMA_API_KEY; the bound is the admin-managed account set; per-user counters surface server-tier usage; no hard spend cap in v0.3 (hard caps v0.4+); keyless localhost remains the zero-cost path"
- "Multi-persona full-milestone review: 4 P0 + 15 P1 + 15 P2 found; all 4 P0s fixed in-phase with attack-proofs (A1 tautological anchoring → real writer-quote extraction; C2 gate denominator → cited-URL scoped; C1 zombie runs → shape-validated plans + catch-all FAILED persist; S1 fetch SSRF → per-hop re-guarding); 6 cheap P1s fixed (approve-with-plan, FAILED status, length caps, probe wiring, README env-var + adapter honesty)"
- "Audit: PASS-WITH-NOTES — reconstruction PASS (all REQ-01..34 covered across v0.2.0–v0.2.4), secrets clean (zero key values in history), branch hygiene exact; critical notes fixed in-phase (REQUIREMENTS traceability un-staled, P5 PLAN written, ROADMAP P5 status, security-engineer persona deactivated per its removal note)"
- "Milestone ship: v0.2.5 tagged as the MVP release candidate — the pipeline then HALTS at the merge_to_main owner approval gate (D-018)"
- "G-009 escalation presented at the gate: the owner ratifies that live mode covers the core research loop only (G-002); remaining prototype surfaces stay mock-backed to v0.3+"
personas:
- backend-engineer
- data-engineer
- frontend-engineer
- "security-engineer (phase-specific P1/P2/P4)"
- ci-lead-developer
requirements: [REQ-69, REQ-70, REQ-71, REQ-72, REQ-73, REQ-74, REQ-75, REQ-76, REQ-77, REQ-78, REQ-79, REQ-80, REQ-81, REQ-82, REQ-83, REQ-84, REQ-85]
requirements: [REQ-01, REQ-02, REQ-03, REQ-04, REQ-05, REQ-06, REQ-07, REQ-08, REQ-09, REQ-10, REQ-11, REQ-12, REQ-13, REQ-14, REQ-15, REQ-16, REQ-17, REQ-18, REQ-19, REQ-20, REQ-21, REQ-22, REQ-23, REQ-24, REQ-25, REQ-26, REQ-31, REQ-32, REQ-33, REQ-34]
---
## User-Facing Surface
The hosted multi-user product: the full approved frontend — a login screen; Ask → Choose → Plan & Run → Reading Room; the app shell with research library, project graph, task inbox, follow-up suggestions, refine dialogue, settings with masked model-key entry and resolved credential tier, avatar user menu with real profile — now live-bound end to end against the FastAPI backend with **zero live-mode delegation to the mock** (D-112), user-scoped (`runs/<user_id>/<run_id>/`), with per-user encrypted model keys resolving user → server → keyless per run, and deployed as a two-service stack (uvicorn single worker + Next.js live build) behind an SSE-safe HAProxy with TLS — real inference as the product default. Releases v0.3.0–v0.3.5.
The MVP release candidate: the approved v0.1 prototype frontend (Ask → Choose → Plan & Run → Reading Room, all 8 cards, shell, library, settings) now backed by a real FastAPI research engine in live mode — methodology registry (Rapid Answer + Deep Report), 12-dial framework, asyncio state machine with durable pause/cancel/resume, visited-only evidence store, attack-proven trust loop (quote anchoring, SSRF-guarded liveness, citation ledger, ≥99% export gate with zero-unanchored requirement), SSE live trace, adversarial suite (14 attacks), E2E happy path on the real stack. Releases v0.2.0–v0.2.5.
## Happy Path (owner review script)
1. `cd backend && uv sync && uv run pytest` — all suites green with zero API keys: auth suite (`test_auth.py`), key-store tamper tests, read-model tests, restart-sweep test, engine/trust/adversarial suites under the `auth_enabled=False` harness; `uv run pytest -m live` skips cleanly.
2. `cd frontend && corepack pnpm install && corepack pnpm test` — component + live-binding suites green; typecheck + lint green.
3. Local live-mode full loop: `uv run uvicorn researchx.api.main:app --port 8000` (env-file with `RESEARCHX_KEK` + server model key) + `NEXT_PUBLIC_RESEARCHX_API_MODE=live NEXT_PUBLIC_API_BASE_URL=http://localhost:8000 corepack pnpm dev` → http://localhost:3000 → register (first user = admin) → login → Settings: enter model key (shows masked status only), config shows resolved tier → Ask a question → Research this → pick Deep Report → Start research → approve the plan (≤3 clicks on defaults) → live SSE trace → Reading Room: hover a citation shows the stored quoted passage; ledger panel; export Markdown/JSON with real fields → Library shows the run in its auto-categorized folder; graph/tasks/suggestions surface the run's open questions; refine dialogue runs on the real planner.
4. Multi-user isolation: admin creates a second user; that user sees zero of the first user's runs, graph, tasks, or suggestions; the first user's key never affects the second user's resolved tier.
5. Hosted: `deploy.sh` → both services up + healthy; `curl https://<origin>/api/runs` → 401 unauthenticated; `curl https://<origin>/healthz` → 200; login through the public URL; SSE trace streams unbuffered through the proxy; per-user key honored end to end (public deployment per owner directive, D-019).
6. Owner decision at the P5 gate: approve → v0.3.5 = milestone release, merge to main; or direct changes → binding fix cycle.
1. `cd backend && uv sync && uv run pytest` — 129 tests green, zero API keys (live tier: `uv run pytest -m live`, skips cleanly)
2. `cd frontend && corepack pnpm install && corepack pnpm test` — 97 tests green (90 component + 5 live-binding + 2 real-stack E2E)
3. `uv run uvicorn researchx.api.app:app --port 8000` + `NEXT_PUBLIC_RESEARCHX_API_MODE=live corepack pnpm dev` → http://localhost:3000 → Ask a question → Research this → pick Deep Report → Start research → approve the plan (3 clicks) → live SSE trace → report with anchored citations → ledger → export Markdown (citation links) / JSON (backend-schema round-trip)
4. `uv run python -m researchx.engine.demo` — full pipeline on fakes, artifacts in a temp runs dir
5. Owner decision: approve → milestone merge to main + v0.2.5 published as the MVP release; or direct changes → binding fix cycle
## UX Acceptance Criteria
- The ≤3-click research path (Ask → Choose → Run on defaults) is preserved for an authenticated user; login is a once-per-session step (14-day sliding) adding zero clicks to the path.
- Reading Room parity with the approved mock: page-anchored citations whose hover shows the exact stored quoted passage; a real ledger panel; exports with real entry data — no fabricated fields anywhere.
- Honesty in live mode: resolved credential tier visible (user|server|keyless), key-store-disabled stated honestly, no synthetic suggestions, honest empty states for fresh users.
- Suggestions click-fills-the-box (never auto-starts); refine is opt-in and model-driven.
- Keys masked everywhere: no secret value is ever rendered, logged, or exported.
- Cross-user isolation is total and invisible: no surface leaks another user's runs, questions, or key status.
- ≤3 clicks from cold start to initiated research, validated on the real stack (REQ-17)
- Every trust claim attack-proven: hallucinated citations fail the export gate; fabricated quotes on real passage IDs flag unanchored; SSRF redirects die; pause/resume/cancel survive the loop
- Honest scope: live mode = core research loop (G-002); library/settings/suggestions/refine stay mock-backed until v0.3+ — ratified at this gate (G-009)
- No secret value in any artifact, trace, or log; zero-key test discipline throughout
## Planned ApiClient Extensions (C3-18 — documented before P1; this list is exhaustive)
## What this phase did
- `login(email, password): Promise<UserProfile>` — POST /api/auth/login (sets the session cookie)
- `logout(): Promise<void>` — POST /api/auth/logout
- `getSession(): Promise<UserProfile | null>` — GET /api/auth/me (401 → null)
- Admin hooks (API-first in v0.3; no dedicated admin screen): `listUsers()`, `createUser(input)`, `deactivateUser(id)` — GET /api/admin/users, POST /api/admin/users, POST /api/admin/users/{id}/deactivate
- MockApiClient gains matching no-op/local implementations (standalone mock preserved — REQ-31/REQ-54); live implements against the routes. The v0.2 pause/resume extensions (G-003) carry unchanged. No other interface changes are planned for v0.3.
1. **Review** (multi-persona, probe-verified): 4 P0 + 15 P1 + 15 P2. The 4 P0s broke the milestone's flagship guarantees and were all fixed with attack-proofs: quote anchoring extracted the stored passage's own quote (always 1.0) instead of the writer's text; the export-gate denominator counted unprobed visited pages as dead; malformed planner JSON left zombie runs; fetch adapters followed redirects without re-guarding. Six cheap P1s also fixed.
2. **Audit**: PASS-WITH-NOTES — reconstruction, secrets sweep (clean), branch/tag hygiene exact; both critical notes (stale traceability rows, missing P5 plan) fixed in-phase.
3. **Ship (this commit sequence)**: PR #14 (phase/05 → milestone), squash-merge, tag v0.2.5 — the MVP release candidate, release with the full milestone summary. Per D-018 and the merge_to_main escalation hook, the milestone merge to main does NOT proceed autonomously — the pipeline halts for the recorded owner decision, with G-009 (live-mode scope) presented at the same gate.
## Phase 1 — Auth + Sessions + User Scoping (ships v0.3.1)
## Scope Guard
**Personas** (lead-developer decomposition by territory): backend-engineer (lead — auth package, storage partitioning, engine/app wiring, sweep/pruning), data-engineer (auth + state pydantic contracts), security-engineer (phase-specific — auth adversarial suite + STRIDE), frontend-engineer (session surface, snapshot gate), ci-lead-developer (ship).
**Requirements covered**: REQ-69, REQ-70, REQ-71
**Task waves**:
*Wave 1 — contracts + stores (parallel)*
- **T-1.1 (backend-engineer)**: Data root + user store — `researchx.config.paths.data_root()` (RESEARCHX_DATA_DIR → `backend/var/researchx` dev; D-107); `auth/users.py`: User model (user_id = 8-char slug, validation `^[a-z0-9]{8}$`, D-105; email; argon2id via argon2-cffi `PasswordHasher()` defaults, hash/verify/rehash all in `asyncio.to_thread`, D-102; display_name; role; active; created_at), `users.json` at the data root, atomic rewrite (D-017 idiom). Pyproject gains `argon2-cffi>=23.1,<26` (no python-multipart).
- **T-1.2 (backend-engineer)**: Session store — `auth/sessions.py`: opaque `token_urlsafe(32)` tokens, in-process dict + durable `sessions.json` (atomic), `Session {token_hash, user_id, created_at, last_seen, expires_at}`, 14-day sliding expiry via `touch()` with >60s persist throttle, token rotation at login, immediate revocation (logout deletes the presented token; deactivation deletes all the user's tokens) (D-101).
- **T-1.3 (data-engineer)**: Auth + state contracts — pydantic User/Session schemas, auth route request/response models, `RunState.owner_id: str | None = None` (serialization-compatible, no migration per D-106); old state files must deserialize.
*Wave 2 — enforcement (depends W1)*
- **T-1.4 (backend-engineer)**: Guard + auth routes + CORS — `auth/guard.py` `require_user` dependency (session resolve → 401; `X-ResearchX-Request` header required on non-GET → 403; inactive user → 401; sliding touch); `auth/routes.py` register/login/logout/me with httpOnly `researchx_session` cookie (SameSite=Lax, path=/, Secure hosted-only flag; no Domain attribute); uniform 401 "invalid email or password"; register open only while the user store is empty — first user `role="admin"`, then 403 (no await between check and insert except the argon2 hash; single worker = race-free, D-104); public router (`/api/auth/*`, `/healthz` outside /api per D-103) + protected router (everything else) via `APIRouter(dependencies=[Depends(require_user)])` — **router dependencies, not middleware** (C3-01; no BaseHTTPMiddleware near SSE); CORS: `allow_credentials=True` + explicit localhost origins, ordered so preflight OPTIONS answers without 401 (C3-04); login/register POSTs also require the CSRF header (C3-05).
- **T-1.5 (backend-engineer)**: Admin routes — POST /api/admin/users (admin-guarded create with admin-set initial password), GET /api/admin/users (list), deactivate (kills all sessions; data retained) — REQ-70.
- **T-1.6 (backend-engineer)**: Storage partitioning — `LocalStorageAdapter(base_dir, owner_id)` with **required** owner_id (validated), `run_dir → base_dir/runs/<owner_id>/<run_id>/`, + `list_run_ids()` (read-models scan later); `StorageAdapter` protocol gains `append_run_artifact` (the actor already calls it); `read_run_artifact` error message embeds the owner path; consolidate the duplicate run-id generators (P-14). Protocol otherwise byte-for-byte unchanged (D-106).
*Wave 3 — engine/app wiring (depends W2)*
- **T-1.7 (backend-engineer)**: Owner-scoped app wiring — `make_run_config(..., owner_id)` stamps `state.owner_id`; RunActor untouched (engine contracts unchanged); `RunRegistry` keyed `(owner_id, run_id)` — cross-owner access 404; routes construct the owner-scoped storage per request from the auth dependency; `create_app(*, auth_enabled=True)` default with **parameter-only** dev override resolving a synthetic dev user (`user_id="dev00000"`, role admin — C3-16, never an env var readable in production); `dev_bridge.py` stays explicitly `auth_enabled=False` (P-5); `engine/demo.py` owner arg (P-9); remove the dead duplicate `return EventSourceResponse` (app.py:331, P-11); mechanical `owner_id` constructor args across the 8 test files (RESEARCH §5 touchpoint list).
- **T-1.8 (backend-engineer)**: Startup sweep + registry pruning — on `create_app` startup, rewrite every non-terminal `state.json` under the data root as CANCELLED (D-115; legal from every non-terminal state; in-flight actors die with the process); registry drops terminal `(actor, bus)` entries once no live SSE subscriber (refcount in the events route — P-6); events-404-after-prune/restart documented (trace persists on disk).
*Wave 4 — adversarial + frontend + snapshot (depends W3)*
- **T-1.9 (security-engineer + backend-engineer)**: Auth suite — `backend/tests/test_auth.py` with `auth_enabled=True` (RESEARCH §11): first-user-admin (+403 register after), wrong-password 401 uniform, logout revocation, 401 on all protected routes, CSRF header rejection, cross-user 404 (all run endpoints), admin create/deactivate (deactivation kills sessions), session expiry, cookie flags, no password-hash material in any response; pins P-5 and the authorization half of P-8.
- **T-1.10 (frontend-engineer)**: Session surface — ApiClient extensions per C3-18 (mock no-ops; live against routes); login screen (email + password, error states); live-mode route guard redirecting unauthenticated users to /login; user menu shows the real profile + logout; fetch discipline: `credentials:"include"` + `X-ResearchX-Request` header on every mutation (C3-05); dev docs pin same-hostname origins (`localhost` both sides — port is not part of same-site computation).
- **T-1.11 (frontend-engineer)**: Snapshot regen gate — export + commit `backend/openapi.json` (auth/admin routes included), regenerate `frontend/src/lib/api/generated/` from the snapshot only, wire the schema-drift check into the gates, operation-ids stable (G-005).
*Wave 5 — ship*
- **T-1.12 (ci-lead-developer)**: Phase ship — all gates green (backend ruff/mypy/pytest incl. `test_auth.py`; frontend lint/typecheck/vitest; drift gate), tag **v0.3.1** + release; REQUIREMENTS/ROADMAP traceability rows updated.
**Must-haves (verification-oriented)**:
- Unauthenticated access to any `/api` route except register/login → 401; register 403s once the user store is non-empty — pinned by `test_auth.py`.
- First registered user is admin; all subsequent accounts admin-created; deactivation kills sessions and retains data.
- No user can read or act on another user's data: owner-keyed registry → cross-owner 404 on run GET/lifecycle/report/ledger/events — pinned by test.
- Storage layout is `runs/<user_id>/<run_id>/`; `owner_id` recorded in every persisted run state.
- Sessions are server-side with httpOnly SameSite=Lax cookies, immediate revocation, sliding expiry; CSRF header enforced on non-GET (EventSource GET exempt).
- Engine contracts unchanged; existing zero-key suites green with only mechanical `owner_id` constructor changes.
- `openapi.json` snapshot committed; generated client regenerated from it; drift gate fails on intentional drift (proof).
- No run survives a restart in a non-terminal state (sweep test); terminal registry entries prune (leak test).
- No password hash or key material in any API response or log.
**Pitfall closures (RESEARCH §12)**: **P-4** (CORS `allow_credentials` + ordering), **P-5** (dev_bridge explicitly auth-off, pinned by the suite), **P-6** (registry pruning), **P-7** (owner_id + partitioning), **P-9** (demo.py owner arg), **P-10** (layout test re-pinned to the partitioned path), **P-11** (dead code removed), **P-14** (run-id generators consolidated). P-8's authorization half (owner-keyed 404s) closes here; its registry-independence half closes in P3.
**Success criteria (testable)**:
- `test_auth.py` green covering the full §11 list; two-user isolation test proves zero visibility across every run endpoint.
- Register → admin; second register → 403; admin creates + deactivates (sessions die) — all route-tested.
- Non-GET without the header → 403; with header + session → 2xx.
- Disk layout test proves `runs/<owner>/<run_id>/`; restart-sweep test proves non-terminal → CANCELLED across an app restart; pruning test proves terminal entries drop with zero SSE subscribers.
- Drift gate: intentional schema change fails the gate; regen green.
- Gates green; patch v0.3.1 shipped.
## Phase 2 — Per-User Settings + Model Keys (ships v0.3.2)
**Personas**: backend-engineer (lead — key store, settings store, model factory, config rework, production entrypoint), data-engineer (settings + key-envelope schemas), security-engineer (phase-specific — key-store adversarial suite), frontend-engineer (settings live-binding, snapshot), ci-lead-developer (ship).
**Requirements covered**: REQ-72, REQ-73, REQ-74
**Task waves**:
*Wave 1 — contracts*
- **T-2.1 (data-engineer)**: Settings + key-envelope schemas — per-user Settings payload (model names, methodology defaults + resolved dials, appearance slot, masked key status, storage path display-only — D-111); key-store envelope `{v, alg, updated_at, entries: {OLLAMA_API_KEY: {nonce, ct, updated_at}}}` (D-108/D-109); KEK decode rules (urlsafe-b64 or hex, exactly 32 bytes; decode/length errors are typed AdapterErrors).
*Wave 2 — stores (depends T-2.1)*
- **T-2.2 (backend-engineer)**: Key store — `auth/key_store.py`: AES-256-GCM via `cryptography` AESGCM (pyproject gains `cryptography>=43`); KEK = `RESEARCHX_KEK` from the env-file **'keys'** scope (`config.json` secrets.scopes gained `"keys": ["RESEARCHX_KEK"]` in P0; provisioned into `.ciagent/.env.secrets` only — never git); AAD = `user_id` (a copied file under another user fails `InvalidTag`); fresh `os.urandom(12)` nonce per write (never reused under the KEK); `keys/<user_id>.json`, atomic, 0600; crypto in `asyncio.to_thread`; honest degradation: no KEK → disabled (PUT rejected with `{"key_store": "disabled"}`), corrupt entries surface as errors, never a silent tier drop (C3-06). KEK change requires process restart (documented).
- **T-2.3 (backend-engineer)**: Settings persistence — `settings/<user_id>.json` store, atomic rewrite, defaults for fresh users, GET/PUT payload shapes (D-111).
*Wave 3 — routes + factory (depends W2)*
- **T-2.4 (backend-engineer)**: Settings + key routes — GET/PUT `/api/config/settings`; model-key PUT accepts a raw key (scope = OLLAMA_API_KEY only, D-109) → encrypted at rest; every response returns configured + masked (last 4) only — the raw key never appears in any response, log, or trace (REQ-73).
- **T-2.5 (backend-engineer)**: Per-run model factory — `adapters/model_factory.py`: `ModelAdapterFactory(secrets, key_store, defaults).resolve(user, settings) → (adapter, tier)` resolving user → server → keyless (REQ-74); `OllamaModelAdapter` constructor gains optional `api_key` (precedence over secrets resolve; secrets path back-compat); **one adapter instance per run** — the `_active_base` fallback mutation is per-instance, never shared (P-2 closure); `create_run` resolves + injects (engine contracts unchanged); state records `owner_id` + `credential_tier`.
- **T-2.6 (backend-engineer)**: Config status rework — GET `/api/config` resolves the **current user's** tier via the factory without creating a run (kills the `runs._actors` private read — P-1 closure; no cross-user probe leak, no zero-run breakage); probe cached in-process keyed `(user_id, tier, key_fingerprint)` with short TTL, running against the resolved tier; per-user cost counters owner-keyed (P-13 closure; G-007 carries); `resolved_tier` + `key_store: enabled|disabled` surfaced.
*Wave 4 — production + adversarial + frontend (depends W3)*
- **T-2.7 (backend-engineer)**: Production entrypoint — new `researchx.api.main` wiring `create_app(data_root=RESEARCHX_DATA_DIR, factory=<live factory from env-file secrets>, auth_enabled=True)` (P-3 backend closure — the `app.py:359` module global serving fakes + cwd storage leaves production); `app.py` stays the test factory; `main.py` never passes `auth_enabled=False`.
- **T-2.8 (security-engineer + backend-engineer)**: Key-store adversarial suite — tamper tests: AAD binding (key file copied under another user_id → `InvalidTag` surfaced), nonce uniqueness across writes, masked-only responses (no raw key material in response/log/trace), disabled-honesty without KEK, corrupt-entry error surfacing; opt-in keyed tier (skipped by default; zero-key CI unchanged) proving a keyed user gets real inference end to end through the REQ-74 chain.
- **T-2.9 (frontend-engineer)**: Settings live-binding — `GeneratedApiClient` getSettings/saveSettings → `/api/config/settings`; masked key entry (password-type input → PUT; masked status display; never echoed); getConnectionStatus maps GET `/api/config` (resolved tier user|server|keyless, key_store status, probe, per-user counters); storage directory display-only in live/hosted mode (D-111 — the free-path picker is the local single-user profile only); mock parity preserved standalone.
- **T-2.10 (frontend-engineer)**: Snapshot regen — settings/key routes change `openapi.json` → snapshot committed, client regenerated, drift gate green (G-005).
*Wave 5 — ship*
- **T-2.11 (ci-lead-developer)**: Phase ship — gates green; tag **v0.3.2** + release; traceability updated.
**Must-haves**:
- Keys never appear in full in any API response, log, or trace — masked only, pinned by test.
- Key store: AES-256-GCM, AAD=user_id, fresh nonce per write, 0600, atomic — tamper-proof pinned by test.
- Without a KEK: key store disabled honestly; resolution proceeds server/keyless; never a silent tier drop.
- Per-run credential chain user → server → keyless; probe runs against the resolved tier; config status surfaces the current user's tier without creating a run and without reading private registry state.
- Shared-adapter fallback bug dead: one adapter per run; `_active_base` mutation can never cross users.
- Cost counters owner-keyed.
- Engine contracts unchanged; zero-key suites green (fakes, `auth_enabled=False`).
- Production entrypoint live-wired; the module-global fake app serves nothing.
**Pitfall closures**: **P-1** (config_status → factory-resolved probe), **P-2** (per-run factory kills the shared `_active_base` mutation), **P-13** (owner-keyed cost counters), **P-3 backend half** (`researchx.api.main`; the hosting half closes in P4).
**Success criteria**:
- Opt-in keyed tier: a keyed user gets fully real inference end to end; `resolved_tier="user"` in config status.
- Zero-key CI: full suite green with no KEK configured — key store reports disabled, tier falls to server/keyless honestly.
- Tamper suite green: AAD tamper → InvalidTag surfaced; nonces unique; masked-only responses.
- Config status correct with zero runs and per-user isolated (no probe leak).
- Drift gate green; gates green; patch v0.3.2 shipped.
## Phase 3 — Live Surfaces, user-scoped (ships v0.3.3)
**Personas**: backend-engineer (lead — artifacts, read-models, all new routes), data-engineer (artifact + read-model payload schemas), frontend-engineer (standalone client, all surface swaps, Reading Room fidelity), ci-lead-developer (ship). Security-engineer inactive this phase (no new secret handling — roster reason).
**Requirements covered**: REQ-76, REQ-77, REQ-78, REQ-79, REQ-80, REQ-81, REQ-82 (+ REQ-75 delegation-end half: after this phase no live-mode surface delegates to the mock)
**Task waves**:
*Wave 1 — artifacts + read-models (parallel)*
- **T-3.1 (backend-engineer + data-engineer)**: report.json + evidence.json at DONE — persisted inside `_verify_phase` right after ledger/checklist (EvidenceStore is GC'd with the actor — this is the only write point; D-113): `report.json {v, run_id, owner_id, title, question, methodology_slug, generated_at, sections:[{heading, body}], citations:[{passage_id, url, title, quote, health, authority_tier}]}`, `evidence.json {v, run_id, passages:[{passage_id, url, title, quote, text}]}`; **marker normalization at the persistence boundary only**: section bodies `[p-N]` → `[^pN]`, ids `p-N` → `pN` (the Reading Room parses with zero frontend changes); `report.md` keeps raw `[p-N]` — trust-loop regexes untouched (C3-10).
- **T-3.2 (backend-engineer + data-engineer)**: Read-models — `api/read_models.py` scans `runs/<owner_id>/` loading `state.json` (+ report.json/ledger.json where needed); skips dotfiles/tmp defensively (D-115); payload schemas shaped to the existing frontend types: library folders (auto-categorization by methodology + deterministic topic tokens from plan sub-questions) + ReportSummary, ReportConnection, ProjectGraph nodes/edges, InboxItem `{id: "<run_id>#<idx>", question, sourceRunId, status}`, SuggestedQuery.
- **T-3.3 (backend-engineer)**: Enriched report endpoint — GET `/api/runs/{id}/report` → `{run_id, markdown, report, evidence}` (markdown retained, back-compat); **disk-read, registry-independent** (P-8 full closure); graceful markdown-only fallback for pre-artifact DONE runs (old artifacts disposable per D-106, but never a 500); ledger endpoint likewise disk-read.
*Wave 2 — routes (depends W1)*
- **T-3.4 (backend-engineer)**: Library + connections + graph — GET `/api/library?sort=recent|title` (folders + ReportSummary[] resolving to real runs); GET `/api/library/connections` (shared-source edges ≥1 ledger-URL overlap + topical edges); GET `/api/graph` (same edges in ProjectGraph shape) — REQ-76/77, D-114.
- **T-3.5 (backend-engineer)**: Tasks + suggestions — GET `/api/tasks` (open questions = plan sub-questions, answered-iff run reached DONE); GET `/api/suggestions` (open items joined with source run titles; no synthetic suggestions; empty state for fresh users) — REQ-78/79, D-114.
- **T-3.6 (backend-engineer)**: Refine + profile — POST `/api/refine/questions {draft}` → 2–3 planner clarifying questions; POST `/api/refine/compose {draft, answers}` → composed refined question + suggested context; planner calls ride the **same per-user credential chain** (P2 factory), honest 503 for a keyless user with no local endpoint — never a hang; GET/PUT `/api/profile` (display_name, email; no deletion) — REQ-80/81. All owner-scoped via the auth dependency; non-GET ⇒ CSRF header applies.
*Wave 3 — frontend standalone (depends routes + snapshot)*
- **T-3.7 (frontend-engineer)**: Snapshot regen — all new routes into the committed `openapi.json` snapshot; client regenerated; drift gate green (G-005).
- **T-3.8 (frontend-engineer)**: GeneratedApiClient standalone — every ApiClient method overridden against real endpoints; `extends MockApiClient` **dropped** (a standalone class implementing ApiClient — D-112); request wrapper: `credentials:"include"` + `X-ResearchX-Request` on every mutation; `EventSource` constructed with `{withCredentials: true}`; mock remains a standalone dev/test fixture.
- **T-3.9 (frontend-engineer)**: Surface swaps — getReportLibrary, getReportConnections, getProjectGraph, getTaskInbox, getSuggestedQueries, getRefinementQuestions, composeRefinedQuestion, getUserProfile, plus the P2 settings/config methods — all live, zero component rewrites (REQ-54); suggestions click-fills never auto-starts; the refine drawer drives the real exchange.
- **T-3.10 (frontend-engineer)**: Reading Room fidelity + honest exports — getReport maps report.json → Report (sections + citations with **real quotes**; tier map official→1, reference→2, else 3); getLedger real entries; exportReport built from real entry data (kills the fabricated `claims_supported: 1` and empty quotes — P-12 closure, REQ-82); hover shows the stored quoted passage — parity with the mock Reading Room v0 (REQ-82); exports round-trip through the backend-schema gate (REQ-34 carries) with citation links intact.
*Wave 4 — tests + ship (depends W3)*
- **T-3.11 (backend-engineer + data-engineer)**: Read-model test suite — `backend/tests/test_read_models.py`: a fixture disk tree (two users; DONE, CANCELLED, in-flight, FAILED runs) asserting library folder categorization + recency/title sorting; shared-source edges (≥1 ledger URL overlap) + topical edges; graph node/edge shape; tasks answered-iff DONE; suggestions = open items joined with run titles, empty for a fresh user, never synthetic; dotfile/tmp skip; **registry-independence test**: all read endpoints correct against a fresh `create_app` over an existing data root with zero in-memory actors (P-8 full-closure proof); artifact tests: report.json/evidence.json written at DONE with normalized markers (`[^pN]` in report.json, raw `[p-N]` in report.md), markdown-only fallback for pre-artifact DONE runs (never a 500).
- **T-3.12 (frontend-engineer)**: Live-surface regression suite — component tests run unchanged against the live binding (zero component rewrites proven, REQ-54); ≤3-click path pinned green in live mode; suggestions click-fills-the-box never auto-starts; refine exchange drives the real endpoint on fakes; drift gate green.
- **T-3.13 (ci-lead-developer)**: Phase ship — all gates green; tag **v0.3.3** + release; traceability rows updated.
**Must-haves**:
- Every prototype surface (library, connections, graph, tasks, suggestions, refine, profile, settings, Reading Room) works against real user data in live mode — `GeneratedApiClient` is standalone, `extends MockApiClient` is gone (D-112), and no live-mode method delegates to the mock.
- Read-models read disk artifacts only, never RunRegistry: every read endpoint is correct after a restart with zero in-memory state; non-terminal runs post-sweep appear as CANCELLED, terminal runs stay readable.
- Reading Room parity with the mock: structured sections, page-anchored citations whose hover shows the stored quoted passage, ledger panel from real ledger.json; exports carry real quotes/claims/health/tier — no fabricated fields.
- Engine persists report.json + evidence.json at DONE inside `_verify_phase`; `report.md` marker syntax untouched (trust-loop regexes unchanged).
- All new routes owner-scoped via the auth dependency — cross-user 404s pinned; non-GET routes CSRF-header-guarded (refine POSTs included).
- ≤3-click path green; suggestions never auto-start; honest empty states; no synthetic suggestions.
- Snapshot committed, client regenerated from the snapshot only, drift gate green (G-005).
- Zero-key discipline intact: refine tests on REQ-31 fakes; keyless refine surfaces an honest 503, never a hang.
**Pitfall closures**: **P-8 full closure** (registry-independent disk reads — auth half closed in P1, read half here, pinned by the restart test), **P-12** (export fabrication killed — real quotes/claims from report.json/evidence.json). Marker-normalization hazard (§6) closed by construction: normalization at the persistence boundary only, `report.md` raw form pinned by test.
**Success criteria**:
- Read-model suite green over the multi-user fixture tree; fresh-app-over-existing-data-root test proves every read endpoint without the registry.
- Live mode: all surfaces live-bound, component tests unchanged, 3-click green, no delegation to mock anywhere in the live client.
- Export JSON payloads contain real quotes, real claims_supported, real health/tier — the fabricated `claims_supported: 1` / empty quotes are gone and a test pins their absence.
- Fresh user: empty library, empty tasks, empty suggestions — honest, no synthetic content.
- Drift gate green; gates green; patch v0.3.3 shipped.
## Phase 4 — Hosting + Real-Inference Default (ships v0.3.4)
**Personas**: ci-lead-developer (lead — deploy.sh, HAProxy config, hosting per its repo-root territory), backend-engineer (Dockerfile, production wiring completion, health endpoint), frontend-engineer (live build profile, build args), security-engineer (phase-specific — hosted exposure audit through the public URL).
**Requirements covered**: REQ-75, REQ-83, REQ-84, REQ-85
**Task waves**:
*Wave 1 — container + production wiring*
- **T-4.1 (backend-engineer)**: Backend Dockerfile — `python:3.12-slim` + uv from the ghcr image; `uv sync --frozen --no-dev`; `useradd --system --uid 10001 researchx`; `mkdir -p /var/lib/researchx` + chown; `USER researchx`; `CMD ["uvicorn", "researchx.api.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1", "--proxy-headers"]` (REQ-83, C3-13). Secrets are runtime-only via `--env-file` (`RESEARCHX_KEK`, `OLLAMA_API_KEY`, `OLLAMA_BASE_URL`, `TAVILY_API_KEY`) — never baked into the image, never in git.
- **T-4.2 (backend-engineer)**: Production wiring completion — `researchx.api.main` verified end to end: `data_root=RESEARCHX_DATA_DIR` (container default `/var/lib/researchx`), live model factory from env-file secrets, `auth_enabled=True` hardwired — **P-3 full closure**: no production path can serve FakeModelAdapter or cwd-based storage; parameter-only auth override confirmed absent from the image (no `RESEARCHX_AUTH`-style env readable anywhere); uvicorn graceful-shutdown behavior documented (in-flight runs exercise `_persist_cancelled` on SIGTERM; `docker rm -f` SIGKILLs — the P1 startup sweep is the safety net).
*Wave 2 — frontend live build profile*
- **T-4.3 (frontend-engineer)**: Live build profile — frontend Dockerfile build stage gains `ARG`/`ENV` before `pnpm build`: `NEXT_PUBLIC_RESEARCHX_API_MODE=live` + `NEXT_PUBLIC_API_BASE_URL=""` (empty = same-origin through HAProxy's `/api` path route); `baseUrl()` treats `""` as same-origin (openapi-fetch relative base, EventSource relative URL); build asserts the empty base (a non-empty base would break same-origin cookies); dev live-mode docs pin `http://localhost:8000` on both sides (hostname rule, C3-15).
*Wave 3 — deploy + proxy*
- **T-4.4 (ci-lead-developer)**: deploy.sh two-service extension — rsync now includes `backend/`; both images built on the host; plain docker-run pairs on the existing LXC idiom (no compose dependency): `researchx-net` bridge network; `researchx-api` backend (env-file, data volume bind, `-p 127.0.0.1:8000:8000` with host-side HAProxy, `--restart` policy); `researchx` frontend (`-p 3000:3000`, live build args); paired rollback containers `researchx-api-prev` + `researchx-prev` both retained; data volume pre-created + `chown 10001` host-side (belt-and-braces against EACCES); health check on BOTH `/healthz` (backend) and `/ask` (frontend) before the swap; idempotent re-run.
- **T-4.5 (ci-lead-developer, security-engineer review)**: HAProxy front config — documented + idempotent update to the host's existing HAProxy 2.8 TLS frontend: `acl is_api path_beg /api`, `acl is_health path /healthz` → `backend_api`; `default_backend frontend_next`; `backend_api`: `option httpchk GET /healthz`, `default-server inter 5s fall 3 rise 2`, `server api1 127.0.0.1:8000 check`, `timeout server 3600s`, `timeout tunnel 3600s`; **NO `option http-buffer-response`, NO compression on the API backend** (gzip buffers text/event-stream); `backend frontend_next`: `timeout server 300s`, `server next1 127.0.0.1:3000 check` (REQ-84, C3-14); `timeout client` exceeds the sse-starlette 15s ping interval; the config file is committed as documentation; application to the live host only on owner deploy directive (D-019).
- **T-4.6 (backend-engineer)**: `/healthz` verification — unauthenticated `GET /healthz → 200 {"status":"ok"}` (public router from P1, outside `/api` per D-103) proven through the proxy path route.
*Wave 4 — smoke + exposure audit + ship*
- **T-4.7 (ci-lead-developer)**: Hosted smoke assertions — scripted against a staged instance: (1) unauthenticated `curl` to public `/api/runs` → **401**; (2) `/healthz` → **200**; (3) **live-mode marker**: authenticated `/api/config` resolves a real tier and the smoke asserts the build inlined live mode (empty base + live mode, not mock); (4) **SSE-through-proxy smoke**: an authenticated EventSource against the public origin streams a full run's events unbuffered end to end — no buffering, no mid-stream cut, 15s pings hold the connection through `timeout server 3600s` (REQ-84/85).
- **T-4.8 (security-engineer)**: Hosted exposure audit — through the public URL: every `/api` route 401s unauthenticated (401-on-public-/api pinned); `/healthz` is the ONLY unauthenticated route; no `auth_enabled` backdoor in the production image (parameter-only, `main.py` never passes False — verified); cookie `Secure` flag set on the TLS-fronted host; secrets sweep over the built image layers and deploy artifacts (zero key values anywhere); STRIDE pass on the proxy surface.
- **T-4.9 (ci-lead-developer)**: Phase ship — all gates green; tag **v0.3.4** + release; traceability updated; deployment to the live host staged only on owner directive (D-019) — the smoke suite is written to run against a staged deploy.
**Must-haves**:
- One-command deploy brings up both services; paired rollback containers exist for BOTH services (REQ-83).
- Backend container: non-root uid 10001, single uvicorn worker (`--workers 1` mandatory — in-memory registry + session store), secrets runtime-only via `--env-file`, data volume writable by uid 10001.
- The hosted instance runs real inference in live mode by default (build-time inlined; REQ-75) with per-user keys honored through the full chain — while the code default stays mock (zero-key dev/CI preserved, REQ-31/D-112).
- SSE streams unbuffered through the proxy: no http-buffer-response, no API compression, timeout server/tunnel 3600s, pings unbroken — verified end to end.
- Unauthenticated public `/api` → 401 on every route; `/healthz` 200 the only unauthenticated route (REQ-85).
- No secret values baked into images, committed, or logged; volume chown 10001.
- HAProxy changes documented + idempotent; applied only on owner directive (D-019).
**Pitfall closures**: **P-3 full closure** (hosting half — the container serves `researchx.api.main` with the live factory; nothing can host fakes). RESEARCH §8/§9 hazards encoded and tested: volume-ownership EACCES (chown 10001 + smoke write), SSE-compression hazard (explicitly absent from the API backend config, asserted in review), ping keepalive (SSE smoke holds ≥ several ping intervals), SIGKILL-on-redeploy (startup sweep as the net, covered by the P1 sweep test).
**Success criteria**:
- Both images build on the host via the extended deploy.sh; staged smoke suite passes all four assertions (401 / healthz 200 / live-mode marker / SSE unbuffered).
- A keyed user on the staged instance gets fully real inference end to end through the proxy (`resolved_tier="user"` visible in the authenticated UI).
- Image-layer secrets sweep clean; no auth-disable path exists in the production image.
- Gates green; patch v0.3.4 shipped.
## Phase 5 — Final Review + Ship (FINAL — not planned in detail here)
Per ROADMAP: multi-persona review of the full milestone diff (REQ-69..85), audit (reconstruction, secrets sweep, branch/tag hygiene), then tag **v0.3.5 = the milestone release**. The pipeline HALTS at the merge_to_main owner approval gate (D-018 carries); deployment to research.coreci.dev proceeds only on owner directive (D-019 carries). The security-engineer's removal note (deactivate after P4) executes at P4 ship; the P5 review runs the standard full multi-persona pass. Detailed planning lands as the P5 PLAN when P4 completes, per the v0.2 precedent.
## Cross-Phase Notes
- **Versioning**: patches v0.3.1 (P1) → v0.3.2 (P2) → v0.3.3 (P3) → v0.3.4 (P4) → v0.3.5 (P5, the milestone release); tags on the v0.3.x line; MAJOR stays 0.
- **Carrying gates**: G-005 snapshot discipline applies in every route-touching phase (P1, P2, P3); G-007 cost counters survive as owner-keyed (P-13, P2); G-008 DDG fixture discipline carries untouched; D-018/D-019 halt + deploy gates bind at P4/P5.
- **REQ coverage assert**: each of REQ-69..85 maps to ≥1 test or contract proof in its phase (auth suite, key-store tamper suite, read-model suite, hosted smoke) — the audit reconstructs the milestone from these.
- **Interface additions**: the C3-18 list is exhaustive; any other ApiClient change requires a PLAN amendment before implementation (the G-002 lesson, restated).
- **Pitfall ledger (C3-19)**: P-1, P-2, P-13 close in P2; P-4, P-5, P-6, P-7, P-9, P-10, P-11, P-14 + P-8 (auth half) close in P1; P-8 (full) + P-12 close in P3; P-3 closes across P2 (backend half) and P4 (hosting half). All fourteen verified closed by v0.3.4.
- **Engine contracts unchanged throughout**: the only engine-file touches in the milestone are additive — `RunState.owner_id` stamping (P1) and the `_verify_phase` artifact writes (P3, D-113); no methodology, state-machine, or protocol rewrites anywhere.
Milestone merge to main, v0.2.5 publication, and any deploy wait on the owner approval gate (D-018/G-009). The pipeline does not proceed autonomously past this commit.
+8 -19
View File
@@ -6,10 +6,9 @@ ResearchX (working name; rename deferred — naming collision audit on file, see
## Core Value
A deep-research studio where you bring your own model, storage, and memory; choose a research methodology before a single query fires; and every claim in the final report is anchored to a visited, verified, quotable source. Simple enough to run in three clicks, transparent enough to audit line by line.
## Objective (three-milestone plan, owner-directed; v0.3 scope directed 2026-09-29)
1. **Milestone v0.1 — Prototype** (COMPLETE, approved 2026-09-27, v0.1.8): UI/UX with mock data — production Next.js frontend behind a typed `ApiClient`/`MockApiClient` boundary.
2. **Milestone v0.2 — MVP** (COMPLETE, approved 2026-09-27, v0.2.5): FastAPI backend, async engine, adapter layer, trust loop, OpenAPI client swap. Live mode = core research loop only (G-002/G-009 ratified).
3. **Milestone v0.3 — Hosted** (owner-directed 2026-09-29): the hosted multi-user product — auth + sessions (user/pass, admin-managed accounts, OAuth deferred), per-user settings incl. model API keys (encrypted at rest; resolution chain user key → server key → keyless localhost), all prototype surfaces live-bound and user-scoped (library, settings persistence, suggestions, refine, graph/inbox, profile), backend hosting for the LXC deployment (SSE-safe), and real inference as the product default (mock demoted to dev/test fixture). The remaining 6 methodologies stay v0.4+ scope.
## Objective (two-milestone plan, owner-directed 2026-09-27)
1. **Milestone v0.1 — Prototype**: UI/UX only, mock data, the production Next.js frontend with all data behind a typed `ApiClient` interface backed by `MockApiClient`. Surfaces: Ask, Choose (all 8 methodology cards), Plan & Run, Reading Room v0, v2 surface mockups (project graph, task inbox). **Ships only after owner approval — no MVP work begins until approved.**
2. **Milestone v0.2 — MVP**: gated on prototype approval. FastAPI backend, custom async engine, adapter layer, trust loop, and integration (MockApiClient swapped for the OpenAPI-generated client).
## Requirements (validated)
1. Five pluggable adapter slots: model (OpenAI-compatible, planner/writer split), storage (local FS), search (Tavily default, DuckDuckGo keyless fallback), memory (off — stubbed), secrets (env — OpenBao later)
@@ -29,18 +28,16 @@ A deep-research studio where you bring your own model, storage, and memory; choo
- Memory scopes (semantic caching), S3/git storage adapters, Exa/Perplexity/Google search, OpenBao secrets, Docker packaging
## Constraints
- ~~Prototype must be approved before MVP work begins~~ (v0.1 approved 2026-09-27; v0.2 approved 2026-09-27)
- Prototype must be approved before MVP work begins (owner directive)
- Versioning starts at v0.1; MAJOR remains 0 until an owner-approved v1 release; all pre-v1 ships are MINOR/PATCH only (owner directive)
- ~~Single-user, local-first tool — no auth/multi-tenancy in MVP~~ **SUPERSEDED v0.3 (owner-directed 2026-09-29)**: multi-user hosted product — user/pass auth, admin-managed accounts (first registered user is admin), OAuth deferred to v0.4+; runs/library/settings partitioned by user
- Per-user model keys resolve per run: user's own key → server env-file key → keyless localhost fallback (owner directive)
- Not a general-purpose chatbot; not a model host; not a closed content index (v1)
- Paywalled academic access out of scope v1 — open APIs (ArXiv/PubMed) only
- Single-user, local-first tool — no auth/multi-tenancy in MVP
- Ollama/vLLM supported via OpenAI-compatible endpoints only (no vendor-specific adapters)
- Secrets never in app config or git; model capability honesty (warn when selected model is weak at structured outputs)
- No deploys without owner directive (D-019 carries); milestone merge to main halts at the owner approval gate (D-018 carries)
## Out of Scope
- Paywalled academic sources; model hosting; open self-serve registration / OAuth (admin-managed accounts only, OAuth v0.4+); native Ollama API adapter; domain registration/rename decision
- Paywalled academic sources; model hosting; multi-tenant/auth; native Ollama API adapter; domain registration/rename decision
## Context
Vision finalized and locked 2026-09-26: v1 methodology composition LOCKED (8 core), product named ResearchX, OpenBao retained as architectural secrets component, criticism matrix and evidence base [E1]–[E15] on file. Owner revisions 2026-09-27: (1) prototype-first — UI/UX prototype with mock data must be approved before MVP work begins; (2) version starts at v0.1, MAJOR remains 0 until MVP released; (3) the prototype is its own milestone. Milestone plan: v0.1 prototype (this milestone) → v0.2 MVP (gated).
@@ -59,16 +56,8 @@ Vision finalized and locked 2026-09-26: v1 methodology composition LOCKED (8 cor
## Key Decisions (v0.2 clarify — D-007..D-020, 2026-09-27)
- Backend layout: `backend/` repo-root package, src-layout `researchx` (config, methodologies, adapters, engine, evidence, trust, api per ARCHITECTURE §Components); uv + uv-managed Python 3.12 (D-007/D-008)
- Ports/binding: backend 127.0.0.1:8000, frontend :3000 via NEXT_PUBLIC_API_BASE_URL; CORS localhost:3000 (D-009); run IDs = 8-char slugs; storage = <dir>/runs/<id>/ (D-010 — **superseded by D-106 for v0.3+: runs/<user_id>/<run_id>/**)
- Ports/binding: backend 127.0.0.1:8000, frontend :3000 via NEXT_PUBLIC_API_BASE_URL; CORS localhost:3000 (D-009); run IDs = 8-char slugs; storage = <dir>/runs/<id>/ (D-010)
- Adapters: Ollama base URLs + key via env-file secrets scopes (OLLAMA_BASE_URL, OLLAMA_API_KEY); DDG keyless fallback = html.duckduckgo.com/html conservative scrape (D-011/D-012)
- Integration: openapi-typescript + openapi-fetch → frontend/src/lib/api/generated, bound behind ApiClient; RESEARCHX_API_MODE=mock|live, mock default (D-013/D-020)
- Transport/trust: sse-starlette SSE + pydantic typed event bus (D-014); difflib quote anchoring ≥ 0.85 (D-015); HTTP liveness live/stale/dead (D-016); JSON durable state, atomic rewrite per transition (D-017)
- Run termination: pipeline ships P1..P4 + P5 final patch, then HALTS at the owner approval gate before milestone → main (merge_to_main hook, v0.1 precedent); no deploys without owner directive (D-018/D-019)
## Key Decisions (v0.3 clarify — D-101..D-115, 2026-09-29)
- Sessions: server-side store (in-process dict + durable sessions.json, D-017 idiom), httpOnly SameSite=Lax cookies, opaque tokens, 14-day sliding expiry, immediate revocation (D-101); argon2id via argon2-cffi, verified via asyncio.to_thread (D-102)
- Auth posture: 401 on all /api/* except login/register (register open only while user store empty — first user is admin, then admin-managed; D-103/D-104); CSRF = SameSite=Lax + custom header on non-GET (EventSource GET exempt); /healthz outside /api for proxy checks
- Identity/partitioning: user_id = 8-char slug, email login (D-105); storage runs/<user_id>/<run_id>/ (D-106 supersedes D-010); data root = RESEARCHX_DATA_DIR (backend/var/researchx dev, /var/lib/researchx container; D-107)
- Key store: AES-256-GCM with RESEARCHX_KEK from the env-file 'keys' scope, user_id as AAD; disabled honestly when no KEK (D-108); scope = model key only (D-109); model adapter constructed per run (fixes shared-adapter fallback bug; D-110)
- Settings: GET /api/config stays read-only adapter status (user-contextualized resolved tier); new GET/PUT /api/config/settings for per-user settings; storage path display-only hosted (D-111)
- Mock fate: code default stays mock (zero-key dev/CI); hosted image built with NEXT_PUBLIC_RESEARCHX_API_MODE=live; no live delegation to mock after P3 (D-112)
- REQ-82 is backend work: engine persists report.json + evidence.json at DONE; endpoints enriched (D-113); open questions = plan sub-questions answered-iff DONE (D-114); read-models read disk artifacts, not RunRegistry; in-flight runs marked cancelled on restart; rehydration out of v0.3 scope (D-115)
- Run termination: pipeline ships P1..P4 + P5 final patch, then HALTS at the owner approval gate before milestone → main (merge_to_main hook, v0.1 precedent); no deploys without owner directive (D-018/D-019)
+1 -32
View File
@@ -80,33 +80,6 @@
- **D-013/D-020**: openapi-typescript + openapi-fetch generated client bound behind ApiClient; RESEARCHX_API_MODE=mock|live, mock default until P3
- **D-014..D-017**: sse-starlette SSE + pydantic event bus; difflib quote anchoring ≥ 0.85; HTTP liveness live/stale/dead; JSON durable state atomically rewritten per transition
## Hosted (Milestone v0.3 — owner-directed 2026-09-29)
### Auth & Sessions
- **REQ-69**: User/pass auth: user model with argon2id password hashing (argon2-cffi, verified off the event loop), register/login/logout/me routes, httpOnly cookie sessions (server-side session store: in-process dict + durable JSON under the data root, immediate revocation on logout/deactivation), CSRF-safe same-origin posture (SameSite=Lax cookie + custom-header check on mutating requests); OAuth explicitly deferred to v0.4+
- **REQ-70**: Admin-managed accounts: registration is open only while the user store is empty; the first registered user becomes admin; subsequently accounts are created by an admin only (admin-set initial password, no email infrastructure); admin can list/create/deactivate accounts (deactivation kills sessions; data retained)
- **REQ-71**: User scoping: every run, stored report, and setting is partitioned by owner — the engine, storage paths, and library read-model are user-scoped; storage layout becomes runs/<user_id>/<run_id>/ (supersedes D-010); owner_id is recorded in run state; no migration (pre-hosted artifacts disposable); no user can read or act on another user's data through the API
### Per-User Settings & Model Keys
- **REQ-72**: Per-user settings persistence (PUT/GET /api/config/settings): methodology defaults, resolved dials, appearance slot, masked key status; storage directory is server-managed per-user — display-only in hosted mode (the free-path picker applies to the local single-user profile only)
- **REQ-73**: Per-user model API keys: masked entry in Settings (never returned in full — masked status only), encrypted at rest (per-user key store; KEK = RESEARCHX_KEK from the env-file secrets 'keys' scope; cipher AES-256-GCM with user_id as AAD; without a configured KEK the key store is disabled with honest status — chain falls to server/keyless); scope = model key (OLLAMA_API_KEY) only in v0.3
- **REQ-74**: Per-run model credential resolution chain: user's own key → server env-file key → keyless localhost:11434 fallback (existing fallback semantics preserved); the model adapter is constructed per run (fixes the shared-adapter fallback bug); the capability probe runs against the resolved tier; config status surfaces which tier resolved (user|server|keyless) for the current user
- **REQ-75**: Real inference as the product default: the code default of RESEARCHX_API_MODE stays mock (zero-key dev/CI per REQ-31); the hosted frontend is built with NEXT_PUBLIC_RESEARCHX_API_MODE=live (build-time inlining); after v0.3 P3 no live-mode surface delegates to the mock; mock demoted to a dev/test fixture
### Live Surfaces (G-002 boundary lifted, user-scoped)
- **REQ-76**: Library backend: GET /api/library — folder-structure listing over the user's runs (auto-categorization by topic + methodology, recency/title sorting); report cards resolve to real runs
- **REQ-77**: Cross-report connections + project graph: GET /api/library/connections and GET /api/graph — shared-source and topical edges computed from the user's ledgers and run metadata
- **REQ-78**: Task inbox: GET /api/tasks — open questions from the user's runs as checkable follow-ups; open questions = plan sub-questions of the user's runs, answered-iff the run reached DONE
- **REQ-79**: Suggestions: GET /api/suggestions — follow-up query suggestions derived from the user's unanswered questions (open inbox items joined with source runs); click-fills-the-box semantics preserved; empty state for users with no runs (no synthetic suggestions)
- **REQ-80**: Refine: POST /api/refine/questions + /api/refine/compose — planner-model-driven clarifying exchange composing a refined question + suggested context (scripted prototype dialogue replaced by real model calls)
- **REQ-81**: User profile: GET/PUT /api/profile — local profile (display name, email) surfaced in the avatar user menu; no account deletion in v0.3
- **REQ-82**: Live Reading Room fidelity: the live report renders structured sections with page-anchored citations (hover shows the stored quoted passage) and the ledger panel from the citation ledger — parity with the mock Reading Room v0; the engine persists report.json (structured sections + anchored citations with quotes) and evidence.json (passage texts) at DONE; report/ledger endpoints expose them; live exports use real entry data (no fabricated fields)
### Hosting
- **REQ-83**: Backend container: Dockerfile for the FastAPI backend (single uvicorn worker, non-root), two-service deployment (frontend + backend) via deploy.sh extension; RESEARCHX_DATA_DIR volume (default /var/lib/researchx); unauthenticated /healthz outside /api for proxy health checks
- **REQ-84**: SSE-safe reverse proxy: HAProxy path routing /api → backend with streaming timeouts that do not buffer or cut long-lived SSE connections; frontend served with NEXT_PUBLIC_API_BASE_URL pointing at the public origin
- **REQ-85**: Hosted live mode: the deployed instance runs real inference in live mode (per REQ-75), per-user keys honored; auth guard covers the backend surface (no unauthenticated API access through the public URL)
## V2 (Post-MVP backlog)
### Methodology Catalog Extension
@@ -139,7 +112,7 @@
|---|---|
| Paywalled academic access | Unsolved in v1 — routed around via open APIs (vision §8) |
| General-purpose chatbot / model host / closed content index | Explicit non-goals (vision §8) |
| Auth / open self-serve registration / OAuth | v0.3 ships user/pass auth with admin-managed accounts (owner-directed 2026-09-29); open registration and OAuth stay out (OAuth v0.4+) |
| Auth / multi-tenancy | Single-user local-first MVP |
| Native Ollama API adapter | Ollama supported via OpenAI-compatible endpoint only |
| Domain registration / product rename | Deferred by owner (vision §11) |
| v1.0.0 release | Owner-approved release only; not created autonomously by the pipeline |
@@ -161,8 +134,4 @@
| REQ-25, REQ-26, REQ-33 (API, client, tooling) | v0.2 | P3 | complete (v0.2.3) |
| REQ-17, REQ-32, REQ-34 (adversarial, E2E, round-trip) | v0.2 | P4 | complete (v0.2.4) |
| All REQ-01..34 (milestone gate) | v0.2 | P5 | complete (v0.2.5 — **owner approval recorded 2026-09-27**, G-009 ratified; milestone merged to main) |
| REQ-69..71 (auth, sessions, scoping) | v0.3 | P1 | pending |
| REQ-72..74 (settings, per-user keys, resolution chain) | v0.3 | P2 | pending |
| REQ-76..82 (live surfaces, user-scoped) | v0.3 | P3 | pending |
| REQ-75 (delegation-end half) + REQ-83..85 (hosting; live default hosted) | v0.3 | P3/P4 (REQ-75 split per G-103: delegation-end P3, hosted-default P4) | pending |
| REQ-35..51 | v2 backlog (post-MVP milestones) | — | pending |
-555
View File
@@ -1,555 +0,0 @@
# Research — v0.3 Hosted (Phase 0)
## Run: 2026-09-29 (stage: research)
Phase-0 research findings for milestone v0.3 (Hosted): auth + server-side
sessions, per-user encrypted model keys, user-scoped storage, live surfaces
backed by disk artifacts, and two-service hosting behind an SSE-safe proxy.
Each topic records concrete code/config shapes and pitfalls that PLAN
consumes. All library claims were verified against the pinned versions in
the repo (starlette 0.46.2, fastapi 0.115.14, sse-starlette, eventsource
polyfill 5.1.2).
## 1. FastAPI auth/session layer
**Recommendation:**
- New package `backend/src/researchx/auth/`: `users.py` (user model +
argon2id), `sessions.py` (server-side session store), `guard.py`
(session + CSRF dependency), `routes.py` (register/login/logout/me).
- **Route dependencies, not middleware, as the enforcement point.** Group
all protected routes under `APIRouter(dependencies=[Depends(require_user)])`
inside `create_app`. The dependency resolves the session cookie, loads
the user, runs the CSRF header check on non-GET, and returns the `User`:
```python
async def require_user(request: Request) -> User:
token = request.cookies.get(SESSION_COOKIE) # "researchx_session"
session = sessions.get(token) # None if absent/expired/revoked
if session is None:
raise HTTPException(status_code=401, detail="authentication required")
if request.method != "GET" and "x-researchx-request" not in request.headers:
raise HTTPException(status_code=403, detail="missing request header")
user = users.get(session.user_id)
if user is None or not user.active:
raise HTTPException(status_code=401, detail="account disabled")
sessions.touch(session) # sliding expiry
return user
```
Rationale vs global middleware: the dependency appears in OpenAPI (401
documented), is overridable per-test via `app.dependency_overrides`
(critical for the zero-key harness, topic 9), and avoids the pure-ASGI
middleware rewrite needed to keep SSE streaming safe. A public router
(`/api/auth/*`, `/healthz`) + protected router (everything else)
enforces "all /api/* except login/register" by construction.
- **Session store (D-101):** opaque token = `secrets.token_urlsafe(32)`;
in-process `dict[token, Session]` + durable `sessions.json` at the data
root, written with the atomic tmp+`os.replace` idiom (D-017).
`Session = {token_hash, user_id, created_at, last_seen, expires_at}`.
14-day sliding expiry via `touch()`; persist only when >60s since last
persist. Immediate revocation = delete token + persist. Logout deletes
the presented token; deactivation deletes all of the user's tokens.
Rotate the token at login (kills session fixation by construction).
- **Cookie:** `researchx_session=<token>`; `httponly=True, samesite="lax",
path="/"`, `secure=True` only in hosted/TLS deploys (config flag), no
`Domain` attribute (host-only).
- **argon2-cffi (D-102):** `PasswordHasher()` defaults are argon2id with
the RFC 9106 low-memory profile — keep the defaults. All calls off the
event loop via `asyncio.to_thread`; `VerifyMismatchError` → uniform 401
"invalid email or password"; `check_needs_rehash` → rehash on login.
- **CORS ordering (verified):** the LAST-added middleware is OUTERMOST
(starlette 0.46.2 `build_middleware_stack`). Add `CORSMiddleware` AFTER
any auth middleware so preflight OPTIONS is answered without a 401.
Cookie auth requires `allow_credentials=True` + explicit localhost
origins (browsers reject wildcard origins with credentials).
- **First-user-is-admin (D-104):** `POST /api/auth/register` checks
`users.empty()` then inserts `role="admin"`; no `await` between check
and insert except the argon2 hash — the single uvicorn worker makes
this race-free by construction. After the first user, register 403s;
`POST /api/admin/users` (admin-guarded) creates accounts.
- **User store:** `users.json` at the data root:
`{"users": [{"user_id", "email", "password_hash", "display_name",
"role": "admin"|"user", "active": true, "created_at"}]}`; `user_id` =
8-char slug from the run-ID alphabet + validation regex `^[a-z0-9]{8}$`
(D-105 — filesystem-safe partition key).
**Pitfalls:**
- Do NOT use `BaseHTTPMiddleware` for the guard — it wraps response bodies
and has a history of SSE/streaming buffering bugs; router dependencies
sidestep the class.
- SameSite site-computation is scheme+registrable domain, NOT port —
`localhost:3000 → localhost:8000` is same-site (Lax cookie flows with
`credentials:"include"`), but `localhost:3000 → 127.0.0.1:8000` is
CROSS-site (Lax cookie silently dropped). Dev docs must pin both sides
to `localhost` (`NEXT_PUBLIC_API_BASE_URL=http://localhost:8000`).
- argon2 verify at default params takes ~50–100ms CPU — `to_thread`
mandatory for hash and verify.
- `EnvFileSecretsProvider` loads at construction; user/session/key stores
are mutable and must reload-or-rewrite atomically instead.
## 2. CSRF posture (D-103)
**Recommendation:**
- Combined defense, no token round-trips: (1) `SameSite=Lax` session
cookie blocks cross-site cookie attachment on background POSTs; (2)
every non-GET `/api` request must carry `X-ResearchX-Request: 1`
(checked in `require_user`). A cross-site attacker can neither attach
the Lax cookie to a background POST nor set custom headers without
CORS preflight approval.
- Frontend: the live client sends the header on every mutating call —
a small `#req()` wrapper or openapi-fetch middleware in
`GeneratedApiClient`: `headers: { "X-ResearchX-Request": "1" }`.
- **EventSource is exempt by design:** GET-only, cannot set custom
headers — the guard skips GETs, so `/api/runs/{id}/events`
authenticates by cookie alone. The eventsource polyfill (5.1.2) used
in E2E supports `withCredentials` and a fetch override.
- Login/register POSTs also require the header (uniform rule).
- Dev split-origin: `fetch(..., {credentials: "include"})` everywhere +
CORS `allow_credentials=True` + same-hostname origins (§1 pitfall).
**Pitfalls:**
- If dev ever needs cross-SITE cookies (localhost vs 127.0.0.1), Lax
fails silently — the fix is same-hostname origins, not `SameSite=None`.
- `Secure` must NOT be set in plain-HTTP dev (browsers drop Secure
cookies on http); MUST be set on the TLS-fronted host.
## 3. Per-user encrypted key store (D-108, D-109)
**Recommendation:**
- New `backend/src/researchx/auth/key_store.py` using
`cryptography.hazmat.primitives.ciphers.aead.AESGCM`
(`AESGCM(key).encrypt(nonce, data, aad)` → ciphertext||16-byte tag;
`decrypt` raises `InvalidTag` on mismatch; nonce = 12 bytes; NEVER
reuse a nonce under the same KEK).
- **KEK (D-108):** `RESEARCHX_KEK` from the env-file **'keys' scope**
(`config.json secrets.scopes` gains `"keys": ["RESEARCHX_KEK"]`),
accepted as urlsafe-base64 or hex of exactly 32 bytes; decode errors
raise `AdapterError("RESEARCHX_KEK is not valid b64/hex")`; wrong
length raises (`must decode to 32 bytes (AES-256)`).
- **Envelope:** AAD = `user_id.encode()` (a copied file under another
user's directory fails `InvalidTag`); fresh `os.urandom(12)` nonce per
write; scope = `OLLAMA_API_KEY` only (D-109).
- **File format** `keys/<user_id>.json` under the data root, atomic
rewrite, `0600`:
```json
{
"v": 1,
"alg": "AES-256-GCM",
"updated_at": "2026-09-29T12:00:00Z",
"entries": {
"OLLAMA_API_KEY": {"nonce": "<b64url(12)>", "ct": "<b64url(ct+tag)>", "updated_at": "..."}
}
}
```
- **API never returns key material** (REQ-73): PUT accepts a raw key, GET
returns `{"configured": true, "masked": "…3f2a"}` (last 4 chars).
Crypto in `asyncio.to_thread` for discipline.
- Honest degradation: no KEK → PUT returns `{"key_store": "disabled"}`;
resolution chain proceeds server/keyless; corrupt entries (InvalidTag)
surface as error status, not silent tier drop.
**Pitfalls:**
- GCM nonce reuse under a fixed KEK is catastrophic — always `os.urandom`.
- KEK rotation is out of v0.3 scope; `v` + `entries` leave room later.
- A changed KEK requires process restart (EnvFileSecretsProvider loads
once) — document it.
## 4. Per-run model adapter factory (D-110)
**Recommendation:**
- Verified shared-adapter bug (`ollama.py:73-79`): a cloud failure
permanently flips `self._active_base` on the SHARED instance — every
subsequent run by every user silently uses localhost. Fix = per-run
adapter instances.
- New `backend/src/researchx/adapters/model_factory.py`:
`ModelAdapterFactory(secrets, key_store, defaults)` with
`async resolve(user, settings) -> (OllamaModelAdapter, tier)` resolving
the chain user → server → keyless (REQ-74) and constructing one
adapter per run.
- `OllamaModelAdapter` constructor gains optional `api_key: str | None`
taking precedence over `secrets.resolve(...)`; the secrets path stays
for back-compat. Fallback mutation stays — now per-instance.
- **Threading through:** `create_app` constructs the factory once;
`create_run` does `model, tier = await factory.resolve(user, settings)`
and passes `model` into `RunActor` — **engine contracts unchanged**;
only the instance origin changes. `RunRegistry` keys become
`(owner_id, run_id)`. `state.json` records `owner_id` and
`credential_tier` for auditability.
- **Probe-per-run:** the existing `supports_probe` + `probe_model_adapter`
+ `actor.attach_probe(...)` flow operates on the adapter passed to
`create_run` — per-tier automatically. `GET /api/config` resolves the
factory for the current user WITHOUT creating a run; cache in-process
keyed by `(user_id, tier, key_fingerprint)` with short TTL.
**Pitfalls:**
- `config_status` reads `runs._actors` (private) to steal the last run's
probe (`app.py:174-177`) — breaks with zero runs and leaks another
user's probe; replace with the factory-resolved probe.
- Don't cache adapters across runs — per-run lifetime keeps
`_last_fallback_reason` honest.
## 5. Storage partitioning (D-106, D-107) — exact touchpoints
**Recommendation:**
- **Owner-scoped adapter instances, not owner-in-signatures.**
`LocalStorageAdapter(base_dir, owner_id)` binds the owner at
construction; `run_dir(run_id) -> base_dir/"runs"/owner_id/run_id`.
The `StorageAdapter` protocol stays byte-for-byte unchanged; only
constructors gain `owner_id` (validated `^[a-z0-9]{8}$`, D-105).
- App wiring: routes construct the owner-scoped adapter from the auth
dependency per request. `RunState` gains `owner_id: str | None = None`
(serialization-compatible; no migration per D-106).
`make_run_config(...)` gains `owner_id` and stamps it on state.
- `RunRegistry` keys `_actors[(owner_id, run_id)]`; cross-owner access
is a 404.
- **D-107 data root:** new `researchx.config.paths.data_root()`:
`os.environ["RESEARCHX_DATA_DIR"]` else `backend/var/researchx` (dev);
container `/var/lib/researchx`. Replaces the cwd-dependent
`Path.cwd()/".researchx"` (`app.py:150`). Layout:
```
$RESEARCHX_DATA_DIR/
users.json sessions.json
runs/<user_id>/<run_id>/{state.json, trace.jsonl, report.md,
report.json, evidence.json, ledger.json, checklist.md}
keys/<user_id>.json
settings/<user_id>.json
```
- **Exact touchpoint list (verified in-tree):**
1. `adapters/local_storage.py` — `__init__(owner_id)`, `run_dir`, +
`list_run_ids() -> list[str]` (read-models scan).
2. `adapters/storage.py` — protocol unchanged; add `append_run_artifact`
(actor uses it, protocol lacks it).
3. `engine/run_state.py` — `owner_id` field.
4. `engine/actor.py` — owner-scoped storage; `make_run_config(owner_id)`;
artifacts at DONE (topic 6).
5. `api/app.py` — per-request storage, registry keys, config display =
data root, `create_app(..., auth_enabled: bool = True)`.
6. `api/dev_bridge.py` — owner + auth off.
7. `engine/demo.py` — storage constructor gains owner.
8. Tests constructing storage/actor: `test_local_storage.py`
(layout assertion + traversal), `test_engine.py`,
`test_probe_wiring.py`, `test_adversarial.py`, `test_resume.py`,
`test_integration_p1.py`, `test_p0_regressions.py`, `test_api.py` —
mechanical `owner_id="u1"`-style constructor args.
9. Frontend untouched by partitioning (ids stay run-scoped).
**Pitfalls:**
- `owner_id` must be REQUIRED in `LocalStorageAdapter` — an optional
default invites silent unpartitioned use.
- `read_run_artifact`'s error message embeds `runs/{run_id}/{name}` —
update to owner path; tests may pin it.
- Consolidate the duplicate run-id generators (`local_storage` vs
`run_config.new_run_id` — different alphabets) when touching both.
## 6. New persisted artifacts: report.json + evidence.json (D-113)
**Recommendation:**
- Persist both at DONE inside `_verify_phase` (actor.py:315-332), right
after `report.md`/`ledger.json` — deterministic, no model calls.
- **Marker normalization (frontend-compat crux):** the writer emits
`[p-N]` (trust loop regex `\[p-(\d+)\]`); the Reading Room parses
`[^pN]` with `[A-Za-z0-9]+` only (`report-body.ts:18` — `[^p-1]` does
NOT match). Normalize at persistence: in `report.json` section bodies
`[p-3]` → `[^p3]`, ids `p-3` → `p3`. `report.md` keeps the raw form.
- **report.json**: `{v, run_id, owner_id, title, question, methodology_slug,
generated_at, sections: [{heading, body}], citations: [{passage_id, url,
title, quote, health, authority_tier}]}`. Sections split from
report.md headings; title = first H1 else `"Deep Report — {question}"`.
- **evidence.json**: `{v, run_id, passages: [{passage_id, url, title,
quote, text}]}` — serialized from `EvidenceStore` (ids renumbered `pN`).
- **Endpoint enrichment:** `GET /api/runs/{id}/report` returns
`{run_id, markdown, report: {...}, evidence: {...}}` (markdown retained,
back-compat); ledger unchanged. Live `exportReport` switches to real
fields — kills the fabricated `claims_supported: 1` and empty quotes
(`live-client.ts:329-384`).
- Live-client `getReport` maps report.json → frontend `Report`
(citations → `Citation` with real quotes; tier map official→1,
reference→2, else 3).
**Pitfalls:**
- Do NOT change `report.md`'s marker syntax — `_claims_per_passage`,
`extract_report_claims`, `_cited_urls` regex on `\[p-(\d+)\]`.
- Old DONE runs lack report.json — the endpoint must fall back to
markdown-only gracefully (pre-hosted artifacts disposable per D-106).
- `EvidenceStore` is per-actor and GC'd after the run — evidence.json
MUST be written inside `_verify_phase` (D-115 disk-reads later).
## 7. Live read-models on disk artifacts (D-114, D-115)
**Recommendation:**
- Read-models read disk, never RunRegistry (D-115): a small
`api/read_models.py` scans `runs/<owner_id>/*/` loading `state.json`
(+ report.json/ledger.json where needed).
- **Restart sweep:** on `create_app` startup, rewrite every non-terminal
`state.json` as CANCELLED (legal from every non-terminal state,
verified `states.py`); in-flight actors die with the process;
rehydration out of scope (D-115).
- **Registry pruning:** `_actors`/`_buses` are never pruned today (slow
multi-user leak); drop terminal entries once no live SSE subscriber
(refcount in the events route); events 404 after prune/restart is
documented, trace persists on disk.
- **Open-question semantics (D-114):** open questions = plan
sub-questions of the user's runs, answered-iff `state == DONE`; no
synthetic suggestions; empty states for new users.
- Route shapes (owner-scoped via the auth dependency):
- `GET /api/library?sort=recent|title` → folders (auto-categorized by
methodology + deterministic topic tokens from plan sub-questions) +
`ReportSummary[]`.
- `GET /api/library/connections` → shared-source edges from ledger URL
overlap (≥1 shared URL); related-topic edges on methodology +
category overlap.
- `GET /api/graph` → same edges in the frontend `ProjectGraph`
node/edge shape.
- `GET /api/tasks` → `InboxItem[] {id: "<run_id>#<idx>", question,
sourceRunId, status}` (done-iff-run-DONE else open).
- `GET /api/suggestions` → open items joined with run titles:
`SuggestedQuery[] {id, question, sourceRunId, sourceReportTitle}`.
- `POST /api/refine/questions {draft}` → planner call (2-3 clarifying
questions); `POST /api/refine/compose {draft, answers}` → planner
composes refined question + context. Non-GET ⇒ CSRF header applies.
- `GET/PUT /api/profile` → display_name/email.
- `GET/PUT /api/config/settings` (D-111): per-user settings
(`settings/<user_id>.json`): model names, methodology defaults,
masked key status, storage path display-only.
- `GET /api/config` stays read-only, user-contextualized:
`resolved_tier: user|server|keyless`, `key_store: enabled|disabled`,
per-user cost counters (owner-keyed), probe (§4).
- Directory scans skip dotfiles/tmp leftovers defensively.
- Refine endpoints spend planner tokens — same credential chain; a
keyless-only user with no local Ollama gets an honest 503, not a hang.
## 8. Hosting: backend Dockerfile + two-service deploy (REQ-83)
**Recommendation:**
- `backend/Dockerfile` (uv idiom, non-root uid 10001, single worker):
`python:3.12-slim` + uv from ghcr image; `uv sync --frozen --no-dev`;
`useradd --system --uid 10001 researchx`; `mkdir -p /var/lib/researchx`
+ chown; `USER researchx`;
`CMD ["uvicorn", "researchx.api.main:app", "--host", "0.0.0.0",
"--port", "8000", "--workers", "1", "--proxy-headers"]`.
`--workers 1` is a hard requirement (in-memory registry + sessions).
- **New production entrypoint `researchx.api.main`:** today `app.py`'s
module-level `app = create_app()` serves **FakeModelAdapter +
cwd-based storage** — hosting that would silently ship fakes.
`main.py` wires `create_app(data_root=RESEARCHX_DATA_DIR, factory=
<live factory from env-file secrets>, auth_enabled=True)`. `app.py`
stays the test factory.
- Secrets at runtime only: `RESEARCHX_KEK`, `OLLAMA_API_KEY`,
`OLLAMA_BASE_URL`, `TAVILY_API_KEY` via `--env-file` — never baked
into the image, never in git.
- **Plain docker-run pairs, not compose** — matches the existing
deploy.sh idiom and avoids a compose dependency on the LXC host:
`researchx-net` bridge network; `researchx-api` backend (env-file,
data volume, `-p 127.0.0.1:8000:8000` if HAProxy is host-side);
`researchx` frontend (`-p 3000:3000`, live build args). Keep
`researchx-api-prev` + `researchx-prev` for rollback.
- **deploy.sh extension:** rsync now includes `backend/`; build both
images on the host; health check BOTH `/healthz` (backend) and `/ask`
(frontend); update the HAProxy backend section idempotently.
- Data volume: pre-create + `chown 10001` host-side (belt-and-braces).
- New pyproject deps: `argon2-cffi>=23.1,<26`, `cryptography>=43`; no
`python-multipart` (JSON login bodies only).
**Pitfalls:**
- The volume must be writable by uid 10001 — a root-owned bind mount
makes every artifact write EACCES.
- uvicorn graceful shutdown default lets in-flight runs exercise
`_persist_cancelled` on redeploy; `docker rm -f` SIGKILLs — the
startup sweep (§7) is the real safety net.
## 9. HAProxy SSE configuration (REQ-84)
**Recommendation:**
- The host ALREADY runs HAProxy 2.8 with TLS for research.coreci.dev —
this is a front-config extension, not a new component:
```
frontend https-in
acl is_api path_beg /api
acl is_health path /healthz
use_backend backend_api if is_api or is_health
default_backend frontend_next
backend backend_api
option httpchk GET /healthz
default-server inter 5s fall 3 rise 2
server api1 127.0.0.1:8000 check
timeout server 3600s
timeout tunnel 3600s
backend frontend_next
timeout server 300s
server next1 127.0.0.1:3000 check
```
- HAProxy 2.x does not buffer response bodies by default in HTTP mode —
SSE streams natively. The three buffering hazards: NO
`option http-buffer-response`, NO compression on the API backend
(gzip buffers text/event-stream), keep default http-keep-alive.
- Verified: `sse-starlette` sends `Cache-Control: no-store` +
`X-Accel-Buffering: no` and emits `: ping` every **15s**
(`DEFAULT_PING_INTERVAL = 15`) — defeats idle timeouts.
- `timeout client` in global/defaults must exceed the 15s ping interval.
- Health check on `/healthz` (unauthenticated, outside /api per D-103):
`GET /healthz → 200 {"status":"ok"}`.
**Pitfalls:**
- Enabling global compression "to save bandwidth" is the classic way SSE
silently degrades — scope it off for the API backend.
- Don't disable the sse-starlette ping.
## 10. Next.js build-time env + EventSource credentials (D-112)
**Recommendation:**
- `NEXT_PUBLIC_*` vars are inlined at BUILD time — two build profiles:
- **dev/test (default):** no env; mode resolves mock; zero keys
(REQ-31 preserved, D-112).
- **hosted:** `pnpm build` with
`NEXT_PUBLIC_RESEARCHX_API_MODE=live NEXT_PUBLIC_API_BASE_URL=""` —
empty base = same-origin through HAProxy's `/api` path route.
`baseUrl()` treats `""` as same-origin (openapi-fetch accepts a
relative base; EventSource accepts relative URLs). Pin dev live-mode
docs to `http://localhost:8000` (§1 hostname rule).
- `deploy.sh` frontend build gains the build args; the Dockerfile build
stage passes them (`ARG`/`ENV` before `pnpm build`).
- EventSource: `new EventSource(url, { withCredentials: true })` — the
polyfill supports the same init dict. Same-origin hosted mode doesn't
strictly need it; set it anyway.
- **Delegation ends (D-112):** `GeneratedApiClient extends MockApiClient`
today; in P3 every method is overridden against real endpoints and the
inheritance is dropped (a standalone class implementing `ApiClient`);
mock remains standalone. OpenAPI snapshot regen (G-005): the new
auth/settings/library/refine/admin routes change `openapi.json` —
commit the snapshot, regen the generated schema, add the ApiClient
interface methods (`login/logout/getSession`, admin hooks) as PLANNED
extensions (the v0.2 G-002 lesson: document every addition up front).
**Pitfalls:**
- Forgetting the build args yields a hosted instance silently running
mock mode — add a deploy.sh smoke assertion: hosted `/api/config`
returns 401 + a live-mode marker.
- A non-empty `NEXT_PUBLIC_API_BASE_URL` in the hosted build breaks
same-origin cookies — assert `""` at build.
## 11. Dev/E2E harness under auth
**Recommendation:**
- `create_app(..., auth_enabled: bool = True)`. When False, the guard
resolves a synthetic dev user (`user_id="dev00000"`, role="admin") —
every route behaves exactly as authenticated, owner-scoped, zero keys.
- Existing suites keep running unchanged: backend API tests
(`test_api.py`, `test_adversarial.py`) and the real-stack E2E
(`dev_bridge.py`) construct with `auth_enabled=False`. Engine/trust
suites only need the `owner_id` constructor args (topic 5).
- **New auth suite (`test_auth.py`) with `auth_enabled=True`:**
first-user-admin (+403 after), wrong password 401, logout revocation,
401 on protected routes, CSRF header rejection, cross-user 404,
admin create/deactivate (kills sessions), expiry, key-store
round-trip + AAD tamper rejection + masked-only responses.
**Pitfalls:**
- Don't gate `auth_enabled` on an env var readable in production images
(a baked `RESEARCHX_AUTH=off` violates REQ-85); parameter-only, and
`main.py` never passes it.
- The E2E custom fetchImpl forwards headers verbatim — once the live
client adds the CSRF header, E2E keeps working via GETs + dev-disabled
guard; note for the plan, don't build now.
## 12. Pitfall scan — current code that breaks under multi-user
1. **`config_status` reads `runs._actors`** (`app.py:174-181`) —
multi-user probe leak + zero-run breakage. → factory probe (§4). [P1]
2. **Shared `model` instance across runs** (`app.py:152`) — the verified
`_active_base` fallback mutation (`ollama.py:73-79`) flips every user
to localhost after one cloud 401. → per-run factory (§4). [P1]
3. **`app = create_app()` module global** (`app.py:359`) — serves fakes
+ cwd storage in production. → `researchx.api.main` (§8). [P2/P4]
4. **CORS lacks `allow_credentials`** (`app.py:143-148`) — cookie auth
cannot work cross-origin in dev. → §1. [P1]
5. **`dev_bridge.py` no auth** — correct for E2E; must stay explicitly
`auth_enabled=False`, never the production entry. [verify P1]
6. **`RunRegistry` unbounded growth** — prune terminal runs. → §7. [P1]
7. **`RunState` no `owner_id`** — cross-user authorization impossible
today. → topic 5. [P1]
8. **Read endpoints require the registry** (`runs.has(run_id)` guards)
— every read dies on restart; a hosted redeploy 404s the whole
library. → disk read-models (D-115). [P1/P3]
9. **`demo.py`** constructs storage directly — mechanical owner fix. [P1]
10. **`test_local_storage.test_run_dir_layout`** asserts unpartitioned
path — changes with D-106 (intentional). [P1]
11. **Dead code** duplicate `return EventSourceResponse` (app.py:331) —
remove while touching the file. [P1]
12. **`exportReport` fabricates data** (`live-client.ts:350-389`) —
fixed by report.json/evidence.json. [P3]
13. **Global `cost_counters`** (`app.py:110`) — per-user (owner-keyed). [P1/P2]
14. **Duplicate run-id generators** (different alphabets) — consolidate. [P1]
## Research Conclusions
- **C3-01** Auth enforcement = router-level `Depends(require_user)` in
`create_app` (public router `/api/auth/*` + `/healthz`; protected the
rest), NOT middleware: OpenAPI-visible, dependency_overrides testable,
no BaseHTTPMiddleware/SSE interaction.
- **C3-02** Sessions: opaque `token_urlsafe(32)` cookies
(`researchx_session`, httpOnly, SameSite=Lax, Secure hosted-only),
server-side dict + atomic sessions.json, 14-day sliding expiry with
>60s persist throttle, token rotation at login, immediate revocation.
- **C3-03** argon2id via argon2-cffi `PasswordHasher()` defaults; hash,
verify, rehash-on-login all in `asyncio.to_thread`; uniform 401.
- **C3-04** CORS: `allow_credentials=True` with explicit localhost
origins, added so it wraps auth (last-added = outermost in starlette
0.46.2); dev origins must share the hostname `localhost` (port is not
part of same-site computation).
- **C3-05** CSRF = SameSite=Lax + `X-ResearchX-Request` header on all
non-GET /api (checked in the auth dependency); EventSource GET exempt;
fetch `credentials:"include"` throughout.
- **C3-06** Key store: AES-256-GCM via cryptography AESGCM, KEK =
32-byte RESEARCHX_KEK (b64/hex) from the env-file 'keys' scope, AAD =
user_id, `os.urandom(12)` nonce per write, `keys/<user_id>.json`
(v1/entries format, atomic, 0600); masked-only API; honest disabled
status; InvalidTag surfaced.
- **C3-07** Per-run model factory: `resolve(user) → (adapter, tier)`,
chain user→server→keyless; OllamaModelAdapter gains optional `api_key`
constructor param; engine contracts unchanged; probe-per-tier falls
out of the existing flow; GET /api/config = current user's tier +
fingerprint-cached probe.
- **C3-08** Partitioning = owner-scoped `LocalStorageAdapter(base_dir,
owner_id)` (protocol unchanged), `runs/<owner_id>/<run_id>/`, owner_id
in RunState, registry keyed (owner_id, run_id), per-request adapters;
~10 enumerated touchpoints incl. 8 test files + demo.py + dev_bridge.
- **C3-09** Data root = RESEARCHX_DATA_DIR (dev backend/var/researchx,
container /var/lib/researchx) via a new `paths.data_root()`; the
module-level create_app() cwd default + fake adapters are replaced by
a production `researchx.api.main` entry.
- **C3-10** report.json + evidence.json persisted at DONE in
_verify_phase; markers normalized `[p-N]`→`[^pN]`, ids `p-N`→`pN` at
the persistence boundary so the Reading Room parses with zero
frontend changes; report.md keeps `[p-N]` for the trust loop.
- **C3-11** Read-models read disk artifacts only; open questions = plan
sub-questions answered-iff DONE; startup sweep marks non-terminal
runs CANCELLED; rehydration out of scope.
- **C3-12** Registry pruning: drop terminal entries once no SSE
subscriber; events 404 after prune/restart documented; trace persists.
- **C3-13** Backend container: uv-based non-root Dockerfile (uid 10001,
chowned volume), `uvicorn --workers 1` mandatory, runtime-only secrets
via --env-file, plain docker-run pairs (no compose dependency),
researchx-api-prev rollback, health on both /healthz and /ask.
- **C3-14** HAProxy: path route /api + /healthz → backend (2.8 on the
LXC host, already TLS-fronting); timeout server/tunnel 3600s, NO
http-buffer-response, NO compression on the API backend; sse-starlette
15s pings + X-Accel-Buffering keep streams alive; httpchk GET /healthz.
- **C3-15** Two build profiles: code default mock; hosted image built
with NEXT_PUBLIC_RESEARCHX_API_MODE=live +
NEXT_PUBLIC_API_BASE_URL="" (same-origin); deploy smoke asserts live
mode; EventSource withCredentials; delegation to MockApiClient ends in
P3 (standalone GeneratedApiClient).
- **C3-16** Auth harness: create_app(auth_enabled=True) default with a
parameter-only dev/test override resolving a synthetic dev user;
existing zero-key suites run with it off; dedicated auth suite on.
- **C3-17** New pyproject deps: argon2-cffi>=23.1,<26, cryptography>=43;
no python-multipart; config.json secrets scopes gain
"keys": ["RESEARCHX_KEK"].
- **C3-18** ApiClient interface additions are PLANNED (G-002 lesson):
login/logout/getSession + admin hooks documented in PLAN before P1;
openapi.json snapshot regen is a P1 gate (G-005 discipline carries).
- **C3-19** Pitfalls P-1..P-14 (§12) each closed by the phase touching
its file: P-1..P-11, P-13, P-14 in P1; P-3 in P2/P4; P-12 in P3;
P-5 verified by the auth suite.
- **C3-20** All decisions D-101..D-115 confirmed workable against the
actual code; refinements folded into PLAN as constraints.
-46
View File
@@ -129,52 +129,6 @@ library/settings/suggestions/refine/graph stay mock-backed until
v0.3+. Milestone branch merged to main and deleted — tags
v0.2.0–v0.2.5 preserve the full history. Next: v0.3 planning.
## Milestone v0.3 — Hosted (feature) — owner-directed 2026-09-29
Scope (owner session 2026-09-29): one milestone combining the G-009-ratified live surfaces + hosting with three owner additions — auth/sessions (user/pass, admin-managed, OAuth deferred), per-user model API keys, and real inference as the product default. Tags run on the v0.3.x line; MAJOR stays 0; feature milestone — each phase ships a patch, the final phase's patch IS the milestone release.
### Phase 0 — Pre-Execution (v0.3)
- **Description**: SPECIFY (v0.3 requirement set REQ-69..85; single-user constraint superseded), CLARIFY (auth/session mechanics, key-encryption, partitioning, mock's fate, Reading-Room fidelity), RESEARCH (FastAPI session/auth patterns, encrypted key stores, SSE-proxying, storage read-model), PLAN (vertical slices), GRILL, MVP/UX check.
- **Requirements covered**: none (planning artifacts)
- **Success criteria**: .ciagent/ files refreshed for v0.3; PLAN.md carries User-Facing Surface + Happy Path + UX Acceptance Criteria; PERSONAS.md v0.3 roster active; grill verdict recorded with binding decisions
- **Depends on**: v0.2 approval (recorded)
- **Status**: complete (PR #16 squash-merged to milestone/v0.3-hosted 1adfa87, tag v0.3.0, release created; D-101..D-115 clarified; RESEARCH-v03 with C3-01..C3-20; PLAN P1..P4 vertical slices; grill Proceed 0.68 with G-101..G-110 bound, G-111 escalated; MVP/UX PASS)
### Phase 1 — Auth + Sessions + User Scoping
- **Description**: User model (argon2id), register/login/logout/me, httpOnly cookie sessions (server-side store), first-user-is-admin + admin-managed accounts (REQ-70), auth guard on every API endpoint, runs/settings partitioned by owner (engine + storage paths). Frontend: login screen, real profile in user menu, logout.
- **Requirements covered**: REQ-69, REQ-70, REQ-71
- **Success criteria**: unauthenticated API access returns 401; two users cannot see each other's runs; admin can create/deactivate accounts; existing engine tests green under an authenticated harness
- **Depends on**: 0
- **Status**: pending
### Phase 2 — Per-User Settings + Model Keys
- **Description**: Encrypted per-user key store (KEK from env-file secrets scope), per-user settings persistence (PUT/GET /api/config), per-run credential resolution chain (user key → server key → keyless localhost; REQ-74) with probe per resolved key, settings UI live-bound with masked keys.
- **Requirements covered**: REQ-72, REQ-73, REQ-74
- **Success criteria**: a keyed user gets fully real inference end-to-end; keys never appear in API responses (masked only), logs, or traces; resolution tier surfaced in config status; zero-key tests stay green via REQ-31 fakes
- **Depends on**: 1
- **Status**: pending
### Phase 3 — Live Surfaces (user-scoped)
- **Description**: Library read-model (GET /api/library), connections + graph (GET /api/library/connections, GET /api/graph), task inbox (GET /api/tasks), suggestions (GET /api/suggestions), refine (POST /api/refine/*, real planner calls), profile (GET/PUT /api/profile), live Reading Room fidelity (structured report, anchored citations, ledger panel). G-002 boundary lifted; frontend live swaps with zero component rewrites (REQ-54).
- **Requirements covered**: REQ-76, REQ-77, REQ-78, REQ-79, REQ-80, REQ-81, REQ-82
- **Success criteria**: every prototype surface works against real user data in live mode; suggestions/refine driven by real model calls; Reading Room parity with mock; ≤3-click path still green
- **Depends on**: 2
- **Status**: pending
### Phase 4 — Hosting + Real-Inference Default
- **Description**: Backend Dockerfile (single worker, non-root), two-service deployment via deploy.sh extension, SSE-safe HAProxy path routing (/api → backend, streaming timeouts), hosted instance runs live mode by default (mock demoted to dev/test fixture), auth guard verified through the public URL.
- **Requirements covered**: REQ-75, REQ-83, REQ-84, REQ-85
- **Success criteria**: one-command deploy brings up both services; SSE streams unbuffered through the proxy; unauthenticated curl to the public /api returns 401; live-mode default proven on the staged instance
- **Depends on**: 3
- **Status**: pending
### Phase 5 — Final Review + Ship (FINAL)
- **Description**: Multi-persona review of the full milestone diff, audit, tag v0.3.5 = milestone release; pipeline HALTS at the merge_to_main owner approval gate (D-018 carries); deployment to research.coreci.dev only on owner directive (D-019 carries).
- **Requirements covered**: milestone gate (REQ-69..85)
- **Success criteria**: review PASS (P0s fixed in-phase); audit PASS; v0.3.5 tagged + release; pipeline halted for the recorded owner decision
- **Depends on**: 4
- **Status**: pending
## Milestone Releases
- **v0.1.4 = extended-prototype candidate** (superseded by the P5/P6 extension cycle)
- **v0.1.5 = app shell + research library + settings + avatar** (P5 patch)
+2 -5
View File
@@ -3,13 +3,13 @@
"projects": [],
"active_project": "researchx",
"milestone": "v0.3",
"milestone_name": "hosted",
"milestone_name": "next-milestone-planning",
"milestone_type": "feature",
"versioning": {
"major_locked": true,
"major_lock_note": "MAJOR remains 0 until an owner-approved v1 release; all pre-v1 ships are MINOR/PATCH only",
"tag_line": "v0.3.x",
"next_milestone": "v0.4 candidates (owner-directed post-v0.3): remaining 6 methodologies (REQ-35..40), OAuth deferred from v0.3 auth, Exa/Perplexity search adapters, S3/git storage, memory scopes"
"next_milestone": "v0.3 scope definition pending owner direction (post-MVP backlog: live-bound library/settings/suggestions/refine, backend hosting, remaining 6 methodologies)"
},
"autonomy": {
"level": "full",
@@ -51,9 +51,6 @@
],
"search": [
"TAVILY_API_KEY"
],
"keys": [
"RESEARCHX_KEK"
]
}
}
+157
View File
@@ -0,0 +1,157 @@
# .gitea/workflows/ci.yml - ResearchX phase-4 CI gate (researchx#24).
#
# This file IS the gate. The pre-code clause (coreci/company#105, c14057):
# a code phase opens only when the product repo's gating workflow file has
# dispatched at least one run on `main` and every job in that file is green
# in that run. Until this file exists, coreci/researchx has no workflow and
# no run at all - that is the case the clause was amended for (researchx#24,
# c14093). A half-green pipeline does not open the phase either.
#
# Two jobs:
# backend - uv + Python 3.12: ruff, mypy, pytest
# frontend - Node 22 + pnpm 12.6.0: typecheck, lint, test, build
#
# Runner label: `ubuntu-latest` - the org CI fleet's published label whose
# image is docker.gitea.com/runner-images:ubuntu-latest (label map read at
# source: coreci/nextcraft deploy/ci/docker-compose.yaml + deploy/ci/README.md,
# R-3). The previous string `employee-runner` was a fleet NAME, not a label:
# the employee fleet's labels are `employee-py` and `employee-coder` (read
# from a live job log of this seat's own home), and the CI fleet's are node /
# ubuntu-latest / python / release - no runner carried the old label, so a
# merged file asking for it could still buy no run.
#
# Toolchain: measured, not assumed. From the runner's own logs (job 56393,
# coreci/nova-platform, 2026-10-26T23:17Z, and job 57005, coreci/inboxos-v1,
# 2026-10-27T03:01Z): the image carries python 3.12.3, pip, git, node (the
# actions run in it), pypi REACHABLE and github REACHABLE; actions/checkout@v4,
# actions/setup-python@v5 and actions/setup-go@v5 all run inside it. It does
# NOT carry uv or pnpm, so this file installs them: uv from pypi (pip), pnpm
# 12.6.0 through corepack - the repo's own documented way (README command map).
# The gates are the repo's documented commands: backend `uv run ruff check src
# tests`, `uv run mypy src`, `uv run pytest`; frontend `pnpm typecheck`,
# `pnpm lint`, `pnpm test`, `pnpm build`. The frontend lane mirrors the
# release image's own recipe (frontend/Dockerfile: node:22-alpine, corepack,
# pnpm install --frozen-lockfile, pnpm build), so the gate tests what the
# product ships.
#
# Owner of a red leg (the clause asks for the owner of the fix next to the
# owner of the product line): toolchain / image / dispatch problems ->
# coreci-sre (employee-runner-fleet + gitea-instance); product-line problems
# -> coreci-researchx-po. Approval for this file rides the merge request
# (merge = approval, constitution v1.3).
name: ci
on:
push:
pull_request:
branches: ["main"]
workflow_dispatch:
jobs:
backend:
name: "backend (uv, Python 3.12: ruff/mypy/pytest)"
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python 3.12
uses: actions/setup-python@v5
with:
python-version: "3.12"
# uv is not in the CI image; pypi is reachable from it (measured). The
# install is a toolchain leg, not a product gate: the gates still run
# from the repo's own committed lockfile (backend/uv.lock).
- name: Install uv (from pypi)
run: |
set -eu
python -m pip install --upgrade uv
uv --version
- name: Install dependencies (uv, from the committed lockfile)
working-directory: backend
run: uv sync --frozen
- name: Lint (ruff)
working-directory: backend
run: uv run ruff check src tests
- name: Typecheck (mypy)
working-directory: backend
run: uv run mypy src
- name: Test (pytest)
working-directory: backend
run: uv run pytest
frontend:
name: "frontend (Next.js on pnpm: typecheck/lint/test/build)"
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- name: Checkout
uses: actions/checkout@v4
# Node 22 to mirror the release image (frontend/Dockerfile: node:22).
- name: Setup Node 22
uses: actions/setup-node@v4
with:
node-version: "22"
- name: Setup Python 3.12 (for the real-stack e2e bridge)
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install uv (from pypi)
run: |
set -eu
python -m pip install --upgrade uv
# pnpm 12.6.0 is pinned by frontend/package.json `packageManager`;
# corepack reads it from there. If the image carries no corepack, npm
# installs the same pinned version.
- name: pnpm 12.6.0 (corepack, pinned by frontend/package.json)
working-directory: frontend
run: |
set -eu
node --version
corepack enable pnpm 2>/dev/null || true
if ! pnpm --version >/dev/null 2>&1; then
npm i -g pnpm@12.6.0
fi
test "$(pnpm --version)" = "12.6.0"
- name: Install dependencies (from the committed lockfile)
working-directory: frontend
run: pnpm install --frozen-lockfile
# frontend/src/lib/api/e2e-real-stack.test.ts drives the real FastAPI
# stack: it spawns `uv run python -m researchx.api.dev_bridge` from
# ../backend (loopback :8000, fakes only, zero API keys). Without a
# synced backend venv that test cannot come up, so the lane provisions
# it. Provisioning only - the backend gates stay in the backend job.
- name: Backend venv for the real-stack e2e bridge
working-directory: backend
run: uv sync --frozen
- name: Typecheck
working-directory: frontend
run: pnpm typecheck
- name: Lint
working-directory: frontend
run: pnpm lint
- name: Test
working-directory: frontend
run: pnpm test
- name: Build
working-directory: frontend
env:
NEXT_TELEMETRY_DISABLED: "1"
run: pnpm build
+7
View File
@@ -1,5 +1,12 @@
# ResearchX
**TL;DR** — ResearchX turns one question into a cited research report: pick a method, approve the plan, and the engine visits real web sources, with every claim anchored to a quoted passage from a page it actually read. The engine milestone (v0.2.5) is shipped and owner-approved; hosting goes live next (v0.3, in progress).
- **What** — FastAPI backend (planning, search, extraction, verification, export gates) plus a Next.js frontend; the full test suite runs with zero API keys.
- **Why** — Readers can check every citation back to its source passage; runs are resumable from durable state.
- **Action** — no action — FYI. Developers: see the Quickstart below.
- **Details** — `frontend/README.md` (web app); `.ciagent/` (pipeline state and roadmap).
A deep-research studio where you bring your own model, storage, and memory; choose a research methodology before a single query fires; and every claim in the final report is anchored to a visited, verified, quotable source.
**Status: MVP shipped (v0.2.5, owner-approved 2026-09-27)** — the full research engine (FastAPI + asyncio state machine + adapter layer + attack-proven trust loop) behind the v0.1 frontend. Live mode covers the core research loop (G-002/G-009): the engine runs locally per the quickstart; library/settings/suggestions/refine stay mock-backed until v0.3+.
+2 -3
View File
@@ -226,6 +226,7 @@ def create_app(
storage=storage,
registry=registry,
event_bus=bus,
on_complete=runs.note_completed,
)
probe_data: dict[str, Any] | None = None
if supports_probe(model):
@@ -240,9 +241,7 @@ def create_app(
runs.add(run_id, actor, task, bus)
async def _watch() -> None:
state = await task
if state.state == RunStateName.DONE:
runs.note_completed()
await task
asyncio.create_task(_watch())
return _summary(run_id)
+5
View File
@@ -1,4 +1,5 @@
import asyncio
from collections.abc import Callable
from datetime import UTC, datetime
from typing import Any
@@ -34,6 +35,7 @@ class RunActor:
storage: LocalStorageAdapter,
registry: MethodologyRegistry,
event_bus: EventBus,
on_complete: Callable[[], None] | None = None,
) -> None:
self._config = run_config
self._model = model
@@ -41,6 +43,7 @@ class RunActor:
self._storage = storage
self._registry = registry
self._bus = event_bus
self._on_complete = on_complete
self._store = EvidenceStore()
self.state = RunState(
run_id=run_config.run_id,
@@ -83,6 +86,8 @@ class RunActor:
async def _checkpoint(self, target: RunStateName) -> None:
if self.state.state != target:
self.state.transition(target)
if target is RunStateName.DONE and self._on_complete is not None:
self._on_complete()
await self._persist()
async def _wait_if_paused(self) -> None:
+7
View File
@@ -1,5 +1,12 @@
# ResearchX Frontend
**TL;DR** — This is the ResearchX web interface: ask a question, choose a research method, approve the plan, watch the live trace, read the cited report. The prototype runs entirely on scripted sample data — no API keys, no backend, no network — so the full product journey is tryable today.
- **What** — A Next.js app (ask, choose, run, report, library, settings) rendering mock data through one typed client interface.
- **Why** — The journey is checkable end-to-end before live inference exists.
- **Action** — no action — FYI. Developers: Quickstart below (Node 20+, pnpm).
- **Details** — Root `README.md`: backend engine, live mode, architecture.
The prototype frontend for ResearchX — methodology-driven deep research
with a trust loop. This milestone (v0.1) runs **entirely on mock data**:
no API keys, no env vars, no network calls beyond static assets.