Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 coreci
|
fd3d6c2e03 | ||
 coreci-bot
|
b485fc0817 | ||
|
coreci
|
ccb373dee1 | ||
|
coreci-bot
|
ee26ff1658 | ||
|
coreci
|
b8b0b68f1b | ||
|
coreci-bot
|
2a61433e6c | ||
|
coreci
|
60f858ab99 | ||
|
coreci-bot
|
1adfa878d7 | ||
|
coreci
|
8db45e478b | ||
|
coreci
|
702d3aae71 | ||
|
coreci-bot
|
b11b202271 |
+42
-31
@@ -1,7 +1,7 @@
|
||||
# Architecture
|
||||
|
||||
## Overview
|
||||
Two-milestone build (owner-directed 2026-09-27). Milestone v0.1 builds the UI/UX prototype as the production Next.js frontend with all data access behind a typed `ApiClient` interface backed by `MockApiClient` — no backend, zero throwaway work. Milestone v0.2 (gated on prototype approval) adds the FastAPI backend, the custom async research engine, and the adapter layer, then swaps `MockApiClient` for the OpenAPI-generated client. The engine is a hand-rolled asyncio state machine; every run persists its resolved 12-dial configuration and full event trace for reproducibility and auditability.
|
||||
Three-milestone build (owner-directed). Milestone v0.1 (COMPLETE) built the UI/UX prototype as the production Next.js frontend behind a typed `ApiClient`/`MockApiClient` boundary. Milestone v0.2 (COMPLETE) added the FastAPI backend, custom async research engine, adapter layer, and trust loop, then swapped MockApiClient for the OpenAPI-generated client. Milestone v0.3 — **Hosted** — converts the product to a hosted multi-user service: auth + server-side sessions, per-user encrypted model keys, user-scoped storage and surfaces, and a two-service deployment behind an SSE-safe reverse proxy. The engine keeps its hand-rolled asyncio state machine; every run persists resolved dials, full trace, and structured report/evidence artifacts for reproducibility and auditability.
|
||||
|
||||
## Components
|
||||
|
||||
@@ -13,61 +13,72 @@ Typed pydantic schema for the 12 dials (source universe, search routing, query s
|
||||
|
||||
### 2. Adapter Layer (`src/researchx/adapters/`)
|
||||
Five pluggable slots with versioned contracts:
|
||||
- **Model**: any OpenAI-compatible endpoint (covers Ollama cloud/local and vLLM); separate planner/writer roles; capability probe on connect (context window estimate, conservative request cap = min(model claim, 8192), embedding batch 8) surfaced as UI overrides; structured-output capability warning. Default: ollama.com/v1 cloud-first with localhost:11434/v1 fallback.
|
||||
- **Model**: any OpenAI-compatible endpoint (covers Ollama cloud/local and vLLM); separate planner/writer roles; capability probe on connect (context window estimate, conservative request cap = min(model claim, 8192), embedding batch 8) surfaced as UI overrides; structured-output capability warning. **v0.3:** adapters are constructed **per run** by the Model Adapter Factory (`model_factory.py`) — no shared instance; the per-instance fallback (cloud → localhost) preserves REQ-08 semantics without cross-run mutation; optional `api_key` constructor param carries the resolved per-user credential.
|
||||
- **Search**: Tavily default; DuckDuckGo keyless fallback; interface ready for Exa/Perplexity later.
|
||||
- **Storage**: local filesystem adapter; reports as plain Markdown/JSON under user-chosen directory; run state as JSON after each transition.
|
||||
- **Memory**: off in v1.0 — interface stubbed for semantic caching.
|
||||
- **Secrets**: env-file provider in v1.0 — interface stubbed for OpenBao.
|
||||
- **Storage**: local filesystem adapter; reports as plain Markdown/JSON under a user-chosen local directory; run state as JSON after each transition. **v0.3:** owner-scoped `LocalStorageAdapter(base_dir, owner_id)` instances — run layout `runs/<user_id>/<run_id>/` (supersedes D-010); data root via `RESEARCHX_DATA_DIR` (`researchx.config.paths.data_root()`); protocol gains `append_run_artifact`.
|
||||
- **Memory**: off — interface stubbed for semantic caching.
|
||||
- **Secrets**: env-file provider — interface stubbed for OpenBao. **v0.3:** the scopes map gains a `'keys'` scope (`RESEARCHX_KEK`) consumed by the per-user key store.
|
||||
|
||||
- **Boundaries:** all external I/O lives here; engine never imports vendor SDKs directly.
|
||||
- **DependsOn:** config (dial schemas only)
|
||||
|
||||
### 3. Research Engine (`src/researchx/engine/`)
|
||||
Custom asyncio state machine: PLANNING → AWAITING_APPROVAL → SEARCHING → EXTRACTING → REFLECTING → (depth loop) → SYNTHESIZING → VERIFYING → DONE. One RunActor per run; durable JSON state persisted to storage after each transition (pause/cancel/resume); typed event bus feeding SSE live trace; plan (sub-questions + source universe + outline) halts at AWAITING_APPROVAL until approved/edited.
|
||||
### 3. Identity & Sessions (`src/researchx/auth/`) — NEW v0.3
|
||||
User model (8-char slug `user_id`, email login, argon2id password hashing via argon2-cffi, all hashing off the event loop), server-side session store (opaque `token_urlsafe(32)` httpOnly SameSite=Lax cookies; in-process dict + durable `sessions.json` at the data root; 14-day sliding expiry; immediate revocation on logout/deactivation; token rotation at login), auth guard as a router-level dependency (`Depends(require_user)`) covering all `/api/*` except login/register, CSRF check (custom header on non-GET), first-user-is-admin + admin-managed accounts (`/api/admin/users`), user store `users.json` at the data root.
|
||||
|
||||
- **Boundaries:** owns user/session data + password hashing; no run knowledge.
|
||||
- **DependsOn:** adapters (secrets for the KEK scope), storage (data-root persistence)
|
||||
|
||||
### 4. Per-User Key Store (`src/researchx/auth/key_store.py`) — NEW v0.3
|
||||
AES-256-GCM envelope keyed by `RESEARCHX_KEK` (env-file 'keys' scope, 32 bytes b64/hex), `user_id` as AAD (ciphertext is bound to its owner — swapped files fail `InvalidTag`), `os.urandom(12)` nonce per write, `keys/<user_id>.json` (v1/entries format, atomic rewrite, 0600). API surface is masked-only (`configured` + last-4). Without a configured KEK the store is disabled with honest status; the resolution chain falls to server/keyless.
|
||||
|
||||
- **Boundaries:** owns key encryption/decryption; never returns key material.
|
||||
- **DependsOn:** Identity, adapters/secrets
|
||||
|
||||
### 5. Model Adapter Factory (`src/researchx/adapters/model_factory.py`) — NEW v0.3
|
||||
Per-run credential resolution (REQ-74): user's own key → server env-file key → keyless localhost:11434; constructs one model adapter per run and returns the resolved tier (`user|server|keyless`); the capability probe runs against the resolved tier; `GET /api/config` resolves the factory for the current user without creating a run (fingerprint-cached probe). Fixes the v0.2 shared-adapter fallback-mutation bug by construction.
|
||||
|
||||
- **Boundaries:** resolves credentials and constructs adapters; no engine knowledge.
|
||||
- **DependsOn:** Identity (user), Key Store, adapters (model, secrets)
|
||||
|
||||
### 6. Research Engine (`src/researchx/engine/`)
|
||||
Custom asyncio state machine: PLANNING → AWAITING_APPROVAL → SEARCHING → EXTRACTING → REFLECTING → (depth loop) → SYNTHESIZING → VERIFYING → DONE. One RunActor per run; durable JSON state persisted to storage after each transition (pause/cancel/resume); typed event bus feeding SSE live trace; plan (sub-questions + source universe + outline) halts at AWAITING_APPROVAL until approved/edited. **v0.3:** `RunState` carries `owner_id`; at DONE the engine persists two additional artifacts — `report.json` (structured sections + anchored citations with quotes) and `evidence.json` (passage texts) — written in the verify phase before the evidence store is discarded; citation markers are normalized `[p-N]` → `[^pN]` at the report.json boundary (report.md keeps the trust-loop form); startup sweep marks in-flight runs CANCELLED on process restart (rehydration out of v0.3 scope).
|
||||
|
||||
- **Boundaries:** orchestrates; contains no HTTP handling and no vendor I/O.
|
||||
- **DependsOn:** adapters, methodologies, trust, evidence
|
||||
|
||||
### 4. Evidence Store (`src/researchx/evidence/`)
|
||||
Visited-page store with stable passage IDs; passage extraction from fetched pages. The only source of writer context — the visited-only citation constraint is enforced by construction: the writer sees nothing else.
|
||||
### 7. Evidence Store (`src/researchx/evidence/`)
|
||||
Visited-page store with stable passage IDs; passage extraction from fetched pages. The only source of writer context — the visited-only citation constraint is enforced by construction: the writer sees nothing else. **v0.3:** serialized to `evidence.json` at DONE (passage texts survive the actor).
|
||||
|
||||
- **Boundaries:** owns evidence data model.
|
||||
- **DependsOn:** adapters (search/storage only)
|
||||
|
||||
### 5. Trust Loop (`src/researchx/trust/`)
|
||||
### 8. Trust Loop (`src/researchx/trust/`)
|
||||
Runs inside every methodology: generation-time visited-only constraint [E1]; post-run URL liveness pass with live/stale/dead classification [E1]; claim-to-passage quote anchoring (fuzzy match against stored passages) [E1b]; citation ledger JSON per run (URL, visited_at, passages, claims supported, health, authority tier); verification checklist export.
|
||||
|
||||
- **Boundaries:** read side of evidence store; verification reports feed the ledger.
|
||||
- **DependsOn:** evidence, adapters (model for verification passes)
|
||||
|
||||
### 6. API Layer (`src/researchx/api/`)
|
||||
FastAPI routes: POST /api/runs, GET /api/methodologies, GET /api/runs/{id}, approve/pause/cancel/resume, GET /api/runs/{id}/events (SSE), GET /api/config. OpenAPI schema generates the typed frontend client.
|
||||
### 9. API Layer (`src/researchx/api/`)
|
||||
FastAPI routes — **v0.3 inventory**: `/api/auth/{register,login,logout,me}` (public router; register open only while the user store is empty), `/api/admin/users*` (admin-guarded), `POST /api/runs`, `GET /api/methodologies`, `GET /api/runs/{id}`, approve/pause/cancel/resume, `GET /api/runs/{id}/events` (SSE), `GET /api/runs/{id}/report` (enriched: markdown + structured report + evidence), `GET /api/runs/{id}/ledger`, `GET /api/config` (read-only, user-contextualized: resolved tier, key-store status, per-user cost counters, probe), `GET/PUT /api/config/settings` (per-user settings), `GET /api/library`, `GET /api/library/connections`, `GET /api/graph`, `GET /api/tasks`, `GET /api/suggestions`, `POST /api/refine/{questions,compose}`, `GET/PUT /api/profile`, and `/healthz` outside `/api` (unauthenticated, for proxy health checks). Read-model routes (library/connections/graph/tasks/suggestions/profile/settings) read disk artifacts only — never the RunRegistry; the registry is owner-keyed `(owner_id, run_id)` and pruned of terminal runs. OpenAPI schema generates the typed frontend client.
|
||||
|
||||
- **Boundaries:** transport only — no business logic.
|
||||
- **DependsOn:** engine, methodologies, config
|
||||
- **DependsOn:** engine, methodologies, config, auth, key store, model factory
|
||||
|
||||
### 7. Frontend (Next.js App Router, TypeScript strict)
|
||||
App shell (root layout, P5): persistent sidebar (primary nav — New research / Library / Settings; adapter connection-status dots as a v2 preview) + top bar (⌘K navigation palette — screens + stored reports; avatar button → user menu with the local single-user profile). Screens: Ask (question + optional context; open-question suggestions below the form — follow-ups from unanswered questions of past runs, click-fills-the-box, REQ-67; opt-in Refine dialogue — inline drawer, scripted clarifying exchange, composed refined question + suggested context, REQ-68), Choose (methodology cards with ETA/cost/why-recommended; dials behind "advanced"), Plan & Run (approve/edit plan, live SSE trace, pause/cancel), Reading Room v0 (rendered report, hover-a-citation → exact quoted passage, ledger panel, Markdown/JSON export), Library (P5: folder-structure report storage — auto-categorized folders, recency/title sorting, report cards into the Reading Room; cross-report Connections panel — shared-sources ⇄ / related-topic → edges; project graph + task inbox as v2-preview panels), Settings (P5: masked API-key status, storage directory, default methodology + resolved 12-dial display, appearance preview, about/version).
|
||||
### 10. Frontend (Next.js App Router, TypeScript strict)
|
||||
App shell (persistent sidebar + top bar with ⌘K slot + avatar). **v0.3 additions:** login screen; session-aware shell (avatar menu → real profile, logout); live binding of ALL surfaces (the G-002 mock-delegation boundary is lifted — `GeneratedApiClient` becomes a standalone `ApiClient` implementation with no `MockApiClient` inheritance); auth + CSRF fetch wrapper (`credentials: "include"`, `X-ResearchX-Request` on mutations); EventSource `withCredentials`; two build profiles (mock dev default; hosted build inlines `NEXT_PUBLIC_RESEARCHX_API_MODE=live` + empty same-origin base URL). Screens: Ask (question + context; suggestions below the form; opt-in refine dialogue — now real planner calls), Choose (methodology cards + dials drawer), Plan & Run (approve/edit, live SSE trace, pause/cancel/resume), Reading Room (structured report with anchored hover citations from report.json/evidence.json, ledger panel, exports), Library (folder structure, report cards, connections), Settings (per-user: masked model key, methodology defaults, model names, display-only storage), profile.
|
||||
|
||||
- **Boundaries:** consumes the typed `ApiClient` interface only — components never import the mock client directly. In v0.1 the interface is bound to `MockApiClient` (fixture data + scripted event replay); in v0.2 phase 3 the binding swaps to the OpenAPI-generated client with no component changes.
|
||||
- **DependsOn:** ApiClient contract (v0.1: mock; v0.2: API layer)
|
||||
- **Boundaries:** consumes the typed `ApiClient` interface only — components never import the mock or generated client directly; the boundary selects the binding by build profile.
|
||||
- **DependsOn:** ApiClient contract
|
||||
|
||||
## Data Flow
|
||||
v0.1 (prototype): Ask (question + context) → Choose (methodology slug + dial overrides) → MockApiClient returns scripted plan → Plan & Run (scripted live trace replay, pause/cancel) → Reading Room (mock report with anchored citations, ledger, real file exports). All flows exercise the exact component paths the MVP will use.
|
||||
v0.3 (hosted): browser → HAProxy (TLS, `/api` + `/healthz` → backend :8000, SSE-safe: no response buffering, no API-backend compression, 3600s timeouts) → FastAPI (session guard resolves user → per-user settings + key store → model factory resolves credential tier user→server→keyless → owner-scoped engine run) → durable artifacts under `runs/<user_id>/<run_id>/` → read-models scan disk → Library/Graph/Tasks/Suggestions render → Reading Room renders report.json with anchored citations (hover quotes from evidence.json) → exports round-trip real data. Engine internals (state machine, evidence, trust loop) unchanged from v0.2.
|
||||
|
||||
v0.2 (MVP): Ask (question + context) → Choose (methodology slug + dial overrides) → POST /api/runs → engine PLANNING (planner model via model adapter, dial bundle resolved) → AWAITING_APPROVAL (plan persisted; SSE events; user approves/edits) → SEARCHING (search adapter, routing per dials) → EXTRACTING (fetch pages → evidence store, stable passage IDs) → REFLECTING (planner model; depth loop per dials) → SYNTHESIZING (writer model; context = evidence passages only, per-claim citation markers) → VERIFYING (liveness pass, quote anchoring, confidence labels) → DONE (report + ledger + checklist persisted to storage; Reading Room renders; export).
|
||||
## Deployment Topology (NEW v0.3)
|
||||
Two containers on the LXC host behind the existing HAProxy/TLS front (research.coreci.dev): `researchx` (Next.js frontend, :3000, live build args) and `researchx-api` (FastAPI backend, :8000 host-published, non-root uid 10001, `uvicorn --workers 1` — mandatory: in-memory registry + session store), `researchx-data` volume at `/var/lib/researchx` (RESEARCHX_DATA_DIR), runtime secrets via `--env-file` only. Rollback pairs `researchx-prev` / `researchx-api-prev`. deploy.sh builds both images at the tag, health-checks `/healthz` (backend) + `/ask` (frontend), and asserts live mode in the smoke test.
|
||||
|
||||
## Build Order
|
||||
Milestone v0.1 (prototype): 1. Frontend scaffold + ApiClient/MockApiClient boundary → 2. Ask + Choose → 3. Plan & Run + Reading Room v0 → 4. v2 mockups + polish + 3-click validation → 5. review + owner approval gate (owner returned gate feedback → extension cycle: 5. app shell + research library + settings + avatar → 6. final review + approval re-presentation (2nd owner gate feedback: 7. ask suggestions + refine dialogue → 8. final re-review + ship)).
|
||||
Milestone v0.1 (prototype — COMPLETE): frontend scaffold → Ask/Choose → Plan & Run/Reading Room → mockups/3-click → owner-gate extensions (app shell, library, settings, avatar; suggestions, refine) → approval.
|
||||
Milestone v0.2 (MVP — COMPLETE): methodology framework → adapters → evidence → trust → engine → API → client swap → adversarial + E2E → approval.
|
||||
Milestone v0.3 (hosted): 1. Auth + sessions + user scoping (storage partitioning, registry owner-keying, auth suite) → 2. Per-user settings + encrypted key store + model factory (credential chain, probe-per-tier) → 3. Live surfaces (read-models on disk artifacts, report.json/evidence.json + Reading Room fidelity, refine/suggestions real calls, frontend live swaps, mock-delegation ends) → 4. Hosting (backend Dockerfile + production entrypoint, two-service deploy.sh, SSE-safe HAProxy, live-default hosted build) → 5. Final review + owner gate.
|
||||
|
||||
Milestone v0.2 (MVP, gated on v0.1 approval):
|
||||
1. Methodology framework (dial schema + 2 seed methodologies + registry)
|
||||
2. Adapter layer (model/search/storage contracts + local implementations; fake adapters for tests)
|
||||
3. Evidence store (visited pages, passage extraction)
|
||||
4. Trust loop (liveness, anchoring, ledger)
|
||||
5. Engine (state machine, event bus, durable state, pause/resume)
|
||||
6. API layer (routes + SSE)
|
||||
7. Integration (swap MockApiClient → OpenAPI-generated client)
|
||||
8. Adversarial test suite (hallucinated-URL detection, quote-mismatch detection, pause/resume mid-depth-loop) + E2E happy path
|
||||
|
||||
Build order is dependency-driven: trust verification requires evidence; engine requires adapters + trust; API requires engine; integration requires API; frontend components carry over unchanged from v0.1.
|
||||
Build order is dependency-driven: auth precedes key store (identity); key store precedes the factory (credential chain); read-models precede hosting (disk-backed reads must exist before redeploys); hosting precedes the gate (the owner reviews the deployed thing).
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"phase": 3,
|
||||
"stage": "complete",
|
||||
"milestone": "v0.3",
|
||||
"phase_role": "execution",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-09-29T23:20:00Z"
|
||||
}
|
||||
@@ -0,0 +1,432 @@
|
||||
# CIAgent Grill Report
|
||||
|
||||
## Run: 2026-09-27 11:25 (mode: mechanical, focus: all)
|
||||
|
||||
### Verdict: Proceed (confidence: 0.72)
|
||||
|
||||
The plan is sound with fixable gaps. The evidence base is unusually strong for a project at P0: a shipped and owner-approved v0.1 (9 tags, decision commit 3fd0e10), a locked two-milestone structure, a typed frontend boundary already in the tree (client.ts), a research file with concrete API shapes, and honest `---ci---` reporting throughout. But the grill found **two internal contradictions and one measurability hole** that would each have surfaced at P3 as rework: (1) PLAN.md P1 claims REQ-07/REQ-12 delivery while ROADMAP/REQUIREMENTS/PLAN's own frontmatter map them to P2; (2) the approved prototype's ApiClient documents seven v0.2 endpoint mappings (`GET /api/graph`, `GET /api/tasks`, `GET /api/library`, `GET /api/library/connections`, `GET /api/suggestions`, `POST /api/refine/*`, `PUT /api/config`) that REQ-25 does not include — a hidden-scope grenade wired to the P3 swap; (3) REQ-23's ≥99% liveness bar cannot be proven in the zero-key test suite it is scheduled into. All three are absorbed below as binding decisions. The MVP does not overreach as specced — REQ-01..34 is a genuine core loop — but the *implicit* live-mode expectation (whole prototype UI working against the backend) does overreach, and is explicitly reduced to the core research loop (G-002, escalated for owner ratification at the P5 gate).
|
||||
|
||||
---
|
||||
|
||||
### Axis 1 — Business Case
|
||||
- **Q1**: What problem does this actually solve, and is that problem still the top priority?
|
||||
- Evidence: PROJECT.md Core Value + Key Decisions ("deep-research agents hallucinate URLs at the highest measured rates; verification cuts failures to <1% [E1]"); owner approved v0.1.8 and immediately unblocked v0.2 (commits 3fd0e10, 42435bc).
|
||||
- Answer: Real, named problem — citation-untrustworthy deep-research agents — converted into a differentiated product thesis (methodology-first + trust loop). The owner's same-day unblocking of v0.2 shows it remains the top priority. Not a project that outlived its justification: the value is entirely unrealized until the MVP ships, which argues for proceeding, not stopping.
|
||||
- Confidence: 0.80
|
||||
- **Q2**: Who is the named executive sponsor, and when did they last make a decision under pressure?
|
||||
- Evidence: Owner decisions under pressure recorded twice in v0.1: P4 gate returned *feedback instead of approval* → binding P5/P6 extension (ROADMAP P4 note, commit 7db53d62); P6 gate returned *2nd feedback* → P7/P8 extension (commit 57c5231, ROADMAP P6 note); final approval 3fd0e10. D-018/D-019 record owner directives for this run.
|
||||
- Answer: The sponsor is pressure-tested, not a rubber stamp — two rejections/extensions before approval. This is the strongest possible sponsor signal available in the record.
|
||||
- Confidence: 0.85
|
||||
- **Q3**: What happens to the business if the project is cancelled?
|
||||
- Evidence: v0.1 is mock-only by design (REQ-53: "no backend, no API keys"); zero research value has shipped; all value lands in v0.2 (REQ-01..34).
|
||||
- Answer: Cancelling now strands 100% of sunk effort — the prototype is a shell without the MVP. The project is not a zombie; it is the opposite: a project whose entire value is still ahead of it, which raises the stakes of P2/P3 execution quality, not the case for stopping.
|
||||
- Confidence: 0.75
|
||||
- **Q4**: Is the ROI calculated against a counterfactual?
|
||||
- Evidence: MISSING — no ROI/counterfactual calculation exists anywhere in .ciagent/.
|
||||
- Answer: No formal ROI. The implicit counterfactual is "ad-hoc LLM prompting with unverified citations," which is plausible but never quantified. For an owner-directed, single-user, local-first tool this is a tolerable omission — the sponsor is the beneficiary — but it means priority calls rest entirely on owner judgment, with no objective measure if enthusiasm fades.
|
||||
- Confidence: 0.55
|
||||
- Decision: G-007 — make the counterfactual *measurable in-product*: per-run request/token/cost counters in the run trace, surfaced via `GET /api/config` (advisory, no hard spend cap in MVP). This is the cheapest honest ROI instrumentation available.
|
||||
|
||||
### Axis 2 — Scope and Requirements
|
||||
- **Q1**: Is the scope expanding, contracting, or genuinely stable?
|
||||
- Evidence: REQ-01..34 locked with traceability (REQUIREMENTS.md table); D-007..D-020 clarify commit recorded (e79102b); v0.1 scope *grew twice* via owner gate feedback (P5/P7 extensions) — proof the gate can grow scope.
|
||||
- Answer: The formal scope is stable and frozen. But the grill found the *informal* scope is ambiguous: the prototype's approved UI implies backend endpoints that REQ-25 does not promise (see Q4). That ambiguity will resolve itself at P3 — either as scope growth or as a visible degraded mode. It must be resolved now, in writing.
|
||||
- Confidence: 0.60
|
||||
- **Q2**: Who owns the requirements, and have they been frozen?
|
||||
- Evidence: Owner-approved milestone structure (PROJECT.md Constraints); REQUIREMENTS.md traceability with per-phase status; clarify decisions D-007..D-020 at confidence 0.75–1.0.
|
||||
- Answer: Owner owns; the pipeline freezes and traces them. Requirements management is this project's strongest discipline.
|
||||
- Confidence: 0.85
|
||||
- **Q3**: What is explicitly out of scope?
|
||||
- Evidence: REQUIREMENTS.md "Out of Scope" table (paywalled sources, chatbot/hosting, auth/multi-tenancy, native Ollama adapter, rename, autonomous v1.0.0); PROJECT.md Out of Scope.
|
||||
- Answer: Explicit, reasoned, and enforced (e.g., OpenAI-compatible-only is restated in constraints and architecture).
|
||||
- Confidence: 0.90
|
||||
- **Q4**: Are there hidden requirements only disclosed late in delivery?
|
||||
- Evidence: **FOUND — two.** (a) `frontend/src/lib/api/client.ts` lines 123–179 document v0.2 mappings to `GET /api/graph`, `GET /api/tasks`, `GET /api/library`, `GET /api/library/connections`, `GET /api/suggestions`, `POST /api/refine/questions`, `POST /api/refine/compose`, `PUT /api/config` — none of which appear in REQ-25 (runs, methodologies, lifecycle, SSE, config-read only) or any v0.2 phase mapping. (b) The same file exposes no `pauseRun`/`resumeRun` while REQ-25 mandates pause/resume in the API and REQ-15 mandates server-side pause/resume across restarts; line 69's comment claims "pause/resume stay client-side concerns," which contradicts the engine requirement.
|
||||
- Answer: The P3 swap ("behind the unchanged ApiClient interface," REQ-26/REQ-54) is wired to detonate: either the backend grows 7+ undocumented endpoints (scope expansion inside the MVP milestone), or live mode silently breaks six approved UI surfaces, or the interface changes anyway (breaking REQ-54's zero-rewrite promise without a plan). This is the single largest finding of this grill.
|
||||
- Confidence: 0.50
|
||||
- Decision: G-002 — declare the v0.2 live-mode surface = **core research loop only** (Ask, Choose, Plan & Run, Reading Room + read-only `GET /api/config`); Library, Settings persistence, Suggestions, Refine, project graph, task inbox, and user profile stay mock-bound behind ApiClient in live mode, honestly marked in-UI, with their backends explicitly deferred to v0.3+. G-003 — add `pauseRun`/`resumeRun` to ApiClient in P3 as the single planned, documented interface extension (mock gains matching implementations), superseding the v0.1 client-side-pause comment; REQ-54's zero-rewrite promise is amended to "no *unplanned* rewrites."
|
||||
- Escalation: [G-009] — owner must ratify the live-mode boundary at the P5 gate (see Escalations).
|
||||
|
||||
### Axis 3 — Architecture and Technical Feasibility
|
||||
- **Q1**: Has the architecture been validated by the people who will build and operate it?
|
||||
- Evidence: RESEARCH-v02.md — 8 topics with concrete API payload shapes, pitfalls, and 14 conclusions written for the builders (pydantic event unions, shielded state writes, `os.replace` atomicity, pytest-asyncio modes, hatchling `packages` key). But `backend/` does not exist yet — zero lines of the architecture have run.
|
||||
- Answer: Design-side validation is strong (this is what a builder-facing research doc should look like); runtime validation is zero, by definition of P0. The risk concentrates at first contact: ollama.com/v1 shapes, Tavily payload, SSE backpressure. P1 Wave 3 is the first live proof.
|
||||
- Confidence: 0.65
|
||||
- **Q2**: What is the integration surface, and where does it quietly double in cost?
|
||||
- Evidence: Five adapter slots (ARCHITECTURE §2); SSE transport (D-014); OpenAPI client generation (D-013, RESEARCH §7 — flags operation-id stability and regen-from-snapshot discipline as pitfalls); the ApiClient surface gap (Axis 2 Q4).
|
||||
- Answer: 5 adapters + SSE + codegen swap = 7 integration seams. The cost-doubling candidate is the P3 swap (seam #7) — precisely where the undocumented-endpoint problem sits. Second candidate: the openapi.json snapshot discipline exists only as a research *pitfall note*, not a P3 must-have.
|
||||
- Confidence: 0.60
|
||||
- Decision: G-005 — P3 must-have: commit the backend `openapi.json` snapshot; regenerate `frontend/src/lib/api/generated/` only from the snapshot (never a live server); add a schema-drift check to the gates; fix FastAPI operation-ids at route creation.
|
||||
- **Q3**: Is there an existing system being replaced — actual data volume, transaction rate, edge-case behavior?
|
||||
- Evidence: Greenfield backend; the "replaced system" is `MockApiClient` (fixtures + scripted replay, frontend/src/lib/api/mock/). REQ-54 defines the swap contract; PERSONAS.md binds the frontend-engineer to it.
|
||||
- Answer: No data migration risk. The migration is one interface binding line (client.ts:187) — provided G-002/G-003 are absorbed; otherwise the "small migration" is the biggest one in the project.
|
||||
- Confidence: 0.70
|
||||
- **Q4**: What technical debt is inherited, and is it budgeted for?
|
||||
- Evidence: (a) the v0.1 "client-side pause" comment (client.ts:69) contradicting REQ-15 — budgeted via G-003. (b) DDG `html.duckduckgo.com/html` scrape as the keyless fallback (D-012, confidence 0.75) — RESEARCH §3 itself warns "rate-limits aggressively... best-effort." (c) Scripted-replay SSE shape must match D-014's event union — verified aligned (D-014 rationale cites the MockApiClient replay contract).
|
||||
- Answer: The DDG fallback is the fragility hotspot: a legacy HTML surface, conservatively scraped, that the research file already expects to fail. The zero-config promise of the product rests partly on it. Mitigation exists in design (surfaced warning, never hang) but is not yet test-enforced.
|
||||
- Confidence: 0.55
|
||||
- Decision: G-008 — DDG fallback is tested against recorded HTML fixtures only (never live-scraped in CI); the degradation contract (partial/empty results + typed warning, never hang, never block the engine) is asserted by test. DDG is documented as degraded-mode, not a reliable second source.
|
||||
|
||||
### Axis 4 — People, Skills, and Organization
|
||||
- **Q1**: Which 2–3 people, if they left, would the project fail?
|
||||
- Evidence: The entire delivery capability is one agent pipeline (coreci) + one owner (decision commit 3fd0e10; all directives owner-recorded). PERSONAS.md is a persona roster, not a human team.
|
||||
- Answer: Key-person dependency is total and structural: owner disengagement stalls every gate by design (that is the governance model); pipeline failure stops all delivery. For a single-user, owner-directed tool this is acceptable — but it means timeline estimates are really "pipeline availability" estimates, and there is no bench.
|
||||
- Confidence: 0.60
|
||||
- **Q2**: Are resources actually allocated at the percentages claimed?
|
||||
- Evidence: Autonomy level "full" (config.json); v0.1 shipped 9 phases, PRs #1–#8, tags v0.1.0–v0.1.8 within ~10 hours of wall clock (commits dd2538f 00:44 → d6ddcfe 10:03, 2026-09-27).
|
||||
- Answer: The pipeline is demonstrably 100% allocated and not fighting BAU fires — velocity is exceptional and verified by artifact trail, not status reports.
|
||||
- Confidence: 0.85
|
||||
- **Q3**: Is there a product owner with actual authority to prioritize?
|
||||
- Evidence: Owner directives recorded as binding decisions (D-018, D-019 at confidence 1.0); two gate rejections prove authority in practice.
|
||||
- Answer: Unambiguous single owner; no committee. Model is working.
|
||||
- Confidence: 0.90
|
||||
- **Q4**: Is the team building capability it doesn't have, hoping to learn as they go?
|
||||
- Evidence: The stack is being built in-milestone: FastAPI, asyncio durability (shielded writes, CancelledError discipline — RESEARCH §5), sse-starlette, httpx adapters, openapi codegen. No backend code exists yet (repo is frontend-only today). RESEARCH-v02.md documents the failure modes in advance (pytest-asyncio event_loop removal, hatchling src-layout wheel pitfall, uvicorn single-worker constraint).
|
||||
- Answer: Yes, capability is being built on the fly — acceptable here because (a) this is a prototype-to-MVP transition with research-backed patterns, and (b) zero-key fakes (REQ-31) mean the engine can be proven without live services. But P2 is where "learning asyncio under load" meets "13 requirements" — see Axis 5.
|
||||
- Confidence: 0.60
|
||||
|
||||
### Axis 5 — Timeline and Estimates
|
||||
- **Q1**: Was the deadline set before or after scope and approach were understood?
|
||||
- Evidence: No wall-clock deadlines exist anywhere in ROADMAP v0.2 — sequencing is dependency-driven (P0→P5), cadence is owner gates (D-018: halt at the owner approval gate).
|
||||
- Answer: No reverse-engineered deadline — the healthiest possible pattern. The only "date" is the owner's attention, which v0.1 proved is present.
|
||||
- Confidence: 0.85
|
||||
- **Q2**: What is the critical path, and what single thing would push it by 3+ months?
|
||||
- Evidence: Critical path P1 → P2 → P3 → P4 (sequential phases, ROADMAP). P2 is the milestone's heaviest phase: 13 requirements (REQ-07, 12, 13–24) including the durable asyncio state machine, trust loop, and live probing. The 3-month pusher is a P3 integration-drift discovery: the ApiClient surface gap (Axis 2 Q4) hitting mid-swap, after P2's engine is built against the wrong boundary assumptions.
|
||||
- Answer: Critical path is P2-heavy and P3-fragile. G-002/G-003 defuse the P3 risk; P2's size needs a pressure valve.
|
||||
- Confidence: 0.70
|
||||
- Decision: G-006 — permit P2 to ship as two patches (P2a: engine + durable state + SSE; P2b: trust loop + live probe/REQ-07/12) or run mid-phase wave gates, without changing milestone scope; the P3 swap plan (endpoint list per G-002) must be drafted at P2 *start*, not P3 start, so drift surfaces before the engine is frozen.
|
||||
- **Q3**: Are the estimates evidence-based?
|
||||
- Evidence: The only velocity base is v0.1: ~1 phase/hour sustained across 9 phases including review+audit each. v0.2 phases are materially heavier (greenfield backend, adversarial tests). MISSING: any per-phase sizing in ROADMAP/PLAN.
|
||||
- Answer: No estimates exist to be wrong — but also none to be right. v0.1 velocity does not transfer 1:1 (frontend screens vs. a durable async engine); treating it as a base rate would understate P2. The mitigation is the phase-patch mechanism (G-006) rather than fake numbers, which this grill endorses over invented estimates.
|
||||
- Confidence: 0.60
|
||||
- **Q4**: Is there a working definition of done?
|
||||
- Evidence: Per-phase success criteria (ROADMAP); PLAN.md gates + "REQ coverage assert: each REQ mapped to ≥1 test or contract proof"; verify stage precedent (PASS-WITH-FIXES with in-phase P0 fixes, PRs #4/#6/#8); final owner gate.
|
||||
- Answer: Yes, and it is enforced — v0.1 reviews fixed 5 P0s and multiple P1s *in-phase*, which is proof the DoD has teeth.
|
||||
- Confidence: 0.85
|
||||
- Decision: G-001 — resolve a DoD contradiction inside P1's own plan: PLAN.md must_haves and tasks T-1.10/T-1.11 claim REQ-07/REQ-12 delivery ("GET /models + structured-output smoke test" = live probing) while PLAN's frontmatter (`requirements: [REQ-01..06, 08..11, 31]`), ROADMAP P1 ("REQ-07, REQ-12 ship in P2... live probing against real endpoints in P2"), and REQUIREMENTS traceability all map them to P2. Binding: P1 ships the typed ProbeResult + structured-output warning *contracts* proven with fake adapters; live endpoint probing executes in P2 with the engine. Secondary bookkeeping fix in the same decision: the "REQ-13..24" range notation includes REQ-17 in both P2 and P4 rows — REQ-17 (3-click on real stack) stays in P4; ranges are corrected to exclude it.
|
||||
|
||||
### Axis 6 — Budget and Financial Realism
|
||||
- **Q1**: What percentage of budget is spent vs. remaining?
|
||||
- Evidence: MISSING — no cost ledger exists. Known spends: pipeline tokens (v0.1: 9 phases of agent execution; v0.2 P0 stages), Ollama cloud key (config.json secrets scope `model`), Tavily key (scope `search`), self-hosted LXC infra (DEPLOY-RECORD.md).
|
||||
- Answer: Untracked. For a single-user local-first tool the absolute numbers are small, but "untracked" is still a finding: no one can currently answer what a Deep Report run costs.
|
||||
- Confidence: 0.60
|
||||
- **Q2**: Are there predictable cost drivers not in the original budget?
|
||||
- Evidence: Deep Report = depth 3/breadth 2 → per run: ~dozens of model calls (planner + depth-loop reflections + writer + verification passes) plus search calls; Ollama cloud is pay-per-use; Tavily free tier is quota-capped; DDG fallback is free-but-fragile (D-012). Localhost:11434 fallback is the zero-cost escape hatch (REQ-08).
|
||||
- Answer: Cost drivers are real and unbounded per run (no request caps on loops beyond dial defaults). The local-fallback design keeps the project from *hard* dependency on spend, but there is no visibility.
|
||||
- Confidence: 0.60
|
||||
- Decision: G-007 — per-run counters (model requests, tokens, search calls) recorded in the run trace and surfaced via `GET /api/config`; advisory only in MVP.
|
||||
- **Q3**: Burn rate and runway?
|
||||
- Evidence: MISSING formal tracking; mitigated by local-first defaults (keyless localhost fallback, DDG keyless fallback) meaning the runway floor is "zero marginal cost, degraded mode."
|
||||
- Answer: The architecture is honest about this: the product's default posture degrades to free. Runway is effectively unbounded if the owner accepts degraded mode; the grill accepts this as sound for the MVP.
|
||||
- Confidence: 0.70
|
||||
- **Q4**: Is the budget contingent on something that hasn't happened yet?
|
||||
- Evidence: No — no sale/board/hiring dependency. Keys exist (config.json scopes), infra exists (LXC), pipeline is running.
|
||||
- Answer: Not contingent. Proceed.
|
||||
- Confidence: 0.80
|
||||
|
||||
### Axis 7 — Risks, Assumptions, and Dependencies
|
||||
- **Q1**: Top 3 assumptions, and evidence for each?
|
||||
- Evidence: (a) *Adapter endpoint compatibility* — ollama.com/v1 OpenAI-shape and Tavily POST payload researched with concrete shapes (RESEARCH §2/§3), but unvalidated live in-repo; P1 Wave 3 is the test. (b) *DDG fallback stability* — RESEARCH §3 itself rates it "best-effort, aggressively rate-limited" (weakest assumption in the plan; D-012 confidence 0.75). (c) *Zero-rewrite client swap* — typed boundary exists (client.ts), RESEARCHX_API_MODE designed (D-020), but the interface gaps (Axis 2 Q4) undercut it until G-002/G-003 land.
|
||||
- Answer: Two of three are well-evidenced; the DDG assumption is the one the project's own research argues against relying on. All three have named first-contact phases (P1 Wave 3, P2, P3).
|
||||
- Confidence: 0.60
|
||||
- Decision: G-008 (fixture-tested DDG degradation contract) — see Axis 3 Q4.
|
||||
- **Q2**: External dependencies outside the team?
|
||||
- Evidence: Tavily quota; Ollama cloud key validity; DDG uncontrolled HTML surface; none of these are under the project's control; all have fallbacks *except* Tavily→DDG quality.
|
||||
- Answer: Dependencies are acknowledged with degradation paths — the correct posture. The unmitigated residual is *degraded search quality* when Tavily is exhausted, which directly attacks the ≥99% liveness bar.
|
||||
- Confidence: 0.65
|
||||
- **Q3**: The single risk that kills the project, and is there a mitigation?
|
||||
- Evidence: P3 swap discovering the ApiClient/surface drift after the engine is frozen (evidence chain: client.ts:123–179 vs REQ-25; ROADMAP P3 "the ApiClient interface absorbs any prototype-era API drift" — an assertion, not a plan).
|
||||
- Answer: Kill risk = integration drift at P3 → rework spiral through a hand-rolled engine → owner patience (already tested twice by extensions) exhausts. Mitigation now exists in writing: G-002 (boundary declared), G-003 (interface change planned), G-005 (snapshot discipline), G-006 (P3 plan drafted at P2 start). This grill's core job is done if those four hold.
|
||||
- Confidence: 0.65
|
||||
- **Q4**: Has a pre-mortem been done?
|
||||
- Evidence: MISSING — no pre-mortem artifact. This grill is the closest proxy.
|
||||
- Answer: "It's 12 months out and v0.2 failed" — the derivable causes, in order: (1) P3 surface gap hit late (mitigated: G-002/003/005/006); (2) P2 asyncio durability bugs (pause/resume across restarts is genuinely hard — mitigated by RESEARCH §5 patterns + REQ-32 pause-mid-depth-loop adversarial tests); (3) quote anchoring thrash on real noisy pages (difflib 0.85 is fixture-proven only — live-page behavior unknown until P4/live runs); (4) DDG dead + Tavily exhausted → un-demoable zero-config mode (mitigated: G-008 + owner-held Tavily key).
|
||||
- Confidence: 0.60
|
||||
|
||||
### Axis 8 — Governance, Decision-Making, and Communication
|
||||
- **Q1**: Who is the decision-maker when two parties disagree?
|
||||
- Evidence: Single owner; all decisions recorded as decision commits (3fd0e10; D-007..D-020 in e79102b); escalation hooks codified in config.json (deploy, delete_data, merge_to_main, prototype_approval).
|
||||
- Answer: Unambiguous. No dual-executive stall is possible in this structure.
|
||||
- Confidence: 0.90
|
||||
- **Q2**: Governance cadence and escalation pattern — useful in a crisis?
|
||||
- Evidence: Per-phase ship + verify + audit with `---ci---` blocks; v0.1 precedent: pipeline *halted itself* at gates twice (7db53d62, 57c5231) rather than self-approving — the escalation pattern is proven under the exact stress it exists for.
|
||||
- Answer: Cadence is per-phase and gates have been exercised under pressure. Crisis-tested by construction.
|
||||
- Confidence: 0.85
|
||||
- **Q3**: What is being omitted from status reports?
|
||||
- Evidence: Honest reporting verified in history: "PASS-WITH-FIXES," "PASS-WITH-NOTES," "approval_pending," P0s disclosed and fixed in-phase; DEPLOY-RECORD.md as a separate ledger. The one systemic omission this grill found lives in *spec comments, not status blocks*: the client.ts v0.2 endpoint mappings that were never promoted into REQUIREMENTS — a silent scope assumption, now surfaced (G-002).
|
||||
- Answer: Status reporting is honest. Spec-side implication drift is the channel where optimism leaked — and it's now on the record.
|
||||
- Confidence: 0.80
|
||||
- **Q4**: Is there a stop-the-project trigger?
|
||||
- Evidence: Owner rejection at gate → binding fix cycle (v0.1 exercised twice); v1.0.0 owner-only (config.json major_locked); D-018 halts at the v0.2 gate; D-019 forbids deploys without directive.
|
||||
- Answer: Yes — and rejection has demonstrably not become politically impossible; the owner said "not yet, extend" twice without ceremony.
|
||||
- Confidence: 0.85
|
||||
|
||||
### Axis 9 — Change, Adoption, and Operational Readiness
|
||||
- **Q1**: Who will use this, how does their work change, and what's in it for them?
|
||||
- Evidence: Single-user owner-researcher (PROJECT.md constraint: "Single-user, local-first"); work change = ad-hoc prompting → structured, reproducible methodology runs with auditable citations (REQ-04 dial snapshots, REQ-21 ledger, REQ-22 checklist).
|
||||
- Answer: The user is the sponsor — adoption politics are minimal, and the value proposition (trustable citations) is the same thesis the sponsor approved twice. The real adoption test is whether live-mode runs *feel* better than the mock — which lands at P4/REQ-17.
|
||||
- Confidence: 0.80
|
||||
- **Q2**: Is ops involved now, or handed a finished product at go-live?
|
||||
- Evidence: The pipeline *is* ops: deploy.sh with rolling swap + `researchx-prev` rollback container (commit 2ffc71c), gates re-run at deploy time (DEPLOY-RECORD.md rows), VERSIONING.md rules. Gap: the deploy system is frontend-only; the MVP's live mode requires a local backend process (uvicorn, single worker — RESEARCH §4) whose serving story at the owner gate is undefined (D-019 forbids deploys without directive, so live-mode *presentation* mode is an open question).
|
||||
- Answer: Ops is built-in and rollback-capable for the frontend. The backend has no deploy story — coherent with local-first, but it means the P5 owner-gate demo of the *live* MVP needs a plan (local backend + keys at demo time, or gate runs on mock mode). This rides with G-009.
|
||||
- Confidence: 0.60
|
||||
- **Q3**: What is the rollback plan?
|
||||
- Evidence: Tags v0.1.0–v0.1.8 preserve history; deploy.sh swaps containers with a `researchx-prev` rollback; durable run state is per-run JSON on local disk (D-017) — engine changes cannot corrupt other runs' state; frontend mock mode (RESEARCHX_API_MODE=mock default, D-020) is itself a rollback for the entire live integration.
|
||||
- Answer: Multi-layer rollback exists and is codified. The mock-default flag is an unusually good one — the MVP can ship with live mode opt-in, making go-live risk nearly zero.
|
||||
- Confidence: 0.80
|
||||
- **Q4**: Have success criteria been validated with the people who will judge success?
|
||||
- Evidence: REQ-17 (≤3 clicks on real stack) is P4-tested, validated on mock in v0.1 (REQ-61 pinned by test). REQ-23 (≥99% citation liveness) — **measurement gap**: the entire adversarial suite is zero-key on a 5–8 page fixture corpus (RESEARCH §5b); 99% liveness on fixture URLs is vacuous, and no keyed/live validation run is scheduled anywhere in P1–P5.
|
||||
- Answer: The 99% bar — the project's headline promise — is currently scheduled to be "verified" in a harness that cannot verify it. If the owner gate treats REQ-23 as CI-green, the status report will be honest-looking but hollow.
|
||||
- Confidence: 0.55
|
||||
- Decision: G-004 — REQ-23 is defined as a **runtime per-run gate** enforced at export (failures listed in the verification checklist, REQ-22) plus an opt-in, keyed, live-URL test tier (skipped by default, run before ships at owner discretion); P4 proves the *gate mechanics* with fixtures and explicitly does not claim live 99% attainment. The MVP definition of done for REQ-23 = the gate exists and enforces, not "CI showed 99%."
|
||||
|
||||
### Meta — Closing Review
|
||||
- **Q1**: If you were the auditor, what would you flag?
|
||||
- Evidence: The two internal contradictions (PLAN vs ROADMAP/REQUIREMENTS on REQ-07/12, PLAN must_haves vs PLAN frontmatter; REQ-17 double-mapped in range notation) and the spec-comment endpoint mappings outside REQ-25 (client.ts:123–179).
|
||||
- Answer: Flag #1: phase-mapping integrity — the traceability table is the single source of truth and PLAN must conform to it (G-001). Flag #2: an approved artifact (client.ts) documents v0.2 API surface that the v0.2 spec does not contain — the grill treats this as the most consequential process lapse found, because it would have surfaced as a P3 "surprise" after the engine was frozen.
|
||||
- Confidence: 0.75
|
||||
- **Q2**: What is the project not doing that it should?
|
||||
- Evidence: Missing items found across axes: openapi snapshot drift check as a gate (G-005), per-run cost counters (G-007), live-URL test tier for REQ-23 (G-004), P3 swap plan at P2 start (G-006), fixture-based DDG tests (G-008), live-mode boundary declaration (G-002).
|
||||
- Answer: All six are now bound. Nothing found suggests the project is doing something it should not.
|
||||
- Confidence: 0.75
|
||||
- **Q3**: What is the simplest version that delivers 80% of the value?
|
||||
- Evidence: The MVP already trimmed to 2 of 8 methodologies (REQ-03) and a core-loop-only API (REQ-25). The remaining 20% of complexity carrying outsized risk: Deep Report's depth-3 breadth-2 loop (multiplies engine, trust, and cost surface) vs. Rapid Answer's depth-1 single-pass core.
|
||||
- Answer: Rapid Answer alone + trust loop = 80% of the thesis proof. Deep Report is the pre-agreed deferral lever if P2 overruns — decided *now*, while it is cheap, rather than mid-crisis.
|
||||
- Confidence: 0.65
|
||||
- Decision: G-010 — contingency lever, pre-agreed: if P2/P4 slip beyond one extension cycle, Deep Report (REQ-03 partial) defers to v0.3 and the MVP ships Rapid Answer end-to-end; the lever requires owner sign-off when pulled, per gate rules.
|
||||
- **Q4**: What must be true for this to succeed in the next 90 days, and is it true today?
|
||||
- Evidence: Required: (a) researched adapter shapes hold live (untested — first proof P1 Wave 3); (b) owner engagement continues (proven: 3 gate decisions in one day); (c) Tavily key active/quota available (held in secrets scope; unverified live); (d) the P3 boundary is as declared in G-002 (currently *not* true in writing — client.ts says otherwise).
|
||||
- Answer: Three of four are true or evidenced; (d) becomes true when the orchestrator promotes G-002/G-003. That promotion is the gating action of this entire grill.
|
||||
- Confidence: 0.70
|
||||
|
||||
---
|
||||
|
||||
### Binding Decisions
|
||||
| ID | Axis | Decision | Confidence |
|
||||
|----|------|----------|-----------|
|
||||
| G-001 | 2/5 | Resolve the REQ-07/REQ-12 phase contradiction in PLAN.md's favor of the ROADMAP/traceability mapping: P1 ships typed probe/warning contracts proven with fakes; live endpoint probing (GET /models, structured-output smoke test) executes in P2. Reword PLAN must_haves T-1.10/T-1.11 to contract level; fix the REQ-13..24 range notation so REQ-17 appears only in P4. | 0.85 |
|
||||
| G-002 | 2/3 | Declare v0.2 live-mode surface = core research loop only (Ask, Choose, Plan & Run, Reading Room, read-only GET /api/config). Library, Settings persistence, Suggestions, Refine, project graph, task inbox, user profile stay mock-bound in live mode, honestly marked; their backends are v0.3+ scope. Update client.ts v0.2-mapping comments accordingly. | 0.80 |
|
||||
| G-003 | 3 | Add `pauseRun`/`resumeRun` to the ApiClient interface in P3 as the single planned, documented interface extension (mock gains matching implementations); run-level pause is server-side per REQ-15/REQ-25, superseding the v0.1 "client-side concerns" comment. REQ-54 amended: no *unplanned* rewrites. | 0.80 |
|
||||
| G-004 | 7/9 | REQ-23 (≥99% liveness) is a runtime per-run gate enforced at export and surfaced in the verification checklist — not a zero-key CI assertion. Add an opt-in keyed live-URL test tier (skipped by default). P4 proves gate mechanics with fixtures; MVP DoD for REQ-23 = the gate exists and enforces. | 0.75 |
|
||||
| G-005 | 3/5 | P3 must-have: commit the backend openapi.json snapshot; regenerate frontend/src/lib/api/generated/ only from the snapshot; add a schema-drift check to gates; fix FastAPI operation-ids at route creation. | 0.85 |
|
||||
| G-006 | 5 | Permit P2 (13 REQs — heaviest phase) to ship as two patches (P2a engine+durable state+SSE, P2b trust loop+live probe) or run mid-phase wave gates without milestone scope change; draft the P3 swap endpoint plan (per G-002) at P2 start, not P3 start. | 0.70 |
|
||||
| G-007 | 1/6 | Per-run advisory counters (model requests, tokens, search calls) recorded in the run trace and surfaced via GET /api/config — the cheapest honest ROI/burn instrumentation; no hard spend cap in MVP. | 0.70 |
|
||||
| G-008 | 3/7 | DDG fallback tested against recorded HTML fixtures only (never live-scraped in CI); degradation contract (partial/empty + typed warning, never hang the engine) asserted by test; DDG documented as degraded-mode, not a reliable second source. | 0.70 |
|
||||
| G-010 | Meta | Pre-agreed contingency lever: if P2/P4 slip beyond one extension cycle, Deep Report (depth 3) defers to v0.3 and the MVP ships Rapid Answer end-to-end; pulling the lever requires owner sign-off at a gate. | 0.65 |
|
||||
|
||||
### Escalations
|
||||
- [G-009] Axis 2 — Owner ratification required at the P5 gate (D-018 halt): the v0.2 MVP demonstrates the **live core research loop only**; the remaining approved-prototype surfaces (Library, Settings persistence, Suggestions, Refine, graph/inbox previews, profile) remain mock-bound until v0.3+, and the live-mode gate demo requires a local backend + keys (or runs in mock mode) since D-019 forbids deploys. This redefines what "MVP done" looks like at the demo and must not be discovered by the owner at the gate — it must be ratified in advance. Engineering-side handling is bound in G-002/G-003; the ratification itself is unresolved until the owner records it.
|
||||
- **RESOLVED 2026-09-27: RATIFIED.** Owner approval recorded via /ci-run 'APPROVED' at the v0.2.5 gate (decision commit 4721c2a), following the owner-directed UAT staging at https://research.coreci.dev. Live-mode scope = core research loop only is the accepted MVP boundary; mock-bound surfaces move to v0.3+.
|
||||
# CIAgent Grill Report
|
||||
|
||||
## Run: 2026-09-29 17:05 (mode: mechanical, focus: all)
|
||||
|
||||
### Verdict: Proceed (confidence: 0.68)
|
||||
|
||||
The plan survives adversarial re-verification better than any artifact this project has produced: the grill independently re-checked the research file's load-bearing claims against the tree and found them exact — the shared-adapter fallback mutation (`ollama.py:73-79`), the `config_status` private-registry read (`app.py:174-177`), the CORS-credentials gap (`app.py:143-148`), the fake-serving module global (`app.py:359`), the missing `append_run_artifact` protocol method, the `[p-N]`/`[^pN]` marker split (`actor.py:337` vs `report-body.ts:18`), the export fabrication (`live-client.ts:296-304/335/376`), and the 8-test-file partitioning blast radius. The auth supersession is traceable end to end (a4afccc constraint rewrite + D-101..D-115 @ 33f4fa0 + ROADMAP session note — matching the owner's "one milestone / admin-managed / server-key-fallback" session picks). Engine-contract preservation is a real design discipline, not a slogan. But the grill found **one "verified" research claim that is false** (the restart sweep's CANCELLED-from-any-state legality — `states.py` forbids PLANNING→CANCELLED), **three phase/bookkeeping contradictions** (REQ-75 triple-mapped across three documents; two stale out-of-scope rows contradicting the same-file supersession; C3-19's pitfall-phase mapping vs PLAN), **one unaddressed prototype-fidelity gap** (the task-inbox checkbox cannot survive the derived live read-model), and **an honestly-unbudgeted public attack surface** (no password policy, no login rate limit on a TLS-fronted public URL). All are absorbed as binding decisions G-101..G-110; one item is genuinely owner-required (G-111: P4's definition-of-done depends on a staged host deployment that D-019 forbids without a directive). The milestone is heavier than v0.2 — the P3 pressure valve absent from the plan is bound here (G-108) as the G-006 precedent. Proceed.
|
||||
|
||||
### Axis 1 — Business Case
|
||||
- **Q1**: What problem does this actually solve, and is that problem still the top priority?
|
||||
- Evidence: v0.2.5 approved and merged 2026-09-27 (8db45e4, b11b202); v0.3 scope owner-directed the next day, 2026-09-29 (a4afccc; ROADMAP "owner session 2026-09-29"); DEPLOY-RECORD.md itself records "live mode additionally needs the backend API on the host, which is v0.3+ deployment work" — the deployment gap is acknowledged in the ship record, not invented by the pipeline.
|
||||
- Answer: Real and owner-named: convert the ratified mock-bound surfaces + single-user local loop into a hosted multi-user product with real inference. The same-day sequencing after v0.2.5 approval is the strongest available priority signal.
|
||||
- Confidence: 0.80
|
||||
- **Q2**: Who is the named executive sponsor, and when did they last make a decision under pressure?
|
||||
- Evidence: Owner pressure-tested through v0.1 (two gate rejections → binding extensions, commits 57c5231, 3fd0e10), G-009 ratified at the v0.2 gate (4721c2a); the v0.3 direction itself is a decision under pressure — superseding a recorded constraint ("single-user, local-first") the same day the MVP shipped.
|
||||
- Answer: Sponsor engaged and decisive; the supersession is a recorded constraint change, not scope drift.
|
||||
- Confidence: 0.85
|
||||
- **Q3**: What happens to the business if the project is cancelled?
|
||||
- Evidence: v0.2.5 live mode = core research loop only, mock-default frontend, no hosting (G-009 ratification, ROADMAP v0.2 complete note); the owner-stated v0.3 purpose is the hosted product.
|
||||
- Answer: Cancelling strands the G-009 promise ("mock-bound surfaces move to v0.3+") and leaves the product undeployable in its owner-intended form. Not a zombie — the payoff milestone.
|
||||
- Confidence: 0.75
|
||||
- **Q4**: Is the ROI calculated against a counterfactual?
|
||||
- Evidence: MISSING — no formal ROI anywhere; G-007 counters carried advisory-only, owner-keyed in P2 (T-2.6, P-13). v0.3 adds real costs v0.2 did not carry: public infrastructure exposure and the server-tier key serving any admin-created user's runs.
|
||||
- Answer: Still owner-judgment — tolerable for a sponsor-beneficiary product with an admin-bounded user set, but the server-key cost posture is now a stated-nowhere exposure; bound as G-110.
|
||||
- Confidence: 0.55
|
||||
- Decision: G-110 — document the cost posture: server-tier runs bill the owner's key; the bound is admin-managed accounts; per-user counters surface server-tier usage; hard caps deferred v0.4+.
|
||||
|
||||
### Axis 2 — Scope and Requirements
|
||||
- **Q1**: Is the scope expanding, contracting, or genuinely stable?
|
||||
- Evidence: Scope grew by owner directive (ROADMAP 2026-09-29 note); REQ-69..85 frozen with per-phase traceability; PLAN frontmatter carries all 17. But the G-001-class phase contradictions persisting: REQ-75 is mapped P3 by REQUIREMENTS traceability, P4-only by ROADMAP P3/P4 lists, and split by PLAN (delegation-end in P3, hosted in P4). C3-19 likewise maps P-1/P-2/P-13 to P1 while PLAN closes them in P2.
|
||||
- Answer: Formal scope stable and owner-frozen; the traceability layer is where the drift lives — three documents disagree about one requirement. Bound in G-103.
|
||||
- Confidence: 0.60
|
||||
- Decision: G-103 — bind the REQ-75 split explicitly; fix the traceability row notation; declare PLAN's pitfall-closure mapping authoritative over C3-19's compressed prose.
|
||||
- **Q2**: Who owns the requirements, and have they been frozen?
|
||||
- Evidence: Owner-directed supersession traceable (a4afccc strikethrough + rationale; D-101..D-115 clarify commit 33f4fa0 at confidence 0.75–0.95).
|
||||
- Answer: Owner owns; pipeline freezes and traces. The supersession discipline is exemplary — the constraint change is visible, dated, and strikethrough-preserved rather than silently rewritten.
|
||||
- Confidence: 0.85
|
||||
- **Q3**: What is explicitly out of scope?
|
||||
- Evidence: **CONTRADICTION FOUND** — REQUIREMENTS.md Out-of-Scope table still carries "Auth / multi-tenancy | Single-user local-first MVP"; PROJECT.md Out of Scope still lists "multi-tenant/auth" — while PROJECT.md:34 records the supersession in the same file and REQ-69..71 define the now-in-scope auth. The honest deferrals (OAuth v0.4+, 6 methodologies v0.4+, KEK rotation, account deletion, rehydration) are all recorded.
|
||||
- Answer: Mostly honest, with two stale rows that directly contradict a recorded decision — exactly the spec-comment drift class the v0.2 grill flagged. Bound in G-102.
|
||||
- Confidence: 0.85
|
||||
- Decision: G-102 — update both stale out-of-scope rows to match the supersession.
|
||||
- **Q4**: Are there hidden requirements only disclosed late in delivery?
|
||||
- Evidence: **FOUND — three.** (a) REQ-78 promises "checkable follow-ups"; `library-view.tsx:430-471` ships a working checkbox; the live read-model (D-114) derives status answered-iff-DONE over a GET-only route — a live user's check silently reverts on refetch; PLAN never addresses it. (b) No password minimum length exists anywhere (grep-verified). (c) No login rate limiting anywhere (grep-verified) — argon2 verify is ~50-100ms CPU, a public flood is a DoS amplifier on a single worker.
|
||||
- Answer: The v0.2 G-002 lesson recurs once (the toggle); the exposure items are new-hosting hidden requirements. All three bound (G-109, G-104).
|
||||
- Confidence: 0.75
|
||||
|
||||
### Axis 3 — Architecture and Technical Feasibility
|
||||
- **Q1**: Has the architecture been validated by the people who will build and operate it?
|
||||
- Evidence: RESEARCH-v03 claims re-verified by this grill: 11 of 12 sampled claims exact. **One claim is FALSE**: RESEARCH §7/C3-11 "rewrite every non-terminal state.json as CANCELLED (legal from every non-terminal state, verified states.py)" — `states.py` `_LEGAL_TRANSITIONS[PLANNING] = {AWAITING_APPROVAL, FAILED}`: no CANCELLED. A crash mid-PLANNING leaves a persisted PLANNING state the sweep cannot legally cancel.
|
||||
- Answer: Design-side validation is the strongest yet — but the single false "verified" claim is the precise failure mode mechanical research invites: verification asserted, not re-run. Bound in G-101.
|
||||
- Confidence: 0.85
|
||||
- Decision: G-101 — startup sweep must handle PLANNING-state runs: add PLANNING→CANCELLED to `_LEGAL_TRANSITIONS` or document the sweep as an explicit transition-validation bypass; pinned by test in T-1.8.
|
||||
- **Q2**: What is the integration surface, and where does it quietly double in cost?
|
||||
- Evidence: Five seams: auth dependency threading (P1), per-run factory + probe-per-tier (P2), disk read-models (P3), snapshot regen ×3 (G-005 wired into P1/P2/P3), HAProxy front extension (P4). The cost-doubler candidate is SSE-through-proxy: the only seam whose live configuration is deferred to deploy time.
|
||||
- Answer: Residual risk is bounded, not eliminated: the P4 smoke asserts a full unbuffered SSE run through the proxy path with the `-prev` rollback pair retained — worst case is one deploy iteration, not milestone rework. Accepted with the G-111 governance condition.
|
||||
- Confidence: 0.65
|
||||
- **Q3**: Is there an existing system being replaced?
|
||||
- Evidence: Greenfield hosting; pre-hosted artifacts explicitly disposable (D-106, owner-recorded); the "replaced system" is the mock delegation layer (live-client.ts:61) — deletion is D-112/T-3.8 with component tests proving zero rewrites (T-3.12). Volume model: admin-managed accounts (REQ-70) bound the user set; single worker mandatory.
|
||||
- Answer: No migration risk; the swap is designed as a deletion with regression pins. The one fidelity trap (task-inbox toggle) is found and bound (G-109).
|
||||
- Confidence: 0.75
|
||||
- **Q4**: What is the technical debt being inherited, and is it budgeted for?
|
||||
- Evidence: The 14-item pitfall ledger is budgeted per-phase (PLAN cross-phase notes) — with C3-19's compressed phase mapping contradicting PLAN (folded into G-103). Marker normalization: one-directional at the report.json boundary only; report.md raw form pinned by test; trust-loop regexes untouched.
|
||||
- Answer: If a future writer emits `[^pN]` natively, `_cited_urls` matches nothing → ledger empties → the G-004 export gate fails visibly at runtime, and the raw-form pin catches it in CI first. Silent corruption is not reachable.
|
||||
- Confidence: 0.70
|
||||
|
||||
### Axis 4 — People, Skills, and Organization
|
||||
- **Q1**: Which 2–3 people, if they left, would the project fail?
|
||||
- Evidence: One agent pipeline + one owner; gates owner-halt by design (D-018). The security-engineer persona is phase-specific P1/P2/P4 with an explicit removal note (PERSONAS.md).
|
||||
- Answer: Total key-person dependency remains structural and accepted; the P5 full multi-persona review covers the security persona's absence.
|
||||
- Confidence: 0.60
|
||||
- **Q2**: Are resources actually allocated at the percentages claimed?
|
||||
- Evidence: v0.1 shipped ~9 phases in ~1 day; v0.2 shipped P0..P5 in one day; v0.3 P0 commits same-day (a4afccc→dca86a7).
|
||||
- Answer: Velocity verified by artifact trail. v0.3's four phases are each multi-domain — the evidence supports capacity, with the G-108 valve for the heaviest phase.
|
||||
- Confidence: 0.80
|
||||
- **Q3**: Is there a product owner with actual authority to prioritize?
|
||||
- Evidence: Owner-directed supersession recorded same-day as MVP ship (a4afccc); G-009 ratified at gate (4721c2a); D-018/D-019 carry.
|
||||
- Answer: Unambiguous single owner, exercised.
|
||||
- Confidence: 0.90
|
||||
- **Q4**: Is the team building capability it doesn't have, hoping to learn as they go?
|
||||
- Evidence: argon2-cffi, AESGCM envelope encryption, server-side sessions, SSE-through-proxy are all new to the codebase; mitigations are research-verified concrete shapes + adversarial suites specced + a dedicated security-engineer persona on exactly the phases that carry the new capability.
|
||||
- Answer: Yes — learning-while-doing on a security-critical public surface, the highest-stakes version of this risk. The mitigation structure is proportionate. Acceptable; the G-104 hardening items close the gaps the research did not self-identify.
|
||||
- Confidence: 0.60
|
||||
|
||||
### Axis 5 — Timeline and Estimates
|
||||
- **Q1**: Was the deadline set before or after scope and approach were understood?
|
||||
- Evidence: No wall-clock deadlines anywhere; dependency-driven phases; owner gates as the only cadence.
|
||||
- Answer: Healthiest pattern, unchanged from v0.2.
|
||||
- Confidence: 0.85
|
||||
- **Q2**: What is the critical path, and what single thing would push it by 3+ months?
|
||||
- Evidence: P1→P2→P3→P4 sequential. P3 is the heaviest (7 REQs + standalone client + every surface swap + Reading Room fidelity + delegation-end); P1 is the widest (auth + partitioning across the 10-touchpoint list). The v0.2 pusher-class risk (integration drift at the swap) is pre-defused by G-005 wired into all three route-touching phases + the C3-18 exhaustive interface list.
|
||||
- Answer: The pusher is P3 overrun with no pre-agreed valve — v0.2 had G-006; v0.3's plan had none. Bound in G-108.
|
||||
- Confidence: 0.70
|
||||
- Decision: G-108 — P3 may ship as two patches (P3a backend; P3b frontend) or run mid-phase wave gates, without scope change; tags extend on the v0.3.x line.
|
||||
- **Q3**: Are the estimates evidence-based?
|
||||
- Evidence: v0.1/v0.2 velocity does not transfer 1:1 (auth/crypto/hosting are new domains); no per-phase sizing exists. The honest mitigation remains the phase-patch mechanism — now with the P3 seam named in advance.
|
||||
- Answer: No estimates to be wrong; the valve replaces fake numbers. Endorsed, per the v0.2 precedent.
|
||||
- Confidence: 0.60
|
||||
- **Q4**: Is there a working definition of done?
|
||||
- Evidence: Per-phase must-haves are concretely testable (401 pins, two-user isolation, tamper pins, sweep test, drift gate, four smoke assertions); P5 owner gate. **One DoD term is undefined**: T-4.7/T-4.9 run the smoke "against a staged deploy" — "staged" is never defined, and the host runs one container today.
|
||||
- Answer: DoD has teeth everywhere except the P4 smoke's target. Bound in G-107; its owner-contingency half escalates (G-111).
|
||||
- Confidence: 0.75
|
||||
- Decision: G-107 — "staged deploy" defined: candidate pair built + run on the LXC host, health green, the four smoke assertions executed host-locally against the candidates; public TLS-front cutover remains a discrete owner-directive act (D-019) with post-cutover public assertions run with the -prev pair armed.
|
||||
|
||||
### Axis 6 — Budget and Financial Realism
|
||||
- **Q1**: What percentage of budget is spent vs. remaining?
|
||||
- Evidence: MISSING — no cost ledger (v0.2 finding, carried). New v0.3 spends: second container family, argon2-cffi + cryptography (OSS), HAProxy front extension (existing instance), owner's Ollama key serving server-tier users.
|
||||
- Answer: Untracked, small in absolute terms, tolerable for sponsor-beneficiary scope — but the server-tier exposure is a stated-nowhere cost posture (G-110).
|
||||
- Confidence: 0.60
|
||||
- **Q2**: Are there predictable cost drivers not in the original budget?
|
||||
- Evidence: (a) Server-key tier: any admin-created user's keyless runs bill the owner (REQ-74 chain; G-007 counters advisory-only). (b) Host disk: deploy.sh (212 lines, verified) builds one image today with no free-space assert and no image prune; P4 doubles image families and runs 4 containers.
|
||||
- Answer: Both are real; (a) is a policy posture to state (G-110), (b) is deploy hygiene to add (G-106).
|
||||
- Confidence: 0.70
|
||||
- Decision: G-106 — deploy.sh gains a pre-build free-disk assertion and a post-swap dangling-image prune; the two-image/rollback-pair footprint is documented as the host budget.
|
||||
- **Q3**: Burn rate and runway?
|
||||
- Evidence: Keyless localhost + DDG fallback keep the zero-marginal-cost degraded posture (G-008 carries); per-user keys shift burn to users; refine endpoints ride the same chain with honest 503.
|
||||
- Answer: Runway effectively unbounded in degraded mode; hosting costs are fixed and small. Sound.
|
||||
- Confidence: 0.70
|
||||
- **Q4**: Is the budget contingent on something that hasn't happened yet?
|
||||
- Evidence: Infra exists (LXC + HAProxy already TLS-fronting research.coreci.dev); keys exist in scopes. One new prerequisite: RESEARCHX_KEK does not exist yet (provisioned into `.ciagent/.env.secrets` at P2 — never git).
|
||||
- Answer: Not contingent in any structural sense; the KEK is a trivial owner action — which is precisely why its loss posture must be stated before it becomes load-bearing (G-105).
|
||||
- Confidence: 0.75
|
||||
- Decision: G-105 — KEK-loss posture stated honestly: loss = per-user encrypted keys unrecoverable, chain degrades to server/keyless honestly, users re-enter keys under a re-provisioned KEK; the owner MUST back up the KEK off-repo; rotation tooling stays out of v0.3 (recorded, not silently dropped).
|
||||
|
||||
### Axis 7 — Risks, Assumptions, and Dependencies
|
||||
- **Q1**: Top 3 assumptions, and evidence for each?
|
||||
- Evidence: (a) *HAProxy live-host config accepts the SSE-safe front extension* — version and TLS front evidenced by two prior deploys; global/default sections unread until P4. Medium evidence. (b) *Single-worker in-memory registry/sessions suffice* — bounded by admin-managed accounts; restart sweep + pruning + disk read-models remove restart-fragility. Good evidence. (c) *Zero-component-rewrite surface swaps hold* — evidenced by the v0.2 P3 precedent + G-005 + T-3.12 regression suite; the task-toggle is the one discovered exception (G-109).
|
||||
- Answer: Two well-evidenced; (a) is the residual, bounded by the P4 smoke + rollback pair.
|
||||
- Confidence: 0.60
|
||||
- **Q2**: External dependencies outside the team?
|
||||
- Evidence: HAProxy host state (owner-held, D-019-gated), Ollama cloud/local + Tavily/DDG (carried v0.2, G-008 carries), Docker host disk (G-106 binds the missing hygiene).
|
||||
- Answer: All acknowledged with degradation or rollback paths; the proxy is the only one without a code-level fallback, and it has a process-level one.
|
||||
- Confidence: 0.65
|
||||
- **Q3**: The single risk that kills the project, and is there a mitigation?
|
||||
- Evidence: Authorization-class auth bugs on a public URL (cross-user leakage) — mitigated by construction: owner-keyed registry 404s + partitioned storage + per-request owner-scoped adapters + the two-user isolation test pinned in P1/P3 + the P4 exposure audit + admin-managed accounts bounding the victim set. The sweep-legality bug (G-101) is the found instance of the adjacent class.
|
||||
- Answer: Kill risk is real but the mitigation stack is the plan's strongest suit; the found bug proves the stack needs the grill, and now it has the fix.
|
||||
- Confidence: 0.70
|
||||
- **Q4**: Has a pre-mortem been done?
|
||||
- Evidence: MISSING as a formal artifact; this grill is the proxy.
|
||||
- Answer: "12 months out, v0.3 failed" — derivable causes: (1) the sweep bug corrupting restarts at first redeploy (bound: G-101); (2) P3 fidelity parity slips — one found (toggle, G-109), the rest pinned by parity tests; (3) proxy/SSE iteration thrash post-cutover (bounded: smoke + rollback); (4) KEK lost with no documented posture (bound: G-105); (5) public brute-force/DoS on a fresh auth surface (bound: G-104). All five now bound or bounded-by-design.
|
||||
- Confidence: 0.65
|
||||
|
||||
### Axis 8 — Governance, Decision-Making, and Communication
|
||||
- **Q1**: Who is the decision-maker when two parties disagree?
|
||||
- Evidence: Single owner; D-018/D-019 carry; escalation hooks codified (config.json).
|
||||
- Answer: Unambiguous; no dual-executive stall possible.
|
||||
- Confidence: 0.90
|
||||
- **Q2**: Governance cadence and escalation pattern — useful in a crisis?
|
||||
- Evidence: Per-phase ship + gates; the pipeline halted itself at the v0.2 gate correctly; deploy is owner-directive-only. **New v0.3 wrinkle**: P4's phase-ship DoD requires a host deployment that D-019 forbids without a directive — the first time a phase's success criteria are contingent on a mid-milestone owner action. T-4.9 acknowledges it but does not resolve the contingency.
|
||||
- Answer: The escalation pattern is crisis-proven, but this dependency is new and unresolved — if the owner declines the directive at P4, REQ-83..85 ship unproven-on-host and the milestone gate presents hosted requirements on faith. Escalated as G-111.
|
||||
- Confidence: 0.70
|
||||
- **Q3**: What is being omitted from status reports?
|
||||
- Evidence: `---ci---` honesty verified historically. The channel where optimism leaked this cycle is the research file: C3-11 asserts "verified states.py" for a claim that is false — asserted verification that was not re-run. This grill re-ran it.
|
||||
- Answer: Status reporting remains honest; asserted-but-unre-run verification is the leak channel, and the fix is cultural (G-101's test pin makes the claim checkable).
|
||||
- Confidence: 0.80
|
||||
- **Q4**: Is there a stop-the-project trigger?
|
||||
- Evidence: Owner rejection precedent (twice in v0.1); D-019 is the hosting-scope-specific stop trigger — no public cutover without directive; merge halts at the gate (D-018).
|
||||
- Answer: Yes, and the hosting design adds a second one: the staged/public split (G-107) means exposure is a discrete owner decision, not a gradual slide.
|
||||
- Confidence: 0.85
|
||||
|
||||
### Axis 9 — Change, Adoption, and Operational Readiness
|
||||
- **Q1**: Who will use this, how does their work change, and what's in it for them?
|
||||
- Evidence: Admin-managed accounts (REQ-70) — the owner creates the user set; work change = login + hosted real inference with per-user keys vs. local setup; UX Acceptance Criteria preserve the ≤3-click path with login as a once-per-session step.
|
||||
- Answer: Small, known, sponsor-adjacent user set; adoption politics minimal, value concrete.
|
||||
- Confidence: 0.70
|
||||
- **Q2**: Is ops involved now, or handed a finished product at go-live?
|
||||
- Evidence: The pipeline is ops; deploy.sh extends with paired rollback for both services (T-4.4); the startup sweep is the redeploy safety net (T-1.8); volume ownership handled (chown 10001, T-4.4). Gaps found: no disk assert/prune (G-106), undefined "staged" (G-107), KEK operational posture undocumented (G-105).
|
||||
- Answer: Ops is built-in; three hygiene gaps bound.
|
||||
- Confidence: 0.65
|
||||
- **Q3**: What is the rollback plan?
|
||||
- Evidence: Paired `-prev` containers for BOTH services (T-4.4); mock-dev default preserved at the code level (REQ-75/D-112); pre-hosted artifacts disposable (D-106); sessions durable across restarts (D-101).
|
||||
- Answer: Multi-layer and codified — the strongest single dimension of this plan.
|
||||
- Confidence: 0.80
|
||||
- **Q4**: Have success criteria been validated with the people who will judge success?
|
||||
- Evidence: ROADMAP v0.3 success criteria are concrete (401 curl, SSE unbuffered, two-user isolation, live-mode default on the staged instance); owner gate precedent enforces criteria. **Found**: REQ-78's own text promises "checkable follow-ups" while D-114/T-3.5 derive answered-iff-DONE over a GET-only route — the approved prototype's working checkbox (`library-view.tsx:463-471`) cannot hold live.
|
||||
- Answer: One criterion-vs-implementation mismatch found; bound as honest degradation (G-109) rather than silent reversion.
|
||||
- Confidence: 0.65
|
||||
- Decision: G-109 — live-mode task status is derived (open unless run DONE); the inbox checkbox becomes display-honest in live mode (disabled/annotated; mock keeps the free toggle for dev); persisted per-question check state is v0.4+ scope (REQ-45 backlog); pinned by a live-mode component test.
|
||||
|
||||
### Meta — Closing Review
|
||||
- **Q1**: If you were the auditor, what would you flag?
|
||||
- Evidence: The false "verified" claim (C3-11 sweep legality vs `states.py` PLANNING); the REQ-75 triple mapping; the two stale out-of-scope rows contradicting the same-file supersession; C3-19 vs PLAN pitfall-phase mapping.
|
||||
- Answer: Flag #1 is the sweep bug — a restart-path correctness claim that would have detonated at the first hosted redeploy, the exact moment the milestone's value proposition is under evaluation. Flags #2-4 are the recurring phase-mapping/bookkeeping class (G-001's relatives), now bound (G-102/G-103).
|
||||
- Confidence: 0.80
|
||||
- **Q2**: What is the project not doing that it should?
|
||||
- Evidence: Found across axes: exposure hardening (password policy, login rate limit — G-104); KEK-loss posture (G-105); deploy disk hygiene (G-106); staged-deploy definition (G-107); P3 pressure valve (G-108); task-toggle honesty (G-109); cost posture statement (G-110).
|
||||
- Answer: Seven omissions, all bound. Nothing found suggests the project is doing something it should not — engine-contract preservation holds everywhere checked.
|
||||
- Confidence: 0.75
|
||||
- **Q3**: What is the simplest version that delivers 80% of the value?
|
||||
- Evidence: The owner explicitly picked "one milestone"; deferring live surfaces would re-create the G-009 mock-bound state the owner just moved past — the lever is not deferral.
|
||||
- Answer: The pre-agreed lever is the P3 split (G-108), decided now while it is cheap — not surface deferral, which would contradict the owner's directive.
|
||||
- Confidence: 0.65
|
||||
- **Q4**: What must be true for this to succeed in the next 90 days, and is it true today?
|
||||
- Evidence: (a) D-101..D-115 workable against the tree — re-verified (11/12 sampled exact, 1 false and bound). (b) Owner engagement — evidenced same-day. (c) The KEK provisioned by P2 — trivial owner action, posture bound (G-105). (d) The P4 staged host deployment authorized — currently NOT true; D-019 requires a directive the plan does not pre-arrange (G-111).
|
||||
- Answer: (a)(b)(c) true or evidenced; (d) becomes true when the owner records the pre-authorization. That escalation is the gating action of this grill.
|
||||
- Confidence: 0.70
|
||||
|
||||
### Binding Decisions
|
||||
| ID | Axis | Decision | Confidence |
|
||||
|----|------|----------|-----------|
|
||||
| G-101 | 3/7 | Startup sweep must handle PLANNING-state runs: RESEARCH §7/C3-11's "CANCELLED legal from every non-terminal state" is false (`states.py`: PLANNING → {AWAITING_APPROVAL, FAILED}). Add PLANNING→CANCELLED to `_LEGAL_TRANSITIONS` or document the sweep as an explicit transition-validation bypass; pin with a PLANNING-fixture→CANCELLED sweep test in T-1.8. | 0.90 |
|
||||
| G-102 | 2 | Fix the stale out-of-scope rows contradicting the recorded supersession: REQUIREMENTS.md Out-of-Scope ("Auth / multi-tenancy") and PROJECT.md Out of Scope ("multi-tenant/auth") — replaced with the honest v0.3 posture (auth/multi-user in scope; open self-serve registration and OAuth out; admin-managed accounts only, OAuth v0.4+). | 0.95 |
|
||||
| G-103 | 2/5 | Phase-mapping integrity (G-001 precedent): bind REQ-75's split — delegation-end half closes in P3 (pinned by the standalone-client/no-mock-delegation tests), hosted-live-default half in P4 (pinned by the live-mode smoke marker); fix the traceability row (REQ-75..82 → REQ-76..82 + REQ-75 split note); declare PLAN's pitfall-closure mapping authoritative over C3-19's compressed phase prose. | 0.85 |
|
||||
| G-104 | 2/3 | Public-exposure hardening, budgeted honestly: (a) minimum password length ≥12 enforced at register + admin-create (P1 T-1.4/T-1.5, uniform error messages preserved); (b) per-IP login rate limit at the HAProxy front (stick-table on /api/auth/*, P4 T-4.5) bounding argon2 CPU burn on the single worker; (c) the T-4.8 exposure audit asserts the throttle and confirms uniform 401s (no user enumeration). | 0.80 |
|
||||
| G-105 | 6/7 | KEK-loss posture stated honestly (PLAN T-2.2 + owner docs): RESEARCHX_KEK's only copy is the env-file (never git); loss = per-user encrypted keys unrecoverable (AES-256-GCM + AAD), chain degrades to server/keyless honestly, users re-enter keys under a re-provisioned KEK; the owner MUST back up the KEK off-repo; rotation/re-wrap tooling stays out of v0.3 (recorded, not silently dropped). | 0.85 |
|
||||
| G-106 | 6 | deploy.sh two-image hygiene: pre-build free-disk assertion, post-swap dangling-image prune, and a documented host budget (2 image families × live+prev, 4 containers) — the current script has neither, and P4 doubles the footprint on a single LXC host. | 0.75 |
|
||||
| G-107 | 5/9 | "Staged deploy" defined: candidate pair (researchx + researchx-api) built + run on the LXC host, health checks green, the four smoke assertions executed host-locally against the candidates; public TLS-front cutover remains a discrete owner-directive act (D-019) with post-cutover public assertions (public 401, SSE through proxy) run with the -prev pair armed. | 0.75 |
|
||||
| G-108 | 5 | G-006 precedent applied to P3: may ship as two patches (P3a backend artifacts + read-models + routes + read-model suite; P3b frontend standalone client + surface swaps + Reading Room fidelity + delegation-end) or run mid-phase wave gates, without scope change; tags extend on the v0.3.x line; the milestone release remains the final phase's patch. | 0.70 |
|
||||
| G-109 | 9 | Task-inbox honesty (G-002 lesson): live-mode status is derived (answered-iff run DONE, D-114); the inbox checkbox (`library-view.tsx:463-471`) becomes display-honest in live mode — disabled/annotated, never a silent reversion; mock keeps the free toggle for dev; persisted per-question check state recorded as v0.4+ (REQ-45 backlog); pinned by a live-mode component test in T-3.12. | 0.75 |
|
||||
| G-110 | 1/6 | Cost posture stated in PLAN/PROJECT: server-tier runs bill the owner's OLLAMA_API_KEY; the bound is the admin-managed account set (registration closed, REQ-70); per-user counters (P-13, T-2.6) surface server-tier usage per user for owner audit; no hard spend cap in v0.3 — hard caps are v0.4+ scope if the user set grows; keyless localhost remains the zero-cost path. | 0.70 |
|
||||
|
||||
### Escalations
|
||||
- [G-111] Axis 5/8 — Owner pre-authorization required before P4 executes: P4's own definition of done ("staged smoke suite passes all four assertions", ROADMAP/PLAN T-4.7/T-4.9) requires deploying the candidate container pair to the LXC host — an action D-019 forbids without an owner directive. Without pre-authorization, P4 ships with REQ-83..85 unproven on the real host (the HAProxy 2.8 live config is unread until then) and the milestone gate would present hosting requirements on faith. The owner must either (a) pre-authorize staged host deployments for P4 DoD purposes (candidate pair + host-local smoke, NO public cutover — cutover stays a separate owner directive per G-107), or (b) accept that P4's phase ship carries hosting proven only on a local docker pair, with the host proofs deferred to the gate. This is the first phase in the project whose success criteria are contingent on a mid-milestone owner action; it must not be discovered at P4 ship time. Engineering-side handling is bound in G-107; the authorization itself is unresolved until the owner records it.
|
||||
+36
-26
@@ -1,46 +1,56 @@
|
||||
# Personas — Milestone v0.1 (Prototype)
|
||||
# Personas — Milestone v0.3 (Hosted)
|
||||
|
||||
---
|
||||
active:
|
||||
- name: backend-engineer
|
||||
- name: data-engineer
|
||||
- name: frontend-engineer
|
||||
- name: ci-lead-developer
|
||||
phase_specific: []
|
||||
reason: "v0.1 is a pure frontend prototype (Next.js, mock data). Only the frontend domain has work; backend/data personas activate in v0.2. Lead-developer coordinates decomposition and enforces the ApiClient boundary."
|
||||
phase_specific:
|
||||
- name: security-engineer
|
||||
phases: [1, 2, 4]
|
||||
reason: "v0.3 is the hosted multi-user milestone: P1 lands auth/sessions/CSRF/user-scoping, P2 lands the encrypted key store + credential chain, and P4 exposes the whole surface through a public proxy — security review is needed at all three, but not at P3 (live surfaces are read-model + binding work with no new secret handling) or P5 (the review phase runs the full multi-persona pass anyway)."
|
||||
---
|
||||
|
||||
## Roster
|
||||
|
||||
### backend-engineer
|
||||
- **Domain**: backend
|
||||
- **Active**: true
|
||||
- **Phase-specific**: false
|
||||
- **Frameworks**: fastapi, pydantic v2, argon2-cffi, cryptography, uv, uvicorn, sse-starlette, httpx, pytest, pytest-asyncio
|
||||
- **Constraints**: single uvicorn worker mandatory (in-memory registry + session store); all external I/O lives in adapters — engine imports no vendor SDKs; no blocking calls in async paths (argon2/AES via `asyncio.to_thread`); durable JSON via atomic tmp+os.replace; auth guard is router-dependency-based (`Depends(require_user)`), never BaseHTTPMiddleware (SSE safety); RunRegistry owner-keyed `(owner_id, run_id)` + pruned; engine contracts unchanged (per-run adapter injection); `create_app(auth_enabled=True)` default with parameter-only dev override — never an env var readable in production images
|
||||
- **Territory**: `backend/**`
|
||||
|
||||
### data-engineer
|
||||
- **Domain**: data
|
||||
- **Active**: true
|
||||
- **Phase-specific**: false
|
||||
- **Frameworks**: pydantic v2 (schemas), pytest
|
||||
- **Constraints**: owns the new pydantic schema contracts: User/Session, per-user Settings, key-store envelope file, `RunState.owner_id` (serialization-compatible), report.json/evidence.json (marker normalization `p-N`→`pN`, frontend Report/Citation compatible), read-model payloads (library/tasks/suggestions/graph/profile); pure data + validation — no I/O in schema modules; every run persists resolved dial values (REQ-04 carries)
|
||||
- **Territory**: `backend/src/researchx/config/**`, `backend/src/researchx/evidence/**` (+ schema review on `backend/src/researchx/auth/**` models)
|
||||
|
||||
### frontend-engineer
|
||||
- **Domain**: frontend
|
||||
- **Active**: true
|
||||
- **Phase-specific**: false
|
||||
- **Frameworks**: next.js (App Router), react 19, typescript (strict), tailwind, vitest
|
||||
- **Constraints**: component-first; server components by default, client components only where interactivity demands; all data access through the ApiClient interface — components never import MockApiClient directly; no secrets or env keys in the frontend; typography care per vision §4
|
||||
- **Frameworks**: next.js (App Router), react 19, typescript (strict), tailwind, vitest, openapi-fetch, eventsource
|
||||
- **Constraints**: components never import the mock or generated client directly — only the ApiClient boundary does; PLANNED interface additions (login/logout/getSession, admin hooks) documented before P1; fetch `credentials:"include"` + `X-ResearchX-Request` on all mutations; EventSource `withCredentials: true`; build-time NEXT_PUBLIC_* inlining profiles (mock dev default / live hosted with empty same-origin base); zero component rewrites in live swaps (REQ-54); no secrets or env keys in the frontend
|
||||
- **Territory**: `frontend/**`
|
||||
|
||||
### security-engineer (phase-specific: P1, P2, P4)
|
||||
- **Domain**: security
|
||||
- **Active**: true while P1/P2/P4 execute; deactivated between and after P4
|
||||
- **Phase-specific**: true
|
||||
- **Frameworks**: argon2-cffi, cryptography (AESGCM), httpx, pytest, STRIDE analysis
|
||||
- **Constraints**: KEK from env-file 'keys' scope only — never shell env, never logged, never in git, never in API responses (masked status only); AAD binds ciphertext to user_id; opaque tokens 32-byte urlsafe; SameSite=Lax + custom-header CSRF verified end-to-end; cross-user 404s provable by test; SSRF guard carries to any new fetch path; hosted exposure audit (401 on public /api, /healthz the only unauthenticated route); no `auth_enabled` env-var backdoor in production images
|
||||
- **Territory**: `backend/src/researchx/auth/**` (P1/P2), `.ciagent/release/**` + hosting config review (P4)
|
||||
- **Removal**: deactivate after P4 ships
|
||||
|
||||
### ci-lead-developer
|
||||
- **Domain**: coordination
|
||||
- **Active**: true
|
||||
- **Phase-specific**: false
|
||||
- **Frameworks**: ciagent workflows, git discipline
|
||||
- **Constraints**: enforce persona territories; ApiClient boundary review; commit discipline with ---ci--- blocks; phase-boundary file updates only (no .ciagent/ writes during task execution)
|
||||
- **Territory**: `.ciagent/**`, repo root config files
|
||||
|
||||
### Deactivated (v0.1)
|
||||
|
||||
#### backend-engineer
|
||||
- **Domain**: backend
|
||||
- **Active**: false
|
||||
- **Reason**: No backend in the prototype milestone; the FastAPI backend is milestone v0.2 (blocked until prototype approval)
|
||||
- **Frameworks**: fastapi, pydantic, uv (activates in v0.2)
|
||||
- **Territory**: `backend/**` (does not exist yet)
|
||||
|
||||
#### data-engineer
|
||||
- **Domain**: data
|
||||
- **Active**: false
|
||||
- **Reason**: No database/schema work in v0.1; evidence store and citation ledger models arrive in v0.2; prototype uses typed fixtures
|
||||
- **Frameworks**: pydantic schemas (activates in v0.2)
|
||||
- **Territory**: `backend/src/researchx/evidence/**` (does not exist yet)
|
||||
|
||||
### Custom personas
|
||||
None required for v0.1. The security-auditor persona activates in v0.2 phase 2 (trust loop: secrets handling, liveness verification). No CLI surface exists in this milestone, so no cli-engineer.
|
||||
- **Constraints**: enforce persona territories; adapter/engine boundary review (I/O only in adapters); commit discipline with ---ci--- blocks; phase-boundary file updates only (no .ciagent/ writes during task execution); gates green before every ship; deploy.sh two-service extension (build both images, paired rollback containers, health on /healthz + /ask, live-mode build args asserted in the smoke test); secrets only from .ciagent/.env.secrets; no deploys without owner directive (D-019) and merge-to-main halts at the owner gate (D-018)
|
||||
- **Territory**: `.ciagent/**`, repo root config files (README, VERSIONING, .gitignore, Dockerfiles/deploy scripts)
|
||||
+244
-35
@@ -1,54 +1,263 @@
|
||||
# PLAN — Milestone v0.1 Phase 8 (FINAL): Review + Approval Re-presentation + Ship
|
||||
# PLAN — Milestone v0.3 (Hosted): Working Plan for Phases 1–4
|
||||
|
||||
---
|
||||
phase: 8
|
||||
milestone: v0.1
|
||||
status: executed
|
||||
phase: 0
|
||||
milestone: v0.3
|
||||
status: planned
|
||||
must_haves:
|
||||
- "Full-milestone 2nd final review: PASS-WITH-FIXES — double-compose race + dead drawer-keyboard P1s fixed with regression tests (90/90); P6/P7 fixes verified holding"
|
||||
- "Audit: PASS-WITH-NOTES — reconstruction, tags, file/branch/commit discipline, secrets sweep green; notes applied (ARCHITECTURE §7 + Build Order, root README /ask row, VERSIONING table, About string, package.json version)"
|
||||
- "Final phase patch v0.1.8 shipped — milestone release candidate, presented at the REQ-62 owner approval gate with UAT staged per owner cadence"
|
||||
- "Engine contracts unchanged: all v0.3 work lands outside the engine — per-run adapter injection, owner-scoped storage constructors, owner_id stamping via make_run_config; no methodology/engine protocol rewrites"
|
||||
- "No secret values in git, logs, traces, or API responses: model keys masked (last 4 chars) at the API boundary and encrypted at rest (AES-256-GCM, KEK from the env-file 'keys' scope), passwords argon2id-hashed and never emitted"
|
||||
- "User scoping provable end to end: runs/<user_id>/<run_id>/ partitioning, owner-keyed RunRegistry, disk read-models — cross-user access 404s, pinned by test in every phase that touches the data flow"
|
||||
- "Every phase ships a patch (v0.3.1..v0.3.4; P5 tags v0.3.5 = the milestone release); gates green before every ship; pipeline halts at the merge_to_main owner approval gate (D-018); no deploys without owner directive (D-019)"
|
||||
- "openapi.json snapshot committed and the generated frontend client regenerated from the snapshot (never a live server) in every phase that touches routes; schema-drift check wired into the gates (G-005)"
|
||||
- "Zero-key test discipline: every CI suite runs on REQ-31 fakes with auth via create_app(auth_enabled=False) or dependency_overrides; keyed/live tiers are opt-in, skipped by default"
|
||||
- "Single uvicorn worker in every serving mode — the in-memory RunRegistry and session store are single-process by design"
|
||||
- "Mock demoted to dev/test fixture: the code default of RESEARCHX_API_MODE stays mock; after P3 no live-mode surface delegates to MockApiClient; hosted builds inline NEXT_PUBLIC_RESEARCHX_API_MODE=live (REQ-75)"
|
||||
- "G-101: startup sweep handles PLANNING-state runs — add PLANNING→CANCELLED to _LEGAL_TRANSITIONS (or document the sweep as an explicit validation bypass); pinned by a PLANNING-fixture→CANCELLED sweep test in T-1.8"
|
||||
- "G-104: public-exposure hardening — password minimum length ≥12 at register + admin-create (T-1.4/T-1.5); per-IP login rate limit at the HAProxy front (stick-table on /api/auth/*, T-4.5); exposure audit asserts the throttle + uniform 401s (T-4.8)"
|
||||
- "G-105: KEK-loss posture documented (T-2.2 + owner docs): loss = per-user keys unrecoverable, honest degradation to server/keyless, re-entry under a re-provisioned KEK; owner backs up the KEK off-repo; rotation tooling out of v0.3 (recorded)"
|
||||
- "G-106: deploy.sh hygiene — pre-build free-disk assertion + post-swap dangling-image prune + documented host budget (2 image families × live+prev)"
|
||||
- "G-107: 'staged deploy' = candidate pair built + run on the LXC host, health green, four smoke assertions host-locally; public TLS-front cutover stays a discrete owner-directive act with -prev armed"
|
||||
- "G-108: P3 pressure valve (G-006 precedent) — may ship as P3a (backend artifacts + read-models + routes + suite) and P3b (standalone client + swaps + fidelity + delegation-end), tags extending on the v0.3.x line, without scope change"
|
||||
- "G-109: task-inbox honesty — live-mode status is derived (answered-iff run DONE); the checkbox is display-honest in live mode (disabled/annotated; mock keeps the free toggle); persisted check state is v0.4+ (REQ-45 backlog); pinned by a live-mode component test in T-3.12"
|
||||
- "G-110: cost posture stated — server-tier runs bill the owner's OLLAMA_API_KEY; the bound is the admin-managed account set; per-user counters surface server-tier usage; no hard spend cap in v0.3 (hard caps v0.4+); keyless localhost remains the zero-cost path"
|
||||
personas:
|
||||
- backend-engineer
|
||||
- data-engineer
|
||||
- frontend-engineer
|
||||
- "security-engineer (phase-specific P1/P2/P4)"
|
||||
- ci-lead-developer
|
||||
requirements: [REQ-62]
|
||||
requirements: [REQ-69, REQ-70, REQ-71, REQ-72, REQ-73, REQ-74, REQ-75, REQ-76, REQ-77, REQ-78, REQ-79, REQ-80, REQ-81, REQ-82, REQ-83, REQ-84, REQ-85]
|
||||
---
|
||||
|
||||
## User-Facing Surface
|
||||
|
||||
The extended prototype presented for owner approval: everything from v0.1.6 (shell, library, settings, avatar) plus the v0.1.7 ask-intelligence layer — open-question suggestions and the refine dialogue on the New Research view. Releases v0.1.0–v0.1.8; v0.1.8 is the extended-prototype release candidate.
|
||||
The hosted multi-user product: the full approved frontend — a login screen; Ask → Choose → Plan & Run → Reading Room; the app shell with research library, project graph, task inbox, follow-up suggestions, refine dialogue, settings with masked model-key entry and resolved credential tier, avatar user menu with real profile — now live-bound end to end against the FastAPI backend with **zero live-mode delegation to the mock** (D-112), user-scoped (`runs/<user_id>/<run_id>/`), with per-user encrypted model keys resolving user → server → keyless per run, and deployed as a two-service stack (uvicorn single worker + Next.js live build) behind an SSE-safe HAProxy with TLS — real inference as the product default. Releases v0.3.0–v0.3.5.
|
||||
|
||||
## Happy Path (owner review script)
|
||||
|
||||
1. `cd frontend && pnpm install && pnpm dev` → http://localhost:3000 → shell renders on /ask
|
||||
2. Ask → question → Research this (click 1)
|
||||
3. Choose → Deep Report card → Start research (click 2)
|
||||
4. Plan & Run → Approve plan (click 3) → live trace → pause/resume/cancel
|
||||
5. Read the report → hover [n] → exact quote; ledger; export Markdown/JSON
|
||||
6. New research → click an open-question suggestion → box fills (no auto-start) → edit → Research this
|
||||
7. New research → type a draft → Refine question → answer the 3 clarifying questions (skip the optional two) → Use this → question + context filled → Research this
|
||||
8. Library → folder tree → stored report opens its own report; Connections panel; ⌘K palette
|
||||
9. Avatar → user menu → Settings → API keys masked, methodology defaults live preview
|
||||
10. Owner decision: approve → v0.1.8 published + milestone merge to main; or request changes
|
||||
|
||||
## What this phase did
|
||||
|
||||
1. **Review** (multi-persona, P7-weighted, probe-verified): verdict PASS-WITH-FIXES. P1s: Enter at the last exchange question composed twice (stale section-level keydown guard re-ran submitAnswer after the input guard advanced — invisible with the deterministic mock, a doubled paid POST the day v0.2 wires the endpoint) and Enter on focused drawer buttons was dead (section preventDefault swallowed the click). Fixed: single Enter path (input guard owns Enter; buttons self-activate); error state gained a Keep original button (mouse-user trap). 2 regression tests (single-compose spy, Enter-on-button); suite 88 → 90. Adversarial probe: no drawer control can submit the outer form. P6/P7 fixes verified holding.
|
||||
2. **Audit**: PASS-WITH-NOTES — tags v0.1.0–v0.1.7 verified (annotated, on-branch, mapped to ship records); DEPLOY-RECORD.md tracked + accurate; branch hygiene exact; commit discipline clean (---/ci--- closer holding since the P6 regression); secrets clean across history; REQ-67/68 spot-checked as genuinely delivered; gates re-run live (90/90). Notes applied: ARCHITECTURE.md §7 + Build Order extended with the ask-intelligence layer; root README /ask row updated; VERSIONING table → v0.1.8; settings About + package.json → v0.1.8.
|
||||
3. **Ship (this commit sequence)**: PR #8 (phase/08 → milestone), squash-merge, tag v0.1.8, release with full milestone summary, v0.1.8-uat staged on the LXC host per owner cadence. Per REQ-62 and the prototype_approval escalation hook, the milestone merge to main does NOT proceed autonomously — the pipeline halts at the gate for the recorded owner decision.
|
||||
1. `cd backend && uv sync && uv run pytest` — all suites green with zero API keys: auth suite (`test_auth.py`), key-store tamper tests, read-model tests, restart-sweep test, engine/trust/adversarial suites under the `auth_enabled=False` harness; `uv run pytest -m live` skips cleanly.
|
||||
2. `cd frontend && corepack pnpm install && corepack pnpm test` — component + live-binding suites green; typecheck + lint green.
|
||||
3. Local live-mode full loop: `uv run uvicorn researchx.api.main:app --port 8000` (env-file with `RESEARCHX_KEK` + server model key) + `NEXT_PUBLIC_RESEARCHX_API_MODE=live NEXT_PUBLIC_API_BASE_URL=http://localhost:8000 corepack pnpm dev` → http://localhost:3000 → register (first user = admin) → login → Settings: enter model key (shows masked status only), config shows resolved tier → Ask a question → Research this → pick Deep Report → Start research → approve the plan (≤3 clicks on defaults) → live SSE trace → Reading Room: hover a citation shows the stored quoted passage; ledger panel; export Markdown/JSON with real fields → Library shows the run in its auto-categorized folder; graph/tasks/suggestions surface the run's open questions; refine dialogue runs on the real planner.
|
||||
4. Multi-user isolation: admin creates a second user; that user sees zero of the first user's runs, graph, tasks, or suggestions; the first user's key never affects the second user's resolved tier.
|
||||
5. Hosted: `deploy.sh` → both services up + healthy; `curl https://<origin>/api/runs` → 401 unauthenticated; `curl https://<origin>/healthz` → 200; login through the public URL; SSE trace streams unbuffered through the proxy; per-user key honored end to end (public deployment per owner directive, D-019).
|
||||
6. Owner decision at the P5 gate: approve → v0.3.5 = milestone release, merge to main; or direct changes → binding fix cycle.
|
||||
|
||||
## UX Acceptance Criteria
|
||||
|
||||
- Owner demo script runs clean on the shipped code (probe- and test-verified)
|
||||
- Suggestions + refine never auto-submit; drafts stay untouched on dismiss
|
||||
- The 3-click criterion still pinned green on AskView (affordances opt-in)
|
||||
- Honest captions everywhere; keyboard accessible throughout
|
||||
- The ≤3-click research path (Ask → Choose → Run on defaults) is preserved for an authenticated user; login is a once-per-session step (14-day sliding) adding zero clicks to the path.
|
||||
- Reading Room parity with the approved mock: page-anchored citations whose hover shows the exact stored quoted passage; a real ledger panel; exports with real entry data — no fabricated fields anywhere.
|
||||
- Honesty in live mode: resolved credential tier visible (user|server|keyless), key-store-disabled stated honestly, no synthetic suggestions, honest empty states for fresh users.
|
||||
- Suggestions click-fills-the-box (never auto-starts); refine is opt-in and model-driven.
|
||||
- Keys masked everywhere: no secret value is ever rendered, logged, or exported.
|
||||
- Cross-user isolation is total and invisible: no surface leaks another user's runs, questions, or key status.
|
||||
|
||||
## Verification
|
||||
## Planned ApiClient Extensions (C3-18 — documented before P1; this list is exhaustive)
|
||||
|
||||
- Gates: typecheck ✓ lint ✓ tests 90/90 ✓ build ✓ (re-run after review fixes, plus by audit)
|
||||
- Regression tests pin: single-compose, Enter-on-button, focus return, Enter-guard, no-auto-submit
|
||||
- Post-hoc v0.2 log: compose-endpoint idempotency key; freeze draft at drawer-open; confirm-overwrite when context non-empty; Try-again from error state; textarea answers; aria-live progress
|
||||
- `login(email, password): Promise<UserProfile>` — POST /api/auth/login (sets the session cookie)
|
||||
- `logout(): Promise<void>` — POST /api/auth/logout
|
||||
- `getSession(): Promise<UserProfile | null>` — GET /api/auth/me (401 → null)
|
||||
- Admin hooks (API-first in v0.3; no dedicated admin screen): `listUsers()`, `createUser(input)`, `deactivateUser(id)` — GET /api/admin/users, POST /api/admin/users, POST /api/admin/users/{id}/deactivate
|
||||
- MockApiClient gains matching no-op/local implementations (standalone mock preserved — REQ-31/REQ-54); live implements against the routes. The v0.2 pause/resume extensions (G-003) carry unchanged. No other interface changes are planned for v0.3.
|
||||
|
||||
## Scope Guard
|
||||
## Phase 1 — Auth + Sessions + User Scoping (ships v0.3.1)
|
||||
|
||||
Milestone merge to main, v0.1.8 publication, v0.2 unblocking, and all v0.2 P1 work wait on the owner approval gate (REQ-62, prototype_approval escalation hook). The pipeline does not proceed autonomously past this commit.
|
||||
**Personas** (lead-developer decomposition by territory): backend-engineer (lead — auth package, storage partitioning, engine/app wiring, sweep/pruning), data-engineer (auth + state pydantic contracts), security-engineer (phase-specific — auth adversarial suite + STRIDE), frontend-engineer (session surface, snapshot gate), ci-lead-developer (ship).
|
||||
|
||||
**Requirements covered**: REQ-69, REQ-70, REQ-71
|
||||
|
||||
**Task waves**:
|
||||
|
||||
*Wave 1 — contracts + stores (parallel)*
|
||||
- **T-1.1 (backend-engineer)**: Data root + user store — `researchx.config.paths.data_root()` (RESEARCHX_DATA_DIR → `backend/var/researchx` dev; D-107); `auth/users.py`: User model (user_id = 8-char slug, validation `^[a-z0-9]{8}$`, D-105; email; argon2id via argon2-cffi `PasswordHasher()` defaults, hash/verify/rehash all in `asyncio.to_thread`, D-102; display_name; role; active; created_at), `users.json` at the data root, atomic rewrite (D-017 idiom). Pyproject gains `argon2-cffi>=23.1,<26` (no python-multipart).
|
||||
- **T-1.2 (backend-engineer)**: Session store — `auth/sessions.py`: opaque `token_urlsafe(32)` tokens, in-process dict + durable `sessions.json` (atomic), `Session {token_hash, user_id, created_at, last_seen, expires_at}`, 14-day sliding expiry via `touch()` with >60s persist throttle, token rotation at login, immediate revocation (logout deletes the presented token; deactivation deletes all the user's tokens) (D-101).
|
||||
- **T-1.3 (data-engineer)**: Auth + state contracts — pydantic User/Session schemas, auth route request/response models, `RunState.owner_id: str | None = None` (serialization-compatible, no migration per D-106); old state files must deserialize.
|
||||
|
||||
*Wave 2 — enforcement (depends W1)*
|
||||
- **T-1.4 (backend-engineer)**: Guard + auth routes + CORS — `auth/guard.py` `require_user` dependency (session resolve → 401; `X-ResearchX-Request` header required on non-GET → 403; inactive user → 401; sliding touch); `auth/routes.py` register/login/logout/me with httpOnly `researchx_session` cookie (SameSite=Lax, path=/, Secure hosted-only flag; no Domain attribute); uniform 401 "invalid email or password"; register open only while the user store is empty — first user `role="admin"`, then 403 (no await between check and insert except the argon2 hash; single worker = race-free, D-104); public router (`/api/auth/*`, `/healthz` outside /api per D-103) + protected router (everything else) via `APIRouter(dependencies=[Depends(require_user)])` — **router dependencies, not middleware** (C3-01; no BaseHTTPMiddleware near SSE); CORS: `allow_credentials=True` + explicit localhost origins, ordered so preflight OPTIONS answers without 401 (C3-04); login/register POSTs also require the CSRF header (C3-05).
|
||||
- **T-1.5 (backend-engineer)**: Admin routes — POST /api/admin/users (admin-guarded create with admin-set initial password), GET /api/admin/users (list), deactivate (kills all sessions; data retained) — REQ-70.
|
||||
- **T-1.6 (backend-engineer)**: Storage partitioning — `LocalStorageAdapter(base_dir, owner_id)` with **required** owner_id (validated), `run_dir → base_dir/runs/<owner_id>/<run_id>/`, + `list_run_ids()` (read-models scan later); `StorageAdapter` protocol gains `append_run_artifact` (the actor already calls it); `read_run_artifact` error message embeds the owner path; consolidate the duplicate run-id generators (P-14). Protocol otherwise byte-for-byte unchanged (D-106).
|
||||
|
||||
*Wave 3 — engine/app wiring (depends W2)*
|
||||
- **T-1.7 (backend-engineer)**: Owner-scoped app wiring — `make_run_config(..., owner_id)` stamps `state.owner_id`; RunActor untouched (engine contracts unchanged); `RunRegistry` keyed `(owner_id, run_id)` — cross-owner access 404; routes construct the owner-scoped storage per request from the auth dependency; `create_app(*, auth_enabled=True)` default with **parameter-only** dev override resolving a synthetic dev user (`user_id="dev00000"`, role admin — C3-16, never an env var readable in production); `dev_bridge.py` stays explicitly `auth_enabled=False` (P-5); `engine/demo.py` owner arg (P-9); remove the dead duplicate `return EventSourceResponse` (app.py:331, P-11); mechanical `owner_id` constructor args across the 8 test files (RESEARCH §5 touchpoint list).
|
||||
- **T-1.8 (backend-engineer)**: Startup sweep + registry pruning — on `create_app` startup, rewrite every non-terminal `state.json` under the data root as CANCELLED (D-115; legal from every non-terminal state; in-flight actors die with the process); registry drops terminal `(actor, bus)` entries once no live SSE subscriber (refcount in the events route — P-6); events-404-after-prune/restart documented (trace persists on disk).
|
||||
|
||||
*Wave 4 — adversarial + frontend + snapshot (depends W3)*
|
||||
- **T-1.9 (security-engineer + backend-engineer)**: Auth suite — `backend/tests/test_auth.py` with `auth_enabled=True` (RESEARCH §11): first-user-admin (+403 register after), wrong-password 401 uniform, logout revocation, 401 on all protected routes, CSRF header rejection, cross-user 404 (all run endpoints), admin create/deactivate (deactivation kills sessions), session expiry, cookie flags, no password-hash material in any response; pins P-5 and the authorization half of P-8.
|
||||
- **T-1.10 (frontend-engineer)**: Session surface — ApiClient extensions per C3-18 (mock no-ops; live against routes); login screen (email + password, error states); live-mode route guard redirecting unauthenticated users to /login; user menu shows the real profile + logout; fetch discipline: `credentials:"include"` + `X-ResearchX-Request` header on every mutation (C3-05); dev docs pin same-hostname origins (`localhost` both sides — port is not part of same-site computation).
|
||||
- **T-1.11 (frontend-engineer)**: Snapshot regen gate — export + commit `backend/openapi.json` (auth/admin routes included), regenerate `frontend/src/lib/api/generated/` from the snapshot only, wire the schema-drift check into the gates, operation-ids stable (G-005).
|
||||
|
||||
*Wave 5 — ship*
|
||||
- **T-1.12 (ci-lead-developer)**: Phase ship — all gates green (backend ruff/mypy/pytest incl. `test_auth.py`; frontend lint/typecheck/vitest; drift gate), tag **v0.3.1** + release; REQUIREMENTS/ROADMAP traceability rows updated.
|
||||
|
||||
**Must-haves (verification-oriented)**:
|
||||
- Unauthenticated access to any `/api` route except register/login → 401; register 403s once the user store is non-empty — pinned by `test_auth.py`.
|
||||
- First registered user is admin; all subsequent accounts admin-created; deactivation kills sessions and retains data.
|
||||
- No user can read or act on another user's data: owner-keyed registry → cross-owner 404 on run GET/lifecycle/report/ledger/events — pinned by test.
|
||||
- Storage layout is `runs/<user_id>/<run_id>/`; `owner_id` recorded in every persisted run state.
|
||||
- Sessions are server-side with httpOnly SameSite=Lax cookies, immediate revocation, sliding expiry; CSRF header enforced on non-GET (EventSource GET exempt).
|
||||
- Engine contracts unchanged; existing zero-key suites green with only mechanical `owner_id` constructor changes.
|
||||
- `openapi.json` snapshot committed; generated client regenerated from it; drift gate fails on intentional drift (proof).
|
||||
- No run survives a restart in a non-terminal state (sweep test); terminal registry entries prune (leak test).
|
||||
- No password hash or key material in any API response or log.
|
||||
|
||||
**Pitfall closures (RESEARCH §12)**: **P-4** (CORS `allow_credentials` + ordering), **P-5** (dev_bridge explicitly auth-off, pinned by the suite), **P-6** (registry pruning), **P-7** (owner_id + partitioning), **P-9** (demo.py owner arg), **P-10** (layout test re-pinned to the partitioned path), **P-11** (dead code removed), **P-14** (run-id generators consolidated). P-8's authorization half (owner-keyed 404s) closes here; its registry-independence half closes in P3.
|
||||
|
||||
**Success criteria (testable)**:
|
||||
- `test_auth.py` green covering the full §11 list; two-user isolation test proves zero visibility across every run endpoint.
|
||||
- Register → admin; second register → 403; admin creates + deactivates (sessions die) — all route-tested.
|
||||
- Non-GET without the header → 403; with header + session → 2xx.
|
||||
- Disk layout test proves `runs/<owner>/<run_id>/`; restart-sweep test proves non-terminal → CANCELLED across an app restart; pruning test proves terminal entries drop with zero SSE subscribers.
|
||||
- Drift gate: intentional schema change fails the gate; regen green.
|
||||
- Gates green; patch v0.3.1 shipped.
|
||||
|
||||
## Phase 2 — Per-User Settings + Model Keys (ships v0.3.2)
|
||||
|
||||
**Personas**: backend-engineer (lead — key store, settings store, model factory, config rework, production entrypoint), data-engineer (settings + key-envelope schemas), security-engineer (phase-specific — key-store adversarial suite), frontend-engineer (settings live-binding, snapshot), ci-lead-developer (ship).
|
||||
|
||||
**Requirements covered**: REQ-72, REQ-73, REQ-74
|
||||
|
||||
**Task waves**:
|
||||
|
||||
*Wave 1 — contracts*
|
||||
- **T-2.1 (data-engineer)**: Settings + key-envelope schemas — per-user Settings payload (model names, methodology defaults + resolved dials, appearance slot, masked key status, storage path display-only — D-111); key-store envelope `{v, alg, updated_at, entries: {OLLAMA_API_KEY: {nonce, ct, updated_at}}}` (D-108/D-109); KEK decode rules (urlsafe-b64 or hex, exactly 32 bytes; decode/length errors are typed AdapterErrors).
|
||||
|
||||
*Wave 2 — stores (depends T-2.1)*
|
||||
- **T-2.2 (backend-engineer)**: Key store — `auth/key_store.py`: AES-256-GCM via `cryptography` AESGCM (pyproject gains `cryptography>=43`); KEK = `RESEARCHX_KEK` from the env-file **'keys'** scope (`config.json` secrets.scopes gained `"keys": ["RESEARCHX_KEK"]` in P0; provisioned into `.ciagent/.env.secrets` only — never git); AAD = `user_id` (a copied file under another user fails `InvalidTag`); fresh `os.urandom(12)` nonce per write (never reused under the KEK); `keys/<user_id>.json`, atomic, 0600; crypto in `asyncio.to_thread`; honest degradation: no KEK → disabled (PUT rejected with `{"key_store": "disabled"}`), corrupt entries surface as errors, never a silent tier drop (C3-06). KEK change requires process restart (documented).
|
||||
- **T-2.3 (backend-engineer)**: Settings persistence — `settings/<user_id>.json` store, atomic rewrite, defaults for fresh users, GET/PUT payload shapes (D-111).
|
||||
|
||||
*Wave 3 — routes + factory (depends W2)*
|
||||
- **T-2.4 (backend-engineer)**: Settings + key routes — GET/PUT `/api/config/settings`; model-key PUT accepts a raw key (scope = OLLAMA_API_KEY only, D-109) → encrypted at rest; every response returns configured + masked (last 4) only — the raw key never appears in any response, log, or trace (REQ-73).
|
||||
- **T-2.5 (backend-engineer)**: Per-run model factory — `adapters/model_factory.py`: `ModelAdapterFactory(secrets, key_store, defaults).resolve(user, settings) → (adapter, tier)` resolving user → server → keyless (REQ-74); `OllamaModelAdapter` constructor gains optional `api_key` (precedence over secrets resolve; secrets path back-compat); **one adapter instance per run** — the `_active_base` fallback mutation is per-instance, never shared (P-2 closure); `create_run` resolves + injects (engine contracts unchanged); state records `owner_id` + `credential_tier`.
|
||||
- **T-2.6 (backend-engineer)**: Config status rework — GET `/api/config` resolves the **current user's** tier via the factory without creating a run (kills the `runs._actors` private read — P-1 closure; no cross-user probe leak, no zero-run breakage); probe cached in-process keyed `(user_id, tier, key_fingerprint)` with short TTL, running against the resolved tier; per-user cost counters owner-keyed (P-13 closure; G-007 carries); `resolved_tier` + `key_store: enabled|disabled` surfaced.
|
||||
|
||||
*Wave 4 — production + adversarial + frontend (depends W3)*
|
||||
- **T-2.7 (backend-engineer)**: Production entrypoint — new `researchx.api.main` wiring `create_app(data_root=RESEARCHX_DATA_DIR, factory=<live factory from env-file secrets>, auth_enabled=True)` (P-3 backend closure — the `app.py:359` module global serving fakes + cwd storage leaves production); `app.py` stays the test factory; `main.py` never passes `auth_enabled=False`.
|
||||
- **T-2.8 (security-engineer + backend-engineer)**: Key-store adversarial suite — tamper tests: AAD binding (key file copied under another user_id → `InvalidTag` surfaced), nonce uniqueness across writes, masked-only responses (no raw key material in response/log/trace), disabled-honesty without KEK, corrupt-entry error surfacing; opt-in keyed tier (skipped by default; zero-key CI unchanged) proving a keyed user gets real inference end to end through the REQ-74 chain.
|
||||
- **T-2.9 (frontend-engineer)**: Settings live-binding — `GeneratedApiClient` getSettings/saveSettings → `/api/config/settings`; masked key entry (password-type input → PUT; masked status display; never echoed); getConnectionStatus maps GET `/api/config` (resolved tier user|server|keyless, key_store status, probe, per-user counters); storage directory display-only in live/hosted mode (D-111 — the free-path picker is the local single-user profile only); mock parity preserved standalone.
|
||||
- **T-2.10 (frontend-engineer)**: Snapshot regen — settings/key routes change `openapi.json` → snapshot committed, client regenerated, drift gate green (G-005).
|
||||
|
||||
*Wave 5 — ship*
|
||||
- **T-2.11 (ci-lead-developer)**: Phase ship — gates green; tag **v0.3.2** + release; traceability updated.
|
||||
|
||||
**Must-haves**:
|
||||
- Keys never appear in full in any API response, log, or trace — masked only, pinned by test.
|
||||
- Key store: AES-256-GCM, AAD=user_id, fresh nonce per write, 0600, atomic — tamper-proof pinned by test.
|
||||
- Without a KEK: key store disabled honestly; resolution proceeds server/keyless; never a silent tier drop.
|
||||
- Per-run credential chain user → server → keyless; probe runs against the resolved tier; config status surfaces the current user's tier without creating a run and without reading private registry state.
|
||||
- Shared-adapter fallback bug dead: one adapter per run; `_active_base` mutation can never cross users.
|
||||
- Cost counters owner-keyed.
|
||||
- Engine contracts unchanged; zero-key suites green (fakes, `auth_enabled=False`).
|
||||
- Production entrypoint live-wired; the module-global fake app serves nothing.
|
||||
|
||||
**Pitfall closures**: **P-1** (config_status → factory-resolved probe), **P-2** (per-run factory kills the shared `_active_base` mutation), **P-13** (owner-keyed cost counters), **P-3 backend half** (`researchx.api.main`; the hosting half closes in P4).
|
||||
|
||||
**Success criteria**:
|
||||
- Opt-in keyed tier: a keyed user gets fully real inference end to end; `resolved_tier="user"` in config status.
|
||||
- Zero-key CI: full suite green with no KEK configured — key store reports disabled, tier falls to server/keyless honestly.
|
||||
- Tamper suite green: AAD tamper → InvalidTag surfaced; nonces unique; masked-only responses.
|
||||
- Config status correct with zero runs and per-user isolated (no probe leak).
|
||||
- Drift gate green; gates green; patch v0.3.2 shipped.
|
||||
|
||||
## Phase 3 — Live Surfaces, user-scoped (ships v0.3.3)
|
||||
|
||||
**Personas**: backend-engineer (lead — artifacts, read-models, all new routes), data-engineer (artifact + read-model payload schemas), frontend-engineer (standalone client, all surface swaps, Reading Room fidelity), ci-lead-developer (ship). Security-engineer inactive this phase (no new secret handling — roster reason).
|
||||
|
||||
**Requirements covered**: REQ-76, REQ-77, REQ-78, REQ-79, REQ-80, REQ-81, REQ-82 (+ REQ-75 delegation-end half: after this phase no live-mode surface delegates to the mock)
|
||||
|
||||
**Task waves**:
|
||||
|
||||
*Wave 1 — artifacts + read-models (parallel)*
|
||||
- **T-3.1 (backend-engineer + data-engineer)**: report.json + evidence.json at DONE — persisted inside `_verify_phase` right after ledger/checklist (EvidenceStore is GC'd with the actor — this is the only write point; D-113): `report.json {v, run_id, owner_id, title, question, methodology_slug, generated_at, sections:[{heading, body}], citations:[{passage_id, url, title, quote, health, authority_tier}]}`, `evidence.json {v, run_id, passages:[{passage_id, url, title, quote, text}]}`; **marker normalization at the persistence boundary only**: section bodies `[p-N]` → `[^pN]`, ids `p-N` → `pN` (the Reading Room parses with zero frontend changes); `report.md` keeps raw `[p-N]` — trust-loop regexes untouched (C3-10).
|
||||
- **T-3.2 (backend-engineer + data-engineer)**: Read-models — `api/read_models.py` scans `runs/<owner_id>/` loading `state.json` (+ report.json/ledger.json where needed); skips dotfiles/tmp defensively (D-115); payload schemas shaped to the existing frontend types: library folders (auto-categorization by methodology + deterministic topic tokens from plan sub-questions) + ReportSummary, ReportConnection, ProjectGraph nodes/edges, InboxItem `{id: "<run_id>#<idx>", question, sourceRunId, status}`, SuggestedQuery.
|
||||
- **T-3.3 (backend-engineer)**: Enriched report endpoint — GET `/api/runs/{id}/report` → `{run_id, markdown, report, evidence}` (markdown retained, back-compat); **disk-read, registry-independent** (P-8 full closure); graceful markdown-only fallback for pre-artifact DONE runs (old artifacts disposable per D-106, but never a 500); ledger endpoint likewise disk-read.
|
||||
|
||||
*Wave 2 — routes (depends W1)*
|
||||
- **T-3.4 (backend-engineer)**: Library + connections + graph — GET `/api/library?sort=recent|title` (folders + ReportSummary[] resolving to real runs); GET `/api/library/connections` (shared-source edges ≥1 ledger-URL overlap + topical edges); GET `/api/graph` (same edges in ProjectGraph shape) — REQ-76/77, D-114.
|
||||
- **T-3.5 (backend-engineer)**: Tasks + suggestions — GET `/api/tasks` (open questions = plan sub-questions, answered-iff run reached DONE); GET `/api/suggestions` (open items joined with source run titles; no synthetic suggestions; empty state for fresh users) — REQ-78/79, D-114.
|
||||
- **T-3.6 (backend-engineer)**: Refine + profile — POST `/api/refine/questions {draft}` → 2–3 planner clarifying questions; POST `/api/refine/compose {draft, answers}` → composed refined question + suggested context; planner calls ride the **same per-user credential chain** (P2 factory), honest 503 for a keyless user with no local endpoint — never a hang; GET/PUT `/api/profile` (display_name, email; no deletion) — REQ-80/81. All owner-scoped via the auth dependency; non-GET ⇒ CSRF header applies.
|
||||
|
||||
*Wave 3 — frontend standalone (depends routes + snapshot)*
|
||||
- **T-3.7 (frontend-engineer)**: Snapshot regen — all new routes into the committed `openapi.json` snapshot; client regenerated; drift gate green (G-005).
|
||||
- **T-3.8 (frontend-engineer)**: GeneratedApiClient standalone — every ApiClient method overridden against real endpoints; `extends MockApiClient` **dropped** (a standalone class implementing ApiClient — D-112); request wrapper: `credentials:"include"` + `X-ResearchX-Request` on every mutation; `EventSource` constructed with `{withCredentials: true}`; mock remains a standalone dev/test fixture.
|
||||
- **T-3.9 (frontend-engineer)**: Surface swaps — getReportLibrary, getReportConnections, getProjectGraph, getTaskInbox, getSuggestedQueries, getRefinementQuestions, composeRefinedQuestion, getUserProfile, plus the P2 settings/config methods — all live, zero component rewrites (REQ-54); suggestions click-fills never auto-starts; the refine drawer drives the real exchange.
|
||||
- **T-3.10 (frontend-engineer)**: Reading Room fidelity + honest exports — getReport maps report.json → Report (sections + citations with **real quotes**; tier map official→1, reference→2, else 3); getLedger real entries; exportReport built from real entry data (kills the fabricated `claims_supported: 1` and empty quotes — P-12 closure, REQ-82); hover shows the stored quoted passage — parity with the mock Reading Room v0 (REQ-82); exports round-trip through the backend-schema gate (REQ-34 carries) with citation links intact.
|
||||
|
||||
*Wave 4 — tests + ship (depends W3)*
|
||||
- **T-3.11 (backend-engineer + data-engineer)**: Read-model test suite — `backend/tests/test_read_models.py`: a fixture disk tree (two users; DONE, CANCELLED, in-flight, FAILED runs) asserting library folder categorization + recency/title sorting; shared-source edges (≥1 ledger URL overlap) + topical edges; graph node/edge shape; tasks answered-iff DONE; suggestions = open items joined with run titles, empty for a fresh user, never synthetic; dotfile/tmp skip; **registry-independence test**: all read endpoints correct against a fresh `create_app` over an existing data root with zero in-memory actors (P-8 full-closure proof); artifact tests: report.json/evidence.json written at DONE with normalized markers (`[^pN]` in report.json, raw `[p-N]` in report.md), markdown-only fallback for pre-artifact DONE runs (never a 500).
|
||||
- **T-3.12 (frontend-engineer)**: Live-surface regression suite — component tests run unchanged against the live binding (zero component rewrites proven, REQ-54); ≤3-click path pinned green in live mode; suggestions click-fills-the-box never auto-starts; refine exchange drives the real endpoint on fakes; drift gate green.
|
||||
- **T-3.13 (ci-lead-developer)**: Phase ship — all gates green; tag **v0.3.3** + release; traceability rows updated.
|
||||
|
||||
**Must-haves**:
|
||||
- Every prototype surface (library, connections, graph, tasks, suggestions, refine, profile, settings, Reading Room) works against real user data in live mode — `GeneratedApiClient` is standalone, `extends MockApiClient` is gone (D-112), and no live-mode method delegates to the mock.
|
||||
- Read-models read disk artifacts only, never RunRegistry: every read endpoint is correct after a restart with zero in-memory state; non-terminal runs post-sweep appear as CANCELLED, terminal runs stay readable.
|
||||
- Reading Room parity with the mock: structured sections, page-anchored citations whose hover shows the stored quoted passage, ledger panel from real ledger.json; exports carry real quotes/claims/health/tier — no fabricated fields.
|
||||
- Engine persists report.json + evidence.json at DONE inside `_verify_phase`; `report.md` marker syntax untouched (trust-loop regexes unchanged).
|
||||
- All new routes owner-scoped via the auth dependency — cross-user 404s pinned; non-GET routes CSRF-header-guarded (refine POSTs included).
|
||||
- ≤3-click path green; suggestions never auto-start; honest empty states; no synthetic suggestions.
|
||||
- Snapshot committed, client regenerated from the snapshot only, drift gate green (G-005).
|
||||
- Zero-key discipline intact: refine tests on REQ-31 fakes; keyless refine surfaces an honest 503, never a hang.
|
||||
|
||||
**Pitfall closures**: **P-8 full closure** (registry-independent disk reads — auth half closed in P1, read half here, pinned by the restart test), **P-12** (export fabrication killed — real quotes/claims from report.json/evidence.json). Marker-normalization hazard (§6) closed by construction: normalization at the persistence boundary only, `report.md` raw form pinned by test.
|
||||
|
||||
**Success criteria**:
|
||||
- Read-model suite green over the multi-user fixture tree; fresh-app-over-existing-data-root test proves every read endpoint without the registry.
|
||||
- Live mode: all surfaces live-bound, component tests unchanged, 3-click green, no delegation to mock anywhere in the live client.
|
||||
- Export JSON payloads contain real quotes, real claims_supported, real health/tier — the fabricated `claims_supported: 1` / empty quotes are gone and a test pins their absence.
|
||||
- Fresh user: empty library, empty tasks, empty suggestions — honest, no synthetic content.
|
||||
- Drift gate green; gates green; patch v0.3.3 shipped.
|
||||
|
||||
## Phase 4 — Hosting + Real-Inference Default (ships v0.3.4)
|
||||
|
||||
**Personas**: ci-lead-developer (lead — deploy.sh, HAProxy config, hosting per its repo-root territory), backend-engineer (Dockerfile, production wiring completion, health endpoint), frontend-engineer (live build profile, build args), security-engineer (phase-specific — hosted exposure audit through the public URL).
|
||||
|
||||
**Requirements covered**: REQ-75, REQ-83, REQ-84, REQ-85
|
||||
|
||||
**Task waves**:
|
||||
|
||||
*Wave 1 — container + production wiring*
|
||||
- **T-4.1 (backend-engineer)**: Backend Dockerfile — `python:3.12-slim` + uv from the ghcr image; `uv sync --frozen --no-dev`; `useradd --system --uid 10001 researchx`; `mkdir -p /var/lib/researchx` + chown; `USER researchx`; `CMD ["uvicorn", "researchx.api.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1", "--proxy-headers"]` (REQ-83, C3-13). Secrets are runtime-only via `--env-file` (`RESEARCHX_KEK`, `OLLAMA_API_KEY`, `OLLAMA_BASE_URL`, `TAVILY_API_KEY`) — never baked into the image, never in git.
|
||||
- **T-4.2 (backend-engineer)**: Production wiring completion — `researchx.api.main` verified end to end: `data_root=RESEARCHX_DATA_DIR` (container default `/var/lib/researchx`), live model factory from env-file secrets, `auth_enabled=True` hardwired — **P-3 full closure**: no production path can serve FakeModelAdapter or cwd-based storage; parameter-only auth override confirmed absent from the image (no `RESEARCHX_AUTH`-style env readable anywhere); uvicorn graceful-shutdown behavior documented (in-flight runs exercise `_persist_cancelled` on SIGTERM; `docker rm -f` SIGKILLs — the P1 startup sweep is the safety net).
|
||||
|
||||
*Wave 2 — frontend live build profile*
|
||||
- **T-4.3 (frontend-engineer)**: Live build profile — frontend Dockerfile build stage gains `ARG`/`ENV` before `pnpm build`: `NEXT_PUBLIC_RESEARCHX_API_MODE=live` + `NEXT_PUBLIC_API_BASE_URL=""` (empty = same-origin through HAProxy's `/api` path route); `baseUrl()` treats `""` as same-origin (openapi-fetch relative base, EventSource relative URL); build asserts the empty base (a non-empty base would break same-origin cookies); dev live-mode docs pin `http://localhost:8000` on both sides (hostname rule, C3-15).
|
||||
|
||||
*Wave 3 — deploy + proxy*
|
||||
- **T-4.4 (ci-lead-developer)**: deploy.sh two-service extension — rsync now includes `backend/`; both images built on the host; plain docker-run pairs on the existing LXC idiom (no compose dependency): `researchx-net` bridge network; `researchx-api` backend (env-file, data volume bind, `-p 127.0.0.1:8000:8000` with host-side HAProxy, `--restart` policy); `researchx` frontend (`-p 3000:3000`, live build args); paired rollback containers `researchx-api-prev` + `researchx-prev` both retained; data volume pre-created + `chown 10001` host-side (belt-and-braces against EACCES); health check on BOTH `/healthz` (backend) and `/ask` (frontend) before the swap; idempotent re-run.
|
||||
- **T-4.5 (ci-lead-developer, security-engineer review)**: HAProxy front config — documented + idempotent update to the host's existing HAProxy 2.8 TLS frontend: `acl is_api path_beg /api`, `acl is_health path /healthz` → `backend_api`; `default_backend frontend_next`; `backend_api`: `option httpchk GET /healthz`, `default-server inter 5s fall 3 rise 2`, `server api1 127.0.0.1:8000 check`, `timeout server 3600s`, `timeout tunnel 3600s`; **NO `option http-buffer-response`, NO compression on the API backend** (gzip buffers text/event-stream); `backend frontend_next`: `timeout server 300s`, `server next1 127.0.0.1:3000 check` (REQ-84, C3-14); `timeout client` exceeds the sse-starlette 15s ping interval; the config file is committed as documentation; application to the live host only on owner deploy directive (D-019).
|
||||
- **T-4.6 (backend-engineer)**: `/healthz` verification — unauthenticated `GET /healthz → 200 {"status":"ok"}` (public router from P1, outside `/api` per D-103) proven through the proxy path route.
|
||||
|
||||
*Wave 4 — smoke + exposure audit + ship*
|
||||
- **T-4.7 (ci-lead-developer)**: Hosted smoke assertions — scripted against a staged instance: (1) unauthenticated `curl` to public `/api/runs` → **401**; (2) `/healthz` → **200**; (3) **live-mode marker**: authenticated `/api/config` resolves a real tier and the smoke asserts the build inlined live mode (empty base + live mode, not mock); (4) **SSE-through-proxy smoke**: an authenticated EventSource against the public origin streams a full run's events unbuffered end to end — no buffering, no mid-stream cut, 15s pings hold the connection through `timeout server 3600s` (REQ-84/85).
|
||||
- **T-4.8 (security-engineer)**: Hosted exposure audit — through the public URL: every `/api` route 401s unauthenticated (401-on-public-/api pinned); `/healthz` is the ONLY unauthenticated route; no `auth_enabled` backdoor in the production image (parameter-only, `main.py` never passes False — verified); cookie `Secure` flag set on the TLS-fronted host; secrets sweep over the built image layers and deploy artifacts (zero key values anywhere); STRIDE pass on the proxy surface.
|
||||
- **T-4.9 (ci-lead-developer)**: Phase ship — all gates green; tag **v0.3.4** + release; traceability updated; deployment to the live host staged only on owner directive (D-019) — the smoke suite is written to run against a staged deploy.
|
||||
|
||||
**Must-haves**:
|
||||
- One-command deploy brings up both services; paired rollback containers exist for BOTH services (REQ-83).
|
||||
- Backend container: non-root uid 10001, single uvicorn worker (`--workers 1` mandatory — in-memory registry + session store), secrets runtime-only via `--env-file`, data volume writable by uid 10001.
|
||||
- The hosted instance runs real inference in live mode by default (build-time inlined; REQ-75) with per-user keys honored through the full chain — while the code default stays mock (zero-key dev/CI preserved, REQ-31/D-112).
|
||||
- SSE streams unbuffered through the proxy: no http-buffer-response, no API compression, timeout server/tunnel 3600s, pings unbroken — verified end to end.
|
||||
- Unauthenticated public `/api` → 401 on every route; `/healthz` 200 the only unauthenticated route (REQ-85).
|
||||
- No secret values baked into images, committed, or logged; volume chown 10001.
|
||||
- HAProxy changes documented + idempotent; applied only on owner directive (D-019).
|
||||
|
||||
**Pitfall closures**: **P-3 full closure** (hosting half — the container serves `researchx.api.main` with the live factory; nothing can host fakes). RESEARCH §8/§9 hazards encoded and tested: volume-ownership EACCES (chown 10001 + smoke write), SSE-compression hazard (explicitly absent from the API backend config, asserted in review), ping keepalive (SSE smoke holds ≥ several ping intervals), SIGKILL-on-redeploy (startup sweep as the net, covered by the P1 sweep test).
|
||||
|
||||
**Success criteria**:
|
||||
- Both images build on the host via the extended deploy.sh; staged smoke suite passes all four assertions (401 / healthz 200 / live-mode marker / SSE unbuffered).
|
||||
- A keyed user on the staged instance gets fully real inference end to end through the proxy (`resolved_tier="user"` visible in the authenticated UI).
|
||||
- Image-layer secrets sweep clean; no auth-disable path exists in the production image.
|
||||
- Gates green; patch v0.3.4 shipped.
|
||||
|
||||
## Phase 5 — Final Review + Ship (FINAL — not planned in detail here)
|
||||
|
||||
Per ROADMAP: multi-persona review of the full milestone diff (REQ-69..85), audit (reconstruction, secrets sweep, branch/tag hygiene), then tag **v0.3.5 = the milestone release**. The pipeline HALTS at the merge_to_main owner approval gate (D-018 carries); deployment to research.coreci.dev proceeds only on owner directive (D-019 carries). The security-engineer's removal note (deactivate after P4) executes at P4 ship; the P5 review runs the standard full multi-persona pass. Detailed planning lands as the P5 PLAN when P4 completes, per the v0.2 precedent.
|
||||
|
||||
## Cross-Phase Notes
|
||||
|
||||
- **Versioning**: patches v0.3.1 (P1) → v0.3.2 (P2) → v0.3.3 (P3) → v0.3.4 (P4) → v0.3.5 (P5, the milestone release); tags on the v0.3.x line; MAJOR stays 0.
|
||||
- **Carrying gates**: G-005 snapshot discipline applies in every route-touching phase (P1, P2, P3); G-007 cost counters survive as owner-keyed (P-13, P2); G-008 DDG fixture discipline carries untouched; D-018/D-019 halt + deploy gates bind at P4/P5.
|
||||
- **REQ coverage assert**: each of REQ-69..85 maps to ≥1 test or contract proof in its phase (auth suite, key-store tamper suite, read-model suite, hosted smoke) — the audit reconstructs the milestone from these.
|
||||
- **Interface additions**: the C3-18 list is exhaustive; any other ApiClient change requires a PLAN amendment before implementation (the G-002 lesson, restated).
|
||||
- **Pitfall ledger (C3-19)**: P-1, P-2, P-13 close in P2; P-4, P-5, P-6, P-7, P-9, P-10, P-11, P-14 + P-8 (auth half) close in P1; P-8 (full) + P-12 close in P3; P-3 closes across P2 (backend half) and P4 (hosting half). All fourteen verified closed by v0.3.4.
|
||||
- **Engine contracts unchanged throughout**: the only engine-file touches in the milestone are additive — `RunState.owner_id` stamping (P1) and the `_verify_phase` artifact writes (P3, D-113); no methodology, state-machine, or protocol rewrites anywhere.
|
||||
+27
-8
@@ -6,9 +6,10 @@ ResearchX (working name; rename deferred — naming collision audit on file, see
|
||||
## Core Value
|
||||
A deep-research studio where you bring your own model, storage, and memory; choose a research methodology before a single query fires; and every claim in the final report is anchored to a visited, verified, quotable source. Simple enough to run in three clicks, transparent enough to audit line by line.
|
||||
|
||||
## Objective (two-milestone plan, owner-directed 2026-09-27)
|
||||
1. **Milestone v0.1 — Prototype**: UI/UX only, mock data, the production Next.js frontend with all data behind a typed `ApiClient` interface backed by `MockApiClient`. Surfaces: Ask, Choose (all 8 methodology cards), Plan & Run, Reading Room v0, v2 surface mockups (project graph, task inbox). **Ships only after owner approval — no MVP work begins until approved.**
|
||||
2. **Milestone v0.2 — MVP**: gated on prototype approval. FastAPI backend, custom async engine, adapter layer, trust loop, and integration (MockApiClient swapped for the OpenAPI-generated client).
|
||||
## Objective (three-milestone plan, owner-directed; v0.3 scope directed 2026-09-29)
|
||||
1. **Milestone v0.1 — Prototype** (COMPLETE, approved 2026-09-27, v0.1.8): UI/UX with mock data — production Next.js frontend behind a typed `ApiClient`/`MockApiClient` boundary.
|
||||
2. **Milestone v0.2 — MVP** (COMPLETE, approved 2026-09-27, v0.2.5): FastAPI backend, async engine, adapter layer, trust loop, OpenAPI client swap. Live mode = core research loop only (G-002/G-009 ratified).
|
||||
3. **Milestone v0.3 — Hosted** (owner-directed 2026-09-29): the hosted multi-user product — auth + sessions (user/pass, admin-managed accounts, OAuth deferred), per-user settings incl. model API keys (encrypted at rest; resolution chain user key → server key → keyless localhost), all prototype surfaces live-bound and user-scoped (library, settings persistence, suggestions, refine, graph/inbox, profile), backend hosting for the LXC deployment (SSE-safe), and real inference as the product default (mock demoted to dev/test fixture). The remaining 6 methodologies stay v0.4+ scope.
|
||||
|
||||
## Requirements (validated)
|
||||
1. Five pluggable adapter slots: model (OpenAI-compatible, planner/writer split), storage (local FS), search (Tavily default, DuckDuckGo keyless fallback), memory (off — stubbed), secrets (env — OpenBao later)
|
||||
@@ -28,16 +29,18 @@ A deep-research studio where you bring your own model, storage, and memory; choo
|
||||
- Memory scopes (semantic caching), S3/git storage adapters, Exa/Perplexity/Google search, OpenBao secrets, Docker packaging
|
||||
|
||||
## Constraints
|
||||
- Prototype must be approved before MVP work begins (owner directive)
|
||||
- ~~Prototype must be approved before MVP work begins~~ (v0.1 approved 2026-09-27; v0.2 approved 2026-09-27)
|
||||
- Versioning starts at v0.1; MAJOR remains 0 until an owner-approved v1 release; all pre-v1 ships are MINOR/PATCH only (owner directive)
|
||||
- ~~Single-user, local-first tool — no auth/multi-tenancy in MVP~~ **SUPERSEDED v0.3 (owner-directed 2026-09-29)**: multi-user hosted product — user/pass auth, admin-managed accounts (first registered user is admin), OAuth deferred to v0.4+; runs/library/settings partitioned by user
|
||||
- Per-user model keys resolve per run: user's own key → server env-file key → keyless localhost fallback (owner directive)
|
||||
- Not a general-purpose chatbot; not a model host; not a closed content index (v1)
|
||||
- Paywalled academic access out of scope v1 — open APIs (ArXiv/PubMed) only
|
||||
- Single-user, local-first tool — no auth/multi-tenancy in MVP
|
||||
- Ollama/vLLM supported via OpenAI-compatible endpoints only (no vendor-specific adapters)
|
||||
- Secrets never in app config or git; model capability honesty (warn when selected model is weak at structured outputs)
|
||||
- No deploys without owner directive (D-019 carries); milestone merge to main halts at the owner approval gate (D-018 carries)
|
||||
|
||||
## Out of Scope
|
||||
- Paywalled academic sources; model hosting; multi-tenant/auth; native Ollama API adapter; domain registration/rename decision
|
||||
- Paywalled academic sources; model hosting; open self-serve registration / OAuth (admin-managed accounts only, OAuth v0.4+); native Ollama API adapter; domain registration/rename decision
|
||||
|
||||
## Context
|
||||
Vision finalized and locked 2026-09-26: v1 methodology composition LOCKED (8 core), product named ResearchX, OpenBao retained as architectural secrets component, criticism matrix and evidence base [E1]–[E15] on file. Owner revisions 2026-09-27: (1) prototype-first — UI/UX prototype with mock data must be approved before MVP work begins; (2) version starts at v0.1, MAJOR remains 0 until MVP released; (3) the prototype is its own milestone. Milestone plan: v0.1 prototype (this milestone) → v0.2 MVP (gated).
|
||||
@@ -51,5 +54,21 @@ Vision finalized and locked 2026-09-26: v1 methodology composition LOCKED (8 cor
|
||||
- Model adapter default: Ollama cloud-first (ollama.com/v1) with keyless localhost:11434/v1 fallback on auth/connection failure
|
||||
- Prototype-first gate (owner): UI/UX prototype with mock data, approved before MVP work; prototype is its own milestone (v0.1)
|
||||
- Versioning (owner): starts at v0.1; MAJOR locked at 0 until owner-approved v1 release; MVP may span multiple 0.x minor milestones
|
||||
- Prototype = production Next.js frontend with typed ApiClient/MockApiClient split — zero throwaway work; phase 4 of v0.2 swaps the mock for the OpenAPI client
|
||||
- Tag-line mapping: 0.x milestones run patches on their own minor line (v0.1.x); supersedes the greenfield deviation noted at init
|
||||
- Prototype = production Next.js frontend with typed ApiClient/MockApiClient split — zero throwaway work; phase 3 of v0.2 swaps the mock for the OpenAPI client
|
||||
- Tag-line mapping: 0.x milestones run patches on their own minor line (v0.1.x); supersedes the greenfield deviation noted at init
|
||||
|
||||
## Key Decisions (v0.2 clarify — D-007..D-020, 2026-09-27)
|
||||
- Backend layout: `backend/` repo-root package, src-layout `researchx` (config, methodologies, adapters, engine, evidence, trust, api per ARCHITECTURE §Components); uv + uv-managed Python 3.12 (D-007/D-008)
|
||||
- Ports/binding: backend 127.0.0.1:8000, frontend :3000 via NEXT_PUBLIC_API_BASE_URL; CORS localhost:3000 (D-009); run IDs = 8-char slugs; storage = <dir>/runs/<id>/ (D-010 — **superseded by D-106 for v0.3+: runs/<user_id>/<run_id>/**)
|
||||
- Adapters: Ollama base URLs + key via env-file secrets scopes (OLLAMA_BASE_URL, OLLAMA_API_KEY); DDG keyless fallback = html.duckduckgo.com/html conservative scrape (D-011/D-012)
|
||||
- Integration: openapi-typescript + openapi-fetch → frontend/src/lib/api/generated, bound behind ApiClient; RESEARCHX_API_MODE=mock|live, mock default (D-013/D-020)
|
||||
- Transport/trust: sse-starlette SSE + pydantic typed event bus (D-014); difflib quote anchoring ≥ 0.85 (D-015); HTTP liveness live/stale/dead (D-016); JSON durable state, atomic rewrite per transition (D-017)
|
||||
- Run termination: pipeline ships P1..P4 + P5 final patch, then HALTS at the owner approval gate before milestone → main (merge_to_main hook, v0.1 precedent); no deploys without owner directive (D-018/D-019)
|
||||
## Key Decisions (v0.3 clarify — D-101..D-115, 2026-09-29)
|
||||
- Sessions: server-side store (in-process dict + durable sessions.json, D-017 idiom), httpOnly SameSite=Lax cookies, opaque tokens, 14-day sliding expiry, immediate revocation (D-101); argon2id via argon2-cffi, verified via asyncio.to_thread (D-102)
|
||||
- Auth posture: 401 on all /api/* except login/register (register open only while user store empty — first user is admin, then admin-managed; D-103/D-104); CSRF = SameSite=Lax + custom header on non-GET (EventSource GET exempt); /healthz outside /api for proxy checks
|
||||
- Identity/partitioning: user_id = 8-char slug, email login (D-105); storage runs/<user_id>/<run_id>/ (D-106 supersedes D-010); data root = RESEARCHX_DATA_DIR (backend/var/researchx dev, /var/lib/researchx container; D-107)
|
||||
- Key store: AES-256-GCM with RESEARCHX_KEK from the env-file 'keys' scope, user_id as AAD; disabled honestly when no KEK (D-108); scope = model key only (D-109); model adapter constructed per run (fixes shared-adapter fallback bug; D-110)
|
||||
- Settings: GET /api/config stays read-only adapter status (user-contextualized resolved tier); new GET/PUT /api/config/settings for per-user settings; storage path display-only hosted (D-111)
|
||||
- Mock fate: code default stays mock (zero-key dev/CI); hosted image built with NEXT_PUBLIC_RESEARCHX_API_MODE=live; no live delegation to mock after P3 (D-112)
|
||||
- REQ-82 is backend work: engine persists report.json + evidence.json at DONE; endpoints enriched (D-113); open questions = plan sub-questions answered-iff DONE (D-114); read-models read disk artifacts, not RunRegistry; in-flight runs marked cancelled on restart; rehydration out of v0.3 scope (D-115)
|
||||
|
||||
@@ -73,6 +73,40 @@
|
||||
- **REQ-33**: Monorepo: uv-managed backend (ruff, mypy, pytest) + Next.js frontend (ESLint, strict TS, Vitest); one-command install/dev
|
||||
- **REQ-34**: Full export round-trip: report + ledger + citations survive as real links
|
||||
|
||||
### Clarified constraints (v0.2 P0 clarify — D-007..D-020)
|
||||
- **D-007/D-008**: backend/ src-layout package `researchx` (config, methodologies, adapters, engine, evidence, trust, api); uv + managed Python 3.12
|
||||
- **D-009/D-010**: backend 127.0.0.1:8000, CORS localhost:3000; run IDs 8-char slugs; runs/<id>/ storage layout
|
||||
- **D-011/D-012**: model/search endpoints + keys resolve from env-file secrets scopes; DDG keyless fallback via html.duckduckgo.com/html scrape
|
||||
- **D-013/D-020**: openapi-typescript + openapi-fetch generated client bound behind ApiClient; RESEARCHX_API_MODE=mock|live, mock default until P3
|
||||
- **D-014..D-017**: sse-starlette SSE + pydantic event bus; difflib quote anchoring ≥ 0.85; HTTP liveness live/stale/dead; JSON durable state atomically rewritten per transition
|
||||
|
||||
## Hosted (Milestone v0.3 — owner-directed 2026-09-29)
|
||||
|
||||
### Auth & Sessions
|
||||
- **REQ-69**: User/pass auth: user model with argon2id password hashing (argon2-cffi, verified off the event loop), register/login/logout/me routes, httpOnly cookie sessions (server-side session store: in-process dict + durable JSON under the data root, immediate revocation on logout/deactivation), CSRF-safe same-origin posture (SameSite=Lax cookie + custom-header check on mutating requests); OAuth explicitly deferred to v0.4+
|
||||
- **REQ-70**: Admin-managed accounts: registration is open only while the user store is empty; the first registered user becomes admin; subsequently accounts are created by an admin only (admin-set initial password, no email infrastructure); admin can list/create/deactivate accounts (deactivation kills sessions; data retained)
|
||||
- **REQ-71**: User scoping: every run, stored report, and setting is partitioned by owner — the engine, storage paths, and library read-model are user-scoped; storage layout becomes runs/<user_id>/<run_id>/ (supersedes D-010); owner_id is recorded in run state; no migration (pre-hosted artifacts disposable); no user can read or act on another user's data through the API
|
||||
|
||||
### Per-User Settings & Model Keys
|
||||
- **REQ-72**: Per-user settings persistence (PUT/GET /api/config/settings): methodology defaults, resolved dials, appearance slot, masked key status; storage directory is server-managed per-user — display-only in hosted mode (the free-path picker applies to the local single-user profile only)
|
||||
- **REQ-73**: Per-user model API keys: masked entry in Settings (never returned in full — masked status only), encrypted at rest (per-user key store; KEK = RESEARCHX_KEK from the env-file secrets 'keys' scope; cipher AES-256-GCM with user_id as AAD; without a configured KEK the key store is disabled with honest status — chain falls to server/keyless); scope = model key (OLLAMA_API_KEY) only in v0.3
|
||||
- **REQ-74**: Per-run model credential resolution chain: user's own key → server env-file key → keyless localhost:11434 fallback (existing fallback semantics preserved); the model adapter is constructed per run (fixes the shared-adapter fallback bug); the capability probe runs against the resolved tier; config status surfaces which tier resolved (user|server|keyless) for the current user
|
||||
- **REQ-75**: Real inference as the product default: the code default of RESEARCHX_API_MODE stays mock (zero-key dev/CI per REQ-31); the hosted frontend is built with NEXT_PUBLIC_RESEARCHX_API_MODE=live (build-time inlining); after v0.3 P3 no live-mode surface delegates to the mock; mock demoted to a dev/test fixture
|
||||
|
||||
### Live Surfaces (G-002 boundary lifted, user-scoped)
|
||||
- **REQ-76**: Library backend: GET /api/library — folder-structure listing over the user's runs (auto-categorization by topic + methodology, recency/title sorting); report cards resolve to real runs
|
||||
- **REQ-77**: Cross-report connections + project graph: GET /api/library/connections and GET /api/graph — shared-source and topical edges computed from the user's ledgers and run metadata
|
||||
- **REQ-78**: Task inbox: GET /api/tasks — open questions from the user's runs as checkable follow-ups; open questions = plan sub-questions of the user's runs, answered-iff the run reached DONE
|
||||
- **REQ-79**: Suggestions: GET /api/suggestions — follow-up query suggestions derived from the user's unanswered questions (open inbox items joined with source runs); click-fills-the-box semantics preserved; empty state for users with no runs (no synthetic suggestions)
|
||||
- **REQ-80**: Refine: POST /api/refine/questions + /api/refine/compose — planner-model-driven clarifying exchange composing a refined question + suggested context (scripted prototype dialogue replaced by real model calls)
|
||||
- **REQ-81**: User profile: GET/PUT /api/profile — local profile (display name, email) surfaced in the avatar user menu; no account deletion in v0.3
|
||||
- **REQ-82**: Live Reading Room fidelity: the live report renders structured sections with page-anchored citations (hover shows the stored quoted passage) and the ledger panel from the citation ledger — parity with the mock Reading Room v0; the engine persists report.json (structured sections + anchored citations with quotes) and evidence.json (passage texts) at DONE; report/ledger endpoints expose them; live exports use real entry data (no fabricated fields)
|
||||
|
||||
### Hosting
|
||||
- **REQ-83**: Backend container: Dockerfile for the FastAPI backend (single uvicorn worker, non-root), two-service deployment (frontend + backend) via deploy.sh extension; RESEARCHX_DATA_DIR volume (default /var/lib/researchx); unauthenticated /healthz outside /api for proxy health checks
|
||||
- **REQ-84**: SSE-safe reverse proxy: HAProxy path routing /api → backend with streaming timeouts that do not buffer or cut long-lived SSE connections; frontend served with NEXT_PUBLIC_API_BASE_URL pointing at the public origin
|
||||
- **REQ-85**: Hosted live mode: the deployed instance runs real inference in live mode (per REQ-75), per-user keys honored; auth guard covers the backend surface (no unauthenticated API access through the public URL)
|
||||
|
||||
## V2 (Post-MVP backlog)
|
||||
|
||||
### Methodology Catalog Extension
|
||||
@@ -105,7 +139,7 @@
|
||||
|---|---|
|
||||
| Paywalled academic access | Unsolved in v1 — routed around via open APIs (vision §8) |
|
||||
| General-purpose chatbot / model host / closed content index | Explicit non-goals (vision §8) |
|
||||
| Auth / multi-tenancy | Single-user local-first MVP |
|
||||
| Auth / open self-serve registration / OAuth | v0.3 ships user/pass auth with admin-managed accounts (owner-directed 2026-09-29); open registration and OAuth stay out (OAuth v0.4+) |
|
||||
| Native Ollama API adapter | Ollama supported via OpenAI-compatible endpoint only |
|
||||
| Domain registration / product rename | Deferred by owner (vision §11) |
|
||||
| v1.0.0 release | Owner-approved release only; not created autonomously by the pipeline |
|
||||
@@ -122,9 +156,13 @@
|
||||
| REQ-62 (approval gate) | v0.1 | P8 | complete (owner approval recorded 2026-09-27) |
|
||||
| REQ-63, REQ-64, REQ-65, REQ-66 (app shell, library, settings, avatar) | v0.1 | P5 | complete |
|
||||
| REQ-67, REQ-68 (ask suggestions, refine dialogue) | v0.1 | P7 | complete |
|
||||
| REQ-01..06, REQ-08..11, REQ-31 (framework, adapters) | v0.2 | P1 | blocked |
|
||||
| REQ-07, REQ-12, REQ-13..24 (probing, engine, trust) | v0.2 | P2 | blocked |
|
||||
| REQ-25, REQ-26, REQ-33 (API, client, tooling) | v0.2 | P3 | blocked |
|
||||
| REQ-17, REQ-32, REQ-34 (adversarial, E2E, round-trip) | v0.2 | P4 | blocked |
|
||||
| All REQ-01..34 (milestone gate) | v0.2 | P5 | blocked |
|
||||
| REQ-01..06, REQ-08..11, REQ-31 (framework, adapters) | v0.2 | P1 | complete (v0.2.1; REQ-07/12 contracts proven, live probing in P2) |
|
||||
| REQ-07, REQ-12, REQ-13..24 (probing, engine, trust) | v0.2 | P2 | complete (v0.2.2) |
|
||||
| REQ-25, REQ-26, REQ-33 (API, client, tooling) | v0.2 | P3 | complete (v0.2.3) |
|
||||
| REQ-17, REQ-32, REQ-34 (adversarial, E2E, round-trip) | v0.2 | P4 | complete (v0.2.4) |
|
||||
| All REQ-01..34 (milestone gate) | v0.2 | P5 | complete (v0.2.5 — **owner approval recorded 2026-09-27**, G-009 ratified; milestone merged to main) |
|
||||
| REQ-69..71 (auth, sessions, scoping) | v0.3 | P1 | complete (v0.3.1; security review P0 register race + P1 corrupt-store fail-closed fixed in-phase; verify PASS 4/4) |
|
||||
| REQ-72..74 (settings, per-user keys, resolution chain) | v0.3 | P2 | complete (v0.3.2; P0 null-poisoning fixed in verify; P-1/P-2/P-13 closed; production main entrypoint P-3 backend half) |
|
||||
| REQ-76..82 (live surfaces, user-scoped) | v0.3 | P3 | complete (v0.3.3; D-112 standalone client, G-109 honest inbox, P-8 full closure, P-12 dead; verify PASS-WITH-FIXES) |
|
||||
| REQ-75 (delegation-end half) + REQ-83..85 (hosting; live default hosted) | v0.3 | P3/P4 (REQ-75 split per G-103: delegation-end P3 complete, hosted-default P4) | REQ-75 complete in P3 |
|
||||
| REQ-35..51 | v2 backlog (post-MVP milestones) | — | pending |
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
# Research — v0.2 MVP (Phase 0)
|
||||
|
||||
## Run: 2026-09-27 (stage: research)
|
||||
|
||||
Phase-0 research findings for milestone v0.2 (MVP): FastAPI backend, custom asyncio engine, adapter layer, trust loop, OpenAPI client integration. Each topic records concrete recommendations and pitfalls that PLAN consumes.
|
||||
|
||||
## 1. Backend foundation (uv + FastAPI)
|
||||
|
||||
**Recommendation:**
|
||||
- uv (>= 0.4) with `backend/pyproject.toml`, src-layout: `backend/src/researchx/__init__.py`, `backend/src/researchx/{config,methodologies,adapters,engine,evidence,trust,api}/__init__.py`
|
||||
- `[project] name = "researchx"`, `requires-python = ">=3.12"`; deps: `fastapi>=0.115,<0.116`, `pydantic>=2.8,<3`, `uvicorn>=0.30,<1`, `sse-starlette>=2.1,<3`, `httpx>=0.27,<1`
|
||||
- Dev group (`uv sync --group dev` or `[dependency-groups]`): `ruff>=0.6`, `mypy>=1.11`, `pytest>=8`, `pytest-asyncio>=0.23`
|
||||
- `[build-system] hatchling`; `[tool.hatch.build.targets.wheel] packages = ["src/researchx"]` — src-layout pitfall: without the explicit packages key, uv/hatch builds fail or ship an empty wheel
|
||||
- Dev run: `cd backend && uv run uvicorn researchx.api.app:app --reload --port 8000`; one-command from root documented in README
|
||||
- mypy strict on `src/`: `[tool.mypy] strict = true, mypy_path = "src"`; ruff: line-length 100, target py312
|
||||
|
||||
**Pitfalls:**
|
||||
- `uv run` resolves the project venv automatically — never `pip install` into it
|
||||
- pytest needs `pythonpath = ["src"]` in `[tool.pytest.ini_options]` or an editable install (`uv sync` does this) — tests import `researchx.*` directly
|
||||
- pytest-asyncio: use `asyncio_mode = "auto"` and `loop_scope = "function"` (the old `event_loop` fixture is removed in recent versions)
|
||||
|
||||
## 2. Ollama OpenAI-compatible endpoints
|
||||
|
||||
**Recommendation:**
|
||||
- Cloud: `https://ollama.com/v1` with `Authorization: Bearer <OLLAMA_API_KEY>`; local fallback `http://localhost:11434/v1` keyless
|
||||
- Chat: POST `/chat/completions` (OpenAI shape: messages, model, temperature; response choices[0].message.content)
|
||||
- Probe: GET `/models` → data[].id; capability check via a tiny structured-completion smoke test at connect time
|
||||
- Context cap: `min(model_claim, 8192)` tokens; model claim unknown → 8192 flat; embedding batch 8
|
||||
- Planner/writer roles = two model names from settings (default: same model both roles)
|
||||
|
||||
**Pitfalls:**
|
||||
- ollama.com/v1 is cloud-keyed — 401 on missing/invalid key → automatic local-fallback attempt, then surfaced error
|
||||
- Some served models don't support function calling/structured outputs → REQ-12 warning path: detect via smoke-test failure, warn in UI, degrade to plain chat + prompt-embedded JSON contract
|
||||
- Model names are server-specific (`q4` suffixes etc.) — settings must store the exact served name; probe lists what's actually available
|
||||
|
||||
## 3. Search adapters
|
||||
|
||||
**Recommendation:**
|
||||
- Tavily: POST `https://api.tavily.com/search`, JSON `{"api_key": key, "query": q, "search_depth": "basic"|"advanced", "max_results": n, "include_answer": false}` → `results[]: {title, url, content, score}`; timeouts 10s
|
||||
- DuckDuckGo keyless fallback: POST `https://html.duckduckgo.com/html/?q=<urlencoded>` with a plain httpx client; parse anchors `a.result__a` (href → URL, text → title) and `a.result__snippet` from the result list; conservative: max 8 results, no retries beyond one backoff, user-agent string honest
|
||||
- Adapter interface: `async search(query, max_results) -> list[SearchResult]`; registry selects Tavily when TAVILY_API_KEY present, else DDG
|
||||
|
||||
**Pitfalls:**
|
||||
- DDG html rate-limits aggressively — the fallback is best-effort; on 403/timeout return partial/empty with a surfaced warning, never hang the engine
|
||||
- Tavily's `content` is already a snippet — evidence extraction still fetches the live page (visited-only constraint requires the visit)
|
||||
- Never put the API key in a query string or log line
|
||||
|
||||
## 4. FastAPI SSE + asyncio durability
|
||||
|
||||
**Recommendation:**
|
||||
- `sse-starlette` EventSourceResponse fed by a per-run `asyncio.Queue`; pydantic event models with a discriminated union on `type` (run_started, query_fired, source_visited, passage_extracted, state_changed, run_done, run_error)
|
||||
- EventSourceResponse generator: `async for event in queue` → `{"event": <type>, "data": <model_dump_json>}`; send on connect a `state_changed` replay of current persisted state
|
||||
- Client disconnect: wrap the generator in try/finally that cancels only the SSE subscription (never the run); runs keep going when browsers disconnect
|
||||
- Backpressure: unbounded queue per run is acceptable single-user; cap at 1000 with drop-oldest `state_changed` coalescing if needed
|
||||
|
||||
**Pitfalls:**
|
||||
- No blocking I/O in the response generator — all fetches via httpx.AsyncClient
|
||||
- uvicorn single worker; multiple workers would break in-memory run registry — document single-user explicitly
|
||||
- On reconnect (Last-Event-ID unused in MVP) → replay current state + live events onward; full event log persisted in the run dir anyway
|
||||
|
||||
## 5. Durable state machine patterns
|
||||
|
||||
**Recommendation:**
|
||||
- `runs/<id>/state.json`: `{run_id, state, dial_values, plan, question, created_at, updated_at, cursor}` — rewritten atomically after every transition: write `.tmp`, `os.replace`
|
||||
- Transitions only at method boundaries of the state machine; the cursor records depth-loop position for resume
|
||||
- Pause = asyncio.Event cleared; cancel = task.cancel() + terminal CANCELLED state persisted; resume on restart = state file read, EventBus rebuilt, loop re-entered at cursor
|
||||
- pytest-asyncio: `asyncio_mode = "auto"`; tests construct the engine with fake adapters and `await engine.run()` directly
|
||||
|
||||
**Pitfalls:**
|
||||
- Never swallow CancelledError — re-raise after persisting the cancel state
|
||||
- Shield the state write: `with contextlib.suppress(asyncio.CancelledError): await asyncio.shield(save())` then re-raise
|
||||
- Torn reads: only ever `os.replace` (atomic on same filesystem); never append — state is a snapshot, trace is a separate append-only file
|
||||
|
||||
## 5b. Testing fakes (REQ-31)
|
||||
|
||||
**Recommendation:**
|
||||
- FakeModelAdapter: canned planner (sub-questions from a fixture), canned writer (deterministic report citing fixture passage IDs) — zero API keys
|
||||
- FakeSearchAdapter: fixture corpus of 5-8 pages with overlapping passages; deterministic ordering
|
||||
- Engine tests drive every transition, depth loop, pause/resume mid-depth-loop, cancel, and adversarial writer outputs
|
||||
|
||||
**Pitfalls:**
|
||||
- Fakes must implement the full adapter interface (no stub methods) — verify layer asserts interface conformance
|
||||
- Keep fixtures small and unicode-diverse (quote anchoring edge cases)
|
||||
|
||||
## 6. §6. Quote anchoring + liveness
|
||||
|
||||
**Recommendation:**
|
||||
- Normalize (unicode NFC → casefold → whitespace collapse) both quote and passage; exact-substring short-circuit; else difflib SequenceMatcher ratio ≥ 0.85
|
||||
- Ledger entry: `{url, visited_at, passages: [{id, quote}], claims: [claim_ids], health, authority_tier}`
|
||||
- Liveness: single GET with redirect-follow (httpx, 10s timeout): 2xx live; 3xx loop/parked-domain or content-gone markers → stale; 4xx/5xx/timeout/unresolvable → dead
|
||||
- ≥99% liveness gate: shipped report citations must pass liveness at export time; failures listed in the verification checklist
|
||||
|
||||
**Pitlingfalls:**
|
||||
- SequenceMatcher on long passages is O(n²)-ish — anchor per-passage (passages are ≤ ~500 chars), not per-document
|
||||
- Fuzzy matches near threshold (0.80-0.90) flagged in the ledger as `weak_match` for transparency
|
||||
|
||||
**Pitfalls:** (corrected)
|
||||
- SequenceMatcher cost is quadratic-ish — always match quote vs its claimed passage only
|
||||
- Near-threshold (0.80–0.90) matches flagged `weak_match` in the ledger — surfaced, not silently accepted
|
||||
|
||||
## 7. OpenAPI client generation
|
||||
|
||||
**Recommendation:**
|
||||
- `openapi-typescript@^7` (types) + `openapi-fetch@^0.13` (typed fetch client), generated into `frontend/src/lib/api/generated/`
|
||||
- Regen: `pnpm dlx openapi-typescript http://127.0.0.1:8000/openapi.json -o frontend/src/lib/api/generated/schema.d.ts` — needs the backend running, or read a committed `openapi.json` snapshot in CI
|
||||
- Binding: a thin `GeneratedApiClient` implements the existing ApiClient interface by delegating to openapi-fetch with baseURL from NEXT_PUBLIC_API_BASE_URL; components never import it directly — only the boundary file does
|
||||
- RESEARCHX_API_MODE=mock|live env flag selects the binding at boundary initialization; mock stays default
|
||||
|
||||
**Pitfalls:**
|
||||
- FastAPI operation-id stability: route order and function names determine ids — fix function names early; regen discipline = regenerate only from a committed snapshot
|
||||
- FastAPI's default unions (None | X) — openapi-typescript emits optional/nullable types that must be handled at the boundary, not in components
|
||||
|
||||
## 8. Monorepo ergonomics
|
||||
|
||||
**Recommendation:**
|
||||
- Root README one-command story: `uv sync` (backend) + `corepack pnpm install` (frontend); dev: `uv run uvicorn ...` + `corepack pnpm dev`
|
||||
- Backend gates: `uv run ruff check src tests`, `uv run mypy src`, `uv run pytest` (all via `uv run` from backend/)
|
||||
- Frontend gates carry over from v0.1 unchanged: `pnpm typecheck`, `pnpm lint`, `pnpm test`, `pnpm build`
|
||||
- Repo-root package.json is NOT created (frontend is the only JS package); document commands in README instead
|
||||
|
||||
**Pitfalls:**
|
||||
- pnpm via corepack (packageManager field pins pnpm@12.6.0) — direct pnpm may be absent; always `corepack pnpm`
|
||||
- openapi.json snapshot committed for reproducible client regen; regenerate the TS client when the snapshot changes
|
||||
|
||||
## Research Conclusions
|
||||
|
||||
1. Backend = uv + src-layout `researchx` in backend/, pyproject with hatchling `packages=["src/researchx"]`, deps fastapi/pydantic/uvicorn/sse-starlette/httpx, dev ruff/mypy/pytest/pytest-asyncio — one-command via `uv sync && uv run uvicorn researchx.api.app:app`.
|
||||
2. All I/O lives in adapters; engine imports no vendor SDKs (httpx only inside adapters).
|
||||
3. Ollama cloud `https://ollama.com/v1` (Bearer) → local `http://localhost:11434/v1` keyless fallback on 401/connection failure; probe via GET /models + structured-output smoke test; context cap min(claim, 8192), embed batch 8.
|
||||
4. REQ-12 structured-output warning: detect via smoke test; degrade to prompt-embedded JSON contract + UI warning — never fail cryptically.
|
||||
5. Tavily POST api.tavily.com/search with api_key in JSON body; DDG fallback scrapes html.duckduckgo.com/html anchors conservatively (max 8, one backoff, surfaced warnings on rate-limit).
|
||||
6. SSE = sse-starlette EventSourceResponse + per-run asyncio.Queue + pydantic discriminated-union event types; state replay on connect; single uvicorn worker (single-user).
|
||||
7. Durable state = runs/<id>/state.json, atomic tmp+os.replace per transition, cursor for depth-loop resume; trace is a separate append-only file; shielded state writes; never swallow CancelledError.
|
||||
8. Quote anchoring = normalize + exact-substring short-circuit + SequenceMatcher ≥ 0.85 per-passage; near-threshold flagged weak_match.
|
||||
9. Liveness = redirect-following GET, 10s: 2xx live / redirect-loop or content-gone stale / 4xx-5xx-timeout dead; ≥99% citation liveness gate at export.
|
||||
10. Fake adapters (REQ-31) implement the full adapter interface with canned planner/writer + fixture corpus — enables the entire engine + trust test suite with zero keys.
|
||||
11. Client gen = openapi-typescript@^7 + openapi-fetch@^0.13 from a committed openapi.json snapshot; GeneratedApiClient binds behind the ApiClient interface; RESEARCHX_API_MODE=mock|live selects binding; mock default.
|
||||
12. Monorepo gates: backend `uv run ruff check`, `uv run mypy`, `uv run pytest`; frontend v0.1 gates unchanged; no root package.json — commands documented in README.
|
||||
13. Config/status endpoints surface probe results + adapter status (REQ-25 GET /api/config) — plan the schema early so the settings surface stays honest.
|
||||
14. All decisions D-007..D-020 confirmed workable; refinements: hatchling packages key (new), openapi.json snapshot discipline (new), single-worker uvicorn note (new), structured-output smoke-test detection pattern (new) — folded into PLAN as constraints.
|
||||
@@ -0,0 +1,555 @@
|
||||
# Research — v0.3 Hosted (Phase 0)
|
||||
|
||||
## Run: 2026-09-29 (stage: research)
|
||||
|
||||
Phase-0 research findings for milestone v0.3 (Hosted): auth + server-side
|
||||
sessions, per-user encrypted model keys, user-scoped storage, live surfaces
|
||||
backed by disk artifacts, and two-service hosting behind an SSE-safe proxy.
|
||||
Each topic records concrete code/config shapes and pitfalls that PLAN
|
||||
consumes. All library claims were verified against the pinned versions in
|
||||
the repo (starlette 0.46.2, fastapi 0.115.14, sse-starlette, eventsource
|
||||
polyfill 5.1.2).
|
||||
|
||||
## 1. FastAPI auth/session layer
|
||||
|
||||
**Recommendation:**
|
||||
- New package `backend/src/researchx/auth/`: `users.py` (user model +
|
||||
argon2id), `sessions.py` (server-side session store), `guard.py`
|
||||
(session + CSRF dependency), `routes.py` (register/login/logout/me).
|
||||
- **Route dependencies, not middleware, as the enforcement point.** Group
|
||||
all protected routes under `APIRouter(dependencies=[Depends(require_user)])`
|
||||
inside `create_app`. The dependency resolves the session cookie, loads
|
||||
the user, runs the CSRF header check on non-GET, and returns the `User`:
|
||||
```python
|
||||
async def require_user(request: Request) -> User:
|
||||
token = request.cookies.get(SESSION_COOKIE) # "researchx_session"
|
||||
session = sessions.get(token) # None if absent/expired/revoked
|
||||
if session is None:
|
||||
raise HTTPException(status_code=401, detail="authentication required")
|
||||
if request.method != "GET" and "x-researchx-request" not in request.headers:
|
||||
raise HTTPException(status_code=403, detail="missing request header")
|
||||
user = users.get(session.user_id)
|
||||
if user is None or not user.active:
|
||||
raise HTTPException(status_code=401, detail="account disabled")
|
||||
sessions.touch(session) # sliding expiry
|
||||
return user
|
||||
```
|
||||
Rationale vs global middleware: the dependency appears in OpenAPI (401
|
||||
documented), is overridable per-test via `app.dependency_overrides`
|
||||
(critical for the zero-key harness, topic 9), and avoids the pure-ASGI
|
||||
middleware rewrite needed to keep SSE streaming safe. A public router
|
||||
(`/api/auth/*`, `/healthz`) + protected router (everything else)
|
||||
enforces "all /api/* except login/register" by construction.
|
||||
- **Session store (D-101):** opaque token = `secrets.token_urlsafe(32)`;
|
||||
in-process `dict[token, Session]` + durable `sessions.json` at the data
|
||||
root, written with the atomic tmp+`os.replace` idiom (D-017).
|
||||
`Session = {token_hash, user_id, created_at, last_seen, expires_at}`.
|
||||
14-day sliding expiry via `touch()`; persist only when >60s since last
|
||||
persist. Immediate revocation = delete token + persist. Logout deletes
|
||||
the presented token; deactivation deletes all of the user's tokens.
|
||||
Rotate the token at login (kills session fixation by construction).
|
||||
- **Cookie:** `researchx_session=<token>`; `httponly=True, samesite="lax",
|
||||
path="/"`, `secure=True` only in hosted/TLS deploys (config flag), no
|
||||
`Domain` attribute (host-only).
|
||||
- **argon2-cffi (D-102):** `PasswordHasher()` defaults are argon2id with
|
||||
the RFC 9106 low-memory profile — keep the defaults. All calls off the
|
||||
event loop via `asyncio.to_thread`; `VerifyMismatchError` → uniform 401
|
||||
"invalid email or password"; `check_needs_rehash` → rehash on login.
|
||||
- **CORS ordering (verified):** the LAST-added middleware is OUTERMOST
|
||||
(starlette 0.46.2 `build_middleware_stack`). Add `CORSMiddleware` AFTER
|
||||
any auth middleware so preflight OPTIONS is answered without a 401.
|
||||
Cookie auth requires `allow_credentials=True` + explicit localhost
|
||||
origins (browsers reject wildcard origins with credentials).
|
||||
- **First-user-is-admin (D-104):** `POST /api/auth/register` checks
|
||||
`users.empty()` then inserts `role="admin"`; no `await` between check
|
||||
and insert except the argon2 hash — the single uvicorn worker makes
|
||||
this race-free by construction. After the first user, register 403s;
|
||||
`POST /api/admin/users` (admin-guarded) creates accounts.
|
||||
- **User store:** `users.json` at the data root:
|
||||
`{"users": [{"user_id", "email", "password_hash", "display_name",
|
||||
"role": "admin"|"user", "active": true, "created_at"}]}`; `user_id` =
|
||||
8-char slug from the run-ID alphabet + validation regex `^[a-z0-9]{8}$`
|
||||
(D-105 — filesystem-safe partition key).
|
||||
|
||||
**Pitfalls:**
|
||||
- Do NOT use `BaseHTTPMiddleware` for the guard — it wraps response bodies
|
||||
and has a history of SSE/streaming buffering bugs; router dependencies
|
||||
sidestep the class.
|
||||
- SameSite site-computation is scheme+registrable domain, NOT port —
|
||||
`localhost:3000 → localhost:8000` is same-site (Lax cookie flows with
|
||||
`credentials:"include"`), but `localhost:3000 → 127.0.0.1:8000` is
|
||||
CROSS-site (Lax cookie silently dropped). Dev docs must pin both sides
|
||||
to `localhost` (`NEXT_PUBLIC_API_BASE_URL=http://localhost:8000`).
|
||||
- argon2 verify at default params takes ~50–100ms CPU — `to_thread`
|
||||
mandatory for hash and verify.
|
||||
- `EnvFileSecretsProvider` loads at construction; user/session/key stores
|
||||
are mutable and must reload-or-rewrite atomically instead.
|
||||
|
||||
## 2. CSRF posture (D-103)
|
||||
|
||||
**Recommendation:**
|
||||
- Combined defense, no token round-trips: (1) `SameSite=Lax` session
|
||||
cookie blocks cross-site cookie attachment on background POSTs; (2)
|
||||
every non-GET `/api` request must carry `X-ResearchX-Request: 1`
|
||||
(checked in `require_user`). A cross-site attacker can neither attach
|
||||
the Lax cookie to a background POST nor set custom headers without
|
||||
CORS preflight approval.
|
||||
- Frontend: the live client sends the header on every mutating call —
|
||||
a small `#req()` wrapper or openapi-fetch middleware in
|
||||
`GeneratedApiClient`: `headers: { "X-ResearchX-Request": "1" }`.
|
||||
- **EventSource is exempt by design:** GET-only, cannot set custom
|
||||
headers — the guard skips GETs, so `/api/runs/{id}/events`
|
||||
authenticates by cookie alone. The eventsource polyfill (5.1.2) used
|
||||
in E2E supports `withCredentials` and a fetch override.
|
||||
- Login/register POSTs also require the header (uniform rule).
|
||||
- Dev split-origin: `fetch(..., {credentials: "include"})` everywhere +
|
||||
CORS `allow_credentials=True` + same-hostname origins (§1 pitfall).
|
||||
|
||||
**Pitfalls:**
|
||||
- If dev ever needs cross-SITE cookies (localhost vs 127.0.0.1), Lax
|
||||
fails silently — the fix is same-hostname origins, not `SameSite=None`.
|
||||
- `Secure` must NOT be set in plain-HTTP dev (browsers drop Secure
|
||||
cookies on http); MUST be set on the TLS-fronted host.
|
||||
|
||||
## 3. Per-user encrypted key store (D-108, D-109)
|
||||
|
||||
**Recommendation:**
|
||||
- New `backend/src/researchx/auth/key_store.py` using
|
||||
`cryptography.hazmat.primitives.ciphers.aead.AESGCM`
|
||||
(`AESGCM(key).encrypt(nonce, data, aad)` → ciphertext||16-byte tag;
|
||||
`decrypt` raises `InvalidTag` on mismatch; nonce = 12 bytes; NEVER
|
||||
reuse a nonce under the same KEK).
|
||||
- **KEK (D-108):** `RESEARCHX_KEK` from the env-file **'keys' scope**
|
||||
(`config.json secrets.scopes` gains `"keys": ["RESEARCHX_KEK"]`),
|
||||
accepted as urlsafe-base64 or hex of exactly 32 bytes; decode errors
|
||||
raise `AdapterError("RESEARCHX_KEK is not valid b64/hex")`; wrong
|
||||
length raises (`must decode to 32 bytes (AES-256)`).
|
||||
- **Envelope:** AAD = `user_id.encode()` (a copied file under another
|
||||
user's directory fails `InvalidTag`); fresh `os.urandom(12)` nonce per
|
||||
write; scope = `OLLAMA_API_KEY` only (D-109).
|
||||
- **File format** `keys/<user_id>.json` under the data root, atomic
|
||||
rewrite, `0600`:
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"alg": "AES-256-GCM",
|
||||
"updated_at": "2026-09-29T12:00:00Z",
|
||||
"entries": {
|
||||
"OLLAMA_API_KEY": {"nonce": "<b64url(12)>", "ct": "<b64url(ct+tag)>", "updated_at": "..."}
|
||||
}
|
||||
}
|
||||
```
|
||||
- **API never returns key material** (REQ-73): PUT accepts a raw key, GET
|
||||
returns `{"configured": true, "masked": "…3f2a"}` (last 4 chars).
|
||||
Crypto in `asyncio.to_thread` for discipline.
|
||||
- Honest degradation: no KEK → PUT returns `{"key_store": "disabled"}`;
|
||||
resolution chain proceeds server/keyless; corrupt entries (InvalidTag)
|
||||
surface as error status, not silent tier drop.
|
||||
|
||||
**Pitfalls:**
|
||||
- GCM nonce reuse under a fixed KEK is catastrophic — always `os.urandom`.
|
||||
- KEK rotation is out of v0.3 scope; `v` + `entries` leave room later.
|
||||
- A changed KEK requires process restart (EnvFileSecretsProvider loads
|
||||
once) — document it.
|
||||
|
||||
## 4. Per-run model adapter factory (D-110)
|
||||
|
||||
**Recommendation:**
|
||||
- Verified shared-adapter bug (`ollama.py:73-79`): a cloud failure
|
||||
permanently flips `self._active_base` on the SHARED instance — every
|
||||
subsequent run by every user silently uses localhost. Fix = per-run
|
||||
adapter instances.
|
||||
- New `backend/src/researchx/adapters/model_factory.py`:
|
||||
`ModelAdapterFactory(secrets, key_store, defaults)` with
|
||||
`async resolve(user, settings) -> (OllamaModelAdapter, tier)` resolving
|
||||
the chain user → server → keyless (REQ-74) and constructing one
|
||||
adapter per run.
|
||||
- `OllamaModelAdapter` constructor gains optional `api_key: str | None`
|
||||
taking precedence over `secrets.resolve(...)`; the secrets path stays
|
||||
for back-compat. Fallback mutation stays — now per-instance.
|
||||
- **Threading through:** `create_app` constructs the factory once;
|
||||
`create_run` does `model, tier = await factory.resolve(user, settings)`
|
||||
and passes `model` into `RunActor` — **engine contracts unchanged**;
|
||||
only the instance origin changes. `RunRegistry` keys become
|
||||
`(owner_id, run_id)`. `state.json` records `owner_id` and
|
||||
`credential_tier` for auditability.
|
||||
- **Probe-per-run:** the existing `supports_probe` + `probe_model_adapter`
|
||||
+ `actor.attach_probe(...)` flow operates on the adapter passed to
|
||||
`create_run` — per-tier automatically. `GET /api/config` resolves the
|
||||
factory for the current user WITHOUT creating a run; cache in-process
|
||||
keyed by `(user_id, tier, key_fingerprint)` with short TTL.
|
||||
|
||||
**Pitfalls:**
|
||||
- `config_status` reads `runs._actors` (private) to steal the last run's
|
||||
probe (`app.py:174-177`) — breaks with zero runs and leaks another
|
||||
user's probe; replace with the factory-resolved probe.
|
||||
- Don't cache adapters across runs — per-run lifetime keeps
|
||||
`_last_fallback_reason` honest.
|
||||
|
||||
## 5. Storage partitioning (D-106, D-107) — exact touchpoints
|
||||
|
||||
**Recommendation:**
|
||||
- **Owner-scoped adapter instances, not owner-in-signatures.**
|
||||
`LocalStorageAdapter(base_dir, owner_id)` binds the owner at
|
||||
construction; `run_dir(run_id) -> base_dir/"runs"/owner_id/run_id`.
|
||||
The `StorageAdapter` protocol stays byte-for-byte unchanged; only
|
||||
constructors gain `owner_id` (validated `^[a-z0-9]{8}$`, D-105).
|
||||
- App wiring: routes construct the owner-scoped adapter from the auth
|
||||
dependency per request. `RunState` gains `owner_id: str | None = None`
|
||||
(serialization-compatible; no migration per D-106).
|
||||
`make_run_config(...)` gains `owner_id` and stamps it on state.
|
||||
- `RunRegistry` keys `_actors[(owner_id, run_id)]`; cross-owner access
|
||||
is a 404.
|
||||
- **D-107 data root:** new `researchx.config.paths.data_root()`:
|
||||
`os.environ["RESEARCHX_DATA_DIR"]` else `backend/var/researchx` (dev);
|
||||
container `/var/lib/researchx`. Replaces the cwd-dependent
|
||||
`Path.cwd()/".researchx"` (`app.py:150`). Layout:
|
||||
```
|
||||
$RESEARCHX_DATA_DIR/
|
||||
users.json sessions.json
|
||||
runs/<user_id>/<run_id>/{state.json, trace.jsonl, report.md,
|
||||
report.json, evidence.json, ledger.json, checklist.md}
|
||||
keys/<user_id>.json
|
||||
settings/<user_id>.json
|
||||
```
|
||||
- **Exact touchpoint list (verified in-tree):**
|
||||
1. `adapters/local_storage.py` — `__init__(owner_id)`, `run_dir`, +
|
||||
`list_run_ids() -> list[str]` (read-models scan).
|
||||
2. `adapters/storage.py` — protocol unchanged; add `append_run_artifact`
|
||||
(actor uses it, protocol lacks it).
|
||||
3. `engine/run_state.py` — `owner_id` field.
|
||||
4. `engine/actor.py` — owner-scoped storage; `make_run_config(owner_id)`;
|
||||
artifacts at DONE (topic 6).
|
||||
5. `api/app.py` — per-request storage, registry keys, config display =
|
||||
data root, `create_app(..., auth_enabled: bool = True)`.
|
||||
6. `api/dev_bridge.py` — owner + auth off.
|
||||
7. `engine/demo.py` — storage constructor gains owner.
|
||||
8. Tests constructing storage/actor: `test_local_storage.py`
|
||||
(layout assertion + traversal), `test_engine.py`,
|
||||
`test_probe_wiring.py`, `test_adversarial.py`, `test_resume.py`,
|
||||
`test_integration_p1.py`, `test_p0_regressions.py`, `test_api.py` —
|
||||
mechanical `owner_id="u1"`-style constructor args.
|
||||
9. Frontend untouched by partitioning (ids stay run-scoped).
|
||||
|
||||
**Pitfalls:**
|
||||
- `owner_id` must be REQUIRED in `LocalStorageAdapter` — an optional
|
||||
default invites silent unpartitioned use.
|
||||
- `read_run_artifact`'s error message embeds `runs/{run_id}/{name}` —
|
||||
update to owner path; tests may pin it.
|
||||
- Consolidate the duplicate run-id generators (`local_storage` vs
|
||||
`run_config.new_run_id` — different alphabets) when touching both.
|
||||
|
||||
## 6. New persisted artifacts: report.json + evidence.json (D-113)
|
||||
|
||||
**Recommendation:**
|
||||
- Persist both at DONE inside `_verify_phase` (actor.py:315-332), right
|
||||
after `report.md`/`ledger.json` — deterministic, no model calls.
|
||||
- **Marker normalization (frontend-compat crux):** the writer emits
|
||||
`[p-N]` (trust loop regex `\[p-(\d+)\]`); the Reading Room parses
|
||||
`[^pN]` with `[A-Za-z0-9]+` only (`report-body.ts:18` — `[^p-1]` does
|
||||
NOT match). Normalize at persistence: in `report.json` section bodies
|
||||
`[p-3]` → `[^p3]`, ids `p-3` → `p3`. `report.md` keeps the raw form.
|
||||
- **report.json**: `{v, run_id, owner_id, title, question, methodology_slug,
|
||||
generated_at, sections: [{heading, body}], citations: [{passage_id, url,
|
||||
title, quote, health, authority_tier}]}`. Sections split from
|
||||
report.md headings; title = first H1 else `"Deep Report — {question}"`.
|
||||
- **evidence.json**: `{v, run_id, passages: [{passage_id, url, title,
|
||||
quote, text}]}` — serialized from `EvidenceStore` (ids renumbered `pN`).
|
||||
- **Endpoint enrichment:** `GET /api/runs/{id}/report` returns
|
||||
`{run_id, markdown, report: {...}, evidence: {...}}` (markdown retained,
|
||||
back-compat); ledger unchanged. Live `exportReport` switches to real
|
||||
fields — kills the fabricated `claims_supported: 1` and empty quotes
|
||||
(`live-client.ts:329-384`).
|
||||
- Live-client `getReport` maps report.json → frontend `Report`
|
||||
(citations → `Citation` with real quotes; tier map official→1,
|
||||
reference→2, else 3).
|
||||
|
||||
**Pitfalls:**
|
||||
- Do NOT change `report.md`'s marker syntax — `_claims_per_passage`,
|
||||
`extract_report_claims`, `_cited_urls` regex on `\[p-(\d+)\]`.
|
||||
- Old DONE runs lack report.json — the endpoint must fall back to
|
||||
markdown-only gracefully (pre-hosted artifacts disposable per D-106).
|
||||
- `EvidenceStore` is per-actor and GC'd after the run — evidence.json
|
||||
MUST be written inside `_verify_phase` (D-115 disk-reads later).
|
||||
|
||||
## 7. Live read-models on disk artifacts (D-114, D-115)
|
||||
|
||||
**Recommendation:**
|
||||
- Read-models read disk, never RunRegistry (D-115): a small
|
||||
`api/read_models.py` scans `runs/<owner_id>/*/` loading `state.json`
|
||||
(+ report.json/ledger.json where needed).
|
||||
- **Restart sweep:** on `create_app` startup, rewrite every non-terminal
|
||||
`state.json` as CANCELLED (legal from every non-terminal state,
|
||||
verified `states.py`); in-flight actors die with the process;
|
||||
rehydration out of scope (D-115).
|
||||
- **Registry pruning:** `_actors`/`_buses` are never pruned today (slow
|
||||
multi-user leak); drop terminal entries once no live SSE subscriber
|
||||
(refcount in the events route); events 404 after prune/restart is
|
||||
documented, trace persists on disk.
|
||||
- **Open-question semantics (D-114):** open questions = plan
|
||||
sub-questions of the user's runs, answered-iff `state == DONE`; no
|
||||
synthetic suggestions; empty states for new users.
|
||||
- Route shapes (owner-scoped via the auth dependency):
|
||||
- `GET /api/library?sort=recent|title` → folders (auto-categorized by
|
||||
methodology + deterministic topic tokens from plan sub-questions) +
|
||||
`ReportSummary[]`.
|
||||
- `GET /api/library/connections` → shared-source edges from ledger URL
|
||||
overlap (≥1 shared URL); related-topic edges on methodology +
|
||||
category overlap.
|
||||
- `GET /api/graph` → same edges in the frontend `ProjectGraph`
|
||||
node/edge shape.
|
||||
- `GET /api/tasks` → `InboxItem[] {id: "<run_id>#<idx>", question,
|
||||
sourceRunId, status}` (done-iff-run-DONE else open).
|
||||
- `GET /api/suggestions` → open items joined with run titles:
|
||||
`SuggestedQuery[] {id, question, sourceRunId, sourceReportTitle}`.
|
||||
- `POST /api/refine/questions {draft}` → planner call (2-3 clarifying
|
||||
questions); `POST /api/refine/compose {draft, answers}` → planner
|
||||
composes refined question + context. Non-GET ⇒ CSRF header applies.
|
||||
- `GET/PUT /api/profile` → display_name/email.
|
||||
- `GET/PUT /api/config/settings` (D-111): per-user settings
|
||||
(`settings/<user_id>.json`): model names, methodology defaults,
|
||||
masked key status, storage path display-only.
|
||||
- `GET /api/config` stays read-only, user-contextualized:
|
||||
`resolved_tier: user|server|keyless`, `key_store: enabled|disabled`,
|
||||
per-user cost counters (owner-keyed), probe (§4).
|
||||
- Directory scans skip dotfiles/tmp leftovers defensively.
|
||||
- Refine endpoints spend planner tokens — same credential chain; a
|
||||
keyless-only user with no local Ollama gets an honest 503, not a hang.
|
||||
|
||||
## 8. Hosting: backend Dockerfile + two-service deploy (REQ-83)
|
||||
|
||||
**Recommendation:**
|
||||
- `backend/Dockerfile` (uv idiom, non-root uid 10001, single worker):
|
||||
`python:3.12-slim` + uv from ghcr image; `uv sync --frozen --no-dev`;
|
||||
`useradd --system --uid 10001 researchx`; `mkdir -p /var/lib/researchx`
|
||||
+ chown; `USER researchx`;
|
||||
`CMD ["uvicorn", "researchx.api.main:app", "--host", "0.0.0.0",
|
||||
"--port", "8000", "--workers", "1", "--proxy-headers"]`.
|
||||
`--workers 1` is a hard requirement (in-memory registry + sessions).
|
||||
- **New production entrypoint `researchx.api.main`:** today `app.py`'s
|
||||
module-level `app = create_app()` serves **FakeModelAdapter +
|
||||
cwd-based storage** — hosting that would silently ship fakes.
|
||||
`main.py` wires `create_app(data_root=RESEARCHX_DATA_DIR, factory=
|
||||
<live factory from env-file secrets>, auth_enabled=True)`. `app.py`
|
||||
stays the test factory.
|
||||
- Secrets at runtime only: `RESEARCHX_KEK`, `OLLAMA_API_KEY`,
|
||||
`OLLAMA_BASE_URL`, `TAVILY_API_KEY` via `--env-file` — never baked
|
||||
into the image, never in git.
|
||||
- **Plain docker-run pairs, not compose** — matches the existing
|
||||
deploy.sh idiom and avoids a compose dependency on the LXC host:
|
||||
`researchx-net` bridge network; `researchx-api` backend (env-file,
|
||||
data volume, `-p 127.0.0.1:8000:8000` if HAProxy is host-side);
|
||||
`researchx` frontend (`-p 3000:3000`, live build args). Keep
|
||||
`researchx-api-prev` + `researchx-prev` for rollback.
|
||||
- **deploy.sh extension:** rsync now includes `backend/`; build both
|
||||
images on the host; health check BOTH `/healthz` (backend) and `/ask`
|
||||
(frontend); update the HAProxy backend section idempotently.
|
||||
- Data volume: pre-create + `chown 10001` host-side (belt-and-braces).
|
||||
- New pyproject deps: `argon2-cffi>=23.1,<26`, `cryptography>=43`; no
|
||||
`python-multipart` (JSON login bodies only).
|
||||
|
||||
**Pitfalls:**
|
||||
- The volume must be writable by uid 10001 — a root-owned bind mount
|
||||
makes every artifact write EACCES.
|
||||
- uvicorn graceful shutdown default lets in-flight runs exercise
|
||||
`_persist_cancelled` on redeploy; `docker rm -f` SIGKILLs — the
|
||||
startup sweep (§7) is the real safety net.
|
||||
|
||||
## 9. HAProxy SSE configuration (REQ-84)
|
||||
|
||||
**Recommendation:**
|
||||
- The host ALREADY runs HAProxy 2.8 with TLS for research.coreci.dev —
|
||||
this is a front-config extension, not a new component:
|
||||
```
|
||||
frontend https-in
|
||||
acl is_api path_beg /api
|
||||
acl is_health path /healthz
|
||||
use_backend backend_api if is_api or is_health
|
||||
default_backend frontend_next
|
||||
|
||||
backend backend_api
|
||||
option httpchk GET /healthz
|
||||
default-server inter 5s fall 3 rise 2
|
||||
server api1 127.0.0.1:8000 check
|
||||
timeout server 3600s
|
||||
timeout tunnel 3600s
|
||||
|
||||
backend frontend_next
|
||||
timeout server 300s
|
||||
server next1 127.0.0.1:3000 check
|
||||
```
|
||||
- HAProxy 2.x does not buffer response bodies by default in HTTP mode —
|
||||
SSE streams natively. The three buffering hazards: NO
|
||||
`option http-buffer-response`, NO compression on the API backend
|
||||
(gzip buffers text/event-stream), keep default http-keep-alive.
|
||||
- Verified: `sse-starlette` sends `Cache-Control: no-store` +
|
||||
`X-Accel-Buffering: no` and emits `: ping` every **15s**
|
||||
(`DEFAULT_PING_INTERVAL = 15`) — defeats idle timeouts.
|
||||
- `timeout client` in global/defaults must exceed the 15s ping interval.
|
||||
- Health check on `/healthz` (unauthenticated, outside /api per D-103):
|
||||
`GET /healthz → 200 {"status":"ok"}`.
|
||||
|
||||
**Pitfalls:**
|
||||
- Enabling global compression "to save bandwidth" is the classic way SSE
|
||||
silently degrades — scope it off for the API backend.
|
||||
- Don't disable the sse-starlette ping.
|
||||
|
||||
## 10. Next.js build-time env + EventSource credentials (D-112)
|
||||
|
||||
**Recommendation:**
|
||||
- `NEXT_PUBLIC_*` vars are inlined at BUILD time — two build profiles:
|
||||
- **dev/test (default):** no env; mode resolves mock; zero keys
|
||||
(REQ-31 preserved, D-112).
|
||||
- **hosted:** `pnpm build` with
|
||||
`NEXT_PUBLIC_RESEARCHX_API_MODE=live NEXT_PUBLIC_API_BASE_URL=""` —
|
||||
empty base = same-origin through HAProxy's `/api` path route.
|
||||
`baseUrl()` treats `""` as same-origin (openapi-fetch accepts a
|
||||
relative base; EventSource accepts relative URLs). Pin dev live-mode
|
||||
docs to `http://localhost:8000` (§1 hostname rule).
|
||||
- `deploy.sh` frontend build gains the build args; the Dockerfile build
|
||||
stage passes them (`ARG`/`ENV` before `pnpm build`).
|
||||
- EventSource: `new EventSource(url, { withCredentials: true })` — the
|
||||
polyfill supports the same init dict. Same-origin hosted mode doesn't
|
||||
strictly need it; set it anyway.
|
||||
- **Delegation ends (D-112):** `GeneratedApiClient extends MockApiClient`
|
||||
today; in P3 every method is overridden against real endpoints and the
|
||||
inheritance is dropped (a standalone class implementing `ApiClient`);
|
||||
mock remains standalone. OpenAPI snapshot regen (G-005): the new
|
||||
auth/settings/library/refine/admin routes change `openapi.json` —
|
||||
commit the snapshot, regen the generated schema, add the ApiClient
|
||||
interface methods (`login/logout/getSession`, admin hooks) as PLANNED
|
||||
extensions (the v0.2 G-002 lesson: document every addition up front).
|
||||
|
||||
**Pitfalls:**
|
||||
- Forgetting the build args yields a hosted instance silently running
|
||||
mock mode — add a deploy.sh smoke assertion: hosted `/api/config`
|
||||
returns 401 + a live-mode marker.
|
||||
- A non-empty `NEXT_PUBLIC_API_BASE_URL` in the hosted build breaks
|
||||
same-origin cookies — assert `""` at build.
|
||||
|
||||
## 11. Dev/E2E harness under auth
|
||||
|
||||
**Recommendation:**
|
||||
- `create_app(..., auth_enabled: bool = True)`. When False, the guard
|
||||
resolves a synthetic dev user (`user_id="dev00000"`, role="admin") —
|
||||
every route behaves exactly as authenticated, owner-scoped, zero keys.
|
||||
- Existing suites keep running unchanged: backend API tests
|
||||
(`test_api.py`, `test_adversarial.py`) and the real-stack E2E
|
||||
(`dev_bridge.py`) construct with `auth_enabled=False`. Engine/trust
|
||||
suites only need the `owner_id` constructor args (topic 5).
|
||||
- **New auth suite (`test_auth.py`) with `auth_enabled=True`:**
|
||||
first-user-admin (+403 after), wrong password 401, logout revocation,
|
||||
401 on protected routes, CSRF header rejection, cross-user 404,
|
||||
admin create/deactivate (kills sessions), expiry, key-store
|
||||
round-trip + AAD tamper rejection + masked-only responses.
|
||||
|
||||
**Pitfalls:**
|
||||
- Don't gate `auth_enabled` on an env var readable in production images
|
||||
(a baked `RESEARCHX_AUTH=off` violates REQ-85); parameter-only, and
|
||||
`main.py` never passes it.
|
||||
- The E2E custom fetchImpl forwards headers verbatim — once the live
|
||||
client adds the CSRF header, E2E keeps working via GETs + dev-disabled
|
||||
guard; note for the plan, don't build now.
|
||||
|
||||
## 12. Pitfall scan — current code that breaks under multi-user
|
||||
|
||||
1. **`config_status` reads `runs._actors`** (`app.py:174-181`) —
|
||||
multi-user probe leak + zero-run breakage. → factory probe (§4). [P1]
|
||||
2. **Shared `model` instance across runs** (`app.py:152`) — the verified
|
||||
`_active_base` fallback mutation (`ollama.py:73-79`) flips every user
|
||||
to localhost after one cloud 401. → per-run factory (§4). [P1]
|
||||
3. **`app = create_app()` module global** (`app.py:359`) — serves fakes
|
||||
+ cwd storage in production. → `researchx.api.main` (§8). [P2/P4]
|
||||
4. **CORS lacks `allow_credentials`** (`app.py:143-148`) — cookie auth
|
||||
cannot work cross-origin in dev. → §1. [P1]
|
||||
5. **`dev_bridge.py` no auth** — correct for E2E; must stay explicitly
|
||||
`auth_enabled=False`, never the production entry. [verify P1]
|
||||
6. **`RunRegistry` unbounded growth** — prune terminal runs. → §7. [P1]
|
||||
7. **`RunState` no `owner_id`** — cross-user authorization impossible
|
||||
today. → topic 5. [P1]
|
||||
8. **Read endpoints require the registry** (`runs.has(run_id)` guards)
|
||||
— every read dies on restart; a hosted redeploy 404s the whole
|
||||
library. → disk read-models (D-115). [P1/P3]
|
||||
9. **`demo.py`** constructs storage directly — mechanical owner fix. [P1]
|
||||
10. **`test_local_storage.test_run_dir_layout`** asserts unpartitioned
|
||||
path — changes with D-106 (intentional). [P1]
|
||||
11. **Dead code** duplicate `return EventSourceResponse` (app.py:331) —
|
||||
remove while touching the file. [P1]
|
||||
12. **`exportReport` fabricates data** (`live-client.ts:350-389`) —
|
||||
fixed by report.json/evidence.json. [P3]
|
||||
13. **Global `cost_counters`** (`app.py:110`) — per-user (owner-keyed). [P1/P2]
|
||||
14. **Duplicate run-id generators** (different alphabets) — consolidate. [P1]
|
||||
|
||||
## Research Conclusions
|
||||
|
||||
- **C3-01** Auth enforcement = router-level `Depends(require_user)` in
|
||||
`create_app` (public router `/api/auth/*` + `/healthz`; protected the
|
||||
rest), NOT middleware: OpenAPI-visible, dependency_overrides testable,
|
||||
no BaseHTTPMiddleware/SSE interaction.
|
||||
- **C3-02** Sessions: opaque `token_urlsafe(32)` cookies
|
||||
(`researchx_session`, httpOnly, SameSite=Lax, Secure hosted-only),
|
||||
server-side dict + atomic sessions.json, 14-day sliding expiry with
|
||||
>60s persist throttle, token rotation at login, immediate revocation.
|
||||
- **C3-03** argon2id via argon2-cffi `PasswordHasher()` defaults; hash,
|
||||
verify, rehash-on-login all in `asyncio.to_thread`; uniform 401.
|
||||
- **C3-04** CORS: `allow_credentials=True` with explicit localhost
|
||||
origins, added so it wraps auth (last-added = outermost in starlette
|
||||
0.46.2); dev origins must share the hostname `localhost` (port is not
|
||||
part of same-site computation).
|
||||
- **C3-05** CSRF = SameSite=Lax + `X-ResearchX-Request` header on all
|
||||
non-GET /api (checked in the auth dependency); EventSource GET exempt;
|
||||
fetch `credentials:"include"` throughout.
|
||||
- **C3-06** Key store: AES-256-GCM via cryptography AESGCM, KEK =
|
||||
32-byte RESEARCHX_KEK (b64/hex) from the env-file 'keys' scope, AAD =
|
||||
user_id, `os.urandom(12)` nonce per write, `keys/<user_id>.json`
|
||||
(v1/entries format, atomic, 0600); masked-only API; honest disabled
|
||||
status; InvalidTag surfaced.
|
||||
- **C3-07** Per-run model factory: `resolve(user) → (adapter, tier)`,
|
||||
chain user→server→keyless; OllamaModelAdapter gains optional `api_key`
|
||||
constructor param; engine contracts unchanged; probe-per-tier falls
|
||||
out of the existing flow; GET /api/config = current user's tier +
|
||||
fingerprint-cached probe.
|
||||
- **C3-08** Partitioning = owner-scoped `LocalStorageAdapter(base_dir,
|
||||
owner_id)` (protocol unchanged), `runs/<owner_id>/<run_id>/`, owner_id
|
||||
in RunState, registry keyed (owner_id, run_id), per-request adapters;
|
||||
~10 enumerated touchpoints incl. 8 test files + demo.py + dev_bridge.
|
||||
- **C3-09** Data root = RESEARCHX_DATA_DIR (dev backend/var/researchx,
|
||||
container /var/lib/researchx) via a new `paths.data_root()`; the
|
||||
module-level create_app() cwd default + fake adapters are replaced by
|
||||
a production `researchx.api.main` entry.
|
||||
- **C3-10** report.json + evidence.json persisted at DONE in
|
||||
_verify_phase; markers normalized `[p-N]`→`[^pN]`, ids `p-N`→`pN` at
|
||||
the persistence boundary so the Reading Room parses with zero
|
||||
frontend changes; report.md keeps `[p-N]` for the trust loop.
|
||||
- **C3-11** Read-models read disk artifacts only; open questions = plan
|
||||
sub-questions answered-iff DONE; startup sweep marks non-terminal
|
||||
runs CANCELLED; rehydration out of scope.
|
||||
- **C3-12** Registry pruning: drop terminal entries once no SSE
|
||||
subscriber; events 404 after prune/restart documented; trace persists.
|
||||
- **C3-13** Backend container: uv-based non-root Dockerfile (uid 10001,
|
||||
chowned volume), `uvicorn --workers 1` mandatory, runtime-only secrets
|
||||
via --env-file, plain docker-run pairs (no compose dependency),
|
||||
researchx-api-prev rollback, health on both /healthz and /ask.
|
||||
- **C3-14** HAProxy: path route /api + /healthz → backend (2.8 on the
|
||||
LXC host, already TLS-fronting); timeout server/tunnel 3600s, NO
|
||||
http-buffer-response, NO compression on the API backend; sse-starlette
|
||||
15s pings + X-Accel-Buffering keep streams alive; httpchk GET /healthz.
|
||||
- **C3-15** Two build profiles: code default mock; hosted image built
|
||||
with NEXT_PUBLIC_RESEARCHX_API_MODE=live +
|
||||
NEXT_PUBLIC_API_BASE_URL="" (same-origin); deploy smoke asserts live
|
||||
mode; EventSource withCredentials; delegation to MockApiClient ends in
|
||||
P3 (standalone GeneratedApiClient).
|
||||
- **C3-16** Auth harness: create_app(auth_enabled=True) default with a
|
||||
parameter-only dev/test override resolving a synthetic dev user;
|
||||
existing zero-key suites run with it off; dedicated auth suite on.
|
||||
- **C3-17** New pyproject deps: argon2-cffi>=23.1,<26, cryptography>=43;
|
||||
no python-multipart; config.json secrets scopes gain
|
||||
"keys": ["RESEARCHX_KEK"].
|
||||
- **C3-18** ApiClient interface additions are PLANNED (G-002 lesson):
|
||||
login/logout/getSession + admin hooks documented in PLAN before P1;
|
||||
openapi.json snapshot regen is a P1 gate (G-005 discipline carries).
|
||||
- **C3-19** Pitfalls P-1..P-14 (§12) each closed by the phase touching
|
||||
its file: P-1..P-11, P-13, P-14 in P1; P-3 in P2/P4; P-12 in P3;
|
||||
P-5 verified by the auth suite.
|
||||
- **C3-20** All decisions D-101..D-115 confirmed workable against the
|
||||
actual code; refinements folded into PLAN as constraints.
|
||||
+84
-16
@@ -75,37 +75,105 @@ prototype release is **v0.1.8** (final phase patch). Milestone branch
|
||||
merged to main and deleted — tags v0.1.0–v0.1.8 preserve the full
|
||||
history. Next milestone: v0.2 MVP (unblocked).
|
||||
|
||||
## Milestone v0.2 — MVP (feature) — BLOCKED until prototype approval
|
||||
## Milestone v0.2 — MVP (feature) — COMPLETE (approved 2026-09-27)
|
||||
|
||||
Phases renumbered from the original v1.0 plan (1-based within the milestone):
|
||||
Phases renumbered from the original v1.0 plan (1-based within the milestone). Tags run on the v0.2.x line; MAJOR stays 0.
|
||||
|
||||
### Phase 0 — Pre-Execution (v0.2)
|
||||
- **Description**: Milestone pre-execution re-run: PLAN.md refresh against approved prototype decisions (layout learnings, API shape implied by the ApiClient interface), research on any gaps surfaced by approval feedback.
|
||||
- **Status**: blocked (gate: v0.1.4)
|
||||
- **Description**: Milestone pre-execution: SPECIFY (spec validated for v0.2), CLARIFY (D-007..D-020: backend layout, uv+Python 3.12, ports/run-IDs, DDG fallback, openapi-typescript client, SSE+pydantic events, difflib anchoring, liveness rules, durable JSON state, owner-gate halt, no-deploy policy, RESEARCHX_API_MODE), RESEARCH (RESEARCH-v02.md — 8 topics, 14 conclusions, 4 D-refinements), PLAN (P1 vertical-slice plan), GRILL (mechanical nine-axis review), MVP/UX check.
|
||||
- **Requirements covered**: none (planning artifacts)
|
||||
- **Success criteria**: .ciagent/ files refreshed for v0.2; PLAN.md carries User-Facing Surface + Happy Path + UX Acceptance Criteria; PERSONAS.md v0.2 roster active; grill verdict recorded with binding decisions
|
||||
- **Depends on**: v0.1 approval (recorded)
|
||||
- **Status**: complete (PR #9 squash-merged to milestone/v0.2-mvp bbf80ce, tag v0.2.0, release created HTTP 201, phase branch deleted; grill Proceed 0.72 with G-001..G-010 promoted; MVP/UX PASS)
|
||||
|
||||
### Phase 1 — Methodology Framework + Adapter Layer
|
||||
- **Description**: Dial schema (12 dials, typed, safe defaults), methodology registry with Rapid Answer + Deep Report, adapter contracts (model/search/storage/memory/secrets) with local implementations, capability probing with conservative caps (8192 context, embedding batch 8, Ollama cloud-first + local fallback), fake model/search adapters.
|
||||
- **Requirements covered**: REQ-01..06, REQ-08..11, REQ-31
|
||||
- **Status**: blocked
|
||||
- **Description**: Dial schema (12 dials, typed, safe defaults), methodology registry with Rapid Answer + Deep Report, adapter contracts (model/search/storage/memory/secrets) with local implementations, capability probing with conservative caps (8192 context, embedding batch 8, Ollama cloud-first + local fallback), fake model/search adapters. Full task waves in PLAN.md.
|
||||
- **Requirements covered**: REQ-01..06, REQ-08..11, REQ-31 (REQ-07, REQ-12 ship in P2 per ROADMAP phase mapping — dial-level + warning contracts are proven here, live probing against real endpoints in P2)
|
||||
- **Success criteria**: uv one-command backend; gates green zero-API-key; registry resolves slugs; dial snapshot round-trips; fakes drive full interface; degradation typed + honest
|
||||
- **Depends on**: 0
|
||||
- **Status**: complete (PR #10 squash-merged to milestone/v0.2-mvp dfb738d, tag v0.2.1, release HTTP 201, phase branch deleted; verify PASS 0 P0 / 2 P1 fixed in-phase / 9 P2 flagged; REQ-07/12 contracts proven with fakes per G-001, live probing in P2)
|
||||
|
||||
### Phase 2 — Research Engine + Trust Loop
|
||||
### Phase 2 — Research Engine + Trust Loop (complete — shipped v0.2.2)
|
||||
- **Description**: Asyncio state machine with durable JSON state and pause/cancel/resume; plan approval gate; typed event bus → SSE; evidence store with stable passage IDs; trust loop (visited-only constraint, URL liveness pass, quote anchoring, citation ledger, verification checklist, confidence labels).
|
||||
- **Requirements covered**: REQ-07, REQ-12, REQ-13..24
|
||||
- **Status**: blocked
|
||||
- **Status**: complete (PR #11 squash-merged 558f21a, tag v0.2.2, release HTTP 201, phase branch deleted; verify found 2 P0 + 9 P1, all fixed in-phase with regression pins; SSE transport itself lands in P3 — the bus + trace artifacts shipped here)
|
||||
|
||||
### Phase 3 — API Layer + Integration
|
||||
### Phase 3 — API Layer + Integration (complete — shipped v0.2.3)
|
||||
- **Description**: FastAPI routes (runs, methodologies, lifecycle, SSE events, config status); OpenAPI-generated typed client; **swap MockApiClient → generated client** (the ApiClient interface absorbs any prototype-era API drift); ETA/cost heuristics for methodology cards.
|
||||
- **Requirements covered**: REQ-25, REQ-26
|
||||
- **Status**: blocked
|
||||
- **Status**: complete (PR #12 squash-merged 75479a9, tag v0.2.3, release HTTP 201, phase branch deleted; verify found 1 P0 mode-flag + 4 P1, all fixed in-phase; REQ-33 one-command story completed here per plan)
|
||||
|
||||
### Phase 4 — Hardening + Adversarial Suite + E2E
|
||||
- **Description**: Adversarial test suite (hallucinated-URL writer, quote-anchor mismatch, pause/resume mid-depth-loop), E2E happy-path (Playwright), monorepo tooling polish (one-command install/dev, lint/typecheck/test green).
|
||||
### Phase 4 — Hardening + Adversarial Suite + E2E (complete — shipped v0.2.4)
|
||||
- **Description**: Adversarial test suite (hallucinated-URL writer, quote-anchor mismatch, pause/resume mid-depth-loop), E2E happy-path (real stack via live binding + dev bridge), monorepo tooling polish (one-command install/dev, lint/typecheck/test green).
|
||||
- **Requirements covered**: REQ-17, REQ-32, REQ-33, REQ-34
|
||||
- **Status**: blocked
|
||||
- **Status**: complete (PR #13 squash-merged dcf2475, tag v0.2.4, release HTTP 201, phase branch deleted; verify found 5 P1 honesty gaps in the suite itself — export gate anchored, weak band pinned, fabricated reports fail export; E2E validated on the real stack, defaults only)
|
||||
|
||||
### Phase 5 — Final Review + Ship (FINAL)
|
||||
- **Description**: Review + audit + milestone ship; final patch tag IS the MVP release. MAJOR stays 0 — the MVP ships as a 0.x release. v1.0.0 comes only after an owner-approved release decision.
|
||||
- **Status**: blocked
|
||||
- **Status**: complete (PR #14 squash-merged 023667e, tag v0.2.5, release HTTP 201, phase branch deleted; review 4 P0 + 15 P1 — P0s attack-proven fixed; audit PASS-WITH-NOTES — criticals fixed; **owner approval RECORDED 2026-09-27** — decision commit 4721c2a, G-009 scope ratified; milestone merged to main, v0.2.5 = the MVP release)
|
||||
|
||||
## Milestone v0.2 — COMPLETE (approved 2026-09-27)
|
||||
|
||||
All requirements complete: REQ-01..26, REQ-31..34 (the MVP set). The
|
||||
approved MVP release is **v0.2.5** (final phase patch). Delivered: the
|
||||
FastAPI research engine (asyncio state machine, durable
|
||||
pause/cancel/resume), methodology framework (12 dials, Rapid Answer +
|
||||
Deep Report), five-adapter layer (Ollama cloud-first fallback,
|
||||
Tavily/DDG, storage, memory, secrets), the attack-proven trust loop
|
||||
(real quote anchoring, cited-scoped ≥99% liveness export gate, SSRF-
|
||||
guarded fetching, citation ledger, verification checklist), SSE live
|
||||
trace, OpenAPI-generated typed client behind the unchanged ApiClient
|
||||
boundary, a 14-attack adversarial suite, and the 3-click E2E on the
|
||||
real stack. G-009 ratified: live mode = core research loop only;
|
||||
library/settings/suggestions/refine/graph stay mock-backed until
|
||||
v0.3+. Milestone branch merged to main and deleted — tags
|
||||
v0.2.0–v0.2.5 preserve the full history. Next: v0.3 planning.
|
||||
|
||||
## Milestone v0.3 — Hosted (feature) — owner-directed 2026-09-29
|
||||
|
||||
Scope (owner session 2026-09-29): one milestone combining the G-009-ratified live surfaces + hosting with three owner additions — auth/sessions (user/pass, admin-managed, OAuth deferred), per-user model API keys, and real inference as the product default. Tags run on the v0.3.x line; MAJOR stays 0; feature milestone — each phase ships a patch, the final phase's patch IS the milestone release.
|
||||
|
||||
### Phase 0 — Pre-Execution (v0.3)
|
||||
- **Description**: SPECIFY (v0.3 requirement set REQ-69..85; single-user constraint superseded), CLARIFY (auth/session mechanics, key-encryption, partitioning, mock's fate, Reading-Room fidelity), RESEARCH (FastAPI session/auth patterns, encrypted key stores, SSE-proxying, storage read-model), PLAN (vertical slices), GRILL, MVP/UX check.
|
||||
- **Requirements covered**: none (planning artifacts)
|
||||
- **Success criteria**: .ciagent/ files refreshed for v0.3; PLAN.md carries User-Facing Surface + Happy Path + UX Acceptance Criteria; PERSONAS.md v0.3 roster active; grill verdict recorded with binding decisions
|
||||
- **Depends on**: v0.2 approval (recorded)
|
||||
- **Status**: complete (PR #16 squash-merged to milestone/v0.3-hosted 1adfa87, tag v0.3.0, release created; D-101..D-115 clarified; RESEARCH-v03 with C3-01..C3-20; PLAN P1..P4 vertical slices; grill Proceed 0.68 with G-101..G-110 bound, G-111 escalated; MVP/UX PASS)
|
||||
|
||||
### Phase 1 — Auth + Sessions + User Scoping
|
||||
- **Description**: User model (argon2id), register/login/logout/me, httpOnly cookie sessions (server-side store), first-user-is-admin + admin-managed accounts (REQ-70), auth guard on every API endpoint, runs/settings partitioned by owner (engine + storage paths). Frontend: login screen, real profile in user menu, logout.
|
||||
- **Requirements covered**: REQ-69, REQ-70, REQ-71
|
||||
- **Success criteria**: unauthenticated API access returns 401; two users cannot see each other's runs; admin can create/deactivate accounts; existing engine tests green under an authenticated harness
|
||||
- **Depends on**: 0
|
||||
- **Status**: complete (PR #17 squash-merged to milestone/v0.3-hosted 2a61433, tag v0.3.1, release HTTP 201, phase branch deleted; verify PASS 4/4 — 184 backend + 119 frontend tests; security P0 register race + P1 corrupt-store fail-closed fixed in-phase; D-116 rate-limit decision recorded)
|
||||
|
||||
### Phase 2 — Per-User Settings + Model Keys
|
||||
- **Description**: Encrypted per-user key store (KEK from env-file secrets scope), per-user settings persistence (PUT/GET /api/config), per-run credential resolution chain (user key → server key → keyless localhost; REQ-74) with probe per resolved key, settings UI live-bound with masked keys.
|
||||
- **Requirements covered**: REQ-72, REQ-73, REQ-74
|
||||
- **Success criteria**: a keyed user gets fully real inference end-to-end; keys never appear in API responses (masked only), logs, or traces; resolution tier surfaced in config status; zero-key tests stay green via REQ-31 fakes
|
||||
- **Depends on**: 1
|
||||
- **Status**: complete (PR #18 squash-merged to milestone/v0.3-hosted ee26ff1, tag v0.3.2, release HTTP 201, phase branch deleted; verify PASS-WITH-FIXES — P0 settings null-poisoning fixed in-phase; 264 backend + 131 frontend tests)
|
||||
|
||||
### Phase 3 — Live Surfaces (user-scoped)
|
||||
- **Description**: Library read-model (GET /api/library), connections + graph (GET /api/library/connections, GET /api/graph), task inbox (GET /api/tasks), suggestions (GET /api/suggestions), refine (POST /api/refine/*, real planner calls), profile (GET/PUT /api/profile), live Reading Room fidelity (structured report, anchored citations, ledger panel). G-002 boundary lifted; frontend live swaps with zero component rewrites (REQ-54).
|
||||
- **Requirements covered**: REQ-76, REQ-77, REQ-78, REQ-79, REQ-80, REQ-81, REQ-82
|
||||
- **Success criteria**: every prototype surface works against real user data in live mode; suggestions/refine driven by real model calls; Reading Room parity with mock; ≤3-click path still green
|
||||
- **Depends on**: 2
|
||||
- **Status**: complete (PR #19 squash-merged to milestone/v0.3-hosted b485fc0, tag v0.3.3, release HTTP 201, phase branch deleted; verify PASS-WITH-FIXES — CSRF test pin P0 fixed; G-108 valve not needed; 314 backend + 164 frontend tests)
|
||||
|
||||
### Phase 4 — Hosting + Real-Inference Default
|
||||
- **Description**: Backend Dockerfile (single worker, non-root), two-service deployment via deploy.sh extension, SSE-safe HAProxy path routing (/api → backend, streaming timeouts), hosted instance runs live mode by default (mock demoted to dev/test fixture), auth guard verified through the public URL.
|
||||
- **Requirements covered**: REQ-75, REQ-83, REQ-84, REQ-85
|
||||
- **Success criteria**: one-command deploy brings up both services; SSE streams unbuffered through the proxy; unauthenticated curl to the public /api returns 401; live-mode default proven on the staged instance
|
||||
- **Depends on**: 3
|
||||
- **Status**: pending
|
||||
|
||||
### Phase 5 — Final Review + Ship (FINAL)
|
||||
- **Description**: Multi-persona review of the full milestone diff, audit, tag v0.3.5 = milestone release; pipeline HALTS at the merge_to_main owner approval gate (D-018 carries); deployment to research.coreci.dev only on owner directive (D-019 carries).
|
||||
- **Requirements covered**: milestone gate (REQ-69..85)
|
||||
- **Success criteria**: review PASS (P0s fixed in-phase); audit PASS; v0.3.5 tagged + release; pipeline halted for the recorded owner decision
|
||||
- **Depends on**: 4
|
||||
- **Status**: pending
|
||||
|
||||
## Milestone Releases
|
||||
- **v0.1.4 = extended-prototype candidate** (superseded by the P5/P6 extension cycle)
|
||||
@@ -113,7 +181,7 @@ Phases renumbered from the original v1.0 plan (1-based within the milestone):
|
||||
- **v0.1.6 = extended-prototype candidate** (superseded by the P7/P8 extension cycle)
|
||||
- **v0.1.7 = ask suggestions + refine dialogue** (P7 patch)
|
||||
- **v0.1.8 = milestone release** (P8 final phase patch — the approved extended prototype release, pending owner decision)
|
||||
- **v0.2.x = MVP release** (MAJOR still 0; ship only if all v0.2 phases complete)
|
||||
- **v0.2.5 = milestone release** (P5 final phase patch — the approved MVP; owner approval recorded 2026-09-27, G-009 ratified)
|
||||
- If MVP scope extends: additional 0.x minor milestones (v0.3, …), each a full milestone cycle
|
||||
- **v1.0.0**: owner-approved release only — not created by the pipeline autonomously
|
||||
|
||||
|
||||
+25
-10
@@ -2,18 +2,23 @@
|
||||
"ci_version": "0.7.0",
|
||||
"projects": [],
|
||||
"active_project": "researchx",
|
||||
"milestone": "v0.2",
|
||||
"milestone_name": "mvp",
|
||||
"milestone": "v0.3",
|
||||
"milestone_name": "hosted",
|
||||
"milestone_type": "feature",
|
||||
"versioning": {
|
||||
"major_locked": true,
|
||||
"major_lock_note": "MAJOR remains 0 until an owner-approved v1 release; the MVP may span multiple 0.x minor milestones; all pre-v1 ships are MINOR/PATCH only",
|
||||
"tag_line": "v0.2.x",
|
||||
"next_milestone": "v0.3+ (post-MVP backlog milestones)"
|
||||
"major_lock_note": "MAJOR remains 0 until an owner-approved v1 release; all pre-v1 ships are MINOR/PATCH only",
|
||||
"tag_line": "v0.3.x",
|
||||
"next_milestone": "v0.4 candidates (owner-directed post-v0.3): remaining 6 methodologies (REQ-35..40), OAuth deferred from v0.3 auth, Exa/Perplexity search adapters, S3/git storage, memory scopes"
|
||||
},
|
||||
"autonomy": {
|
||||
"level": "full",
|
||||
"escalation_hooks": ["deploy", "delete_data", "merge_to_main", "prototype_approval"],
|
||||
"escalation_hooks": [
|
||||
"deploy",
|
||||
"delete_data",
|
||||
"merge_to_main",
|
||||
"prototype_approval"
|
||||
],
|
||||
"decision_confidence_threshold": 0.6,
|
||||
"max_revision_iterations": 3,
|
||||
"clarify_budget": 10
|
||||
@@ -37,9 +42,19 @@
|
||||
"secrets": {
|
||||
"env_file": ".ciagent/.env.secrets",
|
||||
"scopes": {
|
||||
"release": ["GITEA_TOKEN"],
|
||||
"model": ["OLLAMA_API_KEY", "OLLAMA_BASE_URL"],
|
||||
"search": ["TAVILY_API_KEY"]
|
||||
"release": [
|
||||
"GITEA_TOKEN"
|
||||
],
|
||||
"model": [
|
||||
"OLLAMA_API_KEY",
|
||||
"OLLAMA_BASE_URL"
|
||||
],
|
||||
"search": [
|
||||
"TAVILY_API_KEY"
|
||||
],
|
||||
"keys": [
|
||||
"RESEARCHX_KEK"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Deploy Record — UAT stagings of the v0.1 prototype
|
||||
# Deploy Record — UAT stagings and releases
|
||||
|
||||
Owner-directed UAT deployments, recorded per commit discipline. The
|
||||
approval gate (REQ-62) stays closed; UAT variants stage review builds
|
||||
without minting release tags.
|
||||
Owner-directed deployments, recorded per commit discipline. Release
|
||||
deploys carry the owner approval (v0.1.8); UAT variants stage review
|
||||
builds without minting release tags.
|
||||
|
||||
| Date | Version | Commit | Directive | Result |
|
||||
|---|---|---|---|---|
|
||||
@@ -11,6 +11,8 @@ without minting release tags.
|
||||
| 2026-09-27 | v0.1.7-uat | 432d2c0 (P7 ship + boundary docs) | owner cadence: deploy after P7 and P8 | healthy — http://10.0.0.114:3000 |
|
||||
| 2026-09-27 | v0.1.8-uat | 93a7b57 (P8 ship — release candidate) | owner cadence: deploy after P7 and P8 | healthy — http://10.0.0.114:3000 |
|
||||
| 2026-09-27 | **v0.1.8 (release)** | 93a7b57 (tagged) | **owner approval recorded — milestone release published** | healthy — http://10.0.0.114:3000 |
|
||||
| 2026-09-27 | v0.2.5-uat | c28a97e (v0.2 milestone HEAD + v0.2.5 version strings) | owner: "deploy and validate against https://research.coreci.dev" | healthy — https://research.coreci.dev (HAProxy/SSL) |
|
||||
| 2026-09-27 | **v0.2.5 (release)** | b11b202 (main merge; tag v0.2.5) | **owner approval recorded (/ci-run APPROVED) — milestone release published; G-009 ratified** | serving — same container (v0.2.5-uat build = release content); no redeploy needed |
|
||||
|
||||
## v0.1.6-uat (extended prototype for owner review)
|
||||
|
||||
@@ -24,4 +26,25 @@ without minting release tags.
|
||||
- Rollback: `ssh root@10.0.0.114 'docker stop researchx; docker rename
|
||||
researchx-prev researchx; docker start researchx'`
|
||||
- The approval gate remains closed: milestone merge to main still halts
|
||||
for the recorded owner decision (REQ-62)
|
||||
for the recorded owner decision (REQ-62)
|
||||
|
||||
## v0.2.5-uat (MVP release candidate, owner-directed)
|
||||
|
||||
- Deployed via `bash .ciagent/release/deploy.sh v0.2.5-uat` — the v0.2
|
||||
milestone HEAD (c28a97e: tag v0.2.5 + v0.2.5 About/package.json
|
||||
version strings), per the owner's direct request to deploy and
|
||||
validate against the public URL
|
||||
- Gates re-run at deploy time from the milestone HEAD: typecheck,
|
||||
lint, test (97/97), build — green before sync
|
||||
- Image `researchx:v0.2.5-uat` + `latest` on the host; container
|
||||
swapped, `researchx-prev` (v0.1.8) kept for rollback
|
||||
- Health: 2 checks; validated through the HAProxy SSL front:
|
||||
https://research.coreci.dev — /ask /library /settings /choose all
|
||||
200, root 307→/ask, TLS verified; served settings chunk carries
|
||||
"v0.2.5 · MVP release candidate"; the RESEARCHX_API_MODE switch is
|
||||
present in the served bundle (mock default — correct for a
|
||||
frontend-only container; live mode additionally needs the backend
|
||||
API on the host, which is v0.3+ deployment work)
|
||||
- The merge gate (D-018) remains closed: milestone → main still halts
|
||||
for the recorded owner decision, with G-009 (live-mode scope)
|
||||
presented at the same gate
|
||||
@@ -0,0 +1,87 @@
|
||||
# UAT Script — v0.2.5 (MVP release candidate)
|
||||
|
||||
Deployment: https://research.coreci.dev (HAProxy → LXC container
|
||||
`researchx:v0.2.5-uat`, commit c28a97e, mock-backed frontend).
|
||||
|
||||
Scope note (G-009): this container serves the v0.2.5 frontend with
|
||||
scripted mock data. The v0.2 engine (FastAPI, adapters, trust loop)
|
||||
is validated by the automated suites (129 backend + 97 frontend
|
||||
tests, 14 adversarial attacks, real-stack E2E) and runs via the
|
||||
README live-mode locally; hosting the backend is v0.3+ work. This
|
||||
UAT validates the release-candidate build of the product surface.
|
||||
|
||||
## 1. Shell & navigation (REQ-63, REQ-66)
|
||||
|
||||
1. Open https://research.coreci.dev → lands on **/ask** inside the app shell
|
||||
2. Sidebar: **New research / Library / Settings** nav present on every screen
|
||||
3. Top bar: press **⌘K** → navigation palette (screens + stored reports)
|
||||
4. Click the **avatar** (top right) → user menu: local profile, Settings, About
|
||||
5. Pass: shell persistent everywhere, honest preview-marked future slots
|
||||
|
||||
## 2. Version check (this deploy)
|
||||
|
||||
1. Sidebar → **Settings** → scroll to *About & version*
|
||||
2. Pass: reads **"v0.2.5 · MVP release candidate"** (confirms the deployed build)
|
||||
|
||||
## 3. The ≤3-click happy path (REQ-61 / REQ-17 — core criterion)
|
||||
|
||||
1. **/ask**: type a question → click **Research this** (click 1)
|
||||
2. **/choose**: pick a card (Deep Report is the recommended default) → **Start research** (click 2)
|
||||
3. **Plan & Run**: review the plan → **Approve plan** (click 3)
|
||||
4. Pass: exactly 3 clicks, defaults only, research running with a live trace
|
||||
|
||||
## 4. Plan & Run (REQ-57)
|
||||
|
||||
1. Plan approval UI shows sub-questions, source universe, outline
|
||||
2. Watch the live trace stream (scripted replay)
|
||||
3. Press **pause** → trace halts; **resume** → continues; **cancel** → terminal state
|
||||
4. Pass: all three controls behave; no lost events
|
||||
|
||||
## 5. Reading Room + exports (REQ-58, REQ-59, REQ-34)
|
||||
|
||||
1. Open the finished report from the run screen
|
||||
2. Hover/focus any **[n]** citation marker → the exact quoted passage appears
|
||||
3. Open the **citation ledger** panel: URL, quote, health, authority tier
|
||||
4. **Export Markdown** and **Export JSON** → real files download; citations survive as links
|
||||
5. Pass: quotes match exactly; files open; links intact
|
||||
|
||||
## 6. Ask intelligence (REQ-67, REQ-68)
|
||||
|
||||
1. **New research** → below the form: open-question suggestions appear
|
||||
2. Click one → the question box fills; **never auto-starts**
|
||||
3. Type a draft → **Refine question** → answer the 3 clarifying questions → **Use this** → refined question + context fill the form
|
||||
4. Try **Keep original** on a second draft → untouched; keyboard (Enter) works through both flows
|
||||
5. Pass: both helpers are opt-in, keyboard-accessible, drafts preserved on dismiss
|
||||
|
||||
## 7. Library (REQ-64)
|
||||
|
||||
1. Sidebar → **Library**: folder tree, auto-categorized (topic + methodology), sortable (recency/title)
|
||||
2. Open a stored report → its own Reading Room view
|
||||
3. **Connections** panel: shared-source ⇄ and topical → edges
|
||||
4. v2 previews: project graph + task inbox panels render
|
||||
5. Pass: folders, sorting, connections, previews all render from the fixture data
|
||||
|
||||
## 8. Settings (REQ-65)
|
||||
|
||||
1. API keys: masked (never raw); storage directory field
|
||||
2. Methodology defaults: resolved 12-dial display
|
||||
3. Appearance preview; About & version (step 2)
|
||||
4. Pass: everything honest and functional on mock data
|
||||
|
||||
## Optional: hands-on the live engine (not on this URL)
|
||||
|
||||
```bash
|
||||
# locally, two terminals:
|
||||
cd backend && uv sync && uv run uvicorn researchx.api.app:app --port 8000
|
||||
cd frontend && NEXT_PUBLIC_RESEARCHX_API_MODE=live corepack pnpm dev
|
||||
# then repeat steps 3-5 against the real engine (fake adapters, zero keys)
|
||||
```
|
||||
|
||||
## Decision rubric (the gate this UAT feeds)
|
||||
|
||||
- **Approve**: recorded via /ci-run → milestone merges to main,
|
||||
v0.2.5 publishes as the MVP release, v0.3 planning unblocks
|
||||
(with G-009 scope ratified: library/settings/suggestions/refine/
|
||||
graph stay mock-backed until v0.3+)
|
||||
- **Request changes**: feedback becomes binding decisions for a
|
||||
fix cycle (v0.1 precedent: two feedback cycles before approval)
|
||||
@@ -9,7 +9,7 @@ as established by the CIAgent branch strategy and `config.json`.
|
||||
| Milestone | Tags | Deployed by |
|
||||
|---|---|---|
|
||||
| v0.1 prototype | v0.1.0 … v0.1.8 | `bash .ciagent/release/deploy.sh v0.1.8` |
|
||||
| v0.2 MVP | v0.2.0 … | `bash .ciagent/release/deploy.sh v0.2.x` |
|
||||
| v0.2 MVP | v0.2.0 … v0.2.5 | `bash .ciagent/release/deploy.sh v0.2.5-uat` (staged) — see DEPLOY-RECORD.md |
|
||||
|
||||
## Rules
|
||||
|
||||
|
||||
@@ -2,20 +2,99 @@
|
||||
|
||||
A deep-research studio where you bring your own model, storage, and memory; choose a research methodology before a single query fires; and every claim in the final report is anchored to a visited, verified, quotable source.
|
||||
|
||||
**Status: prototype (milestone v0.1)** — UI/UX with mock data. No backend, no API keys, no network calls.
|
||||
**Status: MVP shipped (v0.2.5, owner-approved 2026-09-27)** — the full research engine (FastAPI + asyncio state machine + adapter layer + attack-proven trust loop) behind the v0.1 frontend. Live mode covers the core research loop (G-002/G-009): the engine runs locally per the quickstart; library/settings/suggestions/refine stay mock-backed until v0.3+.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
# frontend (prototype, mock-backed — works today)
|
||||
cd frontend
|
||||
pnpm install
|
||||
pnpm dev
|
||||
corepack pnpm install
|
||||
corepack pnpm dev # http://localhost:3000
|
||||
|
||||
# backend (MVP — uv manages Python 3.12 and the venv)
|
||||
cd backend
|
||||
uv sync # installs deps + editable researchx package
|
||||
uv run uvicorn researchx.api.app:app --reload --port 8000
|
||||
uv run pytest # full suite, zero API keys
|
||||
```
|
||||
|
||||
Open http://localhost:3000 — you land on **Ask**, inside the app shell (left sidebar, top bar with ⌘K search and your avatar). Type a question, click **Research this**, pick a methodology on **Choose** (Deep Report is the recommended default), and press **Start research**.
|
||||
|
||||
Three clicks from cold start to initiated research.
|
||||
|
||||
## Monorepo command map
|
||||
|
||||
| Layer | Install | Dev | Gates |
|
||||
|---|---|---|---|
|
||||
| backend (uv, Python 3.12) | `uv sync` | `uv run uvicorn researchx.api.app:app --reload --port 8000` | `uv run ruff check src tests` · `uv run mypy src` · `uv run pytest` |
|
||||
| frontend (pnpm 12.6) | `corepack pnpm install` | `corepack pnpm dev` | `corepack pnpm typecheck` · `corepack pnpm lint` · `corepack pnpm test` · `corepack pnpm build` |
|
||||
|
||||
## Engine architecture (v0.2 P2)
|
||||
|
||||
The research engine is a hand-rolled asyncio state machine:
|
||||
|
||||
```
|
||||
PLANNING → AWAITING_APPROVAL → SEARCHING → EXTRACTING → REFLECTING ─┐
|
||||
(halt until ↑ depth loop (dials.depth) │
|
||||
approved) └────────────────────────────┘
|
||||
→ SYNTHESIZING → VERIFYING → DONE (+ CANCELLED / FAILED terminals)
|
||||
```
|
||||
|
||||
- Durable state: `runs/<id>/state.json` rewritten atomically after every transition; resume re-enters at the recorded cursor with the persisted plan
|
||||
- Event bus: typed events (`run_started`, `query_fired`, `source_visited`, `passage_extracted`, `state_changed`, `run_done`, `run_error`) — trace per run
|
||||
- Visited-only constraint: the writer's context is the evidence store and nothing else
|
||||
- Trust loop: SSRF-guarded liveness (live/stale/dead), quote anchoring (normalize + fuzzy ≥ 0.85, weak matches flagged), citation ledger JSON, exportable verification checklist, single-source vs corroborated confidence labels
|
||||
|
||||
Run artifacts per run: `state.json`, `report.md`, `ledger.json`, `checklist.md`.
|
||||
|
||||
Try it on fakes (no keys): `uv run python -m researchx.engine.demo`
|
||||
|
||||
## Adversarial guarantees (v0.2 P4)
|
||||
|
||||
Every trust-loop claim is attacked by the adversarial suite (`backend/tests/test_adversarial.py`) — these are tests that try to fool the verifier, and their passing proves the defenses:
|
||||
|
||||
| Attack | Defense (proven) |
|
||||
|---|---|
|
||||
| Writer hallucinates a citation to a passage that was never stored | Unknown passage IDs count as **unanchored claims** in the ledger; the export gate fails the report |
|
||||
| Fabricated quote (matches nothing) | Anchoring verdict: no match, flagged — never silently accepted |
|
||||
| Near-miss quote (0.80–0.85 similarity) | Flagged `weak_match` in the ledger — surfaced, not accepted |
|
||||
| Liveness probe redirected to a private/metadata IP | Per-hop redirect re-guarding classifies it **dead** (SSRF refusal) |
|
||||
| Redirect chain ending at a gone page (410) | Classified **dead** |
|
||||
| Oversized page (5MB) response | 2MB read cap — classifies without unbounded memory |
|
||||
| Pause during the depth loop | Durable state persists; resume completes the run over the HTTP API |
|
||||
| Cancel mid-loop | Terminal CANCELLED state persisted, never swallowed |
|
||||
|
||||
REQ-23 (≥99% citation liveness) is a per-run **export gate** enforced by the verification checklist, plus an opt-in keyed live tier (`cd backend && uv run pytest -m live`) — the zero-key CI suite never asserts it vacuously.
|
||||
|
||||
The E2E happy path (≤3 clicks: ask, choose, approve) is validated against the real FastAPI + engine stack, defaults only — not the mock (`frontend/src/lib/api/e2e-real-stack.test.ts`).
|
||||
|
||||
## Adapter configuration (v0.2)
|
||||
|
||||
All keys resolve from env files only (`.env`, `.env.secrets`, `.env.*` — never the shell environment, never git):
|
||||
|
||||
| Variable | Scope | Used by |
|
||||
|---|---|---|
|
||||
| `OLLAMA_API_KEY` | model (cloud) | Ollama cloud endpoint `https://ollama.com/v1`; absent/keyless falls back to `http://localhost:11434/v1` automatically |
|
||||
| `OLLAMA_BASE_URL` | model | Overrides the default cloud base URL |
|
||||
| `TAVILY_API_KEY` | search | Tavily default search; absent falls back to the keyless DuckDuckGo adapter (degraded mode — surfaced in the run trace) |
|
||||
|
||||
The test suite needs **no** keys: fake model/search adapters (REQ-31) drive every engine test.
|
||||
|
||||
## Live mode (v0.2 P3)
|
||||
|
||||
The frontend runs mock-backed by default (works standalone, zero keys). To run the core research loop against the real backend:
|
||||
|
||||
```bash
|
||||
# terminal 1 — backend (fake adapters by default; real Ollama/Tavily adapters
|
||||
# are used by the engine in v0.2.5+ wiring — the served API currently ships
|
||||
# fake-adapter runs for demos and tests; see Adapter configuration below)
|
||||
cd backend && uv run uvicorn researchx.api.app:app --reload --port 8000
|
||||
|
||||
# terminal 2 — frontend in live mode
|
||||
cd frontend && NEXT_PUBLIC_RESEARCHX_API_MODE=live corepack pnpm dev
|
||||
```
|
||||
|
||||
Live mode covers the core loop (Ask → Choose → approve plan → live trace → report + ledger); the library, settings persistence, suggestions, refine, and graph surfaces stay mock-backed until v0.3. The API surface: `POST /api/runs`, `GET /api/methodologies`, run lifecycle (`approve/pause/cancel/resume`), `GET /api/runs/{id}/events` (SSE), `GET /api/config`, report + ledger artifacts.
|
||||
|
||||
## What's in the prototype
|
||||
|
||||
| Screen | What it does |
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
.venv/
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
.mypy_cache/
|
||||
var/
|
||||
@@ -0,0 +1 @@
|
||||
3.12
|
||||
@@ -0,0 +1,50 @@
|
||||
[project]
|
||||
name = "researchx"
|
||||
version = "0.2.0"
|
||||
description = "ResearchX — methodology-first deep research studio"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"fastapi>=0.115,<0.116",
|
||||
"pydantic>=2.8,<3",
|
||||
"uvicorn>=0.30,<1",
|
||||
"sse-starlette>=2.1,<3",
|
||||
"httpx>=0.27,<1",
|
||||
"argon2-cffi>=23.1,<26",
|
||||
"cryptography>=43",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"ruff>=0.6",
|
||||
"mypy>=1.11",
|
||||
"pytest>=8",
|
||||
"pytest-asyncio>=0.23",
|
||||
"httpx>=0.28.1",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/researchx"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
|
||||
[tool.ruff.lint]
|
||||
extend-select = ["I", "UP"]
|
||||
|
||||
[tool.mypy]
|
||||
strict = true
|
||||
mypy_path = "src"
|
||||
python_version = "3.12"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
pythonpath = ["src"]
|
||||
markers = [
|
||||
"live: opt-in keyed tests against real adapters (skipped without keys)",
|
||||
]
|
||||
asyncio_mode = "auto"
|
||||
testpaths = ["tests"]
|
||||
@@ -0,0 +1,52 @@
|
||||
import json
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.model import ModelAdapter
|
||||
from researchx.adapters.probe import (
|
||||
DEFAULT_EMBEDDING_BATCH,
|
||||
CapabilityProbe,
|
||||
cap_context_window,
|
||||
structured_output_warning,
|
||||
)
|
||||
from researchx.adapters.types import ModelMessage
|
||||
|
||||
|
||||
def supports_probe(adapter: ModelAdapter) -> bool:
|
||||
return getattr(adapter, "probing_supported", False)
|
||||
|
||||
|
||||
async def probe_model_adapter(adapter: ModelAdapter) -> CapabilityProbe:
|
||||
models = await adapter.list_models()
|
||||
smoke_ok = await _structured_output_smoke_test(adapter)
|
||||
writer = adapter.roles().writer
|
||||
warning = structured_output_warning(smoke_ok, writer)
|
||||
endpoint = getattr(adapter, "active_base_url", "unknown")
|
||||
fallback_reason = getattr(adapter, "last_fallback_reason", None)
|
||||
return CapabilityProbe(
|
||||
endpoint=str(endpoint),
|
||||
models=models,
|
||||
context_window=cap_context_window(None),
|
||||
embedding_batch=DEFAULT_EMBEDDING_BATCH,
|
||||
structured_outputs_supported=smoke_ok,
|
||||
structured_output_warning=warning,
|
||||
fallback_reason=fallback_reason,
|
||||
)
|
||||
|
||||
|
||||
async def _structured_output_smoke_test(adapter: ModelAdapter) -> bool:
|
||||
probe_messages = [
|
||||
ModelMessage(
|
||||
role="user",
|
||||
content='Return a JSON object with exactly one key "ok" set to true. No other text.',
|
||||
)
|
||||
]
|
||||
try:
|
||||
result = await adapter.chat(probe_messages, model=adapter.roles().writer, json_mode=True)
|
||||
except (httpx.HTTPError, httpx.HTTPStatusError, ValueError, KeyError):
|
||||
return False
|
||||
try:
|
||||
parsed = json.loads(result.content.strip())
|
||||
except json.JSONDecodeError:
|
||||
return False
|
||||
return isinstance(parsed, dict) and parsed.get("ok") is True
|
||||
@@ -0,0 +1,136 @@
|
||||
import re
|
||||
from datetime import UTC, datetime
|
||||
from html import unescape
|
||||
from urllib.parse import parse_qs, unquote, urlparse
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.secrets import SecretsProvider
|
||||
from researchx.adapters.tavily import TavilySearchAdapter, extract_text, extract_title
|
||||
from researchx.adapters.types import AdapterError, PageContent, SearchResult
|
||||
from researchx.adapters.url_guard import url_allowed
|
||||
|
||||
_DDG_URL = "https://html.duckduckgo.com/html"
|
||||
_SEARCH_TIMEOUT = 10.0
|
||||
_FETCH_TIMEOUT = 10.0
|
||||
_MAX_FETCH_REDIRECTS = 5
|
||||
_BACKOFF_SECONDS = 1.0
|
||||
_USER_AGENT = "researchx/0.2 (local research studio)"
|
||||
_RESULT_RE = re.compile(
|
||||
r'<a[^>]+class="[^"]*result__a[^"]*"[^>]+href="([^"]+)"[^>]*>(.*?)</a>',
|
||||
re.DOTALL,
|
||||
)
|
||||
_SNIPPET_RE = re.compile(r'<a[^>]+class="[^"]*result__snippet[^"]*"[^>]*>(.*?)</a>', re.DOTALL)
|
||||
_TAG_RE = re.compile(r"<[^>]+>")
|
||||
|
||||
|
||||
class DuckDuckGoSearchAdapter:
|
||||
def __init__(self) -> None:
|
||||
self._last_degradation: str | None = None
|
||||
|
||||
async def search(self, query: str, *, max_results: int = 5) -> list[SearchResult]:
|
||||
results = await self._attempt_search(query, max_results)
|
||||
if results is None:
|
||||
self._last_degradation = "duckduckgo rate-limited or unreachable; returning partial results"
|
||||
return []
|
||||
self._last_degradation = None
|
||||
return results
|
||||
|
||||
async def _attempt_search(self, query: str, max_results: int) -> list[SearchResult] | None:
|
||||
import asyncio
|
||||
|
||||
params = {"q": query}
|
||||
headers = {"User-Agent": _USER_AGENT}
|
||||
for attempt in range(2):
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=_SEARCH_TIMEOUT) as client:
|
||||
response = await client.post(_DDG_URL, params=params, headers=headers)
|
||||
if response.status_code in (403, 429):
|
||||
if attempt == 0:
|
||||
await asyncio.sleep(_BACKOFF_SECONDS)
|
||||
continue
|
||||
return None
|
||||
response.raise_for_status()
|
||||
return self._parse_results(response.text, max_results)
|
||||
except httpx.HTTPError:
|
||||
if attempt == 0:
|
||||
await asyncio.sleep(_BACKOFF_SECONDS)
|
||||
continue
|
||||
return None
|
||||
return None
|
||||
|
||||
def _parse_results(self, html: str, max_results: int) -> list[SearchResult]:
|
||||
anchors = _RESULT_RE.findall(html)
|
||||
snippets = [self._strip(s) for s in _SNIPPET_RE.findall(html)]
|
||||
results: list[SearchResult] = []
|
||||
for index, (href, title_html) in enumerate(anchors):
|
||||
url = self._normalize_href(href)
|
||||
if not url:
|
||||
continue
|
||||
snippet = snippets[index] if index < len(snippets) else ""
|
||||
results.append(
|
||||
SearchResult(
|
||||
title=self._strip(title_html),
|
||||
url=url,
|
||||
snippet=snippet,
|
||||
)
|
||||
)
|
||||
if len(results) >= max_results:
|
||||
break
|
||||
return results
|
||||
|
||||
def _normalize_href(self, href: str) -> str:
|
||||
decoded = unescape(href)
|
||||
parsed = urlparse(decoded)
|
||||
if parsed.path.startswith("//duckduckgo.com/l/") or parsed.netloc == "duckduckgo.com":
|
||||
target = parse_qs(parsed.query).get("uddg", [""])[0]
|
||||
return unquote(target)
|
||||
if decoded.startswith("http"):
|
||||
return decoded
|
||||
return ""
|
||||
|
||||
def _strip(self, html: str) -> str:
|
||||
text = _TAG_RE.sub("", html)
|
||||
return re.sub(r"\s+", " ", text).strip()
|
||||
|
||||
async def fetch(self, url: str) -> PageContent:
|
||||
from urllib.parse import urljoin
|
||||
|
||||
current = url
|
||||
async with httpx.AsyncClient(
|
||||
timeout=_FETCH_TIMEOUT,
|
||||
follow_redirects=False,
|
||||
headers={"User-Agent": _USER_AGENT},
|
||||
) as client:
|
||||
for _ in range(_MAX_FETCH_REDIRECTS + 1):
|
||||
if not url_allowed(current):
|
||||
raise AdapterError(
|
||||
"fetch refused: url not allowed (private or non-http scheme)"
|
||||
)
|
||||
response = await client.get(current)
|
||||
if 300 <= response.status_code < 400:
|
||||
location = response.headers.get("location")
|
||||
if not location:
|
||||
raise AdapterError("fetch failed: redirect without location")
|
||||
current = urljoin(current, location)
|
||||
continue
|
||||
response.raise_for_status()
|
||||
return PageContent(
|
||||
url=url,
|
||||
title=extract_title(response.text),
|
||||
text=extract_text(response.text),
|
||||
fetched_at=datetime.now(UTC),
|
||||
)
|
||||
raise AdapterError("fetch failed: too many redirects")
|
||||
|
||||
@property
|
||||
def last_degradation(self) -> str | None:
|
||||
return self._last_degradation
|
||||
|
||||
|
||||
def resolve_search_adapter(
|
||||
secrets: SecretsProvider,
|
||||
) -> tuple[TavilySearchAdapter | DuckDuckGoSearchAdapter, str]:
|
||||
if secrets.resolve("TAVILY_API_KEY"):
|
||||
return TavilySearchAdapter(secrets), "tavily"
|
||||
return DuckDuckGoSearchAdapter(), "duckduckgo-keyless"
|
||||
@@ -0,0 +1,165 @@
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from researchx.adapters.model import ModelRoles
|
||||
from researchx.adapters.types import (
|
||||
AdapterError,
|
||||
ChatResult,
|
||||
ModelMessage,
|
||||
PageContent,
|
||||
SearchResult,
|
||||
)
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
class FakeModelAdapter:
|
||||
def __init__(self, canned: list[ChatResult]) -> None:
|
||||
self._canned = list(canned)
|
||||
self._json_mode_calls: list[bool] = []
|
||||
self.roles_value = ModelRoles(planner="fake-planner", writer="fake-writer")
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ModelMessage],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.2,
|
||||
json_mode: bool = False,
|
||||
) -> ChatResult:
|
||||
del messages, temperature
|
||||
self._json_mode_calls.append(json_mode)
|
||||
if not self._canned:
|
||||
raise AdapterError(f"no canned response for model {model}")
|
||||
return self._canned.pop(0)
|
||||
|
||||
async def list_models(self) -> list[str]:
|
||||
return ["fake-model"]
|
||||
|
||||
def roles(self) -> ModelRoles:
|
||||
return self.roles_value
|
||||
|
||||
@property
|
||||
def last_json_mode(self) -> bool | None:
|
||||
return self._json_mode_calls[-1] if self._json_mode_calls else None
|
||||
|
||||
|
||||
class FakeSearchAdapter:
|
||||
def __init__(self, corpus: list[PageContent]) -> None:
|
||||
self._corpus = list(corpus)
|
||||
self._by_url = {page.url: page for page in corpus}
|
||||
|
||||
async def search(self, query: str, *, max_results: int = 5) -> list[SearchResult]:
|
||||
terms = [t.lower() for t in query.split() if t]
|
||||
results: list[SearchResult] = []
|
||||
for page in self._corpus:
|
||||
haystack = f"{page.title}\n{page.text}".lower()
|
||||
if all(t in haystack for t in terms):
|
||||
snippet = page.text[:200]
|
||||
results.append(SearchResult(title=page.title, url=page.url, snippet=snippet))
|
||||
if len(results) >= max_results:
|
||||
break
|
||||
return results
|
||||
|
||||
async def fetch(self, url: str) -> PageContent:
|
||||
page = self._by_url.get(url)
|
||||
if page is None:
|
||||
raise AdapterError(f"unknown url: {url}")
|
||||
return page.model_copy(update={"fetched_at": _now()})
|
||||
|
||||
|
||||
SHARED_SENTENCE = (
|
||||
"The verification pass re-anchored every quoted claim to its stored passage "
|
||||
"before the report was allowed to ship."
|
||||
)
|
||||
|
||||
|
||||
def corpus() -> list[PageContent]:
|
||||
stamp = _now()
|
||||
pages = [
|
||||
PageContent(
|
||||
url="https://example.org/verification",
|
||||
title="Verification passes in research pipelines",
|
||||
text=(
|
||||
"Deep research pipelines fail when citations are decorative. "
|
||||
f"{SHARED_SENTENCE} The pipeline treats every quote as a claim "
|
||||
"about evidence, not a claim about the world.\n\n"
|
||||
"Verification therefore runs twice: once at generation time and "
|
||||
"once at export time — liveness is rechecked, anchors are "
|
||||
"re-matched, and the ledger records the outcome."
|
||||
),
|
||||
fetched_at=stamp,
|
||||
),
|
||||
PageContent(
|
||||
url="https://example.org/anchoring",
|
||||
title="Quote anchoring and auditability",
|
||||
text=(
|
||||
f"{SHARED_SENTENCE} Anchoring is the mechanism that makes this "
|
||||
"auditability real: each claim carries a fuzzy-matched pointer "
|
||||
"back to the exact stored passage — d’abord normalisé, puis "
|
||||
"comparé. 近似匹配的阈值设定为 0.85.\n\n"
|
||||
"A claim without an anchor is not a finding; it is a hypothesis "
|
||||
"with better typography."
|
||||
),
|
||||
fetched_at=stamp,
|
||||
),
|
||||
PageContent(
|
||||
url="https://example.org/ollama",
|
||||
title="OpenAI-compatible endpoints on Ollama",
|
||||
text=(
|
||||
"Ollama exposes an OpenAI-compatible surface at /v1: chat "
|
||||
"completions accept the standard message array, and model "
|
||||
"listing works via GET /v1/models. Cloud deployments require "
|
||||
"a bearer key; localhost deployments are keyless by default.\n\n"
|
||||
"Capability probing stays conservative: cap context at the "
|
||||
"minimum of the model claim and 8192 tokens, and batch "
|
||||
"embeddings at 8."
|
||||
),
|
||||
fetched_at=stamp,
|
||||
),
|
||||
PageContent(
|
||||
url="https://example.org/search-routing",
|
||||
title="Search routing across keyless and keyed providers",
|
||||
text=(
|
||||
"A keyed provider (Tavily) is the default route; a keyless "
|
||||
"fallback (DuckDuckGo HTML) degrades gracefully under rate "
|
||||
"limiting. The fallback is a degraded mode, not a second "
|
||||
"source — its warnings surface in the run trace.\n\n"
|
||||
"Routing decisions belong to the methodology dials, never to "
|
||||
"the writer."
|
||||
),
|
||||
fetched_at=stamp,
|
||||
),
|
||||
PageContent(
|
||||
url="https://example.org/state-machines",
|
||||
title="Durable state machines for research engines",
|
||||
text=(
|
||||
"A research engine survives restarts by persisting state after "
|
||||
"every transition: an atomic tmp-plus-replace write leaves no "
|
||||
"torn files, and a cursor records the depth-loop position.\n\n"
|
||||
"Pause clears an asyncio event; cancel persists a terminal "
|
||||
"state; resume re-enters the loop at the recorded cursor. "
|
||||
"CancelledError is re-raised after the write — never swallowed."
|
||||
),
|
||||
fetched_at=stamp,
|
||||
),
|
||||
PageContent(
|
||||
url="https://example.org/methodology",
|
||||
title="Methodology-first research selection",
|
||||
text=(
|
||||
"Choosing a methodology before a single query fires converts "
|
||||
"ad-hoc prompt crafting into a structured, reproducible "
|
||||
"decision. Dials — breadth, depth, citation rigor, "
|
||||
"verification passes — resolve into a snapshot that persists "
|
||||
"with every run.\n\n"
|
||||
"The methodology registry ships Rapid Answer for memo-shaped "
|
||||
"answers and Deep Report for executive reports with full "
|
||||
"verification."
|
||||
),
|
||||
fetched_at=stamp,
|
||||
),
|
||||
]
|
||||
return pages
|
||||
|
||||
CORPUS = corpus()
|
||||
@@ -0,0 +1,112 @@
|
||||
import asyncio
|
||||
import os
|
||||
import re
|
||||
import secrets as secrets_module
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.adapters.types import AdapterError
|
||||
from researchx.config.ids import SLUG_RE, generate_slug
|
||||
|
||||
_ARTIFACT_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
|
||||
|
||||
|
||||
def generate_run_id() -> str:
|
||||
"""Generate an 8-char run id (consolidated generator — config/ids.py, P-14)."""
|
||||
return generate_slug()
|
||||
|
||||
|
||||
def _validate_slug(value: str, what: str) -> None:
|
||||
if SLUG_RE.match(value) is None:
|
||||
raise AdapterError(f"invalid {what}")
|
||||
|
||||
|
||||
def _validate_run_id(run_id: str) -> None:
|
||||
_validate_slug(run_id, "run id")
|
||||
|
||||
|
||||
def _validate_owner_id(owner_id: str) -> None:
|
||||
_validate_slug(owner_id, "owner id")
|
||||
|
||||
|
||||
def _validate_artifact_name(name: str) -> None:
|
||||
if not _ARTIFACT_NAME_RE.match(name) or Path(name).name != name:
|
||||
raise AdapterError("invalid artifact name")
|
||||
|
||||
|
||||
class LocalStorageAdapter:
|
||||
"""Owner-scoped local run storage (D-106): runs/<owner_id>/<run_id>/.
|
||||
|
||||
``owner_id`` is REQUIRED (an optional default invites silent
|
||||
unpartitioned use — RESEARCH §5 pitfall) and validated against the
|
||||
shared 8-char slug shape (D-105) so it is a filesystem-safe
|
||||
partition key.
|
||||
"""
|
||||
|
||||
def __init__(self, base_dir: Path, owner_id: str) -> None:
|
||||
_validate_owner_id(owner_id)
|
||||
self._base_dir = base_dir
|
||||
self._owner_id = owner_id
|
||||
|
||||
@property
|
||||
def owner_id(self) -> str:
|
||||
return self._owner_id
|
||||
|
||||
@property
|
||||
def base_dir(self) -> Path:
|
||||
return self._base_dir
|
||||
|
||||
def run_dir(self, run_id: str) -> Path:
|
||||
_validate_run_id(run_id)
|
||||
return self._base_dir / "runs" / self._owner_id / run_id
|
||||
|
||||
def list_run_ids(self) -> list[str]:
|
||||
"""Scan this owner's partition, skipping non-directories/dotfiles."""
|
||||
runs_dir = self._base_dir / "runs" / self._owner_id
|
||||
if not runs_dir.is_dir():
|
||||
return []
|
||||
return sorted(
|
||||
child.name
|
||||
for child in runs_dir.iterdir()
|
||||
if child.is_dir() and not child.name.startswith(".")
|
||||
)
|
||||
|
||||
async def write_run_artifact(self, run_id: str, name: str, content: str) -> Path:
|
||||
_validate_artifact_name(name)
|
||||
target = self.run_dir(run_id) / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp_path = target.with_name(f"{target.name}.{secrets_module.token_hex(4)}.tmp")
|
||||
|
||||
def _write_atomic() -> None:
|
||||
with open(tmp_path, "w", encoding="utf-8") as handle:
|
||||
handle.write(content)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
tmp_path.replace(target)
|
||||
|
||||
try:
|
||||
await asyncio.to_thread(_write_atomic)
|
||||
finally:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
return target
|
||||
|
||||
async def read_run_artifact(self, run_id: str, name: str) -> str:
|
||||
_validate_artifact_name(name)
|
||||
target = self.run_dir(run_id) / name
|
||||
try:
|
||||
return target.read_text(encoding="utf-8")
|
||||
except FileNotFoundError as exc:
|
||||
raise AdapterError(
|
||||
f"artifact not found: runs/{self._owner_id}/{run_id}/{name}"
|
||||
) from exc
|
||||
|
||||
async def append_run_artifact(self, run_id: str, name: str, content: str) -> Path:
|
||||
_validate_artifact_name(name)
|
||||
target = self.run_dir(run_id) / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def _append() -> None:
|
||||
with open(target, "a", encoding="utf-8") as handle:
|
||||
handle.write(content)
|
||||
|
||||
await asyncio.to_thread(_append)
|
||||
return target
|
||||
@@ -0,0 +1,8 @@
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class MemoryAdapter(Protocol):
|
||||
async def remember(self, scope: str, key: str, value: str) -> None: ...
|
||||
|
||||
async def recall(self, scope: str, key: str) -> str | None: ...
|
||||
@@ -0,0 +1,26 @@
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from researchx.adapters.types import ChatResult, ModelMessage
|
||||
|
||||
|
||||
class ModelRoles(BaseModel):
|
||||
planner: str
|
||||
writer: str
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class ModelAdapter(Protocol):
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ModelMessage],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.2,
|
||||
json_mode: bool = False,
|
||||
) -> ChatResult: ...
|
||||
|
||||
async def list_models(self) -> list[str]: ...
|
||||
|
||||
def roles(self) -> ModelRoles: ...
|
||||
@@ -0,0 +1,163 @@
|
||||
"""Per-run model adapter factory (T-2.5, REQ-74 — D-110, P-2 closure).
|
||||
|
||||
Resolves the per-run credential chain for the CURRENT user:
|
||||
|
||||
user key (decrypted per-user key store) -> tier "user"
|
||||
server env-file OLLAMA_API_KEY -> tier "server"
|
||||
neither (localhost:11434 fallback) -> tier "keyless"
|
||||
|
||||
One adapter instance per ``resolve()`` call — NEVER cached across runs.
|
||||
The per-instance ``_active_base`` fallback mutation in
|
||||
OllamaModelAdapter is then per-run by construction, killing the
|
||||
shared-adapter bug where one user's cloud failure flipped every
|
||||
subsequent user's run to localhost.
|
||||
|
||||
The keyless tier constructs an adapter with no key (no Authorization
|
||||
header is sent — see ollama.py) and the existing cloud→local fallback
|
||||
semantics are preserved: a keyless adapter starts on the cloud base
|
||||
and falls back on failure exactly as before, so localhost:11434
|
||||
remains the zero-cost path.
|
||||
|
||||
Tampered/corrupt key-store envelopes surface as ``KeyStoreError`` (a
|
||||
typed ``AdapterError``) from ``resolve()`` — the caller decides how
|
||||
to present the honest error; the factory NEVER silently falls down the
|
||||
chain on a read failure (that would be a silent tier drop, C3-06).
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
|
||||
from researchx.adapters.model import ModelRoles
|
||||
from researchx.adapters.ollama import OllamaModelAdapter
|
||||
from researchx.adapters.secrets import SecretsProvider
|
||||
from researchx.auth.key_store import KEY_SCOPE, KeyStore, KeyStoreError
|
||||
from researchx.auth.models import UserPublic
|
||||
from researchx.auth.settings import DEFAULT_PLANNER_MODEL, DEFAULT_WRITER_MODEL, UserSettings
|
||||
|
||||
TIER_USER = "user"
|
||||
TIER_SERVER = "server"
|
||||
TIER_KEYLESS = "keyless"
|
||||
|
||||
|
||||
class ModelAdapterFactory:
|
||||
"""Resolve (adapter, tier) for a user's run — per call, never cached."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
secrets: SecretsProvider,
|
||||
key_store: KeyStore,
|
||||
*,
|
||||
planner_model: str = DEFAULT_PLANNER_MODEL,
|
||||
writer_model: str = DEFAULT_WRITER_MODEL,
|
||||
cloud_base_url: str | None = None,
|
||||
local_base_url: str | None = None,
|
||||
) -> None:
|
||||
self._secrets = secrets
|
||||
self._key_store = key_store
|
||||
self._planner_model = planner_model
|
||||
self._writer_model = writer_model
|
||||
self._cloud_base_url = cloud_base_url
|
||||
self._local_base_url = local_base_url
|
||||
|
||||
def _construct(
|
||||
self, planner: str, writer: str, api_key: str | None
|
||||
) -> OllamaModelAdapter:
|
||||
return OllamaModelAdapter(
|
||||
self._secrets,
|
||||
planner_model=planner,
|
||||
writer_model=writer,
|
||||
cloud_base_url=self._cloud_base_url,
|
||||
local_base_url=self._local_base_url,
|
||||
api_key=api_key,
|
||||
)
|
||||
|
||||
async def resolve(
|
||||
self, user: UserPublic, settings: UserSettings
|
||||
) -> tuple[OllamaModelAdapter, str]:
|
||||
"""Resolve the user's credential chain and build ONE adapter.
|
||||
|
||||
The adapter's roles honor the user's saved settings
|
||||
(planner_model/writer_model); an unset settings value keeps the
|
||||
factory defaults. Raises ``KeyStoreError`` (an AdapterError)
|
||||
when the user's stored envelope is tampered/corrupt — never a
|
||||
silent tier drop.
|
||||
"""
|
||||
planner = settings.planner_model or self._planner_model
|
||||
writer = settings.writer_model or self._writer_model
|
||||
|
||||
if self._key_store.enabled:
|
||||
# get_key raises KeyStoreError on tamper — honest error, not
|
||||
# a silent fall to server/keyless.
|
||||
user_key = await self._key_store.get_key(user.user_id)
|
||||
if user_key:
|
||||
return self._construct(planner, writer, user_key), TIER_USER
|
||||
|
||||
server_key = self._secrets.resolve(KEY_SCOPE)
|
||||
if server_key:
|
||||
return self._construct(planner, writer, server_key), TIER_SERVER
|
||||
|
||||
return self._construct(planner, writer, None), TIER_KEYLESS
|
||||
|
||||
async def resolve_status(
|
||||
self, user: UserPublic, settings: UserSettings
|
||||
) -> tuple[OllamaModelAdapter, str, str | None]:
|
||||
"""Status variant for GET /api/config: like resolve() but a
|
||||
tampered/corrupt key-store read DEGRADES down the chain and
|
||||
returns the honest error message instead of raising — the
|
||||
config status must never 500 on a corrupt envelope, and the
|
||||
tier must never silently claim 'user' when the key cannot be
|
||||
decrypted. create_run keeps the raising resolve() (a corrupt
|
||||
key there is an honest 409 failure).
|
||||
"""
|
||||
error: str | None = None
|
||||
planner = settings.planner_model or self._planner_model
|
||||
writer = settings.writer_model or self._writer_model
|
||||
|
||||
if self._key_store.enabled:
|
||||
try:
|
||||
user_key = await self._key_store.get_key(user.user_id)
|
||||
except KeyStoreError as exc:
|
||||
# Honest degradation: the tier falls to server/keyless
|
||||
# but the error message is carried alongside — surfaced
|
||||
# by the config status so the user can see WHY.
|
||||
user_key = None
|
||||
error = str(exc)
|
||||
if user_key:
|
||||
return self._construct(planner, writer, user_key), TIER_USER, None
|
||||
|
||||
server_key = self._secrets.resolve(KEY_SCOPE)
|
||||
if server_key:
|
||||
return self._construct(planner, writer, server_key), TIER_SERVER, error
|
||||
|
||||
return self._construct(planner, writer, None), TIER_KEYLESS, error
|
||||
|
||||
async def key_fingerprint(self, user: UserPublic) -> str:
|
||||
"""Stable per-(user, tier, key) probe-cache key component.
|
||||
|
||||
The raw key is NEVER the fingerprint — a short sha256 of the
|
||||
effective key (or the tier itself when keyless) identifies the
|
||||
resolution without exposing material. A key-store read error
|
||||
yields the server/keyless fingerprint; the probe itself will
|
||||
surface the error state through the config status.
|
||||
"""
|
||||
if self._key_store.enabled:
|
||||
try:
|
||||
user_key = await self._key_store.get_key(user.user_id)
|
||||
except KeyStoreError:
|
||||
user_key = None
|
||||
if user_key:
|
||||
return f"{TIER_USER}:{hashlib.sha256(user_key.encode()).hexdigest()[:12]}"
|
||||
server_key = self._secrets.resolve(KEY_SCOPE)
|
||||
if server_key:
|
||||
return f"{TIER_SERVER}:{hashlib.sha256(server_key.encode()).hexdigest()[:12]}"
|
||||
return TIER_KEYLESS
|
||||
|
||||
def roles_defaults(self) -> ModelRoles:
|
||||
return ModelRoles(planner=self._planner_model, writer=self._writer_model)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"TIER_KEYLESS",
|
||||
"TIER_SERVER",
|
||||
"TIER_USER",
|
||||
"ModelAdapterFactory",
|
||||
]
|
||||
@@ -0,0 +1,129 @@
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.model import ModelRoles
|
||||
from researchx.adapters.secrets import SecretsProvider
|
||||
from researchx.adapters.types import AdapterError, ChatResult, ModelMessage, Usage
|
||||
|
||||
_DEFAULT_CLOUD_BASE = "https://ollama.com/v1"
|
||||
_DEFAULT_LOCAL_BASE = "http://localhost:11434/v1"
|
||||
_REQUEST_TIMEOUT = 60.0
|
||||
|
||||
|
||||
class OllamaModelAdapter:
|
||||
probing_supported = True
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
secrets: SecretsProvider,
|
||||
planner_model: str,
|
||||
writer_model: str,
|
||||
cloud_base_url: str | None = None,
|
||||
local_base_url: str | None = None,
|
||||
api_key: str | None = None,
|
||||
) -> None:
|
||||
self._secrets = secrets
|
||||
self._roles = ModelRoles(planner=planner_model, writer=writer_model)
|
||||
self._cloud_base = (cloud_base_url or secrets.resolve("OLLAMA_BASE_URL") or _DEFAULT_CLOUD_BASE).rstrip("/")
|
||||
self._local_base = (local_base_url or _DEFAULT_LOCAL_BASE).rstrip("/")
|
||||
self._active_base = self._cloud_base
|
||||
# Explicit api_key takes precedence over the secrets resolve
|
||||
# (T-2.5: the per-run factory passes the user's decrypted key);
|
||||
# the secrets path stays for back-compat (server tier).
|
||||
self._api_key = api_key if api_key is not None else secrets.resolve("OLLAMA_API_KEY")
|
||||
self._last_fallback_reason: str | None = None
|
||||
|
||||
def _headers(self, base_url: str) -> dict[str, str]:
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if base_url == self._cloud_base and self._api_key:
|
||||
headers["Authorization"] = f"Bearer {self._api_key}"
|
||||
return headers
|
||||
|
||||
async def _post_chat(
|
||||
self,
|
||||
base_url: str,
|
||||
payload: dict[str, object],
|
||||
) -> dict[str, Any]:
|
||||
async with httpx.AsyncClient(timeout=_REQUEST_TIMEOUT) as client:
|
||||
response = await client.post(
|
||||
f"{base_url}/chat/completions",
|
||||
json=payload,
|
||||
headers=self._headers(base_url),
|
||||
)
|
||||
try:
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
raise AdapterError(f"chat request failed ({type(exc).__name__})") from exc
|
||||
data: dict[str, Any] = response.json()
|
||||
return data
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ModelMessage],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.2,
|
||||
json_mode: bool = False,
|
||||
) -> ChatResult:
|
||||
payload: dict[str, object] = {
|
||||
"model": model,
|
||||
"messages": [m.model_dump() for m in messages],
|
||||
"temperature": temperature,
|
||||
}
|
||||
if json_mode:
|
||||
payload["response_format"] = {"type": "json_object"}
|
||||
try:
|
||||
data = await self._post_chat(self._active_base, payload)
|
||||
except AdapterError as first_error:
|
||||
if self._active_base == self._cloud_base:
|
||||
self._last_fallback_reason = f"cloud endpoint failed: {type(first_error.__cause__).__name__ if first_error.__cause__ else 'Error'}"
|
||||
self._active_base = self._local_base
|
||||
data = await self._post_chat(self._local_base, payload)
|
||||
else:
|
||||
raise
|
||||
choice = data["choices"][0]["message"]
|
||||
raw_usage = data.get("usage") or {}
|
||||
usage = Usage(
|
||||
prompt_tokens=raw_usage.get("prompt_tokens"),
|
||||
completion_tokens=raw_usage.get("completion_tokens"),
|
||||
)
|
||||
return ChatResult(content=choice["content"], model=model, usage=usage)
|
||||
|
||||
async def list_models(self) -> list[str]:
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=_REQUEST_TIMEOUT) as client:
|
||||
response = await client.get(
|
||||
f"{self._active_base}/models",
|
||||
headers=self._headers(self._active_base),
|
||||
)
|
||||
try:
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
raise AdapterError(f"models listing failed ({type(exc).__name__})") from exc
|
||||
data: dict[str, Any] = response.json()
|
||||
return [str(entry["id"]) for entry in data.get("data", [])]
|
||||
except (AdapterError, httpx.HTTPError) as exc:
|
||||
# Fallback only if there IS a distinct local base to fall
|
||||
# back to (a caller pinning both bases to one endpoint —
|
||||
# e.g. a private gateway — must not recurse forever), and
|
||||
# never leak a raw httpx error: the adapter's outward
|
||||
# contract is a typed AdapterError.
|
||||
if self._active_base == self._cloud_base and self._local_base != self._cloud_base:
|
||||
self._last_fallback_reason = "cloud models listing failed; using local"
|
||||
self._active_base = self._local_base
|
||||
return await self.list_models()
|
||||
if isinstance(exc, AdapterError):
|
||||
raise
|
||||
raise AdapterError(f"models listing failed ({type(exc).__name__})") from exc
|
||||
|
||||
def roles(self) -> ModelRoles:
|
||||
return self._roles
|
||||
|
||||
@property
|
||||
def active_base_url(self) -> str:
|
||||
return self._active_base
|
||||
|
||||
@property
|
||||
def last_fallback_reason(self) -> str | None:
|
||||
return self._last_fallback_reason
|
||||
@@ -0,0 +1,34 @@
|
||||
from pydantic import BaseModel
|
||||
|
||||
DEFAULT_CONTEXT_CAP = 8192
|
||||
DEFAULT_EMBEDDING_BATCH = 8
|
||||
|
||||
|
||||
class CapabilityProbe(BaseModel):
|
||||
endpoint: str
|
||||
models: list[str]
|
||||
context_window: int
|
||||
embedding_batch: int
|
||||
structured_outputs_supported: bool
|
||||
structured_output_warning: str | None = None
|
||||
fallback_reason: str | None = None
|
||||
|
||||
@property
|
||||
def degraded(self) -> bool:
|
||||
return self.structured_output_warning is not None or self.fallback_reason is not None
|
||||
|
||||
|
||||
def cap_context_window(model_claim: int | None) -> int:
|
||||
if model_claim is None or model_claim <= 0:
|
||||
return DEFAULT_CONTEXT_CAP
|
||||
return min(model_claim, DEFAULT_CONTEXT_CAP)
|
||||
|
||||
|
||||
def structured_output_warning(supported: bool, model_name: str) -> str | None:
|
||||
if supported:
|
||||
return None
|
||||
return (
|
||||
f"Model '{model_name}' did not confirm structured-output (function-calling) support. "
|
||||
"Methodologies needing structured outputs will degrade to a prompt-embedded JSON "
|
||||
"contract — expect looser parsing, not a hard failure."
|
||||
)
|
||||
@@ -0,0 +1,10 @@
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
from researchx.adapters.types import PageContent, SearchResult
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class SearchAdapter(Protocol):
|
||||
async def search(self, query: str, *, max_results: int = 5) -> list[SearchResult]: ...
|
||||
|
||||
async def fetch(self, url: str) -> PageContent: ...
|
||||
@@ -0,0 +1,32 @@
|
||||
from pathlib import Path
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class SecretsProvider(Protocol):
|
||||
def resolve(self, name: str) -> str | None: ...
|
||||
|
||||
|
||||
def _parse_env_file(path: Path) -> dict[str, str]:
|
||||
values: dict[str, str] = {}
|
||||
for raw_line in path.read_text(encoding="utf-8").splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, _, value = line.partition("=")
|
||||
values[key.strip()] = value.strip().strip("'\"")
|
||||
return values
|
||||
|
||||
|
||||
class EnvFileSecretsProvider:
|
||||
def __init__(self, paths: list[Path]) -> None:
|
||||
self._paths = [p for p in paths if p.is_file()]
|
||||
self._values: dict[str, str] = {}
|
||||
for path in self._paths:
|
||||
self._values.update(_parse_env_file(path))
|
||||
|
||||
def resolve(self, name: str) -> str | None:
|
||||
return self._values.get(name)
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"EnvFileSecretsProvider(paths={[p.name for p in self._paths]})"
|
||||
@@ -0,0 +1,13 @@
|
||||
from pathlib import Path
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class StorageAdapter(Protocol):
|
||||
async def write_run_artifact(self, run_id: str, name: str, content: str) -> Path: ...
|
||||
|
||||
async def read_run_artifact(self, run_id: str, name: str) -> str: ...
|
||||
|
||||
async def append_run_artifact(self, run_id: str, name: str, content: str) -> Path: ...
|
||||
|
||||
def run_dir(self, run_id: str) -> Path: ...
|
||||
@@ -0,0 +1,87 @@
|
||||
import re
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.secrets import SecretsProvider
|
||||
from researchx.adapters.types import AdapterError, PageContent, SearchResult
|
||||
from researchx.adapters.url_guard import url_allowed
|
||||
|
||||
_TAVILY_URL = "https://api.tavily.com/search"
|
||||
_SEARCH_TIMEOUT = 10.0
|
||||
_FETCH_TIMEOUT = 10.0
|
||||
_MAX_FETCH_REDIRECTS = 5
|
||||
_USER_AGENT = "researchx/0.2 (local research studio)"
|
||||
_TITLE_RE = re.compile(r"<title[^>]*>(.*?)</title>", re.IGNORECASE | re.DOTALL)
|
||||
_BLOCK_RE = re.compile(r"<(script|style)[^>]*>.*?</\1>", re.IGNORECASE | re.DOTALL)
|
||||
_TAG_RE = re.compile(r"<[^>]+>")
|
||||
|
||||
|
||||
def extract_text(html: str) -> str:
|
||||
text = _BLOCK_RE.sub(" ", html)
|
||||
text = _TAG_RE.sub(" ", text)
|
||||
text = re.sub(r"\s+", " ", text)
|
||||
return text.strip()
|
||||
|
||||
|
||||
def extract_title(html: str) -> str:
|
||||
match = _TITLE_RE.search(html)
|
||||
return match.group(1).strip() if match else ""
|
||||
|
||||
|
||||
class TavilySearchAdapter:
|
||||
def __init__(self, secrets: SecretsProvider) -> None:
|
||||
self._api_key = secrets.resolve("TAVILY_API_KEY")
|
||||
if not self._api_key:
|
||||
raise AdapterError("TAVILY_API_KEY not present in secrets scope")
|
||||
|
||||
async def search(self, query: str, *, max_results: int = 5) -> list[SearchResult]:
|
||||
payload = {
|
||||
"api_key": self._api_key,
|
||||
"query": query,
|
||||
"search_depth": "basic",
|
||||
"max_results": max_results,
|
||||
"include_answer": False,
|
||||
}
|
||||
async with httpx.AsyncClient(timeout=_SEARCH_TIMEOUT) as client:
|
||||
response = await client.post(_TAVILY_URL, json=payload)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
return [
|
||||
SearchResult(
|
||||
title=str(entry.get("title", "")),
|
||||
url=str(entry.get("url", "")),
|
||||
snippet=str(entry.get("content", "")),
|
||||
)
|
||||
for entry in data.get("results", [])
|
||||
][:max_results]
|
||||
|
||||
async def fetch(self, url: str) -> PageContent:
|
||||
from urllib.parse import urljoin
|
||||
|
||||
current = url
|
||||
async with httpx.AsyncClient(
|
||||
timeout=_FETCH_TIMEOUT,
|
||||
follow_redirects=False,
|
||||
headers={"User-Agent": _USER_AGENT},
|
||||
) as client:
|
||||
for _ in range(_MAX_FETCH_REDIRECTS + 1):
|
||||
if not url_allowed(current):
|
||||
raise AdapterError(
|
||||
"fetch refused: url not allowed (private or non-http scheme)"
|
||||
)
|
||||
response = await client.get(current)
|
||||
if 300 <= response.status_code < 400:
|
||||
location = response.headers.get("location")
|
||||
if not location:
|
||||
raise AdapterError("fetch failed: redirect without location")
|
||||
current = urljoin(current, location)
|
||||
continue
|
||||
response.raise_for_status()
|
||||
return PageContent(
|
||||
url=url,
|
||||
title=extract_title(response.text),
|
||||
text=extract_text(response.text),
|
||||
fetched_at=datetime.now(UTC),
|
||||
)
|
||||
raise AdapterError("fetch failed: too many redirects")
|
||||
@@ -0,0 +1,36 @@
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class ModelMessage(BaseModel):
|
||||
role: str
|
||||
content: str
|
||||
|
||||
|
||||
class Usage(BaseModel):
|
||||
prompt_tokens: int | None = None
|
||||
completion_tokens: int | None = None
|
||||
|
||||
|
||||
class ChatResult(BaseModel):
|
||||
content: str
|
||||
model: str
|
||||
usage: Usage | None = None
|
||||
|
||||
|
||||
class SearchResult(BaseModel):
|
||||
title: str
|
||||
url: str
|
||||
snippet: str
|
||||
|
||||
|
||||
class PageContent(BaseModel):
|
||||
url: str
|
||||
title: str
|
||||
text: str
|
||||
fetched_at: datetime
|
||||
|
||||
|
||||
class AdapterError(Exception):
|
||||
pass
|
||||
@@ -0,0 +1,44 @@
|
||||
import socket
|
||||
from ipaddress import IPv4Address, IPv6Address
|
||||
from urllib.parse import urlparse
|
||||
|
||||
|
||||
def _host_is_private(hostname: str) -> bool:
|
||||
try:
|
||||
addr_info = socket.getaddrinfo(hostname, None)
|
||||
except (socket.gaierror, UnicodeError):
|
||||
return True
|
||||
for family, _, _, _, sockaddr in addr_info:
|
||||
try:
|
||||
if family == socket.AF_INET6:
|
||||
ip: IPv4Address | IPv6Address = IPv6Address(sockaddr[0])
|
||||
else:
|
||||
ip = IPv4Address(sockaddr[0])
|
||||
except ValueError:
|
||||
return True
|
||||
if (
|
||||
ip.is_private
|
||||
or ip.is_loopback
|
||||
or ip.is_link_local
|
||||
or ip.is_reserved
|
||||
or ip.is_multicast
|
||||
or ip.is_unspecified
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def url_allowed(url: str) -> bool:
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return False
|
||||
if not parsed.hostname:
|
||||
return False
|
||||
return not _host_is_private(parsed.hostname)
|
||||
|
||||
|
||||
class FetchRefusedError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
__all__ = ["FetchRefusedError", "url_allowed"]
|
||||
@@ -0,0 +1,934 @@
|
||||
import asyncio
|
||||
import json
|
||||
import time
|
||||
from collections.abc import AsyncIterator
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Any
|
||||
|
||||
from fastapi import APIRouter, Depends, FastAPI, HTTPException
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from pydantic import BaseModel, Field, ValidationError
|
||||
from sse_starlette.sse import EventSourceResponse
|
||||
|
||||
from researchx.adapters.capability_probe import probe_model_adapter, supports_probe
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.model_factory import ModelAdapterFactory
|
||||
from researchx.adapters.types import AdapterError, ChatResult, ModelMessage
|
||||
from researchx.api.read_models import connections, graph, library, list_runs, suggestions, tasks
|
||||
from researchx.auth.admin_routes import build_admin_router
|
||||
from researchx.auth.guard import Guard, build_admin_guard, build_dev_guard, build_guard
|
||||
from researchx.auth.key_store import KEY_SCOPE, KeyStore, KeyStoreError
|
||||
from researchx.auth.models import _EMAIL_RE, UserPublic
|
||||
from researchx.auth.routes import build_auth_router
|
||||
from researchx.auth.sessions import SessionStore
|
||||
from researchx.auth.settings import SettingsStore, SettingsStoreError, UserSettings
|
||||
from researchx.auth.users import UserError, UserStore
|
||||
from researchx.config.paths import data_root
|
||||
from researchx.engine.actor import RunActor, make_run_config
|
||||
from researchx.engine.events import EventBus, RunEvent
|
||||
from researchx.engine.run_state import Plan, RunState
|
||||
from researchx.engine.states import TERMINAL_STATES, RunStateName
|
||||
from researchx.methodologies import registry
|
||||
|
||||
_CANNED_PLAN = (
|
||||
'{"sub_questions": ["What is the research question?"], '
|
||||
'"source_universe": ["web"], "outline": ["approach", "findings", "conclusion"]}'
|
||||
)
|
||||
_CANNED_WRITER = (
|
||||
"# Research Memo\n\n"
|
||||
"The verified core finding [p-1]. A second corroborated observation [p-2].\n"
|
||||
)
|
||||
|
||||
_CORS_ORIGINS = ["http://localhost:3000", "http://127.0.0.1:3000"]
|
||||
|
||||
# Config-status probe cache TTL (P-1): in-process, keyed
|
||||
# (user_id, tier, key_fingerprint) so a key change invalidates.
|
||||
_PROBE_CACHE_TTL_SECONDS = 60.0
|
||||
|
||||
|
||||
class CreateRunBody(BaseModel):
|
||||
question: str = Field(min_length=1, max_length=2000)
|
||||
context: str = Field(default="", max_length=8000)
|
||||
methodology: str
|
||||
dial_overrides: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class ApproveBody(BaseModel):
|
||||
plan: dict[str, Any] | None = None
|
||||
|
||||
|
||||
class SettingsBody(BaseModel):
|
||||
"""PUT /api/config/settings body — partial update (D-111)."""
|
||||
|
||||
planner_model: str | None = Field(default=None, min_length=1, max_length=200)
|
||||
writer_model: str | None = Field(default=None, min_length=1, max_length=200)
|
||||
methodology_defaults: dict[str, Any] | None = None
|
||||
appearance: str | None = Field(default=None, max_length=40)
|
||||
|
||||
|
||||
class ModelKeyBody(BaseModel):
|
||||
"""PUT /api/config/settings/model-key body — the raw key, accepted,
|
||||
never echoed (REQ-73)."""
|
||||
|
||||
key: str = Field(min_length=1, max_length=512)
|
||||
|
||||
|
||||
class KeyStatus(BaseModel):
|
||||
"""Masked-only key status — the raw key NEVER appears (REQ-73)."""
|
||||
|
||||
configured: bool
|
||||
masked: str | None = None
|
||||
disabled: bool = False
|
||||
reason: str | None = None
|
||||
|
||||
|
||||
class ConfigStatus(BaseModel):
|
||||
resolved_tier: str # user | server | keyless (REQ-74)
|
||||
key_store: str # enabled | disabled
|
||||
key_status: KeyStatus
|
||||
model_adapter: str
|
||||
search_adapter: str
|
||||
storage_dir: str
|
||||
probe: dict[str, Any] | None = None
|
||||
cost_counters: dict[str, int]
|
||||
|
||||
|
||||
class RunSummary(BaseModel):
|
||||
id: str
|
||||
status: str
|
||||
question: str
|
||||
methodology: str
|
||||
plan: dict[str, Any] | None = None
|
||||
started_at: str
|
||||
|
||||
|
||||
class MethodologyCard(BaseModel):
|
||||
slug: str
|
||||
name: str
|
||||
description: str
|
||||
eta_minutes: int
|
||||
cost_estimate: str
|
||||
why_recommended: str
|
||||
output_shape: str
|
||||
dials: dict[str, Any]
|
||||
|
||||
|
||||
_STATUS_MAP = {
|
||||
RunStateName.PLANNING: "planning",
|
||||
RunStateName.AWAITING_APPROVAL: "awaiting_approval",
|
||||
RunStateName.SEARCHING: "searching",
|
||||
RunStateName.EXTRACTING: "extracting",
|
||||
RunStateName.REFLECTING: "reflecting",
|
||||
RunStateName.SYNTHESIZING: "synthesizing",
|
||||
RunStateName.VERIFYING: "verifying",
|
||||
RunStateName.DONE: "done",
|
||||
RunStateName.CANCELLED: "cancelled",
|
||||
RunStateName.FAILED: "failed",
|
||||
}
|
||||
|
||||
_ETA_TABLE = {"rapid-answer": (2, "low"), "deep-report": (12, "medium")}
|
||||
|
||||
|
||||
def _cost_estimate(slug: str) -> str:
|
||||
methodology = registry.get(slug)
|
||||
units = methodology.dials.depth * methodology.dials.breadth
|
||||
if units <= 1:
|
||||
return "~5k tokens (estimate)"
|
||||
if units <= 4:
|
||||
return "~25k tokens (estimate)"
|
||||
return "~60k tokens (estimate)"
|
||||
|
||||
|
||||
def _why_recommended(slug: str) -> str:
|
||||
return {
|
||||
"rapid-answer": "One pass, one memo — fastest path to a confident, cited answer.",
|
||||
"deep-report": "Iterative depth with full verification — when the answer must stand up to scrutiny.",
|
||||
}.get(slug, "Methodology-first research.")
|
||||
|
||||
|
||||
class RunRegistry:
|
||||
"""In-memory live-run registry, keyed (owner_id, run_id) (REQ-71).
|
||||
|
||||
Cross-owner access is indistinguishable from a missing run: the
|
||||
keyed lookup misses, routes translate the miss to a 404. Single
|
||||
uvicorn worker only — this state is single-process by design.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._actors: dict[tuple[str, str], tuple[RunActor, asyncio.Task[RunState]]] = {}
|
||||
self._buses: dict[tuple[str, str], EventBus] = {}
|
||||
# Live SSE subscriber refcounts, keyed like _actors (P-6).
|
||||
self._subscribers: dict[tuple[str, str], int] = {}
|
||||
# Owner-keyed cost counters (P-13): per-user usage surface.
|
||||
self.cost_counters: dict[str, dict[str, int]] = {}
|
||||
|
||||
def _counters(self, owner_id: str) -> dict[str, int]:
|
||||
return self.cost_counters.setdefault(owner_id, {"runs_created": 0, "runs_completed": 0})
|
||||
|
||||
def add(
|
||||
self,
|
||||
owner_id: str,
|
||||
run_id: str,
|
||||
actor: RunActor,
|
||||
task: asyncio.Task[RunState],
|
||||
bus: EventBus,
|
||||
) -> None:
|
||||
self._actors[(owner_id, run_id)] = (actor, task)
|
||||
self._buses[(owner_id, run_id)] = bus
|
||||
self._counters(owner_id)["runs_created"] += 1
|
||||
|
||||
def actor(self, owner_id: str, run_id: str) -> RunActor:
|
||||
entry = self._actors.get((owner_id, run_id))
|
||||
if entry is None:
|
||||
raise KeyError((owner_id, run_id))
|
||||
return entry[0]
|
||||
|
||||
def bus(self, owner_id: str, run_id: str) -> EventBus:
|
||||
bus = self._buses.get((owner_id, run_id))
|
||||
if bus is None:
|
||||
raise KeyError((owner_id, run_id))
|
||||
return bus
|
||||
|
||||
def has(self, owner_id: str, run_id: str) -> bool:
|
||||
return (owner_id, run_id) in self._actors
|
||||
|
||||
def note_completed(self, owner_id: str) -> None:
|
||||
self._counters(owner_id)["runs_completed"] += 1
|
||||
|
||||
def completed_snapshot(self, owner_id: str) -> dict[str, int]:
|
||||
"""Current counters for *owner_id*, derived where possible.
|
||||
|
||||
``runs_completed`` is computed from the live registry (a run is
|
||||
completed iff its actor sits in a terminal state) and merged
|
||||
with the sticky counter — which keeps counting runs that have
|
||||
since been pruned from the registry. Deriving from actor state
|
||||
removes the scheduling race between the actor finishing and a
|
||||
detached watcher task incrementing the counter.
|
||||
"""
|
||||
counters = dict(self._counters(owner_id))
|
||||
live_done = sum(
|
||||
1
|
||||
for (owner, _), (actor, _) in self._actors.items()
|
||||
if owner == owner_id and actor.state.state == RunStateName.DONE
|
||||
)
|
||||
if live_done:
|
||||
# Prefer live truth when registry entries still exist; the
|
||||
# sticky counter catches pruned/evicted runs afterwards.
|
||||
counters["runs_completed"] = max(counters["runs_completed"], live_done)
|
||||
return counters
|
||||
|
||||
def subscribe_events(self, owner_id: str, run_id: str) -> None:
|
||||
self._subscribers[(owner_id, run_id)] = self._subscribers.get((owner_id, run_id), 0) + 1
|
||||
|
||||
def unsubscribe_events(self, owner_id: str, run_id: str) -> None:
|
||||
count = self._subscribers.get((owner_id, run_id), 0)
|
||||
if count <= 1:
|
||||
self._subscribers.pop((owner_id, run_id), None)
|
||||
else:
|
||||
self._subscribers[(owner_id, run_id)] = count - 1
|
||||
|
||||
def subscriber_count(self, owner_id: str, run_id: str) -> int:
|
||||
return self._subscribers.get((owner_id, run_id), 0)
|
||||
|
||||
def prune(self, owner_id: str, run_id: str) -> bool:
|
||||
"""Drop a terminal entry with zero live SSE subscribers (P-6).
|
||||
|
||||
Terminal entries are otherwise a slow multi-user leak. Events
|
||||
for a pruned run 404 afterwards — documented acceptable; the
|
||||
trace persists on disk.
|
||||
"""
|
||||
entry = self._actors.get((owner_id, run_id))
|
||||
if entry is None:
|
||||
return False
|
||||
actor = entry[0]
|
||||
if actor.state.state not in TERMINAL_STATES:
|
||||
return False
|
||||
if self.subscriber_count(owner_id, run_id) > 0:
|
||||
return False
|
||||
self._actors.pop((owner_id, run_id), None)
|
||||
self._buses.pop((owner_id, run_id), None)
|
||||
self._subscribers.pop((owner_id, run_id), None)
|
||||
return True
|
||||
|
||||
|
||||
def create_app(
|
||||
*,
|
||||
storage_dir: Path | None = None,
|
||||
model_adapter: Any = None,
|
||||
search_adapter: Any = None,
|
||||
auth_enabled: bool = True,
|
||||
cookie_secure: bool = False,
|
||||
data_dir: Path | None = None,
|
||||
key_store: KeyStore | None = None,
|
||||
settings_store: SettingsStore | None = None,
|
||||
model_factory: ModelAdapterFactory | None = None,
|
||||
secrets_provider: Any = None,
|
||||
) -> FastAPI:
|
||||
"""Build the ResearchX API app.
|
||||
|
||||
``auth_enabled=False`` is a dev/test harness parameter ONLY — never
|
||||
an env var readable in production (C3-16). It resolves a synthetic
|
||||
dev admin for every request so zero-key suites keep working.
|
||||
``data_dir`` overrides the auth-store location (defaults to
|
||||
``data_root()``); ``storage_dir`` remains the run-artifact base.
|
||||
|
||||
Per-user stores (T-2.2/T-2.3): ``key_store`` defaults to a KeyStore
|
||||
over the data root (disabled when no KEK is configured — honest
|
||||
degradation); ``settings_store`` likewise. Both are injectable for
|
||||
tests. ``model_factory`` (T-2.5) resolves the per-user credential
|
||||
chain user → server → keyless for ``create_run`` and
|
||||
``GET /api/config``; when absent the fixed ``model_adapter`` is
|
||||
used (test harness), tiered resolution disabled by construction.
|
||||
"""
|
||||
app = FastAPI(title="ResearchX API", version="0.2.0")
|
||||
|
||||
# D-107: run artifacts live under the data root (storage_dir
|
||||
# overrides for tests); auth stores always share the same root.
|
||||
storage_base = storage_dir if storage_dir is not None else data_root()
|
||||
runs = RunRegistry()
|
||||
# The fallback model is constructed ONLY when no factory is wired:
|
||||
# the factory path (production, T-2.5) resolves a real adapter per
|
||||
# run and the production entrypoint can therefore never hold a
|
||||
# FakeModelAdapter in memory (P-3).
|
||||
model: Any
|
||||
if model_adapter is not None:
|
||||
model = model_adapter
|
||||
elif model_factory is not None:
|
||||
model = None # every consumer goes through the factory
|
||||
else:
|
||||
model = FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=_CANNED_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_CANNED_WRITER, model="fake-writer"),
|
||||
]
|
||||
)
|
||||
search = search_adapter or FakeSearchAdapter(CORPUS)
|
||||
|
||||
auth_root = data_dir if data_dir is not None else data_root()
|
||||
users = UserStore(auth_root / "users.json")
|
||||
sessions = SessionStore(auth_root / "sessions.json")
|
||||
|
||||
# Per-user stores wired from the same root as the auth stores;
|
||||
# injectable for tests (key_store=None -> disabled, honest reason).
|
||||
key_store = key_store if key_store is not None else KeyStore(
|
||||
secrets_provider or _NullSecrets(), auth_root
|
||||
)
|
||||
settings_store = settings_store if settings_store is not None else SettingsStore(auth_root)
|
||||
|
||||
guard: Guard = build_guard(users, sessions) if auth_enabled else build_dev_guard()
|
||||
admin_guard = build_admin_guard(guard)
|
||||
|
||||
# In-process probe cache keyed (user_id, tier, key_fingerprint)
|
||||
# with a short TTL (P-1 closure — the probe runs against the
|
||||
# RESOLVED tier, never stolen from the registry, and a key change
|
||||
# rotates the fingerprint which invalidates the entry).
|
||||
probe_cache: dict[tuple[str, str, str], tuple[float, dict[str, Any] | None]] = {}
|
||||
|
||||
def _owner_storage(user: UserPublic) -> LocalStorageAdapter:
|
||||
"""Owner-scoped storage, constructed per request from the auth
|
||||
dependency (REQ-71): runs/<user_id>/<run_id>/."""
|
||||
return LocalStorageAdapter(base_dir=storage_base, owner_id=user.user_id)
|
||||
|
||||
def _summary(user: UserPublic, run_id: str) -> RunSummary:
|
||||
actor = runs.actor(user.user_id, run_id)
|
||||
state = actor.state
|
||||
return RunSummary(
|
||||
id=run_id,
|
||||
status=_STATUS_MAP[state.state],
|
||||
question=state.question,
|
||||
methodology=state.methodology_slug,
|
||||
plan=state.plan.model_dump(mode="json") if state.plan else None,
|
||||
started_at=state.created_at.isoformat(),
|
||||
)
|
||||
|
||||
async def _masked_key_status(user: UserPublic) -> KeyStatus:
|
||||
"""Masked-only key status — never the raw key (REQ-73)."""
|
||||
try:
|
||||
status = await key_store.masked_status(user.user_id)
|
||||
except KeyStoreError as exc:
|
||||
# Stored-but-unreadable: surface the error state honestly.
|
||||
return KeyStatus(
|
||||
configured=False, masked=None, disabled=not key_store.enabled, reason=str(exc)
|
||||
)
|
||||
masked = status.get("masked")
|
||||
reason = status.get("reason")
|
||||
return KeyStatus(
|
||||
configured=bool(status.get("configured")),
|
||||
masked=str(masked) if masked is not None else None,
|
||||
disabled=bool(status.get("disabled")),
|
||||
reason=str(reason) if reason is not None else None,
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# PUBLIC surface: /healthz outside /api (D-103 — plain GET for the
|
||||
# TLS-front proxy, never auth-wrapped) + /api/auth/* (no
|
||||
# require_user; the auth router enforces the CSRF header on its
|
||||
# own POSTs).
|
||||
# ------------------------------------------------------------------
|
||||
public = build_auth_router(users, sessions, cookie_secure=cookie_secure)
|
||||
|
||||
@app.get("/healthz")
|
||||
async def healthz() -> dict[str, str]:
|
||||
return {"status": "ok"}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# PROTECTED router: every /api business route carries the guard.
|
||||
# Admin user management additionally carries the admin guard.
|
||||
# ------------------------------------------------------------------
|
||||
protected = APIRouter(dependencies=[Depends(guard)])
|
||||
|
||||
protected.include_router(build_admin_router(users, sessions, admin_guard))
|
||||
|
||||
@protected.get("/api/config", response_model=ConfigStatus)
|
||||
async def config_status(user: Annotated[UserPublic, Depends(guard)]) -> ConfigStatus:
|
||||
"""Current user's resolved credential tier + status (T-2.6).
|
||||
|
||||
P-1 closure: the probe is resolved via the factory for the
|
||||
CURRENT user — no run creation, no reads of the private
|
||||
registry dict, correct with zero runs, no cross-user leak.
|
||||
A corrupt/tampered stored envelope degrades the tier honestly
|
||||
(server/keyless) and surfaces the error in key_status.reason —
|
||||
never a 500, never a silent drop.
|
||||
"""
|
||||
tier = "keyless"
|
||||
adapter_name = type(model).__name__ if model is not None else "unknown"
|
||||
probe: dict[str, Any] | None = None
|
||||
resolve_error: str | None = None
|
||||
|
||||
if model_factory is not None:
|
||||
settings = await settings_store.get(user.user_id)
|
||||
adapter, tier, resolve_error = await model_factory.resolve_status(user, settings)
|
||||
adapter_name = type(adapter).__name__
|
||||
fingerprint = await model_factory.key_fingerprint(user)
|
||||
cache_key = (user.user_id, tier, fingerprint)
|
||||
now = time.monotonic()
|
||||
cached = probe_cache.get(cache_key)
|
||||
if cached is not None and now - cached[0] < _PROBE_CACHE_TTL_SECONDS:
|
||||
probe = cached[1]
|
||||
elif supports_probe(adapter):
|
||||
try:
|
||||
probe_result = await probe_model_adapter(adapter)
|
||||
probe = probe_result.model_dump(mode="json")
|
||||
probe_cache[cache_key] = (now, probe)
|
||||
except AdapterError:
|
||||
probe = None
|
||||
probe_cache[cache_key] = (now, None)
|
||||
key_status = await _masked_key_status(user)
|
||||
if resolve_error is not None and key_status.reason is None:
|
||||
# The tier degraded honestly; say WHY (never silently).
|
||||
key_status = key_status.model_copy(update={"reason": resolve_error})
|
||||
return ConfigStatus(
|
||||
resolved_tier=tier,
|
||||
key_store="enabled" if key_store.enabled else "disabled",
|
||||
key_status=key_status,
|
||||
model_adapter=adapter_name,
|
||||
search_adapter=type(search).__name__,
|
||||
storage_dir=str(storage_base),
|
||||
probe=probe,
|
||||
cost_counters=runs.completed_snapshot(user.user_id),
|
||||
)
|
||||
|
||||
@protected.get("/api/config/settings", response_model=UserSettings)
|
||||
async def get_settings(user: Annotated[UserPublic, Depends(guard)]) -> UserSettings:
|
||||
return await settings_store.get(user.user_id)
|
||||
|
||||
@protected.put("/api/config/settings", response_model=UserSettings)
|
||||
async def put_settings(
|
||||
body: SettingsBody, user: Annotated[UserPublic, Depends(guard)]
|
||||
) -> UserSettings:
|
||||
"""Persist a partial settings update (REQ-72); returns the
|
||||
updated settings. An explicit null field is treated as absent
|
||||
(partial-merge semantics — the store refuses null-valued
|
||||
fields, so a null can never poison the on-disk payload)."""
|
||||
updates: dict[str, Any] = {
|
||||
key: value
|
||||
for key, value in body.model_dump(exclude_unset=True).items()
|
||||
if value is not None
|
||||
}
|
||||
try:
|
||||
current = await settings_store.get(user.user_id)
|
||||
merged = current.model_copy(update=updates)
|
||||
return await settings_store.update(user.user_id, merged)
|
||||
except SettingsStoreError as exc:
|
||||
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
||||
|
||||
@protected.put("/api/config/settings/model-key", response_model=KeyStatus)
|
||||
async def put_model_key(
|
||||
body: ModelKeyBody, user: Annotated[UserPublic, Depends(guard)]
|
||||
) -> KeyStatus:
|
||||
"""Accept the user's raw model key, encrypt it at rest, return
|
||||
ONLY the masked status (REQ-73 — the raw key never appears in
|
||||
any response, log, or trace). Without a KEK: honest 409."""
|
||||
if not key_store.enabled:
|
||||
# Honest refusal, never a silent no-op.
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail={
|
||||
"key_store": "disabled",
|
||||
"reason": key_store.disabled_reason,
|
||||
},
|
||||
)
|
||||
await key_store.set_key(user.user_id, KEY_SCOPE, body.key)
|
||||
return await _masked_key_status(user)
|
||||
|
||||
@protected.get("/api/config/settings/model-key", response_model=KeyStatus)
|
||||
async def get_model_key_status(user: Annotated[UserPublic, Depends(guard)]) -> KeyStatus:
|
||||
"""Masked-only key status (never the raw key)."""
|
||||
return await _masked_key_status(user)
|
||||
|
||||
@protected.get("/api/methodologies")
|
||||
async def methodologies() -> list[MethodologyCard]:
|
||||
cards: list[MethodologyCard] = []
|
||||
for methodology in registry.list():
|
||||
eta_minutes, _ = _ETA_TABLE.get(methodology.slug, (5, "low"))
|
||||
cards.append(
|
||||
MethodologyCard(
|
||||
slug=methodology.slug,
|
||||
name=methodology.name,
|
||||
description=methodology.description,
|
||||
eta_minutes=eta_minutes,
|
||||
cost_estimate=_cost_estimate(methodology.slug),
|
||||
why_recommended=_why_recommended(methodology.slug),
|
||||
output_shape=methodology.output_shape,
|
||||
dials=methodology.dials.model_dump(mode="json"),
|
||||
)
|
||||
)
|
||||
return cards
|
||||
|
||||
@protected.post("/api/runs", status_code=201)
|
||||
async def create_run(body: CreateRunBody, user: Annotated[UserPublic, Depends(guard)]) -> RunSummary:
|
||||
if body.methodology not in registry:
|
||||
raise HTTPException(status_code=404, detail="unknown methodology")
|
||||
storage = _owner_storage(user)
|
||||
|
||||
# Per-run credential resolution (T-2.5, REQ-74): one adapter
|
||||
# instance per run — the _active_base fallback mutation is
|
||||
# per-instance by construction (P-2 closure).
|
||||
tier: str | None = None
|
||||
if model_factory is not None:
|
||||
settings = await settings_store.get(user.user_id)
|
||||
try:
|
||||
run_model, tier = await model_factory.resolve(user, settings)
|
||||
except KeyStoreError as exc:
|
||||
# A tampered/corrupt stored key is an honest failure —
|
||||
# never a silent tier drop and never a wrong-key run.
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
else:
|
||||
# No factory: the fixed test-harness adapter (create_app
|
||||
# guarantees one exists whenever the factory is absent).
|
||||
run_model = model
|
||||
|
||||
try:
|
||||
run_id, config = make_run_config(
|
||||
question=body.question,
|
||||
context=body.context,
|
||||
methodology_slug=body.methodology,
|
||||
overrides=body.dial_overrides,
|
||||
registry=registry,
|
||||
owner_id=user.user_id,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=422, detail="invalid dial override") from exc
|
||||
bus = EventBus()
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=run_model,
|
||||
search=search,
|
||||
storage=storage,
|
||||
registry=registry,
|
||||
event_bus=bus,
|
||||
)
|
||||
# Stamp the owner on the persisted state (REQ-71). The actor
|
||||
# itself is untouched — engine contracts unchanged.
|
||||
actor.state.owner_id = user.user_id
|
||||
actor.state.credential_tier = tier # REQ-74: auditability
|
||||
if supports_probe(run_model):
|
||||
try:
|
||||
probe = await probe_model_adapter(run_model)
|
||||
await actor.attach_probe(probe.model_dump(mode="json"))
|
||||
except AdapterError:
|
||||
pass
|
||||
|
||||
task = asyncio.create_task(actor.run())
|
||||
runs.add(user.user_id, run_id, actor, task, bus)
|
||||
|
||||
async def _watch() -> None:
|
||||
state = await task
|
||||
if state.state == RunStateName.DONE:
|
||||
runs.note_completed(user.user_id)
|
||||
|
||||
asyncio.create_task(_watch())
|
||||
return _summary(user, run_id)
|
||||
|
||||
@protected.get("/api/runs/{run_id}")
|
||||
async def get_run(run_id: str, user: Annotated[UserPublic, Depends(guard)]) -> RunSummary:
|
||||
if not runs.has(user.user_id, run_id):
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
return _summary(user, run_id)
|
||||
|
||||
@protected.post("/api/runs/{run_id}/approve")
|
||||
async def approve_run(
|
||||
run_id: str,
|
||||
user: Annotated[UserPublic, Depends(guard)],
|
||||
body: ApproveBody | None = None,
|
||||
) -> RunSummary:
|
||||
if not runs.has(user.user_id, run_id):
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
actor = runs.actor(user.user_id, run_id)
|
||||
edited_plan: Plan | None = None
|
||||
if body and body.plan:
|
||||
try:
|
||||
edited_plan = Plan.model_validate(body.plan)
|
||||
except ValidationError:
|
||||
raise HTTPException(status_code=422, detail="invalid plan")
|
||||
await actor.approve(edited_plan)
|
||||
return _summary(user, run_id)
|
||||
|
||||
@protected.post("/api/runs/{run_id}/pause")
|
||||
async def pause_run(run_id: str, user: Annotated[UserPublic, Depends(guard)]) -> RunSummary:
|
||||
if not runs.has(user.user_id, run_id):
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
runs.actor(user.user_id, run_id).pause()
|
||||
return _summary(user, run_id)
|
||||
|
||||
@protected.post("/api/runs/{run_id}/resume")
|
||||
async def resume_run(run_id: str, user: Annotated[UserPublic, Depends(guard)]) -> RunSummary:
|
||||
if not runs.has(user.user_id, run_id):
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
runs.actor(user.user_id, run_id).resume()
|
||||
return _summary(user, run_id)
|
||||
|
||||
@protected.post("/api/runs/{run_id}/cancel")
|
||||
async def cancel_run(run_id: str, user: Annotated[UserPublic, Depends(guard)]) -> RunSummary:
|
||||
if not runs.has(user.user_id, run_id):
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
actor = runs.actor(user.user_id, run_id)
|
||||
actor.cancel()
|
||||
actor.resume()
|
||||
return _summary(user, run_id)
|
||||
|
||||
@protected.get("/api/runs/{run_id}/events")
|
||||
async def run_events(
|
||||
run_id: str, user: Annotated[UserPublic, Depends(guard)]
|
||||
) -> EventSourceResponse:
|
||||
if not runs.has(user.user_id, run_id):
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
bus = runs.bus(user.user_id, run_id)
|
||||
actor = runs.actor(user.user_id, run_id)
|
||||
terminal = TERMINAL_STATES
|
||||
|
||||
async def event_stream() -> AsyncIterator[dict[str, str]]:
|
||||
queue: asyncio.Queue[RunEvent] = asyncio.Queue()
|
||||
|
||||
async def forward(event: RunEvent) -> None:
|
||||
await queue.put(event)
|
||||
|
||||
runs.subscribe_events(user.user_id, run_id) # SSE refcount (P-6)
|
||||
bus.subscribe(forward)
|
||||
try:
|
||||
replay = bus.replay()
|
||||
watermark = replay[-1].seq if replay else 0
|
||||
for prior in replay:
|
||||
yield {"event": prior.type, "data": prior.model_dump_json()}
|
||||
while True:
|
||||
if actor.state.state in terminal:
|
||||
try:
|
||||
event = await asyncio.wait_for(queue.get(), timeout=0.5)
|
||||
except TimeoutError:
|
||||
return
|
||||
else:
|
||||
event = await queue.get()
|
||||
if event.seq <= watermark:
|
||||
continue
|
||||
yield {"event": event.type, "data": event.model_dump_json()}
|
||||
if event.type in ("run_done", "run_error"):
|
||||
return
|
||||
finally:
|
||||
bus.unsubscribe(forward)
|
||||
runs.unsubscribe_events(user.user_id, run_id)
|
||||
# Registry pruning (P-6): a terminal run whose last SSE
|
||||
# subscriber just left is dropped from the registry.
|
||||
# Events for a pruned run 404 afterwards — documented
|
||||
# acceptable; the trace persists on disk.
|
||||
runs.prune(user.user_id, run_id)
|
||||
|
||||
return EventSourceResponse(event_stream())
|
||||
|
||||
@protected.get("/api/runs/{run_id}/report")
|
||||
async def run_report(run_id: str, user: Annotated[UserPublic, Depends(guard)]) -> dict[str, Any]:
|
||||
"""Disk-read, registry-independent (P-8 full closure, T-3.3):
|
||||
the run directory under THIS owner is the existence check — no
|
||||
RunRegistry involvement, correct after a restart with zero
|
||||
in-memory actors. Pre-artifact DONE runs (and any run whose
|
||||
report.json was not yet written) fall back to markdown-only
|
||||
with null report/evidence — never a 500 (D-106: old artifacts
|
||||
disposable)."""
|
||||
storage = _owner_storage(user)
|
||||
if not (storage.run_dir(run_id)).is_dir():
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
try:
|
||||
markdown = await storage.read_run_artifact(run_id, "report.md")
|
||||
except AdapterError:
|
||||
raise HTTPException(status_code=404, detail="report not ready")
|
||||
report: dict[str, Any] | None = None
|
||||
evidence: dict[str, Any] | None = None
|
||||
try:
|
||||
parsed = json.loads(await storage.read_run_artifact(run_id, "report.json"))
|
||||
if isinstance(parsed, dict):
|
||||
report = parsed
|
||||
except (AdapterError, json.JSONDecodeError):
|
||||
pass
|
||||
try:
|
||||
parsed = json.loads(await storage.read_run_artifact(run_id, "evidence.json"))
|
||||
if isinstance(parsed, dict):
|
||||
evidence = parsed
|
||||
except (AdapterError, json.JSONDecodeError):
|
||||
pass
|
||||
return {"run_id": run_id, "markdown": markdown, "report": report, "evidence": evidence}
|
||||
|
||||
@protected.get("/api/runs/{run_id}/ledger")
|
||||
async def run_ledger(run_id: str, user: Annotated[UserPublic, Depends(guard)]) -> dict[str, Any]:
|
||||
"""Disk-read, registry-independent (T-3.3): same owner-dir
|
||||
existence check as the report route."""
|
||||
storage = _owner_storage(user)
|
||||
if not (storage.run_dir(run_id)).is_dir():
|
||||
raise HTTPException(status_code=404, detail="run not found")
|
||||
try:
|
||||
content = await storage.read_run_artifact(run_id, "ledger.json")
|
||||
except AdapterError:
|
||||
raise HTTPException(status_code=404, detail="ledger not ready")
|
||||
data: dict[str, Any] = json.loads(content)
|
||||
return data
|
||||
|
||||
# ------------------------------------------------------------- T-3.2/T-3.3
|
||||
# Live surfaces (REQ-76..79): read-model routes — disk reads only,
|
||||
# owner-scoped via the guard, NEVER the in-memory RunRegistry
|
||||
# (D-115). Every route below is correct after a restart with zero
|
||||
# in-process actors (P-8 full closure, pinned by test).
|
||||
|
||||
@protected.get("/api/runs")
|
||||
async def list_runs_route(user: Annotated[UserPublic, Depends(guard)]) -> list[RunSummary]:
|
||||
"""All of this owner's runs from disk state files, most recent
|
||||
first — the registry-independent run list (post-sweep
|
||||
CANCELLED runs surface cancelled, NOT open)."""
|
||||
records = list_runs(storage_base, user.user_id)
|
||||
summaries: list[RunSummary] = []
|
||||
for record in records:
|
||||
plan = record.state.get("plan")
|
||||
summaries.append(
|
||||
RunSummary(
|
||||
id=record.run_id,
|
||||
status=_STATUS_MAP.get(RunStateName(record.status), "failed"),
|
||||
question=record.question,
|
||||
methodology=record.methodology_slug,
|
||||
plan=plan if isinstance(plan, dict) else None,
|
||||
started_at=record.created_at,
|
||||
)
|
||||
)
|
||||
return summaries
|
||||
|
||||
@protected.get("/api/library")
|
||||
async def library_route(
|
||||
user: Annotated[UserPublic, Depends(guard)], sort: str = "recent"
|
||||
) -> dict[str, Any]:
|
||||
return library(storage_base, user.user_id, sort=sort)
|
||||
|
||||
@protected.get("/api/library/connections")
|
||||
async def connections_route(user: Annotated[UserPublic, Depends(guard)]) -> list[dict[str, Any]]:
|
||||
return connections(storage_base, user.user_id)
|
||||
|
||||
@protected.get("/api/graph")
|
||||
async def graph_route(user: Annotated[UserPublic, Depends(guard)]) -> dict[str, Any]:
|
||||
return graph(storage_base, user.user_id)
|
||||
|
||||
@protected.get("/api/tasks")
|
||||
async def tasks_route(user: Annotated[UserPublic, Depends(guard)]) -> list[dict[str, str]]:
|
||||
return tasks(storage_base, user.user_id)
|
||||
|
||||
@protected.get("/api/suggestions")
|
||||
async def suggestions_route(
|
||||
user: Annotated[UserPublic, Depends(guard)]
|
||||
) -> list[dict[str, str]]:
|
||||
return suggestions(storage_base, user.user_id)
|
||||
|
||||
# ------------------------------------------------------------- T-3.6
|
||||
# Refine + profile (REQ-80/81). The refine planner calls ride the
|
||||
# SAME per-user credential chain as runs (P2 factory); an
|
||||
# unavailable model is an honest 503, never a hang.
|
||||
|
||||
class RefineDraftBody(BaseModel):
|
||||
draft: str = Field(min_length=1, max_length=2000)
|
||||
|
||||
class RefineComposeBody(BaseModel):
|
||||
draft: str = Field(min_length=1, max_length=2000)
|
||||
answers: list[str] = Field(default_factory=list, max_length=10)
|
||||
|
||||
_REFINE_QUESTIONS_PROMPT = (
|
||||
"You are a research question refiner. The user drafted a research "
|
||||
"question. Reply with a JSON object {{\"questions\": [...]}} holding "
|
||||
"exactly three short clarifying questions that would sharpen the "
|
||||
"draft. No commentary, JSON only.\nDraft: {draft}"
|
||||
)
|
||||
_REFINE_COMPOSE_PROMPT = (
|
||||
"You are a research question refiner. The user drafted a research "
|
||||
"question and answered clarifying questions. Compose the refined "
|
||||
"research question and a short suggested-context paragraph. Reply "
|
||||
"with a JSON object {{\"question\": \"...\", \"context\": \"...\"}}, "
|
||||
"JSON only.\nDraft: {draft}\nAnswers: {answers}"
|
||||
)
|
||||
|
||||
def _parse_refine_json(content: str) -> dict[str, Any]:
|
||||
try:
|
||||
parsed = json.loads(content)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise AdapterError("refine planner returned non-JSON") from exc
|
||||
if not isinstance(parsed, dict):
|
||||
raise AdapterError("refine planner returned non-JSON")
|
||||
return parsed
|
||||
|
||||
async def _refine_model(user: UserPublic) -> tuple[Any, str]:
|
||||
"""Resolve the per-user adapter for a refine call — the same
|
||||
chain as create_run (user -> server -> keyless), one fresh
|
||||
adapter instance per call (P-2)."""
|
||||
if model_factory is not None:
|
||||
settings = await settings_store.get(user.user_id)
|
||||
run_model, tier = await model_factory.resolve(user, settings)
|
||||
return run_model, tier
|
||||
if model is None: # factory wired but absent — impossible via create_app
|
||||
raise AdapterError("no model adapter available")
|
||||
return model, "keyless"
|
||||
|
||||
@protected.post("/api/refine/questions")
|
||||
async def refine_questions(
|
||||
body: RefineDraftBody, user: Annotated[UserPublic, Depends(guard)]
|
||||
) -> dict[str, Any]:
|
||||
draft = body.draft.strip()
|
||||
if not draft:
|
||||
raise HTTPException(status_code=422, detail="draft must not be empty")
|
||||
try:
|
||||
adapter, _tier = await _refine_model(user)
|
||||
result = await adapter.chat(
|
||||
[ModelMessage(role="user", content=_REFINE_QUESTIONS_PROMPT.format(draft=draft))],
|
||||
model=adapter.roles().planner,
|
||||
json_mode=True,
|
||||
)
|
||||
except AdapterError as exc:
|
||||
# Honest 503 (T-3.6): the resolved tier is unavailable —
|
||||
# never a hang, never a silent success.
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
parsed = _parse_refine_json(result.content)
|
||||
raw = parsed.get("questions", [])
|
||||
questions = [str(q).strip() for q in raw if str(q).strip()] if isinstance(raw, list) else []
|
||||
if not questions:
|
||||
raise HTTPException(status_code=503, detail="refine planner returned no questions")
|
||||
return {"questions": questions[:3]}
|
||||
|
||||
@protected.post("/api/refine/compose")
|
||||
async def refine_compose(
|
||||
body: RefineComposeBody, user: Annotated[UserPublic, Depends(guard)]
|
||||
) -> dict[str, str]:
|
||||
draft = body.draft.strip()
|
||||
if not draft:
|
||||
raise HTTPException(status_code=422, detail="draft must not be empty")
|
||||
answers = [str(a).strip() for a in body.answers if str(a).strip()]
|
||||
try:
|
||||
adapter, _tier = await _refine_model(user)
|
||||
result = await adapter.chat(
|
||||
[
|
||||
ModelMessage(
|
||||
role="user",
|
||||
content=_REFINE_COMPOSE_PROMPT.format(
|
||||
draft=draft, answers=json.dumps(answers)
|
||||
),
|
||||
)
|
||||
],
|
||||
model=adapter.roles().planner,
|
||||
json_mode=True,
|
||||
)
|
||||
except AdapterError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
parsed = _parse_refine_json(result.content)
|
||||
question = str(parsed.get("question", "")).strip()
|
||||
if not question:
|
||||
raise HTTPException(status_code=503, detail="refine planner returned no question")
|
||||
return {"question": question, "context": str(parsed.get("context", "")).strip()}
|
||||
|
||||
# ------------------------------------------------------------- profile
|
||||
|
||||
class ProfileBody(BaseModel):
|
||||
display_name: str | None = Field(default=None, min_length=1, max_length=120)
|
||||
email: str | None = Field(default=None, pattern=_EMAIL_RE, max_length=254)
|
||||
|
||||
@protected.get("/api/profile")
|
||||
async def get_profile(user: Annotated[UserPublic, Depends(guard)]) -> dict[str, str]:
|
||||
return {
|
||||
"user_id": user.user_id,
|
||||
"display_name": user.display_name,
|
||||
"email": user.email,
|
||||
}
|
||||
|
||||
@protected.put("/api/profile")
|
||||
async def put_profile(
|
||||
body: ProfileBody, user: Annotated[UserPublic, Depends(guard)]
|
||||
) -> dict[str, str]:
|
||||
"""Update display name and/or email (no deletion, REQ-81).
|
||||
The email stays unique (casefolded) across other users."""
|
||||
stored = users.get(user.user_id)
|
||||
if stored is None:
|
||||
raise HTTPException(status_code=401, detail="account disabled")
|
||||
try:
|
||||
updated = await users.update_profile(
|
||||
user.user_id,
|
||||
display_name=body.display_name,
|
||||
email=body.email,
|
||||
)
|
||||
except UserError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
return {
|
||||
"user_id": updated.user_id,
|
||||
"display_name": updated.display_name,
|
||||
"email": updated.email,
|
||||
}
|
||||
|
||||
app.include_router(public)
|
||||
app.include_router(protected)
|
||||
|
||||
# Startup sweep (D-115, G-101): every non-terminal state.json under
|
||||
# the storage base is rewritten as CANCELLED — in-flight actors died
|
||||
# with the previous process; rehydration is out of v0.3 scope. Runs
|
||||
# synchronously at app construction so the very first request
|
||||
# already sees a consistent disk.
|
||||
from researchx.api.sweep import sweep_in_flight_runs
|
||||
|
||||
sweep_in_flight_runs(storage_base)
|
||||
|
||||
# CORS is added AFTER the routers so it is the OUTERMOST middleware
|
||||
# (starlette builds the stack in add-order: last added = outermost)
|
||||
# — preflight OPTIONS is answered without any auth 401 (P-4/C3-04).
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=_CORS_ORIGINS,
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
return app
|
||||
|
||||
|
||||
class _NullSecrets:
|
||||
"""No-secrets provider: the default key store is disabled unless a
|
||||
secrets provider is injected (zero-key test harness stays green)."""
|
||||
|
||||
def resolve(self, name: str) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
# Dev-only module app (README's `uvicorn researchx.api.app:app`).
|
||||
# Pitfall P-12/§8 item 3: this serves FAKES with default storage —
|
||||
# the production entrypoint (researchx.api.main, Phase 2) must never
|
||||
# import this global. Single worker only (in-memory stores by design).
|
||||
app = create_app()
|
||||
@@ -0,0 +1,91 @@
|
||||
"""Dev bridge: serve the FastAPI app on a loopback port for E2E tests.
|
||||
|
||||
Prints READY on stdout once the server is listening; SIGTERM shuts
|
||||
down cleanly. Single-user, loopback-only, disposable — for tests and
|
||||
local dev. Uses an inexhaustible scripted model so many sequential
|
||||
runs work without restocking canned responses.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import uvicorn
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeSearchAdapter
|
||||
from researchx.adapters.types import ChatResult, ModelMessage
|
||||
from researchx.api.app import create_app
|
||||
|
||||
_HOST = "127.0.0.1"
|
||||
_PORT = 8000
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{
|
||||
"sub_questions": ["verification passes"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["approach", "findings"],
|
||||
}
|
||||
)
|
||||
_WRITER = (
|
||||
"# Research Memo\n\n"
|
||||
"The verified core finding [p-1]. A corroborated observation [p-2].\n"
|
||||
)
|
||||
|
||||
|
||||
class EndlessModelAdapter:
|
||||
def __init__(self) -> None:
|
||||
self.calls = 0
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ModelMessage],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.2,
|
||||
json_mode: bool = False,
|
||||
) -> ChatResult:
|
||||
del messages, temperature
|
||||
self.calls += 1
|
||||
if json_mode:
|
||||
return ChatResult(content=_PLAN, model=model)
|
||||
return ChatResult(content=_WRITER, model=model)
|
||||
|
||||
async def list_models(self) -> list[str]:
|
||||
return ["endless-model"]
|
||||
|
||||
def roles(self) -> Any:
|
||||
from researchx.adapters.model import ModelRoles
|
||||
|
||||
return ModelRoles(planner="endless-model", writer="endless-model")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
# Dev/E2E bridge: auth explicitly OFF (P-5) — a loopback-only,
|
||||
# disposable harness. Never the production entrypoint; the
|
||||
# production server (researchx.api.main, Phase 2) never passes
|
||||
# auth_enabled.
|
||||
app = create_app(
|
||||
storage_dir=Path(tmp),
|
||||
model_adapter=EndlessModelAdapter(),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=Path(tmp),
|
||||
)
|
||||
|
||||
@app.on_event("startup")
|
||||
def _announce() -> None:
|
||||
print(f"READY port={_PORT}", flush=True)
|
||||
|
||||
config = uvicorn.Config(app, host=_HOST, port=_PORT, log_level="warning")
|
||||
server = uvicorn.Server(config)
|
||||
with contextlib.suppress(KeyboardInterrupt):
|
||||
server.run()
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,87 @@
|
||||
"""Production entrypoint (T-2.7 — P-3 backend closure).
|
||||
|
||||
``uvicorn researchx.api.main:app`` — the ONLY serving entry the
|
||||
container ever runs (T-4.1). Distinct from ``researchx.api.app`` (the
|
||||
test/dev factory, whose module-global serves fakes):
|
||||
|
||||
- ``auth_enabled=True`` is HARDWIRED — not a parameter here, never an
|
||||
env var, never False on any path that serves traffic (REQ-85).
|
||||
- ``cookie_secure=True`` (E-4 handoff): the cookie is set with Secure
|
||||
because the public surface sits behind the TLS-fronted proxy.
|
||||
- Run artifacts + auth stores + key store + settings live under
|
||||
``RESEARCHX_DATA_DIR`` (container default ``/var/lib/researchx``,
|
||||
D-107) — never cwd-dependent.
|
||||
- Secrets come from the runtime env file (``.ciagent/.env.secrets``
|
||||
by default, overridable via ``RESEARCHX_SECRETS_FILE``; the
|
||||
container passes its own env-file via ``--env-file``):
|
||||
``RESEARCHX_KEK`` (keys scope) enables the per-user key store,
|
||||
``OLLAMA_API_KEY`` the server tier, ``TAVILY_API_KEY`` keyed search.
|
||||
- The model adapter is ALWAYS the real OllamaModelAdapter, constructed
|
||||
per run by the ModelAdapterFactory — this module can never serve
|
||||
FakeModelAdapter (pinned by test_main_entrypoint).
|
||||
- Search: TavilySearchAdapter when TAVILY_API_KEY is present, else the
|
||||
documented keyless DuckDuckGo fallback (degraded mode — G-008).
|
||||
- Single uvicorn worker only (in-memory RunRegistry + session store).
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import FastAPI
|
||||
|
||||
from researchx.adapters.duckduckgo import resolve_search_adapter
|
||||
from researchx.adapters.model_factory import ModelAdapterFactory
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.api.app import create_app
|
||||
from researchx.auth.key_store import KeyStore
|
||||
from researchx.auth.settings import SettingsStore
|
||||
from researchx.config.paths import data_root
|
||||
|
||||
_DEFAULT_SECRETS_PATHS = [
|
||||
Path("/etc/researchx/env.secrets"), # container convention (runtime bind)
|
||||
Path.cwd() / ".ciagent" / ".env.secrets", # repo-local dev
|
||||
Path.cwd().parents[1] / ".ciagent" / ".env.secrets", # tests run from backend/
|
||||
]
|
||||
|
||||
|
||||
def _secrets_paths() -> list[Path]:
|
||||
"""Runtime env-file candidates (first existing wins per merge order)."""
|
||||
import os
|
||||
|
||||
override = os.environ.get("RESEARCHX_SECRETS_FILE")
|
||||
if override:
|
||||
return [Path(override)]
|
||||
return [p for p in _DEFAULT_SECRETS_PATHS if p.is_file()] or _DEFAULT_SECRETS_PATHS
|
||||
|
||||
|
||||
def build_app() -> FastAPI:
|
||||
"""Build the LIVE app: real adapters, real auth, data-root storage.
|
||||
|
||||
``auth_enabled`` and ``cookie_secure`` are not parameters — they
|
||||
are the production posture, fixed here (C3-16/REQ-85).
|
||||
"""
|
||||
secrets = EnvFileSecretsProvider(paths=_secrets_paths())
|
||||
root = data_root()
|
||||
|
||||
key_store = KeyStore(secrets, root)
|
||||
settings_store = SettingsStore(root)
|
||||
factory = ModelAdapterFactory(secrets, key_store)
|
||||
|
||||
# Tavily when keyed, documented keyless DuckDuckGo otherwise (G-008).
|
||||
search, _search_name = resolve_search_adapter(secrets)
|
||||
|
||||
return create_app(
|
||||
storage_dir=root,
|
||||
data_dir=root,
|
||||
search_adapter=search,
|
||||
secrets_provider=secrets,
|
||||
key_store=key_store,
|
||||
settings_store=settings_store,
|
||||
model_factory=factory,
|
||||
auth_enabled=True, # HARDWIRED — the parameter exists for the test factory only
|
||||
cookie_secure=True, # behind the TLS-front proxy (E-4)
|
||||
)
|
||||
|
||||
|
||||
# Module-level app for `uvicorn researchx.api.main:app`. This is the
|
||||
# production object; it is built from live wiring (never fakes).
|
||||
app = build_app()
|
||||
@@ -0,0 +1,505 @@
|
||||
"""Disk read-models (T-3.2, D-114, D-115 — REQ-76..79).
|
||||
|
||||
Pure functions over ``runs/<owner_id>/<run_id>/`` disk artifacts —
|
||||
NEVER the in-memory RunRegistry (D-115): every read endpoint stays
|
||||
correct after a restart with zero in-process actors (P-8 full
|
||||
closure). Payloads are shaped EXACTLY to the existing frontend types
|
||||
(``frontend/src/lib/types.ts``): ``ReportLibrary``/``ReportSummary``,
|
||||
``ReportConnection``, ``ProjectGraph`` nodes/edges, ``InboxItem``,
|
||||
``SuggestedQuery`` — camelCase, field-for-field, so T-3.9's surface
|
||||
swaps need zero component rewrites (REQ-54).
|
||||
|
||||
Categorization is deterministic + dumb by design: a methodology
|
||||
folder (``deep-report``/``rapid-answer``) plus topic tokens from plan
|
||||
sub-questions. No model calls, no synthesis — the same disk tree
|
||||
always categorizes identically.
|
||||
|
||||
Open-question semantics (D-114): inbox items are plan sub-questions
|
||||
of the user's runs, ``done`` iff the run reached DONE — anything
|
||||
else (post-sweep CANCELLED included) stays ``open``; suggestions are
|
||||
those open items joined with source run titles — never synthetic.
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
# --------------------------------------------------------------- wire shapes
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReportFolder:
|
||||
"""matches frontend ReportFolder — id == ReportSummary.category."""
|
||||
|
||||
id: str
|
||||
name: str
|
||||
description: str
|
||||
|
||||
def payload(self) -> dict[str, str]:
|
||||
return {"id": self.id, "name": self.name, "description": self.description}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReportSummary:
|
||||
"""matches frontend ReportSummary — the library card."""
|
||||
|
||||
id: str
|
||||
runId: str
|
||||
title: str
|
||||
question: str
|
||||
methodologySlug: str
|
||||
category: str
|
||||
generatedAt: str
|
||||
citationCount: int
|
||||
|
||||
def payload(self) -> dict[str, Any]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"runId": self.runId,
|
||||
"title": self.title,
|
||||
"question": self.question,
|
||||
"methodologySlug": self.methodologySlug,
|
||||
"category": self.category,
|
||||
"generatedAt": self.generatedAt,
|
||||
"citationCount": self.citationCount,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReportConnection:
|
||||
"""matches frontend ReportConnection — a shared-source/topic edge."""
|
||||
|
||||
fromReportId: str
|
||||
toReportId: str
|
||||
kind: str # "shared-sources" | "related-topic"
|
||||
label: str
|
||||
|
||||
def payload(self) -> dict[str, str]:
|
||||
return {
|
||||
"fromReportId": self.fromReportId,
|
||||
"toReportId": self.toReportId,
|
||||
"kind": self.kind,
|
||||
"label": self.label,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GraphNode:
|
||||
id: str
|
||||
topic: str
|
||||
linkedSources: list[str]
|
||||
findings: list[str]
|
||||
|
||||
def payload(self) -> dict[str, Any]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"topic": self.topic,
|
||||
"linkedSources": self.linkedSources,
|
||||
"findings": self.findings,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GraphEdge:
|
||||
fromId: str
|
||||
toId: str
|
||||
sharedSourceCount: int
|
||||
|
||||
def payload(self) -> dict[str, Any]:
|
||||
return {
|
||||
"fromId": self.fromId,
|
||||
"toId": self.toId,
|
||||
"sharedSourceCount": self.sharedSourceCount,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class InboxItem:
|
||||
"""matches frontend InboxItem — "<run_id>#<idx>" ids (D-114)."""
|
||||
|
||||
id: str
|
||||
question: str
|
||||
sourceRunId: str
|
||||
status: str # "open" | "done" (in_progress is mock-only vocabulary)
|
||||
|
||||
def payload(self) -> dict[str, str]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"question": self.question,
|
||||
"sourceRunId": self.sourceRunId,
|
||||
"status": self.status,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SuggestedQuery:
|
||||
"""matches frontend SuggestedQuery — an open item + run title."""
|
||||
|
||||
id: str
|
||||
question: str
|
||||
sourceRunId: str
|
||||
sourceReportTitle: str
|
||||
|
||||
def payload(self) -> dict[str, str]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"question": self.question,
|
||||
"sourceRunId": self.sourceRunId,
|
||||
"sourceReportTitle": self.sourceReportTitle,
|
||||
}
|
||||
|
||||
|
||||
# ------------------------------------------------------------- disk scanning
|
||||
|
||||
_METHOD_NAMES = {"rapid-answer": "Rapid Answer", "deep-report": "Deep Report"}
|
||||
_QUOTE_MAX = 200
|
||||
|
||||
|
||||
def _is_scannable(run_dir: Path) -> bool:
|
||||
"""Defensive scan filter (D-115): dotfiles/tmp leftovers skipped."""
|
||||
name = run_dir.name
|
||||
return run_dir.is_dir() and not name.startswith(".") and ".tmp" not in name
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunDiskRecord:
|
||||
"""One run as it sits on disk — the read-model's only input."""
|
||||
|
||||
run_id: str
|
||||
state: dict[str, Any]
|
||||
report: dict[str, Any] | None = None
|
||||
ledger: dict[str, Any] | None = None
|
||||
# derived, filled by load:
|
||||
status: str = field(default="", init=False)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
self.status = str(self.state.get("state", ""))
|
||||
|
||||
@property
|
||||
def is_done(self) -> bool:
|
||||
return self.status == "DONE"
|
||||
|
||||
@property
|
||||
def question(self) -> str:
|
||||
return str(self.state.get("question", ""))
|
||||
|
||||
@property
|
||||
def methodology_slug(self) -> str:
|
||||
return str(self.state.get("methodology_slug", ""))
|
||||
|
||||
@property
|
||||
def created_at(self) -> str:
|
||||
return str(self.state.get("created_at", ""))
|
||||
|
||||
@property
|
||||
def sub_questions(self) -> list[str]:
|
||||
plan = self.state.get("plan")
|
||||
if not isinstance(plan, dict):
|
||||
return []
|
||||
raw = plan.get("sub_questions", [])
|
||||
if not isinstance(raw, list):
|
||||
return []
|
||||
return [str(q) for q in raw]
|
||||
|
||||
def report_title(self) -> str:
|
||||
"""The report.json title, else methodology-name — question."""
|
||||
if self.report is not None and self.report.get("title"):
|
||||
return str(self.report["title"])
|
||||
name = _METHOD_NAMES.get(self.methodology_slug, "Research")
|
||||
return f"{name} — {self.question}"
|
||||
|
||||
def citation_count(self) -> int:
|
||||
if self.report is not None:
|
||||
citations = self.report.get("citations")
|
||||
if isinstance(citations, list):
|
||||
return len(citations)
|
||||
return 0
|
||||
|
||||
def ledger_urls(self) -> list[str]:
|
||||
if self.ledger is None:
|
||||
return []
|
||||
entries = self.ledger.get("entries")
|
||||
if not isinstance(entries, list):
|
||||
return []
|
||||
return [str(e.get("url")) for e in entries if isinstance(e, dict) and e.get("url")]
|
||||
|
||||
|
||||
def _read_json(path: Path) -> dict[str, Any] | None:
|
||||
try:
|
||||
raw: object = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError, ValueError):
|
||||
return None
|
||||
return raw if isinstance(raw, dict) else None
|
||||
|
||||
|
||||
def list_runs(base_dir: Path, owner_id: str) -> list[RunDiskRecord]:
|
||||
"""Scan this owner's partition: state.json (+ report.json,
|
||||
ledger.json where present). Dotfiles/tmp skipped defensively
|
||||
(D-115); corrupt or stateless directories are skipped, never
|
||||
fatal. Records are ordered most-recent first (created_at desc,
|
||||
run_id as the stable tiebreaker)."""
|
||||
runs_dir = base_dir / "runs" / owner_id
|
||||
if not runs_dir.is_dir():
|
||||
return []
|
||||
records: list[RunDiskRecord] = []
|
||||
for run_dir in sorted(runs_dir.iterdir()):
|
||||
if not _is_scannable(run_dir):
|
||||
continue
|
||||
state = _read_json(run_dir / "state.json")
|
||||
if state is None:
|
||||
continue
|
||||
records.append(
|
||||
RunDiskRecord(
|
||||
run_id=run_dir.name,
|
||||
state=state,
|
||||
report=_read_json(run_dir / "report.json"),
|
||||
ledger=_read_json(run_dir / "ledger.json"),
|
||||
)
|
||||
)
|
||||
records.sort(key=lambda r: (r.created_at, r.run_id), reverse=True)
|
||||
return records
|
||||
|
||||
|
||||
# ------------------------------------------------------- categorization (dumb)
|
||||
|
||||
_STOPWORDS = {
|
||||
"the", "a", "an", "of", "in", "on", "and", "or", "to", "for", "is",
|
||||
"are", "what", "which", "how", "do", "does", "why", "when", "with",
|
||||
"that", "this", "these", "those", "it", "its", "by", "at", "be",
|
||||
"can", "could", "should", "will", "into", "from", "about", "across",
|
||||
}
|
||||
|
||||
_TOPIC_TOKEN_RE = re.compile(r"[a-z][a-z0-9-]*")
|
||||
|
||||
|
||||
def topic_tokens(record: RunDiskRecord) -> list[str]:
|
||||
"""Deterministic topic tokens: salient words of the plan
|
||||
sub-questions (lowercased, deduped, order-preserving)."""
|
||||
tokens: list[str] = []
|
||||
for question in record.sub_questions:
|
||||
for token in _TOPIC_TOKEN_RE.findall(question.lower()):
|
||||
if token not in _STOPWORDS and token not in tokens:
|
||||
tokens.append(token)
|
||||
return tokens
|
||||
|
||||
|
||||
def categorize(record: RunDiskRecord) -> str:
|
||||
"""The folder id a report files under: methodology + topic token.
|
||||
|
||||
Deterministic + dumb (REQ-76): the methodology forms the folder
|
||||
base and the first topic token of the plan names the sub-folder —
|
||||
the same disk tree always categorizes identically. Runs without a
|
||||
plan (pre-approval) fall back to the bare methodology slug; a run
|
||||
with neither resolves to "research".
|
||||
"""
|
||||
slug = record.methodology_slug or "research"
|
||||
tokens = topic_tokens(record)
|
||||
if not tokens:
|
||||
return slug
|
||||
return f"{slug}-{tokens[0]}"
|
||||
|
||||
|
||||
def _category_name(category: str) -> str:
|
||||
"""Humanize a category id: 'deep-report-battery' -> 'Deep Report:
|
||||
Battery'. The methodology slug (multi-hyphen, e.g.
|
||||
'deep-report') is stripped as a unit before the topic token is
|
||||
titled."""
|
||||
for slug, name in _METHOD_NAMES.items():
|
||||
prefix = f"{slug}-"
|
||||
if category.startswith(prefix):
|
||||
tail = category[len(prefix) :]
|
||||
return f"{name}: {tail.replace('-', ' ').title()}" if tail else name
|
||||
return category.replace("-", " ").title()
|
||||
|
||||
|
||||
def _category_description(category: str) -> str:
|
||||
return f"Auto-filed from {category.replace('-', ' ')} research."
|
||||
|
||||
|
||||
def _summary(record: RunDiskRecord) -> ReportSummary:
|
||||
generated_at = (
|
||||
str(record.report.get("generated_at")) if record.report is not None else record.created_at
|
||||
)
|
||||
return ReportSummary(
|
||||
id=f"rep-{record.run_id}",
|
||||
runId=record.run_id,
|
||||
title=record.report_title(),
|
||||
question=record.question,
|
||||
methodologySlug=record.methodology_slug,
|
||||
category=categorize(record),
|
||||
generatedAt=generated_at or datetime.now(UTC).isoformat(),
|
||||
citationCount=record.citation_count(),
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------- library
|
||||
|
||||
|
||||
def library(base_dir: Path, owner_id: str, sort: str = "recent") -> dict[str, Any]:
|
||||
"""GET /api/library payload: folders + ReportSummary[] (REQ-76).
|
||||
|
||||
Only DONE runs with a report are "stored reports" — a cancelled or
|
||||
failed run produced nothing worth filing. ``sort`` is "recent"
|
||||
(default, created_at desc — the scan order) or "title" (A-Z).
|
||||
"""
|
||||
if sort not in ("recent", "title"):
|
||||
sort = "recent"
|
||||
done = [r for r in list_runs(base_dir, owner_id) if r.is_done]
|
||||
reports = [_summary(r) for r in done]
|
||||
if sort == "title":
|
||||
reports.sort(key=lambda summary: (summary.title.casefold(), summary.runId))
|
||||
seen: dict[str, int] = {}
|
||||
for summary in reports:
|
||||
seen[summary.category] = seen.get(summary.category, 0) + 1
|
||||
folders = [
|
||||
ReportFolder(
|
||||
id=category,
|
||||
name=_category_name(category),
|
||||
description=_category_description(category),
|
||||
).payload()
|
||||
for category in sorted(seen)
|
||||
]
|
||||
return {
|
||||
"folders": folders,
|
||||
"reports": [summary.payload() for summary in reports],
|
||||
}
|
||||
|
||||
|
||||
# ------------------------------------------------------------ connections
|
||||
|
||||
|
||||
def _shared_urls(first: RunDiskRecord, second: RunDiskRecord) -> set[str]:
|
||||
return set(first.ledger_urls()) & set(second.ledger_urls())
|
||||
|
||||
|
||||
def connections(base_dir: Path, owner_id: str) -> list[dict[str, Any]]:
|
||||
"""GET /api/library/connections payload (REQ-77): shared-source
|
||||
edges (>=1 ledger URL overlap) + related-topic edges (same
|
||||
methodology + shared category). Between the user's DONE runs
|
||||
only; edges point older -> newer (the fixture convention).
|
||||
"""
|
||||
done = list(reversed([r for r in list_runs(base_dir, owner_id) if r.is_done]))
|
||||
edges: list[ReportConnection] = []
|
||||
for i, first in enumerate(done):
|
||||
for second in done[i + 1 :]:
|
||||
shared = _shared_urls(first, second)
|
||||
if shared:
|
||||
edges.append(
|
||||
ReportConnection(
|
||||
fromReportId=f"rep-{first.run_id}",
|
||||
toReportId=f"rep-{second.run_id}",
|
||||
kind="shared-sources",
|
||||
label=f"{len(shared)} shared source{'s' if len(shared) != 1 else ''}",
|
||||
)
|
||||
)
|
||||
elif categorize(first) == categorize(second):
|
||||
edges.append(
|
||||
ReportConnection(
|
||||
fromReportId=f"rep-{first.run_id}",
|
||||
toReportId=f"rep-{second.run_id}",
|
||||
kind="related-topic",
|
||||
label="Adjacent topic",
|
||||
)
|
||||
)
|
||||
return [edge.payload() for edge in edges]
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- graph
|
||||
|
||||
|
||||
def _report_finding(record: RunDiskRecord) -> str:
|
||||
"""The node's first 'finding': the report's title (a real artifact
|
||||
value — never fabricated prose)."""
|
||||
return record.report_title()
|
||||
|
||||
|
||||
def graph(base_dir: Path, owner_id: str) -> dict[str, Any]:
|
||||
"""GET /api/graph payload (REQ-77): DONE runs as GraphNodes, the
|
||||
same edges as connections() as GraphEdges."""
|
||||
done = list(reversed([r for r in list_runs(base_dir, owner_id) if r.is_done]))
|
||||
nodes = [
|
||||
GraphNode(
|
||||
id=record.run_id,
|
||||
topic=record.report_title(),
|
||||
linkedSources=sorted(set(record.ledger_urls())),
|
||||
findings=[_report_finding(record)],
|
||||
).payload()
|
||||
for record in done
|
||||
]
|
||||
edges: list[GraphEdge] = []
|
||||
for i, first in enumerate(done):
|
||||
for second in done[i + 1 :]:
|
||||
shared = _shared_urls(first, second)
|
||||
if shared:
|
||||
edges.append(GraphEdge(first.run_id, second.run_id, len(shared)))
|
||||
elif categorize(first) == categorize(second):
|
||||
edges.append(GraphEdge(first.run_id, second.run_id, 0))
|
||||
return {"nodes": nodes, "edges": [edge.payload() for edge in edges]}
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- tasks
|
||||
|
||||
|
||||
def tasks(base_dir: Path, owner_id: str) -> list[dict[str, str]]:
|
||||
"""GET /api/tasks payload (REQ-78, D-114): plan sub-questions as
|
||||
InboxItems, ``done`` iff the source run reached DONE — post-sweep
|
||||
CANCELLED runs surface ``open``, never answered."""
|
||||
items: list[InboxItem] = []
|
||||
for record in list_runs(base_dir, owner_id):
|
||||
status = "done" if record.is_done else "open"
|
||||
for idx, question in enumerate(record.sub_questions):
|
||||
items.append(
|
||||
InboxItem(
|
||||
id=f"{record.run_id}#{idx}",
|
||||
question=question,
|
||||
sourceRunId=record.run_id,
|
||||
status=status,
|
||||
)
|
||||
)
|
||||
return [item.payload() for item in items]
|
||||
|
||||
|
||||
# -------------------------------------------------------------- suggestions
|
||||
|
||||
|
||||
def suggestions(base_dir: Path, owner_id: str) -> list[dict[str, str]]:
|
||||
"""GET /api/suggestions payload (REQ-79): the OPEN inbox items
|
||||
joined with their source run titles — derived from the user's own
|
||||
runs only, never synthetic; empty for a fresh user."""
|
||||
suggested: list[SuggestedQuery] = []
|
||||
for record in list_runs(base_dir, owner_id):
|
||||
if record.is_done:
|
||||
continue
|
||||
for idx, question in enumerate(record.sub_questions):
|
||||
suggested.append(
|
||||
SuggestedQuery(
|
||||
id=f"sug-{record.run_id}#{idx}",
|
||||
question=question,
|
||||
sourceRunId=record.run_id,
|
||||
sourceReportTitle=record.report_title(),
|
||||
)
|
||||
)
|
||||
return [item.payload() for item in suggested]
|
||||
|
||||
|
||||
__all__ = [
|
||||
"GraphEdge",
|
||||
"GraphNode",
|
||||
"InboxItem",
|
||||
"ReportConnection",
|
||||
"ReportFolder",
|
||||
"ReportSummary",
|
||||
"RunDiskRecord",
|
||||
"SuggestedQuery",
|
||||
"categorize",
|
||||
"connections",
|
||||
"graph",
|
||||
"library",
|
||||
"list_runs",
|
||||
"suggestions",
|
||||
"tasks",
|
||||
"topic_tokens",
|
||||
]
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Startup sweep (D-115, G-101).
|
||||
|
||||
On app construction, every non-terminal ``state.json`` under the
|
||||
storage base is rewritten as CANCELLED — in-flight actors died with the
|
||||
previous process; rehydration is out of v0.3 scope. CANCELLED is legal
|
||||
from every non-terminal state (states.py — including PLANNING per
|
||||
G-101, pinned by test). The registry-side pruning lives in app.py
|
||||
(RunRegistry.prune, P-6).
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.engine.run_state import RunState
|
||||
from researchx.engine.states import TERMINAL_STATES, RunStateName
|
||||
|
||||
|
||||
def _atomic_write(path: Path, content: str) -> None:
|
||||
import os
|
||||
import secrets
|
||||
|
||||
tmp_path = path.with_name(f"{path.name}.{secrets.token_hex(4)}.tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as handle:
|
||||
handle.write(content)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
tmp_path.replace(path)
|
||||
|
||||
|
||||
def sweep_in_flight_runs(base_dir: Path) -> list[tuple[str, str]]:
|
||||
"""Rewrite every non-terminal state.json under base_dir as CANCELLED.
|
||||
|
||||
Scans ``base_dir/runs/<owner_id>/<run_id>/state.json`` (the
|
||||
partitioned layout, D-106). Returns the swept (owner_id, run_id)
|
||||
pairs. Corrupt state files are skipped defensively — a sweep must
|
||||
never prevent startup.
|
||||
"""
|
||||
swept: list[tuple[str, str]] = []
|
||||
runs_dir = base_dir / "runs"
|
||||
if not runs_dir.is_dir():
|
||||
return swept
|
||||
for owner_dir in sorted(runs_dir.iterdir()):
|
||||
if not owner_dir.is_dir() or owner_dir.name.startswith("."):
|
||||
continue
|
||||
for run_dir in sorted(owner_dir.iterdir()):
|
||||
if not run_dir.is_dir() or run_dir.name.startswith("."):
|
||||
continue
|
||||
state_path = run_dir / "state.json"
|
||||
if not state_path.is_file():
|
||||
continue
|
||||
try:
|
||||
state = RunState.model_validate_json(state_path.read_text(encoding="utf-8"))
|
||||
except (ValueError, OSError):
|
||||
# Corrupt/unreadable state files are not swept, never
|
||||
# fatal — a sweep must never prevent startup.
|
||||
continue
|
||||
if state.state in TERMINAL_STATES:
|
||||
continue
|
||||
state.state = RunStateName.CANCELLED # legal from every non-terminal state (G-101)
|
||||
_atomic_write(state_path, state.model_dump_json())
|
||||
swept.append((owner_dir.name, run_dir.name))
|
||||
return swept
|
||||
@@ -0,0 +1,24 @@
|
||||
"""Auth package: user accounts, sessions, guard, and routes (REQ-69/70/71)."""
|
||||
|
||||
from researchx.auth.models import (
|
||||
AdminCreateUserBody,
|
||||
LoginBody,
|
||||
RegisterBody,
|
||||
UserPublic,
|
||||
to_public,
|
||||
)
|
||||
from researchx.auth.sessions import Session, SessionStore
|
||||
from researchx.auth.users import User, UserError, UserStore
|
||||
|
||||
__all__ = [
|
||||
"AdminCreateUserBody",
|
||||
"LoginBody",
|
||||
"RegisterBody",
|
||||
"Session",
|
||||
"SessionStore",
|
||||
"User",
|
||||
"UserError",
|
||||
"UserPublic",
|
||||
"UserStore",
|
||||
"to_public",
|
||||
]
|
||||
@@ -0,0 +1,56 @@
|
||||
"""Admin user-management routes (T-1.5, REQ-70).
|
||||
|
||||
Mounted on the PROTECTED router (session + CSRF already enforced by
|
||||
``require_user``); every route additionally requires the admin role.
|
||||
Responses expose ``UserPublic`` only — never a password hash. Account
|
||||
creation enforces the same >=12 password policy as registration;
|
||||
deactivation sets ``active=False`` and immediately revokes every
|
||||
session of the target user (data is retained).
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
|
||||
from researchx.auth.guard import Guard
|
||||
from researchx.auth.models import AdminCreateUserBody, UserPublic, to_public
|
||||
from researchx.auth.sessions import SessionStore
|
||||
from researchx.auth.users import UserError, UserStore
|
||||
|
||||
|
||||
def build_admin_router(
|
||||
users: UserStore,
|
||||
sessions: SessionStore,
|
||||
admin_guard: Guard,
|
||||
) -> APIRouter:
|
||||
"""Build the admin user surface (mounted at /api/admin, admin-only)."""
|
||||
router = APIRouter(
|
||||
prefix="/api/admin",
|
||||
tags=["admin"],
|
||||
dependencies=[Depends(admin_guard)],
|
||||
)
|
||||
|
||||
@router.get("/users", response_model=list[UserPublic])
|
||||
async def list_users() -> list[UserPublic]:
|
||||
return [to_public(u) for u in users.list()]
|
||||
|
||||
@router.post("/users", status_code=201, response_model=UserPublic)
|
||||
async def create_user(body: AdminCreateUserBody) -> UserPublic:
|
||||
try:
|
||||
user = await users.create_user(
|
||||
email=body.email,
|
||||
password=body.password,
|
||||
display_name=body.display_name or body.email.split("@")[0],
|
||||
role=body.role,
|
||||
)
|
||||
except UserError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
return to_public(user)
|
||||
|
||||
@router.post("/users/{user_id}/deactivate", response_model=UserPublic)
|
||||
async def deactivate_user(user_id: str) -> UserPublic:
|
||||
updated = users.deactivate(user_id)
|
||||
if updated is None:
|
||||
raise HTTPException(status_code=404, detail="user not found")
|
||||
sessions.revoke_all_for(user_id) # deactivation kills sessions immediately
|
||||
return to_public(updated)
|
||||
|
||||
return router
|
||||
@@ -0,0 +1,109 @@
|
||||
"""Auth guard — session + CSRF enforcement as route dependencies (T-1.4).
|
||||
|
||||
Deliberately NOT middleware: router dependencies appear per-route, are
|
||||
overridable via ``app.dependency_overrides``, and never wrap SSE
|
||||
response bodies (no BaseHTTPMiddleware near the event stream).
|
||||
|
||||
Enforcement order:
|
||||
1. missing/invalid/expired session -> 401 "authentication required"
|
||||
2. non-GET without the CSRF header -> 403 "missing request header"
|
||||
3. missing or deactivated user -> 401 "account disabled"
|
||||
then the sliding-expiry ``touch()``.
|
||||
|
||||
When auth is disabled (dev/test harness only — a create_app parameter,
|
||||
never an env var readable in production, C3-16), the dev guard
|
||||
resolves a synthetic admin user for every request so all routes behave
|
||||
exactly as authenticated.
|
||||
"""
|
||||
|
||||
from collections.abc import Awaitable, Callable
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import HTTPException, Request
|
||||
|
||||
from researchx.auth.models import UserPublic, to_public
|
||||
from researchx.auth.sessions import Session, SessionStore
|
||||
from researchx.auth.users import User, UserStore
|
||||
|
||||
SESSION_COOKIE = "researchx_session"
|
||||
CSRF_HEADER = "x-researchx-request"
|
||||
|
||||
DEV_USER_ID = "dev00000"
|
||||
|
||||
Guard = Callable[[Request], Awaitable[UserPublic]]
|
||||
|
||||
|
||||
def _dev_user() -> User:
|
||||
"""The synthetic admin every request resolves when auth is disabled."""
|
||||
return User(
|
||||
user_id=DEV_USER_ID,
|
||||
email="dev@localhost",
|
||||
password_hash="", # never persisted, never verified
|
||||
display_name="Dev User",
|
||||
role="admin",
|
||||
active=True,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
|
||||
|
||||
def require_csrf_header(request: Request) -> None:
|
||||
"""CSRF defense for public POSTs (login/register) — same rule as the guard."""
|
||||
if request.method != "GET" and CSRF_HEADER not in request.headers:
|
||||
raise HTTPException(status_code=403, detail="missing request header")
|
||||
|
||||
|
||||
def build_guard(users: UserStore, sessions: SessionStore) -> Guard:
|
||||
"""Build the authenticated-mode require_user dependency."""
|
||||
|
||||
async def require_user(request: Request) -> UserPublic:
|
||||
token = request.cookies.get(SESSION_COOKIE)
|
||||
session: Session | None = sessions.get(token) if token else None
|
||||
if session is None:
|
||||
raise HTTPException(status_code=401, detail="authentication required")
|
||||
if request.method != "GET" and CSRF_HEADER not in request.headers:
|
||||
raise HTTPException(status_code=403, detail="missing request header")
|
||||
user: User | None = users.get(session.user_id)
|
||||
if user is None or not user.active:
|
||||
raise HTTPException(status_code=401, detail="account disabled")
|
||||
sessions.touch(session) # sliding expiry
|
||||
return to_public(user)
|
||||
|
||||
return require_user
|
||||
|
||||
|
||||
def build_dev_guard() -> Guard:
|
||||
"""Build the dev-mode guard: every request resolves the synthetic dev user."""
|
||||
|
||||
async def require_user(request: Request) -> UserPublic:
|
||||
del request # dev mode: no session, no CSRF — routes behave as before
|
||||
return to_public(_dev_user())
|
||||
|
||||
return require_user
|
||||
|
||||
|
||||
def build_admin_guard(guard: Guard) -> Guard:
|
||||
"""Wrap a guard with the admin role check (REQ-70).
|
||||
|
||||
The returned dependency resolves the wrapped guard first, so
|
||||
``app.dependency_overrides`` on either composes correctly.
|
||||
"""
|
||||
|
||||
async def require_admin(request: Request) -> UserPublic:
|
||||
user = await guard(request)
|
||||
if user.role != "admin":
|
||||
raise HTTPException(status_code=403, detail="admin role required")
|
||||
return user
|
||||
|
||||
return require_admin
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CSRF_HEADER",
|
||||
"DEV_USER_ID",
|
||||
"SESSION_COOKIE",
|
||||
"Guard",
|
||||
"build_admin_guard",
|
||||
"build_dev_guard",
|
||||
"build_guard",
|
||||
"require_csrf_header",
|
||||
]
|
||||
@@ -0,0 +1,336 @@
|
||||
"""Per-user encrypted key store (T-2.2, REQ-73 — D-108/D-109).
|
||||
|
||||
AES-256-GCM envelope encryption of per-user model keys:
|
||||
|
||||
- KEK: ``RESEARCHX_KEK`` from the env-file secrets **'keys' scope**,
|
||||
accepted as urlsafe-base64 or hex, exactly 32 bytes (AES-256). Decode
|
||||
or length errors raise typed ``AdapterError`` (never a silent
|
||||
fallback). With no KEK configured the store is DISABLED — honestly,
|
||||
with a reason string — and the credential chain degrades to
|
||||
server/keyless (C3-06).
|
||||
- AAD: ``user_id.encode()`` — a key file copied under another user_id
|
||||
fails ``InvalidTag`` at decrypt time (cross-user binding by
|
||||
construction).
|
||||
- Nonce: fresh ``os.urandom(12)`` per write; a GCM nonce must NEVER
|
||||
repeat under the same KEK.
|
||||
- File: ``keys/<user_id>.json`` under the data root — ``{v, alg,
|
||||
updated_at, entries}`` — written atomically (tmp + ``os.replace``)
|
||||
with mode 0600.
|
||||
- Crypto runs in ``asyncio.to_thread`` so the event loop is never
|
||||
blocked by file I/O.
|
||||
- Scope: ``OLLAMA_API_KEY`` only in v0.3 (D-109).
|
||||
|
||||
KEK-loss posture (G-105, binding):
|
||||
The KEK is NOT recoverable from anything in this repository or in
|
||||
the running process. If the owner loses it, every per-user
|
||||
encrypted key becomes unrecoverable — by design, that is the point
|
||||
of envelope encryption. The honest degradation is: the key store
|
||||
reports disabled, the resolution chain falls to server/keyless,
|
||||
users re-enter their keys under a re-provisioned KEK, and the owner
|
||||
MUST back up the KEK off-repo. Rotation tooling is out of v0.3
|
||||
scope (recorded). A changed KEK requires a process restart — the
|
||||
secrets provider loads once at construction.
|
||||
|
||||
Corruption is never a silent tier drop: ``InvalidTag`` (AAD tamper,
|
||||
wrong KEK, garbage ciphertext) and malformed envelopes both surface as
|
||||
typed errors (``KeyStoreError``), so the credential chain can report an
|
||||
honest error state instead of quietly falling back.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import os
|
||||
import secrets
|
||||
import threading
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from cryptography.exceptions import InvalidTag
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
from pydantic import BaseModel, Field, ValidationError
|
||||
|
||||
from researchx.adapters.secrets import SecretsProvider
|
||||
from researchx.adapters.types import AdapterError
|
||||
|
||||
KEK_NAME = "RESEARCHX_KEK"
|
||||
KEY_SCOPE = "OLLAMA_API_KEY"
|
||||
_NONCE_BYTES = 12
|
||||
_KEY_BYTES = 32
|
||||
|
||||
|
||||
class KeyStoreError(AdapterError):
|
||||
"""Typed key-store failure: corrupt/tampered envelope, bad KEK shape.
|
||||
|
||||
``kind`` lets callers (routes, factory, tests) distinguish the
|
||||
failure mode without parsing messages:
|
||||
|
||||
- ``disabled`` — no usable KEK; the store refuses honestly.
|
||||
- ``unreadable`` — the on-disk envelope is corrupt garbage.
|
||||
- ``tampered`` — decrypt failed (InvalidTag: wrong user AAD,
|
||||
wrong KEK, or bit-rot ciphertext). NEVER a
|
||||
silent None — the tier must not silently drop.
|
||||
- ``bad_kek`` — the configured KEK is malformed (decode/length).
|
||||
"""
|
||||
|
||||
def __init__(self, kind: str, message: str) -> None:
|
||||
super().__init__(message)
|
||||
self.kind = kind
|
||||
|
||||
|
||||
class _KeyEntry(BaseModel):
|
||||
nonce: str
|
||||
ct: str
|
||||
updated_at: str
|
||||
|
||||
|
||||
class _KeyEnvelope(BaseModel):
|
||||
v: int
|
||||
alg: str = "AES-256-GCM"
|
||||
updated_at: str
|
||||
entries: dict[str, _KeyEntry] = Field(default_factory=dict)
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(UTC).isoformat()
|
||||
|
||||
|
||||
def load_kek_bytes(secrets: SecretsProvider) -> bytes | None:
|
||||
"""Resolve ``RESEARCHX_KEK`` to raw 32 key bytes.
|
||||
|
||||
Accepted encodings: urlsafe-base64 (with or without padding) or
|
||||
hex. Anything that decodes but does not yield exactly 32 bytes, or
|
||||
fails to decode at all, raises ``KeyStoreError(kind="bad_kek")``
|
||||
(the research-file's typed AdapterError contract). ``None`` is
|
||||
returned only when the KEK is simply absent — the honest disabled
|
||||
state, distinct from a broken one.
|
||||
"""
|
||||
raw = secrets.resolve(KEK_NAME)
|
||||
if not raw:
|
||||
return None
|
||||
raw = raw.strip()
|
||||
# Try hex first (unambiguous: hex keys are [0-9a-fA-F]{64}); then
|
||||
# urlsafe b64 with flexible padding.
|
||||
try:
|
||||
if len(raw) == _KEY_BYTES * 2 and all(c in "0123456789abcdefABCDEF" for c in raw):
|
||||
return bytes.fromhex(raw)
|
||||
except ValueError as exc: # pragma: no cover — fromhex on checked chars can't fail
|
||||
raise KeyStoreError("bad_kek", f"{KEK_NAME} is not valid hex") from exc
|
||||
padded = raw + "=" * (-len(raw) % 4)
|
||||
try:
|
||||
decoded = base64.urlsafe_b64decode(padded.encode("ascii"))
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise KeyStoreError(
|
||||
"bad_kek", f"{KEK_NAME} is not valid urlsafe-b64 or hex"
|
||||
) from exc
|
||||
if len(decoded) != _KEY_BYTES:
|
||||
raise KeyStoreError(
|
||||
"bad_kek", f"{KEK_NAME} must decode to exactly 32 bytes (AES-256), got {len(decoded)}"
|
||||
)
|
||||
return decoded
|
||||
|
||||
|
||||
def _encode(raw: bytes) -> str:
|
||||
return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
|
||||
|
||||
|
||||
def _decode(token: str) -> bytes:
|
||||
return base64.urlsafe_b64decode(token + "=" * (-len(token) % 4))
|
||||
|
||||
|
||||
class KeyStore:
|
||||
"""Per-user encrypted key store (``keys/<user_id>.json``).
|
||||
|
||||
The store is ENABLED iff a valid KEK resolves at construction.
|
||||
Every public method is async; file/crypto work runs in
|
||||
``asyncio.to_thread``.
|
||||
"""
|
||||
|
||||
def __init__(self, secrets: SecretsProvider, data_root: Path) -> None:
|
||||
self._secrets = secrets
|
||||
self._root = data_root / "keys"
|
||||
self._lock = threading.Lock()
|
||||
self._kek: bytes | None
|
||||
self._disabled_reason: str | None = None
|
||||
try:
|
||||
self._kek = load_kek_bytes(secrets)
|
||||
except KeyStoreError as exc:
|
||||
# A malformed KEK is a hard misconfiguration: treat the
|
||||
# store as disabled with the honest reason, never a crash
|
||||
# and never a silent enable with the wrong key.
|
||||
self._kek = None
|
||||
self._disabled_reason = f"{KEK_NAME} is misconfigured: {exc}"
|
||||
if self._kek is None and self._disabled_reason is None:
|
||||
self._disabled_reason = (
|
||||
f"no {KEK_NAME} configured (env-file secrets 'keys' scope); "
|
||||
"per-user model keys cannot be stored — the credential chain "
|
||||
"falls to the server tier or keyless localhost"
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ state
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
return self._kek is not None
|
||||
|
||||
@property
|
||||
def disabled_reason(self) -> str | None:
|
||||
return self._disabled_reason
|
||||
|
||||
def _require_kek(self) -> bytes:
|
||||
assert self._kek is not None # internal invariant; callers gate on enabled
|
||||
return self._kek
|
||||
|
||||
def _path(self, user_id: str) -> Path:
|
||||
return self._root / f"{user_id}.json"
|
||||
|
||||
# ------------------------------------------------------- sync workers
|
||||
|
||||
def _read_envelope(self, user_id: str) -> _KeyEnvelope | None:
|
||||
path = self._path(user_id)
|
||||
try:
|
||||
raw = path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
try:
|
||||
return _KeyEnvelope.model_validate_json(raw)
|
||||
except ValidationError as exc:
|
||||
raise KeyStoreError(
|
||||
"unreadable",
|
||||
f"key envelope for user {user_id} is corrupt ({exc.error_count()} errors)",
|
||||
) from exc
|
||||
|
||||
def _set_key_locked(self, user_id: str, raw_key: str) -> None:
|
||||
kek = self._require_kek()
|
||||
nonce = os.urandom(_NONCE_BYTES) # fresh per write — NEVER reuse under the KEK
|
||||
ct = AESGCM(kek).encrypt(nonce, raw_key.encode("utf-8"), user_id.encode("utf-8"))
|
||||
envelope = _KeyEnvelope(
|
||||
v=1,
|
||||
alg="AES-256-GCM",
|
||||
updated_at=_now(),
|
||||
entries={KEY_SCOPE: _KeyEntry(nonce=_encode(nonce), ct=_encode(ct), updated_at=_now())},
|
||||
)
|
||||
path = self._path(user_id)
|
||||
self._root.mkdir(parents=True, exist_ok=True)
|
||||
tmp_path = path.with_name(f"{path.name}.{secrets.token_hex(4)}.tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as handle:
|
||||
handle.write(envelope.model_dump_json(indent=2))
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.chmod(tmp_path, 0o600)
|
||||
os.replace(tmp_path, path) # atomic — a crash never leaves a torn file
|
||||
|
||||
def _get_key_locked(self, user_id: str) -> str | None:
|
||||
envelope = self._read_envelope(user_id)
|
||||
if envelope is None:
|
||||
return None
|
||||
entry = envelope.entries.get(KEY_SCOPE)
|
||||
if entry is None:
|
||||
return None
|
||||
kek = self._require_kek()
|
||||
try:
|
||||
plain = AESGCM(kek).decrypt(
|
||||
_decode(entry.nonce), _decode(entry.ct), user_id.encode("utf-8")
|
||||
)
|
||||
except InvalidTag as exc:
|
||||
# AAD binding (user_id) failed, the KEK changed, or the
|
||||
# ciphertext bit-rotted. This is a typed error, NEVER a
|
||||
# silent None — a silent drop would let the tier fall to
|
||||
# server/keyless without the user knowing their key is
|
||||
# unreadable (C3-06).
|
||||
raise KeyStoreError(
|
||||
"tampered",
|
||||
f"key envelope for user {user_id} failed authentication "
|
||||
"(wrong user binding, changed KEK, or tampering)",
|
||||
) from exc
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise KeyStoreError(
|
||||
"unreadable", f"key envelope for user {user_id} has invalid encoding"
|
||||
) from exc
|
||||
return plain.decode("utf-8")
|
||||
|
||||
# ------------------------------------------------------------ API
|
||||
|
||||
async def set_key(self, user_id: str, scope: str, raw_key: str) -> None:
|
||||
"""Encrypt and persist *raw_key* for *user_id* under *scope*.
|
||||
|
||||
Rejected with a typed error when the store is disabled — an
|
||||
honest refusal, never a silent no-op.
|
||||
"""
|
||||
if scope != KEY_SCOPE:
|
||||
raise KeyStoreError("disabled", f"unsupported key scope: {scope} (v0.3: {KEY_SCOPE} only)")
|
||||
if not self.enabled:
|
||||
raise KeyStoreError("disabled", self._disabled_reason or "key store disabled")
|
||||
|
||||
def _work() -> None:
|
||||
with self._lock:
|
||||
self._set_key_locked(user_id, raw_key)
|
||||
|
||||
await asyncio.to_thread(_work)
|
||||
|
||||
async def get_key(self, user_id: str) -> str | None:
|
||||
"""Decrypt and return the user's key, or None when none stored.
|
||||
|
||||
Raises ``KeyStoreError`` on tamper/corruption (never a silent
|
||||
None), and when the store is disabled with a stored envelope
|
||||
that cannot be read.
|
||||
"""
|
||||
if not self.enabled:
|
||||
# Without a KEK nothing can be read — but an envelope that
|
||||
# EXISTS is surfaced as an error state (the user has a key
|
||||
# we cannot serve), not silently None.
|
||||
def _peek() -> bool:
|
||||
return self._path(user_id).is_file()
|
||||
|
||||
if await asyncio.to_thread(_peek):
|
||||
raise KeyStoreError(
|
||||
"disabled",
|
||||
f"a key is stored for user {user_id} but the key store is disabled ({self._disabled_reason})",
|
||||
)
|
||||
return None
|
||||
|
||||
def _work() -> str | None:
|
||||
with self._lock:
|
||||
return self._get_key_locked(user_id)
|
||||
|
||||
return await asyncio.to_thread(_work)
|
||||
|
||||
async def masked_status(self, user_id: str) -> dict[str, object]:
|
||||
"""Masked-only status: ``{configured, masked}`` — NEVER the raw key.
|
||||
|
||||
When the store is disabled the status says so honestly
|
||||
(``disabled: True`` + reason). A tampered/unreadable envelope
|
||||
is surfaced as an error field rather than ``configured: False``
|
||||
— never a silent tier drop.
|
||||
"""
|
||||
status: dict[str, object] = {"configured": False, "masked": None}
|
||||
if not self.enabled:
|
||||
status["disabled"] = True
|
||||
status["reason"] = self._disabled_reason
|
||||
|
||||
def _peek() -> bool:
|
||||
return self._path(user_id).is_file()
|
||||
|
||||
if await asyncio.to_thread(_peek):
|
||||
# An unreadable stored key is an error state, not "not configured".
|
||||
raise KeyStoreError(
|
||||
"disabled",
|
||||
f"a key is stored for user {user_id} but the key store is disabled ({self._disabled_reason})",
|
||||
)
|
||||
return status
|
||||
|
||||
def _work() -> dict[str, object]:
|
||||
with self._lock:
|
||||
envelope = self._read_envelope(user_id)
|
||||
if envelope is None or envelope.entries.get(KEY_SCOPE) is None:
|
||||
return {"configured": False, "masked": None, "disabled": False}
|
||||
key = self._get_key_locked(user_id) # surfaces tamper as KeyStoreError
|
||||
return {
|
||||
"configured": True,
|
||||
"masked": f"…{key[-4:]}" if key and len(key) >= 4 else "…",
|
||||
"disabled": False,
|
||||
}
|
||||
|
||||
return await asyncio.to_thread(_work)
|
||||
|
||||
|
||||
__all__ = ["KEK_NAME", "KEY_SCOPE", "KeyStore", "KeyStoreError", "load_kek_bytes"]
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Auth API request/response contracts (T-1.3).
|
||||
|
||||
These are the wire models for the register/login/logout/me routes and
|
||||
the admin user surface. Password hashes never appear in any of them
|
||||
(REQ-69: no password-hash material in any API response).
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from researchx.auth.users import MIN_PASSWORD_LENGTH, User
|
||||
|
||||
# Lightweight email shape check — the auth surface needs a stable,
|
||||
# normalizable identifier, not a full RFC 5322 validation (and pulling
|
||||
# pydantic[email] would add an unplanned dependency).
|
||||
_EMAIL_RE = r"^[^\s@]+@[^\s@]+\.[^\s@]+$"
|
||||
|
||||
|
||||
class RegisterBody(BaseModel):
|
||||
email: str = Field(pattern=_EMAIL_RE, max_length=254)
|
||||
password: str = Field(min_length=MIN_PASSWORD_LENGTH)
|
||||
display_name: str = Field(default="", max_length=120)
|
||||
|
||||
|
||||
class LoginBody(BaseModel):
|
||||
email: str = Field(pattern=_EMAIL_RE, max_length=254)
|
||||
password: str = Field(min_length=1)
|
||||
|
||||
|
||||
class AdminCreateUserBody(BaseModel):
|
||||
email: str = Field(pattern=_EMAIL_RE, max_length=254)
|
||||
password: str = Field(min_length=MIN_PASSWORD_LENGTH)
|
||||
display_name: str = Field(default="", max_length=120)
|
||||
role: str = Field(default="user", pattern=r"^(admin|user)$")
|
||||
|
||||
|
||||
class UserPublic(BaseModel):
|
||||
"""A user as exposed on the wire — no password_hash, ever."""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
user_id: str
|
||||
email: str
|
||||
display_name: str
|
||||
role: str
|
||||
active: bool
|
||||
created_at: datetime
|
||||
|
||||
|
||||
def to_public(user: User) -> UserPublic:
|
||||
"""Project a stored ``User`` to its public wire shape."""
|
||||
return UserPublic(
|
||||
user_id=user.user_id,
|
||||
email=user.email,
|
||||
display_name=user.display_name,
|
||||
role=user.role,
|
||||
active=user.active,
|
||||
created_at=user.created_at,
|
||||
)
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Auth routes: register / login / logout / me (T-1.4, REQ-69/70).
|
||||
|
||||
Mounted on a PUBLIC router at /api/auth/* (no require_user dependency)
|
||||
— but login/register POSTs still enforce the CSRF header (C3-05) via
|
||||
``Depends(require_csrf_header)``.
|
||||
|
||||
Route semantics:
|
||||
- register: open only while the user store is empty; the first user
|
||||
becomes admin, everything else 403s. The only await between the
|
||||
emptiness check and the insert is the argon2 hash; re-checking under
|
||||
the store lock keeps it race-free (D-104, single worker by design).
|
||||
- login: uniform 401 "invalid email or password" for every failure
|
||||
mode (no user enumeration); a fresh token is issued every login
|
||||
(session-fixation defense by rotation); the httpOnly cookie is set
|
||||
with SameSite=Lax, Path=/, no Domain attribute, Secure when
|
||||
configured for TLS-fronted hosts.
|
||||
- logout: revokes the presented token immediately.
|
||||
- me: the current user (session only — no CSRF header on GETs).
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
|
||||
from researchx.auth.guard import SESSION_COOKIE, require_csrf_header
|
||||
from researchx.auth.models import LoginBody, RegisterBody, UserPublic, to_public
|
||||
from researchx.auth.sessions import SessionStore
|
||||
from researchx.auth.users import UserError, UserStore
|
||||
|
||||
_COOKIE_MAX_AGE = 14 * 24 * 3600
|
||||
|
||||
|
||||
def build_auth_router(
|
||||
users: UserStore,
|
||||
sessions: SessionStore,
|
||||
cookie_secure: bool,
|
||||
) -> APIRouter:
|
||||
"""Build the public auth router (mounted at /api/auth)."""
|
||||
router = APIRouter(prefix="/api/auth", tags=["auth"])
|
||||
|
||||
def _set_session_cookie(response: Response, token: str) -> None:
|
||||
response.set_cookie(
|
||||
key=SESSION_COOKIE,
|
||||
value=token,
|
||||
max_age=_COOKIE_MAX_AGE,
|
||||
httponly=True,
|
||||
samesite="lax",
|
||||
path="/",
|
||||
secure=cookie_secure,
|
||||
)
|
||||
|
||||
def _presented_token(request: Request) -> str | None:
|
||||
return request.cookies.get(SESSION_COOKIE)
|
||||
|
||||
@router.post(
|
||||
"/register",
|
||||
status_code=201,
|
||||
response_model=UserPublic,
|
||||
dependencies=[Depends(require_csrf_header)],
|
||||
)
|
||||
async def register(body: RegisterBody) -> UserPublic:
|
||||
# Fast-path 403 for the common case; the authoritative
|
||||
# emptiness re-check happens under the store lock inside
|
||||
# create_first_admin (D-104) so a concurrent duplicate
|
||||
# register cannot both succeed.
|
||||
if not users.empty():
|
||||
raise HTTPException(status_code=403, detail="registration closed")
|
||||
try:
|
||||
user = await users.create_first_admin(
|
||||
email=body.email,
|
||||
password=body.password,
|
||||
display_name=body.display_name or body.email.split("@")[0],
|
||||
)
|
||||
except UserError as exc:
|
||||
raise HTTPException(status_code=403, detail=str(exc)) from exc
|
||||
return to_public(user)
|
||||
|
||||
@router.post(
|
||||
"/login",
|
||||
response_model=UserPublic,
|
||||
dependencies=[Depends(require_csrf_header)],
|
||||
)
|
||||
async def login(body: LoginBody, response: Response) -> UserPublic:
|
||||
user = users.get_by_email(body.email)
|
||||
if user is None or not await users.verify_password(user, body.password):
|
||||
raise HTTPException(status_code=401, detail="invalid email or password")
|
||||
if not user.active:
|
||||
raise HTTPException(status_code=401, detail="invalid email or password")
|
||||
# Token rotation: a fresh token on every login kills session
|
||||
# fixation by construction (D-101).
|
||||
token = sessions.generate_token()
|
||||
sessions.create(token, user.user_id)
|
||||
_set_session_cookie(response, token)
|
||||
if await users.needs_rehash(user):
|
||||
await users.rehash_password(user, body.password)
|
||||
return to_public(user)
|
||||
|
||||
@router.post(
|
||||
"/logout",
|
||||
dependencies=[Depends(require_csrf_header)],
|
||||
)
|
||||
async def logout(request: Request, response: Response) -> dict[str, bool]:
|
||||
token = _presented_token(request)
|
||||
if token:
|
||||
sessions.revoke(token)
|
||||
response.delete_cookie(key=SESSION_COOKIE, path="/")
|
||||
return {"ok": True}
|
||||
|
||||
@router.get("/me", response_model=UserPublic)
|
||||
async def me(request: Request) -> UserPublic:
|
||||
token = _presented_token(request)
|
||||
session = sessions.get(token) if token else None
|
||||
if session is None:
|
||||
raise HTTPException(status_code=401, detail="authentication required")
|
||||
user = users.get(session.user_id)
|
||||
if user is None or not user.active:
|
||||
raise HTTPException(status_code=401, detail="authentication required")
|
||||
sessions.touch(session)
|
||||
return to_public(user)
|
||||
|
||||
return router
|
||||
@@ -0,0 +1,161 @@
|
||||
"""Session store — server-side, opaque tokens, durable JSON mirror (D-101).
|
||||
|
||||
Tokens are ``secrets.token_urlsafe(32)``; only their sha256 hash is
|
||||
stored (in memory and in ``<data_root>/sessions.json``). Expiry is
|
||||
14-day sliding: every authenticated request calls ``touch()`` which
|
||||
pushes ``expires_at`` forward, persisting at most once per 60s.
|
||||
Revocation is immediate (logout deletes the presented token;
|
||||
deactivation deletes every token of the user).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import threading
|
||||
import time
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, ValidationError
|
||||
|
||||
SESSION_TTL_SECONDS = 14 * 24 * 3600
|
||||
_PERSIST_THROTTLE_SECONDS = 60.0
|
||||
|
||||
|
||||
class Session(BaseModel):
|
||||
token_hash: str
|
||||
user_id: str
|
||||
created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
|
||||
last_seen: datetime = Field(default_factory=lambda: datetime.now(UTC))
|
||||
expires_at: datetime
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
|
||||
def _hash_token(token: str) -> str:
|
||||
return hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class SessionStore:
|
||||
"""In-memory map keyed by token_hash + durable sessions.json mirror."""
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self._path = path
|
||||
self._sessions: dict[str, Session] = {}
|
||||
self._lock = threading.Lock()
|
||||
self._last_persist = 0.0
|
||||
self._load()
|
||||
|
||||
# ------------------------------------------------------------ persistence
|
||||
|
||||
def _load(self) -> None:
|
||||
if not self._path.exists():
|
||||
return
|
||||
try:
|
||||
raw = json.loads(self._path.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError:
|
||||
return
|
||||
for record in raw.get("sessions", []):
|
||||
try:
|
||||
session = Session.model_validate(record)
|
||||
except (ValidationError, KeyError, TypeError):
|
||||
continue # defensive: skip corrupt records on load
|
||||
self._sessions[session.token_hash] = session
|
||||
|
||||
def _persist_locked(self) -> None:
|
||||
self._path.parent.mkdir(parents=True, exist_ok=True)
|
||||
payload = {
|
||||
"sessions": [s.model_dump(mode="json") for s in self._sessions.values()]
|
||||
}
|
||||
tmp_path = self._path.with_name(f"{self._path.name}.{secrets.token_hex(4)}.tmp")
|
||||
content = json.dumps(payload, indent=2)
|
||||
with open(tmp_path, "w", encoding="utf-8") as handle:
|
||||
handle.write(content)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.replace(tmp_path, self._path)
|
||||
self._last_persist = time.monotonic()
|
||||
|
||||
def _persist(self) -> None:
|
||||
with self._lock:
|
||||
self._persist_locked()
|
||||
|
||||
# ------------------------------------------------------------- queries
|
||||
|
||||
def get(self, token: str) -> Session | None:
|
||||
"""Resolve a presented token; expired or unknown tokens yield None."""
|
||||
token_hash = _hash_token(token)
|
||||
session = self._sessions.get(token_hash)
|
||||
if session is None:
|
||||
return None
|
||||
if session.expires_at <= datetime.now(UTC):
|
||||
with self._lock:
|
||||
expired = self._sessions.pop(token_hash, None)
|
||||
if expired is not None:
|
||||
self._persist_locked()
|
||||
return None
|
||||
return session
|
||||
|
||||
# ------------------------------------------------------------- mutation
|
||||
|
||||
def create(self, token: str, user_id: str) -> Session:
|
||||
"""Store a new session for *token* (only its hash is kept)."""
|
||||
now = datetime.now(UTC)
|
||||
session = Session(
|
||||
token_hash=_hash_token(token),
|
||||
user_id=user_id,
|
||||
created_at=now,
|
||||
last_seen=now,
|
||||
expires_at=now + timedelta(seconds=SESSION_TTL_SECONDS),
|
||||
)
|
||||
with self._lock:
|
||||
self._sessions[session.token_hash] = session
|
||||
self._persist_locked()
|
||||
return session
|
||||
|
||||
def touch(self, session: Session) -> None:
|
||||
"""Sliding expiry: push expires_at forward, throttled persist."""
|
||||
now = datetime.now(UTC)
|
||||
renewed = Session(
|
||||
token_hash=session.token_hash,
|
||||
user_id=session.user_id,
|
||||
created_at=session.created_at,
|
||||
last_seen=now,
|
||||
expires_at=now + timedelta(seconds=SESSION_TTL_SECONDS),
|
||||
)
|
||||
with self._lock:
|
||||
current = self._sessions.get(session.token_hash)
|
||||
if current is None or current != session:
|
||||
return # stale or raced touch — nothing to extend
|
||||
self._sessions[renewed.token_hash] = renewed
|
||||
if time.monotonic() - self._last_persist > _PERSIST_THROTTLE_SECONDS:
|
||||
self._persist_locked()
|
||||
|
||||
def revoke(self, token: str) -> None:
|
||||
"""Immediately delete the session for *token* (logout)."""
|
||||
token_hash = _hash_token(token)
|
||||
with self._lock:
|
||||
if self._sessions.pop(token_hash, None) is not None:
|
||||
self._persist_locked()
|
||||
|
||||
def revoke_all_for(self, user_id: str) -> None:
|
||||
"""Immediately delete every session of *user_id* (deactivation)."""
|
||||
with self._lock:
|
||||
doomed = [h for h, s in self._sessions.items() if s.user_id == user_id]
|
||||
for token_hash in doomed:
|
||||
del self._sessions[token_hash]
|
||||
if doomed:
|
||||
self._persist_locked()
|
||||
|
||||
def generate_token(self) -> str:
|
||||
return secrets.token_urlsafe(32)
|
||||
|
||||
# ----------------------------------------------------------- asyncio shim
|
||||
|
||||
async def arevoke(self, token: str) -> None:
|
||||
await asyncio.to_thread(self.revoke, token)
|
||||
|
||||
async def arevoke_all_for(self, user_id: str) -> None:
|
||||
await asyncio.to_thread(self.revoke_all_for, user_id)
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Per-user settings persistence (T-2.3, REQ-72 — D-111).
|
||||
|
||||
``settings/<user_id>.json`` under the data root, written atomically
|
||||
(tmp + ``os.replace``). Fresh users get defaults on read (GET never
|
||||
writes). The ``UserSettings`` model carries the per-user surface the
|
||||
Settings page binds to: planner/writer model names, methodology
|
||||
defaults (resolved dial overrides per methodology — free-form dict in
|
||||
v0.3, validated at use time), and the appearance slot.
|
||||
|
||||
The storage directory is server-managed and display-only (D-111) — it
|
||||
is NOT part of this payload; the config status route surfaces it.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import secrets
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field, ValidationError
|
||||
|
||||
from researchx.adapters.types import AdapterError
|
||||
|
||||
# Default model names — the same defaults the live smoke tier uses
|
||||
# (gpt-oss:20b on ollama.com). These are fallbacks only: the frontend
|
||||
# Settings page always shows the current value.
|
||||
DEFAULT_PLANNER_MODEL = "gpt-oss:20b"
|
||||
DEFAULT_WRITER_MODEL = "gpt-oss:20b"
|
||||
|
||||
|
||||
class UserSettings(BaseModel):
|
||||
planner_model: str = DEFAULT_PLANNER_MODEL
|
||||
writer_model: str = DEFAULT_WRITER_MODEL
|
||||
methodology_defaults: dict[str, Any] = Field(default_factory=dict)
|
||||
appearance: str = "system"
|
||||
|
||||
|
||||
class SettingsStoreError(AdapterError):
|
||||
"""Typed settings-store failure (corrupt on-disk payload)."""
|
||||
|
||||
|
||||
class SettingsStore:
|
||||
"""Per-user settings, one JSON file per user, atomic writes."""
|
||||
|
||||
def __init__(self, data_root: Path) -> None:
|
||||
self._root = data_root / "settings"
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def _path(self, user_id: str) -> Path:
|
||||
return self._root / f"{user_id}.json"
|
||||
|
||||
# ------------------------------------------------------------ sync
|
||||
|
||||
def _read_sync(self, user_id: str) -> UserSettings:
|
||||
path = self._path(user_id)
|
||||
try:
|
||||
raw = path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError:
|
||||
return UserSettings() # fresh user: defaults, no write on read
|
||||
try:
|
||||
return UserSettings.model_validate_json(raw)
|
||||
except ValidationError as exc:
|
||||
raise SettingsStoreError(
|
||||
f"settings payload for user {user_id} is corrupt ({exc.error_count()} errors)"
|
||||
) from exc
|
||||
|
||||
def _update_sync(self, user_id: str, partial: UserSettings) -> UserSettings:
|
||||
current = self._read_sync(user_id)
|
||||
merged = current.model_copy(update=partial.model_dump(exclude_unset=True))
|
||||
# Validate the merged payload BEFORE it touches disk: a null
|
||||
# field slipped past model_copy (it skips validation) would
|
||||
# poison the file — every later read would raise corrupt.
|
||||
merged = UserSettings.model_validate(merged.model_dump())
|
||||
path = self._path(user_id)
|
||||
self._root.mkdir(parents=True, exist_ok=True)
|
||||
tmp_path = path.with_name(f"{path.name}.{secrets.token_hex(4)}.tmp")
|
||||
with open(tmp_path, "w", encoding="utf-8") as handle:
|
||||
handle.write(merged.model_dump_json(indent=2))
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
tmp_path.replace(path) # atomic
|
||||
return merged
|
||||
|
||||
# ------------------------------------------------------------ async
|
||||
|
||||
async def get(self, user_id: str) -> UserSettings:
|
||||
return await asyncio.to_thread(self._read_sync, user_id)
|
||||
|
||||
async def update(self, user_id: str, partial: UserSettings) -> UserSettings:
|
||||
def _work() -> UserSettings:
|
||||
with self._lock:
|
||||
return self._update_sync(user_id, partial)
|
||||
|
||||
return await asyncio.to_thread(_work)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_PLANNER_MODEL",
|
||||
"DEFAULT_WRITER_MODEL",
|
||||
"SettingsStore",
|
||||
"SettingsStoreError",
|
||||
"UserSettings",
|
||||
]
|
||||
@@ -0,0 +1,237 @@
|
||||
"""User store — durable JSON-backed account registry (REQ-69, D-102/D-105).
|
||||
|
||||
Persisted to ``<data_root>/users.json`` with the atomic tmp+``os.replace``
|
||||
idiom (D-017). Password hashing is argon2id via argon2-cffi
|
||||
``PasswordHasher()`` defaults; every hash/verify runs through
|
||||
``asyncio.to_thread`` because argon2 verify at default parameters costs
|
||||
~50-100ms of CPU and must leave the event loop alone.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import threading
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from argon2 import PasswordHasher
|
||||
from argon2.exceptions import VerifyMismatchError
|
||||
from pydantic import BaseModel, ConfigDict, Field, ValidationError
|
||||
|
||||
from researchx.config.ids import generate_slug
|
||||
|
||||
MIN_PASSWORD_LENGTH = 12
|
||||
|
||||
|
||||
class UserError(ValueError):
|
||||
"""Raised for user-store policy violations (duplicate email, weak password)."""
|
||||
|
||||
|
||||
class User(BaseModel):
|
||||
user_id: str = Field(pattern=r"^[a-z0-9]{8}$")
|
||||
email: str
|
||||
password_hash: str
|
||||
display_name: str
|
||||
role: str = "user"
|
||||
active: bool = True
|
||||
created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
|
||||
class UserStore:
|
||||
"""In-memory map keyed by user_id + durable users.json mirror."""
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
self._path = path
|
||||
self._by_id: dict[str, User] = {}
|
||||
self._by_email: dict[str, User] = {}
|
||||
self._lock = threading.Lock()
|
||||
self._hasher = PasswordHasher()
|
||||
self._load()
|
||||
|
||||
# ------------------------------------------------------------ persistence
|
||||
|
||||
def _load(self) -> None:
|
||||
if not self._path.exists():
|
||||
return
|
||||
try:
|
||||
raw = json.loads(self._path.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError as exc:
|
||||
# FAIL CLOSED on a corrupt user store. Swallowing it would
|
||||
# silently report an empty store, reopening registration on
|
||||
# top of existing accounts — the next persist would then
|
||||
# overwrite them (auth-reset-by-corruption). The file is
|
||||
# renamed aside so an operator can recover it; startup is
|
||||
# refused rather than silently degrading.
|
||||
salvage = self._path.with_name(f"{self._path.name}.corrupt.{secrets.token_hex(4)}")
|
||||
os.replace(self._path, salvage)
|
||||
raise RuntimeError(
|
||||
f"corrupt user store {self._path}; moved aside to {salvage}"
|
||||
) from exc
|
||||
for record in raw.get("users", []):
|
||||
try:
|
||||
user = User.model_validate(record)
|
||||
except (ValidationError, KeyError, TypeError):
|
||||
continue # defensive: skip corrupt records on load
|
||||
self._by_id[user.user_id] = user
|
||||
self._by_email[user.email] = user
|
||||
|
||||
def _persist_locked(self) -> None:
|
||||
self._path.parent.mkdir(parents=True, exist_ok=True)
|
||||
payload = {"users": [u.model_dump(mode="json") for u in self._by_id.values()]}
|
||||
tmp_path = self._path.with_name(f"{self._path.name}.{secrets.token_hex(4)}.tmp")
|
||||
content = json.dumps(payload, indent=2)
|
||||
with open(tmp_path, "w", encoding="utf-8") as handle:
|
||||
handle.write(content)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.replace(tmp_path, self._path)
|
||||
|
||||
def _persist(self) -> None:
|
||||
with self._lock:
|
||||
self._persist_locked()
|
||||
|
||||
# ------------------------------------------------------------- hashing
|
||||
|
||||
async def hash_password(self, password: str) -> str:
|
||||
return await asyncio.to_thread(self._hasher.hash, password)
|
||||
|
||||
async def verify_password(self, user: User, password: str) -> bool:
|
||||
try:
|
||||
await asyncio.to_thread(self._hasher.verify, user.password_hash, password)
|
||||
except VerifyMismatchError:
|
||||
return False
|
||||
return True
|
||||
|
||||
async def needs_rehash(self, user: User) -> bool:
|
||||
return await asyncio.to_thread(self._hasher.check_needs_rehash, user.password_hash)
|
||||
|
||||
# ------------------------------------------------------------- queries
|
||||
|
||||
def empty(self) -> bool:
|
||||
return not self._by_id
|
||||
|
||||
def get(self, user_id: str) -> User | None:
|
||||
return self._by_id.get(user_id)
|
||||
|
||||
def get_by_email(self, email: str) -> User | None:
|
||||
return self._by_email.get(email.casefold())
|
||||
|
||||
def list(self) -> list[User]:
|
||||
return sorted(self._by_id.values(), key=lambda u: (u.created_at, u.user_id))
|
||||
|
||||
# ------------------------------------------------------------- mutation
|
||||
|
||||
async def create_user(
|
||||
self,
|
||||
*,
|
||||
email: str,
|
||||
password: str,
|
||||
display_name: str,
|
||||
role: str = "user",
|
||||
) -> User:
|
||||
normalized = email.casefold()
|
||||
if len(password) < MIN_PASSWORD_LENGTH:
|
||||
raise UserError(f"password must be at least {MIN_PASSWORD_LENGTH} characters")
|
||||
if self._by_email.get(normalized) is not None:
|
||||
raise UserError("email already registered")
|
||||
password_hash = await self.hash_password(password)
|
||||
user = User(
|
||||
user_id=generate_slug(),
|
||||
email=normalized,
|
||||
password_hash=password_hash,
|
||||
display_name=display_name,
|
||||
role=role,
|
||||
)
|
||||
with self._lock:
|
||||
if self._by_email.get(normalized) is not None:
|
||||
raise UserError("email already registered")
|
||||
self._by_id[user.user_id] = user
|
||||
self._by_email[user.email] = user
|
||||
self._persist_locked()
|
||||
return user
|
||||
|
||||
async def rehash_password(self, user: User, password: str) -> None:
|
||||
"""Rehash a user's password in place (login-time rehash on parameter drift)."""
|
||||
password_hash = await self.hash_password(password)
|
||||
with self._lock:
|
||||
updated = user.model_copy(update={"password_hash": password_hash})
|
||||
self._by_id[updated.user_id] = updated
|
||||
self._by_email[updated.email] = updated
|
||||
self._persist_locked()
|
||||
|
||||
async def update_profile(
|
||||
self,
|
||||
user_id: str,
|
||||
*,
|
||||
display_name: str | None = None,
|
||||
email: str | None = None,
|
||||
) -> User:
|
||||
"""Update display name and/or email (T-3.6 — GET/PUT /api/profile).
|
||||
|
||||
The email must remain unique across OTHER users (a taken
|
||||
address raises UserError); the email index is maintained
|
||||
atomically under the store lock. No deletion exists.
|
||||
"""
|
||||
with self._lock:
|
||||
user = self._by_id.get(user_id)
|
||||
if user is None:
|
||||
raise UserError("unknown user")
|
||||
new_email = user.email
|
||||
if email is not None:
|
||||
new_email = email.casefold()
|
||||
other = self._by_email.get(new_email)
|
||||
if other is not None and other.user_id != user_id:
|
||||
raise UserError("email already registered")
|
||||
new_display = display_name if display_name is not None else user.display_name
|
||||
updated = user.model_copy(update={"email": new_email, "display_name": new_display})
|
||||
if new_email != user.email:
|
||||
old = self._by_email.get(user.email)
|
||||
if old is not None and old.user_id == user_id:
|
||||
del self._by_email[user.email]
|
||||
self._by_id[user_id] = updated
|
||||
self._by_email[new_email] = updated
|
||||
self._persist_locked()
|
||||
return updated
|
||||
|
||||
async def create_first_admin(self, *, email: str, password: str, display_name: str) -> User:
|
||||
"""Atomically create the first user, who is admin by construction.
|
||||
|
||||
The emptiness check + insert both happen under the store lock;
|
||||
the only await (the argon2 hash) precedes the check. Combined
|
||||
with the single uvicorn worker this makes registration open
|
||||
exactly once (D-104).
|
||||
"""
|
||||
if len(password) < MIN_PASSWORD_LENGTH:
|
||||
raise UserError(f"password must be at least {MIN_PASSWORD_LENGTH} characters")
|
||||
normalized = email.casefold()
|
||||
password_hash = await self.hash_password(password) # the one allowed await
|
||||
with self._lock:
|
||||
if self._by_id:
|
||||
raise UserError("registration closed")
|
||||
if self._by_email.get(normalized) is not None:
|
||||
raise UserError("email already registered")
|
||||
user = User(
|
||||
user_id=generate_slug(),
|
||||
email=normalized,
|
||||
password_hash=password_hash,
|
||||
display_name=display_name,
|
||||
role="admin",
|
||||
)
|
||||
self._by_id[user.user_id] = user
|
||||
self._by_email[user.email] = user
|
||||
self._persist_locked()
|
||||
return user
|
||||
|
||||
def deactivate(self, user_id: str) -> User | None:
|
||||
with self._lock:
|
||||
user = self._by_id.get(user_id)
|
||||
if user is None:
|
||||
return None
|
||||
updated = user.model_copy(update={"active": False})
|
||||
self._by_id[user_id] = updated
|
||||
self._by_email[updated.email] = updated
|
||||
self._persist_locked()
|
||||
return updated
|
||||
@@ -0,0 +1,111 @@
|
||||
from types import MappingProxyType
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
SourceUniverse = Literal["web", "academic", "local", "web+academic"]
|
||||
SearchRouting = Literal["single", "adaptive", "academic-only"]
|
||||
QueryStrategy = Literal["fixed", "iterative", "decomposed"]
|
||||
QualityGates = Literal["none", "standard", "strict"]
|
||||
CitationRigor = Literal["per-claim", "per-section", "per-report"]
|
||||
SynthesisStyle = Literal["memo", "executive", "encyclopedic", "adversarial"]
|
||||
OutputStructure = Literal["brief", "report", "dossier"]
|
||||
MemoryScope = Literal["none", "project", "cross-project"]
|
||||
|
||||
|
||||
class DialDescriptor(BaseModel, frozen=True):
|
||||
"""UI display metadata for a single dial."""
|
||||
|
||||
name: str
|
||||
label: str
|
||||
description: str
|
||||
|
||||
|
||||
class Dials(BaseModel, frozen=True):
|
||||
"""The 12 methodology dials with safe defaults (REQ-01)."""
|
||||
|
||||
source_universe: SourceUniverse = "web"
|
||||
search_routing: SearchRouting = "single"
|
||||
query_strategy: QueryStrategy = "decomposed"
|
||||
breadth: int = Field(default=2, ge=1, le=5)
|
||||
depth: int = Field(default=1, ge=1, le=3)
|
||||
quality_gates: QualityGates = "standard"
|
||||
citation_rigor: CitationRigor = "per-claim"
|
||||
verification_passes: int = Field(default=1, ge=0, le=3)
|
||||
synthesis_style: SynthesisStyle = "memo"
|
||||
output_structure: OutputStructure = "brief"
|
||||
memory_scope: MemoryScope = "none"
|
||||
checkpoints: bool = False
|
||||
|
||||
|
||||
DIAL_REGISTRY: MappingProxyType[str, DialDescriptor] = MappingProxyType(
|
||||
{
|
||||
"source_universe": DialDescriptor(
|
||||
name="source_universe",
|
||||
label="Source universe",
|
||||
description="Which sources may be searched: web, academic, local, or both.",
|
||||
),
|
||||
"search_routing": DialDescriptor(
|
||||
name="search_routing",
|
||||
label="Search routing",
|
||||
description="How queries are routed: single endpoint, adaptive, or academic-only.",
|
||||
),
|
||||
"query_strategy": DialDescriptor(
|
||||
name="query_strategy",
|
||||
label="Query strategy",
|
||||
description="How the question becomes queries: fixed, iterative, or decomposed.",
|
||||
),
|
||||
"breadth": DialDescriptor(
|
||||
name="breadth",
|
||||
label="Breadth",
|
||||
description="Number of parallel sub-question searches (1-5).",
|
||||
),
|
||||
"depth": DialDescriptor(
|
||||
name="depth",
|
||||
label="Depth",
|
||||
description="Number of reflection and refinement loops (1-3).",
|
||||
),
|
||||
"quality_gates": DialDescriptor(
|
||||
name="quality_gates",
|
||||
label="Quality gates",
|
||||
description="How strictly sources are filtered before synthesis.",
|
||||
),
|
||||
"citation_rigor": DialDescriptor(
|
||||
name="citation_rigor",
|
||||
label="Citation rigor",
|
||||
description="Granularity of citations: per claim, per section, or per report.",
|
||||
),
|
||||
"verification_passes": DialDescriptor(
|
||||
name="verification_passes",
|
||||
label="Verification passes",
|
||||
description="Post-synthesis verification passes over claims (0-3).",
|
||||
),
|
||||
"synthesis_style": DialDescriptor(
|
||||
name="synthesis_style",
|
||||
label="Synthesis style",
|
||||
description="Voice of the synthesized report.",
|
||||
),
|
||||
"output_structure": DialDescriptor(
|
||||
name="output_structure",
|
||||
label="Output structure",
|
||||
description="Shape of the final artifact: brief, report, or dossier.",
|
||||
),
|
||||
"memory_scope": DialDescriptor(
|
||||
name="memory_scope",
|
||||
label="Memory scope",
|
||||
description="What prior context a run may draw on.",
|
||||
),
|
||||
"checkpoints": DialDescriptor(
|
||||
name="checkpoints",
|
||||
label="Checkpoints",
|
||||
description="Persist intermediate state so a run can pause and resume.",
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DIAL_REGISTRY",
|
||||
"DialDescriptor",
|
||||
"Dials",
|
||||
]
|
||||
@@ -0,0 +1,25 @@
|
||||
"""8-char slug ids — the single generator + validation (P-14 consolidation).
|
||||
|
||||
Run ids and user ids share one alphabet and one shape: 8 lowercase
|
||||
alphanumeric characters, validated by ``^[a-z0-9]{8}$`` (D-105 — the
|
||||
user_id doubles as a filesystem-safe partition key). Both
|
||||
``adapters.local_storage`` and ``config.run_config`` import from here so
|
||||
there is exactly one generator left in the tree.
|
||||
"""
|
||||
|
||||
import re
|
||||
import secrets
|
||||
|
||||
ALPHABET = "abcdefghjkmnpqrstuvwxyz23456789"
|
||||
|
||||
SLUG_RE = re.compile(r"^[a-z0-9]{8}$")
|
||||
|
||||
|
||||
def generate_slug() -> str:
|
||||
"""Generate a random 8-char slug from the shared alphabet."""
|
||||
return "".join(secrets.choice(ALPHABET) for _ in range(8))
|
||||
|
||||
|
||||
def is_valid_slug(value: str) -> bool:
|
||||
"""Return True when *value* is a well-formed 8-char slug."""
|
||||
return SLUG_RE.match(value) is not None
|
||||
@@ -0,0 +1,31 @@
|
||||
"""Data-root resolution (D-107).
|
||||
|
||||
All mutable, host-persisted state (users.json, sessions.json, runs/,
|
||||
keys/, settings/) lives under one directory:
|
||||
|
||||
- ``RESEARCHX_DATA_DIR`` when set (container: ``/var/lib/researchx``),
|
||||
- else ``backend/var/researchx`` relative to the repository root (dev).
|
||||
|
||||
Replaces the cwd-dependent ``Path.cwd() / ".researchx"`` storage default.
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
_ENV_VAR = "RESEARCHX_DATA_DIR"
|
||||
|
||||
# src/researchx/config/paths.py -> repo root is four parents up.
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[4]
|
||||
_DEV_DEFAULT = _REPO_ROOT / "backend" / "var" / "researchx"
|
||||
|
||||
|
||||
def data_root() -> Path:
|
||||
"""Return the mutable data root, creating it on demand.
|
||||
|
||||
The directory (and its parent chain) is created lazily so that
|
||||
importing this module never touches the filesystem.
|
||||
"""
|
||||
raw = os.environ.get(_ENV_VAR)
|
||||
root = Path(raw).expanduser() if raw else _DEV_DEFAULT
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
return root
|
||||
@@ -0,0 +1,53 @@
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from researchx.config.dials import Dials
|
||||
from researchx.config.ids import generate_slug
|
||||
from researchx.methodologies.registry import MethodologyRegistry
|
||||
|
||||
|
||||
class UnknownOverrideError(ValueError):
|
||||
"""Raised when an override key is not a dial name."""
|
||||
|
||||
|
||||
class RunConfig(BaseModel, frozen=True):
|
||||
"""Persisted per-run snapshot of resolved dials (REQ-04)."""
|
||||
|
||||
run_id: str = Field(pattern=r"^[a-z0-9]{8}$")
|
||||
methodology_slug: str
|
||||
question: str = Field(min_length=1)
|
||||
context: str = ""
|
||||
resolved_dials: Dials
|
||||
created_at: datetime
|
||||
overrides: dict[str, Any] = Field(default_factory=dict)
|
||||
# Owning user (REQ-71). Optional = serialization-compatible with
|
||||
# pre-v0.3 config files (D-106: no migration).
|
||||
owner_id: str | None = None
|
||||
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
|
||||
def resolve_dials(
|
||||
methodology_slug: str,
|
||||
overrides: Mapping[str, Any],
|
||||
registry: MethodologyRegistry,
|
||||
) -> Dials:
|
||||
"""Resolve a methodology's dial bundle with validated overrides applied."""
|
||||
bundle = registry.get(methodology_slug).dials
|
||||
for key in overrides:
|
||||
if key not in Dials.model_fields:
|
||||
raise UnknownOverrideError(f"unknown dial override: {key}")
|
||||
merged = bundle.model_dump()
|
||||
merged.update(overrides)
|
||||
return Dials.model_validate(merged)
|
||||
|
||||
|
||||
def new_run_id() -> str:
|
||||
"""Generate a run id (P-14: single shared generator — config/ids.py)."""
|
||||
return generate_slug()
|
||||
|
||||
|
||||
__all__ = ["RunConfig", "UnknownOverrideError", "new_run_id", "resolve_dials"]
|
||||
@@ -0,0 +1,401 @@
|
||||
import asyncio
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.model import ModelAdapter
|
||||
from researchx.adapters.search import SearchAdapter
|
||||
from researchx.adapters.types import AdapterError, ModelMessage
|
||||
from researchx.adapters.url_guard import url_allowed
|
||||
from researchx.config.dials import Dials
|
||||
from researchx.config.run_config import RunConfig, resolve_dials
|
||||
from researchx.engine.artifacts import build_evidence_artifact, build_report_artifact
|
||||
from researchx.engine.events import EventBus, RunEvent
|
||||
from researchx.engine.run_state import Plan, RunState
|
||||
from researchx.engine.states import RunStateName
|
||||
from researchx.evidence.store import EvidenceStore
|
||||
from researchx.methodologies import MethodologyRegistry
|
||||
from researchx.trust.ledger import build_ledger, export_checklist
|
||||
from researchx.trust.liveness import liveness_report
|
||||
|
||||
|
||||
class EngineError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class RunActor:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
run_config: RunConfig,
|
||||
model: ModelAdapter,
|
||||
search: SearchAdapter,
|
||||
storage: LocalStorageAdapter,
|
||||
registry: MethodologyRegistry,
|
||||
event_bus: EventBus,
|
||||
) -> None:
|
||||
self._config = run_config
|
||||
self._model = model
|
||||
self._search = search
|
||||
self._storage = storage
|
||||
self._registry = registry
|
||||
self._bus = event_bus
|
||||
self._store = EvidenceStore()
|
||||
self.state = RunState(
|
||||
run_id=run_config.run_id,
|
||||
question=run_config.question,
|
||||
context=run_config.context,
|
||||
methodology_slug=run_config.methodology_slug,
|
||||
)
|
||||
methodology = registry.get(run_config.methodology_slug)
|
||||
self._dials: Dials = run_config.resolved_dials
|
||||
self.state.depth_target = self._dials.depth
|
||||
self._methodology = methodology
|
||||
self._pause = asyncio.Event()
|
||||
self._pause.set()
|
||||
self._cancelled = False
|
||||
self._approved = asyncio.Event()
|
||||
|
||||
async def _publish(self, event_type: str, data: dict[str, str] | None = None) -> None:
|
||||
event = RunEvent(
|
||||
type=event_type,
|
||||
run_id=self._config.run_id,
|
||||
data=data or {},
|
||||
)
|
||||
await self._bus.publish(event)
|
||||
try:
|
||||
await self._storage.append_run_artifact(
|
||||
self._config.run_id,
|
||||
"trace.jsonl",
|
||||
event.model_dump_json() + "\n",
|
||||
)
|
||||
except AdapterError:
|
||||
pass
|
||||
|
||||
async def _persist(self) -> None:
|
||||
content = self._snapshot()
|
||||
await self._storage.write_run_artifact(self._config.run_id, "state.json", content)
|
||||
|
||||
def _snapshot(self) -> str:
|
||||
return self.state.model_dump_json()
|
||||
|
||||
async def _checkpoint(self, target: RunStateName) -> None:
|
||||
if self.state.state != target:
|
||||
self.state.transition(target)
|
||||
await self._persist()
|
||||
|
||||
async def _wait_if_paused(self) -> None:
|
||||
await self._pause.wait()
|
||||
|
||||
def pause(self) -> None:
|
||||
self._pause.clear()
|
||||
|
||||
def resume(self) -> None:
|
||||
self._pause.set()
|
||||
|
||||
def cancel(self) -> None:
|
||||
self._cancelled = True
|
||||
|
||||
async def approve(self, plan: Plan | None = None) -> None:
|
||||
if plan is not None:
|
||||
self.state.plan = plan
|
||||
self._approved.set()
|
||||
|
||||
async def run(self) -> RunState:
|
||||
await self._publish("run_started", {"question": self._config.question})
|
||||
try:
|
||||
await self._execute()
|
||||
except asyncio.CancelledError:
|
||||
await asyncio.shield(self._persist_cancelled())
|
||||
raise
|
||||
except AdapterError as exc:
|
||||
await self._fail(str(exc))
|
||||
raise EngineError(str(exc)) from exc
|
||||
except httpx.HTTPError as exc:
|
||||
await self._fail(f"network error ({type(exc).__name__})")
|
||||
raise EngineError(f"network error ({type(exc).__name__})") from exc
|
||||
except EngineError:
|
||||
await self._fail("engine error")
|
||||
raise
|
||||
except Exception as exc:
|
||||
await self._fail(f"unexpected engine failure ({type(exc).__name__})")
|
||||
raise EngineError(f"unexpected engine failure ({type(exc).__name__})") from exc
|
||||
return self.state
|
||||
|
||||
async def attach_probe(self, probe_data: dict[str, Any]) -> None:
|
||||
self.state.probe = probe_data
|
||||
await self._persist()
|
||||
await self._publish("probe_attached", {"endpoint": str(probe_data.get("endpoint", ""))})
|
||||
|
||||
async def _persist_cancelled(self) -> None:
|
||||
self.state.state = RunStateName.CANCELLED
|
||||
await self._persist()
|
||||
|
||||
async def _fail(self, message: str) -> None:
|
||||
await self._publish("run_error", {"message": message})
|
||||
self.state.state = RunStateName.FAILED
|
||||
await asyncio.shield(self._persist())
|
||||
|
||||
async def _execute(self) -> None:
|
||||
current = self.state.state
|
||||
if current == RunStateName.PLANNING:
|
||||
await self._planning_phase()
|
||||
elif current == RunStateName.AWAITING_APPROVAL:
|
||||
pass
|
||||
elif current in (
|
||||
RunStateName.SEARCHING,
|
||||
RunStateName.EXTRACTING,
|
||||
RunStateName.REFLECTING,
|
||||
RunStateName.SYNTHESIZING,
|
||||
):
|
||||
await self._research_phase()
|
||||
await self._verify_phase()
|
||||
return
|
||||
elif current == RunStateName.VERIFYING:
|
||||
await self._verify_phase()
|
||||
return
|
||||
else:
|
||||
raise EngineError(f"cannot resume from state {current}")
|
||||
await self._approval_phase()
|
||||
await self._research_phase()
|
||||
await self._verify_phase()
|
||||
|
||||
async def _planning_phase(self) -> None:
|
||||
await self._checkpoint(RunStateName.PLANNING)
|
||||
await self._wait_if_paused()
|
||||
if self._cancelled:
|
||||
await self._persist_cancelled()
|
||||
raise asyncio.CancelledError
|
||||
prompt = self._methodology.planner_prompt.format(
|
||||
question=self._config.question, context=self._config.context
|
||||
)
|
||||
result = await self._model.chat(
|
||||
[ModelMessage(role="user", content=prompt)],
|
||||
model=self._model.roles().planner,
|
||||
json_mode=True,
|
||||
)
|
||||
plan = self._parse_plan(result.content)
|
||||
self.state.plan = plan
|
||||
await self._publish("state_changed", {"state": "AWAITING_APPROVAL"})
|
||||
|
||||
def _parse_plan(self, raw: str) -> Plan:
|
||||
import json
|
||||
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise AdapterError("planner returned non-JSON plan") from exc
|
||||
if not isinstance(data, dict):
|
||||
raise AdapterError("planner plan is not a JSON object")
|
||||
sub_questions = data.get("sub_questions", [])
|
||||
source_universe = data.get("source_universe", [])
|
||||
outline = data.get("outline", [])
|
||||
if not (
|
||||
isinstance(sub_questions, list)
|
||||
and isinstance(source_universe, list)
|
||||
and isinstance(outline, list)
|
||||
):
|
||||
raise AdapterError("planner plan has malformed lists")
|
||||
return Plan(
|
||||
sub_questions=[str(q) for q in sub_questions if isinstance(q, str)],
|
||||
source_universe=[str(u) for u in source_universe if isinstance(u, str)],
|
||||
outline=[str(o) for o in outline if isinstance(o, str)],
|
||||
)
|
||||
|
||||
async def _approval_phase(self) -> None:
|
||||
await self._checkpoint(RunStateName.AWAITING_APPROVAL)
|
||||
while not self._approved.is_set():
|
||||
if self._cancelled:
|
||||
await self._persist_cancelled()
|
||||
raise asyncio.CancelledError
|
||||
try:
|
||||
await asyncio.wait_for(self._approved.wait(), timeout=0.1)
|
||||
except TimeoutError:
|
||||
continue
|
||||
await self._checkpoint(RunStateName.SEARCHING)
|
||||
|
||||
async def _research_phase(self) -> None:
|
||||
while True:
|
||||
await self._check_cancelled()
|
||||
await self._searching_step()
|
||||
await self._extracting_step()
|
||||
if self.state.depth_cursor + 1 >= self.state.depth_target:
|
||||
break
|
||||
await self._reflecting_step()
|
||||
self.state.depth_cursor += 1
|
||||
await self._persist()
|
||||
await self._synthesizing_step()
|
||||
|
||||
async def _check_cancelled(self) -> None:
|
||||
if self._cancelled:
|
||||
await self._persist_cancelled()
|
||||
raise asyncio.CancelledError
|
||||
|
||||
async def _searching_step(self) -> None:
|
||||
if not self.state.plan or not self.state.plan.sub_questions:
|
||||
await self._checkpoint(RunStateName.SEARCHING)
|
||||
return
|
||||
await self._checkpoint(RunStateName.SEARCHING)
|
||||
await self._wait_if_paused()
|
||||
for sub_question in self.state.plan.sub_questions[: self._dials.breadth]:
|
||||
await self._publish("query_fired", {"query": sub_question})
|
||||
self.state.bump("queries")
|
||||
results = await self._search.search(sub_question, max_results=3)
|
||||
for result in results:
|
||||
await self._publish(
|
||||
"source_found",
|
||||
{"url": result.url, "title": result.title},
|
||||
)
|
||||
|
||||
async def _extracting_step(self) -> None:
|
||||
await self._checkpoint(RunStateName.EXTRACTING)
|
||||
await self._wait_if_paused()
|
||||
if not self.state.plan:
|
||||
return
|
||||
for sub_question in self.state.plan.sub_questions[: self._dials.breadth]:
|
||||
results = await self._search.search(sub_question, max_results=3)
|
||||
for result in results:
|
||||
if not url_allowed(result.url):
|
||||
await self._publish("source_refused", {"url": result.url})
|
||||
continue
|
||||
try:
|
||||
page = await self._search.fetch(result.url)
|
||||
except (AdapterError, httpx.HTTPError):
|
||||
continue
|
||||
self._store.add_page(page.url, page.title, page.text)
|
||||
await self._publish(
|
||||
"source_visited",
|
||||
{"url": page.url, "title": page.title},
|
||||
)
|
||||
self.state.bump("pages")
|
||||
for passage in self._store.passages_for_url(page.url):
|
||||
await self._publish(
|
||||
"passage_extracted",
|
||||
{"passage_id": passage.passage_id, "url": passage.url},
|
||||
)
|
||||
self.state.bump("passages")
|
||||
|
||||
async def _reflecting_step(self) -> None:
|
||||
await self._checkpoint(RunStateName.REFLECTING)
|
||||
await self._wait_if_paused()
|
||||
prompt = self._methodology.planner_prompt.format(
|
||||
question=self._config.question, context=self._config.context
|
||||
)
|
||||
result = await self._model.chat(
|
||||
[ModelMessage(role="user", content=prompt)],
|
||||
model=self._model.roles().planner,
|
||||
json_mode=True,
|
||||
)
|
||||
if self.state.plan:
|
||||
refined = self._parse_plan(result.content)
|
||||
self.state.plan = refined
|
||||
|
||||
async def _synthesizing_step(self) -> None:
|
||||
await self._checkpoint(RunStateName.SYNTHESIZING)
|
||||
await self._wait_if_paused()
|
||||
await self._check_cancelled()
|
||||
context_block = "\n\n".join(
|
||||
f"[{p.passage_id}] ({p.url}) {p.text}" for p in self._store.writer_context()
|
||||
)
|
||||
prompt = self._methodology.render_writer_prompt(
|
||||
question=self._config.question,
|
||||
context=self._config.context or "(none)",
|
||||
plan=self.state.plan.model_dump_json() if self.state.plan else "{}",
|
||||
evidence=context_block or "(no evidence collected)",
|
||||
)
|
||||
result = await self._model.chat(
|
||||
[ModelMessage(role="user", content=prompt)],
|
||||
model=self._model.roles().writer,
|
||||
)
|
||||
report = result.content
|
||||
await self._storage.write_run_artifact(self._config.run_id, "report.md", report)
|
||||
self.state.bump("writer_calls")
|
||||
|
||||
async def _verify_phase(self) -> None:
|
||||
await self._checkpoint(RunStateName.VERIFYING)
|
||||
report = await self._storage.read_run_artifact(self._config.run_id, "report.md")
|
||||
liveness = await liveness_report(self._cited_urls(report))
|
||||
ledger = build_ledger(
|
||||
run_id=self._config.run_id,
|
||||
report_text=report,
|
||||
evidence=self._store,
|
||||
liveness=liveness,
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
await self._storage.write_run_artifact(
|
||||
self._config.run_id, "ledger.json", ledger.model_dump_json(indent=2)
|
||||
)
|
||||
checklist = export_checklist(ledger)
|
||||
await self._storage.write_run_artifact(self._config.run_id, "checklist.md", checklist)
|
||||
# Structured artifacts (T-3.1, D-113): report.json +
|
||||
# evidence.json, written HERE — the only point where the
|
||||
# per-actor EvidenceStore still exists (it dies with the actor;
|
||||
# the disk read-models of D-114/D-115 can never rebuild it).
|
||||
# Marker normalization [p-N] -> [^pN] happens at this
|
||||
# persistence boundary ONLY (C3-10): report.md keeps the raw
|
||||
# form the trust-loop regexes depend on.
|
||||
evidence_artifact = build_evidence_artifact(
|
||||
run_id=self._config.run_id, evidence=self._store
|
||||
)
|
||||
report_artifact = build_report_artifact(
|
||||
run_id=self._config.run_id,
|
||||
owner_id=self.state.owner_id,
|
||||
question=self._config.question,
|
||||
methodology_name=self._methodology.name,
|
||||
methodology_slug=self._config.methodology_slug,
|
||||
report_markdown=report,
|
||||
ledger=ledger,
|
||||
evidence=self._store,
|
||||
generated_at=ledger.generated_at,
|
||||
)
|
||||
await self._storage.write_run_artifact(
|
||||
self._config.run_id, "evidence.json", evidence_artifact.model_dump_json(indent=2)
|
||||
)
|
||||
await self._storage.write_run_artifact(
|
||||
self._config.run_id, "report.json", report_artifact.model_dump_json(indent=2)
|
||||
)
|
||||
await self._checkpoint(RunStateName.DONE)
|
||||
await self._publish("run_done", {"run_id": self._config.run_id})
|
||||
|
||||
def _cited_urls(self, report_text: str) -> list[str]:
|
||||
import re
|
||||
|
||||
pattern = re.compile(r"\[p-(\d+)\]")
|
||||
cited: list[str] = []
|
||||
for match in pattern.finditer(report_text):
|
||||
passage = self._store.get(f"p-{match.group(1)}")
|
||||
if passage and passage.url not in cited:
|
||||
cited.append(passage.url)
|
||||
return cited
|
||||
|
||||
|
||||
def make_run_config(
|
||||
*,
|
||||
question: str,
|
||||
context: str,
|
||||
methodology_slug: str,
|
||||
overrides: dict[str, object],
|
||||
registry: MethodologyRegistry,
|
||||
run_id: str | None = None,
|
||||
owner_id: str | None = None,
|
||||
) -> tuple[str, RunConfig]:
|
||||
"""Build a run config; ``owner_id`` (when given) travels with the
|
||||
config and is stamped on the run state by the wiring layer (T-1.7).
|
||||
"""
|
||||
from researchx.adapters.local_storage import generate_run_id
|
||||
|
||||
rid = run_id or generate_run_id()
|
||||
dials = resolve_dials(methodology_slug, overrides, registry)
|
||||
config = RunConfig(
|
||||
run_id=rid,
|
||||
methodology_slug=methodology_slug,
|
||||
question=question,
|
||||
context=context,
|
||||
resolved_dials=dials,
|
||||
overrides=dict(overrides),
|
||||
created_at=datetime.now(UTC),
|
||||
owner_id=owner_id,
|
||||
)
|
||||
return rid, config
|
||||
@@ -0,0 +1,227 @@
|
||||
"""Structured run artifacts (T-3.1, D-113): report.json + evidence.json.
|
||||
|
||||
Built at DONE inside ``RunActor._verify_phase`` — the ONLY point where
|
||||
the per-actor ``EvidenceStore`` is still alive (it is GC'd with the
|
||||
actor; D-115 disk read-models come later and can never see it).
|
||||
|
||||
**Marker normalization happens HERE ONLY (C3-10).** The writer emits
|
||||
``[p-N]`` — the trust-loop regexes (``anchoring.py``,
|
||||
``ledger.py``) and the actor's own ``_cited_urls`` depend on that raw
|
||||
form, so ``report.md`` is never touched. The Reading Room parses
|
||||
``[^pN]`` (``report-body.ts`` matches ``[A-Za-z0-9]+`` only —
|
||||
``[^p-1]`` does NOT match), therefore report.json section bodies and
|
||||
the id fields of both artifacts carry the normalized ``pN`` id form.
|
||||
|
||||
Both artifacts are versioned (``v: 1``) so later schema changes stay
|
||||
detectable per file (D-106 old artifacts stay disposable/readable).
|
||||
"""
|
||||
|
||||
import re
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from researchx.evidence.store import EvidenceStore
|
||||
from researchx.trust.ledger import CitationLedger
|
||||
|
||||
#: The writer's raw marker form — the ONLY pattern normalized here.
|
||||
_MARKER_RE = re.compile(r"\[p-(\d+)\]")
|
||||
|
||||
#: ATX headings (H1..H6) — section delimiters for report.json.
|
||||
_HEADING_RE = re.compile(r"^#{1,6}[ \t]+(.*)$", re.MULTILINE)
|
||||
|
||||
#: A lone H1 — the report title source.
|
||||
_H1_RE = re.compile(r"^#[ \t]+(.*)$", re.MULTILINE)
|
||||
|
||||
#: Quote display cap — matches EvidenceStore's ``quote`` convention.
|
||||
_QUOTE_MAX_CHARS = 200
|
||||
|
||||
|
||||
def normalize_passage_id(passage_id: str) -> str:
|
||||
"""``p-N`` -> ``pN`` (the id form report.json/evidence.json carry)."""
|
||||
match = re.fullmatch(r"p-(\d+)", passage_id)
|
||||
return f"p{match.group(1)}" if match else passage_id
|
||||
|
||||
|
||||
def normalize_markers(text: str) -> str:
|
||||
"""``[p-N]`` -> ``[^pN]`` — report.json section bodies ONLY."""
|
||||
return _MARKER_RE.sub(lambda match: f"[^p{match.group(1)}]", text)
|
||||
|
||||
|
||||
def extract_title(markdown: str, fallback: str) -> str:
|
||||
"""First H1 of the report, else the methodology/question fallback."""
|
||||
match = _H1_RE.search(markdown)
|
||||
if match is not None:
|
||||
return match.group(1).strip()
|
||||
return fallback
|
||||
|
||||
|
||||
def split_sections(markdown: str) -> list[tuple[str, str]]:
|
||||
"""Split report.md into (heading, body) chunks on ATX headings.
|
||||
|
||||
Every ``^#{1,6}\\s`` heading starts a section and is included as
|
||||
that section's heading (the H1 chunk included — the title is a
|
||||
separate field and the frontend binding decides presentation).
|
||||
Content before the first heading becomes a ``("", body)`` section
|
||||
rather than being dropped — no content loss, no fabrication.
|
||||
"""
|
||||
sections: list[tuple[str, str]] = []
|
||||
matches = list(_HEADING_RE.finditer(markdown))
|
||||
if not matches:
|
||||
body = markdown.strip()
|
||||
if body:
|
||||
sections.append(("", body))
|
||||
return sections
|
||||
preamble = markdown[: matches[0].start()].strip()
|
||||
if preamble:
|
||||
sections.append(("", preamble))
|
||||
for index, match in enumerate(matches):
|
||||
heading = match.group(1).strip()
|
||||
start = match.end()
|
||||
end = matches[index + 1].start() if index + 1 < len(matches) else len(markdown)
|
||||
sections.append((heading, markdown[start:end].strip()))
|
||||
return sections
|
||||
|
||||
|
||||
class EvidencePassage(BaseModel):
|
||||
"""One stored passage — id in the normalized ``pN`` form."""
|
||||
|
||||
passage_id: str
|
||||
url: str
|
||||
title: str
|
||||
quote: str
|
||||
text: str
|
||||
|
||||
|
||||
class EvidenceArtifact(BaseModel):
|
||||
"""evidence.json — the serialized EvidenceStore at DONE."""
|
||||
|
||||
v: int = 1
|
||||
run_id: str
|
||||
passages: list[EvidencePassage] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ReportSection(BaseModel):
|
||||
heading: str
|
||||
body: str
|
||||
|
||||
|
||||
class ReportCitation(BaseModel):
|
||||
"""A page-anchored citation — passage-level, quote included."""
|
||||
|
||||
passage_id: str # pN form
|
||||
url: str
|
||||
title: str
|
||||
quote: str
|
||||
health: str
|
||||
authority_tier: str
|
||||
|
||||
|
||||
class ReportArtifact(BaseModel):
|
||||
"""report.json — the structured Reading-Room payload at DONE."""
|
||||
|
||||
v: int = 1
|
||||
run_id: str
|
||||
owner_id: str | None = None
|
||||
title: str
|
||||
question: str
|
||||
methodology_slug: str
|
||||
generated_at: datetime
|
||||
sections: list[ReportSection] = Field(default_factory=list)
|
||||
citations: list[ReportCitation] = Field(default_factory=list)
|
||||
|
||||
|
||||
def _page_titles(evidence: EvidenceStore) -> dict[str, str]:
|
||||
return {page.url: page.title for page in evidence.pages()}
|
||||
|
||||
|
||||
def _clip(quote: str) -> str:
|
||||
return quote[:_QUOTE_MAX_CHARS]
|
||||
|
||||
|
||||
def build_evidence_artifact(*, run_id: str, evidence: EvidenceStore) -> EvidenceArtifact:
|
||||
"""Serialize the full EvidenceStore — ids renumbered ``pN``.
|
||||
|
||||
Every passage the run collected, full text included: this is the
|
||||
only persisted copy (the in-actor store dies at DONE).
|
||||
"""
|
||||
titles = _page_titles(evidence)
|
||||
passages = [
|
||||
EvidencePassage(
|
||||
passage_id=normalize_passage_id(passage.passage_id),
|
||||
url=passage.url,
|
||||
title=titles.get(passage.url, ""),
|
||||
quote=_clip(passage.quote),
|
||||
text=passage.text,
|
||||
)
|
||||
for passage in evidence.writer_context()
|
||||
]
|
||||
return EvidenceArtifact(run_id=run_id, passages=passages)
|
||||
|
||||
|
||||
def build_report_artifact(
|
||||
*,
|
||||
run_id: str,
|
||||
owner_id: str | None,
|
||||
question: str,
|
||||
methodology_name: str,
|
||||
methodology_slug: str,
|
||||
report_markdown: str,
|
||||
ledger: CitationLedger,
|
||||
evidence: EvidenceStore,
|
||||
generated_at: datetime,
|
||||
) -> ReportArtifact:
|
||||
"""Build report.json from the raw report.md + ledger + evidence.
|
||||
|
||||
Section bodies carry normalized ``[^pN]`` markers (C3-10); the raw
|
||||
``report.md`` on disk is untouched. Citations are the ledger's cited
|
||||
URL entries resolved to their stored passages — real quotes from
|
||||
the evidence in hand, never fabricated.
|
||||
"""
|
||||
title = extract_title(report_markdown, fallback=f"{methodology_name} — {question}")
|
||||
sections = [
|
||||
ReportSection(heading=heading, body=normalize_markers(body))
|
||||
for heading, body in split_sections(report_markdown)
|
||||
]
|
||||
titles = _page_titles(evidence)
|
||||
citations: list[ReportCitation] = []
|
||||
for entry in ledger.entries:
|
||||
for passage_id in entry.passages:
|
||||
passage = evidence.get(passage_id)
|
||||
if passage is None:
|
||||
continue
|
||||
citations.append(
|
||||
ReportCitation(
|
||||
passage_id=normalize_passage_id(passage_id),
|
||||
url=entry.url,
|
||||
title=titles.get(entry.url, "") or entry.url,
|
||||
quote=_clip(passage.quote),
|
||||
health=entry.health,
|
||||
authority_tier=entry.authority_tier,
|
||||
)
|
||||
)
|
||||
return ReportArtifact(
|
||||
run_id=run_id,
|
||||
owner_id=owner_id,
|
||||
title=title,
|
||||
question=question,
|
||||
methodology_slug=methodology_slug,
|
||||
generated_at=generated_at,
|
||||
sections=sections,
|
||||
citations=citations,
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"EvidenceArtifact",
|
||||
"EvidencePassage",
|
||||
"ReportArtifact",
|
||||
"ReportCitation",
|
||||
"ReportSection",
|
||||
"build_evidence_artifact",
|
||||
"build_report_artifact",
|
||||
"extract_title",
|
||||
"normalize_markers",
|
||||
"normalize_passage_id",
|
||||
"split_sections",
|
||||
]
|
||||
@@ -0,0 +1,70 @@
|
||||
import asyncio
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.engine.actor import RunActor, make_run_config
|
||||
from researchx.engine.events import EventBus
|
||||
from researchx.methodologies import registry
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
base = Path(tmp)
|
||||
owner_id = "demo0001" # 8-char owner slug (P-9: demo passes the owner)
|
||||
run_id, config = make_run_config(
|
||||
question="How do verification pipelines keep citations honest?",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
owner_id=owner_id,
|
||||
)
|
||||
plan = (
|
||||
'{"sub_questions": ["verification passes", "quote anchoring"], '
|
||||
'"source_universe": ["web"], "outline": ["intro", "findings"]}'
|
||||
)
|
||||
writer = (
|
||||
"# Research Report\n\n"
|
||||
"The verification pass re-anchored every quoted claim [p-1] "
|
||||
"before the report shipped [p-2].\n"
|
||||
)
|
||||
model = FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=plan, model="fake-planner"),
|
||||
ChatResult(content=writer, model="fake-writer"),
|
||||
]
|
||||
)
|
||||
search = FakeSearchAdapter(CORPUS)
|
||||
storage = LocalStorageAdapter(base_dir=base, owner_id=owner_id)
|
||||
bus = EventBus()
|
||||
actor = RunActor(
|
||||
run_config=config, model=model, search=search, storage=storage, registry=registry,
|
||||
event_bus=bus,
|
||||
)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state.value == "PLANNING":
|
||||
await asyncio.sleep(0.01)
|
||||
await actor.approve()
|
||||
await task
|
||||
|
||||
state = (storage.run_dir(run_id) / "state.json").read_text()
|
||||
report = (storage.run_dir(run_id) / "report.md").read_text()
|
||||
ledger = (storage.run_dir(run_id) / "ledger.json").read_text()
|
||||
checklist = (storage.run_dir(run_id) / "checklist.md").read_text()
|
||||
print(f"run {run_id} → {actor.state.state.value}")
|
||||
print(f"artifacts under {storage.run_dir(run_id)}:")
|
||||
for name, content in [
|
||||
("state.json", state),
|
||||
("report.md", report),
|
||||
("ledger.json", ledger),
|
||||
("checklist.md", checklist),
|
||||
]:
|
||||
print(f"--- {name} ({len(content)} chars) ---")
|
||||
print(f"events: {len(bus.replay())}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,49 @@
|
||||
from collections.abc import Awaitable, Callable
|
||||
from datetime import UTC, datetime
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class RunEvent(BaseModel):
|
||||
type: str
|
||||
run_id: str
|
||||
at: datetime = Field(default_factory=lambda: datetime.now(UTC))
|
||||
data: dict[str, str] = {}
|
||||
seq: int = 0
|
||||
|
||||
|
||||
class EventBus:
|
||||
def __init__(self) -> None:
|
||||
self._subscribers: list[Callable[[RunEvent], Awaitable[None]]] = []
|
||||
self._log: list[RunEvent] = []
|
||||
self._seq = 0
|
||||
|
||||
def subscribe(self, handler: Callable[[RunEvent], Awaitable[None]]) -> None:
|
||||
self._subscribers.append(handler)
|
||||
|
||||
def unsubscribe(self, handler: Callable[[RunEvent], Awaitable[None]]) -> None:
|
||||
if handler in self._subscribers:
|
||||
self._subscribers.remove(handler)
|
||||
|
||||
async def publish(self, event: RunEvent) -> None:
|
||||
self._seq += 1
|
||||
event.seq = self._seq
|
||||
self._log.append(event)
|
||||
for handler in self._subscribers:
|
||||
await handler(event)
|
||||
|
||||
def replay(self, after_seq: int = 0) -> list[RunEvent]:
|
||||
return [e for e in self._log if e.seq > after_seq]
|
||||
|
||||
|
||||
class TraceWriter:
|
||||
def __init__(self, emit: Callable[[str], None]) -> None:
|
||||
self._emit = emit
|
||||
|
||||
async def record(self, event: RunEvent) -> None:
|
||||
self._emit(event.model_dump_json())
|
||||
|
||||
|
||||
def event_type_of(event: RunEvent) -> Literal["run_started", "other"]:
|
||||
return "run_started" if event.type == "run_started" else "other"
|
||||
@@ -0,0 +1,43 @@
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from researchx.engine.states import RunStateName, can_transition
|
||||
|
||||
|
||||
class Plan(BaseModel):
|
||||
sub_questions: list[str] = Field(default_factory=list)
|
||||
source_universe: list[str] = Field(default_factory=list)
|
||||
outline: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class RunState(BaseModel):
|
||||
run_id: str
|
||||
question: str
|
||||
context: str = ""
|
||||
methodology_slug: str
|
||||
state: RunStateName = RunStateName.PLANNING
|
||||
# Owner of the run (REQ-71). None on pre-v0.3 state files — old
|
||||
# artifacts must deserialize unchanged (D-106: no migration).
|
||||
owner_id: str | None = None
|
||||
# Which credential tier resolved for this run (REQ-74: user|
|
||||
# server|keyless). None on pre-v0.3.2 state files — old artifacts
|
||||
# deserialize unchanged (D-106: no migration).
|
||||
credential_tier: str | None = None
|
||||
plan: Plan | None = None
|
||||
depth_cursor: int = 0
|
||||
depth_target: int = 1
|
||||
counters: dict[str, int] = Field(default_factory=dict)
|
||||
probe: dict[str, Any] | None = None
|
||||
created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
|
||||
updated_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
|
||||
|
||||
def transition(self, target: RunStateName) -> None:
|
||||
if not can_transition(self.state, target):
|
||||
raise ValueError(f"illegal transition: {self.state} -> {target}")
|
||||
self.state = target
|
||||
self.updated_at = datetime.now(UTC)
|
||||
|
||||
def bump(self, key: str) -> None:
|
||||
self.counters[key] = self.counters.get(key, 0) + 1
|
||||
@@ -0,0 +1,72 @@
|
||||
from enum import StrEnum
|
||||
|
||||
|
||||
class RunStateName(StrEnum):
|
||||
PLANNING = "PLANNING"
|
||||
AWAITING_APPROVAL = "AWAITING_APPROVAL"
|
||||
SEARCHING = "SEARCHING"
|
||||
EXTRACTING = "EXTRACTING"
|
||||
REFLECTING = "REFLECTING"
|
||||
SYNTHESIZING = "SYNTHESIZING"
|
||||
VERIFYING = "VERIFYING"
|
||||
DONE = "DONE"
|
||||
CANCELLED = "CANCELLED"
|
||||
FAILED = "FAILED"
|
||||
|
||||
|
||||
_LEGAL_TRANSITIONS: dict[RunStateName, set[RunStateName]] = {
|
||||
# G-101: PLANNING -> CANCELLED is legal — the restart sweep rewrites
|
||||
# in-flight PLANNING runs as CANCELLED; semantically a run that has
|
||||
# not produced a plan can always be abandoned.
|
||||
RunStateName.PLANNING: {
|
||||
RunStateName.AWAITING_APPROVAL,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
},
|
||||
RunStateName.AWAITING_APPROVAL: {
|
||||
RunStateName.SEARCHING,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
},
|
||||
RunStateName.SEARCHING: {
|
||||
RunStateName.EXTRACTING,
|
||||
RunStateName.SYNTHESIZING,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
},
|
||||
RunStateName.EXTRACTING: {
|
||||
RunStateName.REFLECTING,
|
||||
RunStateName.SYNTHESIZING,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
},
|
||||
RunStateName.REFLECTING: {
|
||||
RunStateName.SEARCHING,
|
||||
RunStateName.SYNTHESIZING,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
},
|
||||
RunStateName.SYNTHESIZING: {
|
||||
RunStateName.VERIFYING,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
},
|
||||
RunStateName.VERIFYING: {
|
||||
RunStateName.DONE,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
},
|
||||
RunStateName.DONE: set(),
|
||||
RunStateName.CANCELLED: set(),
|
||||
RunStateName.FAILED: set(),
|
||||
}
|
||||
|
||||
TERMINAL_STATES = {
|
||||
RunStateName.DONE,
|
||||
RunStateName.CANCELLED,
|
||||
RunStateName.FAILED,
|
||||
}
|
||||
|
||||
|
||||
def can_transition(current: RunStateName, target: RunStateName) -> bool:
|
||||
return target in _LEGAL_TRANSITIONS[current]
|
||||
@@ -0,0 +1,70 @@
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class Passage(BaseModel):
|
||||
passage_id: str
|
||||
url: str
|
||||
quote: str
|
||||
text: str
|
||||
|
||||
|
||||
class VisitedPage(BaseModel):
|
||||
url: str
|
||||
title: str
|
||||
passage_ids: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
def extract_passages(text: str, *, min_chars: int = 120) -> list[str]:
|
||||
paragraphs = [p.strip() for p in text.split("\n") if p.strip()]
|
||||
return [p for p in paragraphs if len(p) >= min_chars]
|
||||
|
||||
|
||||
class EvidenceStore:
|
||||
def __init__(self) -> None:
|
||||
self._passages: dict[str, Passage] = {}
|
||||
self._pages: dict[str, VisitedPage] = {}
|
||||
self._seen_quotes: set[str] = set()
|
||||
self._counter = 0
|
||||
|
||||
def add_page(self, url: str, title: str, text: str) -> VisitedPage:
|
||||
page = self._pages.setdefault(url, VisitedPage(url=url, title=title))
|
||||
for paragraph in extract_passages(text):
|
||||
self.add_passage(url, paragraph)
|
||||
return page
|
||||
|
||||
def add_passage(self, url: str, text: str) -> Passage:
|
||||
self._counter += 1
|
||||
passage_id = f"p-{self._counter}"
|
||||
passage = Passage(passage_id=passage_id, url=url, quote=text[:200], text=text)
|
||||
self._passages[passage_id] = passage
|
||||
page = self._pages.get(url)
|
||||
if page is None:
|
||||
page = VisitedPage(url=url, title="")
|
||||
self._pages[url] = page
|
||||
page.passage_ids.append(passage_id)
|
||||
self._seen_quotes.add(text[:200])
|
||||
return passage
|
||||
|
||||
def has_seen(self, quote: str) -> bool:
|
||||
return quote[:200] in self._seen_quotes
|
||||
|
||||
def get(self, passage_id: str) -> Passage | None:
|
||||
return self._passages.get(passage_id)
|
||||
|
||||
def passages_for_url(self, url: str) -> list[Passage]:
|
||||
page = self._pages.get(url)
|
||||
if page is None:
|
||||
return []
|
||||
return [self._passages[pid] for pid in page.passage_ids if pid in self._passages]
|
||||
|
||||
def writer_context(self) -> list[Passage]:
|
||||
return [self._passages[pid] for pid in sorted(self._passages, key=self._passage_order)]
|
||||
|
||||
def _passage_order(self, passage_id: str) -> int:
|
||||
return int(passage_id.split("-")[1])
|
||||
|
||||
def pages(self) -> list[VisitedPage]:
|
||||
return list(self._pages.values())
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._passages)
|
||||
@@ -0,0 +1,9 @@
|
||||
from researchx.methodologies.builtin import registry
|
||||
from researchx.methodologies.models import Methodology
|
||||
from researchx.methodologies.registry import MethodologyRegistry
|
||||
|
||||
__all__ = [
|
||||
"Methodology",
|
||||
"MethodologyRegistry",
|
||||
"registry",
|
||||
]
|
||||
@@ -0,0 +1,97 @@
|
||||
from researchx.config.dials import Dials
|
||||
from researchx.methodologies.models import Methodology
|
||||
from researchx.methodologies.registry import MethodologyRegistry
|
||||
|
||||
_RAPID_PLANNER_PROMPT = (
|
||||
"You are a rapid research planner. Decompose the user's question into the fewest "
|
||||
"focused sub-questions that yield a confident answer.\n"
|
||||
"Question: {question}\n"
|
||||
"Context: {context}"
|
||||
)
|
||||
|
||||
_RAPID_WRITER_PROMPT = (
|
||||
"You are a rapid memo writer. Answer the question directly in a short memo.\n"
|
||||
"Cite a source per claim using the passage markers [p-N]. Every cited quote must "
|
||||
"come verbatim from the evidence passages listed below and from nowhere else.\n"
|
||||
"Question: {question}\n"
|
||||
"Context: {context}\n"
|
||||
"Plan: {plan}\n"
|
||||
"Evidence passages:\n{evidence}"
|
||||
)
|
||||
|
||||
_DEEP_PLANNER_PROMPT = (
|
||||
"You are a deep research planner. Decompose the user's question into two "
|
||||
"sub-questions, then propose an outline for an executive report with "
|
||||
"sections, each mapped to the sub-questions it answers.\n"
|
||||
"Question: {question}\n"
|
||||
"Context: {context}"
|
||||
)
|
||||
|
||||
_DEEP_WRITER_PROMPT = (
|
||||
"You are an executive report writer. Synthesize the evidence into a structured "
|
||||
"executive report following the plan: executive summary, findings, risks, and "
|
||||
"recommendations.\n"
|
||||
"Cite a source per claim using the passage markers [p-N]. Every cited quote must "
|
||||
"come verbatim from the evidence passages listed below and from nowhere else.\n"
|
||||
"Question: {question}\n"
|
||||
"Context: {context}\n"
|
||||
"Plan: {plan}\n"
|
||||
"Evidence passages:\n{evidence}"
|
||||
)
|
||||
|
||||
_BUILTIN: dict[str, Methodology] = {
|
||||
"rapid-answer": Methodology(
|
||||
slug="rapid-answer",
|
||||
name="Rapid Answer",
|
||||
description=(
|
||||
"One search pass, one synthesis pass: a short cited memo for questions "
|
||||
"that need a confident answer fast."
|
||||
),
|
||||
dials=Dials(
|
||||
source_universe="web",
|
||||
search_routing="single",
|
||||
query_strategy="decomposed",
|
||||
breadth=1,
|
||||
depth=1,
|
||||
quality_gates="standard",
|
||||
citation_rigor="per-claim",
|
||||
verification_passes=1,
|
||||
synthesis_style="memo",
|
||||
output_structure="brief",
|
||||
memory_scope="none",
|
||||
checkpoints=False,
|
||||
),
|
||||
planner_prompt=_RAPID_PLANNER_PROMPT,
|
||||
writer_prompt=_RAPID_WRITER_PROMPT,
|
||||
output_shape="memo",
|
||||
),
|
||||
"deep-report": Methodology(
|
||||
slug="deep-report",
|
||||
name="Deep Report",
|
||||
description=(
|
||||
"Full-depth research: iterative sub-questions, reflective refinement, and "
|
||||
"a verified executive report with per-claim citations."
|
||||
),
|
||||
dials=Dials(
|
||||
source_universe="web+academic",
|
||||
search_routing="adaptive",
|
||||
query_strategy="decomposed",
|
||||
breadth=2,
|
||||
depth=3,
|
||||
quality_gates="strict",
|
||||
citation_rigor="per-claim",
|
||||
verification_passes=2,
|
||||
synthesis_style="executive",
|
||||
output_structure="report",
|
||||
memory_scope="none",
|
||||
checkpoints=True,
|
||||
),
|
||||
planner_prompt=_DEEP_PLANNER_PROMPT,
|
||||
writer_prompt=_DEEP_WRITER_PROMPT,
|
||||
output_shape="executive-report",
|
||||
),
|
||||
}
|
||||
|
||||
registry = MethodologyRegistry(_BUILTIN)
|
||||
|
||||
__all__ = ["registry"]
|
||||
@@ -0,0 +1,32 @@
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from researchx.config.dials import Dials
|
||||
|
||||
OutputShape = Literal["memo", "executive-report"]
|
||||
|
||||
|
||||
class Methodology(BaseModel, frozen=True):
|
||||
"""A named research methodology: a dial bundle plus prompt templates (REQ-02)."""
|
||||
|
||||
slug: str = Field(pattern=r"^[a-z0-9]+(-[a-z0-9]+)*$")
|
||||
name: str
|
||||
description: str
|
||||
dials: Dials
|
||||
planner_prompt: str
|
||||
writer_prompt: str
|
||||
output_shape: OutputShape
|
||||
|
||||
def render_planner_prompt(self, question: str, context: str) -> str:
|
||||
return self.planner_prompt.format(question=question, context=context)
|
||||
|
||||
def render_writer_prompt(
|
||||
self, question: str, context: str, plan: str, evidence: str
|
||||
) -> str:
|
||||
return self.writer_prompt.format(
|
||||
question=question, context=context, plan=plan, evidence=evidence
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["Methodology", "OutputShape"]
|
||||
@@ -0,0 +1,26 @@
|
||||
from collections.abc import Mapping
|
||||
|
||||
from researchx.methodologies.models import Methodology
|
||||
|
||||
|
||||
class MethodologyRegistry:
|
||||
"""Frozen by-slug registry that hands out immutable copies (REQ-02/REQ-03)."""
|
||||
|
||||
def __init__(self, methodologies: Mapping[str, Methodology]) -> None:
|
||||
self._methodologies: dict[str, Methodology] = dict(methodologies)
|
||||
|
||||
def get(self, slug: str) -> Methodology:
|
||||
return self._methodologies[slug].model_copy(deep=True)
|
||||
|
||||
def list(self) -> list[Methodology]:
|
||||
ordered = sorted(self._methodologies.values(), key=lambda m: m.name)
|
||||
return [m.model_copy(deep=True) for m in ordered]
|
||||
|
||||
def __contains__(self, slug: str) -> bool:
|
||||
return slug in self._methodologies
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._methodologies)
|
||||
|
||||
|
||||
__all__ = ["MethodologyRegistry"]
|
||||
@@ -0,0 +1,89 @@
|
||||
import re
|
||||
import unicodedata
|
||||
from difflib import SequenceMatcher
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from researchx.evidence.store import EvidenceStore, Passage
|
||||
|
||||
ANCHOR_THRESHOLD = 0.85
|
||||
WEAK_MATCH_MIN = 0.80
|
||||
|
||||
|
||||
def normalize(text: str) -> str:
|
||||
nfc = unicodedata.normalize("NFC", text)
|
||||
collapsed = re.sub(r"\s+", " ", nfc)
|
||||
return collapsed.casefold().strip()
|
||||
|
||||
|
||||
class AnchorVerdict(BaseModel):
|
||||
quote: str
|
||||
passage_id: str | None
|
||||
ratio: float
|
||||
matched: bool
|
||||
weak: bool
|
||||
|
||||
|
||||
def anchor_quote(quote: str, passages: list[Passage]) -> AnchorVerdict:
|
||||
normalized_quote = normalize(quote)
|
||||
best_ratio = 0.0
|
||||
best_id: str | None = None
|
||||
for passage in passages:
|
||||
normalized_passage = normalize(passage.text)
|
||||
if normalized_quote and normalized_quote in normalized_passage:
|
||||
return AnchorVerdict(
|
||||
quote=quote, passage_id=passage.passage_id, ratio=1.0, matched=True, weak=False
|
||||
)
|
||||
ratio = SequenceMatcher(None, normalized_quote, normalized_passage).ratio()
|
||||
if ratio > best_ratio:
|
||||
best_ratio = ratio
|
||||
best_id = passage.passage_id
|
||||
matched = best_ratio >= ANCHOR_THRESHOLD
|
||||
weak = WEAK_MATCH_MIN <= best_ratio < ANCHOR_THRESHOLD
|
||||
return AnchorVerdict(
|
||||
quote=quote,
|
||||
passage_id=best_id if (matched or weak) else None,
|
||||
ratio=round(best_ratio, 4),
|
||||
matched=matched,
|
||||
weak=weak,
|
||||
)
|
||||
|
||||
|
||||
_MARKER_RE = re.compile(r"\[p-(\d+)\]")
|
||||
_HEADER_RE = re.compile(r"^#{1,6}\s.*$", re.MULTILINE)
|
||||
|
||||
|
||||
def _claim_fragment(before: str) -> str:
|
||||
cleaned = _HEADER_RE.sub("", before).rstrip()
|
||||
fragment_start = max(
|
||||
cleaned.rfind("."), cleaned.rfind("!"), cleaned.rfind("?"), cleaned.rfind("\n")
|
||||
)
|
||||
quote_text = cleaned[fragment_start + 1 :].strip(" \t>\"'*-")
|
||||
if not quote_text:
|
||||
quote_text = cleaned[-200:].strip(" \t>\"'*-")
|
||||
return quote_text
|
||||
|
||||
|
||||
def extract_report_claims(report_text: str) -> list[tuple[str, str]]:
|
||||
"""Extract (passage_id, quote_text) claims from the report.
|
||||
|
||||
The quote text is what the WRITER actually wrote adjacent to the
|
||||
citation marker — the last sentence fragment before each [p-N]
|
||||
marker — NOT the stored passage's own text. This is the adversarial
|
||||
core of REQ-20: a fabricated quote riding a real passage ID must
|
||||
be visible to the anchorer.
|
||||
"""
|
||||
claims: list[tuple[str, str]] = []
|
||||
for match in _MARKER_RE.finditer(report_text):
|
||||
passage_id = f"p-{match.group(1)}"
|
||||
quote_text = _claim_fragment(report_text[: match.start()])
|
||||
claims.append((passage_id, quote_text))
|
||||
return claims
|
||||
|
||||
|
||||
def extract_quotes_from_report(report_text: str, evidence: EvidenceStore) -> list[str]:
|
||||
return [quote for _pid, quote in extract_report_claims(report_text) if evidence.get(_pid)]
|
||||
|
||||
|
||||
def extract_cited_passage_ids(report_text: str) -> list[str]:
|
||||
return [f"p-{m.group(1)}" for m in _MARKER_RE.finditer(report_text)]
|
||||
@@ -0,0 +1,208 @@
|
||||
import re
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from researchx.evidence.store import EvidenceStore, Passage
|
||||
from researchx.trust.anchoring import (
|
||||
anchor_quote,
|
||||
extract_cited_passage_ids,
|
||||
extract_report_claims,
|
||||
normalize,
|
||||
)
|
||||
from researchx.trust.liveness import LivenessStatus
|
||||
|
||||
_AUTHORITY_TIERS: dict[str, str] = {
|
||||
"example.org": "reference",
|
||||
"gov": "official",
|
||||
"edu": "academic",
|
||||
"org": "reference",
|
||||
"com": "general",
|
||||
}
|
||||
|
||||
|
||||
def authority_tier(url: str) -> str:
|
||||
host = re.sub(r"^https?://", "", url).split("/")[0].lower().split(":")[0]
|
||||
labels = host.split(".")
|
||||
for suffix, tier in _AUTHORITY_TIERS.items():
|
||||
if labels[-1] == suffix or (len(labels) > 1 and labels[-2] == suffix):
|
||||
return tier
|
||||
return "general"
|
||||
|
||||
|
||||
class CitationEntry(BaseModel):
|
||||
url: str
|
||||
visited_at: datetime
|
||||
passages: list[str]
|
||||
claims_supported: int
|
||||
health: str
|
||||
authority_tier: str
|
||||
|
||||
|
||||
class VerificationChecklist(BaseModel):
|
||||
run_id: str
|
||||
citations_checked: int
|
||||
live: int
|
||||
stale: int
|
||||
dead: int
|
||||
anchored_claims: int
|
||||
unanchored_claims: int
|
||||
weak_matches: int
|
||||
confidence_labels: dict[str, str]
|
||||
liveness_ratio: float
|
||||
|
||||
|
||||
class CitationLedger(BaseModel):
|
||||
run_id: str
|
||||
generated_at: datetime
|
||||
entries: list[CitationEntry]
|
||||
checklist: VerificationChecklist
|
||||
|
||||
|
||||
_CLAIM_RE = re.compile(r"\[p-(\d+)\]")
|
||||
|
||||
|
||||
def _claims_per_passage(report_text: str) -> dict[str, int]:
|
||||
counts: dict[str, int] = {}
|
||||
for match in _CLAIM_RE.finditer(report_text):
|
||||
pid = f"p-{match.group(1)}"
|
||||
counts[pid] = counts.get(pid, 0) + 1
|
||||
return counts
|
||||
|
||||
|
||||
def build_ledger(
|
||||
*,
|
||||
run_id: str,
|
||||
report_text: str,
|
||||
evidence: EvidenceStore,
|
||||
liveness: dict[str, LivenessStatus],
|
||||
visited_at: datetime,
|
||||
) -> CitationLedger:
|
||||
claims = _claims_per_passage(report_text)
|
||||
all_passages = evidence.writer_context()
|
||||
report_claims = extract_report_claims(report_text)
|
||||
cited_ids = extract_cited_passage_ids(report_text)
|
||||
known_ids = {p.passage_id for p in all_passages}
|
||||
cited_urls: list[str] = []
|
||||
entries: list[CitationEntry] = []
|
||||
anchored = 0
|
||||
unanchored = 0
|
||||
weak = 0
|
||||
confidence: dict[str, str] = {}
|
||||
|
||||
for passage_id, writer_quote in report_claims:
|
||||
passage = evidence.get(passage_id)
|
||||
if passage is None:
|
||||
unanchored += 1
|
||||
continue
|
||||
verdict = anchor_quote(writer_quote, [passage])
|
||||
if verdict.matched:
|
||||
anchored += 1
|
||||
elif verdict.weak:
|
||||
weak += 1
|
||||
else:
|
||||
unanchored += 1
|
||||
|
||||
for cited_id in cited_ids:
|
||||
if cited_id not in known_ids:
|
||||
unanchored += 1
|
||||
passage = evidence.get(cited_id)
|
||||
if passage is not None and passage.url not in cited_urls:
|
||||
cited_urls.append(passage.url)
|
||||
|
||||
for page in evidence.pages():
|
||||
if page.url not in cited_urls:
|
||||
continue
|
||||
passages = evidence.passages_for_url(page.url)
|
||||
passage_ids = [p.passage_id for p in passages]
|
||||
page_claims = sum(claims.get(pid, 0) for pid in passage_ids)
|
||||
health = str(liveness.get(page.url, LivenessStatus.DEAD))
|
||||
entries.append(
|
||||
CitationEntry(
|
||||
url=page.url,
|
||||
visited_at=visited_at,
|
||||
passages=passage_ids,
|
||||
claims_supported=page_claims,
|
||||
health=health,
|
||||
authority_tier=authority_tier(page.url),
|
||||
)
|
||||
)
|
||||
for passage in passages:
|
||||
confidence[passage.passage_id] = (
|
||||
"corroborated" if _corroborated(passage, evidence) else "single-source"
|
||||
)
|
||||
|
||||
live = sum(1 for e in entries if e.health == "live")
|
||||
stale = sum(1 for e in entries if e.health == "stale")
|
||||
dead = sum(1 for e in entries if e.health == "dead")
|
||||
total = len(entries)
|
||||
ratio = round(live / total, 4) if total > 0 else 1.0
|
||||
checklist = VerificationChecklist(
|
||||
run_id=run_id,
|
||||
citations_checked=total,
|
||||
live=live,
|
||||
stale=stale,
|
||||
dead=dead,
|
||||
anchored_claims=anchored,
|
||||
unanchored_claims=unanchored,
|
||||
weak_matches=weak,
|
||||
confidence_labels=confidence,
|
||||
liveness_ratio=ratio,
|
||||
)
|
||||
return CitationLedger(
|
||||
run_id=run_id, generated_at=visited_at, entries=entries, checklist=checklist
|
||||
)
|
||||
|
||||
|
||||
def _corroborated(passage: Passage, evidence: EvidenceStore) -> bool:
|
||||
normalized = normalize(passage.text)
|
||||
for other in evidence.writer_context():
|
||||
if other.passage_id == passage.passage_id:
|
||||
continue
|
||||
if normalized in normalize(other.text):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
LIVENESS_EXPORT_GATE = 0.99
|
||||
|
||||
|
||||
def liveness_export_gate(ledger: CitationLedger) -> tuple[bool, list[str]]:
|
||||
failures = [e.url for e in ledger.entries if e.health != "live"]
|
||||
liveness_ok = ledger.checklist.liveness_ratio >= LIVENESS_EXPORT_GATE
|
||||
anchoring_ok = ledger.checklist.unanchored_claims == 0
|
||||
return liveness_ok and anchoring_ok, failures
|
||||
|
||||
|
||||
def export_checklist(ledger: CitationLedger) -> str:
|
||||
gate_passed, failures = liveness_export_gate(ledger)
|
||||
lines = [
|
||||
"# Verification Checklist",
|
||||
"",
|
||||
f"Run: {ledger.run_id}",
|
||||
f"Citations checked: {ledger.checklist.citations_checked}",
|
||||
f"Live: {ledger.checklist.live} / Stale: {ledger.checklist.stale} / Dead: {ledger.checklist.dead}",
|
||||
(
|
||||
f"Liveness ratio: {ledger.checklist.liveness_ratio} (export gate >= {LIVENESS_EXPORT_GATE}: "
|
||||
f"{'PASS' if gate_passed else 'FAIL'})"
|
||||
),
|
||||
(
|
||||
f"Anchoring gate: {'PASS' if ledger.checklist.unanchored_claims == 0 else 'FAIL'} "
|
||||
f"({ledger.checklist.unanchored_claims} unanchored)"
|
||||
),
|
||||
f"Anchored claims: {ledger.checklist.anchored_claims}",
|
||||
f"Unanchored claims: {ledger.checklist.unanchored_claims}",
|
||||
f"Weak matches (flagged, not accepted): {ledger.checklist.weak_matches}",
|
||||
"",
|
||||
"## Per-URL ledger",
|
||||
"",
|
||||
]
|
||||
for entry in ledger.entries:
|
||||
lines.append(
|
||||
f"- {entry.url} — health: {entry.health}, tier: {entry.authority_tier}, "
|
||||
f"passages: {len(entry.passages)}, claims: {entry.claims_supported}"
|
||||
)
|
||||
if failures:
|
||||
lines += ["", "## Liveness failures", ""]
|
||||
lines += [f"- {url}" for url in failures]
|
||||
return "\n".join(lines) + "\n"
|
||||
@@ -0,0 +1,65 @@
|
||||
from enum import StrEnum
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.url_guard import url_allowed
|
||||
|
||||
_LIVENESS_TIMEOUT = 10.0
|
||||
_MAX_RESPONSE_BYTES = 2 * 1024 * 1024
|
||||
_MAX_REDIRECTS = 5
|
||||
_USER_AGENT = "researchx/0.2 (liveness check)"
|
||||
|
||||
|
||||
class LivenessStatus(StrEnum):
|
||||
LIVE = "live"
|
||||
STALE = "stale"
|
||||
DEAD = "dead"
|
||||
|
||||
|
||||
async def check_liveness(url: str, client: httpx.AsyncClient | None = None) -> LivenessStatus:
|
||||
own_client = client is None
|
||||
active: httpx.AsyncClient = client or httpx.AsyncClient(
|
||||
timeout=_LIVENESS_TIMEOUT,
|
||||
follow_redirects=False,
|
||||
)
|
||||
current_url = url
|
||||
try:
|
||||
for _ in range(_MAX_REDIRECTS + 1):
|
||||
if not url_allowed(current_url):
|
||||
return LivenessStatus.DEAD
|
||||
response = await active.get(
|
||||
current_url,
|
||||
headers={"User-Agent": _USER_AGENT},
|
||||
)
|
||||
if 300 <= response.status_code < 400:
|
||||
location = response.headers.get("location")
|
||||
if not location:
|
||||
return LivenessStatus.STALE
|
||||
from urllib.parse import urljoin
|
||||
|
||||
current_url = urljoin(current_url, location)
|
||||
continue
|
||||
content = b""
|
||||
async for chunk in response.aiter_bytes():
|
||||
content += chunk
|
||||
if len(content) > _MAX_RESPONSE_BYTES:
|
||||
break
|
||||
status = response.status_code
|
||||
if 200 <= status < 300:
|
||||
return LivenessStatus.LIVE
|
||||
return LivenessStatus.DEAD
|
||||
return LivenessStatus.STALE
|
||||
except (httpx.HTTPError, OSError):
|
||||
return LivenessStatus.DEAD
|
||||
finally:
|
||||
if own_client:
|
||||
await active.aclose()
|
||||
|
||||
|
||||
async def liveness_report(
|
||||
urls: list[str],
|
||||
) -> dict[str, LivenessStatus]:
|
||||
report: dict[str, LivenessStatus] = {}
|
||||
for url in dict.fromkeys(urls):
|
||||
report[url] = await check_liveness(url)
|
||||
return report
|
||||
@@ -0,0 +1,22 @@
|
||||
"""Shared test fixtures."""
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_sse_starlette_loop_binding():
|
||||
"""Reset sse_starlette's module-global shutdown Event before each test.
|
||||
|
||||
``sse_starlette.AppStatus.should_exit_event`` is an ``anyio.Event``
|
||||
memoized at the first SSE response and bound to that event loop.
|
||||
pytest-asyncio gives every test a fresh loop, so a second SSE test in
|
||||
one process would die with "Event is bound to a different event
|
||||
loop". Resetting the global before each test keeps the library's
|
||||
loop-binding per-test. (Production is unaffected: single process,
|
||||
single loop, single uvicorn worker by design.)
|
||||
"""
|
||||
from sse_starlette.sse import AppStatus
|
||||
|
||||
AppStatus.should_exit = False
|
||||
AppStatus.should_exit_event = None
|
||||
yield
|
||||
@@ -0,0 +1,21 @@
|
||||
import secrets as secrets_module
|
||||
from string import ascii_lowercase
|
||||
|
||||
from researchx.adapters.local_storage import generate_run_id
|
||||
|
||||
|
||||
def test_generate_run_id_format() -> None:
|
||||
run_id = generate_run_id()
|
||||
assert isinstance(run_id, str)
|
||||
assert len(run_id) == 8
|
||||
assert all(c in ascii_lowercase + "23456789" for c in run_id)
|
||||
assert "o" not in run_id and "l" not in run_id and "0" not in run_id and "1" not in run_id
|
||||
|
||||
|
||||
def test_generate_run_id_entropy() -> None:
|
||||
ids = {generate_run_id() for _ in range(100)}
|
||||
assert len(ids) == 100
|
||||
|
||||
|
||||
def test_generate_run_id_uses_secrets_module() -> None:
|
||||
assert secrets_module.choice is not None
|
||||
@@ -0,0 +1,363 @@
|
||||
"""Adversarial suite (REQ-32): attacks on the trust loop, proven caught.
|
||||
|
||||
Every test here is an attempt to fool the verifier. If one passes
|
||||
without being flagged, the trust loop is broken — the suite fails.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.types import AdapterError, ChatResult
|
||||
from researchx.api.app import create_app
|
||||
from researchx.evidence.store import EvidenceStore
|
||||
from researchx.trust.anchoring import anchor_quote
|
||||
from researchx.trust.ledger import build_ledger, export_checklist, liveness_export_gate
|
||||
from researchx.trust.liveness import LivenessStatus, check_liveness
|
||||
|
||||
|
||||
def test_attack_hallucinated_url_fails_export_gate() -> None:
|
||||
store = EvidenceStore()
|
||||
store.add_passage("https://real.example.org/a", "A real stored passage about the topic " * 5)
|
||||
report = (
|
||||
"# Report\n\n"
|
||||
"Real claim [p-1].\n\n"
|
||||
"Supporting detail from a hallucinated source [p-2].\n"
|
||||
)
|
||||
ledger = build_ledger(
|
||||
run_id="attack001",
|
||||
report_text=report,
|
||||
evidence=store,
|
||||
liveness={"https://real.example.org/a": LivenessStatus.LIVE},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
claim_counts = _claim_counts(ledger)
|
||||
assert claim_counts.get("p-1", 0) >= 1
|
||||
assert "p-2" in report
|
||||
assert not any("p-2" in pids for pids in [e.passages for e in ledger.entries]), (
|
||||
"hallucinated passage id must not appear in any ledger entry"
|
||||
)
|
||||
unanchored = ledger.checklist.unanchored_claims
|
||||
assert unanchored >= 1, "hallucinated citation must be flagged as unanchored"
|
||||
|
||||
|
||||
def _claim_counts(ledger) -> dict[str, int]:
|
||||
counts: dict[str, int] = {}
|
||||
for entry in ledger.entries:
|
||||
for pid in entry.passages:
|
||||
counts[pid] = counts.get(pid, 0) + entry.claims_supported
|
||||
return counts
|
||||
|
||||
|
||||
def test_attack_fabricated_quote_flagged_not_accepted() -> None:
|
||||
store = EvidenceStore()
|
||||
for page in CORPUS:
|
||||
store.add_page(page.url, page.title, page.text)
|
||||
passages = store.writer_context()
|
||||
|
||||
fabricated = "This sentence appears in exactly zero stored passages anywhere."
|
||||
verdict = anchor_quote(fabricated, passages)
|
||||
assert verdict.matched is False
|
||||
assert verdict.weak is False
|
||||
|
||||
near_miss = passages[0].text[: len(passages[0].text) // 2] + " And a divergent clause."
|
||||
near_verdict = anchor_quote(near_miss, passages)
|
||||
assert near_verdict.matched is False or near_verdict.weak is True
|
||||
|
||||
exact = anchor_quote(passages[0].quote, passages)
|
||||
assert exact.matched is True
|
||||
|
||||
|
||||
def test_attack_quote_from_other_passage_counts_as_corroboration_not_anchor() -> None:
|
||||
from researchx.adapters.fakes import SHARED_SENTENCE
|
||||
|
||||
store = EvidenceStore()
|
||||
first = store.add_passage("https://a.example.org", SHARED_SENTENCE)
|
||||
second = store.add_passage("https://b.example.org", SHARED_SENTENCE + " Extra tail text.")
|
||||
verdict_first = anchor_quote(second.quote, [first])
|
||||
assert verdict_first.matched is True or verdict_first.ratio > 0.5
|
||||
|
||||
|
||||
async def test_attack_liveness_private_redirect_dies(monkeypatch) -> None:
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(302, headers={"location": "http://169.254.169.254/x"})
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
async with httpx.AsyncClient(transport=transport) as client:
|
||||
status = await check_liveness("https://example.org/redirect", client)
|
||||
assert status == LivenessStatus.DEAD
|
||||
|
||||
|
||||
async def test_attack_liveness_redirect_chain_to_dead(monkeypatch) -> None:
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
if "hop1" in str(request.url):
|
||||
return httpx.Response(302, headers={"location": "https://example.org/hop2"})
|
||||
if "hop2" in str(request.url):
|
||||
return httpx.Response(302, headers={"location": "https://example.org/hop3"})
|
||||
return httpx.Response(410)
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
async with httpx.AsyncClient(transport=transport) as client:
|
||||
status = await check_liveness("https://example.org/hop1", client)
|
||||
assert status == LivenessStatus.DEAD
|
||||
|
||||
|
||||
async def test_attack_oversized_response_capped(monkeypatch) -> None:
|
||||
huge = "x" * (5 * 1024 * 1024)
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(200, text=huge)
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
async with httpx.AsyncClient(transport=transport) as client:
|
||||
status = await check_liveness("https://example.org/huge", client)
|
||||
assert status == LivenessStatus.LIVE
|
||||
|
||||
|
||||
async def test_attack_pause_resume_mid_depth_loop_over_http(tmp_path: Path) -> None:
|
||||
plan_template = json.dumps(
|
||||
{
|
||||
"sub_questions": ["verification passes", "quote anchoring"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro", "findings"],
|
||||
}
|
||||
)
|
||||
writer = "# Report\n\nClaim [p-1] and claim [p-2].\n"
|
||||
canned = [
|
||||
ChatResult(content=plan_template, model="fake-planner"),
|
||||
ChatResult(content=plan_template, model="fake-planner"),
|
||||
ChatResult(content=plan_template, model="fake-planner"),
|
||||
ChatResult(content=writer, model="fake-writer"),
|
||||
]
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(canned=canned),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
created = await client.post(
|
||||
"/api/runs",
|
||||
json={"question": "q", "methodology": "deep-report"},
|
||||
)
|
||||
assert created.status_code == 201
|
||||
run_id = created.json()["id"]
|
||||
|
||||
for _ in range(300):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "awaiting_approval":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
assert status == "awaiting_approval"
|
||||
|
||||
await client.post(f"/api/runs/{run_id}/approve")
|
||||
|
||||
paused_seen = False
|
||||
for _ in range(600):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status in ("reflecting", "searching", "extracting"):
|
||||
await client.post(f"/api/runs/{run_id}/pause")
|
||||
await client.post(f"/api/runs/{run_id}/resume")
|
||||
paused_seen = True
|
||||
break
|
||||
if status == "done":
|
||||
break
|
||||
await asyncio.sleep(0.005)
|
||||
|
||||
for _ in range(600):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "done":
|
||||
break
|
||||
await asyncio.sleep(0.005)
|
||||
assert status == "done"
|
||||
assert paused_seen, "mid-loop pause/resume must have been exercised"
|
||||
|
||||
report = await client.get(f"/api/runs/{run_id}/report")
|
||||
assert report.status_code == 200
|
||||
|
||||
|
||||
async def test_attack_cancel_mid_loop_persists(tmp_path: Path) -> None:
|
||||
plan = json.dumps(
|
||||
{
|
||||
"sub_questions": ["verification passes", "quote anchoring"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro"],
|
||||
}
|
||||
)
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(
|
||||
canned=[ChatResult(content=plan, model="fake-planner") for _ in range(10)]
|
||||
+ [ChatResult(content="# never reached", model="fake-writer")]
|
||||
),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "deep-report"}
|
||||
)
|
||||
run_id = created.json()["id"]
|
||||
for _ in range(300):
|
||||
if (await client.get(f"/api/runs/{run_id}")).json()["status"] == "awaiting_approval":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
await client.post(f"/api/runs/{run_id}/approve")
|
||||
for _ in range(600):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status in ("searching", "extracting", "reflecting"):
|
||||
await client.post(f"/api/runs/{run_id}/cancel")
|
||||
break
|
||||
await asyncio.sleep(0.005)
|
||||
for _ in range(300):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "cancelled":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
assert status == "cancelled"
|
||||
|
||||
def test_attack_weak_band_pinned_real_input() -> None:
|
||||
store = EvidenceStore()
|
||||
passage_text = (
|
||||
"The verification pipeline re-anchored every quoted claim to its stored "
|
||||
"passage before the report was allowed to ship to the reading room."
|
||||
)
|
||||
passage = store.add_passage("https://example.org/weak", passage_text)
|
||||
words = passage_text.split()
|
||||
near_miss = " ".join(words[:11]) + " released to the audience."
|
||||
verdict = anchor_quote(near_miss, [passage])
|
||||
assert 0.80 <= verdict.ratio < 0.85, f"expected weak band, got {verdict.ratio}"
|
||||
assert verdict.matched is False
|
||||
assert verdict.weak is True, f"near-threshold must flag weak, got {verdict.ratio}"
|
||||
|
||||
|
||||
def test_attack_fully_fabricated_report_fails_export() -> None:
|
||||
store = EvidenceStore()
|
||||
real = store.add_passage("https://example.org/real", "A genuine stored passage " * 10)
|
||||
fabricated_report = (
|
||||
"# Report\n\nClaim [p-1].\n\nClaim [p-7]. Claim [p-42].\n"
|
||||
)
|
||||
ledger = build_ledger(
|
||||
run_id="attack002",
|
||||
report_text=fabricated_report,
|
||||
evidence=store,
|
||||
liveness={"https://example.org/real": LivenessStatus.LIVE},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
passed, failures = liveness_export_gate(ledger)
|
||||
assert passed is False, "fully-fabricated report must fail the export gate"
|
||||
assert failures == []
|
||||
assert ledger.checklist.unanchored_claims >= 2
|
||||
del real
|
||||
|
||||
|
||||
def test_export_round_trip_json_validates_into_backend_model() -> None:
|
||||
from researchx.trust.ledger import CitationLedger
|
||||
|
||||
store = EvidenceStore()
|
||||
stored_text = "A passage with real content for the round-trip proof."
|
||||
store.add_passage("https://example.org/a", stored_text)
|
||||
report = f"# Report\n\n{stored_text} [p-1].\n"
|
||||
ledger = build_ledger(
|
||||
run_id="attack003",
|
||||
report_text=report,
|
||||
evidence=store,
|
||||
liveness={"https://example.org/a": LivenessStatus.LIVE},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
exported = ledger.model_dump_json()
|
||||
restored = CitationLedger.model_validate_json(exported)
|
||||
assert restored == ledger
|
||||
checklist_md = export_checklist(ledger)
|
||||
assert "Anchoring gate" in checklist_md
|
||||
assert "PASS" in checklist_md
|
||||
|
||||
|
||||
def test_attack_fabricated_quote_on_real_passage_id_fails() -> None:
|
||||
store = EvidenceStore()
|
||||
stored = store.add_passage(
|
||||
"https://example.org/real",
|
||||
"The genuine finding about verification pipelines is that anchoring works. " * 3,
|
||||
)
|
||||
report = (
|
||||
"# Report\n\n"
|
||||
f"THIS FABRICATED QUOTE WAS NEVER STORED ANYWHERE AT ALL [{stored.passage_id}]\n"
|
||||
)
|
||||
ledger = build_ledger(
|
||||
run_id="attack004",
|
||||
report_text=report,
|
||||
evidence=store,
|
||||
liveness={"https://example.org/real": LivenessStatus.LIVE},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
assert ledger.checklist.unanchored_claims >= 1, (
|
||||
"a fabricated quote riding a real passage ID must be flagged (REQ-20)"
|
||||
)
|
||||
passed, _ = liveness_export_gate(ledger)
|
||||
assert passed is False
|
||||
|
||||
|
||||
def test_attack_real_verbatim_quote_on_real_passage_anchors() -> None:
|
||||
store = EvidenceStore()
|
||||
stored_text = (
|
||||
"The genuine finding about verification pipelines is that anchoring works."
|
||||
)
|
||||
stored = store.add_passage("https://example.org/real", stored_text + " " + stored_text)
|
||||
report = f"# Report\n\n{stored_text} [{stored.passage_id}]\n"
|
||||
ledger = build_ledger(
|
||||
run_id="attack005",
|
||||
report_text=report,
|
||||
evidence=store,
|
||||
liveness={"https://example.org/real": LivenessStatus.LIVE},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
assert ledger.checklist.anchored_claims == 1
|
||||
assert ledger.checklist.unanchored_claims == 0
|
||||
passed, _ = liveness_export_gate(ledger)
|
||||
assert passed is True
|
||||
|
||||
|
||||
async def test_attack_search_result_redirecting_to_private_ip_refused(monkeypatch) -> None:
|
||||
import tempfile
|
||||
from pathlib import Path as PathMod
|
||||
|
||||
import httpx as httpx_mod
|
||||
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.tavily import TavilySearchAdapter
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
env_file = PathMod(tmpdir) / ".env.secrets"
|
||||
env_file.write_text("TAVILY_API_KEY=t-key\n", encoding="utf-8")
|
||||
secrets = EnvFileSecretsProvider(paths=[env_file])
|
||||
adapter = TavilySearchAdapter(secrets)
|
||||
|
||||
def handler(request: httpx_mod.Request) -> httpx_mod.Response:
|
||||
if "boobytrap" in str(request.url):
|
||||
return httpx_mod.Response(
|
||||
302, headers={"location": "http://169.254.169.254/latest/meta-data"}
|
||||
)
|
||||
return httpx_mod.Response(200, text="<html><body>ok</body></html>")
|
||||
|
||||
transport = httpx_mod.MockTransport(handler)
|
||||
original = httpx_mod.AsyncClient
|
||||
monkeypatch.setattr(
|
||||
httpx_mod, "AsyncClient", lambda **kw: original(transport=transport, **kw)
|
||||
)
|
||||
|
||||
try:
|
||||
await adapter.fetch("https://example.org/boobytrap")
|
||||
except AdapterError as exc:
|
||||
assert "refused" in str(exc) or "not allowed" in str(exc)
|
||||
else:
|
||||
raise AssertionError("fetch of a private-redirecting URL must be refused")
|
||||
@@ -0,0 +1,160 @@
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.api.app import create_app
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{"sub_questions": ["verification passes"], "source_universe": ["web"], "outline": ["intro"]}
|
||||
)
|
||||
_WRITER = "# Report\n\nClaim [p-1] and claim [p-2].\n"
|
||||
|
||||
|
||||
def _app(tmp_path: Path):
|
||||
# Zero-key dev harness (C3-16): auth explicitly off — the synthetic
|
||||
# dev admin resolves for every request.
|
||||
return create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER, model="fake-writer"),
|
||||
]
|
||||
),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
|
||||
|
||||
async def _client(tmp_path: Path) -> httpx.AsyncClient:
|
||||
return httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=_app(tmp_path)),
|
||||
base_url="http://testserver",
|
||||
)
|
||||
|
||||
|
||||
async def test_config_status(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
response = await client.get("/api/config")
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert body["model_adapter"] == "FakeModelAdapter"
|
||||
assert body["cost_counters"]["runs_created"] == 0
|
||||
|
||||
|
||||
async def test_methodology_cards(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
response = await client.get("/api/methodologies")
|
||||
assert response.status_code == 200
|
||||
cards = response.json()
|
||||
slugs = [c["slug"] for c in cards]
|
||||
assert "rapid-answer" in slugs and "deep-report" in slugs
|
||||
for card in cards:
|
||||
assert card["eta_minutes"] > 0
|
||||
assert "estimate" in card["cost_estimate"]
|
||||
assert card["why_recommended"]
|
||||
assert card["dials"]["depth"] >= 1
|
||||
|
||||
|
||||
async def test_create_run_rejects_unknown_methodology(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
response = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "nonexistent"}
|
||||
)
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
async def _await_status(client: httpx.AsyncClient, run_id: str, status: str) -> None:
|
||||
for _ in range(200):
|
||||
current = (await client.get(f"/api/runs/{run_id}")).json()
|
||||
if current["status"] == status:
|
||||
return
|
||||
await asyncio.sleep(0.01)
|
||||
raise AssertionError(f"run never reached {status}: {current['status']}")
|
||||
|
||||
|
||||
async def test_run_lifecycle_over_http(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "How do engines run?", "methodology": "rapid-answer"}
|
||||
)
|
||||
assert created.status_code == 201
|
||||
run_id = created.json()["id"]
|
||||
|
||||
await _await_status(client, run_id, "awaiting_approval")
|
||||
approved = await client.post(f"/api/runs/{run_id}/approve")
|
||||
assert approved.status_code == 200
|
||||
|
||||
await _await_status(client, run_id, "done")
|
||||
report = await client.get(f"/api/runs/{run_id}/report")
|
||||
assert report.status_code == 200
|
||||
assert "[p-1]" in report.json()["markdown"]
|
||||
|
||||
ledger = await client.get(f"/api/runs/{run_id}/ledger")
|
||||
assert ledger.status_code == 200
|
||||
assert ledger.json()["checklist"]["citations_checked"] >= 1
|
||||
|
||||
config = (await client.get("/api/config")).json()
|
||||
assert config["cost_counters"]["runs_created"] == 1
|
||||
assert config["cost_counters"]["runs_completed"] == 1
|
||||
|
||||
|
||||
async def test_missing_run_404(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
assert (await client.get("/api/runs/zzzz9999")).status_code == 404
|
||||
assert (await client.post("/api/runs/zzzz9999/approve")).status_code == 404
|
||||
assert (await client.get("/api/runs/zzzz9999/events")).status_code == 404
|
||||
|
||||
|
||||
async def test_sse_stream_replays_events(tmp_path: Path) -> None:
|
||||
client = await _client(tmp_path)
|
||||
try:
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}
|
||||
)
|
||||
run_id = created.json()["id"]
|
||||
await _await_status(client, run_id, "awaiting_approval")
|
||||
await client.post(f"/api/runs/{run_id}/approve")
|
||||
await _await_status(client, run_id, "done")
|
||||
|
||||
collected = b""
|
||||
async with client.stream("GET", f"/api/runs/{run_id}/events") as response:
|
||||
assert response.status_code == 200
|
||||
assert response.headers["content-type"].startswith("text/event-stream")
|
||||
async for chunk in response.aiter_bytes():
|
||||
collected += chunk
|
||||
if b"run_done" in collected:
|
||||
break
|
||||
text = collected.decode()
|
||||
assert "event: run_started" in text
|
||||
assert "event: run_done" in text
|
||||
finally:
|
||||
await client.aclose()
|
||||
|
||||
|
||||
async def test_pause_resume_cancel_over_http(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}
|
||||
)
|
||||
run_id = created.json()["id"]
|
||||
await _await_status(client, run_id, "awaiting_approval")
|
||||
|
||||
paused = await client.post(f"/api/runs/{run_id}/pause")
|
||||
assert paused.status_code == 200
|
||||
resumed = await client.post(f"/api/runs/{run_id}/resume")
|
||||
assert resumed.status_code == 200
|
||||
|
||||
cancelled = await client.post(f"/api/runs/{run_id}/cancel")
|
||||
assert cancelled.status_code == 200
|
||||
for _ in range(200):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "cancelled":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
assert status == "cancelled"
|
||||
@@ -0,0 +1,325 @@
|
||||
"""Structured artifacts at DONE (T-3.1, D-113): report.json +
|
||||
evidence.json — marker normalization at the persistence boundary ONLY
|
||||
(C3-10: report.json bodies carry [^pN], report.md keeps raw [p-N])."""
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import re
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.engine.actor import EngineError, RunActor, make_run_config
|
||||
from researchx.engine.artifacts import (
|
||||
build_evidence_artifact,
|
||||
build_report_artifact,
|
||||
extract_title,
|
||||
normalize_markers,
|
||||
normalize_passage_id,
|
||||
split_sections,
|
||||
)
|
||||
from researchx.engine.events import EventBus
|
||||
from researchx.engine.states import RunStateName
|
||||
from researchx.evidence.store import EvidenceStore
|
||||
from researchx.methodologies import registry
|
||||
from researchx.trust.ledger import build_ledger
|
||||
from researchx.trust.liveness import LivenessStatus
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{
|
||||
"sub_questions": ["methodology verification", "quote anchoring"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro", "findings"],
|
||||
}
|
||||
)
|
||||
|
||||
# Multi-section report with [p-N] markers on multiple lines.
|
||||
_WRITER_REPORT = (
|
||||
"# Battery Advances\n\n"
|
||||
"The verification pass re-anchored every quoted claim [p-1].\n\n"
|
||||
"## Findings\n\n"
|
||||
"Methodology-first selection converts ad-hoc prompting [p-1] "
|
||||
"into reproducible decisions [p-2].\n\n"
|
||||
"## Conclusion\n\n"
|
||||
"The anchored loop holds [p-2].\n"
|
||||
)
|
||||
|
||||
|
||||
def _canned() -> list[ChatResult]:
|
||||
return [
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER_REPORT, model="fake-writer"),
|
||||
]
|
||||
|
||||
|
||||
def _storage(tmp_path: Path) -> LocalStorageAdapter:
|
||||
return LocalStorageAdapter(base_dir=tmp_path, owner_id="abcd2345")
|
||||
|
||||
|
||||
async def _run_to_done(tmp_path: Path) -> RunActor:
|
||||
_run_id, config = make_run_config(
|
||||
question="How do verification pipelines keep citations honest?",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
owner_id="abcd2345",
|
||||
)
|
||||
model = FakeModelAdapter(canned=_canned())
|
||||
search = FakeSearchAdapter(CORPUS)
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=model,
|
||||
search=search,
|
||||
storage=_storage(tmp_path),
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
actor.state.owner_id = "abcd2345"
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state != RunStateName.AWAITING_APPROVAL:
|
||||
await asyncio.sleep(0.01)
|
||||
if task.done():
|
||||
break
|
||||
await actor.approve()
|
||||
await task
|
||||
assert actor.state.state == RunStateName.DONE
|
||||
return actor
|
||||
|
||||
|
||||
async def _read_json(tmp_path: Path, name: str) -> dict[str, object]:
|
||||
raw = await _storage(tmp_path).read_run_artifact("abcd2345", name)
|
||||
return json.loads(raw) # type: ignore[no-any-return]
|
||||
|
||||
|
||||
# ------------------------------------------------------------- unit helpers
|
||||
|
||||
|
||||
def test_normalize_markers_converts_p_dash_form() -> None:
|
||||
assert normalize_markers("claim [p-1] and [p-12]") == "claim [^p1] and [^p12]"
|
||||
|
||||
|
||||
def test_normalize_passage_id_renumbers() -> None:
|
||||
assert normalize_passage_id("p-1") == "p1"
|
||||
assert normalize_passage_id("p-42") == "p42"
|
||||
|
||||
|
||||
def test_split_sections_on_headings() -> None:
|
||||
sections = split_sections("# Title\n\nintro body\n\n## A\n\na body\n\n### B\n\nb body")
|
||||
assert sections == [
|
||||
("Title", "intro body"),
|
||||
("A", "a body"),
|
||||
("B", "b body"),
|
||||
]
|
||||
|
||||
|
||||
def test_split_sections_preamble_and_no_headings() -> None:
|
||||
assert split_sections("loose text only") == [("", "loose text only")]
|
||||
assert split_sections("before\n\n# H\n\nafter") == [("", "before"), ("H", "after")]
|
||||
|
||||
|
||||
def test_extract_title_prefers_first_h1() -> None:
|
||||
assert extract_title("# First H1\n\n## Second\n", "fallback") == "First H1"
|
||||
assert extract_title("no heading at all", "Deep Report — q?") == "Deep Report — q?"
|
||||
|
||||
|
||||
# --------------------------------------------------------------- build paths
|
||||
|
||||
|
||||
def _fixture_store() -> EvidenceStore:
|
||||
store = EvidenceStore()
|
||||
store.add_page(
|
||||
"https://example.org/verification",
|
||||
"Verification passes in research pipelines",
|
||||
(
|
||||
"The verification pass re-anchored every quoted claim to its stored "
|
||||
"passage before the report was allowed to ship, so the audit trail "
|
||||
"stays honest claim by claim."
|
||||
),
|
||||
)
|
||||
return store
|
||||
|
||||
|
||||
def test_build_evidence_artifact_renumbers_ids_and_keeps_text() -> None:
|
||||
artifact = build_evidence_artifact(run_id="abcd2345", evidence=_fixture_store())
|
||||
payload = json.loads(artifact.model_dump_json())
|
||||
assert payload["v"] == 1
|
||||
assert payload["run_id"] == "abcd2345"
|
||||
assert len(payload["passages"]) == 1
|
||||
passage = payload["passages"][0]
|
||||
assert passage["passage_id"] == "p1" # p-1 -> p1
|
||||
assert passage["url"] == "https://example.org/verification"
|
||||
assert passage["title"] == "Verification passes in research pipelines"
|
||||
assert passage["quote"] == passage["text"][:200]
|
||||
assert "re-anchored" in passage["text"]
|
||||
|
||||
|
||||
def test_build_report_artifact_sections_and_citations() -> None:
|
||||
store = _fixture_store()
|
||||
markdown = "# Report\n\nClaim [p-1] here.\n\n## Findings\n\nAnother [p-1] claim.\n"
|
||||
ledger = build_ledger(
|
||||
run_id="abcd2345",
|
||||
report_text=markdown,
|
||||
evidence=store,
|
||||
liveness={"https://example.org/verification": LivenessStatus.LIVE},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
artifact = build_report_artifact(
|
||||
run_id="abcd2345",
|
||||
owner_id="abcd2345",
|
||||
question="q?",
|
||||
methodology_name="Rapid Answer",
|
||||
methodology_slug="rapid-answer",
|
||||
report_markdown=markdown,
|
||||
ledger=ledger,
|
||||
evidence=store,
|
||||
generated_at=ledger.generated_at,
|
||||
)
|
||||
payload = json.loads(artifact.model_dump_json())
|
||||
assert payload["v"] == 1
|
||||
assert payload["owner_id"] == "abcd2345"
|
||||
assert payload["title"] == "Report"
|
||||
assert payload["question"] == "q?"
|
||||
assert payload["methodology_slug"] == "rapid-answer"
|
||||
assert [s["heading"] for s in payload["sections"]] == ["Report", "Findings"]
|
||||
# C3-10: normalized markers in section bodies ONLY.
|
||||
assert "[^p1]" in payload["sections"][0]["body"]
|
||||
assert "[p-1]" not in payload["sections"][0]["body"]
|
||||
assert [c["passage_id"] for c in payload["citations"]] == ["p1"]
|
||||
citation = payload["citations"][0]
|
||||
assert citation["url"] == "https://example.org/verification"
|
||||
assert citation["title"] == "Verification passes in research pipelines"
|
||||
assert citation["quote"].startswith("The verification pass")
|
||||
assert citation["health"] == "live"
|
||||
assert citation["authority_tier"] == "reference"
|
||||
|
||||
|
||||
def test_build_report_artifact_title_fallback_uses_methodology_name() -> None:
|
||||
store = _fixture_store()
|
||||
ledger = build_ledger(
|
||||
run_id="abcd2345",
|
||||
report_text="no heading [p-1]",
|
||||
evidence=store,
|
||||
liveness={},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
artifact = build_report_artifact(
|
||||
run_id="abcd2345",
|
||||
owner_id=None,
|
||||
question="what about X?",
|
||||
methodology_name="Deep Report",
|
||||
methodology_slug="deep-report",
|
||||
report_markdown="no heading [p-1]",
|
||||
ledger=ledger,
|
||||
evidence=store,
|
||||
generated_at=ledger.generated_at,
|
||||
)
|
||||
assert artifact.title == "Deep Report — what about X?"
|
||||
|
||||
|
||||
# --------------------------------------------------- at-DONE persistence
|
||||
|
||||
|
||||
async def test_done_writes_report_and_evidence_artifacts(tmp_path: Path) -> None:
|
||||
await _run_to_done(tmp_path)
|
||||
|
||||
report_payload = await _read_json(tmp_path, "report.json")
|
||||
evidence_payload = await _read_json(tmp_path, "evidence.json")
|
||||
|
||||
# report.json shape
|
||||
assert report_payload["v"] == 1
|
||||
assert report_payload["run_id"] == "abcd2345"
|
||||
assert report_payload["owner_id"] == "abcd2345"
|
||||
assert report_payload["title"] == "Battery Advances"
|
||||
assert report_payload["methodology_slug"] == "rapid-answer"
|
||||
assert "generated_at" in report_payload
|
||||
assert [s["heading"] for s in report_payload["sections"]] == [ # type: ignore[call-overload]
|
||||
"Battery Advances",
|
||||
"Findings",
|
||||
"Conclusion",
|
||||
]
|
||||
|
||||
# evidence.json shape
|
||||
assert evidence_payload["v"] == 1
|
||||
assert evidence_payload["run_id"] == "abcd2345"
|
||||
assert len(evidence_payload["passages"]) >= 2 # type: ignore[arg-type]
|
||||
|
||||
|
||||
async def test_marker_normalization_at_boundary_only(tmp_path: Path) -> None:
|
||||
"""C3-10 pinned end to end: report.json bodies carry [^pN]; the raw
|
||||
report.md keeps [p-N] (the trust-loop regexes' form)."""
|
||||
await _run_to_done(tmp_path)
|
||||
|
||||
report_payload = await _read_json(tmp_path, "report.json")
|
||||
markdown = await _storage(tmp_path).read_run_artifact("abcd2345", "report.md")
|
||||
|
||||
sections = report_payload["sections"]
|
||||
assert isinstance(sections, list) and sections
|
||||
joined = "\n".join(str(s["body"]) for s in sections)
|
||||
assert "[^p1]" in joined
|
||||
assert "[p-1]" not in joined
|
||||
assert "[^p2]" in joined
|
||||
# The raw artifact is untouched — trust-loop regexes keep working.
|
||||
assert "[p-1]" in markdown
|
||||
assert "[^p1]" not in markdown
|
||||
|
||||
|
||||
async def test_citations_carry_real_quotes_and_pn_ids(tmp_path: Path) -> None:
|
||||
await _run_to_done(tmp_path)
|
||||
report_payload = await _read_json(tmp_path, "report.json")
|
||||
evidence_payload = await _read_json(tmp_path, "evidence.json")
|
||||
|
||||
citations = report_payload["citations"]
|
||||
assert isinstance(citations, list) and citations
|
||||
for citation in citations:
|
||||
assert re.fullmatch(r"^p\d+$", str(citation["passage_id"]))
|
||||
assert citation["url"].startswith("https://")
|
||||
assert citation["quote"], "citation quote must be a real stored quote"
|
||||
assert len(str(citation["quote"])) <= 200
|
||||
assert citation["health"] in {"live", "stale", "dead"}
|
||||
assert citation["authority_tier"]
|
||||
|
||||
# Evidence carries full texts: quote is the clipped head of text.
|
||||
passages = evidence_payload["passages"]
|
||||
assert isinstance(passages, list) and passages
|
||||
evidence_by_id = {str(p["passage_id"]): p for p in passages}
|
||||
for citation in citations:
|
||||
passage = evidence_by_id[str(citation["passage_id"])]
|
||||
assert str(passage["quote"]) == str(citation["quote"])
|
||||
assert len(str(passage["text"])) >= len(str(passage["quote"]))
|
||||
|
||||
|
||||
async def test_artifacts_only_written_at_done(tmp_path: Path) -> None:
|
||||
"""A run cancelled at AWAITING_APPROVAL leaves no structured
|
||||
artifacts — the write point is _verify_phase, nothing earlier."""
|
||||
_run_id, config = make_run_config(
|
||||
question="q",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
owner_id="abcd2345",
|
||||
)
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=FakeModelAdapter(canned=_canned()),
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=_storage(tmp_path),
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state != RunStateName.AWAITING_APPROVAL:
|
||||
await asyncio.sleep(0.01)
|
||||
actor.cancel()
|
||||
await actor.approve()
|
||||
with contextlib.suppress(asyncio.CancelledError, EngineError):
|
||||
await task
|
||||
run_dir = tmp_path / "runs" / "abcd2345" / "abcd2345"
|
||||
assert not (run_dir / "report.json").exists()
|
||||
assert not (run_dir / "evidence.json").exists()
|
||||
@@ -0,0 +1,590 @@
|
||||
"""Auth route suite (T-1.9 backend half, REQ-69/70 — RESEARCH §11 list).
|
||||
|
||||
Full adversarial surface of the auth layer, run with
|
||||
``auth_enabled=True``: first-user-admin, registration closing, uniform
|
||||
login failures, session/logout semantics, CSRF enforcement, admin
|
||||
create/deactivate (deactivation kills sessions), and the
|
||||
no-password-hash-anywhere guarantee.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.api.app import create_app
|
||||
from researchx.auth.guard import CSRF_HEADER
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{"sub_questions": ["verification passes"], "source_universe": ["web"], "outline": ["intro"]}
|
||||
)
|
||||
_WRITER = "# Report\n\nClaim [p-1] and claim [p-2].\n"
|
||||
|
||||
_ADMIN_EMAIL = "admin@example.org"
|
||||
_ADMIN_PASSWORD = "correct-horse-battery"
|
||||
_SECOND_EMAIL = "second@example.org"
|
||||
_SECOND_PASSWORD = "another-long-password"
|
||||
|
||||
_HEADERS = {CSRF_HEADER: "1"}
|
||||
|
||||
|
||||
def _app(tmp_path: Path):
|
||||
return create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER, model="fake-writer"),
|
||||
]
|
||||
),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=True,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
|
||||
|
||||
async def _client(tmp_path: Path) -> httpx.AsyncClient:
|
||||
return httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=_app(tmp_path)),
|
||||
base_url="http://testserver",
|
||||
)
|
||||
|
||||
|
||||
async def _register_first_admin(client: httpx.AsyncClient) -> httpx.Response:
|
||||
return await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD, "display_name": "Admin"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
|
||||
async def _login(client: httpx.AsyncClient, email: str, password: str) -> httpx.Response:
|
||||
return await client.post(
|
||||
"/api/auth/login", json={"email": email, "password": password}, headers=_HEADERS
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ register
|
||||
|
||||
|
||||
async def test_healthz_outside_api_is_public(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
response = await client.get("/healthz")
|
||||
assert response.status_code == 200
|
||||
assert response.json() == {"status": "ok"}
|
||||
|
||||
|
||||
async def test_first_register_creates_admin(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
response = await _register_first_admin(client)
|
||||
assert response.status_code == 201
|
||||
body = response.json()
|
||||
assert body["role"] == "admin"
|
||||
assert body["email"] == _ADMIN_EMAIL
|
||||
assert body["active"] is True
|
||||
assert len(body["user_id"]) == 8
|
||||
assert "password_hash" not in body
|
||||
assert "password" not in body
|
||||
|
||||
|
||||
async def test_second_register_is_403(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
response = await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert response.status_code == 403
|
||||
assert response.json()["detail"] == "registration closed"
|
||||
|
||||
|
||||
async def test_register_rejects_short_password(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
response = await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": _ADMIN_EMAIL, "password": "short"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert response.status_code == 422 # pydantic contract rejects < 12
|
||||
|
||||
|
||||
async def test_register_requires_csrf_header(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
response = await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD},
|
||||
)
|
||||
assert response.status_code == 403
|
||||
assert response.json()["detail"] == "missing request header"
|
||||
|
||||
|
||||
async def test_concurrent_register_yields_exactly_one_admin(tmp_path: Path) -> None:
|
||||
"""D-104 adversarial pin: a simultaneous register storm on a fresh
|
||||
store must produce exactly one 201/admin — the emptiness check +
|
||||
insert are atomic under the store lock (create_first_admin), so a
|
||||
raced duplicate cannot both win. Regression guard for the
|
||||
create_user-based route that let N concurrent registers through."""
|
||||
racers = [(f"racer{i}@example.org", f"racing-password-{i:02d}") for i in range(8)]
|
||||
async with await _client(tmp_path) as client:
|
||||
responses = await asyncio.gather(
|
||||
*(
|
||||
client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": email, "password": password},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
for email, password in racers
|
||||
)
|
||||
)
|
||||
winners = [(r, racer) for r, racer in zip(responses, racers, strict=True) if r.status_code == 201]
|
||||
losers = [r for r in responses if r.status_code == 403]
|
||||
assert len(winners) == 1, "exactly one register may win the race"
|
||||
assert len(losers) == 7
|
||||
assert winners[0][0].json()["role"] == "admin"
|
||||
|
||||
# The winner can log in; every losing racer has no account.
|
||||
_, (winner_email, winner_password) = winners[0]
|
||||
login = await client.post(
|
||||
"/api/auth/login",
|
||||
json={"email": winner_email, "password": winner_password},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert login.status_code == 200
|
||||
for email, password in racers:
|
||||
if email == winner_email:
|
||||
continue
|
||||
denied = await client.post(
|
||||
"/api/auth/login",
|
||||
json={"email": email, "password": password},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert denied.status_code == 401, "losing racer must have no account"
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- login
|
||||
|
||||
|
||||
async def test_login_sets_httponly_cookie(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
response = await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
assert response.status_code == 200
|
||||
assert response.json()["role"] == "admin"
|
||||
set_cookie = response.headers["set-cookie"]
|
||||
assert "researchx_session=" in set_cookie
|
||||
assert "httponly" in set_cookie.lower()
|
||||
assert "samesite=lax" in set_cookie.lower()
|
||||
assert "path=/" in set_cookie.lower()
|
||||
assert "domain=" not in set_cookie.lower().replace("samesite=lax", "")
|
||||
assert "secure" not in set_cookie.lower() # cookie_secure=False in tests
|
||||
|
||||
|
||||
async def test_login_wrong_password_uniform_401(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
for email, password in [
|
||||
(_ADMIN_EMAIL, "wrong-password-here"),
|
||||
("ghost@example.org", _ADMIN_PASSWORD),
|
||||
]:
|
||||
response = await _login(client, email, password)
|
||||
assert response.status_code == 401
|
||||
assert response.json()["detail"] == "invalid email or password"
|
||||
|
||||
|
||||
async def test_login_token_rotation(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
first = await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
first_cookie = first.headers["set-cookie"].split(";")[0]
|
||||
second = await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
second_cookie = second.headers["set-cookie"].split(";")[0]
|
||||
assert first_cookie != second_cookie # fresh token every login
|
||||
|
||||
|
||||
# ----------------------------------------------------------- guard + CSRF
|
||||
|
||||
|
||||
async def test_unauthenticated_401_on_protected_routes(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
for method, path in [
|
||||
("GET", "/api/methodologies"),
|
||||
("GET", "/api/config"),
|
||||
("GET", "/api/runs/abcd2345"),
|
||||
("GET", "/api/runs/abcd2345/events"),
|
||||
("GET", "/api/runs/abcd2345/report"),
|
||||
("GET", "/api/admin/users"),
|
||||
]:
|
||||
response = await client.request(method, path)
|
||||
assert response.status_code == 401, f"{method} {path} leaked without auth"
|
||||
assert response.json()["detail"] == "authentication required"
|
||||
|
||||
|
||||
async def test_csrf_header_required_on_non_get(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
# Session cookie attached, but no CSRF header on a POST.
|
||||
response = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}
|
||||
)
|
||||
assert response.status_code == 403
|
||||
assert response.json()["detail"] == "missing request header"
|
||||
|
||||
|
||||
async def test_gets_exempt_from_csrf_header(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
assert (await client.get("/api/methodologies")).status_code == 200
|
||||
assert (await client.get("/api/auth/me")).status_code == 200
|
||||
|
||||
|
||||
# ----------------------------------------------------------- me + logout
|
||||
|
||||
|
||||
async def test_me_returns_current_user(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
response = await client.get("/api/auth/me")
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert body["email"] == _ADMIN_EMAIL
|
||||
assert "password_hash" not in body
|
||||
|
||||
|
||||
async def test_logout_revokes_session(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
assert (await client.get("/api/methodologies")).status_code == 200
|
||||
response = await client.post("/api/auth/logout", headers=_HEADERS)
|
||||
assert response.status_code == 200
|
||||
assert (await client.get("/api/methodologies")).status_code == 401
|
||||
|
||||
|
||||
async def test_deactivated_user_401(tmp_path: Path) -> None:
|
||||
app = _app(tmp_path)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _register_first_admin(admin)
|
||||
await _login(admin, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
created = await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert created.status_code == 201
|
||||
second_id = created.json()["user_id"]
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
await second.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert (await second.get("/api/methodologies")).status_code == 200
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _login(admin, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
deactivated = await admin.post(
|
||||
f"/api/admin/users/{second_id}/deactivate", headers=_HEADERS
|
||||
)
|
||||
assert deactivated.status_code == 200
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
# The deactivated user's session was revoked server-side.
|
||||
response = await second.get("/api/methodologies")
|
||||
assert response.status_code == 401
|
||||
|
||||
# A fresh login against the deactivated account also 401s —
|
||||
# uniformly, without revealing the reason (no enumeration).
|
||||
login = await second.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert login.status_code == 401
|
||||
assert login.json()["detail"] == "invalid email or password"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- admin
|
||||
|
||||
|
||||
async def test_admin_list_users_no_hashes(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
response = await client.get("/api/admin/users")
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert len(body) == 1
|
||||
raw = json.dumps(body)
|
||||
assert "password_hash" not in raw
|
||||
assert _ADMIN_PASSWORD not in raw
|
||||
assert "argon2" not in raw
|
||||
|
||||
|
||||
async def test_admin_create_user_then_user_login(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
created = await client.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert created.status_code == 201
|
||||
body = created.json()
|
||||
assert body["role"] == "user"
|
||||
assert body["active"] is True
|
||||
|
||||
login = await _login(client, _SECOND_EMAIL, _SECOND_PASSWORD)
|
||||
assert login.status_code == 200
|
||||
assert login.json()["role"] == "user"
|
||||
|
||||
|
||||
async def test_admin_create_rejects_short_password(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
response = await client.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": "short"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert response.status_code == 422
|
||||
|
||||
|
||||
async def test_admin_deactivate_revokes_sessions(tmp_path: Path) -> None:
|
||||
app = _app(tmp_path)
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as admin:
|
||||
await _register_first_admin(admin)
|
||||
await _login(admin, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
created = await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
second_id = created.json()["user_id"]
|
||||
|
||||
# A separate client (separate cookie jar) for the second user.
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as second:
|
||||
await second.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert (await second.get("/api/methodologies")).status_code == 200
|
||||
|
||||
# Deactivation from the admin session kills the second session.
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as admin:
|
||||
await _register_first_admin(admin) # store shared; already admin
|
||||
await _login(admin, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
deactivated = await admin.post(
|
||||
f"/api/admin/users/{second_id}/deactivate", headers=_HEADERS
|
||||
)
|
||||
assert deactivated.status_code == 200
|
||||
assert deactivated.json()["active"] is False
|
||||
|
||||
response = await second.get("/api/methodologies")
|
||||
assert response.status_code == 401 # session revoked by deactivation
|
||||
|
||||
|
||||
async def test_non_admin_cannot_reach_admin_routes(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
await client.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
await _login(client, _SECOND_EMAIL, _SECOND_PASSWORD)
|
||||
response = await client.get("/api/admin/users")
|
||||
assert response.status_code == 403
|
||||
assert response.json()["detail"] == "admin role required"
|
||||
|
||||
|
||||
async def test_admin_deactivate_unknown_user_404(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
response = await client.post("/api/admin/users/zzzz9999/deactivate", headers=_HEADERS)
|
||||
assert response.status_code == 404
|
||||
|
||||
|
||||
# ------------------------------------------------------ authenticated runs
|
||||
|
||||
|
||||
async def test_cross_owner_access_404_on_every_run_endpoint(tmp_path: Path) -> None:
|
||||
"""Two-user isolation (REQ-71): zero visibility across every run
|
||||
endpoint — cross-owner access is indistinguishable from a missing
|
||||
run (404, not 403: no existence leak)."""
|
||||
app = _app(tmp_path)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _register_first_admin(admin)
|
||||
await _login(admin, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
created = await admin.post(
|
||||
"/api/runs",
|
||||
json={"question": "admin's private question", "methodology": "rapid-answer"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert created.status_code == 201
|
||||
run_id = created.json()["id"]
|
||||
|
||||
await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
login = await _login(second, _SECOND_EMAIL, _SECOND_PASSWORD)
|
||||
assert login.status_code == 200
|
||||
del login # ids don't matter here; counters do
|
||||
|
||||
for method, path in [
|
||||
("GET", f"/api/runs/{run_id}"),
|
||||
("POST", f"/api/runs/{run_id}/approve"),
|
||||
("POST", f"/api/runs/{run_id}/pause"),
|
||||
("POST", f"/api/runs/{run_id}/resume"),
|
||||
("POST", f"/api/runs/{run_id}/cancel"),
|
||||
("GET", f"/api/runs/{run_id}/events"),
|
||||
("GET", f"/api/runs/{run_id}/report"),
|
||||
("GET", f"/api/runs/{run_id}/ledger"),
|
||||
]:
|
||||
response = await second.request(method, path, headers=_HEADERS)
|
||||
assert response.status_code == 404, f"{method} {path} leaked across owners"
|
||||
|
||||
# The owner still sees their run — the 404s were authorization, not
|
||||
# the run vanishing.
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _login(admin, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
response = await admin.get(f"/api/runs/{run_id}")
|
||||
assert response.status_code == 200
|
||||
|
||||
|
||||
async def test_owner_id_recorded_in_persisted_state(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
login = await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
user_id = login.json()["user_id"]
|
||||
created = await client.post(
|
||||
"/api/runs",
|
||||
json={"question": "q", "methodology": "rapid-answer"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
run_id = created.json()["id"]
|
||||
|
||||
state_path = tmp_path / "runs" / user_id / run_id / "state.json"
|
||||
for _ in range(200): # the actor persists asynchronously
|
||||
if state_path.exists():
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
assert state_path.exists(), "storage layout must be runs/<user_id>/<run_id>/"
|
||||
state_text = state_path.read_text()
|
||||
state = json.loads(state_text)
|
||||
assert state["owner_id"] == user_id
|
||||
|
||||
|
||||
async def test_cost_counters_owner_keyed(tmp_path: Path) -> None:
|
||||
app = _app(tmp_path)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _register_first_admin(admin)
|
||||
await _login(admin, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
created = await admin.post(
|
||||
"/api/runs",
|
||||
json={"question": "q", "methodology": "rapid-answer"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert created.status_code == 201
|
||||
config = await admin.get("/api/config")
|
||||
counters = config.json()["cost_counters"]
|
||||
assert counters["runs_created"] == 1 # the admin's own view, owner-keyed
|
||||
|
||||
await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
await _login(second, _SECOND_EMAIL, _SECOND_PASSWORD)
|
||||
created = await second.post(
|
||||
"/api/runs",
|
||||
json={"question": "q2", "methodology": "rapid-answer"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert created.status_code == 201
|
||||
config = await second.get("/api/config")
|
||||
assert config.json()["cost_counters"] == {"runs_created": 1, "runs_completed": 0}
|
||||
|
||||
|
||||
async def test_authenticated_run_lifecycle(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
created = await client.post(
|
||||
"/api/runs",
|
||||
json={"question": "How do engines run?", "methodology": "rapid-answer"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert created.status_code == 201
|
||||
run_id = created.json()["id"]
|
||||
|
||||
for _ in range(200):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "awaiting_approval":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
approved = await client.post(f"/api/runs/{run_id}/approve", headers=_HEADERS)
|
||||
assert approved.status_code == 200
|
||||
for _ in range(200):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "done":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
report = await client.get(f"/api/runs/{run_id}/report")
|
||||
assert report.status_code == 200
|
||||
assert "[p-1]" in report.json()["markdown"]
|
||||
|
||||
|
||||
async def test_no_password_hash_in_any_response(tmp_path: Path) -> None:
|
||||
"""Every response body of a full auth + run session, scanned."""
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_first_admin(client)
|
||||
await _login(client, _ADMIN_EMAIL, _ADMIN_PASSWORD)
|
||||
await client.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
responses = [
|
||||
await client.get("/api/auth/me"),
|
||||
await client.get("/api/admin/users"),
|
||||
await client.get("/api/methodologies"),
|
||||
await client.get("/api/config"),
|
||||
await client.get("/api/runs/abcd2345"),
|
||||
]
|
||||
for response in responses:
|
||||
raw = response.text
|
||||
assert "$argon2" not in raw # no hash material, ever
|
||||
assert '"password_hash"' not in raw # no hash field, ever
|
||||
assert _ADMIN_PASSWORD not in raw
|
||||
assert _SECOND_PASSWORD not in raw
|
||||
@@ -0,0 +1,99 @@
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.capability_probe import probe_model_adapter
|
||||
from researchx.adapters.ollama import OllamaModelAdapter
|
||||
from researchx.adapters.probe import DEFAULT_CONTEXT_CAP, DEFAULT_EMBEDDING_BATCH
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.types import AdapterError
|
||||
|
||||
|
||||
def _secrets(tmp_path, values: dict[str, str]) -> EnvFileSecretsProvider:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("".join(f"{k}={v}\n" for k, v in values.items()), encoding="utf-8")
|
||||
return EnvFileSecretsProvider(paths=[env_file])
|
||||
|
||||
|
||||
def _ok_smoke_response() -> dict[str, object]:
|
||||
return {
|
||||
"choices": [{"message": {"role": "assistant", "content": '{"ok": true}'}}],
|
||||
"usage": {"prompt_tokens": 3, "completion_tokens": 2},
|
||||
}
|
||||
|
||||
|
||||
async def test_probe_green_on_supporting_model(tmp_path, monkeypatch) -> None:
|
||||
secrets = _secrets(tmp_path, {"OLLAMA_API_KEY": "cloud-key"})
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="w")
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
if str(request.url).endswith("/models"):
|
||||
return httpx.Response(200, json={"data": [{"id": "w"}, {"id": "p"}]})
|
||||
return httpx.Response(200, json=_ok_smoke_response())
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
probe = await probe_model_adapter(adapter)
|
||||
assert probe.endpoint == "https://ollama.com/v1"
|
||||
assert probe.models == ["w", "p"]
|
||||
assert probe.context_window == DEFAULT_CONTEXT_CAP
|
||||
assert probe.embedding_batch == DEFAULT_EMBEDDING_BATCH
|
||||
assert probe.structured_outputs_supported is True
|
||||
assert probe.structured_output_warning is None
|
||||
assert probe.degraded is False
|
||||
|
||||
|
||||
async def test_probe_warns_on_weak_structured_output(tmp_path, monkeypatch) -> None:
|
||||
secrets = _secrets(tmp_path, {"OLLAMA_API_KEY": "cloud-key"})
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="writer-model")
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
if str(request.url).endswith("/models"):
|
||||
return httpx.Response(200, json={"data": [{"id": "writer-model"}]})
|
||||
return httpx.Response(200, json={"choices": [{"message": {"content": "not json"}}]})
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
probe = await probe_model_adapter(adapter)
|
||||
assert probe.structured_outputs_supported is False
|
||||
assert probe.structured_output_warning is not None
|
||||
assert "writer-model" in probe.structured_output_warning
|
||||
assert "degrade" in probe.structured_output_warning
|
||||
assert probe.degraded is True
|
||||
|
||||
|
||||
async def test_probe_reports_endpoint_down_as_unsupported(tmp_path, monkeypatch) -> None:
|
||||
secrets = _secrets(tmp_path, {"OLLAMA_API_KEY": "cloud-key"})
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="w")
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(500, text="boom")
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
with pytest.raises(AdapterError):
|
||||
await probe_model_adapter(adapter)
|
||||
|
||||
|
||||
async def test_list_models_falls_back_to_local(tmp_path, monkeypatch) -> None:
|
||||
secrets = _secrets(tmp_path, {"OLLAMA_API_KEY": "cloud-key"})
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="w")
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
if str(request.url).startswith("https://ollama.com"):
|
||||
return httpx.Response(401, json={"error": "unauthorized"})
|
||||
return httpx.Response(200, json={"data": [{"id": "local-model"}]})
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
models = await adapter.list_models()
|
||||
assert models == ["local-model"]
|
||||
assert adapter.active_base_url == "http://localhost:11434/v1"
|
||||
assert adapter.last_fallback_reason is not None
|
||||
@@ -0,0 +1,66 @@
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from researchx.config.dials import DIAL_REGISTRY, DialDescriptor, Dials
|
||||
|
||||
EXPECTED_DIALS = {
|
||||
"source_universe": "web",
|
||||
"search_routing": "single",
|
||||
"query_strategy": "decomposed",
|
||||
"breadth": 2,
|
||||
"depth": 1,
|
||||
"quality_gates": "standard",
|
||||
"citation_rigor": "per-claim",
|
||||
"verification_passes": 1,
|
||||
"synthesis_style": "memo",
|
||||
"output_structure": "brief",
|
||||
"memory_scope": "none",
|
||||
"checkpoints": False,
|
||||
}
|
||||
|
||||
|
||||
def test_defaults_resolve() -> None:
|
||||
assert Dials().model_dump() == EXPECTED_DIALS
|
||||
|
||||
|
||||
def test_all_twelve_dials_present() -> None:
|
||||
assert set(Dials.model_fields) == set(EXPECTED_DIALS)
|
||||
assert len(Dials.model_fields) == 12
|
||||
|
||||
|
||||
def test_registry_covers_all_dials() -> None:
|
||||
assert set(DIAL_REGISTRY) == set(EXPECTED_DIALS)
|
||||
for descriptor in DIAL_REGISTRY.values():
|
||||
assert isinstance(descriptor, DialDescriptor)
|
||||
assert descriptor.label
|
||||
assert descriptor.description
|
||||
|
||||
|
||||
def test_literal_dial_rejects_invalid_value() -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
Dials(source_universe="library")
|
||||
|
||||
|
||||
def test_bounded_int_rejects_out_of_range() -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
Dials(breadth=6)
|
||||
with pytest.raises(ValidationError):
|
||||
Dials(depth=0)
|
||||
with pytest.raises(ValidationError):
|
||||
Dials(verification_passes=-1)
|
||||
|
||||
|
||||
def test_checkpoints_rejects_non_bool() -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
Dials(checkpoints=5)
|
||||
|
||||
|
||||
def test_model_dump_round_trips() -> None:
|
||||
dials = Dials(depth=3, breadth=5, checkpoints=True, synthesis_style="executive")
|
||||
assert Dials.model_validate(dials.model_dump()) == dials
|
||||
|
||||
|
||||
def test_dials_are_immutable() -> None:
|
||||
dials = Dials()
|
||||
with pytest.raises(ValidationError):
|
||||
dials.depth = 3
|
||||
@@ -0,0 +1,183 @@
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.engine.actor import RunActor, make_run_config
|
||||
from researchx.engine.events import EventBus
|
||||
from researchx.engine.states import RunStateName, can_transition
|
||||
from researchx.methodologies import registry
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{
|
||||
"sub_questions": ["methodology verification", "quote anchoring"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro", "findings"],
|
||||
}
|
||||
)
|
||||
|
||||
_WRITER_REPORT = (
|
||||
"# Report\n\n"
|
||||
"The verification pass re-anchored every quoted claim [p-1]. "
|
||||
"Methodology-first selection converts ad-hoc prompting [p-1] "
|
||||
"into reproducible decisions [p-2].\n"
|
||||
)
|
||||
|
||||
|
||||
def _canned() -> list[ChatResult]:
|
||||
return [
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER_REPORT, model="fake-writer"),
|
||||
]
|
||||
|
||||
|
||||
def _actor(tmp_path: Path) -> tuple[RunActor, EventBus, FakeSearchAdapter]:
|
||||
_run_id, config = make_run_config(
|
||||
question="How do verification pipelines keep citations honest?",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
)
|
||||
model = FakeModelAdapter(canned=_canned())
|
||||
search = FakeSearchAdapter(CORPUS)
|
||||
storage = LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4")
|
||||
bus = EventBus()
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=model,
|
||||
search=search,
|
||||
storage=storage,
|
||||
registry=registry,
|
||||
event_bus=bus,
|
||||
)
|
||||
return actor, bus, search
|
||||
|
||||
|
||||
async def test_full_walk_reaches_done(tmp_path: Path) -> None:
|
||||
actor, bus, _ = _actor(tmp_path)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state != RunStateName.AWAITING_APPROVAL:
|
||||
await asyncio.sleep(0.01)
|
||||
if task.done():
|
||||
break
|
||||
await actor.approve()
|
||||
state = await task
|
||||
assert state.state == RunStateName.DONE
|
||||
assert state.plan is not None and len(state.plan.sub_questions) == 2
|
||||
|
||||
saved = await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact("abcd2345", "state.json")
|
||||
assert json.loads(saved)["state"] == "DONE"
|
||||
|
||||
report = await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact("abcd2345", "report.md")
|
||||
assert "[p-1]" in report
|
||||
|
||||
ledger_raw = await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact(
|
||||
"abcd2345", "ledger.json"
|
||||
)
|
||||
ledger = json.loads(ledger_raw)
|
||||
assert ledger["checklist"]["citations_checked"] >= 1
|
||||
assert ledger["checklist"]["liveness_ratio"] >= 0
|
||||
|
||||
checklist = await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact(
|
||||
"abcd2345", "checklist.md"
|
||||
)
|
||||
assert "Verification Checklist" in checklist
|
||||
|
||||
types = [e.type for e in bus.replay()]
|
||||
assert "run_started" in types
|
||||
assert "query_fired" in types
|
||||
assert "source_visited" in types
|
||||
assert "passage_extracted" in types
|
||||
assert "run_done" in types
|
||||
|
||||
|
||||
async def test_halt_at_awaiting_approval_until_approved(tmp_path: Path) -> None:
|
||||
actor, _, _ = _actor(tmp_path)
|
||||
task = asyncio.create_task(actor.run())
|
||||
for _ in range(100):
|
||||
await asyncio.sleep(0.01)
|
||||
if actor.state.state == RunStateName.AWAITING_APPROVAL:
|
||||
break
|
||||
assert actor.state.state == RunStateName.AWAITING_APPROVAL
|
||||
await asyncio.sleep(0.05)
|
||||
assert not task.done()
|
||||
await actor.approve()
|
||||
state = await task
|
||||
assert state.state == RunStateName.DONE
|
||||
|
||||
|
||||
async def test_cancel_persists_cancelled_state(tmp_path: Path) -> None:
|
||||
actor, _, _ = _actor(tmp_path)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state != RunStateName.AWAITING_APPROVAL:
|
||||
await asyncio.sleep(0.01)
|
||||
actor.cancel()
|
||||
await actor.approve()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
saved = await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact("abcd2345", "state.json")
|
||||
assert json.loads(saved)["state"] == "CANCELLED"
|
||||
|
||||
|
||||
async def test_pause_resume_mid_run(tmp_path: Path) -> None:
|
||||
actor, _, _ = _actor(tmp_path)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state != RunStateName.AWAITING_APPROVAL:
|
||||
await asyncio.sleep(0.01)
|
||||
await actor.approve()
|
||||
actor.pause()
|
||||
await asyncio.sleep(0.05)
|
||||
assert not task.done()
|
||||
actor.resume()
|
||||
state = await task
|
||||
assert state.state == RunStateName.DONE
|
||||
|
||||
|
||||
def test_illegal_transitions_rejected() -> None:
|
||||
assert not can_transition(RunStateName.PLANNING, RunStateName.DONE)
|
||||
assert not can_transition(RunStateName.DONE, RunStateName.PLANNING)
|
||||
assert can_transition(RunStateName.PLANNING, RunStateName.AWAITING_APPROVAL)
|
||||
assert can_transition(RunStateName.REFLECTING, RunStateName.SEARCHING)
|
||||
|
||||
|
||||
async def test_planner_non_json_fails_honestly(tmp_path: Path) -> None:
|
||||
_run_id, config = make_run_config(
|
||||
question="q",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
)
|
||||
model = FakeModelAdapter(canned=[ChatResult(content="not json", model="fake-planner")])
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=model,
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4"),
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
task = asyncio.create_task(actor.run())
|
||||
with pytest.raises(Exception, match="planner returned non-JSON plan"):
|
||||
await task
|
||||
saved = await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact("abcd2345", "state.json")
|
||||
assert json.loads(saved)["state"] == "FAILED"
|
||||
|
||||
|
||||
async def test_writer_context_is_visited_only(tmp_path: Path) -> None:
|
||||
actor, _, _ = _actor(tmp_path)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state != RunStateName.AWAITING_APPROVAL:
|
||||
await asyncio.sleep(0.01)
|
||||
await actor.approve()
|
||||
await task
|
||||
assert actor._store.writer_context(), "evidence collected"
|
||||
outside_ids = {p.passage_id for p in actor._store.writer_context()}
|
||||
assert all(pid.startswith("p-") for pid in outside_ids)
|
||||
@@ -0,0 +1,56 @@
|
||||
from researchx.evidence.store import EvidenceStore, extract_passages
|
||||
|
||||
|
||||
def test_extract_passages_min_length() -> None:
|
||||
text = "short\n" + ("x" * 150) + "\nanother short"
|
||||
passages = extract_passages(text)
|
||||
assert passages == ["x" * 150]
|
||||
|
||||
|
||||
def test_store_stable_ids() -> None:
|
||||
store = EvidenceStore()
|
||||
first = store.add_passage("https://a", "first passage text " * 10)
|
||||
second = store.add_passage("https://a", "second passage text " * 10)
|
||||
assert first.passage_id == "p-1"
|
||||
assert second.passage_id == "p-2"
|
||||
|
||||
|
||||
def test_store_add_page_creates_passages() -> None:
|
||||
store = EvidenceStore()
|
||||
body = "\n".join(f"paragraph {i} " + "y" * 150 for i in range(3))
|
||||
store.add_page("https://example.org/a", "Title A", body)
|
||||
assert len(store) == 3
|
||||
page = store.pages()[0]
|
||||
assert page.url == "https://example.org/a"
|
||||
assert page.title == "Title A"
|
||||
assert len(page.passage_ids) == 3
|
||||
|
||||
|
||||
def test_writer_context_contains_only_stored() -> None:
|
||||
store = EvidenceStore()
|
||||
store.add_passage("https://a", "stored passage " * 20)
|
||||
context = store.writer_context()
|
||||
assert len(context) == 1
|
||||
assert context[0].url == "https://a"
|
||||
|
||||
outside = EvidenceStore()
|
||||
outside_context = outside.writer_context()
|
||||
assert outside_context == []
|
||||
|
||||
|
||||
def test_dedupe_by_quote() -> None:
|
||||
store = EvidenceStore()
|
||||
text = "identical passage text " * 10
|
||||
store.add_passage("https://a", text)
|
||||
assert store.has_seen(text[:200])
|
||||
assert not store.has_seen("something never stored " * 20)
|
||||
|
||||
|
||||
def test_passages_for_url() -> None:
|
||||
store = EvidenceStore()
|
||||
store.add_passage("https://a", "from a " * 30)
|
||||
store.add_passage("https://b", "from b " * 30)
|
||||
from_a = store.passages_for_url("https://a")
|
||||
assert len(from_a) == 1
|
||||
assert from_a[0].url == "https://a"
|
||||
assert store.passages_for_url("https://missing") == []
|
||||
@@ -0,0 +1,82 @@
|
||||
import inspect
|
||||
|
||||
from researchx.adapters import fakes
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter, corpus
|
||||
from researchx.adapters.model import ModelAdapter, ModelRoles
|
||||
from researchx.adapters.search import SearchAdapter
|
||||
from researchx.adapters.types import AdapterError, ChatResult, ModelMessage
|
||||
|
||||
|
||||
def _model() -> FakeModelAdapter:
|
||||
return FakeModelAdapter(canned=[ChatResult(content="ok", model="fake-model")])
|
||||
|
||||
|
||||
async def test_fake_model_full_interface() -> None:
|
||||
model = _model()
|
||||
assert isinstance(model, ModelAdapter)
|
||||
result = await model.chat([ModelMessage(role="user", content="hi")], model="fake-model")
|
||||
assert result.content == "ok"
|
||||
assert await model.list_models() == ["fake-model"]
|
||||
assert model.roles() == ModelRoles(planner="fake-planner", writer="fake-writer")
|
||||
|
||||
|
||||
async def test_fake_model_exhausted_raises() -> None:
|
||||
model = _model()
|
||||
await model.chat([ModelMessage(role="user", content="hi")], model="fake-model")
|
||||
try:
|
||||
await model.chat([ModelMessage(role="user", content="again")], model="fake-model")
|
||||
except AdapterError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("expected AdapterError")
|
||||
|
||||
|
||||
async def test_fake_model_json_mode_recorded() -> None:
|
||||
model = _model()
|
||||
await model.chat([ModelMessage(role="user", content="hi")], model="fake-model", json_mode=True)
|
||||
assert model.last_json_mode is True
|
||||
|
||||
|
||||
async def test_fake_search_full_interface() -> None:
|
||||
search = FakeSearchAdapter(corpus())
|
||||
assert isinstance(search, SearchAdapter)
|
||||
results = await search.search("verification quoted claim", max_results=3)
|
||||
assert 0 < len(results) <= 3
|
||||
page = await search.fetch(results[0].url)
|
||||
assert page.url == results[0].url
|
||||
|
||||
|
||||
async def test_fake_search_deterministic_order() -> None:
|
||||
search = FakeSearchAdapter(corpus())
|
||||
first = await search.search("methodology", max_results=5)
|
||||
second = await search.search("methodology", max_results=5)
|
||||
assert [r.url for r in first] == [r.url for r in second]
|
||||
|
||||
|
||||
async def test_fake_search_unknown_url_raises() -> None:
|
||||
search = FakeSearchAdapter(corpus())
|
||||
try:
|
||||
await search.fetch("https://example.org/missing")
|
||||
except AdapterError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("expected AdapterError")
|
||||
|
||||
|
||||
def test_corpus_shape() -> None:
|
||||
pages = CORPUS
|
||||
assert len(pages) >= 5
|
||||
urls = [p.url for p in pages]
|
||||
assert len(set(urls)) == len(urls)
|
||||
texts = "\n".join(p.text for p in pages)
|
||||
assert "—" in texts or "é" in texts or "近" in texts
|
||||
|
||||
|
||||
def test_corpus_shared_sentence() -> None:
|
||||
sharing = [p for p in CORPUS if fakes.SHARED_SENTENCE in p.text]
|
||||
assert len(sharing) == 2
|
||||
|
||||
|
||||
def test_fakes_no_httpx_dependency() -> None:
|
||||
source = inspect.getsource(fakes)
|
||||
assert "httpx" not in source
|
||||
@@ -0,0 +1,87 @@
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.config.run_config import RunConfig, UnknownOverrideError, new_run_id, resolve_dials
|
||||
from researchx.methodologies import registry
|
||||
|
||||
|
||||
async def test_registry_to_fake_model_to_snapshot_round_trip(tmp_path) -> None:
|
||||
methodology = registry.get("deep-report")
|
||||
canned = ChatResult(
|
||||
content='{"sub_questions": ["What is verification?", "How does anchoring work?"]}',
|
||||
model="fake-planner",
|
||||
)
|
||||
model = FakeModelAdapter(canned=[canned])
|
||||
plan_result = await model.chat([], model=model.roles().planner, json_mode=True)
|
||||
assert "sub_questions" in plan_result.content
|
||||
|
||||
run_id = new_run_id()
|
||||
dials = resolve_dials("deep-report", {"breadth": 3}, registry)
|
||||
run_config = RunConfig(
|
||||
run_id=run_id,
|
||||
methodology_slug="deep-report",
|
||||
question="How do verification pipelines keep citations honest?",
|
||||
resolved_dials=dials,
|
||||
overrides={"breadth": 3},
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
assert run_config.resolved_dials.breadth == 3
|
||||
|
||||
storage = LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4")
|
||||
await storage.write_run_artifact(run_id, "run_config.json", run_config.model_dump_json())
|
||||
|
||||
restored = RunConfig.model_validate_json(
|
||||
await storage.read_run_artifact(run_id, "run_config.json")
|
||||
)
|
||||
assert restored == run_config
|
||||
assert restored.resolved_dials.depth == methodology.dials.depth
|
||||
|
||||
|
||||
def test_methodology_behavior_difference() -> None:
|
||||
rapid = registry.get("rapid-answer")
|
||||
deep = registry.get("deep-report")
|
||||
assert rapid.dials.depth < deep.dials.depth
|
||||
assert rapid.dials.breadth < deep.dials.breadth
|
||||
assert rapid.output_shape != deep.output_shape
|
||||
assert rapid.dials.synthesis_style != deep.dials.synthesis_style
|
||||
assert rapid.dials.verification_passes < deep.dials.verification_passes
|
||||
assert rapid.dials.output_structure != deep.dials.output_structure
|
||||
|
||||
|
||||
def test_registry_snapshot_immutability() -> None:
|
||||
from pydantic import ValidationError
|
||||
|
||||
methodology = registry.get("rapid-answer")
|
||||
frozen = registry.get("rapid-answer")
|
||||
with pytest.raises(ValidationError):
|
||||
frozen.dials.depth = 99
|
||||
fresh = registry.get("rapid-answer")
|
||||
assert fresh.dials.depth == methodology.dials.depth
|
||||
assert fresh.dials.depth == 1
|
||||
|
||||
|
||||
def test_override_rejects_unknown_dial() -> None:
|
||||
with pytest.raises(UnknownOverrideError):
|
||||
resolve_dials("rapid-answer", {"nonexistent_dial": 1}, registry)
|
||||
|
||||
|
||||
def test_new_run_id_format() -> None:
|
||||
# P-14: one generator, one alphabet (config/ids.py) — the old
|
||||
# token_hex(4) duplicate is gone.
|
||||
run_id = new_run_id()
|
||||
assert len(run_id) == 8
|
||||
alphabet = set("abcdefghjkmnpqrstuvwxyz23456789")
|
||||
assert all(c in alphabet for c in run_id)
|
||||
|
||||
|
||||
async def test_fake_search_feeds_evidence_shape() -> None:
|
||||
search = FakeSearchAdapter(CORPUS)
|
||||
results = await search.search("methodology dials verification", max_results=5)
|
||||
assert len(results) >= 1
|
||||
for result in results:
|
||||
page = await search.fetch(result.url)
|
||||
assert page.text.strip()
|
||||
@@ -0,0 +1,336 @@
|
||||
"""Key-store adversarial route suite (T-2.8 — security-engineer hat).
|
||||
|
||||
Route-level adversarial coverage beyond the unit suite in
|
||||
test_key_store_settings.py: corrupt/tampered envelopes surfacing
|
||||
through the API (typed error, honest degraded tier, never a silent
|
||||
drop and never a 500), raw-key non-leak across the FULL response
|
||||
surface of a session that touches the key store, and the create_run
|
||||
409 honesty on a corrupt envelope (never a wrong-key run, never a
|
||||
silent tier fall).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.model_factory import ModelAdapterFactory
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.api.app import create_app
|
||||
from researchx.auth.guard import CSRF_HEADER
|
||||
from researchx.auth.key_store import KEK_NAME, KeyStore
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{"sub_questions": ["verification passes"], "source_universe": ["web"], "outline": ["intro"]}
|
||||
)
|
||||
_WRITER = "# Report\n\nClaim [p-1] and claim [p-2].\n"
|
||||
|
||||
_ADMIN_EMAIL = "admin@example.org"
|
||||
_ADMIN_PASSWORD = "correct-horse-battery"
|
||||
_SECOND_EMAIL = "second@example.org"
|
||||
_SECOND_PASSWORD = "second-long-password"
|
||||
|
||||
_HEADERS = {CSRF_HEADER: "1"}
|
||||
_KEK = base64.urlsafe_b64encode(os.urandom(32)).decode()
|
||||
_USER_KEY = "adversarial-user-key-4242"
|
||||
|
||||
|
||||
def _app(tmp_path: Path, secrets_values: dict[str, str]):
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text(
|
||||
"".join(f"{k}={v}\n" for k, v in secrets_values.items()), encoding="utf-8"
|
||||
)
|
||||
secrets = EnvFileSecretsProvider(paths=[env_file])
|
||||
return create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER, model="fake-writer"),
|
||||
]
|
||||
),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=True,
|
||||
data_dir=tmp_path,
|
||||
secrets_provider=secrets,
|
||||
model_factory=ModelAdapterFactory(
|
||||
secrets,
|
||||
KeyStore(secrets, tmp_path),
|
||||
cloud_base_url="http://127.0.0.1:1/v1", # hermetic: unroutable
|
||||
local_base_url="http://127.0.0.1:1/v1",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
async def _login_admin(client: httpx.AsyncClient) -> None:
|
||||
await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD, "display_name": "Admin"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
await client.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
|
||||
async def _store_key(client: httpx.AsyncClient, key: str) -> None:
|
||||
response = await client.put(
|
||||
"/api/config/settings/model-key", json={"key": key}, headers=_HEADERS
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.json()["configured"] is True
|
||||
|
||||
|
||||
def _corrupt_key_file(tmp_path: Path, user_id: str, content: str) -> None:
|
||||
path = tmp_path / "keys" / f"{user_id}.json"
|
||||
assert path.is_file(), "the key must be stored before corrupting it"
|
||||
path.write_text(content, encoding="utf-8")
|
||||
|
||||
|
||||
async def _current_user_id(client: httpx.AsyncClient) -> str:
|
||||
me = await client.get("/api/auth/me")
|
||||
return me.json()["user_id"]
|
||||
|
||||
|
||||
async def _await_terminal_or_approval(client: httpx.AsyncClient, run_id: str) -> str:
|
||||
for _ in range(300):
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status in ("done", "failed", "cancelled", "awaiting_approval"):
|
||||
return status
|
||||
await asyncio.sleep(0.01)
|
||||
return status
|
||||
|
||||
|
||||
# ------------------------------------------------- corrupt envelope at the API
|
||||
|
||||
|
||||
async def test_config_status_degrades_honestly_on_garbage_envelope(tmp_path: Path) -> None:
|
||||
"""Garbage keys/<uid>.json -> config status 200 with a DEGRADED
|
||||
tier (keyless/server) + the error surfaced in key_status.reason —
|
||||
typed error surfacing, never a silent tier drop, never a 500."""
|
||||
app = _app(tmp_path, {KEK_NAME: _KEK})
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _login_admin(client)
|
||||
await _store_key(client, _USER_KEY)
|
||||
user_id = await _current_user_id(client)
|
||||
_corrupt_key_file(tmp_path, user_id, "this is garbage, not json")
|
||||
|
||||
config = await client.get("/api/config")
|
||||
assert config.status_code == 200 # never a 500
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] in ("server", "keyless") # honest degradation
|
||||
assert body["key_store"] == "enabled"
|
||||
assert body["key_status"]["configured"] is False
|
||||
assert body["key_status"]["reason"] is not None
|
||||
assert "corrupt" in body["key_status"]["reason"].lower()
|
||||
|
||||
|
||||
async def test_config_status_degrades_honestly_on_tampered_envelope(tmp_path: Path) -> None:
|
||||
"""A tampered envelope -> same honest degradation + 'failed
|
||||
authentication' reason — never the decrypted key, never a silent
|
||||
'user' tier."""
|
||||
app = _app(tmp_path, {KEK_NAME: _KEK})
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _login_admin(client)
|
||||
await _store_key(client, _USER_KEY)
|
||||
user_id = await _current_user_id(client)
|
||||
original = json.loads((tmp_path / "keys" / f"{user_id}.json").read_text())
|
||||
# Flip ciphertext bytes: the envelope stays valid JSON but
|
||||
# decrypt must fail (InvalidTag -> typed 'tampered').
|
||||
ct = base64.urlsafe_b64decode(original["entries"]["OLLAMA_API_KEY"]["ct"] + "===")
|
||||
flipped = bytes([ct[0] ^ 0xFF]) + ct[1:]
|
||||
original["entries"]["OLLAMA_API_KEY"]["ct"] = (
|
||||
base64.urlsafe_b64encode(flipped).decode().rstrip("=")
|
||||
)
|
||||
_corrupt_key_file(tmp_path, user_id, json.dumps(original))
|
||||
|
||||
config = await client.get("/api/config")
|
||||
assert config.status_code == 200
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] in ("server", "keyless")
|
||||
assert "authentication" in body["key_status"]["reason"].lower()
|
||||
assert _USER_KEY not in config.text
|
||||
|
||||
|
||||
async def test_create_run_corrupt_envelope_is_honest_409(tmp_path: Path) -> None:
|
||||
"""create_run with a corrupt stored envelope -> 409 with the typed
|
||||
error — the run is NOT created (never a wrong-key run, never a
|
||||
silent tier fall)."""
|
||||
app = _app(tmp_path, {KEK_NAME: _KEK})
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _login_admin(client)
|
||||
await _store_key(client, _USER_KEY)
|
||||
user_id = await _current_user_id(client)
|
||||
_corrupt_key_file(tmp_path, user_id, "{corrupt")
|
||||
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}, headers=_HEADERS
|
||||
)
|
||||
assert created.status_code == 409
|
||||
detail = created.json()["detail"].lower()
|
||||
assert "corrupt" in detail or "key" in detail
|
||||
|
||||
|
||||
async def test_model_key_put_overwrites_after_corruption(tmp_path: Path) -> None:
|
||||
"""Recovery path: a user whose envelope corrupted can PUT a fresh
|
||||
key (atomic overwrite) and return to the 'user' tier."""
|
||||
app = _app(tmp_path, {KEK_NAME: _KEK})
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _login_admin(client)
|
||||
await _store_key(client, _USER_KEY)
|
||||
user_id = await _current_user_id(client)
|
||||
_corrupt_key_file(tmp_path, user_id, "garbage")
|
||||
config = await client.get("/api/config")
|
||||
assert config.json()["resolved_tier"] != "user"
|
||||
|
||||
await _store_key(client, "fresh-key-value-9999")
|
||||
config = await client.get("/api/config")
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] == "user"
|
||||
assert body["key_status"]["masked"].endswith("9999")
|
||||
|
||||
|
||||
# ------------------------------------------------------------- leak scan
|
||||
|
||||
|
||||
async def test_no_raw_key_in_full_session_response_surface(tmp_path: Path) -> None:
|
||||
"""Every response of a full session that stores, reads, and runs
|
||||
with a user key — scanned for the raw key material (REQ-73)."""
|
||||
app = _app(tmp_path, {KEK_NAME: _KEK})
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _login_admin(client)
|
||||
put = await client.put(
|
||||
"/api/config/settings/model-key",
|
||||
json={"key": _USER_KEY},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
responses: list[httpx.Response] = [put]
|
||||
for path in [
|
||||
"/api/config",
|
||||
"/api/config/settings",
|
||||
"/api/config/settings/model-key",
|
||||
"/api/auth/me",
|
||||
"/api/methodologies",
|
||||
"/api/admin/users",
|
||||
]:
|
||||
responses.append(await client.get(path))
|
||||
responses.append(
|
||||
await client.put(
|
||||
"/api/config/settings", json={"appearance": "dark"}, headers=_HEADERS
|
||||
)
|
||||
)
|
||||
|
||||
# A run executing under the user tier must not leak the key in
|
||||
# any run response either.
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}, headers=_HEADERS
|
||||
)
|
||||
assert created.status_code == 201
|
||||
responses.append(created)
|
||||
run_id = created.json()["id"]
|
||||
status = await _await_terminal_or_approval(client, run_id)
|
||||
responses.append(await client.get(f"/api/runs/{run_id}"))
|
||||
if status == "awaiting_approval":
|
||||
responses.append(
|
||||
await client.post(f"/api/runs/{run_id}/approve", headers=_HEADERS)
|
||||
)
|
||||
|
||||
for response in responses:
|
||||
assert _USER_KEY not in response.text, "raw model key leaked in a response"
|
||||
# And never on disk in plaintext.
|
||||
for path in (tmp_path / "keys").iterdir():
|
||||
assert _USER_KEY not in path.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
# ---------------------------------------------------- AAD bind via the API
|
||||
|
||||
|
||||
async def test_second_user_cannot_shadow_first_users_key(tmp_path: Path) -> None:
|
||||
"""Two users, each with their own key: the second user's tier is
|
||||
'user' via THEIR key, and the first user's key never affects the
|
||||
second user's resolution (per-user AAD bind end to end)."""
|
||||
app = _app(tmp_path, {KEK_NAME: _KEK})
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _login_admin(admin)
|
||||
await _store_key(admin, "first-user-key-aaaa")
|
||||
config = await admin.get("/api/config")
|
||||
assert config.json()["resolved_tier"] == "user"
|
||||
assert config.json()["key_status"]["masked"].endswith("aaaa")
|
||||
await admin.post(
|
||||
"/api/admin/users",
|
||||
json={
|
||||
"email": _SECOND_EMAIL,
|
||||
"password": _SECOND_PASSWORD,
|
||||
"role": "user",
|
||||
},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
await second.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
# Before storing a key: keyless (no server key configured).
|
||||
config = await second.get("/api/config")
|
||||
assert config.json()["resolved_tier"] == "keyless"
|
||||
assert "first-user-key" not in config.text
|
||||
|
||||
await _store_key(second, "second-user-key-bbbb")
|
||||
config = await second.get("/api/config")
|
||||
assert config.json()["resolved_tier"] == "user"
|
||||
assert config.json()["key_status"]["masked"].endswith("bbbb")
|
||||
assert "aaaa" not in config.text
|
||||
|
||||
|
||||
# ---------------------------------------------- settings store isolation
|
||||
|
||||
|
||||
async def test_settings_corruption_is_typed_not_crashing(tmp_path: Path) -> None:
|
||||
"""A corrupt settings file surfaces a typed SettingsStoreError —
|
||||
the store fails honestly rather than silently serving defaults
|
||||
over a corrupt payload."""
|
||||
from researchx.auth.settings import SettingsStore, SettingsStoreError
|
||||
|
||||
app = _app(tmp_path, {KEK_NAME: _KEK})
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _login_admin(client)
|
||||
user_id = await _current_user_id(client)
|
||||
settings_dir = tmp_path / "settings"
|
||||
settings_dir.mkdir(parents=True, exist_ok=True)
|
||||
(settings_dir / f"{user_id}.json").write_text("{{not json", encoding="utf-8")
|
||||
|
||||
with pytest.raises(SettingsStoreError):
|
||||
await SettingsStore(tmp_path).get(user_id)
|
||||
|
||||
|
||||
def test_key_store_error_is_adapter_error() -> None:
|
||||
"""The typed key-store error IS an AdapterError — the factory and
|
||||
routes can catch one family (C3-06 contract)."""
|
||||
from researchx.adapters.types import AdapterError
|
||||
from researchx.auth.key_store import KeyStoreError
|
||||
|
||||
assert issubclass(KeyStoreError, AdapterError)
|
||||
@@ -0,0 +1,474 @@
|
||||
"""Key store + settings store suite (T-2.2/T-2.3, REQ-72/73).
|
||||
|
||||
The adversarial core of the per-user key store: KEK decode rules
|
||||
(b64/hex/wrong-length/garbage), set/get round-trips, AAD binding
|
||||
(a copied envelope under another user FAILS — never a wrong-key leak),
|
||||
nonce uniqueness, masked-only status, disabled-honesty without a KEK,
|
||||
0600 file permissions, and the settings defaults/update round-trip.
|
||||
|
||||
The STEP-4 adversarial extension (corrupt-entry surfacing, concurrent
|
||||
set_key atomicity, opt-in keyed live tier) lives at the bottom of this
|
||||
file under the "T-2.8 adversarial extension" banner.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.auth.key_store import KEK_NAME, KEY_SCOPE, KeyStore, KeyStoreError, load_kek_bytes
|
||||
from researchx.auth.settings import SettingsStore, UserSettings
|
||||
|
||||
_KEK_RAW = os.urandom(32)
|
||||
_KEK_B64 = base64.urlsafe_b64encode(_KEK_RAW).decode()
|
||||
_KEK_HEX = _KEK_RAW.hex()
|
||||
_USER = "abcd2345"
|
||||
_OTHER_USER = "efgh6789"
|
||||
_RAW_KEY = "sk-live-key-abcd1234"
|
||||
|
||||
|
||||
def _secrets_with(values: dict[str, str], tmp_path: Path) -> EnvFileSecretsProvider:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("".join(f"{k}={v}\n" for k, v in values.items()), encoding="utf-8")
|
||||
return EnvFileSecretsProvider(paths=[env_file])
|
||||
|
||||
|
||||
def _enabled_store(tmp_path: Path) -> KeyStore:
|
||||
return KeyStore(_secrets_with({KEK_NAME: _KEK_B64}, tmp_path), tmp_path)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- KEK decode
|
||||
|
||||
|
||||
def test_kek_decode_b64(tmp_path: Path) -> None:
|
||||
secrets = _secrets_with({KEK_NAME: _KEK_B64}, tmp_path)
|
||||
assert load_kek_bytes(secrets) == _KEK_RAW
|
||||
|
||||
|
||||
def test_kek_decode_hex(tmp_path: Path) -> None:
|
||||
secrets = _secrets_with({KEK_NAME: _KEK_HEX}, tmp_path)
|
||||
assert load_kek_bytes(secrets) == _KEK_RAW
|
||||
|
||||
|
||||
def test_kek_wrong_length_raises(tmp_path: Path) -> None:
|
||||
short = base64.urlsafe_b64encode(b"only-16-bytes!!").decode()
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
load_kek_bytes(_secrets_with({KEK_NAME: short}, tmp_path))
|
||||
assert "32 bytes" in str(exc_info.value)
|
||||
assert exc_info.value.kind == "bad_kek"
|
||||
|
||||
|
||||
def test_kek_garbage_raises(tmp_path: Path) -> None:
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
load_kek_bytes(_secrets_with({KEK_NAME: "!!!not-base64!!!"}, tmp_path))
|
||||
assert exc_info.value.kind == "bad_kek"
|
||||
|
||||
|
||||
def test_kek_absent_is_none_not_error(tmp_path: Path) -> None:
|
||||
assert load_kek_bytes(_secrets_with({}, tmp_path)) is None
|
||||
|
||||
|
||||
def test_bad_kek_disables_store_honestly(tmp_path: Path) -> None:
|
||||
"""A malformed KEK is not a crash and not a silent enable."""
|
||||
store = KeyStore(_secrets_with({KEK_NAME: "garbage-value"}, tmp_path), tmp_path)
|
||||
assert store.enabled is False
|
||||
assert store.disabled_reason is not None
|
||||
assert "misconfigured" in store.disabled_reason
|
||||
|
||||
|
||||
# ------------------------------------------------------------- round trips
|
||||
|
||||
|
||||
async def test_set_get_round_trip(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
assert await store.get_key(_USER) == _RAW_KEY
|
||||
|
||||
|
||||
async def test_get_without_set_is_none(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
assert await store.get_key(_USER) is None
|
||||
|
||||
|
||||
async def test_set_overwrites_previous_key(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, "first-key-value")
|
||||
await store.set_key(_USER, KEY_SCOPE, "second-key-value")
|
||||
assert await store.get_key(_USER) == "second-key-value"
|
||||
|
||||
|
||||
async def test_envelope_shape_on_disk(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
payload = json.loads((tmp_path / "keys" / f"{_USER}.json").read_text(encoding="utf-8"))
|
||||
assert payload["v"] == 1
|
||||
assert payload["alg"] == "AES-256-GCM"
|
||||
assert "updated_at" in payload
|
||||
entry = payload["entries"][KEY_SCOPE]
|
||||
assert set(entry) == {"nonce", "ct", "updated_at"}
|
||||
# Raw key material must not appear anywhere in the file.
|
||||
assert _RAW_KEY not in json.dumps(payload)
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- AAD
|
||||
|
||||
|
||||
async def test_aad_tamper_copy_to_other_user_fails(tmp_path: Path) -> None:
|
||||
"""The AAD is user_id: a file copied under another user's id FAILS
|
||||
decrypt with a typed error — never returns the other user's key."""
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
|
||||
keys_dir = tmp_path / "keys"
|
||||
(keys_dir / f"{_OTHER_USER}.json").write_text(
|
||||
(keys_dir / f"{_USER}.json").read_text(encoding="utf-8"),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
await store.get_key(_OTHER_USER)
|
||||
assert exc_info.value.kind == "tampered"
|
||||
# Never the wrong user's key:
|
||||
assert exc_info.value.args[0] != _RAW_KEY
|
||||
|
||||
|
||||
async def test_aad_tamper_masked_status_also_fails(tmp_path: Path) -> None:
|
||||
"""Status must not silently report configured=False for a tampered
|
||||
envelope — that would be a silent tier drop."""
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
keys_dir = tmp_path / "keys"
|
||||
(keys_dir / f"{_OTHER_USER}.json").write_text(
|
||||
(keys_dir / f"{_USER}.json").read_text(encoding="utf-8"),
|
||||
encoding="utf-8",
|
||||
)
|
||||
with pytest.raises(KeyStoreError):
|
||||
await store.masked_status(_OTHER_USER)
|
||||
|
||||
|
||||
async def test_kek_change_surfaces_as_tamper(tmp_path: Path) -> None:
|
||||
"""A re-provisioned KEK cannot read old envelopes — typed error,
|
||||
never silent None (documented loss posture, G-105)."""
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
|
||||
new_kek = base64.urlsafe_b64encode(os.urandom(32)).decode()
|
||||
rotated = KeyStore(_secrets_with({KEK_NAME: new_kek}, tmp_path), tmp_path)
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
await rotated.get_key(_USER)
|
||||
assert exc_info.value.kind == "tampered"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ nonces
|
||||
|
||||
|
||||
async def test_nonce_unique_across_writes(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
nonces = []
|
||||
for i in range(3):
|
||||
await store.set_key(_USER, KEY_SCOPE, f"key-number-{i}")
|
||||
payload = json.loads((tmp_path / "keys" / f"{_USER}.json").read_text(encoding="utf-8"))
|
||||
nonces.append(payload["entries"][KEY_SCOPE]["nonce"])
|
||||
assert len(set(nonces)) == 3 # a GCM nonce must NEVER repeat under a KEK
|
||||
|
||||
|
||||
async def test_nonce_is_12_bytes(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
payload = json.loads((tmp_path / "keys" / f"{_USER}.json").read_text(encoding="utf-8"))
|
||||
nonce_b64 = payload["entries"][KEY_SCOPE]["nonce"]
|
||||
assert len(base64.urlsafe_b64decode(nonce_b64 + "===")) == 12
|
||||
|
||||
|
||||
# ------------------------------------------------------------ masked status
|
||||
|
||||
|
||||
async def test_masked_status_never_returns_raw_key(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
status = await store.masked_status(_USER)
|
||||
assert status["configured"] is True
|
||||
assert status["masked"].endswith(_RAW_KEY[-4:])
|
||||
assert _RAW_KEY not in json.dumps(status)
|
||||
assert status["masked"].startswith("…"[:1]) or status["masked"][0] in ("…", ".")
|
||||
|
||||
|
||||
async def test_masked_status_unconfigured(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
status = await store.masked_status(_USER)
|
||||
assert status["configured"] is False
|
||||
assert status["masked"] is None
|
||||
|
||||
|
||||
# -------------------------------------------------------- disabled honesty
|
||||
|
||||
|
||||
async def test_disabled_store_set_rejected_honestly(tmp_path: Path) -> None:
|
||||
store = KeyStore(_secrets_with({}, tmp_path), tmp_path) # no KEK
|
||||
assert store.enabled is False
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
assert exc_info.value.kind == "disabled"
|
||||
assert "RESEARCHX_KEK" in str(exc_info.value)
|
||||
# And nothing was written to disk.
|
||||
assert not (tmp_path / "keys" / f"{_USER}.json").exists()
|
||||
|
||||
|
||||
async def test_disabled_store_get_returns_none(tmp_path: Path) -> None:
|
||||
store = KeyStore(_secrets_with({}, tmp_path), tmp_path)
|
||||
assert await store.get_key(_USER) is None
|
||||
|
||||
|
||||
async def test_disabled_store_status_says_disabled(tmp_path: Path) -> None:
|
||||
store = KeyStore(_secrets_with({}, tmp_path), tmp_path)
|
||||
status = await store.masked_status(_USER)
|
||||
assert status["disabled"] is True
|
||||
assert "RESEARCHX_KEK" in str(status["reason"])
|
||||
assert status["configured"] is False
|
||||
|
||||
|
||||
async def test_disabled_store_with_stored_key_surfaces_error(tmp_path: Path) -> None:
|
||||
"""The loss posture made visible: a KEK vanished after keys were
|
||||
stored — reads must surface an error state, never silently None
|
||||
(which would look like 'no key configured')."""
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
|
||||
gone = KeyStore(_secrets_with({}, tmp_path), tmp_path)
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
await gone.get_key(_USER)
|
||||
assert exc_info.value.kind == "disabled"
|
||||
with pytest.raises(KeyStoreError):
|
||||
await gone.masked_status(_USER)
|
||||
|
||||
|
||||
async def test_unsupported_scope_rejected(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
with pytest.raises(KeyStoreError):
|
||||
await store.set_key(_USER, "TAVILY_API_KEY", "should-not-work")
|
||||
|
||||
|
||||
# ------------------------------------------------------------ file perms
|
||||
|
||||
|
||||
async def test_key_file_perms_0600(tmp_path: Path) -> None:
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
mode = (tmp_path / "keys" / f"{_USER}.json").stat().st_mode & 0o777
|
||||
assert mode == 0o600
|
||||
|
||||
|
||||
def test_key_store_docstring_documents_loss_posture() -> None:
|
||||
"""G-105: the loss posture must be stated where the keys live."""
|
||||
import researchx.auth.key_store as key_store_module
|
||||
|
||||
doc = key_store_module.__doc__ or ""
|
||||
assert "KEK-loss" in doc or "unrecoverable" in doc
|
||||
assert "back up the KEK" in doc
|
||||
|
||||
|
||||
# ------------------------------------------------------------- settings
|
||||
|
||||
|
||||
async def test_settings_defaults_for_fresh_user(tmp_path: Path) -> None:
|
||||
store = SettingsStore(tmp_path)
|
||||
settings = await store.get(_USER)
|
||||
assert settings.planner_model
|
||||
assert settings.writer_model
|
||||
assert settings.methodology_defaults == {}
|
||||
assert settings.appearance == "system"
|
||||
# GET never writes:
|
||||
assert not (tmp_path / "settings" / f"{_USER}.json").exists()
|
||||
|
||||
|
||||
async def test_settings_update_round_trip(tmp_path: Path) -> None:
|
||||
store = SettingsStore(tmp_path)
|
||||
updated = await store.update(_USER, UserSettings(planner_model="m1", writer_model="m2"))
|
||||
assert updated.planner_model == "m1"
|
||||
assert updated.writer_model == "m2"
|
||||
assert updated.appearance == "system" # untouched field keeps default
|
||||
|
||||
reloaded = await store.get(_USER)
|
||||
assert reloaded.planner_model == "m1"
|
||||
assert reloaded.methodology_defaults == {}
|
||||
|
||||
|
||||
async def test_settings_partial_update_preserves_other_fields(tmp_path: Path) -> None:
|
||||
store = SettingsStore(tmp_path)
|
||||
await store.update(_USER, UserSettings(planner_model="keep-planner", appearance="dark"))
|
||||
updated = await store.update(_USER, UserSettings(appearance="light"))
|
||||
assert updated.planner_model == "keep-planner" # survived the partial write
|
||||
assert updated.appearance == "light"
|
||||
|
||||
|
||||
async def test_settings_methodology_defaults_round_trip(tmp_path: Path) -> None:
|
||||
store = SettingsStore(tmp_path)
|
||||
payload = UserSettings(methodology_defaults={"rapid-answer": {"depth": 2}})
|
||||
await store.update(_USER, payload)
|
||||
reloaded = await store.get(_USER)
|
||||
assert reloaded.methodology_defaults == {"rapid-answer": {"depth": 2}}
|
||||
|
||||
|
||||
async def test_settings_update_validates_merged_payload_before_write(tmp_path: Path) -> None:
|
||||
"""P2-verify P0 regression: the store never writes a payload it
|
||||
cannot read back — a null field that slipped past model_copy (which
|
||||
skips validation) is refused before touching disk."""
|
||||
store = SettingsStore(tmp_path)
|
||||
await store.update(_USER, UserSettings(planner_model="keep-me"))
|
||||
poisoned = await store.get(_USER)
|
||||
poisoned = poisoned.model_copy(update={"planner_model": None})
|
||||
with pytest.raises(ValueError):
|
||||
await store.update(_USER, poisoned)
|
||||
reloaded = await store.get(_USER)
|
||||
assert reloaded.planner_model == "keep-me" # on-disk payload untouched
|
||||
|
||||
|
||||
async def test_settings_isolated_per_user(tmp_path: Path) -> None:
|
||||
store = SettingsStore(tmp_path)
|
||||
await store.update(_USER, UserSettings(appearance="dark"))
|
||||
other = await store.get(_OTHER_USER)
|
||||
assert other.appearance == "system" # fresh defaults, no bleed
|
||||
assert not (tmp_path / "settings" / f"{_OTHER_USER}.json").exists()
|
||||
|
||||
|
||||
async def test_settings_corrupt_payload_surfaces_typed_error(tmp_path: Path) -> None:
|
||||
store = SettingsStore(tmp_path)
|
||||
settings_dir = tmp_path / "settings"
|
||||
settings_dir.mkdir(parents=True, exist_ok=True)
|
||||
(settings_dir / f"{_USER}.json").write_text("{not json", encoding="utf-8")
|
||||
from researchx.auth.settings import SettingsStoreError
|
||||
|
||||
with pytest.raises((SettingsStoreError, ValueError)):
|
||||
await store.get(_USER)
|
||||
|
||||
|
||||
# ================================================== T-2.8 adversarial extension
|
||||
# (security-engineer hat — corrupt-entry surfacing, concurrent-set
|
||||
# atomicity, and the opt-in keyed live tier. See STEP 4.)
|
||||
|
||||
|
||||
async def test_corrupt_envelope_surfaces_typed_error_never_silent_drop(tmp_path: Path) -> None:
|
||||
"""Garbage in keys/<uid>.json → KeyStoreError('unreadable'), never
|
||||
a silent None (which would silently drop the user's tier)."""
|
||||
store = _enabled_store(tmp_path)
|
||||
keys_dir = tmp_path / "keys"
|
||||
keys_dir.mkdir(parents=True, exist_ok=True)
|
||||
(keys_dir / f"{_USER}.json").write_text("this is not json at all", encoding="utf-8")
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
await store.get_key(_USER)
|
||||
assert exc_info.value.kind == "unreadable"
|
||||
|
||||
|
||||
async def test_corrupt_entry_fields_surface_typed_error(tmp_path: Path) -> None:
|
||||
"""Structurally-valid JSON with missing/invalid fields — typed error."""
|
||||
store = _enabled_store(tmp_path)
|
||||
keys_dir = tmp_path / "keys"
|
||||
keys_dir.mkdir(parents=True, exist_ok=True)
|
||||
(keys_dir / f"{_USER}.json").write_text(
|
||||
json.dumps({"v": 1, "alg": "AES-256-GCM", "entries": {"OLLAMA_API_KEY": {"nonce": "xx"}}}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
with pytest.raises(KeyStoreError):
|
||||
await store.get_key(_USER)
|
||||
|
||||
|
||||
async def test_bit_rot_ciphertext_surfaces_tamper(tmp_path: Path) -> None:
|
||||
"""Flipping ciphertext bytes → InvalidTag → typed 'tampered'."""
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, _RAW_KEY)
|
||||
path = tmp_path / "keys" / f"{_USER}.json"
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
ct = base64.urlsafe_b64decode(payload["entries"][KEY_SCOPE]["ct"] + "===")
|
||||
flipped = bytes([ct[0] ^ 0xFF]) + ct[1:]
|
||||
payload["entries"][KEY_SCOPE]["ct"] = base64.urlsafe_b64encode(flipped).decode().rstrip("=")
|
||||
path.write_text(json.dumps(payload), encoding="utf-8")
|
||||
with pytest.raises(KeyStoreError) as exc_info:
|
||||
await store.get_key(_USER)
|
||||
assert exc_info.value.kind == "tampered"
|
||||
|
||||
|
||||
async def test_concurrent_set_key_never_tears_the_file(tmp_path: Path) -> None:
|
||||
"""Concurrent writes to the same user: every reader (including a
|
||||
second KeyStore handle over the same root) sees a valid envelope —
|
||||
the tmp+replace idiom leaves no torn files."""
|
||||
store = _enabled_store(tmp_path)
|
||||
await asyncio.gather(
|
||||
*(store.set_key(_USER, KEY_SCOPE, f"concurrent-key-{i}") for i in range(16))
|
||||
)
|
||||
# The final file is one of the written keys, decryptable, well-formed.
|
||||
final = await store.get_key(_USER)
|
||||
assert final is not None
|
||||
assert final.startswith("concurrent-key-")
|
||||
|
||||
# A second handle (fresh process view) reads the same valid file.
|
||||
second = KeyStore(_secrets_with({KEK_NAME: _KEK_B64}, tmp_path), tmp_path)
|
||||
again = await second.get_key(_USER)
|
||||
assert again == final
|
||||
payload = json.loads((tmp_path / "keys" / f"{_USER}.json").read_text(encoding="utf-8"))
|
||||
assert payload["v"] == 1 # full envelope, never a partial write
|
||||
# No tmp strays left behind.
|
||||
assert [p.name for p in (tmp_path / "keys").iterdir() if p.name.endswith(".tmp")] == []
|
||||
|
||||
|
||||
async def test_set_key_atomic_under_concurrent_reads(tmp_path: Path) -> None:
|
||||
"""Reads racing a write never observe a torn file."""
|
||||
store = _enabled_store(tmp_path)
|
||||
await store.set_key(_USER, KEY_SCOPE, "initial-key")
|
||||
|
||||
async def _reader() -> str | None:
|
||||
for _ in range(20):
|
||||
try:
|
||||
key = await store.get_key(_USER)
|
||||
if key is None:
|
||||
raise AssertionError("envelope vanished mid-race")
|
||||
except KeyStoreError as exc:
|
||||
raise AssertionError(f"torn read surfaced: {exc}") from exc
|
||||
return None
|
||||
|
||||
async def _writer() -> None:
|
||||
for i in range(10):
|
||||
await store.set_key(_USER, KEY_SCOPE, f"racing-key-{i}")
|
||||
|
||||
await asyncio.gather(_reader(), _writer())
|
||||
assert (await store.get_key(_USER)).startswith("racing-key-")
|
||||
|
||||
|
||||
def test_keyed_live_tier_marker_registered() -> None:
|
||||
"""The opt-in keyed live tier is marked `live` and skipped by
|
||||
default (same pattern as test_live_smoke.py): the marker must be
|
||||
registered in the pytest config so the opt-in test below skips
|
||||
cleanly in zero-key CI."""
|
||||
import tomllib
|
||||
|
||||
config = tomllib.loads((Path(__file__).resolve().parents[1] / "pyproject.toml").read_text())
|
||||
markers = config["tool"]["pytest"]["ini_options"]["markers"]
|
||||
assert any(str(m).startswith("live:") for m in markers)
|
||||
|
||||
|
||||
# ------------------------------------------------------- opt-in live tier
|
||||
|
||||
|
||||
@pytest.mark.live
|
||||
async def test_live_keyed_tier_end_to_end(tmp_path: Path) -> None:
|
||||
"""Opt-in keyed tier (skipped by default, like test_live_smoke):
|
||||
with a real RESEARCHX_KEK + OLLAMA_API_KEY in the secrets scope, a
|
||||
keyed user gets real inference through the REQ-74 chain."""
|
||||
secrets_paths = [
|
||||
Path.cwd() / ".ciagent" / ".env.secrets",
|
||||
Path.cwd().parents[1] / ".ciagent" / ".env.secrets",
|
||||
]
|
||||
env = EnvFileSecretsProvider(paths=secrets_paths)
|
||||
kek = env.resolve(KEK_NAME)
|
||||
model_key = env.resolve("OLLAMA_API_KEY")
|
||||
if not kek or not model_key:
|
||||
pytest.skip("RESEARCHX_KEK/OLLAMA_API_KEY not present — live tier is opt-in")
|
||||
|
||||
store = KeyStore(env, tmp_path)
|
||||
assert store.enabled, "with a configured KEK the store must be enabled"
|
||||
await store.set_key("live0001", KEY_SCOPE, model_key)
|
||||
resolved = await store.get_key("live0001")
|
||||
assert resolved == model_key
|
||||
@@ -0,0 +1,687 @@
|
||||
"""Live-surface route tests (T-3.3, REQ-76..82).
|
||||
|
||||
Every new/reworked route over a fixture disk tree: report/ledger
|
||||
disk-read with pre-artifact DONE fallback, library/connections/graph/
|
||||
tasks/suggestions, refine (200 on fakes, honest 503 on adapter
|
||||
failure), profile round-trip, and the FRESH-APP-OVER-EXISTING-DATA-
|
||||
ROOT test (P-8 full closure — every read route correct against a
|
||||
second create_app with zero in-memory actors).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.types import AdapterError, ChatResult, ModelMessage
|
||||
from researchx.api.app import create_app
|
||||
from researchx.auth.guard import CSRF_HEADER
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{"sub_questions": ["verification passes"], "source_universe": ["web"], "outline": ["intro"]}
|
||||
)
|
||||
_WRITER = "# Research Memo\n\nClaim [p-1] and claim [p-2].\n"
|
||||
|
||||
_HEADERS = {CSRF_HEADER: "1"}
|
||||
|
||||
_DEV = "dev00000" # the synthetic dev admin (auth_enabled=False harness)
|
||||
|
||||
|
||||
def _state(run_id: str, question: str, state: str, created_at: str, plan: dict | None = None) -> dict:
|
||||
return {
|
||||
"run_id": run_id,
|
||||
"question": question,
|
||||
"context": "",
|
||||
"methodology_slug": "deep-report",
|
||||
"state": state,
|
||||
"owner_id": _DEV,
|
||||
"plan": plan if plan is not None else {
|
||||
"sub_questions": ["battery density question"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro"],
|
||||
},
|
||||
"depth_cursor": 0,
|
||||
"depth_target": 1,
|
||||
"counters": {},
|
||||
"probe": None,
|
||||
"created_at": created_at,
|
||||
"updated_at": created_at,
|
||||
}
|
||||
|
||||
|
||||
def _seed_run(
|
||||
tmp_path: Path,
|
||||
run_id: str,
|
||||
*,
|
||||
state: str,
|
||||
question: str = "What about battery research?",
|
||||
created_at: str = "2026-09-20T10:00:00Z",
|
||||
plan: dict | None = None,
|
||||
report_markdown: str | None = None,
|
||||
report_json: dict | None = None,
|
||||
evidence_json: dict | None = None,
|
||||
ledger_json: dict | None = None,
|
||||
owner: str = _DEV,
|
||||
) -> None:
|
||||
run_dir = tmp_path / "runs" / owner / run_id
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
(run_dir / "state.json").write_text(
|
||||
json.dumps(_state(run_id, question, state, created_at, plan)), encoding="utf-8"
|
||||
)
|
||||
if report_markdown is not None:
|
||||
(run_dir / "report.md").write_text(report_markdown, encoding="utf-8")
|
||||
if report_json is not None:
|
||||
(run_dir / "report.json").write_text(json.dumps(report_json), encoding="utf-8")
|
||||
if evidence_json is not None:
|
||||
(run_dir / "evidence.json").write_text(json.dumps(evidence_json), encoding="utf-8")
|
||||
if ledger_json is not None:
|
||||
(run_dir / "ledger.json").write_text(json.dumps(ledger_json), encoding="utf-8")
|
||||
|
||||
|
||||
_REPORT_JSON = {
|
||||
"v": 1,
|
||||
"run_id": "runaaaa1",
|
||||
"owner_id": _DEV,
|
||||
"title": "Battery Advances",
|
||||
"question": "What about battery research?",
|
||||
"methodology_slug": "deep-report",
|
||||
"generated_at": "2026-09-20T10:05:00Z",
|
||||
"sections": [
|
||||
{"heading": "Research Memo", "body": "The verified core finding [^p1]."}
|
||||
],
|
||||
"citations": [
|
||||
{
|
||||
"passage_id": "p1",
|
||||
"url": "https://example.org/x",
|
||||
"title": "Verification passes",
|
||||
"quote": "The verified core finding.",
|
||||
"health": "live",
|
||||
"authority_tier": "reference",
|
||||
}
|
||||
],
|
||||
}
|
||||
_EVIDENCE_JSON = {
|
||||
"v": 1,
|
||||
"run_id": "runaaaa1",
|
||||
"passages": [
|
||||
{
|
||||
"passage_id": "p1",
|
||||
"url": "https://example.org/x",
|
||||
"title": "Verification passes",
|
||||
"quote": "The verified core finding.",
|
||||
"text": "The verified core finding in full, stored at DONE.",
|
||||
}
|
||||
],
|
||||
}
|
||||
_LEDGER_JSON = {
|
||||
"run_id": "runaaaa1",
|
||||
"generated_at": "2026-09-20T10:05:00Z",
|
||||
"entries": [
|
||||
{
|
||||
"url": "https://example.org/x",
|
||||
"visited_at": "2026-09-20T10:04:00Z",
|
||||
"passages": ["p-1"],
|
||||
"claims_supported": 2,
|
||||
"health": "live",
|
||||
"authority_tier": "reference",
|
||||
}
|
||||
],
|
||||
"checklist": {"run_id": "runaaaa1", "citations_checked": 1},
|
||||
}
|
||||
|
||||
|
||||
def _seed_tree(tmp_path: Path) -> Path:
|
||||
"""The route fixture tree under the dev owner:
|
||||
- runaaaa1: DONE with report.md + report.json + evidence.json + ledger
|
||||
- runbbbb1: DONE pre-artifact (report.md only — no report.json)
|
||||
- runcccc1: non-terminal (SEARCHING) — swept to CANCELLED by create_app
|
||||
"""
|
||||
_seed_run(
|
||||
tmp_path,
|
||||
"runaaaa1",
|
||||
state="DONE",
|
||||
report_markdown="# Research Memo\n\nThe verified core finding [p-1].\n",
|
||||
report_json=_REPORT_JSON,
|
||||
evidence_json=_EVIDENCE_JSON,
|
||||
ledger_json=_LEDGER_JSON,
|
||||
)
|
||||
_seed_run(
|
||||
tmp_path,
|
||||
"runbbbb1",
|
||||
state="DONE",
|
||||
question="A pre-artifact done run?",
|
||||
created_at="2026-09-19T10:00:00Z",
|
||||
report_markdown="# Old Report\n\nRaw [p-1] markers only.\n",
|
||||
)
|
||||
_seed_run(
|
||||
tmp_path,
|
||||
"runcccc1",
|
||||
state="SEARCHING",
|
||||
question="Swept mid-flight question?",
|
||||
created_at="2026-09-21T10:00:00Z",
|
||||
)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _fake_app(tmp_path: Path, canned: list[ChatResult] | None = None) -> object:
|
||||
return create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(canned=canned or [ChatResult(content=_PLAN, model="p")]),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
|
||||
|
||||
async def _client(app: object) -> httpx.AsyncClient:
|
||||
return httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), # type: ignore[arg-type]
|
||||
base_url="http://testserver",
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------- report route
|
||||
|
||||
|
||||
async def test_report_route_disk_read_returns_full_payload(tmp_path: Path) -> None:
|
||||
_seed_tree(tmp_path)
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
response = await client.get("/api/runs/runaaaa1/report")
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert body["run_id"] == "runaaaa1"
|
||||
assert "[p-1]" in body["markdown"] # raw form retained (back-compat)
|
||||
assert body["report"] == _REPORT_JSON
|
||||
assert body["evidence"] == _EVIDENCE_JSON
|
||||
|
||||
|
||||
async def test_report_route_pre_artifact_done_fallback(tmp_path: Path) -> None:
|
||||
"""A DONE run from before T-3.1 (report.md only): markdown served,
|
||||
null report/evidence — never a 500 (D-106)."""
|
||||
_seed_tree(tmp_path)
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
response = await client.get("/api/runs/runbbbb1/report")
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert body["run_id"] == "runbbbb1"
|
||||
assert "Raw [p-1] markers" in body["markdown"]
|
||||
assert body["report"] is None
|
||||
assert body["evidence"] is None
|
||||
|
||||
|
||||
async def test_report_route_missing_run_404(tmp_path: Path) -> None:
|
||||
_seed_tree(tmp_path)
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
assert (await client.get("/api/runs/zzzz9999/report")).status_code == 404
|
||||
|
||||
|
||||
async def test_ledger_route_disk_read(tmp_path: Path) -> None:
|
||||
_seed_tree(tmp_path)
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
response = await client.get("/api/runs/runaaaa1/ledger")
|
||||
assert response.status_code == 200
|
||||
assert response.json() == _LEDGER_JSON
|
||||
assert (await client.get("/api/runs/zzzz9999/ledger")).status_code == 404
|
||||
|
||||
|
||||
# ------------------------------------------------------------ run list route
|
||||
|
||||
|
||||
async def test_runs_route_lists_from_disk_recent_first(tmp_path: Path) -> None:
|
||||
"""GET /api/runs from disk state files; the SEARCHING fixture is
|
||||
swept to CANCELLED at create_app — it surfaces cancelled, NOT open
|
||||
(post-sweep honesty)."""
|
||||
_seed_tree(tmp_path)
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
response = await client.get("/api/runs")
|
||||
assert response.status_code == 200
|
||||
runs = response.json()
|
||||
assert [run["id"] for run in runs] == ["runcccc1", "runaaaa1", "runbbbb1"]
|
||||
statuses = {run["id"]: run["status"] for run in runs}
|
||||
assert statuses["runcccc1"] == "cancelled" # swept, honest
|
||||
assert statuses["runaaaa1"] == "done"
|
||||
assert statuses["runbbbb1"] == "done"
|
||||
first = runs[1]
|
||||
assert first["question"] == "What about battery research?"
|
||||
assert first["methodology"] == "deep-report"
|
||||
assert first["plan"]["sub_questions"] == ["battery density question"]
|
||||
|
||||
|
||||
# ----------------------------------------------------------- library routes
|
||||
|
||||
|
||||
async def test_library_route(tmp_path: Path) -> None:
|
||||
_seed_tree(tmp_path)
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
response = await client.get("/api/library")
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert [folder["id"] for folder in body["folders"]] == ["deep-report-battery"]
|
||||
assert [report["runId"] for report in body["reports"]] == ["runaaaa1", "runbbbb1"]
|
||||
first = body["reports"][0]
|
||||
assert first["title"] == "Battery Advances"
|
||||
assert first["citationCount"] == 1
|
||||
assert first["category"] == "deep-report-battery"
|
||||
# title sort: "Battery Advances" < "Deep Report — A pre-artifact..."
|
||||
titled = await client.get("/api/library?sort=title")
|
||||
assert [r["runId"] for r in titled.json()["reports"]] == ["runaaaa1", "runbbbb1"]
|
||||
|
||||
|
||||
async def test_connections_graph_tasks_suggestions_routes(tmp_path: Path) -> None:
|
||||
_seed_tree(tmp_path)
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
connections = await client.get("/api/library/connections")
|
||||
assert connections.status_code == 200
|
||||
# runaaaa1 has a ledger; runbbbb1 has none -> no shared URL, and
|
||||
# same methodology+category -> related-topic edge.
|
||||
assert connections.json() == [
|
||||
{
|
||||
"fromReportId": "rep-runbbbb1",
|
||||
"toReportId": "rep-runaaaa1",
|
||||
"kind": "related-topic",
|
||||
"label": "Adjacent topic",
|
||||
}
|
||||
]
|
||||
|
||||
graph = await client.get("/api/graph")
|
||||
assert graph.status_code == 200
|
||||
payload = graph.json()
|
||||
assert {node["id"] for node in payload["nodes"]} == {"runaaaa1", "runbbbb1"}
|
||||
node = next(n for n in payload["nodes"] if n["id"] == "runaaaa1")
|
||||
assert node["topic"] == "Battery Advances"
|
||||
assert node["linkedSources"] == ["https://example.org/x"]
|
||||
assert payload["edges"] == [{"fromId": "runbbbb1", "toId": "runaaaa1", "sharedSourceCount": 0}]
|
||||
|
||||
tasks = await client.get("/api/tasks")
|
||||
assert tasks.status_code == 200
|
||||
items = {item["id"]: item for item in tasks.json()}
|
||||
assert items["runaaaa1#0"]["status"] == "done"
|
||||
assert items["runbbbb1#0"]["status"] == "done"
|
||||
# The swept CANCELLED run's question stays OPEN (D-114).
|
||||
assert items["runcccc1#0"]["status"] == "open"
|
||||
assert items["runcccc1#0"]["question"] == "battery density question"
|
||||
|
||||
suggestions = await client.get("/api/suggestions")
|
||||
assert suggestions.status_code == 200
|
||||
assert suggestions.json() == [
|
||||
{
|
||||
"id": "sug-runcccc1#0",
|
||||
"question": "battery density question",
|
||||
"sourceRunId": "runcccc1",
|
||||
"sourceReportTitle": "Deep Report — Swept mid-flight question?",
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
async def test_fresh_user_empty_states(tmp_path: Path) -> None:
|
||||
"""Honest empty states: no synthetic content anywhere."""
|
||||
async with await _client(_fake_app(tmp_path)) as client:
|
||||
assert (await client.get("/api/library")).json() == {"folders": [], "reports": []}
|
||||
assert (await client.get("/api/library/connections")).json() == []
|
||||
assert (await client.get("/api/graph")).json() == {"nodes": [], "edges": []}
|
||||
assert (await client.get("/api/tasks")).json() == []
|
||||
assert (await client.get("/api/suggestions")).json() == []
|
||||
assert (await client.get("/api/runs")).json() == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- refine
|
||||
|
||||
|
||||
class _RefineModel:
|
||||
"""Fake model for the refine exchange: answers the questions prompt,
|
||||
then the compose prompt; a second instance answers with an
|
||||
AdapterError to pin the honest 503."""
|
||||
|
||||
def __init__(self, fail: bool = False) -> None:
|
||||
self._fail = fail
|
||||
self._calls = 0
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ModelMessage],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.2,
|
||||
json_mode: bool = False,
|
||||
) -> ChatResult:
|
||||
del messages, temperature, model
|
||||
if self._fail:
|
||||
raise AdapterError("no local model reachable")
|
||||
self._calls += 1
|
||||
if self._calls == 1:
|
||||
return ChatResult(
|
||||
content=json.dumps(
|
||||
{
|
||||
"questions": [
|
||||
"What scope matters most?",
|
||||
"What decision will this inform?",
|
||||
"How deep should the answer go?",
|
||||
]
|
||||
}
|
||||
),
|
||||
model="fake-refine",
|
||||
)
|
||||
return ChatResult(
|
||||
content=json.dumps(
|
||||
{
|
||||
"question": "How do batteries work — focusing on density?",
|
||||
"context": "Feeds a supplier decision. Depth: executive summary.",
|
||||
}
|
||||
),
|
||||
model="fake-refine",
|
||||
)
|
||||
|
||||
async def list_models(self) -> list[str]:
|
||||
return ["fake-refine"]
|
||||
|
||||
def roles(self) -> object:
|
||||
from researchx.adapters.model import ModelRoles
|
||||
|
||||
return ModelRoles(planner="fake-refine", writer="fake-refine")
|
||||
|
||||
|
||||
async def test_refine_questions_and_compose(tmp_path: Path) -> None:
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=_RefineModel(),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
async with await _client(app) as client:
|
||||
asked = await client.post("/api/refine/questions", json={"draft": "how do batteries work?"})
|
||||
assert asked.status_code == 200
|
||||
assert asked.json() == {
|
||||
"questions": [
|
||||
"What scope matters most?",
|
||||
"What decision will this inform?",
|
||||
"How deep should the answer go?",
|
||||
]
|
||||
}
|
||||
composed = await client.post(
|
||||
"/api/refine/compose",
|
||||
json={"draft": "how do batteries work?", "answers": ["density", "supplier choice"]},
|
||||
)
|
||||
assert composed.status_code == 200
|
||||
assert composed.json() == {
|
||||
"question": "How do batteries work — focusing on density?",
|
||||
"context": "Feeds a supplier decision. Depth: executive summary.",
|
||||
}
|
||||
|
||||
|
||||
async def test_refine_honest_503_never_hang(tmp_path: Path) -> None:
|
||||
"""A keyless user with no reachable endpoint gets an honest 503 —
|
||||
never a hang, never a silent success."""
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=_RefineModel(fail=True),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
async with await _client(app) as client:
|
||||
asked = await asyncio.wait_for(
|
||||
client.post("/api/refine/questions", json={"draft": "q"}), timeout=5.0
|
||||
)
|
||||
assert asked.status_code == 503
|
||||
composed = await asyncio.wait_for(
|
||||
client.post(
|
||||
"/api/refine/compose", json={"draft": "q", "answers": ["a"]}, headers=_HEADERS
|
||||
),
|
||||
timeout=5.0,
|
||||
)
|
||||
assert composed.status_code == 503
|
||||
|
||||
|
||||
async def test_refine_requires_csrf_header_when_authed(tmp_path: Path) -> None:
|
||||
"""Refine POSTs are mutations: the CSRF header applies (T-3.6)."""
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=_RefineModel(),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=True,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:
|
||||
# register + login (both with the header)
|
||||
await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": "a@example.org", "password": "long-enough-pass", "display_name": "A"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
await client.post(
|
||||
"/api/auth/login", json={"email": "a@example.org", "password": "long-enough-pass"}, headers=_HEADERS
|
||||
)
|
||||
no_header = await client.post("/api/refine/questions", json={"draft": "q"})
|
||||
assert no_header.status_code == 403
|
||||
with_header = await client.post("/api/refine/questions", json={"draft": "q"}, headers=_HEADERS)
|
||||
assert with_header.status_code == 200
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- profile
|
||||
|
||||
|
||||
async def test_profile_round_trip(tmp_path: Path) -> None:
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=_RefineModel(),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=True,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as client:
|
||||
await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": "first@example.org", "password": "long-enough-pass", "display_name": "First"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
await client.post(
|
||||
"/api/auth/login",
|
||||
json={"email": "first@example.org", "password": "long-enough-pass"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
got = await client.get("/api/profile")
|
||||
assert got.status_code == 200
|
||||
assert got.json() == {
|
||||
"user_id": got.json()["user_id"],
|
||||
"display_name": "First",
|
||||
"email": "first@example.org",
|
||||
}
|
||||
put = await client.put(
|
||||
"/api/profile", json={"display_name": "Renamed", "email": "new@example.org"}, headers=_HEADERS
|
||||
)
|
||||
assert put.status_code == 200
|
||||
assert put.json()["display_name"] == "Renamed"
|
||||
assert put.json()["email"] == "new@example.org"
|
||||
again = await client.get("/api/profile")
|
||||
assert again.json() == put.json()
|
||||
|
||||
|
||||
async def test_profile_email_unique_across_users(tmp_path: Path) -> None:
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=_RefineModel(),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=True,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await admin.post(
|
||||
"/api/auth/register",
|
||||
json={"email": "admin@example.org", "password": "long-enough-pass", "display_name": "A"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
await admin.post(
|
||||
"/api/auth/login", json={"email": "admin@example.org", "password": "long-enough-pass"}, headers=_HEADERS
|
||||
)
|
||||
created = await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": "second@example.org", "password": "long-enough-pass", "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert created.status_code == 201
|
||||
# admin tries to steal the second user's email
|
||||
taken = await admin.put(
|
||||
"/api/profile", json={"email": "second@example.org"}, headers=_HEADERS
|
||||
)
|
||||
assert taken.status_code == 409
|
||||
|
||||
|
||||
# ---------------------------------------------- cross-user isolation (authed)
|
||||
|
||||
|
||||
async def test_cross_user_404_on_run_scoped_routes(tmp_path: Path) -> None:
|
||||
"""Owner-scoped disk reads: user 2 requesting user 1's run gets a
|
||||
404 (no existence leak) — the owner dir simply doesn't exist under
|
||||
user 2's storage view."""
|
||||
app = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=_RefineModel(),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=True,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await admin.post(
|
||||
"/api/auth/register",
|
||||
json={"email": "admin@example.org", "password": "long-enough-pass", "display_name": "A"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
await admin.post(
|
||||
"/api/auth/login", json={"email": "admin@example.org", "password": "long-enough-pass"}, headers=_HEADERS
|
||||
)
|
||||
admin_id = (await admin.get("/api/auth/me")).json()["user_id"]
|
||||
_seed_run(
|
||||
tmp_path,
|
||||
"runaaaa1",
|
||||
state="DONE",
|
||||
report_markdown="# Mine\n\n[p-1]\n",
|
||||
report_json=_REPORT_JSON,
|
||||
evidence_json=_EVIDENCE_JSON,
|
||||
ledger_json=_LEDGER_JSON,
|
||||
owner=admin_id,
|
||||
)
|
||||
await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": "second@example.org", "password": "long-enough-pass", "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
await second.post(
|
||||
"/api/auth/login", json={"email": "second@example.org", "password": "long-enough-pass"}, headers=_HEADERS
|
||||
)
|
||||
assert (await second.get("/api/runs/runaaaa1/report")).status_code == 404
|
||||
assert (await second.get("/api/runs/runaaaa1/ledger")).status_code == 404
|
||||
# The owner-scoped collections are simply empty for user 2.
|
||||
assert (await second.get("/api/runs")).json() == []
|
||||
assert (await second.get("/api/library")).json() == {"folders": [], "reports": []}
|
||||
assert (await second.get("/api/tasks")).json() == []
|
||||
assert (await second.get("/api/suggestions")).json() == []
|
||||
|
||||
|
||||
# ------------------------------------- P-8 full closure: fresh app, same root
|
||||
|
||||
|
||||
async def test_full_run_over_http_writes_and_serves_artifacts(tmp_path: Path) -> None:
|
||||
"""End-to-end marker contract through the API: a run driven to DONE
|
||||
persists report.json/evidence.json (T-3.1), and the reworked report
|
||||
route serves normalized [^pN] bodies while report.md (via the
|
||||
markdown field) keeps raw [p-N]."""
|
||||
app = _fake_app(
|
||||
tmp_path,
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER, model="fake-writer"),
|
||||
],
|
||||
)
|
||||
async with await _client(app) as client:
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "How do engines run?", "methodology": "rapid-answer"}
|
||||
)
|
||||
assert created.status_code == 201
|
||||
run_id = created.json()["id"]
|
||||
|
||||
for _ in range(400): # await AWAITING_APPROVAL
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "awaiting_approval":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
await client.post(f"/api/runs/{run_id}/approve")
|
||||
for _ in range(400): # await DONE
|
||||
status = (await client.get(f"/api/runs/{run_id}")).json()["status"]
|
||||
if status == "done":
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
assert status == "done"
|
||||
|
||||
report = (await client.get(f"/api/runs/{run_id}/report")).json()
|
||||
assert report["report"] is not None
|
||||
assert report["evidence"] is not None
|
||||
# C3-10 through the API: bodies normalized, markdown raw.
|
||||
joined = "\n".join(str(s["body"]) for s in report["report"]["sections"])
|
||||
assert "[^p1]" in joined or "[^p2]" in joined
|
||||
assert "[p-1]" not in joined
|
||||
assert "[p-1]" in report["markdown"]
|
||||
# citations + evidence ids are pN-form with real quotes.
|
||||
for citation in report["report"]["citations"]:
|
||||
assert citation["passage_id"].startswith("p")
|
||||
assert "-" not in citation["passage_id"]
|
||||
assert citation["quote"]
|
||||
passages = report["evidence"]["passages"]
|
||||
assert passages, "evidence must carry the full stored passages"
|
||||
assert all(len(str(p["text"])) >= len(str(p["quote"])) for p in passages)
|
||||
|
||||
# The disk read-models see the run too (library/tasks derive).
|
||||
library = (await client.get("/api/library")).json()
|
||||
assert [r["runId"] for r in library["reports"]] == [run_id]
|
||||
tasks = (await client.get("/api/tasks")).json()
|
||||
assert all(item["status"] == "done" for item in tasks)
|
||||
|
||||
|
||||
async def test_fresh_app_over_existing_data_root(tmp_path: Path) -> None:
|
||||
"""P-8 full closure: a SECOND create_app over the same data root —
|
||||
zero in-memory actors/registry — every read route still correct.
|
||||
(The registry-independence proof; the first app's actor state is
|
||||
unreachable by construction here.)"""
|
||||
_seed_tree(tmp_path)
|
||||
first = _fake_app(tmp_path)
|
||||
del first # the first app instance is discarded; nothing carries over
|
||||
|
||||
second = create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(canned=[ChatResult(content=_PLAN, model="p")]),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=False,
|
||||
data_dir=tmp_path,
|
||||
)
|
||||
async with await _client(second) as client:
|
||||
# report: full payload, disk-read
|
||||
report = (await client.get("/api/runs/runaaaa1/report")).json()
|
||||
assert report["report"] == _REPORT_JSON
|
||||
assert report["evidence"] == _EVIDENCE_JSON
|
||||
assert "[p-1]" in report["markdown"]
|
||||
# pre-artifact fallback survives the restart too
|
||||
legacy = (await client.get("/api/runs/runbbbb1/report")).json()
|
||||
assert legacy["report"] is None and legacy["markdown"]
|
||||
# ledger
|
||||
assert (await client.get("/api/runs/runaaaa1/ledger")).json() == _LEDGER_JSON
|
||||
# run list: swept CANCELLED is cancelled, not open
|
||||
statuses = {run["id"]: run["status"] for run in (await client.get("/api/runs")).json()}
|
||||
assert statuses["runcccc1"] == "cancelled"
|
||||
# library + connections + graph + tasks + suggestions
|
||||
library = (await client.get("/api/library")).json()
|
||||
assert [r["runId"] for r in library["reports"]] == ["runaaaa1", "runbbbb1"]
|
||||
assert (await client.get("/api/library/connections")).status_code == 200
|
||||
assert (await client.get("/api/graph")).status_code == 200
|
||||
tasks = {item["id"]: item for item in (await client.get("/api/tasks")).json()}
|
||||
assert tasks["runcccc1#0"]["status"] == "open"
|
||||
assert (await client.get("/api/suggestions")).status_code == 200
|
||||
# profile (dev harness GET works with zero stored runs)
|
||||
assert (await client.get("/api/profile")).status_code == 200
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Opt-in live smoke tier (G-004): real-adapter probes, keyed only.
|
||||
|
||||
These tests NEVER run in the zero-key CI suite: they are marked
|
||||
`live` and skip cleanly when OLLAMA_API_KEY / TAVILY_API_KEY are
|
||||
absent from the secrets scope. Run explicitly with:
|
||||
|
||||
uv run pytest -m live
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.capability_probe import probe_model_adapter
|
||||
from researchx.adapters.ollama import OllamaModelAdapter
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.tavily import TavilySearchAdapter
|
||||
from researchx.trust.liveness import LivenessStatus, check_liveness
|
||||
|
||||
pytestmark = pytest.mark.live
|
||||
|
||||
_SECRETS_PATHS = [
|
||||
Path.cwd() / ".ciagent" / ".env.secrets",
|
||||
Path.cwd().parents[1] / ".ciagent" / ".env.secrets",
|
||||
]
|
||||
|
||||
|
||||
def _secrets() -> EnvFileSecretsProvider:
|
||||
return EnvFileSecretsProvider(paths=_SECRETS_PATHS)
|
||||
|
||||
|
||||
def _needs_key(key: str) -> str:
|
||||
value = _secrets().resolve(key) or os.environ.get(key)
|
||||
if not value:
|
||||
pytest.skip(f"{key} not present — live tier is opt-in")
|
||||
return value
|
||||
|
||||
|
||||
async def test_live_ollama_probe_and_chat() -> None:
|
||||
_needs_key("OLLAMA_API_KEY")
|
||||
adapter = OllamaModelAdapter(
|
||||
_secrets(), planner_model="gpt-oss:20b", writer_model="gpt-oss:20b"
|
||||
)
|
||||
probe = await probe_model_adapter(adapter)
|
||||
assert probe.endpoint in ("https://ollama.com/v1", "http://localhost:11434/v1")
|
||||
assert probe.context_window <= 8192
|
||||
assert probe.embedding_batch == 8
|
||||
|
||||
|
||||
async def test_live_tavily_search() -> None:
|
||||
_needs_key("TAVILY_API_KEY")
|
||||
adapter = TavilySearchAdapter(_secrets())
|
||||
results = await adapter.search("deep research agent verification", max_results=3)
|
||||
assert 1 <= len(results) <= 3
|
||||
assert all(r.url.startswith("http") for r in results)
|
||||
|
||||
|
||||
async def test_live_liveness_against_real_page() -> None:
|
||||
status = await check_liveness("https://example.org/")
|
||||
assert status in (LivenessStatus.LIVE, LivenessStatus.STALE, LivenessStatus.DEAD)
|
||||
void_httpx = httpx.AsyncClient
|
||||
del void_httpx
|
||||
@@ -0,0 +1,126 @@
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.storage import StorageAdapter
|
||||
from researchx.adapters.types import AdapterError
|
||||
|
||||
_OWNER = "u1a2b3c4"
|
||||
|
||||
|
||||
def _storage(tmp_path: Path) -> LocalStorageAdapter:
|
||||
return LocalStorageAdapter(base_dir=tmp_path, owner_id=_OWNER)
|
||||
|
||||
|
||||
async def test_storage_round_trip(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
assert isinstance(storage, StorageAdapter)
|
||||
target = await storage.write_run_artifact("abcd2345", "state.json", '{"state": "PLANNING"}')
|
||||
assert target.exists()
|
||||
content = await storage.read_run_artifact("abcd2345", "state.json")
|
||||
assert content == '{"state": "PLANNING"}'
|
||||
|
||||
|
||||
async def test_atomic_overwrite(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
await storage.write_run_artifact("abcd2345", "state.json", '{"v": 1}')
|
||||
await storage.write_run_artifact("abcd2345", "state.json", '{"v": 2}')
|
||||
assert await storage.read_run_artifact("abcd2345", "state.json") == '{"v": 2}'
|
||||
leftovers = list((tmp_path / "runs" / _OWNER / "abcd2345").glob("*.tmp"))
|
||||
assert leftovers == []
|
||||
|
||||
|
||||
async def test_run_dir_layout(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
run_dir = storage.run_dir("abcd2345")
|
||||
# D-106: runs/<owner_id>/<run_id>/ partitioning (supersedes D-010).
|
||||
assert run_dir == tmp_path / "runs" / _OWNER / "abcd2345"
|
||||
await storage.write_run_artifact("abcd2345", "report.md", "# Report")
|
||||
assert run_dir.is_dir()
|
||||
|
||||
|
||||
async def test_owner_id_required_and_validated(tmp_path: Path) -> None:
|
||||
for bad_owner in ["", "short", "waytoolongid123", "ABCD2345", "a b c d e", "../evil"]:
|
||||
try:
|
||||
LocalStorageAdapter(base_dir=tmp_path, owner_id=bad_owner)
|
||||
except AdapterError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError(f"expected AdapterError for owner_id: {bad_owner!r}")
|
||||
# owner_id is required — no default partition key.
|
||||
try:
|
||||
LocalStorageAdapter(base_dir=tmp_path) # type: ignore[call-arg]
|
||||
except TypeError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("owner_id must be a required positional parameter")
|
||||
|
||||
|
||||
async def test_read_missing_error_embeds_owner_path(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
try:
|
||||
await storage.read_run_artifact("zzzz9999", "state.json")
|
||||
except AdapterError as exc:
|
||||
assert f"runs/{_OWNER}/zzzz9999/state.json" in str(exc)
|
||||
else:
|
||||
raise AssertionError("expected AdapterError")
|
||||
|
||||
|
||||
async def test_read_missing_raises(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
try:
|
||||
await storage.read_run_artifact("zzzz9999", "state.json")
|
||||
except AdapterError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("expected AdapterError")
|
||||
|
||||
|
||||
async def test_path_traversal_rejected(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
for bad_name in ["../escape.md", "/etc/passwd", "a/b.md", "", "."]:
|
||||
try:
|
||||
await storage.write_run_artifact("abcd2345", bad_name, "x")
|
||||
except AdapterError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError(f"expected AdapterError for name: {bad_name!r}")
|
||||
for bad_run_id in ["../evil", "short", "waytoolongid123", "ABCD2345", "a b c d e"]:
|
||||
try:
|
||||
await storage.write_run_artifact(bad_run_id, "state.json", "x")
|
||||
except AdapterError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError(f"expected AdapterError for run_id: {bad_run_id!r}")
|
||||
|
||||
|
||||
async def test_no_tmp_leftover_after_write(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
await storage.write_run_artifact("abcd2345", "state.json", '{"v": 1}')
|
||||
leftovers = list((tmp_path / "runs" / _OWNER / "abcd2345").glob("*.tmp"))
|
||||
assert leftovers == []
|
||||
|
||||
|
||||
async def test_partition_isolation_between_owners(tmp_path: Path) -> None:
|
||||
alice = LocalStorageAdapter(base_dir=tmp_path, owner_id="aaaa1111")
|
||||
bob = LocalStorageAdapter(base_dir=tmp_path, owner_id="bbbb2222")
|
||||
await alice.write_run_artifact("abcd2345", "state.json", '{"owner": "alice"}')
|
||||
try:
|
||||
await bob.read_run_artifact("abcd2345", "state.json")
|
||||
except AdapterError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("owner partitions must be isolated on disk")
|
||||
|
||||
|
||||
async def test_list_run_ids_scans_owner_partition(tmp_path: Path) -> None:
|
||||
storage = _storage(tmp_path)
|
||||
assert storage.list_run_ids() == []
|
||||
await storage.write_run_artifact("abcd2345", "state.json", "{}")
|
||||
await storage.write_run_artifact("efgh6789", "state.json", "{}")
|
||||
# A file (not a dir) and dotfiles are skipped defensively.
|
||||
(tmp_path / "runs" / _OWNER / "notadir.md").write_text("x")
|
||||
(tmp_path / "runs" / _OWNER / ".hidden").mkdir()
|
||||
assert storage.list_run_ids() == ["abcd2345", "efgh6789"]
|
||||
|
||||
other = LocalStorageAdapter(base_dir=tmp_path, owner_id="zzzz9999")
|
||||
assert other.list_run_ids() == [] # separate partition, empty
|
||||
@@ -0,0 +1,312 @@
|
||||
"""Production entrypoint suite (T-2.7 — P-3 backend closure).
|
||||
|
||||
The live app built by ``researchx.api.main`` must be structurally
|
||||
incapable of serving fakes:
|
||||
|
||||
- The per-run model path is the ModelAdapterFactory — a run executes
|
||||
against an ``OllamaModelAdapter`` (resolved per user), never
|
||||
``FakeModelAdapter``. Pinned by asserting the factory resolves the
|
||||
adapter and that the app instance holds no fake in the model slot.
|
||||
- ``auth_enabled=True`` hardwired: every business route 401s without a
|
||||
session; there is no parameter or env that turns auth off.
|
||||
- ``cookie_secure=True``: the login cookie carries Secure (behind TLS).
|
||||
- The data root + secrets file come from env vars (RESEARCHX_DATA_DIR,
|
||||
RESEARCHX_SECRETS_FILE) — the test seam is the natural runtime
|
||||
config, parameterized not env-gated.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.fakes import FakeModelAdapter
|
||||
from researchx.auth.guard import CSRF_HEADER
|
||||
|
||||
_HEADERS = {CSRF_HEADER: "1"}
|
||||
_KEK = base64.urlsafe_b64encode(os.urandom(32)).decode()
|
||||
|
||||
_ADMIN_EMAIL = "admin@example.org"
|
||||
_ADMIN_PASSWORD = "correct-horse-battery"
|
||||
|
||||
|
||||
def _runtime_env(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, *, secrets: str | None, data_dir: Path
|
||||
) -> None:
|
||||
"""Point the production entry at test-owned locations.
|
||||
|
||||
This is the parameterization seam: main.py reads the same env vars
|
||||
the container runtime passes (RESEARCHX_SECRETS_FILE via --env-file
|
||||
conventions, RESEARCHX_DATA_DIR for the volume) — no test-only
|
||||
gates exist in the module.
|
||||
"""
|
||||
monkeypatch.setenv("RESEARCHX_DATA_DIR", str(data_dir))
|
||||
if secrets is None:
|
||||
monkeypatch.delenv("RESEARCHX_SECRETS_FILE", raising=False)
|
||||
else:
|
||||
env_file = tmp_path / "env.secrets"
|
||||
env_file.write_text(secrets, encoding="utf-8")
|
||||
monkeypatch.setenv("RESEARCHX_SECRETS_FILE", str(env_file))
|
||||
|
||||
|
||||
def _live_secrets(kek: bool = True, server_key: bool = True) -> str:
|
||||
lines = []
|
||||
if kek:
|
||||
lines.append(f"RESEARCHX_KEK={_KEK}")
|
||||
if server_key:
|
||||
lines.append("OLLAMA_API_KEY=server-tier-key-value")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
async def _register_admin(client: httpx.AsyncClient) -> None:
|
||||
await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD, "display_name": "Admin"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
|
||||
async def _login_with_secure_cookie(client: httpx.AsyncClient) -> str:
|
||||
"""Login and return the session cookie header value.
|
||||
|
||||
The production cookie carries Secure (hardwired, E-4); httpx's
|
||||
cookie jar — like a browser over plain http — will not replay a
|
||||
Secure cookie to a non-TLS origin. The tests therefore forward the
|
||||
cookie explicitly: the ASGI transport is the TLS proxy's interior
|
||||
leg, where the Secure flag does not apply.
|
||||
"""
|
||||
login = await client.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert login.status_code == 200
|
||||
return login.headers["set-cookie"].split(";")[0]
|
||||
|
||||
|
||||
def _auth_headers(cookie: str) -> dict[str, str]:
|
||||
return {**_HEADERS, "cookie": cookie}
|
||||
|
||||
|
||||
# --------------------------------------------------------------- structure
|
||||
|
||||
|
||||
async def test_main_factory_resolves_real_adapters(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""The P-3 backend closure: main's factory wiring resolves a real
|
||||
OllamaModelAdapter per user — the production app can never serve
|
||||
FakeModelAdapter (no fixed model adapter is even constructed when
|
||||
a factory is wired; pinned structurally in app.py)."""
|
||||
from researchx.api import main as main_module
|
||||
|
||||
_runtime_env(tmp_path, monkeypatch, secrets=_live_secrets(), data_dir=tmp_path / "data")
|
||||
main_module.build_app() # must import + build cleanly
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from researchx.adapters.model_factory import ModelAdapterFactory
|
||||
from researchx.adapters.ollama import OllamaModelAdapter
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.auth.key_store import KeyStore
|
||||
from researchx.auth.models import UserPublic
|
||||
from researchx.auth.settings import UserSettings
|
||||
|
||||
secrets_paths = main_module._secrets_paths()
|
||||
assert secrets_paths, "the runtime env file must be discoverable"
|
||||
secrets = EnvFileSecretsProvider(paths=secrets_paths)
|
||||
factory = ModelAdapterFactory(secrets, KeyStore(secrets, tmp_path / "data"))
|
||||
user = UserPublic(
|
||||
user_id="abcd2345",
|
||||
email="u@example.org",
|
||||
display_name="U",
|
||||
role="user",
|
||||
active=True,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
adapter, tier = await factory.resolve(user, UserSettings())
|
||||
assert isinstance(adapter, OllamaModelAdapter)
|
||||
assert not isinstance(adapter, FakeModelAdapter)
|
||||
assert tier in ("user", "server", "keyless")
|
||||
|
||||
|
||||
# --------------------------------------------------------------- behavior
|
||||
|
||||
|
||||
async def test_main_app_auth_hardwired(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""No env, no parameter: every business route 401s without a
|
||||
session (REQ-85 — there is no auth_enabled=False path in main)."""
|
||||
from researchx.api import main as main_module
|
||||
|
||||
_runtime_env(tmp_path, monkeypatch, secrets=_live_secrets(), data_dir=tmp_path / "data")
|
||||
app = main_module.build_app()
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
for path in [
|
||||
"/api/config",
|
||||
"/api/config/settings",
|
||||
"/api/config/settings/model-key",
|
||||
"/api/methodologies",
|
||||
"/api/runs/abcd2345",
|
||||
]:
|
||||
response = await client.get(path)
|
||||
assert response.status_code == 401, f"{path} leaked without auth"
|
||||
# POST-only surface rejects unauthenticated POSTs identically.
|
||||
assert (await client.post("/api/runs", json={}, headers=_HEADERS)).status_code == 401
|
||||
# The public surface stays public (D-103).
|
||||
assert (await client.get("/healthz")).status_code == 200
|
||||
|
||||
|
||||
async def test_main_app_cookie_secure(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""cookie_secure=True hardwired (E-4): the session cookie carries
|
||||
Secure because the public surface is behind the TLS proxy."""
|
||||
from researchx.api import main as main_module
|
||||
|
||||
data_dir = tmp_path / "data"
|
||||
_runtime_env(tmp_path, monkeypatch, secrets=_live_secrets(), data_dir=data_dir)
|
||||
app = main_module.build_app()
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _register_admin(client)
|
||||
login = await client.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert login.status_code == 200
|
||||
set_cookie = login.headers["set-cookie"].lower()
|
||||
assert "secure" in set_cookie # hardwired behind TLS
|
||||
|
||||
|
||||
async def test_main_app_config_resolves_tier_with_runtime_env(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""End to end through the production wiring: register + login +
|
||||
GET /api/config shows the resolved tier and key-store state from
|
||||
the runtime env file (KEK + server key -> server tier for a
|
||||
keyless user, key_store enabled)."""
|
||||
from researchx.api import main as main_module
|
||||
|
||||
data_dir = tmp_path / "data"
|
||||
_runtime_env(tmp_path, monkeypatch, secrets=_live_secrets(), data_dir=data_dir)
|
||||
app = main_module.build_app()
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _register_admin(client)
|
||||
cookie = await _login_with_secure_cookie(client)
|
||||
config = await client.get("/api/config", headers=_auth_headers(cookie))
|
||||
assert config.status_code == 200
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] == "server" # server key present, no user key
|
||||
assert body["key_store"] == "enabled"
|
||||
assert body["model_adapter"] == "OllamaModelAdapter" # never the fake
|
||||
|
||||
|
||||
async def test_main_app_keyless_without_keys(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""No KEK, no server key: key_store disabled, tier keyless — the
|
||||
honest zero-key posture through the production wiring."""
|
||||
from researchx.api import main as main_module
|
||||
|
||||
data_dir = tmp_path / "data"
|
||||
_runtime_env(tmp_path, monkeypatch, secrets="", data_dir=data_dir)
|
||||
app = main_module.build_app()
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _register_admin(client)
|
||||
cookie = await _login_with_secure_cookie(client)
|
||||
config = await client.get("/api/config", headers=_auth_headers(cookie))
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] == "keyless"
|
||||
assert body["key_store"] == "disabled"
|
||||
assert body["key_status"]["disabled"] is True
|
||||
|
||||
|
||||
async def test_main_app_key_store_round_trip_with_kek(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""With a runtime KEK the production app accepts a user key and
|
||||
resolves tier 'user' afterwards (REQ-73/74 chain end to end)."""
|
||||
from researchx.api import main as main_module
|
||||
|
||||
data_dir = tmp_path / "data"
|
||||
_runtime_env(tmp_path, monkeypatch, secrets=_live_secrets(), data_dir=data_dir)
|
||||
app = main_module.build_app()
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _register_admin(client)
|
||||
cookie = await _login_with_secure_cookie(client)
|
||||
put = await client.put(
|
||||
"/api/config/settings/model-key",
|
||||
json={"key": "user-tier-key-7777"},
|
||||
headers=_auth_headers(cookie),
|
||||
)
|
||||
assert put.status_code == 200
|
||||
assert put.json()["configured"] is True
|
||||
|
||||
config = await client.get("/api/config", headers=_auth_headers(cookie))
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] == "user"
|
||||
assert body["key_status"]["masked"].endswith("7777")
|
||||
|
||||
# The key file lands under the DATA ROOT (not cwd).
|
||||
me = await client.get("/api/auth/me", headers=_auth_headers(cookie))
|
||||
user_id = me.json()["user_id"]
|
||||
key_file = data_dir / "keys" / f"{user_id}.json"
|
||||
assert key_file.is_file()
|
||||
payload = json.loads(key_file.read_text(encoding="utf-8"))
|
||||
assert "user-tier-key-7777" not in json.dumps(payload) # encrypted at rest
|
||||
|
||||
|
||||
async def test_main_app_storage_is_data_root(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""The config status storage_dir is the data-root string (D-107) —
|
||||
never cwd-dependent."""
|
||||
from researchx.api import main as main_module
|
||||
|
||||
data_dir = tmp_path / "data-root"
|
||||
_runtime_env(tmp_path, monkeypatch, secrets=_live_secrets(), data_dir=data_dir)
|
||||
app = main_module.build_app()
|
||||
|
||||
async with httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=app), base_url="http://testserver"
|
||||
) as client:
|
||||
await _register_admin(client)
|
||||
cookie = await _login_with_secure_cookie(client)
|
||||
config = await client.get("/api/config", headers=_auth_headers(cookie))
|
||||
assert config.json()["storage_dir"] == str(data_dir)
|
||||
|
||||
|
||||
def test_main_module_has_no_auth_disable_surface() -> None:
|
||||
"""The main module exposes no auth_enabled parameter and reads no
|
||||
env var that could disable auth (REQ-85 + C3-16)."""
|
||||
import inspect
|
||||
|
||||
from researchx.api import main as main_module
|
||||
|
||||
signature = inspect.signature(main_module.build_app)
|
||||
assert "auth_enabled" not in signature.parameters
|
||||
assert "cookie_secure" not in signature.parameters
|
||||
source = inspect.getsource(main_module)
|
||||
assert "RESEARCHX_AUTH" not in source
|
||||
assert "auth_enabled=False" not in source
|
||||
# build_app passes auth_enabled=True exactly once (hardcoded).
|
||||
build_source = inspect.getsource(main_module.build_app)
|
||||
assert build_source.count("auth_enabled=") == 1
|
||||
assert "auth_enabled=True" in build_source
|
||||
assert "cookie_secure=True" in build_source
|
||||
@@ -0,0 +1,95 @@
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from researchx.methodologies import registry
|
||||
|
||||
|
||||
def test_registry_resolves_both_slugs() -> None:
|
||||
rapid = registry.get("rapid-answer")
|
||||
deep = registry.get("deep-report")
|
||||
assert rapid.slug == "rapid-answer"
|
||||
assert deep.slug == "deep-report"
|
||||
assert rapid.name == "Rapid Answer"
|
||||
assert deep.name == "Deep Report"
|
||||
|
||||
|
||||
def test_unknown_slug_raises_key_error() -> None:
|
||||
with pytest.raises(KeyError):
|
||||
registry.get("systematic-review")
|
||||
|
||||
|
||||
def test_list_ordered_by_name() -> None:
|
||||
listed = registry.list()
|
||||
assert [m.name for m in listed] == ["Deep Report", "Rapid Answer"]
|
||||
|
||||
|
||||
def test_builtin_dials_match_contracts() -> None:
|
||||
rapid = registry.get("rapid-answer")
|
||||
assert rapid.dials.depth == 1
|
||||
assert rapid.dials.breadth == 1
|
||||
assert rapid.dials.verification_passes == 1
|
||||
assert rapid.dials.citation_rigor == "per-claim"
|
||||
assert rapid.dials.synthesis_style == "memo"
|
||||
assert rapid.dials.output_structure == "brief"
|
||||
assert rapid.output_shape == "memo"
|
||||
|
||||
deep = registry.get("deep-report")
|
||||
assert deep.dials.depth == 3
|
||||
assert deep.dials.breadth == 2
|
||||
assert deep.dials.verification_passes == 2
|
||||
assert deep.dials.citation_rigor == "per-claim"
|
||||
assert deep.dials.synthesis_style == "executive"
|
||||
assert deep.dials.output_structure == "report"
|
||||
assert deep.dials.checkpoints is True
|
||||
assert deep.output_shape == "executive-report"
|
||||
|
||||
|
||||
def test_methodologies_differ_meaningfully() -> None:
|
||||
rapid = registry.get("rapid-answer")
|
||||
deep = registry.get("deep-report")
|
||||
assert rapid.dials.depth != deep.dials.depth
|
||||
assert rapid.dials.breadth != deep.dials.breadth
|
||||
assert rapid.dials.verification_passes != deep.dials.verification_passes
|
||||
assert rapid.dials.synthesis_style != deep.dials.synthesis_style
|
||||
assert rapid.dials.output_structure != deep.dials.output_structure
|
||||
assert rapid.output_shape != deep.output_shape
|
||||
|
||||
|
||||
def test_registry_entries_are_immutable() -> None:
|
||||
snapshot = registry.get("rapid-answer")
|
||||
with pytest.raises(ValidationError):
|
||||
snapshot.dials.depth = 3
|
||||
assert registry.get("rapid-answer").dials.depth == 1
|
||||
|
||||
listed = registry.list()
|
||||
with pytest.raises(ValidationError):
|
||||
listed[0].dials.checkpoints = True
|
||||
assert registry.get("rapid-answer").dials.checkpoints is False
|
||||
|
||||
|
||||
def test_prompt_templates_render_placeholders() -> None:
|
||||
deep = registry.get("deep-report")
|
||||
planner = deep.render_planner_prompt(question="What changed?", context="none")
|
||||
assert "What changed?" in planner
|
||||
assert "none" in planner
|
||||
writer = deep.render_writer_prompt(
|
||||
question="What changed?", context="none", plan="p1", evidence="[p-1] (https://a) quote text"
|
||||
)
|
||||
assert "{plan}" not in writer
|
||||
assert "{evidence}" not in writer
|
||||
assert "p1" in writer
|
||||
assert "[p-1] (https://a) quote text" in writer
|
||||
|
||||
|
||||
def test_writer_prompt_requires_evidence_slot() -> None:
|
||||
for slug in ("rapid-answer", "deep-report"):
|
||||
methodology = registry.get(slug)
|
||||
assert "{evidence}" in methodology.writer_prompt
|
||||
|
||||
|
||||
def test_mutation_of_returned_copy_does_not_affect_registry() -> None:
|
||||
original = registry.get("deep-report").dials.model_dump()
|
||||
copy = registry.get("deep-report").dials.model_copy()
|
||||
mutated = copy.model_copy(update={"depth": 1})
|
||||
assert mutated.depth == 1
|
||||
assert registry.get("deep-report").dials.model_dump() == original
|
||||
@@ -0,0 +1,215 @@
|
||||
import asyncio
|
||||
import json
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.tavily import TavilySearchAdapter
|
||||
from researchx.adapters.types import AdapterError, ChatResult, ModelMessage
|
||||
from researchx.engine.actor import RunActor, make_run_config
|
||||
from researchx.engine.events import EventBus
|
||||
from researchx.engine.run_state import Plan, RunState
|
||||
from researchx.engine.states import RunStateName
|
||||
from researchx.evidence.store import EvidenceStore
|
||||
from researchx.methodologies import registry
|
||||
from researchx.trust.anchoring import anchor_quote, extract_quotes_from_report
|
||||
|
||||
|
||||
class RecordingModel:
|
||||
def __init__(self, canned: list[ChatResult]) -> None:
|
||||
self._canned = list(canned)
|
||||
self.prompts: list[str] = []
|
||||
|
||||
async def chat(
|
||||
self, messages: list[ModelMessage], *, model: str, temperature: float = 0.2,
|
||||
json_mode: bool = False,
|
||||
) -> ChatResult:
|
||||
self.prompts.append(messages[0].content)
|
||||
return self._canned.pop(0)
|
||||
|
||||
async def list_models(self) -> list[str]:
|
||||
return ["fake-model"]
|
||||
|
||||
def roles(self):
|
||||
from researchx.adapters.model import ModelRoles
|
||||
|
||||
return ModelRoles(planner="fake-planner", writer="fake-writer")
|
||||
|
||||
|
||||
_PLAN = json.dumps({"sub_questions": ["verification"], "source_universe": ["web"], "outline": []})
|
||||
|
||||
|
||||
async def test_evidence_reaches_writer_prompt(tmp_path: Path) -> None:
|
||||
_rid, config = make_run_config(
|
||||
question="q", context="", methodology_slug="rapid-answer",
|
||||
overrides={}, registry=registry, run_id="abcd2345",
|
||||
)
|
||||
model = RecordingModel(
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content="# Report\n\nClaim [p-1].\n", model="fake-writer"),
|
||||
]
|
||||
)
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=model, # type: ignore[arg-type]
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4"),
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state == RunStateName.PLANNING:
|
||||
await asyncio.sleep(0.01)
|
||||
await actor.approve()
|
||||
await task
|
||||
|
||||
writer_prompt = model.prompts[-1]
|
||||
assert "[p-1]" in writer_prompt
|
||||
assert "(https://example.org/" in writer_prompt
|
||||
assert "Evidence passages:" in writer_prompt
|
||||
|
||||
|
||||
async def test_engine_refuses_private_urls_at_extract(tmp_path: Path, monkeypatch) -> None:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("TAVILY_API_KEY=t-key\n", encoding="utf-8")
|
||||
secrets = EnvFileSecretsProvider(paths=[env_file])
|
||||
tavily = TavilySearchAdapter(secrets)
|
||||
|
||||
called_urls: list[str] = []
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
called_urls.append(str(request.url))
|
||||
return httpx.Response(200, text="<html><body>never</body></html>")
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kw: original(transport=transport, **kw))
|
||||
|
||||
with pytest.raises(AdapterError, match="fetch refused"):
|
||||
await tavily.fetch("http://169.254.169.254/latest/meta-data")
|
||||
assert called_urls == []
|
||||
|
||||
|
||||
async def test_resume_from_searching_mid_run(tmp_path: Path) -> None:
|
||||
_rid, config = make_run_config(
|
||||
question="q", context="", methodology_slug="rapid-answer",
|
||||
overrides={}, registry=registry, run_id="abcd2345",
|
||||
)
|
||||
storage = LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4")
|
||||
mid_state = RunState(
|
||||
run_id="abcd2345",
|
||||
question="q",
|
||||
methodology_slug="rapid-answer",
|
||||
state=RunStateName.SEARCHING,
|
||||
plan=Plan(sub_questions=["verification"], source_universe=["web"], outline=[]),
|
||||
depth_cursor=0,
|
||||
depth_target=1,
|
||||
)
|
||||
await storage.write_run_artifact("abcd2345", "state.json", mid_state.model_dump_json())
|
||||
|
||||
model = RecordingModel(
|
||||
canned=[
|
||||
ChatResult(content="# Report\n\nClaim [p-1].\n", model="fake-writer"),
|
||||
]
|
||||
)
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=model, # type: ignore[arg-type]
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=storage,
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
actor.state = RunState.model_validate_json(
|
||||
await storage.read_run_artifact("abcd2345", "state.json")
|
||||
)
|
||||
state = await actor.run()
|
||||
assert state.state == RunStateName.DONE
|
||||
|
||||
|
||||
async def test_cancel_after_approval_persists_cancelled(tmp_path: Path) -> None:
|
||||
_rid, config = make_run_config(
|
||||
question="q", context="", methodology_slug="rapid-answer",
|
||||
overrides={}, registry=registry, run_id="abcd2345",
|
||||
)
|
||||
model = FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content="# Report\n\nClaim [p-1].\n", model="fake-writer"),
|
||||
]
|
||||
)
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=model,
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4"),
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state == RunStateName.PLANNING:
|
||||
await asyncio.sleep(0.01)
|
||||
await actor.approve()
|
||||
actor.pause()
|
||||
actor.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
saved = json.loads(
|
||||
await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact("abcd2345", "state.json")
|
||||
)
|
||||
assert saved["state"] == "CANCELLED"
|
||||
|
||||
|
||||
def test_quote_mismatch_flagged_in_ledger_path() -> None:
|
||||
store = EvidenceStore()
|
||||
real_passage = store.add_passage("https://a", "The verified sentence about anchoring " * 5)
|
||||
hallucinated = "This quote was never in any stored passage at all whatsoever."
|
||||
|
||||
verdict = anchor_quote(hallucinated, [real_passage])
|
||||
assert verdict.matched is False
|
||||
assert verdict.weak is False
|
||||
|
||||
good = anchor_quote(real_passage.quote, [real_passage])
|
||||
assert good.matched is True
|
||||
|
||||
report = f"Claim [p-1].\n\n> {hallucinated} [p-1]\n"
|
||||
quotes = extract_quotes_from_report(report, store)
|
||||
assert len(quotes) == 2
|
||||
bad_verdict = anchor_quote(hallucinated, store.writer_context())
|
||||
assert bad_verdict.matched is False
|
||||
|
||||
|
||||
def test_ledger_export_gate() -> None:
|
||||
from researchx.trust.ledger import build_ledger, liveness_export_gate
|
||||
from researchx.trust.liveness import LivenessStatus
|
||||
|
||||
store = EvidenceStore()
|
||||
stored_text = "The verified sentence used for the export gate test."
|
||||
store.add_passage("https://a", stored_text)
|
||||
ledger = build_ledger(
|
||||
run_id="abcd2345",
|
||||
report_text=f"{stored_text} [p-1]",
|
||||
evidence=store,
|
||||
liveness={"https://a": LivenessStatus.LIVE},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
passed, failures = liveness_export_gate(ledger)
|
||||
assert passed is True
|
||||
assert failures == []
|
||||
|
||||
ledger_dead = build_ledger(
|
||||
run_id="abcd2345",
|
||||
report_text=f"{stored_text} [p-1]",
|
||||
evidence=store,
|
||||
liveness={"https://a": LivenessStatus.DEAD},
|
||||
visited_at=datetime.now(UTC),
|
||||
)
|
||||
passed, failures = liveness_export_gate(ledger_dead)
|
||||
assert passed is False
|
||||
assert failures == ["https://a"]
|
||||
@@ -0,0 +1,129 @@
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.capability_probe import probe_model_adapter
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.ollama import OllamaModelAdapter
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.engine.actor import RunActor, make_run_config
|
||||
from researchx.engine.events import EventBus
|
||||
from researchx.engine.states import RunStateName
|
||||
from researchx.methodologies import registry
|
||||
|
||||
|
||||
async def test_probe_attaches_to_run_state(tmp_path: Path, monkeypatch) -> None:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("OLLAMA_API_KEY=cloud-key\n", encoding="utf-8")
|
||||
secrets = EnvFileSecretsProvider(paths=[env_file])
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="w")
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
if str(request.url).endswith("/models"):
|
||||
return httpx.Response(200, json={"data": [{"id": "w"}]})
|
||||
return httpx.Response(
|
||||
200, json={"choices": [{"message": {"content": '{"ok": true}'}}]}
|
||||
)
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kw: original(transport=transport, **kw))
|
||||
|
||||
probe = await probe_model_adapter(adapter)
|
||||
assert probe.structured_outputs_supported is True
|
||||
|
||||
_rid, config = make_run_config(
|
||||
question="q",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
)
|
||||
plan = json.dumps({"sub_questions": ["verification"], "source_universe": ["web"], "outline": []})
|
||||
writer = "# Report\n\nClaim [p-1].\n"
|
||||
fake_model = FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=plan, model="fake-planner"),
|
||||
ChatResult(content=writer, model="fake-writer"),
|
||||
]
|
||||
)
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=fake_model,
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4"),
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
await actor.attach_probe(probe.model_dump(mode="json"))
|
||||
assert actor.state.probe is not None
|
||||
assert actor.state.probe["structured_outputs_supported"] is True
|
||||
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state == RunStateName.PLANNING:
|
||||
await asyncio.sleep(0.01)
|
||||
await actor.approve()
|
||||
state = await task
|
||||
assert state.state == RunStateName.DONE
|
||||
saved = json.loads(
|
||||
await LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4").read_run_artifact("abcd2345", "state.json")
|
||||
)
|
||||
assert saved["probe"]["endpoint"] == "https://ollama.com/v1"
|
||||
|
||||
|
||||
async def test_probe_failure_never_crashes_the_run(tmp_path: Path, monkeypatch) -> None:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("OLLAMA_API_KEY=cloud-key\n", encoding="utf-8")
|
||||
secrets = EnvFileSecretsProvider(paths=[env_file])
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="w")
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(500, text="down")
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kw: original(transport=transport, **kw))
|
||||
|
||||
from researchx.adapters.types import AdapterError
|
||||
|
||||
with pytest.raises(AdapterError):
|
||||
await probe_model_adapter(adapter)
|
||||
|
||||
_rid, config = make_run_config(
|
||||
question="q",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
)
|
||||
plan = json.dumps({"sub_questions": ["verification"], "source_universe": ["web"], "outline": []})
|
||||
writer = "# Report\n\nClaim [p-1].\n"
|
||||
fake_model = FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=plan, model="fake-planner"),
|
||||
ChatResult(content=writer, model="fake-writer"),
|
||||
]
|
||||
)
|
||||
bus = EventBus()
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=fake_model,
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4"),
|
||||
registry=registry,
|
||||
event_bus=bus,
|
||||
)
|
||||
task = asyncio.create_task(actor.run())
|
||||
while actor.state.state == RunStateName.PLANNING:
|
||||
await asyncio.sleep(0.01)
|
||||
await actor.approve()
|
||||
state = await task
|
||||
assert state.state == RunStateName.DONE
|
||||
assert state.probe is None
|
||||
@@ -0,0 +1,508 @@
|
||||
"""Read-model unit tests (T-3.2, D-114/D-115 — REQ-76..79).
|
||||
|
||||
A fixture disk tree — two users; DONE, CANCELLED, FAILED runs; a
|
||||
dotfile and a .tmp leftover — asserting categorization, sorting,
|
||||
shared-source/topic edges, tasks answered-iff DONE, suggestions
|
||||
(never synthetic, empty for fresh users), dotfile/tmp skip, and
|
||||
cross-user isolation at the read-model level.
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.api.read_models import (
|
||||
categorize,
|
||||
connections,
|
||||
graph,
|
||||
library,
|
||||
list_runs,
|
||||
suggestions,
|
||||
tasks,
|
||||
topic_tokens,
|
||||
)
|
||||
|
||||
_U1 = "aaaa1111"
|
||||
_U2 = "bbbb2222"
|
||||
|
||||
_PLAN_A = {
|
||||
"sub_questions": ["battery energy density gains", "solid-state pilot production"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro"],
|
||||
}
|
||||
_PLAN_B = {
|
||||
"sub_questions": ["battery safety testing standards"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro"],
|
||||
}
|
||||
_PLAN_CANCELLED = {
|
||||
"sub_questions": ["abandoned open question"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro"],
|
||||
}
|
||||
_PLAN_FAILED = {"sub_questions": ["failed run question"], "source_universe": ["web"], "outline": []}
|
||||
_PLAN_INFLIGHT = {"sub_questions": ["still running question"], "source_universe": ["web"], "outline": []}
|
||||
|
||||
_REPORT_A = {
|
||||
"v": 1,
|
||||
"run_id": "runaaaa1",
|
||||
"owner_id": _U1,
|
||||
"title": "Battery Advances",
|
||||
"question": "What are the latest advances in solid-state batteries?",
|
||||
"methodology_slug": "deep-report",
|
||||
"generated_at": "2026-09-20T14:32:00Z",
|
||||
"sections": [{"heading": "Report", "body": "Claim [^p1]."}],
|
||||
"citations": [
|
||||
{"passage_id": "p1", "url": "https://example.org/x", "title": "X", "quote": "q", "health": "live", "authority_tier": "reference"}
|
||||
],
|
||||
}
|
||||
_LEDGER_A = {
|
||||
"run_id": "runaaaa1",
|
||||
"generated_at": "2026-09-20T14:32:00Z",
|
||||
"entries": [
|
||||
{"url": "https://example.org/x", "visited_at": "2026-09-20T14:00:00Z", "passages": ["p-1"], "claims_supported": 1, "health": "live", "authority_tier": "reference"},
|
||||
{"url": "https://example.org/y", "visited_at": "2026-09-20T14:00:00Z", "passages": ["p-2"], "claims_supported": 1, "health": "live", "authority_tier": "general"},
|
||||
],
|
||||
"checklist": {},
|
||||
}
|
||||
|
||||
_LEDGER_B = {
|
||||
"run_id": "runbbbb1",
|
||||
"generated_at": "2026-09-22T09:14:00Z",
|
||||
"entries": [
|
||||
{"url": "https://example.org/x", "visited_at": "2026-09-22T09:00:00Z", "passages": ["p-1"], "claims_supported": 2, "health": "live", "authority_tier": "reference"},
|
||||
{"url": "https://example.org/z", "visited_at": "2026-09-22T09:00:00Z", "passages": ["p-2"], "claims_supported": 1, "health": "stale", "authority_tier": "general"},
|
||||
],
|
||||
"checklist": {},
|
||||
}
|
||||
|
||||
|
||||
def _state(
|
||||
run_id: str,
|
||||
question: str,
|
||||
slug: str,
|
||||
state: str,
|
||||
plan: dict | None,
|
||||
created_at: str,
|
||||
) -> dict:
|
||||
return {
|
||||
"run_id": run_id,
|
||||
"question": question,
|
||||
"context": "",
|
||||
"methodology_slug": slug,
|
||||
"state": state,
|
||||
"owner_id": _U1,
|
||||
"plan": plan,
|
||||
"depth_cursor": 0,
|
||||
"depth_target": 1,
|
||||
"counters": {},
|
||||
"probe": None,
|
||||
"created_at": created_at,
|
||||
"updated_at": created_at,
|
||||
}
|
||||
|
||||
|
||||
def _write_run(
|
||||
base: Path,
|
||||
owner: str,
|
||||
run_id: str,
|
||||
state: dict,
|
||||
report: dict | None = None,
|
||||
ledger: dict | None = None,
|
||||
) -> None:
|
||||
run_dir = base / "runs" / owner / run_id
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
(run_dir / "state.json").write_text(json.dumps(state), encoding="utf-8")
|
||||
if report is not None:
|
||||
(run_dir / "report.json").write_text(json.dumps(report), encoding="utf-8")
|
||||
if ledger is not None:
|
||||
(run_dir / "ledger.json").write_text(json.dumps(ledger), encoding="utf-8")
|
||||
|
||||
|
||||
def _tree(tmp_path: Path) -> Path:
|
||||
"""The fixture tree: user 1 has DONE A (deep-report, battery), DONE
|
||||
B (rapid-answer, battery safety), a CANCELLED run, and a FAILED
|
||||
run; user 2 has one DONE run invisible to user 1."""
|
||||
base = tmp_path
|
||||
_write_run(
|
||||
base,
|
||||
_U1,
|
||||
"runaaaa1",
|
||||
_state(
|
||||
"runaaaa1",
|
||||
"What are the latest advances in solid-state batteries?",
|
||||
"deep-report",
|
||||
"DONE",
|
||||
_PLAN_A,
|
||||
"2026-09-20T14:32:00Z",
|
||||
),
|
||||
report=_REPORT_A,
|
||||
ledger=_LEDGER_A,
|
||||
)
|
||||
_write_run(
|
||||
base,
|
||||
_U1,
|
||||
"runbbbb1",
|
||||
_state(
|
||||
"runbbbb1",
|
||||
"How do battery safety standards evolve?",
|
||||
"rapid-answer",
|
||||
"DONE",
|
||||
_PLAN_B,
|
||||
"2026-09-22T09:14:00Z",
|
||||
),
|
||||
ledger=_LEDGER_B,
|
||||
)
|
||||
_write_run(
|
||||
base,
|
||||
_U1,
|
||||
"runcanc1",
|
||||
_state(
|
||||
"runcanc1",
|
||||
"abandoned mid-flight",
|
||||
"deep-report",
|
||||
"CANCELLED",
|
||||
_PLAN_CANCELLED,
|
||||
"2026-09-21T10:00:00Z",
|
||||
),
|
||||
)
|
||||
_write_run(
|
||||
base,
|
||||
_U1,
|
||||
"runfail1",
|
||||
_state(
|
||||
"runfail1",
|
||||
"it broke",
|
||||
"rapid-answer",
|
||||
"FAILED",
|
||||
_PLAN_FAILED,
|
||||
"2026-09-23T10:00:00Z",
|
||||
),
|
||||
)
|
||||
# Defensive-scan fodder (D-115): dotfile + tmp leftovers + junk.
|
||||
(base / "runs" / _U1 / ".hidden").mkdir(parents=True)
|
||||
(base / "runs" / _U1 / "runbbbb1.tmp").mkdir(parents=True)
|
||||
(base / "runs" / _U1 / "orphan001").mkdir(parents=True) # no state.json: skipped
|
||||
# An in-flight run (raw, pre-sweep — the read-model sees the state
|
||||
# as written; the sweep happens at create_app in the route layer).
|
||||
_write_run(
|
||||
base,
|
||||
_U1,
|
||||
"runinflg1",
|
||||
_state(
|
||||
"runinflg1",
|
||||
"still running",
|
||||
"deep-report",
|
||||
"SEARCHING",
|
||||
_PLAN_INFLIGHT,
|
||||
"2026-09-25T08:00:00Z",
|
||||
),
|
||||
)
|
||||
# The other user's run: invisible at every read-model level.
|
||||
_write_run(
|
||||
base,
|
||||
_U2,
|
||||
"runcccc1",
|
||||
_state(
|
||||
"runcccc1",
|
||||
"user two's private research",
|
||||
"deep-report",
|
||||
"DONE",
|
||||
_PLAN_B,
|
||||
"2026-09-24T10:00:00Z",
|
||||
),
|
||||
ledger=_LEDGER_A,
|
||||
)
|
||||
return base
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ scanning
|
||||
|
||||
|
||||
def test_list_runs_scans_owner_partition_only(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
ids = [record.run_id for record in list_runs(base, _U1)]
|
||||
# CANCELLED + FAILED + in-flight included; dotfile/tmp/orphan
|
||||
# skipped; user 2's run absent.
|
||||
assert set(ids) == {"runaaaa1", "runbbbb1", "runcanc1", "runfail1", "runinflg1"}
|
||||
|
||||
|
||||
def test_list_runs_orders_most_recent_first(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
ids = [record.run_id for record in list_runs(base, _U1)]
|
||||
assert ids[0] == "runinflg1" # 2026-09-25
|
||||
assert ids[-1] == "runaaaa1" # 2026-09-20
|
||||
|
||||
|
||||
def test_list_runs_skips_dotfiles_and_tmp(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
ids = {record.run_id for record in list_runs(base, _U1)}
|
||||
assert ".hidden" not in ids
|
||||
assert "runbbbb1.tmp" not in ids
|
||||
|
||||
|
||||
def test_list_runs_empty_for_fresh_user(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
assert list_runs(base, "cccc3333") == []
|
||||
|
||||
|
||||
# ------------------------------------------------------------ categorization
|
||||
|
||||
|
||||
def test_categorize_is_methodology_plus_first_topic_token() -> None:
|
||||
record = list_runs # placeholder to keep import honest
|
||||
del record
|
||||
from researchx.api.read_models import RunDiskRecord
|
||||
|
||||
run = RunDiskRecord(
|
||||
run_id="runaaaa1",
|
||||
state={
|
||||
"run_id": "runaaaa1",
|
||||
"question": "q",
|
||||
"methodology_slug": "deep-report",
|
||||
"state": "DONE",
|
||||
"plan": _PLAN_A,
|
||||
"created_at": "2026-09-20T14:32:00Z",
|
||||
},
|
||||
)
|
||||
assert categorize(run) == "deep-report-battery"
|
||||
|
||||
|
||||
def test_topic_tokens_skip_stopwords_and_dedupe() -> None:
|
||||
from researchx.api.read_models import RunDiskRecord
|
||||
|
||||
run = RunDiskRecord(
|
||||
run_id="r",
|
||||
state={
|
||||
"run_id": "r",
|
||||
"question": "q",
|
||||
"methodology_slug": "rapid-answer",
|
||||
"state": "DONE",
|
||||
"plan": {"sub_questions": ["The what of battery battery gains"]},
|
||||
"created_at": "2026-09-20T14:32:00Z",
|
||||
},
|
||||
)
|
||||
assert topic_tokens(run) == ["battery", "gains"]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ library
|
||||
|
||||
|
||||
def test_library_folders_and_summaries(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
payload = library(base, _U1)
|
||||
# Folders from the two DONE runs' categories, sorted by id.
|
||||
assert [folder["id"] for folder in payload["folders"]] == [
|
||||
"deep-report-battery",
|
||||
"rapid-answer-battery",
|
||||
]
|
||||
assert payload["folders"][0]["name"] == "Deep Report: Battery"
|
||||
# Only DONE runs are filed (cancelled/failed produce nothing).
|
||||
assert {report["runId"] for report in payload["reports"]} == {"runaaaa1", "runbbbb1"}
|
||||
first = next(r for r in payload["reports"] if r["runId"] == "runaaaa1")
|
||||
assert first["id"] == "rep-runaaaa1"
|
||||
assert first["title"] == "Battery Advances"
|
||||
assert first["methodologySlug"] == "deep-report"
|
||||
assert first["category"] == "deep-report-battery"
|
||||
assert first["generatedAt"] == "2026-09-20T14:32:00Z"
|
||||
assert first["citationCount"] == 1
|
||||
# Pre-artifact DONE run: falls back to state created_at + methodology title.
|
||||
second = next(r for r in payload["reports"] if r["runId"] == "runbbbb1")
|
||||
assert second["generatedAt"] == "2026-09-22T09:14:00Z"
|
||||
assert second["title"] == "Rapid Answer — How do battery safety standards evolve?"
|
||||
|
||||
|
||||
def test_library_sort_recent_and_title(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
recent = [r["runId"] for r in library(base, _U1, sort="recent")["reports"]]
|
||||
title = [r["runId"] for r in library(base, _U1, sort="title")["reports"]]
|
||||
# recent: newest first (created_at desc).
|
||||
assert recent == ["runbbbb1", "runaaaa1"]
|
||||
# title: "Battery Advances" < "Rapid Answer — ..." A-Z.
|
||||
assert title == ["runaaaa1", "runbbbb1"]
|
||||
# unknown sort key falls back to recent.
|
||||
assert library(base, _U1, sort="bogus")["reports"][0]["runId"] == "runbbbb1"
|
||||
|
||||
|
||||
def test_library_fresh_user_is_empty(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
payload = library(base, "cccc3333")
|
||||
assert payload == {"folders": [], "reports": []}
|
||||
|
||||
|
||||
def test_library_cross_user_isolation(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
payload = library(base, _U1)
|
||||
assert all(report["runId"] != "runcccc1" for report in payload["reports"])
|
||||
|
||||
|
||||
# ------------------------------------------------------------- connections
|
||||
|
||||
|
||||
def test_connections_shared_source_edge(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
edges = connections(base, _U1)
|
||||
shared = [e for e in edges if e["kind"] == "shared-sources"]
|
||||
# runaaaa1 + runbbbb1 share https://example.org/x (>=1 URL overlap).
|
||||
assert shared == [
|
||||
{
|
||||
"fromReportId": "rep-runaaaa1",
|
||||
"toReportId": "rep-runbbbb1",
|
||||
"kind": "shared-sources",
|
||||
"label": "1 shared source",
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_connections_related_topic_edge_when_no_shared_urls(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
# Strip the shared URL: the two DONE runs then connect only via
|
||||
# category (same methodology + first topic token) — no, they have
|
||||
# DIFFERENT methodologies. Instead verify the pairing rule: with no
|
||||
# shared URL and differing categories there is NO edge.
|
||||
other = base / "runs" / _U1 / "runbbbb1" / "ledger.json"
|
||||
payload = json.loads(other.read_text(encoding="utf-8"))
|
||||
payload["entries"] = [e for e in payload["entries"] if e["url"] != "https://example.org/x"]
|
||||
other.write_text(json.dumps(payload), encoding="utf-8")
|
||||
edges = connections(base, _U1)
|
||||
# deep-report-battery vs rapid-answer-battery: different methodology
|
||||
# folders -> not related-topic, and no shared source -> no edge.
|
||||
assert edges == []
|
||||
|
||||
|
||||
def test_connections_related_topic_edge_on_same_category(tmp_path: Path) -> None:
|
||||
base = tmp_path / "iso"
|
||||
_write_run(
|
||||
base,
|
||||
_U1,
|
||||
"runx0011",
|
||||
_state("runx0011", "battery density", "deep-report", "DONE", _PLAN_A, "2026-09-20T10:00:00Z"),
|
||||
ledger={"entries": [{"url": "https://example.org/x"}], "checklist": {}},
|
||||
)
|
||||
_write_run(
|
||||
base,
|
||||
_U1,
|
||||
"runx0022",
|
||||
_state("runx0022", "battery pilots", "deep-report", "DONE", _PLAN_A, "2026-09-21T10:00:00Z"),
|
||||
ledger={"entries": [{"url": "https://example.org/q"}], "checklist": {}},
|
||||
)
|
||||
edges = connections(base, _U1)
|
||||
assert edges == [
|
||||
{
|
||||
"fromReportId": "rep-runx0011",
|
||||
"toReportId": "rep-runx0022",
|
||||
"kind": "related-topic",
|
||||
"label": "Adjacent topic",
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_connections_cross_user_isolation(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
edges = connections(base, _U1)
|
||||
for edge in edges:
|
||||
assert "runcccc1" not in (edge["fromReportId"], edge["toReportId"])
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- graph
|
||||
|
||||
|
||||
def test_graph_nodes_and_edges(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
payload = graph(base, _U1)
|
||||
node_ids = {node["id"] for node in payload["nodes"]}
|
||||
assert node_ids == {"runaaaa1", "runbbbb1"} # DONE runs only
|
||||
node = next(n for n in payload["nodes"] if n["id"] == "runaaaa1")
|
||||
assert node["topic"] == "Battery Advances"
|
||||
assert set(node["linkedSources"]) == {"https://example.org/x", "https://example.org/y"}
|
||||
assert node["findings"] == ["Battery Advances"]
|
||||
assert payload["edges"] == [
|
||||
{"fromId": "runaaaa1", "toId": "runbbbb1", "sharedSourceCount": 1}
|
||||
]
|
||||
|
||||
|
||||
def test_graph_fresh_user_is_empty(tmp_path: Path) -> None:
|
||||
assert graph(_tree(tmp_path), "cccc3333") == {"nodes": [], "edges": []}
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- tasks
|
||||
|
||||
|
||||
def test_tasks_answered_iff_done(tmp_path: Path) -> None:
|
||||
"""D-114: a sub-question is done IFF its run reached DONE; the
|
||||
post-sweep CANCELLED run's questions stay open."""
|
||||
base = _tree(tmp_path)
|
||||
items = tasks(base, _U1)
|
||||
by_id = {item["id"]: item for item in items}
|
||||
# DONE runs -> done items with <run_id>#<idx> ids.
|
||||
assert by_id["runaaaa1#0"] == {
|
||||
"id": "runaaaa1#0",
|
||||
"question": "battery energy density gains",
|
||||
"sourceRunId": "runaaaa1",
|
||||
"status": "done",
|
||||
}
|
||||
assert by_id["runbbbb1#0"]["status"] == "done"
|
||||
# CANCELLED + FAILED + in-flight runs -> open items.
|
||||
assert by_id["runcanc1#0"]["status"] == "open"
|
||||
assert by_id["runcanc1#0"]["question"] == "abandoned open question"
|
||||
assert by_id["runfail1#0"]["status"] == "open"
|
||||
assert by_id["runinflg1#0"]["status"] == "open"
|
||||
# user 2's runs absent.
|
||||
assert all("runcccc1" not in item["id"] for item in items)
|
||||
|
||||
|
||||
def test_tasks_cross_user_isolation(tmp_path: Path) -> None:
|
||||
items = tasks(_tree(tmp_path), "cccc3333")
|
||||
assert items == []
|
||||
|
||||
|
||||
# -------------------------------------------------------------- suggestions
|
||||
|
||||
|
||||
def test_suggestions_are_open_items_joined_with_titles(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
payload = suggestions(base, _U1)
|
||||
# Open items from the CANCELLED + FAILED + in-flight runs, most
|
||||
# recent run first, attributed to their source run titles; nothing
|
||||
# synthetic, nothing from DONE runs.
|
||||
assert payload == [
|
||||
{
|
||||
"id": "sug-runinflg1#0",
|
||||
"question": "still running question",
|
||||
"sourceRunId": "runinflg1",
|
||||
"sourceReportTitle": "Deep Report — still running",
|
||||
},
|
||||
{
|
||||
"id": "sug-runfail1#0",
|
||||
"question": "failed run question",
|
||||
"sourceRunId": "runfail1",
|
||||
"sourceReportTitle": "Rapid Answer — it broke",
|
||||
},
|
||||
{
|
||||
"id": "sug-runcanc1#0",
|
||||
"question": "abandoned open question",
|
||||
"sourceRunId": "runcanc1",
|
||||
"sourceReportTitle": "Deep Report — abandoned mid-flight",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_suggestions_fresh_user_empty_never_synthetic(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
assert suggestions(base, "cccc3333") == []
|
||||
|
||||
|
||||
def test_suggestions_exclude_done_runs(tmp_path: Path) -> None:
|
||||
base = _tree(tmp_path)
|
||||
questions = [s["question"] for s in suggestions(base, _U1)]
|
||||
assert "battery energy density gains" not in questions # DONE run
|
||||
|
||||
|
||||
# ------------------------------------------------- defensive scanning extras
|
||||
|
||||
|
||||
def test_list_runs_skips_corrupt_state_json(tmp_path: Path) -> None:
|
||||
base = tmp_path
|
||||
run_dir = base / "runs" / _U1 / "corrupt01"
|
||||
run_dir.mkdir(parents=True)
|
||||
(run_dir / "state.json").write_text("{not json", encoding="utf-8")
|
||||
assert list_runs(base, _U1) == []
|
||||
@@ -0,0 +1,203 @@
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.duckduckgo import DuckDuckGoSearchAdapter
|
||||
from researchx.adapters.model import ModelRoles
|
||||
from researchx.adapters.ollama import OllamaModelAdapter
|
||||
from researchx.adapters.probe import CapabilityProbe, cap_context_window, structured_output_warning
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.tavily import TavilySearchAdapter
|
||||
from researchx.adapters.types import AdapterError, ModelMessage
|
||||
|
||||
|
||||
def _secrets(tmp_path, values: dict[str, str]) -> EnvFileSecretsProvider:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text(
|
||||
"".join(f"{k}={v}\n" for k, v in values.items()),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return EnvFileSecretsProvider(paths=[env_file])
|
||||
|
||||
|
||||
def _chat_payload_ok() -> dict[str, object]:
|
||||
return {
|
||||
"choices": [{"message": {"role": "assistant", "content": '{"ok": true}'}}],
|
||||
"usage": {"prompt_tokens": 10, "completion_tokens": 5},
|
||||
}
|
||||
|
||||
|
||||
async def test_ollama_adapter_uses_cloud_when_keyed(tmp_path, monkeypatch) -> None:
|
||||
secrets = _secrets(tmp_path, {"OLLAMA_API_KEY": "cloud-key"})
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="w")
|
||||
|
||||
calls: list[tuple[str, dict[str, str]]] = []
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
calls.append((str(request.url), dict(request.headers)))
|
||||
return httpx.Response(200, json=_chat_payload_ok())
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
result = await adapter.chat([ModelMessage(role="user", content="hi")], model="w")
|
||||
assert result.content == '{"ok": true}'
|
||||
assert calls[0][0].startswith("https://ollama.com/v1")
|
||||
assert "Bearer cloud-key" in calls[0][1]["authorization"]
|
||||
|
||||
|
||||
async def test_ollama_adapter_falls_back_to_local_on_cloud_failure(tmp_path, monkeypatch) -> None:
|
||||
secrets = _secrets(tmp_path, {"OLLAMA_API_KEY": "cloud-key"})
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="p", writer_model="w")
|
||||
|
||||
bases_hit: list[str] = []
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
url = str(request.url)
|
||||
bases_hit.append(url)
|
||||
if url.startswith("https://ollama.com"):
|
||||
return httpx.Response(401, json={"error": "bad key"})
|
||||
return httpx.Response(200, json=_chat_payload_ok())
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
result = await adapter.chat([ModelMessage(role="user", content="hi")], model="w")
|
||||
assert result.content == '{"ok": true}'
|
||||
assert any(u.startswith("http://localhost:11434/v1") for u in bases_hit)
|
||||
assert adapter.last_fallback_reason is not None
|
||||
assert adapter.active_base_url == "http://localhost:11434/v1"
|
||||
|
||||
|
||||
async def test_ollama_roles(tmp_path) -> None:
|
||||
secrets = _secrets(tmp_path, {})
|
||||
adapter = OllamaModelAdapter(secrets, planner_model="planner-model", writer_model="writer-model")
|
||||
assert adapter.roles() == ModelRoles(planner="planner-model", writer="writer-model")
|
||||
|
||||
|
||||
def test_context_cap() -> None:
|
||||
assert cap_context_window(None) == 8192
|
||||
assert cap_context_window(100_000) == 8192
|
||||
assert cap_context_window(4_096) == 4_096
|
||||
assert cap_context_window(0) == 8192
|
||||
|
||||
|
||||
def test_structured_output_warning_contract() -> None:
|
||||
assert structured_output_warning(True, "m") is None
|
||||
warning = structured_output_warning(False, "writer-model")
|
||||
assert warning is not None
|
||||
assert "writer-model" in warning
|
||||
assert "degrade" in warning
|
||||
probe = CapabilityProbe(
|
||||
endpoint="http://localhost:11434/v1",
|
||||
models=["m"],
|
||||
context_window=8192,
|
||||
embedding_batch=8,
|
||||
structured_outputs_supported=False,
|
||||
structured_output_warning=warning,
|
||||
)
|
||||
assert probe.degraded
|
||||
|
||||
|
||||
async def test_tavily_requires_key(tmp_path) -> None:
|
||||
secrets = _secrets(tmp_path, {})
|
||||
with pytest.raises(AdapterError):
|
||||
TavilySearchAdapter(secrets)
|
||||
|
||||
|
||||
async def test_tavily_search_returns_results(tmp_path, monkeypatch) -> None:
|
||||
secrets = _secrets(tmp_path, {"TAVILY_API_KEY": "t-key"})
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
import json
|
||||
|
||||
body = json.loads(request.content.decode())
|
||||
assert body["api_key"] == "t-key"
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"results": [
|
||||
{"title": "T1", "url": "https://example.org/a", "content": "snippet one"},
|
||||
{"title": "T2", "url": "https://example.org/b", "content": "snippet two"},
|
||||
]
|
||||
},
|
||||
)
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
adapter = TavilySearchAdapter(secrets)
|
||||
results = await adapter.search("deep research", max_results=2)
|
||||
assert [r.title for r in results] == ["T1", "T2"]
|
||||
|
||||
|
||||
async def test_ddg_parses_html_results(monkeypatch) -> None:
|
||||
html = """
|
||||
<div class="results">
|
||||
<a rel="nofollow" class="result__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fexample.org%2Fx&rut=abc">Example One</a>
|
||||
<a class="result__snippet" href="#">First snippet text</a>
|
||||
<a rel="nofollow" class="result__a" href="https://example.org/y">Example Two</a>
|
||||
<a class="result__snippet" href="#">Second snippet text</a>
|
||||
</div>
|
||||
"""
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(200, text=html)
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
adapter = DuckDuckGoSearchAdapter()
|
||||
results = await adapter.search("anything", max_results=8)
|
||||
assert [r.url for r in results] == ["https://example.org/x", "https://example.org/y"]
|
||||
assert results[0].snippet == "First snippet text"
|
||||
assert adapter.last_degradation is None
|
||||
|
||||
|
||||
async def test_ddg_rate_limit_degrades_with_typed_warning(monkeypatch) -> None:
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(403, text="forbidden")
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
adapter = DuckDuckGoSearchAdapter()
|
||||
results = await adapter.search("anything")
|
||||
assert results == []
|
||||
assert adapter.last_degradation is not None
|
||||
assert "rate-limited" in adapter.last_degradation
|
||||
|
||||
|
||||
async def test_ddg_fetch_extracts_text(monkeypatch) -> None:
|
||||
html = "<html><head><title>Page Title</title></head><body><script>var x=1;</script><p>Hello world</p></body></html>"
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(200, text=html)
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs))
|
||||
|
||||
adapter = DuckDuckGoSearchAdapter()
|
||||
page = await adapter.fetch("https://example.org/x")
|
||||
assert page.title == "Page Title"
|
||||
assert "Hello world" in page.text
|
||||
assert "var x" not in page.text
|
||||
|
||||
|
||||
def test_resolve_search_adapter_prefers_tavily(tmp_path) -> None:
|
||||
from researchx.adapters.duckduckgo import resolve_search_adapter
|
||||
|
||||
keyed = _secrets(tmp_path, {"TAVILY_API_KEY": "t-key"})
|
||||
adapter, name = resolve_search_adapter(keyed)
|
||||
assert name == "tavily"
|
||||
assert isinstance(adapter, TavilySearchAdapter)
|
||||
|
||||
keyless = _secrets(tmp_path, {})
|
||||
adapter, name = resolve_search_adapter(keyless)
|
||||
assert name == "duckduckgo-keyless"
|
||||
assert isinstance(adapter, DuckDuckGoSearchAdapter)
|
||||
@@ -0,0 +1,105 @@
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.local_storage import LocalStorageAdapter
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.engine.actor import RunActor, make_run_config
|
||||
from researchx.engine.events import EventBus
|
||||
from researchx.engine.run_state import Plan, RunState
|
||||
from researchx.engine.states import RunStateName
|
||||
from researchx.methodologies import registry
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{
|
||||
"sub_questions": ["verification passes"],
|
||||
"source_universe": ["web"],
|
||||
"outline": ["intro"],
|
||||
}
|
||||
)
|
||||
_WRITER = "# Report\n\nClaim [p-1] and claim [p-2].\n"
|
||||
|
||||
|
||||
def test_run_id_generator_is_the_storage_one() -> None:
|
||||
from researchx.engine.actor import make_run_config as mrc
|
||||
|
||||
run_id, _ = mrc(
|
||||
question="q", context="", methodology_slug="rapid-answer", overrides={}, registry=registry
|
||||
)
|
||||
assert len(run_id) == 8
|
||||
alphabet = set("abcdefghjkmnpqrstuvwxyz23456789")
|
||||
assert all(c in alphabet for c in run_id)
|
||||
|
||||
|
||||
async def test_state_file_round_trip_enables_restart(tmp_path: Path) -> None:
|
||||
run_id, config = make_run_config(
|
||||
question="How do engines resume?",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
)
|
||||
storage = LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4")
|
||||
state = RunState(
|
||||
run_id=run_id,
|
||||
question=config.question,
|
||||
methodology_slug="rapid-answer",
|
||||
state=RunStateName.SEARCHING,
|
||||
depth_cursor=1,
|
||||
depth_target=3,
|
||||
)
|
||||
await storage.write_run_artifact(run_id, "state.json", state.model_dump_json())
|
||||
|
||||
restored = RunState.model_validate_json(
|
||||
await storage.read_run_artifact(run_id, "state.json")
|
||||
)
|
||||
assert restored.depth_cursor == 1
|
||||
assert restored.depth_target == 3
|
||||
assert restored.state == RunStateName.SEARCHING
|
||||
|
||||
|
||||
async def test_actor_resumes_from_restored_state(tmp_path: Path) -> None:
|
||||
run_id, config = make_run_config(
|
||||
question="How do engines resume?",
|
||||
context="",
|
||||
methodology_slug="rapid-answer",
|
||||
overrides={},
|
||||
registry=registry,
|
||||
run_id="abcd2345",
|
||||
)
|
||||
storage = LocalStorageAdapter(base_dir=tmp_path, owner_id="u1a2b3c4")
|
||||
mid_state = RunState(
|
||||
run_id=run_id,
|
||||
question=config.question,
|
||||
methodology_slug="rapid-answer",
|
||||
state=RunStateName.AWAITING_APPROVAL,
|
||||
depth_cursor=0,
|
||||
depth_target=1,
|
||||
plan=Plan(sub_questions=["verification passes"], source_universe=["web"], outline=["intro"]),
|
||||
)
|
||||
await storage.write_run_artifact(run_id, "state.json", mid_state.model_dump_json())
|
||||
|
||||
model = FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER, model="fake-writer"),
|
||||
]
|
||||
)
|
||||
actor = RunActor(
|
||||
run_config=config,
|
||||
model=model,
|
||||
search=FakeSearchAdapter(CORPUS),
|
||||
storage=storage,
|
||||
registry=registry,
|
||||
event_bus=EventBus(),
|
||||
)
|
||||
restored = RunState.model_validate_json(
|
||||
await storage.read_run_artifact(run_id, "state.json")
|
||||
)
|
||||
actor.state = restored
|
||||
await actor.approve()
|
||||
state = await actor.run()
|
||||
assert state.state == RunStateName.DONE
|
||||
assert state.plan is not None
|
||||
assert state.plan.sub_questions == ["verification passes"]
|
||||
@@ -0,0 +1,96 @@
|
||||
from datetime import UTC, datetime
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from researchx.config.dials import Dials
|
||||
from researchx.config.run_config import (
|
||||
RunConfig,
|
||||
UnknownOverrideError,
|
||||
new_run_id,
|
||||
resolve_dials,
|
||||
)
|
||||
from researchx.methodologies import registry
|
||||
|
||||
|
||||
def test_resolve_applies_overrides_on_top_of_bundle() -> None:
|
||||
resolved = resolve_dials("deep-report", {"depth": 1, "checkpoints": False}, registry)
|
||||
assert resolved.depth == 1
|
||||
assert resolved.checkpoints is False
|
||||
assert resolved.breadth == 2
|
||||
assert resolved.verification_passes == 2
|
||||
|
||||
|
||||
def test_resolve_empty_overrides_returns_bundle() -> None:
|
||||
resolved = resolve_dials("rapid-answer", {}, registry)
|
||||
assert resolved == registry.get("rapid-answer").dials
|
||||
|
||||
|
||||
def test_resolve_rejects_unknown_override_keys() -> None:
|
||||
with pytest.raises(UnknownOverrideError):
|
||||
resolve_dials("rapid-answer", {"temperature": 0.7}, registry)
|
||||
|
||||
|
||||
def test_resolve_rejects_invalid_override_values() -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
resolve_dials("rapid-answer", {"depth": 9}, registry)
|
||||
with pytest.raises(ValidationError):
|
||||
resolve_dials("deep-report", {"synthesis_style": "poetic"}, registry)
|
||||
|
||||
|
||||
def test_resolve_does_not_mutate_registry_bundle() -> None:
|
||||
resolve_dials("deep-report", {"depth": 1}, registry)
|
||||
assert registry.get("deep-report").dials.depth == 3
|
||||
|
||||
|
||||
def _run_config(**overrides: object) -> RunConfig:
|
||||
return RunConfig(
|
||||
run_id=new_run_id(),
|
||||
methodology_slug="deep-report",
|
||||
question="What changed in the market?",
|
||||
context="Q3 earnings season",
|
||||
resolved_dials=resolve_dials("deep-report", overrides, registry),
|
||||
created_at=datetime.now(UTC),
|
||||
overrides=dict(overrides),
|
||||
)
|
||||
|
||||
|
||||
def test_run_config_accepts_valid_id() -> None:
|
||||
config = _run_config(depth=2)
|
||||
assert config.run_id == "abcd1234" or len(config.run_id) == 8
|
||||
|
||||
|
||||
def test_run_config_rejects_invalid_run_id() -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
RunConfig(
|
||||
run_id="short",
|
||||
methodology_slug="rapid-answer",
|
||||
question="q",
|
||||
resolved_dials=Dials(),
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
|
||||
|
||||
def test_run_config_rejects_empty_question() -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
RunConfig(
|
||||
run_id=new_run_id(),
|
||||
methodology_slug="rapid-answer",
|
||||
question="",
|
||||
resolved_dials=Dials(),
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
|
||||
|
||||
def test_run_config_snapshot_round_trips_json() -> None:
|
||||
config = _run_config(depth=2, checkpoints=False)
|
||||
restored = RunConfig.model_validate_json(config.model_dump_json())
|
||||
assert restored == config
|
||||
assert restored.resolved_dials.depth == 2
|
||||
assert restored.overrides == {"depth": 2, "checkpoints": False}
|
||||
|
||||
|
||||
def test_run_config_is_immutable() -> None:
|
||||
config = _run_config()
|
||||
with pytest.raises(ValidationError):
|
||||
config.question = "different"
|
||||
@@ -0,0 +1,46 @@
|
||||
from pathlib import Path
|
||||
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider, SecretsProvider
|
||||
|
||||
|
||||
def test_secrets_provider_conformance(tmp_path: Path) -> None:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("OLLAMA_API_KEY=test-key-value\n", encoding="utf-8")
|
||||
provider = EnvFileSecretsProvider(paths=[env_file])
|
||||
assert isinstance(provider, SecretsProvider)
|
||||
assert provider.resolve("OLLAMA_API_KEY") == "test-key-value"
|
||||
assert provider.resolve("MISSING_VAR") is None
|
||||
|
||||
|
||||
def test_secrets_later_file_wins(tmp_path: Path) -> None:
|
||||
first = tmp_path / ".env"
|
||||
second = tmp_path / ".env.secrets"
|
||||
first.write_text("TAVILY_API_KEY=first-key\nOLLAMA_API_KEY=ollama-first\n", encoding="utf-8")
|
||||
second.write_text("OLLAMA_API_KEY=ollama-second\n", encoding="utf-8")
|
||||
provider = EnvFileSecretsProvider(paths=[first, second])
|
||||
assert provider.resolve("TAVILY_API_KEY") == "first-key"
|
||||
assert provider.resolve("OLLAMA_API_KEY") == "ollama-second"
|
||||
|
||||
|
||||
def test_secrets_parse_skips_comments_and_blanks(tmp_path: Path) -> None:
|
||||
env_file = tmp_path / ".env"
|
||||
env_file.write_text(
|
||||
"# comment line\n\nGITEA_TOKEN='quoted-token'\nBROKEN_LINE_NO_EQUALS\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
provider = EnvFileSecretsProvider(paths=[env_file])
|
||||
assert provider.resolve("GITEA_TOKEN") == "quoted-token"
|
||||
assert provider.resolve("BROKEN_LINE_NO_EQUALS") is None
|
||||
|
||||
|
||||
def test_secrets_repr_never_leaks_values(tmp_path: Path) -> None:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("OLLAMA_API_KEY=super-secret-value-123\n", encoding="utf-8")
|
||||
provider = EnvFileSecretsProvider(paths=[env_file])
|
||||
assert "super-secret-value-123" not in repr(provider)
|
||||
assert "super-secret-value-123" not in str(provider)
|
||||
|
||||
|
||||
def test_secrets_missing_file_ignored(tmp_path: Path) -> None:
|
||||
provider = EnvFileSecretsProvider(paths=[tmp_path / ".env.absent"])
|
||||
assert provider.resolve("ANY_KEY") is None
|
||||
@@ -0,0 +1,651 @@
|
||||
"""Settings + model-key routes + factory + config-status suite (STEP 2).
|
||||
|
||||
Covers T-2.4 (settings/key routes), T-2.5 (per-run model factory
|
||||
chain: user → server → keyless), and T-2.6 (config status rework —
|
||||
P-1 closure: factory-resolved probe for the CURRENT user, zero runs
|
||||
required, no cross-user leak).
|
||||
|
||||
Run under the ``auth_enabled=True`` harness (two users prove per-user
|
||||
isolation) plus direct factory tests against fake key stores.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from researchx.adapters.fakes import CORPUS, FakeModelAdapter, FakeSearchAdapter
|
||||
from researchx.adapters.model_factory import ModelAdapterFactory
|
||||
from researchx.adapters.secrets import EnvFileSecretsProvider
|
||||
from researchx.adapters.types import ChatResult
|
||||
from researchx.api.app import create_app
|
||||
from researchx.auth.guard import CSRF_HEADER
|
||||
from researchx.auth.key_store import KEK_NAME, KeyStore
|
||||
from researchx.auth.models import UserPublic
|
||||
from researchx.auth.settings import UserSettings
|
||||
|
||||
_PLAN = json.dumps(
|
||||
{"sub_questions": ["verification passes"], "source_universe": ["web"], "outline": ["intro"]}
|
||||
)
|
||||
_WRITER = "# Report\n\nClaim [p-1] and claim [p-2].\n"
|
||||
|
||||
_ADMIN_EMAIL = "admin@example.org"
|
||||
_ADMIN_PASSWORD = "correct-horse-battery"
|
||||
_SECOND_EMAIL = "second@example.org"
|
||||
_SECOND_PASSWORD = "another-long-password"
|
||||
|
||||
_HEADERS = {CSRF_HEADER: "1"}
|
||||
|
||||
_KEK = base64.urlsafe_b64encode(os.urandom(32)).decode()
|
||||
|
||||
_FAKE_KEY = "sk-user-tier-key-9999"
|
||||
|
||||
|
||||
def _env_file(tmp_path: Path, values: dict[str, str]) -> Path:
|
||||
env_file = tmp_path / ".env.secrets"
|
||||
env_file.write_text("".join(f"{k}={v}\n" for k, v in values.items()), encoding="utf-8")
|
||||
return env_file
|
||||
|
||||
|
||||
def _user(user_id: str = "abcd2345") -> UserPublic:
|
||||
from datetime import UTC, datetime
|
||||
|
||||
return UserPublic(
|
||||
user_id=user_id,
|
||||
email="user@example.org",
|
||||
display_name="User",
|
||||
role="user",
|
||||
active=True,
|
||||
created_at=datetime.now(UTC),
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------- harness
|
||||
|
||||
|
||||
def _app(
|
||||
tmp_path: Path,
|
||||
*,
|
||||
secrets_values: dict[str, str] | None = None,
|
||||
with_factory: bool = True,
|
||||
):
|
||||
"""create_app under auth_enabled=True with live per-user stores.
|
||||
|
||||
The model adapter stays the zero-key fake (the fake's chat is what
|
||||
runs execute against); the FACTORY is wired so the tier chain and
|
||||
config-status rework are exercised end to end. The factory's bases
|
||||
point at an unroutable host so config-status probes fail fast and
|
||||
hermetically (no outbound HTTP from the unit tier — the probe
|
||||
contract has its own monkeypatched tests below).
|
||||
"""
|
||||
secrets = EnvFileSecretsProvider(
|
||||
paths=[_env_file(tmp_path, secrets_values or {})]
|
||||
)
|
||||
return create_app(
|
||||
storage_dir=tmp_path,
|
||||
model_adapter=FakeModelAdapter(
|
||||
canned=[
|
||||
ChatResult(content=_PLAN, model="fake-planner"),
|
||||
ChatResult(content=_WRITER, model="fake-writer"),
|
||||
]
|
||||
),
|
||||
search_adapter=FakeSearchAdapter(CORPUS),
|
||||
auth_enabled=True,
|
||||
data_dir=tmp_path,
|
||||
secrets_provider=secrets,
|
||||
model_factory=ModelAdapterFactory(
|
||||
secrets,
|
||||
KeyStore(secrets, tmp_path),
|
||||
cloud_base_url="http://127.0.0.1:1/v1",
|
||||
local_base_url="http://127.0.0.1:1/v1",
|
||||
)
|
||||
if with_factory
|
||||
else None,
|
||||
)
|
||||
|
||||
|
||||
async def _client(tmp_path: Path, **kwargs) -> httpx.AsyncClient:
|
||||
return httpx.AsyncClient(
|
||||
transport=httpx.ASGITransport(app=_app(tmp_path, **kwargs)),
|
||||
base_url="http://testserver",
|
||||
)
|
||||
|
||||
|
||||
async def _register_and_login(client: httpx.AsyncClient) -> dict[str, str]:
|
||||
await client.post(
|
||||
"/api/auth/register",
|
||||
json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD, "display_name": "Admin"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
login = await client.post(
|
||||
"/api/auth/login", json={"email": _ADMIN_EMAIL, "password": _ADMIN_PASSWORD}, headers=_HEADERS
|
||||
)
|
||||
return login.json()
|
||||
|
||||
|
||||
# --------------------------------------------------- settings CRUD routes
|
||||
|
||||
|
||||
async def test_settings_get_returns_defaults(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
me = await _register_and_login(client)
|
||||
response = await client.get("/api/config/settings")
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert body["planner_model"]
|
||||
assert body["writer_model"]
|
||||
assert body["methodology_defaults"] == {}
|
||||
assert body["appearance"] == "system"
|
||||
# never leaks the user's id/email into the settings payload
|
||||
assert "user_id" not in body
|
||||
assert me["email"] not in json.dumps(body)
|
||||
|
||||
|
||||
async def test_settings_put_round_trip(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
put = await client.put(
|
||||
"/api/config/settings",
|
||||
json={"planner_model": "qwen-max", "appearance": "dark"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert put.status_code == 200
|
||||
assert put.json()["planner_model"] == "qwen-max"
|
||||
assert put.json()["appearance"] == "dark"
|
||||
# writer_model untouched -> default preserved
|
||||
assert put.json()["writer_model"]
|
||||
|
||||
got = await client.get("/api/config/settings")
|
||||
assert got.json()["planner_model"] == "qwen-max"
|
||||
assert got.json()["appearance"] == "dark"
|
||||
|
||||
|
||||
async def test_settings_put_partial_preserves(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
await client.put(
|
||||
"/api/config/settings", json={"planner_model": "p-one"}, headers=_HEADERS
|
||||
)
|
||||
put = await client.put(
|
||||
"/api/config/settings", json={"writer_model": "w-two"}, headers=_HEADERS
|
||||
)
|
||||
assert put.json()["planner_model"] == "p-one"
|
||||
assert put.json()["writer_model"] == "w-two"
|
||||
|
||||
|
||||
async def test_settings_put_validates_body(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
response = await client.put(
|
||||
"/api/config/settings", json={"planner_model": ""}, headers=_HEADERS
|
||||
)
|
||||
assert response.status_code == 422
|
||||
|
||||
|
||||
async def test_settings_put_requires_csrf(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
response = await client.put("/api/config/settings", json={"appearance": "dark"})
|
||||
assert response.status_code == 403
|
||||
|
||||
|
||||
async def test_settings_put_explicit_null_treated_as_absent(tmp_path: Path) -> None:
|
||||
"""P2-verify P0 regression: an explicit null field must be treated
|
||||
as absent (partial-merge semantics) — never 200-then-poison the
|
||||
on-disk payload into a corrupt file that 500s every later read."""
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
put = await client.put(
|
||||
"/api/config/settings", json={"planner_model": None}, headers=_HEADERS
|
||||
)
|
||||
assert put.status_code == 200
|
||||
body = put.json()
|
||||
assert body["planner_model"] # default preserved, not null
|
||||
|
||||
# Every subsequent read + write + run creation stays healthy.
|
||||
got = await client.get("/api/config/settings")
|
||||
assert got.status_code == 200
|
||||
assert got.json()["planner_model"]
|
||||
again = await client.put(
|
||||
"/api/config/settings", json={"planner_model": "qwen-max"}, headers=_HEADERS
|
||||
)
|
||||
assert again.status_code == 200
|
||||
assert again.json()["planner_model"] == "qwen-max"
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}, headers=_HEADERS
|
||||
)
|
||||
assert created.status_code == 201
|
||||
|
||||
|
||||
async def test_settings_isolated_per_user(tmp_path: Path) -> None:
|
||||
app = _app(tmp_path)
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _register_and_login(admin)
|
||||
await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
await admin.put(
|
||||
"/api/config/settings", json={"appearance": "dark"}, headers=_HEADERS
|
||||
)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
await second.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
got = await second.get("/api/config/settings")
|
||||
assert got.json()["appearance"] == "system" # no bleed
|
||||
|
||||
|
||||
# --------------------------------------------------- model-key routes
|
||||
|
||||
|
||||
async def test_model_key_put_masked_round_trip(tmp_path: Path) -> None:
|
||||
"""PUT the raw key -> 200 with masked status only; the raw key
|
||||
NEVER appears in any response (REQ-73)."""
|
||||
async with await _client(tmp_path, secrets_values={KEK_NAME: _KEK}) as client:
|
||||
me = await _register_and_login(client)
|
||||
put = await client.put(
|
||||
"/api/config/settings/model-key",
|
||||
json={"key": _FAKE_KEY},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert put.status_code == 200
|
||||
body = put.json()
|
||||
assert body["configured"] is True
|
||||
assert body["masked"] is not None
|
||||
assert body["masked"].endswith(_FAKE_KEY[-4:])
|
||||
assert _FAKE_KEY not in put.text
|
||||
|
||||
status = await client.get("/api/config/settings/model-key")
|
||||
assert status.json()["configured"] is True
|
||||
assert _FAKE_KEY not in status.text
|
||||
assert me["user_id"] not in status.text
|
||||
|
||||
|
||||
async def test_model_key_put_disabled_without_kek_honest_409(tmp_path: Path) -> None:
|
||||
"""No KEK -> 409 with the honest {key_store: disabled} payload —
|
||||
never a silent acceptance."""
|
||||
async with await _client(tmp_path) as client: # no KEK in secrets
|
||||
await _register_and_login(client)
|
||||
put = await client.put(
|
||||
"/api/config/settings/model-key",
|
||||
json={"key": "some-key-value"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
assert put.status_code == 409
|
||||
detail = put.json()["detail"]
|
||||
assert detail["key_store"] == "disabled"
|
||||
assert "RESEARCHX_KEK" in detail["reason"]
|
||||
# Nothing was written to disk.
|
||||
user_dir = tmp_path / "keys"
|
||||
assert not user_dir.exists() or not any(user_dir.iterdir())
|
||||
|
||||
|
||||
async def test_model_key_get_status_disabled_says_so(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
status = await client.get("/api/config/settings/model-key")
|
||||
assert status.status_code == 200
|
||||
body = status.json()
|
||||
assert body["disabled"] is True
|
||||
assert "RESEARCHX_KEK" in body["reason"]
|
||||
assert body["configured"] is False
|
||||
|
||||
|
||||
async def test_model_key_put_requires_csrf(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path, secrets_values={KEK_NAME: _KEK}) as client:
|
||||
await _register_and_login(client)
|
||||
response = await client.put(
|
||||
"/api/config/settings/model-key", json={"key": "no-header-key"}
|
||||
)
|
||||
assert response.status_code == 403
|
||||
|
||||
|
||||
async def test_model_key_never_in_config_or_settings_responses(tmp_path: Path) -> None:
|
||||
"""The raw key must not surface in ANY authenticated response."""
|
||||
async with await _client(tmp_path, secrets_values={KEK_NAME: _KEK}) as client:
|
||||
await _register_and_login(client)
|
||||
await client.put(
|
||||
"/api/config/settings/model-key", json={"key": _FAKE_KEY}, headers=_HEADERS
|
||||
)
|
||||
for path in [
|
||||
"/api/config",
|
||||
"/api/config/settings",
|
||||
"/api/config/settings/model-key",
|
||||
"/api/auth/me",
|
||||
"/api/methodologies",
|
||||
]:
|
||||
response = await client.get(path)
|
||||
assert _FAKE_KEY not in response.text, f"raw key leaked via {path}"
|
||||
|
||||
|
||||
# --------------------------------------------------------- factory chain
|
||||
|
||||
|
||||
class _FakeKeyStore:
|
||||
"""Test double for KeyStore: enabled with a known key or disabled."""
|
||||
|
||||
def __init__(self, key: str | None = None, enabled: bool = True) -> None:
|
||||
self._key = key
|
||||
self._enabled_flag = enabled
|
||||
|
||||
@property
|
||||
def enabled(self) -> bool:
|
||||
return self._enabled_flag
|
||||
|
||||
async def get_key(self, user_id: str) -> str | None:
|
||||
del user_id
|
||||
return self._key
|
||||
|
||||
async def masked_status(self, user_id: str) -> dict[str, object]:
|
||||
del user_id
|
||||
return {
|
||||
"configured": self._key is not None,
|
||||
"masked": f"…{self._key[-4:]}" if self._key else None,
|
||||
"disabled": not self._enabled_flag,
|
||||
}
|
||||
|
||||
|
||||
def _factory(tmp_path: Path, secrets_values: dict[str, str], key_store: object) -> ModelAdapterFactory:
|
||||
secrets = EnvFileSecretsProvider(paths=[_env_file(tmp_path, secrets_values)])
|
||||
return ModelAdapterFactory(
|
||||
secrets,
|
||||
key_store, # type: ignore[arg-type]
|
||||
cloud_base_url="http://127.0.0.1:1/v1", # unroutable: hermetic unit tier
|
||||
local_base_url="http://127.0.0.1:1/v1",
|
||||
)
|
||||
|
||||
|
||||
async def test_factory_user_tier_when_key_present(tmp_path: Path) -> None:
|
||||
factory = _factory(
|
||||
tmp_path,
|
||||
{"OLLAMA_API_KEY": "server-key-value"},
|
||||
_FakeKeyStore(key=_FAKE_KEY),
|
||||
)
|
||||
adapter, tier = await factory.resolve(_user(), UserSettings())
|
||||
assert tier == "user"
|
||||
assert adapter.roles().planner == UserSettings().planner_model
|
||||
|
||||
|
||||
async def test_factory_server_tier_when_no_user_key(tmp_path: Path) -> None:
|
||||
factory = _factory(
|
||||
tmp_path,
|
||||
{"OLLAMA_API_KEY": "server-key-value"},
|
||||
_FakeKeyStore(key=None),
|
||||
)
|
||||
_adapter, tier = await factory.resolve(_user(), UserSettings())
|
||||
assert tier == "server"
|
||||
|
||||
|
||||
async def test_factory_keyless_when_neither(tmp_path: Path) -> None:
|
||||
factory = _factory(tmp_path, {}, _FakeKeyStore(key=None))
|
||||
_adapter, tier = await factory.resolve(_user(), UserSettings())
|
||||
assert tier == "keyless"
|
||||
|
||||
|
||||
async def test_factory_keyless_when_store_disabled(tmp_path: Path) -> None:
|
||||
factory = _factory(
|
||||
tmp_path,
|
||||
{"OLLAMA_API_KEY": "server-key-value"},
|
||||
_FakeKeyStore(key=None, enabled=False),
|
||||
)
|
||||
_adapter, tier = await factory.resolve(_user(), UserSettings())
|
||||
assert tier == "server" # chain proceeds to server when key store disabled
|
||||
|
||||
|
||||
async def test_factory_respects_user_settings_models(tmp_path: Path) -> None:
|
||||
factory = _factory(tmp_path, {}, _FakeKeyStore(key=None))
|
||||
settings = UserSettings(planner_model="my-planner", writer_model="my-writer")
|
||||
adapter, tier = await factory.resolve(_user(), settings)
|
||||
assert tier == "keyless"
|
||||
assert adapter.roles().planner == "my-planner"
|
||||
assert adapter.roles().writer == "my-writer"
|
||||
|
||||
|
||||
async def test_factory_never_caches_adapters(tmp_path: Path) -> None:
|
||||
factory = _factory(tmp_path, {}, _FakeKeyStore(key=None))
|
||||
first, _ = await factory.resolve(_user(), UserSettings())
|
||||
second, _ = await factory.resolve(_user(), UserSettings())
|
||||
assert first is not second # one instance per run (P-2 closure)
|
||||
|
||||
|
||||
async def test_factory_user_key_precedence_over_server_secrets(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""The resolved user key must be the Authorization the adapter sends
|
||||
— not the server env-file key."""
|
||||
seen_headers: dict[str, str] = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen_headers.update(dict(request.headers))
|
||||
return httpx.Response(
|
||||
200, json={"choices": [{"message": {"content": '{"ok": true}'}}]}
|
||||
)
|
||||
|
||||
transport = httpx.MockTransport(handler)
|
||||
original = httpx.AsyncClient
|
||||
monkeypatch.setattr(
|
||||
httpx, "AsyncClient", lambda **kwargs: original(transport=transport, **kwargs)
|
||||
)
|
||||
|
||||
factory = _factory(
|
||||
tmp_path,
|
||||
{"OLLAMA_API_KEY": "server-key-value"},
|
||||
_FakeKeyStore(key=_FAKE_KEY),
|
||||
)
|
||||
adapter, tier = await factory.resolve(_user(), UserSettings())
|
||||
assert tier == "user"
|
||||
await adapter.chat(
|
||||
[__import__("researchx.adapters.types", fromlist=["ModelMessage"]).ModelMessage(
|
||||
role="user", content="hi"
|
||||
)],
|
||||
model="w",
|
||||
)
|
||||
assert seen_headers["authorization"] == f"Bearer {_FAKE_KEY}"
|
||||
|
||||
|
||||
# ------------------------------------------------- config status rework
|
||||
|
||||
|
||||
async def test_config_status_tier_per_user_no_cross_leak(tmp_path: Path) -> None:
|
||||
"""Config status shows the CURRENT user's tier: a keyed user shows
|
||||
'user', a keyless second user shows 'server' — zero runs needed
|
||||
(P-1 closure), zero cross-user bleed."""
|
||||
app = _app(tmp_path, secrets_values={KEK_NAME: _KEK, "OLLAMA_API_KEY": "server-key-value"})
|
||||
transport = httpx.ASGITransport(app=app)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as admin:
|
||||
await _register_and_login(admin)
|
||||
await admin.put(
|
||||
"/api/config/settings/model-key",
|
||||
json={"key": _FAKE_KEY},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
config = await admin.get("/api/config")
|
||||
assert config.status_code == 200
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] == "user"
|
||||
assert body["key_store"] == "enabled"
|
||||
assert body["key_status"]["configured"] is True
|
||||
assert body["key_status"]["masked"].endswith(_FAKE_KEY[-4:])
|
||||
assert body["cost_counters"] == {"runs_created": 0, "runs_completed": 0}
|
||||
|
||||
await admin.post(
|
||||
"/api/admin/users",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD, "role": "user"},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://testserver") as second:
|
||||
await second.post(
|
||||
"/api/auth/login",
|
||||
json={"email": _SECOND_EMAIL, "password": _SECOND_PASSWORD},
|
||||
headers=_HEADERS,
|
||||
)
|
||||
config = await second.get("/api/config")
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] == "server" # no user key of their own
|
||||
assert body["key_store"] == "enabled" # the STORE is enabled; the user has no key
|
||||
assert body["key_status"]["configured"] is False
|
||||
assert _FAKE_KEY not in config.text
|
||||
assert _FAKE_KEY[-4:] not in config.text
|
||||
# Counters owner-keyed: the second user sees zero, not the admin's runs.
|
||||
assert body["cost_counters"] == {"runs_created": 0, "runs_completed": 0}
|
||||
|
||||
|
||||
async def test_config_status_keyless_when_no_keys(tmp_path: Path) -> None:
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
config = await client.get("/api/config")
|
||||
body = config.json()
|
||||
assert body["resolved_tier"] == "keyless"
|
||||
assert body["key_store"] == "disabled"
|
||||
assert body["key_status"]["disabled"] is True
|
||||
assert body["model_adapter"] == "OllamaModelAdapter" # factory-constructed
|
||||
|
||||
|
||||
async def test_config_status_works_with_zero_runs(tmp_path: Path) -> None:
|
||||
"""P-1: correct config status with zero runs in the registry."""
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
response = await client.get("/api/config")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["resolved_tier"] in ("user", "server", "keyless")
|
||||
assert response.json()["cost_counters"]["runs_created"] == 0
|
||||
|
||||
|
||||
async def test_config_status_probe_cached_and_error_safe(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""Probe semantics pinned hermetically: a successful probe is
|
||||
cached in-process (second call does not re-probe), and an
|
||||
AdapterError probe yields probe: None (never a 500)."""
|
||||
import researchx.api.app as app_module
|
||||
|
||||
calls: list[int] = []
|
||||
probe_payload = {
|
||||
"endpoint": "http://127.0.0.1:1/v1",
|
||||
"models": ["m"],
|
||||
"context_window": 8192,
|
||||
"embedding_batch": 8,
|
||||
"structured_outputs_supported": True,
|
||||
}
|
||||
from researchx.adapters.probe import CapabilityProbe
|
||||
|
||||
fake_probe = CapabilityProbe(
|
||||
endpoint="http://127.0.0.1:1/v1",
|
||||
models=["m"],
|
||||
context_window=8192,
|
||||
embedding_batch=8,
|
||||
structured_outputs_supported=True,
|
||||
)
|
||||
|
||||
async def _fake_probe(adapter):
|
||||
calls.append(1)
|
||||
return fake_probe
|
||||
|
||||
async def _failing_probe(adapter):
|
||||
from researchx.adapters.types import AdapterError
|
||||
|
||||
raise AdapterError("unreachable")
|
||||
|
||||
monkeypatch.setattr(app_module, "probe_model_adapter", _fake_probe)
|
||||
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
first = await client.get("/api/config")
|
||||
assert first.status_code == 200
|
||||
assert first.json()["probe"]["endpoint"] == probe_payload["endpoint"]
|
||||
assert len(calls) == 1
|
||||
second = await client.get("/api/config")
|
||||
assert len(calls) == 1 # served from the (user, tier, fingerprint) cache
|
||||
assert second.json()["probe"]["endpoint"] == probe_payload["endpoint"]
|
||||
|
||||
# Fresh app: a failing probe yields None, cached — never a 500, and
|
||||
# the route stays healthy on repeat calls.
|
||||
monkeypatch.setattr(app_module, "probe_model_adapter", _failing_probe)
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
first = await client.get("/api/config")
|
||||
assert first.status_code == 200
|
||||
assert first.json()["probe"] is None
|
||||
second = await client.get("/api/config")
|
||||
assert second.status_code == 200
|
||||
assert second.json()["probe"] is None
|
||||
|
||||
|
||||
async def test_config_status_uses_factory_adapter_not_fixed_one(tmp_path: Path) -> None:
|
||||
"""The rework: model_adapter names the FACTORY-constructed adapter
|
||||
(OllamaModelAdapter), not the fixed injected fake."""
|
||||
async with await _client(tmp_path) as client:
|
||||
await _register_and_login(client)
|
||||
config = await client.get("/api/config")
|
||||
assert config.json()["model_adapter"] == "OllamaModelAdapter"
|
||||
|
||||
|
||||
async def test_config_status_without_factory_names_fixed_adapter(tmp_path: Path) -> None:
|
||||
"""Back-compat: with no factory wired (pure test harness), the
|
||||
config status still answers with the injected adapter's name."""
|
||||
async with await _client(tmp_path, with_factory=False) as client:
|
||||
await _register_and_login(client)
|
||||
config = await client.get("/api/config")
|
||||
body = config.json()
|
||||
assert body["model_adapter"] == "FakeModelAdapter"
|
||||
assert body["resolved_tier"] == "keyless"
|
||||
|
||||
|
||||
# ----------------------------------------------- create_run tier stamping
|
||||
|
||||
|
||||
async def test_create_run_stamps_credential_tier(tmp_path: Path) -> None:
|
||||
"""create_run resolves the factory and stamps state.credential_tier
|
||||
(REQ-74); the engine contracts are otherwise unchanged."""
|
||||
async with await _client(tmp_path) as client:
|
||||
me = await _register_and_login(client)
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}, headers=_HEADERS
|
||||
)
|
||||
assert created.status_code == 201
|
||||
run_id = created.json()["id"]
|
||||
|
||||
state_path = tmp_path / "runs" / me["user_id"] / run_id / "state.json"
|
||||
import asyncio
|
||||
|
||||
for _ in range(200):
|
||||
if state_path.exists():
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
state = json.loads(state_path.read_text(encoding="utf-8"))
|
||||
assert state["credential_tier"] == "keyless"
|
||||
assert state["owner_id"] == me["user_id"]
|
||||
|
||||
|
||||
async def test_create_run_user_tier_uses_user_key(tmp_path: Path) -> None:
|
||||
"""A keyed user's run executes with THEIR key through the factory
|
||||
chain (fake model still serves the run — zero-key CI)."""
|
||||
async with await _client(tmp_path, secrets_values={KEK_NAME: _KEK}) as client:
|
||||
me = await _register_and_login(client)
|
||||
await client.put(
|
||||
"/api/config/settings/model-key", json={"key": _FAKE_KEY}, headers=_HEADERS
|
||||
)
|
||||
created = await client.post(
|
||||
"/api/runs", json={"question": "q", "methodology": "rapid-answer"}, headers=_HEADERS
|
||||
)
|
||||
assert created.status_code == 201
|
||||
run_id = created.json()["id"]
|
||||
|
||||
import asyncio
|
||||
|
||||
state_path = tmp_path / "runs" / me["user_id"] / run_id / "state.json"
|
||||
for _ in range(200):
|
||||
if state_path.exists():
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
state = json.loads(state_path.read_text(encoding="utf-8"))
|
||||
assert state["credential_tier"] == "user"
|
||||
@@ -0,0 +1,5 @@
|
||||
import researchx
|
||||
|
||||
|
||||
def test_package_imports() -> None:
|
||||
assert researchx.__name__ == "researchx"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user