Compare commits

...

1 Commits

Author SHA1 Message Date
CIAgent 0516b1e438 docs(P00): complete payment-engine phase 0 — v0.8.0
CI / node (push) Successful in 1m0s
CI / python (push) Successful in 2m12s
Release / release (push) Successful in 3m56s
Specify (D-074 scope lock: REQ-F-018 activated as REQ-9-001..005),
clarify (D-075..D-084), research (R-33..R-54 — Stripe protocol facts
web-verified; in-tree integration probes), plan (6 vertical-slice phases,
waves binding, must-haves falsifiable), grill (G-301..G-310 binding
decisions applied), MVP/UX gate PASS. Tags on the v0.8.x line; branch
milestone/v0.9-payment-engine.

---ci---
phase: 0
milestone: v0.9
status: complete
---/ci---
2026-09-29 17:23:40 +00:00
10 changed files with 557 additions and 50 deletions
+16
View File
@@ -43,6 +43,22 @@ Nextcraft is a TypeScript monorepo (pnpm workspaces + turborepo) with a Next.js
49. **D-052 tooling conclusions** — coverage: pytest-cov (Python) + c8 wrapped around existing `tsx --test` (no runner swap, no vitest); markers `fast`/`real_server`/`userns` under `--strict-markers`, unmarked defaults to fast, fast run = `-m "not real_server and not userns"`; pytest-timeout `signal` method, 60s global / 180s real-server+userns via collection hook (D-054); CLI fresh-clone E2E env-gated (`CLI_E2E=1`) into `cli:test:e2e` with a 900s budget (D-055); `filterwarnings = ["error", <narrow module+message-scoped upstream ignores>]` (D-056); mypy relaxed-first baseline (`check_untyped_defs`, `warn_unused_ignores`, no full strict) (D-058); recharts 2→3 verified by tsc + static build + existing tests (D-059); shared Python helpers in `tests/helpers/` package, fixtures stay in conftest (D-060); pyproject.toml is the single version source, `FastAPI(version=)` reads package metadata (D-061).
50. **D-062 v0.7 wave order = dependency-ordered: wiring before composition before elimination before hardening** — P1 harness wiring (coverage/timeout/markers/turbo tasks/leak fixes) MUST precede P2 suite composition because the cut/consolidate decisions are evidence-based (P1's coverage data tells P2 what to cut); P3 warning elimination is independent of P1/P2 but ordered after them to avoid rebasing `session.exec()` conversions over moved test files; P4 lint hygiene (ESLint + noqa RUF100) after P3 so the eslint-disable resolutions happen under a warning-clean tree; P5 type+dependency hardening last (mypy adoption and recharts 3 are the riskiest changes — they run against a fully-stabilized harness, and recharts gets the fresh coverage/ESLint net as its safety net); P6 final review. ESLint adoption (D-048) rides P4 because the orphaned eslint-disable comments only become meaningful once a config exists.
### v0.9 Payment Engine Architecture (Phase 0 RESEARCH — Payment Engine, D-075..D-084 binding)
**Module:** `ai_service/billing/` (7th domain module, D-031 pattern — extend ai-service, no new apps). Boundary rule (family discipline): `billing/` never imports `agents/` or `api/`; it imports `config` + its own `base.py` only. Files: `base.py` (PaymentProvider protocol + pydantic contracts + normalized event names), `store.py` (7th D-027 SQLite store), `mock.py` (deterministic provider, Stripe-shaped ids/signatures per R-36/R-48), `stripe.py` (Stripe-compatible REST impl — form-encoded bodies, Basic auth, Idempotency-Key per R-33/R-49), `factory.py` (env-gated selection, UnknownPaymentProviderError → G-11 loud mock fallback), `catalog.py` (seeded catalog constant + idempotent seed).
**Provider protocol (D-075):** `create_checkout_session(spec) -> CheckoutSession`, `retrieve_session(id)`, `cancel_subscription(id, *, at_period_end)`, `update_subscription(id, new_price_ref, ...)` (resolves subscription_item id inside the provider — R-47 dual-charge guard), `verify_webhook(raw_body: bytes, signature_header: str, *, now=None) -> VerifiedEvent` (HMAC-SHA256 over `"{t}.{raw_body}"`, compare_digest, all-v1 iteration for secret rolls, ±300s replay window, ignore non-v1 schemes — R-38..R-40), `normalize_event(provider_event) -> CanonicalBillingEvent` (R-43 mapping).
**Store (D-080, 7 tables):** `packages` (one-time: posts_limit, price_cents, duration_days), `plans` (recurring: interval, price_cents), `purchases` (posts_limit/posts_used/expires_at snapshot + price snapshot — D-078), `subscriptions` (state machine: trialing/active/past_due/cancelled/expired/upgraded, expires_at, provider ref), `subscription_events` (provider event log, verbatim ids, UniqueConstraint event_id — dedupe), `checkout_sessions` (client_reference_id reconciliation), `billing_events` (full audit: provider events + manual admin actions with provenance). Same SQLite file (`Settings().db_path`), WAL pragmas, Protocol + SQLiteBillingStore pair, main.py lifespan wiring with state-injection override.
**Routes:** `api/billing.py` (`/v1/billing/catalog` public GET; `/v1/billing/webhook` public POST — raw-body-first HMAC verify, then parse, then idempotent process, 200-fast) + `api/billing_admin.py` (`/v1/billing/admin/*` — require_admin dep verbatim R-51: checkout on behalf of employer (D-076), entitlement view, manual grant/revoke, event log query) + mock-only confirm route (D-082, mounted only when provider=mock). Entitlement enforcement composes into `create_posting` AFTER require_admin: billing store's `consume_posting(employer_id)` → domain error → route translates to `HTTPException(402, detail={reason, posts_used, posts_limit, upgrade_cta})` (dict-detail precedent = require_verified_age).
**Settings (R-53):** `payment_provider: str = "mock"`, `payment_api_key: SecretStr`, `webhook_secret: str = ""` (unset → per-boot random → webhooks fail closed — correct for a payment surface). Env: `AI_PAYMENT_PROVIDER`, `AI_PAYMENT_API_KEY`, `AI_WEBHOOK_SECRET`.
**Web (REQ-9-002):** pricing page → server shell (metadata retained, A-411/pins) + `PricingClient` component fetching `/v1/billing/catalog` (A-406 board-client pattern, AbortController + honest loading/error/empty states); `engine-client.ts` gains `listBillingCatalog()` + `PaymentRequiredError` (402 dict-detail discrimination); types in `packages/types/billing.ts` dual-schema-pinned in pins.test.ts. Admin entitlement view: moderation surface gains per-employer entitlement state (client fetch, same pattern).
**Money (D-081):** integer cents end-to-end; pydantic `int ge=0`; TS `number` documented as cents; currency `usd` fixed.
### v0.8 Distribution & Release Architecture (Phase 0 RESEARCH — Ship Anywhere, D-063..D-072 binding)
**Runner fleet topology (CT 113, 10.0.0.113):** three org-level gitea-runner containers (official image `docker.io/gitea/runner:3` — the Docker Hub `gitea/act_runner` repo is deprecated, renamed `gitea/runner`) in **sibling-container mode**: each mounts the host `/var/run/docker.sock`, has its OWN `/data` volume holding the `.runner` registration file (identity + credentials; shared state = re-registration), registers once against `https://git.coreci.dev` (Gitea 1.26.1, probe-verified) with the founder-provided org-level registration token (chat-only, reset after fleet green). Hybrid labels via `GITEA_RUNNER_LABELS`: `node` → `docker://node:24-bookworm`, `python` → `docker://python:3.11-bookworm`, `release` → `docker://node:24-bookworm`, `ubuntu-latest` → `docker://docker.gitea.com/runner-images:ubuntu-latest` (label format `name[:schema[:args]]`; a job's `runs-on` matches label names, the `docker://` image selects the job container). Per-runner `runner.capacity: 1` (default; also caps per-job docker subnet consumption) → fleet concurrency 3. Cache: each runner's built-in cache server must be pinned to a job-reachable `cache.host` + fixed published port (dockerized runners otherwise create per-job networks the cache address is unreachable in — `actions/cache` fails ETIMEDOUT); 32 GB disk guarded by `cache.repo_size_limit`, `health_check.min_free_disk_space_mb`, and a host-side `docker system prune` cron. Workspace-path alignment inside/outside the runner container is NOT needed for our jobs (no `container.bind_workdir`; `docker build` TARS the context over the API) — documented as the known sibling-mode gotcha if a future workflow bind-mounts.
+8
View File
@@ -0,0 +1,8 @@
{
"phase": 0,
"stage": "plan",
"milestone": "v0.9",
"phase_role": "pre_execution",
"attempts": 0,
"updated_at": "2026-09-29T17:52:00Z"
}
+45
View File
@@ -1,3 +1,48 @@
# Nextcraft v0.9 — GRILL.md (Adversarial Review Verdict)
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-BINDINGS (REVISE → record G-301..G-310, then SHIP) · **Confidence:** 0.85 · **Date:** 2026-09-29
## Summary
The v0.9 Payment Engine plan (5 requirement phases + final, D-074 locked) is feasible, correctly scoped, and hermetically testable — the operator-mediated checkout model is coherent end-to-end and entitlement needs no employer auth. But red-teaming against the working tree (not the plan's description of it) found 10 binding defects: a mock-complete route mounted on every default-config production box (D-082's "never on a real deployment" rationale is **false** — mock IS the default), webhook handlers that would trust forgeable event payloads (leaked secret = free entitlements), a check-then-increment race in consume_posting, event-id-only dedupe defeated by Stripe's dual invoice events, a phantom `upgraded` state with no transition rows, an un-pinned ingest bypass that would 402 the seeded corpus, a checkout route naming contradiction between REQUIREMENTS/ROADMAP and binding D-076, an admin panel requiring a browser token entry that exists nowhere in apps/web, and a stripe checkout that cannot create sessions (success_url never populated — Happy Path 5's "zero code changes" was false). Two plan citations proved wrong in-tree (the "sitemap env-gated routes" precedent does not exist; admin surface has zero admin-token plumbing). Two Gs bite inside P1's Task 1-1-02 — recording them now prevents store-schema retrofit churn after P1 ships. Nothing re-architects: every G amends an existing task, must-have, or requirement sentence; the phase/wave structure is untouched.
## Per-Axis Findings
| # | Axis | Verdict | Reason |
|---|------|---------|--------|
| 1 | Feasibility | PASS | Operator-mediated checkout coherent end-to-end; entitlement composes after require_admin with employer_id as body field; 6-phase shape matches the proven v0.6 cadence |
| 2 | Scope creep | CONCERN | REQ-9-003/ROADMAP name a public POST /v1/billing/checkout contradicting D-076 (G-307); admin panel hides a browser-token requirement (G-308); "stripe switch = zero code changes" false without success/cancel URLs (G-310) |
| 3 | Security | CONCERN | Mock-complete route unauthenticated on default deployments (G-301); webhook state resolution unspecified — payload trust = forged entitlements (G-302) |
| 4 | Correctness | CONCERN | consume_posting check-then-increment race (G-304); dual invoice events defeat event-id dedupe (G-303); `upgraded` state hole (G-305); out-of-order events against missing targets + grace timing unspecified (G-306) |
| 5 | Testability | PASS | Hermetic by construction: MockTransport byte-contracts, now-injection, direct-handler-call mock-complete — provided G-306 keeps grace lazy |
| 6 | Backward compat | PASS | Fixture strategy workable (~9 test sites grant unlimited to legacy seeded employers); pricing metadata pin survives conversion; ingest bypass unpinned → G-309 |
| 7 | Cost/benefit | PASS | D-074 stands: payments unblocks employer self-serve; no new deps (hand-rolled REST, no Stripe SDK); challenge declined |
| 8 | Plan-vs-reality | PASS (with corrections) | All cited patterns verified in-tree; two rationale citations false (G-301/G-308); suite count 674/625-fast drift noted |
## Binding Decisions (G-301..G-310 — applied to PLAN/REQUIREMENTS/ROADMAP before execute)
**G-301 — mock-complete route: admin-gate it; D-082's "never on a real deployment" rationale is false.** The default provider is mock (D-075) and the pilot IS a real deployment — `POST /v1/billing/mock/complete` mounted "only when provider=mock" is mounted on every default-config production box. An unauthenticated completion route on a payment surface is a hole no matter how unguessable `cs_mock_…` ids are. Decision: keep the provider=mock conditional mount AND compose `require_admin` on the route — the operator completing a session after out-of-band payment is exactly the honest pilot flow D-076 describes. Amends PLAN Task 3-2-03, MH-3c; D-082's rationale corrected (ship-notes record: the sitemap "env-gated routes" precedent does not exist — this is the first factory-conditional mount in-tree). Tests: no-token → 403 under mock; route 404s under stripe; E2E green with admin-gated complete. Confidence 0.85.
**G-302 — webhook events are untrusted ID references; state resolves from local rows.** If checkout.completed reads session status/amount from the event payload, a leaked AI_WEBHOOK_SECRET = free entitlements (forge status=complete, payment_status=paid for any session id). Decision: handlers treat event payloads as ID references only — the session resolves from the local checkout_sessions row (creatable only via the admin route); unknown session id → recorded as an `unmatched` billing_event, NO entitlement, 200. Purchase price/name from the stored snapshot (D-078), never the event body; renewal amounts from the local plan row. Amends Task 3-2-02. Tests: signed checkout.completed with no local row → 200, zero purchases, one unmatched event; inflated payload amount → purchase price = snapshot. Confidence 0.85.
**G-303 — grants are idempotent per session, not per event id.** R-43/D-083 accept invoice.paid AND invoice.payment_succeeded and Stripe's async path fires distinct completed events — different event ids for the same logical completion sail past event-id dedupe and double-grant. Decision: `purchases.checkout_session_id` gets a UniqueConstraint; subscription upsert keys on provider subscription id; entitlement creation is exactly-once per session regardless of event count. Amends Task 1-1-02 (schema), Task 3-2-02; extends MH-3b. Test: two signed checkout.completed events, different event ids, same session → exactly one purchase row (IntegrityError → created=False), one subscription. Confidence 0.85.
**G-304 — consume_posting is a single conditional UPDATE, not check-then-increment.** insert_application survives concurrency via UniqueConstraint; posts_used has no such backstop — two concurrent create_posting calls both read posts_used=9 and both increment. Decision: consume_posting = one statement `UPDATE purchases SET posts_used = posts_used + 1 WHERE id = <newest non-expired with remaining> AND posts_used < posts_limit` (rowcount-checked; 0 rows → PackageExhausted/NoActivePackage). Amends Task 1-1-02. Test: 1 post remaining, two concurrent consumes on separate connections (WAL) → exactly one succeeds. Confidence 0.80.
**G-305 — `upgraded` is not a status state.** D-083's "(+ upgraded)" has no transition-matrix rows and REQ-9-005's canonical states don't include it. Decision: upgrade = plan_id transition within `active` (provider update per R-47 with items[0][id] — pinned by MH-3d), recorded as a billing_event with provenance; `upgraded` removed from the status enum — D-083's "(+ upgraded)" reinterpreted as the auditable upgrade ACTION, not a state. Amends Task 5-1-01, MH-5a. Tests: upgrade on active → status stays active, plan_id changes, billing_event exists; stored-status set contains no `upgraded`. Confidence 0.75.
**G-306 — out-of-order tolerance includes missing targets; grace expiry is lazy, not a timer.** (a) invoice.payment_failed/invoice.paid can arrive before checkout.completed creates the local subscription — handlers never 500 on unknown targets: record as unmatched (G-302 marker), return 200; recovery is the grant or admin manual complement. (b) past_due → expired at 7 days is computed lazily from past_due_at + 7d at read/entitlement time — no background timer, keeping tests hermetic. Amends Task 3-2-02 (tolerance rule), Task 5-1-01 (grace semantics); risks 3-4. Tests: signed invoice.paid for unknown subscription → 200 recorded no crash; entitlement check with injected now at past_due_at+6d → granted, +8d → 402/expired; no sleeps. Confidence 0.80.
**G-307 — align REQ-9-003/ROADMAP route naming to D-076.** REQUIREMENTS REQ-9-003 and ROADMAP phase-3 say `POST /v1/billing/checkout`; binding D-076, PLAN, and ARCHITECTURE say `/v1/billing/admin/checkout` behind require_admin. Decision: amend the wording at this gate (the requirement text must not name a public un-gated checkout route that will not exist). Test: route-inventory pin — no /v1/billing/checkout route exists; /v1/billing/admin/checkout 403s without token. Confidence 0.90.
**G-308 — define the browser-side admin-token entry; the admin panel is otherwise unreachable.** The admin surface is mock-data driven; no X-Admin-Token exists in apps/web. REQ-9-004's "admin surface shows per-employer entitlement state" hides a new requirement: an operator token entry in the browser — Task 4-2-01's A-406 citation is wrong (A-406 is for public routes). Decision: a small token input in the entitlement panel persisted to sessionStorage, sent by a new adminFetch wrapper in engine-client (X-Admin-Token header); 403 renders an honest locked state with the token prompt; the token is never logged. Amends Task 4-2-01. Tests: stubbed 403 → locked-state render; fetch stub asserts the header; no admin-token string in non-test web source beyond the wrapper. Confidence 0.75.
**G-309 — pin the ingest bypass: seed ingestion is never entitlement-gated.** Enforcement composes into the admin create_posting route ONLY, but the seeded-corpus ingest upserts ~20 postings directly through the store and must never hit the 402 path. Decision: explicit rule + test — run_ingest (and any future adapter) bypasses consume_posting by design; enforcement exists only in the admin route. Amends Task 4-1-01 (test list), risk 5 companion. Test: seeded ingest on a store with zero purchases → 20 postings upserted, zero 402s, zero consumption rows. Confidence 0.90.
**G-310 — default success/cancel URLs for the stripe checkout, or delete the "env-only switch" claim.** R-34: success_url is required for hosted checkout; the admin checkout body is {item_id, employer_id} and no task populates it — the stripe path cannot create sessions as planned. Decision: the checkout route defaults success_url = {public_base_url}/marketplace/pricing?checkout=success and cancel_url = …?checkout=cancelled (public_base_url exists at config.py:170); loud G-11 boot warning when stripe is selected and public_base_url is unset. Amends Task 3-2-01 (defaults) + Task 3-1-01 (byte-contract asserts both fields). Tests: MockTransport byte-contract shows success_url/cancel_url present; factory-selection test shows the loud warning when stripe + empty origin. Confidence 0.80.
---
# Nextcraft v0.6 — GRILL.md (Adversarial Review Verdict)
**Stage:** GRILL, Phase 0 pre-execution · **Verdict:** GO-WITH-BINDINGS · **Confidence:** 0.72 · **Date:** 2026-09-21
+44 -42
View File
@@ -2,18 +2,17 @@
## Persona Roster
> **v0.8 update (RESEARCH, lead-developer assessment):** milestone = Ship Anywhere (Feature — release/distribution rails: runner fleet, docker image, deb/arm64 packaging, CLI onboarding glue, admin guides). **Activated:** devops-engineer (NEW custom persona — the runner fleet on CT 113, docker-compose + labels + cache hygiene, .gitea/workflows push + tag lanes, Dockerfile + registry push, pre-commit config; the whole release-consistency surface REQ-8-001/002), cli-engineer (re-activated from its v0.4/v0.4.6 territory — build-binary.mjs arch parameter + arm64 node base, install.sh arch-resolve, deb packaging scripts, doctor container/deb-channel awareness, `guide` embedded text, bootstrap/deploy/dev container refusal — REQ-8-003/004), sandbox-engineer (re-activated for the D-068 capability probe promotion into sandbox/capability.py + the 503 degrade arm in api/sandboxes.py — its unshare_backend/manager/api-surface territory), lead-developer (coordination + wave order + the P5 admin-guide assembly), security-auditor (phase-specific: release secrets handling REGISTRY_TOKEN/GITEA_TOKEN/registration token, image non-root + healthcheck, sibling-mode blast radius, deb postinst). **Deactivated:** backend-engineer (no config/api surface work beyond what sandbox-engineer owns — the D-068 wiring lives in sandbox+main.py boot notice, owned by sandbox-engineer this milestone), frontend-engineer (no web-app changes; apps/web is only a BUILD INPUT to the image), ai-engineer, data-engineer, identity-engineer, voice-engineer, design-system-engineer.
> **v0.9 update (RESEARCH, lead-developer assessment):** milestone = Payment Engine (Feature — ai_service/billing/ module: provider protocol + mock + stripe impl, 7th D-027 store, /v1/billing routes with HMAC webhooks, posting entitlement enforcement, pricing-page + admin-surface runtime wiring). **Activated:** backend-engineer (owns ai_service/billing/* + api/billing*.py + api/deps.py + main.py router/lifespan wiring — REQ-9-001/003/004/005 core), data-engineer (owns the 7-table billing store schema: idempotency UniqueConstraints, purchase price snapshots, subscription state machine, event dedupe — REQ-9-001/005), security-auditor (phase-specific: webhook HMAC verify + replay window + raw-body discipline, fail-closed secrets, cents money handling, 402 CTA payloads — REQ-9-003), frontend-engineer (owns the pricing-page client conversion + engine-client listBillingCatalog/PaymentRequiredError + packages/types/billing.ts + admin entitlement view — REQ-9-002/004), lead-developer (coordination + wave order + dual-schema pin discipline + cross-boundary wire review). **Deactivated:** cli-engineer (no CLI changes — billing is server+web only), sandbox-engineer, devops-engineer (no infra changes; the existing release lane carries the new code automatically), ai-engineer, identity-engineer, voice-engineer, design-system-engineer.
### lead-developer
```yaml
active: true
phase_specific: false
reason: Coordinates the v0.8 Ship Anywhere milestone (fleet → container → debian → onboarding → guides waves); owns the admin-guide assembly (P5) as the single cross-channel runbook; owns wave-order + the release-asset matrix consistency check (every release: x64/arm64 binaries + debs + image tag per D-064/D-065)
reason: Coordinates the v0.9 Payment Engine milestone (foundation → catalog → checkout → enforcement → lifecycle waves); owns wave order + the cross-boundary wire review (billing.ts ↔ base.py dual-schema pins, 402 CTA payload shape) + provider-boundary event normalization consistency (R-43)
domain: coordination
frameworks:
- turborepo
- pnpm
- gitea-actions
- docker
constraints:
- pragmatic
- battle-tested defaults
@@ -21,15 +20,14 @@ constraints:
territory:
- "**/package.json"
- "**/turbo.json"
- "deploy/README.md"
- "README.md"
- "apps/web/tests/pins.test.ts"
```
### devops-engineer
```yaml
active: true
active: false
phase_specific: false
reason: NEW v0.8 custom persona — owns the entire release-consistency surface: 3 org-level gitea/runner:3 containers on CT 113 (sibling mode, host docker.sock, per-runner /data, GITEA_RUNNER_LABELS, capacity 1, cache.host pinning + 32 GB prune cron — REQ-8-001), .gitea/workflows push lane (node/python lanes with the repo's own scripts, D-063 fast-only test lanes) + tag lane (exclusive release owner D-064), multi-stage Dockerfile + git.coreci.dev/coreci/nextcraft push (semver + latest per D-065 — REQ-8-002), root .pre-commit-config.yaml (D-070)
reason: v0.9 DEACTIVATED — no infra changes; billing rides the existing release lane (image + assets build automatically on tag per D-064/D-065). Roster + v0.8 territory retained.
domain: devops
frameworks:
- docker
@@ -39,23 +37,23 @@ frameworks:
- bash
constraints:
- sibling-mode-socket-trust (org-scoped private repos only; accepted-risk documented)
- secrets-via-actions-secrets-or-founder (registration token never in files; REGISTRY_TOKEN self-serve PUT first, founder UI fallback)
- job-token-cannot-publish-packages (upstream-confirmed; registry push needs the PAT secret)
- cache-host-pinning (dockerized runner cache server must be job-reachable or actions/cache ETIMEDOUTs)
- secrets-via-actions-secrets-or-founder
- job-token-cannot-publish-packages
- cache-host-pinning
- fast-lanes-only-in-ci (real_server/userns never in job containers, D-063)
territory:
- ".gitea/workflows/**"
- "Dockerfile"
- ".dockerignore"
- ".pre-commit-config.yaml"
- "deploy/runner/**" # compose + config for CT 113 (admin-guide artifacts, not CT state)
- "deploy/runner/**"
```
### cli-engineer
```yaml
active: true
active: false
phase_specific: false
reason: v0.8 re-activation — owns the distribution artifacts of its own binary: build-binary.mjs arch parameter (--arch x64|arm64; arm64 node-base download replacing process.execPath copy, postject unchanged — D-069 QEMU-free cross-build), scripts/build-deb.sh + postinst advisory doctor (D-067), install.sh arch-gate→arch-resolve (x86_64/aarch64, sha256+version gates byte-identical), doctor container/deb-channel awareness (real userns probe + NC_CONTAINER//.dockerenv//usr/bin/nextcraft detection — REQ-8-004/D-066), `nextcraft guide` offline-first embedded text (D-072), bootstrap/deploy/dev in-container refusal (D-066/D-071), verify container-mode branch
reason: v0.9 DEACTIVATED — no CLI changes; billing is server+web only. Roster + v0.8 territory retained.
domain: cli
frameworks:
- node
@@ -82,9 +80,9 @@ territory:
### sandbox-engineer
```yaml
active: true
active: false
phase_specific: false
reason: v0.8 re-activation — owns the D-068 degrade detection in its sandbox fabric: promote tests/helpers/userns.py's probe into ai_service/sandbox/capability.py (verbatim semantics), the boot notice in main.py lifespan (the G-11 mount-less-warning pattern), the per-create 503 + remediation arm (SandboxUnavailableError mapping beside PoolFullError in api/sandboxes.py) — the honest container-degrade path for REQ-8-002; real_server/userns test tiers stay OUT of CI lanes per D-063
reason: v0.9 DEACTIVATED — no sandbox fabric changes; the D-068 degrade surface ships unchanged. Roster + v0.8 territory retained.
domain: infra
frameworks:
- python
@@ -108,34 +106,33 @@ territory:
```yaml
active: true
phase_specific: true
reason: v0.8 re-activated (phase-specific) — P1 release secrets review (registration token flow, REGISTRY_TOKEN/GITEA_TOKEN Actions secrets, the .env-only resolution of release-assets.sh preserved as manual fallback), P2 image review (non-root user, healthcheck without curl, state volume ownership, sibling-mode socket blast-radius row), P3 deb review (postinst never fails dpkg, empty Depends + honest Recommends), P6 final review (channel consistency: every recipe backed by a test or live-box evidence)
reason: v0.9 re-activated (phase-specific) — P3 webhook security review: HMAC-SHA256 verify discipline (raw-body-first, compare_digest, all-v1 iteration for secret rolls, ±300s replay window, non-v1 schemes ignored — R-38..R-40), fail-closed webhook secret (unset → per-boot random → 401s, correct for a payment surface), event dedupe integrity, cents money handling (no floats — D-081), 402 CTA payload honesty, stripe provider key redaction in errors (the embedding test precedent); P6 final review (no secrets in billing code/logs/releases)
domain: security
frameworks:
- pytest
- httpx
constraints:
- STRIDE-classified
- secrets-never-in-workflow-env (Actions secrets only; never files)
- secrets-via-env-only (D-014; AI_PAYMENT_API_KEY/AI_WEBHOOK_SECRET never logged)
- secrets-never-logged
- registration-token-reset-after-fleet-green (founder directive)
- fail-closed-degrades (unprivileged default; privileged variant documented opt-in)
- webhooks-fail-closed-when-unconfigured
- raw-body-before-parse (signature verification sees exact bytes)
territory:
- ".gitea/workflows/**"
- "Dockerfile"
- "scripts/build-deb.sh"
- "scripts/release-assets.sh"
- "apps/cli/src/commands/deploy.ts"
- "apps/ai-service/ai_service/billing/**"
- "apps/ai-service/ai_service/api/billing*.py"
- "apps/ai-service/tests/billing/**"
```
### backend-engineer
```yaml
active: false
active: true
phase_specific: false
reason: v0.8 DEACTIVATED — the only backend-adjacent work (D-068 probe wiring + boot notice) is owned by sandbox-engineer whose fabric it extends; config.py/pyproject need zero changes (NC_CONTAINER is a CLI-layer marker; the AI_* env surface is untouched). Roster retained; the v0.7 pytest/mypy territory is stable.
reason: v0.9 ACTIVATED — owns the billing module core: ai_service/billing/base.py (PaymentProvider protocol + pydantic contracts + canonical event names), mock.py (deterministic Stripe-shaped provider), stripe.py (form-encoded REST via shared httpx pool, Idempotency-Key per R-49), factory.py (G-11 loud fallback), catalog.py (seeded idempotent), api/billing.py + api/billing_admin.py (require_admin dep verbatim R-51, webhook route raw-body-first, entitlement composition into create_posting AFTER the admin gate), config.py trio (payment_provider/payment_api_key/webhook_secret — R-53), main.py router mounts + lifespan store wiring
domain: backend
frameworks:
- fastapi
- pydantic-settings
- httpx
- bash
- turborepo
- pnpm
@@ -143,22 +140,23 @@ constraints:
- secrets-via-env-only (D-014)
- idempotent-scripts
- fail-closed-admin
- never-call-cloud-in-tests (MockTransport byte-contract only, D-075)
- provider-boundary-normalization (R-43: store provider ids verbatim)
territory:
- "apps/ai-service/ai_service/billing/**"
- "apps/ai-service/ai_service/config.py"
- "apps/ai-service/ai_service/main.py"
- "apps/ai-service/ai_service/api/**"
- "apps/ai-service/scripts/**"
- "apps/ai-service/tests/billing/**"
- "apps/ai-service/.env.example"
- "apps/ai-service/pyproject.toml"
- "package.json"
- "turbo.json"
```
### frontend-engineer
```yaml
active: false
active: true
phase_specific: false
reason: v0.8 DEACTIVATED — no apps/web changes; the web app is only a BUILD INPUT to the docker image (NEXT_PUBLIC_AI_SERVICE_URL=self export, unchanged since D-039). ESLint/c8/pins territory stable from v0.7.
reason: v0.9 ACTIVATED — owns the pricing-page conversion (server shell retained per A-411/pins + PricingClient component with AbortController + honest loading/error/empty states, the A-406 board-client pattern), engine-client listBillingCatalog() + PaymentRequiredError (402 dict-detail discrimination — parseError precedent), packages/types/billing.ts (dual-schema pinned), admin entitlement view in the moderation surface (per-employer active package/posts used/limit/expiry, client fetch)
domain: frontend
frameworks:
- react
@@ -168,16 +166,19 @@ constraints:
- component-first
- server-components-default
- no-build-time-db-reads (D-039 export build stays repo-only)
- honest-states (loading/error/empty never mocked)
- cents-in-types (number documented as cents, D-081)
territory:
- "apps/web/**"
- "packages/ui/**"
- "packages/types/billing.ts"
```
### ai-engineer
```yaml
active: false
phase_specific: false
reason: v0.8 DEACTIVATED — no LLM-facing work; the provider layers ship unchanged inside the docker image (G-11 fallbacks are the only runtime-config behavior).
reason: v0.9 DEACTIVATED — no LLM-facing work; all provider layers ship unchanged.
domain: ai
frameworks:
- pydantic
@@ -196,9 +197,9 @@ territory:
### data-engineer
```yaml
active: false
active: true
phase_specific: false
reason: v0.8 DEACTIVATED — zero schema/store changes; the ~/.nextcraft/data/nextcraft.db path rides into the docker volume via config.py's existing home-dir default (no code change).
reason: v0.9 ACTIVATED — owns the 7th D-027 store: billing/store.py with the 7-table schema (packages, plans, purchases with price snapshots — D-078, subscriptions state machine, subscription_events with verbatim event_id UniqueConstraint dedupe, checkout_sessions, billing_events audit trail), InvalidTransition domain error, transition matrix tests, purchase consumption query (newest non-expired with remaining posts), idempotent catalog seeding
domain: data
frameworks:
- sqlmodel
@@ -208,19 +209,20 @@ constraints:
- schema-first
- type-safe
- dual-schema-sync
- idempotency-by-constraint (UniqueConstraint + check-first, the insert_application pattern)
- snapshots-immutable (catalog edits never rewrite purchase history, D-078)
territory:
- "apps/ai-service/ai_service/marketplace/base.py"
- "apps/ai-service/ai_service/marketplace/store.py"
- "apps/ai-service/ai_service/marketplace/ingest.py"
- "apps/ai-service/ai_service/billing/store.py"
- "apps/ai-service/ai_service/billing/catalog.py"
- "apps/ai-service/tests/billing/test_store.py"
- "packages/types/**"
- "packages/mock-data/**"
```
### identity-engineer
```yaml
active: false
phase_specific: false
reason: v0.5 custom persona — module shipped complete; no v0.8 identity-domain work.
reason: v0.5 custom persona — module shipped complete; no v0.9 identity-domain work.
domain: identity
frameworks:
- fastapi
@@ -239,7 +241,7 @@ territory:
```yaml
active: false
phase_specific: false
reason: v0.5 territory — voice shipped complete; no v0.8 voice work.
reason: v0.5 territory — voice shipped complete; no v0.9 voice work.
domain: ai-media
frameworks:
- httpx
@@ -260,7 +262,7 @@ territory:
```yaml
active: false
phase_specific: false
reason: No design-token or primitive work in v0.8; roster retained.
reason: No design-token or primitive work in v0.9; roster retained.
domain: frontend
frameworks:
- tailwindcss
+165 -1
View File
@@ -1,4 +1,168 @@
# Nextcraft — PLAN.md (v0.8 Ship Anywhere)
# Nextcraft — PLAN.md (v0.9 Payment Engine)
## Overview
**Milestone:** v0.9 — Payment Engine (Feature, D-074): payment processing and subscription management — posting packages (single/bundle/enterprise), talent-access plans, subscription tiers with real checkout, provider webhooks (HMAC-verified, idempotent), posting-creation entitlement enforcement, and a full subscription lifecycle — behind a provider-agnostic `PaymentProvider` protocol, mock-first (D-014 pattern; tests never call Stripe). The pricing page (REQ-017's surface) gains its real backend.
**Tag line:** phase tags advance the v0.8.x line (P0 → v0.8.0 … P6 → v0.8.6 = milestone release), continuing from the v0.7.6 milestone release
**Branch:** milestone/v0.9-payment-engine
**Phase list (requirement mapping):**
| Phase | Requirement | Focus |
|-------|-------------|-------|
| P1 | REQ-9-001 | Billing foundation: billing/ module (protocol + mock + store + HMAC + factory + settings), no routes yet |
| P2 | REQ-9-002 | Catalog + pricing surface: seeded catalog, /v1/billing/catalog, pricing-page client conversion, types + pins |
| P3 | REQ-9-003 | Checkout + webhook flow: admin checkout route, stripe provider, public webhook (raw-body HMAC, dedupe, idempotent processing), mock-complete E2E path |
| P4 | REQ-9-004 | Entitlement enforcement: consume_posting in create_posting, 402 CTA payload, admin entitlement view |
| P5 | REQ-9-005 | Subscription lifecycle: state machine via webhooks, manual grant/revoke, billing event log, past_due grace |
| P6 | — | Final review + audit + milestone ship (v0.8.6 = milestone release) |
**Wave order (binding): foundation before surface before flow before enforcement before lifecycle.** P1 builds the module everything else composes (store + protocol + mock — P2's catalog route reads the store, P3's checkout writes it). P2 precedes P3 because checkout references catalog items by id (the catalog must exist + be surfaced before sessions can be created against it). P3 precedes P4 because enforcement consumes entitlements that only the webhook flow grants. P5 needs the full event set the flow produces. Each phase is a vertical slice: store rows → routes → web surface → tests green → ship.
**Binding contracts:** clarify decisions D-075..D-084 (PROJECT.md) and research R-33..R-54 (RESEARCH.md) are design contracts; this plan operationalizes them, never re-litigates.
**Environment facts (probe-verified, R-51..R-54):** require_admin dep pattern at api/marketplace_admin.py:46-85; D-027 store family replication (marketplace/store.py the template); Settings env trio precedent (config.py: provider selector + SecretStr key + per-boot-random secret); HMAC precedent telemetry/tokens.py; at-least-once dedupe precedent insert_application; pricing page inline constants at page.tsx:41-111 + metadata pinned by pins.test.ts:189-190 + sitemap STATIC_PAGES; engine-client jsonFetch/parseError at lines 90-140; main.py inline router mounts + lifespan state-injection + G-11 fallback pattern.
**NFR note:** this is a Feature milestone — no purity gate; `feat:` commits are expected and scoped to billing + the pricing/admin surfaces.
**Secrets policy (binding):** AI_PAYMENT_API_KEY / AI_WEBHOOK_SECRET env-only (D-014), SecretStr in settings, never logged, never in .env.example with real values; tests use fixed test secrets; the stripe provider redacts keys from exceptions (embedding test precedent).
---
## Phase 1: Billing Foundation (REQ-9-001)
**Goal:** The billing module exists and is proven: protocol + mock provider + 7-table store + HMAC webhook verification + factory + settings — all tested, zero routes mounted, zero web changes.
### Wave 1-1: Module core (backend-engineer + data-engineer, sequential)
- **Task 1-1-01** (backend-engineer): `ai_service/billing/__init__.py` + `base.py` — pydantic contracts: `Package` (id, name, price_cents, posts_limit, duration_days, features), `Plan` (id, name, price_cents, interval, features), `CheckoutSessionSpec` (mode, line item ref, success/cancel refs, client_reference_id, metadata), `CheckoutSession` (id, url, status, payment_status, mode, expires_at), `SubscriptionRef`, `CanonicalBillingEvent` (name enum: checkout.completed, invoice.paid, invoice.payment_failed, subscription.cancelled, subscription.renewed + payload dict). Money = int cents ge=0 (D-081). `PaymentProvider` Protocol: create_checkout_session, retrieve_session, cancel_subscription, update_subscription, verify_webhook, normalize_event.
- **Task 1-1-02** (data-engineer): `billing/store.py` — 7th D-027 store (R-52): 7 tables per D-080; UniqueConstraints (subscription_events.event_id, checkout_sessions.provider_session_id, **purchases.checkout_session_id — G-303: grants idempotent per session, not per event id**); purchases carry price/name snapshot columns; `_validate`/`_normalize`/`_as_utc`/`_sqlite_connect` (WAL/NORMAL/busy_timeout/foreign_keys); Protocol + SQLiteBillingStore pair; `InvalidTransition(ValueError)`; store methods: seed_catalog (idempotent upsert), list_catalog, get_purchase/consume_posting(employer_id) (**G-304: single conditional UPDATE `SET posts_used = posts_used + 1 WHERE id = <newest non-expired with remaining> AND posts_used < posts_limit`, rowcount-checked — never check-then-increment; 0 rows → PackageExhausted/NoActivePackage**), subscription get/upsert/transition (state machine per D-083; **upsert keyed on provider subscription id — G-303**), insert_provider_event (check-first dedupe → created flag), insert_billing_event (audit, manual provenance, **unmatched marker per G-302**), close. Tests: `tests/billing/test_store.py` — tmp_path db, table-existence via sqlite_master, pragma pins, IntegrityError surfacing, dedupe idempotency double-insert (**G-303: same session, different event ids → one purchase**), transition matrix (valid + InvalidTransition; **G-305: no `upgraded` status value**), consume_posting exhaustion + expiry + multi-purchase ordering (**G-304: concurrency test — 1 remaining, two connections → exactly one succeeds**).
- **Task 1-1-03** (backend-engineer): `billing/mock.py` — deterministic mock provider (D-075): Stripe-shaped ids (`cs_mock_...` counter+randbits — the identity-mock unique-mint lesson), create/retrieve session (unpaid → paid on confirm), signature mint/verify identical scheme to real provider (whsec-shaped), normalize_event passthrough (mock emits canonical names directly). Tests: `tests/billing/test_mock.py`.
- **Task 1-1-04** (backend-engineer): `billing/webhooks.py` — verification core shared by providers: parse `Stripe-Signature` (t=,v1= pairs, multi-v1, ignore v0/unknown — R-38), HMAC-SHA256 over `{t}.{raw_body}` bytes, hmac.compare_digest, ±300s window with `now` injection (R-40, risk-4). Tests: `tests/billing/test_webhooks.py` — good/bad signature, stale timestamp (outside window), missing header, multi-v1 (secret roll), v0-scheme rejected, replay-same-event accepted (idempotent at the verify layer).
- **Task 1-1-05** (backend-engineer): `billing/factory.py` + config.py trio (R-53): `payment_provider="mock"`, `payment_api_key: SecretStr`, `webhook_secret=""` (per-boot random); UnknownPaymentProviderError; .env.example additions (commented, no real values). Tests: factory selection + unknown error + secret redaction (repr must not leak).
### Wave 1-2: Foundation must-haves
- P1 runs the full suite (fast lane) green: new tests + existing 670 unaffected (billing module is additive; main.py untouched this phase).
- mypy + ruff green (billing files join [tool.mypy] files set implicitly — same package).
- **Must-have:** mock provider + store + webhooks.py are usable WITHOUT any route — proven by direct-instantiation tests only.
## Phase 2: Catalog + Pricing Surface (REQ-9-002)
**Goal:** The catalog is real and surfaced: seeded packages/plans in the DB, `/v1/billing/catalog` public GET, the pricing page reads it at runtime, wire types pinned both sides.
### Wave 2-1: Backend catalog route (backend-engineer)
- **Task 2-1-01** (backend-engineer): `billing/catalog.py` — the seed constant (single $49/1 post/30d, bundle $399/10 posts/90d, enterprise custom; starter $99/mo, pro $299/mo, enterprise-talent custom — REQ-017's tier structure) + idempotent seed into store. Mounted via `api/billing.py`: `GET /v1/billing/catalog` → `{packages: [...], plans: [...]}` (public — catalog is advertising, not secret; CORS-existing policy applies). Deps: `get_billing_store` (R-51/R-52 wiring: deps.py + main.py lifespan + close in finally). Tests: `tests/billing/test_api.py` — catalog returns the 6 seeded items, shape pinned, unauthenticated 200.
### Wave 2-2: Web pricing conversion (frontend-engineer)
- **Task 2-2-01** (frontend-engineer): `packages/types/billing.ts` — Package/Plan/CatalogResponse types (cents documented, D-081); dual-schema pins in `apps/web/tests/pins.test.ts` (G-108 discipline: field-for-field vs base.py).
- **Task 2-2-02** (frontend-engineer): `engine-client.ts` — `listBillingCatalog()` via jsonFetch; page stays server shell (metadata + pins untouched — A-411); new `components/marketplace/pricing-client.tsx` ('use client', AbortController, loading/error/empty states — board-client pattern); CTA copy per D-079 (contact path + operator-mediated note; no fake purchase buttons). Tests: fetch-stub catalog render, error state render, empty catalog render, cents formatting.
- **Task 2-2-03** (frontend-engineer): pins updated: pricing-page metadata pin retained; new catalog wire pins added.
## Phase 3: Checkout + Webhook Flow (REQ-9-003)
**Goal:** The full payment flow: admin creates a checkout session (mock by default, stripe via env), the provider completes it, the webhook (HMAC-verified, deduped, idempotent) grants the entitlement — proven E2E in tests with zero cloud calls.
### Wave 3-1: Stripe provider (backend-engineer)
- **Task 3-1-01** (backend-engineer): `billing/stripe.py` — form-encoded REST via the shared httpx pool: create session (mode, line_items[0][price_data] from catalog item snapshot, client_reference_id, Idempotency-Key uuid4 — R-33/R-35/R-49), retrieve session, cancel_subscription (DELETE or cancel_at_period_end — R-46), update_subscription (items[0][id] resolution inside provider — R-47 dual-charge guard), verify_webhook (shared webhooks.py), normalize_event (R-43 mapping). Tests: `tests/billing/test_stripe_provider.py` — MockTransport byte-contract: exact form-encoded body, Basic auth header, Idempotency-Key present, item-id in update body, key redaction in exceptions, normalize_event mapping table (Stripe → canonical).
- **Task 3-1-02** (backend-engineer): main.py factory wiring: provider from settings (mock default), G-11 loud fallback on factory error (voice precedent), provider on app.state + deps.
### Wave 3-2: Checkout + webhook routes (backend-engineer)
- **Task 3-2-01** (backend-engineer): `api/billing_admin.py` — `POST /v1/billing/admin/checkout` (require_admin verbatim — R-51): body {item_id, employer_id} → catalog item lookup → create session (operator_ref + employer_id recorded, D-076; **success_url/cancel_url default from public_base_url per G-310: success = {public_base_url}/marketplace/pricing?checkout=success, cancel = …?checkout=cancelled; loud G-11 boot warning when stripe + public_base_url unset**) → persist checkout_sessions row → return {session_id, url}. Tests: fail-closed 403 (no/missing token), 404 unknown item, 201 with persisted row, **byte-contract: success_url + cancel_url present in the form body**.
- **Task 3-2-02** (backend-engineer): `api/billing.py` webhook — `POST /v1/billing/webhook`: `await request.body()` FIRST (risk-1 raw-body discipline), verify HMAC (401 on bad/missing signature), parse event, insert_provider_event (dedupe: existing → 200 already-processed), process by canonical name — **G-302: event payloads are untrusted ID references — the session/subscription resolves from LOCAL rows (creatable only via the admin route); unknown target id → recorded as an unmatched billing_event, NO entitlement, 200; purchase price/name from the stored snapshot, never the event body** — checkout.completed → verify predicate status=complete AND payment_status=paid (R-37) against the local session row → create purchase/subscription from session snapshot; invoice.paid → extend/renew; invoice.payment_failed → past_due; subscription.cancelled → cancelled. All transitions via store (InvalidTransition tolerated-as-duplicate, recorded; **G-306: unknown-target events never 500 — recorded unmatched, 200**). 200 fast. Tests: `tests/billing/test_webhook_route.py` — unsigned 401, bad sig 401, valid + duplicate (idempotent 200, single entitlement), out-of-order tolerated, grace window pinned, mock flow E2E, **G-302: signed event for unknown session → 200 + zero purchases + one unmatched row; inflated payload amount → price = snapshot**, **G-303: distinct event ids same session → one purchase**.
- **Task 3-2-03** (backend-engineer): mock-complete path (D-082, **amended by G-301**): `POST /v1/billing/mock/complete` mounted ONLY under provider=mock **AND composed with require_admin — the operator completing a session after out-of-band payment is the honest pilot flow (G-301: mock is the default provider, so this route exists on every default-config deployment; an unauthenticated completion route on a payment surface is a hole regardless of id unguessability)** — mints a signed event through the REAL webhook processing (direct handler call, same code path) — E2E proof: admin checkout → admin-gated mock complete → webhook → entitlement exists. Route 404s under stripe (factory-conditional mount, test-pinned). Tests: **no token → 403 under mock (G-301)**; 404 under stripe; E2E green.
## Phase 4: Entitlement Enforcement (REQ-9-004)
**Goal:** Posting creation enforces packages honestly; the admin surface shows entitlement state.
### Wave 4-1: Enforcement composition (backend-engineer)
- **Task 4-1-01** (backend-engineer): `api/marketplace_admin.py` create_posting composes billing AFTER require_admin: consume_posting(employer_id) → success: proceed (posts_used incremented); `PackageExhausted`/`NoActivePackage` (billing domain errors) → `HTTPException(402, detail={reason, posts_used, posts_limit, upgrade_cta: {catalog_hint}})` (dict-detail precedent — require_verified_age). Backward-compat: employers with UNLIMITED manual grants bypass (enterprise — D-080). **G-309: the enforcement exists ONLY in the admin route — run_ingest and any future adapter bypass consume_posting by design (seed ingestion is never entitlement-gated)**. Tests: 402 on no package, 402 on exhausted, 201 + consumption on valid, unlimited bypass, expired package degrades to 402, **seeded ingest on a zero-purchase store → all postings upserted, zero 402s (G-309)**.
- **Task 4-1-02** (backend-engineer): `GET /v1/billing/admin/entitlements/{employer_id}` — active package, posts used/limit, expiry, subscription state (require_admin). Tests: shape + fail-closed.
### Wave 4-2: Admin surface (frontend-engineer)
- **Task 4-2-01** (frontend-engineer): admin moderation surface gains per-employer entitlement panel (**G-308 design, replacing the A-406 citation — the entitlements route is admin-gated, A-406 is the public-route pattern**): a small operator-token input persisted to sessionStorage + a new `adminFetch` wrapper in engine-client (X-Admin-Token header, token never logged); 403 renders an honest locked state with the token prompt; panel shows active package, posts used/limit bar, expiry, subscription state badge, manual-grant pointer. Tests: fetch-stub render, 402-CTA payload rendering, **stubbed 403 → locked-state render with prompt (G-308)**, **fetch stub asserts the X-Admin-Token header**, no admin-token string in non-test web source beyond the wrapper.
## Phase 5: Subscription Lifecycle (REQ-9-005)
**Goal:** Full lifecycle: state machine driven by webhooks + manual admin actions, audit log complete, past_due grace honest.
### Wave 5-1: Lifecycle + admin routes (backend-engineer + data-engineer)
- **Task 5-1-01** (data-engineer): store transition matrix complete: trialing→active, active→past_due, past_due→active (recovery), past_due→expired (**G-306: lazily computed from past_due_at + 7-day constant at read/entitlement time — no background timer; test-pinned D-083**), active→cancelled, cancelled terminal; **G-305: no `upgraded` status — upgrade is a plan_id transition within `active`, recorded as a billing_event with provenance**; expires_at extension idempotent on invoice.paid; manual grant/revoke/complement methods (source="manual" + operator provenance in billing_events). Tests: full matrix + grace-elapsed expiry (**injected now: past_due_at+6d → granted, +8d → expired — no sleeps**) + manual-action audit rows + **upgrade keeps status=active, changes plan_id, writes billing_event (G-305)**.
- **Task 5-1-02** (backend-engineer): `api/billing_admin.py` lifecycle routes: POST admin/subscriptions/{id}/cancel, /resume, /upgrade (provider calls for stripe; state transitions for mock), POST admin/employers/{id}/grant (manual unlimited/package), /revoke, GET /v1/billing/admin/events (queryable audit log: by employer, by type, by time). Tests: each route happy path + fail-closed 403 + InvalidTransition 409.
- **Task 5-1-03** (backend-engineer): webhook subscription.renewed handling: renewal invoice.paid extends expires_at idempotently (event dedupe makes double-fire safe — R-45); corroborating customer.subscription.updated recorded. Tests: double-renew single-extension.
## Phase 6: Final Review + Ship (milestone release, v0.8.x line)
**Goal:** Multi-persona review of all v0.9 changes; audit (reconstruction + discipline); milestone ship: merge to main, tag v0.8.6 (final v0.8.x patch = milestone release), full asset matrix + registry image via the release lane (D-064), milestone completion commits, branch cleanup.
- Review: P0 fixes in-phase; P1+ fixed here, never loop back to EXECUTE.
- Audit: git log ↔ .ciagent/ reconstruction; file discipline; branch hygiene; commit discipline (feat: scoped to billing + pricing/admin surfaces).
- Ship: phase/(N+1) branch → milestone → main; delete all milestone branches (tags preserve history).
- Complete: REQUIREMENTS.md (REQ-9-001..005 complete), ROADMAP.md milestone entry, checkpoint cleared.
---
## Must-Haves (per phase — concrete, falsifiable, test-linked)
- **P1 (REQ-9-001):** billing module importable + tested standalone; store tests green (7 tables, dedupe, transitions); webhooks.py verify matrix green; factory + settings trio live; fast suite green; NO routes mounted, NO web changes. **MH-1a** sqlite_master shows 7 billing tables. **MH-1b** duplicate insert_provider_event returns created=False. **MH-1c** bad-signature verify raises. **MH-1d** factory: mock default, unknown raises, key redacted.
- **P2 (REQ-9-002):** catalog route serves 6 seeded items; pricing page renders from runtime fetch with honest states; dual-schema pins green; metadata pin retained. **MH-2a** GET /v1/billing/catalog unauth 200 with exactly the seeded items. **MH-2b** pins.test.ts billing section green. **MH-2c** pricing page: no inline JOB_PLANS/TALENT_PLANS constants remain (grep-clean).
- **P3 (REQ-9-003):** stripe provider byte-contract green under MockTransport; webhook 401/200/idempotent matrix green; mock E2E green (checkout → complete → entitlement). **MH-3a** unsigned webhook → 401, never processed. **MH-3b** duplicate event → 200, single entitlement; **distinct event ids, same session → one purchase (G-303)**. **MH-3c** E2E: purchase exists after admin-gated mock complete with predicate honored; **no-token complete → 403 (G-301)**. **MH-3d** update body contains items[0][id]. **MH-3e** no test touches the network (MockTransport-only, grep-pin). **MH-3f** signed event for unknown session → 200, zero entitlement, unmatched row (G-302).
- **P4 (REQ-9-004):** create_posting enforces; 402 payload shape pinned; admin entitlement view renders. **MH-4a** no-package posting → 402 with upgrade_cta. **MH-4b** exhausted → 402; valid → 201 + posts_used incremented. **MH-4c** unlimited bypass. **MH-4d** expired degrades to 402.
- **P5 (REQ-9-005):** transition matrix complete + grace pinned; manual actions audited; renewal idempotent; event log queryable. **MH-5a** grace-elapsed past_due → expired (7-day constant pinned, lazy evaluation per G-306 — no sleeps); **no `upgraded` status value exists (G-305)**. **MH-5b** double-renew single extension. **MH-5c** manual grant → billing_events row with source=manual. **MH-5d** GET admin/events filters by employer.
## Milestone Must-Haves
- REQ-9-001..005 all complete; the mock payment loop is the machine-checked E2E proof (checkout → webhook → entitlement → consumption → renewal/cancel).
- The stripe provider is byte-contract-proven without ever leaving MockTransport.
- Zero secrets in code/tests/examples; AI_WEBHOOK_SECRET unset → webhooks fail closed.
- Suite counts: pytest + web + cli all green, zero warnings (v0.7 regime holds).
- Docs: .env.example billing section; deploy/README.md unaffected (billing needs no admin-box changes — record this explicitly); REQUIREMENTS/ROADMAP updated.
## Risks
1. **Raw-body discipline** (risk-1): any body pre-parse breaks HMAC — pinned by MH-3a + route-order test.
2. **Dual-charge** (R-47): pinned by MH-3d.
3. **Out-of-order events** (R-45): transitions tolerate duplicates; InvalidTransition recorded not raised for already-reflected state — pinned in webhook tests.
4. **Test clock**: now-injection in verify + store (no sleeps — deterministic).
5. **Backward compat at P4**: existing marketplace admin tests create postings for seeded employers — seed catalog + grant manual unlimited to test employers OR mark route change explicit in fixtures (decided: tests grant unlimited to legacy seeded employers — honest, explicit).
6. **Pins churn**: pins.test.ts grows — keep the G-108 discipline (regression/boundary pins only, no local-grep copy pins).
## User-Facing Surface
1. **Pricing page** — real catalog data at runtime (packages + plans with cents-true prices), honest loading/error states, contact-path CTAs (operator-mediated checkout note).
2. **Admin billing surface** — per-employer entitlement panel (package, posts used/limit, expiry, subscription badge) in the moderation view; operator checkout trigger.
3. **Public API** — GET /v1/billing/catalog (advertises the catalog); POST /v1/billing/webhook (provider-facing).
4. **Operator API** — /v1/billing/admin/* (checkout, entitlements, lifecycle, grants, event log) behind the admin token.
5. **Release notes** — v0.8.x line ships the billing surface through the existing automated lane (no admin-box changes required).
## Happy Path (end-to-end, written BEFORE execute)
1. **Operator buys for an employer (mock pilot):** admin token → POST /v1/billing/admin/checkout {item: bundle, employer: E1} → session created → mock complete → webhook (signed, deduped) → purchase row: 10 posts, $3.99 snapshot → E1's admin panel shows 10/10 posts available.
2. **Employer posts jobs:** create_posting E1 → 201, posts_used 1..N; at 10/10 the 11th → 402 with upgrade_cta → operator buys enterprise grant → unlimited.
3. **Subscription month rolls:** pro plan renews → invoice.paid webhook → expires_at extended exactly once (double-delivery safe) → talent-access stays active.
4. **Payment fails:** card declines → invoice.payment_failed → past_due → 7-day grace → still unpaid → expired (honest downgrade) — OR recovery payment inside grace → active again.
5. **Stripe switch (env-only):** AI_PAYMENT_PROVIDER=stripe + keys → same routes, real hosted checkout, real webhooks — zero code changes.
6. **Visitor checks pricing:** /marketplace/pricing loads → catalog fetched → prices + tiers match the DB exactly.
## UX Acceptance Criteria
1. No dishonest buttons: every CTA states the real pilot path (operator-mediated); no fake "buy now" without a real flow behind it.
2. Prices are always cents-true and consistent: pricing page = catalog API = purchase snapshots; no drift.
3. Failures are loud and actionable: 402 payloads carry upgrade_cta; webhook failures fail closed (401) rather than processing unsigned; past_due is visible in the admin panel before downgrade.
4. The admin never needs to guess: entitlement panel answers "can this employer post?" in one glance; the event log answers "what happened to this subscription?" with full provenance.
5. Existing surfaces unchanged: job board, defense, sandboxes, deploy — billing is additive; any regression there is a P0.
---
# v0.8 PLAN (Ship Anywhere — prior milestone, retained below)
## Overview
+25 -1
View File
@@ -16,7 +16,30 @@ Nextcraft is an AI-native outcome school where graduates prove what they can bui
---
## Milestone v0.8 — Ship Anywhere (ACTIVE, Feature, started 2026-09-27)
## Milestone v0.9 — Payment Engine (ACTIVE, Feature, started 2026-09-29)
**Scope (D-074, auto-decided at SPECIFY under full autonomy D-007; founder scope question left open post-v0.8):** Activates **REQ-F-018 — payment processing and subscription management** as REQ-9-001..005. The pricing page (REQ-017, critical since v0.1) gains its real backend: posting packages (single, bundle, enterprise), talent-access plans for employers, and subscription tiers — with checkout, provider webhooks, entitlement enforcement on posting creation, and a subscription lifecycle (renew/cancel/upgrade/downgrade) under a provider-agnostic protocol (D-014 mock-first pattern). Rationale: the guiding outcome (≥1,000 placements via the marketplace) requires a monetized, self-serve employer side; the marketplace engine (v0.6) built the supply side, ship-anywhere (v0.8) made the stack installable anywhere — payments is the only high-priority requirement deferred since v0.3 that directly unblocks employer self-serve (A-407: "real employer auth lands post-payments with the B2B employer network") and CIRR-style placement tracking needs paid postings to track. Confidence 0.80 ≥ threshold 0.60 (D-007).
**Out of scope:** real employer auth/accounts (B2B employer network — deferred, lands post-payments per A-407), multi-currency, tax/VAT handling beyond passthrough, invoice PDF generation, Stripe-specific UI components, CIRR tracking (REQ-F-020, deferred).
**Success (milestone-level):** an employer (via admin-mediated flow at pilot) can purchase a posting package or talent-access subscription through a real Stripe-compatible checkout; webhooks update entitlements automatically; posting creation enforces package limits honestly; subscription lifecycle transitions (renew/cancel/upgrade) work end-to-end; the pricing page reflects real catalog data; all provider integrations are mock-first (tests never call Stripe).
## Clarify decisions (D-075+, CLARIFY stage full autonomy, 2026-09-29)
| ID | Decision | Rationale | Confidence |
|----|----------|-----------|------------|
| D-075 | Both providers ship in-milestone: `mock` (deterministic, default) and `stripe` (real Stripe-compatible REST via the shared httpx pool — checkout session create/retrieve, subscription cancel/upgrade, event signature verify). Tests exercise the stripe provider ONLY through `httpx.MockTransport` (the openai-audio/embedding precedent — byte-contract pinned, cloud never called); `AI_PAYMENT_PROVIDER=stripe` is env-gated and needs `AI_PAYMENT_API_KEY` — unset key falls back to mock with a loud G-11 boot log (voice/embedding factory pattern, `billing/factory.py` + main.py lifespan catch). | The D-030 openai-audio precedent: a real provider shipped alongside mock with MockTransport byte-contract tests. A protocol without a real implementation is unproven; a real implementation without mock is untestable. Pilot runs mock; the real path is config-away. | 0.80 |
| D-076 | Buyer model at pilot = **operator-mediated** (extends A-401): checkout sessions are created via admin-token-gated routes (`/v1/billing/admin/checkout`) on behalf of an employer ref in the request body; no employer principals exist (A-407). Every purchase records `operator_ref` (admin token identity = "operator") + `employer_id`. Webhook route is public (server-to-server) but HMAC-verified (D-077). | Employers are data, not principals (A-407); the admin token dep (`require_admin` returning Settings) is the only operator identity mechanism in the codebase. Self-serve checkout requires employer auth — explicitly post-payments scope. | 0.85 |
| D-077 | Webhook signature = **HMAC-SHA256 over `"{timestamp}.{raw_body}"`** (Stripe's own scheme shape), verified with `hmac.compare_digest`; secret from `AI_WEBHOOK_SECRET` env, **unset → per-boot random** (the `telemetry_ingest_secret` pattern) — meaning webhooks fail closed when unconfigured, which is correct for a payment surface. Missing/bad signature → 401, event_id dedupe via UniqueConstraint + check-first (the `insert_application` at-least-once pattern). Raw body read via `await request.body()` (the resume route is the nearest raw-request precedent). | The v0.4.7 ingest-token pattern (per-boot secret, compare_digest, length-prefixed binding) is the proven in-tree HMAC precedent; Stripe's timestamp.body scheme is the industry shape and tolerates clock skew via a replay window (±5 min, pinned by test). | 0.80 |
| D-078 | Catalog = **seeded DB tables, not a code constant**: `packages` (one-time: single $49/1 post/30d, bundle $399/10 posts/90d each, enterprise) + `plans` (recurring: starter $99/mo, pro $299/mo, enterprise-talent custom) seeded idempotently at store init from a versioned code constant (the SeededCorpusSource `seed_corpus.json` pattern). Prices = integer cents, currency `usd` fixed. Purchases **snapshot** price/name at buy time (catalog edits never rewrite history). | The pricing page and admin checkout must read ONE source (data-driven tier copy, REQ-9-002); DB tables allow future admin price edits without code changes while snapshots keep financial history immutable. Code-constant-only would make REQ-9-002's "no hardcoded tier copy drifting from backend" harder to prove. | 0.80 |
| D-079 | Pricing page = **server shell retained** (static-export metadata stays — A-411) + new client component fetching `/v1/billing/catalog` (the A-406 board-client pattern: AbortController, honest loading/error/empty states). CTA buttons at pilot = **contact path** (mailto + "operator-mediated checkout" copy) — NOT self-serve purchase (no employer auth, D-076). The real checkout trigger lives in the admin surface (operator buys on behalf of an employer). | Self-serve CTAs without employer auth would be a dishonest button; the admin-mediated purchase is the honest pilot flow. The pricing page's job in v0.9 is truth: real catalog data, real states. | 0.80 |
| D-080 | Entitlement model: `purchases` table (one-time packages: `posts_limit`/`posts_used`/`expires_at` snapshot) + `subscriptions` (recurring plans). Posting creation consumes 1 post from the **newest non-expired purchase with remaining posts**, else 402 with upgrade-CTA dict payload (the `require_verified_age` dict-detail precedent); enterprise = manual unlimited grant (admin route, provenance-recorded). Store tables (7th D-027 family): `packages`, `plans`, `purchases`, `subscriptions`, `subscription_events`, `checkout_sessions`, `billing_events` — `purchases` + `billing_events` added to the REQ-9-001 list (one-time packages + full audit trail need their own tables; a one-time purchase is NOT a degenerate subscription). | Mixing one-time packs into subscriptions overloads the state machine (REQ-9-005's states don't apply to a 10-post bundle); the identity/store.py + marketplace/store.py replication pattern makes an 7-table store cheap. | 0.75 |
| D-081 | Money = **integer cents** end-to-end (pydantic `int` with ge=0, TS `number` documented as cents); no floats anywhere in billing; currency `usd` only (multi-currency out of scope). | Float money is the classic billing bug; Stripe itself is cents-native. The dual-schema pins (pins.test.ts) enforce the int wire type both sides. | 0.90 |
| D-082 | Mock checkout completion = **a mock-provider-only confirm route** (`POST /v1/billing/mock/complete`, mounted ONLY when provider=mock): it mints a properly-signed webhook event and POSTs it through the REAL webhook processing path (fetch to self or direct handler call — same code path, proven by test). This is the E2E payment proof without cloud. The route 404s when provider=stripe. | The pipeline must be provable end-to-end in tests (webhook → dedupe → entitlement); a confirm route mounted only in mock mode is the same shape as the sitemap's env-gated routes and can never be hit on a real deployment (factory mounts it conditionally). | 0.70 |
| D-083 | Subscription lifecycle: states `trialing → active → past_due → cancelled|expired` (+ `upgraded`), transitions driven ONLY by webhook events or admin manual actions. **Event names are OUR canonical names, normalized at the provider boundary** (R-43: Stripe has no `subscription.renewed`/`checkout.completed`): `checkout.completed` ← `checkout.session.completed`; `invoice.paid` ← `invoice.paid` (also accept `invoice.payment_succeeded`); `subscription.cancelled` ← `customer.subscription.deleted`; `subscription.renewed` ← `invoice.paid` on a renewal invoice (corroborated by `customer.subscription.updated`); provider event ids stored + deduped verbatim (never normalized). Completion predicate `status=complete AND payment_status=paid` (R-37). `past_due` grace = 7 days (module constant, test-pinned) before expiry downgrade; renewals extend `expires_at` idempotently (event_id dedupe makes double-fire safe); manual grant/revoke records `source: "manual"` + operator provenance in `billing_events`. Invalid transitions → `InvalidTransition` → 409 (store precedent). | At-least-once webhooks + idempotent state transitions is the A-309 seq-lease lesson applied to billing; manual remediation must be audit-visible (REQ-9-005's "recorded as manual events with provenance"). Research R-41..R-45 verified the state/event mapping against Stripe's actual enums. | 0.80 |
| D-084 | Wire discipline: billing types in `packages/types/billing.ts` + pydantic contracts in `ai_service/billing/base.py`, dual-schema pinned field-for-field in `apps/web/tests/pins.test.ts` (G-108 discipline); `engine-client.ts` gains `listBillingCatalog()` + a `PaymentRequiredError` (402, dict-detail discrimination — the existing `parseError` VerifyRequiredError precedent). | Every prior cross-boundary wire (PostingSummary, SearchResult, facets) is pinned this way; the 402 CTA payload must be machine-readable for the admin surface. | 0.85 |
## Milestone v0.8 — Ship Anywhere (COMPLETE, Feature, 2026-09-27 → 2026-09-29)
**Scope (founder directive 2026-09-26, options 1–4 confirmed + scope decisions answered 2026-09-26/27):** Release and deployment guides for admins. nextcraft must be installable on any Debian-based Linux server, via every mainstream channel, with release consistency enforced by automation:
@@ -298,6 +321,7 @@ The following remain deferred beyond v0.3 and will be activated in subsequent mi
| D-014 | Provider-agnostic LLM layer; ollama-cloud as initial provider | OpenAI-compatible client abstraction with pluggable providers: ollama-cloud (https://ollama.com/v1, default), local OpenAI-compatible endpoint, deterministic mock (tests/CI). Keys in gitignored .ciagent/.env.secrets, never in code or commits. | apps/ai-service llm package with 3 providers; default=ollama-cloud |
| D-015 | All six agents implemented as real LLM services; engines mocked | Coach/Tutor/Mentor fully real. Lab/Assessor/Proctor are real LLM logic over mock inputs (simulated telemetry, pre-baked artifacts) since sandbox fabric, assessment engine, and identity verification are v0.3+. Consistent with v0.1's mock-data approach. | REQ-F-001..006 complete in v0.2; real engines deferred to v0.3+ |
| D-016 | v0.4 = Distribution & Bootstrap CLI (founder directive supersedes previously-named v0.4 seams) | Founder directive 2026-09-12: focus this milestone on streamlining install, a bootstrap CLI with a one-liner install script, ongoing release binaries. Real server STT/TTS, KYC, design/simulation envs, seq-lease move to v0.5. | Milestone scope locked at SPECIFY; binary = CLI-only, linux x64 |
| D-074 | v0.9 = Payment Engine (auto-decided at SPECIFY, full autonomy; founder scope question unanswered post-v0.8) | Activates REQ-F-018 as REQ-9-001..005: posting packages (single/bundle/enterprise), talent-access plans, subscription tiers — checkout, provider webhooks, entitlement enforcement on posting creation, subscription lifecycle. Provider-agnostic protocol, mock-first (D-014 pattern). Rationale: ≥1,000 placements requires a monetized self-serve employer side; payments is the only high-priority REQ deferred since v0.3 that unblocks employer self-serve (A-407) and paid postings precede CIRR tracking. Confidence 0.80 ≥ threshold 0.60 (D-007). | Milestone scope locked at SPECIFY; tags on v0.8.x line; feature milestone; branch milestone/v0.9-payment-engine |
| D-047 | v0.6 = Marketplace Engine (auto-decided at SPECIFY, full autonomy; founder scope question left unanswered 2026-09-21) | Activates REQ-F-011..016 as REQ-6-001..006: real marketplace backend (aggregation, tagging, vector search, enrichment, resume parsing, SEO pages). Rationale: the guiding outcome (≥1,000 placements via the marketplace) runs through the only surface with zero backend; every prior milestone fed the credential side, placement side is the bottleneck. Confidence 0.80 ≥ threshold 0.60 (D-007). External 3M+ job corpus scaled to pilot (adapter contract preserves the full-scale path). | Milestone scope locked at SPECIFY; tags on v0.5.x line; feature milestone |
| D-039 | Single-port deploy + unattended dev (v0.3.6 hotfix, founder directive) | Only :8420 reachable behind HAProxy; site was down (domain root hit the API 404). Web app = static export served same-origin by the ai-service (`AI_WEB_STATIC_DIR`); `dev -d`/`stop`/`log` daemonize ops; durable state (DB, sandboxes, pid/log) moves to `~/.nextcraft/` out of the repo. | One port serves UI+API; unattended deploy recipe in deploy/README.md |
+36 -2
View File
@@ -1,6 +1,40 @@
# Nextcraft — REQUIREMENTS.md
## v0.8 Requirements (ACTIVE — Ship Anywhere, Feature)
## v0.9 Requirements (ACTIVE — Payment Engine, Feature)
**Milestone type: Feature** — payment processing and subscription management (REQ-F-018 activated per D-074). Tags on the v0.8.x line (P0 → v0.8.0 … P6 → v0.8.6 = milestone release); branch `milestone/v0.9-payment-engine`. Provider-agnostic protocol, mock-first (D-014 pattern): tests never call Stripe; the real provider drops in by env config without API changes.
### Billing Foundation
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-9-001 | Billing foundation: `ai_service/billing/` module (D-031 pattern — extend ai-service, no new apps): `PaymentProvider` protocol (create checkout session, retrieve session, cancel subscription, upgrade/downgrade subscription, list subscription events) + deterministic mock provider (approves-on-policy, synchronous state machine); SQLite billing store (7th D-027 family — `packages`, `plans`, `subscriptions`, `subscription_events`, `checkout_sessions` tables) with SQLModel typed access; webhook signature verification (HMAC, per-boot secret default — the v0.4.7 ingest-token pattern); provider selection via `AI_PAYMENT_PROVIDER=mock\|stripe` env (keys env-only, never committed); test-only mode is the default (mock) so the stack runs with zero keys | critical | 1 | open |
### Catalog + Pricing Surface
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-9-002 | Package catalog + real pricing surface: seeded package catalog (single posting, bundle ×5, enterprise-unlimited tiers + talent-access plans: basic/pro/enterprise per REQ-017's existing pricing-page tiers) exposed via `/v1/billing/catalog`; the pricing page switches from mock imports to runtime client fetch (A-406 pattern) with honest loading/error states; package features/limits data-driven from the catalog (no hardcoded tier copy drifting from backend); build-time static metadata retained (A-411) | critical | 2 | open |
### Checkout Flow
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-9-003 | Checkout + webhook flow: `POST /v1/billing/admin/checkout` (admin token-gated per D-076 — operator buys on an employer's behalf; buyer recorded as operator ref + employer ref) creates a provider checkout session for a catalog item; mock provider renders an in-app confirmation path; stripe provider redirects to Stripe Checkout (env-gated, never in tests); `POST /v1/billing/webhook` receives provider events with HMAC signature verification (reject 401 on bad signature — never process unsigned), idempotent event processing (event_id dedupe + per-session grant idempotency per G-303), entitlement granted on `checkout.completed` with state resolved from local rows only (G-302 — event payloads are untrusted ID references); webhook replay tolerance (at-least-once semantics — the A-309 seq pattern applied to billing) | critical | 3 | open |
### Entitlement Enforcement
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-9-004 | Posting entitlement enforcement: posting creation (admin API route per A-401) checks the employer's active package — posting limit reached → honest 402 with upgrade-CTA payload (pointing at catalog/checkout); unlimited/enterprise bypass; expired/cancelled package degrades to the same 402 path; entitlement check composed after the admin token gate (operator route — D-043 learner gate composition untouched); admin surface shows per-employer entitlement state (active package, posts used/limit, expiry) in the existing admin marketplace moderation view | high | 4 | open |
### Subscription Lifecycle
| ID | Description | Priority | Phase | Status |
|----|-------------|----------|-------|--------|
| REQ-9-005 | Subscription lifecycle + billing admin: subscription states (trialing → active → past_due → cancelled/expired) with provider-driven transitions (webhook events `invoice.paid`, `invoice.payment_failed`, `subscription.cancelled`, `subscription.renewed`); renewals extend expiry idempotently; admin routes for manual grant/revoke/complement (operator remediation for failed webhooks — recorded as manual events with provenance); billing event log with full audit trail (every provider event + manual action stored, queryable); honest 402 + grace handling on payment failure (past_due grace window before downgrade) | high | 5 | open |
## v0.8 Requirements (Complete — Ship Anywhere, Feature; shipped as v0.7.6)
**Milestone type: Feature** — new distribution channels (docker image, deb packages, arm64 binaries) and release automation (Gitea Actions, pre-commit). Tags on the v0.7.x line (P0 → v0.7.0 … P6 → v0.7.6 = milestone release); branch `milestone/v0.8-ship-anywhere`.
@@ -290,7 +324,7 @@
| ID | Description | Priority | Milestone | Status |
|----|-------------|----------|-----------|--------|
| REQ-F-017 | Identity verification and age-gating (16+/18+) — real KYC backend | high | v0.5 | activated → complete (REQ-5-003/004) |
| REQ-F-018 | Payment processing and subscription management | high | v0.3+ | deferred |
| REQ-F-018 | Payment processing and subscription management | high | v0.9 | activated → REQ-9-001..005 |
| REQ-F-019 | Human tutor marketplace (third-party courses) | medium | v0.4+ | deferred |
| REQ-F-020 | CIRR-style placement tracking and audit | medium | v0.5+ | deferred |
+195 -1
View File
@@ -1,4 +1,198 @@
# Nextcraft — RESEARCH.md (v0.8 Phase 0 RESEARCH — Ship Anywhere)
# Nextcraft — RESEARCH.md (v0.9 Phase 0 RESEARCH — Payment Engine)
Web-verified census of the payment/billing surface, 2026-09-29. Every
conclusion carries its source (docs.stripe.com page or in-tree file probe).
Binding clarify decisions D-075..D-084 are NOT re-litigated; this research
operationalizes them. Upstream docs consulted: docs.stripe.com —
api/checkout/sessions (create/retrieve/object), webhooks (manual signature
verification + replay), api/subscriptions (object/cancel/update),
billing/subscriptions/webhooks, billing/revenue-recovery/smart-retries,
api/idempotent_requests, api/events/types, keys, billing/testing/test-clocks.
In-tree probes grounding the module design: api/marketplace_admin.py
(require_admin dep), api/deps.py, marketplace/store.py + identity/store.py
(D-027 family), config.py (Settings env trio precedents),
telemetry/tokens.py (HMAC), api/telemetry.py (WS ingest verify),
marketplace/store.py insert_application (at-least-once dedupe precedent),
main.py (lifespan store wiring + inline router mounts + G-11 fallback),
voice/factory.py + marketplace/embedding/factory.py (provider factory),
apps/web pricing page.tsx (inline JOB_PLANS/TALENT_PLANS constants),
lib/engine-client.ts (jsonFetch + parseError), tests/marketplace/conftest.py
(make_client + state injection), tests pins.test.ts (dual-schema pins).
## R-33..R-50 — Stripe-compatible protocol facts (web-verified)
**R-33 (high).** Checkout Session REST: `POST /v1/checkout/sessions`
(create) + `GET /v1/checkout/sessions/{id}` (retrieve). Auth = HTTP Basic
(`-u sk_...:`) or Bearer. Body is **form-encoded** with bracket nesting
(`line_items[0][price_data][currency]=usd`) — NOT JSON. The stripe provider
must encode bodies accordingly; MockTransport byte-contract tests assert the
exact encoded body + Authorization header.
**R-34 (high).** `mode` required, enum `payment|setup|subscription`;
`line_items` required for payment/subscription; `success_url` required
(hosted); `client_reference_id` optional ≤200 chars — our reconciliation key
(checkout_session row id); `metadata` optional map; `expires_at` 30m–24h.
**R-35 (high).** line item: exactly one of `price` (existing Price ID) or
`price_data` {currency (lowercase ISO), unit_amount (int, smallest currency
unit), product_data {name}, recurring {interval: day|week|month|year} for
subscriptions}. quantity optional. One-time vs recurring differs only by the
presence of `price_data[recurring][interval]` — matches D-078's catalog
shape (packages vs plans as one table family with an interval field).
**R-36 (high).** Session response fields we normalize: `id` (`cs_...`),
`url` (hosted checkout URL, present only while open), `status`
(`open|complete|expired`), `payment_status` (`paid|unpaid|
no_payment_required`), `customer` (nullable), `subscription` (nullable,
subscription mode only), `mode`, `amount_total`, `currency`, `metadata`,
`expires_at`, `livemode`.
**R-37 (high).** Completion predicate: `status == "complete" AND
payment_status == "paid"` — never `status` alone (async methods can leave
`complete` with payment still processing; `checkout.session.
async_payment_succeeded` fires later). The billing webhook handler uses this
predicate on `checkout.session.completed` before granting entitlements.
**R-38 (high).** Webhook header: `Stripe-Signature: t=<unix_seconds>,
v1=<hex>` on one line; multiple `v1` during secret rolls; a fake `v0` scheme
exists on test events — ignore all non-`v1` schemes (downgrade protection).
**R-39 (high).** Signed payload = `"{timestamp}.{raw body}"`; HMAC-SHA256
with the endpoint secret (`whsec_...`); constant-time comparison; raw body
must be un-parsed bytes at verification time. **D-077's scheme is
byte-for-byte Stripe's** — adopt as-is; the in-tree precedent
(telemetry/tokens.py) supplies the mint/verify/compare_digest shape.
**R-40 (high).** Replay tolerance: official default **300 s**; "Don't use a
tolerance of 0". Each delivery/retry re-signs (new t + new v1) — attackers
cannot alter t without invalidating. D-077's ±5 min = Stripe default.
**R-41 (high).** Stripe subscription.status enum: `incomplete`,
`incomplete_expired` (23h terminal after failed initial payment),
`trialing`, `active`, `past_due`, `canceled`, `unpaid`, `paused`.
**R-42 (high).** State normalization (design choice on verified facts):
our `trialing` ← Stripe `trialing`/`incomplete`; `active` ← `active`;
`past_due` ← `past_due`; `cancelled` ← `canceled`; `expired` ←
`incomplete_expired`/`unpaid`. D-083's five states map cleanly.
**R-43 (high).** Event normalization — Stripe has no `subscription.renewed`
or `checkout.completed` events; D-083's names are OUR canonical names,
mapped at the provider boundary:
- `checkout.completed` ← `checkout.session.completed`
- `invoice.paid` ← `invoice.paid` (canonical provisioning signal;
`invoice.payment_succeeded` also fires — accept both, treat as one)
- `invoice.payment_failed` ← `invoice.payment_failed`
- `subscription.cancelled` ← `customer.subscription.deleted`
- `subscription.renewed` ← `invoice.paid` on a renewal invoice
(corroborated by `customer.subscription.updated`)
**Action recorded against D-083: the provider protocol carries a
`normalize_event()` boundary; webhook processing stores + dedupes provider
event ids verbatim (never normalized) so dedupe stays truthful.**
**R-44 (medium).** Dunning: after retries exhaust, subscription becomes
`canceled` or `unpaid` per Dashboard settings; Smart Retries default ~8
retries/2 weeks. Our mock dunning = D-083's 7-day past_due grace (module
constant), independent of Stripe settings.
**R-45 (high).** Event delivery: NO ordering guarantee; duplicates happen —
dedupe by event `id`; return 2xx fast before complex logic; retries
exponential backoff (up to 3 days live). Confirms D-077's event_id dedupe +
D-083's idempotent transitions; webhook route returns 200 immediately after
persisting the event row (processing is synchronous but bounded).
**R-46 (high).** Cancel: `DELETE /v1/subscriptions/{id}` (immediate;
`invoice_now`/`prorate` default false) → status `canceled`; or deferred:
POST update `cancel_at_period_end=true` (stays active until period end, then
`customer.subscription.deleted`). Protocol `cancel_subscription(id,
at_period_end=False)` maps 1:1.
**R-47 (high).** Upgrade/downgrade: `POST /v1/subscriptions/{id}` with
`items[0][id]=<subscription_item_id>` + `items[0][price]=<new price>` —
**including the item id is mandatory** (else the new price is ADDED
alongside the old → dual-charge bug). `proration_behavior`:
`create_prorations` (default) | `none` | `always_invoice`. Failed immediate
payment → update succeeds, subscription `past_due`. Protocol
`update_subscription(subscription_id, new_price_ref)` must resolve the
subscription_item id inside the provider (the route must not have to know
it) — recorded as a byte-contract test case.
**R-48 (high).** `unit_amount` = integer in the smallest currency unit —
cents for USD (`unit_amount: 1000` = $10.00). Confirms D-081 (integer cents
end-to-end; no floats anywhere in billing).
**R-49 (high).** `Idempotency-Key` header on all POSTs; Stripe replays the
saved response for 24h on key reuse; same key + different params errors;
no key on a double-submit creates two sessions. The stripe provider sends
`Idempotency-Key: <uuid4>` per logical checkout/subscription operation.
**R-50 (high).** Test mode (`sk_test_` keys) is an isolated sandbox — card
networks never process; every object carries `livemode: false`. Combined
with MockTransport-only tests (D-075), no test path can ever reach Stripe.
Test clocks exist for time-simulated billing — noted as precedent; our
tests use MockTransport + the deterministic mock exclusively.
## R-51..R-54 — In-tree integration facts (probe-verified)
**R-51.** Admin gate composition (REQ-9-003/004 operator routes): the
`require_admin` FastAPI **dependency** (api/marketplace_admin.py:46-85) —
`X-Admin-Token` or `Authorization: Bearer`; fail-closed when
`AI_ADMIN_TOKEN` unset (403 even with a candidate token);
`secrets.compare_digest` constant-time UTF-8 compare; returns `Settings`.
Billing admin routes copy this dep verbatim; billing entitlement check
composes AFTER it (REQ-9-004).
**R-52.** Store wiring: 7th D-027 store = `billing/store.py` mirroring
marketplace/store.py — SQLModel table classes in-module (`__tablename__`,
UniqueConstraints for idempotency keys, `sa_type=String` + `_validate()`
instead of bare Literal annotations, `_as_utc()` re-labeling,
`_sqlite_connect` pragmas WAL/NORMAL/busy_timeout/foreign_keys); Protocol +
SQLiteImpl pair; `Settings().db_path` shared file; main.py lifespan
state-injection override guard + close() in finally; `get_billing_store`
dep in api/deps.py ("routes never import stores directly" territory rule).
**R-53.** Provider factory + settings trio: config.py gains
`payment_provider: str = "mock"` + `payment_api_key: SecretStr = SecretStr("")`
+ `webhook_secret: str = ""` (per-boot random when unset — the
telemetry_ingest_secret shape). `billing/factory.py` mirrors
voice/factory.py: `UnknownPaymentProviderError(ValueError)`; main.py
lifespan catches factory errors → loud G-11 log → falls back to mock
(R-51 boot-warning pattern; voice/embedding precedent at main.py:138-154).
**R-54.** Pricing page conversion (REQ-9-002): page.tsx currently holds
inline `JOB_PLANS`/`TALENT_PLANS` constants (lines 41-111) +
`COMPARISON` matrix; conversion = A-406 board-client pattern — page stays a
server shell (metadata retained per A-411, pinned by pins.test.ts:189-190
+ sitemap STATIC_PAGES), new client component fetches `/v1/billing/catalog`
via engine-client `jsonFetch` with honest loading/error/empty states;
`listBillingCatalog()` + `PaymentRequiredError` (402 dict-detail) added to
engine-client.ts (parseError VerifyRequiredError precedent, lines 90-123);
wire types in packages/types/billing.ts dual-schema-pinned in pins.test.ts.
## Open risks (tracked into PLAN)
1. **Webhook raw-body discipline**: signature verification needs the exact
raw bytes — any FastAPI body-parsing middleware (or pydantic model
binding) before verification breaks it. The webhook route reads
`await request.body()` FIRST, verifies, then parses JSON. Tracked as a
P3 must-have + probe test.
2. **Dual-charge on upgrade** (R-47): `items[0][id]` omission is the
classic Stripe bug — byte-contract test must assert the item id is
present in the update body.
3. **Webhook ordering** (R-45): `invoice.payment_failed` can arrive before
`invoice.paid` under retries; transitions must be event-driven state
machine transitions (InvalidTransition → 409 tolerated as already-reflected
duplicates, recorded not raised) — the at-least-once lesson.
4. **Clock skew in tests**: signature timestamps use `now` injection
(protocol takes `now: float | None = None`) so replay-window tests are
deterministic without sleeps.
5. **Secret rotation**: multiple `v1` schemes (R-38) — verify() iterates
ALL v1 values, accepts any match (supports secret rolls without
downtime); test-pinned.
---
## v0.8 Phase 0 RESEARCH (Ship Anywhere — prior milestone, retained below)
Probe-verified census of the release/distribution surface, 2026-09-27.
Every conclusion carries evidence (file paths probed in this repo, live HTTP
+20
View File
@@ -28,6 +28,26 @@
---
## v0.9 (Active — Payment Engine)
Payment processing and subscription management (REQ-F-018 activated per D-074, auto-decided at SPECIFY under full autonomy): posting packages (single/bundle/enterprise), talent-access plans, subscription tiers with real checkout, provider webhooks (HMAC-verified, idempotent), posting-creation entitlement enforcement, and a full subscription lifecycle (renew/cancel/upgrade/past_due grace) — all behind a provider-agnostic `PaymentProvider` protocol, mock-first (D-014 pattern; tests never call Stripe). The pricing page (REQ-017) gains its real backend. Out of scope: employer auth/accounts (B2B network — post-payments per A-407), multi-currency, tax/VAT beyond passthrough, invoice PDFs, CIRR tracking (REQ-F-020). Tags on the v0.8.x line (P0 → v0.8.0 … P6 → v0.8.6 = milestone release); branch `milestone/v0.9-payment-engine`.
## Phase List
### v0.9 (active)
| # | Name | Status | Depends On | Requirements | Success Criteria |
|---|------|--------|------------|--------------|------------------|
| 0 | Pre-execution | in progress | — | — | Specify (D-074), clarify (D-075+), research (R-33+), plan, grill (G-301+), MVP/UX check complete; shipped as v0.8.0 |
| 1 | Billing foundation | pending | 0 | REQ-9-001 | `ai_service/billing/` module: PaymentProvider protocol + deterministic mock + SQLite store (7th D-027 family) + HMAC webhook verification + env-gated provider selection; tests never call a real provider |
| 2 | Catalog + pricing surface | pending | 1 | REQ-9-002 | Seeded catalog (posting packages + talent-access plans) via `/v1/billing/catalog`; pricing page runtime client fetch with honest loading/error states; tier copy data-driven |
| 3 | Checkout + webhook flow | pending | 2 | REQ-9-003 | `POST /v1/billing/admin/checkout` (admin-gated, D-076) + `POST /v1/billing/webhook` (HMAC-verified, idempotent event processing, at-least-once tolerance, local-row state resolution per G-302); entitlement granted on checkout.completed |
| 4 | Entitlement enforcement | pending | 3 | REQ-9-004 | Posting creation enforces package limits (honest 402 + upgrade CTA); expired/cancelled degrade to 402; admin surface shows per-employer entitlement state |
| 5 | Subscription lifecycle | pending | 4 | REQ-9-005 | Subscription state machine (trialing/active/past_due/cancelled/expired); idempotent renewals; manual admin grant/revoke with provenance; full billing event audit log; past_due grace window |
| 6 | Final review + ship | pending | 1-5 | — | Review clean; audit pass; milestone tagged v0.8.6 (final v0.8.x patch = milestone release); release with full asset matrix + registry image; milestone merged to main |
---
## v0.8 (Complete — Shipped as v0.7.6)
Ship Anywhere: installable on any Debian-based Linux server via docker (Gitea registry image, honest sandbox degrade per D-068), .deb packages (amd64+arm64, advisory postinst per D-067), and arch-aware binary (x64+arm64, R-24 arch-resolve) — with release consistency enforced by pre-commit hooks + Gitea Actions (3-runner fleet on CT 113 as committed fleet-as-code, machine-made releases with the 8-asset matrix + registry image on every release), the CLI carrying the full suite (guide + channel-aware doctor, D-066/D-072), and the evidence-backed admin runbook (G-210 parity-pinned). 7 phases (P0–P6), all 5 requirements (REQ-8-001..005) complete. Tags v0.7.0–v0.7.5 per phase, milestone release v0.7.6 (release 936). Final review CLEAN (G-201..G-210 honored — the smoke gate caught 4 real image bugs pre-ship, each fixed in-phase); audit PASS (reconstruction + discipline green).
+3 -3
View File
@@ -46,9 +46,9 @@
"projects": [],
"active_project": null,
"milestone": {
"version": "v0.8",
"name": "ship-anywhere",
"version": "v0.9",
"name": "payment-engine",
"type": "feature",
"branch": "milestone/v0.8-ship-anywhere"
"branch": "milestone/v0.9-payment-engine"
}
}