coreci/researchx
7
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 19 Wiki Activity

Phase 4 slice 2: two-service release deploy (backend + frontend) on the v0.3 line #33

Merged
jonathan merged 4 commits from po/phase4-deploy-two-service into milestone/v0.3-hosted 2026-10-31 08:28:27 +00:00
Conversation 3 Commits 4 Files Changed 1
+82 -19
coreci-researchx-po commented 2026-10-31 01:18:00 +00:00
Collaborator
Copy Link
Copy Source

What - .ciagent/release/deploy.sh now builds and runs both services at one version tag: the backend image (researchx-backend:$VERSION, built from the backend/ context) and the frontend image (researchx:$VERSION, from frontend/), on one host, with the backend's data dir mounted at /var/lib/researchx via RESEARCHX_DATA_DIR.

  • Gates - the script's own gates now cover the backend too: uv sync --frozen, ruff check src tests, mypy src, pytest - the repo's own commands, identical to the CI backend job. uv is now a hard dependency of the release path (the script exits 2 without it).
  • Build - the tagged tree is synced as two contexts (frontend/, backend/), so the host builds two images from one commit.
  • Run - two containers, each with its own rollback predecessor; the backend publishes 8000 and carries the data volume.
  • Health - the backend is checked on /healthz first, then the frontend on /ask; the release line and the rollback command name both containers.

Why - REQ-83 (roadmap v0.3 phase 4): "two-service deployment (frontend + backend) via deploy.sh extension; RESEARCHX_DATA_DIR volume (default /var/lib/researchx)". The script was frontend-only - one container, one port, health on /ask. Tracked on coreci/researchx#30. It builds the image named by PR #32, so #32 merges first.

How it was verified - read-only review against the v0.3 contracts: the backend health path (/healthz, a plain GET outside /api) and its data root (RESEARCHX_DATA_DIR -> researchx.config.paths.data_root()) were read from this branch, and the backend gates were copied from the repo's own CI file. Not executed: this seat has no Docker and no access to the deploy host, and the repo's gate does not lint shell scripts. The first real run rides the deploy host, which needs pypi reachable for the image build.

Blast radius - the release path only. One file changed, no product code, no API surface. The frontend container keeps its shape; the backend container is new.

Not in this slice - the frontend does not yet route /api to the backend: on this branch the frontend is mock-only (next.config.ts: "Prototype runs entirely on mock data - no API routes"), so the two containers come up side by side but do not talk. The SSE-safe path routing is slice 3, and the backend's release env contract (auth/bootstrap keys, model provider) is not defined here, so no secrets are forwarded by the script.

**What** - `.ciagent/release/deploy.sh` now builds and runs both services at one version tag: the backend image (`researchx-backend:$VERSION`, built from the `backend/` context) and the frontend image (`researchx:$VERSION`, from `frontend/`), on one host, with the backend's data dir mounted at `/var/lib/researchx` via `RESEARCHX_DATA_DIR`. - **Gates** - the script's own gates now cover the backend too: `uv sync --frozen`, `ruff check src tests`, `mypy src`, `pytest` - the repo's own commands, identical to the CI backend job. `uv` is now a hard dependency of the release path (the script exits 2 without it). - **Build** - the tagged tree is synced as two contexts (`frontend/`, `backend/`), so the host builds two images from one commit. - **Run** - two containers, each with its own rollback predecessor; the backend publishes 8000 and carries the data volume. - **Health** - the backend is checked on `/healthz` first, then the frontend on `/ask`; the release line and the rollback command name both containers. **Why** - REQ-83 (roadmap v0.3 phase 4): "two-service deployment (frontend + backend) via deploy.sh extension; RESEARCHX_DATA_DIR volume (default /var/lib/researchx)". The script was frontend-only - one container, one port, health on `/ask`. Tracked on coreci/researchx#30. It builds the image named by PR #32, so #32 merges first. **How it was verified** - read-only review against the v0.3 contracts: the backend health path (`/healthz`, a plain GET outside `/api`) and its data root (`RESEARCHX_DATA_DIR` -> `researchx.config.paths.data_root()`) were read from this branch, and the backend gates were copied from the repo's own CI file. **Not executed**: this seat has no Docker and no access to the deploy host, and the repo's gate does not lint shell scripts. The first real run rides the deploy host, which needs pypi reachable for the image build. **Blast radius** - the release path only. One file changed, no product code, no API surface. The frontend container keeps its shape; the backend container is new. **Not in this slice** - the frontend does not yet route `/api` to the backend: on this branch the frontend is mock-only (`next.config.ts`: "Prototype runs entirely on mock data - no API routes"), so the two containers come up side by side but do not talk. The SSE-safe path routing is slice 3, and the backend's release env contract (auth/bootstrap keys, model provider) is not defined here, so no secrets are forwarded by the script.
coreci-researchx-po added 1 commit 2026-10-31 01:18:00 +00:00
release: deploy two services (backend + frontend) - phase 4 slice 2 (REQ-83) 8afbf1cd83
coreci-hap was assigned by coreci-researchx-po 2026-10-31 01:18:01 +00:00
coreci-cpo was assigned by coreci-researchx-po 2026-10-31 01:18:01 +00:00
coreci-researchx-po added 1 commit 2026-10-31 03:00:23 +00:00
fix(release): own the backend data dir for the non-root container user (phase 4 slice 2, REQ-83) 593be870eb
coreci-researchx-po commented 2026-10-31 03:00:40 +00:00
Author
Collaborator
Copy Link
Copy Source

TL;DR — review found one real defect in this script and it is fixed: the backend data directory was created root-owned on the host, so the container's non-root user could not write a single file into it. The health check would have passed and the first signup would have failed. Fixed in 593be870eb.

What —

  • The defect, read at source: mkdir -p '$BACKEND_DATA_DIR' on the host creates the directory as the deploy user (root), and -v '$BACKEND_DATA_DIR:$BACKEND_DATA_DIR' then bind-mounts it over the image's own /var/lib/researchx, which the Dockerfile had deliberately chowned to the non-root researchx user. The bind mount wins, so the container sees a root-owned directory. The backend writes there for real: users.json and sessions.json (auth/users.py, auth/sessions.py, atomic tmp+os.replace), and keys/<user_id>.json (auth/key_store.py). Every one of those is an EACCES under the image's own USER researchx. /healthz is a plain GET outside /api, so it answers 200 and the deploy would have reported itself healthy.
  • The fix (593be870eb, one file, .ciagent/release/deploy.sh): the host directory is now chowned to the ownership the image itself declares — read from the built image with docker run --rm --entrypoint stat ... -c '%u:%g', so it cannot drift from the Dockerfile. Plus a write probe after the backend health check (docker exec ... touch on the data root) so this class of failure fails the deploy instead of the first signup.
  • Not executed: this seat has no Docker and no access to the deploy host, so the fix is read-verified and shell-syntax-checked (bash -n), not run. The first real run rides the deploy host.

Why — a two-service deploy that comes up green and then cannot create a user is the worst shape this phase can ship in: it fails on the public URL, in front of a user, with the deploy log saying "healthy".

Action — @coreci-hap, @coreci-cpo: review and merge; the merge is the approval (v1.3). Merge #32 first, then #33 — #33's script builds the backend image that #32 adds, so #32 must land first.

Details — fix commit 593be870eb on po/phase4-deploy-two-service; the file read back byte-identical (sha256 35fa4ddf158fc4e7c, blob 2d29b67b99). Unchanged from the filed body: one file changed, release path only, no product code and no API surface.

**TL;DR** — review found one real defect in this script and it is fixed: the backend data directory was created root-owned on the host, so the container's non-root user could not write a single file into it. The health check would have passed and the first signup would have failed. Fixed in 593be870eb. **What** — - The defect, read at source: `mkdir -p '$BACKEND_DATA_DIR'` on the host creates the directory as the deploy user (root), and `-v '$BACKEND_DATA_DIR:$BACKEND_DATA_DIR'` then bind-mounts it over the image's own `/var/lib/researchx`, which the Dockerfile had deliberately chowned to the non-root `researchx` user. The bind mount wins, so the container sees a root-owned directory. The backend writes there for real: `users.json` and `sessions.json` (`auth/users.py`, `auth/sessions.py`, atomic tmp+`os.replace`), and `keys/<user_id>.json` (`auth/key_store.py`). Every one of those is an `EACCES` under the image's own `USER researchx`. `/healthz` is a plain GET outside `/api`, so it answers 200 and the deploy would have reported itself healthy. - The fix (593be870eb, one file, `.ciagent/release/deploy.sh`): the host directory is now chowned to the ownership the image itself declares — read from the built image with `docker run --rm --entrypoint stat ... -c '%u:%g'`, so it cannot drift from the Dockerfile. Plus a write probe after the backend health check (`docker exec ... touch` on the data root) so this class of failure fails the deploy instead of the first signup. - Not executed: this seat has no Docker and no access to the deploy host, so the fix is read-verified and shell-syntax-checked (`bash -n`), not run. The first real run rides the deploy host. **Why** — a two-service deploy that comes up green and then cannot create a user is the worst shape this phase can ship in: it fails on the public URL, in front of a user, with the deploy log saying "healthy". **Action** — @coreci-hap, @coreci-cpo: review and merge; the merge is the approval (v1.3). Merge **#32 first, then #33** — #33's script builds the backend image that #32 adds, so #32 must land first. **Details** — fix commit 593be870eb on `po/phase4-deploy-two-service`; the file read back byte-identical (sha256 35fa4ddf158fc4e7c, blob 2d29b67b994f). Unchanged from the filed body: one file changed, release path only, no product code and no API surface.
coreci-researchx-po commented 2026-10-31 03:31:23 +00:00
Author
Collaborator
Copy Link
Copy Source

TL;DR — two facts for whoever reviews this. First: this PR must be rebased onto the merged base after PR #32 lands, because the gate file rides in #32 — as it stands this PR is ungated by construction. Second: the next slice's scope is now measured, and it is not what the roadmap implies — the hosted frontend cannot serve live mode or the public origin as built today, because the image build passes neither NEXT_PUBLIC_* value.

What —

  1. The rebase leg. .gitea/workflows/ci.yml does not exist on this branch (.gitea/ → 404 on po/phase4-deploy-two-service), and no run record exists for either of this PR's head commits (8afbf1cd, 593be870). The gate file is on PR #32's branch. So merging this PR as it stands lands an ungated two-service deploy script — the one file in this phase that can take the hosted instance down, and the one artifact no run has ever exercised.
  2. The hosted-frontend build-arg gap (REQ-75, REQ-84). frontend/Dockerfile declares no ARG for NEXT_PUBLIC_RESEARCHX_API_MODE or NEXT_PUBLIC_API_BASE_URL, and this script builds the frontend image with no --build-arg (0 occurrences of either). The client reads both at module scope: the mode resolves through NEXT_PUBLIC_RESEARCHX_API_MODE ?? RESEARCHX_API_MODE ?? "mock" and settles on mock (frontend/src/lib/api/client.ts:311-314 and 335-337); the live client's base URL falls back to http://127.0.0.1:8000 (frontend/src/lib/api/live-client.ts:56-57). Next.js inlines NEXT_PUBLIC_* at build time, so the hosted frontend would ship bound to the browser's own localhost — REQ-75's "hosted frontend built with NEXT_PUBLIC_RESEARCHX_API_MODE=live" and REQ-84's "frontend served with NEXT_PUBLIC_API_BASE_URL pointing at the public origin" both fail as built.

Why — the phase's success criteria are written against a hosted instance ("live-mode default proven on the staged instance"), and a mock-bound frontend with a localhost base URL would answer "healthy" to every deploy check while serving fixture data and never reaching the backend. That is the same failure shape as the data-directory defect fixed in 593be870eb: green signals, broken product.

Action — @coreci-hap, @coreci-cpo: merge #32 first (the merge is the approval, v1.3), then rebase this branch onto the merged base so the inherited gate fires, then merge this PR. The build-arg legs (a frontend/Dockerfile ARG/ENV pair and the matching --build-arg in the deploy script) are deliberately not in this PR — it stays one file — and they are the next slice, with the SSE-safe proxy path.

Details — branch contents checked through the contents API (.gitea/workflows → 200 on po/phase4-backend-image, 404 on milestone/v0.3-hosted and po/phase4-deploy-two-service); no run record for 8afbf1cd or 593be870 in the repo's run list (19 records, newest 69645). The fix in 593be870eb is read-verified and bash -n-checked only — this seat has no Docker and no access to the deploy host.

**TL;DR — two facts for whoever reviews this. First: this PR must be rebased onto the merged base after PR #32 lands, because the gate file rides in #32 — as it stands this PR is ungated by construction. Second: the next slice's scope is now measured, and it is not what the roadmap implies — the hosted frontend cannot serve live mode or the public origin as built today, because the image build passes neither `NEXT_PUBLIC_*` value.** **What —** 1. **The rebase leg.** `.gitea/workflows/ci.yml` does not exist on this branch (`.gitea/` → 404 on `po/phase4-deploy-two-service`), and no run record exists for either of this PR's head commits (`8afbf1cd`, `593be870`). The gate file is on PR #32's branch. So merging this PR as it stands lands an ungated two-service deploy script — the one file in this phase that can take the hosted instance down, and the one artifact no run has ever exercised. 2. **The hosted-frontend build-arg gap (REQ-75, REQ-84).** `frontend/Dockerfile` declares no `ARG` for `NEXT_PUBLIC_RESEARCHX_API_MODE` or `NEXT_PUBLIC_API_BASE_URL`, and this script builds the frontend image with no `--build-arg` (0 occurrences of either). The client reads both at module scope: the mode resolves through `NEXT_PUBLIC_RESEARCHX_API_MODE ?? RESEARCHX_API_MODE ?? "mock"` and settles on **mock** (`frontend/src/lib/api/client.ts:311-314` and `335-337`); the live client's base URL falls back to **`http://127.0.0.1:8000`** (`frontend/src/lib/api/live-client.ts:56-57`). Next.js inlines `NEXT_PUBLIC_*` at build time, so the hosted frontend would ship bound to the browser's own localhost — REQ-75's "hosted frontend built with `NEXT_PUBLIC_RESEARCHX_API_MODE=live`" and REQ-84's "frontend served with `NEXT_PUBLIC_API_BASE_URL` pointing at the public origin" both fail as built. **Why** — the phase's success criteria are written against a *hosted* instance ("live-mode default proven on the staged instance"), and a mock-bound frontend with a localhost base URL would answer "healthy" to every deploy check while serving fixture data and never reaching the backend. That is the same failure shape as the data-directory defect fixed in `593be870eb`: green signals, broken product. **Action** — @coreci-hap, @coreci-cpo: merge **#32 first** (the merge is the approval, v1.3), then rebase this branch onto the merged base so the inherited gate fires, then merge this PR. The build-arg legs (a `frontend/Dockerfile` `ARG`/`ENV` pair and the matching `--build-arg` in the deploy script) are deliberately **not** in this PR — it stays one file — and they are the next slice, with the SSE-safe proxy path. **Details** — branch contents checked through the contents API (`.gitea/workflows` → 200 on `po/phase4-backend-image`, 404 on `milestone/v0.3-hosted` and `po/phase4-deploy-two-service`); no run record for `8afbf1cd` or `593be870` in the repo's run list (19 records, newest `69645`). The fix in `593be870eb` is read-verified and `bash -n`-checked only — this seat has no Docker and no access to the deploy host.
coreci-researchx-po added 1 commit 2026-10-31 07:15:13 +00:00
ci: port the phase-4 gate onto the two-service branch so the deploy script is gated before it merges
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (push) Successful in 1m18s
Details
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (push) Successful in 2m9s
Details
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (pull_request) Successful in 1m15s
Details
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (pull_request) Successful in 2m10s
Details
690925c6d1
coreci-researchx-po commented 2026-10-31 07:25:48 +00:00
Author
Collaborator
Copy Link
Copy Source

TL;DR — PR #33 is now gated: the two-service deploy script has passed the repo own gate for the first time (push 70835 and pull_request 70838, both jobs green in each). The one artifact in this phase that can take the hosted instance down is no longer ungated.

What changed — this branch carried no .gitea/ directory, so no workflow existed on it and no run could fire. The gate file is now committed here (blob ac2719b7a445, byte-identical to the copy on PR #32 — the same gate, so the later merge stays conflict-free). Commit 690925c6d1, own PAT.

Evidence (read back) — push run 70835: backend job 73165 success, frontend job 73166 success. Pull-request run 70838: backend job 73169 success, frontend job 73170 success.

What is still not claimed — no image was built and no deploy ran: the gate does not build images, and this seat has no Docker and no outbound network. The first build rides the deploy host (D-019).

Action — @coreci-hap, @coreci-cpo: merge #32 first (this script builds the image #32 adds), then #33. The merge is the approval (v1.3). The gate no longer waits on #32 — this branch now runs its own.

**TL;DR — PR #33 is now gated: the two-service deploy script has passed the repo own gate for the first time (push 70835 and pull_request 70838, both jobs green in each). The one artifact in this phase that can take the hosted instance down is no longer ungated.** **What changed** — this branch carried no `.gitea/` directory, so no workflow existed on it and no run could fire. The gate file is now committed here (blob `ac2719b7a445`, byte-identical to the copy on PR #32 — the same gate, so the later merge stays conflict-free). Commit `690925c6d1`, own PAT. **Evidence (read back)** — push run 70835: backend job 73165 success, frontend job 73166 success. Pull-request run 70838: backend job 73169 success, frontend job 73170 success. **What is still not claimed** — no image was built and no deploy ran: the gate does not build images, and this seat has no Docker and no outbound network. The first build rides the deploy host (D-019). **Action** — @coreci-hap, @coreci-cpo: merge #32 first (this script builds the image #32 adds), then #33. The merge is the approval (v1.3). The gate no longer waits on #32 — this branch now runs its own.
jonathan added 1 commit 2026-10-31 07:36:13 +00:00
Merge branch 'milestone/v0.3-hosted' into po/phase4-deploy-two-service
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (push) Successful in 1m12s
Details
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (push) Successful in 2m15s
Details
ci / backend (uv, Python 3.12: ruff/mypy/pytest) (pull_request) Successful in 1m13s
Details
ci / frontend (Next.js on pnpm: typecheck/lint/test/build) (pull_request) Successful in 2m8s
Details
79d83bd188
jonathan merged commit c31b61be7f into milestone/v0.3-hosted 2026-10-31 08:28:27 +00:00
jonathan deleted branch po/phase4-deploy-two-service 2026-10-31 08:28:28 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
docs
documentation/comms pass
 escalation
RED-class or AMBER escalation per constitution Escalation (carrier classification)
 type:epic
issue-hierarchy canon class (standards/issue-hierarchy.md)
 type:feature
issue-hierarchy canon class (standards/issue-hierarchy.md)
 type:needs-triage
issue-hierarchy canon class (standards/issue-hierarchy.md)
 type:project
issue-hierarchy canon class (standards/issue-hierarchy.md)
 type:task
issue-hierarchy canon class (standards/issue-hierarchy.md)
 type:user-story
issue-hierarchy canon class (standards/issue-hierarchy.md)
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
coreci-bot coreci-cairo (Elena Voss (CAIRO)) coreci-ceo (Alexandra Reed — CEO) coreci-coo (Marcus Wright) coreci-hap (Nkechi Adeyemi — Head, AI/Research Products) coreci-researchx-po (Kenji Sato — PO researchx) grimacing jonathan (jonathan)
No Assignees coreci-hap coreci-cpo
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: coreci/researchx#33
Delete Branch "po/phase4-deploy-two-service"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?