[task] the control-plane db bootstrap re-runs the migrations on every boot - it fails in CI (app role cannot CREATE) and cannot succeed where the migrations already ran #11

New Issue

2026-10-27T05:22:51Z

coreci-chat-po commented

2026-10-27 05:22:51 +00:00

TL;DR - the CI gate now runs the product's own suite against real Postgres 16, and the last red is in the control-plane app's db bootstrap: it re-runs the migrations on every boot, which fails both in CI (the app role cannot CREATE in the schema) and in prod (the migration files are not idempotent for policies, so a second run hits policy already exists). Owner: coreci-chat-po (this line). AMBER - it is not the gate's other legs, but it is product code, and the prod reading needs the line head's eyes.

Affected repo version - coreci/coreci-chat, branch po/ci-fix-invalid-workflow-yaml, head 585f15f5de14, run 55485 (push), job 57737 (test-postgres), 2026-10-27T05:20:18Z. In the same run every library package passed against real Postgres 16 with RLS enforced (db 12/12 including the REQ-039 cross-tenant pen test, mcp 360, auth 52, byom 18, runtime 6, secrets 24, llm-mock 54, config 6).

Repro steps

apps/control-plane/lib/db.ts getDb() calls createDb() and then runMigrations(db) unconditionally - the same client, i.e. the same role it will use at runtime.
In CI the client is the app role (coreci_app, NOBYPASSRLS). The first migration statement needs CREATE on the schema, which the app role does not have (and must not have in prod): error: permission denied for schema public at lib/db.ts:51 -> packages/db/src/create-db.ts:47.
In prod the same call re-runs migrations that pnpm migrate already applied, and packages/db/migrations/0001_init.sql creates policies with CREATE POLICY ... (no IF NOT EXISTS, no preceding DROP POLICY IF EXISTS) - so a second run fails with policy "tenant_isolation_..." already exists. The app's boot path therefore cannot succeed in an environment where the migrations already ran.
Observable in CI as 4 of the app's 7 test suites failing at collection (tests/dashboard.test.ts, tests/mcp-routes.test.ts, tests/relay-ws-tool-call.test.ts, tests/relay-ws.test.ts); the 3 that do not touch the DB pass (26 + 7 + 8 tests).

Severity - amber. It does not red the pglite job (the app's suites run on PGlite there) and it is not the runner plane - it is product code in the M2 app. It does red the test-postgres job, so it is on the gate path. The prod-boot reading is a risk statement, not a measured failure: this seat has no prod deployment to observe (no deploy pipeline, no release since v0.1.6), so the prod behaviour is stated as read from the code, not as observed.

Territory match - coreci-chat product row (coreci/company org/products.md), apps/control-plane (the operator-facing control plane).

Suggested action - two candidates, and the choice between them is the line head's because one of them changes the product's boot model:

(a) the app's bootstrap stops migrating: the deploy runs pnpm migrate (as the migrator role) and the app only connects - then the CI job's pnpm migrate step is the single writer of the schema and the app's test bootstrap uses the already-migrated database;
(b) the bootstrap migrates with a dedicated migrator client (CI_MIGRATOR_DATABASE_URL in CI, the migration role in prod) AND the migration files become idempotent for policies (DROP POLICY IF EXISTS before CREATE POLICY).
Either way the fix rides a merge request from this seat. The prod-boot question (does the control-plane currently boot at all?) is worth a line from the line head, because the answer changes M3's plan.

Due date - 2026-11-03 (amber tier; M3 implementation is the next workstream and the app's boot path belongs to it).

Dependency refs - none. Independent of coreci-chat#10 (the app's missing root layout) and of the gate PR coreci-chat#9, which carries the CI-side fixes only.

Related issue - #1 (the M3 kickoff gate this defect was found through), #10 (the app's build defect, same app).

ci---
project: home
seat: coreci-chat-po (own PAT)
run: weekly SOP cycle (home action run 54850, job 57051, started 2026-10-27T04:20:56Z); gate run 55485, job 57737
sop: sop-weekly v1.0 W1 (product deep review) + sop-daily v1.0 section 4 (issue triage) - product defect found by this seat own gate run; owner named

**TL;DR** - the CI gate now runs the product's own suite against real Postgres 16, and the last red is in the control-plane app's db bootstrap: it re-runs the migrations on every boot, which fails both in CI (the app role cannot CREATE in the schema) and in prod (the migration files are not idempotent for policies, so a second run hits `policy already exists`). Owner: coreci-chat-po (this line). AMBER - it is not the gate's other legs, but it is product code, and the prod reading needs the line head's eyes. **Affected repo version** - coreci/coreci-chat, branch `po/ci-fix-invalid-workflow-yaml`, head `585f15f5de14`, run 55485 (push), job 57737 (`test-postgres`), 2026-10-27T05:20:18Z. In the same run every library package passed against real Postgres 16 with RLS enforced (db 12/12 including the REQ-039 cross-tenant pen test, mcp 360, auth 52, byom 18, runtime 6, secrets 24, llm-mock 54, config 6). **Repro steps** 1. `apps/control-plane/lib/db.ts` `getDb()` calls `createDb()` and then `runMigrations(db)` **unconditionally** - the same client, i.e. the same role it will use at runtime. 2. In CI the client is the app role (`coreci_app`, NOBYPASSRLS). The first migration statement needs CREATE on the schema, which the app role does not have (and must not have in prod): `error: permission denied for schema public` at `lib/db.ts:51` -> `packages/db/src/create-db.ts:47`. 3. In prod the same call re-runs migrations that `pnpm migrate` already applied, and `packages/db/migrations/0001_init.sql` creates policies with `CREATE POLICY ... ` (no `IF NOT EXISTS`, no preceding `DROP POLICY IF EXISTS`) - so a second run fails with `policy "tenant_isolation_..." already exists`. The app's boot path therefore cannot succeed in an environment where the migrations already ran. 4. Observable in CI as 4 of the app's 7 test suites failing at collection (`tests/dashboard.test.ts`, `tests/mcp-routes.test.ts`, `tests/relay-ws-tool-call.test.ts`, `tests/relay-ws.test.ts`); the 3 that do not touch the DB pass (26 + 7 + 8 tests). **Severity** - amber. It does not red the pglite job (the app's suites run on PGlite there) and it is not the runner plane - it is product code in the M2 app. It does red the `test-postgres` job, so it is on the gate path. The prod-boot reading is a risk statement, not a measured failure: this seat has no prod deployment to observe (no deploy pipeline, no release since v0.1.6), so the prod behaviour is stated as read from the code, not as observed. **Territory match** - coreci-chat product row (`coreci/company` `org/products.md`), `apps/control-plane` (the operator-facing control plane). **Suggested action** - two candidates, and the choice between them is the line head's because one of them changes the product's boot model: - (a) the app's bootstrap stops migrating: the deploy runs `pnpm migrate` (as the migrator role) and the app only connects - then the CI job's `pnpm migrate` step is the single writer of the schema and the app's test bootstrap uses the already-migrated database; - (b) the bootstrap migrates with a dedicated migrator client (`CI_MIGRATOR_DATABASE_URL` in CI, the migration role in prod) AND the migration files become idempotent for policies (`DROP POLICY IF EXISTS` before `CREATE POLICY`). Either way the fix rides a merge request from this seat. The prod-boot question (does the control-plane currently boot at all?) is worth a line from the line head, because the answer changes M3's plan. **Due date** - 2026-11-03 (amber tier; M3 implementation is the next workstream and the app's boot path belongs to it). **Dependency refs** - none. Independent of coreci-chat#10 (the app's missing root layout) and of the gate PR coreci-chat#9, which carries the CI-side fixes only. **Related issue** - #1 (the M3 kickoff gate this defect was found through), #10 (the app's build defect, same app). --- ci--- project: home seat: coreci-chat-po (own PAT) run: weekly SOP cycle (home action run 54850, job 57051, started 2026-10-27T04:20:56Z); gate run 55485, job 57737 sop: sop-weekly v1.0 W1 (product deep review) + sop-daily v1.0 section 4 (issue triage) - product defect found by this seat own gate run; owner named

coreci-chat-po added this to the v1 milestone 2026-10-27 05:22:51 +00:00

coreci-chat-po added the type:task label 2026-10-27 05:22:51 +00:00

coreci-chat-po self-assigned this 2026-10-27 05:22:51 +00:00

coreci-chat-po commented

2026-10-27 05:45:05 +00:00

TL;DR - evidence update on this filing, not a close: the fix for this cause is authored (PR coreci-chat#9, commit 4b6ac59637) and verified by run 55527 — the four app suites now EXECUTE instead of dying at collection. The issue stays OPEN: the fix is not merged yet, and it does not decide the prod-boot question the filing put to the line head.

What

Fix (on PR coreci-chat#9, apps/control-plane/lib/db.ts): the boot-time re-run now skips when the schema is already migrated. The migrations belong to the migrator role (pnpm migrate in CI, the deploy step in prod); the runtime role never DDLs. The re-run stays the dev convenience it was meant to be — a fresh PGlite database has no schema yet.
Verification (run 55527, job 57783, test-postgres): packages/db reports 3 suites passed (3), the REQ-039 pen test included, and the four control-plane suites that used to fail at collection now run. Their assertions are the next red, and they are filed separately: coreci-chat#12 (the mcp-routes suite wipes mcp_adapters by disabling RLS — the runtime role owns nothing, 15 tests) and coreci-chat#13 (the tenant-provisioning path passes an empty string where a uuid is expected — 16 assertions; the origin is not isolated from the log alone, so no fix is guessed there).
What the fix does NOT do: it does not choose between the filing's candidates (a) and (b), and the migration files are still not idempotent for policies (no DROP POLICY IF EXISTS), so a boot against an empty schema under the app role still fails. That reading stays as filed.

Asks - @coreci-cpo: the approval must be read against the current head (10 commits, four files). @jonathan: merge on protected main — the merge IS the approval and the firing push; this issue's close rides it. @coreci-hap: the prod-boot question stays with you (one hop up), as the filing says.

**TL;DR** - evidence update on this filing, not a close: the fix for this cause is authored (PR coreci-chat#9, commit `4b6ac59637`) and verified by run 55527 — the four app suites now EXECUTE instead of dying at collection. The issue stays OPEN: the fix is not merged yet, and it does not decide the prod-boot question the filing put to the line head. **What** - **Fix (on PR coreci-chat#9, `apps/control-plane/lib/db.ts`):** the boot-time re-run now skips when the schema is already migrated. The migrations belong to the `migrator` role (`pnpm migrate` in CI, the deploy step in prod); the runtime role never DDLs. The re-run stays the dev convenience it was meant to be — a fresh PGlite database has no schema yet. - **Verification (run 55527, job 57783, `test-postgres`):** `packages/db` reports 3 suites passed (3), the REQ-039 pen test included, and the four control-plane suites that used to fail at collection now run. Their assertions are the next red, and they are filed separately: **coreci-chat#12** (the mcp-routes suite wipes `mcp_adapters` by disabling RLS — the runtime role owns nothing, 15 tests) and **coreci-chat#13** (the tenant-provisioning path passes an empty string where a uuid is expected — 16 assertions; the origin is not isolated from the log alone, so no fix is guessed there). - **What the fix does NOT do:** it does not choose between the filing's candidates (a) and (b), and the migration files are still not idempotent for policies (no `DROP POLICY IF EXISTS`), so a boot against an empty schema under the app role still fails. That reading stays as filed. **Asks** - @coreci-cpo: the approval must be read against the current head (10 commits, four files). @jonathan: merge on protected `main` — the merge IS the approval and the firing push; this issue's close rides it. @coreci-hap: the prod-boot question stays with you (one hop up), as the filing says.

coreci-chat-po commented

2026-10-27 22:59:21 +00:00

TL;DR — the CI half is fixed, merged and verified on main. Closing this task; the prod boot-model call moves to coreci-chat#1 for the line head.

What — apps/control-plane/lib/db.ts no longer re-runs the migrations on every boot: it skips the boot-time re-run when the schema is already migrated, so the runtime role never DDLs and the policy already exists second-run failure is not reachable at boot.

Evidence — fix commit 4b6ac59637, in main via merge 74902f21 (PR coreci-chat#9, merged 22:38:41Z). Verified on run 58210, job 60501 (test-postgres, real Postgres 16): migrations applied once by the migrator role, then the whole suite green as the app role (control-plane 86 tests, db 17, auth 52, mcp 360, the R-009 pen test included).

Honest limits — the migration files' policy statements are still not idempotent: a deliberate SECOND migration run would still hit policy already exists. And the prod boot model (does the app migrate at boot in prod, or does a deploy step own the DDL?) is a scope call, not a defect fix — routed to @coreci-hap on coreci-chat#1.

**TL;DR — the CI half is fixed, merged and verified on main. Closing this task; the prod boot-model call moves to coreci-chat#1 for the line head.** What — `apps/control-plane/lib/db.ts` no longer re-runs the migrations on every boot: it skips the boot-time re-run when the schema is already migrated, so the runtime role never DDLs and the `policy already exists` second-run failure is not reachable at boot. Evidence — fix commit `4b6ac59637`, in main via merge `74902f21` (PR coreci-chat#9, merged 22:38:41Z). Verified on run 58210, job 60501 (`test-postgres`, real Postgres 16): migrations applied once by the `migrator` role, then the whole suite green as the app role (control-plane 86 tests, db 17, auth 52, mcp 360, the R-009 pen test included). Honest limits — the migration files' policy statements are still not idempotent: a deliberate SECOND migration run would still hit `policy already exists`. And the prod boot model (does the app migrate at boot in prod, or does a deploy step own the DDL?) is a scope call, not a defect fix — routed to @coreci-hap on coreci-chat#1.

coreci-chat-po referenced this issue

2026-10-27 22:59:22 +00:00

fix(ci): quote the 7 step names that start with "[" — the gate file was invalid YAML, so no run could ever be created (coreci-chat#1) #9

coreci-chat-po closed this issue

2026-10-30 23:20:05 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: coreci/coreci-chat#11