diff --git a/.ciagent/CHECKPOINT.json b/.ciagent/CHECKPOINT.json index 768ac5f..c2745bf 100644 --- a/.ciagent/CHECKPOINT.json +++ b/.ciagent/CHECKPOINT.json @@ -1,10 +1,10 @@ { "phase": 0, - "stage": "clarify", + "stage": "research", "milestone": "v0.5", "milestone_slug": "distribution", "phase_role": "pre_execution", "attempts": 0, - "updated_at": "2026-08-03T18:05:00Z", + "updated_at": "2026-08-03T18:15:00Z", "milestone_complete": false } \ No newline at end of file diff --git a/.ciagent/PERSONAS.md b/.ciagent/PERSONAS.md index 6ab33f2..1ee2927 100644 --- a/.ciagent/PERSONAS.md +++ b/.ciagent/PERSONAS.md @@ -2,52 +2,42 @@ active_personas: - lead-developer - backend-engineer - - data-engineer - cli-engineer - - security-engineer - - network-engineer + - devops-engineer deactivated_personas: - frontend-engineer - - devops-sre -phase_specific: - - cli-engineer - data-engineer - security-engineer - network-engineer +phase_specific: + - cli-engineer + - devops-engineer reason: | Orca is a CLI-first, offline-first orchestration engine with no web UI and - a single-binary distribution model. The v0.3 milestone is a 2-phase - completion milestone (iter.Seq streaming + doctor network/db) that touches - the CLI, store, doctor, transport, and security layers. The persona roster - reflects this: + a single-binary distribution model. The v0.5 milestone is a distribution + milestone (install, namespace, docker, public releases) that touches the + CLI namespace layer, shell scripts, and container/build infrastructure. + The persona roster reflects this: - lead-developer: coordination, task decomposition, territory adjudication - (e.g. D-039 dbPath relocation between cli-engineer territory and the - doctor package). - - backend-engineer: daemon health endpoint surface that the doctor network - check probes; transport dispatch client reuse. - - data-engineer: iter.Seq[Job|Node] on the store repos (P01) and the - migration-version query + PRAGMA integrity_check in the store layer (P02). - - cli-engineer: the --watch flag on `orca job list` / `orca node list` - (P01) and the doctor subcommand wiring (P02). - - security-engineer: mTLS client config reuse for the doctor network probe - (P02) — TLS config is the security-engineer territory per v0.2. - - network-engineer: the doctor /healthz probe over mTLS reuses the - transport layer (P02) — connection lifecycle / peer reachability is the - network-engineer territory. + between cli-engineer (namespace flag) and devops-engineer (install.sh, + Dockerfile, release pipeline). + - backend-engineer: no new backend surface in v0.5, but owns the + `internal/store` and `internal/certpaths` refactors for namespace + unification (REQ-041) — these are shared-infra concerns that the + backend-engineer adjudicates. + - cli-engineer: the `--system` flag on `rootCmd` and the `init --system` + subcommand (REQ-042) — pure CLI surface. + - devops-engineer (NEW, reactivated): install.sh, Dockerfile, + .coreci.yml container pipeline, scripts/release.sh docker publish step. - Deactivated: - - frontend-engineer: no web UI in Orca (v0.1 onward). NOT relevant to v0.3. - - devops-sre: no container/cloud integrations; release flow is handled by - CoreCI (not a persona territory). - - Phase-specific (v0.3): - - cli-engineer: P01 (--watch flag is a CLI surface) + P02 (doctor - subcommand wiring). - - data-engineer: P01 (iter.Seq on store repos) + P02 (migration version + - integrity check in store layer). - - security-engineer: P02 only (mTLS client config for doctor network probe). - - network-engineer: P02 only (mTLS /healthz probe over transport). + Deactivated for v0.5 (no v0.5 surface): + - frontend-engineer: no web UI (unchanged from v0.1). + - data-engineer: v0.5 has no store/schema work — the store refactor + (R-004) is a 1-line routing change, not schema work. + - security-engineer: v0.5 has no new cert/mTLS surface — the namespace + unification moves cert paths but does not change cert logic. + - network-engineer: v0.5 has no transport/network surface. --- # Personas: Orca @@ -103,59 +93,54 @@ reason: | - **Reason**: Owns the transport layer and peer-to-peer connection lifecycle. For v0.3 P02, the doctor network check is a read-only mTLS `/healthz` probe that reuses `transport.MTLSClient` — the connection lifecycle (dial, per-probe 3s timeout, handshake) is network-engineer territory. Added `bounded-probe-timeout` to constraints (doctor must not stall on one slow peer — RESEARCH_v0.3.md D-038). Territory confirmed: `internal/transport/` holds mtls.go, dispatch.go, retry.go, idempotency.go, handshake_log.go. - **Phase scope**: P02 only (doctor network probe reuses transport layer). +### devops-engineer (reactivated in v0.5) +- **Domain**: devops / packaging +- **Frameworks**: `docker`, `bash`, `curl`, `tea` +- **Constraints**: `idempotent-scripts`, `minimal-image-size`, `no-secret-in-image`, `reproducible-build` +- **Territory**: `Dockerfile`, `scripts/install.sh`, `scripts/release.sh`, `.coreci.yml`, `docs/docker.md` +- **Active**: true +- **Reason**: v0.5 is a distribution milestone. The devops-engineer owns install.sh (REQ-043/044), the Dockerfile + container registry publish (REQ-046), and the `.coreci.yml` release pipeline extension. Reactivated from v0.1 (where it was deactivated as "devops-sre" because CoreCI handled release). In v0.5, container + install surface is first-class devops work. + ### frontend-engineer - **Active**: false -- **Reason**: No web UI in Orca (v0.1 onward). NOT relevant to v0.3 — v0.3 adds no UI surface. Confirmed deactivated. +- **Reason**: No web UI in Orca (v0.1 onward). NOT relevant to v0.5. Confirmed deactivated. -### devops-sre -- **Active**: false -- **Reason**: No container/cloud integrations. Release flow is handled by CoreCI (not a persona territory). Confirmed deactivated. +### data-engineer +- **Active**: false (v0.5) +- **Reason**: v0.5 has no store/schema work. The `internal/store/store.go` refactor (R-004) is a 1-line routing change from hardcoded `~/.orca` to `certpaths.DBPath()` — this is shared-infra, adjudicated by backend-engineer, not data-engineer schema work. + +### security-engineer +- **Active**: false (v0.5) +- **Reason**: v0.5 has no new cert/mTLS surface. The namespace unification moves cert paths via `certpaths.Dir()` (already the source of truth) but does not change cert generation, validation, or TLS config logic. + +### network-engineer +- **Active**: false (v0.5) +- **Reason**: v0.5 has no transport/network surface. No new peer-to-peer, dispatch, or health-probe work. ## Territory Enforcement - **Mode**: `warn` (per `config.json`) - **Behavior**: Out-of-territory file changes log a warning but do not block. -- **Rationale**: Allows flexibility during early development; tighten to `strict` post-v0.1. For v0.3, the main territory-overlap risk is D-039 (moving `dbPath` from `internal/cli` to `internal/certpaths`) which crosses cli-engineer and the shared-infra concern — lead-developer adjudicates. +- **Rationale**: Allows flexibility during early development; tighten to `strict` post-v0.1. For v0.5, the main territory-overlap risk is `internal/store/store.go` (R-004) which is backend-engineer shared-infra territory but touches the data layer — lead-developer adjudicates. -## Phase-Specific Personas (v0.3) +## Phase-Specific Personas (v0.5) | Persona | Active in | Reason | |---------|-----------|--------| -| `cli-engineer` | P01, P02 | P01: `--watch` flag is a CLI surface (signal handling, table/JSON render). P02: doctor subcommand wiring in `internal/cli/doctor.go`. | -| `data-engineer` | P01, P02 | P01: `iter.Seq[Job|Node]` on the store repos + the no-leak polling loop. P02: `MigrationVersion` query + `PRAGMA integrity_check` in the store layer. | -| `security-engineer` | P02 | mTLS client config reuse for the doctor network probe. P01 has no security surface. | -| `network-engineer` | P02 | mTLS `/healthz` probe over the transport layer (connection lifecycle, per-probe timeout). P01 has no network surface. | +| `cli-engineer` | P1 | `--system` flag on `rootCmd` + `init --system` subcommand (REQ-042). Pure CLI surface. | +| `devops-engineer` | P2, P3 | P2: install.sh + in-place update (REQ-043/044). P3: Dockerfile + container registry publish (REQ-046). | +| `backend-engineer` | P1 | `internal/store/store.go` + `internal/certpaths/certpaths.go` namespace routing refactor (REQ-041). Shared-infra. | In full-autonomy mode, all personas are auto-accepted and the phase-scope assignments are applied automatically when a phase is committed. -## v0.3 vs v0.2 Persona Diff +## v0.5 vs v0.3 Persona Diff | Change | Rationale | |--------|-----------| -| `data-engineer` frameworks: added `iter` | P01 introduces `iter.Seq[T]` on the store repos — a new stdlib framework surface for this persona. | -| `data-engineer` constraints: added `no-goroutine-leak` | The iter.Seq polling loop must not leak goroutines (inline pull loop, defer ticker.Stop, rows.Close on every path — RESEARCH D-032). | -| `cli-engineer` constraints: added `signal-handling` | P01 requires `signal.NotifyContext` for ctrl-c propagation to iter.Seq (D-031). | -| `network-engineer` constraints: added `bounded-probe-timeout` | P02 doctor network check must bound each peer probe (3s) so one slow peer doesn't stall diagnostics (D-038). | -| `network-engineer` phase scope: was P02-only (v0.2), now P02-only (v0.3) | Same persona, different phase content — v0.3 P02 is doctor network, not multi-node dispatch. | -| `security-engineer` phase scope: was P01+P02 (v0.2), now P02-only (v0.3) | v0.3 has no new cert/CA work; security surface is limited to reusing the existing mTLS client config in doctor. | -| `frontend-engineer` | Remains deactivated (no UI in v0.3). | -| `devops-sre` | Remains deactivated (CoreCI handles release). | - -## Migration from v0.2 - -- `backend-engineer` territory unchanged: `internal/daemon/**` still owns HTTP - handlers. The `/healthz` endpoint that the doctor network check probes is - backend-engineer territory; the *probing* client is network-engineer. -- `data-engineer` territory expanded scope: still owns `internal/store/**` but - now adds the `iter.Seq` polling implementations (P01) and a public - `MigrationVersion` query (P02). -- `security-engineer` territory unchanged: `internal/security/**` + the TLS - config portion of `internal/transport/**`. The doctor network check calls - into `security.ClientTLSConfig` indirectly via `transport.NewMTLSClient` — - no new security-engineer files, just reuse. -- `cli-engineer` territory unchanged: `internal/cli/**`. P01 modifies - `job.go` and `node.go`; P02 modifies `doctor.go`. The `dbPath` relocation - (D-039) moves a 5-line function out of `internal/cli/node.go` into - `internal/certpaths` — cli-engineer territory loses one function, shared - infra gains it. \ No newline at end of file +| `devops-engineer` reactivated (was `devops-sre`, deactivated in v0.1) | v0.5 is a distribution milestone — install.sh, Dockerfile, container registry publish are first-class devops work. Renamed from `devops-sre` to `devops-engineer` to reflect build/packaging focus (not SRE/ops). | +| `data-engineer` deactivated | v0.5 has no schema/store logic work — the store.go change is a 1-line routing refactor (shared-infra, backend-engineer). | +| `security-engineer` deactivated | v0.5 has no new cert/mTLS surface. | +| `network-engineer` deactivated | v0.5 has no transport/network surface. | +| `cli-engineer` phase scope: was P01+P02 (v0.3), now P1 only (v0.5) | v0.5 P1 is the `--system` flag (CLI surface). P2/P3 are devops territory. | +| `frontend-engineer` | Remains deactivated (no UI in v0.5). | \ No newline at end of file diff --git a/.ciagent/RESEARCH_v0.5.md b/.ciagent/RESEARCH_v0.5.md new file mode 100644 index 0000000..76a349d --- /dev/null +++ b/.ciagent/RESEARCH_v0.5.md @@ -0,0 +1,161 @@ +# Research: Orca v0.5 — Distribution + +Research findings for the v0.5 Distribution milestone (install, namespace, +docker, public releases). Conducted during P0 RESEARCH under full autonomy. + +## R-001: Gitea Container Registry + +**Source**: https://docs.gitea.com/usage/packages/container (Gitea 1.27.1 docs) + +**Findings**: +- Gitea ships a built-in OCI-compliant container registry. +- Image naming convention: `{registry}/{owner}/{image}:{tag}`. + For orca: `git.cloudinit.dev/coreci/orca:{tag}`. +- Auth: `docker login git.cloudinit.dev` with username + personal access + token (or password if no 2FA). The `GITEA_TOKEN` env var already used + for release publishing works as the password. +- Push: `docker push git.cloudinit.dev/coreci/orca:v0.4.4`. +- Pull: anonymous pull works **if the repo is public** (REQ-045 flips + this). For private repos, pull requires auth. +- Tags are case-insensitive — use lowercase image names. +- The registry supports multi-arch manifests via `docker buildx`. + +**Implication for P03**: `scripts/release.sh` must add a `docker build` ++ `docker login` + `docker push` step. The `.coreci.yml` release +pipeline needs a `container-publish` step. Credential is `GITEA_TOKEN` +(reused from the existing release flow — no new secret needed). + +## R-002: `tea repos edit` — Repo Visibility + +**Source**: `tea repos edit --help` (tea 0.14.1 installed locally) + +**Findings**: +- Command: `tea repos edit --private false --repo coreci/orca` +- The `--private` flag accepts `true`/`false` (string, not bool). +- Default login `bot` (cloudinit-bot) is already configured and is the + default login. No extra auth needed. +- The change is immediate and reversible (re-run with `--private true`). + +**Implication for P0 ship**: Run this as an operational step during the +P0 ship. Verify with unauth `curl` against the releases API afterward. + +## R-003: Gitea Releases API — Asset Download URLs + +**Source**: `/api/v1/repos/coreci/orca/releases/latest` (authed probe) + +**Findings**: +- Auth header format: `Authorization: token ` (NOT basic + auth — basic auth returns "invalid username, password or token"). +- Latest release endpoint: `GET /api/v1/repos/coreci/orca/releases/latest` + → JSON with `tag_name`, `name`, `body`, `assets[]`. +- Each asset has `browser_download_url` — the direct download URL. +- **Public access**: once the repo is public (R-002), the releases API + and asset downloads work **without authentication**. This is what + `install.sh` relies on (REQ-043). +- Asset naming convention from existing releases: + `orca-{version}-linux-amd64.tar.gz` (per `scripts/release.sh`). + +**Implication for P02 install.sh**: +1. Query `GET /api/v1/repos/coreci/orca/releases/latest` (unauth, post-R-002). +2. Parse `tag_name` for the version. +3. Find the asset with `name` matching `orca-{tag}-linux-{arch}.tar.gz`. +4. Download `browser_download_url` with `curl -fsSL`. +5. Extract and install. + +## R-004: ORCA_HOME Propagation Points (Codebase Audit) + +**Source**: `grep` for `UserHomeDir|os.Getenv("ORCA|\.orca` across `*.go` + +**Findings** — exactly 3 production code sites determine the namespace +root today: + +| File | Current behavior | Needs change? | +|------|-----------------|----------------| +| `internal/certpaths/certpaths.go:21-26` | `Dir()` honors `ORCA_HOME` → `~/.orca` | **No** — this is the single source of truth. Already correct. | +| `internal/store/store.go:13-19` | `Open("")` hardcodes `~/.orca/orca.db` (ignores `ORCA_HOME`) | **Yes** — route through `certpaths.DBPath()` instead. | +| `internal/cli/init.go:16-22` | Hardcodes `~/.orca` via `os.UserHomeDir()` | **Yes** — route through `certpaths.Dir()`. | + +All other call sites (`node.go:openDB`, `daemon.go`, `job.go`, `doctor.go`, +`cert.go`) already go through `certpaths.DBPath()` or `certpaths.Dir()` +indirectly. **No other files need changes for REQ-041.** + +**For REQ-042 (`--system`)**: Add a `--system` persistent flag on +`rootCmd`. When set, `rootCmd.PersistentPreRunE` sets +`os.Setenv("ORCA_HOME", "/root/.orca")` before any subcommand runs. +This is the minimal-touch approach — all downstream code already +honors `ORCA_HOME`. The flag is a CLI convenience that maps to the +env var, not a parallel mechanism. + +**Backward compatibility**: empty `ORCA_HOME` + no `--system` → +`~/.orca` (unchanged). Existing tests that `t.Setenv("ORCA_HOME", ...)` +continue to work. + +## R-005: Distroless Base Image for CGO-free Go Binaries + +**Source**: Go module audit — `modernc.org/sqlite` (pure Go, CGO-free), +`go.mod` has no CGO dependencies. + +**Findings**: +- `gcr.io/distroless/static-debian12` is the correct base for static + Go binaries with no CGO and no libc dependency. ~2MB image. +- orca uses `modernc.org/sqlite` (pure Go) — no CGO, no libc. ✓ +- Multi-stage Dockerfile: + - Stage 1 (`golang:1.25`): build with `-trimpath -ldflags` (same as + Makefile), output `bin/orca`. + - Stage 2 (`gcr.io/distroless/static-debian12`): `COPY bin/orca /orca`, + `ENTRYPOINT ["/orca"]`. +- `CGO_ENABLED=0` must be set in the build stage to guarantee a static + binary (Go defaults to CGO_ENABLED=1 on platforms with a C compiler). +- The image runs as `nonroot` user by default in distroless — but orca + writes to `~/.orca` (or `/root/.orca` for `--system`). For the + container image, default `ORCA_HOME=/var/lib/orca` and document + volume mount at that path. + +**Implication for P03**: Dockerfile is ~15 lines. The `.coreci.yml` +release pipeline adds a `docker build --build-arg VERSION=$VERSION -t +git.cloudinit.dev/coreci/orca:$VERSION .` step + login + push. + +## R-006: install.sh Conventions (curl|sh pattern) + +**Source**: Common patterns from deno, rustup, homebrew installers. + +**Findings**: +- 1-liner: `curl -fsSL | bash` (or `| bash -s -- --system`). +- The script must be downloadable from a stable URL. orca's script + lives at `scripts/install.sh` in the repo, accessible via + `https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install.sh` + (once repo is public per R-002). +- Args passed via `bash -s -- --system --version v0.4.4`. +- In-place update: detect existing binary at install path, read its + version via `orca version --json` (parse `version` field), print + "updated from X to Y", overwrite binary. **Never** touch the + namespace dir (`~/.orca` or `/root/.orca`) — that's user state. +- User-level default: `~/.local/bin/orca` (XDG-ish, on PATH on most + modern distros). System-level: `/usr/local/bin/orca` (requires root). + +**Implication for P02**: install.sh is ~80-100 lines of bash. Idempotent. +Tested via a `scripts/install_test.sh` that mocks the download and +verifies path selection + update-in-place. + +## Pitfalls (P-001..P-003) + +- **P-001**: `docker` may not be available in the CoreCI release + pipeline container. The `.coreci.yml` release step uses + `image: golang:1.25` which does NOT include docker. **Mitigation**: + the release pipeline must use a `docker:dind` sidecar or a step image + that has the docker CLI. Alternatively, `scripts/release.sh` handles + docker publish only when run locally or in a CI step that has docker. + The `.coreci.yml` container step must use an image with docker CLI + (e.g., `catthehacker/docker:docker-latest` or a custom image). + +- **P-002**: Making the repo public exposes git history including the + pre-existing `.env` SHA-1 leak (commit `00127ce` documented the + rotate-forward decision; `.gitleaks-baseline.json` suppresses it for + scanning). The leak is a **non-secret** (the token was rotated). This + is an accepted risk per the existing decision — no new action needed, + but document it in the P0 ship commit. + +- **P-003**: `CGO_ENABLED=0` must be explicit in the Dockerfile build + stage. Without it, `go build` in `golang:1.25` may produce a + dynamically-linked binary that won't run in distroless. Verified: + orca has no CGO deps, but `CGO_ENABLED=0` is belt-and-suspenders. \ No newline at end of file