docs(P00): research findings
R-001: Gitea container registry (OCI, docker login/push, anon pull when public) R-002: tea repos edit --private false (visibility flip for REQ-045) R-003: Gitea releases API (Authorization: token header, asset download URLs) R-004: ORCA_HOME propagation audit (3 sites: certpaths/store/init) R-005: distroless static-debian12 base (CGO-free, modernc/sqlite) R-006: install.sh curl|sh conventions + in-place update pattern Pitfalls P-001..P-003 (docker-in-CI, public-history leak, CGO_ENABLED=0) PERSONAS.md: devops-engineer reactivated, data/security/network deactivated for v0.5 ---ci--- project: orca phase: 0 milestone: v0.5 status: research ---/ci---
This commit is contained in:
Jon Chery
@@ -1,10 +1,10 @@
|
||||
{
|
||||
"phase": 0,
|
||||
"stage": "clarify",
|
||||
"stage": "research",
|
||||
"milestone": "v0.5",
|
||||
"milestone_slug": "distribution",
|
||||
"phase_role": "pre_execution",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-08-03T18:05:00Z",
|
||||
"updated_at": "2026-08-03T18:15:00Z",
|
||||
"milestone_complete": false
|
||||
}
|
||||
+57
-72
@@ -2,52 +2,42 @@
|
||||
active_personas:
|
||||
- lead-developer
|
||||
- backend-engineer
|
||||
- data-engineer
|
||||
- cli-engineer
|
||||
- security-engineer
|
||||
- network-engineer
|
||||
- devops-engineer
|
||||
deactivated_personas:
|
||||
- frontend-engineer
|
||||
- devops-sre
|
||||
phase_specific:
|
||||
- cli-engineer
|
||||
- data-engineer
|
||||
- security-engineer
|
||||
- network-engineer
|
||||
phase_specific:
|
||||
- cli-engineer
|
||||
- devops-engineer
|
||||
reason: |
|
||||
Orca is a CLI-first, offline-first orchestration engine with no web UI and
|
||||
a single-binary distribution model. The v0.3 milestone is a 2-phase
|
||||
completion milestone (iter.Seq streaming + doctor network/db) that touches
|
||||
the CLI, store, doctor, transport, and security layers. The persona roster
|
||||
reflects this:
|
||||
a single-binary distribution model. The v0.5 milestone is a distribution
|
||||
milestone (install, namespace, docker, public releases) that touches the
|
||||
CLI namespace layer, shell scripts, and container/build infrastructure.
|
||||
The persona roster reflects this:
|
||||
|
||||
- lead-developer: coordination, task decomposition, territory adjudication
|
||||
(e.g. D-039 dbPath relocation between cli-engineer territory and the
|
||||
doctor package).
|
||||
- backend-engineer: daemon health endpoint surface that the doctor network
|
||||
check probes; transport dispatch client reuse.
|
||||
- data-engineer: iter.Seq[Job|Node] on the store repos (P01) and the
|
||||
migration-version query + PRAGMA integrity_check in the store layer (P02).
|
||||
- cli-engineer: the --watch flag on `orca job list` / `orca node list`
|
||||
(P01) and the doctor subcommand wiring (P02).
|
||||
- security-engineer: mTLS client config reuse for the doctor network probe
|
||||
(P02) — TLS config is the security-engineer territory per v0.2.
|
||||
- network-engineer: the doctor /healthz probe over mTLS reuses the
|
||||
transport layer (P02) — connection lifecycle / peer reachability is the
|
||||
network-engineer territory.
|
||||
between cli-engineer (namespace flag) and devops-engineer (install.sh,
|
||||
Dockerfile, release pipeline).
|
||||
- backend-engineer: no new backend surface in v0.5, but owns the
|
||||
`internal/store` and `internal/certpaths` refactors for namespace
|
||||
unification (REQ-041) — these are shared-infra concerns that the
|
||||
backend-engineer adjudicates.
|
||||
- cli-engineer: the `--system` flag on `rootCmd` and the `init --system`
|
||||
subcommand (REQ-042) — pure CLI surface.
|
||||
- devops-engineer (NEW, reactivated): install.sh, Dockerfile,
|
||||
.coreci.yml container pipeline, scripts/release.sh docker publish step.
|
||||
|
||||
Deactivated:
|
||||
- frontend-engineer: no web UI in Orca (v0.1 onward). NOT relevant to v0.3.
|
||||
- devops-sre: no container/cloud integrations; release flow is handled by
|
||||
CoreCI (not a persona territory).
|
||||
|
||||
Phase-specific (v0.3):
|
||||
- cli-engineer: P01 (--watch flag is a CLI surface) + P02 (doctor
|
||||
subcommand wiring).
|
||||
- data-engineer: P01 (iter.Seq on store repos) + P02 (migration version +
|
||||
integrity check in store layer).
|
||||
- security-engineer: P02 only (mTLS client config for doctor network probe).
|
||||
- network-engineer: P02 only (mTLS /healthz probe over transport).
|
||||
Deactivated for v0.5 (no v0.5 surface):
|
||||
- frontend-engineer: no web UI (unchanged from v0.1).
|
||||
- data-engineer: v0.5 has no store/schema work — the store refactor
|
||||
(R-004) is a 1-line routing change, not schema work.
|
||||
- security-engineer: v0.5 has no new cert/mTLS surface — the namespace
|
||||
unification moves cert paths but does not change cert logic.
|
||||
- network-engineer: v0.5 has no transport/network surface.
|
||||
---
|
||||
|
||||
# Personas: Orca
|
||||
@@ -103,59 +93,54 @@ reason: |
|
||||
- **Reason**: Owns the transport layer and peer-to-peer connection lifecycle. For v0.3 P02, the doctor network check is a read-only mTLS `/healthz` probe that reuses `transport.MTLSClient` — the connection lifecycle (dial, per-probe 3s timeout, handshake) is network-engineer territory. Added `bounded-probe-timeout` to constraints (doctor must not stall on one slow peer — RESEARCH_v0.3.md D-038). Territory confirmed: `internal/transport/` holds mtls.go, dispatch.go, retry.go, idempotency.go, handshake_log.go.
|
||||
- **Phase scope**: P02 only (doctor network probe reuses transport layer).
|
||||
|
||||
### devops-engineer (reactivated in v0.5)
|
||||
- **Domain**: devops / packaging
|
||||
- **Frameworks**: `docker`, `bash`, `curl`, `tea`
|
||||
- **Constraints**: `idempotent-scripts`, `minimal-image-size`, `no-secret-in-image`, `reproducible-build`
|
||||
- **Territory**: `Dockerfile`, `scripts/install.sh`, `scripts/release.sh`, `.coreci.yml`, `docs/docker.md`
|
||||
- **Active**: true
|
||||
- **Reason**: v0.5 is a distribution milestone. The devops-engineer owns install.sh (REQ-043/044), the Dockerfile + container registry publish (REQ-046), and the `.coreci.yml` release pipeline extension. Reactivated from v0.1 (where it was deactivated as "devops-sre" because CoreCI handled release). In v0.5, container + install surface is first-class devops work.
|
||||
|
||||
### frontend-engineer
|
||||
- **Active**: false
|
||||
- **Reason**: No web UI in Orca (v0.1 onward). NOT relevant to v0.3 — v0.3 adds no UI surface. Confirmed deactivated.
|
||||
- **Reason**: No web UI in Orca (v0.1 onward). NOT relevant to v0.5. Confirmed deactivated.
|
||||
|
||||
### devops-sre
|
||||
- **Active**: false
|
||||
- **Reason**: No container/cloud integrations. Release flow is handled by CoreCI (not a persona territory). Confirmed deactivated.
|
||||
### data-engineer
|
||||
- **Active**: false (v0.5)
|
||||
- **Reason**: v0.5 has no store/schema work. The `internal/store/store.go` refactor (R-004) is a 1-line routing change from hardcoded `~/.orca` to `certpaths.DBPath()` — this is shared-infra, adjudicated by backend-engineer, not data-engineer schema work.
|
||||
|
||||
### security-engineer
|
||||
- **Active**: false (v0.5)
|
||||
- **Reason**: v0.5 has no new cert/mTLS surface. The namespace unification moves cert paths via `certpaths.Dir()` (already the source of truth) but does not change cert generation, validation, or TLS config logic.
|
||||
|
||||
### network-engineer
|
||||
- **Active**: false (v0.5)
|
||||
- **Reason**: v0.5 has no transport/network surface. No new peer-to-peer, dispatch, or health-probe work.
|
||||
|
||||
## Territory Enforcement
|
||||
|
||||
- **Mode**: `warn` (per `config.json`)
|
||||
- **Behavior**: Out-of-territory file changes log a warning but do not block.
|
||||
- **Rationale**: Allows flexibility during early development; tighten to `strict` post-v0.1. For v0.3, the main territory-overlap risk is D-039 (moving `dbPath` from `internal/cli` to `internal/certpaths`) which crosses cli-engineer and the shared-infra concern — lead-developer adjudicates.
|
||||
- **Rationale**: Allows flexibility during early development; tighten to `strict` post-v0.1. For v0.5, the main territory-overlap risk is `internal/store/store.go` (R-004) which is backend-engineer shared-infra territory but touches the data layer — lead-developer adjudicates.
|
||||
|
||||
## Phase-Specific Personas (v0.3)
|
||||
## Phase-Specific Personas (v0.5)
|
||||
|
||||
| Persona | Active in | Reason |
|
||||
|---------|-----------|--------|
|
||||
| `cli-engineer` | P01, P02 | P01: `--watch` flag is a CLI surface (signal handling, table/JSON render). P02: doctor subcommand wiring in `internal/cli/doctor.go`. |
|
||||
| `data-engineer` | P01, P02 | P01: `iter.Seq[Job|Node]` on the store repos + the no-leak polling loop. P02: `MigrationVersion` query + `PRAGMA integrity_check` in the store layer. |
|
||||
| `security-engineer` | P02 | mTLS client config reuse for the doctor network probe. P01 has no security surface. |
|
||||
| `network-engineer` | P02 | mTLS `/healthz` probe over the transport layer (connection lifecycle, per-probe timeout). P01 has no network surface. |
|
||||
| `cli-engineer` | P1 | `--system` flag on `rootCmd` + `init --system` subcommand (REQ-042). Pure CLI surface. |
|
||||
| `devops-engineer` | P2, P3 | P2: install.sh + in-place update (REQ-043/044). P3: Dockerfile + container registry publish (REQ-046). |
|
||||
| `backend-engineer` | P1 | `internal/store/store.go` + `internal/certpaths/certpaths.go` namespace routing refactor (REQ-041). Shared-infra. |
|
||||
|
||||
In full-autonomy mode, all personas are auto-accepted and the phase-scope
|
||||
assignments are applied automatically when a phase is committed.
|
||||
|
||||
## v0.3 vs v0.2 Persona Diff
|
||||
## v0.5 vs v0.3 Persona Diff
|
||||
|
||||
| Change | Rationale |
|
||||
|--------|-----------|
|
||||
| `data-engineer` frameworks: added `iter` | P01 introduces `iter.Seq[T]` on the store repos — a new stdlib framework surface for this persona. |
|
||||
| `data-engineer` constraints: added `no-goroutine-leak` | The iter.Seq polling loop must not leak goroutines (inline pull loop, defer ticker.Stop, rows.Close on every path — RESEARCH D-032). |
|
||||
| `cli-engineer` constraints: added `signal-handling` | P01 requires `signal.NotifyContext` for ctrl-c propagation to iter.Seq (D-031). |
|
||||
| `network-engineer` constraints: added `bounded-probe-timeout` | P02 doctor network check must bound each peer probe (3s) so one slow peer doesn't stall diagnostics (D-038). |
|
||||
| `network-engineer` phase scope: was P02-only (v0.2), now P02-only (v0.3) | Same persona, different phase content — v0.3 P02 is doctor network, not multi-node dispatch. |
|
||||
| `security-engineer` phase scope: was P01+P02 (v0.2), now P02-only (v0.3) | v0.3 has no new cert/CA work; security surface is limited to reusing the existing mTLS client config in doctor. |
|
||||
| `frontend-engineer` | Remains deactivated (no UI in v0.3). |
|
||||
| `devops-sre` | Remains deactivated (CoreCI handles release). |
|
||||
|
||||
## Migration from v0.2
|
||||
|
||||
- `backend-engineer` territory unchanged: `internal/daemon/**` still owns HTTP
|
||||
handlers. The `/healthz` endpoint that the doctor network check probes is
|
||||
backend-engineer territory; the *probing* client is network-engineer.
|
||||
- `data-engineer` territory expanded scope: still owns `internal/store/**` but
|
||||
now adds the `iter.Seq` polling implementations (P01) and a public
|
||||
`MigrationVersion` query (P02).
|
||||
- `security-engineer` territory unchanged: `internal/security/**` + the TLS
|
||||
config portion of `internal/transport/**`. The doctor network check calls
|
||||
into `security.ClientTLSConfig` indirectly via `transport.NewMTLSClient` —
|
||||
no new security-engineer files, just reuse.
|
||||
- `cli-engineer` territory unchanged: `internal/cli/**`. P01 modifies
|
||||
`job.go` and `node.go`; P02 modifies `doctor.go`. The `dbPath` relocation
|
||||
(D-039) moves a 5-line function out of `internal/cli/node.go` into
|
||||
`internal/certpaths` — cli-engineer territory loses one function, shared
|
||||
infra gains it.
|
||||
| `devops-engineer` reactivated (was `devops-sre`, deactivated in v0.1) | v0.5 is a distribution milestone — install.sh, Dockerfile, container registry publish are first-class devops work. Renamed from `devops-sre` to `devops-engineer` to reflect build/packaging focus (not SRE/ops). |
|
||||
| `data-engineer` deactivated | v0.5 has no schema/store logic work — the store.go change is a 1-line routing refactor (shared-infra, backend-engineer). |
|
||||
| `security-engineer` deactivated | v0.5 has no new cert/mTLS surface. |
|
||||
| `network-engineer` deactivated | v0.5 has no transport/network surface. |
|
||||
| `cli-engineer` phase scope: was P01+P02 (v0.3), now P1 only (v0.5) | v0.5 P1 is the `--system` flag (CLI surface). P2/P3 are devops territory. |
|
||||
| `frontend-engineer` | Remains deactivated (no UI in v0.5). |
|
||||
@@ -0,0 +1,161 @@
|
||||
# Research: Orca v0.5 — Distribution
|
||||
|
||||
Research findings for the v0.5 Distribution milestone (install, namespace,
|
||||
docker, public releases). Conducted during P0 RESEARCH under full autonomy.
|
||||
|
||||
## R-001: Gitea Container Registry
|
||||
|
||||
**Source**: https://docs.gitea.com/usage/packages/container (Gitea 1.27.1 docs)
|
||||
|
||||
**Findings**:
|
||||
- Gitea ships a built-in OCI-compliant container registry.
|
||||
- Image naming convention: `{registry}/{owner}/{image}:{tag}`.
|
||||
For orca: `git.cloudinit.dev/coreci/orca:{tag}`.
|
||||
- Auth: `docker login git.cloudinit.dev` with username + personal access
|
||||
token (or password if no 2FA). The `GITEA_TOKEN` env var already used
|
||||
for release publishing works as the password.
|
||||
- Push: `docker push git.cloudinit.dev/coreci/orca:v0.4.4`.
|
||||
- Pull: anonymous pull works **if the repo is public** (REQ-045 flips
|
||||
this). For private repos, pull requires auth.
|
||||
- Tags are case-insensitive — use lowercase image names.
|
||||
- The registry supports multi-arch manifests via `docker buildx`.
|
||||
|
||||
**Implication for P03**: `scripts/release.sh` must add a `docker build`
|
||||
+ `docker login` + `docker push` step. The `.coreci.yml` release
|
||||
pipeline needs a `container-publish` step. Credential is `GITEA_TOKEN`
|
||||
(reused from the existing release flow — no new secret needed).
|
||||
|
||||
## R-002: `tea repos edit` — Repo Visibility
|
||||
|
||||
**Source**: `tea repos edit --help` (tea 0.14.1 installed locally)
|
||||
|
||||
**Findings**:
|
||||
- Command: `tea repos edit --private false --repo coreci/orca`
|
||||
- The `--private` flag accepts `true`/`false` (string, not bool).
|
||||
- Default login `bot` (cloudinit-bot) is already configured and is the
|
||||
default login. No extra auth needed.
|
||||
- The change is immediate and reversible (re-run with `--private true`).
|
||||
|
||||
**Implication for P0 ship**: Run this as an operational step during the
|
||||
P0 ship. Verify with unauth `curl` against the releases API afterward.
|
||||
|
||||
## R-003: Gitea Releases API — Asset Download URLs
|
||||
|
||||
**Source**: `/api/v1/repos/coreci/orca/releases/latest` (authed probe)
|
||||
|
||||
**Findings**:
|
||||
- Auth header format: `Authorization: token <GITEA_TOKEN>` (NOT basic
|
||||
auth — basic auth returns "invalid username, password or token").
|
||||
- Latest release endpoint: `GET /api/v1/repos/coreci/orca/releases/latest`
|
||||
→ JSON with `tag_name`, `name`, `body`, `assets[]`.
|
||||
- Each asset has `browser_download_url` — the direct download URL.
|
||||
- **Public access**: once the repo is public (R-002), the releases API
|
||||
and asset downloads work **without authentication**. This is what
|
||||
`install.sh` relies on (REQ-043).
|
||||
- Asset naming convention from existing releases:
|
||||
`orca-{version}-linux-amd64.tar.gz` (per `scripts/release.sh`).
|
||||
|
||||
**Implication for P02 install.sh**:
|
||||
1. Query `GET /api/v1/repos/coreci/orca/releases/latest` (unauth, post-R-002).
|
||||
2. Parse `tag_name` for the version.
|
||||
3. Find the asset with `name` matching `orca-{tag}-linux-{arch}.tar.gz`.
|
||||
4. Download `browser_download_url` with `curl -fsSL`.
|
||||
5. Extract and install.
|
||||
|
||||
## R-004: ORCA_HOME Propagation Points (Codebase Audit)
|
||||
|
||||
**Source**: `grep` for `UserHomeDir|os.Getenv("ORCA|\.orca` across `*.go`
|
||||
|
||||
**Findings** — exactly 3 production code sites determine the namespace
|
||||
root today:
|
||||
|
||||
| File | Current behavior | Needs change? |
|
||||
|------|-----------------|----------------|
|
||||
| `internal/certpaths/certpaths.go:21-26` | `Dir()` honors `ORCA_HOME` → `~/.orca` | **No** — this is the single source of truth. Already correct. |
|
||||
| `internal/store/store.go:13-19` | `Open("")` hardcodes `~/.orca/orca.db` (ignores `ORCA_HOME`) | **Yes** — route through `certpaths.DBPath()` instead. |
|
||||
| `internal/cli/init.go:16-22` | Hardcodes `~/.orca` via `os.UserHomeDir()` | **Yes** — route through `certpaths.Dir()`. |
|
||||
|
||||
All other call sites (`node.go:openDB`, `daemon.go`, `job.go`, `doctor.go`,
|
||||
`cert.go`) already go through `certpaths.DBPath()` or `certpaths.Dir()`
|
||||
indirectly. **No other files need changes for REQ-041.**
|
||||
|
||||
**For REQ-042 (`--system`)**: Add a `--system` persistent flag on
|
||||
`rootCmd`. When set, `rootCmd.PersistentPreRunE` sets
|
||||
`os.Setenv("ORCA_HOME", "/root/.orca")` before any subcommand runs.
|
||||
This is the minimal-touch approach — all downstream code already
|
||||
honors `ORCA_HOME`. The flag is a CLI convenience that maps to the
|
||||
env var, not a parallel mechanism.
|
||||
|
||||
**Backward compatibility**: empty `ORCA_HOME` + no `--system` →
|
||||
`~/.orca` (unchanged). Existing tests that `t.Setenv("ORCA_HOME", ...)`
|
||||
continue to work.
|
||||
|
||||
## R-005: Distroless Base Image for CGO-free Go Binaries
|
||||
|
||||
**Source**: Go module audit — `modernc.org/sqlite` (pure Go, CGO-free),
|
||||
`go.mod` has no CGO dependencies.
|
||||
|
||||
**Findings**:
|
||||
- `gcr.io/distroless/static-debian12` is the correct base for static
|
||||
Go binaries with no CGO and no libc dependency. ~2MB image.
|
||||
- orca uses `modernc.org/sqlite` (pure Go) — no CGO, no libc. ✓
|
||||
- Multi-stage Dockerfile:
|
||||
- Stage 1 (`golang:1.25`): build with `-trimpath -ldflags` (same as
|
||||
Makefile), output `bin/orca`.
|
||||
- Stage 2 (`gcr.io/distroless/static-debian12`): `COPY bin/orca /orca`,
|
||||
`ENTRYPOINT ["/orca"]`.
|
||||
- `CGO_ENABLED=0` must be set in the build stage to guarantee a static
|
||||
binary (Go defaults to CGO_ENABLED=1 on platforms with a C compiler).
|
||||
- The image runs as `nonroot` user by default in distroless — but orca
|
||||
writes to `~/.orca` (or `/root/.orca` for `--system`). For the
|
||||
container image, default `ORCA_HOME=/var/lib/orca` and document
|
||||
volume mount at that path.
|
||||
|
||||
**Implication for P03**: Dockerfile is ~15 lines. The `.coreci.yml`
|
||||
release pipeline adds a `docker build --build-arg VERSION=$VERSION -t
|
||||
git.cloudinit.dev/coreci/orca:$VERSION .` step + login + push.
|
||||
|
||||
## R-006: install.sh Conventions (curl|sh pattern)
|
||||
|
||||
**Source**: Common patterns from deno, rustup, homebrew installers.
|
||||
|
||||
**Findings**:
|
||||
- 1-liner: `curl -fsSL <url> | bash` (or `| bash -s -- --system`).
|
||||
- The script must be downloadable from a stable URL. orca's script
|
||||
lives at `scripts/install.sh` in the repo, accessible via
|
||||
`https://git.cloudinit.dev/coreci/orca/raw/branch/main/scripts/install.sh`
|
||||
(once repo is public per R-002).
|
||||
- Args passed via `bash -s -- --system --version v0.4.4`.
|
||||
- In-place update: detect existing binary at install path, read its
|
||||
version via `orca version --json` (parse `version` field), print
|
||||
"updated from X to Y", overwrite binary. **Never** touch the
|
||||
namespace dir (`~/.orca` or `/root/.orca`) — that's user state.
|
||||
- User-level default: `~/.local/bin/orca` (XDG-ish, on PATH on most
|
||||
modern distros). System-level: `/usr/local/bin/orca` (requires root).
|
||||
|
||||
**Implication for P02**: install.sh is ~80-100 lines of bash. Idempotent.
|
||||
Tested via a `scripts/install_test.sh` that mocks the download and
|
||||
verifies path selection + update-in-place.
|
||||
|
||||
## Pitfalls (P-001..P-003)
|
||||
|
||||
- **P-001**: `docker` may not be available in the CoreCI release
|
||||
pipeline container. The `.coreci.yml` release step uses
|
||||
`image: golang:1.25` which does NOT include docker. **Mitigation**:
|
||||
the release pipeline must use a `docker:dind` sidecar or a step image
|
||||
that has the docker CLI. Alternatively, `scripts/release.sh` handles
|
||||
docker publish only when run locally or in a CI step that has docker.
|
||||
The `.coreci.yml` container step must use an image with docker CLI
|
||||
(e.g., `catthehacker/docker:docker-latest` or a custom image).
|
||||
|
||||
- **P-002**: Making the repo public exposes git history including the
|
||||
pre-existing `.env` SHA-1 leak (commit `00127ce` documented the
|
||||
rotate-forward decision; `.gitleaks-baseline.json` suppresses it for
|
||||
scanning). The leak is a **non-secret** (the token was rotated). This
|
||||
is an accepted risk per the existing decision — no new action needed,
|
||||
but document it in the P0 ship commit.
|
||||
|
||||
- **P-003**: `CGO_ENABLED=0` must be explicit in the Dockerfile build
|
||||
stage. Without it, `go build` in `golang:1.25` may produce a
|
||||
dynamically-linked binary that won't run in distroless. Verified:
|
||||
orca has no CGO deps, but `CGO_ENABLED=0` is belt-and-suspenders.
|
||||
Reference in New Issue
Block a user