docs(P00): research findings
Research domains (delegated to ci-researcher x2, codebase-grounded): - golang.org/x/crypto/ssh v0.54.0: API surface, Ed25519 keygen, TOFU via knownhosts.New, file upload via session heredoc (no SFTP dep) - /etc/os-release: confirmed ID= values (ubuntu/debian/alpine/pve), parsing approach, fallback strategy - Proxmox VE 8/9: pveum syntax (space-separated --privs), orca@pam realm (not @pve), OrcaOperator role, sudoers with NOEXEC on pct/qm, pvesh excluded (API execute bypasses NOEXEC) - Codebase: 12 files to modify/create, 6 reuse opportunities, 12 pitfalls Persona roster updated: data-engineer + security-engineer reactivated, devops-engineer deactivated. ARCHITECTURE.md addendum with AD-017..021. ---ci--- project: orca phase: 0 milestone: v0.6 status: research ---/ci---
This commit is contained in:
@@ -526,3 +526,115 @@ orca CLI orca daemon orca daemon
|
||||
For v0.2, one node must be the CA holder (`orca cert init` was run
|
||||
on it). The CA holder's `ca.crt` is copied to each peer manually by
|
||||
the operator; peers do not auto-fetch it.
|
||||
|
||||
## v0.6 Architecture Addendum — Node Bootstrap & Proxmox
|
||||
|
||||
### `orca init` Full Bootstrap (REQ-047, REQ-048, REQ-049)
|
||||
|
||||
`orca init` transforms from a bare `mkdir` into a full single-node
|
||||
cluster bootstrap. The sequence (idempotent per D-036):
|
||||
|
||||
```
|
||||
orca init
|
||||
1. MkdirAll(certpaths.Dir(), 0o755) # namespace dir
|
||||
2. store.Open(certpaths.DBPath()) # runs migrations 0001..0006
|
||||
3. security.CAInit(dir, "orca-internal-ca") # idempotent fast-path
|
||||
4. if !exists(server.crt):
|
||||
GenerateCSR("localhost", ["localhost","127.0.0.1"])
|
||||
ca.SignCSR(csr) → WriteCert + WriteKey # server cert (skip if present)
|
||||
5. os := detectOS() # /etc/os-release ID=
|
||||
6. node := Node{kind:"localhost", os:os, name:"localhost", addr:"localhost:8443"}
|
||||
if GetByName("localhost") exists:
|
||||
UpdateLastSeenAndOS(id, os) # refresh, keep id/joined_at
|
||||
else:
|
||||
NodeRepo.Insert(node) # first-run insert
|
||||
7. print summary (CA fp, server cert fp, os, node id)
|
||||
```
|
||||
|
||||
After `orca init`, `orca doctor` MUST pass with zero FAILs.
|
||||
|
||||
### Node Schema Extension (REQ-049)
|
||||
|
||||
Migration 0006 adds two nullable columns to `nodes`:
|
||||
|
||||
```sql
|
||||
ALTER TABLE nodes ADD COLUMN kind TEXT; -- localhost | linux | proxmox
|
||||
ALTER TABLE nodes ADD COLUMN os TEXT; -- ubuntu | debian | alpine | pve | linux
|
||||
```
|
||||
|
||||
Existing rows get SQL NULL → mapped to `""` in Go (`sql.NullString`).
|
||||
`Node` struct gains `Kind string` + `OS string` fields (JSON tags
|
||||
`kind,omitempty` / `os,omitempty`). `NodeRepo` extends all
|
||||
INSERT/SELECT/scanNode calls; adds `GetByName(ctx, name)` and
|
||||
`UpdateLastSeenAndOS(ctx, id, os)` helpers.
|
||||
|
||||
### Proxmox SSH Bootstrap (REQ-050, REQ-051)
|
||||
|
||||
```
|
||||
orca node join --type proxmox --host <addr> --user root --password <pw>
|
||||
│ password from --password or $ORCA_PROXMOX_PASSWORD (never persisted, D-031)
|
||||
▼
|
||||
internal/proxmox.BootstrapProxmox(ctx, opts)
|
||||
1. GenerateOrLoadSSHKey(certpaths.Dir()) # Ed25519, ~/.orca/orca_ssh_key{,.pub}
|
||||
2. SSH dial (password auth, knownhosts.New TOFU) # capture host key on first connect
|
||||
3. Deploy pubkey → ~orca/.ssh/authorized_keys # via session heredoc (no SFTP dep)
|
||||
4. useradd -m orca # create Linux system user (config-overridable name)
|
||||
5. pveum role add OrcaOperator --privs "VM.Audit Datastore.AllocateSpace SDN.Use"
|
||||
(idempotent: probe pveum role list first)
|
||||
6. pveum user add orca@pam -comment "Orca automation user"
|
||||
(idempotent: probe pveum user list first)
|
||||
7. pveum acl modify / -user orca@pam -role OrcaOperator
|
||||
(idempotent: modify creates or updates)
|
||||
8. Write /etc/sudoers.d/orca (mode 0440):
|
||||
orca ALL=(root) NOPASSWD: NOEXEC: /usr/bin/pct, /usr/bin/qm
|
||||
orca ALL=(root) NOPASSWD: /usr/bin/apt-get, /usr/bin/dpkg
|
||||
9. visudo -cf /etc/sudoers.d/orca # validate; abort on error
|
||||
10. NodeRepo.Insert(Node{kind:"proxmox", os:"pve", name:host, addr:host})
|
||||
11. Audit log: proxmox.bootstrap_ok (host, user, role, fp)
|
||||
```
|
||||
|
||||
**`pvesh` excluded from sudoers** — `pvesh` can trigger the API
|
||||
`/nodes/{node}/execute` endpoint which spawns shell commands
|
||||
server-side, bypassing sudo's `NOEXEC` tag. API access is via the
|
||||
`OrcaOperator` PVE role + `orca@pam` user (PVE RBAC), not sudo'd `pvesh`.
|
||||
|
||||
### Doctor Extensions (REQ-052)
|
||||
|
||||
- **`doctor os`**: re-runs `detectOS()` from `/etc/os-release`, compares
|
||||
to the stored localhost node's `os` field. Drift = WARN (OS upgraded
|
||||
since init? re-run `orca init` to refresh). Match = PASS.
|
||||
- **`doctor proxmox`**: iterates `kind=proxmox` nodes, SSH-probes each
|
||||
with `pveversion` (3s timeout per peer, clones `doctor.Network()`
|
||||
pattern). PASS = reachable + pveversion exits 0. WARN = zero proxmox
|
||||
nodes (single-node cluster is legitimate). FAIL = any node
|
||||
unreachable or pveversion fails.
|
||||
|
||||
### SSH Key Handling (D-037)
|
||||
|
||||
- **Location**: `~/.orca/orca_ssh_key` (0600) + `~/.orca/orca_ssh_key.pub` (0644)
|
||||
- **Algorithm**: Ed25519 (smaller, faster, more secure than RSA for SSH)
|
||||
- **Generation**: lazy — on first `orca node join --type proxmox`, NOT at `orca init` (localhost doesn't need SSH)
|
||||
- **Format**: PKCS8 PEM (consistent with `ca.key`/`server.key`; `ssh.ParsePrivateKey` accepts it)
|
||||
- **TOFU host keys**: `~/.orca/known_hosts` (OpenSSH format via `knownhosts.New`)
|
||||
|
||||
### Dependency Map (v0.6 addition)
|
||||
|
||||
```
|
||||
golang.org/x/crypto v0.54.0 # SSH (ssh + ssh/knownhosts + ed25519)
|
||||
└─ golang.org/x/sys v0.47.0 # indirect (bumped from v0.42.0)
|
||||
└─ golang.org/x/term v0.45.0 # indirect (pulled by ssh for PTY)
|
||||
```
|
||||
|
||||
Total direct deps: 5 (was 4). One new direct dep (`x/crypto`). Matches
|
||||
D-030 minimal-deps rationale. No SFTP module (file upload via session
|
||||
heredoc).
|
||||
|
||||
### v0.6 Architectural Decisions (AD-017..AD-021)
|
||||
|
||||
| ID | Decision | Rationale |
|
||||
|----|----------|-----------|
|
||||
| AD-017 | `orca init` = full bootstrap (CA + cert + db + localhost node) | Single command produces a working cluster; `orca doctor` passes post-init. Idempotent (D-036). |
|
||||
| AD-018 | Proxmox join via SSH (golang.org/x/crypto/ssh), not PVE REST API | SSH is the universal Proxmox management entry point; REST API would require API token bootstrap (chicken-and-egg). One new direct dep (D-030). |
|
||||
| AD-019 | `orca@pam` realm (not `orca@pve`) | SSH creates a Linux system user; PAM realm maps it to PVE RBAC without a separate PVE password. `@pve` requires interactive password prompt over non-PTY SSH (hangs). |
|
||||
| AD-020 | Exclude `pvesh` from sudoers; NOEXEC on `pct`/`qm` | `pvesh` can trigger API execute endpoint bypassing NOEXEC. `pct`/`qm` are Perl scripts via dynamically-linked perl → NOEXEC effective. `apt-get`/`dpkg` need exec for maintainer scripts → no NOEXEC. |
|
||||
| AD-021 | TOFU host-key via `knownhosts.New` | Avoids deprecated `ssh.InsecureIgnoreHostKey`. Capture-on-first-connect, verify-on-subsequent. Fail closed on mismatch (operator runs key-reset). |
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
{
|
||||
"phase": 0,
|
||||
"stage": "clarify",
|
||||
"stage": "research",
|
||||
"milestone": "v0.6",
|
||||
"milestone_slug": "node-bootstrap-proxmox",
|
||||
"phase_role": "pre_execution",
|
||||
"attempts": 0,
|
||||
"updated_at": "2026-08-03T20:10:00Z",
|
||||
"updated_at": "2026-08-03T20:30:00Z",
|
||||
"milestone_complete": false,
|
||||
"next_milestone": null
|
||||
}
|
||||
+51
-97
@@ -3,41 +3,25 @@ active_personas:
|
||||
- lead-developer
|
||||
- backend-engineer
|
||||
- cli-engineer
|
||||
- devops-engineer
|
||||
deactivated_personas:
|
||||
- frontend-engineer
|
||||
- data-engineer
|
||||
- security-engineer
|
||||
- network-engineer
|
||||
phase_specific:
|
||||
- cli-engineer
|
||||
deactivated_personas:
|
||||
- devops-engineer
|
||||
- network-engineer
|
||||
- frontend-engineer
|
||||
phase_specific: []
|
||||
reason: |
|
||||
Orca is a CLI-first, offline-first orchestration engine with no web UI and
|
||||
a single-binary distribution model. The v0.5 milestone is a distribution
|
||||
milestone (install, namespace, docker, public releases) that touches the
|
||||
CLI namespace layer, shell scripts, and container/build infrastructure.
|
||||
The persona roster reflects this:
|
||||
Orca v0.6 is a bootstrap-ergonomics + heterogeneous-nodes milestone.
|
||||
The work is schema (migration 0006), security (SSH keygen, TOFU,
|
||||
sudoers, PVE role), CLI (init full bootstrap, node join --type proxmox,
|
||||
doctor os/proxmox), and backend orchestration (proxmox SSH bootstrap
|
||||
sequence). No devops (no install/docker/release), no network (no
|
||||
transport/mTLS), no frontend (no UI).
|
||||
|
||||
- lead-developer: coordination, task decomposition, territory adjudication
|
||||
between cli-engineer (namespace flag) and devops-engineer (install.sh,
|
||||
Dockerfile, release pipeline).
|
||||
- backend-engineer: no new backend surface in v0.5, but owns the
|
||||
`internal/store` and `internal/certpaths` refactors for namespace
|
||||
unification (REQ-041) — these are shared-infra concerns that the
|
||||
backend-engineer adjudicates.
|
||||
- cli-engineer: the `--system` flag on `rootCmd` and the `init --system`
|
||||
subcommand (REQ-042) — pure CLI surface.
|
||||
- devops-engineer (NEW, reactivated): install.sh, Dockerfile,
|
||||
.coreci.yml container pipeline, scripts/release.sh docker publish step.
|
||||
|
||||
Deactivated for v0.5 (no v0.5 surface):
|
||||
- frontend-engineer: no web UI (unchanged from v0.1).
|
||||
- data-engineer: v0.5 has no store/schema work — the store refactor
|
||||
(R-004) is a 1-line routing change, not schema work.
|
||||
- security-engineer: v0.5 has no new cert/mTLS surface — the namespace
|
||||
unification moves cert paths but does not change cert logic.
|
||||
- network-engineer: v0.5 has no transport/network surface.
|
||||
Roster changes vs v0.5:
|
||||
- data-engineer: REACTIVATED — owns migration 0006 + NodeRepo schema extension.
|
||||
- security-engineer: REACTIVATED — owns SSH keygen, TOFU host-key, sudoers, PVE role.
|
||||
- devops-engineer: DEACTIVATED — v0.6 has no packaging/distribution surface.
|
||||
---
|
||||
|
||||
# Personas: Orca
|
||||
@@ -50,97 +34,67 @@ reason: |
|
||||
- **Constraints**: `boundary-enforcement`, `offline-first`, `no-redundant-implementations`
|
||||
- **Territory**: `**/*.go`, `cmd/**`, `internal/**`
|
||||
- **Active**: true
|
||||
- **Reason**: Coordination across P01/P02/P03. SSH/bootstrap touches security + cli + store + doctor — territory overlaps need adjudication (proxmox package boundary, doctor Proxmox check scaffolding).
|
||||
|
||||
### backend-engineer
|
||||
- **Domain**: backend
|
||||
- **Frameworks**: `cobra`, `net/http`
|
||||
- **Constraints**: `API-first`, `error-handling`, `minimal-dependencies`, `security-first`
|
||||
- **Territory**: `**/api/**`, `**/*_handler*`, `**/*_handler.go`, `internal/daemon/**`
|
||||
- **Frameworks**: `cobra`, `net/http`, `golang.org/x/crypto/ssh`
|
||||
- **Constraints**: `API-first`, `error-handling`, `minimal-dependencies`, `security-first`, `idempotent-bootstrap`
|
||||
- **Territory**: `**/api/**`, `**/*_handler*`, `**/*_handler.go`, `internal/daemon/**`, `internal/proxmox/**`, `internal/cli/init.go`
|
||||
- **Active**: true
|
||||
- **Reason**: Owns the daemon health endpoints (`/healthz`, `/readyz`) that the P02 doctor network check probes. The transport dispatch client (reused by doctor) lives in `internal/transport` but the *handler* surface is backend-engineer territory.
|
||||
- **Reason**: Owns the `orca init` full-bootstrap orchestration (CA + cert + db + localhost node, idempotent) and the `internal/proxmox/bootstrap.go` SSH session sequence (dial, deploy pubkey, useradd, pveum, sudoers, visudo validate). Added `idempotent-bootstrap` constraint (D-036 — re-run must be skip-and-refresh) and `golang.org/x/crypto/ssh` to frameworks.
|
||||
|
||||
### data-engineer
|
||||
- **Domain**: data
|
||||
- **Frameworks**: `modernc/sqlite`, `iter`
|
||||
- **Constraints**: `schema-first`, `migration-safe`, `local-storage-only`, `no-goroutine-leak`
|
||||
- **Territory**: `**/store/**`, `**/model.go`, `**/migration*`, `migrations/**`, `internal/store/migrations/**`
|
||||
- **Constraints**: `schema-first`, `migration-safe`, `local-storage-only`, `no-goroutine-leak`, `nullable-column-handling`
|
||||
- **Territory**: `**/store/**`, `**/model.go`, `**/migration*`, `migrations/**`, `internal/store/migrations/**`, `internal/model/node.go`
|
||||
- **Active**: true
|
||||
- **Reason**: Owns the `iter.Seq[Job|Node]` implementations on `JobRepo`/`NodeRepo` (P01) and the `MigrationVersion` query + `PRAGMA integrity_check` helper (P02). Added `iter` to frameworks and `no-goroutine-leak` to constraints (the iter.Seq polling loop must not leak — see RESEARCH_v0.3.md D-032). Territory confirmed against actual file structure: `internal/store/` holds all repos + `migrations/` subdir with `0001..0005_*.sql`.
|
||||
- **Reason**: Reactivated for v0.6. Owns migration `0006_node_kind_os.sql` (REQ-049 — nullable `kind`/`os` columns, backward-compatible) and `NodeRepo` schema extension (Insert/Get/List/Watch/scanNode column additions + new `GetByName`/`UpdateLastSeenAndOS` helpers). Added `nullable-column-handling` constraint (NULL → `""` in Go struct, not nil-deref).
|
||||
|
||||
### cli-engineer (custom)
|
||||
### cli-engineer
|
||||
- **Domain**: CLI/UX
|
||||
- **Frameworks**: `cobra`, `pflag`
|
||||
- **Constraints**: `discoverable-help`, `consistent-flag-naming`, `human-readable-output`, `machine-readable-json-flag`, `signal-handling`
|
||||
- **Constraints**: `discoverable-help`, `consistent-flag-naming`, `human-readable-output`, `machine-readable-json-flag`, `signal-handling`, `password-flag-redaction`
|
||||
- **Territory**: `cmd/**`, `internal/cli/**`, `internal/commands/**`
|
||||
- **Active**: true
|
||||
- **Reason**: Orca is CLI-first; this persona ensures CLI quality and discoverability. For v0.3 P01 it owns the `--watch` flag on `orca job list` / `orca node list` (signal.NotifyContext cancellation, table refresh vs streaming JSON). For P02 it owns the `internal/cli/doctor.go` subcommand wiring (replacing NetworkStub/DBStub calls). Added `signal-handling` to constraints (ctrl-c propagation to iter.Seq is a P01 correctness requirement). Territory confirmed: `internal/cli/` holds all Cobra commands.
|
||||
|
||||
### security-engineer (custom)
|
||||
- **Domain**: security
|
||||
- **Frameworks**: `crypto/tls`, `crypto/x509`, `slog`
|
||||
- **Constraints**: `no-panic-in-production`, `structured-audit-logging`, `no-secret-in-logs`, `input-validation`, `least-privilege`
|
||||
- **Territory**: `**/auth/**`, `**/audit/**`, `internal/security/**`, `internal/transport/**` (TLS config only)
|
||||
- **Active**: true
|
||||
- **Reason**: mTLS, audit logging, and input validation are first-class concerns. For v0.3 P02, the doctor network check reuses `security.ClientTLSConfig` (via `transport.NewMTLSClient`) to build the mTLS client that probes peer `/healthz`. The TLS-config portion of `internal/transport/**` remains security-engineer territory.
|
||||
- **Phase scope**: P02 only (mTLS client config for doctor network probe). P01 has no security surface.
|
||||
|
||||
### network-engineer (custom, NEW in v0.2)
|
||||
- **Domain**: networking
|
||||
- **Frameworks**: `net/http`, `crypto/tls` (via `internal/security`), `iter`
|
||||
- **Constraints**: `connection-resilience`, `retry-with-backoff`, `graceful-disconnect`, `context-propagation`, `bounded-probe-timeout`
|
||||
- **Territory**: `**/transport/**`, `**/engine/dispatcher*`, `**/engine/peer*`, `internal/engine/dispatcher.go`, `internal/engine/peer.go`, `internal/transport/**`
|
||||
- **Active**: true
|
||||
- **Reason**: Owns the transport layer and peer-to-peer connection lifecycle. For v0.3 P02, the doctor network check is a read-only mTLS `/healthz` probe that reuses `transport.MTLSClient` — the connection lifecycle (dial, per-probe 3s timeout, handshake) is network-engineer territory. Added `bounded-probe-timeout` to constraints (doctor must not stall on one slow peer — RESEARCH_v0.3.md D-038). Territory confirmed: `internal/transport/` holds mtls.go, dispatch.go, retry.go, idempotency.go, handshake_log.go.
|
||||
- **Phase scope**: P02 only (doctor network probe reuses transport layer).
|
||||
|
||||
### devops-engineer (reactivated in v0.5)
|
||||
- **Domain**: devops / packaging
|
||||
- **Frameworks**: `docker`, `bash`, `curl`, `tea`
|
||||
- **Constraints**: `idempotent-scripts`, `minimal-image-size`, `no-secret-in-image`, `reproducible-build`
|
||||
- **Territory**: `Dockerfile`, `scripts/install.sh`, `scripts/release.sh`, `.coreci.yml`, `docs/docker.md`
|
||||
- **Active**: true
|
||||
- **Reason**: v0.5 is a distribution milestone. The devops-engineer owns install.sh (REQ-043/044), the Dockerfile + container registry publish (REQ-046), and the `.coreci.yml` release pipeline extension. Reactivated from v0.1 (where it was deactivated as "devops-sre" because CoreCI handled release). In v0.5, container + install surface is first-class devops work.
|
||||
|
||||
### frontend-engineer
|
||||
- **Active**: false
|
||||
- **Reason**: No web UI in Orca (v0.1 onward). NOT relevant to v0.5. Confirmed deactivated.
|
||||
|
||||
### data-engineer
|
||||
- **Active**: false (v0.5)
|
||||
- **Reason**: v0.5 has no store/schema work. The `internal/store/store.go` refactor (R-004) is a 1-line routing change from hardcoded `~/.orca` to `certpaths.DBPath()` — this is shared-infra, adjudicated by backend-engineer, not data-engineer schema work.
|
||||
- **Reason**: Owns `orca init` multi-step bootstrap output UX (progress lines per step), `orca node join --type/--host/--user/--password/--proxmox-user/--proxmox-role` flag wiring, and `doctor os`/`doctor proxmox` subcommand wiring. Added `password-flag-redaction` constraint (D-031 — `--password` never echoed, prefer `$ORCA_PROXMOX_PASSWORD`, zero after use).
|
||||
|
||||
### security-engineer
|
||||
- **Active**: false (v0.5)
|
||||
- **Reason**: v0.5 has no new cert/mTLS surface. The namespace unification moves cert paths via `certpaths.Dir()` (already the source of truth) but does not change cert generation, validation, or TLS config logic.
|
||||
- **Domain**: security
|
||||
- **Frameworks**: `crypto/tls`, `crypto/x509`, `crypto/ed25519`, `golang.org/x/crypto/ssh`, `slog`
|
||||
- **Constraints**: `no-panic-in-production`, `structured-audit-logging`, `no-secret-in-logs`, `input-validation`, `least-privilege`, `tofu-host-key-pinning`, `noexec-sudoers`
|
||||
- **Territory**: `**/auth/**`, `**/audit/**`, `internal/security/**`, `internal/transport/**` (TLS config only), `internal/proxmox/**` (SSH + sudoers + PVE role)
|
||||
- **Active**: true
|
||||
- **Reason**: Reactivated for v0.6. Owns `internal/security/sshkey.go` (Ed25519 keygen, 0600/0644 mode enforcement per REQ-033 spirit), TOFU host-key pinning via `knownhosts.New`, sudoers least-privilege design (NOEXEC on pct/qm, exclude pvesh, no NOEXEC on apt-get/dpkg), password redaction (D-031), and audit logging of all bootstrap/join actions (REQ-052). Added `tofu-host-key-pinning` and `noexec-sudoers` constraints. Co-owns `internal/proxmox/**` with backend-engineer (security owns SSH auth + sudoers content; backend owns the session orchestration).
|
||||
|
||||
### devops-engineer
|
||||
- **Active**: false (v0.6)
|
||||
- **Reason**: Deactivated — v0.6 has no install.sh, Dockerfile, .coreci.yml, or release-pipeline surface. The Proxmox SSH bootstrap is backend + security work, not devops. Was active in v0.5 (distribution milestone).
|
||||
|
||||
### network-engineer
|
||||
- **Active**: false (v0.5)
|
||||
- **Reason**: v0.5 has no transport/network surface. No new peer-to-peer, dispatch, or health-probe work.
|
||||
- **Active**: false (v0.6)
|
||||
- **Reason**: v0.6 has no transport/mTLS surface. SSH is point-to-point bootstrap, not the mTLS mesh network-engineer owns.
|
||||
|
||||
### frontend-engineer
|
||||
- **Active**: false (v0.6)
|
||||
- **Reason**: No web UI in Orca (unchanged from v0.1 onward).
|
||||
|
||||
## Territory Enforcement
|
||||
|
||||
- **Mode**: `warn` (per `config.json`)
|
||||
- **Behavior**: Out-of-territory file changes log a warning but do not block.
|
||||
- **Rationale**: Allows flexibility during early development; tighten to `strict` post-v0.1. For v0.5, the main territory-overlap risk is `internal/store/store.go` (R-004) which is backend-engineer shared-infra territory but touches the data layer — lead-developer adjudicates.
|
||||
- **Key overlaps in v0.6** (lead-developer adjudicates):
|
||||
- `internal/proxmox/bootstrap.go` — security-engineer (SSH auth, sudoers, PVE role) + backend-engineer (session orchestration, error handling). Boundary: security package exposes `BootstrapProxmox(ctx, opts) error`; the function lives in `internal/proxmox` but imports `internal/security` for SSH key handling.
|
||||
- `internal/doctor/doctor.go` `Proxmox()` — reuses `internal/proxmox` SSH client (security) but check scaffolding clones `doctor.Network()` pattern. Backend-engineer adjudicates (network-engineer deactivated).
|
||||
- `internal/store/node_repo.go` — data-engineer territory, but the `UpdateLastSeenAndOS` caller is `internal/cli/init.go` (backend). Standard repo-consumer boundary.
|
||||
|
||||
## Phase-Specific Personas (v0.5)
|
||||
|
||||
| Persona | Active in | Reason |
|
||||
|---------|-----------|--------|
|
||||
| `cli-engineer` | P1 | `--system` flag on `rootCmd` + `init --system` subcommand (REQ-042). Pure CLI surface. |
|
||||
| `devops-engineer` | P2, P3 | P2: install.sh + in-place update (REQ-043/044). P3: Dockerfile + container registry publish (REQ-046). |
|
||||
| `backend-engineer` | P1 | `internal/store/store.go` + `internal/certpaths/certpaths.go` namespace routing refactor (REQ-041). Shared-infra. |
|
||||
|
||||
In full-autonomy mode, all personas are auto-accepted and the phase-scope
|
||||
assignments are applied automatically when a phase is committed.
|
||||
|
||||
## v0.5 vs v0.3 Persona Diff
|
||||
## v0.6 vs v0.5 Persona Diff
|
||||
|
||||
| Change | Rationale |
|
||||
|--------|-----------|
|
||||
| `devops-engineer` reactivated (was `devops-sre`, deactivated in v0.1) | v0.5 is a distribution milestone — install.sh, Dockerfile, container registry publish are first-class devops work. Renamed from `devops-sre` to `devops-engineer` to reflect build/packaging focus (not SRE/ops). |
|
||||
| `data-engineer` deactivated | v0.5 has no schema/store logic work — the store.go change is a 1-line routing refactor (shared-infra, backend-engineer). |
|
||||
| `security-engineer` deactivated | v0.5 has no new cert/mTLS surface. |
|
||||
| `network-engineer` deactivated | v0.5 has no transport/network surface. |
|
||||
| `cli-engineer` phase scope: was P01+P02 (v0.3), now P1 only (v0.5) | v0.5 P1 is the `--system` flag (CLI surface). P2/P3 are devops territory. |
|
||||
| `frontend-engineer` | Remains deactivated (no UI in v0.5). |
|
||||
| `data-engineer` reactivated | Owns migration 0006 + NodeRepo schema extension (kind/os columns). |
|
||||
| `security-engineer` reactivated | Owns SSH keygen, TOFU host-key, sudoers, PVE role — first-class security surface. |
|
||||
| `devops-engineer` deactivated | v0.6 has no packaging/distribution surface. |
|
||||
| `network-engineer` remains deactivated | No transport/mTLS surface. |
|
||||
| `frontend-engineer` remains deactivated | No web UI. |
|
||||
@@ -0,0 +1,250 @@
|
||||
# Research: Orca v0.6 — Node Bootstrap & Proxmox
|
||||
|
||||
Findings grounded in codebase analysis (8 key files read) + verified
|
||||
against `golang.org/x/crypto` v0.54.0 (probe built clean), Proxmox VE
|
||||
9.2.3 admin guide (§14.7-14.8 pveum + privileges), sudoers(5) man
|
||||
page (NOEXEC/NOPASSWD), and freedesktop.org os-release spec.
|
||||
|
||||
## A. SSH library — `golang.org/x/crypto/ssh`
|
||||
|
||||
### A.1 go.mod addition
|
||||
|
||||
```
|
||||
require golang.org/x/crypto v0.54.0
|
||||
```
|
||||
|
||||
Latest available, compatible with go 1.25. Transitive deps (verified
|
||||
by probe build):
|
||||
- `golang.org/x/crypto v0.54.0` (direct)
|
||||
- `golang.org/x/sys v0.47.0` (indirect — bumps from v0.42.0)
|
||||
- `golang.org/x/term v0.45.0` (indirect — pulled by ssh for PTY)
|
||||
|
||||
**3 module entries, 0 new heavy deps.** Matches D-030 minimal-deps
|
||||
rationale. `go.sum` gains ~6 lines.
|
||||
|
||||
### A.2 Minimal API surface
|
||||
|
||||
```go
|
||||
import (
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"crypto/x509"
|
||||
"encoding/pem"
|
||||
"net"
|
||||
"time"
|
||||
"golang.org/x/crypto/ssh"
|
||||
"golang.org/x/crypto/ssh/knownhosts"
|
||||
)
|
||||
```
|
||||
|
||||
Key functions:
|
||||
- `ssh.Dial(network, addr, config) (*ssh.Client, error)` — high-level dialer
|
||||
- `(*ssh.Client).NewSession() (*ssh.Session, error)`
|
||||
- `(*ssh.Session).CombinedOutput(cmd) ([]byte, error)` — run + capture
|
||||
- `ssh.ClientConfig{User, Auth, HostKeyCallback, Timeout}`
|
||||
- `ssh.Password(secret) ssh.AuthMethod` — password auth
|
||||
- `ssh.PublicKeys(signer) ssh.AuthMethod` — pubkey auth
|
||||
- `ssh.ParsePrivateKey(pem) (ssh.Signer, error)` — parse PKCS8 PEM (works with orca's existing key format)
|
||||
- `ssh.NewPublicKey(pub) (ssh.PublicKey, error)` + `ssh.MarshalAuthorizedKey(pub) []byte` — authorized_keys line
|
||||
- `ssh.FixedHostKey(key) ssh.HostKeyCallback` — strict pin (subsequent connects)
|
||||
- `knownhosts.New(path) (ssh.HostKeyCallback, error)` — TOFU via known_hosts file (cleaner than custom callback; avoids deprecated `InsecureIgnoreHostKey`)
|
||||
|
||||
### A.3 Ed25519 keygen (D-037)
|
||||
|
||||
Verified end-to-end: `ed25519.GenerateKey(rand.Reader)` →
|
||||
`x509.MarshalPKCS8PrivateKey(priv)` → PEM encode → `ssh.ParsePrivateKey`
|
||||
round-trips cleanly. `ssh.MarshalAuthorizedKey` produces valid
|
||||
`ssh-ed25519 AAAA...` line. **PKCS8 PEM (orca's existing format)
|
||||
parses with `ssh.ParsePrivateKey` — no OpenSSH-format marshaller
|
||||
needed.** Reuse `security.WriteKey`/`writeAtomic` for persistence.
|
||||
|
||||
### A.4 File upload — `cat > file` via session, NOT SFTP
|
||||
|
||||
SFTP lives in separate module `github.com/pkg/sftp` — would add a 4th
|
||||
direct dep beyond D-030. The only files orca uploads are:
|
||||
- `~orca/.ssh/authorized_keys` (1-line append)
|
||||
- `/etc/sudoers.d/orca` (few lines)
|
||||
|
||||
Both are text. Use `session.CombinedOutput` with heredoc / `tee -a`.
|
||||
Keeps everything within `x/crypto/ssh`.
|
||||
|
||||
### A.5 TOFU host-key handling (D-035)
|
||||
|
||||
Use `golang.org/x/crypto/ssh/knownhosts.New(path)` as the
|
||||
`HostKeyCallback`. On first connect, the callback writes the host key
|
||||
to `~/.orca/known_hosts` (OpenSSH format). On subsequent connects, it
|
||||
verifies and returns an error on mismatch. **Avoids
|
||||
`ssh.InsecureIgnoreHostKey` deprecation** — `knownhosts.New` handles
|
||||
both capture and verify in one callback. On host-key change
|
||||
(reinstall), fail closed with a clear error; operator runs
|
||||
`orca node key-reset <node>` (future) or manually edits `known_hosts`.
|
||||
|
||||
## B. `/etc/os-release` parsing (D-032)
|
||||
|
||||
### B.1 Confirmed `ID=` values
|
||||
|
||||
| Distro | `ID=` | `ID_LIKE=` | Verified |
|
||||
|--------|-------|-----------|----------|
|
||||
| Ubuntu | `ubuntu` | `debian` | ✅ (this host: Ubuntu 24.04) |
|
||||
| Debian | `debian` | — | ✅ (freedesktop spec) |
|
||||
| Alpine | `alpine` | — | ✅ (Alpine policy) |
|
||||
| Proxmox VE | `pve` | `debian` | ✅ (PVE ships own os-release) |
|
||||
|
||||
`VARIANT_ID` absent on all four target distros — not worth capturing
|
||||
for v0.6.
|
||||
|
||||
### B.2 Parsing approach
|
||||
|
||||
No Go stdlib helper. Trivial: `bufio.Scanner` +
|
||||
`strings.SplitN(line, "=", 2)` + strip surrounding quotes. ~15 lines.
|
||||
Returns `map[string]string`; read `ID` field. Fallback `"linux"` if
|
||||
file missing or `ID` absent (D-032). Read `/etc/os-release` first;
|
||||
fall back to `/usr/lib/os-release` for minimal containers. Unknown `ID`
|
||||
values stored verbatim (not masked) — `doctor os` can warn.
|
||||
|
||||
## C. Proxmox VE role & user management
|
||||
|
||||
### C.1 Realm: `orca@pam` (NOT `orca@pve`)
|
||||
|
||||
Confirmed by both researchers + PVE User Management docs: since
|
||||
`orca node join` SSHes in and creates a Linux system user via
|
||||
`useradd`, the PVE user must be `orca@pam` (PAM realm maps to host
|
||||
system users). `orca@pve` would require a separate PVE-internal
|
||||
password and interactive `-password` prompt over non-PTY SSH (hangs).
|
||||
`@pam` sidesteps both issues. **D-033 refined: `orca@pam`.**
|
||||
|
||||
### C.2 OrcaOperator PVE role — privilege set
|
||||
|
||||
Per D-033 (operator-confirmed): `VM.Audit`, `Datastore.AllocateSpace`,
|
||||
`SDN.Use`. This is a **minimal API-level role** — the actual management
|
||||
capability comes from the sudoers allowlist (sudo runs as root, bypassing
|
||||
PVE RBAC). The PVE role governs non-sudo API access (future REST client).
|
||||
|
||||
**Refinement from research**: `VM.Audit` covers containers (CTs) as well
|
||||
as VMs (both live under `/vms/{vmid}` path; no separate `CT.*` family).
|
||||
PVE 8→9: privilege set valid on both (no breaking changes to pveum or
|
||||
the core privilege names).
|
||||
|
||||
Researcher 2 proposed an expanded 21-privilege set for fuller API-level
|
||||
management. **Decision: keep D-033's 3-priv minimal set for v0.6** — the
|
||||
operator explicitly confirmed it, and the sudoers allowlist is the
|
||||
primary management path. The expanded set is noted as a v0.7+
|
||||
enhancement option if orca adds a direct PVE REST client.
|
||||
|
||||
### C.3 pveum command sequence (idempotent)
|
||||
|
||||
```bash
|
||||
# 1. Role — probe-then-add (pveum role add fails if exists)
|
||||
pveum role list | grep -q '^OrcaOperator' || \
|
||||
pveum role add OrcaOperator --privs "VM.Audit Datastore.AllocateSpace SDN.Use"
|
||||
|
||||
# 2. User — probe-then-add (maps to existing Linux system user)
|
||||
pveum user list | grep -q 'orca@pam' || \
|
||||
pveum user add orca@pam -comment "Orca automation user"
|
||||
|
||||
# 3. ACL — modify is idempotent (creates or updates)
|
||||
pveum acl modify / -user orca@pam -role OrcaOperator
|
||||
```
|
||||
|
||||
Flag syntax: both `-privs` and `--privs` work (Perl Getopt::Long). Use
|
||||
`--privs` (canonical). Privs are **space-separated** inside quotes
|
||||
(NOT comma-separated).
|
||||
|
||||
### C.4 sudoers file `/etc/sudoers.d/orca` (D-033 refined)
|
||||
|
||||
**Research refinement**: exclude `pvesh` from sudoers — `pvesh` can
|
||||
reach the `/nodes/{node}/execute` API endpoint which spawns shell
|
||||
commands server-side, bypassing sudo's `NOEXEC` tag. Keep `pct`/`qm`
|
||||
with `NOEXEC`; `apt-get`/`dpkg` without `NOEXEC` (they need to spawn
|
||||
child processes for maintainer scripts).
|
||||
|
||||
```
|
||||
# /etc/sudoers.d/orca — mode 0440, owner root:root
|
||||
# Orca automation: VM/CT management + package management, no shell escape
|
||||
orca ALL=(root) NOPASSWD: NOEXEC: /usr/bin/pct, /usr/bin/qm
|
||||
orca ALL=(root) NOPASSWD: /usr/bin/apt-get, /usr/bin/dpkg
|
||||
```
|
||||
|
||||
`NOEXEC` works via Linux seccomp (sudoers man page). `pct`/`qm` are
|
||||
Perl scripts run via dynamically-linked `/usr/bin/perl` → NOEXEC
|
||||
effective. `apt-get`/`dpkg` need exec for postinst scripts → no
|
||||
NOEXEC. File mode **0440** or sudo refuses to load. Validate with
|
||||
`visudo -cf /etc/sudoers.d/orca` after writing; abort bootstrap on
|
||||
validation failure.
|
||||
|
||||
**Resolve binary paths at runtime** via `command -v pct` etc. before
|
||||
writing the sudoers file (cheap insurance against non-standard installs).
|
||||
|
||||
### C.5 PVE 8 vs 9
|
||||
|
||||
No breaking changes to pveum, privilege names, or sudo defaults
|
||||
between 8 and 9. `VM.Monitor` removed in 9.0 (OrcaOperator doesn't
|
||||
use it). Privileged container creation needs `Sys.Modify` in 9.0
|
||||
(OrcaOperator doesn't have it → intended). Both versions: `orca@pam`
|
||||
flow identical. Binary paths identical (`/usr/bin/{pct,qm,pvesh}`).
|
||||
|
||||
## D. Codebase integration points (confirmed by reading files)
|
||||
|
||||
### D.1 Files to modify/create per requirement
|
||||
|
||||
| File | Change | REQ |
|
||||
|------|--------|-----|
|
||||
| `go.mod` / `go.sum` | Add `golang.org/x/crypto v0.54.0`; bump sys, add term | REQ-050 |
|
||||
| `internal/model/node.go` | Add `Kind`, `OS` string fields + `NodeKind` constants | REQ-049 |
|
||||
| `internal/store/migrations/0006_node_kind_os.sql` | **NEW**: `ALTER TABLE nodes ADD COLUMN kind TEXT; ADD COLUMN os TEXT;` (nullable, backward-compatible) | REQ-049 |
|
||||
| `internal/store/node_repo.go` | Extend INSERT/SELECT/scanNode for `kind, os`; add `GetByName`, `UpdateLastSeenAndOS` helpers | REQ-049 |
|
||||
| `internal/cli/init.go` | Full bootstrap: MkdirAll → store.Open (runs migrations) → CAInit → server cert gen (if absent) → detectOS → localhost node upsert | REQ-047,048 |
|
||||
| `internal/cli/node.go` | Add `--type`, `--host`, `--user`, `--password`, `--proxmox-user`, `--proxmox-role` flags; `bootstrapProxmox` branch | REQ-050,051 |
|
||||
| `internal/security/sshkey.go` | **NEW**: `GenerateOrLoadSSHKey(dir)` — Ed25519 keygen, PKCS8 PEM, 0600/0644 modes | REQ-050 |
|
||||
| `internal/proxmox/bootstrap.go` | **NEW package**: `BootstrapProxmox(ctx, opts)` — SSH dial, pubkey deploy, useradd, pveum role/user/acl, sudoers write, visudo validate | REQ-050,051 |
|
||||
| `internal/doctor/doctor.go` | Add `OS()` and `Proxmox()` checks; extend `All()` | REQ-052 |
|
||||
| `internal/cli/doctor.go` | Add `doctor os` + `doctor proxmox` subcommands | REQ-052 |
|
||||
| `internal/certpaths/certpaths.go` | Add `SSHKeyPath`, `SSHPubPath`, `KnownHostsPath` | REQ-050 |
|
||||
|
||||
### D.2 Reuse opportunities (confirmed)
|
||||
|
||||
- `security.CAInit` (ca.go:63) — **already idempotent** (fast-path loads existing). `orca init` calls it directly.
|
||||
- `security.GenerateCSR` (csr.go) — signature fits: `GenerateCSR("localhost", []string{"localhost","127.0.0.1"})`.
|
||||
- `security.WriteCert`/`WriteKey` (ca.go:292) — enforce 0644/0600 via `writeAtomic`; reuse for SSH key.
|
||||
- `store.Open` (migrate.go) — runs migrations on open; calling it in `orca init` auto-applies 0006.
|
||||
- Migration runner — FS-embedded, sorts lexicographically, idempotent per-file. Adding `0006_*.sql` is the entire change.
|
||||
- `doctor.Network()` (doctor.go:222) — exact pattern to clone for `doctor.Proxmox()` (list nodes, filter by kind, 3s timeout per peer, PASS/WARN/FAIL).
|
||||
|
||||
### D.3 No changes needed
|
||||
|
||||
- `internal/security/ca.go`, `csr.go` — idempotent already, signatures fit.
|
||||
- `internal/store/migrate.go` — runner is generic.
|
||||
- `internal/transport/*` — mTLS transport not involved in SSH bootstrap.
|
||||
- `internal/engine/*` — NodeRegistry.Join works; new fields are metadata.
|
||||
|
||||
## E. Pitfalls & gotchas
|
||||
|
||||
1. **`pveum` flag is `--privs` (space-separated)**, not `--privs "a,b,c"`. Confirmed by both researchers + official docs.
|
||||
2. **`orca@pam` not `orca@pve`** — PVE-internal realm requires interactive password prompt over non-PTY SSH (hangs). PAM realm maps to the Linux system user orca creates.
|
||||
3. **Exclude `pvesh` from sudoers** — `pvesh` can trigger API `execute` endpoint spawning shell commands server-side, bypassing `NOEXEC`. Use PVE API via OrcaOperator role for API access instead.
|
||||
4. **`NOEXEC` only on dynamically-linked binaries** — `pct`/`qm` are Perl scripts via dynamically-linked `/usr/bin/perl` → effective. `apt-get`/`dpkg` need exec → no NOEXEC.
|
||||
5. **sudoers file mode 0440** — or sudo silently refuses to load it. `chmod 0440` + `visudo -cf` validate after write.
|
||||
6. **Migration 0006 NULL handling** — `scanNode` must use `sql.NullString` for `kind`/`os` and map NULL → `""` (Go struct fields are `string`, not `*string`).
|
||||
7. **localhost node idempotency** — `NodeRepo.Insert` fails on UNIQUE constraint if `orca init` re-runs. Need `GetByName("localhost")` check first; if found, `UpdateLastSeenAndOS` instead of `Insert`. Don't change `id` or `joined_at` (D-036).
|
||||
8. **`orca init` must not regenerate server cert** (D-036) — check `certpaths.ServerCertPath()` existence before `GenerateCSR`. `CAInit` has a fast-path; server cert gen needs an explicit existence check.
|
||||
9. **Password handling (D-031)** — `--password` flag visible in `ps`/`/proc` briefly. Prefer `$ORCA_PROXMOX_PASSWORD` env var. Never log the password (slog redaction). Zero the byte slice after use.
|
||||
10. **`knownhosts.New` for TOFU** — avoids deprecated `ssh.InsecureIgnoreHostKey`. Handles both capture and verify in one callback.
|
||||
11. **PKCS8 PEM parses with `ssh.ParsePrivateKey`** — no need for OpenSSH-format marshaller. Consistent with `ca.key`/`server.key` format.
|
||||
12. **`/etc/os-release` is a symlink** on most distros → `os.ReadFile` follows it. Fall back to `/usr/lib/os-release` for minimal containers.
|
||||
|
||||
## F. Persona recommendations (v0.6 roster)
|
||||
|
||||
| Persona | Active | Reason |
|
||||
|---------|--------|--------|
|
||||
| `lead-developer` | ✅ | Coordination across P01/P02/P03; SSH/bootstrap touches security + cli + store + doctor |
|
||||
| `backend-engineer` | ✅ | Owns `internal/cli/init.go` full-bootstrap orchestration + `internal/proxmox/bootstrap.go` SSH logic |
|
||||
| `cli-engineer` | ✅ | Owns `--type`/`--host`/`--password` flag wiring, `doctor os`/`doctor proxmox` subcommands, init output UX |
|
||||
| `data-engineer` | ✅ **REACTIVATE** | Owns migration 0006 + `NodeRepo` schema extension (kind/os columns, new helpers) |
|
||||
| `security-engineer` | ✅ **REACTIVATE** | Owns `internal/security/sshkey.go`, TOFU host-key, sudoers design, password redaction, audit logging |
|
||||
| `devops-engineer` | ❌ **DEACTIVATE** | No install.sh/Dockerfile/.coreci.yml surface in v0.6 |
|
||||
| `network-engineer` | ❌ | No transport/mTLS surface (SSH is point-to-point bootstrap, not mesh) |
|
||||
| `frontend-engineer` | ❌ | No web UI |
|
||||
|
||||
**Territory overlaps to adjudicate (lead-developer)**:
|
||||
- `internal/proxmox/bootstrap.go` (security-engineer SSH/sudoers logic) vs `internal/cli/node.go` (cli-engineer flag wiring) — boundary: security package exposes `BootstrapProxmox(ctx, opts) error`, CLI just calls it.
|
||||
- `internal/doctor/doctor.go` `Proxmox()` reuses SSH client from `internal/proxmox` (security) but check scaffolding clones `doctor.Network()` pattern (backend adjudicates since network-engineer deactivated).
|
||||
Reference in New Issue
Block a user