coreci/orca
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 91 Wiki Activity

docs(P00): research findings — v0.16 release binary asset fix

Browse Source 
Validated root cause against CoreCI source code:
- .coreci.yml uses invalid format (pipelines:/steps:/image:/commands:)
  → CoreCI silently drops unknown fields → empty Jobs map → zero jobs
- Gitea Actions clone of private coreci repo has no credentials
- CoreCI executor chain, env forwarding, secret resolution all
  validated against source (run.go, isolated_shell.go, pass_through.go,
  github.go, pipeline.go)

---ci---
project: orca
phase: 0
milestone: v0.16
status: research
---/ci---
This commit is contained in:
Jon Chery
2026-08-12 21:03:36 +00:00
parent 82dfe7a941
commit 1c2843a39d
2 changed files with 161 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.ciagent/CHECKPOINT.json
+2 -2
View File
@@ -1,11 +1,11 @@
{
"phase": 0,
"stage": "clarify",
"stage": "research",
"milestone": "v0.16",
"milestone_slug": "release-binary-fix",
"phase_role": "pre_execution",
"attempts": 0,
"updated_at": "2026-08-12T21:05:00Z",
"updated_at": "2026-08-12T21:10:00Z",
"milestone_complete": false,
"previous_milestone": "v0.15",
"phases_shipped": [],
.ciagent/RESEARCH_v0.16.md
+159
View File
@@ -0,0 +1,159 @@
# RESEARCH v0.16: Release Binary Asset Fix
## Root Cause: Two Compounding Bugs
### Bug 1: Gitea Actions workflow cannot clone private coreci repo
`.gitea/workflows/release.yml` step "Install CoreCI" runs:
```bash
git clone --depth=1 https://git.cloudinit.dev/coreci/coreci.git /tmp/coreci
```
The `coreci` repo is **private** (confirmed: `curl -s -o /dev/null -w "%{http_code}" https://git.cloudinit.dev/coreci/coreci` returns 404 without auth; the Gitea API reports `"private": true`).
The `actions/checkout@v4` step injects auth only for the orca repo:
```
[command]/usr/bin/git config --local http.https://git.cloudinit.dev/.extraheader AUTHORIZATION: basic ***
```
The subsequent bare `git clone` of the coreci repo has no credentials. Runner logs confirm the failure:
```
fatal: could not read Username for 'https://git.cloudinit.dev': No such device or address
❌ Failure - Main Install CoreCI
exitcode '128': failure
```
Both Gitea Actions runs (v0.14.1 run #3540, v0.14.2 run #3544) failed at this step. The `container-orca` and `container-traefik` jobs (with `needs: ci`) were skipped. `coreci run` never executed.
**Fix (REQ-183):** Pass `GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}` as env to the "Install CoreCI" step and embed it in the clone URL:
```bash
git clone --depth=1 https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/coreci/coreci.git /tmp/coreci
```
### Bug 2: .coreci.yml uses a format CoreCI does not understand
The orca `.coreci.yml` uses:
```yaml
pipelines:
validate:
steps:
- name: go-version
image: golang:1.25.12
commands: [go version, gofmt -l ., go vet ./...]
release:
when:
ref: "refs/tags/v*"
steps: [...]
```
CoreCI's actual format (validated against `workflows/pipeline/pipeline.go`):
```go
type Pipeline struct {
Jobs map[string]Job `yaml:"jobs"`
Services map[string]Service `yaml:"services,omitempty"`
Env EnvBlock `yaml:"env,omitempty"`
}
type Job struct {
Needs []string `yaml:"needs,omitempty"`
Plugin string `yaml:"plugin,omitempty"`
Invoke string `yaml:"invoke,omitempty"`
Vars map[string]string `yaml:"vars,omitempty"`
// ...
}
```
`yaml.Unmarshal` into a struct silently drops unknown fields (no `KnownFields(true)`). The `pipelines:` top-level key doesn't map to any struct field → `Jobs` is an empty map. CoreCI's `validate()` does NOT reject empty jobs:
```go
func validate(p *Pipeline) error {
seen := make(map[string]struct{}, len(p.Jobs))
for name, job := range p.Jobs { ... }
return nil // empty Jobs → no iterations → nil error
}
```
So `coreci run` loads an empty pipeline, runs zero jobs, and exits successfully — no build, no tarball, no asset upload.
Additional format mismatches in the current `.coreci.yml`:
- `when: ref: "refs/tags/v*"` — CoreCI has no `when` at pipeline/job level; it uses `rules: [{if: "CI_COMMIT_TAG"}]` on jobs. And CoreCI's `github.go` doesn't set `CI_COMMIT_TAG` for Gitea Actions. But the Gitea Actions workflow already gates on `on: push: tags: ['v*']`, so no conditional is needed.
- `${CI_COMMIT_TAG:-dev}` in `env:` blocks — CoreCI does not expand `${VAR}` at YAML parse time. Only `${{ secrets.KEY }}` is interpolated. Shell expansion works inside `invoke:` via `sh -c`.
- `GITEA_TOKEN: ${GITEA_TOKEN}` in `env:` blocks — CoreCI's `env` block doesn't do env-var interpolation. Secrets must go in job `vars:` via `${{ secrets.GITEA_TOKEN }}`.
**Fix (REQ-184):** Full rewrite to CoreCI native `jobs:` format with a proper DAG.
## CoreCI Execution Model (validated against source)
### Executor Chain
Default: `podman,docker,shell-isolated` (`NewChainExecutorFromConfig`).
- `ErrExecutorUnavailable` (executor missing) → fall through to next.
- `ErrExecutorFailed` (non-zero exit) → stop immediately, no fallback.
### Job Execution with `plugin` + `invoke`
A job with BOTH `plugin: docker://golang:1.25` AND `invoke: "go build ..."`:
- **Podman/Docker executor**: pulls image, runs `podman/docker run ... <image> sh -c "<invoke>"`. Env vars from the `env` map are injected via `-e K=V`.
- **Shell-isolated executor**: checks `invoke` first (line 52). If non-empty, runs `sh -c "<invoke>"` directly, ignoring the `plugin` image. This is the fallback when no container runtime is available.
- This dual pattern is correct: container if available, shell fallback if not.
### Environment Variables in Jobs
`buildIsolatedEnv` (`isolated_shell.go:34`):
```go
func buildIsolatedEnv(env map[string]string) []string {
var out []string
for _, key := range envAllowlist { // PATH, HOME, LANG, TMPDIR, TERM, CI
if val, ok := os.LookupEnv(key); ok { out = append(out, ...) }
}
for _, kv := range os.Environ() {
if strings.HasPrefix(kv, "CORECI_") || strings.HasPrefix(kv, "CI_") { out = append(out, kv) }
}
for k, v := range env { out = append(out, fmt.Sprintf("%s=%s", k, v)) } // env map
out = append(out, "CORECI_SANDBOX=isolated-shell")
return out
}
```
The `env` map (3rd loop) is built in `run.go` from:
1. `cicontext.Detect()` → CI context (`CI_COMMIT_SHA`, `CI_COMMIT_BRANCH`, etc.)
2. `envfwd.PassThroughEnv()` → user-defined host env vars NOT in deny-list
3. Job `vars` (with `${{ secrets.* }}` interpolation)
So in a job's `invoke:` script, these env vars are available:
- `CI_COMMIT_BRANCH` — tag name on tag push (from `GITHUB_REF_NAME`)
- `CI_COMMIT_SHA` — commit SHA
- `GITEA_TOKEN` — forwarded by PassThroughEnv (not in deny-list)
- `CORECI_*` prefixed vars — always forwarded
- `CI_*` prefixed vars — always forwarded
- Any job `vars` values
### Secret Resolution
`run.go:146-153`:
```go
resolver := func(key string) (string, error) {
if v, ok := secretMap[key]; ok { return v, nil } // local DB first
if v := os.Getenv(key); v != "" { return v, nil } // then env
return "", secrets.ErrMissingKey
}
```
Secret interpolation `${{ secrets.GITEA_TOKEN }}` in job `vars:` is resolved by checking the local CoreCI secret store first, then `os.Getenv`. Since `GITEA_TOKEN` is in the env (from the Gitea Actions step env), `os.Getenv("GITEA_TOKEN")` succeeds.
## Gitea Actions CI Context
On a tag push (`refs/tags/v0.15.1`), Gitea Actions sets:
- `GITHUB_ACTIONS=true`
- `GITHUB_REF=refs/tags/v0.15.1`
- `GITHUB_REF_NAME=v0.15.1`
- `GITHUB_REF_TYPE=tag`
- `GITHUB_SHA=<commit>`
CoreCI's `github.go` maps:
- `GITHUB_ACTIONS=true` → `isGitHub()` returns true → `IsRunningInCI()` true
- `GITHUB_SHA` → `CI_COMMIT_SHA`
- `GITHUB_REF_NAME` → `CI_COMMIT_BRANCH` (= `v0.15.1` on tag push)
`PassThroughEnv()` runs (CI detected). `GITHUB_REF` is in `predefinedCIVars` (NOT forwarded), but `GITEA_TOKEN` is NOT in any deny-list → forwarded.
## Conclusion
The fix is two file changes:
1. `.gitea/workflows/release.yml` — auth the coreci clone (embed token in URL)
2. `.coreci.yml` — rewrite to CoreCI native `jobs:` format with DAG
Both are validated against CoreCI source code and docs. No API calls needed — `coreci run` executes the `.coreci.yml` pipeline locally on the runner.