82dfe7a941
7 decisions (D-269..D-275) validated against CoreCI source code: - D-270: .coreci.yml uses invalid format (pipelines:/steps:/image:/ commands:) — CoreCI silently drops unknown fields → empty Jobs map - D-272: CI_COMMIT_BRANCH contains tag name on tag pushes (CoreCI github.go maps GITHUB_REF_NAME → CI_COMMIT_BRANCH) - D-273: handle duplicate release (ship workflow creates release first; coreci run attaches assets later via API fallback) - D-274: shell-friendly jobs (runner is ubuntu, not alpine — no apk) ---ci--- project: orca phase: 0 milestone: v0.16 status: clarify ---/ci---
117 lines
7.4 KiB
Markdown
117 lines
7.4 KiB
Markdown
# CLARIFY v0.16: Release Binary Asset Fix
|
|
|
|
## Decisions
|
|
|
|
| ID | Decision | Rationale | Confidence |
|
|
|----|----------|-----------|------------|
|
|
| D-269 | Auth the coreci clone via URL embedding | The `actions/checkout@v4` step injects auth only for the orca repo. The bare `git clone https://git.cloudinit.dev/coreci/coreci.git` has no credentials and fails with `fatal: could not read Username`. Embedding `https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/...` in the clone URL is the simplest fix — no git credential helper config needed. | 0.95 |
|
|
| D-270 | Rewrite .coreci.yml to CoreCI native `jobs:` format | CoreCI's `Pipeline` struct (`workflows/pipeline/pipeline.go`) only has `Jobs`/`Services`/`Env` fields. The orca `.coreci.yml` uses `pipelines:`/`steps:`/`image:`/`commands:` which are silently dropped by `yaml.Unmarshal` → empty `Jobs` map → zero jobs execute. `validate()` does not reject empty jobs. | 0.97 (validated against CoreCI source) |
|
|
| D-271 | No tag-conditional rules in .coreci.yml | The Gitea Actions workflow already gates on `on: push: tags: ['v*']`. Every `coreci run` invocation is already a release run. CoreCI's `cicontext/github.go` doesn't even set `CI_COMMIT_TAG` for Gitea Actions (it maps `GITHUB_REF_NAME` → `CI_COMMIT_BRANCH`). | 0.90 |
|
|
| D-272 | Use `CI_COMMIT_BRANCH` for tag name | On a tag push, Gitea Actions sets `GITHUB_REF_NAME=v0.15.1`. CoreCI's `github.go` maps this to `CI_COMMIT_BRANCH`. So `CI_COMMIT_BRANCH` contains the tag name on tag pushes. This is the env var to use for version injection in `invoke:` scripts. | 0.90 |
|
|
| D-273 | Handle duplicate release gracefully | The CIAgent ship workflow creates releases (title+body, no binary) via the Gitea API. The `coreci run` release job runs later (after the Gitea Actions workflow triggers). `tea releases create` fails if the release exists. Fallback: query the release ID by tag and attach assets via the Gitea API `POST /releases/{id}/assets` endpoint. | 0.92 |
|
|
| D-274 | Shell-friendly jobs (no `apk add`) | The Gitea Actions runner runs the `ci` job inside `docker.gitea.com/runner-images:ubuntu-latest` (ubuntu, not alpine). CoreCI's shell-isolated executor (the likely fallback if podman/docker aren't in the runner container) runs `sh -c <invoke>` directly. `apk add` won't work on ubuntu. Use `curl` (pre-installed) for tool downloads. | 0.85 |
|
|
| D-275 | `GITEA_TOKEN` via PassThroughEnv | CoreCI's `PassThroughEnv()` forwards env vars not in the deny-list. `GITEA_TOKEN` is not in `systemVars` or `predefinedCIVars`, so it IS forwarded when `IsRunningInCI()` is true (Gitea Actions sets `GITHUB_ACTIONS=true`). The Gitea Actions workflow sets `GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}` in the `coreci run` step env. So `GITEA_TOKEN` is available in job `invoke:` scripts. Belt-and-suspenders: also declare `vars: { GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} }` so CoreCI's secret resolver picks it up via `os.Getenv` fallback. | 0.88 |
|
|
|
|
## Research: CoreCI .coreci.yml format (validated against source)
|
|
|
|
CoreCI's `Pipeline` struct (`workflows/pipeline/pipeline.go`):
|
|
```go
|
|
type Pipeline struct {
|
|
Jobs map[string]Job `yaml:"jobs"`
|
|
Services map[string]Service `yaml:"services,omitempty"`
|
|
Env EnvBlock `yaml:"env,omitempty"`
|
|
}
|
|
type Job struct {
|
|
Needs []string `yaml:"needs,omitempty"`
|
|
Plugin string `yaml:"plugin,omitempty"`
|
|
Invoke string `yaml:"invoke,omitempty"`
|
|
Vars map[string]string `yaml:"vars,omitempty"`
|
|
MemoryLimitMb int `yaml:"memory_limit_mb,omitempty"`
|
|
TimeoutMs int `yaml:"timeout_ms,omitempty"`
|
|
Rules []Rule `yaml:"rules,omitempty"`
|
|
Tags []string `yaml:"tags,omitempty"`
|
|
}
|
|
```
|
|
|
|
Key facts (from `docs/CORECI-YML.md` + source):
|
|
- `plugin` and `invoke` are mutually exclusive on the same job — BUT the
|
|
isolated_shell executor checks `invoke` first (line 52) and only falls
|
|
back to `plugin` if `invoke` is empty. So a job with BOTH `plugin:
|
|
docker://golang:1.25` AND `invoke: "go build ..."` works: container
|
|
executors run the invoke inside the container; shell-isolated runs
|
|
the invoke directly (ignoring the image). This is the correct pattern.
|
|
- Unknown YAML fields are silently dropped (no strict decode). This is
|
|
why the current `pipelines:`/`steps:`/`image:`/`commands:` format
|
|
produces an empty `Jobs` map with no error.
|
|
- `validate()` does NOT reject empty `Jobs` maps — it only checks for
|
|
duplicate names and plugin/invoke mutual exclusivity within existing
|
|
jobs.
|
|
- Secret interpolation `${{ secrets.KEY }}` works only in job `vars:`
|
|
values. The resolver (run.go:146-153) checks the local secret store
|
|
first, then falls back to `os.Getenv(key)`.
|
|
- Shell `${VAR}` expansion works inside `invoke:` strings at runtime
|
|
(via `sh -c`), but NOT in YAML field values at parse time.
|
|
- `env.from_ci` controls which CI vars are injected; if empty, all
|
|
detected CI vars are merged.
|
|
|
|
## Research: Gitea Actions runner environment
|
|
|
|
The `ci` job (no `container:` field) runs inside
|
|
`docker.gitea.com/runner-images:ubuntu-latest`. The `Set up Go` step
|
|
installs Go 1.25. CoreCI's executor chain is `podman,docker,shell-isolated`.
|
|
If podman/docker aren't in the runner container, jobs fall back to
|
|
`shell-isolated` which runs `sh -c <invoke>` directly. Go commands work
|
|
in shell-isolated mode (Go is on PATH). Tool installation via `go install`
|
|
works (needs Go + network). `gitleaks` binary download via `curl` works.
|
|
|
|
## Research: Gitea Actions CI context (CoreCI detection)
|
|
|
|
CoreCI's `cicontext/github.go`:
|
|
```go
|
|
func isGitHub() bool { return os.Getenv("GITHUB_ACTIONS") == "true" }
|
|
func normalizeGitHub() map[string]string {
|
|
return map[string]string{
|
|
"CI": "true",
|
|
"CI_COMMIT_SHA": envOrDefault("GITHUB_SHA", ""),
|
|
"CI_COMMIT_BRANCH": envOrDefault("GITHUB_REF_NAME", ""),
|
|
// ... (no CI_COMMIT_TAG)
|
|
}
|
|
}
|
|
```
|
|
|
|
On a tag push: `GITHUB_REF_NAME=v0.15.1` → `CI_COMMIT_BRANCH=v0.15.1`.
|
|
`CI_COMMIT_TAG` is NOT set — CoreCI doesn't populate it for Gitea Actions.
|
|
|
|
## Plan
|
|
|
|
### Phase 1 (only execution phase)
|
|
|
|
**Files to modify:**
|
|
|
|
1. `.gitea/workflows/release.yml` — move `GITEA_TOKEN` env to the
|
|
"Install CoreCI" step and embed it in the clone URL:
|
|
```yaml
|
|
- name: Install CoreCI
|
|
env:
|
|
GITEA_TOKEN: ${{ secrets.PAT_TOKEN }}
|
|
run: |
|
|
git clone --depth=1 https://cloudinit-bot:${GITEA_TOKEN}@git.cloudinit.dev/coreci/coreci.git /tmp/coreci
|
|
cd /tmp/coreci
|
|
CGO_ENABLED=0 go build -tags sqlite_go,embed -o /usr/local/bin/coreci ./cmd/coreci
|
|
coreci version
|
|
```
|
|
|
|
2. `.coreci.yml` — full rewrite to CoreCI native `jobs:` format:
|
|
- DAG: `go-vet` → fan-out to `verify-reqs`, `gosec`, `govulncheck`,
|
|
`gitleaks` → `build` → `test` → `release`
|
|
- Each job: `plugin: docker://golang:1.25.12` + `invoke: |` (multi-line)
|
|
- `build` job: version injection via `CI_COMMIT_BRANCH` (tag) +
|
|
`CI_COMMIT_SHA` + `date` for build time
|
|
- `release` job: build tarball + SHA256SUMS, install `tea` via curl,
|
|
create release with assets (fallback to API asset attachment if
|
|
release exists), verify asset count ≥ 2 (REQ-097 gate C-21)
|
|
- `GITEA_TOKEN` via `vars: { GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} }`
|
|
- No tag-conditional rules (workflow already gates on tags)
|
|
|
|
3. `scripts/trigger_coreci.sh` — no changes needed (Gitea Actions is the
|
|
trigger; the hook is for branch-push CI only). |