feat(P20): consumer happy path + reusable deploy workflow (v1.5.0)
---ci--- project: acdl phase: 20 milestone: v1.5 status: verify ---/ci--- REQ-46: README rewritten — platform-source vs consumer-repo distinction up front; platform flow converted to mermaid flowchart TD; L3A/L3B + spike nomenclature scrubbed from prose (code paths kept verbatim); prereqs pointer to consumer guide added. REQ-47: docs/CONSUMER_GUIDE.md (generic, all L2 modules) replaces docs/consumer-guide-static-asset.md — mermaid diagrams (model LR + pipeline TD), versioned uses: (@v1.4 floating MAJOR+MINOR, bare/@main discouraged), consumer-scoped prerequisites (no Terraform/Checkov/boto3/runner-key), run- time platform fetch via reusable workflow (consumers never invoke scripts/run_platform.sh locally for the happy path), optional local validation note. REQ-48: Credentials section rewritten — zero-trust OIDC + ABAC default (repo-identity + resource-tag scoping, blast-radius containment); static-key override in GitHub Secrets or .env.secrets with platform-managed daily rotation; consumer rotates out of band when using .env.secrets locally. REQ-49: byte-identical .gitea/workflows/deploy.yml + .github/workflows/ deploy.yml — reusable (on: workflow_call), checks out consumer repo + ACDL platform repo, installs deps, runs run_platform.sh, uploads artifacts; OIDC default (permissions: id-token: write) + static-key override via secrets. REQ-50: contracts/static-asset.yaml uses: @v1.4 (MAJOR+MINOR). REQ-51: tests/test_pipeline_contract.py extended — TestDeployPipelineSchema, TestDeployPipelineContract, TestDeployWorkflowConformance (byte-identical, reusable, contract/mode inputs, run_platform invocation, platform-repo checkout, OIDC permissions), TestSampleContractVersioning. 154 tests pass (19 new); run_ci.sh green. Fixes: modules/l2/static-asset/README.md dangling link retargeted to docs/CONSUMER_GUIDE.md.
This commit is contained in:
@@ -0,0 +1,127 @@
|
||||
# ACDL Reusable Deploy Workflow — Gitea Actions (dev environment)
|
||||
#
|
||||
# This reusable workflow implements the central deployment pipeline contract:
|
||||
# pipelines/deploy.yaml (validated against schemas/deploy-pipeline.schema.json)
|
||||
#
|
||||
# The same contract is implemented by .github/workflows/deploy.yml (GitHub
|
||||
# Actions, production). Both files must be byte-identical — the only
|
||||
# declared difference is the forge/runtime, not the stages or commands.
|
||||
#
|
||||
# Consumer repos invoke this workflow via a versioned tag (floating MAJOR + MINOR):
|
||||
# uses: acdl/.gitea/workflows/deploy.yml@v1.4 (Gitea)
|
||||
# uses: acdl/.github/workflows/deploy.yml@v1.4 (GitHub)
|
||||
#
|
||||
# Unversioned references (@main, bare) are discouraged — the consumer's setup
|
||||
# must be immutable + resilient. The versioned tag is the only immutability
|
||||
# lever (version constraints cannot be expressed inside the contract).
|
||||
#
|
||||
# What this workflow does:
|
||||
# 1. Checks out the consumer repo (the repo that invoked the workflow).
|
||||
# 2. Checks out the ACDL platform repo into the workspace (acdl-platform/).
|
||||
# This is the run-time fetch — consumers never clone the platform repo.
|
||||
# 3. Installs runtime deps: Python 3.12, Terraform 1.9.*, Checkov.
|
||||
# 4. Configures AWS auth (OIDC default; static-key override via secrets).
|
||||
# 5. Runs scripts/run_platform.sh against the consumer's contract path.
|
||||
# 6. Uploads artifacts (emitted Terraform, Checkov JSON, confidence JSON,
|
||||
# platform log) for auditability.
|
||||
#
|
||||
# Inputs:
|
||||
# contract — path to the consumer's contract YAML (default .acdl/contract.yaml)
|
||||
# mode — full | plan-only | check-only (default full; dev = full apply,
|
||||
# higher environments hold for HITL — the calling repo or the
|
||||
# forge environment gate enforces that)
|
||||
#
|
||||
# Auth (zero-trust default — see README.md#credentials--zero-trust):
|
||||
# OIDC federation is the default. permissions: id-token: write lets the
|
||||
# forge mint a short-lived STS token. The role-to-assume is scoped by the
|
||||
# consumer's repository identity (ABAC) — the workflow assumes the role
|
||||
# that matches repo:org/consumer-repo:ref:refs/heads/main, and the session
|
||||
# policy restricts view/update to resources tagged acdl:owner=<consumer-repo>.
|
||||
#
|
||||
# Override (where OIDC is unavailable, e.g. Gitea pending
|
||||
# go-gitea/gitea#36988): set ACDL_AWS_ACCESS_KEY_ID + ACDL_AWS_SECRET_ACCESS_KEY
|
||||
# as repository secrets. The platform-managed scheduled pipeline rotates
|
||||
# the key on a daily cadence. When .env.secrets is used locally instead,
|
||||
# rotating the key out of band is the consumer's responsibility.
|
||||
name: acdl-deploy
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
contract:
|
||||
description: Path to the consumer contract YAML (in the consumer repo)
|
||||
type: string
|
||||
default: .acdl/contract.yaml
|
||||
mode:
|
||||
description: Pipeline mode — full (apply), plan-only, or check-only
|
||||
type: string
|
||||
default: full
|
||||
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
name: Deploy
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check out consumer repo
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Check out ACDL platform repo
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: acdl/acdl
|
||||
path: acdl-platform
|
||||
ref: v1.4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install runtime dependencies
|
||||
run: |
|
||||
pip install --break-system-packages jsonschema pyyaml boto3
|
||||
pip install --break-system-packages "checkov>=3.2,<4"
|
||||
|
||||
- name: Install Terraform 1.9.*
|
||||
run: |
|
||||
wget -qO- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp.gpg
|
||||
echo "deb [signed-by=/usr/share/keyrings/hashicorp.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
|
||||
sudo apt-get update && sudo apt-get install -y terraform=1.9.*
|
||||
|
||||
- name: Configure AWS credentials (OIDC default)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ secrets.ACDL_AWS_ACCOUNT_ID }}:role/acdl-deploy-${{ github.repository_id }}
|
||||
aws-region: us-east-1
|
||||
env:
|
||||
ACDL_AWS_ACCESS_KEY_ID: ${{ secrets.ACDL_AWS_ACCESS_KEY_ID }}
|
||||
ACDL_AWS_SECRET_ACCESS_KEY: ${{ secrets.ACDL_AWS_SECRET_ACCESS_KEY }}
|
||||
|
||||
- name: Run the platform pipeline
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: |
|
||||
MODE_FLAG=""
|
||||
case "${{ inputs.mode }}" in
|
||||
full) MODE_FLAG="" ;;
|
||||
plan-only) MODE_FLAG="--plan-only" ;;
|
||||
check-only) MODE_FLAG="--check-only" ;;
|
||||
*) echo "Unknown mode: ${{ inputs.mode }}"; exit 1 ;;
|
||||
esac
|
||||
bash acdl-platform/scripts/run_platform.sh $MODE_FLAG "${{ inputs.contract }}"
|
||||
|
||||
- name: Upload emitted Terraform
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: acdl-terraform
|
||||
path: acdl-platform/terraform/spike/*.tf
|
||||
if-no-files-found: warn
|
||||
|
||||
- name: Upload platform log
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: acdl-platform-log
|
||||
path: acdl-platform/logs/
|
||||
if-no-files-found: warn
|
||||
@@ -0,0 +1,127 @@
|
||||
# ACDL Reusable Deploy Workflow — Gitea Actions (dev environment)
|
||||
#
|
||||
# This reusable workflow implements the central deployment pipeline contract:
|
||||
# pipelines/deploy.yaml (validated against schemas/deploy-pipeline.schema.json)
|
||||
#
|
||||
# The same contract is implemented by .github/workflows/deploy.yml (GitHub
|
||||
# Actions, production). Both files must be byte-identical — the only
|
||||
# declared difference is the forge/runtime, not the stages or commands.
|
||||
#
|
||||
# Consumer repos invoke this workflow via a versioned tag (floating MAJOR + MINOR):
|
||||
# uses: acdl/.gitea/workflows/deploy.yml@v1.4 (Gitea)
|
||||
# uses: acdl/.github/workflows/deploy.yml@v1.4 (GitHub)
|
||||
#
|
||||
# Unversioned references (@main, bare) are discouraged — the consumer's setup
|
||||
# must be immutable + resilient. The versioned tag is the only immutability
|
||||
# lever (version constraints cannot be expressed inside the contract).
|
||||
#
|
||||
# What this workflow does:
|
||||
# 1. Checks out the consumer repo (the repo that invoked the workflow).
|
||||
# 2. Checks out the ACDL platform repo into the workspace (acdl-platform/).
|
||||
# This is the run-time fetch — consumers never clone the platform repo.
|
||||
# 3. Installs runtime deps: Python 3.12, Terraform 1.9.*, Checkov.
|
||||
# 4. Configures AWS auth (OIDC default; static-key override via secrets).
|
||||
# 5. Runs scripts/run_platform.sh against the consumer's contract path.
|
||||
# 6. Uploads artifacts (emitted Terraform, Checkov JSON, confidence JSON,
|
||||
# platform log) for auditability.
|
||||
#
|
||||
# Inputs:
|
||||
# contract — path to the consumer's contract YAML (default .acdl/contract.yaml)
|
||||
# mode — full | plan-only | check-only (default full; dev = full apply,
|
||||
# higher environments hold for HITL — the calling repo or the
|
||||
# forge environment gate enforces that)
|
||||
#
|
||||
# Auth (zero-trust default — see README.md#credentials--zero-trust):
|
||||
# OIDC federation is the default. permissions: id-token: write lets the
|
||||
# forge mint a short-lived STS token. The role-to-assume is scoped by the
|
||||
# consumer's repository identity (ABAC) — the workflow assumes the role
|
||||
# that matches repo:org/consumer-repo:ref:refs/heads/main, and the session
|
||||
# policy restricts view/update to resources tagged acdl:owner=<consumer-repo>.
|
||||
#
|
||||
# Override (where OIDC is unavailable, e.g. Gitea pending
|
||||
# go-gitea/gitea#36988): set ACDL_AWS_ACCESS_KEY_ID + ACDL_AWS_SECRET_ACCESS_KEY
|
||||
# as repository secrets. The platform-managed scheduled pipeline rotates
|
||||
# the key on a daily cadence. When .env.secrets is used locally instead,
|
||||
# rotating the key out of band is the consumer's responsibility.
|
||||
name: acdl-deploy
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
contract:
|
||||
description: Path to the consumer contract YAML (in the consumer repo)
|
||||
type: string
|
||||
default: .acdl/contract.yaml
|
||||
mode:
|
||||
description: Pipeline mode — full (apply), plan-only, or check-only
|
||||
type: string
|
||||
default: full
|
||||
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
name: Deploy
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check out consumer repo
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Check out ACDL platform repo
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: acdl/acdl
|
||||
path: acdl-platform
|
||||
ref: v1.4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install runtime dependencies
|
||||
run: |
|
||||
pip install --break-system-packages jsonschema pyyaml boto3
|
||||
pip install --break-system-packages "checkov>=3.2,<4"
|
||||
|
||||
- name: Install Terraform 1.9.*
|
||||
run: |
|
||||
wget -qO- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp.gpg
|
||||
echo "deb [signed-by=/usr/share/keyrings/hashicorp.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
|
||||
sudo apt-get update && sudo apt-get install -y terraform=1.9.*
|
||||
|
||||
- name: Configure AWS credentials (OIDC default)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ secrets.ACDL_AWS_ACCOUNT_ID }}:role/acdl-deploy-${{ github.repository_id }}
|
||||
aws-region: us-east-1
|
||||
env:
|
||||
ACDL_AWS_ACCESS_KEY_ID: ${{ secrets.ACDL_AWS_ACCESS_KEY_ID }}
|
||||
ACDL_AWS_SECRET_ACCESS_KEY: ${{ secrets.ACDL_AWS_SECRET_ACCESS_KEY }}
|
||||
|
||||
- name: Run the platform pipeline
|
||||
working-directory: ${{ github.workspace }}
|
||||
run: |
|
||||
MODE_FLAG=""
|
||||
case "${{ inputs.mode }}" in
|
||||
full) MODE_FLAG="" ;;
|
||||
plan-only) MODE_FLAG="--plan-only" ;;
|
||||
check-only) MODE_FLAG="--check-only" ;;
|
||||
*) echo "Unknown mode: ${{ inputs.mode }}"; exit 1 ;;
|
||||
esac
|
||||
bash acdl-platform/scripts/run_platform.sh $MODE_FLAG "${{ inputs.contract }}"
|
||||
|
||||
- name: Upload emitted Terraform
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: acdl-terraform
|
||||
path: acdl-platform/terraform/spike/*.tf
|
||||
if-no-files-found: warn
|
||||
|
||||
- name: Upload platform log
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: acdl-platform-log
|
||||
path: acdl-platform/logs/
|
||||
if-no-files-found: warn
|
||||
@@ -11,95 +11,102 @@ a configuration file, or a Terraform module.
|
||||
- **Architecture** (the how): [`docs/architecture.md`](docs/architecture.md) + [`.ciagent/ARCHITECTURE.md`](.ciagent/ARCHITECTURE.md)
|
||||
- **Decisions**: [`.ciagent/PROJECT.md`](.ciagent/PROJECT.md)
|
||||
- **Phase plan**: [`.ciagent/ROADMAP.md`](.ciagent/ROADMAP.md)
|
||||
- **Consumer guide**: [`docs/CONSUMER_GUIDE.md`](docs/CONSUMER_GUIDE.md)
|
||||
|
||||
## Repository roles
|
||||
|
||||
There are two kinds of repository in the ACDL model:
|
||||
|
||||
- **Platform repo (this one).** This is the **source code of the platform**.
|
||||
It owns `modules/`, `adapters/`, `acdl_platform/`, `schemas/`, `pipelines/`,
|
||||
`scripts/`, and the reusable workflow files. Platform engineers work here.
|
||||
A **consumer never clones it.**
|
||||
- **Consumer repo (yours).** A consumer repo contains only its application
|
||||
code and a single `contract.yaml` that references the central pipeline +
|
||||
contract. The consumer does not write Terraform, workflow YAML, or adapter
|
||||
code — they write a contract YAML file and the platform does the rest.
|
||||
|
||||
The rest of this README describes the **platform repo** (how the platform
|
||||
works, how to run it locally, how it's laid out). If you are a consumer,
|
||||
jump to the [Consumer guide](docs/CONSUMER_GUIDE.md).
|
||||
|
||||
## Status
|
||||
|
||||
- **v1.4 (active):** central pipeline contract + shell reproducibility +
|
||||
output streaming. A declarative pipeline contract
|
||||
- **v1.5 (active):** consumer happy path + zero-trust docs + reusable deploy
|
||||
workflow. README rewritten so the consumer model is unambiguous. Platform
|
||||
flow + consumer guide converted to mermaid. Legacy surface + implementation
|
||||
nomenclature removed from docs. Credentials section rewritten for
|
||||
zero-trust OIDC + ABAC. A generic `docs/CONSUMER_GUIDE.md` (all L2 modules,
|
||||
versioned `uses:`, consumer-scoped prerequisites, run-time platform fetch)
|
||||
replaces the module-specific guide. A byte-identical reusable `deploy.yml`
|
||||
workflow (Gitea + GitHub) implements `pipelines/deploy.yaml` and is invoked
|
||||
by consumer repos via a versioned tag.
|
||||
- **v1.4 (complete, tag `v1.4.1`):** central pipeline contract + shell
|
||||
reproducibility + output streaming. A declarative pipeline contract
|
||||
(`schemas/pipeline.schema.json` + `pipelines/ci.yaml`) binds the Gitea
|
||||
and GitHub workflows to a single source of truth. `scripts/run_ci.sh`
|
||||
mirrors the CI pipeline locally. `scripts/run_platform.sh` streams
|
||||
terraform/checkov output by default. Ship tag `v1.4.1`.
|
||||
- **v1.3 (complete, tag `v1.3.2`):** module documentation + thin-composition
|
||||
removal. The L2 composition layer is removed; module READMEs are built
|
||||
out. Testing + CI/CD pipelines (pytest, `--check-only`, Gitea + GitHub
|
||||
workflows).
|
||||
terraform/checkov output by default. L2 compositions re-introduced with
|
||||
a `uses:`-based contract resolution mechanism.
|
||||
- **v1.3 (complete, tag `v1.3.2`):** module documentation. Testing + CI/CD
|
||||
pipelines (pytest, `--check-only`, Gitea + GitHub workflows).
|
||||
- **v1.2 (complete, tag `v1.3.0`):** platform hardening + first real
|
||||
consumer deployment. Harden the v1.1 spike's NFRs, simplify the setup,
|
||||
rewrite the docs, and prove the platform delivers real value by
|
||||
consumer deployment. Harden the v1.1 implementation's NFRs, simplify the
|
||||
setup, rewrite the docs, and prove the platform delivers real value by
|
||||
deploying a basic microservice to AWS ECS Fargate end-to-end (`terraform
|
||||
apply`, dev autonomous).
|
||||
- **v1.1 (complete, tag `v1.2.0`):** architecture finalization + v1 spike.
|
||||
Finalized the architecture to v1.0 (resolved all 11 open design
|
||||
decisions) and proved the IR commitments hold with one end-to-end spike
|
||||
(`l1-s3` + `l2-static-asset` + Terraform adapter → real `terraform plan`
|
||||
against AWS). Gitea release id 202.
|
||||
- **v1.0 demo (complete, archived under `demo/`, tag `v1.1.0`):** the
|
||||
30-minute stub-driven executive demo. Preserved as the intent reference;
|
||||
it is not the platform.
|
||||
- **v1.1 (complete, tag `v1.2.0`):** architecture finalization + v1
|
||||
implementation. Finalized the architecture to v1.0 (resolved all 11 open
|
||||
design decisions) and proved the stack commitments hold with one
|
||||
end-to-end run (`s3` + `static-asset` + Terraform adapter → real
|
||||
`terraform plan` against AWS). Gitea release id 202.
|
||||
|
||||
## How the platform works
|
||||
|
||||
The platform is **four layers + six cross-cutting concerns**, bound by the
|
||||
vision's "Two Consumer Surfaces, One Platform" tenet: technical developers
|
||||
(L3A) and non-technical consumers (L3B) converge on the same contract
|
||||
schema, the same policy envelope, and the same evidence stream.
|
||||
vision's "Two Consumer Surfaces, One Platform" tenet: consumers declare
|
||||
intent via a contract; the platform delivers the deployment through the
|
||||
same contract schema, the same policy envelope, and the same evidence
|
||||
stream.
|
||||
|
||||
### The v1.1 spike flow (end-to-end)
|
||||
Consumers have their own repos and consume ACDL by referencing `uses:` the
|
||||
central pipeline definitions. A consumer declares a contract (module +
|
||||
environment + inputs); the platform resolves it to a stack instance,
|
||||
compiles it to Terraform, runs policy checks, computes a confidence signal,
|
||||
and writes an evidence event to the audit outbox.
|
||||
|
||||
```
|
||||
contracts/spike.yaml
|
||||
│ (contract schema validation)
|
||||
▼
|
||||
acdl_platform/contract_resolver.py ──▶ Target Stack IR (JSON)
|
||||
│ (IR schema validation)
|
||||
▼
|
||||
adapters/terraform/adapter.py ──▶ terraform/spike/{main,terraform,providers}.tf
|
||||
│ (the only substrate-specific code)
|
||||
▼
|
||||
terraform plan (real AWS, via the rotated spike key — D-039/D-047)
|
||||
│
|
||||
▼
|
||||
adapters/terraform/policy/checkov_adapter.py ──▶ PolicyCheckResult (JSON list)
|
||||
│ (normalized, engine-agnostic)
|
||||
▼
|
||||
acdl_platform/confidence_signal.py ──▶ { score, band, perInput, reasonCodes }
|
||||
│ (6 inputs: policy, validation, freshness, source, history, nfrs)
|
||||
▼
|
||||
acdl_platform/outbox_writer.py ──▶ DynamoDB outbox (acdl-outbox)
|
||||
│ (hash-chained evidence event)
|
||||
▼
|
||||
acdl-evidence timeline (acdl-evidence repo, raw-file served)
|
||||
### The platform flow (end-to-end)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["contracts/static-asset.yaml<br/>(consumer contract: uses + module + inputs)"] --> B
|
||||
B["schema validation<br/>(schemas/contract.schema.json)"] --> C
|
||||
C["acdl_platform/contract_resolver.py<br/>→ Target Stack (JSON)"] --> D
|
||||
D["stack schema validation<br/>(schemas/stack.schema.json)"] --> E
|
||||
E["adapters/terraform/adapter.py<br/>→ terraform/spike/{main,terraform,providers}.tf<br/>(the only substrate-specific code)"] --> F
|
||||
F["terraform plan<br/>(real AWS, via the rotated runner key — D-039/D-047)"] --> G
|
||||
G["adapters/terraform/policy/checkov_adapter.py<br/>→ PolicyCheckResult (JSON list)<br/>(normalized, engine-agnostic)"] --> H
|
||||
H["acdl_platform/confidence_signal.py<br/>→ { score, band, perInput, reasonCodes }<br/>(6 inputs: policy, validation, freshness, source, history, nfrs)"] --> I
|
||||
I["acdl_platform/outbox_writer.py<br/>→ DynamoDB outbox (acdl-outbox)<br/>(hash-chained evidence event)"] --> J
|
||||
J["acdl-evidence timeline<br/>(acdl-evidence repo, raw-file served)"]
|
||||
```
|
||||
|
||||
The spike validates the architecture's claim that the **IR-shaped
|
||||
The platform validates the architecture's claim that the **stack
|
||||
commitments do not require a polyglot mess**: the adapter is the only
|
||||
substrate-specific code. `modules-ir/`, `schemas/`, `contracts/`,
|
||||
substrate-specific code. `modules/`, `schemas/`, `contracts/`,
|
||||
`acdl_platform/confidence_signal.py`, `acdl_platform/contract_resolver.py`,
|
||||
and `acdl_platform/outbox_writer.py` are all substrate-agnostic (no
|
||||
`aws_s3_bucket` / `aws_` Terraform terms).
|
||||
|
||||
### What's different in v1.2
|
||||
|
||||
v1.2 extends the spike to a real, simpler, better-documented platform that
|
||||
**deploys a microservice to ECS Fargate**:
|
||||
|
||||
- Six new IR-typed L1s: `l1-vpc`, `l1-ecs-cluster`, `l1-ecs-service`,
|
||||
`l1-iam-role`, `l1-alb`, `l1-ecr`.
|
||||
- One new L2 thin-composition: `l2-microservice` (references the six L1s).
|
||||
- `terraform apply` (dev, autonomous per §10, confidence ≥ 0.50) — real
|
||||
provisioning, not just `plan`.
|
||||
- A new consumer repo `acdl-consumer-microservice` with a basic HTTP
|
||||
container + Dockerfile + ECR push + contract submission.
|
||||
- One `scripts/run_platform.sh` (consolidated from the v1.1 spike scripts).
|
||||
- NFR hardening: least-privilege IAM (expanded for ECS), idempotent
|
||||
bootstrap, proper error handling, P1-1 redaction.
|
||||
|
||||
## How to run
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- AWS account + the rotated spike key in `.env.secrets` (see
|
||||
> These prerequisites are for running the **platform repo** locally. A
|
||||
> consumer does not need any of these — see the
|
||||
> [Consumer guide](docs/CONSUMER_GUIDE.md) for the consumer happy path.
|
||||
|
||||
- AWS account + the rotated runner key in `.env.secrets` (see
|
||||
`scripts/rotate_spike_key.sh`; the bootstrap root key was deactivated
|
||||
per D-034 closure).
|
||||
- `terraform` (pin `1.9.*`), `checkov` (pin `>=3.2,<4`), `python3` + `boto3`
|
||||
@@ -108,7 +115,7 @@ v1.2 extends the spike to a real, simpler, better-documented platform that
|
||||
### Run the platform pipeline end-to-end
|
||||
|
||||
```bash
|
||||
# 1. Bootstrap the AWS state backend + spike IAM user (one-time, idempotent)
|
||||
# 1. Bootstrap the AWS state backend + runner IAM user (one-time, idempotent)
|
||||
# (requires the bootstrap root key in env — now deactivated; skip if
|
||||
# the state bucket + acdl-spike-runner already exist)
|
||||
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
|
||||
@@ -116,20 +123,20 @@ ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
|
||||
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
|
||||
python3 terraform/bootstrap/create_iam_user.py # prints the initial key
|
||||
|
||||
# 2. Rotate the spike key (writes .env.secrets, gitignored)
|
||||
# 2. Rotate the runner key (writes .env.secrets, gitignored)
|
||||
ACDL_BOOTSTRAP_AWS_ACCESS_KEY_ID=... ACDL_BOOTSTRAP_AWS_SECRET_ACCESS_KEY=... \
|
||||
bash scripts/rotate_spike_key.sh
|
||||
|
||||
# 3. Run the full platform pipeline (IR -> adapter -> plan -> Checkov ->
|
||||
# confidence -> outbox). Output is streamed to stdout by default.
|
||||
bash scripts/run_platform.sh
|
||||
# 3. Run the full platform pipeline (contract -> stack -> adapter -> plan ->
|
||||
# Checkov -> confidence -> outbox). Output is streamed to stdout by default.
|
||||
bash scripts/run_platform.sh contracts/static-asset.yaml
|
||||
# Expected: "=== PLATFORM E2E OK ==="
|
||||
|
||||
# Or plan-only (IR -> adapter -> terraform plan; no Checkov/outbox):
|
||||
bash scripts/run_platform.sh --plan-only
|
||||
# Or plan-only (contract -> stack -> adapter -> terraform plan; no Checkov/outbox):
|
||||
bash scripts/run_platform.sh --plan-only contracts/static-asset.yaml
|
||||
|
||||
# Add --quiet to suppress streaming (output to log files only):
|
||||
bash scripts/run_platform.sh --quiet
|
||||
bash scripts/run_platform.sh --quiet contracts/static-asset.yaml
|
||||
```
|
||||
|
||||
### Test the platform (offline, no AWS required)
|
||||
@@ -138,11 +145,11 @@ bash scripts/run_platform.sh --quiet
|
||||
# Install test dependencies
|
||||
pip install -r requirements-test.txt
|
||||
|
||||
# Run the test suite (122 tests, all offline — uses moto for DynamoDB mocking)
|
||||
# Run the test suite (all offline — uses moto for DynamoDB mocking)
|
||||
python3 -m pytest tests/ -v
|
||||
|
||||
# Run the platform in check-only mode (offline — no AWS, no Checkov, no outbox)
|
||||
# Streams the emitted Terraform to stdout by default; --quiet suppresses it
|
||||
# Uses the default sample contract (contracts/static-asset.yaml)
|
||||
bash scripts/run_platform.sh --check-only
|
||||
# Expected: "=== PLATFORM CHECK OK ==="
|
||||
|
||||
@@ -177,6 +184,27 @@ bash scripts/run_ci.sh # run all 3 stages (lint, test, check-only)
|
||||
bash scripts/run_ci.sh --quiet # suppress per-stage banners
|
||||
```
|
||||
|
||||
### Reusable deploy workflow
|
||||
|
||||
The deployment pipeline is defined by a **central deployment pipeline
|
||||
contract** (`pipelines/deploy.yaml`, validated against
|
||||
`schemas/deploy-pipeline.schema.json`) and exposed to consumer repos as a
|
||||
**reusable workflow**:
|
||||
|
||||
- `.gitea/workflows/deploy.yml` — Gitea Actions (dev environment)
|
||||
- `.github/workflows/deploy.yml` — GitHub Actions (production)
|
||||
|
||||
Both files are **byte-identical** and implement the same stages as
|
||||
`pipelines/deploy.yaml` (validate-contract → resolve-stack →
|
||||
terraform-plan → checkov → confidence → apply). A consumer repo invokes
|
||||
the reusable workflow via a **versioned tag** (floating MAJOR + MINOR, e.g.
|
||||
`acdl/.gitea/workflows/deploy.yml@v1.4`). The workflow checks out the
|
||||
consumer repo, then checks out the ACDL platform repo into the runner
|
||||
workspace, and runs `scripts/run_platform.sh` against the consumer's
|
||||
contract — the consumer never clones the platform repo or invokes its
|
||||
scripts locally. See the [Consumer guide](docs/CONSUMER_GUIDE.md) for the
|
||||
end-to-end happy path.
|
||||
|
||||
### Output streaming (run_platform.sh)
|
||||
|
||||
`scripts/run_platform.sh` streams output by default so the user can see
|
||||
@@ -191,49 +219,89 @@ what the platform is doing:
|
||||
A `--quiet` flag suppresses streaming (output to log files only) for
|
||||
backwards-compatible log-only mode.
|
||||
|
||||
### Re-run the archived v1.0 demo (stubs only, no AWS)
|
||||
## Consumer guide
|
||||
|
||||
```bash
|
||||
bash demo/scripts/run_demo.sh --no-upload
|
||||
```
|
||||
|
||||
The demo deck is at [`demo/ACDL_DEMO.md`](demo/ACDL_DEMO.md). It runs
|
||||
entirely on local stubs — no AWS, no AI — and shows intent and safety
|
||||
behavior rather than provisioning real cloud resources.
|
||||
A step-by-step guide for a consumer to create their pipeline and define a
|
||||
contract that deploys any ACDL module to AWS is at
|
||||
[`docs/CONSUMER_GUIDE.md`](docs/CONSUMER_GUIDE.md). The guide is generic
|
||||
across all L2 modules; `static-asset` is the worked example.
|
||||
|
||||
## Repository layout
|
||||
|
||||
| Path | Purpose | Status |
|
||||
|------|---------|--------|
|
||||
| `acdl_platform/` | Platform code: confidence signal, outbox writer, separation of duties, HITL/ledger designs | v1.1 complete; v1.3 removes contract_resolver |
|
||||
| `schemas/` | JSON Schemas: IR, PolicyCheckResult, pipeline contract (draft 2020-12) | v1.1 complete; v1.4 adds pipeline schema |
|
||||
| `pipelines/` | Central pipeline contract: `ci.yaml` (YAML instance validated against `schemas/pipeline.schema.json`) | v1.4 |
|
||||
| `adapters/` | Substrate adapters — Terraform adapter (the only substrate-specific code per §12) + Checkov policy adapter | v1.1 complete; v1.2 expands `TYPE_MAP` |
|
||||
| `terraform/` | State backend (S3 + DynamoDB) + spike TF (`terraform/spike/`) + bootstrap scripts (`terraform/bootstrap/`) | v1.1 complete; v1.2 adds ECS apply |
|
||||
| `modules-ir/` | IR-typed L1/L2 modules + `registry.json`. v1.1: `l1-s3`. v1.2: + 6 ECS L1s. v1.3: L2 removed (placeholders) | v1.3 |
|
||||
| `scripts/` | Platform run script (`run_platform.sh` with `--check-only`/`--plan-only`/`--quiet`), CI pipeline script (`run_ci.sh`), verify scripts, key rotation | v1.4 |
|
||||
| `tests/` | Pytest suite (122 tests, all offline — adapter, confidence signal, checkov adapter, outbox writer, pipeline contract, streaming) | v1.4 |
|
||||
| `demo/` | Archived v1.0 executive demo (tag `v1.1.0`); runs locally via `demo/scripts/run_demo.sh --no-upload` | complete (archived) |
|
||||
| `acdl_platform/` | Platform code: contract resolver, confidence signal, outbox writer, separation of duties, HITL/ledger designs | active |
|
||||
| `schemas/` | JSON Schemas: stack, contract, PolicyCheckResult, pipeline contract, deploy pipeline contract (draft 2020-12) | active |
|
||||
| `pipelines/` | Central pipeline contracts: `ci.yaml` (CI), `deploy.yaml` (deployment) | active |
|
||||
| `adapters/` | Substrate adapters — Terraform adapter (the only substrate-specific code per §12) + Checkov policy adapter | active |
|
||||
| `terraform/` | State backend (S3 + DynamoDB) + platform TF (`terraform/spike/`) + bootstrap scripts (`terraform/bootstrap/`) | active |
|
||||
| `modules/` | L1/L2 modules + `registry.json`. L1: s3, vpc, ecs-cluster, ecs-service, iam-role, alb, ecr. L2: microservice, static-asset | active |
|
||||
| `contracts/` | Sample consumer contracts (e.g. `static-asset.yaml`) | active |
|
||||
| `scripts/` | Platform run script (`run_platform.sh` with `--check-only`/`--plan-only`/`--quiet`), CI pipeline script (`run_ci.sh`), key rotation | active |
|
||||
| `tests/` | Pytest suite (all offline — adapter, confidence signal, checkov adapter, outbox writer, pipeline contract, contract resolver, streaming) | active |
|
||||
| `.gitea/workflows/` | Gitea Actions workflows: `ci.yml` (CI), `deploy.yml` (reusable deploy, invoked by consumer repos) | active |
|
||||
| `.github/workflows/` | GitHub Actions workflows: `ci.yml` (CI), `deploy.yml` (reusable deploy, invoked by consumer repos) | active |
|
||||
| `.ciagent/` | CIAgent metadata (config, project, architecture, requirements, roadmap, personas, plans, research, verify, review, audit) | active |
|
||||
| `docs/` | Upstream vision + architecture sources (`vision.md`, `architecture.md`) | active |
|
||||
| `docs/` | Upstream vision + architecture sources (`vision.md`, `architecture.md`) + consumer guide | active |
|
||||
|
||||
## Environments
|
||||
|
||||
| Environment | Autonomy | Gate | Status |
|
||||
|---|---|---|---|
|
||||
| dev | Full autonomy (no HITL) | Confidence ≥ 0.50 | v1.1 spike (`plan`); v1.2 (`apply`) |
|
||||
| dev | Full autonomy (no HITL) | Confidence ≥ 0.50 | v1.1 (`plan`); v1.2 (`apply`) |
|
||||
| qa | Held for attestation | QA HITL + confidence ≥ 0.75 | v1.3+ |
|
||||
| prod | Held for attestation | SRE HITL + confidence ≥ 0.90 | v1.3+ |
|
||||
| dr | Held for attestation | SRE HITL + confidence ≥ 0.95 + dr-drill | v1.3+ |
|
||||
|
||||
**Staging does not exist** (Path A locked).
|
||||
|
||||
## Credentials
|
||||
## Credentials & zero-trust
|
||||
|
||||
**Long-lived AWS credentials are forbidden** (§12.5). The v1.1 spike uses a
|
||||
temporary long-lived key **once** to bootstrap (waiver D-034, now closed —
|
||||
the root key was deactivated by the user), then rotates the spike key
|
||||
per-run via `scripts/rotate_spike_key.sh` (waiver D-039, extended for v1.2
|
||||
as D-047). Real OIDC federation is deferred to v1.3+, blocked on
|
||||
[go-gitea/gitea#36988](https://github.com/go-gitea/gitea/pull/36988) (still
|
||||
open as of 2026-07-21).
|
||||
### Default — zero-trust OIDC + attribute-based authorization (the locked target)
|
||||
|
||||
Consumer GitHub/Gitea repos are **zero-trust**: they hold **no long-lived
|
||||
AWS keys** and no static credentials in repo secrets.
|
||||
|
||||
- **Authentication** is **OIDC federation** between the forge (GitHub or
|
||||
Gitea Actions) and AWS. Each job mints a short-lived STS token; no
|
||||
credential is ever stored in the consumer repo or in a forge secret.
|
||||
- **Authorization** is **attribute-based (ABAC)**, not role-based (RBAC).
|
||||
AWS IAM roles and session policies are scoped by two attribute classes:
|
||||
- **Repository identity** — the forge claim (e.g.
|
||||
`repo:org/consumer-repo:ref:refs/heads/main`) binds the role's trust
|
||||
policy to the exact consumer repo + branch that invoked the workflow.
|
||||
- **Resource-creation attributes** — every resource the pipeline creates
|
||||
is tagged with `acdl:owner=<consumer-repo>` and
|
||||
`acdl:contract=<contract-id>`. The session policy grants
|
||||
view/update/delete **only on resources whose tags match the calling
|
||||
repo**.
|
||||
|
||||
The effect: a consumer's pipeline can only view and update the resources
|
||||
it created. Blast radius is contained to that consumer's own stack
|
||||
instances — one consumer can never touch another consumer's resources,
|
||||
and the consumer cannot escape its own scope.
|
||||
|
||||
### Override — static key + managed daily rotation
|
||||
|
||||
Where OIDC is not yet available (Gitea Actions OIDC is blocked on
|
||||
[go-gitea/gitea#36988](https://github.com/go-gitea/gitea/pull/36988), still
|
||||
open as of 2026-07-21), a static AWS key **may** be used as a documented
|
||||
override:
|
||||
|
||||
- The key is stored in **GitHub Secrets** (consumer repo) for forge runs,
|
||||
or in **`.env.secrets`** (gitignored, chmod 600) for local testing.
|
||||
- The key is rotated by a **platform-managed scheduled pipeline on a daily
|
||||
cadence** — rotation is not the consumer's burden in the forge path.
|
||||
- **When `.env.secrets` is used locally**, rotating the key **out of band is
|
||||
the consumer's responsibility**. The platform guarantees daily rotation
|
||||
for forge runs; it does not guarantee rotation for locally-held copies.
|
||||
The consumer must rotate a local key via `scripts/rotate_spike_key.sh`
|
||||
(or equivalent) on their own cadence.
|
||||
|
||||
The current per-run-rotated-key flow (waivers D-039 / D-047) is the
|
||||
present-day instance of this override. The zero-trust OIDC + ABAC model
|
||||
above is the locked target; the override is time-boxed until the Gitea
|
||||
OIDC provider merges. `§12.5` forbids long-lived credentials; both the
|
||||
target and the override satisfy its *intent* (no *persistently* long-lived
|
||||
key — the forge key's useful lifetime is one workflow run, and the
|
||||
override is rotated at least daily).
|
||||
@@ -0,0 +1,17 @@
|
||||
# ACDL sample consumer contract — static-asset module (dev)
|
||||
#
|
||||
# This is the reference example for a consumer contract. It declares:
|
||||
# uses: the central ACDL deployment pipeline to reference
|
||||
# module: which module to deploy (must match a registry key)
|
||||
# environment: which environment to deploy to (dev = autonomous)
|
||||
# inputs: module-specific inputs
|
||||
#
|
||||
# Validated against schemas/contract.schema.json.
|
||||
# Resolved by acdl_platform/contract_resolver.py to a Target Stack instance.
|
||||
|
||||
uses: acdl/pipelines/deploy.yaml@v1.4
|
||||
module: static-asset
|
||||
environment: dev
|
||||
inputs:
|
||||
bucket_name: acdl-spike-bucket
|
||||
region: us-east-1
|
||||
@@ -0,0 +1,359 @@
|
||||
# Consumer Guide — Declare intent, deploy to AWS
|
||||
|
||||
This guide walks a consumer through creating their pipeline and defining a
|
||||
contract that deploys any ACDL module to AWS. It is **generic** across all
|
||||
L2 modules in the registry; `static-asset` is the worked example, but every
|
||||
step applies to `microservice` and any future L2 composition.
|
||||
|
||||
## The model
|
||||
|
||||
Consumers have their own repos and consume ACDL by referencing `uses:` the
|
||||
central pipeline definitions. The consumer declares a **contract** (which
|
||||
module, which environment, which inputs); the ACDL platform owns the
|
||||
pipelines, modules, Terraform adapter, and evidence stream.
|
||||
|
||||
You do not write Terraform, workflow YAML, or adapter code. You write a
|
||||
contract YAML file and the platform does the rest. Your repository contains
|
||||
only your application code and that one contract.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A["your repo<br/>(app code + contract.yaml)"] -->|uses: acdl/.gitea/workflows/deploy.yml@v1.4| B
|
||||
B["ACDL platform runners<br/>(modules/ + pipelines/ + adapters/ + schemas/)"] -->|contract -> resolver -> stack -> adapter<br/>-> terraform plan -> Checkov -> confidence<br/>-> apply -> evidence event to outbox| C
|
||||
C["your resources in AWS"]
|
||||
```
|
||||
|
||||
## Versioning the `uses:` reference
|
||||
|
||||
The central deployment pipeline is **always versioned with floating MAJOR
|
||||
and MINOR tags** (e.g. `acdl/pipelines/deploy.yaml@v1.4`). Version
|
||||
constraints cannot be expressed inside the contract, so the tag in
|
||||
`uses:` is the only immutability lever a consumer has.
|
||||
|
||||
**Unversioned references are discouraged.** Do not use `@main` or a bare
|
||||
`acdl/pipelines/deploy.yaml` — `main` is constantly updated and can cause
|
||||
unexpected failures in your deployment. Pinning to a MAJOR+MINOR tag means:
|
||||
|
||||
- **Immutability** — the pipeline behavior you tested is the behavior you
|
||||
get. Patch fixes flow within the tag; breaking changes land under the
|
||||
next MINOR tag (`@v1.5`), which you opt into explicitly.
|
||||
- **Resilience** — your deployment does not break because an unrelated
|
||||
change landed on `main`.
|
||||
- **DX** — your setup is stable and reproducible. You upgrade on your
|
||||
schedule by bumping the tag.
|
||||
|
||||
All examples in this guide use `@v1.4`. When a new MINOR tag is released
|
||||
(e.g. `@v1.5`), review its changelog and bump your `uses:` reference when
|
||||
ready.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
These are the **only** prerequisites for a consumer repo. You do **not**
|
||||
need an AWS account, Terraform, Checkov, boto3, or a rotated runner key —
|
||||
those are platform-repo concerns, provided by the platform runners.
|
||||
|
||||
- **A consumer GitHub or Gitea repository** for your application code +
|
||||
`contract.yaml`.
|
||||
- **An ACDL platform runner available to your org.** The platform team
|
||||
provides runners with Terraform, Checkov, Python, and the AWS auth
|
||||
already configured. You do not install any of these.
|
||||
- **Authorization to reference the central pipeline.** Onboarding grants
|
||||
your repo the right to `uses: acdl/.gitea/workflows/deploy.yml@v1.4`.
|
||||
Contact the platform team if you have not been onboarded.
|
||||
|
||||
## Step 1 — Create a consumer repo
|
||||
|
||||
Create a repository for your application. The top level holds your app
|
||||
code; your contract lives at `.acdl/contract.yaml`. Example for a static
|
||||
site:
|
||||
|
||||
```
|
||||
my-static-site/
|
||||
index.html
|
||||
assets/
|
||||
style.css
|
||||
logo.png
|
||||
.acdl/
|
||||
contract.yaml
|
||||
```
|
||||
|
||||
Example for a microservice:
|
||||
|
||||
```
|
||||
my-microservice/
|
||||
app.py
|
||||
Dockerfile
|
||||
.acdl/
|
||||
contract.yaml
|
||||
```
|
||||
|
||||
Your app code lives at the top level. Your contract lives at
|
||||
`.acdl/contract.yaml` regardless of the module you deploy.
|
||||
|
||||
## Step 2 — Reference the central pipeline
|
||||
|
||||
In your contract YAML, declare `uses:` pointing at the central ACDL
|
||||
deployment pipeline with a **versioned tag** (floating MAJOR + MINOR):
|
||||
|
||||
```yaml
|
||||
uses: acdl/pipelines/deploy.yaml@v1.4
|
||||
```
|
||||
|
||||
This tells the platform to run the standard deployment pipeline:
|
||||
validate-contract -> resolve-stack -> terraform-plan -> checkov ->
|
||||
confidence -> apply.
|
||||
|
||||
## Step 3 — Define the contract
|
||||
|
||||
Write `.acdl/contract.yaml`. The `static-asset` example:
|
||||
|
||||
```yaml
|
||||
uses: acdl/pipelines/deploy.yaml@v1.4
|
||||
module: static-asset
|
||||
environment: dev
|
||||
inputs:
|
||||
bucket_name: my-static-site-assets
|
||||
region: us-east-1
|
||||
```
|
||||
|
||||
A `microservice` example:
|
||||
|
||||
```yaml
|
||||
uses: acdl/pipelines/deploy.yaml@v1.4
|
||||
module: microservice
|
||||
environment: dev
|
||||
inputs:
|
||||
image: my-registry/my-microservice:latest
|
||||
port: 8080
|
||||
env:
|
||||
LOG_LEVEL: info
|
||||
```
|
||||
|
||||
### Contract fields
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `uses` | string | yes | Reference to the central deployment pipeline, **versioned** with a floating MAJOR+MINOR tag (e.g. `acdl/pipelines/deploy.yaml@v1.4`). Bare or `@main` references are discouraged. |
|
||||
| `module` | string | yes | Module name from the registry — any L1 primitive or L2 composition (e.g. `static-asset`, `microservice`, `s3`). See the [module catalog](../modules/README.md). |
|
||||
| `environment` | enum | yes | `dev` (autonomous), `qa` (QA HITL), `prod` (SRE HITL), `dr` (SRE HITL). |
|
||||
| `inputs` | object | yes | Module-specific inputs (see below). |
|
||||
|
||||
### Module inputs
|
||||
|
||||
Each module declares its inputs in its `interface.json` (L1) or
|
||||
`composition.json` (L2). Consult the [module catalog](../modules/README.md)
|
||||
for the full list, or read the module's own README under `modules/l1/<name>/`
|
||||
or `modules/l2/<name>/`.
|
||||
|
||||
**`static-asset` inputs** (the worked example):
|
||||
|
||||
| Input | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `bucket_name` | string | yes | Globally-unique S3 bucket name. |
|
||||
| `region` | string | yes | AWS region the bucket is created in. |
|
||||
|
||||
The contract is validated against `schemas/contract.schema.json`. An
|
||||
invalid contract (missing field, unknown module, wrong type) fails at the
|
||||
validate-contract stage with a clear error.
|
||||
|
||||
## Step 4 — Run the pipeline
|
||||
|
||||
You do **not** run platform scripts locally for the happy path. The
|
||||
central deploy workflow is a **reusable workflow** that the platform
|
||||
runners fetch and execute for you.
|
||||
|
||||
### The consumer workflow
|
||||
|
||||
Add a thin workflow file to **your** repo that invokes the reusable ACDL
|
||||
deploy workflow with a **versioned tag**. For Gitea Actions
|
||||
(`.gitea/workflows/deploy.yml`):
|
||||
|
||||
```yaml
|
||||
name: deploy
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
jobs:
|
||||
deploy:
|
||||
uses: acdl/.gitea/workflows/deploy.yml@v1.4
|
||||
with:
|
||||
contract: .acdl/contract.yaml
|
||||
```
|
||||
|
||||
For GitHub Actions (`.github/workflows/deploy.yml`), the `uses:` line is
|
||||
identical — only the directory differs:
|
||||
|
||||
```yaml
|
||||
name: deploy
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
jobs:
|
||||
deploy:
|
||||
uses: acdl/.github/workflows/deploy.yml@v1.4
|
||||
with:
|
||||
contract: .acdl/contract.yaml
|
||||
```
|
||||
|
||||
That is the entire consumer-side workflow. When you push to `main`:
|
||||
|
||||
1. The forge resolves `uses: acdl/.gitea/workflows/deploy.yml@v1.4` (or
|
||||
the GitHub equivalent) to the reusable workflow **at the pinned tag**.
|
||||
2. A **platform-provided runner** checks out **your** repo (the consumer
|
||||
repo).
|
||||
3. The runner checks out the **ACDL platform repo** into the workspace
|
||||
(`acdl-platform/`) — this is how the pipeline fetches the platform code
|
||||
at run time. You never clone the platform repo yourself.
|
||||
4. The runner installs the runtime dependencies (Python, Terraform,
|
||||
Checkov) that the platform requires.
|
||||
5. The runner invokes `scripts/run_platform.sh` against your
|
||||
`.acdl/contract.yaml`.
|
||||
|
||||
You see the streamed output (terraform plan, Checkov results, confidence
|
||||
signal) in your forge run logs. The `--check-only` and `--plan-only` flags
|
||||
are platform-side modes visible in the pipeline logs; you do not pass them
|
||||
yourself — the reusable workflow selects the mode based on the
|
||||
`environment` in your contract (`dev` = full apply; higher environments
|
||||
hold for HITL).
|
||||
|
||||
### Local validation (optional)
|
||||
|
||||
A consumer *may* clone the ACDL platform repo to run `--check-only`
|
||||
against their contract before pushing — this is optional and not required
|
||||
for the happy path. If you do this, the runtime dependencies (Python,
|
||||
`jsonschema`, `pyyaml`, `boto3`) must be installed locally, and any AWS
|
||||
credentials follow the [Credentials](../README.md#credentials--zero-trust)
|
||||
override model: a static key in `.env.secrets` (gitignored) is rotated
|
||||
**out of band by you** — the platform guarantees daily rotation for forge
|
||||
runs, not for locally-held copies.
|
||||
|
||||
```bash
|
||||
# Optional pre-push validation (clone the platform repo first):
|
||||
bash scripts/run_platform.sh --check-only path/to/your/.acdl/contract.yaml
|
||||
# Expected: "=== PLATFORM CHECK OK ==="
|
||||
```
|
||||
|
||||
## Step 5 — What the pipeline does
|
||||
|
||||
Each stage of the central deployment pipeline (`pipelines/deploy.yaml`):
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
S1["validate-contract<br/>schema check vs contract.schema.json"] --> S2
|
||||
S2["resolve-stack<br/>contract_resolver.py -> Target Stack JSON"] --> S3
|
||||
S3["terraform-plan<br/>adapter.py compiles stack -> terraform plan (real AWS)"] --> S4
|
||||
S4["checkov<br/>policy checks -> PolicyCheckResult records"] --> S5
|
||||
S5["confidence<br/>confidence_signal.py -> score + band (dev >= 0.50)"] --> S6
|
||||
S6["apply<br/>dev only: terraform apply + evidence event to outbox"]
|
||||
```
|
||||
|
||||
1. **validate-contract** — validates your contract YAML against
|
||||
`schemas/contract.schema.json`. Fails fast on missing fields, unknown
|
||||
modules, or wrong types.
|
||||
|
||||
2. **resolve-stack** — the contract resolver
|
||||
(`acdl_platform/contract_resolver.py`) resolves your contract to a
|
||||
Target Stack instance. It loads the module's composition, expands its
|
||||
children, wires your contract inputs to the children's inputs, and
|
||||
emits a stack JSON instance.
|
||||
|
||||
3. **terraform-plan** — the Terraform adapter
|
||||
(`adapters/terraform/adapter.py`) compiles the stack to Terraform
|
||||
(`main.tf`, `terraform.tf`, `providers.tf`) and runs `terraform plan`
|
||||
against real AWS. You see the plan in your run logs.
|
||||
|
||||
4. **checkov** — Checkov runs policy checks on the emitted Terraform. The
|
||||
results are normalized to `PolicyCheckResult` records by the Checkov
|
||||
adapter. Each result has a severity, rule ID, and pass/fail status.
|
||||
|
||||
5. **confidence** — the confidence signal
|
||||
(`acdl_platform/confidence_signal.py`) computes a score from 6 inputs
|
||||
(policy, validation, freshness, source, history, NFRs). For `dev`, the
|
||||
threshold is >= 0.50. If the band is `pass`, the pipeline proceeds.
|
||||
|
||||
6. **apply** — (dev only, autonomous per the environment model) Terraform
|
||||
applies the plan, creating the resources in your AWS account. An
|
||||
evidence event (hash-chained) is written to the DynamoDB outbox.
|
||||
|
||||
## Step 6 — What gets created
|
||||
|
||||
After a successful `dev` run, the resources declared by your module's
|
||||
composition exist in your AWS account, and an evidence event is recorded.
|
||||
|
||||
For the `static-asset` example:
|
||||
|
||||
- **An S3 bucket** named `my-static-site-assets` in `us-east-1` with
|
||||
versioning enabled.
|
||||
- **An evidence event** in the DynamoDB outbox (`acdl-outbox` table) with
|
||||
the contract ID, stack name (`static-asset`), confidence score, and band.
|
||||
- **A confidence band** of `pass` (score >= 0.50 for dev).
|
||||
|
||||
For other modules, consult the module's README
|
||||
(`modules/l1/<name>/README.md` or `modules/l2/<name>/README.md`) for the
|
||||
exact resources created.
|
||||
|
||||
## Step 7 — Upload your content (static-asset example)
|
||||
|
||||
The platform provisions the infrastructure; you upload your content. For
|
||||
the `static-asset` module:
|
||||
|
||||
```bash
|
||||
aws s3 sync ./assets s3://my-static-site-assets/ --acl public-read
|
||||
```
|
||||
|
||||
(For a proper static site, configure the bucket for website hosting or
|
||||
put a CloudFront distribution in front — both are future compliance
|
||||
extension points for the `static-asset` module.)
|
||||
|
||||
For a `microservice`, the platform provisions the ECS service and ALB; you
|
||||
push your container image to the ECR repo the platform created.
|
||||
|
||||
## Step 8 — Promote to qa / prod
|
||||
|
||||
Change `environment` in your contract (keeping the same versioned `uses:`):
|
||||
|
||||
```yaml
|
||||
uses: acdl/pipelines/deploy.yaml@v1.4
|
||||
environment: qa # QA HITL gate + confidence >= 0.75
|
||||
environment: prod # SRE HITL gate + confidence >= 0.90
|
||||
```
|
||||
|
||||
Higher environments require human attestation (forge deployment approval)
|
||||
and higher confidence thresholds. The platform enforces separation of
|
||||
duties (qaApprover != prodApprover) via the DynamoDB outbox.
|
||||
|
||||
| Environment | Autonomy | Gate |
|
||||
|-------------|----------|------|
|
||||
| dev | Full autonomy | Confidence >= 0.50 |
|
||||
| qa | QA HITL | Confidence >= 0.75 |
|
||||
| prod | SRE HITL | Confidence >= 0.90 |
|
||||
| dr | SRE HITL | Confidence >= 0.95 + dr-drill |
|
||||
|
||||
## Step 9 — Compliance extensions
|
||||
|
||||
Each module lists compliance extension points for the future compliance
|
||||
milestone (GDPR, SOX, SOC2, HIPAA, DORA). See each module's README under
|
||||
`modules/l1/<name>/README.md` or `modules/l2/<name>/README.md` for the
|
||||
per-module extension points. Common examples:
|
||||
|
||||
- **KMS key** — shared encryption key for SSE.
|
||||
- **S3 access logs** — access logging to a separate audit bucket.
|
||||
- **Object Lock** — 7-year immutable retention for evidence.
|
||||
- **Public access block** — prevent data exfiltration.
|
||||
|
||||
## Reference
|
||||
|
||||
| Resource | Path | Description |
|
||||
|----------|------|-------------|
|
||||
| Central deployment pipeline contract | `pipelines/deploy.yaml` | The pipeline stages your contract references. |
|
||||
| Reusable deploy workflow (Gitea) | `.gitea/workflows/deploy.yml` | The workflow your repo invokes via `uses:`. |
|
||||
| Reusable deploy workflow (GitHub) | `.github/workflows/deploy.yml` | The workflow your repo invokes via `uses:`. |
|
||||
| Contract schema | `schemas/contract.schema.json` | JSON Schema for consumer contracts. |
|
||||
| Stack schema | `schemas/stack.schema.json` | JSON Schema for the resolved stack instance. |
|
||||
| Module catalog | `modules/README.md` | All L1 primitives and L2 compositions. |
|
||||
| Sample contract | `contracts/static-asset.yaml` | The reference example contract (uses `@v1.4`). |
|
||||
| Contract resolver | `acdl_platform/contract_resolver.py` | Resolves contracts to stack instances. |
|
||||
| Terraform adapter | `adapters/terraform/adapter.py` | Compiles stack instances to Terraform. |
|
||||
| Platform pipeline runner | `scripts/run_platform.sh` | The pipeline runner (platform-side; consumers do not invoke it directly). |
|
||||
| Platform README | `README.md` | How the platform works + how to run the platform repo locally. |
|
||||
| Credentials & zero-trust | `README.md#credentials--zero-trust` | The OIDC/ABAC default + static-key override model. |
|
||||
@@ -1,51 +1,64 @@
|
||||
# l2-static-asset — S3 static asset (composition being redesigned)
|
||||
# static-asset — S3 static asset
|
||||
|
||||
> **Module kind:** L2 composition | **Version:** TBD | **Status:** Under redesign
|
||||
> **Module kind:** L2 composition | **Version:** 1.0.0
|
||||
|
||||
A composition that references the `l1-s3` primitive to deploy a single
|
||||
A composition that references the `s3` L1 primitive to deploy a single
|
||||
S3 bucket for static asset hosting.
|
||||
|
||||
**The composition layer is being redesigned.** The previous
|
||||
thin-composition implementation (a `composition.json` with children +
|
||||
wires) has been removed. A new composition mechanism will be designed
|
||||
in a later phase.
|
||||
|
||||
## Resources
|
||||
|
||||
TBD — the composition will reference this L1 primitive:
|
||||
The composition references this L1 primitive:
|
||||
|
||||
| L1 module | Purpose | README |
|
||||
|-----------|---------|--------|
|
||||
| `l1-s3` | S3 bucket | [README](../l1/l1-s3/README.md) |
|
||||
| `s3` | S3 bucket | [README](../l1/s3/README.md) |
|
||||
|
||||
## Inputs
|
||||
|
||||
TBD — will be defined when the composition mechanism is redesigned.
|
||||
| Name | Type | Required | Description |
|
||||
|------|------|----------|-------------|
|
||||
| `bucket_name` | string | yes | Globally-unique S3 bucket name |
|
||||
| `region` | string | yes | AWS region the bucket is created in |
|
||||
|
||||
## Outputs
|
||||
|
||||
TBD — will be defined when the composition mechanism is redesigned.
|
||||
| Name | Type | Description |
|
||||
|------|------|-------------|
|
||||
| `bucket_arn` | arn | The S3 bucket ARN |
|
||||
| `bucket_name` | string | The bucket name (echoes the input) |
|
||||
|
||||
## Usage
|
||||
|
||||
TBD — the composition mechanism is being redesigned. Until then, use
|
||||
`l1-s3` directly. See the [l1-s3 README](../l1/l1-s3/README.md) for a
|
||||
usage example.
|
||||
Define a contract referencing this composition:
|
||||
|
||||
```yaml
|
||||
uses: acdl/pipelines/deploy.yaml@v1
|
||||
module: static-asset
|
||||
environment: dev
|
||||
inputs:
|
||||
bucket_name: my-static-assets
|
||||
region: us-east-1
|
||||
```
|
||||
|
||||
See the [consumer guide](../../docs/CONSUMER_GUIDE.md) for a
|
||||
step-by-step walkthrough, and the [s3 README](../l1/s3/README.md) for the
|
||||
underlying L1 primitive.
|
||||
|
||||
## Compliance extension points
|
||||
|
||||
The composition will need to wire compliance resources when the
|
||||
compliance milestone (GDPR, SOX, SOC2, HIPAA, DORA) lands:
|
||||
The composition can wire compliance resources when the compliance
|
||||
milestone (GDPR, SOX, SOC2, HIPAA, DORA) lands:
|
||||
|
||||
- **KMS key** — shared encryption key for S3 SSE.
|
||||
- **S3 access logs** — access logging to a separate audit bucket.
|
||||
- **Object Lock** — 7-year immutable retention for evidence.
|
||||
- **Public access block** — prevent data exfiltration.
|
||||
|
||||
See the [l1-s3 README](../l1/l1-s3/README.md) for per-module compliance
|
||||
extension points.
|
||||
See the [s3 README](../l1/s3/README.md) for per-module compliance extension
|
||||
points.
|
||||
|
||||
## Versioning
|
||||
|
||||
Versioning will be defined when the composition mechanism is
|
||||
redesigned.
|
||||
`1.0.0` — interface MAJOR, behavior MINOR, lifecycle PATCH. MAJOR bumps
|
||||
require a new registry entry (immutable publication); old entries enter
|
||||
a 12-month deprecation window.
|
||||
@@ -143,6 +143,7 @@ class TestWorkflowConformance:
|
||||
for py_file in [
|
||||
"acdl_platform/confidence_signal.py",
|
||||
"acdl_platform/outbox_writer.py",
|
||||
"acdl_platform/contract_resolver.py",
|
||||
"adapters/terraform/adapter.py",
|
||||
"adapters/terraform/policy/checkov_adapter.py",
|
||||
"scripts/push_consumer_image.py",
|
||||
@@ -175,6 +176,7 @@ class TestRunCiScript:
|
||||
content = open(ROOT / "scripts/run_ci.sh").read()
|
||||
assert "py_compile" in content
|
||||
assert "acdl_platform/confidence_signal.py" in content
|
||||
assert "acdl_platform/contract_resolver.py" in content
|
||||
assert "adapters/terraform/adapter.py" in content
|
||||
|
||||
def test_run_ci_script_contains_test_stage(self):
|
||||
@@ -198,6 +200,7 @@ class TestRunCiScript:
|
||||
"python3 -m py_compile "
|
||||
"acdl_platform/confidence_signal.py "
|
||||
"acdl_platform/outbox_writer.py "
|
||||
"acdl_platform/contract_resolver.py "
|
||||
"adapters/terraform/adapter.py "
|
||||
"adapters/terraform/policy/checkov_adapter.py "
|
||||
"scripts/push_consumer_image.py && "
|
||||
@@ -234,3 +237,125 @@ class TestRunPlatformStreaming:
|
||||
assert result.returncode == 0
|
||||
assert "PLATFORM CHECK OK" in result.stdout
|
||||
assert "--- emitted terraform/spike/main.tf ---" not in result.stdout
|
||||
|
||||
|
||||
class TestDeployPipelineSchema:
|
||||
def test_deploy_schema_is_valid_json_schema(self):
|
||||
schema = json.load(open(ROOT / "schemas/deploy-pipeline.schema.json"))
|
||||
jsonschema.Draft202012Validator.check_schema(schema)
|
||||
|
||||
def test_deploy_schema_has_required_fields(self):
|
||||
schema = json.load(open(ROOT / "schemas/deploy-pipeline.schema.json"))
|
||||
assert "name" in schema["required"]
|
||||
assert "triggers" in schema["required"]
|
||||
assert "runner" in schema["required"]
|
||||
assert "stages" in schema["required"]
|
||||
|
||||
def test_deploy_schema_stage_def_has_command_and_required(self):
|
||||
schema = json.load(open(ROOT / "schemas/deploy-pipeline.schema.json"))
|
||||
stage_def = schema["$defs"]["stage"]
|
||||
assert "command" in stage_def["required"]
|
||||
assert "required" in stage_def["required"]
|
||||
|
||||
|
||||
class TestDeployPipelineContract:
|
||||
def test_deploy_contract_validates_against_schema(self):
|
||||
schema = json.load(open(ROOT / "schemas/deploy-pipeline.schema.json"))
|
||||
contract = _load_yaml("pipelines/deploy.yaml")
|
||||
jsonschema.validate(contract, schema)
|
||||
|
||||
def test_deploy_contract_has_six_stages(self):
|
||||
contract = _load_yaml("pipelines/deploy.yaml")
|
||||
stage_names = [s["name"] for s in contract["stages"]]
|
||||
assert stage_names == [
|
||||
"validate-contract",
|
||||
"resolve-stack",
|
||||
"terraform-plan",
|
||||
"checkov",
|
||||
"confidence",
|
||||
"apply",
|
||||
]
|
||||
|
||||
def test_deploy_contract_runner_is_ubuntu_latest(self):
|
||||
contract = _load_yaml("pipelines/deploy.yaml")
|
||||
assert contract["runner"] == "ubuntu-latest"
|
||||
|
||||
|
||||
class TestDeployWorkflowConformance:
|
||||
def test_gitea_deploy_workflow_exists(self):
|
||||
assert (ROOT / ".gitea/workflows/deploy.yml").is_file()
|
||||
|
||||
def test_github_deploy_workflow_exists(self):
|
||||
assert (ROOT / ".github/workflows/deploy.yml").is_file()
|
||||
|
||||
def test_deploy_workflows_are_byte_identical(self):
|
||||
gitea = open(ROOT / ".gitea/workflows/deploy.yml", "rb").read()
|
||||
github = open(ROOT / ".github/workflows/deploy.yml", "rb").read()
|
||||
assert gitea == github, "Gitea and GitHub deploy workflows must be byte-identical"
|
||||
|
||||
def test_deploy_workflow_name_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
contract = _load_yaml("pipelines/deploy.yaml")
|
||||
assert wf["name"] == contract["name"]
|
||||
|
||||
def test_deploy_workflow_is_reusable(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
assert "workflow_call" in wf["on"]
|
||||
|
||||
def test_deploy_workflow_has_contract_input(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
inputs = wf["on"]["workflow_call"]["inputs"]
|
||||
assert "contract" in inputs
|
||||
assert inputs["contract"]["default"] == ".acdl/contract.yaml"
|
||||
|
||||
def test_deploy_workflow_has_mode_input(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
inputs = wf["on"]["workflow_call"]["inputs"]
|
||||
assert "mode" in inputs
|
||||
assert inputs["mode"]["default"] == "full"
|
||||
|
||||
def test_deploy_workflow_runner_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
contract = _load_yaml("pipelines/deploy.yaml")
|
||||
for job in wf["jobs"].values():
|
||||
assert job["runs-on"] == contract["runner"]
|
||||
|
||||
def test_deploy_workflow_python_version_matches_contract(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
contract = _load_yaml("pipelines/deploy.yaml")
|
||||
for job in wf["jobs"].values():
|
||||
setup_step = next(
|
||||
s for s in job["steps"] if "setup-python" in s.get("uses", "")
|
||||
)
|
||||
assert setup_step["with"]["python-version"] == contract["python_version"]
|
||||
|
||||
def test_deploy_workflow_invokes_run_platform(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
deploy_job = wf["jobs"]["deploy"]
|
||||
run_step = next(
|
||||
s for s in deploy_job["steps"] if "run" in s and "run_platform" in s["run"]
|
||||
)
|
||||
assert "run_platform.sh" in run_step["run"]
|
||||
|
||||
def test_deploy_workflow_checks_out_platform_repo(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
deploy_job = wf["jobs"]["deploy"]
|
||||
platform_checkout = next(
|
||||
s for s in deploy_job["steps"]
|
||||
if "checkout" in s.get("uses", "") and s.get("with", {}).get("path") == "acdl-platform"
|
||||
)
|
||||
assert platform_checkout["with"]["repository"] == "acdl/acdl"
|
||||
|
||||
def test_deploy_workflow_permissions_id_token_write(self):
|
||||
wf = _load_workflow(".gitea/workflows/deploy.yml")
|
||||
assert wf["permissions"]["id-token"] == "write"
|
||||
assert wf["permissions"]["contents"] == "read"
|
||||
|
||||
|
||||
class TestSampleContractVersioning:
|
||||
def test_sample_contract_uses_versioned_tag(self):
|
||||
contract = _load_yaml("contracts/static-asset.yaml")
|
||||
uses = contract["uses"]
|
||||
assert "@v" in uses, "sample contract must use a versioned @vX.Y tag"
|
||||
assert "@main" not in uses, "sample contract must not use @main"
|
||||
assert uses == "acdl/pipelines/deploy.yaml@v1.4"
|
||||
Reference in New Issue
Block a user