The runbook §6 section ordering was 6.1 → 6.4 → 6.2 → 6.3
because the v1.1 EXECUTE stage inserted §6.4 "Secret
rotation" (REQ-INB-21, commit 6067008) between §6.1
"Secret inventory" and §6.2 "Why we never commit secrets".
The non-monotonic ordering was filed as v1.1 backlog P1-1
(CI-LOG.md:217-218) and v1.1 explicitly deferred the fix
to a follow-up milestone. v1.2 was a 0/1 disposition so
the carry-forward is v1.3.
This commit restores the monotonic ordering:
- §6.1 Secret inventory (unchanged)
- §6.2 Secret rotation (renumbered from §6.4)
- §6.3 Why we never commit secrets (renumbered from §6.2)
- §6.4 Data-sovereignty posture (renumbered from §6.3)
Plus 4 cross-reference updates in the runbook body:
- line 509: "see §6.4 for the procedure" → "see §6.2 for the procedure"
- line 625: "§6.4 (new in v1.1)" → "§6.2 (new in v1.1, renumbered in v1.3 from §6.4)"
- line 767: "§6.4 documents" → "§6.2 documents"
- line 790: "§5.7 + §6.4" → "§5.7 + §6.2"
Verified: grep -n '^### 6\.' docs/operations-runbook.md
returns 4 sections in 6.1 → 6.2 → 6.3 → 6.4 order.
grep -n '§6\.' docs/operations-runbook.md returns no
stale §6.4 cross-references pointing at the old
"Secret rotation" position.
This commit closes v1.1 backlog item P1-1 (runbook §6
ordering, per CI-LOG.md:217-218).
---ci---
project: inboxos
phase: 1
milestone: v1.3
status: execute
tasks_completed: [T2.5]
candidates_closed: [REQ-INB-36]
backlog_closures:
v1_1_P1_1: REQ-INB-36
files_touched:
- docs/operations-runbook.md (4 heading renumbers + 4 cross-ref updates, lines 509, 511, 561, 568, 625, 767, 790)
---/ci---
The runbook §4.5 "Forward-pointer" paragraph (lines 285-289)
said "Alerting is out of scope for v1.1; the alert layer is
the next v1.2 deliverable." v1.2 has shipped (tag v1.2.0 on
63c0f59) and was a 0/1 disposition — no alerting deliverable
landed. The forward-pointer was stale.
This commit updates the paragraph to reflect v1.3 reality:
alerting remains out of scope; v1.2 closed on 0/1; the
alert layer is the next v1.x candidate cycle to re-evaluate.
The metrics substrate (REQ-INB-23) is referenced as the
current observability surface. The rest of §4.5 (the
metrics tail snippet at lines 280-283, the opt-out block
at 291-294) is unchanged.
---ci---
project: inboxos
phase: 1
milestone: v1.3
status: execute
tasks_completed: [T2.2]
candidates_closed: [REQ-INB-30]
files_touched:
- docs/operations-runbook.md (1 paragraph, lines 285-289, 5 lines)
---/ci---
The runbook footer (line 823) still said "Next review: after
the v1.1 ship (2026-06-08, milestone `milestone/v1.1-initial`)"
even though v1.1 has shipped, v1.2 has shipped (tag v1.2.0 on
63c0f59), and v1.3 is in the ideate phase on
milestone/v1.3-initial. The footer pointer was stale by 2
milestones.
This commit updates the footer to "Next review: after the
v1.3 ship (2026-06-08, milestone `milestone/v1.3-initial`)"
so the v1.3 audit trail is consistent. The rest of the
footer (the trailing sentence about divergence) is
preserved verbatim.
---ci---
project: inboxos
phase: 1
milestone: v1.3
status: execute
tasks_completed: [T2.1]
candidates_closed: [REQ-INB-29]
files_touched:
- docs/operations-runbook.md (1 line, line 823)
---/ci---
P0 from v1.1 multi-persona code review (Pass 1, doc-drift sweep).
The runbook referenced the old cto-inboxos owner at two locations
that the v1.1 work introduced (the §5.1 runner recovery example
and the §10 RTO breakdown) and had a stale 'Next review' footer
that still pointed at the v1.0 Phase 5 close.
- docs/operations-runbook.md:319 §5.1 (Gitea runners API URL)
- docs/operations-runbook.md:792 §10 (Gitea restore push URL)
- docs/operations-runbook.md:823 footer (Next review pointer)
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: review
---/ci---
P0 from v1.1 multi-persona code review (Pass 1, correctness sweep).
The three v1.1-introduced shell helpers (backup.sh, restore.sh,
rotate_secrets.sh) hardcoded the old cto-inboxos owner in the clone URL,
push URL, and default REPO_OWNER. With the org migration to jc-capital
landed at a068c8a, those hardcodes would fail at first invocation:
- scripts/backup.sh:151 clone URL
- scripts/restore.sh:145 push --mirror URL
- scripts/rotate_secrets.sh:32-33 default REPO_OWNER
- scripts/rotate_secrets.sh:101,131 string-literal fallback
The runbook §6.2 service-account line (562) also referenced the old
owner; the runbook lines that are CPO-managed were left alone.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: review
---/ci---
REQ-INB-15..25 close v1.1 Phase 1. The new §9.4 'v1.1 acceptance
gate' is appended to docs/operations-runbook.md, mirroring the
v1.0 §9.1 + §9.2 structure but only for the 11 new
requirements. Each must-have row cites:
- the REQ-INB-N it satisfies;
- the file:line (or commit SHA, when the line is the test)
that proves it;
- the verification command (where applicable);
- the owner persona.
The 11 candidates are explicitly marked '[CANDIDATE] until the
CPO authors the matching entries in docs/requirements.md'. The
trailing paragraph notes that the .ciagent/inboxos/ROADMAP.md
Phase 1 status row is not flipped by the lead-developer; the
flip is the CPO's call. The PR description in git log reads:
'v1.1 closure: acceptance gate appended; all 11 must-haves
cross-referenced to source'.
No source code is modified by this task; the entire change is
documentation. The 14 v1.1 execution commits (T1.1..T1.7 +
T2.1..T2.5 + T3.1..T3.2) are visible in 'git log --oneline' on
phase/01-v1.1-execute.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
REQ-INB-22 lands the backup + restore helper and the runbook RTO/RPO.
scripts/backup.sh (new, ~120 lines, set -euo pipefail):
- Accepts a positional argument: nocodb | seaweedfs | gitea.
- nocodb: POST /api/v2/meta/bases/<id>/export to a timestamped
file under backups/nocodb/<ts>/nocodb-export.json.
- seaweedfs: requires SEAWEEDFS_S3_BACKUP_BUCKET and
SEAWEEDFS_S3_BACKUP_ENDPOINT. Mirrors the inboxos bucket to
the backup bucket via 'aws s3 sync' (or boto3 fallback).
Errors with exit code 4 + a clear 'no backup target
configured' message when the env vars are unset.
- gitea: 'git bundle create --all' of the cto-inboxos/inboxos
repo, or 'git clone' + 'git bundle create' fallback.
- Prints the documented one-line summary:
backup_id=<ts> service=<svc> bytes=<int> target=<path>.
scripts/restore.sh (new, ~100 lines, set -euo pipefail):
- Accepts two positional arguments: <backup_id> <service>.
- Verifies the backup directory layout: backups/<svc>/<id>/.
- For nocodb: prints the import instructions (NocoDB has no
server-side import on the free / self-hosted plan; the
operator uses the NocoDB UI to upload the JSON).
- For seaweedfs: reverse-sync from the backup bucket to the
primary bucket via 'aws s3 sync' (or boto3 fallback).
- For gitea: clones the bundle to a temp dir, force-pushes
via the Gitea REST API.
- Prints the documented one-line summary:
restored backup_id=<id> service=<svc> verified=<bool>.
pipelines/_tests/test_restore_offline.py (new, 7 tests):
- All 7 tests pass on the offline test host. The git-bundle
round-trip test pins the integration shape that the
operator's host will exercise end-to-end.
docs/operations-runbook.md:
- New §5.7 'Backup and restore' after §5.6 'Historical pipeline
run cancelled mid-way'. Documents the daily cron, the
retention (7 daily + 4 weekly + 0 monthly), the copy-paste
invocation examples, and the test-or-die gate.
- New §10 'Disaster recovery targets' after §9.3 'Closing the
gate'. States RTO < 4 hours and RPO < 24 hours with the
time-breakdown rationale. Marks the values as explicit and
CPO-acknowledged (the ideation report Tier 2 T2-C flagged
them as 'guesses' pending CPO sign-off; the values recorded
here are the v1.1 ship defaults).
- §8.2 'Cross-references' gains the §5.7 + §10 cross-
reference bullet, removing the trailing 'added by T3.1
itself' note (the bullet is now present and the list is
internally consistent).
scripts/apply_schemas.py / pipelines/_tests/test_papra_index_offline.py
are not touched (the schema freeze is preserved). The
PAPRA_INDEX_TABLE env var is read by the helper but is not
required by the backup / restore procedure; the Gitea
secrets for the new backup target bucket are the only new
env vars.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
REQ-INB-15/19/20/21/23/25 all land new sections or
new artifacts in this v1.1 plan. The runbook §8.2 'Cross-
references' list gains three new bullets to keep it internally
consistent with the v1.1 deliverables:
- Papra write policy: extended to note the live-write gate
(REQ-INB-20) and the papra_index audit table (REQ-INB-19).
- Secret rotation (NEW): §6.4 (REQ-INB-21).
- Metrics surface (NEW): §4.5 (REQ-INB-23).
The §5.7 + §10 cross-reference for backups (REQ-INB-22) is
deliberately NOT added here; T3.1 inserts the §5.7 + §10
sections and adds the cross-reference bullet in the same PR so
the §8.2 list is always internally consistent. A trailing
paragraph notes this.
The existing bullets (Schemas, Extraction contract, Gitea
Actions, Pipelines, Papra write policy, Phase 1.1 verification)
remain in place. The Papra write policy bullet is extended (not
duplicated) to mention the new sub-sections.
No source code is modified; the entire change is documentation.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
REQ-INB-24 closes the post-hoc-review backlog items P1-1 and P2-4
that the v1.0 close did not fit.
.ciagent/inboxos/plans/05-PLAN.md:
- The 'Parallelization plan (one PR per task)' table at
lines 318-330 gains a one-line footnote under the table:
'PR #C originally opened with 2 files; the third (papra-write-
policy.md §Rule 1 Live pipeline note:) was a follow-on added
in a hotfix during review. P1-1 was the absence of an explicit
three-file split in the plan; this footnote is the explicit
split.'
docs/operations-runbook.md:
- §9.2 row 19 (line 559 in pre-T1.2 numbering) gains a one-line
annotation in the 'Closed by' cell: '... owner: backend-
engineer (the P-7 papra_decide investigation; the asymmetry
is structural, the doc is the deliverable)'. The row now
explicitly names the persona.
.ciagent/inboxos/CI-LOG.md:
- P1-1 entry gains a 'closed_in: <T2.4 commit SHA>' pointer
pointing at the plans/05-PLAN.md footnote.
- P2-4 entry gains a 'closed_in: <T2.4 commit SHA>' pointer
pointing at the runbook §9.2 row 19 owner annotation.
- The 'remaining_for_phase_6' list loses both P1-1 and P2-4
(down to [P1-3, P1-5, P2-1, P2-2]). P1-3, P1-5, P2-1, P2-2
remain in the Phase 6 backlog (these are v1.1 candidate REQ-INB
IDs, closed by the v1.1 plan itself, not by T2.4).
CI-LOG.md is a CIAgent-side document and is not CPO-owned; the
lead-developer can edit it as part of this task. The exact
lines (96-97, 112-113, 115) are cited in the plan's must_haves.
No source code is modified; the entire change is documentation.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
T2.2 closes the call-site wiring for the T1.5 Papra live-write gate
and lands the per-run metrics.jsonl surface (REQ-INB-23). The
T1.5 + T2.2 work is folded into one commit so the two pipeline
files are touched by exactly one Wave 2 task.
pipelines/common.py:
- LLM_METRICS_ENABLED env-var gate around the
llm_call_seconds_sum / llm_call_count accumulators in
_ollama_call_one_model. The accumulators are stored in
os.environ so the main() functions can read them when emitting
the per-run metrics line. Default is 'true' (gate enabled).
pipelines/live_pipeline.py:
- Import emit_papra_write_gate_event and emit_metric.
- process_one_message: emit the papra_write_gate event at the
start of the papra branch (REQ-INB-20 call-site).
- main: emit one metrics line with the COUNTER_KEYS / GAUGE_KEYS
vocabulary from pipelines/_metrics/emit.py, including
llm_call_seconds_sum and llm_call_count read from the
os.environ accumulators.
pipelines/historical_pipeline.py:
- Import emit_papra_write_gate_event and emit_metric.
- process_one_message: emit the papra_write_gate event at the
start of the body-decision branch.
- main: emit one metrics line with the same vocabulary.
pipelines/_metrics/emit.py (new):
- emit_metric(run_id, pipeline, *, counters, gauges) appends one
JSONL line to <pipeline>-metrics-<run_id>.jsonl. The
COUNTER_KEYS and GAUGE_KEYS module constants pin the vocabulary
so the test can assert exact equality.
pipelines/_metrics/__init__.py (new):
- Empty / near-empty package marker.
pipelines/_tests/test_metrics_offline.py (new, 2 tests):
- test_emit_metric_creates_file_with_one_line_per_call: asserts
the line shape, the exact COUNTER_KEYS / GAUGE_KEYS, and the
per-run file naming.
- test_emit_metric_does_not_overwrite_existing_file: asserts two
calls in a row append, never truncate.
docs/operations-runbook.md:
- New §4.5 'Metrics surface' (after §4.4 'SeaweedFS bucket
layout' at line 238). Documents the file location, the counter
and gauge vocabulary, the tail-and-summarize procedure, the
opt-out via LLM_METRICS_ENABLED, and a forward-pointer noting
alerting is a v1.2 candidate (out of scope for v1.1).
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
REQ-INB-21 lands the secret-rotation helper and the runbook procedure.
scripts/rotate_secrets.sh (new, ~150 lines, set -euo pipefail):
- Accepts --dry-run (default) and --apply flags plus a positional
secret name.
- Reads the new value from STDIN (never a CLI arg) to avoid ps leaks.
- Calls pipelines.common.gitea_request via a python3 subshell to PUT
the new value to Gitea's actions-secrets endpoint. The Gitea
actions-secrets PUT is in-place (overwrites the existing secret
at the same name); the helper then verifies via GET that the
secret still exists under the new value.
- Exits 0 on success, 2 on missing env / missing value, 3 on Gitea
API failure, 4 on helper-internal error.
- Dry-run prints a redacted diff (sha256:8hex) of the new value
against an unknown placeholder for the old value (Gitea does
not return secret plaintexts via the API).
pipelines/_tests/test_rotate_secrets_offline.py (new, ~3 tests):
- test_dry_run_prints_diff_and_exits_zero_no_put
- test_apply_branch_runs_python_helper
- test_helper_fails_loud_when_env_unset
The 3 tests pass on the offline test host (the apply branch test
documents that exit code 0 or 3 is acceptable in this environment:
0 on a real Gitea, 3 on connection failure with no Gitea reachable).
docs/operations-runbook.md:
- §6.1 line 401 'future enhancement' parenthetical replaced with
'The helper landed in v1.1 (REQ-INB-21); see §6.4.'
- New §6.4 'Secret rotation' between §6.1 and §6.2: rotation
ordering (NOCODB_API_TOKEN -> SEAWEEDFS_* -> IMAP_PASSWORD ->
GITEA_USER/PASS), test-or-die gate, and a copy-paste invocation
example.
.env.example is not touched (env vars are unchanged; only the
helper and the procedure are new).
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
The table has 22 data rows: 6 IMAP + 4 SeaweedFS + 4 NocoDB + 2 Ollama + 3 Papra + 3 Gitea (INSTANCE_URL, USER, PASS). The v1.0 ship counted 21 because GITEA_USER / GITEA_PASS was treated as one credential. Replace the stale count and the stale rationale with the corrected count, the rationale (GITEA_USER / GITEA_PASS row was treated as one credential), and a corrected rounding note that pins the source of the '15' to the Papra trio being optional.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
The previous glob cited upload-artifact@v3 across {live,historical,ci-pr}.yml. Verification: 'grep -n upload-artifact .gitea/workflows/ci-pr.yml' returns zero hits. The pin is real only in live-pipeline.yml:150 and historical-pipeline.yml:170. Replace the misleading glob with the two concrete line citations and a parenthetical noting that ci-pr.yml has no upload-artifact step.
The §'v3 pinning' paragraph in docs/gitea-actions.md:247 already correctly scopes the v3 pin to live + historical only; no edit required.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
- P1-4: Redraw runbook §1 system diagram to remove the OpenCode CLI
node (the runner now calls Ollama's /v1/chat/completions directly).
- P1-6: Fix CI-LOG.md ED-1 effect: value to be a single-line string
(was multi-line continuation that YAML interprets as nested keys).
- P1-2: Update plan §Phase 5 acceptance gate row 16 to use §6.1
reference (was :487; the runbook grew 13 lines during Wave 1+2).
- Update plan row 19 to reflect the ED-1 no-op (live papra_decide
at line 490 is already the post-classify call).
Remaining P1/P2 issues are filed in the COMPLETE commit's
post-hoc-review block as Phase 6 backlog.
---ci---
project: inboxos
phase: 5
milestone: v1.0
status: review
refs_review_issues: [P1-2, P1-4, P1-6]
files: [docs/operations-runbook.md, .ciagent/inboxos/CI-LOG.md, .ciagent/inboxos/plans/05-PLAN.md]
---/ci---
The runbook's state-file inventory drifted from the actual code.
Field names now match pipelines/live_pipeline.py:600-603 and
pipelines/historical_pipeline.py:516-521.
---ci---
project: inboxos
phase: 5
milestone: v1.0
status: execute
pr: H
task: T2.1
persona: devops-engineer
ref: P-6
refs_requirements: [REQ-INB-3, REQ-INB-6]
files: [docs/operations-runbook.md]
---/ci---
- docs/operations-runbook.md: 8-section runbook (system overview,
service inventory, workflow ops, observability, failure modes,
security, cost, glossary) so a new operator can recover from
common failures without re-deriving the architecture.
- README.md: top-level entry point that links to the runbook as
the day-1 incident-response reference.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
cloudinit-bot
cto