The v1.1 P0 sweep (commits 7c20d5d, d530921) updated the 3
new shell helpers, the runbook, and 2 runbook lines to point
at jc-capital. It missed 2 files:
- docs/gitea-actions.md:95 (last touched at Phase 4 / INB-33,
before the v1.0→v1.1 org migration; the v1.1 sweep was
scoped to .md / .sh created or modified in v1.1)
- scripts/setup_gitea_secrets.sh:4,14,18,28 (pre-existing at
the v1.0 close; not in the v1.1 diff; filed as v1.1 P1-2
for v1.2 follow-up but v1.2 was a 0/1 disposition so the
carry-forward is v1.3)
This commit closes the cto-inboxos owner drift in both files
using the same env-var-resolved default pattern as the v1.1
P0 sweep:
- docs/gitea-actions.md:95 now uses
${GITEA_OWNER:-jc-capital}/inboxos
- scripts/setup_gitea_secrets.sh:4,14 use the same
${GITEA_OWNER:-jc-capital} pattern in the header
- scripts/setup_gitea_secrets.sh:18 defaults
GITEA_OWNER to jc-capital
- scripts/setup_gitea_secrets.sh:28 uses
${GITEA_OWNER:-jc-capital} in the example
Verified: grep -n cto-inboxos docs/gitea-actions.md
scripts/setup_gitea_secrets.sh returns empty. The script is
still chmod +x with set -euo pipefail (unchanged).
This commit also closes v1.1 backlog item P1-2
(setup_gitea_secrets.sh owner drift, per CI-LOG.md:219-220).
---ci---
project: inboxos
phase: 1
milestone: v1.3
status: execute
tasks_completed: [T1.1]
candidates_closed: [REQ-INB-28, REQ-INB-37]
backlog_closures:
v1_1_P1_2: REQ-INB-37
files_touched:
- docs/gitea-actions.md (1 line, line 95)
- scripts/setup_gitea_secrets.sh (4 lines, 4, 14, 18, 28)
---/ci---
P0 from v1.1 multi-persona code review (Pass 1, correctness sweep).
The three v1.1-introduced shell helpers (backup.sh, restore.sh,
rotate_secrets.sh) hardcoded the old cto-inboxos owner in the clone URL,
push URL, and default REPO_OWNER. With the org migration to jc-capital
landed at a068c8a, those hardcodes would fail at first invocation:
- scripts/backup.sh:151 clone URL
- scripts/restore.sh:145 push --mirror URL
- scripts/rotate_secrets.sh:32-33 default REPO_OWNER
- scripts/rotate_secrets.sh:101,131 string-literal fallback
The runbook §6.2 service-account line (562) also referenced the old
owner; the runbook lines that are CPO-managed were left alone.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: review
---/ci---
REQ-INB-22 lands the backup + restore helper and the runbook RTO/RPO.
scripts/backup.sh (new, ~120 lines, set -euo pipefail):
- Accepts a positional argument: nocodb | seaweedfs | gitea.
- nocodb: POST /api/v2/meta/bases/<id>/export to a timestamped
file under backups/nocodb/<ts>/nocodb-export.json.
- seaweedfs: requires SEAWEEDFS_S3_BACKUP_BUCKET and
SEAWEEDFS_S3_BACKUP_ENDPOINT. Mirrors the inboxos bucket to
the backup bucket via 'aws s3 sync' (or boto3 fallback).
Errors with exit code 4 + a clear 'no backup target
configured' message when the env vars are unset.
- gitea: 'git bundle create --all' of the cto-inboxos/inboxos
repo, or 'git clone' + 'git bundle create' fallback.
- Prints the documented one-line summary:
backup_id=<ts> service=<svc> bytes=<int> target=<path>.
scripts/restore.sh (new, ~100 lines, set -euo pipefail):
- Accepts two positional arguments: <backup_id> <service>.
- Verifies the backup directory layout: backups/<svc>/<id>/.
- For nocodb: prints the import instructions (NocoDB has no
server-side import on the free / self-hosted plan; the
operator uses the NocoDB UI to upload the JSON).
- For seaweedfs: reverse-sync from the backup bucket to the
primary bucket via 'aws s3 sync' (or boto3 fallback).
- For gitea: clones the bundle to a temp dir, force-pushes
via the Gitea REST API.
- Prints the documented one-line summary:
restored backup_id=<id> service=<svc> verified=<bool>.
pipelines/_tests/test_restore_offline.py (new, 7 tests):
- All 7 tests pass on the offline test host. The git-bundle
round-trip test pins the integration shape that the
operator's host will exercise end-to-end.
docs/operations-runbook.md:
- New §5.7 'Backup and restore' after §5.6 'Historical pipeline
run cancelled mid-way'. Documents the daily cron, the
retention (7 daily + 4 weekly + 0 monthly), the copy-paste
invocation examples, and the test-or-die gate.
- New §10 'Disaster recovery targets' after §9.3 'Closing the
gate'. States RTO < 4 hours and RPO < 24 hours with the
time-breakdown rationale. Marks the values as explicit and
CPO-acknowledged (the ideation report Tier 2 T2-C flagged
them as 'guesses' pending CPO sign-off; the values recorded
here are the v1.1 ship defaults).
- §8.2 'Cross-references' gains the §5.7 + §10 cross-
reference bullet, removing the trailing 'added by T3.1
itself' note (the bullet is now present and the list is
internally consistent).
scripts/apply_schemas.py / pipelines/_tests/test_papra_index_offline.py
are not touched (the schema freeze is preserved). The
PAPRA_INDEX_TABLE env var is read by the helper but is not
required by the backup / restore procedure; the Gitea
secrets for the new backup target bucket are the only new
env vars.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
REQ-INB-21 lands the secret-rotation helper and the runbook procedure.
scripts/rotate_secrets.sh (new, ~150 lines, set -euo pipefail):
- Accepts --dry-run (default) and --apply flags plus a positional
secret name.
- Reads the new value from STDIN (never a CLI arg) to avoid ps leaks.
- Calls pipelines.common.gitea_request via a python3 subshell to PUT
the new value to Gitea's actions-secrets endpoint. The Gitea
actions-secrets PUT is in-place (overwrites the existing secret
at the same name); the helper then verifies via GET that the
secret still exists under the new value.
- Exits 0 on success, 2 on missing env / missing value, 3 on Gitea
API failure, 4 on helper-internal error.
- Dry-run prints a redacted diff (sha256:8hex) of the new value
against an unknown placeholder for the old value (Gitea does
not return secret plaintexts via the API).
pipelines/_tests/test_rotate_secrets_offline.py (new, ~3 tests):
- test_dry_run_prints_diff_and_exits_zero_no_put
- test_apply_branch_runs_python_helper
- test_helper_fails_loud_when_env_unset
The 3 tests pass on the offline test host (the apply branch test
documents that exit code 0 or 3 is acceptable in this environment:
0 on a real Gitea, 3 on connection failure with no Gitea reachable).
docs/operations-runbook.md:
- §6.1 line 401 'future enhancement' parenthetical replaced with
'The helper landed in v1.1 (REQ-INB-21); see §6.4.'
- New §6.4 'Secret rotation' between §6.1 and §6.2: rotation
ordering (NOCODB_API_TOKEN -> SEAWEEDFS_* -> IMAP_PASSWORD ->
GITEA_USER/PASS), test-or-die gate, and a copy-paste invocation
example.
.env.example is not touched (env vars are unchanged; only the
helper and the procedure are new).
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
REQ-INB-19 adds a new NocoDB table papra_index (per-message audit trail
for Papra write intents). Register it in scripts/apply_schemas.py
so 'python scripts/apply_schemas.py' creates it on first run and
is a no-op on subsequent runs (matching the existing idempotency
contract for the four module tables).
The shape is a separate AUDIT_TABLES list (parallel to MODULES),
not an entry in MODULES, because the papra_index table has its
own column set (papra_document_id, module, kind, decision_at,
outcome, error) and does not use the COMMON_COLUMNS shared by the
module tables. The MODULES list and the v1.0 module-table freeze
are preserved unchanged.
The audit-tables loop in main() mirrors the module loop: find-or-
create the table, verify the Message_ID UNIQUE constraint, add any
missing columns. --check mode verifies the new table's existence
and UNIQUE constraint.
---ci---
project: inboxos
phase: 1
milestone: v1.1
status: execute
---/ci---
After a29b971 bypassed the opencode CLI on the live path, run 674
got all the way to extract-crm on the first Substack email — and
then hit finish_reason=length on the 86KB body. Two structural
causes plus one ops papercut:
1. ollama_run_command capped max_tokens at 1024. lfm2.5:8b always
spends ~100-300 tokens on chain-of-thought reasoning, so only
~700 tokens were left for the 11-field CRM extract schema. A
richer body burns through that and the response comes back with
content='' and finish_reason='length' after the budget is gone.
- Raise default max_tokens to 4096 (env: INBOXOS_LLM_MAX_TOKENS).
- Route extract-* commands to a bigger model
(INBOXOS_LLM_MODEL_EXTRACT, default gemma4:12b) so the 12B
has more headroom. Classify stays on the cheap lfm2.5:8b.
2. The first email in run 674 was a raw HTML Amazon SES bounce.
lfm2.5:8b literally responded 'It looks like you've shared a
block of HTML...' and never emitted JSON. Strip HTML to plain
text in message_summary_for_prompt() so the LLM sees prose,
not markup. Pure stdlib (re, html.unescape); no new deps.
3. .gitea/workflows/live-pipeline.yml used actions/upload-artifact@v4
which hits GHES-only endpoints and fails on Gitea Actions with
'GHESNotSupportedError'. Pin to v3 (the v1 Gitea-compatible API).
4. The body cap in message_summary_for_prompt was 20000 chars. 2000
is enough for the 11-field schema on a 4096-token completion
budget and keeps a 86KB Substack from eating the whole run.
Override via INBOXOS_BODY_CHARS.
5. historical-pipeline.yml still uses the opencode CLI (separate
from the live bypass). Pin OPENCODE_NPM_VERSION to 1.15.13 —
1.16.x wraps every 4xx from a provider in 'UnknownError' inside
SessionPrompt.command (INB-20). Add a version-mismatch check
so a future runner image can't silently regress.
6. tests/test_live_pipeline_offline.py: cover the new
HTML-stripping + extract-routing code paths with 5 new tests.
10/10 pass locally.
7. .gitignore: ignore .tools/ (local opencode 1.15.13 install for
offline smoke tests).
8. scripts/setup_gitea_secrets.sh: set OPENCODE_NPM_VERSION secret
alongside the workflow default for parity with the historical
pipeline.
End-to-end check against the live Ollama (FRP-tunneled):
llm_run_command('inboxos:classify', QuickBooks invoice payload)
→ {lmm-2.5:8b, 4096 max} → {'modules': ['finance']}
llm_run_command('inboxos:extract-crm', Jane Doe Q3 pilot payload)
→ {gemma4:12b, 4096 max} → 11-field CRM JSON
Refs: INB-17, INB-20
Co-Authored-By: Paperclip <noreply@paperclip.ing>
opencode 1.16.2 server returns UnknownError on every `--command`
invocation (ref: err_xxx). The local-only `opencode run "what is 2+2?"`
default-agent path still works, but `SessionPrompt.command` (the path
the live pipeline uses) crashes immediately on every prompt. Diagnosis:
- Local reproduction: `opencode run --command inboxos:classify` →
UnknownError, server-side stacktrace in
`SessionPrompt.command (/$bunfs/root/chunk-...).js`.
- The Ollama server itself is healthy: `GET /v1/models` returns 7
models including `lfm2.5:8b`; `POST /v1/chat/completions` with the
`inboxos:classify` prompt body returns clean JSON.
The fix is to drop the opencode CLI from the hot path. The pipeline now
calls Ollama's OpenAI-compatible `/v1/chat/completions` endpoint
directly:
- `pipelines/common.py`: new `LLMConfig` (backend selector), new
`ollama_run_command`, new `llm_run_command` dispatcher. The legacy
`opencode_run_command` is kept for offline tests.
- `pipelines/live_pipeline.py`: switch from `opencode_run_command` /
`OpenCodeConfig` to `llm_run_command` / `LLMConfig`. The model
name is now read from the prompt frontmatter (`model: ollama/...`)
with a configurable default (`INBOXOS_LLM_MODEL`).
- `.gitea/workflows/live-pipeline.yml`: drop the "Ensure OpenCode is
installed" step; add `INBOXOS_LLM_BACKEND`, `INBOXOS_LLM_MODEL`,
`INBOXOS_LLM_TIMEOUT`, `OLLAMA_API_KEY` env vars.
- `pipelines/_tests/test_live_pipeline_offline.py`: rebind the fake
onto `lp.llm_run_command` and use `LLMConfig(backend="opencode", …)`
so the offline smoke stays green.
- `.env.example` and `scripts/setup_gitea_secrets.sh`: switch the
default S3 endpoint to `https://storage.cloudinit.dev` (MinIO) per
the platform migration in INB-16.
Local end-to-end check (single message against the live Ollama):
LLMConfig(backend="ollama", ollama_base_url="https://ollama-frp.cloudinit.dev/v1", ...)
→ ollama_run_command("inboxos:classify", {subject="QuickBooks invoice #1234", ...})
→ {"modules": ["finance"]} ✓
Offline tests: 8/8 pass. No secrets touched.
Refs: INB-17
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The live NocoDB base at psmc7y7hx3t57ry already has four tables
(pl_finance, personal_crm, daily_news, promotions) but with
column layouts that predate the current extraction prompts.
This change makes the script idempotent against that pre-existing
state:
- Map each MODULE.name to its actual table title in the base.
The MODULES list names are the pipeline routing keys
(finance/crm/intel/promotions) and stay as the JSON output keys;
the table title is the existing NocoDB label.
- Use uidt (UI Data Type) instead of dt in column-add payloads.
Sending the NocoDB UI type as dt (e.g. dt="SingleSelect" or
dt="Date") triggers SQLITE_ERROR on the underlying SQLite
ALTER TABLE; uidt is accepted and NocoDB picks the correct
storage type. Verified against SingleLineText, LongText, Email,
URL, Date, DateTime, Decimal, SingleSelect, Checkbox.
- Switch the table-detail re-fetch in main() from
/api/v2/meta/tables/{id} (returns 404 on v2026.04.x) to
/api/v1/db/meta/tables/{id}.
SingleSelect options are intentionally not pushed via the column
payload: the API path triggers SQLITE_ERROR, and NocoDB stores
arbitrary string values for select columns with no pre-defined
colOptions. The pipeline writes option names by string value, so
this is safe.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
After the board confirmed IMAP_PASSWORD is set, the remaining
Gitea Actions secrets for the live and historical pipelines
have no reasonable source of truth for the board to use. This
script takes the board's owner-scoped GITEA_TOKEN and sets every
other known-value secret in one call:
- IMAP_HOST/PORT/USER/USE_SSL/MAILBOX
- SEAWEEDFS_S3_ENDPOINT/BUCKET (keys remain operator-supplied)
- NOCODB_BASE_URL (token/project/tables remain
operator-supplied -- see INB-12)
- OLLAMA_BASE_URL
- GITEA_INSTANCE_URL
It is idempotent (re-runs update the same secrets) and lists
every secret that is still missing, so the board knows what to
source next. IMAP_PASSWORD is intentionally NOT touched (the
board already set it via the web UI; this script never sees the
plaintext).
Co-Authored-By: Paperclip <noreply@paperclip.ing>
cloudinit-bot
cto