cloudinit-bot
a8ec7b80de
---ci--- project: inboxos phase: 1 milestone: v1.4 status: execute wave: 2 tasks: [T3, T4, T5, T6, T7, T8] ---/ci---
52 lines
3.6 KiB
Markdown
52 lines
3.6 KiB
Markdown
# InboxOS
|
|
|
|
**Self-hosted personal ERP / CRM pipeline.** A single Gmail inbox, polled over IMAP, archived to SeaweedFS, classified and extracted by a local LLM (Ollama), and written to NocoDB as structured records. Curated attachments are mirrored to Papra. No third-party SaaS touches PII.
|
|
|
|
## Where to start
|
|
|
|
- **30-minute walkthrough** — [`docs/operator-tutorial.md`](docs/operator-tutorial.md) (REQ-INB-53). Guided tour for the new operator: setup → first run → day-to-day → incident response → hardening.
|
|
- **Operations runbook** — [`docs/operations-runbook.md`](docs/operations-runbook.md). System overview, service inventory, workflow operations, observability, failure modes, security, cost, glossary. The first thing to read on day 1 of incident response.
|
|
- **Extraction contract** — [`docs/extraction-contract.md`](docs/extraction-contract.md). How a raw email becomes a NocoDB record. Source of truth for the `inboxos:classify` + `inboxos:extract-<module>` prompt family.
|
|
- **NocoDB schema** — [`docs/schemas.md`](docs/schemas.md). The four module tables (`finance`, `crm`, `intel`, `promotions`) and the Message-ID idempotency contract.
|
|
- **Gitea Actions setup** — [`docs/gitea-actions.md`](docs/gitea-actions.md). Runner provisioning (including the second runner for live + historical parallelism), repo secrets, the Basic Auth model.
|
|
- **Pipelines** — [`pipelines/README.md`](pipelines/README.md). Flow diagrams, idempotency model, rollback procedure for the live and historical pipelines.
|
|
- **Papra write policy** — [`docs/papra-write-policy.md`](docs/papra-write-policy.md). When the pipeline writes to Papra (attachments + selected bodies), and why the first ship is dry-run.
|
|
|
|
## Quick layout
|
|
|
|
```
|
|
inboxos/
|
|
├── README.md # this file
|
|
├── .env.example # placeholder env (no real secrets)
|
|
├── docker/
|
|
│ └── docker-compose.yml # self-hosted stack: gitea, runners, nocodb, seaweedfs, papra, ollama
|
|
├── .gitea/workflows/
|
|
│ ├── live-pipeline.yml # cron */5 * * * *, polls IMAP
|
|
│ ├── historical-pipeline.yml # workflow_dispatch, streams .mbox
|
|
│ └── ci-pr.yml # fast-tests + 100MB e2e + live Ollama smoke
|
|
├── pipelines/ # shared Python: common.py, live_pipeline.py, historical_pipeline.py, mbox_stream.py
|
|
├── .opencode/commands/ # version-controlled extraction prompts (prompt-as-code)
|
|
├── config/ # n8n config (legacy, no longer in use)
|
|
└── docs/ # this runbook + schemas + contracts + reports
|
|
```
|
|
|
|
## Status
|
|
|
|
Phase 5 — Full Deployment. Parent goal: [INB-1](/INB/issues/INB-1). Current phase issue: [INB-7](/INB/issues/INB-7). The runbook in `docs/operations-runbook.md` is the canonical reference for ongoing operation.
|
|
|
|
## Historical archive (Google Takeout .mbox)
|
|
|
|
The full Google Takeout `.mbox` export lives in SeaweedFS at the canonical S3 URI:
|
|
|
|
```
|
|
s3://inboxos/eml/historical/all-mail.mbox
|
|
```
|
|
|
|
That is the value to pass as the `mbox_object_key` workflow_dispatch input to `historical-pipeline.yml`. To verify it exists, run from any host with `mc` configured against the `inboxos` bucket:
|
|
|
|
```bash
|
|
mc ls --recursive inboxos/inboxos/ | grep '\.mbox$'
|
|
```
|
|
|
|
The historical pipeline streams this object message-by-message, UPSERTs each into NocoDB on `Message-ID`, and writes the raw `.eml` backup under `s3://inboxos/eml/historical/<yyyy>/<mm>/<dd>/<Message-ID>.eml`. See [`docs/operations-runbook.md` §3.2](docs/operations-runbook.md) for the dispatch recipe and §4.4 for the full bucket layout.
|