93 lines
4.3 KiB
Markdown
93 lines
4.3 KiB
Markdown
# archive-bot
|
|
|
|
Matrix → oCIS attachment archiver with encrypted export bundles (phase B).
|
|
|
|
The bot mirrors attachments from deliberately-unencrypted archive rooms into
|
|
oCIS group libraries (`Library/<workspace>/<group>/`), indexes every file
|
|
(sha256, mime class, sender, Matrix event) in `chat_media_archive`, and — when
|
|
phase B is configured — periodically packs a room's archive into an
|
|
**encrypted export bundle** uploaded to R2.
|
|
|
|
## Runtime contract
|
|
|
|
| Variable | Required | Meaning |
|
|
|---|---|---|
|
|
| `DATABASE_URL` | yes | Postgres with `chat_media_archive` |
|
|
| `MATRIX_HOMESERVER_URL` | yes | tuwunel homeserver base URL |
|
|
| `MATRIX_USER` / `MATRIX_PASSWORD` | yes | archive bot account |
|
|
| `ROOM_MAPPINGS` | yes | JSON `[{roomId, workspaceId, groupId}]` |
|
|
| `FILES_OCID_WEBDAV_URL` / `FILES_OCID_USER` / `FILES_OCID_PASSWORD` | yes | oCIS WebDAV (basic auth) |
|
|
| `ARCHIVE_BOT_HEALTH_PORT` | yes | `/healthz` listener |
|
|
| `MATRIX_MEDIA_REDIRECT_HOSTS` | no | Allowed media redirect hosts |
|
|
|
|
All phase-B variables are optional: with none set the bot behaves exactly as
|
|
before (no exporter is constructed, `/healthz` reports `export: null`).
|
|
|
|
## Phase B: encrypted export bundles
|
|
|
|
| Variable | Default | Meaning |
|
|
|---|---|---|
|
|
| `ARCHIVE_EXPORT_AGE_RECIPIENTS` | — | Comma-separated bech32 `age1…` public keys (one per admin) |
|
|
| `ARCHIVE_EXPORT_R2_ACCESS_KEY` / `ARCHIVE_EXPORT_R2_SECRET_KEY` | — | R2 S3 credentials |
|
|
| `ARCHIVE_EXPORT_R2_ENDPOINT` | — | `https://<account>.r2.cloudflarestorage.com` (https enforced) |
|
|
| `ARCHIVE_EXPORT_R2_REGION` | `auto` | R2 region |
|
|
| `ARCHIVE_EXPORT_R2_BUCKET` | `hub-matrix-media` | Target bucket |
|
|
| `ARCHIVE_EXPORT_EVERY` | `500` | Auto-export after N newly archived attachments per room (`0` = manual only) |
|
|
| `ARCHIVE_EXPORT_TMP_DIR` | `$TMPDIR/archive-bot-export` | Ciphertext spill dir (see below) |
|
|
| `ARCHIVE_EXPORT_ENABLED` | — | Set `false` to force-disable despite other variables |
|
|
|
|
### What gets encrypted
|
|
|
|
A bundle is `tar(zstd(age(...)))` containing:
|
|
|
|
- `manifest.json` — room identity, export time, recipients, one metadata row
|
|
per archived file (event id, names, mime, size, **sha256**, sender, timestamp,
|
|
availability). Payload bytes are never in the manifest.
|
|
- `files/<eventId>/<safeName>` — original payload bytes, one entry per
|
|
*available* archive row (redacted rows keep their manifest row but no bytes).
|
|
|
|
The age file key is wrapped for every configured recipient, so any admin
|
|
identity opens a bundle. **Private keys never exist on the server**: generate
|
|
keypairs on admin machines with `age-keygen`, publish only the `age1…` lines in
|
|
`ARCHIVE_EXPORT_AGE_RECIPIENTS`.
|
|
|
|
### Triggers
|
|
|
|
- **Manual**: `!archive export` in a mapped room, from a sender with power
|
|
level ≥ 50 (`m.room.power_levels`). The bot posts the ciphertext object key
|
|
back as a notice.
|
|
- **Rolling**: every `ARCHIVE_EXPORT_EVERY` newly archived attachments in a room.
|
|
|
|
### The spill file (single documented disk touch)
|
|
|
|
The final object length is only known once zstd+age complete, so
|
|
`R2ExportStore` streams the ciphertext to **one** temp file (mode 0600,
|
|
O_EXCL, random name) and PUTs it with an explicit `Content-Length`, then
|
|
unlinks it in `finally`. It holds **ciphertext only** — never payload bytes —
|
|
and a failed PUT leaves nothing behind. Mount `ARCHIVE_EXPORT_TMP_DIR` on
|
|
tmpfs if any disk write is unacceptable.
|
|
|
|
### Encryption-state linter
|
|
|
|
On startup the bot walks every joined room and WARNs on any **unmapped,
|
|
unencrypted** room — an encrypted-by-default policy violation. Mapped rooms
|
|
are expected to be unencrypted (that is the archive policy); an encrypted
|
|
mapped room is logged INFO (the processor already refuses to mirror it).
|
|
The last report is surfaced in `/healthz` under `encryption`.
|
|
|
|
## Restoring a bundle
|
|
|
|
```sh
|
|
node dist/restore.js --bundle <file.tar.zst.age> --identity <age-identity-file> --out <dir>
|
|
```
|
|
|
|
The identity file holds `AGE-SECRET-KEY-1…` lines (`#` comments allowed).
|
|
Every restored file is sha256-verified against `manifest.json`; exit code is
|
|
non-zero unless **all** files verify.
|
|
|
|
## Operations
|
|
|
|
- `/healthz` — rooms, cursor, pending outbox, last export object key,
|
|
encryption lint state.
|
|
- Egress: WebDAV (443) + homeserver + R2 endpoint only; the R2 credential is
|
|
PUT-scoped by policy — this bot never lists, reads or deletes objects.
|