Track the files scaffold; sync deployed oCIS compose requirements
infra/occupymedia/files/ was gitignored in phase one ('nothing deployed');
the oCIS deployment is now live, and the storage doc + DEPLOY.md link to its
runbook. Track the secret-free scaffold (compose.yml, csp.yaml, README,
tuwunel example, .env.example); keep only .env + secrets/ ignored.
compose.yml carries the deployment-proven fixes: ocis runs as user 1000 with
OCIS_CONFIG_DIR=/etc/ocis and HOME=/var/lib/ocis (source-built images do not
inherit the official image's conventions), and 9200/9980 publish on the LAN
for the CT150 tunnel connector.
This commit is contained in:
parent
fc5f94400f
commit
6cdffa60b2
6 changed files with 382 additions and 1 deletions
183
infra/occupymedia/files/README.md
Normal file
183
infra/occupymedia/files/README.md
Normal file
|
|
@ -0,0 +1,183 @@
|
|||
# OccupyMedia files/docs foundation
|
||||
|
||||
Status: **scaffolding only; nothing deployed**. This is a new `occ-files`
|
||||
compose project. Do not merge it into `occ-chat`, `occ-app`, or the historical
|
||||
one-box compose. Live chat remains conduwuit v0.5.0; this phase does not switch
|
||||
its image, config, database or volumes. See [storage architecture and contracts](../../../docs/occupymedia-storage.md).
|
||||
|
||||
## Payload boundary
|
||||
|
||||
R2 is the only durable destination for file/media payloads. oCIS S3NG requires
|
||||
local POSIX metadata; Matrix requires local state. These are backup-critical
|
||||
exceptions, not permission to store file bodies, thumbnails, pad revisions or
|
||||
document caches on Proxmox disks. Transient processing uses bounded RAM/tmpfs.
|
||||
Disable swap for those workloads (including host/CT swap) and payload-bearing
|
||||
core dumps. An R2 outage must fail writes, not fall back to disk.
|
||||
|
||||
The compose masks the expected upload staging path with tmpfs and omits the
|
||||
thumbnail service. This is a **candidate configuration**, not proof that every
|
||||
path in an arbitrary upstream image complies. Audit actual paths and upload
|
||||
behavior for the pinned releases before admitting any user data. tmpfs upload
|
||||
sessions disappear on restart and must be retried. Start with a 20 MiB request
|
||||
cap enforced by the ingress and one upload at a time; large media goes through
|
||||
the existing dedicated media/video plane.
|
||||
|
||||
## Configuration
|
||||
|
||||
`.env.example` deliberately leaves images, account endpoint and secrets empty.
|
||||
Select compatible release digests; do not use `latest` or assume the research's
|
||||
version availability is sufficient validation. Use the same oCIS image for both
|
||||
oCIS and collaboration. Confirm S3NG signing/path-style compatibility against
|
||||
R2 with that release. The native `STORAGE_USERS_S3NG_*` settings explicitly select
|
||||
the exclusive files bucket. `files-config.ts` is an opt-in validation interface,
|
||||
not startup wiring; a later adapter must invoke it and enforce the same checks.
|
||||
Verify the bucket inventory manually before provisioning: `FILES_RESERVED_BUCKETS`
|
||||
must include every other product's actual bucket names, including both PeerTube
|
||||
buckets. Never change live app bucket names or its KEK as part of this phase.
|
||||
|
||||
Source-built images (Apache-2.0): the repo Dockerfile build runs as root and
|
||||
defaults its config search to `$HOME/.ocis` — it does NOT inherit the official
|
||||
image's conventions. The compose therefore pins `user: "1000"`,
|
||||
`HOME: /var/lib/ocis`, and `OCIS_CONFIG_DIR: /etc/ocis` for the ocis service,
|
||||
and publishes 9200/9980 on the LAN (no WAN forward exists; the CT150 tunnel
|
||||
connector reaches occ-files over the bridge). Do not remove these when
|
||||
swapping images.
|
||||
|
||||
Use the built-in oCIS identity provider for an isolated first deployment.
|
||||
Do not point its issuer at chat: current conduwuit is not the proposed OIDC
|
||||
provider. External OIDC and account mapping need a separate design/test pass.
|
||||
|
||||
For CODE 26+, generate its WOPI proof key privately with the selected image's
|
||||
upstream procedure, store it outside the repository, and set
|
||||
`COLLABORA_PROOF_KEY_FILE` to an existing regular file. Keep WOPI proof
|
||||
verification enabled. The bind mount never creates a directory on a missing
|
||||
file path. Review required ownership/permissions for the selected images,
|
||||
including tmpfs/jail and oCIS UID/GID (scaffold assumes 1000).
|
||||
|
||||
## Exact next deployment steps (operator work; not executed in phase one)
|
||||
|
||||
1. Inventory the existing buckets and CTs. Reserve a **new** files CT/project
|
||||
with **8 GiB RAM / 4 vCPU** for the complete docs target; keep API and PeerTube
|
||||
in their current separate CTs. Provide an XFS/ZFS metadata filesystem with
|
||||
working xattrs, initial 10–20 GiB capacity, monitoring and off-host backups.
|
||||
Do not reuse existing Matrix volumes. Allocate/verify no swap and no payload
|
||||
dumps before user uploads.
|
||||
2. Verify/create `occupymedia-files-blobs` if absent. Keep it private and exclusive
|
||||
to oCIS. Provision a bucket-scoped read/write key just for files; Matrix and
|
||||
AnarchoPoster credentials remain separate. Verify lifecycle rules cannot
|
||||
silently expire active opaque oCIS blobs. Define a separate backup account
|
||||
and scheduled copy/restore job; durability alone is not a backup.
|
||||
3. Copy the scaffold to the new CT. Run `cp .env.example .env` and `chmod 600 .env`
|
||||
in that directory. Fill real private credentials, endpoint and approved
|
||||
digest-pinned images, public URLs/hosts, independent random admin/JWT/WOPI
|
||||
secrets and proof-key path. Back up secrets in the password manager. Keep
|
||||
generated `/etc/ocis/ocis.yaml` and all keys out of git. No secrets are supplied
|
||||
by this repository.
|
||||
4. Validate without launching containers:
|
||||
|
||||
```bash
|
||||
cd infra/occupymedia/files
|
||||
docker compose --env-file .env -f compose.yml --profile docs config --quiet
|
||||
```
|
||||
|
||||
This checks syntax/required variables only. Compare the files bucket against
|
||||
Matrix, AnarchoPoster and PeerTube names; verify HTTPS account R2 endpoint,
|
||||
host/URL agreement, image digests and private key-file ownership. Do not print
|
||||
rendered config or `docker inspect` into shared logs (they contain secrets).
|
||||
5. On the new project only, initialize once:
|
||||
|
||||
```bash
|
||||
docker compose --env-file .env -f compose.yml run --rm ocis init
|
||||
docker compose --env-file .env -f compose.yml up -d ocis
|
||||
```
|
||||
|
||||
Protect initialization output (it may contain generated credentials). Do not
|
||||
use an `init || true` startup loop; inspect unexpected init errors. Verify the
|
||||
persisted generated config contains no local user-blob driver override.
|
||||
6. Route `files.occupymedia.cc` through the existing Tunnel to the **new CT's**
|
||||
`127.0.0.1:9200`, and `office.occupymedia.cc` to `127.0.0.1:9980`. These binds
|
||||
require a Tunnel connector on that CT, or an explicit authenticated private
|
||||
relay; the app CT connector cannot reach another CT's loopback. Do not expose
|
||||
these ports publicly. Preserve HTTPS origin/Host headers, WebSockets, encoded
|
||||
WOPI URLs and adequate WebDAV/TUS timeouts. Test actual Cloudflare request-size
|
||||
limits and chunking; Tunnel does not automatically remove those limits.
|
||||
7. Before opening docs, start the optional pair:
|
||||
|
||||
```bash
|
||||
docker compose --env-file .env -f compose.yml --profile docs up -d collaboration collabora
|
||||
```
|
||||
|
||||
oCIS Web permits only the configured office frame origin via `csp.yaml`.
|
||||
Collaboration registers with oCIS over private NATS/CS3 and exposes WOPI only
|
||||
on the compose network. Collabora uses `http://collaboration:9300` as its
|
||||
allowlisted WOPI source; the browser uses HTTPS office URLs. Verify discovery,
|
||||
proof keys, locks, read/write grants and CSP. Do not bypass proof checks or
|
||||
disable TLS verification to make a failed launch appear successful.
|
||||
8. Admission rehearsal: upload a disposable ODT/image, interrupt a TUS upload,
|
||||
restart services, retrieve and checksum the saved object, then co-edit an ODT
|
||||
with two authorized users and reload it after closing both clients. Observe
|
||||
uploads, jail/temp paths, writable container layers, system storage (including
|
||||
avatars), and metadata volumes: **no payload may persist outside R2**. Inspect
|
||||
all release-specific spill/cache paths; mask/disable them or block deployment
|
||||
if unresolved. Disable avatar/system-file features if they store payloads
|
||||
locally. Check R2 failure returns an error, revoked users cannot fetch files,
|
||||
tmpfs exhaustion fails safely, and a backup restores an independent project.
|
||||
Production is blocked until these checks pass.
|
||||
9. Create restricted oCIS Spaces and explicit group/room mappings. Implement the
|
||||
`HubArchivePort` sync-loop adapter + metadata index described in the contract
|
||||
before enabling mirroring. No bot image, API route or schema migration exists
|
||||
in this phase. Reserve an unencrypted archive room per group; require explicit
|
||||
membership/grant reconciliation. Keep encrypted rooms excluded.
|
||||
10. Rehearse Tuwunel separately with fresh state and disposable media using
|
||||
`tuwunel-r2.toml.example` merged into the selected release's full config.
|
||||
Privately supply `endpoint`, `key`, `secret`; the example deliberately lacks
|
||||
usable credentials. Retain the deployed server name for any eventual live
|
||||
upgrade, but do not connect the rehearsal to live federation/volumes. A future
|
||||
authorized migration needs snapshots, compatibility checks, rollback and a
|
||||
media-copy plan. This phase authorizes none of those live changes.
|
||||
11. Add a compliant pad adapter only after it can durably save **text, revisions,
|
||||
attachments and exports** to R2. Leave `PADS_ENABLED=false` until then. An
|
||||
ordinary local Postgres pad deployment plus periodic R2 exports does not
|
||||
satisfy this requirement. Add optional Tika only after extraction spill paths
|
||||
and the policy for storing derived document text have been resolved.
|
||||
|
||||
## Resources and recovery
|
||||
|
||||
| New files/docs component | Planning RAM / CPU |
|
||||
|---|---|
|
||||
| oCIS incl. bounded upload scratch | 0.5–1.5 GiB / 1 vCPU cap |
|
||||
| WOPI collaboration | 0.1–0.25 GiB / 0.25 vCPU cap |
|
||||
| Collabora incl. tmpfs jail/temp | 1–2.5 GiB / 1.5 vCPU cap |
|
||||
| Future hub-archive | 0.1–0.25 GiB / 0.25 vCPU allowance |
|
||||
| Optional future Tika | 0.5–0.75 GiB / 0.5 vCPU allowance |
|
||||
| Future pad adapter | 0.2–0.5 GiB / 0.25 vCPU allowance |
|
||||
|
||||
Current compose caps total **4.25 GiB / 2.75 vCPU**. Reserve **8 GiB / 4 vCPU**
|
||||
for the complete docs target and OS/headroom; actual Collabora concurrency must
|
||||
be measured. A 4 GiB CT is only a reduced files pilot, not this complete stack.
|
||||
RAM/tmpfs counts toward limits. The 20 MiB/one-transfer initial limit is deliberate;
|
||||
large originals and multiple active ODT sessions require measurements before
|
||||
raising it. No pad, bot or Tika RAM is being consumed by this scaffold.
|
||||
|
||||
Daily RPO target: consistent oCIS metadata/config snapshots preserving xattrs,
|
||||
symlinks and ownership **plus** independent R2 object backups; store a manifest
|
||||
linking the snapshot and object-copy checkpoints. Back up generated identity
|
||||
and signing configuration and the Collabora proof key on change. Later add Hub
|
||||
DB index/mapping/outbox/sync-cursor dumps, and Matrix state snapshots via its
|
||||
supported backup method. Protect the AnarchoPoster KEK separately. Restore-test
|
||||
monthly in isolated projects; ensure metadata references resolve before reopening
|
||||
access. R2 alone cannot reconstruct the oCIS directory/ACL tree. Do not back up
|
||||
tmpfs scratch or document jails to local storage. Retention/deletion actions must
|
||||
be coordinated across copies, trash/versions and backup expiry, with legal holds.
|
||||
|
||||
## Upstream verification
|
||||
|
||||
The research is design input, not executable deployment instructions. Settings
|
||||
were checked against [oCIS storage-users](https://doc.owncloud.com/ocis/8.2/admin/deployment/services/s-list/storage-users.html),
|
||||
[storage drivers](https://owncloud.dev/ocis/storage/storagedrivers/),
|
||||
[collaboration](https://doc.owncloud.com/ocis/8.2/admin/deployment/services/s-list/collaboration.html),
|
||||
[the office compose example](https://github.com/owncloud/ocis/blob/master/deployments/examples/ocis_full/collabora.yml),
|
||||
and [Tuwunel's config reference](https://github.com/matrix-construct/tuwunel/blob/main/tuwunel-example.toml).
|
||||
Recheck the exact pinned release before running it; current upstream examples
|
||||
may differ (notably CODE 26 proof-key initialization). The research's mixed local/R2
|
||||
Matrix provider sketch is an eventual transition design, not an R2-only end state.
|
||||
Loading…
Add table
Add a link
Reference in a new issue