T-HUB/docs/occupymedia-storage.md

# OccupyMedia shared files, documents and pads — phase one

**Implemented: contracts, config validation and independent compose scaffolding.**
No services, Matrix upgrade, archive bot, index API, database migration or UI have
been deployed/implemented by this phase. Existing API/media-worker storage paths
and deployment edits are preserved. [Deployment steps and resources](../infra/occupymedia/files/README.md)
are the next operator runbook. Design input: `runs/wp4/fileshare-research.md`.

## Storage ownership

| Payload bucket | Exclusive owner | Status |
|---|---|---|
| `occupymedia-matrix-media` | Future Tuwunel native S3 media | Existing bucket; live conduwuit still local |
| `hub-files-blobs` | oCIS S3NG, working files/docs and pad exports | Existing bucket (provisioned 2026-10-08); oCIS deployed 2026-10-10 on occ-files CT 320. S3 key scoped to this bucket only |
| `hub-pads` | Future compliant pad persistence | Existing bucket; unused until a compliant pad adapter exists |
| `hub-media-library` | Reserved — library copies requiring separate attribution | Existing bucket; unused |
| `occupymedia-media-encrypted` | Tactical Media Broadcast Hub media/library, existing envelope format | Existing configuration; preserve KEK |
| `occupymedia-peertube-videos` | PeerTube video objects | Existing dedicated bucket |
| `occupymedia-peertube-playlists` | PeerTube playlists | Existing dedicated bucket |

Reality note (2026-10-10): the provisioned buckets use the `hub-` prefix, not
the `occupymedia-files-blobs` name first written here. The `hub-` names are the
contract now; do not create duplicates.

Verify actual inventory; names here are ownership contracts, not proof of
provisioning. Do not let the bot write directly into oCIS's opaque bucket. It must
upload via authorized WebDAV/CS3 so metadata and ACLs remain coherent. Each
product uses its own scoped credentials; bucket prefixes are not isolation.

The target is **R2-only durable file/media payloads, minimal local metadata/config
allowed, no local payload cache by design**. POSIX oCIS metadata, Matrix
RocksDB/state, Hub metadata rows, identity/config and keys remain local and need
backups. Source files, document bodies, thumbnails, extracted full text, pad
revisions, attachments and staging/cache copies are payloads. Calling pad text
or full-text extraction a database column does not turn it into metadata.
Bounded transient RAM processing is allowed; disk/swap/core-dump payload copies
are not. This phase does not make existing local Matrix bytes disappear.

## Telegram-like group file contract

The shared TypeScript boundary is `packages/core/src/shared-files.ts`; future
HTTP/DB adapters must validate these shapes at runtime. Use stable
`workspaceId + groupId` (existing Hub groups) and immutable `roomId` to map to an
oCIS `spaceId + folderId`. Start with one restricted Space per group; allow
multiple rooms to map explicitly to that group. Never infer identity/ACL from
room names, aliases or bridge membership. Membership changes must reconcile
both Matrix and file grants; mismatches suspend sharing/mirroring until corrected.

A `GroupFile` records room/group/workspace, sender MXID (not a display name),
Matrix event ID, origin (`matrix`, provider-described `bridge`, direct `upload`,
or `pad-export`), original name, MIME, verified byte size, SHA-256, UTC ISO
`createdAt` and `indexedAt`, stable library resource/version, share link,
retention and availability. For bridged sources preserve both Matrix sender
and provider/remote event attribution when supplied by the trusted bridge;
do not invent attribution. Original `createdAt` comes from event time or accepted
upload time, not the replay/index time. Direct uploads and exports use null event
IDs until a room announcement supplies one. Hash actual bytes while streaming;
S3 ETags, claimed Matrix MIME/size and filenames are not proof. Normalize a safe
display filename and sniff MIME; reject over-limit transfers, including when
Content-Length is missing or lies.

Default links require current group membership; downloads/edit launches must
recheck ACLs and retention. Public links require an explicit group grant and
expiry; revoke them on removal. Persist library links/share IDs and expiry, not
presigned object URLs, WOPI tokens or passwords. Keep retention policy separate
from link expiry: `keep` is the phase-one default with no automated purge;
`until` requires an explicit timestamp, `legalHold` prevents deletion.
Future policy must account for source Matrix media and library copies separately;
one message redaction does not authorize deleting a blob used by another record.

## Metadata index and hub-archive seam

Future `chat_media_archive` persistence is metadata-only. Required constraints:
unique `(workspace_id, room_id, event_id)` for attachment events, stable file
identity/version, indexes on `(workspace_id, group_id, created_at, id)`,
sender, MIME class, name and checksum. List/search return only available,
unexpired rows visible through current group/file ACLs. Use a `(createdAt,id)`
cursor scoped to workspace/group. Filename/MIME/sender/date/origin queries and
oCIS metadata search are the initial searchable/retrievable media experience.
Full-text/OCR/Tika is deferred: neither a local text index nor an extra JVM is
silently included. Archive rows never contain file bodies or encrypted-file keys.

`HubArchivePort` defines future archive/revoke operations. A plain sync-loop bot
(not an appservice) uses a narrowly scoped Matrix account and a scoped oCIS
service identity. Required processing contract:

1. Load an explicitly enabled group mapping and current room encryption state.
   Exclude own bot/reply events to prevent loops. Abort on unknown state, lost
   membership/ACL, redaction, encrypted events or `content.file` attachments.
2. Accept only `m.room.message` attachments (`m.file`, `m.image`, `m.audio`,
   `m.video`) with a valid `mxc://` URI. Use authenticated Matrix media download
   endpoints (with version negotiation), never arbitrary user-supplied HTTP URLs.
   Validate any redirect to the configured R2 endpoint and drop Matrix credentials
   before following; never forward bearer auth to the object store.
3. Claim `attachmentArchiveKey(workspace,room,event)` durably. Use a deterministic
   upload destination/resource lookup and reconcile partial retries to avoid
   duplicate files across crashes. Stream bounded bytes/hash to WebDAV/CS3;
   **no temp files on local disk**. Record the committed library reference,
   verified metadata and a durable reply/outbox entry before advancing the
   stored Matrix sync cursor. Advancing past a failed event requires a durable
   retry record; R2/ACL failures must not lose attachments.
4. Reply once with an authenticated file link using a stable Matrix transaction
   ID derived from the replay key. Search is an index query; retrieve by library
   resource through current ACLs. An encrypted Tactical Media Broadcast Hub library copy, if
   needed for publishing, goes through the existing encryption path into its
   own bucket; no cross-product object-key reuse.
5. On redaction/retention or membership removal, revoke links and mark the index
   unavailable; stop serving immediately. A later audited deletion workflow
   handles library versions/trash, references, source media and legal holds.
   Backfill, retries and live sync obey the same policy and idempotency rules.

No implementation of this loop or SQL table is present in phase one; config's
`HUB_ARCHIVE_*` variables reserve its interface. `canArchiveAttachment` is a
tested pure eligibility check, not complete authorization or byte validation.

## Encryption policy

**Encrypted Matrix rooms cannot be server-side mirrored by this architecture.**
No bot encryption keys, decryption engine or silent downgrade is added. Group
admins designate a separate **unencrypted archive room**, tell members its
attachments enter the shared library, and enable its immutable room mapping
explicitly. Users may deliberately upload/share a file there; encrypted rooms
remain untouched. Treat bridged rooms the same: verify encryption state instead
of assuming the bridge made them plaintext. If an archive room becomes encrypted,
or its state cannot be established, mirroring stops. R2 can retain encrypted
Matrix media as ciphertext in a future Tuwunel deployment, but that does not
make it server-searchable or give the bot access to plaintext.

## Collaborative ODT and pad boundaries

ODT editing is **oCIS file -> collaboration WOPI host -> Collabora -> save/version
back through oCIS -> R2**. Matrix posts links and notifications; it does not edit
ODT. The editor gets short-lived, permission-scoped launch tokens and file locks.
Test simultaneous edits, reconnects, saves, versions and grant revocation; keep
Collabora's working files in transient RAM and block launch if that cannot be
guaranteed for the selected image.

`SharedPad` / `PadExportPort` allow Etherpad- or HedgeDoc-compatible group pad
links and revision exports. Pad content, revisions and attachments must be
durable R2 payloads; local storage may retain only identifiers, ACLs and session
metadata. Standard pad deployments typically persist document text/revisions
in their database: they need a separately validated R2 persistence adapter or
another compliant service. R2 attachment support or periodic exports alone is
insufficient. Consequently there is **no running pad service** in this scaffold,
and `PADS_ENABLED=false`. Approved exports become ordinary versioned oCIS
resources with checksums and group retention; live ODT collaboration remains
Collabora's responsibility. This preserves the lightweight pad option without
introducing an incompatible local payload database.