docs: producer & media ingest, ops changes, library updates (October 2026)

- engineering-guide §10: media attachments, alt-text delivery, producer,
  audio-less silence bed, chunked ingest, upload caps
- admin-guide §10: oCIS retired from the payload path, storage_key (041),
  publishing media resolver + dedupe origins, produce-worker, api restart
  rate-limit note; §6.3 marked legacy
- user-guide §9: 2 GiB Library uploads, compose attachments + alt text,
  producer, chunked large-video uploads
This commit is contained in:
glass beadian 2026-10-16 02:33:46 +00:00
parent 2d881f5e41
commit b3c2085436
3 changed files with 62 additions and 4 deletions

View file

@ -128,6 +128,7 @@ Operational notes:
- Every switch is idempotent (`Idempotency-Key`) and lands in the audit chain as `director.<operation>` — incident review replays commands from the audit log. - Every switch is idempotent (`Idempotency-Key`) and lands in the audit chain as `director.<operation>` — incident review replays commands from the audit log.
## 6.3 Files SSO (Hub as the OIDC issuer for oCIS) ## 6.3 Files SSO (Hub as the OIDC issuer for oCIS)
> **Retired (October 2026):** oCIS is out of the payload path and the in-app Drive view is gone — see §10.1. This section remains only as reference for the legacy `files.occupymedia.cc` host.
`files.occupymedia.cc` signs in through the Hub: the issuer is `https://api.occupymedia.cc/oidc`, client id `web`, PKCE only, auto-consent for that first-party client. Phase 2 (migration 034) makes sessions revocable and adds rotating refresh tokens bound to the Hub session — Hub logout reaches Drive within one access-token TTL (≤1 h). Operational notes: `files.occupymedia.cc` signs in through the Hub: the issuer is `https://api.occupymedia.cc/oidc`, client id `web`, PKCE only, auto-consent for that first-party client. Phase 2 (migration 034) makes sessions revocable and adds rotating refresh tokens bound to the Hub session — Hub logout reaches Drive within one access-token TTL (≤1 h). Operational notes:
- `OIDC_KEY_DIR` must be a persistent volume (the API generates and seals its ES256 keyset there on first boot; losing it logs every Drive session out). - `OIDC_KEY_DIR` must be a persistent volume (the API generates and seals its ES256 keyset there on first boot; losing it logs every Drive session out).
@ -172,7 +173,29 @@ The governance sim is re-runnable (all fixtures prefixed `e2e-gov-`) and writes
| MinIO image pull denied | registry degradation | retry later or point `S3_ENDPOINT` at any S3 | | MinIO image pull denied | registry degradation | retry later or point `S3_ENDPOINT` at any S3 |
| Streams columns empty in UI | mock-registry data (API not wired yet) | expected pre-WP3-integration | | Streams columns empty in UI | mock-registry data (API not wired yet) | expected pre-WP3-integration |
## 10. Upgrading ## 10. Operations changes (October 2026)
### 10.1 oCIS retired from the payload path
- **All media** — Files tab, video, submissions, media posts, produced files — now stores in the **R2 bucket** (`S3_*` env, `S3_KEK` envelope encryption). oCIS holds no payloads.
- `files.occupymedia.cc` / the **Drive tab** is gone from the web UI.
- `192.168.13.210` is no longer in the serving path and may be decommissioned.
### 10.2 Payload pointer (migration 041)
- `chat_media_archive.storage_key` is the **authoritative** payload pointer.
- `ocis_resource_id` remains only as a legacy column — do not build against it.
### 10.3 Publishing media
- The scheduler resolves media at dispatch (**PublishingMediaResolver**: RLS workspace lookup + R2 download + SPENC1 open).
- `MASTODON_PUBLISH_DEDUPE_ORIGINS=https://social.occupymedia.cc` **must stay set**. Media posts are two provider mutations (media upload + status create) and the dispatch gate refuses the second without a verified dedupe origin.
### 10.4 Produce worker
- Runs as a **separate container** (`produce-worker`) and needs **ffmpeg + a font** — `DejaVuSans-Bold` at `/usr/share/fonts/truetype/dejavu/`.
- `MEDIA_DEBUG_STDERR=1` routes ffmpeg stderr to `/tmp/ffmpeg-debug.log` for debugging.
### 10.5 Known ops note
- Restarting the api container repeatedly can trip the homeserver login rate limit (`M_LIMIT_EXCEEDED`) and crash bootstrap — wait **~2–3 min** between api restarts.
## 11. Upgrading
1. Back up DB + confirm KEK backups (§4.3). 1. Back up DB + confirm KEK backups (§4.3).
2. `git pull && pnpm install`. 2. `git pull && pnpm install`.

View file

@ -162,7 +162,35 @@ Shipped and wired end-to-end:
Still planned: media-engine graph rewiring beyond the current FFmpeg-normalize path, LLM video AI (design only), content search rollout. Still planned: media-engine graph rewiring beyond the current FFmpeg-normalize path, LLM video AI (design only), content search rollout.
## 10. Key file map ## 10. Producer & media ingest (October 2026)
### 10.1 Media attachments for publishing
- `POST /media` (multipart `file`); content type is **magic-byte sniffed** (jpeg/png/webp/mp4/webm) — the extension is never trusted.
- Caps: image **10 MiB**, video **64 MiB**, and at most **4 attachments per post** (`MAX_MEDIA_PER_POST`).
- Read back via `GET /media/:id` (auth-scoped). Bytes live in R2 object storage — SPENC1 envelope-encrypted, content-addressed `sha256` keys (§3.8 scheme).
### 10.2 Alt texts
- Drafts carry `altTexts` positionally aligned with `mediaAssetIds` (0–4 entries; `null` = no alt text for that slot).
- At publish, the scheduler maps them into `providerMetadata.altText` (asset-id keyed); Mastodon media uploads receive them as descriptions.
### 10.3 Video producer
- `POST /video/:id/produce` accepts `{ overlays?, voiceover?, outputLabel?, lane? }` — closed enums with a strict key allowlist; unknown keys are rejected. Overlay kinds: `lower_third | logo | clock`; overlay text runs through the filtergraph-metacharacter blocklist before it reaches an FFmpeg graph.
- Jobs execute in the **produce-worker** (its own container, `MEDIA_WORKER_LANE=local`): claim/lease via polling with `SKIP LOCKED`; stale leases are re-queued.
- The output is a **derived library asset** (`derived_from`), publishable like any other asset.
### 10.4 Audio-less sources
- The command-builder synthesizes an `anullsrc` silence bed — configured at the output preset's rate/layout — instead of emitting `[0:a]` maps that ffmpeg rejects.
### 10.5 Chunked video ingest
- `POST /video/upload/init` → `PUT /video/upload/:uploadId/chunk/:index` → `POST /video/upload/:uploadId/complete`.
- 10 MiB chunks (`CHUNK_SIZE_BYTES`) sidestep Cloudflare's ~100 MB proxy body cap. The server reassembles, seals, stores to R2, then runs the normal probe pipeline.
- Max chunked upload: **100 GiB** (`MAX_CHUNKED_UPLOAD_BYTES`).
### 10.6 Upload caps
- Whole-body video upload cap: **2 GiB** (`MAX_VIDEO_UPLOAD_BYTES`).
- Files group upload cap: **2 GiB** (was 20 MiB).
## 11. Key file map
| Concern | Path | | Concern | Path |
|---|---| |---|---|

View file

@ -59,7 +59,14 @@ Each account card shows: platform, handle, **capability chips** (what this conne
Every meaningful action (post published, role changed, connection added, credential rotated, approval granted…) is recorded in a **tamper-evident hash chain**. Filter by actor or action; click a row to open the **diff viewer** — what changed, field by field, with removed/added values. Secret values always show as `[REDACTED]`: keys and tokens are never written into the audit log, by construction. Export to CSV for compliance. Every meaningful action (post published, role changed, connection added, credential rotated, approval granted…) is recorded in a **tamper-evident hash chain**. Filter by actor or action; click a row to open the **diff viewer** — what changed, field by field, with removed/added values. Secret values always show as `[REDACTED]`: keys and tokens are never written into the audit log, by construction. Export to CSV for compliance.
## 9. What's real vs. coming ## 9. New in the Library & Publishing
- **Library (Files) tab:** upload up to **2 GiB per file**. Files are encrypted and stored in the hub's own object storage. The old 20 MiB limit and the separate Drive tab are gone — the Library replaces both.
- **Compose:** attach up to **4 images or videos** per post (10 MiB per image, 64 MiB per video) and add **alt text** descriptions — they appear on Mastodon.
- **Video producer:** apply **lower-third, logo, or clock overlays** and **voiceover** to any library video. Clips with no audio get a synthesized silence bed, so they always work. Results land back in the library and can be published like any other asset.
- **Large videos:** uploads are automatically split into 10 MB chunks — big files work even through Cloudflare.
## 10. What's real vs. coming
**Now (Level 1):** monitoring streams, compose with previews, approvals, scheduling, publishing pipeline, encrypted credential vault, audit trail. *(The screens currently run on representative sample data while the backend wiring lands.)* **Now (Level 1):** monitoring streams, compose with previews, approvals, scheduling, publishing pipeline, encrypted credential vault, audit trail. *(The screens currently run on representative sample data while the backend wiring lands.)*
@ -68,7 +75,7 @@ Every meaningful action (post published, role changed, connection added, credent
- **Crosspost** — one action moves content across platforms with your commentary (an Instagram image quoted into Mastodon, a Mastodon video into Facebook), flowing through approvals. - **Crosspost** — one action moves content across platforms with your commentary (an Instagram image quoted into Mastodon, a Mastodon video into Facebook), flowing through approvals.
- **Level 2** — video uploads, livestreams, the Virtual Channel / Auto Director master-control system, and the operator Control Room. - **Level 2** — video uploads, livestreams, the Virtual Channel / Auto Director master-control system, and the operator Control Room.
## 10. FAQ ## 11. FAQ
**Why can't I retweet on some columns?** That platform's API doesn't expose resharing. We render only real capabilities. **Why can't I retweet on some columns?** That platform's API doesn't expose resharing. We render only real capabilities.