docs: producer & media ingest, ops changes, library updates (October 2026)
- engineering-guide §10: media attachments, alt-text delivery, producer, audio-less silence bed, chunked ingest, upload caps - admin-guide §10: oCIS retired from the payload path, storage_key (041), publishing media resolver + dedupe origins, produce-worker, api restart rate-limit note; §6.3 marked legacy - user-guide §9: 2 GiB Library uploads, compose attachments + alt text, producer, chunked large-video uploads
This commit is contained in:
parent
2d881f5e41
commit
b3c2085436
3 changed files with 62 additions and 4 deletions
|
|
@ -128,6 +128,7 @@ Operational notes:
|
|||
- Every switch is idempotent (`Idempotency-Key`) and lands in the audit chain as `director.<operation>` — incident review replays commands from the audit log.
|
||||
|
||||
## 6.3 Files SSO (Hub as the OIDC issuer for oCIS)
|
||||
> **Retired (October 2026):** oCIS is out of the payload path and the in-app Drive view is gone — see §10.1. This section remains only as reference for the legacy `files.occupymedia.cc` host.
|
||||
`files.occupymedia.cc` signs in through the Hub: the issuer is `https://api.occupymedia.cc/oidc`, client id `web`, PKCE only, auto-consent for that first-party client. Phase 2 (migration 034) makes sessions revocable and adds rotating refresh tokens bound to the Hub session — Hub logout reaches Drive within one access-token TTL (≤1 h). Operational notes:
|
||||
|
||||
- `OIDC_KEY_DIR` must be a persistent volume (the API generates and seals its ES256 keyset there on first boot; losing it logs every Drive session out).
|
||||
|
|
@ -172,7 +173,29 @@ The governance sim is re-runnable (all fixtures prefixed `e2e-gov-`) and writes
|
|||
| MinIO image pull denied | registry degradation | retry later or point `S3_ENDPOINT` at any S3 |
|
||||
| Streams columns empty in UI | mock-registry data (API not wired yet) | expected pre-WP3-integration |
|
||||
|
||||
## 10. Upgrading
|
||||
## 10. Operations changes (October 2026)
|
||||
|
||||
### 10.1 oCIS retired from the payload path
|
||||
- **All media** — Files tab, video, submissions, media posts, produced files — now stores in the **R2 bucket** (`S3_*` env, `S3_KEK` envelope encryption). oCIS holds no payloads.
|
||||
- `files.occupymedia.cc` / the **Drive tab** is gone from the web UI.
|
||||
- `192.168.13.210` is no longer in the serving path and may be decommissioned.
|
||||
|
||||
### 10.2 Payload pointer (migration 041)
|
||||
- `chat_media_archive.storage_key` is the **authoritative** payload pointer.
|
||||
- `ocis_resource_id` remains only as a legacy column — do not build against it.
|
||||
|
||||
### 10.3 Publishing media
|
||||
- The scheduler resolves media at dispatch (**PublishingMediaResolver**: RLS workspace lookup + R2 download + SPENC1 open).
|
||||
- `MASTODON_PUBLISH_DEDUPE_ORIGINS=https://social.occupymedia.cc` **must stay set**. Media posts are two provider mutations (media upload + status create) and the dispatch gate refuses the second without a verified dedupe origin.
|
||||
|
||||
### 10.4 Produce worker
|
||||
- Runs as a **separate container** (`produce-worker`) and needs **ffmpeg + a font** — `DejaVuSans-Bold` at `/usr/share/fonts/truetype/dejavu/`.
|
||||
- `MEDIA_DEBUG_STDERR=1` routes ffmpeg stderr to `/tmp/ffmpeg-debug.log` for debugging.
|
||||
|
||||
### 10.5 Known ops note
|
||||
- Restarting the api container repeatedly can trip the homeserver login rate limit (`M_LIMIT_EXCEEDED`) and crash bootstrap — wait **~2–3 min** between api restarts.
|
||||
|
||||
## 11. Upgrading
|
||||
|
||||
1. Back up DB + confirm KEK backups (§4.3).
|
||||
2. `git pull && pnpm install`.
|
||||
|
|
|
|||
|
|
@ -162,7 +162,35 @@ Shipped and wired end-to-end:
|
|||
|
||||
Still planned: media-engine graph rewiring beyond the current FFmpeg-normalize path, LLM video AI (design only), content search rollout.
|
||||
|
||||
## 10. Key file map
|
||||
## 10. Producer & media ingest (October 2026)
|
||||
|
||||
### 10.1 Media attachments for publishing
|
||||
- `POST /media` (multipart `file`); content type is **magic-byte sniffed** (jpeg/png/webp/mp4/webm) — the extension is never trusted.
|
||||
- Caps: image **10 MiB**, video **64 MiB**, and at most **4 attachments per post** (`MAX_MEDIA_PER_POST`).
|
||||
- Read back via `GET /media/:id` (auth-scoped). Bytes live in R2 object storage — SPENC1 envelope-encrypted, content-addressed `sha256` keys (§3.8 scheme).
|
||||
|
||||
### 10.2 Alt texts
|
||||
- Drafts carry `altTexts` positionally aligned with `mediaAssetIds` (0–4 entries; `null` = no alt text for that slot).
|
||||
- At publish, the scheduler maps them into `providerMetadata.altText` (asset-id keyed); Mastodon media uploads receive them as descriptions.
|
||||
|
||||
### 10.3 Video producer
|
||||
- `POST /video/:id/produce` accepts `{ overlays?, voiceover?, outputLabel?, lane? }` — closed enums with a strict key allowlist; unknown keys are rejected. Overlay kinds: `lower_third | logo | clock`; overlay text runs through the filtergraph-metacharacter blocklist before it reaches an FFmpeg graph.
|
||||
- Jobs execute in the **produce-worker** (its own container, `MEDIA_WORKER_LANE=local`): claim/lease via polling with `SKIP LOCKED`; stale leases are re-queued.
|
||||
- The output is a **derived library asset** (`derived_from`), publishable like any other asset.
|
||||
|
||||
### 10.4 Audio-less sources
|
||||
- The command-builder synthesizes an `anullsrc` silence bed — configured at the output preset's rate/layout — instead of emitting `[0:a]` maps that ffmpeg rejects.
|
||||
|
||||
### 10.5 Chunked video ingest
|
||||
- `POST /video/upload/init` → `PUT /video/upload/:uploadId/chunk/:index` → `POST /video/upload/:uploadId/complete`.
|
||||
- 10 MiB chunks (`CHUNK_SIZE_BYTES`) sidestep Cloudflare's ~100 MB proxy body cap. The server reassembles, seals, stores to R2, then runs the normal probe pipeline.
|
||||
- Max chunked upload: **100 GiB** (`MAX_CHUNKED_UPLOAD_BYTES`).
|
||||
|
||||
### 10.6 Upload caps
|
||||
- Whole-body video upload cap: **2 GiB** (`MAX_VIDEO_UPLOAD_BYTES`).
|
||||
- Files group upload cap: **2 GiB** (was 20 MiB).
|
||||
|
||||
## 11. Key file map
|
||||
|
||||
| Concern | Path |
|
||||
|---|---|
|
||||
|
|
|
|||
|
|
@ -59,7 +59,14 @@ Each account card shows: platform, handle, **capability chips** (what this conne
|
|||
|
||||
Every meaningful action (post published, role changed, connection added, credential rotated, approval granted…) is recorded in a **tamper-evident hash chain**. Filter by actor or action; click a row to open the **diff viewer** — what changed, field by field, with removed/added values. Secret values always show as `[REDACTED]`: keys and tokens are never written into the audit log, by construction. Export to CSV for compliance.
|
||||
|
||||
## 9. What's real vs. coming
|
||||
## 9. New in the Library & Publishing
|
||||
|
||||
- **Library (Files) tab:** upload up to **2 GiB per file**. Files are encrypted and stored in the hub's own object storage. The old 20 MiB limit and the separate Drive tab are gone — the Library replaces both.
|
||||
- **Compose:** attach up to **4 images or videos** per post (10 MiB per image, 64 MiB per video) and add **alt text** descriptions — they appear on Mastodon.
|
||||
- **Video producer:** apply **lower-third, logo, or clock overlays** and **voiceover** to any library video. Clips with no audio get a synthesized silence bed, so they always work. Results land back in the library and can be published like any other asset.
|
||||
- **Large videos:** uploads are automatically split into 10 MB chunks — big files work even through Cloudflare.
|
||||
|
||||
## 10. What's real vs. coming
|
||||
|
||||
**Now (Level 1):** monitoring streams, compose with previews, approvals, scheduling, publishing pipeline, encrypted credential vault, audit trail. *(The screens currently run on representative sample data while the backend wiring lands.)*
|
||||
|
||||
|
|
@ -68,7 +75,7 @@ Every meaningful action (post published, role changed, connection added, credent
|
|||
- **Crosspost** — one action moves content across platforms with your commentary (an Instagram image quoted into Mastodon, a Mastodon video into Facebook), flowing through approvals.
|
||||
- **Level 2** — video uploads, livestreams, the Virtual Channel / Auto Director master-control system, and the operator Control Room.
|
||||
|
||||
## 10. FAQ
|
||||
## 11. FAQ
|
||||
|
||||
**Why can't I retweet on some columns?** That platform's API doesn't expose resharing. We render only real capabilities.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue