glass beadian
aa5e262f3a
Self-hosted multi-platform broadcast newsroom for activist collectives: publishing pipeline with group approval and DST-proof scheduling, TweetDeck-style column wall with hostile-content-safe rendering, live control room with Auto Director, bridged chat governance with polls and quorum, distributed sandboxed transcode workers, encrypted content-addressed archive, monthly analytics. MIT licensed. See README.md and docs/.
540 lines
48 KiB
Markdown
540 lines
48 KiB
Markdown
# Streaming Architecture — Control Room, Overlays, Fallback, Chat, Telegram Ingest
|
||
|
||
Status: decision document (synthesis of four WP4 research reports). No code was changed while writing it.
|
||
Date: 2026-10-07
|
||
Inputs:
|
||
- **[flash]** `runs/wp4/control-room-research-flash.md`
|
||
- **[glm]** `runs/wp4/control-room-research-glm.md`
|
||
- **[as]** (integration research, kept outside the repository)
|
||
- **[tg]** `runs/wp4/telegram-amplify-research.md` (WP15 middle)
|
||
- **[scope]** `runs/wp4/control-room-scope.md`; **[dm]** `docs/design/design-merged.md`; **[wb]** `docs/work-breakdown.md` WP9–WP13, WP15; **[prop]** `docs/proposal.md` §4.4/§4.5
|
||
|
||
**Provenance legend.** `[flash]`, `[glm]`, `[as]`, `[tg]` mark the report a decision comes from. `[both]` means flash and glm agree. `[SYN]` marks a decision this document makes where neither report settled the question, or where the two disagree. Each `[SYN]` explains why it wins.
|
||
|
||
---
|
||
|
||
## 0. Decisions at a glance
|
||
|
||
| # | Question | Decision | Source |
|
||
|---|---|---|---|
|
||
| D1 | Where is program composited? | **On the server**, in the media worker's FFmpeg compositor. The browser is a control surface and a preview consumer only. | [both] |
|
||
| D2 | One FFmpeg process per channel, or split processes? | **Split**: ingest×N → compositor (replaceable) → **frame gate** (clock master) → persistent encoder → push relays. A graph change starts a new compositor (A/B swap). The encoder and the RTMP session are never restarted. | [SYN] over [flash]'s one-process invariant |
|
||
| D3 | Multiview | Every tile plays the worker's **preview tap**, never the remote source. 2x1/2x2 tiles are live LL-HLS. 3x3/4x4 tiles are thumbnails, with live LL-HLS only on the PGM tile and the focused tile. Hard cap of **4 live decodes on desktop and 1 on mobile**. | [both]; caps [SYN] |
|
||
| D4 | Audio | **Monitor mix** runs in the browser (one shared `AudioContext`, gain per tile, same-origin preview audio). **Program mix** runs in the compositor (`amix` with weights from desired state, gain in 3 dB steps). | [both]; steps [glm] |
|
||
| D5 | Overlay rendering | **Pre-rendered assets.** The API renders an SVG template to a PNG with alpha. The worker composites it with `overlay`. A chyron is one wide PNG strip scrolled by an `overlay` x-expression. FFmpeg never sees user text. | [flash] over [glm]'s `drawtext`+`textfile` |
|
||
| D6 | Playlist fallback | **On the worker.** A playlist is an ordinary ingest kind feeding a SLATE/PLAYLIST scene into the same encoder. PeerTube runs as `permanentLive`. PeerTube-native playlists are only the programming source of truth. | [both]; "Tier-2 PeerTube-only mode" from [glm] deferred |
|
||
| D7 | Auto Director | **Rules only**: a priority ladder over monitor states, plus dwell time per source class, ending in the fallback scene. Automation writes the same buses through the same transaction as a human TAKE. | [both]; dwell-by-class [glm] |
|
||
| D8 | PGM/PST transitions | v1 ships **CUT**. FADE follows behind a gate, as a transition graph swapped twice (old+new with `xfade`, then new only). The transition is part of desired state, not an imperative command. | [SYN] from [flash] Q6 + [glm] Q6 |
|
||
| D9 | Chat transport to the app | **WebSocket**: subscribe/ack, auth sent in the first frame, `lastSeq` resume from a server ring buffer. REST cursor polling (`?after=seq`) is the fallback. **No FCM.** | [as]; polling fallback aligned with [dm] §7 |
|
||
| D10 | Telegram "join and amplify" | Bot-as-channel-admin webhook → `archived_feed_items` → review draft → existing `amplify()`. MTProto stays a flagged Phase 2 and is not in this plan. | [tg] |
|
||
|
||
---
|
||
|
||
## 1. THE composition decision
|
||
|
||
### 1.1 Options
|
||
|
||
- **(A) Server-side FFmpeg compositor.** Runs inside `apps/media-worker` under the supervisor and receives commands over the existing Ed25519 signed-snapshot bus.
|
||
- **(B) Browser-composited program** (the StreamYard model). The browser draws onto a canvas, then `captureStream()`/WebCodecs → WebRTC/WHIP upstream → the worker transcodes to RTMP.
|
||
|
||
### 1.2 Evidence
|
||
|
||
| Criterion | (A) server compositor | (B) browser compositor | Source |
|
||
|---|---|---|---|
|
||
| **Operator laptop sleeps, tab crashes, Wi-Fi drops** | Program keeps running. The supervisor, output lease and reconcile loop already live on the worker. | **Program ends.** For a self-hosted product there is no cloud fallback, so the operator's machine becomes a single point of failure for a 24/7 channel. | [flash] Q3.1, [glm] Q3 |
|
||
| **24/7 premise** (scope item 5: a playlist plays when nothing is live) | Fallback is just another scene, and the encoder never idles. | Unattended hours have no browser to run in, so a second, server-side path is needed anyway. | [glm] Q3 "disqualifying on its own" |
|
||
| **Mobile (later)** | The phone is a pure control-plane client sending desired-state commands. No media runs on the device. | `HTMLMediaElement.captureStream` is `false` on Safari. WebCodecs `AudioEncoder` arrives only in Safari 26. Firefox Android has no `VideoEncoder`. | [flash] Q3 (BCD), [glm] Q3 (BCD) |
|
||
| **Signed command bus** | Fits 1:1. A TAKE writes a bus row, bumps `desiredVersion`, writes an outbox row and publishes a signed full snapshot. The worker converges idempotently. The fence, seen-set, reconcile and assignment epochs all carry over unchanged. | Needs a second "what is program" authority in the browser, a WHIP ingest trust boundary, and a WebRTC signalling path outside egress mediation, secret handling and the reconcile gate. | [flash] Q3.2, [glm] Q3 "Fit with the signed-command worker" |
|
||
| **Latency at the viewer** | Adds about one encode generation. | Can be under a second from glass to RTMP. | Both say **PeerTube's live→HLS chain dominates** (latency modes 1/2/3), so the gap mostly disappears at the viewer. [flash] Q3.4, Q5 |
|
||
| **CPU placement** | Server decodes and encodes. This is already priced in encoder units by WP9 admission. | Moves CPU to the operator machine. | [glm] Q3 |
|
||
| **Security** | Secrets, sandbox and egress pinning stay server-side, where the proposal already solved them. | The browser must decode every source at full quality, which breaks the "operator never pulls the remote source" rule. | [glm] Q3, [prop] §4.4 previews |
|
||
|
||
### 1.3 Decision — (A), server-side compositor [both]
|
||
|
||
The worker composes. The browser only watches previews and sends desired-state commands [glm]. A WebRTC/WHIP low-latency path, if ever needed, enters as **one more source kind** feeding the server compositor. Composition stays server-side either way [flash].
|
||
|
||
**Dissenting view (recorded).** Option (B) is the right answer for *interactive call-in shows*: sub-second guest interaction, zero server encode cost, and WYSIWYG editing on the operator's screen. StreamYard/Restream ship it (their internals are not verified in either report). We reject it because our product is a continuous Virtual Channel, not a studio session. If call-in guests become a requirement, the shape is vMix Call's [glm]: browsers *contribute* sources over WHIP, and the mix still happens on our worker.
|
||
|
||
### 1.4 Where the two reports disagree inside (A): process topology [SYN]
|
||
|
||
- **[flash]** proposes the invariant "**one FFmpeg process per channel**", with every TAKE, layout or overlay change rebuilding the graph.
|
||
- **[glm]** and **[prop] §4.4** use separate ingest / compositor / persistent encoder / pusher processes: "scene switch = graph rewiring… encoder and pushers are never restarted".
|
||
|
||
**Decision: split topology, plus a frame gate.** FFmpeg cannot rewire a running filtergraph. Runtime commands only change parameters of filters that already exist. So in flash's single process, *every* scene change restarts the encoder and the RTMP publish. That breaks WP9's "output that does not stop" and the WP11/WP12 continuity acceptance. flash's own Risk 2 ("a bad rebuild drops frames or kills program") is the symptom. The split topology moves the rebuild away from the encoder:
|
||
|
||
```
|
||
ingest_i (normalize → raw video + PCM over pipes, per source; also emits preview taps)
|
||
│ (N inputs, plus a playlist ingest and overlay asset inputs)
|
||
▼
|
||
compositor[A] ── active graph for desiredVersion v
|
||
compositor[B] ── started for v+1, warmed until its first frame is ready
|
||
│ rawvideo (yuv420p, fixed frame size) + s16le 48 kHz (fixed bytes per frame)
|
||
▼
|
||
FRAME GATE (clock master: paced at the profile fps; on compositor stall it repeats the
|
||
last frame and writes silence; switches A→B at a frame boundary once B
|
||
has produced K frames; then SIGTERMs A)
|
||
▼
|
||
encoder (persistent; reads gate pipes; never restarted by scene changes)
|
||
▼
|
||
push relay ×M (WP9/WP13: lease-bound key over fd, pinned destination)
|
||
```
|
||
|
||
- **Why the gate is the clock master.** Raw video and PCM have fixed byte sizes per frame, so the gate needs no demuxer and no timestamp parsing. Frames are counted, and the encoder sees a perfectly regular input with no gaps. Sync between audio and video holds by construction.
|
||
- **Cost.** About 93 MB/s of raw 1080p30 through one more pipe hop, plus a brief doubled compositor CPU during each swap. The doubled CPU is admission-accounted as a transient burst inside the worker's 20 % failover reserve.
|
||
- **Spike gate (first task of S2).** A gate written in Node with preallocated ring buffers must sustain 1080p60 at under 5 % of a core with no GC pauses above one frame interval. If it fails, the gate becomes a small static Rust binary behind the same pipe contract. The WP9 GStreamer decision gate ([wb] WP9) is folded into this spike: if A/B swap with the gate is not frame-continuous across 1 000 swaps, escalate to GStreamer `input-selector`.
|
||
|
||
flash's argument for a single process was simplicity and keeping "WP11's continuity risk class" in one place. The gate keeps that risk in one place, the gate, while making continuity a property of the design instead of a hope about rebuild speed.
|
||
|
||
---
|
||
|
||
## 2. Multiview + audio spec
|
||
|
||
### 2.1 What a tile plays [both]
|
||
|
||
Tiles never touch remote HLS. Each active ingest emits one **shared preview tap** ([prop] §4.4):
|
||
- a JPEG thumbnail, every 2 s by default (configurable 1–10 s);
|
||
- an on-demand **360p / ≤500 kbps LL-HLS** rendition;
|
||
- **[SYN] an on-demand audio-only rendition** (AAC 64 kbps LL-HLS). Thumbnails carry no audio, so without it a 4x4 operator could not hear a thumbnail tile.
|
||
|
||
These are served from the authenticated preview endpoint with `ETag`, single-flight fill, and an LRU of 6 segments or fewer per source. The endpoint is **same-origin with correct CORS**. This is load-bearing: a cross-origin `MediaElementAudioSourceNode` outputs silence per the Web Audio spec [flash] Q2 and [glm] Q2.
|
||
|
||
The **program tap** is the same mechanism applied to the gate output (one extra tap per channel). The PGM tile always shows what the encoder actually receives.
|
||
|
||
### 2.2 Grid tiers and the decode budget
|
||
|
||
No browser documents a hard limit on concurrent decoders; both reports flag this as **not verified**. The binding constraints are pixel throughput, per-tab MSE memory and mobile hardware-decoder sessions [glm] Q1. Budget: **live decodes ≤ 4 on desktop, ≤ 1 on mobile** [SYN]. This is flash's "never more than 4" plus glm's mobile "thumbnails + one live tile".
|
||
|
||
| Layout | Desktop tiles | Live (LL-HLS 360p) tiles | Mobile |
|
||
|---|---|---|---|
|
||
| 2x1 | 2 | both live | PGM live, other thumbnail |
|
||
| 2x2 | 4 | all 4 live | PGM live, others thumbnail |
|
||
| 3x3 | 9 | **PGM + focused** live; the other 7 are thumbnails | 1 live (PGM or focused) |
|
||
| 4x4 | 16 | **PGM + focused** live; the other 14 are thumbnails | 1 live (PGM or focused) |
|
||
|
||
Additional rules:
|
||
- **"Upgrade more tiles"** is an explicit user action, capped at 4 total, with a capability warning [glm].
|
||
- **Thumbnail cadence**: 2 s by default and 1 s for the PST-selected tile. Off-screen tiles stop polling (`IntersectionObserver` + `visibilitychange`) [prop].
|
||
- **hls.js config per tile** [glm] Q1:
|
||
- `lowLatencyMode: true`;
|
||
- `maxBufferLength: 4`, `maxMaxBufferLength: 8`, `backBufferLength: 4` (the default `Infinity` is wrong for a multiview);
|
||
- `liveSyncDurationCount: 2`.
|
||
- **Player selection** [flash] Q1: if `Hls.isSupported()`, use MSE. Otherwise, if `ManagedMediaSource` exists and `canPlayType(m3u8)`, use native HLS (Safari/iOS 17.1+). Otherwise the client is thumbnail-only.
|
||
- **Autoplay**: every tile boots `muted` + `playsinline`. One "Arm audio" gesture unlocks the shared `AudioContext` [flash] Q1.
|
||
- **Tile chrome**: monitor badge (`LIVE/UNSTABLE/OFFLINE/…` from the live-monitor WS fan-out), tally border (red = PGM, green = PST), and a tile-state label ("thumbnail · 2 s", "live preview").
|
||
- **Tally reflects only state stamped with `observed_version`.** It is never optimistic [glm] Q6, Risk 4.
|
||
- **PST rendering** [SYN] (adopting [glm] Q3's browser *preview* compositor):
|
||
- The PST monitor is rendered **client-side** onto a canvas, from the preview taps of the slots in the PST scene plus the scene's overlay SVGs. These are the same SVGs the server rasterizes (§4), so the look matches.
|
||
- The worker does **not** composite a second (preview) program. That would double compositor CPU for every channel.
|
||
- Caveat shown in the UI: "PST is a look-ahead; PGM is the truth".
|
||
|
||
### 2.3 Audio: two mixes, two owners [both]
|
||
|
||
**Monitor mix** (what the operator hears; browser):
|
||
- **Graph**: one `AudioContext` per page. Each audible tile's media element → `MediaElementAudioSourceNode` → its own `GainNode` → master `GainNode` → destination [glm] Q2.
|
||
- **Modes**:
|
||
- *Follow program* (default): plays the program tap's audio, which is exactly what viewers hear, delayed by roughly the tap latency.
|
||
- *Solo source X*: plays that source's audio rendition.
|
||
- *Custom monitor mix*: up to 4 sources with independent faders. These are continuous UI faders, because the mix is local.
|
||
- **Bounds**: never more than 4 audio renditions plus the program tap. Thumbnails gain audio by attaching the audio-only rendition (§2.1).
|
||
- **Privacy**: per operator and never sent to the server, so no command traffic.
|
||
- **Mobile**: the monitor collapses to "the one tile you are watching" [glm].
|
||
|
||
**Program mix** (what viewers hear; compositor):
|
||
- **Per-slot audio intent**: `scene_slots.audio_mode ∈ off|on|primary`, with at most one `primary` per scene (partial unique index, [prop] §4.4). The default is primary-only, meaning one feed's audio. "Mix N feeds" means several `on` slots.
|
||
- **Graph**: `volume` per input → `amix inputs=N duration=longest dropout_transition=2 normalize=1` [glm] Q2. A dead source fades out instead of cutting.
|
||
- **Gain** is desired state, in **3 dB steps from −24 to 0 dB**. No continuous program fader, because continuous faders would turn the signed bus into a UI-rate control channel [glm] Q2.
|
||
- **Applying a change**: a program gain change bumps `desiredVersion` and triggers an A/B compositor swap (§1.4). It is inaudible because the gate splices at a frame boundary.
|
||
- **Deferred optimization**: `amix`/`volume` runtime commands via `azmq` on an `ipc://` socket in job scratch [flash] Q2. **Not verified**: whether our pinned FFmpeg has libzmq, and whether it is compatible with the seccomp profile.
|
||
|
||
---
|
||
|
||
## 3. Chat unification contract (Opticast ↔ Tactical Media Broadcast Hub)
|
||
|
||
### 3.1 Roles [as] §2.1
|
||
|
||
**Tactical Media Broadcast Hub is the chat aggregation authority.** Connectors ingest server-side into a normalized stream. A new `chat-hub` module persists, dedupes, sequences and fans out. Opticast, the web Control Room and the Tactical Media Broadcast Hub mobile web shell are all **thin authenticated realtime clients**. The phone never holds Twitch/YouTube/PeerTube credentials.
|
||
|
||
The privacy pledge in Opticast's README ("no network access beyond the configured server") remains true. The integration is opt-in and off by default, and the only endpoint is a user-configured, self-hosted Tactical Media Broadcast Hub.
|
||
|
||
### 3.2 Ingestion (platform → `LiveChatMessage`) [as] §2.3
|
||
|
||
| Platform | Read | Reply | Moderate | Mechanism / governance |
|
||
|---|---|---|---|---|
|
||
| Twitch | ✅ | ✅ | delete, timeout, ban | EventSub WS `channel.chat.message` plus delete/clear events. Helix `POST /chat/messages` for replies; Helix moderation and ban endpoints. **No replay after a WS drop.** The hub ring buffer covers clients; the platform gap is accepted and marked with a `system` gap message. Token bucket of 100 per 30 s (elevated). |
|
||
| YouTube | ✅ | ✅ | delete, ban | `liveChatMessages.list` long-poll honouring `pollingIntervalMillis`, **one shared poller per broadcast**. Quota ledger: list = 1 unit, insert/delete/ban = 50, 10 000 per day by default. Quota exhaustion is a first-class `paused:quota` state. |
|
||
| PeerTube + livechat plugin, C2S on | ✅ | ✅ | MUC moderation | Server-side XMPP client joins as the streamer with the livechat long-term token, which is **"not officially supported"** per the plugin docs. |
|
||
| PeerTube, embed only | read-only | ❌ | ❌ | Read-only embed pane, web only. |
|
||
| PeerTube, no plugin | comments | comment reply | ✅ | Existing `getComments` on the connector, labelled "comments". |
|
||
| Mastodon | mentions | status reply | delete own | Streaming `/api/v1/streaming` (user), `source: "social-reply"`. |
|
||
| Mock livestream | ✅ | ✅ | ✅ | Contract and E2E harness. |
|
||
|
||
**Capability probing.** Each connection carries a probed capability row, with `pt-chat: xmpp-c2s | readonly-embed | comments-only | none` for PeerTube.
|
||
|
||
**Storage.** `live_chat_messages` is partitioned monthly per the [wb] retention row. `live_chat_seen_events` uses `ON CONFLICT DO NOTHING`, mirroring `monitor_seen_events`. Dedup key: `(workspace_id, provider, provider_message_id)`. **Ordering**: a per-live monotonic `seq` assigned at hub ingress. Provider timestamps are display-only, because clocks skew [as] §2.4.
|
||
|
||
**Local display filters.** Hide `!commands`, hide bots, mute user, word filter. These run once, hub-side, so web and mobile inherit them [as] §2.8.
|
||
|
||
### 3.3 Transport decision [SYN] (from [as] §2.5 + [dm] §7)
|
||
|
||
| Option | Verdict |
|
||
|---|---|
|
||
| **WebSocket** (subscribe/ack + resume) | **Primary.** Chat volume is far beyond review-chat volume. OkHttp WS on Android can authenticate in the first frame. The existing `authorizeSubscription` port is the gate. |
|
||
| SSE | Rejected. `EventSource` cannot send our Bearer header ([dm] §7 rationale), and it offers no upstream channel for acks. |
|
||
| REST cursor polling | **Fallback** for constrained clients and for resync after `RESYNC_REQUIRED`. It uses the same `after=seq` semantics as [dm] `review_events`, so one cursor discipline applies across the codebase. |
|
||
| FCM push | **Rejected** for v1, because it breaks Opticast's privacy pledge. A later opt-in UnifiedPush or self-hosted web-push is possible. |
|
||
|
||
**WS lifetime on the phone.** The socket stays open only while the chat sheet is open, or while `StreamingService`, the existing camera/mic foreground service, is live. There is no background socket when idle. Backoff from 1 s to 8 s mirrors `ReconnectController` [as] §2.5, Risk 4.
|
||
|
||
**Fan-out.** The hub publishes to Redis `chat:{workspaceId}:{liveId}`. The gateway subscribes only to authorized and viewed topics, using WP11 fan-out rules: bounded per-socket queues, `RESYNC_REQUIRED` on overflow. **Prerequisite:** the API-side `RespControlRedis` has PUBLISH but not SUBSCRIBE, so a subscribe-capable client must land first [as] §2.2.
|
||
|
||
### 3.4 Device pairing [SYN]
|
||
|
||
Opticast authenticates with a **device token**, not a user session cookie.
|
||
1. The web UI calls `POST /devices/pairings` and shows a QR code containing `{serverUrl, pairingCode}` (TTL 5 min, single use).
|
||
2. The app redeems the code with `POST /devices/pairings/redeem` and receives a device token. The token is bound to `(workspaceId, userId)`, scoped to `livechat.view|send|moderate` + `stream.view`, epoch-gated, and revocable from the web UI.
|
||
3. The app stores the token in `EncryptedSecretStore` [as] §4 Phase 2.
|
||
|
||
### 3.5 Endpoint shapes (both apps implement)
|
||
|
||
Routes follow repo convention: no `/workspaces/{wid}` prefix, because the workspace comes from the session or device token; compare `@Controller("publishing/drafts")`. [as] Phase 0 shapes are adapted accordingly.
|
||
|
||
```
|
||
REST (Tactical Media Broadcast Hub API; RbacGuard + RLS; Idempotency-Key on POSTs)
|
||
GET /live-chat/lives → { items: LiveSummary[] } # stream.view
|
||
GET /live-chat/lives/{liveId}/messages?after=<seq>&limit=<1..500, default 100>&platform=<p>
|
||
→ { items: ChatMessage[], nextCursor?: number } # livechat.view
|
||
POST /live-chat/lives/{liveId}/messages
|
||
body { text (1..500 chars), targets: "all" | connectionId[], replyTo?: messageId }
|
||
→ 202 { message: ChatMessage, delivery: Delivery[] } # livechat.send
|
||
POST /live-chat/lives/{liveId}/messages/{messageId}/moderation
|
||
body { action: "delete"|"timeout"|"ban", durationSeconds?: 1..1209600 }
|
||
→ 202 { delivery: Delivery[] } (audited via audit_append) # livechat.moderate
|
||
GET /live-chat/lives/{liveId}/capabilities
|
||
→ [{ connectionId, provider, read, reply, moderate, state: "ok"|"paused:quota"|"paused:rate"|"degraded", reason? }]
|
||
POST /devices/pairings → { pairingCode, qrPayload, expiresAt } # session user
|
||
POST /devices/pairings/redeem body { pairingCode, deviceName } → { deviceToken, workspaceId }
|
||
DELETE /devices/{deviceId} (revoke; bumps the device's epoch)
|
||
|
||
Delivery = { connectionId, provider, status: "queued"|"sent"|"failed"|"unsupported", error?: string }
|
||
|
||
WS /realtime (one socket, multiplexed topics; also used by the Control Room)
|
||
C→S { op:"auth", token } # first frame; never in the URL
|
||
S→C { op:"ready" } | { op:"close", reason }
|
||
C→S { op:"subscribe", id, topic:"livechat", liveId, lastSeq? }
|
||
C→S { op:"subscribe", id, topic:"channel", channelId } # Control Room: buses/monitor/observed
|
||
S→C { op:"ack", id, fromSeq } | { op:"nack", id, reason } # reason ∈ authorizeSubscription reasons
|
||
S→C { op:"msg", topic, seq, type, payload }
|
||
type ∈ chat.message | chat.moderated | chat.delivery | chat.gap | stream.state | capability
|
||
S→C { op:"resync", topic } # client re-fetches via REST ?after=
|
||
C→S { op:"unsubscribe", id } | { op:"ping" } S→C { op:"pong" }
|
||
|
||
ChatMessage = { seq, id, liveStreamId, connectionId, provider, providerMessageId,
|
||
author: { handle, displayName?, avatarUrl?, badges?: string[], color },
|
||
text, sentAt, receivedAt, isModerated, replyTo?, source: "live-chat"|"social-reply"|"system" }
|
||
```
|
||
|
||
Contract rules:
|
||
- **Resume.** `lastSeq` replays from a ring buffer of the last 500 messages per live, held in Redis. If `lastSeq` is older than the ring, the server sends `resync` and the client pages via REST.
|
||
- **Epoch.** An authz-epoch bump (role change, device revoke) closes every socket of that principal within 5 s, matching the WP11 revocation SLO.
|
||
- **Reply semantics.** The composer target defaults to "All capable". An incapable target is refused client-side with its reason. Delivery ticks per target reconcile from `chat.delivery` events, following Restream's pattern [as] §2.7.
|
||
- **Identity.** No cross-platform identity fusion. The author colour is a hash of `provider:handle` [as] §2.6.
|
||
- **Licensing.** Opticast is GPLv3 and the Tactical Media Broadcast Hub licence is **not verified**. The relationship stays API-only, with no shared code [as] Risk 5.
|
||
|
||
### 3.6 Opticast UX (for the client slice) [as] §3
|
||
|
||
- Chat is a bottom sheet on `LiveScreen` with snap points at 40/70/100 %. It is not a third route.
|
||
- One unified list with platform badges, filter chips `All · Twitch · YouTube · PeerTube · Mentions`, and swipe moderation (left = delete, right = ban; long-press for timeout).
|
||
- A "↓ N new" pill appears when scrolled up.
|
||
- A `ChatViewModel` sits beside `StreamViewModel`. Streaming code is untouched.
|
||
- **Opticast as a camera source** for the Control Room needs a push ingest kind (`rtmp-push`/`srt-listen`) that Tactical Media Broadcast Hub lacks today (`source.kind` is `"hls"` only). It is **out of scope for this plan** and listed as a follow-on in §6.4.
|
||
|
||
---
|
||
|
||
## 4. Overlay system
|
||
|
||
### 4.1 The disagreement
|
||
|
||
- **[glm] `drawtext` + `textfile` + `reload=1`.** Text edits land without a graph restart, and scrolling uses `x = main_w − mod(t·speed, main_w + text_w)`. [prop] §4.4 also mentions "overlay text → written to a `textfile`".
|
||
- **[flash] pre-rendered assets.** SVG template → PNG with alpha (via resvg/sharp) → `overlay`. The chyron is one wide strip scrolled by an `overlay` x-expression. FFmpeg never sees text.
|
||
|
||
### 4.2 Decision — pre-rendered assets [flash], with glm's no-restart property recovered by the A/B swap [SYN]
|
||
|
||
Why pre-rendered assets win:
|
||
1. **Same pixels in PST and PGM.** The browser PST look-ahead (§2.2) renders the *same SVG* the server rasterizes. With `drawtext`, the browser would have to imitate FreeType layout and would never match.
|
||
2. **Brand-quality typography.** The geometric design language (#FF1610, solid disk mark, rounded/angled boxes) is trivial in SVG and painful in `drawtext` box parameters.
|
||
3. **Smaller attack surface inside the sandbox.** `drawtext` needs freetype, harfbuzz, fontconfig and fribidi in the pinned FFmpeg, plus font files mounted into every compositor. [glm] Risk 3 itself names "font bombs". Rasterizing once in an API-side sandboxed job keeps fonts and SVG parsing out of the 24/7 process.
|
||
4. **Cost and determinism.** A static PNG overlay is cheaper than per-frame glyph rendering. [glm] flags `drawtext` scroll CPU at 1080p60 as a risk, and its own fallback is "render the band into a strip", which is flash's design. PNGs are also golden-frame deterministic.
|
||
5. **glm's advantage is no longer needed.** glm's argument for `drawtext` (an edit without a graph restart) matters only if restarts are visible. With the frame gate (§1.4), a graph swap is frame-continuous, so an overlay edit is just another `desiredVersion`.
|
||
|
||
`drawtext` survives in one narrow role: a **server-generated clock/countdown** using `expansion=normal` with `localtime`. The format string is a typed enum, never user text.
|
||
|
||
### 4.3 Pipeline
|
||
|
||
```
|
||
Operator edits a lower third (web/mobile)
|
||
│ POST /production/overlays/{overlayId} { templateId, fields: {title ≤ 80, subtitle ≤ 120}, style enum }
|
||
▼
|
||
API validation: length, Unicode category allowlist, no control characters → overlay_revisions row
|
||
▼
|
||
render job (sandboxed WP4 probe class): SVG template (server-owned, typed field slots, bundled fonts)
|
||
→ resvg (no scripts, no external fetch) → PNG with alpha at output resolution
|
||
→ StorageService (sha256 key, dedup) → asset id
|
||
▼
|
||
1 tx: scene overlay slot → new assetId + desired_version++ + outbox row (same path as a TAKE)
|
||
▼
|
||
signed snapshot → worker → compositor B built with the new overlay input → gate swap at a frame boundary
|
||
```
|
||
|
||
Overlay kinds:
|
||
- **Lower third**: PNG composited with `overlay=x:y`. There is no timed `enable=` expression. Toggling is a slot flag in desired state, so the next graph simply includes or omits it.
|
||
- **Chyron**: render the *whole ticker strip* (all items with separators) as one wide PNG. The height of the band is fixed. Width is capped at **16 384 px** [SYN]; longer text is split into a carousel of strips. Scroll with `overlay=x='W-mod(t*S\,w+W)':y=Y`, where speed S is a typed integer of 40–400 px/s.
|
||
- A chyron text edit renders a new strip and swaps graphs. The scroll position restarts, and that is accepted.
|
||
- **Logo from an SVG input**: a tenant upload goes through the WP4 sandboxed probe and is rasterized once per asset version at 1×/2× sizes. Position and size are typed values (corner enum + margin, or explicit numbers).
|
||
|
||
Security gates:
|
||
- **Injection.** Overlay text never reaches argv or a graph, because the graph references `assetId` only. The injection tests from [wb] WP12 apply to the render job too: SVG text nodes are XML-escaped, and the template engine is typed field-slot substitution, not string concatenation.
|
||
- **Worker storage access.** The worker fetches assets through presigned GETs scoped to its assignment ([prop] §4.5 object-storage tenant ACLs).
|
||
|
||
**Not verified:**
|
||
- resvg and sharp APIs. flash fetched the resvg README; glm did not verify either. **Pick resvg-js.**
|
||
- Whether the image2 demuxer re-reads a file on `-loop 1`. flash and glm do not discuss it. It is **no longer needed**, because edits go through the swap.
|
||
|
||
---
|
||
|
||
## 5. Playlist fallback + Auto Director
|
||
|
||
### 5.1 Fallback venue [both]
|
||
|
||
**Decision: worker-side fallback.** The PeerTube live is `permanentLive: true`, so the channel URL stays live forever. "Nothing is live" means the director puts the **FALLBACK scene** on program.
|
||
- No live/VOD flapping, no RTMP reconnect, and no change of PeerTube latency mode. Overlays and program audio stay consistent [flash] Q5, [glm] Q5.
|
||
- There is no PeerTube API to play a playlist into a live [flash] Q5 (verified absence). Switching a channel between live and VOD relies on unverified viewer-UI behaviour [glm] Q5.
|
||
- `permanentLive` and `saveReplay` are mutually exclusive (verified 400 error). Fallback channels therefore run **without replay**, and the channel config UI says so [flash] Risk 5.
|
||
|
||
**Deferred.** [glm]'s **Tier-2 mode** ("live windows + PeerTube playlist page", where the worker stops pushing when dark) is a cost-saving option for channels that do not need continuity. It is **not in this plan**. Revisit it after we have measured the 24/7 encoder cost.
|
||
|
||
**Mechanics** [SYN] (fits the WP9 sandbox: FFmpeg reads only pipes):
|
||
- **A playlist is a source kind** (`kind: "playlist"`). A worker-side **asset feeder** walks the ordered asset list:
|
||
- assets are presigned GETs through the ingest egress relay;
|
||
- they are piped one after another into a normal ingest process (`-re`, normalize stage);
|
||
- the compositor sees a continuous input like any other source.
|
||
- Asset boundaries are hidden by the ingest normalize stage plus the gate (PTS regenerated). Concat-protocol tricks are not used; they are banned by the WP9 protocol rules.
|
||
- **Assets are pre-normalized at upload** to the channel profile by the existing WP4 video pipeline. Loops never transcode. Pricing therefore follows the [glm] "normalize-on-ingest, not in the loop" note.
|
||
- **Programming source of truth.** The FALLBACK scene's playlist is either a local asset list or **mirrored from a PeerTube channel playlist** through the connector (`GET /video-playlists/{id}/videos`), downloaded to S3 and normalized [flash] Q5.
|
||
- **Slate floor.** A static slate (logo + "back soon" + optional clock) is generated by the overlay renderer. It is always available, independent of asset availability. When the playlist ingest is unhealthy (asset missing, decode error), the scene degrades to the slate automatically.
|
||
|
||
### 5.2 Auto Director rules [both]
|
||
|
||
The evaluation context and plumbing already exist and are not redesigned here:
|
||
- **Monitor states**: the reducer emits `OFFLINE|PROBING|LIVE|UNSTABLE|ERROR`, with STARTING/ENDING/UNKNOWN added in the DB. Hysteresis: `liveConfirmations=2`, `offlineConfirmations=3`, `noMediaMs=10 s`, `minHoldMs=2 s`, `cooldownMs=10 s` (`apps/api/src/live-monitor/reducer.ts`).
|
||
- **Evaluation plumbing**: WP10's 250 ms debounce, one evaluation in flight per channel, and `continue-as-new`.
|
||
|
||
Rule ladder, evaluated top-down. The first match writes program.
|
||
|
||
1. **Program lock / operator override** (four WP10 release modes). Automation never undoes a human TAKE while an override or lock holds [both].
|
||
2. **Media-health failover**: if the program source's media-health tier reports failure, the debounce is bypassed. Fail over to the next eligible source, otherwise to FALLBACK. This works during an API outage (WP10).
|
||
3. **Scheduled windows**: WP13 schedules can force a scene inside a window.
|
||
4. **Priority list**: the highest-priority source in `LIVE` wins. A source already on program that turns `UNSTABLE` keeps program until `noMediaMs` plus confirmations expire.
|
||
5. **Terminal**: the FALLBACK scene (playlist, else slate).
|
||
|
||
Anti-flap at the director level [glm] Q7:
|
||
- **Dwell time by source class.** Defaults: a live source holds at least 15 s; FALLBACK holds at least 60 s before a newly LIVE source may preempt it, unless that source's priority is "interrupt". This complements, and does not duplicate, monitor hysteresis.
|
||
- **Cooldown** of 10 s per source after it leaves program before it may return automatically.
|
||
- Out of scope: "real direction" such as audio-activity speaker switching or content-aware cuts. Future work adds it as new *conditions* in the rule `condition` jsonb, not as a new engine [both].
|
||
|
||
**Monitor-state integration in the Control Room.** The UI renders:
|
||
- AUTO/MANUAL/LOCKED status;
|
||
- for each rule, a "**would fire now**" indicator computed from the same monitor snapshot, so operators see *why* the director acted [flash] Q7;
|
||
- a dwell countdown on the program tile;
|
||
- **RETURN TO AUTO**, which re-arms the ladder.
|
||
|
||
Every automated action uses the same bus transaction as a human TAKE: one `desired_version` bump and one outbox row. Automation has no special-cased path [glm] Q7.
|
||
|
||
### 5.3 PGM/PST control set (web now, same commands on mobile later) [both]
|
||
|
||
The v1 control set:
|
||
- **PST select** (tile or scene);
|
||
- **CUT** (TAKE, `transition: {type:"cut"}`);
|
||
- **overlay on/off** for each overlay slot (armed on PST, live after TAKE);
|
||
- **program audio** primary pick and gain steps;
|
||
- **monitor audio** (local only);
|
||
- **LOCK**;
|
||
- **RETURN TO AUTO**.
|
||
|
||
Layout picker, position swap and PIP all operate on the **PST scene** and reach program only through TAKE [flash] Q6.
|
||
|
||
FADE (`{type:"fade", ms: 300|500|1000|2000}`) is added in S4 behind the continuity gate (D8). If a fade graph fails to start within 1 s, the worker degrades to CUT and reports `TRANSITION_DEGRADED` [flash] Risk 2.
|
||
|
||
Every command carries an `Idempotency-Key` and is checked against `director.*`. The target is under 300 ms from command to observed (WP11).
|
||
|
||
Mobile "go live from preview" is the same PST select followed by TAKE. It needs no new server surface.
|
||
|
||
---
|
||
|
||
## 6. Phased build plan
|
||
|
||
Six dispatchable slices. Migration numbers are **pre-assigned** so parallel slices do not collide. `011` stays reserved for the review workflow ([dm] §4), and `012` exists.
|
||
|
||
Default gate for every slice: `pnpm typecheck && pnpm test` green from the repo root (turbo), plus the slice-specific gates below. Each slice writes `runs/<wp>/<slice>-report.md`.
|
||
|
||
```
|
||
┌── S2 worker compositor ──┐
|
||
S1 ──────┤ ├── S4 fallback + director + FADE
|
||
└── S3 control room ───────┘
|
||
S5 chat hub + contract (independent; starts day 1) ── S5b Opticast client (anarchostreamer repo, after S5 contract freeze)
|
||
S6 telegram WP15 middle (independent; starts day 1)
|
||
```
|
||
|
||
**Parallelism:** S1, S5 and S6 can start together. S2 and S3 run in parallel once S1 lands. S3 develops against a fake worker that echoes realized versions. S4 needs both S2 and S3.
|
||
|
||
### S1 — Desired-state v2 + production schema (small; blocks S2/S3)
|
||
|
||
- **Scope:**
|
||
- Extend `DesiredChannel`/`DesiredSpec` from a single `source:{kind:"hls"}` to `{ scenes, buses, sources[], overlays[], transition, audio }`. Old v1 snapshots must still validate.
|
||
- Keep the strict key allowlist and secret rejection.
|
||
- Add migration `013_production.sql`: `production_scenes`, `scene_slots` (layout enum, geometry CHECKs, `audio_mode` with the one-primary partial unique index, `gain_db` in −24…0 in steps of 3), `production_buses` (preview|program, unique per channel), `overlay_slots`/`overlay_revisions` (by asset id), and `source.kind` gaining `playlist`.
|
||
- All tables use channel-scoped composite FKs for the same-channel invariant, plus RLS.
|
||
- **Files:** `apps/api/src/media-control/spec.ts`, `wire.ts`, `apps/media-worker/src/types.ts`, `apps/api/migrations/013_production.sql`, plus tests.
|
||
- **Gates:**
|
||
- Old snapshots round-trip.
|
||
- The [wb] WP12 same-channel invariant tests fail at the DB constraint.
|
||
- Injection payloads in labels (`;[,'`, `movie=`) are rejected by the type layer.
|
||
- The RLS denial suite covers the new tables.
|
||
|
||
### S2 — Worker compositor + frame gate + taps (XL, highest risk; after S1, ∥ S3)
|
||
|
||
- **Scope:**
|
||
- The **frame-gate spike** comes first (§1.4 gate: Node vs Rust; 1 000-swap continuity).
|
||
- `MediaCommandBuilder` graph builders emit `xstack` (2x1/2x2/3x3/4x4 plus custom), PIP `overlay`, overlay asset inputs, chyron x-expression, and `volume`+`amix`, all from typed params only.
|
||
- Ingest×N normalize stage; compositor A/B lifecycle in the supervisor; persistent encoder fed by the gate; playlist feeder ingest.
|
||
- Preview taps: JPEG, 360p LL-HLS, audio-only, and the program tap.
|
||
- The supervisor/bus/fence stay **untouched** apart from the meaning of a RUNNING snapshot [glm] Phase 1.
|
||
- **Files:** `apps/media-worker/src/compositor/*` (new), `src/gate/*` (new), `src/relay/ffmpeg.ts` (generalized), `src/supervisor/index.ts` (A/B lifecycle), `src/preview/*` (new).
|
||
- **Gates:**
|
||
- Golden-frame hashes for each layout.
|
||
- **Zero gap at the encoder input** across 1 000 scripted scene, overlay and gain swaps (frame counter plus PCM continuity check).
|
||
- A 4-input 1080p30 CPU budget test on the reference host.
|
||
- `kill -9` of the compositor mid-swap leaves the encoder and RTMP session up.
|
||
- The ≥8 h soak is wired into the nightly run (not blocking merge).
|
||
- Existing WP9 sandbox, bus and secret-leak suites stay green.
|
||
|
||
### S3 — Control plane + Control Room UI + overlay pipeline (XL; after S1, ∥ S2)
|
||
|
||
- **Scope:**
|
||
- **Bus writes**: TAKE/CUT/PST-select/lock/override in `media-control.service.ts`. One tx = bus write + `desired_version++` + outbox, with `Idempotency-Key` and `director.*`.
|
||
- **Preview endpoint**: ETag, single-flight fill, same-origin, signed short TTL.
|
||
- **Realtime gateway**:
|
||
- wire `/realtime` for real, including the **subscribe-capable Redis client**, which is shared with S5;
|
||
- **S3 owns the gateway** and S5 builds on it;
|
||
- if S5 starts first, S5 lands the gateway and S3 rebases.
|
||
- **Web**: multiview tiers per §2.2, Web Audio monitor per §2.3, PST canvas look-ahead, tally from observed state only, overlay editor.
|
||
- **Overlay pipeline**: SVG templates → resvg-js render job → StorageService → asset id → desired-state bump.
|
||
- **Files:** `apps/api/src/media-control/*`, `apps/api/src/realtime/*`, `apps/api/src/production/*` (new: scenes, overlays, preview), `apps/web/src/app/control-room/*`, `apps/web/src/components/control-room/*`.
|
||
- **Gates:**
|
||
- Idempotency matrix: duplicate, concurrent duplicate, crash mid-tx.
|
||
- Command → observed under 300 ms against the fake worker.
|
||
- Playwright E2E: PST → CUT → tally flips only on observed state.
|
||
- Grid tier caps enforced: a 5th live tile is refused.
|
||
- Overlay injection tests through the render job.
|
||
- Revocation evicts the operator within 5 s.
|
||
|
||
### S4 — Fallback scene + Auto Director v1 + FADE (L; after S2 + S3)
|
||
|
||
- **Scope:**
|
||
- FALLBACK scene: playlist source mirrored from a PeerTube channel playlist via `packages/connector-peertube`, plus the slate floor.
|
||
- `permanentLive` wiring and the replay conflict surfaced in channel config.
|
||
- Rule ladder per §5.2 with dwell-by-class and cooldown, consuming live-monitor outbox events (WP10 intake rules).
|
||
- "Would fire now" indicators and RETURN TO AUTO.
|
||
- FADE via the two-swap transition graph, degrading to CUT.
|
||
- **Files:** `apps/api/src/director/*` (new), `apps/scheduler/*` (Temporal evaluator workflow), `packages/connector-peertube/src/*` (playlist mirror), `apps/media-worker/src/compositor/transition.ts`, and the Control Room AUTO panel.
|
||
- **Gates:**
|
||
- [wb] WP10 matrix: 0/1/2/3+ live, high-priority interrupt, TAKE not undone, lock stability, flap damping, failover without output kill, API-outage scenario.
|
||
- Event-storm test (50 sources at 10 Hz → ≤4 evaluations/s).
|
||
- Playlist asset removed mid-loop → slate within 2 s with no output gap.
|
||
- FADE continuity holds across 200 fades.
|
||
|
||
### S5 — Chat hub + unified contract + web chat panel (L; independent)
|
||
|
||
- **Scope:**
|
||
- **Phase 0 contract** (§3.5) frozen in `packages/core` as types and JSON schema.
|
||
- Migration `014_live_chat.sql`: `live_chat_messages` partitioned monthly, `live_chat_seen_events`, `devices`, `device_pairings`.
|
||
- `apps/api/src/live-chat/` (new):
|
||
- Twitch EventSub WS and YouTube shared-poller ingestion;
|
||
- seq assignment and ring buffer;
|
||
- send governors (token buckets, YT quota ledger);
|
||
- reply fan-out and moderation fan-out with per-target `Delivery`;
|
||
- display filters.
|
||
- Twitch connector gains Helix send/moderate.
|
||
- Device pairing.
|
||
- Web chat panel in the Control Room.
|
||
- PeerTube XMPP adapter = **S5 phase 2**. Ship capability-probe + read-only-embed + comments first.
|
||
- **Files:** `apps/api/src/live-chat/*`, `apps/api/src/devices/*`, `apps/api/src/realtime/*` (shared with S3, see S3), `packages/connector-twitch/src/index.ts`, `packages/connector-mock-livestream/*`, `apps/web/src/components/live-chat/*`, `apps/api/migrations/014_live_chat.sql`.
|
||
- **Gates:**
|
||
- Mock-livestream E2E: ingest → hub → Redis → WS → client, including `lastSeq` resume and the `resync` path.
|
||
- Dedupe: the same provider id twice gives one row.
|
||
- Authz-epoch bump closes sockets within 5 s.
|
||
- The YT quota ledger refuses a send below budget and surfaces `paused:quota`.
|
||
- Delivery `unsupported` per target never fails the batch.
|
||
- Retention sweep test per the [wb] cross-cutting row.
|
||
- **S5b — Opticast client** (separate repo `anarchostreamer`; dispatch after the S5 contract freeze):
|
||
- OkHttp (its only new network dependency), a pairing settings screen, `ChatViewModel` + bottom sheet, WS lifetime tied to `StreamingService`, the composer with target pill, and swipe moderation.
|
||
- Gate: `testDebugUnitTest assembleDebug` green, plus a contract test against a recorded S5 WS transcript.
|
||
|
||
### S6 — WP15 Telegram middle (M; independent)
|
||
|
||
- **Scope** [tg] §5 Phase 1:
|
||
- Migration `015_archive.sql`: `feed_subscriptions` (`{type:"telegram_channel", connectionId, channelId}`) and `archived_feed_items` (RLS; dedup on `(workspace_id, remote_ref=chat_id:message_id)` and content hash).
|
||
- Channel-post normalizer: `media_group_id` album debounce of 2 s, `edited_channel_post` → revision, `forward_origin` and `author_signature` kept as attribution.
|
||
- `my_chat_member` → subscription health ("bot removed").
|
||
- Media harvest via `getFile` into StorageService. Files over 20 MB are recorded as "Bot API cannot fetch", not failed.
|
||
- "Create draft from archived item" → the existing review states in `publishing.service.ts`.
|
||
- Approval calls the existing `amplify()`. Protected content shows "amplify unavailable".
|
||
- Rights note in the UI, and attribution on by default.
|
||
- No MTProto.
|
||
- **Dependency note.** The draft binding targets today's `editing → in_review → approved` states. When the review workflow (`011`, [dm] WP5.1–5.2) lands, the archived-item draft enters the review cycle unchanged, because it is just a draft. Amplifying others' content must **never auto-publish** [tg] §2.3.
|
||
- **Files:** `apps/api/src/archive/*` (new), `packages/connector-telegram/src/index.ts` (normalizer hook only), `apps/api/src/publishing/*` (draft-from-item), `apps/web/src/app/archive/*`, `apps/api/migrations/015_archive.sql`.
|
||
- **Gates:**
|
||
- Webhook fixture → one archived row for a 3-photo album.
|
||
- The same post via two subscriptions → one row.
|
||
- The same image → one blob.
|
||
- Tenant B cannot read tenant A's rows despite a shared blob.
|
||
- A protected-content forward surfaces `ConnectorError`.
|
||
- E2E: post → archive → draft → approve → `forwardMessage` on the mock.
|
||
|
||
### 6.4 Not in this plan (follow-ons)
|
||
|
||
- Opticast as a camera **source**: an `rtmp-push`/`srt-listen` ingest kind with per-channel ingest keys.
|
||
- WHIP guest contribution.
|
||
- Tier-2 PeerTube-playlist mode.
|
||
- MTProto grabber ([tg] Phase 2: flagged, ToS disclosure).
|
||
- Chat relay bot.
|
||
- Runtime `azmq` gain control.
|
||
- Mobile Control Room screens. These reuse the S3 endpoints and need no new server surface.
|
||
|
||
---
|
||
|
||
## 7. Top risks and mitigations
|
||
|
||
| # | Risk | Likelihood / impact | Mitigation |
|
||
|---|---|---|---|
|
||
| 1 | **The frame gate / A/B swap is not frame-continuous**, or costs too much CPU. Every TAKE, overlay and gain change depends on it. | Medium / critical | It is the first task of S2 and gates the whole slice: 1 000-swap continuity plus a CPU budget; a Rust gate if Node fails; GStreamer `input-selector` as the escalation path (the WP9 decision gate). The transient doubled-compositor CPU is reserved inside the 20 % failover headroom. |
|
||
| 2 | **Multiview expectations versus browser physics.** Users expect 16 live tiles. Decode limits are undocumented ([both]: not verified), and iPhone MSE exists only from 17.1. | High / medium | Thumbnail-default tiers with a hard cap of 4 (1 on mobile) and explicit tile-state copy. Run a device-matrix test before any marketing claim. Tiles never decode remote sources. |
|
||
| 3 | **Worker CPU at scale.** A 1080p30 N-input grid plus overlays and `amix`, plus an always-on fallback encoder 24/7 for every channel. | High / high | Input caps per profile in admission. Playlist assets pre-normalized, never transcoded in the loop. FALLBACK priced in capacity units, not treated as free. Tier-2 "dark" mode as the documented future cost lever. CI CPU-budget test in S2. |
|
||
| 4 | **Chat platform fragility**: Twitch has no EventSub replay and harsh rate enforcement; the YouTube quota is 10k/day; the PeerTube livechat plugin varies by instance and its bot token is unsupported. | High / medium | Ring-buffer resume plus `chat.gap` markers. Token buckets and the quota ledger with `paused:*` states. Capability probe with tiered PeerTube degradation (C2S → read-only embed → comments). Request a YouTube quota increase for production. |
|
||
| 5 | **PeerTube coupling**: `permanentLive` excludes replay, Small latency disables P2P, and live↔VOD switching is unmeasured. | Medium / high | Worker-side FALLBACK from day one. Replay and latency mode are explicit channel config with in-UI consequences. Measure live-end→reconnect on a test instance before promising Tier-2. |
|
||
| 6 | **Overlay and asset attack surface**: SVG parsing, fonts, render-job resource exhaustion. | Medium / high | Server-owned templates with typed field slots. resvg with no scripts and no external fetch, run in the WP4 sandbox with time and memory limits. Rendered PNGs are content-addressed and cached. FFmpeg only ever sees asset ids. Injection tests in S1 and S3. |
|
||
| 7 | **Telegram amplification ethics and ToS**: re-posting others' content, protected channels, the 20 MB bot cap. | Certain / medium | A human review gate is mandatory. Attribution is on by default. Protected content and oversize media are surfaced honestly. MTProto stays out of this plan. |
|
||
| 8 | **Opticast privacy pledge erosion**: pressure to add FCM, analytics or a cloud relay. | Medium / medium | The contract offers no push and only the self-hosted endpoint. The integration is opt-in and off by default. Any push must be UnifiedPush/self-hosted and opt-in. The relationship stays API-only, so the unresolved GPLv3 question remains moot. |
|
||
|
||
---
|
||
|
||
## Appendix — where the two control-room reports disagree
|
||
|
||
| Topic | [flash] | [glm] | Pick | Why |
|
||
|---|---|---|---|---|
|
||
| Process topology | One FFmpeg per channel | Split ingest/compositor/encoder/pusher ([prop]) | **Split + frame gate** | FFmpeg cannot rewire graphs, so a single process restarts the RTMP session on every change. |
|
||
| Overlay text | Pre-rendered PNG (SVG→resvg) | `drawtext` + `textfile` `reload` | **PNG** | PST/PGM pixel parity, brand typography, fonts kept out of the 24/7 sandbox, cheaper CPU. glm's no-restart edge is covered by the A/B swap. |
|
||
| Live tile cap at 3x3/4x4 | Focused + PST live (≤4 total) | 1–2 live (focus + PGM) | **PGM + focused live, cap 4 on explicit upgrade** | The PGM tile must be real, so its truth comes from the program tap. PST is a client-side look-ahead, not a decoded stream. |
|
||
| Fallback modes | Worker-side only | Worker-side Tier 1 + PeerTube-playlist Tier 2 | **Tier 1 now, Tier 2 deferred** | Tier 2 depends on unverified PeerTube UI behaviour; it stays as a later cost lever. |
|
||
| Program gain changes | `amix` runtime commands | Desired-state bump in coarse 3 dB steps | **3 dB steps via swap; runtime commands later** | `azmq` availability is unverified, and swaps are already continuous. |
|
||
| PST rendering | Not specified (implied worker) | Browser preview compositor (canvas) | **Browser canvas look-ahead** | Avoids a second server composite for each channel. |
|
||
| Transitions | CUT plus FADE presets; fall back to CUT | CUT plus 300 ms fade default | **CUT in v1; FADE in S4, degrading to CUT** | FADE needs a two-swap transition graph and must be proven continuous first. |
|