# Tactical Media Broadcast Hub — Engineering Guide

**Covers:** everything built as of `499c9fe` (WP0–WP3-in-flight), plus Level-2 systems as marked *(planned)*. Companion docs: [admin-guide.md](./admin-guide.md) · [user-guide.md](./user-guide.md).

---

## 1. System overview

Tactical Media Broadcast Hub is a self-hosted, multi-tenant social-media management platform (Level 1: post & schedule) growing into a video/livestream master-control system (Level 2, planned). Three planes:

```
CONTROL PLANE                     MEDIA PLANE (planned, WP9+)
  apps/web  (Next.js UI)            media-worker: ingest → compositor →
  apps/api  (NestJS)                  persistent encoder → outputs
  apps/scheduler (Temporal)        FFmpeg-first; never through the API
  PostgreSQL · Redis · Temporal
STORAGE PLANE
  S3-compatible object storage (MinIO dev / any S3 prod)
  ALL media client-side encrypted; content-addressed
```

Hard rules enforced in code: media bytes never transit API processes; provider branching outside connector packages fails lint; passwords never transit the system (OAuth tokens / bot tokens only); every tenant table is RLS-isolated with `FORCE ROW LEVEL SECURITY`.

## 2. Repository layout

```
apps/
  web/            Next.js 14 App Router frontend (all screens, mock-registry layer)
  api/            NestJS control plane (health, auth, rbac, vault, audit, publishing, storage)
  scheduler/      Temporal worker (publish workflow + activities)
  media-worker/   health-only stub (WP9 fills it)
packages/
  core/           normalized domain types (permissions, connections, publishing, VC family)
  connector-sdk/  PlatformConnector interface, manifest, registry, HTTP client, contract kits
  connector-example/ connector-mock-social/ connector-mock-video/ connector-mock-livestream/
  connector-mastodon/ connector-telegram/          (WP3, in flight)
  testing/        live-source simulator (dod-flap scenario) + mock RTMP sink
infra/            docker-compose (postgres:5433, redis, temporal, minio, otel, prometheus, grafana)
docs/             this documentation set
```

Monorepo: pnpm workspaces + Turborepo. `pnpm lint|typecheck|build|test` at root runs all 13+ packages.

## 3. Identity, tenancy & security (WP2)

### 3.1 Database roles — two-pool discipline
- **Owner pool** (`SUPER_DATABASE_URL`, the postgres superuser in dev): runs migrations, provisions. Never serves requests.
- **App pool** (`DATABASE_URL`, role `sp_app`): neither superuser nor table owner → `FORCE ROW LEVEL SECURITY` binds for real. Created by migration `003_app_role.sql` with table/sequence grants + default privileges.

### 3.2 Row-level security
Every tenant table (memberships, connections, credentials, audit_log, drafts, publications, scheduled_publications) has:
```sql
USING (workspace_id = NULLIF(current_setting('app.workspace_id', true), '')::uuid)
```
`NULLIF` matters: pooled sessions carry an empty-string GUC placeholder after `SET LOCAL` commits; without it the cast throws instead of failing closed. Tenant context is set per-transaction via `withTenant(workspaceId, fn)` (`apps/api/src/db/client.ts`) using `set_config(..., true)` — transaction-local, cannot leak.

### 3.3 SECURITY DEFINER discipline (audit finding #3)
Cross-tenant functions take an explicit `p_workspace_id` and call `fn_assert_tenant()`, which asserts the parameter against the session's tenant context. No unguarded `EXECUTE` grants for `sp_app`.

### 3.4 Continuous authorization (audit finding #2)
Any membership insert/update/delete fires `bump_authz_epoch()` — `COALESCE(NEW.workspace_id, OLD.workspace_id)` (OLD is NULL on INSERT; the original bug silently no-oped). Sessions embed the epoch at issue; `RbacGuard` compares against the live `workspaces.authz_epoch` and fails closed (`401, epoch stale`) on mismatch. Revocation propagates to live sessions without waiting for re-login.

### 3.5 Sessions, logout & the Hub as OIDC issuer
HMAC-SHA256 signed tokens (`apps/api/src/auth/session.ts`), 12h TTL, constant-time verify. Since migration `034`, every session also carries a `sid` backed by an `auth_sessions` row: login/reissue write the row, `POST /auth/logout` revokes it, and `reissue` refuses both revoked rows and pre-`034` tokens (no `sid`) so revocation applies from the next epoch bump. Request validation (`RbacGuard`) stays stateless — rows are consulted only by reissue and the OIDC refresh path.

The Hub is also its own OpenID Connect **provider** for oCIS (`apps/api/src/oidc/`, issuer `https://api.occupymedia.cc/oidc`): discovery, JWKS (ES256, keys sealed via the vault pattern), `/oidc/authorize` → SPA consent → single-use code (60 s, PKCE S256 enforced) → tokens. Only the first-party client `web` (the oCIS web app) is registered; anything else fails closed. Phase 2 added **session-bound rotating refresh tokens** (`oidc_refresh_tokens`, SHA-256 at rest, `family_id` chains): a chain lives and dies with the consenting Hub session, reuse of a consumed token revokes the whole family (OAuth BCP), and every accept re-checks that the session row is live and the membership active — so Hub logout propagates to oCIS at the next refresh instead of at the next silent authorize. The unauthenticated token endpoint resolves tokens through the definer-rights `oidc_refresh_lookup()` (the `audit_append` pattern). Matrix SSO (`apps/api/src/matrix-sso/`) is a separate 2-minute `m.login.token` broker and intentionally out of propagation scope: Element holds its own Matrix-native session after mint.

### 3.6 Credential vault
Envelope encryption (`vault/vault.ts`): per-workspace DEK (AES-256-GCM) wrapped by master KEK (`SUPERPOSTER_KEK`, 32B base64; production-required). Rotation re-wraps; old DEK dropped → pre-rotation ciphertext unreadable (tested). Connectors receive opaque `CredentialRef` handles via `CredentialAccessor` — never ciphertext, never raw tokens.

### 3.7 Audit log
Hash-chained per branch (`audit_append` for tenant rows, `audit_append_system` for workspaceless events). Branch selection is by **tenant-context presence, never by role** (findings #1/#6 — `sp_scheduler` with tenant context appends to the tenant chain). `audit_verify_chain(workspace)` recomputes hashes — tampering is detectable (tested). Secret-shaped keys are scrubbed to `[REDACTED]` before append (`audit/audit.service.ts`).

### 3.8 Storage encryption (user mandate)
`storage/storage.ts` — client-side envelope: body = `SPENC1 | iv(12) | KEK-wrapped CEK(48) | AES-256-GCM ciphertext+tag`. `S3_KEK` required in production. SSE-at-rest header set additionally. Keys are the **plaintext** sha256 (`sha256/<hash>.<ext>`), so put-dedup survives; `getObject` re-verifies the hash → storage tampering throws. Cross-workspace media dedup is storage-level only; row visibility stays tenant-scoped.

## 4. Connector architecture (WP1)

### 4.1 The contract
`PlatformConnector` (`connector-sdk/src/types.ts`): `manifest()`, `authorize()`, `getIdentity()`, optional `publish/uploadMedia/uploadVideo/getFeed/getComments/reply/getMessages/amplify/unamplify/subscribeWebhooks/handleWebhook/...`. Every list method returns `Page<T, cursor>`.

### 4.2 Manifest & capabilities
`ConnectorManifest` declares `id`, `displayName`, `capabilities` (30+ booleans incl. `amplifyRepost/amplifyQuote/amplifyShare`), `connectionFields` (e.g. Mastodon `instanceUrl`, Telegram `botToken` secret), `oauth` flavor. **The manifest validator rejects password fields outright.** UI renders strictly from capabilities — an undeclared capability renders nothing, never a grayed-out lie.

### 4.3 Registry & the no-branching rule
`ConnectorRegistry.register/get` — unknown provider throws `UnknownProviderError`. The ESLint rule `no-restricted-syntax` (root `eslint.config.mjs`) fails any provider string literal outside `packages/connector-*/`, `packages/testing/`, and `apps/web/src/lib/` (the mock-registry boundary standing in for the real registry until the API wires up — same discipline, documented in the config).

### 4.4 HTTP discipline
`RateLimitedHttpClient`: per-host token bucket (default 1 rps; Mastodon 0.5), exponential backoff with jitter on 429/5xx, `Retry-After` honored.

### 4.5 Amplify truth table (verified per platform, 2026)
| Platform | Repost | Quote | Share | Notes |
|---|---|---|---|---|
| Mastodon | ✅ reblog | ✅ `quoted_status_id` (≥4.5) | — | native scheduling via `scheduled_at` |
| Telegram | — | — | ✅ forward/copy | `has_protected_content` blocks |
| Facebook | ⚠️ no native reshare API | — | link-share approximation | manifest marks degraded |
| Instagram | ❌ | ❌ | own-media stories only | API exposes no amplify |
| Bluesky / Threads / LinkedIn | ✅ / ✅ / ✅(reshare w/ commentary) | ✅ / ✅ / — | — | Level 1b wave |
| Reddit | — | — | crosspost (stable-but-undocumented) + vote | no vote automation ever |
| X | ✅ (metered) | ❌ self-serve (Apr 2026) | — | deferred; pay-per-post model when added |

### 4.6 Contract test kits
`connectorContractTests / videoConnectorContractTests / livestreamConnectorContractTests` — capability-aware: declared capability without implementing method = FAIL; methods behind undeclared capabilities must be absent. Kits caught two real violations during development.

## 5. Publishing pipeline (WP3, sol-delivered & verified)

- **State machine** (`publishing.service.ts`): `editing → in_review → approved → scheduled → published` (+ `rejected`, `cancelled`). Every transition permission-checked (`approval.request` / `approval.grant` / `publication.schedule`) and audited.
- **Durability** (`006_publishing.sql`): `scheduled_publications` carries a `UNIQUE idempotency_key`; the DB row commits first; if Temporal is unreachable the row stays `pending` and the worker's startup sweep picks it up.
- **Temporal** (`scheduler/src/workflows/publish.workflow.ts`): `scheduledPublish(id)` activity (5-min timeout) claims pending-only, resolves the connector via registry, publishes, records the publication, idempotent on re-run.
- **Receipts**: `GET /publishing/drafts/:draftId/publications` returns per-attempt outcomes (`ready/in_progress/completed/failed/unknown/abandoned`, `remoteId/remoteUrl`); the calendar page renders the latest outcome per scheduled draft as a badge with an open-remote link.

### 5.1 Per-connection publish governance (migration 033)
`connections.fields.publishGovernance` decides what happens when an approved draft targets that connection:

| Setting | Behavior on approval |
|---|---|
| unset | today's manual flow: a `publication.publish` holder calls `POST :draftId/publish` |
| `auto` | the approval itself schedules the publish (console approve or passing proposal poll) |
| `admin1` | a second chatgov poll (`subject_kind='draft_publish'`) opens; one admin yes releases |
| `admin_majority` | same, `quorum [{admins, x: floor(n/2)+1}]` snapshotted at open |
| `admin_consensus` | same, `[{admins, x: n}] join all`, veto allowed |

Route: `PATCH /connections/:id/publish-governance` (requires `chatgov.manage`). The publish route answers `202 {state:"awaiting_publish_approval", pollId, closesAt, rule}` while held; a passing poll schedules the publish as the original requester (re-checked at claim). The scheduler's **E3 pre-dispatch check** refuses any dispatch to a gated destination without a passed publish poll for the current approved revision — including schedules created before the gate was switched on. One open publish poll per (draft, destination), index-enforced, never colliding with review polls.

### 5.2 Governed boost (amplify)
`POST /publishing/drafts/:draftId/amplify {connectionId, kind: "repost"|"quote"}` executes the connector's `amplify` against `providerMetadata.sourceRemoteRef` recorded at draft creation — never a caller-supplied id. Target gates: draft state `approved` + the connection's publish governance. A gated boost holds with `202 awaiting_publish_approval` on the same publish-phase poll and is released by re-invoking amplify after the pass (hold-and-retry: no durable amplify queue exists, and the poll-close transaction must not do network I/O). Unset/`auto` connections behave exactly as before (`200` + execute).

## 6. Frontend (apps/web)

Tailwind v4 `@theme` tokens (dark-first OKLCH, `@theme inline` brand hook for tenant theming), hand-rolled primitives, cmdk palette, TanStack Virtual streams (500+ items/column, scroll-into-view), Tiptap compose with per-platform char counters/previews. **Streams flagship:** capability-gated amplify bars, keyboard `j/k` navigate, `r` repost, `q` quote. The Control Room (`app/(main)/control-room/`) runs on the live API — signed preview taps, realtime tally over the websocket client, fixture fallback when offline. QA: `apps/web/scripts/visual-qa.mjs`.

## 7. Testing strategy

| Suite | What it proves | Runs |
|---|---|---|
| connector contract kits | connectors don't lie about capabilities | CI, always |
| pure units (vault, session, oidc, scrub, storage-crypto, key-scheme) | crypto/format invariants incl. tamper detection | CI, always |
| `tenancy.integration.test.ts` | RLS isolation, tenant-branch audit, epoch bump, chain tamper, immutability triggers | needs `DATABASE_URL` (sp_app) + `SUPER_DATABASE_URL` (owner), else skips |
| `storage.integration.test.ts` | S3 roundtrip, dedup, presign | needs `S3_ENDPOINT`, else skips |
| visual-qa (playwright) | screens render, interactions fire, console clean | local/CI with chromium |

Live-DB invocation (from repo root, postgres on :5433):
```
DATABASE_URL='postgresql://sp_app:sp_app_dev@localhost:5433/tmbh' \
SUPER_DATABASE_URL='postgresql://tmbh:tmbh_dev@localhost:5433/tmbh' \
pnpm --filter @tactical-media-broadcast-hub/api test
```

## 8. Observability & ops surface

`/healthz` (liveness), `/readyz` (TCP probes: postgres/redis/temporal with per-component status). OTel OTLP tracing skeleton (disabled-safe without collector). Structured pino logs with credential-key redaction. Grafana (3001) provisioned against Prometheus (9090). Full metrics rollout is WP14.

## 9. Level 2 systems — status

Shipped and wired end-to-end:

- **Media control / director** (`apps/api/src/media-control/`, `apps/api/src/production/`): the `PUT desired` / `take` / `cut` / `pst-select` / `lock` / `override` surface. Each command is a single PostgreSQL transaction — channel row `FOR UPDATE`, idempotency receipt lookup, director-state fold (locked/override), bus moves, capacity admission (§4.4 quota; over-quota rolls back everything, a stop drains the pending queue), desired-version floor across DB+Redis, and an `audit_append` receipt (`director.<op>`, the transactional outbox). The worker observes desired snapshots over signed channels and reports the **realized version**; tally in the UI is worker-observed state, never optimistic.
- **Production spec** (`production.ts`, mirrored in `apps/media-worker`): scenes (`single/pip/2x1/2x2/3x3/4x4/custom`, per-slot `fit`, per-slot audio `off/on/primary` + gain, ≤1 primary slot), sources (HLS URL or video-asset playlist), overlays (`lower_third/chyron/logo/clock`, layers 0–7, rendered revisions as content-addressed PNGs, chyron speed/strip params, server clock), transitions (`cut` or `fade` 300–2000 ms), program master gain. Strict parsers reject injectable strings (`;`, `[`, `,`, `'`, `movie=`) so nothing rides along into FFmpeg graphs.
- **Control Room UI** (`apps/web/src/app/(main)/control-room/`): multiview grid (2x1–4x4/custom, up to 4 live HLS tiles, tap-to-focus, visibility-gated 1 s/2 s preview polling), switcher bar (PST/TAKE-at-frame-boundary/CUT-immediate, lock, override modes `until_released/five_minutes/until_source_offline/until_next_event`), PST preview, audio monitor, overlay editor (`director.change_layout`), fixture mode when offline. Permission gates: `media.view`, `director.view`, `director.take`, `director.lock_program`, `director.override_auto`, `director.change_layout`, `media.manage`.
- **Signed previews** (`production/preview.ts` + controller): capability URLs (expires+HMAC) minted by the authenticated endpoint; the public read route is same-origin only, serves plaintext renders, never redirects encrypted S3 objects, ETag/304-aware.
- **Worker fence** (`apps/api/src/director/`): internal `state/fence/apply` endpoints the worker uses to apply desired snapshots under a fencing token; web ingest, PeerTube and the transcode pool per their own docs.

Still planned: media-engine graph rewiring beyond the current FFmpeg-normalize path, LLM video AI (design only), content search rollout.

## 10. Producer & media ingest (October 2026)

### 10.1 Media attachments for publishing
- `POST /media` (multipart `file`); content type is **magic-byte sniffed** (jpeg/png/webp/mp4/webm) — the extension is never trusted.
- Caps: image **10 MiB**, video **64 MiB**, and at most **4 attachments per post** (`MAX_MEDIA_PER_POST`).
- Read back via `GET /media/:id` (auth-scoped). Bytes live in R2 object storage — SPENC1 envelope-encrypted, content-addressed `sha256` keys (§3.8 scheme).

### 10.2 Alt texts
- Drafts carry `altTexts` positionally aligned with `mediaAssetIds` (0–4 entries; `null` = no alt text for that slot).
- At publish, the scheduler maps them into `providerMetadata.altText` (asset-id keyed); Mastodon media uploads receive them as descriptions.

### 10.3 Video producer
- `POST /video/:id/produce` accepts `{ overlays?, voiceover?, outputLabel?, lane? }` — closed enums with a strict key allowlist; unknown keys are rejected. Overlay kinds: `lower_third | logo | clock`; overlay text runs through the filtergraph-metacharacter blocklist before it reaches an FFmpeg graph.
- Jobs execute in the **produce-worker** (its own container, `MEDIA_WORKER_LANE=local`): claim/lease via polling with `SKIP LOCKED`; stale leases are re-queued.
- The output is a **derived library asset** (`derived_from`), publishable like any other asset.

### 10.4 Audio-less sources
- The command-builder synthesizes an `anullsrc` silence bed — configured at the output preset's rate/layout — instead of emitting `[0:a]` maps that ffmpeg rejects.

### 10.5 Chunked video ingest
- `POST /video/upload/init` → `PUT /video/upload/:uploadId/chunk/:index` → `POST /video/upload/:uploadId/complete`.
- 10 MiB chunks (`CHUNK_SIZE_BYTES`) sidestep Cloudflare's ~100 MB proxy body cap. The server reassembles, seals, stores to R2, then runs the normal probe pipeline.
- Max chunked upload: **100 GiB** (`MAX_CHUNKED_UPLOAD_BYTES`).

### 10.7 Direct-to-R2 video ingest (worker-probed)
- `GET /video/direct/config` → `POST /video/direct/init` (presigned PUT) → `POST /video/direct/complete` (HEAD-gated register + queued probe job). The browser seals SPENC2 and PUTs straight to object storage — the api box never touches video bytes.
- The produce worker claims the probe job (042 widened `produce_job_claim`), fetches the sealed object off R2, opens by magic (SPENC1 kek / SPENC2 direct key via `DIRECT_EC_PRIVATE_KEY`), ffprobes, writes metadata, completes. Compose/mix jobs read direct objects the same way.
- Requirements: `DIRECT_EC_PRIVATE_KEY` on api + media-worker + produce-worker (compose appenv).

### 10.8 Files direct-to-R2 + media viewer
- `GET /files/direct/config`, `POST /files/direct/init|complete`, `POST /files/:id/direct-download` (presigned GET + CEK ticket; the browser decrypts — the server never streams direct/ payloads).
- Viewer suite: image lightbox (zoom/pan/rotate + filmstrip), brand video player (scrubber/buffered/volume/speed/PiP/fullscreen + keyboard), image EDITOR (crop with aspect presets, rotate 90s, flip H/V, brightness/contrast/saturation — saves as a NEW library file), video TRIM (in/out → the cut renders into the Video library via the assemble lane; the original is never modified), PDF (native iframe), text (+copy), HTML (sandboxed `allow-scripts` run + source toggle).
- Thumbnails decrypt client-side; video stills are captured in-browser. Session cache dedupes decryption per item.
- Browser memory caps the direct path at 512 MiB (sealing holds plaintext + frame); larger files fall back to the server path (Cloudflare-capped ~100 MB) until multipart lands.

### 10.9 Multi-file upload queues
- Both dropzones accept multiple files: sequential queue, per-file status (queued / % / done / failed), one failure never aborts the queue.
- Video per-file transport: direct (when enabled, ≤512 MiB) → chunked (>10 MiB) → whole-body (≤10 MiB). Files per-file transport: direct (when enabled, ≤512 MiB) → server multipart.


## 11. Key file map
- Whole-body video upload cap: **2 GiB** (`MAX_VIDEO_UPLOAD_BYTES`).
- Files group upload cap: **2 GiB** (was 20 MiB).

## 11. Key file map

| Concern | Path |
|---|---|
| Tenant context / pools | `apps/api/src/db/client.ts` |
| Migrations 001–034 | `apps/api/migrations/` |
| Vault / storage crypto | `apps/api/src/vault/vault.ts`, `apps/api/src/storage/storage.ts` |
| Sessions / OIDC provider (issuer + refresh) | `apps/api/src/auth/`, `apps/api/src/oidc/` |
| RBAC guard | `apps/api/src/rbac/rbac.guard.ts` |
| Audit chain | `migrations/001_tenants.sql` (SQL) + `apps/api/src/audit/` |
| Publishing + publish governance + amplify | `apps/api/src/publishing/`, `apps/scheduler/src/activities/dispatch-gate.ts` |
| Media control / director / production spec | `apps/api/src/media-control/`, `apps/api/src/production/`, `apps/media-worker/src/production.ts` |
| Control Room UI | `apps/web/src/app/(main)/control-room/`, `apps/web/src/components/control-room/` |
| Matrix SSO broker | `apps/api/src/matrix-sso/` |
| Connector SDK | `packages/connector-sdk/src/` |
| Frontend mock-registry | `apps/web/src/lib/` |
| Lint: no-provider-branching | `eslint.config.mjs` |
| E2E: governance sim (S1–S9), brand palette guard | `scripts/e2e/governance-sim.mjs`, `scripts/e2e/brand-palette-check.mjs` |