Tactical Media Broadcast Hub — initial release
Self-hosted multi-platform broadcast newsroom for activist collectives: publishing pipeline with group approval and DST-proof scheduling, TweetDeck-style column wall with hostile-content-safe rendering, live control room with Auto Director, bridged chat governance with polls and quorum, distributed sandboxed transcode workers, encrypted content-addressed archive, monthly analytics. MIT licensed. See README.md and docs/.
This commit is contained in:
commit
aa5e262f3a
880 changed files with 111042 additions and 0 deletions
190
infra/peertube-workers/WINDOWS.md
Normal file
190
infra/peertube-workers/WINDOWS.md
Normal file
|
|
@ -0,0 +1,190 @@
|
|||
# Windows native NVENC worker
|
||||
|
||||
Use Windows 10/11 x64 with an NVIDIA GPU and a working driver. WSL is the SSH
|
||||
transport only: Node, FFmpeg, the shim, admission, and IPC all run as Windows
|
||||
processes. The initial worker name is `worker-laptop-win`; root is `C:\pt-worker`.
|
||||
The canonical instance is `https://tube.occupymedia.cc`; worker traffic uses
|
||||
`http://TAILSCALE_IP:9000` inside encrypted Tailscale. Viewers use PeerTube.
|
||||
No server configuration change is needed: remote VOD is already enabled.
|
||||
|
||||
## Prepare and enroll
|
||||
|
||||
Run the installer from this directory. Python 3.12+ and Windows Tailscale are
|
||||
build/access prerequisites. Python is embedded in the frozen executables, so
|
||||
services do not depend on the developer's Python installation. Pins are in
|
||||
[win/dependencies.json](win/dependencies.json); the FFmpeg URL is a dated archive,
|
||||
verified by SHA256. The two executables are built from reviewed Python source
|
||||
using the versions in `win/build-requirements.txt`. Runner and IPC dependencies
|
||||
use the existing runtime lockfile. Do not substitute a `.cmd` shim: Node's
|
||||
shell-free spawn rejects it with EINVAL.
|
||||
|
||||
```powershell
|
||||
# Elevated PowerShell: installs Node MSI silently plus isolated Node 22.
|
||||
.\install-peertube-worker-windows.ps1 -Phase Prepare -Name WORKERNAME-win
|
||||
# A filtered SSH/WSL token can prepare everything except the MSI:
|
||||
.\install-peertube-worker-windows.ps1 -Phase Prepare -UserLevel -Name WORKERNAME-win
|
||||
.\install-peertube-worker-windows.ps1 -Phase SelfTest
|
||||
```
|
||||
|
||||
Both H.264 and HEVC must produce a real 5-second 1080p encode **and positive
|
||||
NVIDIA encoder utilization**. GPU identity, UUID, driver, boot, device index,
|
||||
exit codes, samples, and session statistics are recorded in `state\nvenc.json`.
|
||||
No proof means no enrollment. The installer keeps admission PAUSED. A failed
|
||||
init, absent telemetry signal, reboot, or driver mismatch closes admission.
|
||||
Session-0 proof runs again when services are installed; enrollment-session
|
||||
success alone does not prove service-context access.
|
||||
|
||||
Supply the **existing** registration token through stdin. Never pass it as a
|
||||
PowerShell/Node argument, environment variable, transcript, or source file.
|
||||
For SSH transport, pipe PostgreSQL's selected token output directly into the
|
||||
Windows Node `bin\ipc-nvenc.cjs register` process. The version-controlled
|
||||
`win/register-from-ssh.py --worker USER@HOST --name NAME` implements this using two subprocesses and pipe handles; it fetches
|
||||
`runnerRegistrationToken.registrationToken`, database `peertube`, using the
|
||||
PostgreSQL container's existing `POSTGRES_USER`. No token is printed or retained.
|
||||
Before doing so, run `bin\enroll-server.cjs`: it requires concurrency 0 and drops
|
||||
all upstream logs. After registration use `ipc-nvenc.cjs graceful-shutdown`, wait
|
||||
for the listener to exit, and set TOML concurrency to 2. Alternatively use:
|
||||
|
||||
```powershell
|
||||
# The calling process must give this process the secret on inherited stdin.
|
||||
.\install-peertube-worker-windows.ps1 -Phase Enroll -RegistrationTokenStdin
|
||||
```
|
||||
|
||||
The **registration secret** is never saved. The official runner necessarily
|
||||
saves its separate issued **runner credential** in
|
||||
`profile\Roaming\peertube-runner-nodejs\Config\default\config.toml`.
|
||||
NTFS ACLs restrict this tree to the owner, SYSTEM, administrators, and LocalService.
|
||||
Never print that TOML, arbitrary runner logs, database token columns, or HTTP
|
||||
headers. Enrollment pending markers prohibit blind duplicate registrations.
|
||||
|
||||
## Services and operations
|
||||
|
||||
```powershell
|
||||
# ServiceFiles prepares concrete WinSW XML/exe files without touching SCM.
|
||||
.\install-peertube-worker-windows.ps1 -Phase ServiceFiles
|
||||
# Elevated PowerShell; services initially PAUSED, session-0 proof first.
|
||||
.\install-peertube-worker-windows.ps1 -Phase Services
|
||||
C:\pt-worker\pt-worker-win.ps1 status
|
||||
C:\pt-worker\pt-worker-win.ps1 network-test
|
||||
C:\pt-worker\pt-worker-win.ps1 start
|
||||
C:\pt-worker\pt-worker-win.ps1 drain
|
||||
C:\pt-worker\pt-worker-win.ps1 restart
|
||||
C:\pt-worker\pt-worker-win.ps1 self-test
|
||||
```
|
||||
|
||||
WinSW 2.12.0 wraps `pt-runner` (Node supervisor) and `pt-admission` (frozen Python).
|
||||
Both use the [WinSW 2.12 service-account and stop configuration](https://github.com/winsw/winsw/blob/v2.12.0/doc/xmlConfigFile.md), running as LocalService with automatic delayed startup, a 30-second service
|
||||
restart delay, and limited rotated logs. The supervisor uses a further 15-second
|
||||
runner restart delay. It retains only safe milestones and observer events;
|
||||
unfiltered upstream logs are discarded because they can contain tokens.
|
||||
A service stop asks the supervisor to drain via the official named-pipe IPC,
|
||||
with an explicit one-hour SCM stop ceiling. For jobs that can exceed this ceiling,
|
||||
use `pt-worker-win drain`, wait for `runner_running=false`, then stop services.
|
||||
No admission loss forcibly kills an active encode.
|
||||
|
||||
State decisions use the Linux Meter/Admission semantics: AVAILABLE, BUSY,
|
||||
DRAINING, NETWORK_DEGRADED, GPU_UNAVAILABLE, LOW_DISK, THERMAL_THROTTLED, OFFLINE,
|
||||
PAUSED. Status preserves Linux field names (`status`, `disk_free_bytes`,
|
||||
`registration_state`, `last_job_duration_seconds`, `last_job_realtime_factor`,
|
||||
`util`, etc.) and adds `state`, NVENC capability flags, and fleet aliases. QSV
|
||||
flags remain false; a successful NVENC test is never reported as QSV success.
|
||||
The supervisor requires a fresh admission document (<30 seconds), an explicit
|
||||
unpaused control document, and no GPU-refusal marker. Missing/stale admission
|
||||
always triggers the official runner graceful-shutdown request.
|
||||
|
||||
AC policy uses `GetSystemPowerStatus`: battery or unknown AC drains. Thermal
|
||||
zones are read through `root/wmi:MSAcpi_ThermalZoneTemperature` when supported;
|
||||
unavailable CPU sensors remain null. NVIDIA temperature remains a real reading.
|
||||
Defaults: 20 GiB disk reserve, 85 C thermal ceiling, 120 s thermal cooldown,
|
||||
10 Mbps each direction, 1.25x bandwidth recovery margin, three good/two bad
|
||||
samples, 900 s bandwidth expiry, no DERP admission. The existing Tailscale-only
|
||||
server probe (`TAILSCALE_IP:9044`) uses bounded 8 MiB GET download and PUT upload
|
||||
checks, EWMA, and runs while the runner is drained. GPU failure events remain
|
||||
latched until a later real self-test. Boot/driver changes trigger new proof.
|
||||
`powercfg /change standby-timeout-ac 0` is an elevated install step. The laptop
|
||||
must remain on AC; lid-close/Modern Standby behavior still needs owner review.
|
||||
|
||||
## Adapter contract and tests
|
||||
|
||||
The pinned runner requests `libx264`. The executable accepts only reviewed
|
||||
command shapes and rewrites them before executing FFmpeg: CUDA/NVDEC hardware
|
||||
frames -> `scale_cuda` -> `h264_nvenc`, preset p4, preserving bitrate cap/GOP.
|
||||
The original libx264 command is never executed. H.264, SDR HEVC, and VP9 sources
|
||||
have reviewed hardware-decode routes; rotated, HDR, unsupported pixel formats,
|
||||
unknown options, stream codec overrides, extra outputs, and unknown codecs fail
|
||||
closed with exit 78. Audio-only and explicit stream-copy/remux jobs are permitted
|
||||
without video encoding. Static audio-merge images use software image processing
|
||||
and `hwupload_cuda`; video encoding remains NVENC. AV1 input is refused until a
|
||||
reviewed decode route is tested; AV1 capability is not inferred from FFmpeg's
|
||||
encoder list. A local diagnostic can set `PT_NVENC_TARGET=hevc_nvenc`; any other target is refused. Never change the PeerTube service target without a reviewed codec contract. HEVC encoding is proven by the gate but the PeerTube default output
|
||||
remains H.264, matching the pinned runner/server contract.
|
||||
|
||||
```bash
|
||||
python3 -m unittest discover -s infra/peertube-workers/win/tests -p test_nvenc.py -v
|
||||
```
|
||||
|
||||
A running encode that fails, or lasts >=3 seconds without a readable positive
|
||||
encoder signal, returns failure and latches GPU_UNAVAILABLE. Very short encodes
|
||||
may finish within NVIDIA's utilization averaging window; their explicit NVENC
|
||||
encoder and current boot/driver proof remain required. For a real E2E test record
|
||||
positive encode telemetry, job UUID, server Pending/Processing/Completed states,
|
||||
input/output codec and resolution, wall time, RTF, and playable result. Local
|
||||
synthetic success is not a PeerTube E2E result.
|
||||
|
||||
Failure checks: drain -> runner gracefully exits; start/restart -> reconnect
|
||||
milestone; create `state\gpu-unavailable` -> admission GPU_UNAVAILABLE and shim
|
||||
exit 78; remove marker, drain, rerun self-test, then start. Do not unload the
|
||||
owner's desktop GPU driver to simulate failure. Test reboot recovery and AC loss
|
||||
only after services and the owner's physical AC/lid policy can be exercised.
|
||||
|
||||
## Deployment-specific elevation blocker / worker 2
|
||||
|
||||
The verified route is SSH into WSL using `~/.ssh/devserver`, then native
|
||||
PowerShell/Node through `/mnt/c`. Interop works. The Windows token is medium
|
||||
integrity with administrators marked deny-only. `sc create` fails with
|
||||
OpenSCManager error 5; registering a Highest scheduled task returns access denied.
|
||||
Enabling alternate-port Windows sshd needs the same unavailable elevation and
|
||||
would not remove this blocker. No existing recovery task, UAC policy, driver, or
|
||||
server setting was changed to bypass it. The owner must run the prepared Services
|
||||
phase in an elevated Windows PowerShell. An interactive native process deployment
|
||||
can be validated over the working route but is not a boot-persistent service.
|
||||
|
||||
Worker 2 changes only name, root (if wanted), GPU device index, and local network
|
||||
access. Reuse this server probe and existing authorized token; give each worker a
|
||||
unique runner name and isolated profile. Do not copy worker 1's credential TOML.
|
||||
Require a fresh real hardware gate on that GPU and service context. Start with
|
||||
concurrency 2; benchmark equivalent real workloads at 1–4 before increasing it.
|
||||
The RTX 3090's stock session ceiling is not a two-job constraint; no driver patch
|
||||
is deployed. Its measured limit is 350 W; an optional 200–250 W cap needs driver
|
||||
support and throughput/thermal measurement before adopting it. No power limit or
|
||||
driver was changed during deployment. Hardware throughput/session/codec claims
|
||||
from research remain estimates until measured on the specific worker.
|
||||
|
||||
## Authorized E2E fixture
|
||||
|
||||
`win/e2e-peertube.mjs` is an explicit, privileged server test, never an installer
|
||||
autostart step. Run it only for an authorized private upload, with other workers
|
||||
drained if a specific worker must own the job. Supply a synthetic H.264 file as
|
||||
`/tmp/pt-worker-e2e-source.mp4` inside the PeerTube container, then run the script
|
||||
there through Node stdin with `NODE_DB_LOG=false` and `LOGGER_LEVEL=error`. It
|
||||
requires the inspected PeerTube 7.3.0 model layout. It registers the installed
|
||||
models without the initializer's SQL function/extension mutations, creates one
|
||||
10-minute admin OAuth session through the installed token model, performs one
|
||||
private upload through the HTTP API, records safe state metadata, checks private
|
||||
HLS/media HTTP responses and full decoding, then revokes/removes that session.
|
||||
It does not change the root password or runner registration token. Credentials
|
||||
remain in process memory and the temporary normal OAuth database row; they never
|
||||
appear in command arguments, report JSON, or source. Upstream ORM logging is
|
||||
disabled before the token is created. The private test video remains available
|
||||
for owner review; delete disposable server/container scratch files afterward.
|
||||
|
||||
First upload with the target worker drained to capture Pending, then start the
|
||||
worker while the helper polls. Report JSON goes to `/tmp/pt-worker-e2e-result.json`
|
||||
inside the container; it contains no credentials. This host's successful private
|
||||
fixture is [Windows NVENC worker E2E](https://tube.occupymedia.cc/w/7d3EHVeikq3JZ29XqjKJyd),
|
||||
video UUID `324566b4-3506-417d-9cca-cc2a3b7a14cc`. Jobs 1/2 completed on runner 1.
|
||||
The cold WebVideo encode took 36.359 s with positive 100% NVIDIA encoder samples;
|
||||
its warm HLS encode took 2.594 s. These are observed 5-second-fixture timings,
|
||||
not a concurrency benchmark. The short HLS run sampled 0% in NVIDIA's averaging
|
||||
window; its command was still explicitly NVENC, and the WebVideo job supplies
|
||||
mandatory job-correlated GPU proof. Root OAuth sessions were verified back at 0.
|
||||
Loading…
Add table
Add a link
Reference in a new issue