Add video-AI architecture research (transcription, diarization, faces, GDPR)

Research: use PeerTube
native transcription (core since 6.2, whisper-ctranslate2, remote runners on
GPU nodes; large-v3-turbo MIT via model_path); pyannote community-1
diarization (CC-BY-4.0, clean); WhisperX fusion.

Face recognition license audit: NO accurate open FR model has clean weights
(InsightFace/SCRFD non-commercial, yolo-face GPL, AdaFace unstated) —
posture is detection+tracking+blur by default; identity matching opt-in,
human-review-gated, collective decision + DPIA required (Art. 9 biometric
data; automated FR over non-consenting third parties effectively unlawful).

DB DDL sketch: persons/person_face_embeddings/video_face_detections/
transcript_segments/video_speaker_person/identity_audit with review gating,
visibility tiers, erasure workflow.

docs/design/2026-10-11-video-ai-architecture.md

@@ -0,0 +1,53 @@
+Report saved to `research/2026-10-11-av-transcription-diarization-faces-gdpr.md`. Full findings below — everything marked "verified" was fetched from primary sources today (2026-10-11); everything else is explicitly flagged.
+
+# Decision-ready summary
+
+## (a) Transcription — use PeerTube's, don't build one
+
+**Verified from PeerTube v8.3.1 source today** (current release, 2026-09-28):
+- Transcription is **core since 6.2** (`video_transcription:` in `config/default.yaml`), default **off**. Engines: **`whisper-ctranslate2`** (default) or `openai-whisper`; models `tiny|base|small|medium|large-v2|large-v3` — **no `turbo` in the list**, but `model_path:` accepts a CTranslate2 model directory, so drop in `faster-whisper-large-v3-turbo` (MIT) via path. Set `remote_runners.enabled: true` and run the official `peertube-runner` on a Linux 3090 node — same runner contract as your transcode pool.
+- Subtitles land as **VTT** in the caption system (storage prefix `captions/` on R2 is supported — recent changelog fixes caption CORS for object storage). API verified in `server/core/controllers/api/videos/captions.ts`: `GET /api/v1/videos/{id}/captions`, `PUT/DELETE .../captions/{lang}`, `POST .../captions/generate`.
+- **No webhooks in PeerTube (verified — zero hits in repo tree)** → your NestJS api should poll captions-on-publish, or write a tiny plugin (plugin `Hooks` are imported by the caption controller; specific transcription hook names unverified — flagged).
+- Model licensing is clean: **whisper-large-v3 = Apache-2.0, large-v3-turbo = MIT, faster-whisper-large-v3 = MIT** (HF API today). distil-whisper is **English-only → excluded** for ES (card check: marked UNVERIFIED-MEMORY, but nothing suggests multilingual).
+- **Spanish numbers honestly missing**: current HF cards no longer publish FLEURS/CV per-language tables (checked both — absent). From memory: FLEURS-es ~5–7% for large-v3; **expect 15–35% on street audio** (wind/PA/chants). Whisper cards explicitly warn of *"repetitive texts"* (hallucinations) — a real risk on chants/music; VAD + human spot-check needed.
+- VRAM (faster-whisper README, measured on a 3070 Ti): large-v3 int8 batched = **4500 MB, 5× realtime**. Turbo ≈ half that (ESTIMATE). whisper.cpp v1.9.5 (2026-10-06) verified as CPU/Windows fallback with `-ovtt` VTT output, CUDA/Vulkan, Q5_0.
+- Fallback serving if you outgrow the runner: **whisper-asr-webservice v1.10.0** (faster_whisper/whisperx engines, VTT/SRT/word timestamps — README-verified; ideal for media-plane blobs that never enter PeerTube) or **Speaches** (MIT, but v0.9.0-rc.3 — pre-1.0).
+
+## (b) Diarization — pyannote is fine for this use; the gate is not a license
+
+- **`pyannote/speaker-diarization-community-1`: license CC-BY-4.0, gated `auto`** (HF API today); 3.1/segmentation-3.0 are **MIT, gated `auto`**; pyannote.audio code is **MIT** (CNRS). The gate = HF account + accept conditions + read token (card fetch without auth returned 401 — verified empirically). **No non-commercial clause anywhere** → OK for you, add CC attribution to a credits page.
+- Quality (official DER table, community-1 vs 3.1): AMI 17.0/18.8, DIHARD 3 20.2/21.4, **MSDWild 22.8/25.4, Ego4D 46.8/51.2** — the two "wild" benchmarks are the honest proxies for street footage; **no Spanish benchmark exists** (flagged).
+- **WhisperX** (BSD-2, v3.8.6) fuses whisper + word alignment + diarization; its README now requires accepting **`speaker-diarization-community-1`** + `--hf_token`. Spanish alignment model = `VOXPOPULI_ASR_BASE_10K_ES` (verified in `alignment.py`; that model's license unverified — flagged).
+- **NeMo Sortformer weights are CC-BY-NC-4.0 → excluded** (non-commercial; NeMo toolkit itself is Apache-2.0). diart (MIT) is streaming-oriented, maintenance unclear.
+- Speaker→name UX: `speaker_key` per video + one `video_speaker_person` row; confirmed mappings + face-track co-occurrence give cross-video suggestions (label once per speaker).
+
+## (c) Faces — the license audit is the headline
+
+| Component | Code | **Weights** | Verdict |
+|---|---|---|---|
+| InsightFace buffalo_l / ArcFace-w600k | MIT | **NON-COMMERCIAL** — README: models *"available for non-commercial research purposes only"*, applies to pip auto-download too; **2025-11-24 update: contact recognition-oss-pack@insightface.ai for licensing** (both quoted from README today) | ❌ for production index |
+| SCRFD detector weights | MIT code | same non-commercial policy | ❌ if strict / ⚠️ pragmatic |
+| AdaFace | MIT | weights on Google Drive, **no license stated**; WebFace4M/MS1MV2-lineage | ⚠️ gray zone |
+| deepface | MIT | explicitly **inherits** wrapped-model licenses (README quote) — launders nothing | ⚠️ |
+| MediaPipe (BlazeFace detect) | Apache-2.0 | per-file license not clearly stated (flag) | ✅ detection; **no recognition model exists in MediaPipe** |
+| yolov8-face / yolov5-face | **GPL-3.0** | — | ❌ copyleft hazard |
+
+**No accurate FR model in the open ecosystem has clean weights today.** Recommended posture: **detection+tracking+blur by default; identity matching opt-in, human-review-gated, collective decision recorded in writing** (non-commercial terms bar *commercialization* — whether a self-hosted activist tool qualifies is a legal question, not a technical one). Quality limits on your footage: masks (NIST IR 8331 PDF confirmed live at nvlpubs; the "5×–50× more errors" figure = UNVERIFIED-MEMORY, don't quote yet), motion blur, low light, side profiles — these push embeddings into the band where clustering **silently merges the wrong person**, which is your most dangerous failure mode (§GDPR).
+Pipeline: PySceneDetect (0.7.1, BSD-3) → 1–2 keyframes/shot capped at ~2 fps → detect → track → 1 embedding/track → HDBSCAN/Chinese-Whispers offline → incremental assignment vs reviewed centroids (≥τ auto, τ-band → review queue). Cosine threshold ~0.4–0.5 same-person = UNVERIFIED-MEMORY; calibrate on your data. Store centroids in pgvector (512-d fits fine) — reuse the existing pattern.
+
+## (d) Identity model + GDPR (the part that gates everything)
+
+- DDL in the report: `persons` (status `auto|member_confirmed|hidden|erased`, visibility tiers), `person_face_embeddings` (reviewed centroids only), `video_face_detections` (bbox + timestamps + `assignment` state machine + **transient** embedding column wiped by retention), `transcript_segments`, `video_speaker_person`, `identity_audit` (append-only; merge/undo/erasure all reversible). Label once → `UPDATE ... WHERE match_sim >= τ` propagates; τ-band goes to `pending_review`; nothing `auto` ever renders in UI/API.
+- **Verified legal text** (EUR-Lex today): Art. 4(14) definition; **Art. 9(1) prohibits processing biometric data "for the purpose of uniquely identifying"** absent a 9(2) condition — and for bystanders/police, none of (a) explicit consent, (e) "manifestly made public", (f) vital interests, (g) substantial public interest (requires specific law) applies cleanly to an identity index → **an automated FR gallery over non-consenting third parties is effectively unlawful by default; detection+blur that doesn't uniquely identify arguably isn't Art. 9 data at all**. Art. 35(3)(b) makes the **DPIA mandatory** before phase 3. Design mitigations mapped: minimization (coords-not-crops, transient embeddings, pinned models/thresholds), human-review gating, visibility tiers enforced server-side, per-person erasure workflow (index-level; originals stay), audit trail, "high-risk person" flag for police-adjacent IDs, Tailscale-only exposure.
+- **Could not verify this session (flagged):** EDPB Guidelines 05/2022 (all URL attempts 404'd), AEPD guidance + Zaragoza enforcement case (**aepd.es returned 500/503 all day**), NIST mask headline figure. Re-verify before citing any of them.
+
+## VRAM / cost (3090)
+TEI+bge-m3 ~2–3 GB persistent + turbo int8 ~3 GB burst + pyannote ~2–4 GB + faces ~1–2 GB ≈ **11–13 GB concurrent worst case — fits 24 GB, but serialize jobs anyway** (queue, reuse pool-manager pressure signals; keep NVENC transcode floors). Throughput ESTIMATES: turbo ≈ 20–40× RT, large-v3 ≈ 8–12× RT; full pipeline ≈ **5–25 GPU-min per video-hour**; 500-h backfill ≈ 2–4 GPU-days for transcription alone.
+
+## Top unknowns before you commit
+1. One smoke test: does PeerTube's **runner** transcription path pick up CUDA for whisper-ctranslate2, and does `model_path` accept a turbo CT2 dir there?
+2. Internal ES/EN eval set (20–30 real clips) — no published benchmark matches protest audio.
+3. Collective decision + legal advice on FR weights (non-commercial) and on Art. 9 posture **before any phase-3 code**.
+4. WhisperX VoxPopuli-ES alignment weights license; EDPB/AEPD doc URLs (fetch failed today).
+
+Full details, all quotes, DDL, and per-claim source URLs are in the saved report.