glass beadian
aa5e262f3a
Self-hosted multi-platform broadcast newsroom for activist collectives: publishing pipeline with group approval and DST-proof scheduling, TweetDeck-style column wall with hostile-content-safe rendering, live control room with Auto Director, bridged chat governance with polls and quorum, distributed sandboxed transcode workers, encrypted content-addressed archive, monthly analytics. MIT licensed. See README.md and docs/.
22 lines
2.6 KiB
Markdown
22 lines
2.6 KiB
Markdown
# Decision: Content intake posture — 2026-10-09
|
||
|
||
**Decision:** metadata-only + community submissions + Tier-1 sanctioned sources. **No gray-market account farming.**
|
||
|
||
## Context
|
||
The collective wants an intake firehose: continuous content ingestion from social platforms for media work — hashtags, accounts, search — feeding Streams columns, Compose, and the archive.
|
||
|
||
## Research base (live-verified 2026-10-09; the research archive lives outside this repository)
|
||
- `2026-10-09-intake-firehose-architecture.md` — per-source feasibility, verified volumes (Jetstream ~52 posts/s measured, Mastodon 300 req/5min anon verified, YouTube channel RSS keyless verified), architecture, storage math (~37 GB/day modeled), phased rollout.
|
||
- `2026-10-09-scraping-feasibility.md` — farm-account + browser-automation scraping: every platform rated Brittle or worse; enforcement record (X C&D'd Nitter Aug 2026; Meta won injunctions specifically on fake-account scraping).
|
||
- `2026-10-09-gray-market-services.md` — the vendor layer exists and is priced ($1–50 aged accounts, SMS pools, captcha farms, proxies, anti-detect browsers, pre-scraped datasets) but delivers +15–20% coverage for 10–30× the cost of the alternative posture, with 60–90% farm-detection probability within a year, material contagion risk to legitimate accounts, and documented identity-crime supply chains.
|
||
|
||
## Decision
|
||
1. **Tier 1 (build now, zero ban risk, no accounts):** RSS/Atom (incl. YouTube channel RSS), Mastodon public tag/account timelines, Bluesky Jetstream with server-side keyword gate, PeerTube REST, per-URL public fetchers (gallery-dl/instaloader-class).
|
||
2. **Tier 2 (small bounded pool, later):** 1–3 Telegram MTProto session accounts for public channel reading (quiet, warmed, never org-linked); optionally 3–5 read-only X accounts, dedicated egress each, 90-day sunset, kill switches.
|
||
3. **Rejected:** mass account farming, hashtag firehoses via farms, gray-market datasets, managed-scraping retainers — 10–30× cost, existential tail risk, documented identity-crime supply chain (see gray-market ledger §9.4).
|
||
4. **Per-URL ingestion** covers locked platforms: community-submitted public post URLs are fetched individually (gallery-dl/instaloader-class) — categorically different from authenticated crawling.
|
||
|
||
## Consequences
|
||
- Intake adapters draw from `packages/egress` per-host budgets.
|
||
- Locked-platform columns show community-submitted + metadata-only content; full firehose coverage concentrates on sanctioned sources.
|
||
- Decision revisitable if platform APIs open or a funded data vendor passes legal review.
|