diff --git a/README.md b/README.md index 650f286..5f763d7 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,229 @@ Every feature is multi-tenant with row-level security. - **Archive** — content-addressed encrypted S3, dedup, full-text search, CSV/JSON export. Deleted platform content stays citable. - **Analytics** — month-partitioned snapshots, current + previous month with deltas, per-provider breakdown. +## Architecture + +Three flows explain the whole system. + +### 1. One account + +One person connects a collective's account once. From then on: draft, someone +approves, the Hub publishes. The social network's password never passes through +the Hub — the network returns a token and the Hub stores it encrypted. + +```mermaid +flowchart LR + P([Person]) -->|1. Connect| H[Hub] + H -->|2. Permission screen| R[(Social network)] + R -->|3. Token| H + P -->|4. Drafts| H + H -->|5. Approval| A([Another crew member]) + A -->|6. Sign-off| H + H -->|7. Publishes on time| R +``` + +### 2. Several shared accounts + +Each collective has its own accounts and approval rules. A draft can go to +several accounts at once: the Hub creates one copy per destination and each +passes its own account rule (automatic, one person, majority or consensus). +Anything meant for the whole network rises to the parent collective's queue. + +```mermaid +flowchart TB + subgraph TTS[NETWORK QUEUE] + Q[Shared queue] --> IG1[Instagram international] + Q --> M1[Mastodon @network] + end + subgraph S26[COLLECTIVE A] + D[Draft] --> IG2[Instagram @collectiveA] + D --> M2[Mastodon @collectiveA] + D --> T2[Telegram collectiveA] + end + subgraph SUB[Local collectives] + B[Barcelona] --> IG3[Own Instagram] + S[Sevilla] --> IG4[Own Instagram] + end + S26 -->|Uploads for the network| Q + SUB -->|Uploads to A| D + IG2 -.->|Rule: 2 approvals| G{{Approval}} + M2 -.->|Rule: automatic| G +``` + +### 3. The technical half + +The web only talks to the API. The API keeps state in Postgres with +per-collective row-level security, credentials live in an encrypted vault, and +scheduled publishing runs through the scheduler with Temporal. Each network is +a connector; outside the connectors there is no code that distinguishes +networks. Media bytes are end-to-end encrypted (SPENC1 server-sealed, +SPENC2 browser-sealed) in content-addressed R2 storage; heavy processing runs +in workers that pull sealed jobs — the api box never touches plaintext. + +```mermaid +flowchart LR + subgraph Client + W[apps/web — Next.js] + end + subgraph Control + API[apps/api — NestJS] + SCH[apps/scheduler — Temporal] + REG[ConnectorRegistry — packages/connector-*] + PW[produce-worker — probe / compose / assemble] + end + subgraph Data + PG[(PostgreSQL — RLS per collective)] + RD[(Redis — OAuth state 10 min)] + V[Vault — AES-256-GCM + KEK] + S3[(R2 — SPENC1/SPENC2 sealed media)] + end + subgraph Media + MW[apps/media-worker — FFmpeg] + PT[PeerTube — tube.occupymedia.cc] + end + subgraph Networks + MA[Mastodon API] + IG[Instagram Graph API] + TG[Telegram Bot API] + end + + W -->|HTTPS + session| API + API --> PG + API --> RD + API --> V + V --> PG + API -->|connect / identity| REG + SCH -->|publish| REG + SCH --> PG + W -->|SPENC2 sealed| S3 + PW -->|decrypt + render| S3 + PW --> PG + REG -->|providerFetch egress control| MA + REG --> IG + REG --> TG + MW --> S3 + MW -->|RTMP| PT +``` + +Connecting an account (OAuth with one-time `state` and PKCE where the network +supports it): + +1. `POST /connections/authorize`: the API stores the request state encrypted in + Redis and returns the network's URL. +2. The person accepts on the network, which returns to + `GET /connections/callback`. +3. The API checks `state`, cookie and permissions, redeems the code and stores + the token in the vault. + +## Network status + +| Network | Connect | Publish | Notes | +|---|---|---|---| +| Mastodon | Yes | Text | No server keys; each account registers the app on its instance. | +| Telegram | Yes | Text | Bot token from BotFather. | +| Instagram | Yes, with `INSTAGRAM_CLIENT_ID/SECRET` | Not yet | Meta fetches the image from a public URL and media are encrypted: a temporary endpoint is pending. See [docs/oauth-connect.md](docs/oauth-connect.md). | +| PeerTube | No | Live only | The OAuth bridge on tube.occupymedia.cc is pending. | +| X | No | No | No connector. Paid per-use API. | +| TikTok | No | No | Out of scope: the API requires an audit; without it only private publishing. | + +## Outreach (hand-written DMs) + +A list of public accounts (artists, collectives, media) to write to privately +asking for amplification. Each contact has its drafted message, fillable from a +template (`{name}`, `{handle}`), marked sent / replied / declined. CSV import: + +``` +name,platform,handle,profileUrl,category,language +Climáximo,bluesky,climaximopt.bsky.social,https://bsky.app/profile/climaximopt.bsky.social,climate,pt +``` + +The Hub **does not send** the messages. "Copy & open profile" copies the text +and opens the profile; the person pastes and sends it from the network's own +DM box. Reasons: + +| Network | Open a DM via API | Problem | +|---|---|---| +| Instagram | No | The API only allows replying to whoever wrote you in the last 24 h. | +| X | Yes, paid | Mass DMs to strangers break automation rules: suspension. | +| Mastodon | Yes (private mention) | Instances mute or block accounts doing it in bulk; the whole instance can suffer. | +| Bluesky | Yes | Same spam criteria; stranger chats are usually closed. | + +The screen counter warns from 20 first messages per day per collective. +Spread them across people and days, and personalise each one. + +## Requirements + +Measured on the current deployment. Media goes to R2, so the disk does not +grow with videos. + +| Tier | CPU | RAM | Disk | For | +|---|---|---|---|---| +| Minimum | 4 vCPU | 8 GB | 160 GB SSD | The whole stack, transcoding off-box | +| Recommended | 8 vCPU | 16 GB | 240 GB SSD | Livestreams, on-box transcoding, Collabora | +| Render node (optional) | 4+ cores or a GPU with NVENC | 8 GB | 50 GB | Joins over Tailscale; never holds keys | + +Real usage today: ~2.6 GB of RAM across all services. + +Bandwidth: + +| Use | Needed | +|---|---| +| Panel, publishing, network APIs | < 1 Mbps sustained | +| One outgoing live (1080p30 to PeerTube) | 6 Mbps upload per live; 3 Mbps at 720p | +| Viewers | Served by Cloudflare from the HLS cache; the origin does not scale with the audience | +| Video uploads | Bursts as fast as the line allows; 100 GiB per file via chunked ingest | + +## Network and ports + +Inbound from the internet: **none direct**. Everything enters through the +Cloudflare tunnel (with an ingress failover to a standby proxy); the edge +nginx only accepts Cloudflare ranges on 80/443. + +| Port | Service | Expose to | +|---|---|---| +| 443/tcp | Edge nginx | Cloudflare ranges only | +| 80/tcp | Edge nginx (redirect) | Cloudflare ranges only | +| 3000/tcp | API | 127.0.0.1 only | +| 3001/tcp | Web | 127.0.0.1 only | +| 9000/tcp | PeerTube | 127.0.0.1 or Tailscale | +| 1935/tcp | PeerTube RTMP | Tailscale only (media worker) | +| 6167/tcp | Matrix (conduwuit) | 127.0.0.1 | +| 9200, 9980/tcp | oCIS, Collabora | 127.0.0.1 | +| 5432, 6379, 7233 | Postgres, Redis, Temporal | Never outside the Docker network | + +Required egress: 443/tcp to the networks' APIs and to R2, 7844/tcp+udp for +`cloudflared`, 41641/udp for Tailscale. + +Docker publishes ports bypassing ufw. To close them: publish on +`127.0.0.1:PORT`, on the Tailscale IP, or add rules to the `DOCKER-USER` chain. + +## Security and monitoring + +Practices for any deployment. Report security failures privately to the +maintainers — never in public issues. + +- **Invite-only registration.** Open registration hands anyone a collective of their own with Owner rights on it. +- **One secret per service.** The web and workers never need `SUPERPOSTER_KEK`, `S3_KEK` or `SESSION_SECRET`. Different keys for the vault and for media. +- **FFmpeg always sandboxed** (`NODE_ENV=production`, `MEDIA_SANDBOX=on`), no network, no inherited environment variables. +- **Headers**: strict CSP, HSTS, `X-Content-Type-Options`, `Referrer-Policy` on web and API. +- **Request limits** on `/auth/*`, invitations, OIDC and public uploads, keyed by `CF-Connecting-IP`. +- **Metadata out**: strip EXIF/GPS from photos and videos before storing or publishing. +- **Encrypted backups** (`age` or GPG) kept off-machine; test the restore. +- **Tamper-proof audit**: revoke `UPDATE`/`DELETE` on `audit_log` from the `sp_app` role and anchor the chain head off-box. + +What to watch (Prometheus + Grafana ship in `infra/`; Grafana never anonymous): + +| Signal | Alert if | +|---|---| +| `401`/`403` on `/auth/login` per IP | > 20 in 5 min | +| New registrations / redeemed invitations | Anything unplanned | +| `audit_log` events: `member.*`, `connection.removed`, `group.deleted`, `outreach.removed` | Spikes or odd actors | +| `connections` in `needs_credentials` state | Any: token revoked or expired | +| API 5xx rate | > 1 % over 5 min | +| API and worker memory | > 80 % | +| Publishing queue `failed`/`unknown` | Any | +| Cloudflare panel logins and DNS changes | Always | + ## Quickstart ```bash