T-HUB/infra/occupymedia/edge

Edge — second ingress path + DNS failover

The primary ingress is the Cloudflare Tunnel (cloudflared, token-managed — ingress lives in the Zero Trust dashboard). Its edge-IP assignment can go bad for whole regions (2026-10-10: the tunnel hostnames were served from 188.114.96.0/24, blackholed on several networks for hours). This directory is the independent second path and the automation that keeps the site reachable when either path is sick.

The three paths

Mode	DNS	Path	Tradeoff
TUNNEL (default)	proxied CNAME → <tunnel>.cfargotunnel.com	CF edge → tunnel → apps	no inbound ports; edge-pool assignment out of our control
ORIGIN	proxied A → current public IP	CF edge → :443 → nginx edge → apps	needs router forward 443; edge pool may still land badly (pool is per-hostname, opaque)
GREY	unproxied A → current public IP	visitors → :443 → nginx edge → apps	deterministic reachability; origin IP public, no CF WAF

nginx edge mirrors the tunnel ingress by host:

• app.occupymedia.cc → web:3001 (occ-app compose network)
• api.occupymedia.cc → api:3000
• tube.occupymedia.cc → 100.92.231.98:9000 (occ-tube via Tailscale)
• chat.occupymedia.cc → occ-element:80 (client); /_matrix → 100.117.210.52:6167 (occ-chat conduwuit)
• ping.occupymedia.cc → 200 "edge-ok" (activation probe target)
• unknown SNI → ssl_reject_handshake

TLS: Let's Encrypt wildcard *.occupymedia.cc via dns-01 (no port 80 needed). Renewal: occ-edge-certbot.timer, twice daily, reloads nginx.

Chat's signed-out client uses the Hub auth stylesheet on both ingress paths. See the Element auth restyle report for verified 1.12.30 configuration keys, the config merge patch, and exact deployment steps.

Files

File	Role
nginx.conf	edge reverse proxy (deployed to CT 150)
docker-compose.edge.yml	occ-edge-1 on the occ-app_default network
edge-firewall.sh	DOCKER-USER rules: 80/443 from CF ranges only, or OPEN (grey) via firewall.mode
edge-watcher.sh	60s systemd timer: public-IP sync (the IP is dynamic), dual-path health checks, DNS failover with hysteresis
watcher.env.example	CF token + EDGE_ENABLED + TARGET_MODE (secrets stay on the CT)
cloudflare.ini.example	certbot dns-01 credentials (secrets stay on the CT)
occ-edge-*.service/.timer	systemd units for firewall, watcher, certbot

Install (CT 150)

cd /opt/anarchoposter/infra/occupymedia/edge
cp cloudflare.ini.example cloudflare.ini   # + real token, chmod 600
cp watcher.env.example watcher.env         # + real token
docker compose -f docker-compose.edge.yml up -d
sudo cp occ-edge-*.service occ-edge-*.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now occ-edge-firewall.service occ-edge-watcher.timer occ-edge-certbot.timer

Router: forward TCP 443 (80 optional) → the CT's LAN IP.

Activation

The workstation watcher (edge-activate.sh) probes ping.occupymedia.cc through a reachable edge IP. Two consecutive 200s after the port-forward goes live → it sets EDGE_ENABLED=true, flips the four hostnames, verifies the public path for 6 minutes, and reverts to TUNNEL automatically if the new pool assignment is broken too. The CT watcher then owns ongoing failover (2 tunnel failures → ORIGIN/GREY; 3 origin failures → TUNNEL).

Run it from a machine that (a) has the repo .env with the Cloudflare token, (b) can SSH to the CT, and (c) is OUTSIDE the affected region if you need to distinguish "edge pool broken" from "origin unreachable":

TARGET_MODE=ORIGIN ./edge-activate.sh   # or GREY for direct DNS

Target mode: TARGET_MODE=ORIGIN (default) or TARGET_MODE=GREY in watcher.env / the activation environment.

Troubleshooting

• edge.state — current mode + failure counters. edge-watcher.log — every decision. Both in this directory on the CT.
• EDGE_ENABLED=false gates ALL DNS writes — observation only.
• docker logs occ-edge-1 — nginx. docker logs occ-app-cloudflared-1 — tunnel.
• Rollback to tunnel only: ./edge-firewall.sh stays; set mode TUNNEL via the watcher's set_mode TUNNEL <ip> logic or restore the four CNAMEs by hand.