T-HUB/packages/egress/README.md

# @tactical-media-broadcast-hub/egress

Shared HTTP(S) egress policy and PostgreSQL host budgets for WP7 callers. This slice
only adds the package; API and media-worker endpoint validators are migrated later.

```ts
import { Pool } from "pg";
import { createEgressClient, EgressError } from "@tactical-media-broadcast-hub/egress";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const { egressFetch, policy } = createEgressClient({ pool });
const response = await egressFetch("https://instance.example/api/v1/videos", {}, {
  hostClass: "monitor",
  pinnedOrigin: true,
});
```

The caller owns/closes the pool. `egressFetch(url, init, {pool, hostClass, ...config})`
is also exported for one-off calls. `createEgressClient` reuses configuration and
returns `{egressFetch, policy, budgets}`. No global pool, application migration or
in-process budget is created. Requests use Node HTTP(S), bypass proxy environment
variables, and open a fresh connection for each hop. There is no hidden retry.

## Policy and requests

`validateUrl(url, options)` and `policy.validateUrl(url)` check syntax, literals and
ports without DNS or database access. `await policy.validate(url)` returns
`{url, pinnedAddress, family, hostKey}` after resolving once, rejecting every
non-public answer (including mixed public/private answers), and choosing one
address. The request lookup returns only this IP. The original hostname remains
in Host and TLS certificate verification/SNI. A failed address is not replaced
by an unvalidated address.

HTTP port 80 and HTTPS port 443 are allowed by default. Operator configuration can
set per-scheme allowlists, e.g. `ports: {"https:": [443, 8443]}`. Credentials in
URLs, localhost names, whitespace/control characters and backslashes are rejected.
Query strings are supported; fragments are stripped. Both local blocklists are
covered, including metadata/link-local, CGNAT, reserved/documentation/benchmark,
multicast, IPv4-mapped IPv6 and IPv6 transition ranges. Legacy IPv4-compatible,
site-local and local-use NAT64 ranges are also denied.

301/302/303/307/308 redirects get fresh validation, DNS pins and budget consumption
per hop. Cycles and more than five redirects fail. Fetch `redirect: "manual"`
returns the redirect without following; `"error"` rejects it. Cross-origin
redirects strip Authorization, Proxy-Authorization and cookies; redirects that
would replay a body across origins are rejected. Use `pinnedOrigin: true` for
secret-bearing requests and discovery/JWKS fetches: every off-origin redirect is
rejected before DNS or connection, including a different scheme/port. A URL value
for `pinnedOrigin` also checks the initial request against that origin.

Responses are buffered into a standard `Response` with final `url` and
`redirected`. Defaults: 15-second total deadline (DNS, DB, redirects and body),
8 MiB request/response limits, five redirects. Configure `timeoutMs`,
`maxRequestBytes`, `maxResponseBytes`, `maxRedirects`; caller abort signals are
honored. The transport requests identity encoding and rejects upstream compressed
bodies rather than returning incorrectly decoded bytes. It does not maintain a
cookie jar. No application-specific authorization is inferred from `hostClass`.

`EgressError.code` distinguishes `DENIED_PRIVATE_ADDRESS`, `SCHEME_DENIED`,
`PORT_DENIED`, `DNS_FAILED`, `BUDGET_EXHAUSTED`, `REDIRECT_LOOP`, `ORIGIN_DENIED`,
`TIMEOUT`, `ABORTED`, size limits and invalid input/configuration. Budget errors
include `retryAfterMs`; messages omit URLs and upstream exception strings.

## Migration and shared buckets

The application migration owner runs `migrations/001_host_budgets.sql` inside a
transaction, recording **`@tactical-media-broadcast-hub/egress/001_host_budgets.sql`** in the
existing `schema_migrations(name, applied_at)` ledger in the same transaction.
Use a namespaced ledger key to avoid colliding with the API's `001_tenants.sql`.
The SQL is exported via `@tactical-media-broadcast-hub/egress/migrations/001_host_budgets.sql`.
It neither creates nor writes a separate migration ledger. Grant the runtime
pool role SELECT/INSERT/UPDATE on `host_budgets` as part of application deployment.
This is infrastructure state, so no workspace column or RLS policy is installed.

Keys are IDNA/punycode lowercase `host:effective-port` (IPv6 bracketed); trailing
DNS dots are stripped. All callers, replicas and host classes draw from the same
row for a key. Default capacity is 3, continuous refill is 10 tokens per fixed
10-second interval. `hostClasses` configures initial capacity/refill for new hosts,
e.g. `{monitor: {capacity: 3, refillPerInterval: 10}}`. The first initializer wins;
later callers cannot enlarge or reset an existing bucket. Administrative changes
to existing rows must be applied explicitly in the database.

Initialization, row locking, refill and conditional UPDATE/debit occur in one
transaction. Time is sampled from PostgreSQL after acquiring the lock, so worker
clock skew and lock waits cannot overfill the bucket. The optional `now` callback
is a test clock; omit it in production.

429, Retry-After (seconds or HTTP date), and X-RateLimit-Remaining=0 park tokens at
zero using a future `updated_at`. X-RateLimit-Reset accepts Unix seconds,
milliseconds, or small delta-second values. Missing/invalid 429 hints default to
ten seconds. Parking deadlines never shorten due to concurrent responses; refill
starts at the deadline and the next token becomes available after its refill
time. Remaining/Limit hints only tighten tokens/capacity/rate; absent a longer
reset window, Limit is conservatively treated as tokens per ten seconds. Hints
never increase local limits. Parking is applied at response headers, even if the
body later fails its size limit. Callers decide when to retry budget errors.

## HLS

`await validateMasterPlaylist(text, playlistUrl, policy)` requires a master
playlist and returns validated/pinned child endpoints, including variant lines,
renditions, iframe variants, keys/maps and other URI-bearing attributes.
`await rewriteSegmentUris(text, playlistUrl, policy)` accepts master/media
playlists, validates their base and every child, and returns absolute URLs.
Relative, root-relative, protocol-relative and absolute references are supported.
Quoted commas, other attribute values, comments and LF/CRLF style are preserved.
Content-steering SERVER-URI is validated too. Malformed/unquoted/duplicate URI
attributes and unexpanded HLS variables are rejected.

Rewritten URLs must still be fetched through `egressFetch`, which validates and
pins at connection time. These helpers do not fetch playlists or hand remote
URLs to media processes. Relay-local IDs and RTMP/SRT mediation belong to later
integration work.

## Verification

From this package, with Node 22+ and pnpm available:

```sh
pnpm install --filter @tactical-media-broadcast-hub/egress --lockfile=false --ignore-scripts
pnpm exec tsc --noEmit
pnpm exec vitest run
```

Database tests use a unique temporary schema and drop it afterward. Set
`EGRESS_TEST_DATABASE_URL` to a PostgreSQL 15+ database whose test role can create
schemas; the default is the repository Compose development database at
`postgres://superposter:superposter_dev@127.0.0.1:5433/superposter`.
No production tables or application migrations are touched. Tests cover the
address matrix, DNS pinning/rebinding, redirect policy, PostgreSQL concurrency,
fake-clock refill, Retry-After/rate-limit parking, request caps and HLS rewriting.