Snipers

Appearance

ADR 0001: WebTransport datagrams for realtime traffic ​

Status: Accepted, implemented (2026-09-05)

Context ​

Netcode quality is the product ("fair shots" is a design pillar). The realtime traffic — inputs up, snapshots down — originally rode the WebSocket, i.e. TCP: reliable, ordered delivery. For a 60 Hz simulation with 30 Hz snapshots, those guarantees are actively harmful on imperfect connections:

  • Head-of-line blocking. When one TCP segment is lost, everything behind it is withheld until the retransmission lands (~1 RTT). One lost snapshot on a 100 ms connection means ~150–250 ms of frozen remote players, then a burst.
  • Retransmitted state is worthless. Snapshot N is superseded by snapshot N+1 before its retransmission arrives; the same goes for inputs (the server already repeats the last held input across gaps). Resending stale realtime state is strictly worse than dropping it.

Browsers offer exactly two UDP-like escapes: WebRTC DataChannels (unreliable mode) and WebTransport (HTTP/3 datagrams). The original plan chose WebRTC for Safari compatibility. WebTransport reached Baseline in March 2026 (Chrome 97+, Firefox 114+, Safari 26.4+), which removed that rationale. Chrome is the primary browser target; broad support still matters, but is guaranteed by a fallback, not by the primary transport.

Relevant architecture at the time of the decision: a fleet of game servers where a directory records which server owns each room, connections reach the fleet through a load balancer (ALB in prod, traefik in dev), and mis-routed WebSockets are forwarded to the owner.

Decision ​

Upgrade realtime traffic to WebTransport datagrams, negotiated over the existing WebSocket, which remains the handshake, control channel, and universal fallback.

  • Handshake: the join flow is unchanged. The welcome message additionally offers wt: { url, hash, token } — the owning server's public UDP endpoint, the SHA-256 of its certificate, and a per-player session token. The client dials WebTransport directly at the owner, then repeats a hello datagram carrying the token until snapshots arrive on the session (hello may drop; the first WT snapshot is the implicit ack). From then on inputs and snapshots ride datagrams and the server stops sending snapshots on the WS.
  • Direct-to-owner UDP. QUIC cannot traverse the ALB (or traefik), so WebTransport bypasses the load-balancer tier entirely. This dovetails with the room-ownership architecture: the WS handshake may be forwarded through a non-owner, but the welcome is authored by the owner, so the WT offer always points at the right server and no UDP-level forwarding is ever needed.
  • Certificates without infrastructure. Each server self-signs an ECDSA P-256 certificate and the client pins it via serverCertificateHashes, receiving the hash over the trusted WSS channel. The spec caps such certs at 14 days validity, so servers rotate them weekly (new welcomes carry the new hash; live sessions are unaffected). No CA, no per-server DNS, no cert distribution.
  • Datagram size. QUIC datagrams don't fragment (~1.2 KB usable). Messages over the threshold are sent one-per-unidirectional-stream instead — still no head-of-line blocking between snapshots, since each stream is independent.
  • Fallback is total and per-player. No WebTransport support, blocked UDP, an attach timeout, or a mid-game session drop all leave (or return) that player on the WebSocket with the exact pre-upgrade behavior. The WS path must therefore remain sufficient on its own, forever.

Value ​

  • Eliminates TCP-induced hitching for players on lossy connections (the netsim harness, ?loss=5, shows the before/after). A lost snapshot now costs one 33 ms snapshot interval instead of a multi-RTT stall.
  • Strictly additive: per-player opt-in with graceful fallback means zero regression risk for unsupported browsers or hostile networks.
  • Far simpler than the WebRTC alternative — no ICE/STUN/SDP, no signaling dance; the client side is a URL, a hash, and two streams.
  • No congestion-control backoff after loss bursts; recovery is immediate.

Tradeoffs and risks accepted ​

  • Server library maturity. Node has no built-in WebTransport; the server depends on a third-party HTTP/3 implementation (native libquiche binding). It is the least battle-tested link. Mitigation: if it fails to load or bind, the server logs and runs WS-only — the game never depends on it.
  • UDP reachability. Corporate/hotel networks and some firewalls block outbound UDP; those players silently stay on WS. This is acceptable by design but means WT adoption should be observed, not assumed.
  • Infrastructure coupling. Servers must be directly reachable on their UDP port from the internet (public IPs, security group open on the WT port). Any future move to private subnets/NAT, or to a proxied topology, breaks the datagram path (but not the game, thanks to fallback).
  • Two live transports to keep working: every netcode change must be tested under both, and the netsim applies at the app layer to both.
  • Transient events can now be lost. Shot tracers and kill-feed entries are flushed into exactly one snapshot; over TCP that was reliable, over datagrams a dropped snapshot drops them forever (a missed tracer or feed line — never scores or health, which are absolute state in every snapshot). Known follow-up: transient events should be repeated across a few snapshots, acked, or moved to the reliable channel.

Constraints future work must respect ​

  • Everything on the datagram path must be loss-tolerant and superseding. Snapshots are dropped by the receiver if older than the newest seen; inputs are dropped by the server if their sequence is not strictly newer than both the last processed and last queued. Any new realtime message must either carry absolute state (safe to lose), or ride the WebSocket / a stream.
  • Keep snapshots under the datagram budget. The stream fallback works but gives up per-packet independence; delta compression (already planned) is also what keeps snapshots in datagram territory as player counts grow.
  • Certificate rules: ECDSA (not RSA), ≤14 days validity, SHA-256 hash — and rotation must outpace expiry on long-lived servers.
  • The welcome is the negotiation point. Anything a client needs to reach its room owner out-of-band (new transports, endpoints, credentials) belongs there, because it is authored by the owner even when the WS is forwarded.

Alternatives considered ​

  • WebRTC DataChannel (unreliable mode): the original plan; broadest legacy support. Rejected once WebTransport went Baseline — WebRTC's ICE/SDP/DTLS machinery and heavier server stack bought nothing we still needed.
  • Stay on WebSocket + app-layer mitigation (delta compression, adaptive interpolation): cheaper, universal, and still worth doing — but it cannot remove head-of-line blocking, only shrink it. Deferred, not rejected; the compression half remains planned.
  • WebTransport via a proxy (keeping the ALB in the path): no AWS-managed HTTP/3-to-target support, and a proxy would reintroduce a hop the direct-to-owner design exists to avoid.

Revisit when ​

  • Telemetry shows low WT adoption (UDP-hostile networks dominating) — the complexity would then serve few players.
  • Node ships native WebTransport (its QUIC core has landed) — swap off the third-party dependency.
  • The topology changes (private subnets, more proxying, non-public tasks) — the direct-UDP assumption is the first casualty.