Remote Play host: hard-coded 512 KB SO_SNDBUF on streaming socket causes packet loss and bitrate collapse at high bitrates

[gehhilfe]

Your system information

• Steam client version (build number): 1785799196
• Distribution: CachyOS (Arch-based), kernel 7.2.0-1-cachyos
• Opted into Steam client beta?: No
• Have you checked for system updates?: Yes
• Session: gamescope-session (gamescope 3.16.25), PipeWire 1.6.8, Mesa 26.2.1
• GPU: AMD Radeon RX 9060 XT (VAAPI hardware encoding, working)
• Network: wired gigabit (Realtek RTL8125B, r8169), verified clean

Issue description

The Remote Play host creates its streaming UDP socket (port 27031) with a hard-coded SO_SNDBUF of 512 KB (ss reports tb524288). At higher stream resolutions/bitrates, a single encoded frame burst — especially recovery keyframes at 1440p/4K — exceeds this buffer. The kernel then silently discards the tail of the burst (netstat -su "send buffer errors" increments), the client observes sequence gaps and requests a keyframe, the keyframe is an even larger burst that overflows again, and the session enters a loss/keyframe spiral. The adaptive bitrate collapses (observed: 2.5 Mbit/s at 4K) and the client reports ~10% frame loss on a network that is demonstrably lossless.

This is easy to misdiagnose as network packet loss: iperf3 shows 0% loss on the same path even at 500 Mbit/s in 300-packet bursts (-b 500M/300), because iperf paces its packets while the video stream sends whole frames as line-rate microbursts (host log shows inst: values of 400–700 Mbit/s while the average is ~10 Mbit/s). NIC and qdisc counters stay clean because the drops happen at the socket layer.

Raising net.core.wmem_default does not help, because the client calls setsockopt(SO_SNDBUF) explicitly, overriding the system default.

The macOS Steam client appears to have the mirror problem on its receive socket (small hard-coded SO_RCVBUF); enlarging it there was also required to fully stop the loss.

Steps for reproducing this issue

1. Linux host with hardware encoding, wired gigabit LAN, any client.
2. Stream at 2560x1440 or 3840x2160, 60+ FPS, manual bandwidth 50+ Mbit/s.
3. Watch netstat -su | grep "send buffer" on the host grow in lockstep with client-reported frame loss, while ethtool -S, tc -s qdisc, and iperf3 (including burst mode) show a clean network.
4. Confirm the socket size: ss -aum 'sport = :27031' → tb524288.

Expected behavior

The streaming socket send buffer should scale with the configured bitrate/resolution (or simply be a few MB) so that a full keyframe burst fits.

Workaround

Raising SO_SNDBUF on the live socket to 8 MB (via pidfd_getfd + setsockopt from a root helper) immediately and completely eliminates the loss: the bitrate then sustains 250 Mbit/s at 1440p60 with zero lost packets on the same setup. Steam re-applies its small buffer whenever it recreates the socket (each Steam start and periodically around session setup), so the helper must re-patch — which also demonstrates the buffer size is the only variable changing.

[kisak-valve]

Previously posted at #9364 (comment).