Description
Control plane interfaces (ctlplane TAP, VRF loopback TUN) currently use raw read()/write() on kernel TUN/TAP fds with no offload negotiation. Each packet is a separate syscall limited to MTU size (~1500 bytes). This becomes a bottleneck for control-plane TCP flows (BGP sessions via FRR, route synchronization, etc.).
Linux TUN/TAP supports IFF_VNET_HDR which prepends a virtio_net_hdr to each packet, allowing the kernel to send large (up to 64K) GSO segments in a single read(). Grout can then either forward these to a TSO-capable NIC as-is, or segment in software at the TX boundary.
Proposed implementation
GSO mbuf pool for TUN/TAP interfaces
The TUN/TAP RX path allocates from the per-interface control plane pool (iface->pool). With IFF_VNET_HDR enabled, the kernel can deliver packets up to 64K, so these pools must use large mbufs.
Add a separate pool creation path (e.g. gr_pktmbuf_gso_pool_get()) that allocates mbufs sized to GROUT_MAX_GSO_SIZE (environment variable, default 65536), following the pattern from #697. Assign this pool to iface->pool for ctlplane and loopback interfaces at creation time instead of the normal MTU-sized pool. All other consumers (physical ports, datapath nodes) keep using the normal-sized pools.
IFF_VNET_HDR on device creation
Set IFF_VNET_HDR on both TAP (ctlplane.c) and TUN (loopback.c) devices. After TUNSETIFF, call TUNSETOFFLOAD with TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6 | TUN_F_USO4 | TUN_F_USO6.
RX path (kernel -> grout)
Allocate from iface->pool (now GSO-sized). Use readv() to receive the virtio_net_hdr alongside the packet data. Parse gso_type to set RTE_MBUF_F_TX_TCP_SEG or RTE_MBUF_F_TX_UDP_SEG, tso_segsz, l2_len/l3_len/l4_len on the mbuf. The GSO metadata travels through the graph untouched (needs to be checked, maybe we need to adjust it when encapsulating).
TX path (grout -> kernel)
Always prepend a virtio_net_hdr on write()/writev(). For normal MTU-sized packets, gso_type is VIRTIO_NET_HDR_GSO_NONE. For hairpin GSO packets (VRF-to-VRF), populate the virtio header from ol_flags so the kernel handles segmentation.
Skip MTU check for GSO packets
In ip_output and ip6_output, bypass the MTU check / fragmentation path when the mbuf carries GSO flags, since segmentation is deferred to the TX side.
Software GSO fallback at port_tx
Before rte_eth_tx_burst(), check if the mbuf carries GSO flags and the egress NIC lacks hardware TSO support. If so, call rte_gso_segment() to split into MTU-sized segments. Request RTE_ETH_TX_OFFLOAD_TCP_TSO / UDP_TSO in port txmode.offloads (already masked against NIC capabilities). The output segments come from the port's own MTU-sized pool. Link librte_gso.
Description
Control plane interfaces (ctlplane TAP, VRF loopback TUN) currently use raw read()/write() on kernel TUN/TAP fds with no offload negotiation. Each packet is a separate syscall limited to MTU size (~1500 bytes). This becomes a bottleneck for control-plane TCP flows (BGP sessions via FRR, route synchronization, etc.).
Linux TUN/TAP supports IFF_VNET_HDR which prepends a virtio_net_hdr to each packet, allowing the kernel to send large (up to 64K) GSO segments in a single read(). Grout can then either forward these to a TSO-capable NIC as-is, or segment in software at the TX boundary.
Proposed implementation
GSO mbuf pool for TUN/TAP interfaces
The TUN/TAP RX path allocates from the per-interface control plane pool (
iface->pool). WithIFF_VNET_HDRenabled, the kernel can deliver packets up to 64K, so these pools must use large mbufs.Add a separate pool creation path (e.g. gr_pktmbuf_gso_pool_get()) that allocates mbufs sized to
GROUT_MAX_GSO_SIZE(environment variable, default 65536), following the pattern from #697. Assign this pool toiface->poolfor ctlplane and loopback interfaces at creation time instead of the normal MTU-sized pool. All other consumers (physical ports, datapath nodes) keep using the normal-sized pools.IFF_VNET_HDRon device creationSet
IFF_VNET_HDRon both TAP (ctlplane.c) and TUN (loopback.c) devices. AfterTUNSETIFF, callTUNSETOFFLOADwithTUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6 | TUN_F_USO4 | TUN_F_USO6.RX path (kernel -> grout)
Allocate from
iface->pool(now GSO-sized). Usereadv()to receive thevirtio_net_hdralongside the packet data. Parsegso_typeto setRTE_MBUF_F_TX_TCP_SEGorRTE_MBUF_F_TX_UDP_SEG,tso_segsz,l2_len/l3_len/l4_lenon the mbuf. The GSO metadata travels through the graph untouched (needs to be checked, maybe we need to adjust it when encapsulating).TX path (grout -> kernel)
Always prepend a
virtio_net_hdronwrite()/writev(). For normal MTU-sized packets, gso_type isVIRTIO_NET_HDR_GSO_NONE. For hairpin GSO packets (VRF-to-VRF), populate the virtio header fromol_flagsso the kernel handles segmentation.Skip MTU check for GSO packets
In
ip_outputandip6_output, bypass the MTU check / fragmentation path when the mbuf carries GSO flags, since segmentation is deferred to the TX side.Software GSO fallback at port_tx
Before
rte_eth_tx_burst(), check if the mbuf carries GSO flags and the egress NIC lacks hardware TSO support. If so, callrte_gso_segment()to split into MTU-sized segments. RequestRTE_ETH_TX_OFFLOAD_TCP_TSO/UDP_TSOin porttxmode.offloads(already masked against NIC capabilities). The output segments come from the port's own MTU-sized pool. Linklibrte_gso.