I run an internet-facing file storage server for customers, so throughput matters. I've been studying Linux kernel networking behavior and sysctl documentation rather than blindly applying tuning guides, and I'm unsure about net.ipv4.tcp_mtu_probing.
Many guides recommend setting it to 1 or 2 instead of the default 0. If enabling it is generally better, why isn't it the default? My concerns are that MTU probing might mask an underlying path-MTU problem, interpret ordinary packet loss as an MTU issue and hurt congestion control, or trigger problems with firewalls that dislike packets of varying sizes.
Should I leave it disabled, enable it globally, or use it only after observing specific PMTU-related failures?
3 Answers
For most public internet traffic, start with the normal interface MTU and let TCP use the MSS advertised by each client. Don’t assume every customer path supports the same maximum packet size, and don’t optimize for jumbo frames unless you control the entire local path.
If you see connections that work briefly and then hang, repeated retransmissions at a consistent packet size, or evidence that ICMP PMTU errors are being blocked, then tcp_mtu_probing may be worth testing. Setting it to 1 is the cautious option because probing is activated when the kernel suspects a PMTU black hole; setting it to 2 forces probing more broadly and should have a specific justification.
Test from real customer networks or representative providers rather than relying only on a local benchmark. There is no single setting that will be optimal for every path to an internet-facing server.
I’d leave it at the default unless you have evidence of a PMTU black-hole problem. tcp_mtu_probing is primarily a resilience workaround, not a general performance optimization.
Normal Path MTU Discovery uses ICMP “Fragmentation Needed” messages. If those messages are being filtered by a firewall, NAT device, or router, a connection can stall because the sender never learns that packets are too large. MTU probing provides an alternate way for TCP to infer a usable packet size, so it can keep working despite broken ICMP handling.
If PMTU Discovery is functioning normally, enabling probing adds another way for packet loss or transient network problems to be interpreted. Measure before and after any change, and only keep it if it fixes a reproducible problem.
That’s also my rule for networking sysctls: don’t tune around a hypothetical problem. Test with representative client paths and monitor retransmissions, stalls, throughput, and the negotiated MSS.
Your concerns are partly right, but the important distinction is between ordinary ICMP-based PMTU Discovery and packetization-layer probing. Probing does use successful delivery and loss of deliberately sized packets to estimate the path limit, so congestion or unrelated loss can potentially confuse it. That tradeoff is why it isn’t universally enabled by default.
However, a smaller path MTU is not necessarily a defect you can fix. Customers may connect through tunnels, VPNs, PPPoE, unusual access networks, or paths that change during routing failures. You may not control the link where the smaller MTU exists. In those cases, probing can be a useful fallback rather than simply hiding a problem.
Variable TCP segment sizes are normal, and ordinary firewalls generally do not reject packets merely because their sizes differ. The bigger concern is misdiagnosing loss, not firewall hostility. For a public service, a conservative default with good monitoring is usually preferable to forcing probing everywhere.
Exactly. If the service needs different behavior for different regions or network types, routing, regional endpoints, or separate service tiers may help more than one global sysctl value.