Hey everyone! I've recently set up two HPE SN3420M switches for a data center environment, operating in MLAG, and connected to four Windows 2025 servers via Broadcom NICs. While we've optimized our transfer rates with iperf and ntttcp, I'm noticing fluctuating ICMP response times between hosts that are directly connected to these switches. I'm curious if it's typical to see these kinds of ICMP ping times, and I've read conflicting views about using ICMP to gauge network performance. Here are some ping results I'm getting:
Pinging a host at XX.XXX.XX.212 shows response times mostly around 1ms, but they fluctuate:
- Values range from approximately 0.9ms to 1.3ms
Is this normal for such a high-speed switch setup? Any insights would be appreciated! Cheers!
4 Answers
I've seen similar fluctuations with different transceivers, especially when they were only handling about 17GB or 25GB of traffic. It might be worthwhile to run iperf on each individual interface to see if one particular path is underperforming. Those response times you're seeing, mostly around 1ms, seem okay to me; it's just a ping response, not a full bandwidth measure. Have you tried swapping out for a DAC cable to rule out layer 1 issues?
Which program are you using for ping tests? Have you tried connecting hosts directly to each switch and running ping tests between them? Examining packet captures could also reveal more clues about what's happening.
When you mention high ICMP response times, it's important to consider that many network devices prioritize other traffic over ICMP, which might be causing these delays. For example, in my setup with Catalyst switches, I notice that echo replies can vary significantly, and pinging through the switch often results in lower latency. Make sure you're checking the response times from host to host through the switch. If they exceed 1ms, that's unusual and warrants more investigation, especially with your MLAG setup. Also, what model Broadcom NIC are you using?
While ICMP isn't the best metric for measuring performance, the times you're reporting are a bit higher than expected for that switch. You might want to investigate further.
We are using 25GBPS DAC cables for the connections, no transceivers involved. I've noticed that this fluctuation occurs across all the hosts connected to the switches. I expected a more consistent response time around 0.3ms. Should I adjust my expectations?