Host A in subnet 1 cannot ping host B in subnet 2. The ping times out, and no incoming ICMP packets appear on host B. Host B has a correct but stale ARP entry for its gateway. If B pings the gateway, the entry changes to reachable and A can then reach B. The connection continues working even after the entry becomes stale again, but fails after roughly one or two hours.
There is another strange pattern: host B also has a correct but stale ARP entry for host C in subnet 2. If C pings B, the entry becomes reachable, and A can suddenly reach B. Deleting B's ARP entry for C and then having C ping B produces the same result.
This happens with both Windows and Linux hosts, and it affects multiple systems in subnet 2. There are no active host firewalls, NIC power-saving settings, ARP inspection, or IP Source Guard configurations that I know of. A permanent ARP entry for the gateway does not resolve the problem. Packet captures on B show no incoming ICMP traffic while it is unreachable.
I do not have administrative access to the router or gateway, so I cannot inspect its ARP tables, routing configuration, or logs. Could this be caused by duplicate IP or MAC addresses, proxy ARP, a rogue DHCP server, or some gateway behavior that is triggered by fresh ARP traffic?
3 Answers
The router or gateway is still a major suspect. It may have stale or incorrect ARP information, proxy ARP enabled, an unexpected static route, or a duplicate gateway address. A rogue DHCP server could also be distributing a different gateway. Since B sees no incoming ICMP while A is trying to ping it, the request may never be forwarded to B at all. Ask the network administrator to check the gateway's ARP table, routing table, proxy ARP settings, DHCP sources, and switch MAC-address tables during a failure. If the network uses Cisco equipment, checking redirect entries may also reveal an unexpected next hop.
Work through it hop by hop rather than focusing only on the stale flag. Record each host's IPv4 and IPv6 configuration, MAC addresses, ARP or neighbor tables, routes, and relevant kernel network settings. Capture traffic on host B and, if possible, host C while testing. You want to determine whether B sends replies, whether the gateway sends the original ICMP request, and which MAC address is used at every hop. Static ARP entries can help isolate a failure, but they will not fix a duplicate address or incorrect upstream ARP state.
The strongest possibility is a duplicate IP address, a duplicate MAC, or another device answering ARP on behalf of one of these addresses. Proxy ARP or a second device claiming the gateway could produce exactly this sort of intermittent behavior: upstream equipment learns the wrong MAC, then fresh traffic causes it to relearn the correct path temporarily. Check every active NIC and VM for duplicate MAC and IP addresses, and compare the MAC addresses learned by the switches with the ARP responses seen in packet captures.
The issue affects every host and VM in subnet 2, not just one endpoint. I will still compare all MAC addresses and look for another device answering for the gateway or affected hosts.
That is a useful checklist. Wireshark is already available, and I will document the MAC addresses and compare the ARP exchanges during both the working and broken states.