When I power off the node announcing a VIP, another node starts announcing it within a few seconds, but then withdraws the announcement with reason notOwner. It starts announcing again about five minutes later. Meanwhile, there are repeated memberlist connection failures to the powered-off node.
What could explain the withdrawal and long delay? I expected failover to be closer to 10 seconds.
The timing is worth separating out. The surviving node appears to announce the service quickly, then stops because it no longer considers itself the owner. The later announcement may be a separate change in its view of the cluster, not simply a five-minute connection timeout.
Check the speaker logs on all surviving nodes around the withdrawal, along with node readiness and service endpoint changes. Compare which node each speaker thinks should announce the VIP. A disagreement about membership or eligibility could explain the notOwner message, though the log excerpt alone does not establish the cause.
Memberlist failure detection is one possible place to look. Suspicion and retransmission behavior can affect how quickly a failed member is treated as gone. But repeated attempts to reconnect do not by themselves show that those attempts are blocking failover.
Check the MetalLB version and its documentation or configuration for the memberlist settings actually exposed by that release. Don't assume the underlying gossip library's knobs are all available as MetalLB configuration options.
Also check the client side before measuring the outage solely from speaker logs. L2 failover depends on clients updating their ARP or neighbor cache after the VIP moves. Compare the VIP's observed MAC address and connectivity from a client with the announcement timestamps.
That won't explain the notOwner withdrawal on its own, but it can distinguish a delayed announcement from a client that kept using an old neighbor entry.
Answers are reviewed before publication.
Kexby-28:
Is there documentation for changing those memberlist settings? I haven't found an obvious configuration option.
6b+inkpad~31:
That depends on the release, and the excerpt doesn't establish that those settings can be changed. Check the version-specific configuration first rather than assuming a suspicion multiplier is exposed.