How are you handling repeated Azure management sign-in attempts?

Start with the sign-in logs before changing policies. Check which users are affected, whether the attempts are failing or succeeding, and the resource and client shown for each event. Attempts may appear as a rich client accessing Azure Resource Manager, with addresses spread across networks, including IPv6. That can help distinguish a password spray from a problem with a particular user or client.

Keep monitoring for successful sign-ins and investigate any account that may have been accessed. A changing list of source addresses is a poor basis for a simple blocklist.

If most users don't need to manage Azure, restrict that access rather than trying to disable something called Azure CLI without checking what it affects. A Conditional Access policy can target Azure Resource Manager and the users who need it. You can allow only the appropriate admin group, and consider restricting access to known locations where that fits your setup.

Test the policy carefully, including IPv6 behavior, and keep emergency access accounts in mind. Conditional Access can deny access to the resource, but it may not stop the password attempts themselves, so the failed events or lockouts could continue. Seeing attempts in the logs does not by itself show that the policy failed.

Also review your smart lockout settings and the authentication methods available to the affected accounts. Stronger sign-in methods can reduce the risk of a stolen password being enough to get in, but they won't necessarily prevent someone from repeatedly submitting bad passwords and triggering lockouts.

I wouldn't loosen lockout protections just to make the alerts go away. If the attempts keep causing disruption after access is restricted, collect the relevant sign-in details and raise it with your identity support channel.