How can I check if disabling direct send is effective?

You can also use SMTP testing tools like telnet or putty. They can show you the results of your connection attempts and whether the messages are being blocked as expected.

You can use PowerShell for testing too. Here’s a quick script:
```powershell
$from = "[email protected]"
$to = "[email protected]"
$subject = "Test Email via Direct Send"
$body = "This is a test message sent using Direct Send to Exchange Online."
$smtpServer = "your-tenancy.mail.protection.outlook.com"
$smtpPort = 25
Send-MailMessage -From $from -To $to -Subject $subject -Body $body -SmtpServer $smtpServer -Port $smtpPort
```
If that one fails, you’ve successfully confirmed direct send is off. Just make sure any necessary modules are installed.

Here’s a simple way to check: Before you disabled direct send, send an email to yourself using that method and make sure you got it. Then, after disabling it, try sending that same email again. If you don’t receive it this time, it means direct send is indeed turned off.

Just to clarify on PowerShell: which modules do I need to install for that script? I thought Send-MailMessage was a native cmdlet, right?

Just try sending an email using direct send and see what happens. If it fails to go through, you're good! But if it still sends, then you might need to dig deeper into the settings.