I'm having trouble setting up AWS Database Migration Service (DMS) to connect to an MSK (Managed Streaming for Kafka) cluster across two AWS accounts using PrivateLink. The setup involves Account A having the DMS replication instance and Account B hosting the MSK cluster. The region is us-west-2, and I'm using SASL/SCRAM for authentication. The MSK cluster is private with no public access, and I've verified that PrivateLink is enabled. However, when I test the DMS Kafka endpoint, I encounter the error: 'Application-Status: 1020912 Failed to connect to database.' This setup works fine in the development environment but fails in production. I've ensured that the DMS security group allows TCP access on ports 14001-14100. Any advice on how to troubleshoot this?
1 Answer
Have you considered trying a plain authentication method instead of SASL/SCRAM? Sometimes simplifying the auth can help identify issues. Also, check if there are any ACLs that might be affecting the connection specifically in your production environment. MSK can be tricky, and misconfigurations with auth are often the culprit. If you can access the cluster without any authentication but still face issues with DMS, then the config on the DMS side might need a closer look.
You hit the nail on the head! I discovered that the missing ACLs were the root of the problem. I spent too long trying to troubleshoot without checking that first.