I'm running into a connection problem trying to set up AWS Database Migration Service (DMS) to connect to an Amazon Managed Streaming for Apache Kafka (MSK) cluster across different AWS accounts. The setup involves:
- An AWS DMS replication instance in Account A (located in `us-west-2a`) and a private MSK cluster in Account B (with brokers accessible only via PrivateLink).
- The connection uses SASL/SCRAM authentication and goes through ports 14001 to 14100.
When I test the DMS Kafka endpoint, I receive the error:
- Application-Status: 1020912
- Application-Message: Failed to connect to database.
This issue only occurs in the production environment, as the same architecture has worked perfectly in development. I've verified that PrivateLink is enabled on MSK and have added a security group rule on the DMS side for the required TCP ports. Any advice on resolving this would be greatly appreciated!
1 Answer
It might help to try connecting without authentication, or using sasl/plain instead. Also, check with your Site Reliability Engineers about any specific ACLs in the production environment. Sometimes, MSK can be tricky, and misconfigurations with authentication can be a major headache. If you can connect to the cluster without auth but DMS still fails, you might need to double-check how DMS is set up.
You were right! I found out that ACLs were missing. I can't believe I spent almost 20 days on that issue.