I'm trying to set up an AWS Lambda function within a private subnet that should process messages from an SQS queue and then write those messages to a PostgreSQL database hosted on RDS Aurora in the same VPC. My infrastructure is built using CDK, where I've defined a VPC with public and private subnets. The private security group currently allows traffic from the same security group and HTTP traffic from the VPC's CIDR block.
The Lambda function seems appropriately set up in the private subnet, but I'm having trouble getting SQS messages to it. I've seen conflicting information online about what needs to be in place for the Lambda to effectively consume the messages from SQS. Here's the basic setup for my SQS queue and Lambda:
```javascript
const sourceQueue = new sqs.Queue(this, "sourceQueue");
```
```javascript
const myLambda = new NodejsFunction(
this,
"myLambda",
{
entry: "path/to/index.js",
handler: "handler",
runtime: lambda.Runtime.NODEJS_22_X,
vpc,
securityGroups: [privateSG],
},
);
myLambda.addEventSource(
new SqsEventSource(sourceQueue),
);
```
Also, I've read that to allow the Lambda to receive messages from SQS, I might need to create a VPC endpoint:
```javascript
const vpcEndpoint = new ec2.InterfaceVpcEndpoint(this, "VpcEndpoint", {
service: ec2.InterfaceVpcEndpointAwsService.SQS,
vpc,
securityGroups: [privateSG],
});
```
However, I'm not able to create VPC endpoints directly due to organizational policies, and I'm concerned about security implications of allowing access. My main goal is to allow the Lambda to receive SQS messages securely and communicate with the database without exposing services unnecessarily. Is there another way to achieve this, or is my understanding of using VPC endpoints correct?
4 Answers
Honestly, the main reason to have the Lambda in a private subnet is to access resources like your database securely. For dealing with SQS, stick to controlling the IAM permissions carefully so only the right services can interact with your queues. You shouldn't really need the VPC endpoint setup unless there's a specific network reason you're handling this.
It sounds like you're getting mixed signals! You don't actually need a VPC endpoint for your Lambda to receive messages from SQS if it is triggered directly. The Lambda service manages polling for you, provided that the IAM role attached to the Lambda has the right permissions for SQS. Just make sure your IAM policies for both the queue and any encryption keys are set to restrict creation and access as needed.
If you're using SQS, just make sure your IAM role for the Lambda has the right permissions to receive messages. That's usually enough. You might only need a VPC endpoint if your Lambda needs to call the SQS service directly as part of its function logic, which typically isn’t the case for standard SQS event sources.
You're on the right track with your setup! AWS services like SQS are generally public, but when your Lambda is in a private subnet, it loses direct access to them. VPC endpoints allow your private resources to access public AWS services without exposing them to the internet, which is statically more secure. If your organization restricts VPC endpoint creation, consider using a NAT Gateway instead for internet access, though it can add cost and complexity.
Also, you should ensure that your SQS is secured with proper IAM policies to prevent unauthorized message sending. As for your concern about a coworker potentially sending messages, restricting IAM permissions to limit who can push messages to the queue should help.
Totally agree! You really just need to lock down those IAM permissions for the queue when it comes to who can send messages.
Exactly! The Lambda polls SQS for messages, so it handles the interaction for you. Just focus on correct permissions.