I'm running Node.js workloads on ECS (using Fargate, no spot instances), but I'm not seeing any logs for 5xx errors. I typically track errors through the API Gateway and ALB, where I can spot 5xx errors, but when checking the logs from ECS, I don't find anything that correlates. I'm looking for suggestions on how to address this visibility issue and ensure that I capture those 5xx errors properly throughout my stack, which includes API Gateway, ALB, ECS, and RDS.
4 Answers
You should verify if you’re currently capturing any logs at all. If not, it’s likely that your IAM role isn’t configured correctly for logging to CloudWatch. That could be why those error logs are missing.
First off, double-check your logging configuration in ECS to make sure your application is set up to log 5xx errors. Additionally, ensure that your CloudWatch logs are configured properly to capture all relevant logs. Often, the issue lies in the logging setup itself.
It's important to differentiate between ALB metrics and target metrics. Check if the 5xx errors are occurring at the ALB itself or are target errors. You won't find an app log if the problem arises at the ALB level. AWS recently launched ALB health check logs, which could help pinpoint issues like your Node process running out of memory.
I totally understand the frustration here! You're already capturing a lot of logs, but if there's no visibility on the 5xx errors, it might be worth checking if those errors are actually occurring in your application. Try logging from the ALB or API Gateway as well, as those logs can provide additional insight.
That does sound like a good approach, but keep in mind that ALB logs can be a bit minimal. They won’t provide the complete picture.