I've noticed a frustrating trend that many DevOps teams might recognize: while our infrastructure is often scrutinized during reviews, most unexpected increases in cloud costs actually stem from the application code. Let me share some examples of issues that frequently slip through the cracks: SDK calls in loops (leading to N+1 problems), recreating clients excessively, polling every few seconds instead of leveraging events, background jobs without termination limits, and changes in Lambda or Glue that increase runtime or data scanned without us realizing it. These code-level issues look fine during discussions but quietly escalate costs every month when deployed at scale.
To tackle this, we've started implementing cost-aware checks directly in our pull requests, which include scanning both Infrastructure as Code (IaC) and application code, estimating runtime amplification (like monthly calls, data scanned, and execution time), and highlighting the financial impact right in the PR comments to prompt immediate changes. Surprisingly, we've found that code-related cost problems occur 3 to 4 times more than infrastructure-related ones and that engineers tend to fix these issues when they receive immediate, contextual feedback. I'm looking to hear from others:
- Have you encountered cost regressions predominantly caused by code instead of infrastructure?
- Do you actively review costs in your PRs, or is that only considered after the bill arrives?
- What specific patterns have led to the most significant cost increases for you?
1 Answer
I've definitely seen code cause cost issues! For instance, I worked on a service where records were being written to storage as individual files. This approach generated huge write operation costs because each file was just a few hundred bytes. We modified the code to batch records together before saving them, and that cut the writing costs down by about 90%! Sometimes the code is the weakest link when it comes to cloud costs, especially if infrastructure is already optimized.
Exactly! While infrastructure choices sure can drive costs, I’ve found that code can create significant variance in overall spending without changing any infrastructure setup. A simple retry loop could turn an otherwise stable service into a costly mess.