Hey everyone! I'm working on a little study to figure out what typically leads to those nightmare cloud bills we all dread. I'd love to hear your personal experiences. What was the worst cloud bill you've ever received? What caused it? Was it a mistake on your part, a co-worker's, or something else? How did you deal with it afterward? Did you implement any measures to prevent it from happening again?
2 Answers
Just get in touch with support when these things happen. I used to work in billing support for Azure. People request refunds for everything from minor miscalculations to major crypto mining incidents. Honestly, just be straightforward with them. Once, someone said they couldn't afford milk because of their bill! Also, filling out their surveys can help you out in the long run—those reviews matter!
One time, a co-worker accidentally pushed an AWS root key to a public GitHub repo, and within minutes, we had a bunch of massive GPU instances mining crypto. We shut everything down and managed to get Amazon to refund the bill—but man, that was a stressful hour! In general, I've noticed that Datadog's costs can skyrocket, especially when you're dealing with a ton of logs. My last job had over 30 billion log entries monthly, and the costs were insane!
How did no one catch that in a PR review?
Happened to me too! I forgot some restricted credentials and ended up getting blocked by AWS pretty fast.
How often did you see big refund requests like that?