I'm really curious about how different teams deal with performance regression testing when it comes to pull requests (PRs). At what point does automating checks like latency, throughput, and resource usage become essential in your workflow, rather than just a bonus? What factors lead to this change in approach within your teams?
3 Answers
Only if performance is critical for our users; they can usually tolerate an extra half second. So, it’s not always a priority for us at every stage.
Are you guys actually doing regression testing for performance? That’s impressive!
We don’t do it on PRs because our tests take about 40 minutes during nightly runs. It helps keep the scripts maintained and gives us a quick overview before deeper checks before a release.
How do you manage your nightly tests? I think having that quick check is a great practice.
I just wanted to know how to frame the conversation around it.