I'm dealing with a significant performance issue on our build server, which is running Windows Server 2019. During our nightly builds and even during high load times throughout the day, we're experiencing tasks that take about three times longer than they should. Specifically, actions like git cloning, NuGet restores, and the overall build process are really dragging.
We have a well-specified environment with plenty of CPU cores and RAM. Our setup includes three parallel Azure DevOps 2020 self-hosted agents, and our workflow focuses primarily on .NET products. We've observed that the server clones GitHub repos and restores packages from an internal NuGet server.
From my findings, it seems like Windows Defender might be the source of the slowdown. I've run various PowerShell queries that suggest antivirus activities are contributing significantly to the performance drop. I attempted to add exclusions for the relevant paths, including build folders and agent directories, but Windows Defender still appears to be scanning heavily during builds. I'm running out of options to troubleshoot this further since backups are not running during peak times. I'm looking for any methodologies or tips on additional checks I can perform to identify the problem.
2 Answers
It sounds like you might be dealing with disk I/O issues during builds. When your build process modifies too many files or a big file repeatedly, it can trigger Windows Defender and slow things down significantly. I'd recommend checking disk I/O during your builds to confirm if that's the case. Also, using Resource Monitor (resmon) to see what's consuming resources could give you more insights.
Do you have any monitoring tools set up? I suggest installing Telegraf, InfluxDB, and Grafana to get detailed metrics about your server. There are plenty of guides available for setting this up, and it can provide you with valuable insights into where the bottlenecks might be.
Related Questions
Can't Load PhpMyadmin On After Server Update
Redirect www to non-www in Apache Conf
How To Check If Your SSL Cert Is SHA 1
Windows TrackPad Gestures