I'm new to implementing CIS benchmarks and looking for ways to apply them uniformly across our servers (2019, 2022, 2025). I've been running CIS CAT scans on individual servers, but I often encounter failures that require me to fork the scans, which is frustrating. I recently tested OSConfig on an Azure Arc onboarded 2025 server, but I'm disappointed with the lack of central reporting. Why should I have to access Windows Admin Center to check each server individually? Additionally, I've explored some security benchmark options in the Defender portal but haven't delved too deep yet. I've thought about the Sentinel workbook for NIST 800, but it seems outdated since a lot of data isn't filling in due to moving to AMA. I'm looking for a centralized dashboard in Azure that can display NIST compliance for all our servers. By the way, I've also struggled with getting the OSConfig score from Windows Admin Center into a dashboard or workbook in Azure.
3 Answers
Have you checked out the CIS Build Kits? They're basically GPOs and bash scripts that help you quickly implement CIS benchmarks. You can get sample kits for Windows Server, and they come with CIS-CAT reports to show compliance. It might help you gather NIST compliance evidence! Worth a look, even if it's not a perfect fix.
Don't forget about the Security Baseline Assessment in Defender. It might not cover the latest CIS 4.0, though. Still worth checking out!
Yeah, I've been testing that out too. I added a tag to a server and created a policy specific to its OS, but it said no servers were assigned. The documentation mentioned it might take time to sort itself out. It's a bit of a hassle.
I remember when OSConfig was pitched to us, we pointed out the centralized reporting issue and they mentioned some Splunk dashboards related to the ingested OSConfig data. We thought it was cool but ultimately decided to pass on it.
You'd think integration with Sentinel would be a no-brainer. I tried getting it to work with on-prem servers using ChatGPT's assistance, but no luck. Maybe it’s better for Azure VMs, but on-prem is a no-go.
I've used those kits, but they can be tricky if the GUI fails. It ends up forcing me to fork benchmarks, which I really dislike. Plus, I tried importing one in my lab and it blocked me from running another benchmark due to SMB version conflicts. It's tough when the procedures feel rushed!