I'm encountering challenges with two VM Scale Sets in our Azure environment where the Custom Script Extension is failing due to issues with storage account access keys. We use these keys to download scripts from a Storage Account when the VMSS instances start, but we're facing authorization failures during our deployment pipeline because of key refresh and rotation problems. We're considering alternatives to improve the situation.
One approach is switching from access keys to Managed Identity with RBAC to access the Storage Account, but it seems the Custom Script Extension doesn't work reliably with Managed Identity. We're also looking into other methods for running startup commands on VMSS instances without depending exclusively on Custom Script Extension.
I have a few questions:
1. How can I configure Custom Script Extension with Managed Identity for blob access effectively?
2. Has anyone had success setting this up in a stable manner?
3. Are there better alternatives to execute startup scripts or commands on VMSS instances (like cloud init, pre-baked images, run-command, etc.)?
Any insights or experience regarding this would be greatly appreciated!
3 Answers
We've had success using Packer for our images instead, which eliminates the need for a Custom Script Extension to run any scripts. If the Managed Identity is configured correctly, there shouldn't be issues with reliability when downloading. Instead of executing scripts at startup, consider baking your scripts into the images themselves. It can streamline your deployments significantly!
Managed identities are designed specifically for scenarios like this, so it's definitely the way to go. Using storage keys just complicates things and poses risks. If you must use an alternate method, consider SAS tokens, but they should really be a last resort!
Absolutely! Managed identities simplify access management. +1 for avoiding SAS tokens unless really necessary.
I'm also curious about the managed DevOps pools as a migration path from VM Scale Sets. They support Packer-based images, which could simplify things. Anyone tried transitioning to this? What was your experience like?
We transitioned our VMSS agent images to be based off the runner-images repository from Microsoft and added some custom tools. Hosted pools work great but just waiting for MS to resolve capacity issues in our region before we move to production.
Thanks for the tip! I'll definitely check out Packer for image creation.