Hey everyone! I'm managing a large internal CI/CD platform for a company with over 1,000 developers and 100+ DevOps engineers, using a variety of tech stacks (Java, Node, Python, Go, .NET, etc.). My aim is to make a standardized, secure, and efficient path to production. I'm currently focusing on a tag-based promotion strategy for releases and I'd love to hear your thoughts on it.
Our current workflow starts from the `main` branch, which is always production-ready, and when merged, it triggers a build that gets deployed to staging. A production release can only occur by creating a Git tag from a commit to the main branch. My concern is balancing the need to reduce developer friction while some teams are resistant to this change because they feel their custom pipelines already work well.
What are your views on using explicit Git tags for production compared to automatic deployment on merges? Also, how do you handle teams with varying levels of testing maturity? If you have experience in coaxing teams toward a more standardized model, what worked for you? Any insights would be greatly appreciated!
4 Answers
It's tough to say why some teams resist change, but I can relate. When I’ve had a setup that works, I see no reason to switch to something new unless there's a pressing issue. Teams often prioritize maintaining their current workflows over transitioning to what seems like a better option, even if it eventually makes sense to migrate.
In my experience at a large bank, I reverted to a custom pipeline because the shared one was too slow. Steps that could go parallel didn't, and different teams have unique needs that a centralized pipeline might not meet. It helps to have a pipeline champion from each team to bridge these gaps!
That’s a valid point. But I’ve programmed mine to allow for parallel steps, while still supporting customization for odd cases. The pushback seems more focused around the tagging process than anything else.
I really like this approach! We use a GitLab pipeline that includes a manual step for tagging. We’ve also got some tooling that links tickets to the tag and automatically generates release notes. We handle versioning pretty well with semver; if a new pipeline on main isn't published, it just updates to `rc2`. This keeps everything organized and clear!
Have you thought about using commit message titles to determine the release type? Like using `feat:` for minor versions and `fix:` for patches? It adds a bit of structure!
You should consider how often you push updates to the staging branch. If QA is testing something and a new update randomly comes in, it could really throw them off. I’m leaning towards tagging a version first, pushing that to staging, and then releasing after QA gives the green light. What are your thoughts on that?
Ideally, staging should be stable and ready for production once QA signs off. It’s about managing expectations — if something is ready, it goes to staging. Tagging main just to push to staging seems unnecessary if everything is being tested in your pipeline.
I totally get that. But I’d argue that every hour spent fixing a custom pipeline is an hour missed for developing features. A centralized platform can save maintenance time and make upgrading easier across the board!