We need to coordinate data pipelines, infrastructure provisioning, and ML retraining in a production environment with no internet access. Cron, scripts, and a shared Jenkins instance are getting hard to maintain.
What tools are worth considering if we want to avoid cloud-dependent licensing and package downloads, preferably without needing a lot of Kubernetes expertise?
Ansible is a reasonable starting point, especially for provisioning and running existing scripts across machines. Ansible Tower is also worth looking at if you need a management interface rather than just playbooks.
Check whether your organization already has an approved product or deployment process before investing much time. Also check the target operating systems and how you'll supply collections, Python dependencies, and updates inside the network.
A self-hosted GitLab setup is worth evaluating if you want the repository and job orchestration close together. It could give the team a way to define and run pipelines without leaning on the shared Jenkins instance.
Test the exact edition and features you need offline, including licensing, runners, dependencies, and updates. Self-hosted doesn't automatically mean every feature is independent of external services.
Take a look at Rundeck for scheduled and manually triggered operational jobs. It may be a simpler fit if most of the work is calling scripts and tools you already have, rather than building a whole new pipeline platform.
Try a small workflow with dependencies and failure handling first, and check how it would run your longer ML jobs.
Kestra is another candidate. A Docker Compose deployment may be appealing if Kubernetes is more machinery than you want.
I'd verify the offline story with the specific version and plugins you plan to use. Bring the images and dependencies into an isolated test environment, then check scheduling, plugin execution, licensing, and upgrades without external access.
The orchestrator is only part of the problem. You'll need an approved way to bring in and maintain packages, container images, and other dependencies. An internal repository and hardened images can keep jobs from trying to fetch things directly from the internet.
That applies whether you go with Ansible, a self-hosted Git repository and job runner, or another tool. Include updates and security review in the plan, not just the initial installation.
Answers are reviewed before publication.
Related Discussions