Hey everyone! I'm working on a personal project where I've set up an Ubuntu server on an old laptop and built a web app using Python, Flask, Nginx, Gunicorn, and PostgreSQL. I've managed to configure GitLab for version control and set up automated backups to S3, plus everything runs well with Docker. Now, I'm trying to align my development process with industry best practices, and I would love your feedback on my roadmap for going forward.
The main features I plan to implement include fetching data from APIs and doing some automated scraping to populate my database.
Right now, everything runs over HTTP, but I plan to switch to HTTPS soon, as I have a domain registered with Cloudflare.
Next, I want to establish a Dev/Test/Prod pipeline and implement various testing phases including Unit, Integration, and Security Testing using GitLab, pytest, and Allure for results visualization. Should I set up a staging environment for further testing before production?
Also, I'm considering using a .gitlab-ci.yml file to create a development container automatically when new branches are created. Is this a common practice? And should a Test environment always be running, or can it be instantiated only during pushes?
I'm thinking of separating my configurations into different .env files for each environment, and I'm prepared to transition to the cloud with Terraform for resource management later on. My goal includes moving to Kubernetes for deployment and possibly looking into blue/green deployments and load balancing.
Does this roadmap look solid or is there anything crucial I might be overlooking? I appreciate any insights!
1 Answer
Your plan looks great overall! One thing to consider is that the number of environments can vary by project and organization. While many people stick to a dev/stage/prod model, the main goal should be to ensure that each environment serves a clear purpose. This clarity will not only help in justifying costs as you move to the cloud but also make for a compelling story during interviews. The staging phase, for example, often serves as a final testing ground before production, catching any bugs that might slip through earlier tests.
Thanks for the insight! Since I'm currently running everything on my personal server, costs aren't an issue yet, but I'll definitely keep that in mind for the future. What kind of testing processes do typically happen during staging?