I'm currently in my 7th semester of college and brainstorming ideas for my project. I'm considering developing a predictive autoscaling solution that collects metrics from a cluster or target group. These metrics would then be sent to an agent, processed through machine learning, and based on predictions, a new virtual machine would be added to the target group or cluster. I know it sounds simple, but I'm realizing it involves a lot, especially since I need to train an ML model and I'm struggling with calculating the average of all server metrics. I'm looking for advice: should I pursue this project or opt for something simpler? I want to make sure I can complete it on time and handle any unforeseen challenges. I'd also appreciate a rating on my project idea!
1 Answer
It seems like a cool idea, but it feels a bit premature without doing some feasibility research first. You should really dig into how you're going to collect those metrics and whether you even need machine learning for what you're trying to achieve. A simple load trend calculation using time-series logging might already do the trick, and this kind of solution has existed long before ML became a trend.
Data collection is about done! Right now, I'm building the data pipeline. I might need ML since I'm not just looking at CPU usage; I'm also diving into network I/O and real-time response times for websites and apps. For instance, if there's just a basic spike for a couple of minutes, it might not be worth spinning up a new VM, as the traffic could drop within that time. It takes a while to spin up a VM, you know?