I'm working on building a coding platform similar to Leetcode, where users can submit their code for testing. My goal is to ensure that the solution is scalable, so I'm considering using k3s to set up a cluster that distributes the workload across different pods. However, I'm a bit confused about whether to focus on thread-level or pod-level parallelism. Should I scale up the number of pods when demand is high, or do I need to consider scaling additional nodes? Also, is it a good idea for pods to create threads to run the submitted code? If so, what would be the optimal number of threads for each pod to manage? I know that threads generally have less context-switching overhead, but scaling pods might be slower in that regard. What are the best practices for scaling code execution in this kind of environment?
3 Answers
For a platform like yours, I'd advise leaning towards pod-level scaling due to the need for security and isolation when executing potentially harmful user code. Setting up Horizontal Pod Autoscalers (HPA) allows you to adjust pod numbers based on CPU or memory usage dynamically. If you're frequently maxing out resources, consider scaling your nodes. As for threading, while it can be efficient, it can complicate resource sharing when working with untrusted submissions. You might want to limit pods to 2-4 threads and monitor performance accordingly.
Security takes precedence over scaling when dealing with untrusted user submissions. Running user code directly in threads tied to your application could lead to serious vulnerabilities. It's better to use isolated environments for user execution. Using tools like ioi/isolate, which containerizes code execution, helps maintain security while allowing you to manage resources effectively.
When starting with k3s, it's often best to get your code execution platform running first before implementing complex scaling strategies. You'll want to determine how much CPU and memory your application needs, and then scale based on those needs. Normally, you'll start with a certain number of pods and monitor the cluster's performance as users submit code. Don't get too bogged down in threading just yet; focusing on pod scaling initially is a good approach. If your application is resource-intensive, consider starting simple and observe how it manages under load.
Related Questions
How To: Running Codex CLI on Windows with Azure OpenAI
Set Wordpress Featured Image Using Javascript
How To Fix PHP Random Being The Same
Why no WebP Support with Wordpress
Replace Wordpress Cron With Linux Cron
Customize Yoast Canonical URL Programmatically