I have a machine with 64 CPU cores and 256 GB of RAM, and I'm trying to figure out the optimal number of threads I can run in a program for everything to operate smoothly. I'm looking for guidelines or documentation that can help clarify this.
4 Answers
The ability to separate tasks is key here. If you have a million operations but each one depends on the previous one, you're not going to benefit from multiple threads, as you'd barely be able to use even two.
It really depends on your workload! You could have a ton of threads—like 100K—just waiting for data, and your CPU usage might be zero. But if you're running complex math operations, those threads could max out your CPU usage. It's all about what your tasks are doing!
Also, consider your CPU's capabilities—does it have hyperthreading? The specific workload you're running matters too. Keep in mind that operating system threads, CPU threads, and virtual threads are not the same thing.
Generally, your hardware allows for at least 64 threads to run in parallel. If your program involves a lot of I/O waiting, you might actually benefit from having more threads than cores to keep things moving while some threads are stalled. It’s all about maximizing throughput!
Yeah, the classic rule of thumb is one thread per core, but really, it all depends on what those threads are up to.