I'm looking to build a server for hosting a single RTX Pro 6000, primarily for running large language models for inference, fine-tuning, agentic, and RAG workflows. I'll also use it for training YOLO models occasionally. Budget isn't an issue, so I'm hoping to get the best compatible CPU, RAM, cooler (either air or water), and server rack.
1 Answer
If you're going for a high-performance setup, you should consider using an AMD Threadripper PRO 7975WX as your CPU. It offers 32 cores, which is great for handling intensive workloads. For RAM, I'd recommend 256GB of DDR5 ECC RDIMM to ensure stability, especially for LLMs. You might want to check out the Noctua NH-U14S TR5-SP6 for cooling; it's efficient and quiet. As for the rack server, the Dell PowerEdge R760xa or HPE ProLiant DL380 Gen11 are solid choices. Make sure to keep airflow in mind with rack-mounted equipment.
Totally agree, those server models are reliable. I've had great experiences with Dell servers myself!