I got a desktop for free that has neither an integrated GPU nor a dedicated graphics card. I already have an NVIDIA GT 610 with 2GB of VRAM and a GT 730 with 1GB available. For running the Gemma 3:1B local language model, would either card be useful, or should I run it entirely on the CPU?
3 Answers
The realistic options are CPU inference or upgrading to a newer GPU. A card such as an RTX 2060 12GB or RTX 3060 12GB would be far more suitable because it has modern CUDA support and enough VRAM for local models.
Neither card is a practical choice for this. Their CUDA support is too old for current LLM software, and getting a compatible build of llama.cpp working would be difficult and likely very slow. The CPU will probably perform better than either GPU.
You can test both, but don’t expect much from them. Between the two, the extra VRAM on the 2GB GT 610 is more useful than the GT 730’s 1GB, although neither has enough modern GPU support to make it a good LLM accelerator.
Related Questions
Lenovo Thinkpad Stuck In Update Loop Install FilterDriverU2_Reload