I'm currently using a system with 8GB of soldered RAM and running Ubuntu 22.04. There's a service I need that requires approximately 5GB of free memory when it's active. The issue I'm facing is that the RAM cache builds up over time, making this service unusable. I've set up a cronjob to run "drop_caches" every two hours to free up some memory, but I'm encountering a few problems: the RAM cache can sometimes fill up even before the two-hour mark, and if "drop_caches" runs while the service is in use, it could disrupt it. I've heard that frequently using "drop_caches" can be tough on the hardware, so I'm looking for a better solution to manage the memory effectively without causing more strain. The specific service I'm working with is ollama with llama3.2:3b.
3 Answers
Have you tried using zswap or zram? They can help manage memory better and reduce the reliance on "drop_caches."
What exactly do you mean by 'renders the service unusable'? Linux is generally good at handling RAM, and if you need more space, you could consider using swap space. Maybe "drop_caches" isn't the answer you're looking for?
The hardware has enough RAM for the service to run well at first, but as cache builds, it becomes an issue. I've checked htop, and the Python backend, presumably from an Open WebUI container, is consuming a lot of RAM. After a few hours, the memory doesn't free up on its own, hence my cronjob to prevent CUDA malloc errors.
Typically, Linux manages RAM pretty well on its own. The unused RAM is used for cache, and it should automatically free up when a process needs it. Is it not working that way for you?
It doesn't seem to be happening; I keep getting malloc errors. I think my OEM Jetson Orin Nano might have something to do with it since it's an ARM setup.
Yeah, but wouldn't relying on swap memory wear out the drive just like constantly clearing the cache would?