I'm struggling to free up space in the `/var/lib/docker/overlay2` directory because I can't seem to remove orphaned image layers. I've been running a cron job daily with `sudo docker image prune -a -f; sudo docker system prune -a -f`, but it only clears up space that is detected by the `docker system df` command, which shows I should have more than the 2.728GB it claims. My goal is to delete abandoned image layers without breaking anything, as whenever I delete directories that aren't in use and pull new images, I encounter initialization errors. I'm looking for a dependable method to clean this up without having to fix my current script that inspects every image.
6 Answers
Could you share the output of `docker ps -a`? It might help to see if any active containers are writing to their own filesystems. Remember, overlay2 works like a Git repo; deleted files might still take up space due to how layers are structured.
You can also use `ncdu -x /var/lib/docker/` to see which specific directories are using a lot of space. Quite often, logs or temp files could be the culprit! If you need to write files, try utilizing Docker volumes or mounts instead.
Check the logs in `/var/lib/docker/containers/*/*.log`. You might find large log files there that can be truncated with `truncate -s 0 /var/lib/docker/containers/[container-id]/[container-id]-json.log`. This isn't exactly your main issue, but it might help you reclaim some space.
Didn’t the old implementation of overlay2 leak space, and it had to be manually cleaned up? You might want to ensure that your stopped containers are completely removed. Checking with `docker diff` on each one can help identify filesystem edits that are lingering around.
I’ve verified everything with `docker volume ls -qf dangling=true` and `docker system df`, and it still shows similar results.
You might want to check what’s taking up space by running `docker system df -v`. This command gives a detailed breakdown, and you can find out which overlay2 directories are currently in use with `docker ps -q | xargs -r docker inspect --format '{{ .Name }} -> {{ .GraphDriver.Data.MergedDir }}' | grep overlay2`. From there, you can decide to delete the unused directories or restart the Docker service if needed.
The `df -v` command shows only one container and it’s just 333.9MB, while other values are zero.
Have you tried `docker system prune --volumes`? The layers you’re seeing might be anonymous volumes, and you can check them with `docker volume ls` to confirm.
I’m still seeing the overlay directory consuming the majority of the space based on my previous checks.
I see that the space is mostly taken up by the overlay directory based on my checks.