I'm looking for a sanity check from other Linux administrators. We have several data-heavy services running on cloud instances and bare-metal systems. During a patching and cleanup window, one job generated a large amount of temporary data and logs, so we expanded a volume to prevent an outage. After cleanup, usage dropped back down, leaving us with a mostly empty volume that costs more than necessary.
Expanding storage is straightforward, but shrinking it is much more involved. XFS cannot be shrunk, while ext4 generally requires taking the filesystem offline and carefully reducing it. In practice, that often means creating a smaller volume, synchronizing the data, stopping services, performing a final sync, switching mounts or UUIDs, and hoping the application behaves afterward.
Monitoring and cost tools can identify the wasted capacity, but the practical Linux response is often to accept the waste rather than risk a stable system. How are you handling this? Do you simply treat live filesystems as effectively one-way and overprovisioned, or are there cleaner ways to reclaim space without a full migration?
4 Answers
For a one-off cleanup, I would not try to force an in-place shrink on XFS. Create a replacement volume, copy the data with rsync, stop the services for a final synchronization, switch the mount, and keep the original volume as a rollback until everything is verified. It is still a migration, but it is predictable and much safer than experimenting on a production filesystem.
The safest approach is usually to separate bursty data before it causes trouble. Put logs, caches, and temporary files on their own filesystem or volume. If one suddenly grows, create a new smaller filesystem, stop the affected services briefly, swap the mount, and deal with the old volume afterward. Centralized logging and sensible rotation can reduce the problem even further.
Offloading logs to centralized storage is especially useful because the local filesystem then only needs to handle a bounded buffer instead of retaining everything indefinitely.
For immutable or fully automated systems, rebuilding the instance or volume from configuration is often cleaner than shrinking anything. That works well when important state is in managed databases, object storage, or separate persistent volumes. It is less convincing for long-lived machines with local state, unusual workloads, or applications that are not completely reproducible, so a careful volume replacement is still a reasonable operational answer.
The practical compromise is to grow conservatively, isolate unpredictable workloads, keep snapshots and backups, and accept that an occasional replacement-volume migration is safer than making filesystem shrinking part of normal operations.
If reclaiming capacity is a recurring requirement, choose the storage layer with that in mind. LVM thin provisioning can return unused blocks to a shared pool when discard or fstrim is configured correctly. Btrfs and ZFS also make dataset resizing easier, although each has its own operational and hardware trade-offs. Thin provisioning does not magically make every filesystem shrinkable, but it can avoid allocating all the physical capacity up front.
LVM alone is not enough if the filesystem inside the logical volume cannot shrink. The filesystem and the storage layer both need to support the operation.
A temporary filesystem or symlink works well for obvious caches and disposable directories, but it does not solve the problem when the already-expanded filesystem contains tightly coupled application data.