I have a system with Debian 13 installed. I was installing something on the machine and ran out of space on the root partition (big Docker image). After it aborted the install, here's the output of `df`:
# df -h
Filesystem Size Used Avail Use% Mounted on
...
/dev/mapper/brick_lvm-os2 24G 21G 2.6G 90% /
...
Here's the output of `ncdu -x /`:
8.4 GiB [#############] /usr
3.2 GiB [#### ] /var
296.6 MiB [ ] /boot
99.5 MiB [ ] /root
19.4 MiB [ ] /etc
16.0 KiB [ ] /media
16.0 KiB [ ] /opt
e 16.0 KiB [ ] /lost+found
12.0 KiB [ ] /mnt
...
*Total disk usage: 11.9 GiB Apparent size: 11.6 GiB Items: 344362
You see the problem? `ncdu` thinks 12G is used, but `df` thinks that 21G is used. That's one hell of a discrepancy. A web search yielded the idea of running `lsof | grep deleted` which turned up Firefox as a major offender ... but killing FF got rid of its hundreds of "deleted" items while not changing what `df` thought at all.
I remember you can hide disk usage under mount points. But that was an old trick one hundred years ago on multi-user systems. One that I'd be unlikely to play on myself as this isn't a multi-user system.
Other things I checked: no, there isn't a giant swap file sitting in the root. No, this problem doesn't exist on my main machine (or, I think, any of my others). The one unusual thing about this system is that ZFS is installed and there are two other mirrored drives. Root is hosted on a separate drive, so I don't think ZFS would be quietly eating huge chunks of this non-ZFS drive? (I am not a ZFS expert.)
This is an LVM system and I have spare space that I can throw at the problem - but I'd really really like to understand what's going on!
Anybody seen anything like this? Any ideas what would cause this discrepancy or how to track it down? Thanks.