I have a system with Debian 13 installed. I was installing something on the machine and ran out of space on the root partition (big Docker image). After it aborted the install, here's the output of `df`: # df -h Filesystem Size Used Avail Use% Mounted on ... /dev/mapper/brick_lvm-os2 24G 21G 2.6G 90% / ... Here's the output of `ncdu -x /`: 8.4 GiB [#############] /usr 3.2 GiB [#### ] /var 296.6 MiB [ ] /boot 99.5 MiB [ ] /root 19.4 MiB [ ] /etc 16.0 KiB [ ] /media 16.0 KiB [ ] /opt e 16.0 KiB [ ] /lost+found 12.0 KiB [ ] /mnt ... *Total disk usage: 11.9 GiB Apparent size: 11.6 GiB Items: 344362 You see the problem? `ncdu` thinks 12G is used, but `df` thinks that 21G is used. That's one hell of a discrepancy. A web search yielded the idea of running `lsof | grep deleted` which turned up Firefox as a major offender ... but killing FF got rid of its hundreds of "deleted" items while not changing what `df` thought at all. I remember you can hide disk usage under mount points. But that was an old trick one hundred years ago on multi-user systems. One that I'd be unlikely to play on myself as this isn't a multi-user system. Other things I checked: no, there isn't a giant swap file sitting in the root. No, this problem doesn't exist on my main machine (or, I think, any of my others). The one unusual thing about this system is that ZFS is installed and there are two other mirrored drives. Root is hosted on a separate drive, so I don't think ZFS would be quietly eating huge chunks of this non-ZFS drive? (I am not a ZFS expert.) This is an LVM system and I have spare space that I can throw at the problem - but I'd really really like to understand what's going on! Anybody seen anything like this? Any ideas what would cause this discrepancy or how to track it down? Thanks. -- Giles https://www.gilesorr.com/ gilesorr@gmail.com
I did a web search for *"df" "ncdu" different*. This is top result https://unix.stackexchange.com/questions/576764/why-is-ncdu-reporting-differ... (BTW I am not being snarky by explaining my web search, I just think its more helpful to show how an answer was obtained.) I won't copy paste the whole accepted answer here but it begins:
The output of `df` is based on filesystem-level statistics, whereas `ncdu` (and a regular `du`) generate their results by scanning through directories, reading the sizes of individual files, and summarizing that.
And later is talking about how btrfs could further complicate but I am not sure if it also applies to zfs. Could try repeating the above search with a param for file system. On Wed, May 13, 2026, at 7:49 PM, Giles Orr via Talk wrote:
I have a system with Debian 13 installed. I was installing something on the machine and ran out of space on the root partition (big Docker image). After it aborted the install, here's the output of `df`:
# df -h Filesystem Size Used Avail Use% Mounted on ... /dev/mapper/brick_lvm-os2 24G 21G 2.6G 90% / ...
Here's the output of `ncdu -x /`:
8.4 GiB [#############] /usr 3.2 GiB [#### ] /var 296.6 MiB [ ] /boot 99.5 MiB [ ] /root 19.4 MiB [ ] /etc 16.0 KiB [ ] /media 16.0 KiB [ ] /opt e 16.0 KiB [ ] /lost+found 12.0 KiB [ ] /mnt ... *Total disk usage: 11.9 GiB Apparent size: 11.6 GiB Items: 344362
You see the problem? `ncdu` thinks 12G is used, but `df` thinks that 21G is used. That's one hell of a discrepancy. A web search yielded the idea of running `lsof | grep deleted` which turned up Firefox as a major offender ... but killing FF got rid of its hundreds of "deleted" items while not changing what `df` thought at all.
I remember you can hide disk usage under mount points. But that was an old trick one hundred years ago on multi-user systems. One that I'd be unlikely to play on myself as this isn't a multi-user system.
Other things I checked: no, there isn't a giant swap file sitting in the root. No, this problem doesn't exist on my main machine (or, I think, any of my others). The one unusual thing about this system is that ZFS is installed and there are two other mirrored drives. Root is hosted on a separate drive, so I don't think ZFS would be quietly eating huge chunks of this non-ZFS drive? (I am not a ZFS expert.)
This is an LVM system and I have spare space that I can throw at the problem - but I'd really really like to understand what's going on!
Anybody seen anything like this? Any ideas what would cause this discrepancy or how to track it down? Thanks.
-- Giles https://www.gilesorr.com/ gilesorr@gmail.com ------------------------------------ Description: GTALUG Talk Unsubscribe via Talk-unsubscribe@lists.gtalug.org Start a new thread: talk@lists.gtalug.org This message archived at https://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/MLJ3USG...
Web search is correct. "du -x /" gives sum of files under "/" mount point only ("-x"). Whereas "df -h" gives the whole filesystem. I'm running btrfs, so * df -h / * btrfs filesystem df / * btrfs filesystem usage / all give same answer. ZFS should have comparable utility. On 2026-05-13 21:57, bitmap via Talk wrote:
I did a web search for *"df" "ncdu" different*. This is top result https://unix.stackexchange.com/questions/576764/why-is-ncdu-reporting-differ...
(BTW I am not being snarky by explaining my web search, I just think its more helpful to show how an answer was obtained.) I won't copy paste the whole accepted answer here but it begins:
The output of |df| is based on filesystem-level statistics, whereas |ncdu| (and a regular |du|) generate their results by scanning through directories, reading the sizes of individual files, and summarizing that.
And later is talking about how btrfs could further complicate but I am not sure if it also applies to zfs. Could try repeating the above search with a param for file system.
On Wed, May 13, 2026, at 7:49 PM, Giles Orr via Talk wrote:
I have a system with Debian 13 installed. I was installing something on the machine and ran out of space on the root partition (big Docker image). After it aborted the install, here's the output of `df`:
# df -h Filesystem Size Used Avail Use% Mounted on ... /dev/mapper/brick_lvm-os2 24G 21G 2.6G 90% / ...
Here's the output of `ncdu -x /`:
8.4 GiB [#############] /usr 3.2 GiB [#### ] /var 296.6 MiB [ ] /boot 99.5 MiB [ ] /root 19.4 MiB [ ] /etc 16.0 KiB [ ] /media 16.0 KiB [ ] /opt e 16.0 KiB [ ] /lost+found 12.0 KiB [ ] /mnt ... *Total disk usage: 11.9 GiB Apparent size: 11.6 GiB Items: 344362
You see the problem? `ncdu` thinks 12G is used, but `df` thinks that 21G is used. That's one hell of a discrepancy. A web search yielded the idea of running `lsof | grep deleted` which turned up Firefox as a major offender ... but killing FF got rid of its hundreds of "deleted" items while not changing what `df` thought at all.
I remember you can hide disk usage under mount points. But that was an old trick one hundred years ago on multi-user systems. One that I'd be unlikely to play on myself as this isn't a multi-user system.
Other things I checked: no, there isn't a giant swap file sitting in the root. No, this problem doesn't exist on my main machine (or, I think, any of my others). The one unusual thing about this system is that ZFS is installed and there are two other mirrored drives. Root is hosted on a separate drive, so I don't think ZFS would be quietly eating huge chunks of this non-ZFS drive? (I am not a ZFS expert.)
This is an LVM system and I have spare space that I can throw at the problem - but I'd really really like to understand what's going on!
Anybody seen anything like this? Any ideas what would cause this discrepancy or how to track it down? Thanks.
-- Giles https://www.gilesorr.com/ gilesorr@gmail.com ------------------------------------ Description: GTALUG Talk Unsubscribe via Talk-unsubscribe@lists.gtalug.org Start a new thread: talk@lists.gtalug.org This message archived at https://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/MLJ3USG...
------------------------------------ Description: GTALUG Talk Unsubscribe viaTalk-unsubscribe@lists.gtalug.org Start a new thread:talk@lists.gtalug.org This message archived athttps://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/CQKIQDR...
On Wed, 13 May 2026 at 19:49, Giles Orr <gilesorr@gmail.com> wrote:
I have a system with Debian 13 installed. I was installing something on the machine and ran out of space on the root partition (big Docker image). After it aborted the install, here's the output of `df`:
# df -h Filesystem Size Used Avail Use% Mounted on ... /dev/mapper/brick_lvm-os2 24G 21G 2.6G 90% / ...
Here's the output of `ncdu -x /`:
8.4 GiB [#############] /usr
3.2 GiB [#### ] /var 296.6 MiB [ ] /boot 99.5 MiB [ ] /root 19.4 MiB [ ] /etc 16.0 KiB [ ] /media 16.0 KiB [ ] /opt e 16.0 KiB [ ] /lost+found 12.0 KiB [ ] /mnt ... *Total disk usage: 11.9 GiB Apparent size: 11.6 GiB Items: 344362
You see the problem? `ncdu` thinks 12G is used, but `df` thinks that 21G is used. That's one hell of a discrepancy. A web search yielded the idea of running `lsof | grep deleted` which turned up Firefox as a major offender ... but killing FF got rid of its hundreds of "deleted" items while not changing what `df` thought at all.
I remember you can hide disk usage under mount points. But that was an old trick one hundred years ago on multi-user systems. One that I'd be unlikely to play on myself as this isn't a multi-user system.
Other things I checked: no, there isn't a giant swap file sitting in the root. No, this problem doesn't exist on my main machine (or, I think, any of my others). The one unusual thing about this system is that ZFS is installed and there are two other mirrored drives. Root is hosted on a separate drive, so I don't think ZFS would be quietly eating huge chunks of this non-ZFS drive? (I am not a ZFS expert.)
This is an LVM system and I have spare space that I can throw at the problem - but I'd really really like to understand what's going on!
Anybody seen anything like this? Any ideas what would cause this discrepancy or how to track it down? Thanks.
To clarify: - the system has three spinning disks - two are mirrored 6TB ZFS drives - but the boot drive is a separate 1TB drive managed by LVM and the root partition and all of the other partitions on that drive are ext4, not ZFS - I understand there are differences between how `du` and `df` calculate things, but we're looking at a discrepancy of 9G (about 40% of the partition capacity) so it's a stretch for me to believe that this is the problem ... -- Giles https://www.gilesorr.com/ gilesorr@gmail.com
On Wed, May 13, 2026 at 07:49:25PM -0400, Giles Orr via Talk wrote:
I have a system with Debian 13 installed. I was installing something on the machine and ran out of space on the root partition (big Docker image). After it aborted the install, here's the output of `df`:
# df -h Filesystem Size Used Avail Use% Mounted on ... /dev/mapper/brick_lvm-os2 24G 21G 2.6G 90% / ...
Here's the output of `ncdu -x /`:
8.4 GiB [#############] /usr
3.2 GiB [#### ] /var 296.6 MiB [ ] /boot 99.5 MiB [ ] /root 19.4 MiB [ ] /etc 16.0 KiB [ ] /media 16.0 KiB [ ] /opt e 16.0 KiB [ ] /lost+found 12.0 KiB [ ] /mnt ... *Total disk usage: 11.9 GiB Apparent size: 11.6 GiB Items: 344362
You see the problem? `ncdu` thinks 12G is used, but `df` thinks that 21G is used. That's one hell of a discrepancy. A web search yielded the idea of running `lsof | grep deleted` which turned up Firefox as a major offender ... but killing FF got rid of its hundreds of "deleted" items while not changing what `df` thought at all.
I remember you can hide disk usage under mount points. But that was an old trick one hundred years ago on multi-user systems. One that I'd be unlikely to play on myself as this isn't a multi-user system.
Other things I checked: no, there isn't a giant swap file sitting in the root. No, this problem doesn't exist on my main machine (or, I think, any of my others). The one unusual thing about this system is that ZFS is installed and there are two other mirrored drives. Root is hosted on a separate drive, so I don't think ZFS would be quietly eating huge chunks of this non-ZFS drive? (I am not a ZFS expert.)
This is an LVM system and I have spare space that I can throw at the problem - but I'd really really like to understand what's going on!
Anybody seen anything like this? Any ideas what would cause this discrepancy or how to track it down? Thanks.
If you were to place a 10GB file in /mnt and then mount a different drive on top of /mnt, df would know the 10GB was used, but du would not be able to see it since the file is hidden by the mount on top of the directory. Also in the case of zfs, df accounts for snapshot used space, which du does not. Also zfs does asynch deletion, so until a large file is actually fully deleted, it won't show as free in df. I doubt that is the case here though. My guess is the snapshot part. -- Len Sorensen
I think Giles said that this / partition is an ordinary LVM partition and not zfs. I don't know much about lvs snapshots and might not expect them to affect df output of the mounted filesystem. Other than filesystems mounted over top of directories full of stuff, I see that the df used size is roughly double the du used size. I wonder if the lvm device is perhaps a mirror, and df shows the total space, but files use up double their actual size? I think - could be wrong - zfs df (sometimes?) tells you the total size of all the disks in a dataset, and the reported used size is what is taken up including the space needed for redundancy. Does the output of the lvs command provide any hints? Hope that's helpful John | On Wed, May 13, 2026 at 07:49:25PM -0400, Giles Orr via Talk wrote: | > You see the problem? `ncdu` thinks 12G is used, but `df` thinks that 21G | > is used. That's one hell of a discrepancy. A web search yielded the idea | > of running `lsof | grep deleted` which turned up Firefox as a major | > offender ... but killing FF got rid of its hundreds of "deleted" items | > while not changing what `df` thought at all.
participants (5)
-
bitmap -
Giles Orr -
John Sellens -
Lennart Sorensen -
William Park