On Sun, 23 Aug 2026 at 15:35, Giles Orr <gilesorr@gmail.com> wrote:
I have a machine running Debian 13 that's used for: - ripping DVDs - ZFS 2x6TB mirror - backing up the contents of my primary machine
But for several months it's been unstable. In the last month I've concluded it's got a memory leak. `htop`, `free`, and `stacer` have led me to this conclusion, but none of these seem to be able to tell me what's causing the leak. I've watched the memory gauge in `stacer` go from "5.0GiB of 15.6GiB" to "10.2GiB" used in about 5 minutes - while I was doing nothing on the machine, and `htop` said no processes were active and nothing in particular was using more memory than usual.
It would be easiest to blame my home-built version of GNU HandBrake, but the surge of memory usage mentioned above occurred before I ever started `ghb`. So pretty sure it's not that. Could it be ZFS memory caching? Is there a way to check (or control) that?
The result is that after about five days the machine is choking because it's trying to swap everything to disk (very slow because spinning disks) and around seven days it dies completely. Prior to this supposed leak, the machine was stable and usable for months at a time, so I'd really love to hunt down and kill this bug ... Any suggestions on how to hunt memory leaks would be much appreciated.
Big thanks to both Randy and Aurelian. I'm really pleased to have learned the use of `pmap` as a side effect of understanding Aurelian's comment. But ... I admit to following the easier route first, namely investigating Randy's suggested setting for ZFS which led me to `arcstat` and `arc_summary` through https://blog.thalheim.io/2025/10/17/zfs-ate-my-ram-understanding-the-arc-cac... ... which pointed out that the ZFS cache is very aggressive and believes it should have ~94% of your memory ... and then doesn't always give it up quickly. With that in mind, I've created /etc/modprobe.d/zfs.conf with max and min restrictions limiting the cache 2GiB<cache<6GiB. `arc_summary` before and after the reboot: ARC status: Total memory size: 15.6 GiB Min target size: 3.1 % 498.5 MiB Max target size: 93.6 % 14.6 GiB Target size (adaptive): 57.9 % 8.5 GiB Current size: 57.9 % 8.4 GiB Free memory size: 1.9 GiB Available memory size: 1.3 GiB ARC status: Total memory size: 15.6 GiB Min target size: 12.8 % 2.0 GiB Max target size: 38.5 % 6.0 GiB Target size (adaptive): 1.6 % 2.0 GiB Current size: 1.6 % 96.3 MiB Free memory size: 14.3 GiB Available memory size: 13.7 GiB Note particularly the "target" values. I'm aware that ZFS loves more cache, and will slow down the smaller the cache is. Given my use of ZFS, a bit of slow-down is _not_ an issue (single (patient) user, ZFS used almost entirely for backups and not really serving content), so I feel like 6G should be plenty. This is probably the fix to my issues but I'll watch closely. Thanks again! -- Giles https://www.gilesorr.com/ gilesorr@gmail.com