On Tue, Jun 30, 2020 at 3:53 AM D. Hugh Redelmeier via talk <talk@gtalug.org> wrote: snip
I was going to consider BTRFS when RHEL dropped it. I took that as a Bad Sign. Do you know why they did this?
I started my use of BTRFS not that long ago on the recommendation of my linux guru who indicated that BTRFS offered positive options as compared to ext4 and was a 'better' option than ZFS. As he died just recently I can no longer ask me for more details re: his recommendation. He styled himself as a 'dinosaur' when it came to linux computing relying very strongly on the most often well thought out and crafted UNIX tenets to the more contemporary everything AND the kitchen sink kind of thinking.
BTRFS progress has seemed painfully slow. To me, that's a sign that it might be too complex. Complexity is my enemy.
- too many places for bugs
- too much code that isn't tested much
- harder to adjust to changing environments
Why does compression help with write amplification on SSDs? My understanding is that SSD firmware compresses already. Possible answer: compression of encrypted stuff is ineffective. But BTRFS doesn't encrypt (LUKS can do so underneath BTRFS, it seems).
The following are some random thoughts about filesystems. I'm interested in any reactions to these.
The UNIX model of a file being a randomly-accessed array of fixed-size blocks doesn't fit very well with compression. Even if a large portion of files are accessed purely as a byte stream. That's perhaps a flaw in UNIX but it is tough to change.
When UNIX was developed the idea of files being as big as tera and peta bytes really wasn't on the radar. As things have evolved files have gotten, in general, quite a bit larger. Mayhap the question here should be - - - how much space gets wasted in files that are smaller than 4k? Or how much space is wasted in the carryover (files filing blocks of space but with a bit more in the next block)?
In modern systems, with all kinds of crazy containerization, I guess de-duplication might be very useful. As well as COW, I think. Is this something for the File System, or a layer below, like LVM?
I was quite eager to try containerization and spent quite a bit of time and effort in trying to get things working. Then I found that those containerization systems would then govern my system use - - by that I mean that I was forced to upgrade often multitudinous files at someone else's behest. Making things more interesting was when one of these updates locked up a whole bunch of things. At that point my interest in containerization disappeared - - - especially when I couldn't even easily remove the infection on my system. I was actually forced to reinstall - - - - absolutely couldn't find another way to remove this cancerous software. The concept of containerization is great - - - - maybe there is an implementation out there that isn't so hamstrung but I'm no longer looking and likely won't look due to the incredible levels of frustration and large amount of wasted time in my effort.
There's something appealing about modularizing the FS code by composable layers. But not if the overhead is observable. Or the composability leaves rough edges.
Here's a natural order for layers: FS (UNIX semantics + ACLS etc, more than just POSIX) de-duplication compression encryption aggregation for efficient use of device?
I don't know where to fit in checksums. Perhaps it's a natural part of encryption (encryption without integrity checking has interesting weaknesses).
I don't know how to deal with the variable-sized blocks that come out of compression. Hardware has co-evolved with file-systems to expect blocks of 512 or 4096 bytes. (I remember IBM/360 disk drives which supported a range of block sizes as if each track was a short piece of magnetic tape.)
I don't know how to have file systems more respectfully reflect the underlying nature of SSDs and shingled HDDs
I'm thinking that most consumers don't give a rip about any issues with either item - - - they think they like the positives and so they buy. It seems that few consider the long term operating costs of items. Most seem to look only at purchase price which often but definitely not always has some correlation with value.
I also am still waiting for translucent mounts like Plan 9.
I think that many or most drives do whole-volume encryption invisible to the OS. This really isn't useful to the OS since the whole volume has a single key.
The most secure encryption is end-to-end. It tends to be less convenient. Maybe my placement of encryption near the bottom of the stack isn't good enough.
If encryption were easier to 'do' perhaps one could even use multiple layers of encryption (encryption of file, directory and disk) to enhance the resilience to intrusion? (Hoping I haven't turned over too many big rocks!) Regards