Yesterday I had a power failure and when things came back a btrfs volume came back missing the last month few months of data. The most current date I can see on a file is Mar 29. It almost seems like the result of rolling back a snapshot but I having never used the snapshot features of btrfs it has me baffled. I guess its time to move my data to XFS. -- Alvin Starr || land: (647)478-6285 Netvel Inc. || Cell: (416)806-0133 alvin@netvel.net ||
I suffered something like that a year ago: I had a hardware failure on my HDD and all my files were from 6 months back. I could recover my Windows partition without much trouble, but the btrfs was a mess. I got a new HDD, reinstalled everything, and got back to ext4. I have to research a little more about XFS... Mauro http://mauro.limeiratem.com - registered Linux User: 294521 Scripture is both history, and a love letter from God. Em ter., 16 de jun. de 2020 às 09:10, Alvin Starr via talk <talk@gtalug.org> escreveu:
Yesterday I had a power failure and when things came back a btrfs volume came back missing the last month few months of data. The most current date I can see on a file is Mar 29.
It almost seems like the result of rolling back a snapshot but I having never used the snapshot features of btrfs it has me baffled.
I guess its time to move my data to XFS.
-- Alvin Starr || land: (647)478-6285 Netvel Inc. || Cell: (416)806-0133 alvin@netvel.net ||
--- Post to this mailing list talk@gtalug.org Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk
I don't like btrfs, this seems to be the default fs for the boot partition with openSUSE Tumbleweed and I think on 2 occasions it got messed up so bad that I stop using it. Anything is better than btrfs, even fat32 which I ended up using for the boot partition! Been running solid with no issues upgrade after rolling upgrades. On 6/16/20 8:10 AM, Alvin Starr via talk wrote:
Yesterday I had a power failure and when things came back a btrfs volume came back missing the last month few months of data. The most current date I can see on a file is Mar 29.
It almost seems like the result of rolling back a snapshot but I having never used the snapshot features of btrfs it has me baffled.
I guess its time to move my data to XFS.
On Tue, Jun 23, 2020 at 4:28 PM Dev Guy via talk <talk@gtalug.org> wrote:
I don't like btrfs, this seems to be the default fs for the boot partition with openSUSE Tumbleweed and I think on 2 occasions it got messed up so bad that I stop using it.
Anything is better than btrfs, even fat32 which I ended up using for the boot partition! Been running solid with no issues upgrade after rolling upgrades.
People love talking smack about btrfs. Here's some real insight from Josef Bacik though at https://lwn.net/ml/fedora-devel/03fbbb9a-7e74-fc49-c663-32722d6f75a2@toxicpa... and https://lwn.net/ml/fedora-devel/cf5b4944-74c4-b4a4-0d65-71d9821a4a71@toxicpa.... Read through the chain. For those who don't know him, Josef is a core btrfs developer. I also quite like Josef personally, he is a good developer, great to work with and is personally invested in helping folks out. This is a solid fs, there are times when I see the reactions from users and I really wonder why I do open source work anyway. I have lost data on all sorts of filesystems, and I have also been at the end of stupid mistakes made. Almost everything is recoverable if you stop, and ask for help and keep your ego out of it. Dhaval
On 6/29/20 6:13 PM, Dhaval Giani via talk wrote:
On Tue, Jun 23, 2020 at 4:28 PM Dev Guy via talk <talk@gtalug.org <mailto:talk@gtalug.org>> wrote:
I don't like btrfs, this seems to be the default fs for the boot partition with openSUSE Tumbleweed and I think on 2 occasions it got messed up so bad that I stop using it.
Anything is better than btrfs, even fat32 which I ended up using for the boot partition! Been running solid with no issues upgrade after rolling upgrades.
People love talking smack about btrfs. Here's some real insight from Josef Bacik though at https://lwn.net/ml/fedora-devel/03fbbb9a-7e74-fc49-c663-32722d6f75a2@toxicpa... and https://lwn.net/ml/fedora-devel/cf5b4944-74c4-b4a4-0d65-71d9821a4a71@toxicpa.... Read through the chain. For those who don't know him, Josef is a core btrfs developer. I also quite like Josef personally, he is a good developer, great to work with and is personally invested in helping folks out.
This is a solid fs, there are times when I see the reactions from users and I really wonder why I do open source work anyway. I have lost data on all sorts of filesystems, and I have also been at the end of stupid mistakes made. Almost everything is recoverable if you stop, and ask for help and keep your ego out of it.
I have used a number of file systems over the years. from ext in the pre-1.0 kernels through Reiser and XFS. And I have had problems of one form or another in all of them but none managed to blow away all the data that was less than 3 months old after a power failure. fortunately the several hundred files were not critical. Before this my take on the various BTRFS complaints were that people were likely using corner case features. But having my data scrubbed without a hardware failure is just down right confidence destroying. If I cannot replace the functionality of ext without having data corrupted then I am not going to use any of the other wonderful features of BTRFS. Not sure how you would define it but my take is that bulk data loss with no indication of errors is the opposite of solid. just my $0.02 -- Alvin Starr || land: (647)478-6285 Netvel Inc. || Cell: (416)806-0133 alvin@netvel.net ||
Warning: it is the middle of the night and I'm going to ramble. | From: Dhaval Giani via talk <talk@gtalug.org> | People love talking smack about btrfs. Here's some real insight from Josef | Bacik though at | https://lwn.net/ml/fedora-devel/03fbbb9a-7e74-fc49-c663-32722d6f75a2@toxicpa... | and | https://lwn.net/ml/fedora-devel/cf5b4944-74c4-b4a4-0d65-71d9821a4a71@toxicpa.... | Read through the chain. For those who don't know him, Josef is a core btrfs | developer. I also quite like Josef personally, he is a good developer, | great to work with and is personally invested in helping folks out. | | This is a solid fs, Interesting. I've only read a bit so far. Generally, I don't wish to be a guinea pig when it comes to file systems. So I switch very rarely. The EXT series has been good to me. I never switched to Reiser or XFS, mostly due to my conservatism. If I remember correctly, inode numbers in Reiser and perhaps XFS didn't have the properties I expected (i.e. within a file system, there was a stable isomorphism between the numbers and the files). I was going to consider BTRFS when RHEL dropped it. I took that as a Bad Sign. Do you know why they did this? BTRFS progress has seemed painfully slow. To me, that's a sign that it might be too complex. Complexity is my enemy. - too many places for bugs - too much code that isn't tested much - harder to adjust to changing environments Why does compression help with write amplification on SSDs? My understanding is that SSD firmware compresses already. Possible answer: compression of encrypted stuff is ineffective. But BTRFS doesn't encrypt (LUKS can do so underneath BTRFS, it seems). The following are some random thoughts about filesystems. I'm interested in any reactions to these. The UNIX model of a file being a randomly-accessed array of fixed-size blocks doesn't fit very well with compression. Even if a large portion of files are accessed purely as a byte stream. That's perhaps a flaw in UNIX but it is tough to change. In modern systems, with all kinds of crazy containerization, I guess de-duplication might be very useful. As well as COW, I think. Is this something for the File System, or a layer below, like LVM? There's something appealing about modularizing the FS code by composable layers. But not if the overhead is observable. Or the composability leaves rough edges. Here's a natural order for layers: FS (UNIX semantics + ACLS etc, more than just POSIX) de-duplication compression encryption aggregation for efficient use of device? I don't know where to fit in checksums. Perhaps it's a natural part of encryption (encryption without integrity checking has interesting weaknesses). I don't know how to deal with the variable-sized blocks that come out of compression. Hardware has co-evolved with file-systems to expect blocks of 512 or 4096 bytes. (I remember IBM/360 disk drives which supported a range of block sizes as if each track was a short piece of magnetic tape.) I don't know how to have file systems more respectfully reflect the underlying nature of SSDs and shingled HDDs I also am still waiting for translucent mounts like Plan 9. I think that many or most drives do whole-volume encryption invisible to the OS. This really isn't useful to the OS since the whole volume has a single key. The most secure encryption is end-to-end. It tends to be less convenient. Maybe my placement of encryption near the bottom of the stack isn't good enough. I have more questions than answers (or even opinions) and my systems are modest, so I stay with EXT4 directly on old fashioned (GUID) partitions. | there are times when I see the reactions from users and | I really wonder why I do open source work anyway. I'd like to understand this comment better. I work on open source and don't remember feeling anything like that from comments in this thread. Other kinds of comments, perhaps. | I have lost data on all | sorts of filesystems, and I have also been at the end of stupid mistakes | made. Almost everything is recoverable if you stop, and ask for help and | keep your ego out of it. More concretely, stop, make an image copy on the disk, and experiment with that. Ones first actions in a disaster may well compound it. But I find filesystems and editors that lose your data are unforgivable. Sometimes it's just a filesystem that is fragile in the face of hardware errors.
On Tue, Jun 30, 2020 at 3:53 AM D. Hugh Redelmeier via talk <talk@gtalug.org> wrote: snip
I was going to consider BTRFS when RHEL dropped it. I took that as a Bad Sign. Do you know why they did this?
I started my use of BTRFS not that long ago on the recommendation of my linux guru who indicated that BTRFS offered positive options as compared to ext4 and was a 'better' option than ZFS. As he died just recently I can no longer ask me for more details re: his recommendation. He styled himself as a 'dinosaur' when it came to linux computing relying very strongly on the most often well thought out and crafted UNIX tenets to the more contemporary everything AND the kitchen sink kind of thinking.
BTRFS progress has seemed painfully slow. To me, that's a sign that it might be too complex. Complexity is my enemy.
- too many places for bugs
- too much code that isn't tested much
- harder to adjust to changing environments
Why does compression help with write amplification on SSDs? My understanding is that SSD firmware compresses already. Possible answer: compression of encrypted stuff is ineffective. But BTRFS doesn't encrypt (LUKS can do so underneath BTRFS, it seems).
The following are some random thoughts about filesystems. I'm interested in any reactions to these.
The UNIX model of a file being a randomly-accessed array of fixed-size blocks doesn't fit very well with compression. Even if a large portion of files are accessed purely as a byte stream. That's perhaps a flaw in UNIX but it is tough to change.
When UNIX was developed the idea of files being as big as tera and peta bytes really wasn't on the radar. As things have evolved files have gotten, in general, quite a bit larger. Mayhap the question here should be - - - how much space gets wasted in files that are smaller than 4k? Or how much space is wasted in the carryover (files filing blocks of space but with a bit more in the next block)?
In modern systems, with all kinds of crazy containerization, I guess de-duplication might be very useful. As well as COW, I think. Is this something for the File System, or a layer below, like LVM?
I was quite eager to try containerization and spent quite a bit of time and effort in trying to get things working. Then I found that those containerization systems would then govern my system use - - by that I mean that I was forced to upgrade often multitudinous files at someone else's behest. Making things more interesting was when one of these updates locked up a whole bunch of things. At that point my interest in containerization disappeared - - - especially when I couldn't even easily remove the infection on my system. I was actually forced to reinstall - - - - absolutely couldn't find another way to remove this cancerous software. The concept of containerization is great - - - - maybe there is an implementation out there that isn't so hamstrung but I'm no longer looking and likely won't look due to the incredible levels of frustration and large amount of wasted time in my effort.
There's something appealing about modularizing the FS code by composable layers. But not if the overhead is observable. Or the composability leaves rough edges.
Here's a natural order for layers: FS (UNIX semantics + ACLS etc, more than just POSIX) de-duplication compression encryption aggregation for efficient use of device?
I don't know where to fit in checksums. Perhaps it's a natural part of encryption (encryption without integrity checking has interesting weaknesses).
I don't know how to deal with the variable-sized blocks that come out of compression. Hardware has co-evolved with file-systems to expect blocks of 512 or 4096 bytes. (I remember IBM/360 disk drives which supported a range of block sizes as if each track was a short piece of magnetic tape.)
I don't know how to have file systems more respectfully reflect the underlying nature of SSDs and shingled HDDs
I'm thinking that most consumers don't give a rip about any issues with either item - - - they think they like the positives and so they buy. It seems that few consider the long term operating costs of items. Most seem to look only at purchase price which often but definitely not always has some correlation with value.
I also am still waiting for translucent mounts like Plan 9.
I think that many or most drives do whole-volume encryption invisible to the OS. This really isn't useful to the OS since the whole volume has a single key.
The most secure encryption is end-to-end. It tends to be less convenient. Maybe my placement of encryption near the bottom of the stack isn't good enough.
If encryption were easier to 'do' perhaps one could even use multiple layers of encryption (encryption of file, directory and disk) to enhance the resilience to intrusion? (Hoping I haven't turned over too many big rocks!) Regards
On 6/30/20 4:53 AM, D. Hugh Redelmeier via talk wrote:
Warning: it is the middle of the night and I'm going to ramble.
[snip]
The following are some random thoughts about filesystems. I'm interested in any reactions to these.
The UNIX model of a file being a randomly-accessed array of fixed-size blocks doesn't fit very well with compression. Even if a large portion of files are accessed purely as a byte stream. That's perhaps a flaw in UNIX but it is tough to change.
Fixed size disk blocks is not just a UNIX thing. Admittedly I have not seen all the hardware out there but I do not believe that there has been a disk drive that has not been formatted with fixed block sizes outside of some very old or very new stuff. Think of it from a hardware perspective If you have random sized blocks you need to manage fragmentation and then likely some form of free space clean up. And that level of compute power was not available in a disk controller until fairly recently. By which time the standard design was so entrenched that any other layouts were not gain enough traction to be worth designing and trying to sell. For example there are disk drives with Ethernet interfaces and have a native object store. The Segate Kinetic drives never seemed to get beyond the sampling phase.
In modern systems, with all kinds of crazy containerization, I guess de-duplication might be very useful. As well as COW, I think. Is this something for the File System, or a layer below, like LVM? I have a problem with de-duplication. I am not sure how well it actually works in practice. At the file system level its just linking of the 2 identical files until one is changed so you need COW.
At the block level you have to look at the overhead of the hash function and the storage of the hash data. The size of the hash key relates to the likelihood an error duplicate match but the size of the block. The duplicate blocks still need to be compared causing an extra reads. Lets say you use SHA-2 for your hash you have a key of 32 bytes if you use 512 bytes for your block size then your hash table is about a 6% overhead. If you go for larger blocks then you will get less hits because the filesystems want to allocate smaller blocks for small file efficiency. If you use LVM extents then the hit rate drops even more. It may work well where you have a large number of VMs where the disk images tend to start out all the same and where the OS will tend to stay static leaving large parts of the disk untouched for a long time. It may also be possible to develop file systems that are amenable to de-duplication.
There's something appealing about modularizing the FS code by composable layers. But not if the overhead is observable. Or the composability leaves rough edges.
Here's a natural order for layers: FS (UNIX semantics + ACLS etc, more than just POSIX) de-duplication compression encryption aggregation for efficient use of device?
This appears to be what Redhat is pushing with their VDO(Virtual Data Optimizer )
I don't know where to fit in checksums. Perhaps it's a natural part of encryption (encryption without integrity checking has interesting weaknesses).
nothing beats dd if=/dev/zero of=your_secret_file for security ;)
I don't know how to deal with the variable-sized blocks that come out of compression. Hardware has co-evolved with file-systems to expect blocks of 512 or 4096 bytes. (I remember IBM/360 disk drives which supported a range of block sizes as if each track was a short piece of magnetic tape.)
Move from disks to object stores(Key/Value).
I don't know how to have file systems more respectfully reflect the underlying nature of SSDs and shingled HDDs
I also am still waiting for translucent mounts like Plan 9.
How would translucent mounts compare to overlay mounts?
I think that many or most drives do whole-volume encryption invisible to the OS. This really isn't useful to the OS since the whole volume has a single key.
The most secure encryption is end-to-end. It tends to be less convenient. Maybe my placement of encryption near the bottom of the stack isn't good enough. I would argue that encryption should be as high in the stack as possible. encrypting the disk provides "at rest" security so when the drives are sold to someone at the bankrupcy sale they cannot use the data. It does not help the hacker who has gained access to the system from dumping the database of credit card info.
[snip] -- Alvin Starr || land: (647)478-6285 Netvel Inc. || Cell: (416)806-0133 alvin@netvel.net ||
Red Hat stopped supporting BTRFS. It's not in RHEL 8. Why? The best "guesses" that I've seen are in <https://access.redhat.com/discussions/3138231> To paraphrase the upstream opinion, btrfs has been "almost production ready" for many years now, but never quite got to the stage where it actually was ready. In the meantime, many of the features that btrfs provides are now available via other more mature and stable storage technologies like ext4, XFS, LVM, etc. We've put considerable effort into improving these technologies to the point where current Red Hat offerings already cover almost the entire btrfs feature set. Red Hat acquired Stratis and their (open source) sofware handles most of what BTRFS does. Using layers (eg. LVM). On the other hand, it looks like the Fedora Poject is considering making BTRFS the default filesystem in Fedora 33. <https://fedoraproject.org/wiki/Changes/BtrfsByDefault> I added a question to that page related to RHEL's dropping of BTRFS but it was deleted because the Fedora change proponent was unable to speak for Red Hat.
participants (6)
-
Alvin Starr -
D. Hugh Redelmeier -
Dev Guy -
Dhaval Giani -
Mauro Souza -
o1bigtenor