
Btrfs Coming to Fedora 33 - caution
https://fedoramagazine.org/btrfs-coming-to-fedora-33/
======
bkor
I've quickly skimmed the discussion on fedora-devel regarding btrfs. I
wondered mainly how they'd handle the various cases where btrfs does not work
well, e.g. files that change often inline (databases, VMs, etc). Apparently an
application can tell to treat those files differently. So it's basically a
matter of fixing various software to work nicely with btrfs as well as any
similar filesystem. As mentioned in the thread, openSUSE already uses btrfs
for loads of years. I do wonder why it isn't more supported by upstream
software (postgres, VM things, etc).

I was also surprised how good the discussion was on fedora-devel. It seems
that people didn't break into camps of "over my dead body", and "if we do not
do this I'll leave", etc. It seemed more like a "this is my really thought out
proposal, but is it actually feasible or did I forget anything". Then people
raised problems, sometimes the proposer(s) already had a solution, sometimes
it was something they didn't think of. What's nice is that despite seeing
various problems everyone seemed to understand that people are trying to
improve things.

I didn't read fedora-devel for many years so that was a welcome change over
the past.

~~~
throw0101a
> _So it 's basically a matter of fixing various software to work nicely with
> btrfs as well as any similar filesystem. I do wonder why it isn't more
> supported by upstream software (postgres, VM things, etc)._

The concept of an application 'supporting a (specific) file system' sounds
slightly ridiculous to me.

Why is Btrfs so special that apps should be rewritten for it? The always-
compared ZFS seems to work just fine without alterations being necessary, so
what make Btrfs worthy of extra considerations?

At most perhaps copy-on-write (COW) file systems _as a general class_ could
perhaps be considered, as that may help with things like shingled drives and
SSDs (i.e., zoned drives may be aligned with COW FSes).

~~~
tinus_hn
> The concept of an application 'supporting a (specific) file system' sounds
> slightly ridiculous to me.

Clearly you weren’t around in the days when NFS was a thing.

~~~
throw0101a
> Clearly you weren’t around in the days when NFS was a thing.

I was and am. I currently help admin an HPC environment with several petabytes
of NFS storage.

And the main problems I've experienced with NFS have generally been with Linux
locking. I've had a lot less drama with Solaris and BSD NFS (though I did
uncover two Solaris NFS server bugs over the years).

Whenever I have NFS issues I generally start with the assumption that there's
a bug in the NFS client code.

~~~
tinus_hn
If you think applications work on NFS by just expecting general Unix
filesystem semantics you’re in for a surprise.

The fact you’re not having trouble just means either the application writers
put in the effort to properly support it, or your use case is simple enough
not to run into problems.

~~~
throw0101a
> either the application writers put in the effort to properly support it

Given the hiccups that we've had with MySQL and locking, I'm not sure that's
the case. (Less so with Postgres.)

I have only a moderate amount of knowledge of the applications that the dozen
research groups use on the cluster, I just make sure the bits flow and haven't
heard too many complaints over the years.

~~~
tinus_hn
Clearly packages like MySQL and PostGreSQL made sure they work on NFS or they
would very clearly tell you not to use it because they’d know it’s a recipe
for disaster. The issues I am referring to typically are with things like
homegrown shell scripts or research software.

------
ge0rg
btrfs has made some headlines in the past about its severely broken checksum
computation in RAID modes, which rules them out for production use, i.e.
[https://phoronix.com/scan.php?page=news_item&px=Btrfs-
RAID-5...](https://phoronix.com/scan.php?page=news_item&px=Btrfs-RAID-56-Is-
Bad)

 _The wrong parity and unrecoverable errors has been confirmed by multiple
parties. The Btrfs RAID 5 /6 code has been called as much as fatally flawed_

It would be interesting to know if this has been addressed already.

The big fat red warning on btrfs' own wiki page is not inspiring trust either:
[https://btrfs.wiki.kernel.org/index.php/RAID56](https://btrfs.wiki.kernel.org/index.php/RAID56)

 _For data, it_ should _be safe as long as a scrub is run immediately after
any unclean shutdown_

~~~
jorritposthuma
It's interesting to see how Synology "solves" this problem:
[https://www.synology.com/en-
global/knowledgebase/DSM/tutoria...](https://www.synology.com/en-
global/knowledgebase/DSM/tutorial/Storage/What_was_the_RAID_implementation_for_Btrfs_File_System_on_SynologyNAS)

~~~
tobias3
Synology has solved it by running btrfs on top of mdraid, then patching it so
that btrfs reports errors to mdraid and putting repair code into mdraid. This
reliably works.

I'm sure some would be interested in this, but Synology seems to not take
GPLv2 too seriously and nobody seems to care about this.

RAID 5/6 in btrfs is relatively neglected because the big users that pay
developers to work on btrfs (and put the work up-stream) don't care that much
about it. E.g. Facebook is probably using it for their HDFS storage, so HDFS
gives them the RAID functionality.

~~~
throwaway8941
They use it for everything except MySQL databases, which are stored on XFS.

There was a recent discussion on LKML where Chris Mason (I think) described
all this.

~~~
petre
Does XFS have better performance than ext4 for MySQL DBs?

------
bleepblorp
For desktop use, what problem does btrfs solve better than lvm+ext4?

Btrfs is slower than lvm+ext4, doesn't like working with large files, requires
more ongoing maintenance (scrub, rebalancing), and is more prone to data
corruption. Given that lvm can do snapshots under ext4, the only real benefit
of btrfs is btrfs send, but for most use cases that doesn't seem like a large
enough benefit to be worth the rest of brtfs' drawbacks.

~~~
viraptor
> requires more ongoing maintenance (scrub, rebalancing)

I'm not sure that's a good description. Scrubbing is an option that you get
extra. You don't need to use it and the behaviour won't be different than for
example ext4 with regards to bad data detection. It's purely an extra feature.

If rebalancing is useful for desktop users (wasn't really in my experience),
I'm sure it will get a system-provided job that balances the resource use and
amount of reclaimed space.

~~~
chasil
There are concerns with ZFS of the "scrub of death" on a system lacking ECC
ram:

[https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-
kill-y...](https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-
data/)

There is some debate on this question:

[https://arstechnica.com/civis/viewtopic.php?f=2&t=1235679&p=...](https://arstechnica.com/civis/viewtopic.php?f=2&t=1235679&p=26303271#p26303271)

I'm curious if the situation is improved for BtrFS.

~~~
pseudalopex
The first link actually explains why ZFS is no more dangerous than any other
filesystem.

------
Tsiklon
Interesting - As RHEL is downstream of Fedora, I thought BTRFS was not being
explored further by Red Hat, on account of their deprecation of it downstream
in RHEL.

If I recall this was because they didn't have the developers able to work on
the software, preferring to use XFS + LVM to accomplish some of the goals of
BTRFS as their STRATIS project.

I wonder what this means for RHEL going forward in RHEL 9?

~~~
rwmj
It means nothing. This is only for the Fedora Desktop spin, and it's lead by
the Fedora community not Red Hat.

~~~
freedomben
_Disclaimer: I work for Red Hat but I have zero internal insight into this. I
'm just a happy Fedora desktop user_

I wouldn't say it means _nothing_ because Fedora is looked at as a proving
ground for inclusion in RHEL, but I would agree that one shouldn't read much
into it.

There are plenty of software packaged/supported on Fedora that isn't and won't
be shipped in RHEL. BTRFS may or may not just be yet another one like that.
I've heard/seen more excitement about Stratis (which does seem awesome so far)
than I have btrfs.

------
rwmj
A few points:

\- No this doesn't affect RHEL.

\- It's only for Fedora Desktop spin (which for various reasons including
this, but also others, you shouldn't use even on a Desktop - I install Fedora
Server on my laptop).

\- Only a subset of btrfs features will be used, especially avoiding the ones
which are known to be problematic.

~~~
rkangel
Can you elaborate on the reasons for using Fedora server instead?

I use Fedora Workstation on several laptops and desktops (without issue). I'm
curious if I'm missing some problem/opportunity.

~~~
rwmj
Defaults to GNOME, firewall disabled, ext4 (and now btrfs) instead of XFS.

(These spins are all about defaults - so installing Server doesn't make it any
less useful for desktops, but you may have to dnf install a few things the
first time you use it.)

~~~
rkangel
What do you mean "defaults to GNOME"? Server defaults to no DE at all I
thought (been a year or two since I've used it) and Fedora defaults to GNOME
anyway.

If it by default doesn't install email client and music players I don't use
that might be nice though.

~~~
boredishBoi
I think the defaults they’re referring to are for fedora desktop, not server

~~~
rkangel
Yes you're right - thank you. On a re-read I see that I had the assertions the
wrong way round.

------
KaiserPro
I have no issues with BTRFS, apart from the documentation is really poor.

I don't mind it becoming popular, but please for the love of $Deity can we
please make the docs as good as ZFS. I will contribute cash if needs be.

Facebook are utterly shit at documenting things, so yes it might work for
their usecase, but they essentially store knowledge through Shamanism, which
is terrible unless you're inducted into the world of the spirits.

~~~
eptcyka
I for one support Facebook's push to make more problems solvable solely by
entering a mud hut and ingesting hallucinogenics.

~~~
blaser-waffle
That won't make them less evil, just less coherent.

~~~
KaiserPro
Its only less coherent if youre not off your tits.

------
noodlesUK
It’s funny that fedora is moving to supporting btrfs as the default FS, when
red hat has stopped supporting it altogether. I’m a big fan. In a world where
we can’t have GPL compatible ZFS, btrfs is the next best thing.

------
AdmiralAsshat
There's probably no way to convert the legacy ext4+LUKS filesystem over, so, I
guess I'll decide when F33 comes out whether I want BTRFS enough to do a clean
install rather than an upgrade.

~~~
filmor
You can convert an ext4 filesystem to btrfs using btrfs-convert.

~~~
symlinkk
Is that stable?

~~~
LanternLight83
Alright, so, disclaimer, I have lost data doing this but it was purely
operator error! Closed my SSH session at the worst possible time. One of the
podcast hosts at Linux Unplugged made the same mistake.

The btrfs-convert tool hypothetically leaves the ext filesystem all but
untouched, and COW's the needed filesystem metadata onto the end of it, with
data modifications coming thereafter (or intelligently stored within the free
space of the ext system). You wait until you feel comfortable with BTRFS, then
delete the preserved ext system and run a balance, which rewrites all data to
disk in the usual structure. Alternatively, the preserved system can be
restored, although I don't actually see instructions for that.

[https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3](https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3)

I love btrfs and would trust it but am glad to have backups of irreplaceable
data.

~~~
AdmiralAsshat
I suppose I wouldn't need to worry about an SSH disconnect while working on
localhost, but I'd still be a little concerned about trying it. I recently had
a Pop!_OS/Ubuntu system irreparably damaged during an upgrade because the
_lock screen_ come on during the five minutes I had stepped away from
monitoring to use the bathroom.

The wiki also doesn't mention how well this process plays with LUKS, so, given
how sensitive headers and such can be, I'll probably wait to hear from some
F33 early-adopters on the conversion tool to see how it turns out. I'm
reasonably happy with my ext4 system as it is, and most of my core data is on
backup disks anyway, so I'm not sure how much the integrity checking of BTRFS
would help. Seems like it would be more critical to have on the backup disks
to make sure I'm not propagating the bit-rot.

------
flas9sd
> The switch to Btrfs will use a single-partition disk layout, and Btrfs’
> built-in volume management. The previous default layout placed constraints
> on disk usage that can be a difficult adjustment for novice users. Btrfs
> solves this problem by avoiding it.

good call, subvolumes came to the rescue of the /home partitioning. It's
useful, but the inflexibility was in the proper split of available disk space.
Novice users then saw warnings for /home - when there was still plenty of
space on the root partition.

------
48bb-9a7e-dc4f2
It's painful to see how much disinformation (even in subtle ways or to go off
the rails) some topics are getting in here.

If you want to get a better picture, also about Zfs and Fedora, read the
previous Btrfs threads where the developers took the time to discuss it. And
to kill some FUD.

I have no association with Btrfs or Fedora but I'd like to have a modern FS
in-tree as battle tested as it can be.

------
kev009
Surprising, I thought RH had drawn pretty clear lines in the sand in favor of
evolving XFS to be the universal disk FS for Linux.

------
knorker
Is btrfs still super slow at deleting large files?

It can take multiple seconds (like 5-10) on my fileserver to delete just one
~10GB file.

~~~
cmurf
The 'rm' command should complete pretty quickly. Actually freeing up space
takes time since a delete is subject to delayed allocation. The default
transaction commit time is 30 seconds. And if there are snapshots or reflink
copies, a backref walk is needed before extents can be freed.

~~~
knorker
I'm talking about the time it takes for the `rm` command to finish. I've
started the habit of running all `rm` commands in the background on btrfs.

~~~
cmurf
If it's reproducible, my suggestion is to strace the rm command and find out
what it's doing that's taking so long; and post it to the mailing list:

[https://btrfs.wiki.kernel.org/index.php/Btrfs_mailing_list](https://btrfs.wiki.kernel.org/index.php/Btrfs_mailing_list)

~~~
WGH_
Recording the kernel profile with perf would be more useful, I think. strace
would probably only show long "unlink" syscall.

~~~
cmurf
Yeah, and if perf is unrevealing, sysrq+t.

------
kevin_b_er
So how long until systemd-somethingmajor requires btrfs?

------
znpy
I just herd my roommate saying "i don't want to play too much with my amd cpu
overclocking because if i crash my system too much btrfs might get corrputed
again".

 _laughs in zfs_

~~~
vetinari
Don't worry, zfs has it's own share of problems.

I have one CentOS machine where I can't update ZFS from 0.7 to 0.8, if I want
to boot again (zfs#8885).

~~~
DiabloD3
Or switch distros.

I'm pretty sure I can't replicate this bug on Debian or Ubuntu.

~~~
diffeomorphism
That is like asking to move to a different house because you don't like the
color of the walls.

~~~
DiabloD3
To use that analogy, no, its more like moving to a different house because
lead paint remediation is too costly and error prone.

~~~
diffeomorphism
That seems like an entirely different analogy and basically just says this
apartment is bad, you should move. Then why did you move in in the first
place?

The point is, you chose your distro for a reason and one little piece of
software is far from enough to override that.

