Btrfs/ZFS/bcachefs under workloads classic benchmarks skip

farlight

https://github.com/fenio/modern-fs-benchmark

Farmadupe

@farlight assuming that you're the creator do you think you'd be able to rework the HTML/CSS? I'm sure you've got good data but speaking on behalf of my eyeballs, the results page is... hard to read!

skerit

Oh, so bcachefs is doing pretty well.

blop

For peace of mind I'm still using zfs (since the last 15+ years) but I'm definitely not impressed by the performance...

loeg

What is md-raid10 doing that is so much worse than lvm-raid10? In terms of "I/O" and "responsiveness." It's not really obvious to me from either the linked page or https://github.com/fenio/modern-fs-benchmark . In principle they should be similar?

Farmadupe

> CI runs use loop devices on shared ephemeral VMs (one VM per filesystem): compare shapes and ratios, not absolute MB/s. Each job records a host-calibration anchor — see the table. I think if you're not using baremetal for such tests, it's likely that the results are simply not comparable at all? What if another tenant is also using the disk?

blop

I think the reviews should also include the social aspect of these filesystems... There is and have been many promising and exciting FS to replace the old boring ones, but for storage you not only want to avoid technical issues but also maintainer(s) drama...

markhahn

what does "integrity" fail mean in the first table? that the case didn't recover from the 2G corruption?

sippingabonedry

So two filesystems that are essentially shunned from the Linux kernel and permanent second-class citizens, and one that was removed from Red Hat and has a questionable history of reliability. Oh boy which do I choose? I'm saying ZFS on another OS.

magicalhippo

> Every push/2-hourly cron builds each filesystem across 4 loop devices backed by sparse files, runs the suite, and publishes a results table in the job summary plus JSON artifacts. I get that real hardware costs (author mentions EUR 70 a month for a suitable server), but without at least a baseline snapshot comparison run between real hardware, both SSD and HDD, and the sparse file-backed loop devices, it's hard to take much away from this. Sadly the AI apocalypse isn't making stuff like this easy to do as a hobby.

fenio

The author of the benchmark here. I went over some comments and I'll try to tackle them here. I'm pretty clear that GH runner based benchmark is far from perfect due to noisy neighbours etc. Thus every test first is running so called calibration... to reject completely unreliable VMs. I'm fully aware that this can't completely fix the issue. Can limit it but not fix. But as of now there are 593 runs recorded so average should still be quite meaningful. Having that said I'm desperately trying to get REAL hardware to run that benchmark. With some successes ;) Few months ago I got Hetzner machine from Kent Overstreet and I was able to finish 3 runs before machine died... Results: https://bartosz.fenski.pl/modern-fs-benchmark/real-hw/ Currently I've got even more interesting machine with tons of disks and I'm running new set of benchmarks but it's really in its initial stage. https://bartosz.fenski.pl/modern-fs-benchmark/sas-hdd/ 2nd run in progress... one run on REAL hardware takes much more time than on GH runner so it's slow. But this new hardware has also so many disks that the plan is to try also more complex, tiered cache topologies. I'm working on it. I'm happy to answer any other questions, sources of every piece of this benchmark are freely available and I'm not saying they are 100% correct. I'm open to improvements.

irusensei

BCacheFS is the best Linux filesystem now that storage costs a premium. You can mix devices of different sizes and types on bcachefs. You can have foreground and background devices to balance performance and also different compression settings for foreground and background transactions. You can set replicas=N to the individual file or directory on bcachefs. For example files you can just re-download or re-build. Likewise you can set a higher number of copies to important files.

azatom

clicked, ctrl+f ssd, hdd, not found, closed

bhaney

Seeing great results from bcachefs just makes me more sad that Kent and the other kernel devs couldn't come to an understanding to keep bcachefs in-tree. I want to use it for my storage arrays so badly, but I'm still stuck with btrfs as the only available in-tree filesystem with modern features.

viraptor

It could use some explanation for the exact setups. Like how many drives are there in each raid, what does the ec config actually look like, etc. As far as I can see it's not described.

nullc

It'll be interesting when people start applying similar effort that is going into LLM search for security vulnerabilities to finding conventional reliability/performance bugs.

pzmarzly

xfs is by far the slowest in "ftruncate empty file to 1G" but the fastest in "ftruncate 256M file to 512M" - does anyone know why?