10 ms·
OpenZFS 2.0
- hlandau 6y agoWill OpenZFS on Linux ever be integrated with the Linux page cache?
- nix23 6y agoNo, they have ARC and ARCL2, if you want the traditional thing go to NILFS2 or BTRFS or in the future XFS (when they have full check-summing).
- curt15 6y ago>in the future XFS (when they have full check-summing). Is this actually planned?
- nix23 6y agoYES! Step by step and keep XFS as stable as it is (the most trustworthy linux FS of them all)
- filmor 6y agoDuring my last >15 years of Linux usage, I had exactly the two filesystems you are advocating for here crashing on me: - XFS, long time ago, had a bug that made it lose files silently - BTRFS twice, most recently about a year ago in a super simple setup (no raid or any other fanciness). I wasn't able to recover it, after a while I at least got it to mount as ro and copied the contents away These were all on Gentoo, so with relatively recent tools and vanilla kernels. The only filesystems that I never had problems with were ext4 and reiserfs.
- nix23 6y ago>reiserfs That was exactly the FS that eat my data back in ~2005. Never had problems with XFS or ZFS. With Btrfs well i just use it regularly since 2 years so i cant say much, but i think Redhat chose XFS for a reason.
- rleigh 6y agoThe reason is they hired the XFS developers from SGI. And they bought Sistina for LVM. As a result, they have been wedded to both XFS and LVM for many years now, because there is likely a combination of wanting to maximise their investment into these technologies and developing the in-house expertise to support them very well, and also in having a number of staff who are deeply committed to them and don't want to change. At some level, they must understand that both XFS and LVM are over 25 years old, and when compared with e.g. ZFS, are completely outclassed. Their current efforts developing Stratis, which is an attempt to provide more ZFS-like functionality by extending XFS, adding LVM thin pools, and managing it all with an unholy complex combination of daemons, D-BUS and Python looks like a logical progression based upon what they have to hand in house, but a strategic mistake when it can never approach ZFS in functionality or reliability simply because these technologies can only be extended so far because of fundamental design limitations. I'll be morbidly interested to see what they can stretch XFS to do. But I won't be using it myself. What I find really surprising here is that Linux in general, and RedHat in particular, don't have a competitive filesystem to offer. There is absolutely nothing which matches ZFS.
- nix23 6y ago>because there is likely a combination of wanting to maximise their investment into these technologies and developing the in-house expertise to support them very well, and also in having a number of staff who are deeply committed to them and don't want to change. Not sure if you would risk your customers data just because of that. I never had any problems with XFS. >At some level, they must understand that both XFS and LVM are over 25 years old Being a User of ZFS (on FreeBSD) myself, zfs is not much younger 2006. >and RedHat in particular, don't have a competitive filesystem to offer. That i really don't understand too. Maybe they think for "small" stuff HW-Raid or LVM is good enough and everything bigger is Ceph or Gluster anyway.
- rleigh 6y agoXFS is one of the only filesystems I've suffered serious catastrophic dataloss with. The other one is of course Btrfs which was and is the worst of the lot. What was worrying was that the XFS dataloss was due an action totally out of our hands: a power outage at a substation which took out a whole area of the city. The whole datacentre lost power, and the XFS filesystems on some massive storage arrays were completely hosed. Just from power loss. It took days to put it all back from tape backups. XFS has long been known to have problems with unclean shutdowns, but total loss from a power outage is about as bad as it gets.
- RantyDave 6y agoZFS isn't really designed for local 'temporary' file systems (IMHO). You don't really need to nest checksums, create snapshots or volume manage when you're slugging pages between ram and nvme.
- keeperofdakeys 6y agoProbably never. ZFS isn't just a filesystem, it was developed to be an entire storage system that's vertically integrated, so ARC is a fundamental part of the filesystem design. ZFS also has a huge legacy. Right now the license (probably) prevents you from legally shipping a compiled zfs module with the linux kernel, just solving that seems insurmountable. It's also supported on Illumos and FreeBSD, trying to refactor it to use the linux page cache would have a chance of introducing bugs to these platforms.
- mholt 6y agoI'm looking at setting up my first ZFS pool ('zpool'?) in a few weeks, on Linux. Will I be using OpenZFS or something else? Ubuntu 20.04. (Sorry if noise; I'm just trying to get an idea of how relevant this 2.0 release is to me.)
- iotku 6y ago> The ZFS on Linux project has been renamed OpenZFS! Both Linux and FreeBSD are now supported from the same repository making all of the OpenZFS features available on both platforms. Previously it was called ZFS on Linux, but now ZFS development is unified on the "OpenZFS" codebase shared both between Linux and FreeBSD as much of the development effort for ZFS in general ended up there.
- mholt 6y agoAh, I was wondering what happened since I stopped hearing about "ZFS on Linux" so now I know what to search for. Thanks!
- codetrotter 6y agoThis is huge! And very exciting :D One thing I am wondering about is this: > Redacted zfs send/receive - Redacted streams allow users to send subsets of their data to a target system. This allows users to save space by not replicating unimportant data within a given dataset or to selectively exclude sensitive information. #7958 Let’s say I have a dataset tank/music-video-project-2020-12 or something and it is like 40 GB and I want to send a snapshot of it to a remote machine on an unreliable connection. Can I use the redacted send/recv functionality to send the dataset in chunks at a time and then at the end have perfect copy of it that I can then send incremental snapshots to?
- 0xCMP 6y agoI think it's more if you want to not send scratch or cached files you can have it automatically remove it from the snapshot being sent > Redacted send/receive is a three-stage process. First, a clone (or clones) is made of the snapshot to be sent to the target. In this clone (or clones), all unnecessary or unwanted data is removed or modified. This clone is then snapshotted to create the "redaction snapshot" (or snapshots). Think of it like a selective sync in Dropbox or SyncThing at the FS level.
- vorpalhex 6y agoThat's a protocol problem, use a protocol such as rsync. You don't need to use redacted sends/recvs.
- anderspitman 6y agoI'd love to get rid of my FreeNAS VM and run ZFS directly on my Linux desktop, but having to mess with the kernel has kept me from attempting it so far. Maybe I'm worrying about nothing. btrfs seems like the main alternative if you want native kernel support, but when I checked a couple years ago there seemed to be a lot of concerns about the stability. Is that still the case?
- paulsmal 6y agoYou know Ubuntu support ZFS since 20.04. Experimental, but quite stable for me. Just select file system during installation process.
- nix23 6y ago>ZFS directly on my Linux desktop Use BTRFS trust me it's stable now...well the commands are terrible compared to ZFS. All my Server are FreeBSD but on the Laptop and on one Workstation i have openSUSE Tumbleweed since like 2 years and it works great.
- neolog 6y ago> the commands are terrible what does that mean?
- vetinari 6y agoFor zfs specific features, there are 'zfs' and 'zpool' commands (well, binaries, and the first parameter is a command). For btrfs, there is 'btrfs'. So I guess that the GP considers /usr/sbin/{zfs,zpool} more intuitive than /usr/sbin/btrfs.
- throw0101a 6y agoSadly dRAID (parity Declustered RAIDz) just missed the cut-off for 2.0, but it looks like it will be in 2.1: * https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAID%20Howto.html https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI... * https://www.youtube.com/watch?v=jdXOtEF6Fh0 https://www.youtube.com/watch?v=jdXOtEF6Fh0
- Nican 6y agodRAID looks really fascinating, but presentation is pretty abstract. Would it allow to add/remove drives from a pool, and allow ZFS to rebalance itself? Would be great for home use, where I have a lot of drives that I collected over the years that are not the same size. EDIT: The more I read into this, it still seems assume that all drives must be of the same size.
- pantalaimon 6y agoYou can already do that with btrfs.
- Nican 6y agoI currently use btrfs with RAID1 at home, and it works great. But btrfs also does not have the track record for being the most stable filesystem as compared to ZFS. [1] https://lore.kernel.org/linux-btrfs/20200627032414.GX10769@hungrycats.org/ https://lore.kernel.org/linux-btrfs/20200627032414.GX10769@h... [2] https://lore.kernel.org/linux-btrfs/20200627030614.GW10769@hungrycats.org/ https://lore.kernel.org/linux-btrfs/20200627030614.GW10769@h... [3] https://lore.kernel.org/linux-btrfs/20200520013255.GD10769@hungrycats.org/ https://lore.kernel.org/linux-btrfs/20200520013255.GD10769@h...
- rewtraw 6y agoYou can do that with ZFS too, at least for mirrored sets (i.e. RAID10). It's possible to remove a vdev, and the pool will migrate the data to the remaining vdevs.
- hardwaresofton 6y ago
- voltagex_ 6y agoAnyone know what version of Ubuntu Server this will land in?
- mlex 6y agoJust built a FreeNAS system over the past couple weeks and finished doing burn-in tests of my hard drives, wonder if I should wait and see how to install OpenZFS 2.0.0 before I create my storage config.
- 1over137 6y agoFreeNAS 12 (now named TrueNAS) is already using OpenZFS 2.0, or very nearly.
- nraynaud 6y agoDoes it support NFS4.2?(fallocate, sparse files and server side copy)
- ed25519FUUU 6y agoAren't ZFS upgrades to existing vdevs really simple? I don't see any reason why you need to wait.
- rodgerd 6y agoThey certainly seem to be within OpenZFS over the past few years.
- mlex 6y agoThat’s the idea I’ve gotten when looking around online. I figured I was in the uncommon situation of having a completely blank and ready system, so I could afford to just wait a few days.
- 1over137 6y agoYes, ZFS upgrades are really simple, but they are one-way, you can't downgrade after.
- rodgerd 6y agoCongratulations - it's great to see the code unification on the two key ZFS platforms, and continuing to add useful features, especially around at-rest encryption. Many thanks to the various OpenZFS contributors.
- boston_sre87 6y agoHas there been any progress on the zfs on linux Linus disagreement front since this article? https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-statements-arent-right-heres-the-straight-dope/ https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-s...
- ed25519FUUU 6y agozfs on linux available as root partition since 20.04. Working quite well I might add!
- gilrain 6y agoIt's still a big pain if you like to keep your kernel relatively up to date. I switched to btrfs; it just working is worth the few extra warts over ZFS.
- xenophonf 6y agoI've had zero problems with kernel updates on Ubuntu 20.04 with ZFS on a natively encrypted root. I followed the instructions in the wiki, lightly modified for my hardware and workload: https://gist.github.com/xenophonf/76fd44ae24772e457cb63d00c00a5967 https://gist.github.com/xenophonf/76fd44ae24772e457cb63d00c0... `apt-get update && apt-get dist-upgrade -y` works as expected. I plan to switch to a similar config on my Lenovo laptop when I upgrade it to the next Ubuntu LTS release.
- accelbred 6y agoUbuntu's kernel isn't exactly keeping up to date though. I assume the person you were replying to may be following mainline. As someone using new kernel version as they are released, I'm not willing to use a filesystem that may break with a kernel update. It also seems openzfs only supports up to kernel 5.6, according the the github release. I'm on 5.9, so its not even an option.
- 6y ago
- ed25519FUUU 6y agoSide note, they really should have in big-bold letters "DO NOT ENABLE DEDUPLICATION UNLESS YOU HAVE A TON OF RAM!" on their readme. That was a huge mistake on my part. The ram requirements are VERY high for good performance. I realized how bad the performance was when it took about 2 hours to delete 1000 files.
- 1over137 6y agoThat's not new with 2.0 though. It's forever been the case with ZFS. Everything that discusses dedupe basically says: 'don't use it'.
- freddie_mercury 6y agoIt does already say that. This is what it says: Deduplication is the process for removing redundant data at the block level, reducing the total amount of data stored. If a file system has the dedup property enabled, duplicate data blocks are removed synchronously. The result is that only unique data is stored and common components are shared among files. Deduplicating data is a very resource-intensive operation. It is generally recommended that you have at least 1.25 GiB of RAM per 1 TiB of storage when you enable deduplication. Calculating the exact requirement depends heavily on the type of data stored in the pool. Enabling deduplication on an improperly-designed system can result in performance issues (slow IO and administrative operations). It can potentially lead to problems importing a pool due to memory exhaustion. Deduplication can consume significant processing power (CPU) and memory as well as generate additional disk IO.
- Mashimo 6y agoMost guides I read tell you that you should not enabled DEDUP unless you know what you are doing and it will use a lot of ram.
- zmix 6y agoTo me this sounds more like you didn't RTFM ;-)
- deleted 6y ago[deleted]
- kzrdude 6y agoOpenZFS is in fact a more prestigeous name and it already sounds better than ZFS on Linux.
- solarengineer 6y agoIf you get on the calls, you’ll find zero hostility across the operating systems devs. The focus is on OpenZFS, with the Linux branch gradually becoming baseline for the FreeBSD work as well. Illumos ( where OpenZFS originated after Illumos was formed post the OpenSolaris shutdown) hasn’t moved to this baseline yet due to the significant OS level differences and instead code is pulled between the “branches” as needed. The collaboration happens via email and regular calls.
- justinclift 6y agoSlightly off topic, but it seems like GitHub can't/won't display the user profile page for one of the OpenZFS developers: https://github.com/behlendorf https://github.com/behlendorf For me, that gives a unicorn 100% of the time (tried across several minutes), instead of showing the developer profile. Anyone else seeing that?
- jclulow 6y agoIt does, indeed, report that "This page is taking too long to load."!
- justinclift 6y agoYeah, it's still unicorning for me, about a day later. :(
- bromonkey 6y agoIt loaded for me earlier today, I think github is just having issue.
- rincebrain 6y agoLoaded in under 5 seconds flat for me, perhaps it's something strange with whatever edge server you're hitting?
- justinclift 6y agoCould be, but if so it's persistent. It's about a day later now, and the page still won't load.
- justinclift 6y agoTesting now, the page is finally loading. Page load time of ~2 days... that's different. ;)
- jstrong 6y agohooray for zstd compression!
- qalmakka 6y agoFinally, this means we've a way to share "real" filesystems on both FreeBSD and Linux. The only other filesystems you could open without issues on both are FAT and NTFS (thought NTFS-3G), both of which are less than ideal for data you care about.
- tpetry 6y agoZstd compression with configurable levels is really interesting: You could write every block first with a level comparable to lz4 for very fast performance. And if a block has not been rewritten for some time you recompress them with a compression level allowing more compression and comparable decompression performance. So cold data (cold write, cold/hot read) will take less and less space over time while still having the same read performance.
- KMag 6y agoThat would be an even more interesting feature for NILFS2, as I understand it, its ring buffer structure requires moving the oldest unmodified blocks as the ring buffer write frontier approaches. Any blocks that are forced to be copied are by definition old and unmodified, and need to be moved anyway, so why not recompress? AFAIK, there are no plans for compression in NILFS, but I think it's an interesting idea.
- rcthompson 6y agoMy understanding is that for ZFS, things like this would require a mythical feature called "block pointer rewrite", the same feature required to implement out-of-band deduplication.
- rincebrain 6y agoYou are correct - ZFS hardcodes the assumption that data's location on disk will never change once written very deeply, and offline dedup/data migrating of any sort would require that. (It would also be a performance nightmare - you'd have a permanent indirection table you'd need to use for _everything_, and if you've ever seen how ZFS dedup performs with its indirection table not on dedicated SSDs, you can understand why this is terrible.)
- tpetry 6y agoThe block could still be rewritten from the view of zfs as long as it does not update the last-written timestamp (does zfs have this?). I was just describing how it would look like from the birds eye.
- KMag 6y agoHow's the memory consumption of ZFS without deduplication these days? I've got a couple of 4 TB drives connected to a single board ARM computer with 2 GB of RAM. I used to use btrfs, but switched to XFS after I accidentally filled up a drive and was unable to recover.
- rincebrain 6y agoZFS without dedup will just run slower with less RAM available for caching, up to a point (I think the lowest I've seen someone run it with ARC configured to use in recent memory is 128 MB? I believe 32 MB or so is the minimum below which OpenZFS will just ignore you if you try to tell it to use less...) I've seen people use it as a rootfs on RPis, and have personally run it on Pis for brief occasions without encountering any RAM problems.