5 ms·

- ZFS with data loss protection - Ceph with data loss protection - Any filesystem on linux-md without data loss protection, BTRFS on linux-md on synology

tecleandor 3y ago
I'm not sure what are you referring to with "data loss protection" in ZFS, but if you refer to the 'copies' parameter, that's not erasure coding, that's just full copies of the data blocks [0]. They mostly only work for some sector errors on disk. If you lose a whole disk, THEY DO NOTHING. [1] "The block pointer can store up to three copies of the data each pointed by a unique DVA. These blocks are referred to as “ditto” blocks in ZFS terminology." [2] And although VMWare calls RAID5/6 "Erasure Coding" [3](!!), I'd say it's not [4]. 0: https://docs.oracle.com/en/operating-systems/solaris/oracle-solaris/11.4/manage-zfs/copies-property.html 1: https://jrs-s.net/2016/05/02/zfs-copies-equals-n/ 2: https://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf 3: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.virtualsan.doc/GUID-AD408FA8-5898-4541-9F82-FE72E6CD6227.html 4: https://www.itprotoday.com/storage/erasure-coding-vs-raid-which-right-and-when -- Edit: some extra info and links
- matja 3y ago
  > For double or triple parity, we use a special case of Reed-Solomon coding https://github.com/openzfs/zfs/blob/zfs-2.1-release/module/zfs/vdev_raidz.c#L47-L109 https://github.com/openzfs/zfs/blob/zfs-2.1-release/module/z...
- naranha 3y ago
  Sorry I meant to write protection against data corruption (checksums)
  - tecleandor 3y ago
    Note that ZFS (and probably other fs) checksums are not data corruption protection, but data corruption detection ;) If a checksum error is detected, you depend (mostly) on your RAIDZ protection to recover from it [0]. If you're on a single drive or striped array without mirrors or raidz1/2/3... you're SOL. (ditto blocks can help but they aren't the best. I'd only recommend them if you're stuck on a single drive setup without raid) But you at least know that your data is corrupt, and that's very important too :) 0: https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Checksums.html

tecleandor 3y ago

I'm not sure what are you referring to with "data loss protection" in ZFS, but if you refer to the 'copies' parameter, that's not erasure coding, that's just full copies of the data blocks [0]. They mostly only work for some sector errors on disk. If you lose a whole disk, THEY DO NOTHING. [1] "The block pointer can store up to three copies of the data each pointed by a unique DVA. These blocks are referred to as “ditto” blocks in ZFS terminology." [2] And although VMWare calls RAID5/6 "Erasure Coding" [3](!!), I'd say it's not [4]. 0: https://docs.oracle.com/en/operating-systems/solaris/oracle-solaris/11.4/manage-zfs/copies-property.html 1: https://jrs-s.net/2016/05/02/zfs-copies-equals-n/ 2: https://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf 3: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.virtualsan.doc/GUID-AD408FA8-5898-4541-9F82-FE72E6CD6227.html 4: https://www.itprotoday.com/storage/erasure-coding-vs-raid-which-right-and-when -- Edit: some extra info and links

matja 3y ago
> For double or triple parity, we use a special case of Reed-Solomon coding https://github.com/openzfs/zfs/blob/zfs-2.1-release/module/zfs/vdev_raidz.c#L47-L109 https://github.com/openzfs/zfs/blob/zfs-2.1-release/module/z...
naranha 3y ago
Sorry I meant to write protection against data corruption (checksums)
- tecleandor 3y ago
  Note that ZFS (and probably other fs) checksums are not data corruption protection, but data corruption detection ;) If a checksum error is detected, you depend (mostly) on your RAIDZ protection to recover from it [0]. If you're on a single drive or striped array without mirrors or raidz1/2/3... you're SOL. (ditto blocks can help but they aren't the best. I'd only recommend them if you're stuck on a single drive setup without raid) But you at least know that your data is corrupt, and that's very important too :) 0: https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Checksums.html