7 ms·

For home usage, if you have backups raidz1 is fine (just do an incremental backup at the first sign of trouble). If you don’t have backups, then you probably sh

by cm2187 2mo ago

For home usage, if you have backups raidz1 is fine (just do an incremental backup at the first sign of trouble). If you don’t have backups, then you probably shouldn’t be running a NAS in the first place.

layer8 2mo ago
This isn’t black and white. You might have huge amounts of non-essential data that aren’t worth the cost of off-site backups, but worth the cost of an extra disk of redundancy to lessen the risk. Even when you do have backups, it will reduce the risk of extended downtime (and possibly egress costs) caused by having to restore large amounts of data from backups.
bluGill 2mo ago
For most people snapshots are enough backup. It is still useful to have an offsite backup, but realistically fire is rare enough that you can risk it, and that is about the only risk most people have (you can have your NAS in a location likely far from where a fire might break out to reduce the risk farther).
- cm2187 2mo ago
  Fire, malware, accidental deletions, capricious RAID controller (pre-ZFS). And that’s only the stuff that happened to me. Add power surge, theft, correlation in SSD failures (eg power on counter overflow firmware bug), damaging the array while moving, etc.
  - bluGill 2mo ago
    Malware, accidental deletion, RAID controller issues - all things ZFS with snapshots are immune to. You can decide if the other worry you or not - the risk is not zero, but it may be acceptable. Your backups are all subject to similar issues.
    - cm2187 2mo ago
      Snapshots don’t protect you from malware. I bet 99% of home users use the same credentials on all their machines, once a malware compromised one, the others are compromised within seconds. [edit] also snapshots aren’t really workable for large files. Remux a movie file and now it occupies twice the space.
Cyph0n 2mo ago
The middle ground I go with is to structure the pool using datasets and zvols, and then periodically backup the critical ones using zfs send. For example, I can live without my movie collection, but I cannot afford to lose personal photos.
ppchain 2mo ago
In the Linux ISO community the math is different because none of the data is essential, however there is a lot of it. Backing it all up even one time to an off-site location would be a 2x expense for a hobby. So 3-2-1 is hard to justify. Instead the math can be different e.g. it's ok to lose some data but not all. Therefore you might prefer the unraid approach over zfs where losing more than parity doesn't kill the whole pool.
- dwedge 2mo ago
  I'm actually in this boat right now. I have 2x26TB drives in a 4 base nas (two empty, plus 3x M2 empty) and I'm trying to decide the best way to set them up. I have around 3TB of backups that I care about as backups (but they don't need to be online, just archive backups at this point), and around 5TB of media that can be replaced. I don't want to lose 26TB just for a mirror, and I have a spare 8TB USB HDD. I'm torn between unraid and just Debian, and I'm torn between just two separate devices and one RAID 1 partition one RAID 0
bensyverson 2mo ago
The risk of z1 is that if you get a read error during resilvering, that data is permanently corrupted. The odds of this happening go way up the larger your individual drives are. This is why I chose RAIDZ2 for my NAS. I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data.
- Dylan16807 2mo ago
  > The odds of this happening go way up the larger your individual drives are. This is a common claim, but honestly, citation needed. We can't just apply bit error rates from the datasheet that haven't been updated in 15 years. I'm sure a 2-day rebuild is a little more risky than a 4-hour rebuild, but I'm not convinced it's by all that much. Especially if you had a monthly scrub going to prevent disk rot and disks secretly getting super fragile. > I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data. And how many times did you have a read error on one of your other disks during a resilver?
  - bell-cot 2mo ago
    >> The odds of this happening go way up the larger your individual drives are. > This is a common claim, but honestly, citation needed. FWIW, here's the quick & dirty argument for that: Assume: - We're working within a given generation of disk technology, with comparable drive and interface tech. - The drives' I/O speed is relatively constant, not scaling with the drive capacity. - The chance of any given bit (of data on a disk) going bad, in a given time interval, is relatively constant. - The drives, regardless of size, will be some fixed-ish percentage full. (If nothing else, the boss won't sign off on overspending for unneeded capacity.) From there - the time needed for resilvering scales linearly with the drive size. As does the number of non-redundant bits that could go bad in any given second of the resilvering. Linear time linear - so the chance of data loss during resilvering rises with the square of the disk size.
    - Dylan16807 2mo ago
      > - The chance of any given bit (of data on a disk) going bad, in a given time interval, is relatively constant. We can't just assume this, though. Especially for data that's being checked regularly, because the drive can see how many pre-ECC bits have gone bad and rewrite iffy sectors. And even if it's true, it's normal to go years without errors. If you have a 15% chance of getting an error each year, and your degraded array has 4 drives without redundancy, then the chance of getting an error in 4 hours is .02% and your chance of getting an error in 48 hours is .3%. Having that risk show up once every few years is pretty negligible compared to everything else that could go wrong with your array. On top of that, if you don't have a hot spare and it takes 5 days to get a new drive, the difference in resilvering speeds is now 5.2 days versus 7 days and that barely matters. The main situation where you have to worry about extra drives failing is when you think the resilvering process will greatly accelerate drive deaths. And that depends very much on the specific numbers you have in mind.

by cm2187 2mo ago

layer8 2mo ago

This isn’t black and white. You might have huge amounts of non-essential data that aren’t worth the cost of off-site backups, but worth the cost of an extra disk of redundancy to lessen the risk. Even when you do have backups, it will reduce the risk of extended downtime (and possibly egress costs) caused by having to restore large amounts of data from backups.

bluGill 2mo ago

For most people snapshots are enough backup. It is still useful to have an offsite backup, but realistically fire is rare enough that you can risk it, and that is about the only risk most people have (you can have your NAS in a location likely far from where a fire might break out to reduce the risk farther).

cm2187 2mo ago
Fire, malware, accidental deletions, capricious RAID controller (pre-ZFS). And that’s only the stuff that happened to me. Add power surge, theft, correlation in SSD failures (eg power on counter overflow firmware bug), damaging the array while moving, etc.
- bluGill 2mo ago
  Malware, accidental deletion, RAID controller issues - all things ZFS with snapshots are immune to. You can decide if the other worry you or not - the risk is not zero, but it may be acceptable. Your backups are all subject to similar issues.
  - cm2187 2mo ago
    Snapshots don’t protect you from malware. I bet 99% of home users use the same credentials on all their machines, once a malware compromised one, the others are compromised within seconds. [edit] also snapshots aren’t really workable for large files. Remux a movie file and now it occupies twice the space.

Cyph0n 2mo ago

The middle ground I go with is to structure the pool using datasets and zvols, and then periodically backup the critical ones using zfs send. For example, I can live without my movie collection, but I cannot afford to lose personal photos.

ppchain 2mo ago

In the Linux ISO community the math is different because none of the data is essential, however there is a lot of it. Backing it all up even one time to an off-site location would be a 2x expense for a hobby. So 3-2-1 is hard to justify. Instead the math can be different e.g. it's ok to lose some data but not all. Therefore you might prefer the unraid approach over zfs where losing more than parity doesn't kill the whole pool.

dwedge 2mo ago
I'm actually in this boat right now. I have 2x26TB drives in a 4 base nas (two empty, plus 3x M2 empty) and I'm trying to decide the best way to set them up. I have around 3TB of backups that I care about as backups (but they don't need to be online, just archive backups at this point), and around 5TB of media that can be replaced. I don't want to lose 26TB just for a mirror, and I have a spare 8TB USB HDD. I'm torn between unraid and just Debian, and I'm torn between just two separate devices and one RAID 1 partition one RAID 0

bensyverson 2mo ago

The risk of z1 is that if you get a read error during resilvering, that data is permanently corrupted. The odds of this happening go way up the larger your individual drives are. This is why I chose RAIDZ2 for my NAS. I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data.

Dylan16807 2mo ago
> The odds of this happening go way up the larger your individual drives are. This is a common claim, but honestly, citation needed. We can't just apply bit error rates from the datasheet that haven't been updated in 15 years. I'm sure a 2-day rebuild is a little more risky than a 4-hour rebuild, but I'm not convinced it's by all that much. Especially if you had a monthly scrub going to prevent disk rot and disks secretly getting super fragile. > I've had to resilver 2-3 times over the past 10 years, and never lost a byte of data. And how many times did you have a read error on one of your other disks during a resilver?
- bell-cot 2mo ago
  >> The odds of this happening go way up the larger your individual drives are. > This is a common claim, but honestly, citation needed. FWIW, here's the quick & dirty argument for that: Assume: - We're working within a given generation of disk technology, with comparable drive and interface tech. - The drives' I/O speed is relatively constant, not scaling with the drive capacity. - The chance of any given bit (of data on a disk) going bad, in a given time interval, is relatively constant. - The drives, regardless of size, will be some fixed-ish percentage full. (If nothing else, the boss won't sign off on overspending for unneeded capacity.) From there - the time needed for resilvering scales linearly with the drive size. As does the number of non-redundant bits that could go bad in any given second of the resilvering. Linear time linear - so the chance of data loss during resilvering rises with the square of the disk size.
  - Dylan16807 2mo ago
    > - The chance of any given bit (of data on a disk) going bad, in a given time interval, is relatively constant. We can't just assume this, though. Especially for data that's being checked regularly, because the drive can see how many pre-ECC bits have gone bad and rewrite iffy sectors. And even if it's true, it's normal to go years without errors. If you have a 15% chance of getting an error each year, and your degraded array has 4 drives without redundancy, then the chance of getting an error in 4 hours is .02% and your chance of getting an error in 48 hours is .3%. Having that risk show up once every few years is pretty negligible compared to everything else that could go wrong with your array. On top of that, if you don't have a hot spare and it takes 5 days to get a new drive, the difference in resilvering speeds is now 5.2 days versus 7 days and that barely matters. The main situation where you have to worry about extra drives failing is when you think the resilvering process will greatly accelerate drive deaths. And that depends very much on the specific numbers you have in mind.