13 ms·

Not a great time price-wise to be building a NAS, but I have been doing so for the last two weeks. Inside a Jonsbo N6 case, which is pretty nice with an 8x SATA

by Confiks 2mo ago

Not a great time price-wise to be building a NAS, but I have been doing so for the last two weeks. Inside a Jonsbo N6 case, which is pretty nice with an 8x SATA backplane and drive bays (unlike the earlier Jonsbo variants). I ended up on shucking 4x the 14 TB WD Elements Desktop. They contain helium drives, the WD140EDGZ in my case, and are about a third cheaper than 4x the 12 TB WD Red Plus drives (which are air-filled). The shucking was easier than I expected too, and the performance seems very comparable. The warranty is a definite downside (European, so no Magnuson-Moss), but I think I can even get them back in their enclosure should they fail during the 2-year warranty period. I've put some second hand 256 GB M.2 SSDs in there as boot drives. It was a bit of a struggle to get it to work in a way that failure of one of the drives doesn't hold up booting, combined with LUKS, TPM keys and ZFS on root. Learned a lot about systemd-boot which I have never used before, but feels a lot saner to me than grub ever was. So now I have a large script which debootstraps a Debian based NAS into being. I noticed that there are a lot of ZFS myths and cargo culting. For example TFA mentions ECC RAM, which in some circles is a must-have because ZFS would wreck your pool during a scrub otherwise, which is a myth. It's also very expensive, especially this year. You also don't need much RAM for ZFS, L2ARC doesn't use much RAM at all, to name a few others. Still doubting about setting `dnodesize=auto` (which is the default), because there are some horror stories about that [1]. And it seems impossible to find a cloud storage provider with reasonable prices that supports `zfs send`. Rsync.net upped their minimum order to 10 TiB recently, which is far too much for my use case. [1] https://github.com/openzfs/zfs/issues/11353 https://github.com/openzfs/zfs/issues/11353 [2] https://www.rsync.net/products/zfsintro.html https://www.rsync.net/products/zfsintro.html

nuker 2mo ago
> combined with LUKS, TPM keys Does it work? Server can reboot and use TPM to unlock rootfs? What about /boot - encrypted and tamper proof? Resists evil maid attack? Rabbit hole, I know, but so fascinating if you solved it all :)
- Gigachad 2mo ago
  If you are worried about someone breaking in to your house and replacing the bootloader while leaving your drives in place I probably wouldn't use the TPM auto unlock even if in theory secure boot should be able to handle this. But in reality that will never happen and the only actual attack you need to be worried about is junkies breaking in and flogging the drives on facebook marketplace. For which, this level of security is fine.
  - nuker 2mo ago
    > junkies breaking in and flogging the drives For this you dont need TPM. Just a LUKS key in rootfs /etc. He said TPM :)
- toast0 2mo ago
  I run my cheap hosting box with a cleartext boot setup that uses ssh to automatically grab the key for the real root from my home server (or an alternate at my MILs house). Using FreeBSD, but similar concepts. A previous hoster once gave me someone else's drives without wiping them. I don't want random customers snooping around on my data if a similar mistake happens with my disks. For home use, I run without disk encryption. If I ever need to do data recovery, it's not going to be possible with encrypted disks and one point of a centralized NAS is to have stable long term storage.
- NekkoDroid 2mo ago
  > What about /boot - encrypted and tamper proof? Resists evil maid attack? > Rabbit hole, I know, but so fascinating if you solved it all :) Encrypting `/boot/` is the wrong thing to do. One can just replace it with something that logs the encryption key instead (and sends it off to somewhere). You actually need to verify that what you expect to be there is actually there and a encryption isn't gonna do that for you. A reasonably secure boot chain looks roughly as follows: bootloader is checked by the UEFI (read secure-boot), bootloader chain loads a signed UKI which also uses secure boot verification (or just skip the bootloader if you only have a single kernel with no fallback), this then loads a signed dm-verity `/usr/` which brings you to the login screen. Root is just data (e.g. mounted `noexec`) and is encrypted using e.g. PCR 11 so that it only unlocks if the UKI is the one that is expected (with some other backup unlock method). This is basically #2 of the "Design Goals" of Poetterings "Fitting everything together"[0]. It's a good read if you are looking to designing your own Linux image and are looking for some inspiration. There is ParticleOS[1] which somewhat dogfoods this kind of system, though I don't think it is anything remotely considered for a production system. [0]: https://0pointer.net/blog/fitting-everything-together.html https://0pointer.net/blog/fitting-everything-together.html [1]: https://github.com/systemd/particleos https://github.com/systemd/particleos
jsr0 2mo ago
Have you looked at https://zfs.rent/ https://zfs.rent/ for cloud storage? You have to provide your own drive, but the monthly cost is relatively low.
devy 2mo ago
Yep. The 4 Samsung SSD 990 PRO 4TB NVMe SSDs are $1100 apiece. Yikes! https://www.samsung.com/us/memory-storage/nvme-ssd/990-pro-pcie-4-0-nvme-ssd-4tb-sku-mz-v9p4t0b-am/ https://www.samsung.com/us/memory-storage/nvme-ssd/990-pro-p...
tomasff 2mo ago
> So now I have a large script which debootstraps a Debian based NAS into being. Is part of this publicly available? After seeing, - https://words.filippo.io/frood/ https://words.filippo.io/frood/ - https://0pointer.net/blog/fitting-everything-together.html https://0pointer.net/blog/fitting-everything-together.html I've been looking at setting something up with mkosi/systemd for my home server/NAS - it's interesting seeing how different setups are solving the problem for inspiration.
watermelon0 2mo ago
I've been thinking about building my own NAS as well. Mind sharing how much did you pay for those hard drives, and what motherboard did you choose?
- Confiks 2mo ago
  It was €329 per hard drive (through a reputable store), and I chose the ASRock B650M-HDV/M.2 mATX motherboard combined with a Ryzen 5 8500G. Stock CPU cooler, replaced the Jonsbo case fans with Arctic P12 Pro PST LN. I only slightly regret the PSU (MSI MAG A650GL), which could be quieter. Not that it's very noisy, and it's a great PSU otherwise, but I should've chosen one that just shuts the fan down at low power usage.
- conradev 2mo ago
  https://diskprices.com/ https://diskprices.com/ is great for this You have SATA or SAS to pick from. The CPU requirements for a storage server are not high. On a typical ATX board you have motherboard SATA and can put SAS controllers in the spare PCIe slots. My first "NAS" was two 22TB hard drives in a ZFS pool on my motherboard SATA
  - dapperdrake 2mo ago
    ATA (SATA), SCSI (SAS), and NVM (NVMe).
    - Brian_K_White 2mo ago
      no. just sata & sas. There is no nvme spinning drive and flash loses bits just from sitting, it's not archival. flash is good for working not for storing.
      - Dylan16807 2mo ago
        A NAS is plugged in. Flash works fine for it.
      - QuiEgo 2mo ago
realityfactchex 2mo ago
> Not a great time price-wise to be building a NAS That under-states the matter. It is a terrible time, price-wise, to build a NAS. I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times. (If there were a magical choice between having AI and significantly more expensive storage, and having no AI and some program to dump that investment money into getting and somehow leveraging significantly more available storage, that is.)
- randusername 2mo ago
  I'm doing it anyway and I've found some secondhand deals. Maybe COVID homelabbers are offloading gear? I built a 24 TB HDD NAS in a 36 bay chassis for around $1000 all-in: mb, ram, rails, chassis, rack, disks, hba, nic Awful Watts/TB but the plan is to run it and the GPU rig on solar.
  - trollbridge 2mo ago
    I obtained a used Dell PowerEdge and just stuffed a bunch of drives in there. It's old enough that the RAM is not overly expensive (DDR3), so it has 192GB of RAM to effectively cache the data, and then 32TB of raw storage. (Of course I regret not spending the extra money to take it to 384GB now.) The nice thing about the array being big is that I can just RAID 1 it instead of worrying about tinkering with RAID 5, and leave a drive as a hot spare. PCIe 2.5G NIC for the uplink, and then it can serve over SMB or iSCSI. The main use case for this thing, incidentally, is it just is a caching proxy that holds Docker images, models off of Huggingface, and so on.
  - bix6 2mo ago
    36 bay? Why so many?
    - randusername 2mo ago
      The 36 bay was all I could find on fb marketplace. I fell in love with the silly idea of squeezing 10G NIC + 10G writes (to a 2x12 RAID 10 HDD array) on the thunderbolt bus of my 12 year old macbook pro. Then I got distracted with the server mb and truenas and ZFS.
Hamuko 2mo ago
I'm also planning a new home server build now and the prices are definitely relatively ass. So far I've spent 820€ on two 22 TB WD Elements HDDs, 375€ on 2x16 GB DDR5 kit, and 520€ on two 2 TB M.2 SSDs (cache). About 1700€ and I still have no server to show for it. Doesn't help that I've been procrastinating on picking the CPU and motherboard.
Fr0styMatt88 2mo ago
I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real? Like I’m already giving up two full drives for redundancy (which saved my ass - I recently had two drives fail on me in quick succession — both SSDs from what looks like an identical batch) but then the advice is kinda saying I need to keep at least another drive worth of space free for the pool to perform well and not crap itself. That hurts with current prices for sure.
- vladvasiliu 2mo ago
  I don't think they are. I've already filled a ZFS pool almost to the brim with no adverse effects.
  - barrkel 2mo ago
    My experience has been escalating fragmentation past 90%, along with lower performance owing to the fragmentation.
- cm2187 2mo ago
  For home usage, if you have backups raidz1 is fine (just do an incremental backup at the first sign of trouble). If you don’t have backups, then you probably shouldn’t be running a NAS in the first place.
  - layer8 2mo ago
    This isn’t black and white. You might have huge amounts of non-essential data that aren’t worth the cost of off-site backups, but worth the cost of an extra disk of redundancy to lessen the risk. Even when you do have backups, it will reduce the risk of extended downtime (and possibly egress costs) caused by having to restore large amounts of data from backups.
  - bluGill 2mo ago
    For most people snapshots are enough backup. It is still useful to have an offsite backup, but realistically fire is rare enough that you can risk it, and that is about the only risk most people have (you can have your NAS in a location likely far from where a fire might break out to reduce the risk farther).
neilpanchal 2mo ago
> 4x the 14 TB WD Elements Desktop. I recommend staying away from SATA drives (huge consumer rush) and look for SAS drives on eBay, particularly HC520 Helium drives from HGST/WD. Need a SAS3008 PCIe adapter ($20) and a SFF-8643 splitter ($30). No backplane is required. Huge lots of decommissioned drives frequently appear. I bought Qty 10* of HC520 (12TB) SAS drives for $1000 about 3 weeks ago, avg age is about 2.5 years, still well within its rated lifetime. Yea may be some of the stuff is fear mongering and cargo culting. I was told ECC is necessary for ZFS. When the article was written, it was cheap af (2024) to buy ECC RAM so not much consideration was given to it. -- (*) I have no idea what to do with it. Anyone has any good idea for using 120TB space? I have about 40TB unused bandwidth in the datacenter, may be host a Debian mirror? Donate storage/bandwidth to Internet Archive? Please contact me, appreciate it.
- Confiks 2mo ago
  > I bought Qty 10* of HC520 (12TB) SAS drives for $1000 about 3 weeks ago, avg age is about 2.5 years, still well within its rated lifetime. That's cheap indeed. Enough headroom for some failing disks too. How is the noise and power usage? I didn't look at SAS drives at all, because my impression was that they're very noisy. I can place my NAS in a closed off room, but it's not too far away and I was afraid SAS drives would be audible through the wall. At the same time, the shucked drive I'm using presents itself as an WD Ultrastar, which comes very close to a SAS drive, and isn't very noisy.
  - neilpanchal 2mo ago
    > How is the noise and power usage? I think they're same as SATA drives, just different interface. AFAIK they have the same physical dimensions and same internals.
    - bobbob1921 2mo ago
      This reply is exactly right about using enterprise drives (hdd), and ideally SAS. Note the power usage of SAS vs sata for an identical drive make/model is very small sometimes 0.5 W to 1.5 W at idle (idle power draw is what matters for spinning HDD’s as underload it doesn’t increase much and idle is what you’ll be at 98% of the time). Also enterprise SSDs are much better than consumer SSD‘s for ZFS, and be aware that when you get above the 4 TB size SSD drives frequently use more power than HDD drives (i’m mainly referring to enterprise non-consumer level drives as that’s all I run for 100s of disks across many zfs systems over past 15 yrs). Also something people forget for home nas or figuring out cost for a nas, the power draw of the entire system times your electricity price you then have to multiply this times 2x to 4x times if your climate requires cooling air conditioning of any kind
ylyn 2mo ago
You assert that ECC RAM being necessary for ZFS is just a myth but provide no justification for why that is untrue. Is it not the case that if you don't have ECC memory, ZFS could end up writing a checksum that does not match the data if you get a bitflip in just the right (wrong) spot?
- vardalab 2mo ago
  Had it happen to my PBS backups running on ZFS without ECC just the other day. Turned out to be heat related, memory was fine but during long backups heat was causing bit flips.
- naturalmovement 2mo ago
  > You assert that ECC RAM being necessary for ZFS is just a myth but provide no justification for why that is untrue. ZFS without ECC is no more risky than any other file system / software RAID without ECC. As no one owes you an explanation, it would take you five seconds to Google this and discover: 1. It's been disproven, with one of the original ZFS developers chiming in. 2. The original source of the rumor was a forum post that somehow became canon.
  - wpm 2mo ago
    What an incredibly rude comment
  - cormorant 2mo ago
    > five seconds to Google this and discover ...(after much longer) that it's a rabbit hole with nuance. https://news.ycombinator.com/item?id=18481910 https://news.ycombinator.com/item?id=18481910
- Confiks 2mo ago
  Yes, indeed. ECC RAM is better than non-ECC RAM, also for ZFS. The myth, popularized by a notorious thread on the TrueNAS forums [1], is specifically that ZFS requires ECC RAM, and will do worse than other filesystems without it, because scrubbing will multiply a single bitflip into a failed pool. A ZFS core developer says that that isn't the case [2]. Here's some more reasoning [3], also about many other myths. [1] https://www.truenas.com/community/threads/ecc-vs-non-ecc-ram-and-zfs.15449/ https://www.truenas.com/community/threads/ecc-vs-non-ecc-ram... [2] https://news.ycombinator.com/item?id=18480016 https://news.ycombinator.com/item?id=18480016 [3] https://kldload.com/zfs-wiki/myths https://kldload.com/zfs-wiki/myths

by Confiks 2mo ago

nuker 2mo ago

> combined with LUKS, TPM keys Does it work? Server can reboot and use TPM to unlock rootfs? What about /boot - encrypted and tamper proof? Resists evil maid attack? Rabbit hole, I know, but so fascinating if you solved it all :)

Gigachad 2mo ago
If you are worried about someone breaking in to your house and replacing the bootloader while leaving your drives in place I probably wouldn't use the TPM auto unlock even if in theory secure boot should be able to handle this. But in reality that will never happen and the only actual attack you need to be worried about is junkies breaking in and flogging the drives on facebook marketplace. For which, this level of security is fine.
- nuker 2mo ago
  > junkies breaking in and flogging the drives For this you dont need TPM. Just a LUKS key in rootfs /etc. He said TPM :)
toast0 2mo ago
I run my cheap hosting box with a cleartext boot setup that uses ssh to automatically grab the key for the real root from my home server (or an alternate at my MILs house). Using FreeBSD, but similar concepts. A previous hoster once gave me someone else's drives without wiping them. I don't want random customers snooping around on my data if a similar mistake happens with my disks. For home use, I run without disk encryption. If I ever need to do data recovery, it's not going to be possible with encrypted disks and one point of a centralized NAS is to have stable long term storage.
NekkoDroid 2mo ago
> What about /boot - encrypted and tamper proof? Resists evil maid attack? > Rabbit hole, I know, but so fascinating if you solved it all :) Encrypting `/boot/` is the wrong thing to do. One can just replace it with something that logs the encryption key instead (and sends it off to somewhere). You actually need to verify that what you expect to be there is actually there and a encryption isn't gonna do that for you. A reasonably secure boot chain looks roughly as follows: bootloader is checked by the UEFI (read secure-boot), bootloader chain loads a signed UKI which also uses secure boot verification (or just skip the bootloader if you only have a single kernel with no fallback), this then loads a signed dm-verity `/usr/` which brings you to the login screen. Root is just data (e.g. mounted `noexec`) and is encrypted using e.g. PCR 11 so that it only unlocks if the UKI is the one that is expected (with some other backup unlock method). This is basically #2 of the "Design Goals" of Poetterings "Fitting everything together"[0]. It's a good read if you are looking to designing your own Linux image and are looking for some inspiration. There is ParticleOS[1] which somewhat dogfoods this kind of system, though I don't think it is anything remotely considered for a production system. [0]: https://0pointer.net/blog/fitting-everything-together.html https://0pointer.net/blog/fitting-everything-together.html [1]: https://github.com/systemd/particleos https://github.com/systemd/particleos

jsr0 2mo ago

Have you looked at https://zfs.rent/ https://zfs.rent/ for cloud storage? You have to provide your own drive, but the monthly cost is relatively low.

devy 2mo ago

Yep. The 4 Samsung SSD 990 PRO 4TB NVMe SSDs are $1100 apiece. Yikes! https://www.samsung.com/us/memory-storage/nvme-ssd/990-pro-pcie-4-0-nvme-ssd-4tb-sku-mz-v9p4t0b-am/ https://www.samsung.com/us/memory-storage/nvme-ssd/990-pro-p...

tomasff 2mo ago

> So now I have a large script which debootstraps a Debian based NAS into being. Is part of this publicly available? After seeing, - https://words.filippo.io/frood/ https://words.filippo.io/frood/ - https://0pointer.net/blog/fitting-everything-together.html https://0pointer.net/blog/fitting-everything-together.html I've been looking at setting something up with mkosi/systemd for my home server/NAS - it's interesting seeing how different setups are solving the problem for inspiration.

watermelon0 2mo ago

I've been thinking about building my own NAS as well. Mind sharing how much did you pay for those hard drives, and what motherboard did you choose?

Confiks 2mo ago
It was €329 per hard drive (through a reputable store), and I chose the ASRock B650M-HDV/M.2 mATX motherboard combined with a Ryzen 5 8500G. Stock CPU cooler, replaced the Jonsbo case fans with Arctic P12 Pro PST LN. I only slightly regret the PSU (MSI MAG A650GL), which could be quieter. Not that it's very noisy, and it's a great PSU otherwise, but I should've chosen one that just shuts the fan down at low power usage.
conradev 2mo ago
https://diskprices.com/ https://diskprices.com/ is great for this You have SATA or SAS to pick from. The CPU requirements for a storage server are not high. On a typical ATX board you have motherboard SATA and can put SAS controllers in the spare PCIe slots. My first "NAS" was two 22TB hard drives in a ZFS pool on my motherboard SATA
- dapperdrake 2mo ago
  ATA (SATA), SCSI (SAS), and NVM (NVMe).
  - Brian_K_White 2mo ago
    no. just sata & sas. There is no nvme spinning drive and flash loses bits just from sitting, it's not archival. flash is good for working not for storing.
    - Dylan16807 2mo ago
      A NAS is plugged in. Flash works fine for it.
    - QuiEgo 2mo ago

realityfactchex 2mo ago

> Not a great time price-wise to be building a NAS That under-states the matter. It is a terrible time, price-wise, to build a NAS. I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times. (If there were a magical choice between having AI and significantly more expensive storage, and having no AI and some program to dump that investment money into getting and somehow leveraging significantly more available storage, that is.)

randusername 2mo ago
I'm doing it anyway and I've found some secondhand deals. Maybe COVID homelabbers are offloading gear? I built a 24 TB HDD NAS in a 36 bay chassis for around $1000 all-in: mb, ram, rails, chassis, rack, disks, hba, nic Awful Watts/TB but the plan is to run it and the GPU rig on solar.
- trollbridge 2mo ago
  I obtained a used Dell PowerEdge and just stuffed a bunch of drives in there. It's old enough that the RAM is not overly expensive (DDR3), so it has 192GB of RAM to effectively cache the data, and then 32TB of raw storage. (Of course I regret not spending the extra money to take it to 384GB now.) The nice thing about the array being big is that I can just RAID 1 it instead of worrying about tinkering with RAID 5, and leave a drive as a hot spare. PCIe 2.5G NIC for the uplink, and then it can serve over SMB or iSCSI. The main use case for this thing, incidentally, is it just is a caching proxy that holds Docker images, models off of Huggingface, and so on.
- bix6 2mo ago
  36 bay? Why so many?
  - randusername 2mo ago
    The 36 bay was all I could find on fb marketplace. I fell in love with the silly idea of squeezing 10G NIC + 10G writes (to a 2x12 RAID 10 HDD array) on the thunderbolt bus of my 12 year old macbook pro. Then I got distracted with the server mb and truenas and ZFS.

Hamuko 2mo ago

I'm also planning a new home server build now and the prices are definitely relatively ass. So far I've spent 820€ on two 22 TB WD Elements HDDs, 375€ on 2x16 GB DDR5 kit, and 520€ on two 2 TB M.2 SSDs (cache). About 1700€ and I still have no server to show for it. Doesn't help that I've been procrastinating on picking the CPU and motherboard.

Fr0styMatt88 2mo ago

I’m running an 8-drive ZFS RAIDZ2 pool. I’m wondering if you know — are the free space recommendations around ZFS cargo or real? Like I’m already giving up two full drives for redundancy (which saved my ass - I recently had two drives fail on me in quick succession — both SSDs from what looks like an identical batch) but then the advice is kinda saying I need to keep at least another drive worth of space free for the pool to perform well and not crap itself. That hurts with current prices for sure.

vladvasiliu 2mo ago
I don't think they are. I've already filled a ZFS pool almost to the brim with no adverse effects.
- barrkel 2mo ago
  My experience has been escalating fragmentation past 90%, along with lower performance owing to the fragmentation.
cm2187 2mo ago
For home usage, if you have backups raidz1 is fine (just do an incremental backup at the first sign of trouble). If you don’t have backups, then you probably shouldn’t be running a NAS in the first place.
- layer8 2mo ago
  This isn’t black and white. You might have huge amounts of non-essential data that aren’t worth the cost of off-site backups, but worth the cost of an extra disk of redundancy to lessen the risk. Even when you do have backups, it will reduce the risk of extended downtime (and possibly egress costs) caused by having to restore large amounts of data from backups.
- bluGill 2mo ago
  For most people snapshots are enough backup. It is still useful to have an offsite backup, but realistically fire is rare enough that you can risk it, and that is about the only risk most people have (you can have your NAS in a location likely far from where a fire might break out to reduce the risk farther).

neilpanchal 2mo ago

> 4x the 14 TB WD Elements Desktop. I recommend staying away from SATA drives (huge consumer rush) and look for SAS drives on eBay, particularly HC520 Helium drives from HGST/WD. Need a SAS3008 PCIe adapter ($20) and a SFF-8643 splitter ($30). No backplane is required. Huge lots of decommissioned drives frequently appear. I bought Qty 10* of HC520 (12TB) SAS drives for $1000 about 3 weeks ago, avg age is about 2.5 years, still well within its rated lifetime. Yea may be some of the stuff is fear mongering and cargo culting. I was told ECC is necessary for ZFS. When the article was written, it was cheap af (2024) to buy ECC RAM so not much consideration was given to it. -- (*) I have no idea what to do with it. Anyone has any good idea for using 120TB space? I have about 40TB unused bandwidth in the datacenter, may be host a Debian mirror? Donate storage/bandwidth to Internet Archive? Please contact me, appreciate it.

Confiks 2mo ago
> I bought Qty 10* of HC520 (12TB) SAS drives for $1000 about 3 weeks ago, avg age is about 2.5 years, still well within its rated lifetime. That's cheap indeed. Enough headroom for some failing disks too. How is the noise and power usage? I didn't look at SAS drives at all, because my impression was that they're very noisy. I can place my NAS in a closed off room, but it's not too far away and I was afraid SAS drives would be audible through the wall. At the same time, the shucked drive I'm using presents itself as an WD Ultrastar, which comes very close to a SAS drive, and isn't very noisy.
- neilpanchal 2mo ago
  > How is the noise and power usage? I think they're same as SATA drives, just different interface. AFAIK they have the same physical dimensions and same internals.
  - bobbob1921 2mo ago
    This reply is exactly right about using enterprise drives (hdd), and ideally SAS. Note the power usage of SAS vs sata for an identical drive make/model is very small sometimes 0.5 W to 1.5 W at idle (idle power draw is what matters for spinning HDD’s as underload it doesn’t increase much and idle is what you’ll be at 98% of the time). Also enterprise SSDs are much better than consumer SSD‘s for ZFS, and be aware that when you get above the 4 TB size SSD drives frequently use more power than HDD drives (i’m mainly referring to enterprise non-consumer level drives as that’s all I run for 100s of disks across many zfs systems over past 15 yrs). Also something people forget for home nas or figuring out cost for a nas, the power draw of the entire system times your electricity price you then have to multiply this times 2x to 4x times if your climate requires cooling air conditioning of any kind

ylyn 2mo ago

You assert that ECC RAM being necessary for ZFS is just a myth but provide no justification for why that is untrue. Is it not the case that if you don't have ECC memory, ZFS could end up writing a checksum that does not match the data if you get a bitflip in just the right (wrong) spot?

vardalab 2mo ago
Had it happen to my PBS backups running on ZFS without ECC just the other day. Turned out to be heat related, memory was fine but during long backups heat was causing bit flips.
naturalmovement 2mo ago
> You assert that ECC RAM being necessary for ZFS is just a myth but provide no justification for why that is untrue. ZFS without ECC is no more risky than any other file system / software RAID without ECC. As no one owes you an explanation, it would take you five seconds to Google this and discover: 1. It's been disproven, with one of the original ZFS developers chiming in. 2. The original source of the rumor was a forum post that somehow became canon.
- wpm 2mo ago
  What an incredibly rude comment
- cormorant 2mo ago
  > five seconds to Google this and discover ...(after much longer) that it's a rabbit hole with nuance. https://news.ycombinator.com/item?id=18481910 https://news.ycombinator.com/item?id=18481910
Confiks 2mo ago
Yes, indeed. ECC RAM is better than non-ECC RAM, also for ZFS. The myth, popularized by a notorious thread on the TrueNAS forums [1], is specifically that ZFS requires ECC RAM, and will do worse than other filesystems without it, because scrubbing will multiply a single bitflip into a failed pool. A ZFS core developer says that that isn't the case [2]. Here's some more reasoning [3], also about many other myths. [1] https://www.truenas.com/community/threads/ecc-vs-non-ecc-ram-and-zfs.15449/ https://www.truenas.com/community/threads/ecc-vs-non-ecc-ram... [2] https://news.ycombinator.com/item?id=18480016 https://news.ycombinator.com/item?id=18480016 [3] https://kldload.com/zfs-wiki/myths https://kldload.com/zfs-wiki/myths