5 ms·

Buddy, I have 24Tb HDDs in my pool today. If anything the opposite has occurred. HDD scaling has largely flattened. Going from 1986 -> 2014, HDD size increased

Buddy, I have 24Tb HDDs in my pool today. If anything the opposite has occurred. HDD scaling has largely flattened. Going from 1986 -> 2014, HDD size increased by 10x every 5.3 years [1]. If anything we should have 100Tb+ drives if scaling kept going. I say this not as a but there have been directly implications for ZFS. All this data stuck behind an interface who's speed is (realistically after a file system & kernel involved) hard limited to 200MiB/s-300MiB/s. Recovery times sky rocket. As you simply cannot re-build parity/copy data. The whole reason stuff like draid [2] were created is so larger pools can recover in less than a day by doing sequential parity & hot-spairs loaded 1/N of each drives data ahead of time. --- 1. Not the most reliable source, but it is a friday afternoon https://old.reddit.com/r/DataHoarder/comments/spoek4/hdd_capacity_by_year_by_power_of_10_some_thoughts/ https://old.reddit.com/r/DataHoarder/comments/spoek4/hdd_cap... 2. https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAID%20Howto.html https://openzfs.github.io/openzfs-docs/Basic%20Concepts/dRAI... for concept, for motivations & implementation details see -> https://www.youtube.com/watch?v=xPU3rIHyCTs https://www.youtube.com/watch?v=xPU3rIHyCTs

godelski 11mo ago
Not quite that level, but you can get 8TB nvmes. You'll pay $500 a pop though...[0]. Weirdly that's the cheapest NewEgg lists for anything above 8TB and even SSDs are more expensive. It's a gen4 PCIe M.2 but a SATA SSD is more? It's better going the next bracket down but still surprising to me that the cheapest 4TB SSD is just $20 cheaper than the cheapest NVMe[1] (a little more and you're getting recognizable names too!) It kinda sucks that things have flatlined a bit, but still cool that a lot of this has become way cheaper. I think the NVMes at these prices and sizes really makes caching a reasonable thing to do for consumer grade storage [0] https://www.newegg.com/western-digital-8tb-black/p/N82E16820250270 https://www.newegg.com/western-digital-8tb-black/p/N82E16820... [1] https://www.newegg.com/p/pl?N=100011693%20600551612&Order=1 https://www.newegg.com/p/pl?N=100011693%20600551612&Order=1
- nick__m 11mo ago
  If your budget allows it you can get 120TB .5 dwdp ssd like that drive http://www.atic.ca/index.php?page=details&psku=319207 http://www.atic.ca/index.php?page=details&psku=319207
- wtallis 11mo ago
  > Weirdly that's the cheapest NewEgg lists for anything above 8TB and even SSDs are more expensive. Please don't perpetuate the weird misconception that "SSD" refers specifically to SATA SSDs and that NVMe SSDs aren't SSDs.
- necovek 11mo ago
  In terms of production, SSD flash chips that go into SATA and NVMe drives can be pretty much the same: only the external interface can be different. The biggest cost driver for flash chips is not the speed they can be read from and written to in bursts, but how resilient they are (how many times can they be written over) and sustained speed (both based on the tech in use, TLC, SLC, MLC, 3D NAND, wear levelling logic...): even for SATA speeds, you need the very best for sustained throughput. Still, SATA SSDs make sense since they can use the full SATA bandwidth and have low latency compared to HDDs. So the (lack of) price difference is not really surprising.

godelski 11mo ago

Not quite that level, but you can get 8TB nvmes. You'll pay $500 a pop though...[0]. Weirdly that's the cheapest NewEgg lists for anything above 8TB and even SSDs are more expensive. It's a gen4 PCIe M.2 but a SATA SSD is more? It's better going the next bracket down but still surprising to me that the cheapest 4TB SSD is just $20 cheaper than the cheapest NVMe[1] (a little more and you're getting recognizable names too!) It kinda sucks that things have flatlined a bit, but still cool that a lot of this has become way cheaper. I think the NVMes at these prices and sizes really makes caching a reasonable thing to do for consumer grade storage [0] https://www.newegg.com/western-digital-8tb-black/p/N82E16820250270 https://www.newegg.com/western-digital-8tb-black/p/N82E16820... [1] https://www.newegg.com/p/pl?N=100011693%20600551612&Order=1 https://www.newegg.com/p/pl?N=100011693%20600551612&Order=1

nick__m 11mo ago
If your budget allows it you can get 120TB .5 dwdp ssd like that drive http://www.atic.ca/index.php?page=details&psku=319207 http://www.atic.ca/index.php?page=details&psku=319207
wtallis 11mo ago
> Weirdly that's the cheapest NewEgg lists for anything above 8TB and even SSDs are more expensive. Please don't perpetuate the weird misconception that "SSD" refers specifically to SATA SSDs and that NVMe SSDs aren't SSDs.
necovek 11mo ago
In terms of production, SSD flash chips that go into SATA and NVMe drives can be pretty much the same: only the external interface can be different. The biggest cost driver for flash chips is not the speed they can be read from and written to in bursts, but how resilient they are (how many times can they be written over) and sustained speed (both based on the tech in use, TLC, SLC, MLC, 3D NAND, wear levelling logic...): even for SATA speeds, you need the very best for sustained throughput. Still, SATA SSDs make sense since they can use the full SATA bandwidth and have low latency compared to HDDs. So the (lack of) price difference is not really surprising.