7 ms·
Nice to see this idea written about in detail. I had thought about it in the context of terrible availability bargain bucket storage (iDrive E2), where the cost
by dmw_ng 2y ago
Nice to see this idea written about in detail. I had thought about it in the context of terrible availability bargain bucket storage (iDrive E2), where the cost of (3,2) erasure coding an object and distributing each segment to one of 3 regions would still be dramatically lower than paying for more expensive and more reliable storage.
Say 1 chunk lives in Germany, Ireland and the US each. Client races GETs to all 3 regions and cancels the request to the slowest to respond (which may also be down). Final client latency is equivalent to that of the 2nd slowest region, with substantially better availability due to the ability to tolerate any single region being down
Still wouldn't recommend using E2 for anything important, but ^ was one potential approach to dealing with its terribleness. It still doesn't address the reality of when E2 regions go down, it is often for days and reportedly sometimes weeks at a time. So reliable writing in this scenario would necessitate some kind of queue with capacity for weeks of storage
There are variants of this scheme where you could potentially balance the horrible reliability storage with some expensive reliable storage as part of the same system, but I never got that far in thinking about how it would work
- ddorian43 2y ago> Client races GETs to all 3 regions and cancels the request to the slowest to respond Dropbox does this internally in magic pocket (see their eng blog)
- derefr 2y agoHaving not heard of E2 before (it never seems to come up in the comparisons the other object-storage providers do to make themselves look good) — do you know why it has such bad availability? "Weeks of downtime" sounds crazy for a business focused on storage. If they aren't also failing at durability, then it wouldn't be any of the classical problems associated with running a storage cluster. Do they just... not bother with online maintenance / upgrades / hardware transitions?
- dmw_ng 2y agoMy best guess is their primary product is a Windows (I think) backup client that has a place for smarts allowing them to paper over problems, or something along those lines. Feels like an "oh Germany is down again, when is Frank back from holiday so he can catch a plane to replace the switch?" type affair. If you google around the Reddit data hoarder communities you'll find plenty of war stories about E2