9 ms·
Backblaze has ordered 100 petabytes of hard drives
- bacoboy 9y agoI guess all the crashplan personal users are jumping over there en-masse. Good for business...
- kyriakos 9y agosomething happened to crashplan?
- cosmie 9y agoThey're shutting down their consumer service to focus on the B2B market. https://blog.eogn.com/2017/08/22/crashplan-is-shutting-down-its-cloud-backup-service-for-consumers/ https://blog.eogn.com/2017/08/22/crashplan-is-shutting-down-...
- atYevP 9y agoYev from Backblaze here -> An influx of Computer Backup customers has certainly played a factor, but also Backblaze B2 is growing quickly!
- no1youknowz 9y ago[Resolved]
- atYevP 9y agoI'll respond! Ping me at yevgeniy@backblaze.com :D
- corobo 9y agoI have a hunch the page you're after is https://www.backblaze.com/company/partners.html https://www.backblaze.com/company/partners.html
- no1youknowz 9y agoContacting via HN was the last option. I tried there and sales on numerous occasions.
- deleted 9y ago[deleted]
- xhrpost 9y agoEven before that they were seeing 25PB in-bound per month, so at the previous level this only a 4 month supply.
- buro9 9y ago@atYevP how much does the Backblaze business plan cost? Or rather, if I have 10TB on a NAS and a Linux machine, what would it cost to back that up to Backblaze as a consumer given that you don't support this and I'd need to purchase the business thing?
- votepaunchy 9y ago$50/month at $0.005/GB/month. https://www.backblaze.com/b2/cloud-storage-pricing.html https://www.backblaze.com/b2/cloud-storage-pricing.html
- buro9 9y agoAha! I thought B2 was like S3 and figured that Business Backup was what I needed. TIL.
- brianwski 9y agoBrian from Backblaze here. > I thought B2 was like S3 It is. Backblaze actually has two separate product lines: 1) Online Backup for $5/month/computer where you have to use our client we wrote with all of it's policies. This is widely targeted at "ease of use" type customers. There is a free feature inside of "Online Backup" called "Groups" where one person can pay for many others. You can use this for a business, we absolutely do NOT charge differently just because it is a business. Or you can use "Groups" to pay for a family, or whatever. "Groups" is free, we honestly don't care if you use it or not. 2) Backblaze B2 - this is "object storage for half a penny per GByte per month". There are MANY clients available for you to choose from, you can see a list on this page: https://www.backblaze.com/b2/integrations.html https://www.backblaze.com/b2/integrations.html The idea here was many people kept approaching Backblaze saying they needed to store stuff cheaply, but didn't want to use our client, or maybe it wasn't "backup" they needed but to host live images on a website on the internet, stuff like that. So we built an API anybody can program to. https://www.backblaze.com/b2/docs/ https://www.backblaze.com/b2/docs/
- atYevP 9y agoHey there! The business and consumer computer backup service have the same feature-set, so our business service would also not work with a NAS or Linux box. I'd recommend looking in to Backblaze B2. It'll be a bit more expensive as it's $0.005/GB but you'll have more control over your backups and lots of integrators make it easy.
- chr4004 9y agoCurious where one would buy those, directly from the manufacturer?
- atYevP 9y agoYev from Backblaze here -> Most of our purchasing is done through distributors. We'd love to buy direct at some point, but for now we have good manufacturer/distributor relationships! We're getting there though!
- linsomniac 9y agoDo you mean the storage pod? You want to look at https://www.backblaze.com/blog/open-source-data-storage-server/ https://www.backblaze.com/blog/open-source-data-storage-serv... and https://www.backuppods.com/ https://www.backuppods.com/
- tooltalk 9y agowonder if Backblaze is buying directly from HDD manufacturers or wholesalers and how much volume discount?
- chiph 9y agoThey've talked about this before -- despite orders like this, they're still not a big enough customer to be on the manufacturer's radar. So they're ordering from a middle-man. Back during the hard-drive crisis after the tidal wave that disrupted production ... they were buying from anyone that had drives to sell, including people going into their local big-box store to source drives.
- gist 9y agoYes I think the 'shucking' blog post was the one that caught my attention first.
- dtech 9y agoI wonder which customers are big enough if this is the case. I can only think of PC manifacturers (Dell et al., but they are switching to SSD's) and companies with a truly massive amount of datacenters (Google, Microsoft, Amazon). Ipods and similar electronics have switched from HDD's for a while now.
- agumonkey 9y agoSo only Dell, HP, Apple and the likes have a direct line ? 10K units of the largest models feels huge to my layman self.
- ksec 9y agoStill? 100PB of HDD is still not big Enough? So the question would be, In unit of Backblaze, what are the size for Amazon, Apple, Google, Facebook?
- ac29 9y agoCurious on that too. I wouldn't be surprised if Amazon needs 10,000 drives a day (many of which are probably going to replace dead drives, or upgrade lower capacity ones). Google is probably about the same.
- thebiglebrewski 9y agoLove these articles! Thanks for writing it. Must feel so good to buy that many HDDs at once...something really gratifying about that.
- brianwski 9y agoAs the guy who signs the check, "gratifying" isn't quite the right word. :-) Some stress, some amazement that everything got this large. When we started 10 years ago, we could only afford to rent a quarter of one cabinet at the datacenter at 365 Main Street, San Francisco. We had "right of first refusal" on the rest of the cabinet we occupied. We could pile all of the hard drives ever deployed in that cabinet on my kitchen table. Now it is semi trucks filled with fork lift pallets of drives. But it just "snuck up on us". Each day it was a little larger, each drive order was just slightly larger than the previous.
- atYevP 9y agoYes, something taking 10 years to sneak up on you is very sneaky.
- amelius 9y agoDo you keep them powered on at all times? Does a storage pod include a software-switch to turn them off selectively?
- atYevP 9y agoYev from Backblaze here -> Yes, unless the storage pod is in maintenance, it remains on so that backup and B2 data can be uploaded and accessed.
- amelius 9y agoInteresting. Have you ever computed how much power you would save if you turned them off when not needed? Or is every pod constantly in use? Would it help if smart algorithms repartition the data on storage pods, to optimize power use?
- NikolaeVarius 9y agoWhy wouldn't that be handled already by the drives themselves?
- amelius 9y agoPerhaps partly. But a drive can't repartition data according to frequency of use.
- x4d66a5ce 9y agoI wonder about this, too. Since access latency is the primary concern, a neat solution would be to create some dynamic "fastup" storage scheme. For large files, loosen the always-on requirement; to fulfill a request the first N bytes are served from the always-on pod, and the tail is stored and served from pods that dynamically power up drives only at access time. The size of N is computed such that the IO time it takes to make it through the fragment stored on the always-on pod is > the power-on time for the pods where the tail is stored.
- ericfrederich 9y agoI love the articles from Backblaze. I enjoy them publishing the failure rates of drives and anything else on the blog. Two things missing from them in my opinion 1) a Linux client and 2) a second location I guess I could get away without a Linux client if there was an unRAID plugin for B2.
- mshook 9y agoRegarding 1): rclone does the job. That's what I use with B2. https://rclone.org/ https://rclone.org/
- giobox 9y agoB2 plus rclone works well, but for many it's potentially significantly more expensive than the standard "all you can eat" $5 a month plan. In my case B2 would workout at roughly 4x the monthly cost and that's before factoring in the fact B2 charges for downloads as well as uploads. An official Linux client would be significantly better value.
- 086421357909764 9y agoAs they've said though, the usage of most linux users isn't on par with Windows / Mac users. They simply would crush themselves with people attempting to backup 200TB nas devices for $5.00 a month. If you want any perspective on how crazy the fringes are with data storage and how they impact providers (such as amazon, google, backblaze, etc.) simply read the stories on reddit /r/datahoarder and you'll quickly see why this decision makes sense from a business perspective.
- giobox 9y agoVery small numbers of people already do this on Mac/Windows using tricks like mouting the NAS as an iSCSI device to backup absurd amounts on the $5 plan. The business model is clearly that the legions of small users subsidize the "whales" and Linux desktops are a drop in the ocean vs the Windows/Mac world too. Do you have a source for where Backblaze claimed the usage of most Linux users isn't on par?
- ape4 9y agoHow about security? If Backblaze can recover files for users then they can get the same files for any TLA.
- grincho 9y agoThat's the advantage of B2: straight storage, bring your own (encrypting) client. If Backblaze proper had a decent restore story (rather than typing your key into a web page and getting a zip), I'd recommend it wholeheartedly.
- sp332 9y agohttps://help.backblaze.com/hc/en-us/articles/217664798-Security-Question-Round-up- https://help.backblaze.com/hc/en-us/articles/217664798-Secur...
- xhrpost 9y agoI imagine you'll have to reduce pricing on Backblaze B2 eventually in order to stay competitive. What's it like having to flip such a massive switch where your revenue is X one minute and then X - 20% the next?
- teach 9y agoI don't understand. B2 is currently about half the cost of S3 for outbound data and 1/4 the cost for data at rest. What do you mean "stay competitive?"
- plantain 9y agoThey're not equivalent. B2 is singly homed.
- liquidgecka 9y agoSo is S3. You have to pick the region (home) for the data when you create the bucket.
- gamegoblin 9y agoFrom https://aws.amazon.com/s3/ https://aws.amazon.com/s3/ and https://aws.amazon.com/s3/details/ https://aws.amazon.com/s3/details/ Data is automatically distributed across a minimum of three physical facilities that are geographically separated by at least 10 kilometers within an AWS Region Designed for 99.999999999% durability and 99.99% availability of objects over a given year. Designed to sustain the concurrent loss of data in two facilities. From https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html Uptime: 99.9% SLA Reliability: 99.999999% durability So S3 is 10x more available, 1000x more durable, and datacenter redundant.
- shawn-furyan 9y agoEleven 9s of durability is an utterly farcical guarantee from a decade old service. A single big durability event in the next century would irrevocably decimate that record. They really might as well claim 100% durability, because it is no less plausible. I mean, what are the odds that S3 will even be operating in 100 years? I'll certainly put $1 against $1,000,000,000 that it won't, and that's only one many ways it could fail the standard.
- linsomniac 9y agoI've watched Backblaze closely, and I love their detailed reports of drive stats. I've always been curious about their choices of manufacturers though. In my past I found that Hitachi/HGST drives were far more reliable than others. We had a couple people doing hardware and software support on ~120 machines, and hard drive replacements were expensive (in man-hours) for us, so whenever possible we got Hitachi drives. The Backblaze numbers seem to support this as well, far fewer failures of HGST drives vs Seagate, but they seem to buy mostly Seagate. I guess when you have tens of thousands of drives, you're always going to need to be replacing them. So maybe a 10x reduction in failures isn't worth it? Or maybe they just couldn't get HGST drives in quantity?
- cyberpunk0 9y agoI've never had a Seagate drive last any reasonable amount of time
- dspillett 9y agoThey don't buy direct from manufacturers so it might be partly down to availability numbers from their local distributors. Also you want to buy from a wide range - if most of the drives come from one place and they have a manufacturing disaster that causes everything made in one month to fail a few months into use then that might take you too close to redundancy requirements for comfort. I would expect the range picked will be based on individual design (platter design, platter layout, controller type, ...) rather than per manufacturer - maybe HGST has fewer designs available at any one time and that tips the balance a bit.
- atYevP 9y agoYev from Backblaze here -> We have relationships with manufacturers but don't directly buy from them (yet - fingers-crossed). It really is a combination of price/reliability. At our scale we can afford slightly higher failure rates if it means less expensive drives, so our #1 data point for purchases is the cost per gigabyte. After that we do look at failure rates, but unless it's something wildly out of "normal" we tend to live with the occasional failures. Thanks to Vaults the back-end is designed in such a way that if a drive fails we don't have to run and replace it - so individual failures tend not to stress us out too much anymore!
- godzillabrennus 9y agoI’ve been a customer for years and recently had a catastrophic failure of a computer and it’s direct attached backup drive. I have spent the last four days waiting for backblaze to create a restore a backup for a computer on and last I checked it was at 9%. I chatted with support and they said this is normal. That progress may jump to completed any time because the restore only tracks file count. I wish I could say I’m happy about this but I’m four days into trying to pull down my data and I still have no idea when I’ll be able to start the download. This sucks.
- 7b64f0f2 9y agoMakes me wonder how much data/files you are trying to get back from their storage.
- finnh 9y agoThen why not ask?
- monochromatic 9y agoWhy would it take any time at all to "create a restore"?
- brianwski 9y agoBrian from Backblaze here. All the data is encrypted and spread throughout our datacenters. So we fetch each file and decrypt it (with the password you provided when kicking off the restore). If this is less than 1 GByte it is instantaneous. But if it is a TByte of data, it takes a few hours to decrypt and add to the ZIP file.
- monochromatic 9y agoThat makes sense, thanks for the response. (4 days and counting still seems excessive though.)
- chrisper 9y ago@atYevP Do you guys do anything special regarding HDD vibrations? I am assuming your special case is built with this in mind. It is crazy how many things one has to consider at such scale. Things people normally do not care about!
- brianwski 9y agoNot Yev but I also work at Backblaze. Most of the anti-vibration tech is straight forward, like the fans aren't bolted to the chassis, they are held by rubber rivets, and we have slowly but steadily improved the "pressure" that drives are held in place to limit failures, etc. From time to time, a fan will "half fail" causing large vibrations which can massively increase drive failures in the rack that bad fan is in.
- chrisper 9y agoThanks. Another question I got for you is: What is louder? The fans or all the disks together? I had a workstation with 6 spinning disks and that was already quite loud.
- atYevP 9y agoNow it's Yev - you'd have to ask our DC techs. Every time I go to the datacenter it "feels" like the loudest part are the fans, but my ears aren't attuned to what is causing which ruckus :P
- pcunite 9y agoCould you let us know about failure rates over time? Thanks for sharing.
- atYevP 9y agoYev here -> These drives will be part of our ongoing "Hard Drive Stats" posts on our blog!
- philfrasty 9y agoIf anyone from the Backblaze team reads this: I love your service BUT please provide proper invoices. The ones provided in the „My Account“ page are an absolute nightmare. Look at services like Stripe or Digitalocean and make them downloadable as PDF. Best case: send them out via email or auto-upload to specific Dropbox folder.