5 ms·

I'm sure you would welcome a data center in your backyard

by 1234letshaveatw 1mo ago

wizzwizz4 1mo ago
Datacentres are a resource sink that serves private interests, not a resource source that serves public interests.
- 1234letshaveatw 1mo ago
  does this site not serve the public interest? I hope you didn't hurt anyone's feelings
  - wizzwizz4 1mo ago
    This site only recently (a few years ago) got multithreading. I'm pretty sure it could run off a second-hand laptop. It doesn't require a datacentre: just the two(?) moderators' salaries, and some rounding-error electricity and ISP costs.
    - 1234letshaveatw 1mo ago
      I'm fond of Netflix?
      - wizzwizz4 1mo ago
        Wouldn't need datacentres if we had IP multicast routing. And actually, Netflix doesn't need that anyway: anycast and telco CDNs get most of the advantages, albeit at the cost of only being available to established players. Netflix uses AWS because it's cheaper and easier for them, not because it's technically necessary: if AWS shut down (with enough notice), Netflix would migrate to an alternative system, and nothing much would change. I wouldn't be surprised if the majority of Netflix's true overhead were actually DRM and account management, rather than the international distribution of perhaps a petabyte of multimedia data, most of which doesn't change. (https://news.ycombinator.com/item?id=33456782 https://news.ycombinator.com/item?id=33456782 estimates half that; https://redlib.catsarch.com/r/NoStupidQuestions/comments/6mfzuj/how_much_data_is_contained_in_the_netflix_library/ https://redlib.catsarch.com/r/NoStupidQuestions/comments/6mf... estimates a tenth of that.)
        Anon1096 1mo ago
        You have no clue what you're talking about while trying to act smug and talking down to someone who knows at least a little bit more than you. Netflix cannot run without massive DCs, whether that be theirs or Amazon's.
        wizzwizz4 1mo ago
        I'm not trying to act smug. Open Connect (Netflix's telco CDN system) doesn't seem like it'd take "massive DCs" to maintain: per [DG], each Open Connect Appliance (OCA) draws between 350W and 750W (depending on model) at peak use, plus cooling to keep it below 26°C (with peaks up to 40°C permitted). By its nature, the OCAs should be geographically distributed, not centralised. With the Open Connect network handling the most popular shows (i.e., the peak load), you'd only need to serve the long tail of the back-catalogue from elsewhere. I don't know the true numbers, because I'm lazy, but [DG] says that storage OCAs can take 6.5 TB of traffic for a fill window, so we can take this as a lower bound for the storage. I'll use my earlier 1 PB overestimate for the total catalogue. Assuming that every OCA has the same data on it, this is (at least) 0.65% of the catalogue. Assuming a Zipf's law distribution of popularity for 20k titles, this means the Open Connect network can serve the top 130 titles: >>> range1 = lambda i: range(1, i+1) >>> sum(1/i for i in range1(130)) / sum(1/i for i in range1(20000)) 0.5198676299332914 which is over half of all Netflix traffic. In reality, this number is surely higher: different parts of the Open Connect network hold different titles depending on regional popularity, storage isn't 100% redundant, and so on: I wouldn't be surprised if it were closer to 70% or 80%. Modify Netflix so you have to request an item from the back-catalogue a day in advance, or something, and the cost to serve the long tail of Netflix goes waaaay down. (Considering that Netflix used to work by sending you physical DVDs, which have more than a day's delay to them, I don't think this is unreasonable.) If you want to get costs down even further, combine this with some custom UDP-ish torrent protocol, and make it peer-to-peer among users (not just the Open Connect network). But if you go too far in this direction, people will start asking why they're even paying Netflix in the first place. Netflix has to remain an infrastructure provider, not just a rentier, if they want to preserve their business model. Or, pie-in-the-sky, if we implemented a worldwide pubsub multicast system, then the top 200 90-minute titles multicasted at 30-second intervals would cost Netflix the bandwidth of just 36000 simultaneous streams, and they could run all of Netflix off the hardware of 200 OCAs… which would fit in one (1) datacentre. Or I could assemble a custom rig that would fit in… ten racks, ballpark estimate, given generous application of duct tape: which probably means I've way underestimated the capacity of each OCA. (Serving unpopular titles on demand would be cheaper than sending out multicast broadcasts that nobody's subscribed to: it would actually be even cheaper than this estimate.) It's such a shame that the only internet-routeable protocols are TCP, unicast UDP, and ICMP. [DG]: https://openconnect.netflix.com/deploymentguide.pdf https://openconnect.netflix.com/deploymentguide.pdf
scott_w 1mo ago
Wind turbines are not data centres. The sea is not part of Donald Trump’s property.
- 1234letshaveatw 1mo ago
  your neighbors property is not yours either, and hmm, I wonder if datacenters and wind turbines have anything in common?
  - justsid 1mo ago
    Let’s go with what this is actually about, windmills. I lived next door to a farm with windmills on it and I actually found them mesmerizing and nice to look at. Also a fun talking point with people visiting, but if we all had them in our backyard I suppose that novelty would wear off. So tl;dr: I would absolutely want to have windmills as my neighbours again. Datacenter not so much, but they are also substantially bigger, though not taller, than windmills.
    - 1234letshaveatw 1mo ago
      they are both noisy
  - scott_w 1mo ago
    Speak plainly. You’re making no sense whatsoever.
    - 1234letshaveatw 1mo ago
      I'm sorry you were unable to grasp the point, which was that both are loud, and both can can have a impact even if located on an adjacent property. I hope that clears it up for you, but, as an aside, I suggest that instead of trying to make demands of people that you simply ask for clarification if you are have difficulty understanding something
      - 1mo ago

I'm sure you would welcome a data center in your backyard

wizzwizz4 1mo ago

Datacentres are a resource sink that serves private interests, not a resource source that serves public interests.

1234letshaveatw 1mo ago
does this site not serve the public interest? I hope you didn't hurt anyone's feelings
- wizzwizz4 1mo ago
  This site only recently (a few years ago) got multithreading. I'm pretty sure it could run off a second-hand laptop. It doesn't require a datacentre: just the two(?) moderators' salaries, and some rounding-error electricity and ISP costs.
  - 1234letshaveatw 1mo ago
    I'm fond of Netflix?
    - wizzwizz4 1mo ago
      Wouldn't need datacentres if we had IP multicast routing. And actually, Netflix doesn't need that anyway: anycast and telco CDNs get most of the advantages, albeit at the cost of only being available to established players. Netflix uses AWS because it's cheaper and easier for them, not because it's technically necessary: if AWS shut down (with enough notice), Netflix would migrate to an alternative system, and nothing much would change. I wouldn't be surprised if the majority of Netflix's true overhead were actually DRM and account management, rather than the international distribution of perhaps a petabyte of multimedia data, most of which doesn't change. (https://news.ycombinator.com/item?id=33456782 https://news.ycombinator.com/item?id=33456782 estimates half that; https://redlib.catsarch.com/r/NoStupidQuestions/comments/6mfzuj/how_much_data_is_contained_in_the_netflix_library/ https://redlib.catsarch.com/r/NoStupidQuestions/comments/6mf... estimates a tenth of that.)
      - Anon1096 1mo ago
        You have no clue what you're talking about while trying to act smug and talking down to someone who knows at least a little bit more than you. Netflix cannot run without massive DCs, whether that be theirs or Amazon's.
        wizzwizz4 1mo ago
        I'm not trying to act smug. Open Connect (Netflix's telco CDN system) doesn't seem like it'd take "massive DCs" to maintain: per [DG], each Open Connect Appliance (OCA) draws between 350W and 750W (depending on model) at peak use, plus cooling to keep it below 26°C (with peaks up to 40°C permitted). By its nature, the OCAs should be geographically distributed, not centralised. With the Open Connect network handling the most popular shows (i.e., the peak load), you'd only need to serve the long tail of the back-catalogue from elsewhere. I don't know the true numbers, because I'm lazy, but [DG] says that storage OCAs can take 6.5 TB of traffic for a fill window, so we can take this as a lower bound for the storage. I'll use my earlier 1 PB overestimate for the total catalogue. Assuming that every OCA has the same data on it, this is (at least) 0.65% of the catalogue. Assuming a Zipf's law distribution of popularity for 20k titles, this means the Open Connect network can serve the top 130 titles: >>> range1 = lambda i: range(1, i+1) >>> sum(1/i for i in range1(130)) / sum(1/i for i in range1(20000)) 0.5198676299332914 which is over half of all Netflix traffic. In reality, this number is surely higher: different parts of the Open Connect network hold different titles depending on regional popularity, storage isn't 100% redundant, and so on: I wouldn't be surprised if it were closer to 70% or 80%. Modify Netflix so you have to request an item from the back-catalogue a day in advance, or something, and the cost to serve the long tail of Netflix goes waaaay down. (Considering that Netflix used to work by sending you physical DVDs, which have more than a day's delay to them, I don't think this is unreasonable.) If you want to get costs down even further, combine this with some custom UDP-ish torrent protocol, and make it peer-to-peer among users (not just the Open Connect network). But if you go too far in this direction, people will start asking why they're even paying Netflix in the first place. Netflix has to remain an infrastructure provider, not just a rentier, if they want to preserve their business model. Or, pie-in-the-sky, if we implemented a worldwide pubsub multicast system, then the top 200 90-minute titles multicasted at 30-second intervals would cost Netflix the bandwidth of just 36000 simultaneous streams, and they could run all of Netflix off the hardware of 200 OCAs… which would fit in one (1) datacentre. Or I could assemble a custom rig that would fit in… ten racks, ballpark estimate, given generous application of duct tape: which probably means I've way underestimated the capacity of each OCA. (Serving unpopular titles on demand would be cheaper than sending out multicast broadcasts that nobody's subscribed to: it would actually be even cheaper than this estimate.) It's such a shame that the only internet-routeable protocols are TCP, unicast UDP, and ICMP. [DG]: https://openconnect.netflix.com/deploymentguide.pdf https://openconnect.netflix.com/deploymentguide.pdf

scott_w 1mo ago

Wind turbines are not data centres. The sea is not part of Donald Trump’s property.

1234letshaveatw 1mo ago
your neighbors property is not yours either, and hmm, I wonder if datacenters and wind turbines have anything in common?
- justsid 1mo ago
  Let’s go with what this is actually about, windmills. I lived next door to a farm with windmills on it and I actually found them mesmerizing and nice to look at. Also a fun talking point with people visiting, but if we all had them in our backyard I suppose that novelty would wear off. So tl;dr: I would absolutely want to have windmills as my neighbours again. Datacenter not so much, but they are also substantially bigger, though not taller, than windmills.
  - 1234letshaveatw 1mo ago
    they are both noisy
- scott_w 1mo ago
  Speak plainly. You’re making no sense whatsoever.
  - 1234letshaveatw 1mo ago
    I'm sorry you were unable to grasp the point, which was that both are loud, and both can can have a impact even if located on an adjacent property. I hope that clears it up for you, but, as an aside, I suggest that instead of trying to make demands of people that you simply ask for clarification if you are have difficulty understanding something
    - 1mo ago