8 ms·
On that note, what are the viable alternatives? All I'm really aware of are Google Proprietary Magic™ (motherfuckers. I want colossus baaaaad), CFS from DataSt
by heretohelp 14y ago
On that note, what are the viable alternatives?
All I'm really aware of are Google Proprietary Magic™ (motherfuckers. I want colossus baaaaad), CFS from DataStax, Storm (sorta-not-really?), and Spark.
- chad_walters 14y agoThere is also S4: http://incubator.apache.org/s4/ http://incubator.apache.org/s4/
- batgaijin 14y agoWTF that project is awesome and doesn't even have a page on Wikipedia?!?!? I don't get it... what's the point of creating awesome software if you don't even make the effort to put the links out to help people find it?
- seanmcdirmid 14y agoYou should write a page for it. You aren't supposed to create pages for your own projects/products on wikipedia; they should come from neutral parties.
- bunderbunder 14y agoSector/Sphere is what the Sloan Digital Sky Survey uses. Instead of supplying map and reduce routines, you implement generic "user defined functions". This gives you some more flexibility about how the work is handled, though if you want to just implement map and reduce UDFs, it supposedly gets better performance than Hadoop. It's also designed to support distributing work over WANs. I think Hadoop really wants every compute node to be on the same LAN.
- heretohelp 14y ago>I think Hadoop really wants every compute node to be on the same LAN. Fucking a-right it does. You should see the labyrinthine depths people descend to in order to scale Hadoop. Sub-clusters of sub-clusters, rack-local clusters, Zookeeper nodes all over the place. It's like fuckin' 'Nam all over again man.