12 ms·
In the next year I can imagine most people deploying Spark will be doing it on Hadoop, since Cloudera 5 will support Spark. It's a natural fit, most people don'
by alanctgardner2 13y ago
In the next year I can imagine most people deploying Spark will be doing it on Hadoop, since Cloudera 5 will support Spark. It's a natural fit, most people don't hate HDFS but their use case doesn't naturally fit the MR programming model.
- gclaramunt 13y agoyeah, I mostly complain about trying to fit everything to the Map/Reduce model
- TallGuyShort 13y agoThe "Hadoop ecosystem" is getting less and less about MapReduce, and more about other execution frameworks that can share HDFS with it.
- alanctgardner2 13y agoYou don't happen to work at Cloudera, do you? I noticed you have some submissions about Impala and Oracle being evil, which seems to be a pretty common view among the ex-Oracle DBAs there
- TallGuyShort 13y agoI do happen to work at Cloudera (hence the Impala submissions), although I'm neither an ex-Oracle DBA nor a huge believer that they're evil. I really don't have a lot of first-hand experience with Oracle as a company - which as you'll see, is why my submission was actually a question about the community's perception and why that's a common view.
- bcbrown 13y agoHey, I've recently begun the interview process at Cloudera, would you mind sending me an email? My address is in my profile, and I'd love to ask you a couple questions.
- enigmo 13y agoI'd like to see other distributed filesystems catch on too. HDFS has a lot of room for improvement.
- monstrado 13y agoWhich areas would you say HDFS needs improvement the most? Just so you know, HDFS is still very actively developed, and keeps introducing features / improving functionality (e.g Native NFS, In-Memory Caching, Short Circuit Reads, High Availability, Namespace Federation, etc) on a pretty regular basis. Feel free to suggest new features, or contribute to the project yourself. https://issues.apache.org/jira/browse/HDFS https://issues.apache.org/jira/browse/HDFS
- enigmo 13y agoI know of the JIRA, thanks. I'm aware that it is being actively developed, but I don't necessarily believe that activity is progress. Keeping a few different players in the mix helps keep everyone focused on progress.
- dundun 13y agoWe've been using it on HDP2 for about a month now. Everything works fairly well and it was super easy to setup because of YARN (and the work the Spark team put in).