7 ms·

Elasticsearch Raises $70 Million

232 points by asm89 12y ago

asm89 12y ago
What I find interesting about the Elasticsearch story is the success it has given that the core product (the search server) is completely open source.
- hackerboos 12y ago
  My fear is that with money comes an 'Enterprise' edition.
  - SEJeff 12y ago
    I don't think the "open core" model is in their best interests. Look at a company like Zenoss, where their OSS version is their biggest competitor. It doesn't make sense for ES to do that. Now marvel[1] is where they've started monetizing and I'm sure they make a good bit of money from "professional services" teaching companies how to deploy ES at scale. Hopefully they pull it off, as the world needs a good competitor to splunk. ES has the backend tech, but kibana has a loooooong way before it rivals the interface for searching splunk. Here's to hoping! [1] http://www.elasticsearch.org/overview/marvel/download/ http://www.elasticsearch.org/overview/marvel/download/
    - Argorak 12y ago
      Marvel is quite interesting. It is a product everyone _could_ build on their own (basically, it is Kibana over their own metrics data), but at that price point, there is no reasonable reason to do so.
  - aembleton 12y ago
    What would be so bad about an enterprise edition? Many companies require extra support and would pay for the enterprise edition and wouldn't even consider the product without this option.
  - dchuk 12y ago
    ElasticSearch is a sufficiently complex system that can be implemented in a ton of different ways to solve a ton of different problems. I'm sure there are more than enough consulting gigs floating around for them make a good bit of money both on actual consulting and selling additional software packages.
  - Karunamon 12y ago
    I was in a class with the founders a few months ago - for whatever it's worth, they're vehemently opposed to segregating functionality like this. Their plans are to offer support, and make their bones that way. Elasticsearch is a complex enough product at scale that this would probably be quite lucrative.
  - thedevopsguy 12y ago
    Elastic Search make their money from Enterprise support licenses. There are so many ways of installing it , scaling and optimizing depending on application. Don't forget the training around. You have IBs like Goldman Sachs using it for log analytics. So the services around that are needed and real. I don't think they will need an enterprise version.
conjugate 12y ago
all the vcs who have poured money into hortonworks and elasticsearch are the greatest contributers to open-source - and they don't even know it.
pyrox420 12y ago
We love Elasticsearch. It's fast and accurate for huge amounts of data, super easy to scale, and incredibly easy to get started. That cash can only make the product better. Good for them!
ig1 12y ago
If people are curious as to why VCs are interested in open-source businesses: http://indexventures.com/news-room/blog/a-perfect-storm-now-is-the-time-for-multibillion-dollar-open-source-companies http://indexventures.com/news-room/blog/a-perfect-storm-now-...
- wslh 12y ago
  I also recommend the JBoss Business Case Study: "Lessons from Leaders: How JBoss did it" [1] [1] http://www.forentrepreneurs.com/lessons-from-leaders/jboss-example/ http://www.forentrepreneurs.com/lessons-from-leaders/jboss-e...
- michaelochurch 12y ago
  Open-source is becoming the only way to develop (and, especially, maintain) complex software. The fact that the engineers can gain employer-independent reputations gives them an incentive that's astronomically expensive (as hedge fund compensation goes) to replicate otherwise. The world is finally figuring out that it's impossible to employ top talent, but that it can be quite lucrative to sponsor it.
  - deleted 12y ago
    [deleted]
  - clubhi 12y ago
    It's not impossible to employ top talent. That is a very ignorant statement.
    - michaelochurch 12y ago
      In technical fact, not impossible but, in fact, common. In spirit, you don't really employ top talent, in the sense that subordinating it wastes it. You sponsor it and become a beneficiary of it.
    - sheepmullet 12y ago
      Top talent has access to thousands of other well paying roles. You cannot treat them the same way companies treat regular employees (badly). If you give them pointless work, or work without career growth, they can up and leave in a heartbeat. If you try and bully them or pressure them to work long hours or place pointless restrictions (dress code, start times, vacation policy, etc) on them they will leave.
gshakir 12y ago
Great product and wish them best of luck. Had great success using with Hive with their Apache Hadoop plugin for doing quick analysis.
mooreds 12y ago
I saw these folks at gluecon. Haven't looked at a ton of dashboard solutions, but I found Kibana to be pretty compelling, simply because it was trivial to get the elk stack up and running and input arbitrary data. I am not as interested in log data, more in business metrics.
jasonkolb 12y ago
I'm sitting in Elasticsearch training right now (during a caffeine break). These are some really great guys that know their stuff, and they're committed to contributing back to the OSS version everything that makes sense. They contribute a ton of code back to Lucene and employ a lot of the brightest minds in this space. On the product side, I'm sitting here being amazed at some of the problems they've solved very elegantly. Elasticsearch has a bright future.
- mewwts 12y ago
  I'm starting a new job in August where I will focus on building search-based applications with Elasticsearch. I honestly can't wait. This news makes me happy as I know I'll be learning relevant technology.
- cbeach 12y ago
  I'm also in Elasticsearch training (Victoria, London). Impressed with the trainers, both core developers with a passion for the product. Bodes well.
- latj 12y ago
  How is it better than Solr? I havent looked at Elastic Search in a long time, so I really do want to know. Not trying to pick a fight. ;)
  - diminish 12y ago
    I m using it since earlier versions for almost 3 yers, i guess. It makes config easier, offers a well designed, easy API. Especially for Rails, PHP, Django crowd, an easy choice to create a sensible text search. The main benefit was an easier Lucene with JSON/http API. Yes a lot more advanced features are there too.
  - mhitza 12y ago
    Advantageous may be the fact that you can extend ElasticSearch with JavaScript, whereas I think as easy for someone who understands the Solr codebase, I've wanted to try that once, but I had to resort to doing the necessary computation in the language calling the Solr service.
arecurrence 12y ago
Awesome! I make heavy use of elasticsearch and am very happy with the performance. It's typically the rest of the pipeline now that causes most of the latency whereas search used to be the bulk of the duration of a request. It has also spurred other entities to improve their search performance :)
jack_jennings 12y ago
Hopefully this means that the documentation gets some help, especially some of the official libraries. So far working with ES has been a mixed bag… seems good in theory but I've had a hard time getting over the learning curve.
- spencera 12y ago
  Want to highlight some of your specific pain points?
  - jack_jennings 12y ago
    I guess I'm in the minority based on the other comments here… After spending a few hours with the documentation, I felt like I had a generally good feel for how you would interact with ES through curl, but then jumping into using the Ruby library there seemed to be a big leap and I felt like I needed to have a much more intimate knowledge of how ES worked to "get" it. A lot of guess and check before I figured out how to query properly. I've also been unsuccessful in figuring out how to implement accent folding. At this point I assume that this has something to do with "mapping", but I couldn't figure out where that was supposed to be set up (again, through the Ruby lib, but I was also unsure where to start to just accomplish that via curl…). Who knows, maybe I just need to spend more time reading the docs, and maybe the information that I need is in there somewhere, but to me it felt incomplete or disorganized. I'd love to stop using Solr for document search, but making ES do things that I know how to do in Solr ended up being too time consuming. I ended up feeling like I probably had to go to a training session to really get it, which is a shame…
    - sujeetsr 12y ago
      I found the elasticsearch tutorials on youtube by Clinton Gormley pretty helpful in understanding the concepts. Also came across this book (which I haven't really read so I don't know how good it is, just posting it here) http://exploringelasticsearch.com/ http://exploringelasticsearch.com/
  - dguaraglia 12y ago
grouma 12y ago
I have been prototyping with Elasticsearch for the last couple of months. I have nothing but great things to say about the software and documentation. Several other partner teams have taken notice of my work and will likely incorporate the software as well. Very exciting!
enscr 12y ago
Could someone explain in a few lines why & how is elasticsearch a revolutionary thing (technically) ?
- craigching 12y ago
  Do you need something beyond "highly scalable search solution"? A lot of enterprises (and I'd guess startups too!) need good search solutions and your two players in the Java space (that I know of) are Solr and Elasticsearch. We are using Solr in our product, but the more I hear about Elasticsearch the more I want to give it a serious try. My reason for not using it originally was that we had prototyped our solution using Compass (which was the project before Elasticsearch) only to have it abandoned to work on Elasticsearch. So I was concerned about the "one dev" model and losing something that was a key component of what we needed. Since then I've not been impressed with SolrCloud (would love to hear good experiences, but it seems their distributed model isn't right) and have been giving Elasticsearch serious thought again.
  - michielvoo 12y ago
    "In the Java space" meaning deployed in a Java servlet container. Both are stand-alone search servers, exposing their API for search and indexing through HTTP using XML and/or JSON.
    - craigching 12y ago
      > "In the Java space" meaning deployed in a Java servlet container. I didn't say nor mean to imply that. Technically, Solr is based on servlet technology whether stand-alone or not, you can choose to deploy it in a web container of your choice or use the Jetty instance it comes with for the "stand-alone" experience. I don't know much about Elasticsearch's architecture personally.
      - Argorak 12y ago
        Elasticsearch is very component-based and can be deployed as a servlet if you really wish. Just use a different transport implementation: https://github.com/elasticsearch/elasticsearch-transport-wares https://github.com/elasticsearch/elasticsearch-transport-war...
alexmorse 12y ago
Maybe they can finally write some docs
- mlawlerau 12y ago
  Ha!
- room271 12y ago
  The docs are pretty good and have been for a while: http://www.elasticsearch.org/guide/ http://www.elasticsearch.org/guide/ Admittedly they were pretty poor early on, but they've matured with the product.
edwintorok 12y ago
I always thought that Elasticsearch was something provided by Amazon, and hence not really interested in it. So I was rather surprised to see in the title that they raised money. Apparently what Amazon provides is called Elastic MapReduce, not Elastic Search.
malux85 12y ago
Elasticsearch is great. I have spent the last 6 months working on a project using it as the primary search technology, it has been nothing but great. I have not had any training and tried to figure it out through the online docs (which in the last overhaul have gotten much better) so forgive me if any of what I'm about to say is wrong, or better yet please correct me so I can learn :) here's the things I noticed when using it: - Since it's schemaless by default it will guess the data type of a document based on the first value it sees, as far as I'm aware you're not able to change a data type later on, so I found it best to create a schema (aka mapping) being explicit about the fields data types up front. I think explicit is better anyway (Zen of python ;) ) - It's blazingly fast. Like crazy quick. - Use the geo data type if you're going to be doing radius queries. I've got 50M documents in the index and it queries insanely fast. It's been just as fast as PostGIS (which I also love) - Use this as the GUI: http://mobz.github.io/elasticsearch-head/ http://mobz.github.io/elasticsearch-head/ - Do some proper research on filtering before you start, start here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-snowball-tokenfilter.html http://www.elasticsearch.org/guide/en/elasticsearch/referenc... the correctly configured snowball filter will make sure things like "rückwärts" will match "ruckwarts",

232 points by asm89 12y ago

asm89 12y ago

What I find interesting about the Elasticsearch story is the success it has given that the core product (the search server) is completely open source.

hackerboos 12y ago
My fear is that with money comes an 'Enterprise' edition.
- SEJeff 12y ago
  I don't think the "open core" model is in their best interests. Look at a company like Zenoss, where their OSS version is their biggest competitor. It doesn't make sense for ES to do that. Now marvel[1] is where they've started monetizing and I'm sure they make a good bit of money from "professional services" teaching companies how to deploy ES at scale. Hopefully they pull it off, as the world needs a good competitor to splunk. ES has the backend tech, but kibana has a loooooong way before it rivals the interface for searching splunk. Here's to hoping! [1] http://www.elasticsearch.org/overview/marvel/download/ http://www.elasticsearch.org/overview/marvel/download/
  - Argorak 12y ago
    Marvel is quite interesting. It is a product everyone _could_ build on their own (basically, it is Kibana over their own metrics data), but at that price point, there is no reasonable reason to do so.
- aembleton 12y ago
  What would be so bad about an enterprise edition? Many companies require extra support and would pay for the enterprise edition and wouldn't even consider the product without this option.
- dchuk 12y ago
  ElasticSearch is a sufficiently complex system that can be implemented in a ton of different ways to solve a ton of different problems. I'm sure there are more than enough consulting gigs floating around for them make a good bit of money both on actual consulting and selling additional software packages.
- Karunamon 12y ago
  I was in a class with the founders a few months ago - for whatever it's worth, they're vehemently opposed to segregating functionality like this. Their plans are to offer support, and make their bones that way. Elasticsearch is a complex enough product at scale that this would probably be quite lucrative.
- thedevopsguy 12y ago
  Elastic Search make their money from Enterprise support licenses. There are so many ways of installing it , scaling and optimizing depending on application. Don't forget the training around. You have IBs like Goldman Sachs using it for log analytics. So the services around that are needed and real. I don't think they will need an enterprise version.

conjugate 12y ago

all the vcs who have poured money into hortonworks and elasticsearch are the greatest contributers to open-source - and they don't even know it.

pyrox420 12y ago

We love Elasticsearch. It's fast and accurate for huge amounts of data, super easy to scale, and incredibly easy to get started. That cash can only make the product better. Good for them!

ig1 12y ago

If people are curious as to why VCs are interested in open-source businesses: http://indexventures.com/news-room/blog/a-perfect-storm-now-is-the-time-for-multibillion-dollar-open-source-companies http://indexventures.com/news-room/blog/a-perfect-storm-now-...

wslh 12y ago
I also recommend the JBoss Business Case Study: "Lessons from Leaders: How JBoss did it" [1] [1] http://www.forentrepreneurs.com/lessons-from-leaders/jboss-example/ http://www.forentrepreneurs.com/lessons-from-leaders/jboss-e...
michaelochurch 12y ago
Open-source is becoming the only way to develop (and, especially, maintain) complex software. The fact that the engineers can gain employer-independent reputations gives them an incentive that's astronomically expensive (as hedge fund compensation goes) to replicate otherwise. The world is finally figuring out that it's impossible to employ top talent, but that it can be quite lucrative to sponsor it.
- deleted 12y ago
  [deleted]
- clubhi 12y ago
  It's not impossible to employ top talent. That is a very ignorant statement.
  - michaelochurch 12y ago
    In technical fact, not impossible but, in fact, common. In spirit, you don't really employ top talent, in the sense that subordinating it wastes it. You sponsor it and become a beneficiary of it.
  - sheepmullet 12y ago
    Top talent has access to thousands of other well paying roles. You cannot treat them the same way companies treat regular employees (badly). If you give them pointless work, or work without career growth, they can up and leave in a heartbeat. If you try and bully them or pressure them to work long hours or place pointless restrictions (dress code, start times, vacation policy, etc) on them they will leave.

gshakir 12y ago

Great product and wish them best of luck. Had great success using with Hive with their Apache Hadoop plugin for doing quick analysis.

mooreds 12y ago

I saw these folks at gluecon. Haven't looked at a ton of dashboard solutions, but I found Kibana to be pretty compelling, simply because it was trivial to get the elk stack up and running and input arbitrary data. I am not as interested in log data, more in business metrics.

jasonkolb 12y ago

I'm sitting in Elasticsearch training right now (during a caffeine break). These are some really great guys that know their stuff, and they're committed to contributing back to the OSS version everything that makes sense. They contribute a ton of code back to Lucene and employ a lot of the brightest minds in this space. On the product side, I'm sitting here being amazed at some of the problems they've solved very elegantly. Elasticsearch has a bright future.

mewwts 12y ago
I'm starting a new job in August where I will focus on building search-based applications with Elasticsearch. I honestly can't wait. This news makes me happy as I know I'll be learning relevant technology.
cbeach 12y ago
I'm also in Elasticsearch training (Victoria, London). Impressed with the trainers, both core developers with a passion for the product. Bodes well.
latj 12y ago
How is it better than Solr? I havent looked at Elastic Search in a long time, so I really do want to know. Not trying to pick a fight. ;)
- diminish 12y ago
  I m using it since earlier versions for almost 3 yers, i guess. It makes config easier, offers a well designed, easy API. Especially for Rails, PHP, Django crowd, an easy choice to create a sensible text search. The main benefit was an easier Lucene with JSON/http API. Yes a lot more advanced features are there too.
- mhitza 12y ago
  Advantageous may be the fact that you can extend ElasticSearch with JavaScript, whereas I think as easy for someone who understands the Solr codebase, I've wanted to try that once, but I had to resort to doing the necessary computation in the language calling the Solr service.

arecurrence 12y ago

Awesome! I make heavy use of elasticsearch and am very happy with the performance. It's typically the rest of the pipeline now that causes most of the latency whereas search used to be the bulk of the duration of a request. It has also spurred other entities to improve their search performance :)

jack_jennings 12y ago

Hopefully this means that the documentation gets some help, especially some of the official libraries. So far working with ES has been a mixed bag… seems good in theory but I've had a hard time getting over the learning curve.

spencera 12y ago
Want to highlight some of your specific pain points?
- jack_jennings 12y ago
  I guess I'm in the minority based on the other comments here… After spending a few hours with the documentation, I felt like I had a generally good feel for how you would interact with ES through curl, but then jumping into using the Ruby library there seemed to be a big leap and I felt like I needed to have a much more intimate knowledge of how ES worked to "get" it. A lot of guess and check before I figured out how to query properly. I've also been unsuccessful in figuring out how to implement accent folding. At this point I assume that this has something to do with "mapping", but I couldn't figure out where that was supposed to be set up (again, through the Ruby lib, but I was also unsure where to start to just accomplish that via curl…). Who knows, maybe I just need to spend more time reading the docs, and maybe the information that I need is in there somewhere, but to me it felt incomplete or disorganized. I'd love to stop using Solr for document search, but making ES do things that I know how to do in Solr ended up being too time consuming. I ended up feeling like I probably had to go to a training session to really get it, which is a shame…
  - sujeetsr 12y ago
    I found the elasticsearch tutorials on youtube by Clinton Gormley pretty helpful in understanding the concepts. Also came across this book (which I haven't really read so I don't know how good it is, just posting it here) http://exploringelasticsearch.com/ http://exploringelasticsearch.com/
- dguaraglia 12y ago

grouma 12y ago

I have been prototyping with Elasticsearch for the last couple of months. I have nothing but great things to say about the software and documentation. Several other partner teams have taken notice of my work and will likely incorporate the software as well. Very exciting!

enscr 12y ago

Could someone explain in a few lines why & how is elasticsearch a revolutionary thing (technically) ?

craigching 12y ago
Do you need something beyond "highly scalable search solution"? A lot of enterprises (and I'd guess startups too!) need good search solutions and your two players in the Java space (that I know of) are Solr and Elasticsearch. We are using Solr in our product, but the more I hear about Elasticsearch the more I want to give it a serious try. My reason for not using it originally was that we had prototyped our solution using Compass (which was the project before Elasticsearch) only to have it abandoned to work on Elasticsearch. So I was concerned about the "one dev" model and losing something that was a key component of what we needed. Since then I've not been impressed with SolrCloud (would love to hear good experiences, but it seems their distributed model isn't right) and have been giving Elasticsearch serious thought again.
- michielvoo 12y ago
  "In the Java space" meaning deployed in a Java servlet container. Both are stand-alone search servers, exposing their API for search and indexing through HTTP using XML and/or JSON.
  - craigching 12y ago
    > "In the Java space" meaning deployed in a Java servlet container. I didn't say nor mean to imply that. Technically, Solr is based on servlet technology whether stand-alone or not, you can choose to deploy it in a web container of your choice or use the Jetty instance it comes with for the "stand-alone" experience. I don't know much about Elasticsearch's architecture personally.
    - Argorak 12y ago
      Elasticsearch is very component-based and can be deployed as a servlet if you really wish. Just use a different transport implementation: https://github.com/elasticsearch/elasticsearch-transport-wares https://github.com/elasticsearch/elasticsearch-transport-war...

alexmorse 12y ago

Maybe they can finally write some docs

mlawlerau 12y ago
Ha!
room271 12y ago
The docs are pretty good and have been for a while: http://www.elasticsearch.org/guide/ http://www.elasticsearch.org/guide/ Admittedly they were pretty poor early on, but they've matured with the product.

edwintorok 12y ago

I always thought that Elasticsearch was something provided by Amazon, and hence not really interested in it. So I was rather surprised to see in the title that they raised money. Apparently what Amazon provides is called Elastic MapReduce, not Elastic Search.

malux85 12y ago

Elasticsearch is great. I have spent the last 6 months working on a project using it as the primary search technology, it has been nothing but great. I have not had any training and tried to figure it out through the online docs (which in the last overhaul have gotten much better) so forgive me if any of what I'm about to say is wrong, or better yet please correct me so I can learn :) here's the things I noticed when using it: - Since it's schemaless by default it will guess the data type of a document based on the first value it sees, as far as I'm aware you're not able to change a data type later on, so I found it best to create a schema (aka mapping) being explicit about the fields data types up front. I think explicit is better anyway (Zen of python ;) ) - It's blazingly fast. Like crazy quick. - Use the geo data type if you're going to be doing radius queries. I've got 50M documents in the index and it queries insanely fast. It's been just as fast as PostGIS (which I also love) - Use this as the GUI: http://mobz.github.io/elasticsearch-head/ http://mobz.github.io/elasticsearch-head/ - Do some proper research on filtering before you start, start here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-snowball-tokenfilter.html http://www.elasticsearch.org/guide/en/elasticsearch/referenc... the correctly configured snowball filter will make sure things like "rückwärts" will match "ruckwarts",