12 ms·
OpenSearch: AWS fork of Elasticsearch and Kibana
- VWWHFSfQ 5y agoI have mixed feelings about this server side license stuff that mongo db started. Imagine where the internet would be today if the creators of apache and mysql had tried to prevent shared hosting providers in the early days of the web from using their software
- brainless 5y agoThe time when Apache or MySQL started out was very different. Imagine where the internet would be if cloud computing itself didn't take off. Do you remember a time when there were hundreds of hosting providers? Do you remember WebHostingTalk where admins would go to check hosting offers from suppliers around the world? The monopolies finished that era. So I don't think that software companies trying to adapt now can be seen through the lens of what was 15 years ago.
- paxys 5y agoThere are more hosting providers today than in the era you are talking about. AWS has a "monopoly" simply because large companies are using it, and back in the day those companies would have run their own datacenters not used a shared PHP host. For a personal site or startup you have a thousand other options.
- darkwater 5y agoI'm not even that sure this is true in absolute numbers, but I'm pretty sure it is not in relative figures. There are waaaay more customer/businesses on the Internet today than there were 15 years ago and not that many providers more.
- carlhjerpe 5y agoDo you have a source for the data that there isn't significantly more hosting providers today? The ease of setting up Plesk or CPanel to become a shared hosting provider today is incredible, essentially 0 maintenance so I doubt your statement holds true, considering how many more people are on the Internet today compared to 15 years ago.
- that_guy_iain 5y agoI wouldn't say AWS has a monopoly. There are tons of providers ranging from Digital Ocean to Google Cloud. AWS is just the largest because they've been around the longest. They're so deeply entrenched in Cloud that most people automatically think of them. I think things such as Netflix talking about their engineering and how they use AWS was a major boost at the start. Now a days people are literally studying to become AWS certified. We even have a AWS specialised working at my company. AWS and other cloud providers are also very smart in locking in start ups by offering them thousands upon thousands of free credit. Build your MVP on there and then end up vendor locked in but think it's a good thing. I heard a tidbit that I don't know if it's true or not but I would like to think it is. AWS has become so expensive for some companies that they're starting to migrate back to their own datacentres.
- brainless 5y agoI think you need to get a reality check. I have worked as an early engineer to 12 or more startups in the last 15 years and am very active in communities ranging from HN, IndieHackers, OnDeck or a dozen others. I have not come across a single founder in the last 5-6 years who does not start with AWS credits or is not craving for them. AWS is nothing but monopoly and they have utilized access to their cash to buy early customers. The same goes for Google and MS. The smaller hosting providers that you believe exist actually are stuck with whatever customers they used to have or a chance new WordPress blog.
- carlhjerpe 5y agoEverything doesn't run on AWS. In Europe where a company might only sell to one country, like Sweden where I live for example many won't see the traffic required to scale beyond a single box, those same smaller companies might also not want to pay with their kidneys to host that one VM. Cloudflare is making this possible, so for smaller businesses and sites one could argue that Cloudflare is the real monopoly (entirely different markets). Now companies like Shopify is enabling people to run a shop without any ops for peanuts. Or smaller SMB it environments, we run a small datacenter at my company running VMware software to run our customers domain controllers, erps, fileservers and such, though this is decreasing, we used to run Exchange too, but migrated every customer to Office365 because it's cheaper than on-premise licensing, and we have 0 ops. The fact that we're local means we can offer dark fibre to many customer sites, giving them 0ms latency to us, making even the chattiest shit system run like things were on their premises. I guess we're "computing at edge" :p Not saying you're wrong that AWS is a cloud monopoly, but everyone isn't purchasing in the cloud market (the majority of the worlds money doesn't flow through startups going global). And there's also Azure which is growing at an incredible rate challenging AWS at migrating legacy workloads to the cloud.
- catern 5y agoThe server side license stuff doesn't prevent shared hosting providers from using software. It just requires hosting providers to open source their infrastructure. I think the internet would be even better today, if shared hosting providers had been sharing infrastructure technology since 25 years ago.
- VWWHFSfQ 5y agoI think that's missing the forest for the trees. The license is designed to prevent hosting providers from selling the software as a service to their customers. The requirement to open source their entire infrastructure and operations is just a means to do that.
- antpls 5y agoTechnically it doesn't just prevent from selling SaaS. It prevents from selling SaaS without having the consent of the open source project first, which means negotiating a deal.
- pas 5y agoThe value in AWS is not the software. (What I heard about it a few years ago it's very explicitly not the cobbled together software.) It's the whole enterprise, the structure, the teams, the operations staff.
- zokier 5y agoI don't think sspl is realistic to be able to comply with. This clause is ridiculously wide: > Corresponding Source for all programs that you use to make the Program or modified version available as a service All programs you use Every single one of them. There is literally no limit on what sources you would need to publish. You don't need to think very long to realize how impossible that is.
- waisbrot 5y agoI mean... there's very obviously a limit, right? You don't use an infinite number of programs. And it's not "all programs you use", it's "all programs that you use to make the Program ... available as a service". You don't need to provide the code for Microsoft Solitare, but you do need to provide the code for Apache httpd and your Linux distro. Which is pretty easy to do!
- joking 5y agoImagine where elastic would be, as the whole success of elastic is based on a apache licensed project (lucene).
- madeofpalk 5y agoIn the same place as it would require elastic’s changes to Lucerne to be made open source also?
- judge2020 5y agoDepends on the license, but if it was SSPL then Elastic wouldn't be able to run their hosted service without open-sourcing every bit of it, including all of the security, infrastructure monitoring, and authentication that is the backend of their business model. See https://news.ycombinator.com/item?id=26784552 https://news.ycombinator.com/item?id=26784552
- asien 5y ago> Imagine where the internet would be today if the creators of apache and mysql had tried to prevent shared hosting PHP + MySQL was the foundation of the Internet not so long ago , Wordpress is stil the backbone of lots of platform. With SS License definitely this would not have been possible.
- pas 5y agoWhy? Dockerized versions of countless server-side software are there for free. In a lot of cases the value is that someone maintains it and operates it for you.
- carlhjerpe 5y agoYes, but would we have TimescaleDB, CockroachDB, ElasticSearch, Docker (containers at all) and projects like these if there wasn't any money at the table? Not saying you're wrong, but it's a multidimensional problem. One could argue AWS is nothing like shared hosting providers(compare scale), and a webserver is essentially "stateless" which means easier to build than say... A database holding sensitive information that doesn't break. I assume this is why postgres HA and horizontal scaling still really isn't a thing, while CockroachDB funded by VC "solved" this problem. I think it's fair to let companies monetize on the service they built, while allowing people to run it on their own if they can. A problem here though is that the companies incentives mismatch the opensource project they're eunning. CockroachDB enterprise having killer features that the opensource version doesn't have and that noone will be able to PR because the company will reject it. TimescaleDB went ahead and opensourced all their features, aligning their incentives with the project, but I don't know of anyone else who has done this.
- deleted 5y ago[deleted]
- rafaelturk 5y agoTLDR: It’s all Apache 2.0.
- phd514 5y agoFrom the announcement: "You should consider the initial code to be at an alpha stage — it is not complete, not thoroughly tested, and not suitable for production use. We are planning to release a beta in the next few weeks, and expect it to stabilize and be ready for production by early summer (mid-2021)." Given that Amazon announced the fork in January and they don't expect it to be production-ready until summer, I'm guessing they've underestimated the amount of work required to package and distribute a product as complex as Elasticsearch. Given that, I doubt they will be well-equipped to keep pace with new feature development.
- lornajane 5y agoI'm not sure I agree with that assessment. Now that the fork is publicly available, others can contribute to get it ready, which wasn't possible until now.
- phd514 5y agoYes, others can contribute, but significant feature development on large-scale OSS projects tends to be driven by developers paid to work on the project full-time and coordinated by an organized steering committee with clear governance (or company if the product is owned by a single company). I don't see any of that in place for OpenSearch and getting that all started up is not at all a trivial endeavor.
- busterarm 5y agoGiven how poorly of a job Elastic themselves did with keeping the full ecosystem of tools working in lockstep for YEARS, I'm sure Amazon will do fine. I remember all through Elasticsearch 5 where none of their packaged Kibana dashboards flippin' worked.
- colechristensen 5y agoI would assume they’re doing more than just packaging. A beginning is the time for taking the most delicate care that the balances are correct
- fastball 5y ago
- TechBro8615 5y agoIt's hard for me to know whether to feel bad for ES in this case. Did they bring it on themselves? Is Amazon too big and a bully? From my perspective, Amazon has made most of its profit price gouging consumers on bandwidth after vendor locking them into their ecosystem, where they bootstrap new services by wrapping open source software with some provisioning scripts, management dashboards and cookie-cutter API / console templates. Indeed, most of this is templated -- AFAIU, for example, each AWS service autogenerates its Boto bindings and parts of its console frontend via code generators. Amazon has really mastered the factory process of churning out new services, and when they find a popular one, they can invest more resources into developing it than the original team ever could. And therein lies the rub. If Amazon is improving the software in a way that the original team couldn't, it's hard to say that the community isn't benefiting. I think what strikes me the wrong way is that Amazon is not doing it for any altruistic reason. In fact, Amazon contributes very little to open source in general, considering how much they take from it. Compare them to Facebook (React, etc) or Google (tons of dev tools) or Microsoft (VSC, TypeScript). What does Amazon have? Firecracker, kind of? And now a fork of ES because that's the only way they could continue making money off it without violating the license a small startup put in place to stop them? Well, good for Amazon, I suppose, but I find myself instinctively disliking them for this. I'm not sure what the solution is. Hopefully technologies like Kubernetes and Terraform will encourage big customers to become at least cloud-agnostic, if not cloud-independent. At the very least it would be great if Amazon / Google / Microsoft stopped gouging bandwidth at such absurd margins. Or not. Maybe it will be their downfall as startups differentiate along those lines. That would be ironic, coming from the originators of "your margin is my opportunity." Personally I'm doing my part by not building anything with vendor lock-in. It's great to be able to deploy to any cloud, if you value either robustness or flexibility.
- busterarm 5y agoAs a longtime ElasticSearch cluster admin/developer and Elastic Cloud customer, I don't feel bad for Elastic in the slightest and I'm psyched about this fork. The way they operate their cloud service leaves a lot to be desired and encourages maximum spend if you end up wanting to use it for anything demanding in production.
- candiddevmike 5y agoWill be interesting if other cloud providers (Google, Azure) offer this or you see other software companies offer support for it. Will also be an interesting case study if the community shifts to this project and it dwarfs elastic for features.
- Pirate-of-SV 5y agoIt's nice that they announce it and that there's some sort of future effort promised. From my perspective we might not upgrade the elastic-stack (with current Elastic projects) too far to not bacome accidentally incompatible in case we want to make a switch.
- andreygrehov 5y agoDoes that mean ElasticSearch documentation won't be relevant soon?
- bombcar 5y agoElasticSearch will need to relatively quickly come out with a feature the OpenSearch doesn’t replicate or people will just use the minimum that both support (see MySQL vs MariaDB).
- denysvitali 5y agoOkay... and what's the difference here from Open Distro for ElasticSearch? I guess it's just a rebranding, isn't it?
- hello_moto 5y agoOpenDistro for ES is the surrounding tools for ES => plugins, index-state-management, basically a suite clone of ES Enterprise offerings (X-pack) because AWS can't ship AWS ES with X-Pack. OpenSearch is the ES (core) itself. If I'm not mistaken.
- bbest123 5y agoOpen Distro for ElasticSearch was not a fork rather an Apache 2.0-licensed distribution of Elasticsearch enhanced with enterprise security, alerting, SQL etc... OpenSearch is a community-driven, open source search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2.
- xvilka 5y agoThey also could sponsor Elasticsearch alternatives in Rust - Sonic[1] and Toshi[2]. Even more, integration[3] with Vector. [1] https://github.com/valeriansaliou/sonic https://github.com/valeriansaliou/sonic [2] https://github.com/toshi-search/Toshi https://github.com/toshi-search/Toshi [3] https://github.com/timberio/vector/issues/988 https://github.com/timberio/vector/issues/988
- kizer 5y agoWhat about the thing that is already named OpenSearch?
- kizer 5y agoIt’s what Chrome’s tab to search utilizes. If you implement the protocol users can tab-to-search your site even with autocomplete.
- kizer 5y agoTurns out that was developed by Amazon according to Wikipedia. So maybe they’re merging that usage into this offering (since that is a spec for search results)?
- carlfmeadows 5y agoYeah the original OpenSearch project is a different enough domain that I think confusion will be minimal. We have talked to the maintainer and he is supportive. We have also posted disambiguation in case anyone does get confused. https://opensearch.org/disambiguation.html https://opensearch.org/disambiguation.html
- jrochkind1 5y agoIn general I'm supportive of an Amazon open source fork here. But the name re-use is unfortunate. Amazon's argument seems to be "Don’t worry, we own the trademark for ‘OpenSearch’ cause it came out of Amazon originally, so it’s cool!” That is really poor stewardship of the Intellectual Property of the trademark of a name that was part of a standard that was meant to be an open multi-vendor standard. Amazon owned the trademark to protect it's use under that standard, not to re-use it for something totally different harming the standard further. But it's just another indication that the original Opensearch, like the era of believing in open web standards for inter-operability that it was part of, is dead.
- DetroitThrow 5y agoI agree, it speaks poorly for their stewardship
- ddevault 5y agoI'm happy to see a couple of good choices made here: - Sticking with Apache 2.0 - Asking for a Developer Certificate-of-Origin rather than a copyright assignment This bodes well for the future of this fork. Amazon also has the resources necessary to keep up consistent and quality maintenance of a project on this scale. Elastic would definitely like you to view AWS as the Big Bad here, but their response to the Elastic betrayal is very good, and I would like to see more like this in the future.
- rurban 5y agoThey took over FreeRTOS for good, CBMC for good, with Xen they were a bit unlucky, but it still has much better security than KVM, and now they take over ElasticSearch. Good Open Source efforts, much better than until a few years ago.
- luke2m 5y agoThis is just sad, amazon playing the victim.
- oever 5y agoThe new name clashes with the Open Search Foundation. https://opensearchfoundation.org/ https://opensearchfoundation.org/
- noitpmeder 5y agoLike they care
- DetroitThrow 5y agoThat's extremely frustrating
- busterarm 5y agoAmazon was using the term "OpenSearch" themselves (via former subsidiary A9) back in 2012. https://github.com/dewitt/opensearch/ https://github.com/dewitt/opensearch/
- ignoramous 5y agoOpenSearch was once was an initiative founded at A9, Amazon subsidiary, to create a personalized, cross-service, search engine: https://archive.is/PCKWq https://archive.is/PCKWq OpenSearch is from an era when Amazon and Google were covertly competitive. Google didn't get anywhere with Froogle and AppEngine; whilst Alexa and A9 didn't move any mountains. Code: https://github.com/dewitt/opensearch https://github.com/dewitt/opensearch
- williesleg 5y agoOh this will not end well.
- mgr86 5y agoNaive aside, but why would I want to use ElasticSearch or OpenSearch over Solr? Are ES and Solr not both based on Lucene?
- binarymax 5y agoGreat blog post on this subject: https://opensourceconnections.com/blog/2019/02/28/stop-worrying-solr-elasticsearch/ https://opensourceconnections.com/blog/2019/02/28/stop-worry...
- vervas 5y agoNot familiar with Solr but I believe the analogy to linux would be choosing the preferred distro while they all use the same kernel. There are a bunch of long comparison lists if you search for it.
- _tom_ 5y agoElasticsearch has done a good job of focusing on the log analytics space, and Kibana is a great tool. I'd probably use ES for any log analytics and Solr for things like website and ecommerce search. Of course, you could do both with either.
- seabrookmx 5y agoElasticSearch seems to have more mindshare. It can be easier to find resources online to help solve your problems.. though ES moves through versions fast (and they do break backwards compatibility on major version bumps) so sometimes this can still be an issue. Other benefit is that you don't have to rely on Zookeeper if you're horizontally scaling. I don't have a ton of experience with Solr but they seem pretty comparable.
- elric 5y agoIn all three cases, Lucene is used as a "low level" (Java) API which provides search capabilities. OS, ES and Solr turn Lucene into a server, with features like horizontal scaling (ES Cluster, Solr Cloud). The major differences are in how well that all works, how easy it is to administer, how much caching and optimization is done on top of Lucene, etc. I haven't extensively used ES, but I've used Solr a lot (and contributed to), and I can say that it's a mess. The community is not one of the better ones I've seen. Bugs and stability issues are often ignored. Patches sit around gathering dust. There are some gems and very clever people in the community, of course, but it seems like there are too few of them to cope with the large beast that Solr has become. If I were starting a new project in the search space, Solr would not make my shortlist.
- parhamn 5y ago> and we don’t ask for a contributor license agreement (CLA) Makes me wonder what these are for (copyright transfer) and why they decided it’s not needed. It also makes me wonder if this sort of thing has ever been taken/tested in court or if it’s paranoid friction with little value add.
- resoluteteeth 5y ago> Makes me wonder what these are for (copyright transfer) and why they decided it’s not needed. It also makes me wonder if this sort of thing has ever been taken/tested in court or if it’s paranoid friction with little value add. Some companies/projects might use them purely to avoid possible future legal headaches (I think GNU does this), and I'm not sure to what degree that has actually been tested, but they can also allow re-licensing under a different license which is more clear cut and I think that's more the issue here Amazon is trying to say that they'll never relicense the code, so they have no need to take ownership over contributions.
- Boulth6 5y agoIndeed that is likely but I wonder why didn't they at least require a Developer's Cerificate of Origin [0] that kernel.org uses. This is really lightweight (just append one line to git commit message) and supposedly provides a minimum legal base for the change. IANAL. [0]: https://blog.chef.io/introducing-developer-certificate-of-origin https://blog.chef.io/introducing-developer-certificate-of-or...
- 5y ago
- sdesol 5y agoWill be interesting to see the resources that AWS will throw at this. You can get a sense of the resource that elastic.co is throwing at elasticsearch at https://public-001.gitsense.com/insights/github/repos?r=github%3Aelastic%2Felasticsearch&v=repo https://public-001.gitsense.com/insights/github/repos?r=gith... I'm currently indexing the fork, so in about an hour or two, I'll provide the insights for the fork as well.
- binarymax 5y agoElastic spends so much time and effort on making sure that their search is performant (and they are not shy about deprecating and removing features that are slow). I think this is where Elastic will continue to shine. It's one thing to add features, it's another to make it so they work well and make sure the integrating product team doesn't shoot themselves in the foot.
- sdesol 5y agoWhat may hurt them though, is the number of customers that currently feel things are currently "good enough". I don't know what their sales engagement looks like, so I'm not sure if this will really hurt them or not.
- binarymax 5y agoPerhaps! My money is on Elastic including an approximate nearest neighbor search in 8.0 which uses the new HNSW feature in Lucene 9, which is going to be hard to do in a distributed capacity and will be a significant feature if they can pull it off. PS - I've never seen gitsense before and it's really cool. I especially like the focus quadrant!
- sdesol 5y agoSince search is your domain expertise, I'll take your word for it :-) As for GitSense, checkout the impact section and sort contributors by "First Commit". With this, you'll get a very good idea of the developer's expertise level.
- thayne 5y agoI'm glad they changed the name. Although it's probably too much to ask that the AWS service be renamed to OpenSearch Service as well.
- ElijahLynn 5y ago> We plan to rename our existing Amazon Elasticsearch Service to Amazon OpenSearch Service.
- thayne 5y agoOh, I missed that :)
- fallat 5y agoSo as someone who has heard about Elasticsearch for years and years, and seen all this, right this moment I've decided to see what it really is. On their home page, "Why use Elastic search?", the reasons are basically: * It's fast! * It does a lot of stuff! * It has some tools to visualize data! * It's distributed!! I have to say this is not very appealing to me since it sounds like something any database could do.
- deleted 5y ago[deleted]
- villasv 5y agoYou're starting from the wrong place if you're comparing Elasticsearch with a database. And you're also arriving at the wrong place if you think that any database can be distributed.
- WrtCdEvrydy 5y agoElastic gives you a lot of the fancy stuff that SQL kinda needs extras and hard work for... but it's just a document store with fancy weighting features.
- xapata 5y ago> just a document store With a particular profile of efficiency choices and interfaces that might appeal to your project.
- manigandham 5y agoThe Elastic website has a dedicated page to explain: https://www.elastic.co/what-is/elasticsearch https://www.elastic.co/what-is/elasticsearch "Elasticsearch is a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on Apache Lucene and was first released in 2010..." I suggest understanding what it is first before comparing it to other databases.
- 5y ago
- sunilkumarc 5y agoInteresting! On a different note, recently I was looking to learn AWS concepts through online courses. After so much of research I finally found this e-book on Gumroad which is written by Daniel Vassallo who has worked in AWS team for 10+ years. I found this e-book very helpful as a beginner. This book covers most of the topics that you need to learn to get started: If someone is interested, here is the link :) https://gumroad.com/a/238777459/MsVlG https://gumroad.com/a/238777459/MsVlG
- fakedang 5y agoOn a tangential note, how is Meilisearch compared to Elasticsearch?
- binarymax 5y agoMeilisearch is a new niche engine for instant-search experiences, and is far less flexible and less mature than Elasticsearch
- shankspeaks 5y agoImagine if they went after Mongo next? Atlas is a virtual monopoly for Mongo solely due to SSPL, and it has created a ridiculously overpriced ecosystem for hosted and managed services, and tooling around it. Parking the technical merits to one side, considering the sheer number of devs and early-stage products that are built on Mongo, I'd love for someone to go after them next.
- calmoo 5y agoAmazon already have DocumentDB which clones the Mongo API. I don't think its forked though, they just use a barely mongo compatible wrapper around their own db engine.
- vio2 5y agoIt's not nearly as compatible as you might think. Interestingly enough, MongoDB's CTO managed RDS at AWS.
- bgorman 5y agoThis already exists https://aws.amazon.com/documentdb/ https://aws.amazon.com/documentdb/
- Graphguy 5y agoKinda blocked on the compatibility front after the 4.0 API though, eh?
- bmcahren 5y agoNot if the recent Oracle vs Google supreme court ruling is to be acknowledged. DocumentDB is to MongoDB 4.0+ as Dalvik runtime was to JVM. Here is their 4.0 compatibility update: https://docs.aws.amazon.com/documentdb/latest/developerguide/compatibility.html https://docs.aws.amazon.com/documentdb/latest/developerguide...
- 5y ago
- chatman 5y agoIf this project were to be governed by Apache Software Foundation, I'd have associated more credibility to this effort.
- binarymax 5y agoHey All, if you're interested in getting a good understanding of this vs Elasticsearch, we invited the team to give a Haystack LIVE talk where they outlined the details and goals of the project: https://www.youtube.com/watch?v=J_6U1luNScg https://www.youtube.com/watch?v=J_6U1luNScg
- deleted 5y ago[deleted]
- whydoineedthis 5y agoanyone know if OpenSearch still uses "/" as a special character? Largest pita when trying to use ES for logging web applications and quite frankly, made it near unusable. If Amazon fixed that, I would be firmly on their side. Also, any improvement over Kibana would be welcome.
- znpy 5y agoRIP Elastic
- bmcahren 5y agoI feel Amazon took the feedback from the DocumentDB/MongoDB fiasco to heart and made positive change in their approach. DocumentDB is a closed source proprietary database created by Amazon to emulate the MongoDB API. Think Google's Dalvik runtime vs Sun/Oracle's JVM. This time around we have an open source fork of ES with big backers all contributing and very permissive licensing. In both cases, Amazon gets to implement AWS-specific upgrades to management to depend heavily on EBS replication rather than application-layer replication. Would it be nice to have that secret sauce that makes Aurora/DocumentDB so nice to use compared to self-hosting or RDS? Of course. Do we have to have it to consider using or contributing to the open source software? No.
- busterarm 5y agoOn the other hand, MongoDB is already sort of obsolete and trending towards death by the time that all ended up happening while ElasticSearch is hot and "new".
- rabbidruster 5y agoWhere do you get the impression MongoDB is trending towards death? Seems to be growing by some metrics; the stock price has more than doubled in the last year. Not a fan myself, but still seems a long way from death to me and seem to be doing something right in enterprise market.
- busterarm 5y agoThere's a difference between trending towards death and dead. IBM has been trending towards death for decades now and it's nowhere near dead. MongoDB is certainly on the road to death, IMO. As has Oracle DBMS since the 90s. Most companies tend to make more money as their product's growth stalls out (extracting more money from existing customers). The fact that Mongo had to create shitty-license underpins huge revenue problems. If you look at the major trends and surveys, the ones targeted at people who actually drive database adoption within companies, MongoDB is sliding YoY for 3 years now. The place you see Mongo growing is Atlas. Yes, as their competitors can no longer offer MongoDB in their clouds, revenue shifts to MongoDB. That does not mean that use of the database itself is growing.
- bambam24 5y agoThis is wonderful
- mumblemumble 5y agoIt's been said that the best way to fortify your business is to use your clout to make the world inhospitable for adjacent businesses. As someone who is not currently in the cloud, that idea strikes me as being very pertinent to what's happening here. Increasingly many technologies are becoming cloud-only, or have non-cloud offerings that are decidedly second-class. Elastic offers on-prem support. I doubt Amazon will be doing the same with OpenSearch. It may be a subtle effect, but it's pushing the world in a direction that makes me uncomfortable. If it's harder for non-cloud-based companies to maintain non-cloud-based offerings, then that will push the industry even more toward being dominated by SaaS products. And these products often leave clients and users locked in, with limited control over their own data, and, by extension, reduced ability to control their own fates. What I worry about is that we may be witnessing a return of Embrace, Extend, Extinguish, only in a new form that's even more dangerous because it's harder to see. I appreciate the discomfort people have about the SSPL. It is a departure from the original ideas behind FOSS. But, at least as I see it, those open source principles were never an end in and of themselves. They're a means to a greater end: digital autonomy. To the extent that very large companies seem to be learning how to co-opt FOSS in order to re-assert control, FOSS's ability, in its current incarnation, to serve that end may be waning.
- busterarm 5y agoElastic's on prem support amounts to little more than an onsite where they explain to you how the Java Garbage Collector works. EVERY detail about tuning your clusters derives from keeping the Java GC from ruining your day. There's some index template optimizations but any semi-competent engineer or dba should be able to figure all that out (it's literally all in the documentation about what not to do). You'll still be able to pay a consultant to come help you -- they don't have to be from Elastic. In fact, it seems like Amazon just created an industry for third party consultants here.
- jimmy2020 5y agoIs there any good that done by Amazon to support OSS? Like ever. They started with cloning MongoDB and now Elastic with actually zero contribution to the community regardless of their insane profits. This is a clear single. Amazon can always clone and redistribute any open source software then lock it in for AWS. If we've started to witness declining in OSS, well at least we know now who started the wave.
- unethical_ban 5y agoYou're asserting that they have forked OSS and then not provided back the source code for their own improvements. I guess we can check their github repos to see if that's the case.
- jimmy2020 5y agoAs far as I know DocumentDB is closed source. Btw: I can name zero OSS projects from Amazon and this is not the same for Microsoft (VScode, TS) and FB (React, Jest).
- yandie 5y agoFirecracker is a famous one that I can think of.
- invertedreversi 5y agoFork of Google crosvm
- jimmy2020 5y agothere's always a fork.
- Twirrim 5y agoFirecracker, s2n, chalice, jsii. On top of that AWS has done a lot of contributions upstream, particularly around the linux kernel, https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?qt=author&q=%40amazon.com https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin..., (particularly the virtualisation stacks involved), some around OpenJDK at what looks like an increasing pace, and the like. To be clear: not an AWS employee.
- deknos 5y agoi bet this will only really work in AWS. :D
- mekster 5y agoAre people actually required to use ELK? What are your use cases? The interface is completely cluttered and it takes loads of resource and it feels like it's waiting to be replaced with lighter and more focused products. Graylog (though it uses Elasticsearch internally) does a decent job at log handling and creating all the visual items out of logs and Grafana/Loki can do quite good at it as well with a very small memory footprint. Besides, most of the "business intelligences" aren't actionable but just some visual arts you wouldn't need but to stare at when you're bored.
- mdaniel 5y agoI recently learned that Graylog changed their license and it's now one of those vanity licenses: https://github.com/Graylog2/graylog2-server/blob/master/LICENSE https://github.com/Graylog2/graylog2-server/blob/master/LICE...
- AtNightWeCode 5y agoI have been perfectly happy with ES cloud services. Is this done by honest intentions from AWS or is it simply based on the fact that ES are making a lot of money of the cloud services?
- judge2020 5y agoWell, now nobody can provide a competitor to ES cloud services for newer versions. If you upgrade to v7.11 or above, you're locking in your choice for 'managed hosting for ES/Kibana' to ES cloud services. > Is this done by honest intentions from AWS or is it simply based on the fact that ES are making a lot of money of the cloud services? Is wanting to make money considered honest intentions? ES released v7.10 under the Apache license on purpose, so they knew that any form of license change would mean people can still use earlier releases without having to adhere to the SSPL, and anyone could legally fork it or run it on non-ES hardware without having to pay the piper.
- InTheArena 5y agoI suspect this is mostly AWS trying to stop using "ElasticSearch" in the title of something, probably for trademark reasons.
- galaxyLogic 5y agoI wonder if ES had originally been AGPL licensed would that have helped them? If Amazon adapts AGPL code to integrate it with their own infra=structure doesn't that in fact mean that all of Amazons' software-based infra-structure would become AGPL as well, and thus easily reproduced by Google Cloud, MS Cloud, Oracle Cloud etc.? Or even inhouse? In other words wouldn't it mean it would be easy to replicate the Amazon Cloud-business (on a smaller scale)?
- lacker 5y agoAmazon just wouldn't do that. They would either not offer it as a service, or make a clone from the beginning like they did with MongoDB. In general none of the cloud providers are actually willing to comply with the AGPL license.
- pessimizer 5y agoOffering the unaltered software as a service, or forking it and releasing all of your changes under AGPL does comply with AGPL.
- deleted 5y ago[deleted]
- hit8run 5y agoPerfect case for a megacorp destroying open source plus business models. I start to hate amazon with a passion. Craziest thing is they are not paying taxes in Europe though they dominate the market. Amazon needs be broken up. It's too big and too mighty.
- themolecularman 5y agocancel amazon prime? It's not too difficult to wait an extra day or two for products.
- fastball 5y agoIsn't this an example of an open source business model. Amazon is supporting development of this Apache 2.0 licensed OSS, which they plan to make money off of...
- hit8run 5y agoIndeed. But they are not the company supposed to make money of that project. They start to dominate several markets by cross financing and therefore need to be broken up.
- fastball 5y agoWhy not? Elastic built their product off of Lucene, why are they the only people that get to make money of this particular piece of OSS?
- galaxyLogic 5y agoI think this thread is much about shared source licenses like SSPL vs. "orthodox" open source licenses like GPL. Based on the link below it seems to me the difference is that SSPL etc. have a clause which prevents me from making money by selling the use of the licensed software over the network for instance. GPL puts some rather strict rules on users of copyleft software, mainly that you MUST distribute your modifications with the same license. What I don't quite get is why adding a rule that says "if you make this software usable over the network you must make it usable for free" would be considered categorically less ethical than GPL. GPL says you must give out your modifications for free. SSPL says you must also give out the rights to use that software for free as well. Isn't SSPL more ethical in the sense that it requires you to give out more for free? https://techcrunch.com/2018/11/29/the-crusade-against-open-source-abuse/ https://techcrunch.com/2018/11/29/the-crusade-against-open-s...
- pas 5y agoEthics requires a framework. Just because something is free it doesn't become good. For example free heroin samples! It's a complex problem to even phrase the question of what do we mean by having a healthy software/IT ecosystem. Do simply count the number of users? GDP of the Internet? Number of git repos? Naturally those doesn't even begin to capture the self-balancing dynamics we are after. We want to encourage folks to start new ventures, but also to give back. But by giving back what if they eliminate old ventures? (Eg. Google "giving back" Chrome might make the Firefox venture non-viable.) How can we describe healthy competition? (It'd be good if the browser market wouldn't be cross-financed from ads, but - let's say - every user would tell their ISP to direct some of their subscription fee to one of the browser vendors.) Okay, but what does this have to do with licenses!? Yeah, it's a fairly hard problem.
- galaxyLogic 5y agoGood questions for discussion. There seems to be a commonly held or propagated assumption that GPL good other licenses un-ethical.
- pmarreck 5y agoWhat is the sell for ES over something like the fulltext search built into Postgres, considering that the cost of adding another dependency is not insignificant?
- blackoil 5y agoAm I missing something here? Elastic says this is a free sw, which you can install and use, but if you want someone else to manage the hosting, we are practically the only option. How did that became an 'ethical position' ? If I am OSS dev, why should I contribute to them vs OpenSearch?
- Redsquare 5y agoAlgolia has won the search race, move on
- Wintereise 5y agoEh, far from it. Their per record / per query pricing is extremely prohibitive for any real production use.
- redwood 5y agoEmbrace extend extinguish
- peterohler 5y agoOne of the more discouraging aspects of being an OSS developer is that successful companies that use your software never consider contributing to the OSS developers. I suppose that is the nature of business though. Take what you can get.
- heroHACK17 5y ago"If you're going to shoot the king, you'd better be god damn sure you kill him." - Jack Barker
- imnoscar 5y agoI want to love elasticsearch and they keep making it harder. On top of this open source backflip, their sales staff would put Oracle to shame. Pressure tactics, making clients pay 5 figures for few hours of ES consultant time, very expensive training/certification (and cert expires every 2 years) - among few of them. AWS gave us consultant time for no charge, guess who we picked to run our ES load.
- wejick 5y agoOne of the best part of elasticsearch is excellent documentation, hope this project can replicate that well.
- galettesaucisse 5y agoMaybe Amazon treating open source developers like it treats its blue collar workers will open people's eyes about working conditions in 21st century American capitalism.
- brainless 5y agoI feel this is a very scary trend starting. I have not come across a single founder in the last 5-6 years who does not start with AWS credits or is not craving for them. AWS is a monopoly and they use their cash to buy early customers. Initially it was Amazon's money, but now AWS has enough cash of their own to push whatever they wish to. The same goes for Google and Microsoft. AWS directly building up the software side of what started out as IaaS (Infrastructure as a Service) is only going to hurt software vendors. We can only expect new software players or ones with low capital to restrict their licenses even more. Open source licenses are not only for ideological freedom, but very necessary for companies (end users) to integrate and modify products on their own. We will migrate more toward source-available licenses instead since big giants are going to corner the small companies. () Edits
- judge2020 5y agoI find it fine for software to not be open source. Source-available but closed-source (in terms of freedom) is perfectly fine from a commercial standpoint and should be the gold-standard for mission-critical software in the backend. The problem comes from companies touting their software as open source purely for the marketing aspect in order to bring in customers and get free work done by people who don't get paid.
- est 5y agoOff topic, is there a lightweight alternative to ES? Preferably can be run in small RAM vps.
- tannhaeuser 5y agoJust want to mention that "OpenSearch" is/was also an AWS^H^H^HAmazon initiative for websites to expose a search URL to browsers in HTML metadata, similar to exposing an RSS feed URL. They may want to consider renaming it to avoid complete and utter confusion, like searching "OpenSearch" (no the other one) using "OpenSearch" (no the ES fork).
- beshrkayali 5y agoI think even with their "permissive usage guidelines" of the OpenSearch trademark, their own way of how they've been doing Amazon ElasticSearch would not be allowed... For example, you can't do: `Microsoft OpenSearch`, you have to do "for OpenSearch" or "with OpenSearch compatibility". From their "permissive" trademark policy [1] > You may also use the “OpenSearch” word mark to make accurate statements about compatibility and interoperability using relational phrases such as “works with,” “runs on,” “compatible with,” and the like (e.g., “Foocorp Software powered by OpenSearch” or “Foocorp Software for OpenSearch” or “Foocorp Software with OpenSearch compatibility”). 1: https://opensearch.org/trademark-usage.html https://opensearch.org/trademark-usage.html
- jbub 5y agoso what will happen to opendistro now ?
- saint_abroad 5y agoIrrespective of how big Elastic is, this is an escalation of Amazon's behaviour of suppressing the tough weeds of competition. > First they came for the socialists, and I did not speak out—because I was not a socialist.
- based2 5y agohow Beats compability is preserved?