
On “Open” Distros, Open Source, and Building a Company - christophilus
https://www.elastic.co/blog/on-open-distros-open-source-and-building-a-company
======
timdorr
For context, this is in response to Amazon's release of "Open Distro for
Elasticsearch": [https://opendistro.github.io/for-
elasticsearch/](https://opendistro.github.io/for-elasticsearch/)

Related discussions:

[https://news.ycombinator.com/item?id=19359602](https://news.ycombinator.com/item?id=19359602)

[https://news.ycombinator.com/item?id=19363961](https://news.ycombinator.com/item?id=19363961)

~~~
hodgesrm
Clicking through to the AWS blog article [1] we find this:

> At AWS, we believe that maintainers of an open source project have a
> responsibility to ensure that the primary open source distribution remains
> open and free of proprietary code so that the community can build on the
> project freely, and the distribution does not advantage any one company over
> another. This was part of the promise the maintainer made when they gained
> developers’ trust to adopt the software.

I have enormous respect for Adrian's work on cloud systems but the quote above
seems very self-serving for Amazon.

Open source is bound by licenses that define the terms of open source code
use. There's nothing whatsoever in the licenses that binds the project
developers to continue to work on the project, to maintain similar license
terms in later releases, or to take (or not take) any other action in their
lives. To argue otherwise seems contrary to the notions of freedom that are
the foundation of open source software.

[1] [https://aws.amazon.com/blogs/opensource/keeping-open-
source-...](https://aws.amazon.com/blogs/opensource/keeping-open-source-open-
open-distro-for-elasticsearch/)

Edit: typo corrected

~~~
bigiain
Amazon can believe whatever they want, but when they say "maintainers of an
open source project have a responsibility to ensure ... " \- I call bullshit.

Those maintainers owe Amazon nothing. The have no responsibility to Amazon.
None.

Same as they owe me nothing.

By definition (if you use any of the more-that-two-or-so-year-old definitions
of "open source"), the project is defined by "remaining free and open", in
that you can always use the code and license you've been using - but there's
no way anybody can make it a "responsibility" of a maintainer to to anything
in the future. They could die, they could get bored, they could get an all-
encompassing job, they could get a family, they could get venture capital and
license any future work with a non-open source license (as Redis and Elastic
have done, at least in some people's view).

The "the promise the maintainer made" was "here's some code, under GPL or
aGPL, or Apache Licence, or Artistic license of BSD three clause, or whatever
the promise at the time was. Take it or leave it. No future responsibility
assumed or enforced. They might even have a track record of releasing more
stuff under the same license/promise. But they can stop or change things
whenever they want.

If Amazon wants the maintainers to do anything else (including ever committing
another line of code or fixing a bug), they can wait for them to do (or not)
it at their discretion, or they can pay the maintainer (or someone else) to do
it (and then, depending on the license, potentially be required to release
that code under the same license.

Sorry for the rant, but that line caught my eye too, and I got immediately
pissed off at Amazon's sense of entitlement there. No open source maintainer
owes you shit. They've shared what they've shared, and you can take it or
leave it, but you taking it doesn't place any obligation or responsibility on
them.

(deep breath)

~~~
SpicyLemonZest
Maintainers are certainly free to adopt a "take it or leave it" attitude if
they want.

But most maintainers (including Elastic) don't want that. They proudly declare
that their developers and users are a community, and they're steering the
community to produce a great and open project on an ongoing basis. The
maintainers certainly have a responsibility to follow through on such promises
- or at least, if Amazon is correct and they aren't following through, they
have no right to complain about a hostile fork.

~~~
hodgesrm
Amazon is welcome to fork and welcome to point out that they might be better
for users than Elastic. But the issue here is that Amazon--like many others
before them--are asserting a 'right' that simply does not exist.

~~~
SpicyLemonZest
I don't think Amazon's trying to claim that Elastic has done something legally
or morally wrong. They just don't feel Elastic has lived up to their standards
for community management, so they're going to take their ball and go start a
new community with higher standards.

------
amirathi
Why write emotionally charged letter instead of one backed with facts? Amazon
is claiming [1][2] that Elastic has been intentionally unclear about what part
of Elastic stack is open source vs proprietary. It would best serve Elastic if
they respond to that with facts.

If the claims are true, and done intentionally, it's a really bad move on part
of Elastic. The ambiguity around licensing hurts all their users and not just
Amazon.

[1] "Unfortunately, since June 2018, we have witnessed significant
intermingling of proprietary code into the code base. While an Apache 2.0
licensed download is still available, there is an extreme lack of clarity as
to what customers who care about open source are getting and what they can
depend on. For example, neither release notes nor documentation make it clear
what is open source and what is proprietary."

[2] "Individual code commits also increasingly contain both open source and
proprietary code, making it very difficult for developers who want to only
work on open source to contribute and participate"

~~~
ldubost
While as an open source software and company creator I highly connect to the
emotional content of his response and feel the pain for Elastic when it comes
to the competition by Amazon, I do agree that he does not address the factual
issues at hand and uses badly the word "open" for stuff that is "shared
source".

One aspect Elastic is also not directly attacked on and therefore not
addressing is the choice of features that are not free and the price point.
Elastic's business model is quite smart. You can easily develop on top of
elastic, however once you have done it and want to go to production some key
features are missing and they are not cheap. Particularly securing your
cluster data is one of the features in xpack which is problematic not to have.
I'm pretty sure you can search for open ports 9200 over the internet or
company intranets and find unwanted unprotected data. And it's not cheap as
you need to go for Gold subscription if I'm correct. I would be surprised
nobody has proposed a pull request with competing code.

Elastic has and is investing a lot as open source and they do need a good
business model that allows investment in the core open source code. The thing
is I'm not sure it's missing here. Elastic.co is quite healthy. It's hard to
know if it would be without this business model, but anyway there is little
chance they would do differently given Elastic.co is VC backed. In the end the
real fight is between the big money players, not about whether Elastic the
software is sustainable. The reality is that given the community, it would
very probably be even without the "xpack" package. Now it's not sure
Elastic.co would be able to aggressively invest in both oss and non oss as
much and therefore compete with Amazon.

OSS is tough. When you go for it, you should go in it for OSS, not for the
company, because keeping the company at the level of the OSS success is hard
both for small and big OSS projects.

Ludovic Dubost (XWiki)

~~~
bragh
You can put a reverse proxy in front of it to provide at least basic
authentication measures and force HTTPS. Better than nothing at least.

But the main problem is that unsecured clusters by default have caused a lot
of reputation loss to the brand. When every few months news hit about yet
another unsecured Elasticsearch cluster that leaked huge amounts of data, it
is getting harder and harder to explain to the less informed how that is the
fault of those people who did not even bother running a reverse proxy, not the
fault of Elasticsearch itself.

------
andreareina
Clicking through to the other blog posts, it becomes apparent the the author
plays fast and loose with the word "open", to their benefit. They "opened up"
by providing gratis licenses to some of their software. They don't call it
"open source", but the phrase "open source" is also scattered around the
articles, so that "open" benefits by proximal association.

They're free to license their software however they want; I don't use it so I
really don't care. But it really rankles when someone extols the virtues of
open source and waxes rhetoric about how open they are, and yet the software
they offer is proprietary by default. They "double down on open", but the open
source distribution is opt-in.

I cannot help but feel that the articles are deliberately constructed to
mislead. That's a serious accusation and I wish I could be more charitable in
how I interpret them but the terminology is so consistent that it cannot be an
accident -- gratis-but-proprietary is always "open", libre is never just
"open" but always "open source".

Technically correct isn't enough. No-one's explicitly staked "open" as being
the same as "open source". But in the context of this conversation "open" is
an obvious contraction of "open source" and to use the obvious contraction to
mean something diametrically opposed is not reasonable or ok; especially when
"source available" is widely understood. The same thing was attempted with the
Commons License.

The intent behind words is important. Persuasive oratory is fine but using
those tactics to mislead does not engender trust, especially when the
beneficiary is the one speaking and not the audience.

~~~
Jedd
I came away with a similar impression.

OP uses the word free twice, in both cases meaning $-free.

"We [...] open sourced most of it, and made all of it free."

From an enterprise licensing perspective, 'open source' as a subset of 'free'
just means I shall have to spend more time with lawyers and accountants.

------
pauldix
This is a really great response and I'm curious how AWS will respond. It's
particularly damning because AWS was arguing from a position of moral
authority on the move, which I think was disingenuous. They wanted to create a
fork and act like they weren't. Shay just called them out on it.

I also wrote up my own thoughts, but it's obviously not as relevant as this.
I'm a little more blunt about what AWS is doing. I even throw in a comparison
to Microsoft abusing their monopoly power in the 90's by distributing Explorer
for free to kill off Netscape: [https://www.influxdata.com/blog/aws-intends-
for-their-new-pr...](https://www.influxdata.com/blog/aws-intends-for-their-
new-project-to-be-an-elasticsearch-fork/)

Longer term, the real question is how Elastic will respond to this
commercially. Will they create more closed source? Go higher up the stack into
specific applications? It's going to be an interesting and tumultuous couple
of years in open source infrastructure software.

~~~
andreareina
Elastic doesn't have much moral authority either when they keep using the word
"open" in reference to code that isn't open source. But they never say "open
source" so technically correct.

EDIT: you've remarked elsewhere on the thread about commercial and economic
realities, so I want to take the opportunity to qualify my statement: they're
free to license and monetize their code however they want. It's what I see as
deceptive intent that I'm having a (very strong) reaction to.

~~~
Drdrdrq
Iiuc, they are basically using open core model, with core app being
free/open/libre under Apache license, and extensions proprietary with code
available non-free/non-open/non-libre Elastic license. At least to me the
situation seems pretty clear, and Amazon complaining that they can't take all
of the code to create a competing service seems pretty disingenious to me.
OP's post is thus in line with what I would expect - but it seems to irk you
in some way. What is it that I'm missing?

~~~
andreareina
I said more here[1] but basically the use of "open" around their
gratis/proprietary code. Never "open source" so it's not an untruth but it's
still misleading especially in a context where they're also talking about
their contributions to open source, a commitment to being open, etc.

[1]
[https://news.ycombinator.com/item?id=19369774](https://news.ycombinator.com/item?id=19369774)

------
jrochkind1
It's interesting that the HN collective results seems to be complaining about
the letter from either party.

When Amazon wrote their letter, the [HN
response]([https://news.ycombinator.com/item?id=19363961](https://news.ycombinator.com/item?id=19363961))
seemed to tip towards not liking what Amazon had to say or was doing.

When ES writes effectively a response, the initial comments don't like ES's
response or what they're doing either.

It may be that the collective HN vibe is critical no matter what. But also
maybe "we" in aggregate legitimately don't like either option given.

I know open source devs need to make money somehow. I also wish there was a
way to do it that was open source. I'm not sure there is. I'm not sure what it
means for the continued viability of open source.

In the original period of open source, people generally worked on open source
on the clock at their existing jobs, a building tools they needed for that
job, a job not dedicated to that particular product but to getting something
done that product helped with. People were getting paid to write open source,
but it wasn't by "productizing" it. The results were still shared freely.

(this is how/where apache httpd, for one, came from; people with jobs who
needed an http server writing one on the clock, even though their job was not
"writing an http server." They needed an http server to accomplish what was
needed for their employers, but their employers had no desire or capacity to
be in the business of selling an http server. Why not collaborate on one we
can all use?)

But those days are gone, software has just gotten _so much more complex_ and
time-consuming to develop. The open source software we have and need can't be
developed in someone's "spare" (even on-the-clock spare) time. Some open
source is still written by people working for a large employer, who pay them
to be _dedicated_ to that product. But we see the perils of that when an
employer decides "wait, why aren't we making money selling this? Why are we
contributing our employee's time to it?" (thanks Oracle).

So that leaves people trying to productize it one way or another. It is not
clear to me how to have a healthy open source ecosystem in that market
environment. I don't think "proprietary layer on top" is actually good for
open source, especially if it's gonna lead (naturally) to rejecting
contributions from outside to add those same features as open source (why
would we want to spend time working with a contributor to make the feature
solid, when we _already wrote it_, and when accepting the contribution would
_harm our business model_ because that feature is what we want to charge
for?), and then resistance to the "right to fork" which I consider the
fundamental basis of open source freedom.

~~~
zokier
> But those days are gone

I have to strongly disagree here. Stuff like Kafka from LinkedIn, Envoy from
Lyft, Prometheus from Soundcloud are just the tip of the iceberg of people
writing tools because they need them, not because they are in the business of
making infrastructure software.

I believe that selling software, be it foss, open core, or fully proprietary
is not good business model, neither from business nor from ethics point of
view. Sell your expertise or use the software yourself to generate value, but
trying to sell it is just futile.

~~~
elcomet
I think a lot of successful companies, Microsoft for instance, would disagree
with you.

~~~
zokier
Considering the direction Microsoft has taken, I think they in particular
would agree. It is no coincidence that MS is more and more straight up giving
away their software, or at least bundling it as a service.

------
wgjordan
Actions speak louder than words. Amazon's words [1] were backed up by action-
specifically, the public release of Open Distro for Elasticsearch and several
useful, advanced features under an Apache 2.0 license.

This post from Elastic consists of vague claims of 'FUD' and of their
commercial code being 'bluntly copied', without any further clarifying details
or responsive action to back them up. (E.g., if you have real evidence of
Amazon copying proprietary code, which I doubt, then announce the copyright
infringement lawsuit you have just filed, don't weakly allude to your
grievances in a blog post.)

Conspicuously, the post did not contain any defense or even mention of
Amazon's key accusations of "significant intermingling of proprietary code
into the code base", or its claim that "the innovation focus has shifted from
furthering the open source distribution to making the proprietary distribution
popular." The only response offered is a vague, generic "we believe in open
source" \- again with no concrete action backing up those empty words.

If this is the best and only response Elastic is able to offer, I fully expect
Amazon to keep the high ground in this controversy.

[1] [https://aws.amazon.com/blogs/opensource/keeping-open-
source-...](https://aws.amazon.com/blogs/opensource/keeping-open-source-open-
open-distro-for-elasticsearch/)

~~~
hypto
actions? I think this is speaking for itself:
[https://www.elastic.co/downloads/elasticsearch-
oss](https://www.elastic.co/downloads/elasticsearch-oss) What has Amazon
contributed to yet? Repackage other open source projects that was already
existing? I don't call it a contribution.

~~~
nemo44x
"Repackage other open source projects that was already existing?"

Most people don't realize this yet. They are taking other, existing projects
which anyone could install themselves for a long time now and framing it like
they built these things for their new fork. So much of their "distribution" is
just a collection of other open source projects.

In the case of the security functionality, that is backed by a company named
Search Guard who have an Enterprise Version in addition to the Community
Version included in the AWS fork. They forked another project, renamed it and
now call it their own.

~~~
pm215
Isn't "a collection of other open source projects" pretty much exactly what a
"distribution" usually implies? What you get is the curation/selection of
what's included and some reassurance that all the pieces really do work
together and other people are using them that way.

------
__blockcipher__
I didn’t see much done to address Amazon’s claims (intermingling of
proprietary/open code in same conmit, etc). Mainly they just claimed FUD and
reminisced a bit.

If I were Elastic I’d be quaking in my boots right now - their entire business
model is under assault. Hell, the whole open core model is being called into
question. Personally I’m glad to see Amazon’s distro.

Anyone have a different take?

~~~
rooam-dev
Just my 2 cents. I don't care about Amazon's distro, I think it won't work.
Others have tried, e.g. MariaDB vs MySQL, OpenOffice vs LibreOffice. Time will
show though...

Edit: The thought was about big companies with their own agenda behind open
source initiatives. Sorry about confusion.

~~~
yjftsjthsd-h
What pattern are you seeing in those examples? Most Linux distros today ship
with LibreOffice and have MariaDB in-repo. From where I sit, the open forks
won.

~~~
mises
Won in terms of user base, maybe. But in terms of money made and commercial
adoption? Mysql beats Mariadb, though I can' t say for openoffice/libreoffice.

~~~
SpicyLemonZest
Is that a fair metric to judge on? The entire point of forking MariaDB was
that the developers expected Oracle to make compromises for the sake of money.

------
car12
Yeah, slice it whichever way, but this basically looks shitty on AWS.

At the end of the day, Open source developers need to be able to put food on
the table. What AWS is doing here, is like a logging company, recklessly
destroying things in its path as long as what they do is "legal", they don't
seem to care.

Thing is, we should look at Open source software just like a precious
rainforest or any other natural resource. If every company started doing what
AWS were doing, soon there would be very few companies like Elastic in
business and that kind of _open_ software will cease to exist.

But hey, at least HNers won't be able to argue about technicalities of the
license then, right?

~~~
geofft
Open source developers need to be able to put food on the table. Open source
developers do not need to be able to put food on the table _while being
employed by Elasticsearch_.

Frankly, yesterday's announcement made me seriously consider whether I wanted
to apply for a job working at AWS, since I expect they have many positions
where you're spending most of the time working on open source code like
Elasticsearch.

The argument you're making is equivalent to the old argument against open-
source software itself: if companies can't make money by selling proprietary
software, who's going to want to be a software developer? Sure, it's great
that some bored Finnish guy wrote an OS that's competitive with Solaris, but
if he drives Sun out of business what's the guarantee anyone else will want to
work on his OS? Don't we need proprietary software companies to ensure that
quality software gets developed?

~~~
car12
> if companies can't make money by selling proprietary software, who's going
> to want to be a software developer?

And that did happen, we don't have a single decent _affordable_ personal
computing ecosystem at this point. The only good personal computing ecosystem
is with Apple, which makes its money by selling _proprietary_ software mostly
but is not affordable to ordinary joe.

What an ordinary person can buy, is an ad-riddled machine like a windows
desktop or a chromebook, where although the software is not proprietary,
they're basically selling you ads(directly or indirectly through your data).

To tie it back in, in fact open source software did rise up and proprietary
software did go down, but we lost the pure software aspect that came with
proprietary software. Basically, open source software is being used to sell
you ads and if the ads based business goes down/stops so does open source
ecosystem.

If you haven't realized, the biggest parts of the open source economy are
propped by FAANG and Microsoft. If these big whales go away, open source's
vibrancy will vanish in a poof.

In fact open source software creation is hugely concentrated to North America
and by extension FAANG. So if FAANG were to stop sponsoring open source, we'll
be back to proprietary software age soon. So, I'm not convinced of your
argument that "open source" has won conclusively.

~~~
geofft
I think my argument is that open source has in fact won, it just hasn't
brought the benefits some people expected it to bring. (And it's totally fine
to say, "Having seen the results, I'm no longer a supporter of open source,"
if you want.) The open-source-is-viable argument was that people will find a
business model other than selling the software itself, and the software
industry won't collapse when proprietary software becomes unviable - and
that's exactly what happened. It turned out the most profitable business model
wasn't support contracts or custom development, it was largely software as a
service and advertising. It also turned out that this business model was so
profitable that there are tons of free-as-in-beer but non-open-source software
products out there, including the vast majority of what Facebook, Google, etc.
offer.

------
dhd415
With the rise of cloud computing, the effort by cloud providers to commoditize
their complements, i.e., software, inevitably leads to this particular kind of
conflict between companies backing OSS software who wish to deploy their own
SaaS offerings and cloud providers with their own SaaS offerings. A common
response is that OSS backers can build businesses based on support and
consulting but I believe that's overly restrictive. Those kinds of businesses
don't scale nearly as well as managed services and it will definitely have a
chilling effect on OSS development if the kinds of companies that are
permitted to be built on them are simply support and consulting. I don't
believe that companies backing OSS have an exclusive right to offer managed
services for their software, but the current situation with cloud providers
offering managed services with only token engagement with the OSS community
(and then only when forced into it by licensing restrictions) is not
beneficial to OSS, either.

~~~
SpicyLemonZest
The post describes how Amazon offered more than a token engagement. Elastic
rejected them, as such a partnership would have included "preferential
treatment that would place them above our users". That's consistent with what
I've seen from other OSS-backing companies; they want cloud providers to
contribute to their community, but flatly refuse to go out of their way to
enable it.

------
mythz
OSS is a great business model if you're one of the 3 multi-billion dollar
major cloud vendors co's who are collectively the primary beneficiaries of
everyone else's OSS investments since relatively no-one pays to acquire/use
free OSS software, only to host it, so it's of course in their best interest
to keep everything OSS so they can collect rent at the point where people pay
to utilize its value - for hosting their production systems.

It's easy for AWS, Azure or GCP to be altruistic and suggests everything
should be OSS for the betterment of humanity, since they've become the primary
beneficiaries of the wealth created by OSS which benefits them more than
anything else. Then use the profits to develop billion dollar cloud
infrastructure moats to ensure a barrier to entry that prohibits anyone else
from partaking in.

This is ultimately the core issue between companies behind developers of OSS
software like Redis Labs, Confluent, Elastic.co, etc. They've put in the
investment to build and support their OSS products but it's the cloud vendors
end up reaping most of the revenue derived from it given they're the only
entity Customers pay to host their production systems. This is what the
extended OSS licenses are designed to target which are typically free for
everyone else but prohibits the cloud vendors from absorbing all the rewards
from hosting their software without revenue sharing anything back.

------
prabhatjha
I think that AWS are the "bad guy" here. I am happy to be proven wrong but
they don’t have a track record of contributing code to the OSS projects they
provide as a service. Whenever they are forced like it was the case with Mongo
and now with Elastic they are using their power to have a competing OSS
project. There is nothing wrong with this business wise but it's totally
against spirit of OSS.

If you look at Red Hat on other hand when they decided that Kubernetes was the
way to go for their OpenShift project they put lots of engineering resources
for upstream k8s.

------
jalaziz
This letter is clearly not written by someone who interacts with customers. As
someone who has tried to work with Elastic, their pricing is atrocious even
for the most basic of features.

You can't get alerting without paying a king's ransom for features you'll
likely never use.

------
vegardx
It would be interesting if someone pushed a PR that contains more or less all
the "secret sauce" of X-Pack, and see what reasons they'd use for rejecting
it.

Clarification: Not push X-Pack itself, but features comparable. Like they
suggested people do in the post.

------
raiyu
While the original movement behind open source was one of benevolence that
isn't to say that the economic model of open source at scale actually works as
a competitive advantage.

There are costs associated with running a business and one of the largest is
marketing. What open source allows you to do is defer these costs by charging
nothing for adoption. This allows you to acquire users, build up an ecosystem,
and create a moat against competitors to your business. Competitors not in the
sense of AWS which is adjacent, but competitors in the sense of another
replacement for Elastic Search itself.

At the beginning you are losing 100% of your revenue opportunity, but you have
to remember that you are either modeling yourself as a consumer company or
business company. Simply put, consumers companies make more money from
consumers while businesses make more money from businesses.

AWS is a business company so the majority of their revenue is from businesses.
With most open source projects you are also a business, which means if you
find 10 years of success 90% of your revenue will eventually come from
businesses.

Those early adopters are usually smaller entities so they aren't really a
large opportunity loss and the big businesses want to see massive adoption
before they really consider switching. So effectively you are cutting out 20%
of your revenue opportunity to build up an ecosystem moat as well as marketing
budget. It isn't money directly out of your pocket, but instead is paid purely
in head count, which by the way is much cheaper and more effective than trying
to market your software to hundreds of thousands of people across the world.

The "free" component isn't just one of benevolence but actually an attack on
any future competitor because why would those early adopters pay $5 for
something that they can get for free. Most early businesses rely on these
smaller customers and open source actually cuts that out of the equation,
reducing competition.

So it's a very interesting dynamic in the sense that something that was
started for a "good" purpose can accidentally actually stifle future
competition, when you would assume otherwise.

------
newaccoutnas
I'm sure Shay et al have good intentions, but to have any degree of usability
with elastic.co you need to buy x-packs which goes against the grain a little.
I'd still like to have auth (not just basic-http) on personal projects.

~~~
cheald
Searchguard is a plugin for ES which gives you TLS + auth. They offer a paid
version with more features (ActiveDirectory integration, HIPAA-compliant
auditing, etc - actual enterprise features), but if you just need
auth/ACLs/transport TLS, it's a great product.

~~~
newaccoutnas
Oh, thanks, I'll take a look

------
rushabh
"When companies came to us, seeing our success, and asked for special working
relationship in order to collaborate on code, demanding preferential treatment
that would place them above our users, we told them no. This happened numerous
times over the years, and only recently again, this time with Amazon. Some
have aligned and became wonderful partners to us and the community. Others,
sadly, didn't follow through. We have a commitment that we will treat a single
developer contributing to our products the same as others. There is no
preference, and we will reject any ask to have one. Our answer has always been
a constant: send a pull request, like everybody else does. The quality will
speak for itself."

The way this is written, it seems like there was a difference of opinion in
terms of the product architecture, and not a purely commercial decision by
Amazon. I would think it would make commercial sense for the team at Elastic
to align with Amazon than the other way round. On the other hand Amazon claims
that Elastic was moving away towards proprietary code [1].

Overall, being in open source for a decade, I am against this trend in dual
licensing (which I fear is pushed by Venture Capitalists). People forget that
the fact that you have freedom and zero friction, is what makes open source
awesome. There are other ways to make money, like building a kick-ass
consulting practice around the open source offering.

[1] [https://opendistro.github.io/for-
elasticsearch/faq.html](https://opendistro.github.io/for-
elasticsearch/faq.html)

------
cestith
Very specifically, the open core of Elastic's offering doesn't do
authentication. It's not that there's a limited number of users. It's not that
it's a single user and password in a config file somewhere. It's that you go
from having all the features of a licensed version including multiple users
with configurable access to different data sets down to needing to run elk
with no native authentication and authorization and then wrap authentication
around it one way or another if you want the community edition. This is 2019.
A data aggregation and inspection system that by default sits open on a port
for anyone to use is not quite fit for purpose.

------
scriptkiddy
I've never really thought that elastic search is all that great, to be honest.
I use it every day at work for an application that does a ton of aggregations.

That said, I can't shake the feeling that we could get the same level of
performance with RethinkDb with the benefit of a better API and less overhead.

Assuming that the use case does not require large quantities of full-text
search, is there really any reason to choose ElasticSearch over RethinkDb?
ElasticSearch does have a larger ecosystem, but with how good RethinkDb's API
is, I'm not entirely sure any 3rd party packages would actually be necessary.

Add to that RethinkDb's ability to do joins and changefeeds and this decision
becomes a no-brainer for me.

~~~
brylie
From what I gather, RethinkDB hasn't seen a release since July 2017:

[https://www.rethinkdb.com/blog/2.3.6-release/](https://www.rethinkdb.com/blog/2.3.6-release/)

The project may be transitioning to new governance.

~~~
scriptkiddy
Yeah, it was taken over by the Linux Foundation.

I can't really say that RethinkDb NEEDS a new release.It's feature complete in
my eyes. All it really needs at this point is maintenance.

------
KorematsuFred
I think a lot of open source people forget that large companies worry a lot
about licenses and clarity around that. Without that clarity and control they
get sued left and right and on this very forum we end up discussing how evil
this large company has been.

Elastic search is a great product and there is no doubt about it. However I
understand Amazon's viewpoint that it should be clear which part of ES are
open source and which are not.

A the end of the day Amazon or Google can easily buuld a clone of ES from
ground up.

------
shareinnovation
Aws is copying others’ efforts and earning billions. Nothing is being given
back to the community, instead Amazon execs buy Bay Area luxuries. Stop aws
manipulating innovative projects in the name of managed services. Elastic,
mongo, Kafka, redis, all are being preyed upon.

~~~
mises
> Stop aws manipulating innovative projects in the name of managed services.

How? Redis is trying, and got backlash. Mongo is trying, and got backlash.
Confluent is trying, and got backlash. We've rather got to pick our poison
here: do we want companies to open their code and have it used for managed
services/cloned, or do we want them to close it and keep making cool tech?

I recall reading about the model Google used to use for this sort of thing:
open your source code about three to five years after you stop using it. This
gives something to the community, while still allowing you to maintain a
competitive advantage, run a managed service, etc. Particularly for those who
run a SAAS model, it lets them gain widespread adoption before a competitor
can spin up an instance of their stuff and undercut them slightly.

Oh, and for those saying "use the Red Hat model", the Red Hat model can't work
for everyone. Red Hat had a new idea and essentially no competition for
enterprise linux at the time, and so didn't have to deal with the competition.
They got a chance to build a customer base before Oracle knocked their product
off. And even then, Oracle did completely rip off their work. That's the risk
you run with open-sourcing your codebase, and not every company is built to
work that way; not our job to tell them they should redo their business model
to allow it.

~~~
andreareina
A lot of conversation around the backlash was about the insinuation that the
new licenses were open when they weren't. "Apache 2 + Commons Clause" is an
abuse of both the Apache license and the concept of the commons; the SSPL was
submitted to the OSI for approval when it clearly did not meet the definition.

------
mdnormy
This response doesn't address why Netflix and Expedia Group is joining AWS
open distro effort. Licensing is major headache for them I assumed.

Personally, as current and former administrator of multiple ES cluster, I'm
quite worried how this "intermingling of open source and proprietary code"
will affect my current and previous team. We certainly not aware of it before
this week.

------
dominotw
This was bad. I was expecting a move towards transparency that could've
outmaneuvered amazon. This is just appeal to emotion.

------
shasts
what would be the next move from Amazon? Open distro for Kafka, MongoDB and
Redis, and all other successful opensource tools?

Just don't know where Amazon is going with this approach of re-packaging
existing opensource tools and calling it their opensource contribution.

~~~
striking
Amazon is rewriting the proprietary enterprise features of ES from scratch and
giving them away for free.

Also, their MongoDB "distro" is just "Amazon DocumentDB (with MongoDB
compatibility)". That's not going to be open source, because it took a lot of
work to basically rewrite most of Mongo.

------
tjolls12
[https://aws.amazon.com/blogs/aws/new-open-distro-for-
elastic...](https://aws.amazon.com/blogs/aws/new-open-distro-for-
elasticsearch/)

------
blinkingled
Lot of bitterness about people forking and bundling ES. If it was so painful
for Elastic maybe they should have gone for a more suitable model - you get
everything open source and pay for support and priority /early access to
security updates. Unlike the current model where you get basic features as
open source and have to pay for and use different license if you want to use
essential features like monitoring and performance analysis.

Someone else with a business model that benefits from providing these features
for free and making them open source to reduce their development/maintenance
burden - cloud providers like Amazon - are going to do just that. And they're
well within their right to do so.

~~~
yjftsjthsd-h
> pay for support and security updates.

Impeding security updates seems rather unethical. Unless you mean backports to
old/stable versions, which is still iffy in my mind but does at least maintain
a way to stay secure for everyone.

~~~
blinkingled
I meant priority/early access to security updates really. Updated my post to
reflect that.

For example enterprises may want quicker access to well tested bug or security
fixes - they can get that first and then the patches appear on GitHub for open
source users.

------
linuxdude314
The claim that Amazon says they work with Elastic for their service is bogus.
I've asked many times, and its been made clear to me that they have some
additional services they run, but aside from those run upstream Elasticsearch.
(Same with Kubernetes).

Elastic's licensing is very hard to navigate, and the per node nature of it
often makes it more cost prohibitive than Splunk.

Hopefully this is a wakeup call to Elastic to start delivering value that
can't simply be added by third party contributions. If thats the case, what
purpose is Elastic (corporation) actually fulfilling?

------
vikaskyadav
This is why I like Thanos. Bullshit is everywhere. Even a person doing behind
a great open source software is getting criticism now. Good days!

------
Infinity15
They write this big post and end the last paragraph saying "It is always Day 0
at Elastic". What does that remind you of? :D

------
ignoramous
> When companies came to us, seeing our success, and asked for special working
> relationship in order to collaborate on code, demanding preferential
> treatment that would place them above our users, we told them no.

I think this may have been the crux of the friction between the companies.
Elastic sure looks to have collaborated with GCP [0][1], if not other
enterprises.

> Our answer has always been a constant: send a pull request, like everybody
> else does. The quality will speak for itself.

I look around and spot at least three substantial bug/fixes from AWS employees
(?) [2][3][4], so I don't see any lack of effort from AWS? AWS engs were
sending pull requests, just like the others. That said, any sort of heavy
handedness from maintainers of elasticsearch might also play a role in forcing
enterprises to take control of software they depend upon so much where legally
allowed. We have seen this happen with Node.js/IO.js and MariaDB/MySQL before.

> The FUD mostly comes from large(r) companies that fear what such a movement
> can cause.

The irony of this statement is not lost on me. Elastic is that larger company
(one that crucially controls a majority of the ecosystem around elasticsearch:
kibana, logstash, beats, xpack to name a few), and AWS is an upstart here with
their investment in open-distro, presumably to fight FUD.

> We all sometimes need to self reflect on what and why we did that made us
> successful, to make sure we stay true to our course. [..] to others out
> there, that face so many reasons to be distracted, keep your focus [..] And
> last, to express our shared commitment to continue to build great products
> [..] It is our true north.

I think one right way for Elastic to show commitment to opensource would be to
entrust the projects to a community agreed governance model, like others
before them have done [5][6][7][8].

Also, what's interesting is that its not just Amazon here, but Expedia and
Netflix as well putting their weight behind open-distro [9].

Like someone pointed out, this response from Elastic does seem to be
emotionally charged [10].

\---

[0] [https://www.elastic.co/about/partners/google-cloud-
platform](https://www.elastic.co/about/partners/google-cloud-platform)

[1] blog.google "With the Elastic partnership in place, we’re kicking off our
joint engineering work and will start rolling out managed Elasticsearch on GCP
later this year." [https://www.blog.google/products/google-cloud/google-
cloud-p...](https://www.blog.google/products/google-cloud/google-cloud-
platform-partners-elastic-offer-managed-open-source-search-and-analytics-gcp/)

[2]
[https://github.com/elastic/elasticsearch/pull/27628/files](https://github.com/elastic/elasticsearch/pull/27628/files)

[3]
[https://github.com/elastic/elasticsearch/issues/31479](https://github.com/elastic/elasticsearch/issues/31479)

[4]
[https://github.com/elastic/elasticsearch/issues/29531](https://github.com/elastic/elasticsearch/issues/29531)

[5] GridGrain and [https://ignite.apache.org](https://ignite.apache.org)

[6] Various networking companies and
[http://opennetworking.org](http://opennetworking.org)

[7] Cloudera and [https://impala.apache.org](https://impala.apache.org)

[8] LinkedIn/Confluent and
[https://kafka.apache.org/](https://kafka.apache.org/)

[9] [https://aws.amazon.com/blogs/opensource/keeping-open-
source-...](https://aws.amazon.com/blogs/opensource/keeping-open-source-open-
open-distro-for-elasticsearch/)

[10] "It is always Day 0 at Elastic (like the developers we serve, we use zero
based numbering)."

~~~
sciurus
"Elastic sure looks to have collaborated with GCP"

AFAICT that partnership just resulted in Elastic Cloud running on GCP in
addition to AWS. If I had to guess, I'd say that Elastic didn't think it was
worth their time to support GCP, but Google provided some resources to
encourage them to add it as an option. That way Google gets to tell
prospective GCP customers that they have an option for hosted Elasticsearch
without having to build it themselves.

------
yowlingcat
Yeah, Elastic doesn't look great here. Anyone else getting deja vu about
Mongo?

------
buryat
so, any license changes?

------
ilrwbwrkhv
time to forget elastic and use open distros

------
scoutt
As an alternative to parsing 3 HN links (plus comments) and reading between
the lines, can someone please summarize what's going on ?

Thank you.

------
explodingcamera
Wow elastic seems salty. I don't think this blog post was a great idea. While
I understand that they need to monetize their project, I have been put off by
them putting their security features behind a paywall. They literally
advertised "Security" as non-existent on their XPack pricing page for their
basic free plan. Furthermore, I am still laughing about how the author
litteraly wrote that Amazon "bluntly copied" them, "sadly, painfully, with
critical bugs".

------
AbraKdabra
I read all of that as a rant about "pleople cloning my open source project and
making it better, stop plz".

Duh, I'm glad about the direction Amazon is going.

------
benatkin
Elastic, like Redis and MongoDB before it, decided that Open Core isn't
enough, but that they need to go against the Open Source Initiative (a
grassroots organization that represents the Open Source movement very well,
and runs [https://opensource.org/](https://opensource.org/) which contains the
open source definition) and confuse users by misusing the well understood, but
non-trademarked term "open source". This phrase in the post from the article:

"When others closed down, we opened up."

...links to another article called Doubling Down on Open [1] where they say
introduce their _source available_ strategy and in some places call it open
source (with and without capital letters) and in other places differentiate
between it and open source.

Please don't call things Open Source that don't fit the Open Source
Definition. You have a right to do so since it's not trademarked, and I'll
defend that right, but it adds confusion (the D part of FUD). And while the
OSI doesn't have a trademark to Open Source (perhaps rightfully so since it's
a generic term, though more generic terms like Apple have been trademarked),
they do own opensource.org.

[1]: [https://www.elastic.co/blog/doubling-down-on-
open](https://www.elastic.co/blog/doubling-down-on-open)

