
Who eats who in open source - mathattack
http://www.platformonomics.com/2019/11/dining-preferences-of-the-cloud-and-open-source-who-eats-who/
======
tinalumfoil
[https://webcache.googleusercontent.com/search?q=cache:www.pl...](https://webcache.googleusercontent.com/search?q=cache:www.platformonomics.com/2019/11/dining-
preferences-of-the-cloud-and-open-source-who-eats-who/)

~~~
neonate
Also
[https://web.archive.org/web/20191123091956/http://www.platfo...](https://web.archive.org/web/20191123091956/http://www.platformonomics.com/2019/11/dining-
preferences-of-the-cloud-and-open-source-who-eats-who/)

------
jandrewrogers
The root of the issue is that licensing costs, the only costs that open source
is guaranteed to reduce, can be a small percentage of operational costs in
practice. If open source makes little effort to optimize the vast majority of
other operational costs at scale, and this is largely the case in my
experience, it can literally be economically uncompetitive with a closed
source product that does optimize those other operational costs even if it has
a license overhead or creates vendor lock-in.

I see this dynamic more and more, companies have figured this out. Sure,
buyers _prefer_ open source infrastructure without vendor lock-in, but they
don't prefer it enough to spend 2-10x the OpEx, which is often the case in
these discussions. The poor optimization of operational costs (except for
license fees) is a critical weakness in open source, and it is increasingly
being attacked successfully. Open source is a nice idea that most companies
love, but they aren't going to spend mountains of extra money in operational
costs to get it.

~~~
henryfjordan
> If open source makes little effort to optimize the vast majority of other
> operational costs at scale, and this is largely the case in my experience

I feel the opposite is true. There are many open source projects that are at
least on par with their proprietary counterparts. Postgres is competitive with
every SQL DB I've heard of except maybe in some niche use-cases. Redis is open
source and incredibly performant. Same with Kafka (if you use it well). Linux
sure runs better than Windows in my experience.

There is also a whole class of open-source libraries like React that have
taken over. I can't even think of a proprietary frontend framework.

What software are you thinking of when you make these claims?

~~~
jandrewrogers
There are two major dimensions to the operational cost weaknesses of open
source. Neither is guaranteed to be meaningfully exploited by proprietary
software in any particular case. Since we are talking about operational costs,
this is almost entirely about data infrastructure software, not things like
React.

First, there is a vast amount of supported operational tooling that is simply
missing from open source ecosystems. PostgreSQL, which I still use
extensively, is a perfect example of this. The code bases around this tooling
are _several times the size of the product they support_ , and it isn't the
kind of code that most developers aspire to write despite its high value to
the customer. Oracle and SQL Server have architectures that are as obsolete as
PostgreSQL, they don't compete on the basis of being modern, efficient designs
but on the basis of having dramatically better operational tooling. Cloud
RDBMS attack a different aspect of this.

Second, is the operational infrastructure costs i.e. how much hardware is
required to run a given workload. This is the larger threat to open source,
particularly as the data intensity of business increases. PostgreSQL, Kafka,
and Redis (I've used all three operationally) have several times the hardware
requirements to deliver a workload in practice than is required with a state-
of-the-art architecture. This isn't hypothetical, I've designed engines that
replaced them when warranted. An 80% reduction in infrastructure cost is
attractive when you are already spending tens or hundreds of millions of
dollars per year on it. Some popular proprietary software is just as
inefficient but that isn't guaranteed to remain the case (and I've replaced
those systems too). Designing efficient software architecture isn't magic
(see: ScyllaDB), you simply see very few examples of it in open source because
it isn't a priority (see: use of managed languages for data infrastructure
despite this being a known limitation).

The reality is that I can guarantee data intensive businesses who make
competent use of open source that I can reduce their infrastructure costs by
4x with ease. You can bury a _massive_ amount licensing and engineering costs
in those savings.

Also, some companies are explicitly looking at this as a major "green"
initiative. The wasteful infrastructure footprint to deliver a workload with
open source is viewed as _environmentally unfriendly_ , and this is pushing
companies to consider proprietary solutions even if they are not sensitive to
operational costs. That isn't a conversation open source is currently prepared
to have but it is coming.

~~~
exceptione
> _I can guarantee data intensive businesses who make competent use of open
> source that I can reduce their infrastructure costs by 4x with ease_

I am interested in your perspective and experiences.

> _Oracle and SQL Server have architectures that are as obsolete as
> PostgreSQL, they don 't compete on the basis of being modern, efficient
> designs but on the basis of having dramatically better operational tooling._

What dramatically better tooling does SQL Server has?

> _PostgreSQL, Kafka, and Redis (I 've used all three operationally) have
> several times the hardware requirements to deliver a workload in practice
> than is required with a state-of-the-art architecture._

What should I think of when you talk about state-of-the-art architecture? And
what would replace aforementioned tools?

What should the open source offerings do to win you over?

~~~
wayneftw
> What dramatically better tooling does SQL Server has?

I don't know about Oracle but SQL Server Data Tools (SSDT) and SQL Server
Management Studio (SSMS) are vastly better than anything I've seen for open
source databases.

SSMS is just a really good visual client with an excellent table/view
designer, diagram builder and query builder with GUI management features that
let you manage virtually every facet of your server and databases.

SSDT lets you treat your SQL schema, database settings and deployments like
code, with versioning. All of your table schemas, views, functions,
procedures, settings, seed data, etc. are stored as text in your git repo.
When you're creating, it has excellent intellisense/autocomplete. You can use
it to diff your schema and data against a given SQL Server database to
generate migration scripts. You can also do data comparisons between your seed
data and your server and you can diff schemas and data between 2 different
servers without even having created an SSDT schema. Another very useful
feature is that you can reverse engineer a SQL Server database that was
already created in order to start your SSDT project. It also has visual
designers for all of the aforementioned things. It used to be a separate tool,
but now it's part of the free Visual Studio community edition. Both tools are
free, as are various editions of SQL Server.

Also, SQL Server itself has had features for a long time that PG is just
starting to get such as real Stored Procedures, ones that can return multiple
heterogeneous result-sets, run transactions and do control of flow with SQL-
like statements (IF/THEN, DO, WHILE, etc.) It also has table variables, which
are like in-memory tables, that you can use as a faster type of temp table for
certain sets of data.

Since I moved off of Windows for my workstations, I've been running SQL Server
in Docker on Linux for the past few months - it's rock solid.

~~~
exceptione
Afaik, those things are available in DBeaver [0] and MySQL Workbench [1] as
well.

@jandrewrogers also mentioned a 4-fold performance increase by switching to
proprietary solutions from pgsql. I am still wondering what those solutions
might be...

\---

0\. [https://dbeaver.io/](https://dbeaver.io/)

1\.
[https://www.mysql.com/products/workbench/](https://www.mysql.com/products/workbench/)

~~~
wayneftw
I’ve used both and I can assure you that both the products you mention fail to
approach the utility and more importantly, the quality of either SSMS or SSDT.

Have you tried the SQL Server based products at all?

~~~
exceptione
Not that much, so I might have to take a better look then. :)

------
gwern
> The unexpected and asymmetric competition from the clouds confounds open
> source companies, who must confront the fact the competitive advantage of
> knowing their software better than anyone else isn’t the insurmountable moat
> they had hoped. It is never fun to wake up and discover your product is now
> just a feature of a broader offering, but this is what is happening with
> software. Claiming open source is eating the cloud is like coffee bean
> farmers claiming they’re eating Starbucks: it willfully (or just out of
> delusion) ignores the vast majority of what the customer is buying.

As always, 'commoditize your complement':
[https://www.gwern.net/Complement](https://www.gwern.net/Complement)

~~~
leoc
> (in Robert Metcalfe’s infamous expression, cross-platform web browsers & the
> Internet would reduce Windows to a “poorly debugged set of device drivers”)

Compare the number of platforms nowadays which (to varying extents) treat
Linux as a free set of buggy device drivers.

------
awinter-py
Missing piece of the argument -- the AGPL service provider clause (that didn't
make it into GPL v3).

The author is talking about physical infrastructure as adding value, and
they're right, but there's a middle layer that cloud vendors add to their
managed tools -- things like auto backup, upgrade and resize for mysql.

These are enhancements to open source DBs etc that cloud vendors keep in-house
as secret sauce. This makes the stock version of open source software hard to
operate in the way AMZN / GOOG operate it, while still allowing AMZN / GOOG to
benefit from community effort without giving much back.

~~~
wmf
With AGPL you still get eaten, just in a different way. The cloud providers
will rewrite your software from scratch while maintaining protocol/API
compatibility.

~~~
jdsully
That's just simple competition that we've always had. Microsoft working on a
competitor to your product was the huge fear of the 90s. Cloud providers
competing with your own source code is a new phemonema.

The original thesis would be that brand loyalty would be strong enough to
prevent forking like this. "You don't want that third-rate version, get the
real thing from the original authors!". It appears that the customer base
assumes quality is roughly equal considering its mostly the same code.

~~~
detaro
In many businesses, getting permission to buy/license from a new supplier is
way more effort than just selecting another option at the cloud provider
you're already set up with.

------
athrowaway3z
I'll play the 'open source eating cloud' argument because what merit it has,
isn't addressed in this piece.

How do computers earn people money? Speaking in 2019 that's obvious. Cloud
providers earn the most.

There used to be other ways people got rich off computers. But all those
businesses imploded against the awesome might of '0 marginal cost', 'only good
solutions survive', world of open source.

Open source has 'removed more profits'/'provided more value' then the cloud.
Cloud is simply the high profit game until competition kicks in, open source
tools become the norm, and only the value of the virtual machine is sold at 1%
the current price.

------
jacques_chester
I work at Pivotal and disagree with a fair amount of the characterisation
thereof, but my bias is mostly informed by actually working there for the last
6 years. Let's look at the heart of the argument instead.

> _If you squint, open source could be seen as a very generous charitable
> donation to some of the largest and wealthiest corporations on the planet._

Broadly: yes. Except backwards.

FLOSS can be seen as a public good -- it is non-excludable and non-rivalrous.
Excludability is the property that someone can be prevented from using the
good (eg, requiring payment to consume a can of soft-drink). Rivalrousness is
the property that utility from consumption of a good by one person diminishes
utility for another person (if I drink the soft-drink, you get less of it).

Economics predicts that public goods will be underprovided by a pure market.
This is because of the free rider problem. Since I can't be excluded from the
good, I can consume it without giving something up for it. Since it's non-
rivalrous, there's no meaningful back pressure that eventually raises the cost
of consumption to an unacceptable level.

A strictly rational agent will _always_ free ride on a public good. And most
of the time, most rational agents will not provide a public good, because the
cost of paying for everyone else's consumption exceeds the benefits of their
own consumption.

This hints at one of the ways that public goods get provided: through subsidy
by benefactors, who will capture some but not all of the value created by
their benefaction. A wealthy person may so enjoy seeing the opera that they
will donate heavily to the local opera house. They don't capture the full
benefit -- other folks can watch the same shows -- but they capture _enough_
value that they are satisfied with the arrangement. They might also get value
from other factors, such as social approval.

The cloud providers do _not_ sell public goods. Their services are fully
excludable. If you refuse to pay, service will end. They _are_ rivalrous at
the limit, though, putting them into the category of club goods. These are
much more likely to be provisioned in a pure market, since the benefits and
costs fall more "correctly" on those who obtain or bear them.

~~~
syshum
>>Economics predicts that public goods will be underprovided by a pure market.
This is because of the free rider problem. Since I can't be excluded from the
good, I can consume it without giving something up for it. Since it's non-
rivalrous, there's no meaningful back pressure that eventually raises the cost
of consumption to an unacceptable level.

>>A strictly rational agent will always free ride on a public good. And most
of the time, most rational agents will not provide a public good, because the
cost of paying for everyone else's consumption exceeds the benefits of their
own consumption.

Thus the reason Free (As in Freedom) software advocates promote the use of
Copy-Left Licensing, and why non-copyleft (like MIT, BSD, and others) are
slowly eroding "open source" to less fully formed software and more just the
tooling, libraries and dev environments used to create software

~~~
pjmlp
Being an old dog at tech, given how the community has been anti-GPL licensing,
pushing for non-copyleft licenses, I predict that in a couple of years we will
be back to the PD, Shareware, Demoware, Beerware, PostcardWare, whatever were
all the way to share code during the 80 and 90's on home computers.

And it is already too late to change course back to GPL being widespread.

Every year there is a new project replacing GCC with LLVM, on Android the
kernel is the only major GPL piece still standing (with Fuchsia on the
horizon), on embedded there is a rise of non-copyleft OSes (including Zephyr,
a Linux Foundation project), and so on.

~~~
syshum
<sarcasm> Dont worry Microsoft Loves Linux and Microsoft Loves Open Source. I
am sure they will be the protectors we need </sarcasm>

------
tyingq
I'm skeptical about _" multi cloud eats cloud"_. If you're going to be boxed
into using a lowest common denominator of services, cloud has less value. You
might as well jump straight to hosted bare metal. The skill sets needed aren't
much different than multi-cloud, and it's much cheaper.

------
zby
The cloud is eating business relying on licensing. This is relevant for both
open and closed licenses. By the way I have wrote a post on that when the
MongoDB debate was raging: [https://medium.com/hackernoon/aws-and-mongo-and-
open-source-...](https://medium.com/hackernoon/aws-and-mongo-and-open-source-
efcdcfb00514). The way for Open Source to survive is to make it attractive for
cloud providers.

~~~
marktangotango
How does one make their project attractive to cloud providers? For the sake of
discussion; an infrastructure project like a job scheduler or some such?

~~~
zby
I was more thinking about making the licensing attractive to cloud providers.
The idea is that cloud providers are the dominant players - so for Open Source
to survive it needs to be adopted by them. But yeah - I don't know how to do
that. Licensing is not a core of their proposal, open sourcing their software
does not impact their revenue directly - but might indirectly by letting other
cloud providers compete with them using their own code.

------
siscia
I wonder if advancement in orchestrator will shift the balance. There is a lot
of effort into putting human knowledge into software, so that it can react to
unexpected situation and self heal. K8s operators to be specific.

If this software works well enough, what would be the competitive advantage of
AWS?

~~~
zzzcpan
K8s is not an advancement in orchestration and is very much on the AWS side in
this, deeply integrating with proprietary cloud services that do their own
operations and infrastructure, not even considering open sourcing those
services and their supporting infrastructure nor working towards alternatives
to replace them. It's just that the same thing that makes AWS piles of money
is the same thing the company behind k8s is after, so they are definitely not
going to advance k8s in a way that commoditizes AWS. Not that they are even
capable of doing so.

------
Andrex
> Pivotal got eaten by “Dockernetes” aka containers (ironically because Google
> was pissed off about Hadoop, but that is another story)

Would anyone care to give a brief summary? It's the first I've heard of this.

~~~
tyingq
I believe it's just saying that Google was disappointed that their MapReduce
research didn't give them any market presence or power. Hadoop took the
concepts and ran with it, with no direct benefit to Google. Pivotal benefited
a lot with their commerical distro of Hadoop.

Google didn't make the same mistake with K8S and it killed Pivotal's PCF.

~~~
jacques_chester
Hadoop looked to be a big part of Pivotal's overall strategy early on but
didn't pan out. Labs and Cloud Foundry have contributed the lion's share of
revenue for most of Pivotal's life, though data products like Greenplum,
Gemfire and RabbitMQ have also contributed a lot.

Kubernetes was never really about Pivotal in any universe I can think of. It
was about AWS.

Disclosure: I work at Pivotal.

------
lacker
People rarely mention in these debates that Bitcoin is open source, along with
essentially all other cryptocurrencies. Indeed, the cryptocurrency business
model could hardly exist without open source. It isn’t the same community at
all as Linux / Postgres / etc, but it certainly should be considered a great
success for open source.

------
gtirloni
This conflates open source with companies using open source as a competitive
advantage. For the latter, yes, cloud is eating their cake.

------
unraveller
Open source thought they could sell consulting and hosting services to
fatigued devs, if they fatigued them enough. All those single project focuses
left the door wide open for multi-clouds to "just add glue" between disparate
open source projects and actually aim to be easy.

We now await to see if compute will become a commodity or just priced that way
to prevent it becoming so. Is any challenger bold enough to bite off more of
the pie than they can chew?

------
mlinksva
This analysis seems correct if OSS = OSS ISVs.

------
beatpanda
I was hoping this article was going to be more like “who makes enough money to
eat working on Open Source Software?”

~~~
nojvek
Me too. I don’t think there are many people making enough money to eat working
on Open Source software. If you see GitHub, the largest most active projects
are funded by Mega Corps like Micro Face Goog.

I wonder how many companies are successful with open source but paid licence
for commercial use.

------
dandanio
"whom"... Who eats whom...

------
xwdv
Who eats closed source software?

~~~
rmoriz
The cloud, too.

Think of Office365, SAP Cloud Platform. It's benefical for a closed source
vendor to switch to a service model, collect monthly recurrent fees and get
free ultimate leverage against the customer.

------
orastor
Right now your webserver is eating your post

------
brodyprice
[https://web.archive.org/web/20191123091956/http://www.platfo...](https://web.archive.org/web/20191123091956/http://www.platformonomics.com/2019/11/dining-
preferences-of-the-cloud-and-open-source-who-eats-who/)

~~~
dang
That one's pretty clearly past the tipping point in usage, so I think we can
leave it above.

~~~
brodyprice
Agreed, sorry. I wanted to add the archive link but felt like being snarky for
some reason

