
When AWS, Azure, or GCP Becomes the Competition - gk1
https://www.gkogan.co/blog/big-cloud/
======
vivekl
While AWS has ruffled a bunch of feathers with their Elasticsearch and Kafka
managed offerings which can be easily construed as attempts to steam roll the
respective open source-first entities, I am actually quite impressed by the
mechanism that Azure has employed with Azure Managed Applications:
[https://docs.microsoft.com/en-us/azure/managed-
applications/...](https://docs.microsoft.com/en-us/azure/managed-
applications/overview). Hashicorp recently announced their collaboration with
Azure to bring forth managed consul offering using this:
[https://www.hashicorp.com/blog/announcing-consul-service-
on-...](https://www.hashicorp.com/blog/announcing-consul-service-on-azure/)

This in my opinion is the right way to solve the problem, i.e. provide the
customers that want a managed offering the means to get a managed version
directly from the entity most active behind the project, (e.g. Hashicorp)
while getting the most from the cloud provider's infrastructure. I would be
willing to trust Confluent or Hashicorp with operating/managing my Kafka or
consul cluster but taking another dependency on their respective cloud
offerings should be of concern.

I am quite surprised that its Azure that has shown creativity here while AWS
and Google have limited themselves to offering marketplace AMIs/images at
ridiculous markups.

~~~
a13n
> While AWS has ruffled a bunch of feathers with their Elasticsearch and Kafka
> managed offerings

Also DynamoDB (vs MongoDB)

~~~
ccbean
Instead of DynamoDB, perhaps you meant DocumentDB which is AWS's managed
MongoDB compatible database service.

~~~
a13n
Whoops, right! Hard to keep track of all those AWS services

------
RcouF1uZ4gsC
> Think Twice Before Open-Sourcing

I think it is a problem that it is very hard to monetize open source
infrastructure libraries. In the past, you could try to do consulting or else
offer paid hosting. Now with the cloud providers doing "zero management"
hosting of open source libraries, both those avenues have been seriously
curtailed.

Honestly, it seems like if you are the primary author of a popular open source
library, your best bet is try to leverage that notoriety into getting hired by
one of the cloud companies, and try to work on the open source library as a
side project. Trying to use that open source library to support yourself any
other way, seems destined for penury and misery.

~~~
jcadam
These days, I wouldn't open source anything other than a small hobby project
or utility library/app.

~~~
jdc
Well.. there is the AGPL.

------
m_ke
If you're doing anything ML related your best bet is to niche down and solve a
very narrow problem for a specific industry / market segment. Building generic
AI tooling sounds great but rarely solves anyones problems.

~~~
mlthoughts2018
As an ML practitioner in a large ecommerce company, I totally agree. When I
evaluate third party tools, 99% have no use at all for me. Things like Algolia
or Rekognition or Clarifai.

Even Sagemaker and Fargate are mostly not useful to me as my company can
easily operate its own k8s platform.

Open source tools for model building are great and a small team of engineers
can write whatever odds & ends that aren’t covered, or make scalable
implementations.

Notebook platforms like Databricks are similarly useless. A notebook is not a
frontend to a cluster and notebooks are universally poor development
environments even for the very tasks they are advertised.

The main thing that matters for ML development is how quickly & easily you can
put an arbitrary container or VM into a deployment environment. You’re always
going to need to rapidly change your container or VM, have common ancestors
that get specialized to support GPU training or some web server application
layer. Making the shortest possible distance between the definition of this
environment and deployment of it is the whole game.

The type of tools I’m willing to pay for are specialized database engines,
rapid data annotation tools, and specialized hardware environments.

I have no time for some “platform” tool for data provenance, notebook
environments, model tracking, model-as-a-service APIs like Rekognition, or
anything where I upload data and get back a model.

~~~
tixocloud
Could you care to elaborate further on specialized database engines and rapid
data annotation tools? There are plenty of data annotation tools out there
that seem to do the job so not sure what rapid would mean in this instance.

~~~
mlthoughts2018
I mean things like vertica or kdb+ that have specialized performance
properties for some use cases. Also to some minor extent managed cluster
pipeline tooling like Spark. I don’t mind paying for managed versions of these
(not Databricks though). For annotation tools I mean things like Prodigy.

~~~
tixocloud
Thanks. Why the interest in managed cluster pipeline but not Databricks? They
seem to be the big name in the game for Spark. Would some tooling around
snapshotting, sequential AB testing, staged rollouts, ghost models be of use?

~~~
mlthoughts2018
Databricks is a notebook frontend to cluster computing. Cluster computing is
useful and I’m willing to pay for it as long as I have total & complete
control of the environments and tooling used in the cluster workflow.

A notebook frontend however is less than useless, and is actively harmful by
propagating poor notebook environments even further into aspects of computing
where they cause harm and hurt reproducability and code factoring.

Given this, even if Databricks offered perfectly complete features for all
aspects of cluster computing, it would still be inferior to just my own
managed EC2 or EMR clusters or equivalent with other providers, where there is
no “notebook as control plane” garbage.

But when you add to that the fact that Databricks lacks full features for me
to totally own every customized detail of my cluster environment (e.g. how can
I run plain Python multiprocessing tasks with zero Spark in Databricks? How
can I bring my own custom defined GPU container with custom compiled
Tensorflow?) it makes the deal even worse.

Databricks is just another Spark / Hadoop style snake oil seller banking on
capturing a bunch of data science teams before people realize that it’s a
conceptually junk way to work.

As for the other tooling you mention, I’d almost always say to build it in
house. For example, I don’t know of a single A/B testing provider that
actually uses frequentist sequential testing to correctly avoid early stopping
bias. You actually need real statisticians hired in-house to solve these
problems, and the engineering work to set up an extensible A/B test as a
service internal tool is not bad. (I’ve built Bayesian A/B test frameworks
with teams of 2-3 engineers in 3 different companies). It’s just not cost
effective to outsource it on the false hopes of not needing to hire your own
in-house statistics experts. Just pony up the dough and hire them.

------
mrskitch
I think a lot of this depends on what your product or audience is. If you're a
domain-expert in your niche then lots of folks would rather go with you since
they'll have _you_ in their corner versus someone just trying to get it
running half-assed. There's also the "you're not amazon so we're going with
you" mentality amongst a lot of folks out there. Simply not being one of these
giants can be a differentiator, especially for folks outside the US.

My anecdotal story is when GCP announced support for running chromium out-of-
the-box. I thought that that would be the end of browserless.io. I held back
my knee-jerk reaction to shutdown, and after a few months I believe we only
lost a single customer. There's probably been a few others that we've lost
along the way since the announcement, but it didn't seem to stifle our growth
in any significant way.

In any case it really depends on what your product or service is. Just wanted
to chime-in and say that the writing isn't necessarily on the wall if this
happens to you. Stay the course and offer them the much-needed competition.

~~~
pm90
Support is where the money is at. Every major cloud vendor has garbage support
and smaller, niche companies value their customers more, especially if you're
not a huge account.

Also, If I'm using your service, and a competitor announces a similar service,
unless they provide something that's much much cheaper, I will probably not
switch.

------
taf2
Sounds like a lot of free marketing! If you're well positioned and a bigger
company enters your space take it for what it is they're advertising to people
in your space. Sure some will just default to one of them but most will shop
around and because these guys are going to be dumping money into your space -
you'll see more eyes on your product. Be on your A game it'll be good for you.
Maybe they'll acquire you because you do it better.

------
aynsof
It misses a couple of big ones:

1\. Offer a great user experience. Many cloud providers are just pumping
products out the door with little to no concern for UX.

2\. Have great personality. That could be through offering outstanding
support, or through being the kind of company that people root for.
(Honeycomb.io is a good example of this.)

~~~
hinkley
Of the many people who hated Microsoft during their rise to power, one of the
most coherent groups were people who were mad because MS put a competitor with
an arguably better product out of business.

That is in fact how FUD got into the vernacular. They would make vague
promises that you couldn't quite call outright lies about having a product
coming in that market, and a year later they'd put out something tepid
(thereby proving they were lying about their progress earlier), but with some
bulk licensing deal.

By the time the MS product came out the competitor would already be
experiencing reduced sales growth because people would play wait-and-see for
the MS product.

There is nothing stopping any of these folks from pulling the same trick.
Having a better product will not save you. A brilliant product might, but even
that's not a given.

~~~
user5994461
Who is gonna play wait-and-see in this day and age? We're not in the 90's
where one had to wait for the product to be announced, then built, then
shipped on a CD.

~~~
adrianmsmith
If a product has lots of integrations within your company, or if you store
lots of data in it which would need to be migrated to another product (e.g. a
database), or if lots of people in your company need to receive training for
the product, then it might make sense to wait 6 months to see what product the
big player is going to come out with, rather than choosing a smaller niche
vendor.

------
elamje
I wonder if this is a driving force behind why Cognitect has kept Datomic
closed source. I’ve seen people rally against Datomic more than once for that
reason, but honestly, how can you open-source something truly unique and
monetize without a large risk of cloud provider hijacking.

~~~
dustingetz
RICH HICKEY: To those who think that Datomic ought to be open source: We don't
see a viable economic model there.

The source is an r/clojure flamewar which I'm not going to link.

------
gk1
There are many more examples that I did not include. Like Redis (and AWS
Elasticache), MongoDB, Cassandra, Akamai (and AWS Cloudfront), etc.

Any others?

~~~
reilly3000
Timescale/Influx, Neptune/multiple graphdbs, activeMQ/MQ, Kafka, Kubernetes,
Hadoop/EMR, Athena/presto, Spark/Glue+Kinesis, Iron.io, even CloudWatch log
insights eating other log platforms.

------
manigandham
Uses want to pay for (managed) services, not software. The rise of AWS and
cloud and downfall of on-prem licensing models over the last decade has proven
this beyond any doubt.

It's completely a failure of vendors to keep trying to sell software instead
of competing on the services. The ones who have rolled out their own offerings
are generating plenty of revenue, but the overall breadth, flexibility and
quality of is still rather poor. I don't understand why these vendors are
moving slowly here. They're the best at running their own software. Writing
some blog posts isn't going to fix anything.

------
sammanna
This is a good post from a16z -

[https://a16z.com/2019/10/04/commercializing-open-
source/](https://a16z.com/2019/10/04/commercializing-open-source/)

"As software has eaten the world, open source is eating software.

Today, almost every major technology company, from Facebook to Google, is
written on the backs of open source software. Increasingly, these companies
are building their own open source projects as well – Airbnb, for example, has
more than 30 open source projects, and Google more than 2000!

In the future, the virtuous cycle will continue. Technologically, AI, open
source data, and block chain are some examples of emerging innovations. The
next generation of business models may include ad-supported OSS, as when a
large proprietary enterprise supports open source projects; data-driven
revenue; and crypto tokens, which monetize blockchain.

I believe Open Source 3.0 will expand how we think of and define open source
businesses. Open source will no longer be RedHat, Elastic, Databricks, and
Cloudera; it will be – at least in part – Facebook, Airbnb, Google, and any
other business that has open source as a key part of its stack. When we look
at open source this way, then the renaissance underway may only be in its
infancy. The market and possibilities for open source software are far greater
than we have yet realized."

------
known
[http://www.joelonsoftware.com/articles/fog0000000052.html](http://www.joelonsoftware.com/articles/fog0000000052.html)
can help Win Against Big Cloud

------
avaika
Wait, isn't it like any other market which already has long lasting players?

If you want no competition -- find a new market (and win it). Otherwise you
have to be smarter and faster than existing players.

~~~
cj
You can probably draw a parallel between the retail sector and the phenomenon
discussed in the article. For example:

Decades ago (before Walmart, Target, etc), it was common to have many small
specialized stores that did just 1 thing really well (ie. selling just books,
selling just shoes, selling just office supplies). Over time, Walmart/Target
came along and bundled all of those specialty stores into 1 giant store that
sells everything, causing a lot of the small specialty stores to eventually
fade away.

AWS/GCP/Azure is sort of like the "Walmart of B2B Internet Services" in that
they are in direct competition with companies like MongoDB, Akamai, and the
other examples from the article re: offering comparable managed services.

As a market matures, the market consolidates into a small number a big
players. To survive, specialty shops need to do things that the big players
won't or can't, which I think the article does a good job discussing.

------
9nGQluzmnq3M
The other option is to partner up with Big Cloud. They all offer marketplace
programs that let customers install third-party software quickly and easily,
and take care of the billing too (for a price, of course).

~~~
gk1
I address this in the article. Unfortunately (for the startup) being in one of
the Big Cloud partner programs does not prevent them from launching directly
competitive products.

~~~
bern4444
It probably only makes it easier for them as well since they can see what add
ons/tools are the most consumed. Then they can use that data to figure out
what features they should build next for their tools

------
hmart
There will not be another Dropbox.

~~~
mac01021
What does that mean?

------
sdan
Is there anyway to create your own AWS/Azure/GCP on your own server?

~~~
Redsquare
Azure has Azure Stack, you need a deep wallet [https://azure.microsoft.com/en-
gb/overview/azure-stack/](https://azure.microsoft.com/en-gb/overview/azure-
stack/)

~~~
sdan
Interesting, because I was seriously thinking of building a product where you
get a good amount of features from these cloud providers and run it on your
own servers... although obviously there's going to be some drawbacks, you may
save a lot of money.

~~~
ceejayoz
That's [https://www.openstack.org/](https://www.openstack.org/).

~~~
sdan
Oh. Haha. Looks like I learned something new today. Thanks for the info!

------
shrimpi
This one is easy: patents

------
gawi
Nice plug in the middle of middle of the article, exactly at the right spot:

> By the way, I write an article like this every month or so, covering lessons
> learned from growing B2B software startups. Get an email update when the
> next one is published

This is the closest I got to subscribing to a newsletter.

~~~
gk1
> This is the closest I got to subscribing to a newsletter.

Almost got you, huh? :)

I've A/B tested various locations and the inline-embed is the best, by far.
This is also why many news sites now include related stories in the middle of
the article, not at the end.

~~~
mey
Wish the rss/atom was better represented, rather than only existing in
metadata on your site. I must be a dying breed, but I refuse to sign up for
email newsletters. I refuse to hand over that much of my attention.

Edit: [https://www.gkogan.co/feed.xml](https://www.gkogan.co/feed.xml)

~~~
alexis_fr
That’s interesting. So RSS didn’t stick because it didn’t allow advertisers to
harvest email addresses.

Which is stupid because you still can’t reach to your audience if you just buy
harvested addresses, so if it’s just people who subscribed to your newsletter,
you’re just better off letting them subscribe to your RSS. Perhaps the
quantifiability of newsletters vs RSS makes email win. If only RSS had
embedded ways to count views and see what users looked at.

------
caniszczyk
Terrible blog post, they completely forget to mention Mesos which owned the
orchestration market before Docker Swarm and Kubernetes

~~~
elamje
Yikes, that’s a little harsh considering the good discussion going on before
and after your comment.

~~~
caniszczyk
There is good discussion but there is a lot of hyperbole, how many companies
can you name that have been killed by these hyperscale cloud providers?

Many seem to be doing well: [https://a16z.com/2019/10/04/commercializing-open-
source/](https://a16z.com/2019/10/04/commercializing-open-source/)

Some just may not be worth their oversized valuations in the long run but made
choices to be valued much larger than their actual worth.

