
How AWS came to be - codehusker
https://techcrunch.com/2016/07/02/andy-jassys-brief-history-of-the-genesis-of-aws/
======
parasubvert
This is mostly a puff piece of revisionist history, as told by the person who
wants you to think he started it all. No mention of Ben Black, Chris Pinkham,
or Chris Brown, the team in South Africa, the memo to Bezos they wrote that
actually started the idea, etc. (This side of the story is all readily Google-
able if you're interested).

~~~
deanCommie
The very first link in the very first sentence of the article mentions most of
these: [http://www.networkworld.com/article/2891297/cloud-
computing/...](http://www.networkworld.com/article/2891297/cloud-
computing/the-myth-about-how-amazon-s-web-service-started-just-won-t-die.html)

They're not attempting to provide the definitive history, just some new info
from Jassy

------
jakozaur
Would add two points:

1\. Internal Jeff Bezos memo from around 2002 that everything should be a web
service, which eventually can be externalize:
[http://apievangelist.com/2012/01/12/the-secret-to-amazons-
su...](http://apievangelist.com/2012/01/12/the-secret-to-amazons-success-
internal-apis/)

"Anyone who doesn’t do this will be fired. Thank you; have a nice day!"

2\. AWS started in 2006 with EC2 (provision service through API) and S3 (some
file storage). Simple, but primitive building blocks.

Google entered game early in 2008 with App Engine. Very powerful, but
restricted Platform as a Service. It pretty much requires to rewrite whole
application from scratch and doesn't let you do tons of stuff.

Eventually AWS is keep adding services with more functionality (SimpleDB,
DynamoDB, SQS), while it took quite a while Google to realize that it needs to
provide bare servers too. So both approaches converge, but Amazon capture
magnitude more revenues from cloud infrastructure.

~~~
ignoramous
So, from the stories I hear; S3 came into being when an engineer in the Retail
Org in Seattle ran out of disk storage and couldn't provision a server fast
enough. He wrote a '6 pager' and pitched the idea to the S-team (SVPs
reporting to Bezos back in the day) who then decided to pursue the business
[0].

EC2 was, in some capacity, a tech borrowed from the Alexa (analytics)
acquisition, which teams in Cape Town built upon.

SimpleDB (worked upon since 2005, and managed by people acquihired from
Junglee.com), morphed into DynamoDB (started 2009-11) pretty immediately as
they realised that it was going to be no where near enough.

Bezos personally pursued Werner Vogels to lead the web services initiative
sometime in 2003. A lot of folks from the Retail Org, who solved problems of
enormous complexities for Amazon.com, the website, got together to create AWS.

[0] Side note: Amazon's top hierarchy meets once every year and decides to
invest in two new buinesses Amazon has never been involved in, or markets it
can transform due to its involvement in multiple sectors... that's why there's
so much happening at any given point in time at Amazon. It's really an
incredible place if innovation is what drives you.

~~~
lamontcg
I worked as a system engineer there from 2001-2006. That all sounds a wee bit
apocryphal. At the time S3 came into being Amazon was rather insanely jealous
of Google and Google Mail's storage capacity and the Google File System
whitepaper published in 2003. I can certainly believe that aggravation over
not being able to provision storage may have been used to 'sell' that to the
S-team and/or might have been the proverbial last straw, but those ideas were
all in the air -- "we should build our own GoogleFS" was very common talk. The
number of home-grown, shitty distributed filesystems that had no replication
and N single points of failure caused all the SEs to pull their hair out --
"oh another royal clusterf __*, hand built by yet another software dev team,
and tossed over the proverbial chinese wall for us to maintain, when we had no
input into the design, awesome... "

I also don't think that EC2 was borrowed from Alexa at all. There were
political reasons why that development happened in Cape Town (it insulated
them from Seattle politics and in particular from the VP who was over my head
who hated the whole idea).

I think Alexa led to the Amazon search engine -- which i thought was called
something like a6 (kind of like i18n is short for "internationalization" but
with a6 being short for "amazon" \-- but that doesn't quite sound right and I
think I'm thinking of an Audi -- it was something like that though). They were
in SF and setting up a satellite there to hire engineers away from Apple and
Google. I also recall them doing some good work on SEO and fixing our URLs and
such to get better search placement on Google which drove significant revenue.
I don't think they had anything to do with EC2, but there could be some kernel
behind the scenes that I wasn't aware of -- however it all smells wrong to me.

~~~
dannylandau
I though it was called A9 -- [https://www.a9.com/](https://www.a9.com/)

~~~
lamontcg
ah right it was (or 'is' i guess) short for "amazon.com" but with the last 9
characters deleted, including the period...

~~~
dastbe
It is short for algorithm.

~~~
lamontcg
That'd be a8

------
SwellJoe
I'm always, _always_ , amazed at how much of Amazon is below the surface of
the general public's awareness. They're an iceberg. Sure, the top is huge;
Amazon is, I guess, the world's biggest retailer. But, damn, what's under the
water that few people see is just incredible. AWS is, by far, the most popular
cloud service; it's the default, and hosts several other billion dollar
businesses. Bezos has made good on his promise of making computing like the
power company: You just plug in your stuff, and pay based on what you use.

I don't love any of the AWS APIs, but they got them out the door faster than
anyone else (often by several years)...and they _work_. They can be clunky,
but if you make the right incantations, you get the results you need and a
limitless pool of resources, if you have the money.

But, more importantly, they've led rather than followed...because they knew
what a service-based architecture needed to work, because they'd built one of
the largest ones in the world before anybody else. So, as companies have grown
on AWS, they've always found that Amazon had already thought of the growing
pains they were going to run into and had already engineered solutions. So,
Amazon is reading your mind, because they've been there and shared that
particular pain. So, when you get to that crossroads, there's already an
outpost with a note saying, "We went this way, here's a map and some
supplies."

Yegge's rant on the subject is enlightening, but probably could have mostly
been deduced from the outside without prior knowledge. Someone high up had to
make the proclamation that Amazon would become SOA, at all costs, and someone
had to make the call that it would be built to share, from the beginning. And,
it's why so few companies have been able to catch up; only Google and
Microsoft have come close, I think, and it's because they have tremendous
resources, and some of the same internal forces at work.

~~~
bruce_one
I don't think they are the world's biggest retailer?

[http://www.forbes.com/sites/laurengensler/2016/05/27/global-...](http://www.forbes.com/sites/laurengensler/2016/05/27/global-2000-worlds-
largest-retailers/)

(It might depend on metric used :-) )

~~~
SwellJoe
Fair enough; not my area of expertise. They are the largest online retailer,
though, right? (Walmart has an online division, but their online sales surely
aren't comparable to Amazon.)

------
nashashmi
What started out as an idea to extend only Amazon's core capabilities as a
service to other retailers transformed into a challenge to extend even the
core of the core, which was the digital backbone infrastructure. Maybe this
can be a tl;dr of the whole article.

Such business philosophies require companies to unselfishly give up developed
competitive advantages and provide that as a service to the entire industry.
Imagine if this same philosophy was implemented at other companies: the past-
time heavyweights would be serving a far higher calling today. And entire
industries of innovations of would have picked up and developed on where the
old left off.

Examples like Intel offering the capabilities of its foundries or Kodak
offering its patents to filmmaking or Microsoft offering others to build on
its Office Doc format would have prevented both their fall and promoted the
industries to greater heights.

~~~
rpgmaker
> the past-time heavyweights would be serving a far higher calling today

Higher calling? It's a business dude, plain and simple.

~~~
nashashmi
Business doesn't last long if the only goal is profit and money. There has to
be a meaning that transcends the financials. In terms of the businesses, the
end goal was progress in the science and engineering these companies
pioneered.

~~~
quesera
That sounds nice, but it's empirically untrue.

------
ktamura
Missing from this article entirely is AWS's shrewd marketing and ability to
define and grow a market.

As other commenters and the article noted, when AWS was launched, it just
provided simple, primitive building blocks like compute (EC2) and object
storage (S3). At the time, AWS's value proposition was entirely around
elasticity of infrastructure and its archenemy was data centers. In the
process, they created a new market that came to be called Infrastructure as a
Service.

Since then, AWS has gone up the stack steadily, adding infrastructure
application services like RDS, ELB, etc. These infrastructure applications
have made it easier and faster to deploy applications on AWS, but unlike pure
infrastructure, they come with a degree of vendor lock-in. But developers
didn't blink at all because it was so much more convenient than upgrading your
own MySQL or setting up an HAProxy.

And now, AWS is squarely in the applications business. Last year, they
announced Quicksight, signaling to the world that they are going to squeeze
out business intelligence vendors. Few know that Amazon has Microsoft
Outlook/Gmail for Work competitor called WorkMail. Slowly but surely, AWS is
trying to swallow the entire enterprise software stack. I noticed that most
people no longer describe AWS as "IaaS" but as "cloud infrastructure." I.e.,
their scope and market is expanding.

Key to this success is their strategic prescience and marketing execution,
both for the developers and the suits. They knew what made developers happy
(free/cheap to start, great API and documentation, scales well) and took the
time to build solid infrastructure. For the suits, they created a clear vision
of "old v. new" and used customers like Netflix to convey how the new world is
going to run on AWS.

As a keen observer and practitioner of enterprise software marketing, AWS's
rise to dominance is a textbook example.

~~~
justicezyx
"They knew what made developers happy (free/cheap to start, great API and
documentation, scales well) and took the time to build solid infrastructure."

I think most big tech companies know "what made developers happy". Microsoft
did, Oracle did, Google did. The difference is that Amazon decided that they
want to shared with the developers their secrets, and apparently many other
companies do not think in that way.

------
hinkley
> like many startups, when it launched in 1994, it didn’t really plan well for
> future requirements. Instead of an organized development environment, they
> had unknowingly created a jumbled mess.

Somehow this sums up many of the frustrations I've had working in software. I
point out to people when they're making these kinds of decisions, and I call
them on it when they romanticize it into an unhappy accident instead of a form
of insidious, latent neglect.

I find it infinitely amusing when software developers become homeowners and
complain about the exact same sorts of myopic shortcuts and idiotic decisions
made by the former owners, without a hint of dissonance.

~~~
sidcool
I would like to hear your experiences with software.

~~~
hinkley
Michael Feathers and I are of an accord on many topics, including the use of
salty language to describe untenable situations. He has books but I really
have enjoyed some of his presentations and one-on-ones on other people's
podcasts. Googlestalk him.

------
mblode
This reminds me of Steve Yegge's rant about Amazon and Google. Talking about
Accessibility and Platforms.

[https://gist.github.com/chitchcock/1281611](https://gist.github.com/chitchcock/1281611)

~~~
dmit
Follow-up:
[https://plus.google.com/110981030061712822816/posts/AaygmbzV...](https://plus.google.com/110981030061712822816/posts/AaygmbzVeRq)

------
dexterdog
How is Werner Vogels not even mentioned in this article? I know he wasn't at
Amazon in 2003 when the discussions began, but he was within a year.

~~~
chaostheory
I'm guessing because it's hard to keep track of all of that history, even
though it's available in multiple places.

I'm one of the few crazy people trying to do it.

[http://www.theymadethat.com/things/dd846efa-9679-4c16-ad96-6...](http://www.theymadethat.com/things/dd846efa-9679-4c16-ad96-6375e1b189be/amazon-
web-services)

Feel free to join me =)

(Constructive criticism is also welcome)

------
oli5679
I wonder if Amazon will ever offer AFS (Amazon Fulfilment Service)? Imagine
how much lower their costs of delivering goods from warehouse to customers
would be. I understand this would push people away from listing on Amazon
directly and there are some strategic/information issues but the cost
reductions could be so dramatic that they outweigh this!

~~~
dlgeek
[https://services.amazon.com/fulfillment-by-amazon/multi-
chan...](https://services.amazon.com/fulfillment-by-amazon/multi-channel.htm)
\- been there for a long time.

------
timf
Alexa, SQS, and the "E-Commerce Service" were the first services announced for
beta testing and what made up the first set of services billed as AWS.

Have a look at the AWS page from 2005, pre-S3:

[https://web.archive.org/web/20050504101424/http://www.amazon...](https://web.archive.org/web/20050504101424/http://www.amazon.com/gp/browse.html?node=3435361&)

~~~
jhspaybar
The ecommerce service was also known as ECS, but when the EC2 Container
Service was named internally, only after naming was it discovered that ECS was
a long ago service which still had its end points running for the people still
using it. After a bit of fumbling around, we got a domain name that worked,
but going through the history from the inside was fun!

------
nevir
Holy FUCK TechCrunch's mobile site is frustrating to use.

Tap on "read full article" \- NOPE, you just clicked on an ad instead.

('cause it relayouts at some point after the page loads and right around the
time you want to tap on that)

~~~
chinathrow
That is made on purpose. Welcome to clickjacking.

------
jomamaxx
What's missing from this is their fairly innovative product marketing
decisions:

They put full descriptions of their products, very comprehensive, out on full
display, including pricing.

Not only is pricing transparent, but it's not profit maximizing in the short
run ... prices go down over time.

You can sign up with a credit card and get going.

Most companies, like Oracle - would have put this product behind a wall of
idiot sales people and obfuscation.

I've been using AWS since the start and I've never needed support - not a bit.

This part of their operating paradigm should not be overlooked. It was a very
gutsy decision to just throw everything over the fence and let 'whoever' use
it.

We take this for granted now, but it could have been another way entirely.

Many other established companies still have not gotten the memo.

~~~
bigiain
> Most companies, like Oracle - would have put this product behind a wall of
> idiot sales people and obfuscation.

This can be really important. I've often had conversations that went something
like:

"How much will this cost to run?"

"On AWS, $76US/month plus something under $5/month in bandwidth"

"How much if we run it no Telstra's cloud"

"No idea - the webste says 'call your account manager'".

"Have you called them?"

"No. I don't have a Telstra account manager. Let me know if you want to sort
that out - or if we should just get it all running this morning on Amazon."

~~~
jomamaxx
Big dumb companies and government agencies pay the price.

Layers and layers of business people in between the actual product, nobody
really sure what's going on... many of them are on 'cost plus' programs so
they are incented to drive up cost.

I worked for (then) Spar aerospace, making the Space Shuttle robot arm, paid
for mostly by NASA and Canadian gov. We paid through the teeth for everything.
And then collected 10% on top. It's a perverse system. That said, it keeps a
lot of people employed.

Oracle spends much more $ on sales than they do R&D, about 1/2 of their
operation costs are 'sales' \- meaning, that guy sitting there telling you
about how great Oracle products are? 1/2 of your purchase price goes towards
paying his salary.

'Business people' and 'sales people' are important components of the process,
but when there is massive institutional gestation around these projects, it
gets unwieldy very quickly.

AWS is a move in the right direction and should help drive efficiencies in a
lot of places.

------
juliangamble
_At the same time, Bezos became enamored with a book called Creation, by Steve
Grand, the developer of a 1990s video game called Creatures that allowed
players to guide and nurture a seemingly intelligent organism on their
computer screens. Grand wrote that his approach to creating intelligent life
was to focus on designing simple computational building blocks, called
primitives, and then sit back and watch surprising behaviors emerge._

 _The book…helped to crystallize the debate over the problems with the
company’s own infrastructure. If Amazon wanted to stimulate creativity among
its developers, it shouldn’t try to guess what kind of services they might
want; such guesses would be based on patterns of the past. Instead, it should
be creating primitives — the building blocks of computing — and then getting
out of the way. In other words, it needed to break its infrastructure down
into the smallest, simplest atomic components and allow developers to freely
access them with as much flexibility as possible._

[https://stratechery.com/2016/the-amazon-
tax/](https://stratechery.com/2016/the-amazon-tax/)

------
ww520
AWS became successful because Amazon could massively dogfood its own services
as the biggest online merchant.

~~~
serge2k
The biggest myth about amazon (that they love to push) is that AWS is just the
retail stuff externalized/we run on aws.

~~~
bckygldstn
What makes you say that? The post linked to above suggests that's exactly how
AWS was started. Is it no longer the case?

[https://plus.google.com/+RipRowan/posts/eVeouesvaVX](https://plus.google.com/+RipRowan/posts/eVeouesvaVX)

~~~
serge2k
THe culture certainly supports it.

Wasn't trying to take anything away, it's just a separate product is all I
meant. Their tagline has always kinda been "Run on the same platform as
amazon.com!"

~~~
beastcoast
My understanding is that AWS was built from the ground up and is not a fork of
the actual code that runs Amazon. This is apparent in that Amazon has internal
competitors to DynamoDB and SWF. I'm sure there are other examples.

To add to that, many of the older backend systems are still running Oracle,
although they are now pushing to migrate to noSQL solutions. Dogfooding is
generally encouraged but not required internally.

~~~
discodave
Bingo, also lots of examples of teams trying to build things for internal
Amazon that will then be "easily" externalized as AWS services... and failing.

~~~
beastcoast
I've seen that situation exactly.. Large product with SVP support that gets
built for AWS, (nearly) launched and then shot down by Jeff himself :)

------
albasha
Still, don't use AWS for your new startup. Move to it once you outgrow the
offers from Linode and DigitalOcean. You'll save a ton of money.

~~~
Johnie
As a startup, how is this cheaper than the AWS free tier?[1] Once you outgrow
the free tier, you'd also outgrow Linode and DigitalOcean.

[1] [https://aws.amazon.com/free/](https://aws.amazon.com/free/)

~~~
billmalarky
AWS is incredibly convenient, but it's priced at a premium. The main cost
liability to building on AWS is the cost of bandwidth is insane, 5X the cost
of DigitalOcean or Linode, 10X the cost of co-location.

Of course this depends on the type of startup you are building, a video
startup is gonna be crazy expensive on AWS, Non-media heavy SAAS it may not
matter.

Of course you can always build on AWS and place a caching layer in front of
your bandwidth heavy media using DigitalOcean boxes to save big.

~~~
webtechgal
> Of course you can always build on AWS and place a caching layer in front of
> your bandwidth heavy media using DigitalOcean boxes to save big.

This IS a brilliant idea!!

~~~
billmalarky
I like reactions like that to my suggestions!

I'm building a video startup so bandwidth solutions have been on my mind these
days.

------
wallflower
Slightly OT: At the current stock price and AWS growing 40% YOY, do you think
Amazon stock is a good buy and hold investment?

~~~
Spooky23
Amazon is a high-flying stock that is great until it isn't. If they have a
significant earning miss it other misstep, the stock will be punished.

It's a great company, but you need to understand the business and understand
what Wall St expectations are.

~~~
wallflower
Thanks for your quick and honest answer. I was over-weighting the effect of
AWS as the cash cow for revenues (60% of profits) and forgot about how Wall
Street punishes earning misses (the eternal treadmill of expectations).

