This is mostly a puff piece of revisionist history, as told by the person who wants you to think he started it all. No mention of Ben Black, Chris Pinkham, or Chris Brown, the team in South Africa, the memo to Bezos they wrote that actually started the idea, etc. (This side of the story is all readily Google-able if you're interested).
"Anyone who doesn’t do this will be fired. Thank you; have a nice day!"
2. AWS started in 2006 with EC2 (provision service through API) and S3 (some file storage). Simple, but primitive building blocks.
Google entered game early in 2008 with App Engine. Very powerful, but restricted Platform as a Service. It pretty much requires to rewrite whole application from scratch and doesn't let you do tons of stuff.
Eventually AWS is keep adding services with more functionality (SimpleDB, DynamoDB, SQS), while it took quite a while Google to realize that it needs to provide bare servers too. So both approaches converge, but Amazon capture magnitude more revenues from cloud infrastructure.
So, from the stories I hear; S3 came into being when an engineer in the Retail Org in Seattle ran out of disk storage and couldn't provision a server fast enough. He wrote a '6 pager' and pitched the idea to the S-team (SVPs reporting to Bezos back in the day) who then decided to pursue the business [0].
EC2 was, in some capacity, a tech borrowed from the Alexa (analytics) acquisition, which teams in Cape Town built upon.
SimpleDB (worked upon since 2005, and managed by people acquihired from Junglee.com), morphed into DynamoDB (started 2009-11) pretty immediately as they realised that it was going to be no where near enough.
Bezos personally pursued Werner Vogels to lead the web services initiative sometime in 2003. A lot of folks from the Retail Org, who solved problems of enormous complexities for Amazon.com, the website, got together to create AWS.
[0] Side note: Amazon's top hierarchy meets once every year and decides to invest in two new buinesses Amazon has never been involved in, or markets it can transform due to its involvement in multiple sectors... that's why there's so much happening at any given point in time at Amazon. It's really an incredible place if innovation is what drives you.
I worked as a system engineer there from 2001-2006. That all sounds a wee bit apocryphal. At the time S3 came into being Amazon was rather insanely jealous of Google and Google Mail's storage capacity and the Google File System whitepaper published in 2003. I can certainly believe that aggravation over not being able to provision storage may have been used to 'sell' that to the S-team and/or might have been the proverbial last straw, but those ideas were all in the air -- "we should build our own GoogleFS" was very common talk. The number of home-grown, shitty distributed filesystems that had no replication and N single points of failure caused all the SEs to pull their hair out -- "oh another royal clusterf*, hand built by yet another software dev team, and tossed over the proverbial chinese wall for us to maintain, when we had no input into the design, awesome..."
I also don't think that EC2 was borrowed from Alexa at all. There were political reasons why that development happened in Cape Town (it insulated them from Seattle politics and in particular from the VP who was over my head who hated the whole idea).
I think Alexa led to the Amazon search engine -- which i thought was called something like a6 (kind of like i18n is short for "internationalization" but with a6 being short for "amazon" -- but that doesn't quite sound right and I think I'm thinking of an Audi -- it was something like that though). They were in SF and setting up a satellite there to hire engineers away from Apple and Google. I also recall them doing some good work on SEO and fixing our URLs and such to get better search placement on Google which drove significant revenue. I don't think they had anything to do with EC2, but there could be some kernel behind the scenes that I wasn't aware of -- however it all smells wrong to me.
Oh, you can't just give us the set-up and not include the punchline like that. Here's a fuller version of it, in its original point form from Steve Yegge's[1] infamous rant:
...
6)Anyone who doesn’t do this will be fired.
7)Thank you; have a nice day!
Ha, ha! You 150-odd ex-Amazon folks here will of course realize immediately that #7 was a little joke I threw in, because Bezos most definitely does not give a shit about your day.
#6, however, was quite real, so people went to work...
"Google entered game early in 2008 with App Engine. Very powerful"
Empirical observations show that Google never managed to (or never tried to) make App Engine meet the general customers' needs for building their applications. Of cuz, that does not include Snapchat.
Is it because Google did not invest enough, or Google failed to educate and/or engage developers, or Google made a wrong assumption that App Engine would eventually catch up, there is no clear answer.
So overall, I do not think App Engine is 'powerful'. Sure, it has fancy feature and advanced designs. But that does not make something powerful. A powerful tool is one that help users do more with higher efficiency, or enable users to do things that otherwise is impossible or too costly. App Engine might be able to do that, but it was not proved to be true in general.
I'm always, always, amazed at how much of Amazon is below the surface of the general public's awareness. They're an iceberg. Sure, the top is huge; Amazon is, I guess, the world's biggest retailer. But, damn, what's under the water that few people see is just incredible. AWS is, by far, the most popular cloud service; it's the default, and hosts several other billion dollar businesses. Bezos has made good on his promise of making computing like the power company: You just plug in your stuff, and pay based on what you use.
I don't love any of the AWS APIs, but they got them out the door faster than anyone else (often by several years)...and they work. They can be clunky, but if you make the right incantations, you get the results you need and a limitless pool of resources, if you have the money.
But, more importantly, they've led rather than followed...because they knew what a service-based architecture needed to work, because they'd built one of the largest ones in the world before anybody else. So, as companies have grown on AWS, they've always found that Amazon had already thought of the growing pains they were going to run into and had already engineered solutions. So, Amazon is reading your mind, because they've been there and shared that particular pain. So, when you get to that crossroads, there's already an outpost with a note saying, "We went this way, here's a map and some supplies."
Yegge's rant on the subject is enlightening, but probably could have mostly been deduced from the outside without prior knowledge. Someone high up had to make the proclamation that Amazon would become SOA, at all costs, and someone had to make the call that it would be built to share, from the beginning. And, it's why so few companies have been able to catch up; only Google and Microsoft have come close, I think, and it's because they have tremendous resources, and some of the same internal forces at work.
Fair enough; not my area of expertise. They are the largest online retailer, though, right? (Walmart has an online division, but their online sales surely aren't comparable to Amazon.)
What started out as an idea to extend only Amazon's core capabilities as a service to other retailers transformed into a challenge to extend even the core of the core, which was the digital backbone infrastructure. Maybe this can be a tl;dr of the whole article.
Such business philosophies require companies to unselfishly give up developed competitive advantages and provide that as a service to the entire industry. Imagine if this same philosophy was implemented at other companies: the past-time heavyweights would be serving a far higher calling today. And entire industries of innovations of would have picked up and developed on where the old left off.
Examples like Intel offering the capabilities of its foundries or Kodak offering its patents to filmmaking or Microsoft offering others to build on its Office Doc format would have prevented both their fall and promoted the industries to greater heights.
You forget the it's implicit in Amazon's service(at least in the early ears) that no major competitor will use their services out of care for data privacy, reliability, etc. So Amazon gets the best of both worlds:leading eCommerce and leading cloud(while
"hurting" other cloud platforms who might help competitors).
Such strategy could also work well for others - if Intel offers fab services but doesn't sell to pc/server competitors ?
This reminds me of some open source angle on business: try to provide more value than you capture. I believe this to be a healthy attitude, hopefully more companies and entities can live up to this challenge.
Business doesn't last long if the only goal is profit and money. There has to be a meaning that transcends the financials. In terms of the businesses, the end goal was progress in the science and engineering these companies pioneered.
Missing from this article entirely is AWS's shrewd marketing and ability to define and grow a market.
As other commenters and the article noted, when AWS was launched, it just provided simple, primitive building blocks like compute (EC2) and object storage (S3). At the time, AWS's value proposition was entirely around elasticity of infrastructure and its archenemy was data centers. In the process, they created a new market that came to be called Infrastructure as a Service.
Since then, AWS has gone up the stack steadily, adding infrastructure application services like RDS, ELB, etc. These infrastructure applications have made it easier and faster to deploy applications on AWS, but unlike pure infrastructure, they come with a degree of vendor lock-in. But developers didn't blink at all because it was so much more convenient than upgrading your own MySQL or setting up an HAProxy.
And now, AWS is squarely in the applications business. Last year, they announced Quicksight, signaling to the world that they are going to squeeze out business intelligence vendors. Few know that Amazon has Microsoft Outlook/Gmail for Work competitor called WorkMail. Slowly but surely, AWS is trying to swallow the entire enterprise software stack. I noticed that most people no longer describe AWS as "IaaS" but as "cloud infrastructure." I.e., their scope and market is expanding.
Key to this success is their strategic prescience and marketing execution, both for the developers and the suits. They knew what made developers happy (free/cheap to start, great API and documentation, scales well) and took the time to build solid infrastructure. For the suits, they created a clear vision of "old v. new" and used customers like Netflix to convey how the new world is going to run on AWS.
As a keen observer and practitioner of enterprise software marketing, AWS's rise to dominance is a textbook example.
"They knew what made developers happy (free/cheap to start, great API and documentation, scales well) and took the time to build solid infrastructure."
I think most big tech companies know "what made developers happy". Microsoft did, Oracle did, Google did. The difference is that Amazon decided that they want to shared with the developers their secrets, and apparently many other companies do not think in that way.
> like many startups, when it launched in 1994, it didn’t really plan well for future requirements. Instead of an organized development environment, they had unknowingly created a jumbled mess.
Somehow this sums up many of the frustrations I've had working in software. I point out to people when they're making these kinds of decisions, and I call them on it when they romanticize it into an unhappy accident instead of a form of insidious, latent neglect.
I find it infinitely amusing when software developers become homeowners and complain about the exact same sorts of myopic shortcuts and idiotic decisions made by the former owners, without a hint of dissonance.
Michael Feathers and I are of an accord on many topics, including the use of salty language to describe untenable situations. He has books but I really have enjoyed some of his presentations and one-on-ones on other people's podcasts. Googlestalk him.
sigh, my secret worry is that Steve has been publishing stuff like this ever since his blog went silent, only now it's for Googlers only. His essays have been and still are so very much worth reading for anyone in the software field.
I wonder if Amazon will ever offer AFS (Amazon Fulfilment Service)? Imagine how much lower their costs of delivering goods from warehouse to customers would be. I understand this would push people away from listing on Amazon directly and there are some strategic/information issues but the cost reductions could be so dramatic that they outweigh this!
The ecommerce service was also known as ECS, but when the EC2 Container Service was named internally, only after naming was it discovered that ECS was a long ago service which still had its end points running for the people still using it. After a bit of fumbling around, we got a domain name that worked, but going through the history from the inside was fun!
What's missing from this is their fairly innovative product marketing decisions:
They put full descriptions of their products, very comprehensive, out on full display, including pricing.
Not only is pricing transparent, but it's not profit maximizing in the short run ... prices go down over time.
You can sign up with a credit card and get going.
Most companies, like Oracle - would have put this product behind a wall of idiot sales people and obfuscation.
I've been using AWS since the start and I've never needed support - not a bit.
This part of their operating paradigm should not be overlooked. It was a very gutsy decision to just throw everything over the fence and let 'whoever' use it.
We take this for granted now, but it could have been another way entirely.
Many other established companies still have not gotten the memo.
> Most companies, like Oracle - would have put this product behind a wall of idiot sales people and obfuscation.
This can be really important. I've often had conversations that went something like:
"How much will this cost to run?"
"On AWS, $76US/month plus something under $5/month in bandwidth"
"How much if we run it no Telstra's cloud"
"No idea - the webste says 'call your account manager'".
"Have you called them?"
"No. I don't have a Telstra account manager. Let me know if you want to sort that out - or if we should just get it all running this morning on Amazon."
Big dumb companies and government agencies pay the price.
Layers and layers of business people in between the actual product, nobody really sure what's going on... many of them are on 'cost plus' programs so they are incented to drive up cost.
I worked for (then) Spar aerospace, making the Space Shuttle robot arm, paid for mostly by NASA and Canadian gov. We paid through the teeth for everything. And then collected 10% on top. It's a perverse system. That said, it keeps a lot of people employed.
Oracle spends much more $ on sales than they do R&D, about 1/2 of their operation costs are 'sales' - meaning, that guy sitting there telling you about how great Oracle products are? 1/2 of your purchase price goes towards paying his salary.
'Business people' and 'sales people' are important components of the process, but when there is massive institutional gestation around these projects, it gets unwieldy very quickly.
AWS is a move in the right direction and should help drive efficiencies in a lot of places.
At the same time, Bezos became enamored with a book called Creation, by Steve Grand, the developer of a 1990s video game called Creatures that allowed players to guide and nurture a seemingly intelligent organism on their computer screens. Grand wrote that his approach to creating intelligent life was to focus on designing simple computational building blocks, called primitives, and then sit back and watch surprising behaviors emerge.
The book…helped to crystallize the debate over the problems with the company’s own infrastructure. If Amazon wanted to stimulate creativity among its developers, it shouldn’t try to guess what kind of services they might want; such guesses would be based on patterns of the past. Instead, it should be creating primitives — the building blocks of computing — and then getting out of the way. In other words, it needed to break its infrastructure down into the smallest, simplest atomic components and allow developers to freely access them with as much flexibility as possible.
Wasn't trying to take anything away, it's just a separate product is all I meant. Their tagline has always kinda been "Run on the same platform as amazon.com!"
My understanding is that AWS was built from the ground up and is not a fork of the actual code that runs Amazon. This is apparent in that Amazon has internal competitors to DynamoDB and SWF. I'm sure there are other examples.
To add to that, many of the older backend systems are still running Oracle, although they are now pushing to migrate to noSQL solutions. Dogfooding is generally encouraged but not required internally.
Bingo, also lots of examples of teams trying to build things for internal Amazon that will then be "easily" externalized as AWS services... and failing.
AWS is incredibly convenient, but it's priced at a premium. The main cost liability to building on AWS is the cost of bandwidth is insane, 5X the cost of DigitalOcean or Linode, 10X the cost of co-location.
Of course this depends on the type of startup you are building, a video startup is gonna be crazy expensive on AWS, Non-media heavy SAAS it may not matter.
Of course you can always build on AWS and place a caching layer in front of your bandwidth heavy media using DigitalOcean boxes to save big.
I agree, the AWS free tier is incredible for getting a project off the ground. For a side project of mine, I'm spending a little over $10/mo on AWS:
- 2x 1GB instances (one free, one $10/mo)
- Free ELB to load-balance between them
- Free 1GB Postgres RDS
Then I spend less than a buck or two a month when I spin up a c4.large for a few minutes to compile a new AMI periodically. This would all cost at least 3-4x on something like Linode or Digital Ocean.
This comment breaks the HN guidelines, obviously, and I've already warned you recently not to do this (https://news.ycombinator.com/item?id=11784145). It doesn't matter whether the other person is breaking the rules, being provocative, or whatever. Please don't do it again or we'll end up having to ban you as well.
Thanks for your quick and honest answer. I was over-weighting the effect of AWS as the cash cow for revenues (60% of profits) and forgot about how Wall Street punishes earning misses (the eternal treadmill of expectations).
