
A Comprehensive Guide to Building a Scalable Web App on AWS - jkresner
https://www.airpair.com/aws/posts/building-a-scalable-web-app-on-amazon-web-services-p1?wed
======
tieTYT
The scariest thing to me about AWS is that I might accidentally bankrupt
myself while I learn to use it. I've seen horror stories on HN before.
Articles titled "How I spent $32k with AWS, a for loop, and a simple typo" or
something like that.

Normally when I learn something new, I learn by tinkering and breaking stuff.
I don't feel comfortable doing that with AWS. I'm hoping people will tell me
I'm way off base because this fear has stopped me from getting the ball
rolling.

~~~
themartorana
I run a million-dollar business on it for $4500/m, give or take. It's not
Webfaction (who I love and highly recommend) or Linode. It's virtualized data-
center territory that can support some massive stacks.

They give you enough rope to hang yourself. I'm ok with that, because it works
brilliantly when you have the time to put in to it. (They also almost always
help out or completely forgive someone that accidentally ran up a massive bill
in a short period of time.)

Without AWS, my company wouldn't exist - I could never have initially afforded
dedicated hosting or collocation. AWS changed the game. You can argue they're
not the cheapest or even the best anymore, but it works this way because of
Amazon.

Edit: what's with the down-votes?

~~~
oafitupa
What's your point? What you said changes nothing for people like tieTYT and
me.

~~~
themartorana
If you want to play with VPS servers, use DO. If you want to virtualize a
data-center or series of globally-connected data centers with racks of
servers, use AWS.

So maybe AWS isn't the right solution for you. You say in another comment that
warnings about billing overages aren't enough. AWS can't hold your hand and
provide Netflix-capable services at the same time.

Amazon chose the latter. I would recommend you not choose AWS until you have
the time to dedicate to it.

~~~
oafitupa
How is being able to set up a $ limit "holding my hand"? You guys are
defending something that is indefensible.

I don't want to "play" with VPS servers, I want to work knowing that I won't
go bankrupt just because I'm using at the same time a service that won't let
me set up a limit, and a pull payment system (ie. credit cards, as opposed to
push systems like Bitcoin).

~~~
mentat
If you don't want to bankrupt yourself then check your usage every day or
hour. It's still your responsibility not your vendor's.

~~~
oafitupa
Like it or not this is a problem many of us have, and are avoiding AWS for
that sole reason. You can keep blaming the victim, or you can accept not being
able to set up a limit is a usability problem.

It's like saying "this website is stupid, it's storing passwords in
plaintext", and you answering "hurr, it's the user's responsibility to create
and manage cryptographically secure passwords".

~~~
jalfresi
You're not a victim! Youre inability to control your costs, given the tools,
alarms and general common sense doesn't entitle you to some magic off button!
Learn to use the tools appropriately and you won't incur unexpected costs.

And as an analogy, Aws is more like C; if you are willing to put the time in
to learn, you can do some amazing things that are impossible with other
providers. But you must take responsibility for them.

~~~
RemoteWorker
How is a simple configurable limite a "magic button"? What extreme technical
difficulties do you see in implementing it, that warranted you calling it
"magic"?

~~~
tobz
Because it would tear down the entire stack? If you're a real business,
depending on servers to be up and data to stick around, then it makes
absolutely no sense to have machines shut off and volumes deleted if you hit
some arbitrary marker. There's nothing you get for free in AWS (besides the
Free tier, and not many people are running their entire, highly profitable
business on that) and the only solution to "not spend more than $X per month"
is to literally shut down and delete things.

I'd love to hear use cases where legitimate businesses, who make money off of
the products or services they offer, can literally afford to have their
business just stop working. It sounds totally contrived.

~~~
spdustin
A CloudWatch alarm on a billing metric could also be used to send you SMS,
email, hell even call you if you want to wire a webhook up to a quick and
dirty twilio app (via SNS)

In fact, we use an SNS->Slack gateway running on a free Heroku dyno to get out
alerts in a Slack channel (which is pushed to phones/etc), along with other
CloudWatch alarms related to performance.

Honestly, this issue of "I don't have any visibility into what I'm spending"
is a waste of energy. You do, and you can have AWS bug you as intensely as you
want with updates as frequent and as urgent as you need

------
Rapzid
This article appears to include a lot of very good advice(speaking as an AWS
solutions architect). I might suggest a emphasising a few things such as not
having keys on login accounts(they negate multi-factor auth if leaked), and to
ALWAYS pick or create a new IAM role if you aren't sure an existing one fits
for the EC2 instances.. But perhaps this sort of advice is not appropriate for
the article.

Much respect for the amount of work that went into this. I'll try to get
through it all here at some point :)

------
jkresner
Apologies in advance for the table of contents going way off the screen. This
is the biggest post published so far. We'll be doing some UX work on the table
of contents widget in the next week.

~~~
subrat_rout
Josh. This article is great and very helpful. Also I would like to know if you
have any plan to write more articles on AWS that are more in depth.

Few examples such as: 1\. What are the security measures somebody needs to put
in place to host a HIPAA compliant Web app etc.

2\. What are the popular stacks that can be set up on AWS without much hassles
and with less expertise.

3\. What are day to day activities required to maintain few web apps on AWS.
etc

~~~
rwfilice
Agree would love to see the HIPAA compliant article.

------
artifaxx
Out of curiousity, why go with AWS when Linode, Digitalocean, etc appear to be
so much more cost effective? Is the simplicity of spinning up AWS instances
really great enough to counterbalance what appears to be a significantly
greater cost? Is it the flexibility of different AWS services?

~~~
joshpadnick
Article author here. So, you're right that while AWS does continue to lower
prices, they're still not the cheapest game in town. Frankly, they're not even
necessarily the most performant game in town.

What they are really competing on is breadth and depth of service. The article
goes into a lot of those services, but, as one example, if you launch an
instance in EC2 you can allow it to access secured buckets in S3 without any
need to store keys/passwords on the instance itself thanks to IAM roles.

Another example is services like AWS Lambda, which is a hosted way to run a
function without any need to manage servers.

The list goes on and on...direct VPN connectivity, Hosted Active Directory,
CloudHSM. While I'm biased, my perception is that AWS is pretty far ahead of
the pack.

~~~
artifaxx
Thanks for all the replies. I am just researching all of this for my own
startup and it is important to understand all the tradeoffs. And it is clear
from your article that AWS has a very deep feature set. Its a good article!

~~~
seunosewa
But do you need the features? Are they important enough to be worth getting
locked into an expensive ecosystem over?

Look at Reddit, they have very few employees yet they can't make a profit
because their hosting costs are massive.

Contrast them with StackExchange, which uses a small number of powerful, well-
optimized servers and is very profitable.

~~~
seunosewa
Actually, technically, StackExchange is not profitable yet:
([http://www.joelonsoftware.com/items/2015/01/20.html](http://www.joelonsoftware.com/items/2015/01/20.html)).
However, "We could just slow down our insane hiring pace and get profitable
right now, but it would mean foregoing some of the investments that let us
help more developers."

------
chacham15
These guides are really hard to read (at least for me) because there isnt
really a point (until the end) where I can stop and go try out some of what
I've learned. Perhaps, if it were formatted in the way of "steps to setup a
scalable web app on AWS" it would be more palatable.

------
jwilliams
This links to a HN discussion on the relative merits of VPN access versus
bastion/jump boxes
([https://news.ycombinator.com/item?id=8637154](https://news.ycombinator.com/item?id=8637154)).

This conversation didn't ever seem to bottom out to conclusion? In particular
I was wondering how servers connected _outwards_ in the VPN scenario.

~~~
maslam
We (Appuri) use VPCs exclusively. There are pros and cons. I'll try to list
the top:

Pros: \- Logical isolation. You can put instances (and RDS, Redshift) etc.
inside logical subnets that are not addressable from the outside world. \-
VPNs. If you really want extra security, you can wire up VPN so one of your
VPC subnets shows up on your corporate subnet.

Cons: \- A complete pain to manage with SSH-based tools. Most deployment tools
(Ansible, for example) and even lower-level tools like fleetctl don't play
well (if at all) with jump boxes. Example - Ansible Tower requires instances
that are publicly addressable OR placing a Tower instance inside a VPC (which
means we can't use it to manage multiple VPCs) \- We have had to write our own
workarounds for the above con. \- Complexity. There are more concepts to learn
about. \- Lack of portability. I don't know if all cloud providers (Azure, DO
etc.) even support VPCs the same way AWS does. This makes our infrastructure
less portable than I'd like

~~~
jwilliams
I think you'll find you can solve this via ssh config. Specifically using
ProxyCommand -- in the case of Ansible anyway. You can then ssh reference an
internal address.

I was asking less about VPCs in general, more the use of the VPN->VPC or
Bastion approach to bridge into that network.

------
chuckcode
Great to see article putting it all together. Also going to be great to send
this to people who think that using AWS cloud means that you can skip hiring
people with skills in systems administration.

------
AliCollins
Does anyone know of a similar article for the Google Cloud infrastructure?

------
skc
This is a great article.

I knew I'd see a bunch of people stating that AWS is expensive and you should
use a dedicated server or a VPS. But there are many applications built by
people like me who are lone developers or small teams of developers who either
don't have the admin skills or simply don't want to admin their own servers
and the fact that AWS handles quite a lot of this for you is sometimes worth
the the added cost.

------
minikomi
Amazing post! Thank you very much. Would love to be able to read this on
kindle - you should make an epub! haha

------
jacques_chester
Missing from the list of PaaS offerings is Cloud Foundry, which already runs
on AWS.

Obvious disclaimer: I work on CF for my dayjob.

------
lazyfunctor
Nice post! Thanks for writing it. Do you know of any similar resource for
google compute platform?

