
Serving 39M Requests for $370/Month - gk1
https://trackchanges.postlight.com/serving-39-million-requests-for-370-month-or-how-we-reduced-our-hosting-costs-by-two-orders-of-edc30a9a88cd
======
munns
I'll throw myself into the gauntlet here because its my job. (Chris Munns -
Senior Developer Advocate for Serverless @ AWS)(munns@amazon.com)

A lot of the comments here are comparing bare metal/virtual servers to a
combination of two+ products with literally dozens of built in capabilities,
security controls, built in scaling, monitoring, logging, deployment
mechanisms, and built in high availability. These are not apples to apples
comparisons in the least.

Fwiw I've run my own metal at some fairly large websites(can go find me on
Linkedin) and I can honestly say that to discredit the amount of time you
spend on solving the above capabilities WELL is to probably toss more than
120% of the cost out the window. People time to manage what goes on the
hardware is almost always higher than the cost of the hardware. I've spent 5+
years working with everyone from startups up through massive enterprises and
only the top 10% does any of this well when doing it themselves.

Taking some of the other examples here, let's say you do try and solve this
with 2 virtual/physical servers from a traditional hosting provider. What now
do you do for: high availability, nearly instant scalability, monitoring,
logging, alarming, deployment and security controls. Those last few all scale
on their own too (oh you've never filled a harddrive with logs before?). Are
you going to run all of your software on these 2-3 hosts? What happens when
they go down? How quickly are you recovering? What impact is there to your
customers during this time? What value do you place on that downtime? How many
of your tools just failed? Lost data?

How about patching the OS, the software you run on it, your tools, your
languages/packages? In busier environments where I ran my own logging,
monitoring, metrics, alarming, backups, software, firewalls, etc + the
operating systems those things ran on + the modules/packages most developers
use these days in languages such as node.js and python, you are talking about
whole days in a week to track, test, and understand these changes and the
risks associated.

I'm a #serverless zealot by trade sure, and I am happy to be called out on
that, but these comparisons of a very high level platform that essentially
removes most of the day to day care and feeding of your own "iron" can not be
compared to running your own iron at just that iron's cost.

~~~
hedora
At $370/month, you're right: Operating all of these services is definitely
costing more in human time than machine time.

I think the confusion stems from the fact that we don't really know how
expensive a "request" is. Many applications can do 1000's of requests per
second per machine (say 15K/sec).

Extrapolating the 15 requests per second = $370/month in the article, my
hypothetical one machine application will cost $370K/month on lambda. For
that, I can rent colo space, hire people to run all the services you
mentioned, and gold plate the racks while I'm at it. That is crazy overhead
for one machine worth of workload.

The GB*sec of a request in my extrapolated workload is certainly wrong, but I
suspect a lot of people ran this calculation in their heads.

I don't really know what a request does in this context, except that version
N-1 was 100x more expensive than the current one.

I think the right takeaway from the article is that spending some time to
optimize workloads that are costing O(one engineer salary) to operate is often
worth the effort, and, for low traffic stuff, it is hard to beat one-size-
fits-all infrastructure like lambda.

~~~
munns
It's true, "spring cleaning" from time to time can pay back in ways that
people don't expect once they clean up months and years worth of additive work
on top of something else. I agree that is a big take away.

On per request pricing this is something you can do with Lambda now semi-
trivially. You can figure out average Lambda execution cost based on duration
of execution and memory you have configured + API-GW related request costs.
Depending on your DB you can potentially tie this back as well (harder with
some DBs vs. others)

I'd say that while we see cost savings for most serverless platform customers,
like an overwhelming amount honestly, we also see companies adopting it for
the benefits of time to market, reduced operation burden, and the overall
capabilities that they don't need to write themselves. Hard to ROI some of
that. While this article did take on a cost based view I'd love to see more
folks talk about how these "softer" benefits impact them.

Thanks for your reply!

\- Chris

------
Veratyr
Hmmm... 39M requests per month = 15 requests per second. "20.3MM out of 39MM"
of those are hitting a cache, leaving 7.2 requests per second that actually
need to be run. The article says requests typically take 2.4s but it's a
scraping API and many of those seconds are going to be I/O rather than CPU
time. Assuming that there's say 1.0s of CPU time used per request, you should
only need an average of ~8 cores/hyperthreads for this.

I suspect you could fit this into a pair of dedicated server s from somewhere
like OVH or Hetzner for <$150/month (two servers to provision for peak load,
which I'm assuming is ~3x as high as the troughs).

Is there any reason why this wouldn't work?

~~~
rtpg
you still have to run the cache from somewhere, and you probably want to be
able to deploy during peak load times.

Not sure about the price, to me $75 for a decently beefy server seems low?
Compared to the cheap stuff that most people use because most people are only
at like 1 or 2 r/s.

Also serverless might sound silly but isn't it nice to not have to do ops
works? Surely worth $200/month.

~~~
kogepathic
_> Not sure about the price, to me $75 for a decently beefy server seems low?_

You can argue whether Hetzner is reputable hosting or not, but $75/month will
get you a v3 Xeon with 32GB of ECC RAM.

 _> Also serverless might sound silly but isn't it nice to not have to do ops
works? Surely worth $200/month._

Hosting something on Amazon doesn't absolve you of operations work, it's just
a different kind of operations. Instead of managing the OS/software updates,
you need to manage S3, IAM, CloudWatch, optimise time/memory consumption to
reduce cost, etc.

Maybe we just have different definitions of "ops works" but it bothers me when
people claim that you don't need to have anyone minding AWS because it's
serverless. You still need to have people monitoring, managing permissions,
optimising to reduce cost, pushing updates, etc.

~~~
mgkimsal
> Maybe we just have different definitions of "ops works" but it bothers me
> when people claim that you don't need to have anyone minding AWS because
> it's serverless. You still need to have people monitoring, managing
> permissions, optimising to reduce cost, pushing updates, etc

Thank you. Instead of being able to find a server ops guy from a large pool of
experienced folks, you now have to find someone specifically with AWS-ops
experience - a growing but much smaller pool of people, ime. Getting IAM
permissions correct for complex stuff is just not simple, for example, and
there's some use cases IAM stuff doesn't (or didn't) cover.

Had a client who couldn't give me EC2 access because - at least at the time -
they'd have to give me access to all their EC2 instances, not just the ones I
needed. Now... this may have been IAM-inexperience, although, at the time, my
digging seemed to indicate it was the correct conclusion.

~~~
vidarh
I keep recommending people look at places like Hetzner over AWS. People keep
wanting to do AWS anyway, and then end up paying 2x-3x as much for the servers
_and_ more for the ops work. I shouldn't complain - people who opt for AWS
even when it doesn't make sense are what my profit margin is made of.

(I have clients I _recommend_ AWS to as well, but reducing ops time and/or
reduced cost are rarely if ever part of those discussions; AWS is the luxury
option)

~~~
rtpg
out of curiosity: What makes you recommend AWS in those cases?

~~~
vidarh
It would typically be if they have use cases where they either want to do
large batch jobs, or have other needs where "add-on" services AWS provides is
worthwhile. Often it's a matter of urgency, and AWS will be a good first
approximation, and we can start migrating onto a hybrid or fully dedicated
environment depending on what's most cost effective down the road when they
know more about what they actually need.

Sometimes it's a matter of politics and trst - AWS is trusted, and if their
hosting cost doesn't add up to much of their total costs, then that trust is
sometimes more worthwhile than squeezing the cost down. E.g. I'm working on a
project for a VC now, and their yearly hosting costs are likely to be below
what I'm charging them for the initial development of their system, so
spending time selling them on a cheaper provider is just not worth it for
either them or me.

------
iDemonix
A slightly older article, but here's 40M requests a day for $10/m

[http://reviewsignal.com/blog/2014/06/25/40-million-hits-a-
da...](http://reviewsignal.com/blog/2014/06/25/40-million-hits-a-day-on-
wordpress-using-a-10-vps/)

~~~
bdcravens
Apples and oranges. The article you linked referred to a Wordpress site; the
article in this post refers to an actual parsing application.

~~~
iDemonix
Completely agree, different topics, but for anyone interested in low-cost
scaling it's an interesting read.

------
TekMol
This needs an explanation why the requests are so expensive.

39M Requests for $370/Month is an order of magnitude _more_ expensive then the
average web application.

I read through the article and it did not give any explanation except that it
hints at a big database: "database had grown to store a massive slice of the
internet".

Without actual numbers and a description of the workload, it's hard to say if
the described solution is good or bad.

~~~
gravypod
Just for context at 39M requests at $370 you are spending ~0.001 cent per
request. That's only counting the API. That's not counting their Database
costs (which if they are archiving a large part of the internet would be
sizable).

For reference I've had times where I was getting ~1000 requests/day/site on a
$17/year VM. This was done with Cloudflare caching and caching PHP output.

This comes out to...

    
    
        * `ls /var/www/ -1 | wc -l`: 17
        * (1000 * 30 days) = 30000req/month/site = 30k * 17/month
        * 17 / 12 = $1.5/month
    

0.00029411764 cents / request

That means my request, on a crappy VPS serving Wordpress, some custom static
sites, and a few other services (PayPal API handling, emailing, uptime
monitoring, etc), cost only 30% as much as the requests described in this
post!

PHP-FPM, Lighttpd, and MySQL's small config are magical.

------
tayo42
That title is a little deceptive about scale? 39m requests a month works out
to 14/s, then half of that is cached? Why did it cost over $10,000 to begin
with? Seems like a simple workload. 2 small vms could handle that

~~~
pfg
It's definitely not quite an apples-to-apples comparison. The article mentions
the old API used to do things like storing the results in a database which
"[...] had grown to store a massive slice of the internet", while the new
service appears to be stateless, aside from the cache. Not needing a couple of
large database servers alone is going to save a lot of money.

That's not to say Lambda was the wrong choice here, the overall cost is pretty
low and it's hard to argue that maintaining their own servers would end up
being less than that.

------
grayrest
> Rewrite the Readability Parser API

That's at least the third rewrite of the parser. The initial version was a js
bookmarklet, I did the rewrite into Python in 2008 which kind of sat around
and then got turned into the app by a different group about a year later. Fun
to see it go back to JS.

------
markonen
One key takeaway here is that in a typical serverless API setup, the AWS API
Gateway will actually end up costing more than the Lambda compute. According
to the breakdown in this post the Mercury Web Parser API isn't an exception.

I don't want to be the guy saying that 39 million requests isn't a lot, but
especially if your API serves trivial responses to a lot more requests, API
Gateway can end up being the opposite of cheap.

~~~
nikon
Agreed that the API Gateway is overpriced, but what is your alternative -
build a Lambda that somehow is your API gateway?

~~~
markonen
There is no truly serverless alternative, although since the API for invoking
Lambda functions synchronously is free, you can possibly roll your own API
Gateway (assuming you are OK with having _servers_ for that).

But my point was more about simply being cognizant of the API Gateway cost.
Too many Serverless tutorials/testimonials assume no significant request
volume and end up ignoring this aspect of AWS billing.

~~~
vidarh
The genius of AWS lies in a pricing structure that is easy to get approval
for, and where developers once they've started using it are rarely aware of
how much it costs...

I see so many cases where people are bewildered at how they've come to spend
as much as they do because nobody understands the cost model.

------
luhn
The low cost is really impressive, especially given that the requests are
relatively long-running (2-3s). Color me surprised that their use-case could
benefit so much from Lambda's pricing.

Obviously YMMV. Does anybody else have anecdotes about Lambda's pricing
helping or hurting their application?

~~~
fweespeech
I wouldn't call it a low price.

Maybe I'm just spoiled because I can put crawl/parse tasks in a work queue for
each url without much concern for latency but I've gotten 1000 r/s (on async
calls) out of $100/€100 dedicated servers.

Their setup _really_ only makes sense for WAN facing public architecture that
can spike because you have no control over the load. Otherwise, I'd suggest
you just get some leased dedicated server(s) to meet your workload.

------
samtoday
They mentioned the cost of storing all the articles in a database at the
start. I wonder how much of the cost decrease was from removing that?

------
endorphone
Many of the comments make a good case for why requests per month is a close to
meaningless metric. My blog, hosted on a single Amazon spot instance for about
$11/month, serves close to that measure at times.

How big is each request? How peaky is it? Etc.

------
kasey_junk
Is it common to quote requests in requests per month?

I had to read a big chunk of the article before I realized what units were and
did the math to get the 15 rps number.

~~~
dsacco
No it's not you, that really threw me off as well. I've worked on fairly large
data mining software and we almost always talked about requests on a per
second basis (sometimes requests per day if we were discussing aggregate
projects and total output). Using requests per month would be...astronomical.
Hell, one product was running 100M requests _per day_ (though to be fair not
every request was 1:1 bound to a corresponding database read or write op).

That brings me to my next observation: I don't want to demean the author, but
15 rps is really not extraordinary, so I'm wondering why these requests are so
expensive. I assume there is a good reason they need to focus on "scaling"
that many, I just can't figure out exactly what it is. They might be doing a
lot of very inconsistent throughput with hard to predict peaks, I guess...

------
tjholowaychuk
I'm a huge fan of Lambda, but these prices really need to go down. If anyone
thinks API Gateway is cheap, just do the math, this is a trivial amount of
traffic for even the smallest of EC2 instances.

Head over to [http://serverlesscalc.com/](http://serverlesscalc.com/) and
enable the "HTTP Requests" radio.

------
tapirl
If you do well, you can get 39M requests per month for less than $30 on App
Engine.

------
tracker1
It's worth noting, that they could use route53, and parallel deployments to
get lower latency as well, with regional scaling. Not sure what this would do
to request times, already over a second on average.

------
davewasthere
On an aside, I do like their Mercury Reader extension.

You find some website that's utterly polluted with content, click the Mercury
Reader and you've got a very clean readable article (similar to Firefox's
Reading mode).

I keep thinking, it'd be nice to have a Lynx Browser-like proxy that reformats
existing webpages into readable, but minimum markup. Sort of like AMP-
everywhere perhaps. The purist in me only wants to download the 2kB of actual
information from most sites, and not the extra few Meg for tracking, ads,
pointless styles, frameworks etc...

------
m0shen
While several other commenters have pointed out how expensive API
Gateway/Lambda is. It is highly dependent on your workload.

For example: We have a fairly consistent load of very small, short api
requests (Billions a month). These requests can be easily served by a couple
of small servers. We calculated API Gateway + Lambda to cost 30 times more
than the solution we rolled on Elastic Beanstalk.

We have other services which run for nothing on API Gateway + Lambda. Most of
them aren't doing very much though (< 1M requests a month).

------
bdcravens
> Our previous costs also included database expenses, which we chose to forego
> in our serverless setup, opting instead for short-term caching.

Most web applications require a higher degree of persistence.

------
desireco42
This is actually good use-case for Lambda, majority of other attempts that I
see are mostly not great for Lambda. I personally wouldn't use it without
really good need and would be very cautious because costs can spike without
you having a limit on them with Amazon.

It is good but some more traditional way would also be fine as well. I kind of
expected to see Elixir or Go in the backend handling the load more
efficiently.

------
CyberDildonics
This is 15 requests per second, which should be easily handled with just about
anything connected to the internet.

------
baybal2
Back in 2014, I worked on an in-memory DB API that handled 1m http requests
per hour with ease on a desktop machine with 10G card.

DPDK + DMA aware programming in C++

------
avip
>39 Million Requests ... $/Month

39 days/Month

80K sec/day

1M req/day ==> 12 req/s

~~~
nickpsecurity
There's not 39 days in a month.

------
spullara
If you can't make $370/month from 39M requests, maybe you shouldn't be making
those requests in the first place?

~~~
lilbobbytables
Sounds like they can just well.

They mentioned orders (plural) of magnitude difference in cost. Which means it
was closer to $37,000/mo previously.

------
candiodari
TLDR: amazon serverless (lambda) free tier.

~~~
elwesties
What do you mean by this comment?

~~~
luhn
He means that the cloud is so obviously expensive that any low-cost
infrastructure must be running on bare metal.

------
PhilWright
Pretty amazing. Looking at their numbers...

39 million requests X 2.39 seconds per request = 93.21 million seconds of
compute time performed each month. That works out at 2.95 years of computation
for $370.

Although he mentioned that about half the traffic comes from cached results.
So maybe it is more like 1.5 years of actual new computing each month. Compare
the $370 against the AWS cost of running a VM for 1.5 years to do the same
workload and I think your saving a lot of money.

~~~
fweespeech
> That works out at 2.95 years of computation for $370.

It is 2.95 years of computation on less than 1 CPU core.

[http://docs.aws.amazon.com/lambda/latest/dg/lambda-
introduct...](http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction-
function.html)

> AWS Lambda allocates CPU power proportional to the memory by using the same
> ratio as a general purpose Amazon EC2 instance type, such as an M3 type. For
> example, if you allocate 256 MB memory, your Lambda function will receive
> twice the CPU share than if you allocated only 128 MB.

[https://aws.amazon.com/ec2/instance-
types/](https://aws.amazon.com/ec2/instance-types/)

> m3.large 2 7.5 1 x 32

> Each vCPU is a hyperthread of an Intel Xeon core except for T2 and
> m3.medium.

1 Core per 7.5GB RAM. Or ~1/28th of 1 core.

(2.95*365.25)/28 = ~39 Core Days.

That is really only impressive when you are comparing AWS prices to AWS prices
which is a trap people seem to frequently fall into imo.

~~~
vacri
The computation costs for Lambda were $174, not the full $370.

~~~
fweespeech
Using AWS Lambada created that cost structure. Using an alternative to AWS
would not.

$150 (at most) dedicated server for instance.

~~~
sudshekhar
As an aside (and a noob question), how many requests can a single server
handle? I know frameworks such as Node can do handle many more connections
when compared to something like Django but what would be a reasonable
ballpark?

As a concrete example, say you have an ecommerce app. Read requests take 80ms
while write takes 200ms. Any good article/book I can read to get more insight
regarding this?

~~~
falcolas
I've personally handled hundreds of requests per second using Django behind
Nginx on a m3.medium EC2 instance. The trick to scaling like this is to take
full advantage of caching, and the cores available to you. With Django (and
Node), this means spinning up multiple instances on a single server (one per
available core).

With even moderate caching (say, 1m), you can saturate your VM's bandwidth
before running out of CPU or memory.

Ultimately, you will always have a bottleneck. All you can do is identify the
bottleneck (via load testing) and address it. If it's network (and you're in
Amazon), you have to scale vertically until they remove that limit, or
horizontally if you're already on unrestricted instances. If it's the CPU,
you'll need to be more aggressive at caching completed computations, or work
in a language with less overhead. If it's memory, start evaluating the
tradeoffs between CPU and memory. If it's your DB, start investigating why
it's slow (it's probably a poor index or configuration; DBs on decent hardware
can handle millions of requests per second before degrading).

I don't have any book recommendations; I just recommend creating something and
find its limits with a simple load test tool. Then figure out your bottleneck
and fix that. Rinse and repeat, and learn.

