
9 million hits/day with 120 megs RAM - verisimilitude
http://tumbledry.org/2011/08/31/9_million_hits_day_with_120
======
jroes
Blogs really don't need a PHP, Rails, or anything backend. It's static
content.

Here's how I think blogging should work:

1\. Visit a web app where you create your blog post, add pictures, use your
rich text editor, that sort of thing.

2\. Click the "Publish" button, which generates static HTML and runs any other
processing like tag generation or pngcrush.

3\. Your static HTML gets pushed out to some server that does the hosting for
you. It could even be one of those really cheap shared hosting providers.

If you really want comments, let someone like Disqus or Intense Debate handle
it. Pretty much any dynamic feature you need can be outsourced.

~~~
jacques_chester
That's pretty much how MovableType used to work.

Fun fact: it was terrible. Most bloggers I know hated it.

Wordpress is a bucket of poop too, but it has the quality that when you press
'publish', the site is updated in seconds, not minutes.

(I believe MT has changed since then).

~~~
pwaring
It's how Greymatter and a few other systems used to work as well. The main
problem was determining which pages should be updated, which usually resulted
in the entire blog being rebuilt. This took several minutes if you had a few
years of archives, and if anything went wrong part-way through the process you
had to start it all again.

~~~
jacques_chester
I've ruminated in the past about a better blogging system based on updating
on-disk caches based on POST and not GET. I think everybody who watches
Wordpress trip over and burst into flames when you so much as _breathe_ on it
has.

The main problem to solve is inter-page dependency. The rest is not that hard
/famouswords.

~~~
pwaring
You can get too carried away with scalability though. I run half a dozen
WordPress blogs, plus some static sites and email for multiple domains (with
AV and spam filtering) within 500MB of RAM. I don't need to implement any
caching because I already have sufficient resources.

~~~
mkr-hn
Get this: <http://wordpress.org/extend/plugins/w3-total-cache/>

It's called a caching plugin, but it also compresses and minifies stuff. Any
site can benefit from loading faster. You can turn it on and let it go. It has
sane defaults.

~~~
rawsyntax
Thanks for the info, after hearing people praise wordpress as the holy grail
of blogging, I found it to be very very slow. Hopefully this plugin will help

~~~
byoung2
WordPress is usually praised for usability, not performance. To make WordPress
faster and more efficient, you need a caching plugin like W3 Total Cache. It
caches your pages in static files, caches queries and results in memcache, and
rewrites css, images, and js to CDN paths if you have a provider. In front of
this, I would have Varnish cache as much as possible in memory, for even
faster access. W3 Total cache can even invalidate your Varnish cache when
pages are updated.

~~~
tombot
Cannot give Varnish enough praise, it can tubo charge even the most expensive
/ slowest loading pages and uses minimal resources.

------
michael_dorfman
I guess my age is catching up with me-- my gut reaction on seeing the headline
was: _120MB? That's a lot of RAM-- who has that? Oh, wait..._

~~~
Luyt
Indeed! I read it as:

 _9 million hits/day with 120 gigs RAM_

Especially because of the scalability discussion around it. Only later in the
article it dawned on me that he is using a cheap cloud instance.

~~~
harrisreynolds
LOL... me too... I was thinking... who _couldn't_ hit 9M hits a day with 120
gigs of RAM... just load everything in memory and walk away! 120 MB is
certainly more impressive.

------
mfjordvald
Cross posting this from a comment I made on reddit: This is something I've
actually worked extensively on solving and it's not quite as easy as this
article claims it to be. In fact, there are quite a few too many draw backs to
this method to any site that isn't largely static or updated very rarely.

* Whenever anything is updated the entire cache is invalidated and each item needs to be fetched again. This means you'll have some page loads being slow and others being fast. If you have a very dynamic website you will hardly ever even see a cached version.

* You can't cache things forever, primarily because when anything is updated the entire version namespace is invalidated. This means that if you have a site that isn't updated at all in a long time then the cache is still invalidated by the TTL and has to be updated. Of course, if you decide to cache forever and the version namespace is incremented then...

* You never know when your cache is full. Since the method of updating the cache isn't to invalidate keys but rather to just fill it with new keys, you will have a lot of stale data. This data will eventually have to get evicted from the cache. This means you don't reliably know when you need to upgrade your cache memory.

All that said. Version namespacing your cache is better than not caching at
all and it's usually also better than having a lot of stale data as active
keys. If you want to do proper cache invalidation in case you have a highly
dynamic site then it's still possible, but it requires a lot more work,
there's a reason for this famous quote:
<http://martinfowler.com/bliki/TwoHardThings.html>

------
api
This underscores how ridiculously overspecced modern servers are due to the
bloat of a lot of modern software.

~~~
jacques_chester
Virtual machines change the economics of hosting _for the host_. A given VM
has a lower administrative overhead per customer than shared hosting does.
This will eventually drive out shared hosting, IMO -- except for single-
application purposes.

~~~
lsc
"for the host" is key, really. the customer has to do a lot more work on a VPS
than on a shared host, so I think shared hosting isn't going away.

Now, I agree with you that the shared hosting market will continue to move
more towards constraining the user to a particular language within that shared
hosting environment, as that allows the host to provide better service, but
this has been true for a while. For the last 10+ years, if a shared-hosting
provider allowed you to run an arbitrary binary, it only allowed you to do so
as a second-class citizen, behind several layers of setuid wrappers and other
(usually slow) security. Back in the day, if you wanted to run at reasonable
speed on a shared host, you'd write in php, and use the supplied MySQL
database, which conceptually isn't that different from what many of the
language specific cloud providers do today.

The interesting thing is that this means shared hosting, usually with a well-
supported but language constrained environment, is actually becoming the
premium hosting option. VPSs, formerly the high-end option, are now the cheap
stuff. Which makes sense; as a provider, sure, the VPS costs me more ram, but
I can get an 8GiB reg. ecc ddr3 ram module for under a hundred bucks now. Ram
is cheap. what costs me money is the support, and it is a lot easier to rent
out unsupported VPSs than it is to rent out unsupported shared hosting.

If anything, with this new 'premium' image and advances in billing customs, I
think we are seen a renascence in 'premium' shared hosting services.

~~~
Nick_C
> The interesting thing is that this means shared hosting, usually with a
> well-supported but language constrained environment, is actually becoming
> the premium hosting option. VPSs, formerly the high-end option, are now the
> cheap stuff.

Exactly this. I have been using shared hosting for around $100/yr, which was
cheap 3 years ago when private servers were double or triple that. But
recently I bought a VPS for _$15/year_. Sure, I have to do my own admin, but
I'm fine with that. If the cost for the shared hosting doesn't come down, I'm
going to have to move everything to my VPS.

~~~
lsc
$15/year? is it an OpenVZ (advanced chroot jail) VPS?

That's an interesting counterpoint to this discussion; advanced chroot jail
hosts are still cheaper than Xen hosts, and OpenVZ (the common advanced chroot
jail software on linux) is in between shared hosting and full virtualization
with regards to how many resources are shared and how much host sysadmin
effort is required.

With OpenVZ, all users share a kernel and share swap and pagecache; you can
fit more OpenVZ guests on a particular bit of hardware than Xen guests, and
generally speaking, more host sysadmin involvement is required when running an
OpenVZ vs. a xen or kvm host.

The interesting part of the xen/OpenVZ dichotomy is that it goes the other
direction; so far the market price for OpenVZ guests is much lower than the
market price for Xen guests; OpenVZ mostly occupies the very low end of the
VPS market.

~~~
Nick_C
> $15/year? is it an OpenVZ (advanced chroot jail) VPS?

As far as I can tell (from the kernel name), it is OpenVZ. I only discovered
that last night, coincidentally.

It is tiny, only 256MB, bursting to 512MB, of RAM. But at that price it is
fine for my needs; I even get my own static IP address (the full cost of the
VPS is cheaper than just adding a static IP address on my shared site!).

I was reading up on OpenVZ vs Xen or KVM last night, and I take on board your
comments above. The virtual server market seems to be transitioning right now,
with pricing yet to settle down within a common range like they were a few
years ago.

~~~
lsc
yeah, at $15/year, you are getting quite a steal, OpenVZ or not. for a 256M
xen VPS I charge $76.80. Heck. even if your provider is being heavy on the
swap (my understanding is that OpenVZ 'RAM' numbers include swap, while under
Xen, I give you ram, and you can add swap if you like, so when directly
comparing OpenVZ and Xen guests, you should compare a xen guest with less ram
to an OpenVZ guest with more.) and you are only getting 128M of actual ram I'm
charging you $57.60 for that, and you haven't said, but if they are following
industry norms they are giving you more disk than I am, and I primarily
compete on price; I'm fairly low-end. Did you get a special? or is that the
sort of thing you can order every day?

~~~
Nick_C
It was a special, but one that they offer regularly, in the sense that every
so often they offer say 1,000 of them until the offer is filled up. Then it is
closed off until the next round.

Disk space is 10GB and traffic is 500GB.

I found the vendor on www.lowendbox.com .

------
jacques_chester
The key points:

    
    
        1. Use caching.
        2. Use Nginx.
        3. Use PHP-FPM.

~~~
easp
I think people are too quick to throw out Apache + mod_php. It remains the
defacto deployment environment for most open source PHP apps, which means that
it is the best understood and best supported. For example, if you deploy using
Apache, you don't have to worry about converting .htaccess rules that
wordpress and some caching plugins create into an Nginx equivalent.

People bitch a lot about the memory consumption of apache, but it's often
overstated. They add up the resident set for each individual apache process
and come up with a huge number, missing the fact that a significant amount is
actually shared between processes.

Apache's memory consumption problems have more to do with how it handles
clients. Each client ties up a worker process for the entire duration of a
request, or beyond if keepalives are enabled (as they should be). That means
that the memory overhead of the PHP interpreter is locked up doing nothing
after the page is generated while it's being fed to the client. Even worse,
that overhead is incurred while transmitting a static file.

I use nginx as a reverse proxy to Apache. It lets me deploy PHP apps easily
and efficiently. Apache returns the dynamic and static requests as quickly as
it can and moves on to the next request. Nginx buffers the result from apache
and efficiently manages returning the data to all the clients. I used to have
Nginx serve the static files directly, which is more efficient, but added
complexity to my config. I chose simplicity.

A PHP opcode cache, like APC, is also a big win, because it cuts the overhead
of parsing and loading the PHP source files. I'm not convinced of the value of
other caching for most uses. CPU time usually isn't the scarce resource, RAM
is. The DB and filesystem cache are already trying to keep needed data in RAM.
Adding more caching layers generally means more copies of the same data in
different forms, which means less of the underlying data fits in RAM.

~~~
snorkel
So true. Apache + mod_php is very reasonable choice. Anyone who advocates
using FastCGI instead of mod_php to "save memory" just doesn't understand what
the actual memory footprint of Apache + mod_php really is, and how adjust the
number of Apache processes.

In fact FastCGI _still_ ties up an Apache process for the duration of the
request: Apache hands the PHP request off to a FastCGI worker, then waits for
that PHP worker to send back the output, so the Apache process is still
blocked waiting on PHP in either scenario.

Also the overhead of Apache serving static content is miniscule compared to
the amount of work a PHP does per dynamic request, unless the static content
is very large, like large media files.

~~~
runningdogx
It's not just about steady state memory footprint. It's about the whole stack,
how apache and mod_php and mysql (if you use it) interact.

Excessive traffic means a spike in simultaneously served connections, which
means a spike in apache threads (assuming worker MPM, iirc there's 1 process
per 25 threads by default). With the mod_php model, the per-thread php memory
usage can be very expensive when you have dozens or hundreds of apache threads
serving requests. A spike in running php instances leads to a spike in mysql
connections for a typical web app. If you haven't tuned mysql carefully, which
most typical environments have not, mysql memory usage will also skyrocket.

Then for the coup de grace you get stupid apps which think it's perfectly fine
to issue long-running queries occasionally (occasionally meaning something
like .1% to a few percent of page loads). When that happens, if you're using
myisam tables which were the default with mysql < 5.5 (and which therefore
dominate deployments, even if "everyone knows" you're supposed to be using
innodb), then those infrequent long-running queries block the mysql thread
queue, leading to an often catastrophically severe mysql thread backlog. Since
php threads are issuing those queries, the apache+mod_php threads stack up as
well, and they do not use trivial amounts of memory.

The result is that you have to severely over-engineer the machine with excess
memory if you want to survive large traffic spikes. If you don't, you can
easily hit swap which will kill your site temporarily, or worse, run out of
swap too and have the oomkiller kill something... either your webserver or
mysql.

The benefit to fastcgi is it takes the memory allocation of php out of
apache's hands, so every new apache thread is more limited in how much bloat
it adds to the system. With a limited pool of fastcgi processes, you can also
limit the number of db connections which further improves the worst-case
memory usage scenario.

The advantage of in-apache-process php is that it serves php faster when there
are few parallel requests, but it's on the order of single-digit milliseconds
difference (the extra overhead of sending requests through a fastcgi socket),
which is dwarfed by network rtt times even if none of the above pathologies
rear their heads.

The apache+mod_php model is to do php processing for all active connections in
parallel. The fastcgi model is to do php processing for at most x connections
where x is the php fastcgi pool size, leaving all other requests to wait for
an open slot. It may intuitively seem like the fastcgi model is going to be
slower because some requests have to wait for a fastcgi process to become
free, but if you think about average php request time it's going to be better
for high parallelism, because the limiting factors are cpu and i/o. The apache
model ends up using ridiculous amounts of resources just so no php request has
to wait to begin getting processed by php. The high contention particularly
for i/o created by apache and mysql when they have large numbers of threads is
what makes the fastcgi model superior.

------
noelwelsh
I'd like to know how much the Joyent Smartmachine contributed to this. They
make some bold claims on their website, and really do seem like a great
alternative to EC2 (disk IO that doesn't suck!) if they deliver. Anyone have
any experience?

------
dan_manges
Some interesting techniques in here (e.g. Faking Dynamic Features Using Inline
Caching), but otherwise it seems easy to scale to this level when the majority
of page content can be cached.

------
todsul
The difference between Apache and Nginx is that out of the box, Nginx is built
for speed. Both are capable of thousands of requests per second, but Nginx
arguably does it better with its event-based architecture (opposed to Apache
being process based). The config syntax is also refreshingly simple, so
converting .htaccess rules couldn't be easier.

We were recently paying a small fortune for hosting one of our websites. It
was bumping up against memory limits even after a serious code rework and
aggressive caching. Instead of upgrading we decided to test a new config using
Nginx.

Now we run three sites, one fairly popular, on a 512Mb Linode with Nginx, APC,
FPM, Varnish and a CDN, and it can take an amazing amount of load. Varnish
needs memory, but without Varnish we could run this setup on a box a fraction
of the size.

This plan costs $19/month! I still can't believe we're paying so little.

Instead of focussing just on the server though, and like the TumbleDry article
somewhat suggests, HTTP cache is probably the best place to start in terms of
performance. Varnish, CDNs, etc all rely on intelligent HTTP caching. And if
you do it right, you don't need to worry (too often) about cache invalidation.

What I'm really looking forward to is making use of ESI in Symfony2 and
Varnish. That will mean setting different cache headers for portions of pages,
which will further reduce the need to manually invalidate cache.

For now though, I'm loving Nginx + FPM + APC.

------
silverbax88
I agree that this should be the standard, not an exception.

~~~
g-garron
I've been once on /., two times on Digg and all times my server crashed. I've
improved it a lot since then, using different techniques. But never been hit
by one of those again :(

New self hosted blogs can't get the load of peak times, because new bloggers
start a WordPress blog, install on it a lot of plugins and never think about
performance. I think most of us have made that mistake some time.

~~~
jellicle
Sure. But it's not a mistake; it's a correct choice given current
expectations.

What I don't understand is why the various CMS's don't offer automatic on-the-
fly reconfiguration. It's 2011. We should be able to have the best of both
worlds:

\--when load is light, your blog software hits the database 42 times with
every page loaded, no problem

\-- when site load shifts from 1 page per hour to ten pages per second, the
CMS should automatically, with no user intervention, say "oh shit" (perhaps
audibly in the server room) and then automatically generate a static front
page, and static article pages for whatever pages are being linked to, turn
off the bells, turn off the whistles, email and tweet at the server
administrator, etc. The CMS should cheerfully weather a storm all by itself.
And when the load dies down, the static page should revert to a dynamic,
42-queries-on-the-database page, again without any intervention from the
server administrator.

Does this exist anywhere, out of the box?

~~~
silverbax88
I don't know, but I do know that my architectural approach is to get any web
app I write to only hit the database ONCE per page view. Ten calls is a lot
for me. I've seen some that hit the DB 100 times on each view and I was able
to reduce all 100 down to a single call.

~~~
pestaa
I heard one specific Joomla site used 800 queries to render the front page. A
dev needs natural talent to reach that point, I guess.

~~~
arghnoname
I worked on a site once that loaded at an okay rate for the live site, but the
staging server would take about 15 minutes (seriously) to load once every
other hour or so. I thought this was strange, same code and all, so I looked
into it.

There was a loop on the page that instantiated objects with an id, the number
of ids dependent on the results of a previous query. What did that object
instantiation do under the hood? It performed a query to fetch state. I
calculated that 5000 queries were being run, and it only cropped up every once
and a while (and seemingly 'never' on the live site) because queries were
automatically cached for a set amount of time.

I was new on the project, and in what is probably poor form, went around the
whole office, letting my horror be fully known. People just shrugged though.

edit: I forgot to add, modifying it to only perform one query was trivial.

------
dan_manges
It's not apparent if the 9 million+ daily hits number is taking into account
that peak hours will be higher than off hours. It would take 100 reqs/sec if
the traffic is even throughout the day, but 375 reqs/sec if 15% of the day's
traffic is in the peak hour.

~~~
onemoreact
Yea, I see several diffrent numbers being tossed about but:

 _This caching (I’m using APC right now) got my page load times down to about
170 microseconds for most pages, and 400 microseconds_

400 microseconds ~= 2,500hits/sec; 170 microseconds ~= 5,880hits/sec; Both of
which seem reasonable for a simple PHP site using a single core CPU @ 1+Ghz.

------
g-garron
As a lot of you have said: Static content is the key to success. You can name
it:

\- Movable type

\- Drupal + boost

\- Wordpress + SuperCache

\- Jekyll or other static website generators

Better if Nginx is serving those static files, LAMP can be behind creating the
static files.

I used that way with Drupal+boost for a lot of time and worked.

------
drv
The title says "120 megs of RAM", but I wonder if that's at all comparable to
a real machine with 120 MB. I imagine that the "120 MB" VM is running on a
beefy host with tens (or hundreds) of gigabytes of RAM shared between the
guest VMs and also used for disk cache. It seems likely that accessing a
guest's virtual disk would actually hit the host's disk cache a lot of the
time (especially when that guest has been busy recently); that would improve
the speed of disk access for that VM enough that it could make up for the lack
of memory for disk cache within the guest.

This is purely speculation, but I would be interested to see if there is any
actual research to back it up.

I suppose if the guest in this instance is not swapping very often, then this
is fairly irrelevant, but the article didn't mention anything about swap.

------
wingo
Very nicely done. Using JS to give personalized experiences seems to be the
way to go. I suppose you could generate a JSON list of id,date pairs to reduce
page bloat, if it matters.

[Edit: This is 100 qps. It's a lot for a blog, but is not an unreasonable load
by any means.]

------
gtklocker
He could just say "tl;dr\n<machine specs>\n<I use static pages>".

------
ez77
For static, high-traffic, small-size content, doesn't it make sense to load a
minimal OS entirely to RAM and serve it from there? Has anybody tried this? (I
guess this rules out VPSes...)

Note: This is a variation of a previous comment I made, but a variation
nonetheless. Sorry to belabor the point.

------
ez77
Naive question: suppose that you serve a low-throughput site, say with a total
of 3MB of data (probably text files). What's the simplest way to ensure that
those 3MB of content (very little compared to 120MB) live always in RAM? By
this I mean not giving the server a choice =).

~~~
sbierwagen
<https://secure.wikimedia.org/wikipedia/en/wiki/RAM_disk>

------
senthilnayagam
discovered blitz.io will keep me and my servers busy this weekend

~~~
tudorizer
Kinda expensive :(

~~~
jbhelms
I use <http://loadimpact.com/>

The free version can give you an idea of how your server will handle a small
spike, and the other versions are per day so if you only need to test once or
so a week as you finish a sprint then it wouldn't be a horrible cost.

~~~
icebraining
It's odd that they let you just test any website without proving its yours (by
e.g. putting special file in the root or something). Can't unauthorized
testing be considered a DoS attack?

~~~
kowsik
I can only speak for <http://blitz.io>. We generate an account-specific UUID
which becomes a URL that you need to make available on your app (adding a
route in ruby/node.js or uploading a txt file). Before each load test we check
that _your_ UUID is available on the app. Even if your UUID is leaked, this is
not a problem, since it's unique to your account. Unless of course, your
Google/Facebook account gets compromised. We currently, do not support
password logins. It's either OAuth/FB Connect or SSO through our partners.

------
antihero
That's about 100requests/sec, which isn't particularly amazing.

~~~
pestaa
On average that means 1.2 Mb memory available/request, which isn't
particularly boring either.

------
TylerE
One thing to point out, based on my experience, is that you need about 10x _or
more_ peak throughput to handle a given average throughput. Spikes kill you.

------
waffle_ss
Wonder how much faster it would be if PHP was taken out of the mix (looks like
he's just just serving static pages anyway).

~~~
j_col
But he is using the APC cache, which is part of PHP, so I'm not sure he could
pull that off? PHP-FPM is the fastest way he could be using that interpreter
anyhow, as it maintains a persistent bank of PHP instances in-memory, with
nginx acting as a proxy to it via a Unix socket.

~~~
icebraining
You wouldn't need APC if you weren't running PHP...

------
krmmalik
Quick Question. Would using Nginx as a front-end to Node improve performance
in the same way it has done for serving PHP?

~~~
jjm
Putting any proxy cache in front of your app will help. You can 'simulate'
static caching of dynamic content. You will have to take a look at your
headers however as most caches obey upstream caching headers to spec tightly.

------
michael_h
By inlining the comments, he's reducing cpu time by...transferring extra data
across the network?

------
winsbe01
love it. gives me faith that the archaic machines i have serving can still
hold their own!

------
schiptsov
There was some post about a happy Win/IIS/CF guy - he definitely should read
this.. ^_^

------
njharman
hits/day and megs ram are orthogonal.

------
kahawe
I have to say I haven't tried that myself nor have I looked at the prices so
my question is: All the fun of having your own server aside, why wouldn't I
rather just run a site like that on something like amazon ec2 and stop
worrying about hits and load even if it is just a personal blog?

------
RyanKearney
I'm not too sure I'd go with a load testing company that can't even keep their
own website up.

<http://blitz.io/>

> Internal Server Error

~~~
mkr-hn
It's always exactly the time when you need your service to be up the most that
it goes down. What matters is how they respond to it.

~~~
kowsik
thanks for the vote of confidence, will write up what happened soon.

~~~
kowsik
Here's the blog on what we are learning as we scale:
[http://blog.mudynamics.com/2011/09/01/blitz-io-path-
finding-...](http://blog.mudynamics.com/2011/09/01/blitz-io-path-finding-with-
couchdb/)

------
luigionline
how about just using a CDN service. There is no need to play around anymore.

------
j_col
Very impressive, it's amazing how the "LAMP" stack continues to evolve.

~~~
Joakal
Looks more like a LNMP stack.

~~~
j_col
I'm using PHP-FPM myself and it is fantastic, major increase in performance
when compared to mod_php. I still have not let go of Apache2 yet, but nginx is
calling to me due to it's non-blocking design...

~~~
Joakal
It's pretty painless beyond understanding it. Plus curbs most DDoS issues
lately.

~~~
troutwine
Not to mention that the folks in #nginx on freenode are really very helpful.

