
Bulletproof your blog – survive the Hacker News effect - masnick
http://www.maxmasnick.com/guides/bulletproof_your_blog/
======
tikhonj
He forgot the geekiest option: write your own blogging engine. It's a perfect
excuse to use that new language and framework you're learning! Or, if you're
not learning anything now, it's a great excuse to start learning.

~~~
masnick
While this is a great learning experience, it is hard to write a blog engine
that will stand up to traffic spikes on the first try.

If your goal is learning a new language or framework, this is a great idea. If
your goal is to have a reliable, traffic-resistant blog I think it's best to
stick with a tried and true platform.

~~~
seunosewa
It's not hard. Just write a blog engine that generates static html pages.

~~~
masnick
Right, if you write something akin to Jekyll it will work fine.

However, I think most people would probably be trying to learn a framework
like Rails or Django if they were writing a blogging engine to learn a new
language/framework.

~~~
mapleoin
So write your blogging engine as you normally would and then make a button
which says: dump to html for each page and then figure out when you want that
button to be pressed (i.e. manually or automatically).

More things to learn. Yay!

~~~
sunkencity
Build it anyway you like but then wget --mirror before rsync to deploy.

------
kijin
It's a shame that several years after the term "slashdot effect" was coined,
self-hosted personal blogs are still vulnerable to this easily preventable
phenomenon. The companies that host the majority of the world's small web
sites, such as BlueHost, DreamHost, and (heavens forbid) GoDaddy, have not
made any innovations in this department. Most VPS's are no different.

NearlyFreeSpeech.NET is really good at handling traffic spikes, for both
static and dynamic sites. It combines the convenience of shared hosting with
the scalability of cloud computing. You just need to insert more coins when
you run out of your prepaid traffic.

~~~
marquis
Where is the financial incentive for these hosting companies to do so? Most
users will never see a traffic spike that can take down your average VPS or
bluehost site. It seems there is a gap in the market for self-hosted sites
that have built-in structures to build upon that allow for traffic spiking.

~~~
edanm
Isn't the gap filled by companies like Posterous/Tumbler? For the average
user, they answer the problem of "put something online and make sure it stays
there" perfectly well.

------
riobard
_Disadvantages: Not for the non-geeky. You can only post from a computer where
you have your static site generator set up_

Not necessarily. I have a very simple static site generator (basically a
single Python script, plus a few dependencies) running on a EC2 Micro
instance. It watches changes in a shared Dropbox folder (using inotify and
dropboxd) of a bunch of Markdown files and templates, and generates HTML
posts, index page, and RSS automatically. I could post from any machine as
long as it has my Dropbox account, even on my iPhone! No more command lines or
git stuff, just write. Dropbox will take care of the rest.

Joe Hewitt's Nerve [1] works this way. I believe Marco Arment's Second Crack
works similarly, too, but it's not open sourced yet.

[1]: <https://github.com/joehewitt/nerve/>

~~~
bsphil
Sounds like it still requires more setup than the average non-geek is willing
to do.

~~~
masnick
Yes -- if you're comfortable with the command line and a scripting language,
you can set up a script to handle your static blog. However, I suspect most
people who want to blog don't have these skills or don't want to spend the
time messing with the tech.

------
danneu
I wasn't a big fan of the Jekyll workflow, especially when I wanted to write a
simple post from another device. My solution to that, an app that wrapped
Jekyll, just felt hacky.

Instead, I just made a Rails app with fifteen minutes spent dropping in Active
Admin and RDiscount to render @post.body Markdown into @post.rendered_body
HTML. Then, I aggressively cached pages into the static `public/` directory. I
chose Rails simply because I'm using Rails for more serious apps and I wanted
a casual app that I could take seriously enough to practice on.

Otherwise, I'd have as much fun building the same thing in any other Ruby
framework for Ruby practice and then cache the pages into a static directory.

Isn't that what any other static site generator does?

------
chubs
Don't most people with high-traffic blogs simply use posterous or tumblr these
days so they don't have to worry about these kinds of things? That's what
works for me... then the next problem rears its ugly head: bulletproofing your
DNS!

~~~
kmfrk
People with basic blogging needs are spoiled beyond belief with services like
Tumblr, but if you need more advanced things to play with, it becomes a little
overwhelming.

ExpressionEngine understood the need for people with a lot of needs, but I
think everyone eventually have to roll their own (using some sort of
framework), if they want something that satisfies their perfectionism.

------
jyap
When my blog hit number 1 on Hacker News earlier this month
(<http://news.ycombinator.com/item?id=3328619>) it never went down. Running
Jekyll on GitHub with my own domain julianyap.com.

30,000 hits in 24 hours.

I remember the "move your site off of GoDaddy" post last week going down this
week which was run on Tumblr. Elsewhere Tumblr wasn't down so it depends on
the resources they allocate to you. I suspect MG Seigler's blog is on a top
tier Tumblr resource pool so I wouldnt recommend Tumblr unless you have
constant popularity to your blog.

------
tvorryn
I was a little disappointed that when he talked about Jekyll blogs, he didn't
mention Octopress. The default theme looks pretty good, it has a decent set of
plugins and the code display looks great. It was actually pretty easy to set
up too.

<http://octopress.org/> My blog using the default theme:
<http://www.trevoroakes.com>

~~~
masnick
Good point, I'll add this.

------
ggwicz
The Jekyll recommendation is a great one. For quick one-off articles something
like pen.io is great, as well. And GitHub Pages are just fantastic if you
didn't know already :)

In my experience, MediaTemple handles traffic spikes remarkably well. Maybe
not ideal for the super-techies, but running WordPress on it with a caching
plugin handles traffic pretty well at the $20 / month plan. And using S3 to
serve static sites is fantastic. I'm on my phone right now so it's a pain to
find the specific article, but Kyle Shank uses S3 and wrote a pretty handy
article at <http://kyleshank.com>. Check it out and try to find the article.
Great read.

------
jjcm
I personally would recommend hosting on a VPS with a reverse proxy like
varnish set up. Even if you've had very minimal experience with linux systems,
it's a cinch to setup and you can get quick, fast caching up in a couple of
minutes.

------
ck2
Save dynamic page as html from browser.

Upload to server as html.

Add.htaccess rule to redirect old dynamic page to static page. (when traffic
dies, remove rule)

Or if you are running WordPress, just install wp-super-cache and when a
traffic spike happens, click the "lockdown" button.

~~~
sirn
Easier: just put this into your .htaccess on traffic spike.

    
    
        Header append X-Coral-Control "redirect-home"
        RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
        RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
        RewriteRule ^(.*)$ http://your.domain.nyud.net/$1 [R,L]
    

Or even better, just redirect all traffic with news.ycombinator.com or
reedit.com referer (before RewriteRule):

    
    
        RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?ycombinator\.com [OR]
        RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?reddit\.com [OR]
    

You don't even have to manually disable caching after traffic dies down this
way.

------
apsurd
Since discovering Jekyll and the fact that GitHub Pages will host your content
for free, I think this would be the ideal solution. However Jekyll is a pain
in the ass to use, which is why I've been working on
<http://jekyllbootstrap.com/>

Jekyll bootstrap is intended to deploy straight-away to GitHub pages so
hosting is free. I'm not finished though so any more eyeballs I get on the
source will definitely be helpful. Merry Christmas =)

------
hajrice
Surprised the author didn't mention WP Engine. We (along with SmartBear,
Foursquare and many others) use it at Helpjuice.com for our blog and it works
great.

~~~
masnick
I hadn't heard of WP Engine. I'll have to check it out. Thanks for the tip.

------
zrail
I wrote kind of a hybrid static blog setup the other day[1]. You write posts
in markdown, check them in along side the app, and then push to Heroku or
wherever else you want to host it. It's just a little Sinatra app, no database
or anything.

[1]: [http://bugsplat.info/2011-12-14-yet-another-not-very-
static-...](http://bugsplat.info/2011-12-14-yet-another-not-very-static-blog-
generator.html)

------
oldstrangers
It is easier than ever today to "bulletproof" your blog. I'm running a hybrid
server and have found wordpress + cloudflare + w3tc to be a very solid setup.
I got hit with 73,230 unique visitors (94.46% new visitors) in a 24 hour
period earlier this month and you would've never noticed.

------
jaequery
i'm sorry but this author has failed to point out the most commong cause for
why a site goes down. it's not because of failing to cache...etc...most dbs
will cache for you anyway... the main reason is because you probably are on
apache and serving static assets directly w/o hosting them off a CDN or
nginx/litespeed. if your page had 150 images/js/css combined, that's 150 hits
hitting your server per page load. now multiply that by 100's or 1,000's+
visitors HN brings to your site at any given moment, ofcourse it will go down.
apache isn't really good at serving static files, just relieving static assets
off to cdn/fileserver is huge.

------
buster
or: <http://cmdrtaco.net/2011/09/scaling-in-the-cloud/>

------
dsl
I'm really surprised by this article. I thought this was a pretty much solved
problem now that CloudFlare exists...

------
nodemaker
I find it funny (and perhaps ironic!) that people would want their thoughts
kept in some obscure row and obscure column of a remote SQL database.

But perhaps it is better than not storing and sharing your thoughts at all!

In any case...Merry Christmas HN!

~~~
zrail
How is that ironic? Thoughts have to be stored somewhere and databases happen
to be really good at storing things.

~~~
nodemaker
It is ironic because the real deep reason we share our thoughts is because we
want them to stand out!

Databases are indeed good at storing millions and billions of things and they
are designed for it.But no human being is individually capable of creating
content that really justifies a "dynamic programming oriented query plan
generation" each time someone wishes to access his thoughts!

tl;dr My argument is that static site generation for personal blogs is common
sense!

~~~
zrail
Sure. My counter-argument is that for multi-tenant hosting like tumblr where
millions of people keep their personal blogs, keeping posts in a database is
common sense because they're a known quantity. Backup strategies, optimization
techniques, operations procedures are all well known and battle tested.

Also, creating content using a web-based editor that saves to a database is
very easy for not-so-technical people to wrap their minds around. Static site
generators? Not so much.

~~~
nodemaker
Yes sites like tumblr definitely make sense...My argument is against running
your self-hosted personal blog with an SQL server! And of course...my argument
is for the technical people only....perhaps someone among us can make a static
content generation solution for everyone!

