
Stack Exchange Engineering: How We Built Our Blog - jonhmchan
http://blog.stackexchange.com/2015/07/how-we-built-our-blog/
======
allendoerfer
They lost a simple interface designed to edit blog posts and now their
marketing team has to use the Github interface and learn Markdown.

They lost the comment system and had to move the data to an external provider,
which instantly bit them.

They had to write Python scripts to munge the data.

But now they have the performance of static html. I guess just changing two
lines in the nginx configuration did not came to mind.

I think this was a very poor investment and will never pay off.

~~~
nnq
They've also lost COMPLEXITY, and this always pays off, even if in roundabout
ways that you cannot measure, and even if it's only for the _positive effects
on the mental-health of the engineers working on this!_

Really... modern WordPress is a very complicated beast and keeping all sorts
of details about how you need to optimize it, server, caching, db in you head
is just no worth it. Yeah, "changing two lines in the nginx configuration"
could have worked, but imho simply the cost of keeping in your head the
information about what two lines to change is just too large. As it grows, it
simply drives you insane by how many technical trivialities you need to keep
in your head to keep it working smoothly, when you could instead use that
brainpower so solve the really interesting problems.

An the fact that "their marketing team has to use the Github interface and
learn Markdown" is also a hidden benefit, if you think about all the subtle
frontend bugs that happen when somebody uses the WYSIWYG editor of WP and they
end up with a combination of styles that breaks the layout. For any project
above a certain size, the "dumb WIM" (What I Mean) interface of Markdown
always trumps the WYSIWYG.

Also, the benefit of mostly NOT HAVING TO THINK ABOUT SECURITY AT ALL is
_freaking HUGE!_ Developers can simply say "it's the server guys' problem"
now. Forget about having to manually audit the code of 3rd party plugins if
you really care about security. And forget about updating and realizing
something breaks and you have to fix it.

For any fast growing company, keep this in mind: REDUCE COMPLEXITY. ALWAYS. NO
MATTER HOW LARGE THE SHORT TERM COST.

~~~
pp19dd
Just as an aside to WordPress optimization, so there are two ways to do that.
One of them is to analyze content flow (how often are things updated) and
traffic flow and then come up with a reasonable strategy. Things like use
memcaching for sidebars, separate those out depending whether you need to,
then work on caching plugins. The caching plugins need work because they tend
to be programmed to allow robots to spider uncached content, which makes no
sense to me. Robot traffic on high volume sites is just insane. With a hundred
blogs or so on a multisite installation you could easily spend a month doing
postmortems and hunting down the least common denominator (oh hello
switch_to_blog, why do you use 36 MB of RAM on each call?)

Or, you can slap some ram sticks in the machine, install varnish-cache, write
a few rules for it and go play minecraft the other 29 days.

------
danso
Spoiler alert: it takes awhile to get there, but the answer is: Jekyll. Which
is neat...even as active as their blogging efforts are, as I was reading the
intro grafs, I kept thinking, "Yeah, could be done just fine as Jekyll"...I
was mostly expecting them to announce new in-house built blogging software,
but was thrilled to see that the rest of the post is about how to migrate from
WordPress to Jekyll.

------
ne01
At SunSed we are trying to solve this exact problem: A blogging platform for
professional whith batteries included. Kind of excited to see WordPress does
not work for everyone. Thanks for sharing!

------
cddotdotslash
I'm all for spending engineering time on non-project related tasks
occasionally, but this seems like an excessive use of valuable time to
reinvent the wheel. I'm sure they could have found a blogging platform that
didn't require custom markdown interfaces, Python scripting, and re-writing
everything from scratch. It's just a blog.

~~~
hayksaakian
I disagree. It seems like the biggest time-wasters were exporting all their
data from wordpress without breaking anything.

Even if they found "a blogging platform that iddn't require custom markdown
interfaces" that would not have solved the problems they faced getting data
out of wordpress.

~~~
hueving
So why didn't they just fix WordPress. If export was expensive and they were
willing to burn engineering time on it, the solution seems obvious.

However, WordPress is not cool so they went to the tired old "reinvent
blogging". No open source contributions, just building castles in their own
sandbox.

~~~
danso
I don't want to make this sound like an appeal to authority...but have you
created/maintained both WordPress and static-generated blogs (either Jekyll,
or something similar like Middleman)? Without having done both, I'd tend to
agree with you: a blog is a blog.

The opportunity cost from system migration (and re-invention) is non-trivial.
But so are the opportunities gained with a new infrastructure. The speed gains
from serving static files is nice, but to me, they are no where near the kind
of gains I make by being able to work with plain text files.

When I want to write a file, I pop open my (probably already open) text
editor, bang it out, save, switch to Terminal, push to S3. Did I make a typo?
OK, fix the typo, save, re-run the Terminal command. Maybe I want to push a
few posts, but I want to make sure they all look right...OK, run the jekyll
server and look at localhost. Then run the push command. Oh wait, maybe one of
those posts should be a draft. OK, set it to draft in the metadata, re-push,
etc. etc.

I have a particularly bad experience with WordPress because I've been running
my blog on it for five years on the cheapest Dreamhost shared server. However,
I've spent a good amount of time researching and installing WordPress plugins
to do caching...but the problem is that every goddamnned database action,
including setting a post from draft to publish, or vice versa, is a 5 to 50
second process.

Obviously, Stack Exchange doesn't suffer under that kind of hosting issue. And
they could probably set up their WordPress on a local instance so that it can
be locally browsed (in my experience, this can be _profoundly difficult_ , and
quite the weekend project). But no matter what their hosting infrastructure
is, it still doesn't negate the fact that every posting action requires a
database -- everything from creating users, setting user roles, setting post
status, and of course, editing the post.

Sure, you have to learn those things in Jekyll too...but the conventions are
easy...and most importantly, you can do them within the confines of your text
editor. That means everyone who contributes to the blog, engineers and
marketers, _gets better at their text editor_ (and operating system). This is
a much more worthwhile skill than getting better at WordPress...which is a
non-trivial skill...WordPress, to its credit, is fast moving in its
development...for occasional publishers like me, this means I spent a good
amount of cognitive energy re-learning the interface. With Jekyll, even as the
software changes, I just have to learn my text editor. And that's just
wonderful, considering how much of my work involves a text editor.

For non-techies, I imagine it's a little daunting at first...but they're
fully-functioning adults. They can handle plain text. And working with Jekyll
not only makes it easier for the engineers to do the occasional hacking of
templates, it allows the non-techies to actually _see_ it without having to
jump through the many plugin submenus in the WordPress admin. It is literally
a Cmd-T (in Sublime Text) to look up a file. Or, a Find All across a
project...which is another _huge benefit_...blog contributors often need to
look up past material...the text editor search is far superior to what can be
done in the WP Admin or other web search interfaces.

So it's not that "WordPress is not cool"...and the fact that you think that
one solution is to "just fix WordPress" makes me think you don't understand
the problem. WordPress doesn't need to be "fixed" to be more like Jekyll, it
is a fundamentally different product, with certain tradeoffs and benefits
worth pursing. To say that the OP should've "just fixed WordPress" is like
saying, "Why use Facebook? Just get better at using the CC: field in email"

~~~
rokhayakebe
_" Why use Facebook? Just get better at using the CC: field in email"_

:)

------
vlad
That’s exactly what I just did for my daily video / podcasting site Rate My
App.com.

\- Everything in markdown

\- Generate the podcasts as an RSS feed

\- Generate the list of videos as JSON, filtered inside the AngularJS app

Such an approach is not great for SEO in the short-term, so I won't be
surprised if they lose ranking because their posts and comments are missing
from the the html until their jQuery code fetches and renders them.

For me, I don't have 700 posts so that's easy to address later once I have
content worth indexing, keeping the urls the same of course. :)

~~~
chaselee
Check out [prerender]([https://prerender.io/](https://prerender.io/)) to get
around the SEO issue.

------
misterjinx
It is not possible to subscribe to feeds individually ? For instance I would
like to subscribe only for the engineering RSS feed. At the moment all I could
find was a general feed for the entire blog.

------
kmfrk
Static blogs are doing pretty well, but setting up a sepparate interface or
flow for writers - not admins and developers - is still a big of a chore,
unless you don’t mind giving people full access to the site repo.

I’m thinking about setting up a separate draft and post repo and importing
them into the root with git subtree.

------
dylz
Damn. Was hoping for a self hosted (and full featured) disqus alternative or
something to appear one day.

~~~
jonhmchan
Discourse was really, really close for us. The breaker was the lack of a real
WordPress import support: [https://meta.discourse.org/t/is-it-possibile-to-
import-exist...](https://meta.discourse.org/t/is-it-possibile-to-import-
existing-wordpress-comments-along-with-post-into-discourse/18157)

That being said, that evaluation was done _before_ we found out how difficult
it would be with Disqus.

------
philippnagel
Are there any good, corporate solutions for organizations that lack in-house
engineering resources?

~~~
saosebastiao
Jekyll is pretty low impact as far as engineering goes...all the engineering
work here went into _migration_ of the existing blog. I would think getting
Jekyll set up would be even easier than a self-hosted Wordpress installation.

------
jlarocco
It seems like this was a big waste of time reinventing Octopress.

~~~
tarr11
Octopress is very difficult to use, and I would not recommend it. Using jekyll
directly is a much better choice.

~~~
jacquesm
Octopress is built on top of jekyll and what is a real nuisance is how long it
takes to re-generate a site with more than a few posts.

~~~
Nick-Craver
This isn't true anymore. Octopress uses an older version of Jekyll. If you're
on 3.0 (which we're using) has incremental generation. Here's the log of a
recent incremental build:

    
    
      [Step 1/3] Jekyll build (11s)
      [20:52:40][Jekyll build] Jekyll build --config '_blog.config.yml' --no-watch
      [20:52:43][Jekyll build] Configuration file: C:/BuildAgent/work/4e27a39603eb2bf1/blog/_config.yml
      [20:52:43][Jekyll build]             Source: C:/BuildAgent/work/4e27a39603eb2bf1/blog
      [20:52:43][Jekyll build]        Destination: C:/BuildAgent/work/4e27a39603eb2bf1/blog/_site
      [20:52:43][Jekyll build]  Incremental build: disabled
      [20:52:43][Jekyll build]       Generating... 
      [20:52:52][Jekyll build]                     done in 9.082 seconds.

~~~
jacquesm
Cool, time to upgrade then!

------
rebootthesystem
They had me at "Not Wordpress or PHP"

