

Minecraftwiki serving more traffic than Stackoverflow with 4 servers (and PHP) - Keyframe
http://www.reddit.com/r/Minecraft/comments/drg0n/minecraftwikinet_and_minecraftforumnet_now_serve/c12ckss

======
citricsquid
I'm part of the Minecraft forum/wiki team (I created them both originally) so
if you guys have any questions, feel free to shoot them at me, or you can join
#rsw on irc.esper.net.

(in advance I'm not one of the sys admins, although I guess I'm a php
developer, but I suck too much to work on the sites at these traffic levels, I
just do the whole community stuff)

Also to clarify, the forum + wiki combined when ignoring our downtime are
pushing more traffic, not _just_ the wiki.

Edit: Also to hijack my own comment, if any of you guys with your fancy
startups want to advertise to a community of indie gamers feel free to email
sam@redstonewire.com :-)

------
points
Is this the shape of things to come?

Comparisons of apples to oranges... It's not about traffic, it's about what
sort of website you have, how dynamic/static it is, etc

A wiki likely has relatively few writes compared to reads, so caching should
work very well.

That said, always nice to see people optimizing properly and using a sane
number of servers.

~~~
superjared
That's kind of the point. Yesterday Spolsky asked why Digg had 500 servers and
less traffic than StackOverflow, yet StackOverflow had 5 servers.

False dichotomies huzzah!

~~~
bad_user
Yes but StackOverflow is very similar to Digg:

you've got articles that have rating, you've comments to those articles that
have rating, articles and comments are sorted according to rating, people earn
karma from posting good comments / articles, everything has tags.

So personally I don't see a false dichotomy; even if Digg is more dynamic /
complex ... WTF are they doing with those 500 servers?

~~~
qeorge
Digg is more like Facebook and Twitter than StackOverflow. Each of Digg's
logged in users gets their own "News Feed" based on the users they
follow/friended/are most similar to.

StackOverflow on the other hand is much simpler - questions and responses,
plus users and voting. AFAIK there's no collaborative filtering going on at
SO, be it user or item based.

I don't know why Digg needs so many boxen, but I did find Spolsky's comparison
disingenuous.

------
rythie
I think these types of stories are misleading for startups.

Many startups would do better to add server capacity in the short term, rather
than spend lots of time optimizing to cut costs, when this is typically hidden
from the user.

For example, a 4GB linode VPS is $160/month, so you can have 34 of those
($5440/month) for the cost of one developer (Salary of $67k based on:
[http://www.simplyhired.com/a/salary/search/q-php+developer/l...](http://www.simplyhired.com/a/salary/search/q-php+developer/l-california)).
Also, many startups struggle to recruit good developers, so would it make
sense for them to spend all their time optimising code to perform on cheap
hardware? rather than improving the product in a visible way to the user?

~~~
patrickgzill
For $200 per month you can get a quad core X3220 with 8 GB RAM and 2x 500GB
disk with a large amount of bandwidth included: <http://www.100tb.com/> .

I don't fully understand the love for large VPSes (that aren't even all that
large) compared to dedicated hardware that have a better chance of having
higher memory bandwidth, more RAM, and faster disk access; though I do
understand that many are very happy with Linode as a business.

~~~
citricsquid
The ability to grow with a vps is much easier than with dedicated hardware.

Also on a semi related note, I (like you) suggested 100tb.com but we tried
them out (just to test speeds) and they're pretty poor...

~~~
patrickgzill
May I ask what part of the speeds were poor for you? I am curious.

~~~
citricsquid
I'll talk to the guy who actually tested them when he wakes up, but from what
I understand network speeds were terrible. I'll get back to you when I know
:-)

------
jbk
Sorry, I don't get the fuss about this.

I do 1M pages/day in average since more than 1 year on one unique server that
is a bi-opteron 250 at 2,4Ghz with a load average of 0.3...

We just serve mostly static content, and most php content is cached. I just
think that this comparison to SO and /. is flawed.

~~~
necro
I agree. Over here we do 70m (high write ratio) pages per month on 1 server
handling all apache/php/mysql. Hardware is really fast these days if you tune
it to any degree.

Heck, if we're showing off, here is how we do it...pretty graphs and all.
<http://www.pinkbike.com/news/pinkbike-speed-efficiency.html>

~~~
StavrosK
That was a very interesting article, thanks. One question, if I may: When
using a reverse proxy, it makes no sense to have keepalives on for Apache,
correct? The proxy takes care of the keepalive and leaves Apache free for
other requests?

~~~
necro
Correct. The reverse proxy pulls from the fast, local network apache, and then
passed the data to the slow clients. Apace is connected for a shorter time.
Basically you're trying minimize the time a "memory expensive" process like
apache is open per client.

~~~
StavrosK
Yep, makes perfect sense, thank you. I've disabled keepalive and increased my
mancrush on varnish.

------
bill-nordwall
If they enabled gzip compression on their CSS/Javascript files could cut down
their page weight by several hundred kb.

Even just running their pngs through a lossless compression tool like Smush.it
would probably be worth it: <http://news.ycombinator.com/item?id=1796101>

PageSpeed and YSlow yielded many other fruit-bearing ideas.

Update: Added link to Autosmush HN item.

------
astrange
I'm running a website with 15% more quantcast visits than stackoverflow and
~200MiB/s traffic from 3 servers, one of which is nearly at 100% idle.

The trick is not to have any of those dynamic page things.

------
citricsquid
Also a request if any of you are experience with it, we're interested in new
ad strategies (that retain our "minimalist" approach, allowing expansion
without upsetting users) so if anyone either works for an advert company _or_
has experience at our volume (or similar) we'd _love_ to hear from you.
sam@redstonewire.com :-)

(Hopefully this is okay, I'm not a regular HN user, mainly lurk, it isn't
mentioned in the guidelines but it might be one of those secret rules that are
learned as you go along... be gentle!)

~~~
benologist
Isn't minecraft making like, a house a day in revenue? Is it really worth
tossing ads in forums?

Anyway I'd suggest you guys chat with CPMStar - <http://www.cpmstar.com>,
they're the king of gaming-ad-stuff.

Drop me an email (check my profile) and I'll pass you the addresses I know
there.

~~~
citricsquid
We're _not_ connected with the people behind Minecraft, we're totally separate
entities. The forums and wiki are community ran and while we've had brief
discussions with the Minecraft company (Mojang) nothing has come of it. The
general consensus is that we're best operating as separate entities, it means
Mojang can focus on game development and we can focus on growing the
community, it means that we can remain impartial (although whether or not that
is an issue, I have no idea). We've never had a penny of Minecraft proceeds
:-)

I'll send an email now, it'd be great to talk to some people over at cpmstar!

~~~
benologist
Oh sorry mate my mistake heh.

------
sams99
It has been quite a while since we served one million page views a day....
even on public holiday weekends we serve more.

I'm surprised this got upvoted so much, I could easily serve 2 million static
pages a day off one server, if I needed to and the pages were static.

The assumption that we are using the same hardware and have similar workloads
and so on, is clearly wrong.

We could spend months and months tweaking everything so we need 2/3/4 less
servers .... but ... servers are cheap, developer time is expensive.

Also, we happen to have backup servers, we are not running at 100% utilization
an we also happen to run chat off the same servers.

I think it is awesome that minecraft are serving lots of traffic, I love
nginx, we use haproxy. But the headline is misleading.

~~~
citricsquid
Just to clarify, the OP here titled it in a manner that misrepresents what we
were saying. Also, we're far from serving static pages. Granted the wiki
(which is 50% of our traffic) is pretty static and we could easily run that
from a single server, the reason we have such high number of servers is
because of the forum, which is the other 30m page views and it's phpbb,
it's... well, let's not go there.

This submission is poorly titled, our intention was never to claim we're
better than SO (we're very different... just like SO is very different to
Digg) it was just a good comparison to make, as in "Joel said they're serving
60m page views a month and SO is huge, well we're doing the same, now you can
see how big we are!" not "We serve the same as SO, therefore they suck!".

~~~
sams99
You really should be using community tracker, my other baby :)
<http://community.mediabrowser.tv/> I'm so happy I moved off phpbb it was
causing nothing but grief.

No hard feeling here, I think you are building an awesome business

~~~
citricsquid
We're in the process of moving, it's just a lot of work at our size, we have
to make sure everything works :) We actually started out with fluxbb (my
choice) but users got tetchy and as we moved from being "just a forum" to
being a "community" we had to go forward with new features, but this was back
before we had adverts and the $250+ for "proper" forums wasn't something we
wanted to do. Here's an idea of how much we've grown:
<http://i.imgur.com/eenut.jpg>

That software looks _interesting_ although as I'm not a sys admin, all that
matters to me is how pretty it is and that doesn't have enough rounded corners
;)

------
spolsky
I don't get how Minecraftwiki is serving "more traffic than StackOverflow." I
think we have at least twice their daily traffic. All our numbers are on
Quantast--feel free to check.

~~~
citricsquid
As I've said elsewhere, the person who titled this is a silly person, they
misrepresented what we said.

Wiki + Forum = 60m page views a month, so _combined_ we serve the same amount
of traffic as you, as per this tweet:
<http://twitter.com/#!/spolsky/status/27244766467>

We weren't challenging you or anything, I just noticed that tweet (it was
mentioned here) and I thought "hey we're doing the same, we can use them as an
example of how big we are!". I'm just a dumb kid who has never had anything he
created this successful before and being able to say "We're as big as
stackoverflow" is _crazy_.

~~~
Keyframe
_As I've said elsewhere, the person who titled this is a silly person, they
misrepresented what we said._

Yes, you've said that several times. Intention was two-part. One, to show how
previous post by spolsky was false dichotomy, and second to point out to your
success - which almost everyone here understood as such.

However, your original post on reddit (well your sysadmin) was titled:
"Minecraftwiki.net and minecraftforum.net now serve more traffic than Slashdot
and Stackoverflow" - so you can't blame that part on me, just the server count
and php part.

~~~
citricsquid
I'm not good at the whole English thing (even though it's my only language). I
didn't mean to imply you were at fault, just that your title didn't represent
what was _actually_ happening. Also I didn't realise that you'd posted a
comment here pointing out it was supposed to be a joke, I'm used to reddit
where it points out that a comment is by the submitter.

Poor wording and a mistake on my part, sorry! :-)

------
joshu
iirc 4chan is doing radically more traffic on even less hardware. different
sites have different performance... so what?

~~~
enneff
Does anyone have any actual numbers on 4chan's traffic or hardware setup?

~~~
joshu
I do, but am not at liberty to share.

------
Fluxx
This is more a testament to HTTP caching and varnish than PHP, 4 servers or
Mediawiki. If you can cache the entire page and serve it out of cache for most
of your requests, you're in a very position.

~~~
swah

      very position
          ^- insert word here

~~~
Fluxx
"good" :)

------
JoelSutherland
Is minecraft that big, or is the tech world that small?

~~~
citricsquid
No, Minecraft is seriously that big. If you trust Alexa much you'll find that
we (forum/wiki) are in the top ~5k for both sites, Minecraft is top 3k last I
checked. It's been insane recently... what really hammers it home is that this
is a product people have purchased, so it's going to be around for a long
while. While we probably won't maintain the current traffic once the game
settles down into a normal routine, we sure won't be dying for many years,
which is what I love about this.

Minecraft is like garrymods, the game is what _you_ the player want to make
it, this will lead to a lot of future success along side this current success.

Also if you want to see the sales figures, I've been tracking them for the
past few months: <http://m00d.net/minecraft/sales/> :)

*If you're interested, here's a (not very accurate) list of where Minecraft has been featured: [http://www.minecraftforum.net/viewtopic.php?f=3&t=2162](http://www.minecraftforum.net/viewtopic.php?f=3&t=2162) which includes Australian TV, Physical magazines, huge tech blogs, gaming blogs, forums... everywhere! I don't think I'll ever see anything happen like this again in my life (and I'm only young) -- Minecraft is incredibly unique.

~~~
mayank
> the game is what you the player want to make it

Interesting -- I know nothing about it, but it sounds like it might appeal to
people who want to learn video game programming, at least perhaps the ones
that don't want to go into hardcore engine programming.

~~~
AdamTReineke
It's more like playing with LEGO than anything else.

------
piotrSikora
I'm just wondering... Why do they use Varnish _and_ HAProxy _and_ nginx? This
is quite redundant setup. It would be _a lot_ more efficient to put nginx on
lb01 and leave only PHP on fe* nodes.

