
Ask HN: I built a site that just went ridiculously viral. What do I do now? - markbao
Hey HN,<p>So I built http://threewords.me - literally an MVP that I posted on Facebook and my friends started using. Fast forward 3 days. The entire Twitter results page for "threewords.me" is of tweets that happened less than five minutes ago. The site grew 2x in pageviews over the past <i>hour</i>.<p>Two problems: 1) what do I do now? 2) how do I afford this?<p>Advertising? Hosting partner? ...daresay... <i>investment</i>?<p>Sorry to be brief. If you're curious, the stack is Rails + Ruby Enterprise Edition + Passenger + nginx, which isn't cheap like PHP to host. On the $40 Linode right now but maxing out CPU at 350%.<p>227K pageviews today. 50K uniques. 71% traffic referred. 8,285 users. 3,100 new users in the past 1 hour.<p>What happens now?<p>EDIT: Hello! This thread is not going unnoticed while I take my metaphorical fire extinguisher to the fires that are happening. Will reply soon.<p>EDIT: cranked the Linode up to 4096. $160 server, woo!<p>EDIT: David from Duostack (http://duostack.com) is helping with the load on his cloud Ruby platform. Many many thanks to him.
======
jeromec
No offense, but putting it on HN in the middle of traffic spike pressures
would not help things. ;)

I was able to sign up and see just enough to get a feel before the site became
unresponsive. I like that it's dead simple. I can also see how it's incredibly
viral as it asks you to ask others to describe you -- very smart. At first
glance it looks to me like something that could enjoy tons of traffic, but
which would probably be short-lived. The challenge would be finding a way to
have that traffic stick around...

I view this as a bit of a long shot, but I would do a few things. First,
you've _got_ to have the site handle the traffic. No advice on how, but find a
way to get that done. Next, I would try to become known as _the place_ to get
a quick summary on anybody -- even celebrities. I imagine people will receive
multiple adjectives from different people, but I'd have the site tally the
three most popular and promote those as best describing of the user. :) I see
a couple of possible pages for "viewer stickiness". The homepage could feature
very popular users. Imagine showing the three most submitted words to describe
/paulg or /marrington and other users with large responses... Next, on each
user's page it could show their "friends" and the words that describe them.
The site also provides the most recently submitted three words on the user's
page. This might at least make each user curious about checking back at their
own profile to see how people were labeling them. As for monetizing, that's
probably with ads, as usual, but you have to be careful about when and where
to put them. Like I said, probably a long shot, but who knows? Good luck!

~~~
astrofinch
>As for monetizing, that's probably with ads, as usual, but you have to be
careful about when and where to put them.

How about analytics? For each person who writes about me, I could pay to see
where in the world they lived, how long it took them to type in the three
words (or even watch a movie of them typing them in), whether they clicked the
link that was on my Facebook or the one on my Twitter feed (based on HTTP
referrer), etc.

If you were confident you could give each user the ability to see this
information for 1 user for free, and then have them pay if they wanted to see
it for the rest of the users.

~~~
jeromec
Hmm, that's interesting. Very popular users might find some use for that kind
of thing. For example, up and coming music artists trying to build a following
and connect with fans maybe... I definitely think sites should consider
charging when and where possible, but sometimes there is good opportunity for
it, and sometimes not.

------
gdeglin
I recently built a Facebook application that approached the same kind of
growth.

Sites like this typically see eCPM between $0.10 and $0.30 for advertising. So
you're looking at around $30/day revenue right now.

Consider switching to EC2. This will allow you to easily scale up to a more
powerful server if growth continues or down to a less powerful one if traffic
tapers off.

Depending on how you set things up, a common bottleneck for Facebook
applications is API calls. Ensure you are not making any API calls from Rails,
since this will cause Rails processes to hang until Facebook returns results.
With high traffic this could easily cause your server to become overwhelmed.
Move all API calls to Delayed Job or another background processing system if
you haven't already.

It looks like you have relatively few remotely loaded assets, which is great.
You can likely speed things up more by making sure that your database is well
optimized for the server (Make sure query cache is enabled, and you have
indexes on the right columns for example), and identify any actions that take
a long time to return.

As for what to expect in the future, growth will not be infinite. There are a
finite number of people on earth that are going to love your site and Facebook
integration is a fantastic way to reach nearly all of those people in a very
short amount of time. After a while most of those people will get bored and
you will see traffic begin to fall. The best ways to address this are by
consistently releasing new features to reengage users or by referring your
users to other similar sites that you build.

~~~
klbarry
Would it be possible to see your Facebook app? I'm curious to see what traits
these fast growing apps have in common in person.

~~~
gdeglin
Sorry. It got banned from platform for a few reasons including tagging photos
that didn't have people in them. (Similar to the cartoons that people upload
and tag their friends as characters).

In general the biggest trait is that the app is built around sharing. Apps
where sharing is a secondary feature generally do much worse in comparison to
those where the whole point of the app is to share something. Even successful
games on the Facebook platform universally require users to share content with
friends to progress.

Many of the most basic viral apps are simply based around sending gifts to
friends. If 10% of users that receive some sort of virtual gift then go on to
send it to an average of 11 friends, your app will be highly viral in no time.

~~~
zachster
That's really interesting. My recent viral app, Profile Banner, also utilizes
tagging of photos without people. How long did it take before your app was
banned?

------
jacquesm
hey Mark, if you need server capacity I can set you up with four machines on
very short notice, let me know. j@ww.com (no charge, help you handle the spike
and see if it has staying power without having to get in to long term
contracts).

~~~
jasonlbaptiste
^ This is reason why I love this place and never believe the occasional
"hacker news is going to shit" posts.

~~~
alnayyir
To be fair, Reddit has a noted capacity for charity and generosity, but I do
agree.

~~~
pavs
>To be fair, Reddit has a noted capacity for charity and generosity

Understatement of the year.

~~~
alnayyir
>Understatement

Entirely necessary here, even a hint of hyperbole and the pedants come out of
the woodwork.

~~~
izaidi
_Entirely_ necessary?

------
cd34
Where is your current bottleneck? mysql? check your slow query log, turn on
the log-queries-not-using-indexes, do a quick analysis of those to see if you
have queries that aren't using an index or queries that can be optimized.
Probably 90% of the scaling problems are database issues.

The quick solution is to scale up your current linode package, however, I
think you might find that your traffic is going to peak and will wane next
week after everyone has gone back to school/work and is out of the holiday
mindset at which point you can scale back.

Put in a throttle - /proc/loadavg

Limit it to X signups per hour, asking them to check back next hour. Collect
email and notify when they can sign up. Point them at a facebook page, ask
them to like the page, do announcements when the next 'batch' of users is able
to sign up.

If you want some help analyzing the slow query logs and/or the normal logs,
email me a url of a .gz/.bz2 of a reasonable snippet at
hnusername@hnusername.com.

Since you're running nginx, consider doing an alias and use proxy caching. I
would doubt your static files are causing too many issues, but, if you can
cache computationally expensive pages a little, that can help.

~~~
Luyt
Last week I optimized some MySQL queries which triggered the 'not-using-index'
warning, even when I had all fields which were used for selecting and sorting
indexed. I think a JOIN on a nested subselect somehow confused MySQL: when I
duplicated some extra restraining WHERE clauses into the subselect, response
shot up from 800msec to 12msec per query. Very nice. But the query still shows
up as 'not-using-index', which is strange.

~~~
cd34
MySQL's query plan is odd at times. What does an explain look like on that
query? If any portion of the query, including subselects isn't able to use an
index, it will report it. You can try to force indicies with USE INDEX/FORCE
INDEX which can sometimes help.

~~~
cd34
<http://cd34.com/blog/scalability/mysql-query-optimization/>
<http://cd34.com/blog/scalability/mysql-5-1s-query-optimizer/>

------
kabuks
Your site seems to be down.

Think about moving the code over to Heroku now, and keep scaling to meet
demand.

If you don't have the cash, email me and I'll front you what you need to keep
it up until you figure out how you want to roll with this. This is not a time
to trip over pennies.

Congratulations.

~~~
latch
Might not be a time to trip over pennies, but there's also no reason to rush
into things. Heroku has always seemed to be horribly overpriced as a scaling
solution.

A more traditional approach of 1 web and 1 db server (possibly even physical
boxes) might end up providing better value and give him enough time to figure
out how to scale this horizontally on EC2 (the web part should be
easy...remove any in memory state data, the db might be a bit more work).

~~~
steveklabnik
The release of Hackety was fairly large, I got LifeHackered, TUAWd... 50k
uniques, 100k pageviews over three days. Heroku's free account didn't blink.

~~~
moe
Sorry but that's nowhere near "huge" in terms of server load.

Even a small machine can trivially handle 150k pageviews per _hour_ , even
with ruby strapped around its neck.

Spiky traffic is an issue of its own, but it sounds like Mark's site is
growing fairly steadily (albeit steeply).

I agree with latch that heroku becomes (ridiculously) overpriced the further
you depart from the free plan. If a stopgap is urgently needed then it may of
course still be a valid choice. But in the midterm, if you're pushing serious
traffic, look elsewhere.

~~~
steveklabnik
I'm just saying that it was more than I expected out of a totally free
account, that's all.

I have yet to use them at higher loads. We shall see.

Also, it's my understanding that they're still less expensive than something
like EngineYard, and still generally cheaper than paying for a sysadmin.
Dedicated hardware will of course outperform any of this stuff, but I also
don't have to think about it at all. Even VPSes involve spending initial setup
time and then periods of upkeep, you can't just install and say "done."

Then again, I'm coming from a mostly theoretical standpoint. I don't have real
world experience at true scale.

~~~
moe
Yea sorry if I came across rude, didn't mean to attack you or heroku, just
wanted to set the figures straight.

With regard to heroku you have a point about the "not needing a sysadmin"
part. That, however, only works for a fairly short period during the lifecycle
of a business; heroku is ideal for bootstrapping.

Once your site grows to the scale that mark is seeing (if that persists, which
I doubt) then the heroku value proposition rapidly shrinks.

At that point you start to need more customization than heroku can provide
[for a reasonable price] and you also need at least one person with systems
knowledge in your company to prevent expensive mistakes in the software
architecture.

Gladly that person will then more or less pay for itself, simply by moving the
app to a cheaper hosting platform.

~~~
steveklabnik
No worries, absolutely no offense taken. :)

------
dools
EDIT: HN doesn't render that code very nicely, here's a pastie:
<http://pastie.org/1416631>

Hi there, your home page is very simple, render it in plain HTML and post the
signup to queue.php.

In queue.php put:

<?php $data = serialize($_POST); mysql_connect('localhost','dbuser','dbpass');
mysql_select_db('dbname'); mysql_query('INSERT INTO cache(cache_data)
VALUES(\''.mysql_real_escape_string($data).'\')'); echo 'YOUR REAL PAGE SHOULD
BE HERE WITH A MESSAGE ABOUT BEING IN THE QUEUE'; ?>

The cache table should just be a PK cache_id and a text field (or perhaps
mediumtext). I've suggested using a database instead of a filesystem based
cache because, if you're doing this quickly, doing it using MySQL means you're
less likely to run into file permissions or security problems.

Then you're at least capturing everyone's information whilst you figure out
how to scale.

------
nzadrozny
You've since mentioned that you've gone for a 4 gig box with 55 Passenger
workers. Definitely a good start.

If I were in your shoes, these three items would be next on my emergency-
scaling checklist:

1\. Database sanity check: Are you using indexes for calls like "Find user by
username"? Does your database need its own dedicated box?

2\. Pick up another box for memcached. Full-page cache everything you can with
rack-cache. Don't worry about expiration, just use some reasonably low TTL to
put a throttle on your read requests for now.

3\. Move all of your static assets to S3 + CloudFront. Gzipped, with a far-
future Expires header. While you're in there, move the JS to the footer of
your layouts. No sense serving any of that stuff from your Linode boxes if
they're as pegged as they sound from your other comments.

As to making money? No clue. Maybe I'll have an idea when I can get a request
through and see what the app is all about ;)

One thought occurs: this seems to be a very vanity-driven kind of app. Find a
way to charge $1, $2 or even $5 for some simple premium vanity-stoking feature
and you could be in good shape. Better than advertising, in my book.

Have fun!

------
jimboyoungblood
1) In general, the database is _always_ the bottleneck. It's easy to throw up
more webservers- can't do that with db's.

2) Throw up an ad or two, with the goal of having the ad revenue pay your
server costs. This will allow you to not worry about the financial situation
while you're fighting fires, so you can defer thinking about investment etc
until later.

(now after you have the servers tamed...)

3) Make sure you are incorporated. You probably want to be an S corp or LLC.

4) Look at your metrics and decide whether you have a sustainable property or
not. (Look at your retention numbers) Based on the data, decide whether you
have

(a) a self-sustaining cash cow, (b) something that can be huge and take over
the world, or (c) a fad that will grow and die quickly.

If (b), and you need money to get there, start talking to investors. But don't
take money too hastily- it's a long term commitment and you'd better know what
you're getting yourself into.

If (c), try to sell it to someone who thinks it's a (b) before it dies.

Congrats and good luck!

~~~
Natsu
Not so sure about incorporating, but that's because I spent my day writing up
notes on Camz's talk on incorporation. He gave the three reasons to
incorporate:

1\. You have significant profits or net income. 2\. You have high liability.
3\. You're getting funding.

I doubt this meets any of those criteria, unless someone offered to fund him.

Here are the relevant links if you want to see what I'm talking about: HN
Story - <http://news.ycombinator.com/item?id=2047818> Podcast -
<http://cameronkeng.com/podcast-ep-1-incorporation/> Summary/Notes -
<http://pastebin.ca/2033868>

~~~
escoz
One thing he doesn't mention is how much "significate profits" is.. Would 1k
be significant? 10k?

For example, how much would I save by incorporating if I make 1k versus 10k
versus 100k a year?

------
rbxbx
<http://markbao.com/about>

You're 17/18?

Kids these days. Geeze.

Fine work sir, though I don't have much advice to offer beyond what has
already been stated.

~~~
mickdarling
Yeah, and I met Mark 3 or 4 years ago and he had already started at least one
and maybe 2 successful startups.

~~~
swombat
According to <http://juniorbiz.com/interview-mark-bao> (February this year)
Mark has started 11 businesses and 3 foundations, and sold 3 of them already.

------
Skywing
Well, it's a nifty idea and I know I've seen that trend on Facebook before.
But that's also the problem - it's a 1 time use fad type thing. I don't know
how much I'd invest in the current incarnation - perhaps just enough to keep
the site functioning. Aim for some kind of traffic-based business strategy
like ads or something. Maybe it can help pay for the site itself. If you could
perhaps take the current idea and evolve it into something with a little more
sustaining power then perhaps you could turn it into a nice little side income
thing.

~~~
phlux
let the words become tags linked to the FB account. Such that if users have a
tag like "Snowboarder" associated with their FB account - he can build out a
tagDB of FB accounts.

This way - you could target ads to FB accounts/users that are all
"snowboarders"

Or you can at least search who among your friends have the tag [snowboarder].

Or if you give more categories to the tags, you can ask your friends to fill
in _their_ favorite 3 [X] -- so you can simply ask all your friends a question
and then build an interesting DB of tags/info/interests of all FB users.

~~~
spullara
You mean like this?

<http://www.facebook.com/pages/Snowboarding/107496599279538> 1,784,641 people
like this 9 Friends Like This

------
rapind
The following until you no longer have issues handling the traffic (which
might be right after step 2).

1\. Upgrade your linode to something around 2 Gig for now.

2\. Start playing with your MySQL or Postgresql config (assuming you're not
already using redis or something similar, in which case just throw more RAM at
the box). Google performance settings for your DB of choice and look for
relevant posts to the memory you have.

3\. If there's anything cachable, install memcached and implement it.

4\. Purchase a second linode and move your database to it.

5\. Upgrade each VPS as needed.

6\. At this point you get into fancy scaling and there are loads of options.
You should have plenty of breathing room by this point though and can plan it
out.

~~~
markbao
Upgrading to the 4 gig linode as we speak. this is getting kind of ridiculous.

~~~
rapind
The linode guys are great. You might want to flip them an email and let them
know you're getting hit. Could speed up the resizing etc.

------
ElbertF
This has happened to me a few times and usually the Twitter/social media storm
blows over fairly quickly. I suggest getting an extra Linode for now and wait
a few days to see what happens. If the numbers do stay up the site will
probably be able to pay for itself with ads. Your site is down at the moment
so I can't really see what it's about.

Edit: Google Cache got it:
[http://webcache.googleusercontent.com/search?sclient=psy&...](http://webcache.googleusercontent.com/search?sclient=psy&hl=en&safe=off&q=cache%3Ahttp%3A%2F%2Fthreewords.me&aq=f&aqi=&aql=&oq=&gs_rfai=&pbx=1)

~~~
jackowayed
Rather than get another Linode, for the very short term, I'd just crank the
current Linode as big as they go (I assume Linode has super-quick resizing)
and tell passenger to run a bunch more processes.

~~~
markbao
Just cranked it to 2048MB. Cranked Passenger to 55 workers in the pool. Using
400% CPU still. Linode's probably not happy.

EDIT: Cranked to 4096

~~~
jackowayed
I doubt they're unhappy. Resources are shared in such a way that everyone is
guaranteed their fair share. I think the 400% reflects that you have access to
multiple cores. 100%=maxing out one core, I believe. (And you might not be
truly maxing it out, just maxing out your share of it.)

~~~
lsc
If you want to see how much CPU is being denied to you because other XEN VMs
on the box are using it, in top, look at "steal" - that's the percentage of
the CPU that you want that you are not getting because it's been allocated to
other people.

Xen is pretty great about partitioning everything except for I/O. Network I/O
probably won't be a problem for you (xen doesn't really start getting tripped
up until you are well above 1G/sec. I /believe/ linode has a 10,000 packet per
second limit on top of that, but that's a whole lot of packets. This is
probably not your problem.) but disk I/O will be.

Cache everything. If you cache enough, and have enough ram, slow disk I/O may
not be a problem at all. watch your linode to make sure you don't go over
quota on your disk I/O allocation... maybe email support to make sure you are
in the clear. My understanding is that linode has better disk I/O than I do
because they have a process in place for limiting heavy disk users. (my
understanding is that it's a good process, they warn you before limiting you
and generally do it in a fair manner. I plan on emulating them in this regard.
)

No two ways around it; I/O to spinning disk and virtualization go together
poorly. What would have been a sequential access on a dedicated server becomes
random access... and on a good day, that means 1/10th the performance.

If you are using a lot of disk I/O, a dedicated server is about your only
option. Considering the prices nearly all American providers charge, if you
plan on keeping it for more than 2 months, you are nearly always better off
buying. (and that way you can buy better disk, too.)

Of course, if you aren't limited by I/O, or if you can solve the I/O problem
using ram caches, then it might make sense to stay 'in the cloud' on linode or
ec2 or the like until you are sure you are going to need the space long-term.

------
david_shaw
Adsense. With only about 10k hits a day, my bedtime calculator
<http://sleepyti.me> brings in $10-$20/day. At the rate you're getting users,
you could probably bring in a significant load of cash. Good luck!

~~~
KeithMajhor
"Sleepyti.me is provided as a free service thanks to support from our ads."

I don't know if you read the "Sacked by a Google Algorithm" article from
yesterday.
[http://www.duckworksmagazine.com/11/columns/guest/winter/ind...](http://www.duckworksmagazine.com/11/columns/guest/winter/index.htm)
The comment you make on the results page sounds dangerously close to what got
Adsense to boot him.

~~~
david_shaw
Thanks for the tip. I saw that article on HN the other day, but the wording is
designed to encourage people to leave adblock disabled to see relevant, sleep-
related ads. You might be right though, it's always better safe than sorry;
I'll look into clarifying the wording there. Wouldn't want to offend my Google
overlords!

------
blader
Advice from a guy who's gone through this a few times:

1\. Monitor your retention. Are people coming back or just checking your site
out once? You can go viral very quickly but tank just as quickly once the
initial interest dies down.

2\. Start thinking about a business model. Highly viral sites that don't
retain well could really hurt your bank account if you're not monetizing. If
you can't can come up with an interesting one, throw up some ads to tide you
over.

3\. As soon as possible, get your database on dedicated hardware. Your
bottleneck is going to be the database: so memcache, add the right db indexes
(get a backup db so you can hot-swap while you do this), and add lots of RAM
if needed. You don't want to be scrambling to migrate your database once you
hit the limit.

------
pathik
I don't suppose this traffic will sustain. Slap an ad right in the center and
monetize the hell out of it. Or get a sponsor.

Also, for the short term, Amazon EC2 would be the best option to handle the
traffic.

~~~
epi0Bauqu
I'll sponsor, or more accurately, DDG will.

------
catshirt
the site got twenty two hundred hits within two hours?

~~~
mindcrime
Thousand... twenty-two thousand.

~~~
jaekwon
wow

------
spullara
My guess is this app doesn't need the resources he is throwing at it. If you
look at the pages it is dead simple. I can't imagine what was using all the
CPU. Would have been good to get a TOP or something to see what was going on.
Since it is on Ruby, I recommend getting it on NewRelic and have it tell you
where the bad parts are.

------
cmelbye
Heh, I was wondering what this was when I first saw it a few hours ago. I've
seen 5-6 people I follow on Tumblr use this thing already.

I suggest migrating it to Heroku. Push your code, copy over the database, and
crank your dynos up. They charge by the hour, so as an example if you have ~10
dynos running (roughly equivalent to 10 thin instances, but slightly better)
for 5 days (or however long it takes for the initial traffic surge to
subside), you're only going to pay about $50. Then, you can turn down your
dyno count and pay a reasonable monthly fee.

~~~
jarin
There's a bit of magic involved with things like compass/sass and Paperclip on
Heroku (you have to use Amazon S3 for storage and there is some other setting
I can check on).

Anyway Mark, I have a bunch of apps on Heroku that use all kinds of wacky
stuff—including the aforementioned gems—so email me at jarin (at)
robotmodehq.com if you decide to try out Heroku and get stuck. I can send over
working code snippets that you should be able to just drop in.

~~~
jarin
I got a few emails about the Compass/SASS on Heroku thing, so I posted it up
on my blog in case anyone who didn't email me needs it. Compass and SASS are
notoriously painful to get up and running on Heroku, and everyone says to use
the Hassle gem but it never works. So here's the SUPER MAGIC WAY:
[http://jarinheit.posterous.com/the-ultimate-secret-to-
gettin...](http://jarinheit.posterous.com/the-ultimate-secret-to-getting-
compass-to-wor)

------
markbao
I haven't forgotten about this. I need sleep (exhausted and have been in bad
health lately). Will get back to this soon.

------
rms
Don't forget to apply for the Thiel Fellowship! Email me for help with your
essays (that offer applies to anyone).

~~~
rdl
Yes, if there is anyone I know who should apply for the Thiel deal, it is
Mark. Especially since he has said he doesn't exactly love his college. Thiel
should make moving to SF a requirement too.

------
adebelov
awesome, on Techcrunch 10 min ago:<http://techcrunch.com/2010/12/30/three-
words/>

------
sjtgraham
Hey Mark, first of all congratulations. Do you have a caching layer? If not
now is the time to implement one.

------
ivankirigin
Consider adding a revenue stream immediately. Let users pay to see what others
have said about friends. Charge 1 facebook credit. Also let users pay to block
that access: 10 credits. Also pay to let others ignore the block: 100 credits.
Etc...

You'll need to redirect to a facebook page to use credits because they aren't
available off facebook.com yet. They don't have many good examples of offsite
apps using credits, so they could streamline the approval process I bet. Email
me with questions.

~~~
bertil
I'm not sure all your users are familiar with Facebook credits; I'm not sure
you want to set up barriers before you know what you have.

~~~
ivankirigin
If he isn't making money with an app like this, he has nothing. Ohh and
millions of people understand facebook credits. I can tell you with pretty
good knowledge that the 1-credit transaction is a good idea for this kind of
application.

Certainly this hasn't been proven with apps off of facebook. It's an excellent
experiment though.

~~~
far33d
I thought Facebook credits weren't even available as an option for Connect
apps?

If you really want to grow even faster:

1) scrap the standalone website and build it as an iframe app (should be very
little work). This will simplify login significantly and will keep people
inside FB, which in my experience, makes a world of difference in the viral
funnel.

2) replace the "share this link" with a real Stream.publish call. Consider
adding like buttons for every single user's page.

3) Allow people to see which of their friends have answers as well.

Then, if you add the 1 credit transaction, you should do pretty well. Beware,
however... Facebook has given out a lot of free credits to users and they do
not give app devs any payback when free credits are used. These chargebacks
can affect actual revenue from small transactions like this significantly.

------
Void_
Why did you build it? What was your motivation? I tried it, and I have no idea
why would I spam my friends with link so they would describe me. I have
absolutely no interest in that, and I would feel really stupid sending anyone
link like that.

Did you actually felt the need to have such a website? Or you just figured
people will like it? How did you figure that out?

Please, share.

~~~
astrofinch
Most people have a strong interest in how others perceive them, including
yourself: "I would feel really stupid sending anyone link like that." You're
probably part of a culture where publicly asking others what they think of you
is frowned on.

~~~
Void_
It's not the culture. I care about what people think about me, but not as much
as asking my friends to describe me it three words.

I think I understand it now. People use this, because they wanna hear people
saying good things about them. I think most people would send link to their
friends, because they don't feel very confident about themselves.

It's the basic principle of Facebook (or of whatever Facebook turned into) -
people post things so they would get positive feedback and they would feel
better about themselves.

~~~
astrofinch
Well speaking for myself, I am pretty clueless on stuff like whether people
think the things I say are generally interesting or generally boring. I
wouldn't be ashamed to use Mark's site to try and learn stuff about myself. Of
course, I'm sure the factors you mention provide subconscious motivation as
well.

------
mickdarling
Hey Mark, it's been a while. Nicely done. You said it was just an MVP so I
assume you have a product designed beyond this point. So, NOW is the time to
do some validation on what those other ideas are. You have a large audience
who already know who you are and are willingly giving you input.

Maybe in put down your follow ideas in short little phrases and have people
provide three words on that, if they wish, along with three words on their
friends. It will give you great insight into what THIS audience thinks is a
good or bad next step for you.

This is a site that will not maintain interest as it is beyond a short period
unless you add more to it. So the key here is to move forward quickly, even if
it is in tiny incremental steps. letting the users help you make some of those
choices of what steps to take can only help.

------
harscoat
Mark you just started 2011 for me. Very inspiring/smart what you do!
Congrats!!!

------
jjoe
It looks like it's already been moved to EC2 and switched to Varnish cache and
Nginx for static files. I think the decision to tie the "user" view to the
main FQDN (<http://threewords.me/joe>) will come back and bite hard if this
project turns into a large user base.

It would have perhaps been better to setup a wildcard DNS/vhost and set it as
<http://joe.threewords.me>. It can still be done but the earlier the better.
Time is of essence here.

Regards

Joe

~~~
pdx
Can you help me understand how this helps with scaling? I prefer this url
format myself and noticed it's used rarely by the big guys. If there's a
scaling advantage it provides, however, it's not obvious to me.

~~~
jjoe
DNS is much easier to scale out than HTTP. By having full FQDN for users, you
have the capability to easily scale out without needing a load balancer in
front of say, threewords.me.

So in your DNS zone you can tell it to route all accounts that start with the
letter 'a' to server with IP 1.1.1.1, letter b to IP 2.2.2.2, etc. Server with
IP 1.1.1.1 will not only run a headless web app but also route SQL queries to
a db shard (all relational data for users starting with the letter a).

This level of "user" partitioning can be deeper. So it's not just the first
letter of the user but two or three more letters/numbers. So joe.threewords.me
would be on a different server than j4mes.threewords.me (That's
j.o.threewords.me vs j.4.threewords.me).

Regards

Joe

~~~
adrianscott
twitter has servers dedicated to high-volume users without having to use
urls/dns like that.

~~~
jjoe
"twitter has servers dedicated to high-volume users without having to use
urls/dns like that"

That's why they need so many load balancers w/ failover:

twitter.com has address 128.242.240.148 twitter.com has address
128.242.245.212 twitter.com has address 128.121.243.228

Load balancers aren't flexible to work with especially when dealing with
sessions. I imagine Twitter's front end nodes are doing nothing but routing
URLs to backends, which are doing the real work. So the front end balancer
takes <http://www.twitter.com/unixy> and routes the actual server-side request
internally (where scalability is implemented).

------
dedward
Find your current bottleneck.

Go to static content wherever you can and get it replicated out and hosted
somewhere.

consider something like HAProxy (well.. haproxy!) in the stack out front to
control concurrency to resources that are bottlenecked - that way you can at
least get linear degradation instead of out of control degradation.

EDIT: Drop the dynamic content wherever you can, shut off whatever dynamic
parts you can and then scale out horizontally. Then fix the bottlenecks and
bring them back online.

------
vinyl
About monetizing: you could show the first 2 words only, and charge a small
fee for the third ? I guess people would pay to see what is said about them at
least.

~~~
smokey221
This is a terrible terrible terrible idea. You don't want your users to feel
that you are exploiting them. It will just pi$$ them off. Just go with a bunch
of well placed ads.

------
jamesbritt
"If you're curious, the stack is Rails + Ruby Enterprise Edition + Passenger +
nginx, which isn't cheap like PHP to host. On the $40 Linode right now but
maxing out CPU at 350%."

Interesting. Makes me think that an MVP should be done in PHP or something
else that is cheap to host and handles these sorts of conditions well, and
then if things look bright build the real site in whatever seems appropriate
for the expected demand.

~~~
jpcx01
Nah. MVP should be done in the technology you are most proficient at,
regardless of how fast/slow it runs. Optimize after you've proven your idea.

I think he just should have hosted on Heroku as he could have scaled it a bit
faster.

------
Tycho
Seems like a good place to ask this:

are those 'sign in with Facebook' buttons trust-worthy?

I've only clicked one of them, and that was a 'sign-up' button for an event i
will be physically attending

everytime i see one, I worry that

a) it's going to broadcast to my entire friendlist that i signed up for this
service

b) it's not a REAL facebook button, but something to phish my account

c) this site will know my Facebook login

d) some other party will intercept the login...

tell me it's completely safe?

------
Maxious
Haha I recieved links to threewords.me before I even opened HN this morning. I
don't know how to solve your fires but I've been asked to tell you to let the
text color be customised because otherwise you can make unreadable on certain
backgrounds. Maybe you should setup a getsatisfaction-ish feedback/support
area?

threewords for threewords.me: hot new cool

------
andreasklinger
Congrats on the App.

Regarding the BizModel. Forget it and grow as long as the hype works.

If you can't: Offer the possibility to add an optional reponse email. An ask
for a penny if people want to contact the anonymous poster. Cooperate with FB
for FB Credits to see this work.

Expansion: Threewords for all type of content (blog articles, third parties
like celebrities, etc)

------
Timothee
Just curious: what does it do? (no offense, but it asks for too many things
and permissions for me to try it right now)

~~~
markbao
You don't even need Facebook connect to try it out, just sign up. Send the
site to your friends, they give you three words anonymously, you see them.

I need to up the copy on the site.

~~~
guelo
Then why do you need all those Facebook permissions?

~~~
markbao
Ah, you're right. This is literally an MVP I threw together in 4 hours.

------
harscoat
Pls Mark, do keep us updated with the Stats

------
rdl
This is exactly why it is nice to have a bunch of credit cards with
comparatively high limits, even if you are frugal. There is no investment or
other deal which can be negotiated as quickly as just swiping a card and
worrying about it 20+ days later. (combined with EC2 or other scale with API
services)

------
jarin
Oooh, hey this is bad. If you go to <http://threewords.me/username.xml> you
can see the user's Facebook access token, secret, session key, and hashed
password. You should probably remove that view or modify the to_xml in the
user model.

~~~
jarin
It's fixed now.

------
davepm
[http://techcrunch.com/2010/12/30/three-
words/?utm_source=fee...](http://techcrunch.com/2010/12/30/three-
words/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Techcrunch+%28TechCrunch%29&utm_content=My+Yahoo)

;)

------
gsteph22
FYI: Heroku is fantastic for scaling apps. But if it's your database that's
the bottleneck...you've got problems. Because your options with SQL are
scaling up or sharding, each quite painful. If you want any advice, ping me.

~~~
AlexC04
Perhaps rewriting the database to use MongoDB? Their <http://www.mongodb.org/>
tutorial took about 10 minutes and is really easy to get to know.

As a DB they say it's designed for SPEED and SCALE. It might be worth looking
at. (who's got the time right?) :)

~~~
dasil003
Alternatively, I've gotten a lot of mileage out of using redis as a cache to
selectively optimize data piece by piece.

------
zachahack
First of all, congrats on your problems :) I've signed up, and things are
responsive.

1\. Implement caching. 2\. Get faster servers to handle your 'debutant' phase'
ex: EC2 3\. bask in limelight, enjoy!

------
iAmVinnieV
Congrats on executing the idea and the traffic surge. Looks like you're on the
front page of TechCrunch so you'll have more traffic coming your way.

------
anthony_franco
What's your main bottleneck? What's driving that high CPU?

Without knowing more about your setup I'd offload assets to S3 and switch from
Passenger to Unicorn.

~~~
markbao
User-specific assets are on S3... will need to offset the javascript and CSS
there too.

Good tip on Unicorn. Any benchmarks on the performance gain over
Passenger+REE?

~~~
anthony_franco
Not sure if there's any benchmarks out there. But I had your similar setup
when my app went viral. Switching to Unicorn w/REE replaced an expensive
engineyard setup. Learned about it from github
(<https://github.com/blog/517-unicorn>)

But something you could do immediately to help with the load is to run haproxy
on your current linode and funnel some of the traffic to a new linode that you
spin up.

------
pathik
Mark,

How did it go? Is the traffic still sustaining? Also tell us if you have been
able to monetize it? Share your experiences with a blog post.

------
whalesalad
Cache the hell out of it! I'd spend all of my time looking into caching
mechanisms for Rails right now.

------
cloudwalking
"502 Bad Gateway" - hosting troubles.. Can you explain a bit more about what
threewords.me is?

~~~
auston
It's a site where you can ask people to describe you, in three words.

~~~
shashank261
I am not sure how long it is going to last. i mean you will run out of friends
to write about, Isn't it?

~~~
jasonlbaptiste
then it starts suggesting people. apply it to more than people. then it starts
suggesting more than people. you get the picture.

------
kmfrk
Yoggrt.com seems like a decent ad network if Fusion Ads and Deck Network won't
bite.

~~~
drivingsouth
InfluAds.com also does

------
vanni
Clickable: <http://threewords.me>

------
davidj
turn on mysql connection pooling and install Google Web Accelerator, you'll
have to switch to apache to do Google Web Accelerator though.

------
elvirs
could not you make it pull my photo from facebook as well? also ad an option
of inviting friends from facebook?

------
EGreg
yo mark, congrats. So what are your next steps? Grab an advertising network
quick and throw it on there for now.

------
HackrNwsDesignr
What does MVP stand for?

~~~
harscoat
Minimum Viable Product <http://en.wikipedia.org/wiki/Minimum_viable_product>

------
grandalf
short term: heroku

long term: app engine (just port it to webapp framework)

~~~
gry
Why?

~~~
grandalf
Heroku in the short term b/c you wrote it in rails and you can easily scale it
up or down based on virality.

App engine after that b/c it should be fairly easy to port and it's an
extremely headache-free way to host an app. Now that they have always on
instances, it's _very_ fast and scales efficiently and effortlessly.

------
stef25
Great idea! BRB, working on sixwords.me

------
WALoeIII
Heroku.

------
jagira
You are on Techcrunch!!!!

I will pray for you.

------
jetz
how is the load now?

------
binaryfinery
That's why I use google appengine. It would have handled what you describe
without even going over the daily free limit. Of course, you wouldn't be cool
as rails.

~~~
gcv
You could be cool as Clojure, though: <https://github.com/gcv/appengine-magic>

~~~
binaryfinery
Whats the start-up time? i.e. when google spins up a new instance to handle
traffic, how quickly can it come up?

~~~
gcv
In the free configuration, I've clocked it between 4 and 6 seconds. (Sometimes
up to 10 seconds, but not recently.) With the 1.4.0 release of the App Engine
SDK, the cold start time has become less important, though. First, you can pay
$9/month to have three instances running all the time. Second, App Engine can
send a "warmup" request to your application.

------
to
minimize the server work; get some cheap nodes at rackspace cloud; use one
node for sessions in memcache; use two nodes for mongodb (master/slave) read
from both; try to minimize xss/html/js and gzip everything; cache as much as
possible; maybe even queue inserts/updates with redis and a cron;

thats how i survived a mini fb app thatwent from zero to 44 million users in 7
days... rackspace cloud + memcache + redis insert&update queue + round robin.
after a month i even killed the rackspace cloud interface and api with over 80
nodes and growing. had to get the memory limit on my account raised twice.
peak were around 40k req/sec. after the second month we moved it to four bulky
servers (16 core, raid5, 32gb ram) each cost around 12k dollar.

the whole secret is caching. everything, everywhere and as much as possible.

~~~
phoenix24
which was mini fb-app, if you don't mind me asking?

------
mkramlich
Seeing the title on the front page, I came here to post "Monetize it, baby!".
Now that I've read more detail I'm in the camp, "Put out those fires NAOW and
put some extra water buckets nearby too."

Then monetize it, baby.

------
shashank261
First, do not put ads on your site as it will annoy the users. Rather, take
feedback and see what it takes to get more traffic. I guess you can double up
your existing infrastructure to handle any surge in traffic. You do not want
your users to see your page down due to excessive traffic.

------
Devilboy
So... what is it?

------
anilbioma
I just don't get it, what problem does this app solves anyway? Plus, HN has
become a promotional playground, this really piss me off.

------
dotcoma
what do you do? stick ads all over the place! what do you think, you have a
business there? like twitter? yeah, sure, raise another 200 million...

~~~
dotcoma
make fun of twitter, and you get down-voted. I know, I know, Google will buy
them for 4 billion in 2011, but that is the only exit strategy they have. No
biz model there, sorry.

~~~
revorad
Have you seen this? - <http://business.twitter.com/>

~~~
dotcoma
how much money do you think they can make with that?

~~~
revorad
I don't really want to offer my random guess against your random guess of "not
much money". Why not wait and see? What vested interest do you have in seeing
Twitter fail?

I hope Twitter makes a lot of money and lasts for a long long time. It will be
good for the world.

~~~
dotcoma
I have no vetted interest in seeing Twitter fail, nor do I think it will. I
think Google and Bing will keep it afloat for a long time until they figure
out how - or if - they are really interested in the so-called "real-time web".

------
veb
I created at stupid app a week ago, called <http://wonklyrics.com> where the
point is to enter a songs lyrics, and change one word to make it more...
funnier.

Sadly to say, it's received a total of 300 visits. :-P

~~~
aparadja
Just a heads up, the app doesn't protect against xss at all. For example:
<http://wonklyrics.com/24>

