

Ask HN: My web startup is growing fast what next? - digamber_kamat

Dont ask me whats the url of the startup. Let us assume the question to be hypothetical.<p>Our website is in beta phase and I am the only coder in the team. Looking at the idea some investors have given us a decent amount of money.<p>We launched this site through invitations to 100 people. Within a week we hit the 1000 registered users mark. Till then we had hosted our site on rackspace cloud with a server with 256MB (thats not a typo) machine. The tech stack was LAMP.<p>Sensing the growth we increased the RAM to 1 GB. The number of users at any given time was around 40. The RAM was more than sufficient for such a scenario but still the site started slowing down.<p>I immediately shifted the mysql server to a different machine. That really did not make much difference.<p>So I looked at the most frequently used operations (which was some queries with joins). I indexed all the tables accordingly and installed memcache to cache the queries for half an hour or so (wherever possible).<p>Now the number of online at any given time is increased to 120. Total registered users is 9000. 
Our investors are happy and co-founders think we are successful and now trying to buy an office for ourselves and build a proper team.<p>However I am clueless as to think how we can scale hence forth.<p>Can people give me some pointers as to the different paths I can follow?
Please note that we cant host the server ourselves, we would love to continue with rack-space.
======
rjamestaylor
I would be happy to look at your configuration and suggest options for scaling
further (does not necessarily mean more resources or $$!). Please send an
email with your account number to twitter@rackspace.com. Myself or one of my
teammates will continue our discussion from there!

Robert Taylor Sr Systems Engineer (Linux) Rackspace Hosting
robot@rackspace.com

~~~
digamber_kamat
This is the reason I love rackspace :)

------
pilom
From a scale/speed perspective, step one is figure out your next bottleneck.
You should do this before it actually becomes a bottleneck. Obvious things
come to mind:

1\. Memory - very easy to scale up to a machine with more, maybe throw a Nginx
frontend on it if you are getting too many connections 2\. Bandwidth - move
resources to a CDN assuming you already gzip and minify text, combine
javascript files, and have appropriately sized photos. 3\. Disk I/O - change
DB structure again, more/better caching, set up load balancers (there are tons
of other optimizations to do before you get to this point) 4\. CPU - profile
your PHP to see what takes so long. Flush your output as soon as possible so
users see something faster.

Lastly be sure you have a redundant setup with auto-failover. 9000 users get
very unhappy very quickly when you start having unexpected downtime more than
0 seconds.

~~~
digamber_kamat
Let me look closely at your checklist

1\. Memory: We have 2 GB and "free -m" shows that more than 200MB (least) is
free at any given time. Do I still need to add more memory?

2\. Our site scores perfect A on Yslow. So minification , gzip, far future
headers etc is in place.

3\. We have improved out DB structure a lot. Since we are adding more features
continuously we are constantly changing the DB structure. With the help of
some more experienced programmers I will get these things tweaked. But in
MySQL okay if our user base will hit 1 Lakh in next three months ?

4\. CPU profiling is something that we haven't done yet. Will do this asap.
Can you suggest some resources starting points ?

You also said something about load balancers. Will you please give me some
resources to start up with ( I know I can google but ...) ?

5\. Fail-over is something that I did not think of initially. What are the
best tools around ?for the same ?

The whole discussion might sound as if I want you guys to spoon feed me. But
that is not the case. Let us look at the whole questions as an opportunity to
share each others views on the same topic, even the experts might get to learn
new things.

~~~
sagacity
> But in MySQL okay if our user base will hit 1 Lakh in next three months ?

I doubt if many here will understand what is 1 Lakh. Better make it 100,000.
:-)

(BTW, where do you live?)

~~~
digamber_kamat
I live in Mumbai, India :)

------
bee
Maybe you should change the db structure, to optimize sql queries, use a
caching system to delivery static content, or you can ask a more experienced
developer to look over db structure and give you ideas. 9000 users don't seem
that many to make the server run slow...

~~~
sagacity
> 9000 users don't seem that many to make the server run slow...

I second that, especially when you have moved mysql on a separate server. I'm
sure you're missing something basic here (as happens to many from time to time
:-)).

~~~
digamber_kamat
Our server is not slow at the moment. It is pretty responsive. But as 9000
reaches closer to 100000 I am sure existing safe-gaurds will not be enough, so
I am asking what else can be done.

------
stevenklein
It's really to comment on things like an office, raising money, growing the
team, doing more customer development, etc. without knowing much (if anything)
about your situation. I do have just one quick comment though -- you mentioned
the speed of your site being a factor pretty early on. There are many factors
that go into a user's overall experience but perceived speed of your site is a
HUGE one. Don't let that be an issue for you right now as it can severely
impede your growth.

------
badkins
When optimizing, don't do it blindly. Do some actual tests or profiling to
find the true bottleneck. It might be a specific query, the DB as a whole,
etc. But unless you fix the biggest bottleneck, other improvements won't make
much diference. Once you fix the biggest bottleneck, go back and do it again
for the next biggest.

This way, you will know for sure if you need another machine, more RAM, or
just another table index.

~~~
digamber_kamat
Yes true. I realized this during my initial optimization. I realized that some
queries involving joins were taking unusually long time, a slight tweaking
tremendous gain in performance.

------
sagacity
This is pretty basic, but in addition to mysql table structure and query
optimization, have you done/considered any tweaking of the mysql config
itself? (Usually at /etc/my.cnf)

We've repeatedly experienced _huge_ performance gains even with basic/minor
tweaks of my.cnf.

HTH

~~~
digamber_kamat
Cool. Something that I dint know. So far I have changed some table types to
innoDB from MyISAM. Rewritten a lot of queries and in some cases maintained
redundancy instead of normalization.

~~~
sagacity
Likewise, you might want to check/tweak your apache config too (usually at
/etc/httpd/conf/httpd.conf).

As you probably know, the default configs of both - mysql and apache are
usually pretty minimalistic so tuning these in accordance with your
load/traffic scenarios can pay you off big-time in terms of performance boost.

On that note, I'm sure they have some _highly_ competent people out there at
RackSpace who should also be able to help you out with all this.

------
digamber_kamat
I finally found a great guide about the topic:
[http://www.amazon.com/dp/067232699X/?tag=stackoverfl08-20#re...](http://www.amazon.com/dp/067232699X/?tag=stackoverfl08-20#reader_067232699X)

