Hacker Newsnew | comments | show | ask | jobs | submit login

Assuming a simple system a single box could be averaging around 5k requests per second. That's 18 million pages per hour, and just under 1/2 a billion pages per day. That's around 2.5 pages per month to every person on the planet but let's say you get 100 million people that's 150 pages per month. Now if you think you are going to get to that point in the next year feel then it's an issue but IMO 99% of people are far from that point.

Granted the real question becomes storing data not handling that number of requests, but a database that knows where a bunch of dumb files scales really well. (If you look into things this is Facebook's basic approach.)

I wonder where you are taking those figures from. The article stated about 100 queries/sec on a MacBook without any write load (if I interpreted that correctly). If 5k/sec are sustainable on a single box, including writes, then yes, that will probably go a long way.

I'd venture the guess that you'd be talking quite a different budget than a bunch of pizzaboxes in a horizontal setup though. The SAN to handle 5k IOPS alone will set you back by an interesting amount (even more so when you consider mirroring, which you'd probably want to have at that scale). I'd also be worried about the network - GBit/s is probably not going to cut it at that rate anymore.

So, all in all this is precisely why I asked about horizontal scalability. A setup of 5 machines that handle 1000 reqs/sec each is usually cheaper than a single machine to handle all of the 5000/sec.


If his MacBook can handle the entire database so I don't see the need for a SAN (yet). I don't know how much benefit there would be to increasing his systems RAM, but if the database fit's on a laptops HDD, then you can probably get most of it into ram which would make things insanely faster. My guess is upgrading to a 10,000$ box with 64Gb of ram and a 1 + 0 RAID of SSD he could probably get 50x increase in speed which would be ~5k operations per second. Granted he might develop issues with network bandwidth or some other bottleneck, but even just averaging 1k/second represents huge revenue potential relative to the cost of that system. And a back of the envelope calculation should give him a rough estimate of it's value.

PS: Upgrading 10gb Ethernet is not really that expensive now days if he is only linking a few web servers to two databases.

EDIT: To give you some idea what flash can do http://advancedstorage.micronblogs.com/2008/11/iops-like-you... (Granted, it's a stupid video, but 150,000 Read IO's and 80,000 write IO's and 800MB/second of bandwidth on two PCie Cards in 09 / 10 with fusion IO doing the same type of thing today).


If his MacBook can handle the entire database so I don't see the need for a SAN (yet).

The SAN comes into play when a single box can't deliver the IOPS anymore - remember it's not just a matter of adding SSDs. At those rates you start touching the controller and bus limits. Likewise a saturated 10Gb ethernet link causes a significant interrupt-rate (older cards would bottleneck on a single core) that often exposes interesting corner-cases in your OS and hardware of choice.

I'm not saying it's not doable and I know what SSDs are capable of (we just fitted a server with X25's). I'm just saying that your estimate of $10.000 is very optimistic, add a zero and you'll be closer to home. That's because I still think you'd definately be talking an xfire 4600 class machine and a SAN.

Anyways, this is all speculation. Wheels made some reasonable statements that they have it on their radar and I'm definately looking forward to some real-world benchmarks with a concurrent write-load.


Applications are open for YC Winter 2016

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact