

Ask HN: Cost of a server farm - bravura

If I want to run a commodity hardware farm, say 100 eight-core machines, how much will this cost? I am also asking about incidentals like space to house the servers, T3 internet to the outside world, gigabit intranet, power, etc.
======
patrickg-zill
100 8-core servers, each 1U, figure $2K apiece as a rough guess. That is
$200K.

Next you are talking about 3 racks of space (you can fit 42U in a rack), plus
extra power, figure $3000 per month in a less-expensive colocation space.

Gigabit switches, cabling, etc. figure another 5K or more, depending on the
quality and features you want.

Bandwidth will be metered by your colocation provider, prices can vary,
usually you will be given 5-10Mbps as part of the rack.

So figure $210K plus $3K per month or so.

~~~
thwarted
You can physically fit 42U in a single rack, but you can't run that many
servers off the regular per-circuit (which is often per-rack/cabinet) AC power
limits most datacenters provide. Each circuit is usually 16-20 amps, this is
in my experience about 8-10 2-3U servers (your hardware should have wattage
and amperage documentation). You can get higher power density (and sometimes
cheaper prices) if you use DC powered hardware. You can get more circuits, but
they often charge incrementally more because as the servers per square foot
ratio goes up, cooling requirements go up also (you pay for cooling as part of
the price for the physical space), and the power limits are a way to help keep
that under control, predictable, and homogenous across the datacenter.

~~~
patrickg-zill
Well, 100 1U servers is about 33 per rack if you have 3 racks; switches should
not be more than 2U or 4U on top of that, and they do not use a lot of power.

Not sure exactly how much power, but 2x 30A power feeds will give 48A usable
power, which is just about 1.5 amps per system. Whether that is enough for 8
cores (2x quad core), I don't know; will depend on amount of memory and disks.

------
gojomo
$58,400/month plus bandwidth -- as 100 Amazon EC2 "High-CPU Extra-Large
Instances", with 8 virtual cores, 7GB RAM, and 1690GB local disk.

