

Ask HN: Cost of servers & bandwidth  - paul9290

I am curious the infrastructure cost of running a streaming audio site with 50K to 100K users per month.  Users would be uploading files to server and streaming them.<p>Does anyone know of a link resource that details this type of information or able to detail their experiences?  Does not need to be specific regarding stated monthly users.
======
brk
Committed bandwidth can be had in many datacenters these days for around
$50/Mbs ($50 for every 1Mbps of committed bandwidth). Or you can do 95th
percentile burstable bandwidth as well (or combine the two ($100/mo for 2Mbs
plus ~$100 for every 1Mbs over 2Mbps, billed 95th percentile)).

About $400/mo for a half cabinet and some power. You're in the neighborhood of
probably $500-$900/mo.

Why colocation instead of some shared or pseudo-dedicated server some place?
Because (IMO and IME) you're better off with any site that is going to do
streaming and potential sustained throughput in colo facility with fixed
costs. Places like Rackspace just love when you go 10x over their pitiful
limits and your $200 monthly plan is suddenly $4000. Kind of hard to budget
for.

Of course, I'm also biased, having an interest in a colo facility :)

~~~
staunch
> _About $400/mo for a half cabinet and some power. You're in the neighborhood
> of probably $500-$900/mo._

And that doesn't even factor in buying a server if it's a colo. Which is
exactly why he should be renting a dedicated and buying bandwidth at
$10-$20/Mbit. It doesn't make sense to setup your own rack unless you're going
to have more than a few machines. It's more expensive and a lot more work.

~~~
carl_
If you're talking about providing a high capacity and quality experience then
there's no alternative to colo.

I think the /Mbps prices quoted are a bit on the high side (you can get cheap
transit at 5GBP/Mbps if you deal in reasonable commits). Apart from dropping
your own line in there's no cheaper way I know of to achieve the quality.

Use of a CDN would allow less capital outlay on the standard http hosting
(could use dedis, vm, cloud) and somewhat instant scalability. Once you know
your traffic patterns your implementation options sometimes get made for you.

~~~
brk
Yes, the prices were a bit high. They're very regional (and depend on what
carriers are at a given site, etc), so I was trying to give a decent average,
erring towards worst case (my own prices in MA are a good bit lower, but I'm
not trying to troll for customers on HN).

Good point on the CDN, "streaming" can sometimes be non-CDN compatible
(although "CDN" is a wide ranging term).

Were it me, I'd start with an $800 1U box in a decent colo somewhere. Add
another one on the opposite coast when the first started to get above 50%
utilization and then scale from there.

~~~
Retric
Going colo can be worth it when you need serious hardware but for 800$ / month
you can pick up:

Four of these: Xeon 3210 - SATA 250 GB IDE/SATA HDD 2 GB 2000 GB 5 IPs $199.00

Or one of these ~Dual Xeon 2.8, 8 x 300 GB 10K RPM SCSI/SAS HDD 2 x 73 GB 10K
RPM SCSI/SAS HDD 8 GB RAM 3000 GB Bandwidth 5 IPs $799.00

Which can take a lot of traffic. IMO Colo is great when you want to use lots
of heavy duty HW and little bandwidth but for front end web servers you need a
lot of traffic before it's reasonable.

------
staunch
If you mean serving audio files 100k times (once per "user") then it's not
expensive unless the files are especially large. If you're serving over HTTP
then use high performance web server (like nginx or lighttpd) and a dedicated
server.

I highly recommend ServerBeach myself -- especially if you're going to use any
significant amount of bandwidth. For $75/mo you can have a dedicated server
and 1200 GB/mo transfer.

[http://www.serverbeach.com/catalog/cust_ref_landing_new.php?...](http://www.serverbeach.com/catalog/cust_ref_landing_new.php?REF=7XYHDMBU8A)
($100 off w/my referral code ($250 for me))

<http://www.serverbeach.com> (no referral code :D)

~~~
anotherjesse
I also recommend checking serverbeach. My box for userscripts.org runs on a
$180/mo box (the extra cost is for 3GB of RAM).

I'm currently doing 50k visitors a day and dark traffic of 10x that -
averaging 1.2Mbps without a problem.

You can even get 10Mbps or 100Mbps unmetered for $100/mo and $1000/mo roughly

------
ashleyw
S3 based on 100k members uploading on average 5x5MB audio files each, and a
growth (new uploads) of around 10%/month:

    
    
      Storage: 1500GB
      Transfer in: 150GB
      Transfer out: 30000GB
      PUT requests: 50000
      GET requests: 10000000
      ———————————————
      Total cost: $4,560.10
    

Not too bad if you ask me! (btw its late and I'm tired, so sorry if my figures
are _WAY_ off! :P)

~~~
aupajo
Shouldn't that be POST? PUT would be for updating the file that was originally
POSTed.

~~~
tlrobinson
No, POST is typically for submitting data to some other "resource" for
processing, while PUT is for storing data to a specific URL.

HTTP/1.1 RFC: _"The fundamental difference between the POST and PUT requests
is reflected in the different meaning of the Request-URI. The URI in a POST
request identifies the resource that will handle the enclosed entity. That
resource might be a data-accepting process, a gateway to some other protocol,
or a separate entity that accepts annotations. In contrast, the URI in a PUT
request identifies the entity enclosed with the request -- the user agent
knows what URI is intended and the server MUST NOT attempt to apply the
request to some other resource."_

And from S3 documentation: _"Objects are stored using the HTTP PUT method."_

S3 and similar file storage services map particularly well to GET, PUT, and
DELETE.

------
scumola
I go through <http://serverpronto.com> for my hosted machine. It's a real
machine (not virtual or shared) and I've got real root access to it. It comes
with Fedora 3 or 4 and I upgraded it to Fedora 6 remotely, but I'm a little
intimidated to upgrade beyond that without having physical access to the
machine. $30/mo, 200GB bandwidth @ 100Mbit, 30GB HD, 256M ram for starters.
There are other options. I use S3 for storage of large data items. All-in-all
not too bad for a good hosting setup. I host about 10 semi-active sites on
that one machine. My S3 monthly charges are $1 for my personal backups of
important data and $30 for website data.

I know that there are plans out there for unmetered monthly data that's capped
at a certain speed for a fixed low $$/mo (30-ish), but they tend to be virtual
hosts. You could get a real host (for the site and good performance) with
capped bandwidth and a second host with unmetered bandwidth for the storage
and delivery of your streaming media. Best of both worlds kinda, but again S3
charges so little, that it may be better to just go through Amazon for the
media.

~~~
listic
But they offer only 200 GB of bandwith with that plus overages are $0.89 per
GB. Yes you've got root acess but they charge $29 for reboot (yuck!) and $75
for half an hour of custom services. I think that one would be better off
using VPS like Linode or Slicehost: you can reboot, repartition and reinstall
your virtual machine to your liking and the expansion should be easy and
straightforward. .

------
patrickg-zill
Go to webhostingtalk.com and look on the colo hosting offers section. That
will give you your lower-end pricing for colo space and bandwidth pricing.

Then add some kind of servers, I am partial to the Sun servers that have LOM
(lights out management) or the HP ones that have iLO (same idea). This lets
you connect to a service processor on the system, and lets you power-
on/off/reboot it, control the BIOS, and even install the OS on it, remotely.
You can get them used on eBay for $1000 to $2000 each.

------
alecco
<http://www.simplecdn.com/savings> (On-Demand Flash Streaming part)

Don't know them yet, but there are some good reviews. Perhaps it can be a
starting point even if it doesn't deliver completely. I am planning to use
them in the near future on top of AWS and Google App Engine, but not for
streaming.

------
jsn
<http://www.gandi.net/hosting/proposal/price/>

it's pretty much an elastic cloud. i didn't try it myself yet, though.
unmetered bandwidth in 5mbps increments, plenty of other resources, and very
attractive pricing. i think it's as good as it gets for home-made small cdns
and whatnot.

------
paul9290
Thanks ....

What about using Amazon S3 to store/stream users uploaded files, but the site
being hosted elsewhere - a dedicated box at home for instance?

They click to play a file and it starts streaming it from S3. Is this doable?
Looks to be cheap this way?

~~~
scumola
Sure. <http://mediawombat.com> does that very thing. The website is stored on
a hosted machine somewhere in Florida and the actual media is stored on
Amazon's S3 and loaded via the browser on demand.

~~~
paul9290
Cool ... that looks to be the cheapest solution.

Possibly use a shared deluxe to host site on 1.2TB bandwidth a month, then all
the storage and streaming of audio files from S3.

~~~
prateekdayal
> Possibly use a shared deluxe to host site on 1.2TB bandwidth a month, then
> all the storage and streaming of audio files from S3.

If you mean some kind of shared hosting, I would strongly advice against it.
If you are looking at 50k - 100k users a month, you would need good amount of
RAM and CPU to do stuff like re-encoding mp3 files (you will have to do this
as flash does not support many kinds of mp3 files)

There is other stuff too .. atleast go for a VPS. We are hosted on one and the
experience has been brilliant so far.

------
comforteagle
youtube scaled on serverbeach. 'nuff said frankly.

I however would do AWS without question. (already on AWS, was on serverbeach -
loved them though)

~~~
kvogt
Actually, youtube was on serverbeach _until_ they had to scale. They
immediately moved to colo and CDN.

------
fallentimes
I wonder what Justin.TV uses?

~~~
wave
I think in addition to their own servers, they use AWS
[http://www.slideshare.net/tracylaxdal/justintv-aws-the-
start...](http://www.slideshare.net/tracylaxdal/justintv-aws-the-startup-
project)

~~~
kvogt
That's a little out of date. We have a few racks and several 10,000mbps
uplinks.

------
niels_olson
does user=visit for small user sets? We get about 15k visits/mo but only 300
users.

