
My $300 Home Cloud Server - theonewolf
http://xrds.acm.org/blog/2014/09/my-300-home-cloud-server-a-story-of-blood-sweat-and-ebay/
======
Someone1234
Hmm seems like he made the classical mistake of thinking that an x86-64 server
is somehow special and more worthwhile than just buying a bare-bones PC.

I've managed Dell servers before (real ones) and aside from a bunch of stuff
to make managing them in mass easier (like DRAC), there isn't very much value
in trying to retrofit one for home use.

VT-x or 24 GB of RAM support isn't exactly a hard requirement in 2014. Plus
the Xeon E5410 is a 2007 chip (7 years old!) and gets it butt kicked by even a
low end i3 from 2012 [0].

Plus modern systems consume less electrical power which makes running them
cheaper and a PC is certainly quieter than any rack mounted server (which are
designed for maximum cooling, noise be damned).

Additionally if he had purchased off-the-shelf parts he would likely have a
warranty for 12 months (or more). With the eBay purchased Dell machine he not
only doesn't get any warranty but ALSO Dell themselves don't even support
it(!) so no firmware updates.

So while this was a fun read, it just reminds me of my more nieve days when I
thought doing stuff like this was a good idea. Go over to any SysAdmin forum
(e.g. spiceworks community) and they'll tell you the same thing.

[0][http://cpuboss.com/cpus/Intel-Xeon-E5410-vs-Intel-
Core-i3-32...](http://cpuboss.com/cpus/Intel-Xeon-E5410-vs-Intel-Core-i3-3220)

edit: Originally linked to an 2012 i5 comparison ($180 CPU). Altered it to an
2012 i3 comparison ($100 CPU) which still out-performance the 7 year old Xeon.
The $100 price tag seems a lot more comparable than a near $200 CPU. Plus the
i3 supports 32 GB of RAM and VT-x (the OP's requirements).

~~~
drzaiusapelord
Sysadmin here. Agreed. 1 or 2U servers for home use is fairly ridiculous. In
fact, its kinda ridiculous for some business use now. We're probably only a
few years away from a mainstream and popular form-factor that's a fraction of
the size, with tiny SSDs instead of giant spinning drives and power-sipping
x86 or ARM chips. Imagine something the size of a stack of 4 or 5 ipads
replacing the typical 19" racked server.

For small and perhaps even medium business, this could be a good fit. The 19"
rack really is something from a past era. Soon people will look at it the same
way I look at old mainframes and minicomputers.

As far as the article goes, $300 is impressive, but the costs of wasted
electricity will eat up the savings eventually compared to just dropping some
cash on a new Synology NAS, and you get a warranty.

~~~
curun1r
I don't think a Synology would have the horsepower that the story author was
after. He specified that he was going to be chewing through large datasets, so
the dual Xeons @2.3GHz were important to him. Synologies are great for what
they do, but underpowered for CPU/RAM-intensive tasks...at least in the sub
$500 range, which is what he was looking to spend.

I'm a happy Synology owner, but I don't use it for my dev server. It sits
behind my VPN and serves media files. With the variable speed drives (WD
Green) and ARM chip, it draws very little power. In short, I love it for the
specific purpose for which it was designed, but I wouldn't use it for much
else. My dev server is in AWS. It's just too easy to spin up and spin down
whenever I need it and keeps me from having to devote space in my apartment
(SF aint cheap!). Also, not having a hefty power bill is nice.

If I were advising the story author, I'd tell him that unless part of the
enjoyment is building and running the thing, he should opt for a cloud
provider instead. He could get a DO droplet spec'd beyond the physical server
he bought for $0.12/hr and spin it up whenever he wanted to use it. Unless
he's planning on using it almost continuously, you can fit a lot of 12-cent
hours into a $310 budget, and that's before you consider the increased energy
usage from running a server at home.

~~~
arthurfm
> With the variable speed drives (WD Green)

None of Western Digital's HDDs change their RPM. It is fixed.

~~~
curun1r
The WD docs for Intellipower indicate that RPM is one of the things they vary
to conserve power. I can't find much in the way of specifics, but since they
list the RPM of the drives as "Intellipower", the assumption would be that
they don't operate at a fixed RPM.

~~~
arthurfm
The RPM varies between different models. Each drive has a fixed RPM [1][2].

From SPCR:

> Western Digital doesn't want to say that they're selling 5,400 RPM drives —
> those became second class in the desktop market years ago. Instead, they
> rate the drive's speed as "IntelliPower" and take pains to emphasize that
> there are other factors that affect performance.

Western Digital has caught a lot of flak for withholding the rotation speed of
the Green Power, especially when the product was first launched and the
marketing material listed the rotation speed as 5,400-7,200 RPM. This led some
to speculate that the rotation speed changed dynamically during use — which
would have been an impressive engineering feat had it been true. The reality
is revealed by a sentence that Western Digital added to the description of
IntelliPower: "For each GreenPower™ drive model, WD may use a different,
invariable RPM." In other words, Western Digital reserves the right to release
both 5,400 RPM and 7,200 RPM drives under the Green Power name — without
telling you which are which.

We were able to confirm that our 750 GB Green Power had a spindle speed of
5,400 RPM by analyzing its sound spectrum. Why sound? Sound is vibration; the
pitch of the sound corresponds to the frequency of the vibration. Hard drives
vibrate at the speed of their motor, so they produce a noise at the same
frequency as their rotation speed. Our sample had a sharp spike at exactly 90
Hz (cycles per second). Multiplying that number by 60 (to get cycles per
minute) yielded a measured rotation speed of 5,400 RPM.

[1]
[http://www.silentpcreview.com/article786-page2.html](http://www.silentpcreview.com/article786-page2.html)

[2]
[http://arstechnica.com/civis/viewtopic.php?p=23526214#p23526...](http://arstechnica.com/civis/viewtopic.php?p=23526214#p23526214)

------
psophis
Why use the term cloud? It's just a server, no cloud-ness.

[https://en.wikipedia.org/wiki/Cloud_computing](https://en.wikipedia.org/wiki/Cloud_computing)

~~~
ryanthejuggler
> _Oh yeah, did I mention I also wanted a beefier machine at home so I could
> manipulate large virtual machine images?_

I think he's looking to create a setup where he can spin up and destroy
networked VMs, which would indeed make the project count as a cloud. From your
Wikipedia link:

> The main enabling technology for cloud computing is virtualization.
> Virtualization software allows a physical computing device to be
> electronically separated into one or more "virtual" devices, each of which
> can be easily used and managed to perform computing tasks. With operating
> system–level virtualization essentially creating a scalable system of
> multiple independent computing devices, idle computing resources can be
> allocated and used more efficiently. Virtualization provides the agility
> required to speed up IT operations, and reduces cost by increasing
> infrastructure utilization. Autonomic computing automates the process
> through which the user can provision resources on-demand. By minimizing user
> involvement, automation speeds up the process, reduces labor costs and
> reduces the possibility of human errors.[27]

Admittedly, it depends on what he wants to do with this that would determine
whether it's "really" a cloud or not.

~~~
theonewolf
Cloud research.

Working with modified hypervisor (QEMU), and modified cloud software
(OpenStack in my research).

------
shiftpgdn
Don't bring a server into your home if you value your sanity. Those little
fans spin at 20,000RPM and can be heard throughout your house.

~~~
bmelton
I remember when the company I was working at bought a bunch of new Sun
hardware, and ended up throwing out all their old Sun 420Rs. Despite being
_slightly_ outdated, they were still in perfect condition, so I took the 4 of
them to the house (carefully, because they were heavy) and set up a pretend
data center.

The first shock, I can confirm, was the noise they made. I ended up relegating
them to the basement, which was otherwise unused.

The second shock was when my utility bill came in the next month. It's very
easy to know the power consumption of enterprise-class servers for planning
purposes. There's a giant abstraction between the power consumption on paper,
and the power consumption on your home utility bill that I think everybody
working around servers should experience at least once.

They were retired, shortly after.

~~~
mrbill
The noisiest system I've ever owned was a Sun T1000 1U server. Here's some
examples. I tried to mute it a bit by putting it in a huge tupperware-like
under-bed storage box, but even that didn't help one bit.

[http://www.mrbill.net/t1k/noise/](http://www.mrbill.net/t1k/noise/)

~~~
greedo
I worked on a stack of T1000s. We had a room next to our sysadmin hovel that
we used for setting up systems before putting them in the datacenter. I worked
remotely a lot, and would periodically reboot these servers. When they booted,
part of their POST was running their fans at 99% of their RPM capability. The
noise was astounding, and my coworkers grew pretty tired of me...

------
shanselman
Maybe I'm missing something, why not just use a cloud VM? On and off as like
and pricing turns off. Surely that would be less than $300, wouldn't be in
your house blowing air, and would be (more) easily accessible anywhere.

~~~
theonewolf
I do cloud research.

I modify the hypervisor.

I do things that no cloud in the world can do.

~~~
dimxasnewfrozen
This is particularly funny when you use the Cloud to Butt chrome extension.

~~~
wlesieutre
How do you know when someone uses a text replacement extension?

Don't worry, they'll tell you.

------
dorfsmay
This is a home server, not a cloud server:

\- no remote access to the console if ssh stops working

\- no possibility to remotely power it down/up

\- if the hardware dies, no possibility to restart on a different hardware
remotely

There's nothing wrong with a home server, I have one or three myself, but
these limitations are why I move "essential" services (e.g.: email) to the
cloud.

Home servers are better suited for exactly this, need of high CPU, and also
need of a lot of storage, for a backup server for example. I used to run
everything on home servers and back them out to the cloud, but I have now
started to reverse this, run the services in the cloud (cheap for small
service, remotely accessible etc...), back them up to the home server (cheaper
large storage, it's fine if they go offline for a few days while on holiday
for example).

~~~
theonewolf
[https://news.ycombinator.com/item?id=8374003](https://news.ycombinator.com/item?id=8374003)

Not sure if I should repeat myself a lot or not.

~~~
mserdarsanli
Latter would be more preferable.

------
andmarios
I once had a 2U rack server. Proper double Pentium III's inside, ECC RAM,
serial console, alarms on the case, everything. Doubled as a coffee table in
my living room. Alas, I was young.

Nowadays I have an Array R2 pc case with six HDDs, one SSD for the OS, a mini-
itx board and an Intel CPU with virtualization support (I really don't
remember the model) and a few gigs of RAM. All (except the HDDs) were bought
new at about the same cost. The SSD is a 32GB chinese model from eBay. The
machine is powerful, quiet when the HDDs are spinned down, relatively small
and elegant and acts as NAS, Server, testbed, etc. Also at idle it consumes
about 20W.

------
post_break
$300 for the server, $50 a month to power that heater.

~~~
theonewolf
With my 400W and around 10 cents per kilowatt hour in my area I'm seeing for
750 hours a cost of $30.00 estimated.

Now, I do not have this running 24/7.

Only when I or she wants to run a research job.

So we're talking 8-10 hours, 2-4 times a week.

Pretty cheap actually overall :-)

------
ja27
If you have a local makerspace / hackerspace, check with them. We get quite a
few decommissioned servers donated to ours but we generally don't have much
use for them. Noise and heat we can sorta manage, but the power usage is a bit
steep. We'd be happy to host them for people if they pay for the power and
aren't doing anything too crazy with the network.

~~~
theonewolf
Thanks for the tip!

------
guyzero
Is this super-loud? I have an old 1U server which seemed cool until I booted
it up. It's like vacuum-cleaner-loud.

~~~
theonewolf
It's not horrible, sounds like a blow drier on low setting when idling.

------
noonespecial
My Kill-A-Watt meter helped me determine that the best home server is a almost
always going to be a fast laptop (around 2 years old) stashed in your bedroom
closet. You'll get the most computer per dollar of purchase price/power price
available plus a built-in UPS and quiet operation as a bonus.

~~~
theonewolf
That makes sense.

Unless you need, like me, 24-32 GB++ of RAM. Dual HDDs, and a whole lot of
cores to run 3-4 VMs at a time.

Then the laptop is so resource-poor that it is rendered useless for the
workloads I described in the article.

This article wasn't about how to make a "cheap home server"\---it was about
replicating real data-center-class hardware in the home and how to do that
cheaply.

~~~
noonespecial
Ymmv I guess. 20gig ram (2 8s plus onboard 4), 1.5 tb 7200 rpm and an i7 was
enough for me to do my handful of vms.

Granted no ecc, no mirroring, no future expansion. But then no 18$/month power
bill either. To each their own.

~~~
theonewolf
How much did your 20gig of RAM cost you?

We don't run 24/7, only when we need to do our experiments.

So the power bill is much lower than what a lot of people are worried about.

This setup is definitely used on-demand :-)

------
sauere
Unless you need ECC memory for some reason, next time, do your ears and your
wallet a favor and just buy regular x86 consumer hardware. It's half as loud
(those 10k RPM fans must be killing your ears) and half the price and will
consume half the energy.

~~~
theonewolf
So you believe I can buy 24 GB of non-ECC RAM for less than $30 ?

------
_asciiker_
how is this a cloud server? that's like calling every picture a "selfie"...

~~~
theonewolf
First, what I meant, was in its past life it was in a cloud probably.

Secondly, I do cloud research.

I run a modified KVM/QEMU hypervisor.

I run a cloud with features you can not find anywhere else in the world today.

So, it is, indeed, a mini-home cloud :-)

~~~
_asciiker_
well now you're assuming virtualization is the same as cloud, it is not, a
cloud researcher should know what a "cloud" actually is instead of the
marketing usage for it. for one, redundancy is always implicit in any cloud
solution.

~~~
theonewolf
I don't believe redundancy, what I think you mean---fault tolerance---is
implicit in the definition of cloud computing.

The most general form of the term 'cloud' in my mind is elastic access to
computational resources.

There is no guarantee of high availability (in fact, we see failed AWS
instances and horror stories all the time), nor automated
failover/redundancy/fault tolerance.

Chaos Monkey by Netflix is a great example of the shortcomings of the modern
cloud in terms of fault tolerance and redundancy.

The cloud doesn't give it to you, you have to build that yourself.

I'm a very open-minded person, perhaps because of my academic background, so I
would be happy to say that I agree that a subtype of the fundamental cloud
could be a redundant/fault-tolerant one.

But the fundamental definition of cloud does not include fault tolerance.

Don't trust me.

Check out the more formal definition from US NIST (years of work, 15 drafts),
there is no mention of redundancy or fault-tolerance:

[http://faculty.winthrop.edu/domanm/csci411/Handouts/NIST.pdf](http://faculty.winthrop.edu/domanm/csci411/Handouts/NIST.pdf)

------
hnha
Warning, this page loads 20 megabytes of huge images that might crash your
mobile browser. It does that for no good reason, the images are only shown in
thumbnail size..

~~~
theonewolf
Fixed. Sorry, I was hosting them myself and keeping them hi-res for the
magazine.

ACM is fixing something at their end to host them, and at the smaller size.

~~~
hnha
thanks!

------
walterbell
Dell (T20) and Lenovo (TS140) sell quiet and low-power servers which are
targeted at homes and small business. Prices start at $300. Model with VT-d
and vPro Xeon E3 is $500. Dell model officially supports RedHat Linux, so
there's no Windows tax.

[http://www.rackrealm.com/dell-t20-server-and-
openmediavault-...](http://www.rackrealm.com/dell-t20-server-and-
openmediavault-review-rolling-your-own-nas/)

~~~
theonewolf
How much are they with 8 real cores and 24 - 32 GB RAM? And two hard drives?

------
cherry_su
I'm pretty sure the E5410s (or any Harpertown/Woodcrest processor) in a
standard configuration use fully buffered (FB-DIMM) RAM, as opposed to
unbuffered. Super cool that Dell did some trickery in the CS24-SC (apparently
CS means "custom server") to bypass that obnoxious requirement.

EDIT: Apparently this family of servers use a bastardized chipset that doesn't
require FB-DIMMs; nothing special on Dell's part.

------
manacit
I've got something like this sitting in my living room, but instead of using a
1U rackmount server, I opted for a Mini-ITX system with a LIAN LI PC-Q25B
case.

It's quiet, holds tons of drives and actually looks like it belongs next to
the TV (I run XBMC on it as well). It was pricer than the OP's configuration,
but it won't keep you up at night (and bleed your wallet in power bills).

~~~
theonewolf
Mine runs at 400W, less at idle.

We don't run it 24/7, only when we have some research jobs to run :-)

------
Animats
If you want previous-generation rackmount gear, try Weird Stuff Warehouse, in
Sunnyvale, CA. Weird Stuff is where Silicon Valley data centers go to die.
Typical: "HP Proliant DL160 G5 2x Quad-Core Xeon E5472 @ 3.0GHz 4GB RAM 1U
Server - $189.95" They have lots of older Cisco gear and all the accessories
you need to build a retro data center. Not that you'd want to.

~~~
theonewolf
That's already more expensive than what I paid for the 1U server :-)

And I packed in 24 GB RAM for less than $190 (I think $188).

Only the HDDs bring the price up, and a couple other accessories, to $300.

------
zacharycohn
How is that cloud? Isn't that just... a server?

~~~
morganvachon
I think he meant it's a repurposed cloud server, i.e. in its original home it
was used in a cloud computing scenario.

I'm with everyone else though, in thinking that a good quality workstation
would be a better fit for his purposes. For the same $300 you can get a decent
workstation with a quad core CPU and hardware virtualization on eBay that can
handle 32GB of memory, and room in the case for at least 2 HDDs and 4 SSDs if
you use a stacking mount.

------
mdellabitta
> Figure 4. The HDDs connect fine without caddies.

Uhhh....

~~~
theonewolf
So the SATA/SAS ports are low enough that the hard drives fit fine in there
and connect nicely. They're at a very slight angle.

Are you worried about vibration? Or something else? I am a little bit.

~~~
mdellabitta
Yeah, vibration, or accidental removal. I have cats, dogs, and kids.

------
esaym
"We’d need CPUs with VT-x or AMD-V so we could run VMs with accelerated
hardware support. VMs run slow as a snail without acceleration."

This is false. This is why I use xen on a debian host with paravirtualization.

I can then have many linux guests all served with cheap older hardware that
doesn't have the hardware hooks.

~~~
cthalupa
If you do have hardware extensions for virtualization optimization in your
CPU, you're probably missing out on performance with Xen by using paravirt.

System calls and memory access are slower in PV than HVM (See: Problems
introduced with paravirt when x86_64 removed two of the protection rings from
CPU architecture) , and PV-on-HVM drivers for block/network devices get you PV
level performance.

------
rwmj
HP-built Itaniums go for $100 on eBay. I bought one the other day (for kicks).
The performance is pretty nice.

~~~
theonewolf
It's pretty fun :-)

Also, if you're the real RWMJ, hello :-)

I work on introspecting virtual disks with QEMU/KVM (someday hoping to patch
QEMU to support my research).

We've interacted a few times before.

~~~
rwmj
Interesting .. got a link?

~~~
theonewolf
Off and on, little threads like this:

[https://lists.gnu.org/archive/html/qemu-
devel/2013-05/msg016...](https://lists.gnu.org/archive/html/qemu-
devel/2013-05/msg01687.html)

------
sciurus
If you want old hardware, Ebay is one route. If you don't want to deal with
bidding or need a warranty, there are places like
[http://www.servermonkey.com](http://www.servermonkey.com) that specialize in
refurbishing and reselling old servers and workstations.

~~~
theonewolf
Totally agree, as a hobby eBay is fine.

I avoided these guys (and others like them) because of the premium cost on
top.

------
kordless
On-the-fly survey for what I'm currently working on:
[http://utter.io/](http://utter.io/). Vote up if you have a server(s) at home
which you are already virtualizing. I have 5 machines running OpenStack at the
house.

------
fredsted
I hope he has a basement or something to put it with all the noise it's going
to make. I wonder if he would be better off financially buying a second-hand
desktop PC due to power usage.

~~~
theonewolf
This guy uses 400W, less at idle.

If I could find a cheap desktop with 24-32 GB RAM (again, also cheap, like $60
cheap), than that would be an option.

------
Xdes
Personally I want to make a huge Raspberry PI and Netduino cluster.

~~~
theonewolf
Sounds awesome :-)

------
bnejad
Home and cloud are mutually exclusive I think. Cool build though.

~~~
theonewolf
Au contraire.

See the cloudlets concept:
[http://elijah.cs.cmu.edu/](http://elijah.cs.cmu.edu/)

~~~
RickHull
Perhaps this is a cloudlet then? Cloud computing is not merely VMs on demand.
An essential nature of cloud is multiple distributed machines with the ability
to transfer applications (or VMs) between machines for various reasons
including load balancing, fault tolerance, and high availability. Furthermore,
the application user is abstracted from which physical machine the application
is running on. In your case, you have a single machine which you are likely
addressing directly. This is not cloud computing.

Perhaps you mean "cloud storage", where your personal devices are able to
store and retrieve things "in the cloud". The equivalent of a local FTP server
(or NAS) does not a cloud make.

I appreciated the writeup nonetheless.

------
mimog
Just buy a Synology NAS and be done with it.

~~~
kstrauser
That was my first thought. For storage, a Synology would wipe the floor with
his setup. He's using it as a compute node, though, and even the newest DS415+
wouldn't hold a candle to this server project.

------
RexRollman
As a geek hobbyist, I have always liked the 1U form factor, but the noise out
of those things can be crazy.

~~~
theonewolf
The CS24-CS isn't so bad.

Sounds like a hair drier on low, as others have reported.

It's only super loud if you load it (I calculated Pi to 5 billion digits). It
would spin up fans, be loud for 30 seconds, and spin down for a few minutes
and repeat as it computed.

~~~
micro-ram
Don't let the sound bother you. Put it down in the basement or somewhere where
you can't hear it and use the DRAC to power it on and off.

~~~
theonewolf
Precisely. We leave it in a more unused space, and the BMC is useful in
remotely powering up and checking on it :-)

------
mp4box
How long it took to complete the Memtest ? just curious.

