
Six-Year homelab history in pictures - devy
https://blog.networkprofile.org/6-year-homelab-history-in-pictures/
======
whalesalad
This is very awesome. For those interested, the Reddit.com/r/homelab section
is super great for this kind of stuff.

I just got a Dell R720 off Craigslist to do an experiment with the Firecracker
MicroVM of all things. It's got 48gb of ram and dual hex-core Xeons. It's a
lot of fun to play with this kind of gear.

Ikea sells a 'rack' (not the Lackrack,
[https://wiki.eth0.nl/index.php/LackRack](https://wiki.eth0.nl/index.php/LackRack))
that is really great for holding gear:
[https://www.ikea.com/us/en/catalog/products/S39031411/](https://www.ikea.com/us/en/catalog/products/S39031411/)

Some pix of my recent setup:
[https://imgur.com/a/xXc6720](https://imgur.com/a/xXc6720)

Back in ~2014 I had a similar tiny setup in my basement with a UPS and an
older (and much louder) Poweredge 2950:
[http://i.imgur.com/zxYxuLq.jpg](http://i.imgur.com/zxYxuLq.jpg)

~~~
jcims
The Dell C62x0 series high density compute systems are pretty cool too. Up to
4 nodes per chassis, dual cpu 48 gb per node.

Biggest issue is fan noise. You have to tuck them away somewhere.

e.g. [https://www.ebay.com/itm/Dell-PowerEdge-C6220-GEN-
II-4-x-Nod...](https://www.ebay.com/itm/Dell-PowerEdge-C6220-GEN-II-4-x-Node-
Server-8-x-SIX-
CORE-2-60GHz-E5-2630V2-128GB/132907453614?hash=item1ef1e6ccae:g:q6AAAOSwzd9cLj5B)

------
_jcwu
Since I got fiber gbit up and down I try to relocate as much as possible of my
homelab to real data centers (Hetzner) since I can't stand the noise. (I have
10ms ping to my server)

Then I use wireguard to connect it to my home network and make it act like a
server in the home network. (All the traffic even gets directed through home)

I still have the main storage server locally though because otherwise I'd not
be able to even watch movies if my internet went down.

My dream would be an all SSD storage server being near silet.

~~~
drudru11
What fiber did you get?

~~~
chrisper
I live in Switzerland so I have Sunrise.

~~~
drudru11
Awesome - thx for sharing.

------
jclay
I recently set up Pfsense on an old box I had lying around. The practical uses
i’ve found are: setting up the PfsenseNG DNS adblocker and setting up an
OpenVPN server so I can VPN into my LAN away from home instead of port
forwarding. I also just set up Dynamic DNS so that I can do this with a
domain.

Can anyone share other use cases they’ve found to be practical with their
homelab setup?

~~~
Ajedi32
\- Fine-grained QoS controls via PfSense

\- Guest Wi-Fi VLAN with UniFi + PfSense

\- NAS with automatic backups of client machines via Syncthing

\- Automatic file history of backups with ZFS Snapshots via FreeNAS

\- Automatic mirroring of backups to the cloud with Duplicati + Sia

\- Home media server via Plex

\- Home automation / web dashboard via Home Assistant

\- Minecraft server with MineOS in a VM

Those are all things I either have working, or am currently working on setting
up on my own network.

~~~
jclay
Can you expand on the QoS setup you’re using? I’ve recently read about the
fq_codel addition to Pfsense and wondering if anyone here has been using that.

I’m also looking for more performance related tunes as well. To that end, I’ve
configured Cloudflares DNS but not sure what else yields results. It sounds
like squid caching is mostly useless due to most traffic being HTTPS.

~~~
Ajedi32
It's actually a pretty simple queue-based setup right now. My primary intent
was to ensure that bulk downloads and uploads don't interfere with my latency
while gaming.

I did look into fq_codel but I don't believe that was a practical option on
PfSense back when I was setting everything up. I might have to revisit that
now that it has native support in PfSense 2.4.4.

------
cbdumas
"I then got hold of a 3.2TB Micron 9100 PRO NVMe SSD"

How does one "get hold" of a $3k SSD? This is awesome though, you clearly put
a lot of work into this.

------
JohnJamesRambo
What do you store on the servers and use them for?

~~~
vvanders
Not my lab but it varies from homelab to homelab. A lot of it is using it as a
safe place to understand/experiment with new tech without breaking anything in
a production/work environment.

Some people also use them to run Plex/Pihole/Pfsense and the like as well.

~~~
awat
I think this is a pretty good summation. I'm fairly active at the r/homelab
subreddit and this is pretty on point. Usually there is some combination of
upgraded home media/routing and then on top of that a learning and
experimenting section based on your specific goals.

For others though it really is a hobby in the sense of building to build.

------
mirimir
These days, I mainly use three old i5 boxes with 8GB RAM and SSD RAID10. Two
with 4x128GB and one with 6x256GB. They're all VirtualBox hosts, and run
various pfSense VPN-gateway VMs, Whonix VMs, and other workspace VMs.

I also have some old rack-mount servers. But their fan-control systems are all
Windows-based, and they're impossibly loud with Linux. I keep meaning to do
some crude electrical hack -- basically a manual speed control -- but just
haven't gotten around to it. And there's also the fact that I don't need that
much muscle locally. Because I like to keep my network activity minimal.

Everything is on a 0.25kWh UPS (3kW maximum load). With a kill switch on my
desk. The UPS and a pfSense perimeter router/firewall (and a rack with the old
servers) live in a well-ventilated closet.

------
ajmarsh
Nice write up and photos. I used to maintain a lab like this only my cable
management was nowhere near as nice as his. I've replaced this all with a
combination of AWS and an Antsle. I miss the hands-on hardware time but don't
miss the fan noise.

------
johnflan
Looks like a damn expensive hobby, one which I have a certain affinity for.

~~~
vvanders
Usually people buy old servers instead of building supermicro. You can pick up
an R610/R710 for $2-300 and even the newer R620/R720 for $3-400.

As long as you doing mind the noise + power they're not a bad deal.

------
SteveNuts
I used to run a full size rack with servers and gear but ended up downsizing
to three NUCs with 32GB in each, a simple managed Ubiquiti switch, and an 8
bay Synology like the one in the article. Doesn't have the "wow factor" of the
big rack of equipment, but it's a lot easier to work on, quieter, and cheaper
on the electric bill.

------
slyall
Worried to see the rackmount hardware close to the carpeted floor with a wall
directly behind them. Potential for dust and airflow issues.

I've had bits and pieces of a home setup over the years (currently just an HP
microserver with disks and VMs) but lots of cost and noise for something I
probably won't use a lot of.

------
sgt
Cool. This puts my homelab to shame. I've basically got an old i5 server
running Linux (non SSD), an unstable HP DL160 (never buy HP, kids!) and a
Raspberry Pi for home automation. I've also got lots of additional hardware
that I rarely power up such as Sun Blade 2000's, SGI's etc.

~~~
blattimwind
I've been trying to get rid of my PowerPC and SPARC machines forever... it is
a very slow going business.

------
woolvalley
The casual handgun on the desk in pictures is quite hilarious.

~~~
choward
You have to protect your data from both remote and local attacks.

------
kasey_junk
My problem with homelabs (mine and others) is the power management. Finding
hardware that triangulates to energy efficient/good proxy for production/cost
effective is _really_ hard to do.

~~~
icedchai
I just ignore it. I have about 10 servers in my basement. Total electric bill
is like $160/month.

~~~
bjelkeman-again
We had a small cluster in the basement, with a server for all the backups, a
firewall etc. initially for our software development and later mainly for
email.

I just decommissioned it all (still have a few network hard drives for local
backup of laptops etc). The power it was drawing was about 300-400 watt. That
is 7.2-9.6 kWh/day and 2600-3500 kWh/year. (At about $0.1/kWh.)

That is about equivalent of powering a Tesla Model 3 (160 Wh/km) 16000-22000
km, which is about what I drive per year.

Our garage has a south east facing roof which with solar panels could deliver
about 3000 kWh/year. That installation isn’t terribly efficent due to scale,
and would cost me about $15000 to install.

Instead we decided to move the email to a Swiss email provider (Migadu) that
cost about $4/month. We may install solar panels later amyway, but moving the
servers into the cloud cost $48/year and gave us enough equivalent electricity
to drive the whole year.

------
vonseel
"After messing around with the ERL for months and going through flash drive
after flash drive (What they boot from) I decided it was time to move on."

Hmm, if he is talking about the EdgeRouter Lite, I have one, going strong
since ~2014, didn't have any clue it has problems with flash drives, and there
certainly isn't any need to open the case and mess with internal components.
It's a good little router and I have few complaints with my Ubiquiti gear. I
have not placed mine in a rack without adequate ventilation, however.

~~~
DKnoll
I've heard the same complaint about the USB flash drives failing. I run an
ER-8 but have had no issues. I think it may have been a bad batch or
something.

Some people have also had issues after installing OpenBSD on them without
limiting writes.

------
walrus01
Anyone who is thinking of doing this should consider how loud and annoying 1RU
sized servers and switches are. They're not designed to be used in the same
room as people for any extended amount of time. If you have a whole room in
your house that you can dedicate to this, or a corner of a garage, sure. But
don't buy a used Dell R610 1U server off eBay and expect to put it comfortably
in your home office.

~~~
vonseel
Agreed. Workstation towers like the Z420 are probably a lot quieter for small
homelabs. I have a TS140 and Microserver, while the old Microserver is nearly
silent, the TS140 can be irritating if seeking a dead silent room.

------
indigo945
I, on the other hand, use a very old pink netbook I got from a garage sale to
host my syncthing backups and as a CalDAV server.

Hey, it even comes with a USP!

~~~
ipsum2
I love this idea. You can get decent laptops (i7, 8gb rm) with broken displays
for cheap on EBay/Craigslist, and install Linux. Best way to set up a homelab
on the cheap.

~~~
WrtCdEvrydy
This was actually one of the most cost efficient ways to do scaled low power
datacenters for the longest time.

Set the fans to max and do direct DC off of a DC UPS.

------
dawnerd
I used to run a super loud HP server I got off craigslist for something like
70 bucks. Thing was a beast. Since have switched to a dual Xeon storinator
with 30 8tb hdds. My next project is getting another storinator and finishing
wiring the house for 10ge

------
code4tee
+1 for using old equipment for home labs.

There’s lots of perfectly good hardware out there for cheap that may be beyond
it’s useful life for top end production workloads but is still great for other
projects or just tinkering.

------
chooseaname
That's an amazing setup.

My little TS140 with it's Xeon running Proxmox does just about all I need. My
Synology does the backups and my EdgeRouter-X does the routing to round things
out.

------
Vaslo
I've always wanted to do something like this. Where would someone go about
learning how? Would you need a background or are there other resources you
could use to learn?

~~~
ranger207
reddit.com/r/homelab has a decent wiki for the hardware side. For the software
side, it's mostly DevOps/sysadmin work. DigitalOcean has extensive tutorials
for that kind of stuff.

------
jtl999
I just have a Skylake Xeon E3 quad core from 2016 as my main sever and an
i3-6100 with 8GB of RAM and Intel NICs as my router, but I want more goodies
:D

------
znpy
This (wo|)man is living the dream.

~~~
jgys
“Person” is way more inclusive and way less clunky.

------
jackallis
for someone who has zero knowledge of networking, looking at those pics -
WOOOOW!!

~~~
kazen44
If you are actually interested in learning more about networking, i cannot
recommend starting your own homelab enough.

networking seems to be a hit or miss for people, some people find it very
logical and "easy" while others seem convinced it's black magic.

networking basics is rather easy, the problem is complexity that gets added
when you scale. (although i doubt you would reach that point in a homelab
though).

~~~
jgys
I have slightly more than zero knowledge of networking and would like to
remedy this.

You seem convinced it’s not black magic, so recommend a book/mooc/resource
that’ll prove it. :-P

~~~
DrPhish
This will explain the basics of LANs in a fun, but very thorough way:
[https://www.amazon.com/All-New-Switch-Book-Switching-
Technol...](https://www.amazon.com/All-New-Switch-Book-Switching-
Technology/dp/0470287152)

And this one will fill out your knowledge on the routing side, although it is
rather dry: [https://www.amazon.com/Network-Routing-Algorithms-
Architectu...](https://www.amazon.com/Network-Routing-Algorithms-
Architectures-Networking-ebook/dp/B005HRE41W)

