
Anatomy of My Kubernetes Cluster - astefanutti
https://ttt.io/anatomy-of-my-kubernetes-cluster
======
asdfman123
More software engineers should build Kubernetes clusters like this at home...

So they don't have to do it at their work.

(Unless they work at a large enough company to justify one.)

~~~
philsnow
A $500 per person "get it out of your system at home" fund, I love it

~~~
justinclift
$500 might be on the low side, but it would cover a good percentage of the
cost. :)

------
rcarmo
This is neat. I've been doing the same with Pi 1s and 2s for a few years
([http://github.com/rcarmo/raspi-cluster](http://github.com/rcarmo/raspi-
cluster)) but stopped short of upgrading to 3s and 4s because of
heat/power/cost.

I've found that it is possible to run k3s on SD cards quite well if you're
patient, but that the Pi 2 master node spent a _lot_ of time just doing
internal stuff even on an otherwise idle cluster, so I went back to Swarm.

(I also had quite a bit of trouble with setting up a container registry, which
was hard and undocumented in the first few releases of k3s. Ingress was also a
bit of a pain, it's much easier to get traefik working in Swarm...)

~~~
tbrock
This seems to be a Kubernetes thing. Why the hell does it idle using 25% of my
4 CPUs?

~~~
snupples
That makes no sense. Try looking at logs and such at least.

~~~
rcarmo
The logs don't say much. It's mostly the API server getting hammered by nodes
and various components.

~~~
snupples
Ok so it's just the API servers. Not every node which is what it sounded like.
Typically you wouldn't run normal workloads on the same host as the API.
Although these quick "how I did it" articles have a tendency to skip best
practice. Still sounds like something isn't right in your environment. They
shouldn't be hammered that hard unless you have a very large cluster.

------
justinclift
As a general thought, this kind of power supply seems like what the author was
first looking for but couldn't source at the time:

[https://www.ebay.com.au/itm/Switching-Power-Supply-
DC-5V-12V...](https://www.ebay.com.au/itm/Switching-Power-Supply-
DC-5V-12V-24V-36V-48V-CCTV-LED-Strip-Source-Transformer/372916962602)

It's a 5V, 20Amp one (thus 100W). There seem to be several of them on Ebay
from various vendors.

Using that would mean the 12V laptop power supply, and buck converter to
change the 12V to 5V wouldn't be needed. Would probably bring down the cost a
little bit too.

------
namelosw
Wow it's really beautiful, looks like a real rack.

I wonder, the wires seems to be very hard to bend. When I built custom PC,
there's custom build soft power supply / SATA wires for sale. Maybe there
should be similar things for USB or other wires that could be easier to tuck
in the case.

~~~
youngtaff
Wonder what a setup using PiBlades might look like

[http://www.bitscope.com/product/blade/?p=about](http://www.bitscope.com/product/blade/?p=about)

------
tyingq
_" 59 Watts, That’s well above the power supplied by any AC mains to 5V DC
external power supply I could find"_

MeanWell has their RS-75-5, which is 60 Watts. Usually around $20.

------
captn3m0
If you’d like to do something similar, check out the Turing Pi, which uses RPI
compute modules and runs off a ATX power supply.

[https://turingpi.com/](https://turingpi.com/)

------
kl4m
Network throughput should be tested with iperf (
[https://iperf.fr/](https://iperf.fr/) ), certainly not scp

------
zulu-inuoe
Real question coming from a non-k8-user. What would you use something like
this for? Server hosting or?

~~~
bashinator
Ultimately the question is, "how much of the total cpu/memory is used just
running kubernetes?" I know prior to rpi4, clusters like this were basically
demo/education/bragging rights projects.

------
tra3
Nicely done. I've been meaning to dive into k8s clusters (with either k3s or
kind) -- what is the advantage of building a Pi cluster over something like
multiple vagrant nodes? Not criticizing at all, just trying to understand.

~~~
rcarmo
Not the OP (see my other comment). For me it was the cheap hardware and the
fun aspects of the project. It's a tangible thing.

But I also like ARM chips and do occasional embedded development, so I had
other motives :)

------
hinkley
Power supply:

I took my desire to build a Pi cluster as an excuse to upgrade to a bigger,
newer Anker charging hub.

------
caseyf7
Beautiful. Like a work of art.

------
m0zg
Why tho? Why not just launch N KVM virtual machines on your linux box? This
literally takes minutes, unlike this project which probably took days. And
it's much faster once you get it up and running.

~~~
rubiquity
Because tinkering is fun and fun is good.

~~~
m0zg
It just seems like there are much more interesting projects available for
"tinkering" than building a slow, barely usable Kubernetes cluster.

~~~
moondev
Like what? Interested to hear what you deem interesting enough to tinker on.

~~~
m0zg
Just about anything not Kubernetes related. I'm currently building a cat toy
which combines object detection and reverse kinematics to play with my cat
using a laser pointer when she steps into view of the camera.

~~~
chickenpotpie
Why tho? Why not just use a laser pointer yourself? This literally takes
minutes, unlike this project which probably took days. And it's much faster
once you get it up and running.

~~~
m0zg
AI and robotics tho. Way cooler than some stupid Kubernetes that doesn't even
work well on this hardware.

~~~
tjbiddle
I know it's a novel concept that not many know about, but: Everyone has their
own interests and curiosities.

------
ablekh
Nice write-up.

