
Converting an old MacBook into an always-on personal Kubernetes cluster - spalas
https://devopsdirective.com/posts/2020/03/always-on-minikube/
======
wpietri
For what it's worth, I built one like this on an old Linux laptop. I then
moved my home lighting controller onto it as a test. Some months later my
house lights stopped working, and I had to spend a few hours applying snippets
semi-randomly from Stack Overflow. It could be the problem was due to expired
Kubernetes certificates, or it could be something else.

Either way, my conclusion was that Kubernetes is really meant for more scale
and attention than I like to give my home infrastructure, so although I
enjoyed the experiment, I would encourage people not to run important services
on something like this.

~~~
odensc
FYI, if you're looking to deploy a hobbyist/side-project Kubernetes cluster
I'd highly recommend k3s [1] - it's a lightweight certified distribution, and
is ridiculously easy to set up. I haven't needed to do any maintenance besides
the periodic update.

It also works fine for production in my experience, I have a mildly popular
website running on a 4GB VPS with it and haven't had any issues related to k3s
itself.

[1]: [https://k3s.io/](https://k3s.io/)

~~~
ukd1
Similar, and probably slightly easier from my experience, plus supports things
like clustering and easily managing service, is microk8s[1] from canonical.

[1]: [https://microk8s.org](https://microk8s.org)

~~~
odensc
Interesting, I'll have to look more into that and the differences to k3s. But
k3s does support clustering BTW.

------
s-km
What sort of fun projects/experiments do people do with a super low powered
k8s cluster locally?

I'm kind of in this weird position where I understand the benefits and use of
k8s, but I:

a) Can't think of any cutesy distributed systems/microservices type thing that
I could or would want to run on a low power machine locally (lack of
processing power or ISP getting pissed off @ massive amount of traffic if
you're e.g. scraping a ton of data and doing stream processing on it in your
little cluster)

b) Don't really understand the point in investing time in it, as it feels like
one of those things you learn on the job as it comes up. And for a lot of
people (the majority, probably?) it'll probably never even come up unless they
just are hunting for new tech to introduce at work regardless of if the
business actually needs it. Which IMO, most businesses don't even have a
compelling reason to switch from the old 3 tier monolith architecture.

~~~
michaelt
_> Don't really understand the point in investing time in it, as it feels like
one of those things you learn on the job as it comes up_

I don't need to spend my time developing skills for the job I've _already got_
, I need to develop skills for the _next job_ I'll have

 _> most businesses don't even have a compelling reason to switch from the old
3 tier monolith architecture._

...thus showing I can't rely on my employer to keep my skills up to date for
me.

Not that there's no market for specialists in older technology - back in 1999
I heard rumours that COBOL experts were commanding huge salaries to work on
millennium bug mitigation in banks. But people following that career path
should be choosing it consciously, not by accident :)

~~~
spottybanana
You're betting on the technology demand to move to certain direction. To me it
feels like investing in stocks, but instead of betting with your money you are
betting with your time and brain cycles invested. What makes you so confident
that this piece of technology will flourish compared to so many others?

~~~
bigj0n
Kuburnetes is lightweight, extensible and based on open standards, which is
the recipe for a long-term solution in this space. It also has first class
support in all of the major cloud providers and has an established tooling
ecosystem around it.

~~~
geerlingguy
I'll agree with everything here besides 'lightweight'.

~~~
rumanator
> I'll agree with everything here besides 'lightweight'.

Kubernetes is pretty lean. It does require a significant mental load to get up
and running, but that's mostly due to how it forces developers to venture into
the old and largely unfamiliar sysadmin territory, where you need to pay
attention to more stuff other than the compiler finishing a build job.

------
mosselman
You could sell the MacBook and build a decent raspberrypi cluster with the
money you make on it.

~~~
rmdashrfstar
A big problem is local storage provisioner for kubernetes is still not GA. I
want to use an SSD attached to one master, and use that as storage for the
other RPIs. Doing this is still undocumented/the Wild West.

~~~
vetinari
Disclaimer: k8s beginner here.

As far as I understand, local storage provisioner is for the node local
storage; the storage doesn't follow the workload to whatever node is the pod
scheduled, but schedules the pod to the node containing the storage device. It
doesn't allow for pods to access local storage outside the node.

So for worker nodes using storage on the master node, isn't it better to use
either iSCSI or NFS?

~~~
escardin
This is exactly right. Local storage is conceptually the persistent volume
equivalent to an emptydir volume mount.

If you want to have a 'storage node' in a simple way, the NFS storage provider
is the way to go. You install the nfs client libs on each node, setup an NFS
share and configure and run the provisioner[1].

My experience with iSCSI is to stay the heck away from it. It is not what you
want. iSCSI is really meant for people who already have iSCSI SANs and not
people who have a disk they want to share. The more I learned about it, the
more I learned that I should have picked something else for every use. It's
not that it's bad, it's that it solves a much different problem than I
expected given the networked nature of it.

[1] [https://github.com/kubernetes-incubator/external-
storage/tre...](https://github.com/kubernetes-incubator/external-
storage/tree/master/nfs) (I think this is the right one, been a while).

~~~
zimmertr
NFS-Provisioner and NFS-Client have been around for a while. They're about to
be promoted from Incubator actually.

~~~
yebyen
That is fantastic, I remember when they were new ideas without any
implementation but just a few people trying some things on GitHub issues.

[https://engineering.bitnami.com/articles/scaling-
wordpress-i...](https://engineering.bitnami.com/articles/scaling-wordpress-in-
kubernetes.html)

This article does the NFS approach justice I think, I was pleased to find it
has been a working strategy for a while!

------
ivan4th
For me there's very good reason for running home k8s cluster: dogfood factor.
I run my smarthome and home surveillance (zoneminder) stuff, along with UniFi
controller on k8s, and I must say that helps me a great deal with SRE part of
my job. I do learn some important things before I encounter them in the work
setting, such as:
[https://twitter.com/ivan4th/status/1236481744477532171](https://twitter.com/ivan4th/status/1236481744477532171)

------
bluedino
An HP or Dell USFF PC would be a great alternative. Something like an OptiPlex
7010 or 800 G2. You can pick them up for $50 used, if you need more power you
can add as large of an SSD as you want plus 16GB of RAM, they're small and
quiet and can run Linux.

~~~
dangus
Yes but this cost the author $0

------
captn3m0
I used to run minikube on my MBA till last year (Was running Arch). It was
very underpowered to run a cluster, as I guess they'll soon find out. This was
a MBA 2015, so the OP's 2012 MBA is even less powerful.

The fans would start whirring, and the device was immediately unusable for
anything else. I switched to using microk8s, which is slightly better, but
still makes the device crawls. The MBA also only has 4GB RAM, which is very
low on what you might count as k8s-ready.

~~~
bluedino
Installing Linux on the MacBook Air would be a huge improvement. Less overhead
and not having to use VirtualBox to run Linux anyway...

I have an old T460S, I ran VirtualBox on it for a short while, it was so slow
it felt like something was wrong.

------
johnchristopher
There are things I don't understand with Kubernetes and the "spin up
containers as needed". How do you spin up different databases ? It won't sync,
unless your code takes that into account... Say, I have a simple WP site
running with a web, a php and a db container... What's the schema to give 2 or
3 more database container and still have visitors see the same content ? (and
the databases being consistent) Set up replication ?

~~~
usui
To tackle this issue in Kubernetes (specifically for a db), I have created
persistent volume claims that are mounted across replicas in a db deployment,
with the permission of multiple readers but only one writer. If we know that
Postgres stores all its data in /var/x, then we can mount /var/x in all
replicas as a shared volume.

In terms of the point related to taking into account in your application, as
long as you have all the db replicas under one umbrella as a
deployment/service, then having one endpoint for the db is fine and it is no
concern of the application.

Keep in mind I am still learning Kubernetes, but this is what I have done to
scale up separate back end components. Are there any objectionable/wrong
practices being done?

~~~
q3k
This is not the right approach.

First, ReadWriteMany implementations (which depend on your cluster) might not
guarantee the sort of POSIX filesystem consistency that databases expect.

Second, does Postgres in read-only replica expect to be run on a read-only,
possibly-changing volume? What's the consistency model then?

The standard way of doing this is to run a single postgres instance on a
single PVC/PV (that replicates across the cluster anyway), letting the cluster
move the pod if it dies. In addition, you can run read-only postgres replicas
for some semblance of read-only HA while the master reschedules on failure.
You can also go deeper into faster failover mechanisms (without having the k8s
scheduler in the hot path of that) using any of the tons of postgres HA
systems.

------
DmitryOlshansky
Or how to elegantly waste a perfectly fine piece of hardware in a neatly over-
engineered way ;)

------
greyskull
I wonder what it would take to do something like this with kind[0] so you can
have something resembling an actual multi-node cluster, if it's even possible.

I've used kind successfully on WSL for experimenting locally, and even found a
script to open up ports on the firewall and set up a port "forwarding" of
sorts using the netsh utility, which let me access a program bound to a port
within WSL. Though I suspect additional hurdles considering however the
networking for kind works.

[0] [https://kind.sigs.k8s.io/](https://kind.sigs.k8s.io/)

~~~
monadic2
Why run windows at all?

~~~
akiselev
Dual booting/VMs might not be practical or worth the effort if you're a gamer
or work in one of these fields: mechanical/electrical engineering, firmware,
lab or factory automation, architecture, etc.

~~~
tsm212
I guess the only limiting factor in dual booting would be a lack of skill to
set up dual boot. It's quite intimidating at first. I have been dual-booting
for ~8 years and I have had my fair share of blunders but nowadays setting up
a dual boot pc is very easy especially for the tech savvy (people who are
running a k8 cluster) people. I cannot even recall if there was any time where
I could restore atleast windows part of the os.

~~~
yjftsjthsd-h
I don't think it's just a skill thing; multibooting is more surface for
"interesting" bugs and annoyances as you force interactions between systems
that weren't designed to work together. Sure, you might know how to fix
Windows overwriting the bootloader for the nth time, but it's still a hassle.
Yes, you know how to get the UEFI settings the way you want them, but if you
were single-booting the system would have done it for you. I get that it's not
that bad once you're used to it, but there is a cost to these things.

~~~
akiselev
It starts with messed up clocks and ends up with a bricked SecureBoot.

------
punnerud
If you want to use your own domain you can turn Cloudflare into a DDNS by
frequently checking your public-IP for changes and update the records on
Cloudflare using their API if it change (this is free, except the domain
cost): [https://github.com/punnerud/cloudflare-
ddns](https://github.com/punnerud/cloudflare-ddns)

~~~
jyrkesh
I'm doing exactly this now on an RPi running Docker Swarm. Only difference is
that I pay Namecheap for the domain, so Cloudflare is free...I know Swarm is
outdated, but it was super easy to start being useful on just one node, now
it's running all sorts of things for me.

But yeah, it's just a shell script that hits the API every 10 mins.

------
viro
Am I the only person that gets bugged when people use the word cluster for a
single computer?

~~~
Spivak
If I take a physical machine, install ESXi on it, spin up three VMs, install
etcd on them, and join them together I have a cluster.

If it's instead three etcd processes in the same VM it's still a cluster.

If you evacuate two of the processes and allow a single node to maintain
quorum it's still a cluster.

"I always get bugged when people use the word array to refer to an array of
length 1."

------
fulafel
Anyone have a theory why they're running OS X on it?

~~~
diffeomorphism
> I happen to have a 2012 MacBook Air sitting around unused now that it is no
> longer my daily driver.

It already had MacOS installed and there was no strong reason not to use it.

~~~
e12e
Does it run the latest os x OK?

------
fortran77
I've had bad luck keeping laptops powered 24/7 for months or years. The
battery always swells and fails.

~~~
teejmya
Yes, this is inevitable if you're going to run this plugged in over a
significant amount of time. Should remove the battery.

I found no mention of this in the article, which I think is dangerous.
Unfortunately I didn't find any way to contact the author on his site (except
an unused comment plugin).

~~~
spalas
Hi -- Author here.

Thanks for the word of warning. I'll take a look at removing the battery to
avoid any issues.

~~~
Terretta
I wouldn't change your setup based on anecdata.

For instance, my MacBooks tend to be in use 5 - 10 years because they get
handed down, and I essentially _only_ run MacBooks while they're plugged in
(e.g., off power less than one day a week), and have never had that happen.

I certainly see battery capacities drop after 3 years or more, and simply buy
a new battery.

OWC "MacSales" batteries:
[https://eshop.macsales.com/shop/Apple/Laptop/Batteries](https://eshop.macsales.com/shop/Apple/Laptop/Batteries)

~~~
teejmya
I'm glad you haven't had this issue before, but I'm willing to bet that the
(admittedly not often) time you spend unplugged has prevented it.

I've run several laptops as you describe without issue, however I've also run
two macbooks at two different times plugged in 24/7 as "servers" and both had
this issue within 3 years. The first of these two shattered the glass trackpad
which was a safety issue in itself. Apple agreed and fixed it for free even
though the warranty had expired!

This happened to one of my Dell laptops last week since it had been docked for
a year and already had a worn battery. So that's why it was fresh on my mind
when I commented. Luckily I caught it because the plastic case bent upwards...

------
spalas
Initially, I tried doing this with a 2010 Mac Mini (which claims to support
VMX), but couldn't get Minikube running on it...

For the past two weeks though, I haven't had any issues with this setup on the
Macbook Air!

~~~
josteink
> which claims to support VMX

I though Kubernetes was all about containers, not VMs?

Why not just boot proper Linux on that thing?

~~~
jrockway
Your Kubernetes node itself is Linux, so requires virtualization if your host
machine is not Linux. Docker for Mac also works this way; when you run a
container, it's running on a Linux VM that Docker sets up, not on your Mac
directly.

Even on Linux, minikube uses a separate VM. It's just cleaner than having the
kubelet running on your workstation directly. (microk8s takes a different
approach and runs on your machine directly. The last time I interacted with
it, it destroyed my coworker's workstation and we had to reinstall the machine
completely. k8s is pretty invasive and really wants an entire machine at its
disposal. VMs are just perfect for that.)

~~~
josteink
> Your Kubernetes node itself is Linux, so requires virtualization if your
> host machine is not Linux.

Thus my question: If the aim is to use the machine for a Kubernetes "cluster",
why not boot proper Linux on it, so Kubernetes can run at full speed, without
any VM overhead?

~~~
jrockway
With processor support for virtualization, the overhead is minimal. I
personally use a Windows machine at home (for games, sigh) and run VMs for a
Linux development machine and a few k8s nodes for testing. The performance
inside the VMs is excellent.

I think right now if you want to have a single-node testing cluster, you will
be very happy with minikube and the VM it creates. If you want multiple nodes,
you will be very happy with VMs; you can create, destroy, and inject errors
right from the command-line without having to walk over to physical machines
and manipulate them.

------
api
Works just as well on any sufficiently powerful old laptop. I find that any
damaged laptop or one with a battery that doesn't hold a charge converts
nicely into a home or lab server.

------
xwdv
A better thing to do with an old MacBook is using it for an Arcade cabinet.

I wish the person who wrote this article described what he did next with this
K8s cluster. Probably nothing.

~~~
spalas
Author here --

Something like this? [https://www.imore.com/mac-mini-mame-arcade-cabinet-
project](https://www.imore.com/mac-mini-mame-arcade-cabinet-project) (Looks
cool! Some time down the line I'll have to try it out)

The first thing I did with the setup was to learn about the differences
between Helm 2 and Helm 3. I had used Helm 2 in my previous job and wanted to
get some hands-on experience with the latest version by installing and
modifying some helm charts.

This is certainly something that could be accomplished with a similar setup
running locally on my primary computer, but I like the reduced (mental)
activation energy of always having it ready to go.

------
brian_herman__
Should I do this to my 2009 macbook?

~~~
spalas
I don't think the CPU will support virtualization to use this approach.

You might be able to install Linux directly and then using something like
[https://microk8s.io/](https://microk8s.io/) instead...

------
drceng
But we are always at home nowadays…

------
jariel
Why would one want a 'personal kube cluster'?

~~~
yjftsjthsd-h
You're on _Hacker_ News. Playing with fun new technology is 100% a valid
"usecase" here:)

~~~
jariel
Yes, of course! Which is why it's a rhetorical question.

Many technologies, I can see how people would want to hack on.

But Kube, to me, is going way down the rabbit hole - a tech to support another
tech, to support another tech, to support another tech, to do maybe something
at scale, which few will ever do.

I feel as the Kubes is one of those almost entirely arbitrary forms of
complexity that pulls our nerdy attention into the netherworld.

I feel lately that tech people are creating a fully on dystopia of total
complexity: more than any one individual can grasp in a lifetime, and a
situation in which it's nary impossible to know even which direction someone
should head to be a 'pragmatic contributor' who also has 'some semblance of a
life' without woefully falling behind.

It's one thing to have enthusiasts, it's another to have a situation wherein
only kids coding since 18 and running their own 'kube clusters' and 10-layer
stacks at home have the chops to do what's necessary.

------
jstewartmobile
macbook air is a good computer. put it in a drawer. it might still come in
handy one day.

shitting it up and wearing it down with kubernetes is a guaranteed wrong move.

