
Gokrazy: A pure-Go userland for Raspberry Pi 3 appliances - type0
https://gokrazy.org/
======
simmons
This is great! I think it could have uses far beyond the Raspberry Pi. As
someone who's worked in the consumer electronics industry for years, I think
we desperately need to rethink our approach to Linux-based electronics with
respect to both the userland and the primary application. A legacy-free
framework written in a safe language could go a long way to improving our
currently lousy reputation with respect to security and reliability. (Yes,
fresh code will have fresh bugs even in a safe language -- but it's a start!)
I'm also keeping my eyes open for similar ideas using Rust.

~~~
hugelgupf
Shameless self-promotion: also take a look at
[https://github.com/u-root/u-root](https://github.com/u-root/u-root) :)

(The gokrazy and u-root people all work at Google, I just contacted them to
see if we can help each other. Not a competing project.)

~~~
thisacctforreal
Any relation to
[https://github.com/u-boot/u-boot](https://github.com/u-boot/u-boot) ?

~~~
hugelgupf
No, what sigjuice said is right. The name was derived off u-boot, though, and
u-root can be used (and is in use) with LinuxBoot.

------
jononor
Super nice. Glad to see A/B partiton updating scheme, that is the way to go on
appliance / embedded device.

Want something like this for one of the languages that I know... Node.js,
Python or Rust. I'd include C++11, though for this kind of deployment method I
think one needs packages easily available in a way separate from Linux distro
(since there is none).

~~~
londons_explore
The A/B updating scheme is a massive waste of space in most implementations...
In many hardware devices, flash storage space for the OS is actually one of
the most expensive components, and one of the main reasons that many devices
run little hard-to-program embedded OS's rather than linux or something else
high level.

Instead, a filesystem like btrfs should be used which can give the appearance
of multiple versions, yet only use actual storage space where versions differ.

I'd prefer the root filesystem to be something using a FUSE-like layer which
can store format specific binary diff/patches between versions, and then
transparently apply (and cache) them on access.

Taking it a step further, I wish desktop linux distros had a package-manager-
fs, which is a readonly virtual filesystem which displayed all the files
installed by all packages, but the underlying real filesystem only contained
all the original compressed packages and a cache of recently used files.

~~~
jononor
Sure it is quite inefficient. Both in storage and in transmission of updates.

But it is very simple and robust. What one tests pre rollout is identical to
what will go on production devices, down to the byte. Each device with same
version is identical. It can be cryptographically verified easily. Automatic
rollback can be performed on update fail, including self-checks.

This approach of shipping a custom userland without any of the generic Linux
userland should drastically bring down the size.

~~~
dividuum
Transmission of updates can be quite small: For the info-beamer hosted
([https://info-beamer.com/hosted](https://info-beamer.com/hosted)) OS, we also
do A/B booting. Updates are implemented using zsync.

The initial installation of the OS is simply done by unzipping a carefully
crafted [1] install.zip file to the Raspberry Pi SD card. By exploiting zsync
features, it's quite efficient to reconstruct the original install.zip from
the unpacked files. Once that's done, all future updates also use zsync and
only fetch changes to the locally stored 'install.zip' file. After updating
this file we then unpacked everything again to the next A/B partition, so the
exact state of the system is always known. Updates are quite small that way.
It also helps that the complete OS is only ~37MB compressed at the moment.

[1] The root file system is squashfs and is stored uncompressed in the zip
file to avoid double compression and allows zsync to use the squashfs
blocksize to find a minimal set of changes. Similarly the initramfs.gz is
compressed --rsyncable and also also stored uncompressed in the zip file.

------
shurcooL
> The motivation is that @stapelberg spends way more time on C software and
> their various issues than he would like. Hence, he is going Go-only where
> feasible.

This is so cool! I’m very happy to learn an effort like this exists, and
looking forward to perusing the code for it... in Go (a language I really
enjoy reading).

------
sigjuice
It is really interesting that Gokrazy uses an upstream kernel (with a couple
of minor patches).

[https://github.com/gokrazy/kernel/blob/3ecdc901da51c2c5b6bf7...](https://github.com/gokrazy/kernel/blob/3ecdc901da51c2c5b6bf7ab003bf764ee4b95426/cmd/gokr-
build-kernel/build.go#L15)

~~~
colemickens
Why is that interesting? Just that they're not having to carry special patches
for the RPi3 or what?

~~~
sigjuice
Yes. I have only tried Raspbian and I'm not used to getting RPi kernels from
kernel.org

~~~
muep
E.g. Fedora has for years been using for RPi the same close-to-mainline kernel
source that is used for other ARM boards and x86-64 and any other supported hw
platform.

------
tigeba
Does anyone know of a good tool for building custom PI distributions based on
the Raspbian / Raspbian Lite or other distributions that does not involve
forking the official image builder and hacking it up? My ideal tool would
allow me to do this with a Dockerfile and just export the image after the
configuration is complete.

~~~
snops
[http://resin.io](http://resin.io) is close to this, but also handles updates
for you.

------
dom96
I'm having a really hard time figuring out what problem this project solves,
can anyone explain?

~~~
leaveyou
They say it right in the header:

"For a long time, we were unhappy with having to care about security issues
and Linux distribution maintenance on our various Raspberry Pis."

~~~
elliotec
So it’s like it’s own OS almost? What’s a userland?

~~~
AYBABTME
It's everything the kernel doesn't do. In general, that means the surface with
which you interact as a user. So if you're used to using a terminal, your SSH
connection, your Bash session, are all userland programs (that use kernel
features to do things). You can think of it as kernel is a framework like
Rails and userland stuff is developers using and expending the interfaces...
sort of.

When your computer runs headless, the init/process tree system is userland.
Background (cron) jobs are userland, daemons are userland. Etc.

~~~
candiodari
So it's a distribution ?

... that will have security issues, require updates, distribution maintenance,
and all that stuff they claim is unacceptable ?

~~~
secure
It’s not a full-blown distribution; it doesn’t distribute third-party
software. Instead, it allows application authors to build their own
distribution (at which point “appliance” is a more fitting word than
“distribution”).

Updates are easy to fully automate with gokrazy (and I’ve been using that
mechanism for a year at this point) because assembling a new image and
installing it over the network is just one simple command. New kernel and
firmware versions are provided automatically once they are verified to boot on
real hardware.

There is no maintenance aspect in the actual installations (i.e. you don’t
need to SSH into them, ran commands to update, etc.) — the root file system is
read-only, and an update overwrites it completely. Unless the applications you
want to use with gokrazy introduce state, gokrazy defaults to being stateless.

Hope that clarifies things, let me know if you have further questions.

~~~
djhworld
Forgive me if this sounds dumb, but if the image just contains the firmware +
kernel + a light layer for managing processes, do this mean it doesn't include
stuff like SSHD?

~~~
secure
That’s correct: there is no SSH daemon by default. You can optionally include
[https://github.com/gokrazy/breakglass](https://github.com/gokrazy/breakglass),
though, which is an SSH daemon that can be started on demand to facilitate
interactive debugging.

------
Ixiaus
Why is Go considered safe?

~~~
secure
Go is memory-safe. See also
[https://en.wikipedia.org/wiki/Memory_safety](https://en.wikipedia.org/wiki/Memory_safety)

~~~
Ixiaus
Are there not many statically typed, compiled languages that ship a runtime
with a garbage collector? Why is Go a good choice here?

Why is this choice better than a language that provides memory safety using
linear types so you don't even need a runtime or garbage collector?

~~~
secure
You can read more about why I like Go at
[https://michael.stapelberg.de/posts/2017-08-19-golang_favori...](https://michael.stapelberg.de/posts/2017-08-19-golang_favorite/).

Of course I used my favorite programming language for building gokrazy :).

~~~
pknopf
Hey, the creator of i3!

I love your stuff! I enjoyed your episode on Go Time as well!

Cheers!

~~~
secure
Glad you like my work; thanks for the kind words!

------
cjdell
Could/should one use this to build a touch screen user interface for an RPi as
part of a kiosk based application?

I really like Go, and suspect a Go app would beat the pants off anything
powered by Electron (performance wise).

~~~
secure
I’d love to see this, but note that you’ll need to use a pure-Go graphical
toolkit (or implement all graphics yourself), and likely interface the
touchscreen yourself, too. If you’re not comfortable with this, it might be
better to use something else.

~~~
tmzt
Maybe a Flutter port would make sense here?

------
velodrome
This is a repost from last year:

[https://news.ycombinator.com/item?id=13792287](https://news.ycombinator.com/item?id=13792287)

~~~
bringtheaction
"Reposts" are fine. Better to formulate it as _previous discussion_.

------
Kabukks
Looks great. I'd love to use it. How is the GPIO/PWM support?

~~~
secure
Have a look at [https://periph.io/](https://periph.io/).

In case any of periph.io’s features require changes to the kernel
configuration, I’m happy to change it accordingly.

