
Gopher-OS: A Proof of Concept OS Written in Go - ingve
https://github.com/achilleasa/gopher-os
======
achilleasa
Hi, I am the author of gopher-os. It started as a fun research project to
learn more about the Go runtime internals and I didn't really expect it making
it to HN.

If you take a look at the Go runtime sources you will notice that all the low-
level arch/os-related bits have been split into separate files which usually
invoke some syscalls (e.g. the memory allocator eventually calls mmap)

The idea I am currently investigating is to first provide an implementation
for things such as physical memory allocation and virtual memory mapping which
would ultimately allow me to bootstrap the Go allocator. From that point
onwards the plan is to incrementally add more features and gradually
initialize the rest of the runtime.

This is much more difficult than it sounds as all code must be written in such
a way to prevent the compiler from calling the runtime allocator (no variables
allowed to escape to the heap).

~~~
lloeki
Your project reminds me that I wish GOOS=efi to be a thing.

~~~
uluyol
Wasn't there once a "tiny" runtime?

[https://codereview.appspot.com/3996047/](https://codereview.appspot.com/3996047/)

------
josteink
Maybe I'm old, but I immediately assumed this was an OS built around the
Gopher-protocol[1], whatever that would mean in practice.

Gopher is already an established name, so the naming seems a bit unfortunate.

[1]
[https://en.m.wikipedia.org/wiki/Gopher_(protocol)](https://en.m.wikipedia.org/wiki/Gopher_\(protocol\))

~~~
Jaruzel
The subsummation of the word 'Gopher' into the Go language eco-sphere
irritates me also. I believe it has something to do with the Language having
adopted the Gopher (animal) as their mascot?

Just because something is no longer in common/widespread use, does not mean
you can come along and use its meaning or name on your shiny new product.

Gopher the protocol maybe niche these days, but it's not dead. The continued
use of the term 'Gopher-XYZ' by the Go language people will only accelerate
the death of the Gopher protocol, which although inevitable it does not mean
it should be erased from history by something else.

~~~
canes123456
There a tons of name conflicts for things that are more popular than the
gopher protocol. If the creators wanted to avoid this they need to pick a
unique name.

~~~
treve
My pet peeve is Atom the protocol and Atom the text editor. They're obscuring
a format and a protocol that's much more important for the web than another
text editor is, and they're doing the world a disfavor as a result.

~~~
elinchrome
Even worse is when Justin Timberlake did that song "Cry Me A River", when
there was already an excellent song by the same name. It just obscures the
existing song, and does the world a disservice.

~~~
treve
Pretty rude way to disagree with someone. I'd appreciate any criticism, but
not sure what I'm supposed to do with this comment, other than feeling bad.

~~~
kminehart
I'm getting the impression that he wasn't actually telling you to "cry him a
river".

------
weberc2
I don't know anything about OS development, but how do you implement an OS in
a language whose runtime depends on an OS?

~~~
dullgiulio
Well, the Go runtime is written in Go itself. And you can write Go without the
runtime, the same way the compiler is.

The hard part is that the runtime is indeed build on OS primitives (syscall
mostly), that you have to implement yourself.

You have to forgo all the niceties of having a runtime but you can do it.
Definitely not the most productive use of Go, but it's fun and you learn a
lot.

~~~
haberman
> Well, the Go runtime is written in Go itself.

That's a bit of a stretch. The runtime uses special pragmas that are not
available to normal programs. I don't think it would be possible to write Go's
GC in Go unless these special pragmas existed:

[https://github.com/golang/go/blob/57df2f802f0417f08100ff8002...](https://github.com/golang/go/blob/57df2f802f0417f08100ff8002f3b062e695e148/src/runtime/HACKING.md)

Also, there are magical variables in the runtime that the compiler knows about
and special-cases:
[https://github.com/golang/go/blob/57df2f802f0417f08100ff8002...](https://github.com/golang/go/blob/57df2f802f0417f08100ff8002f3b062e695e148/src/runtime/mgc.go#L241)

> And you can write Go without the runtime, the same way the compiler is.

How would you write Go without using the GC?

~~~
pjmlp
That is the magic of cross compilation.

There is already a compiler that knows those special pragmas, one just needs
to add a new bare-metal backed to it.

As for writing Go without a GC, just like in any other GC enabled systems
programming language, by not using language features that require GC in the
lowest layer, which the other packages then depend on.

Example in Oberon,
[https://people.inf.ethz.ch/wirth/ProjectOberon/Sources/Kerne...](https://people.inf.ethz.ch/wirth/ProjectOberon/Sources/Kernel.Mod.txt)

Also even ANSI C requires some Assembly or compiler extensions to fully
implement libc.

------
eberkund
Probably should have let this develop a little further before posting it.

~~~
edoceo
No way, raw stuff on HN, moar please. Lots of learnding for all

------
mhh__
Is Go suitable for OS/Systems development? Inline Assembly would be nice to
have.

I'm guessing a project like this will need to implement the Go runtime given
that (I'm assuming) it uses posix (Or windows etc.) system calls.

~~~
lloeki
Go has an assembler, but its goal is not to build, say, a bootloader:

[https://golang.org/doc/asm](https://golang.org/doc/asm)

~~~
driusan
The Go assembly language is actually one of the things that make doing
something like this difficult.

Go uses the Plan9 assembly syntax but doesn't support GNU asm syntax. gccgo
uses GNU asm syntax but doesn't support Plan9 assembly.

The Go linker doesn't allow you to not link in the Go standard library
runtime, which means you need to use gccgo to write a kernel.

The end result is that you can't do something that's buildable with the
standard Go toolchain, and even if Go were to add a "don't link the runtime"
option there'd be no way to incrementally port your ASM over one bit at a
time. It's all or nothing.

~~~
achilleasa
I take a slightly different approach for my implementation that allows me to
use the standard go toolchain. I use nasm for the early ASM code and patch the
go build tool's output (see: [https://github.com/achilleasa/gopher-
os/blob/master/Makefile...](https://github.com/achilleasa/gopher-
os/blob/master/Makefile#L39)) to bypass the link step and replace it with a
manual call to ld that links the go object files with the output from nasm.

I will be talking about this approach in more detail in GolangUK '17.

~~~
driusan
Do you know if your talk going to be online afterwards?

~~~
achilleasa
Yes, the talk will be recorded and posted online with the slides and some
getting started code.

------
saintfiends
Interesting project.

What's all this voodoo inside Makefile, I would love more comments in there. I
think that would be the first step for anyone wanting to contribute, to
understand the Makefile.

    
    
        @echo "[go] compiling go sources into a standalone .o file"
        @GOARCH=$(GOARCH) GOOS=$(GOOS) go build -n 2>&1 | sed \
    	   -e "1s|^|set -e\n|" \
    	   -e "1s|^|export GOOS=$(GOOS)\n|" \
    	   -e "1s|^|export GOARCH=$(GOARCH)\n|" \
    	   -e "1s|^|WORK='$(BUILD_ABS_DIR)'\n|" \
    	   -e "1s|^|alias pack='go tool pack'\n|" \
    	   -e "/^mv/d" \
    	   -e "s|-extld|-tmpdir='$(BUILD_ABS_DIR)' -linkmode=external -extldflags='-nostartfiles -nodefaultlibs -nostdlib -r' -extld|g" \
    	    | sh 2>&1 | sed -e "s/^/  | /g"
    

_edit_ I think I understand what's happening here, funny how writing it down
helps one understand. It's just building up a build script with sed and
executing it. The thing was I didn't know sed could do that. TIL

------
infogulch
This seems to be in very early development with the first commits in late
March and a single line readme. Interested to see where it goes and what
problems they encounter.

------
barsonme
nice! btw, wrt the `Memset` function, if you have something like

    
    
        func foo(b []byte) {
            for i := range b {
                b[i] = 0
            }
        }
    

the compiler turns it into a `memclr` call which can be pretty fast:
[https://github.com/golang/go/commit/f03c9202c43e0abb13066985...](https://github.com/golang/go/commit/f03c9202c43e0abb130669852082117ca50aa9b1)

Perhaps there could be a performance boost to a simple for-range loop if
`value == 0`?

------
pjmlp
Good luck with the project, nice to learn about it.

------
ungerik
A lightweight Go OS would be nice for virtualization environments. Deploy your
Go server/service with the minimal needed OS into the cloud...

------
solidsnack9000
Perhaps it can have a JS shell?

And store data in HTML elements instead of files?

------
fenwick67
With the S not capitalized in "Gopher-Os" I read it like a cereal (like
Cheerios).

------
Numberwang
I've never really understood OS:es. Why are so many people creating them, yet
so few have any success? I've must have seen a dozen OS:es over the past two
years on HN, most developed by people having no confidence in calling it a
serious effort.

Why care about OS:es when there is no room for anyone on that level of
abstraction apart from a handful developers at MS, Apple and Google?

~~~
skocznymroczny
Having no confidence in the project is part of the path to success

"I’m doing a (free) operating system (just a hobby, won’t be big and
professional like gnu)"

~~~
Numberwang
Hey, good luck with it! I'm thinking maybe I'm just wired differently. If I
have a side project or hobby I still want to feel maybe not success but a
sense of completion.

An OS project I suspect will always end when you no longer have time for it or
are stuck somewhere. I'd be annoyed by that.

~~~
garfij
It might have been helpful for GP to note that the quote comes from Linus
Torvalds. He said it while he was first developing Linux

