

Game Development in Go - jaimeyap
http://www.j15r.com/blog/2015/01/25/Game_Development_in_Go

======
lloeki
When I decided to take part in LD48 #29 (which was the first time I did a
gamejam), I went with golang. I did some research beforehand on the language,
a few basic libraries and got me back on track with OpenGL which I did not use
for years (and only ever as a toy). I maintained a log[0] and fed twitter
regularly about my progress.

To sum up I was 1) on the move, 2) in a new language, 3) in a domain I only
superficially know. Not precisely the best conditions to say the least, but I
managed to read and map keyboard and mouse input, draw sprites, animated water
(screenshot missing), and (almost) working point physics. Were I not aiming
for the realistic physics but some crude old skool implementation, I'd
definitely have a (very basic) platform game out as the last screenshot.

So, even at the lowest levels and using a few techniques I gleaned from
watching Notch's Twitch stream on LD48 #28 it was loads of fun and extremely
productive, even without support libraries or a dedicated engine. Hint:
binding a reload hack thingy[1] to cmd+R[2] was the best thing I did, being
stubborn about the Real physics thing was the worst.

[0]:
[https://github.com/lloeki/ld48-29/blob/master/log.mdown](https://github.com/lloeki/ld48-29/blob/master/log.mdown)

[1]:
[https://github.com/lloeki/ld48-29/blob/master/ld48-29.go#L31](https://github.com/lloeki/ld48-29/blob/master/ld48-29.go#L31)

[2]:
[https://github.com/lloeki/ld48-29/blob/master/ld48-29.go#L65](https://github.com/lloeki/ld48-29/blob/master/ld48-29.go#L65)

~~~
joelgwebber
Nice work. Assuming my experiments convince me that writing games in Go is
worthwhile, I'd love for the ecosystem to evolve to a point that it's
trivially easy to get up and running. The fast iteration time is very nice for
these competitions (now I just need a debugger).

I think it shouldn't be too much trouble to get to the point where we have a
set of basic composable libraries for loading and rendering meshes and other
graphics, as well as sound/input/etc. that doesn't require so much wiring. I
haven't done the exercise yet, but I'm also hopeful that SWIG will provide
good enough bindings to Bullet Physics, so that you won't stub your toe on
"real physics" again :)

------
agmcleod
My concern with using go is definitely the GC. I mostly build games in
Javascript, which has a nasty GC. But to invest in a new language, I feel like
the gains need to be better than that. Java definitely has better perf than JS
no doubt, and there's good tooling around it. But I'm happy working in the web
space of things.

The only thing im considering looking at for gamedev is Rust, but i need to
let that grow a bit first.

~~~
joelgwebber
See my comment elsewhere on this thread, about techniques for limiting garbage
in Go. I'm far from proving that this is sufficient for avoiding significant
GC pauses, but I'm tentatively hopeful. I'll post more as I get more data on
large scenes.

I did play with Rust a bit, and I do find a lot to like there. Unfortunately,
I quickly found myself dealing with an overwhelming explosion of type
parameters (both of the garden variety, and the 'lifetime' variety). Some of
this may have been my own naïveté in the language, and some bad library design
(the graphics library I was using ended up forcing me to pollute nearly every
type with three or for type parameters). But that, coupled with my own Go
experience, a slow-ish (though better than C++) compiler, and no better
debugging support than Go, led me to stick with the latter for the time being.

~~~
hedgehog
Generating garbage slowly will reduce the frequency of collections but it
won't reduce the amount of time each individual collection takes. If you need
to reduce collection times then reduce the amount of data that the garbage
collector needs to traverse each time, that is move some of your data off-
heap. The concurrent GC in 1.5 should help as long as the STW deadline can be
reduced enough (10ms is too long for a game) but going to a generational
algorithm may be necessary to get GC overhead as low as you'd want.

~~~
joelgwebber
Yes, sort of. Not generating a large amount of garbage can still give the
runtime an opportunity to reduce the amount of work done per collection pass.
A naïve heuristic for determining when to run GC will give you the typical
"sawtooth" pattern, because you generate garbage until hitting a fixed ceiling
threshold, collect it all, then wash, rinse, repeat. But a game that's not
completely pegging the CPU will have some idle time that can be used for small
collections, so limiting the amount of garbage produced on each frame should
put an upper bound on the amount of work done during each frame.

When I was helping Rovio port Angry Birds to the web a few years ago, we ran
into serious frame hitches that were being triggered by GC pauses in Chrome.
Two things fixed this -- the first was fixing a bug that caused it to run a
full mark/sweep far too aggressively; but the second was when V8 committed an
incremental collector (not concurrent, just able to spread the work out more
by being able to run a partial mark/sweep and resume it later). After that,
the GC pauses disappeared into tiny ~N00µs pauses that never impacted the
game.

~~~
hedgehog
Right now Go uses the naive heuristic you describe, there's no concurrent or
incremental collection so no matter how long it takes to get to the GC it'll
be a stop-the-world collection and potentially take a long time. I forgot to
mention that besides reducing the amount of on-heap data you can also speed up
collections in the current GC by lowering the GC threshold (SetGCPercent).
This does come at the cost of collecting more often though so it's only really
helpful if you're generating a lot of garbage on a relatively small heap. The
impression I get is that for Angry Birds you would be fine with the 1.4 GC but
Sim City might have some issues.

[https://docs.google.com/document/d/16Y4IsnNRCN43Mx0NZc5YXZLo...](https://docs.google.com/document/d/16Y4IsnNRCN43Mx0NZc5YXZLovrHvvLhK_h0KN8woTO4/edit)

Go might eventually get a generational GC but my understanding is that that
pretty much requires a copying GC and there are concerns with C interop (right
now you can point to Go memory from C).

~~~
pcwalton
> Go might eventually get a generational GC but my understanding is that that
> pretty much requires a copying GC and there are concerns with C interop
> (right now you can point to Go memory from C).

It does. Also note that incremental GC in Go is going to have different
performance characteristics than incremental GC in JavaScript, because of the
fact that Go's GC has to be thread-safe.

------
prottmann
"Over Quota This application is temporarily over its serving quota. Please try
again later."

Lol, last time i have seen such a message was ten years ago ;-)

~~~
joelgwebber
Damn, that was fast. Thought billing was enabled for that site, but I guess
not. Fixed!

Edit: Or at least it should be. Might take a couple of minutes to clear up.

------
karka91
I was experimenting with the idea of learning game dev with golang but shortly
after starting I realized that there's very little tooling for that and I'll
probably drop that project midway. Needing to make my own
engine/tools/bindings on every step drained all the enthusiasm.

Would love to see something like LÖVE made available for golang

~~~
MoOmer
But, but, love + Lua is just beautiful. I've only messed around with it some,
but it is really a thing of beauty.

Haxe is one of my biggest current interests, and Snokit + Luxe is looking
better every day.

------
wsc981
Currently the site is unreachable (for me at least). But here I've got a Coral
Cache version of the content:
[http://www.j15r.com.nyud.net/blog/2015/01/25/Game_Developmen...](http://www.j15r.com.nyud.net/blog/2015/01/25/Game_Development_in_Go)

------
zamalek
Did you at all find that the printf methodology of debugging got in the way of
the gamedev? I've been keeping an eye on Go, but have been hesitant to
approach it seriously because of that debugging approach.

~~~
falcolas
I have worked with GDB and Go before; it's not intractable, just more
difficult to interpret.

Printf frequently works very well for most debugging, especially server side.
I use the regular logging module pointed at stdout instead of printf
specifically, and increasing the debugging level works most of the time.

Its also nice that with a bit of metaprogramming and goroutines/channels, you
can make log printing nearly free. I once saw a technique where you simply
write your really detailed log messages to a ring buffer (using a lockfree
algorithm), and spit them out when you fail. This is even easier to do in Go,
since you can just send the messages over a channel and not have to worry
about properly implementing a lock free algorithm, and spit them out in a
`recover()` function surrounding your main (or look at them in GDB).

~~~
zamalek
> I once saw a technique where you simply write your really detailed log
> messages to a ring buffer

While this is incredibly cool, my initial concern was mostly this:

> I have worked with GDB and Go before; it's not intractable, just more
> difficult to interpret. > Printf frequently works very well for most
> debugging, especially server side.

The thing about your non-niche dev is that, unless you are debugging prod (
_gasp_ ), you can control the influx of data/output of data. E.g. If you want
to debug a specific webpage you can simply hit that webpage in the browser
yourself, and you are guaranteed to only see code executing that has to do
with that request.

The problem with gamedev is that things are happening 60 times a second. There
is a firehose of data and there is nothing you can do about that. Logging is
definitely used, but it is more useful on client machines once you have
actually shipped a working product.

I guess what you could do is force a fail once a condition is met, to access
the last data in the ringbuffer (and pray that it is still there, 60 times a
second is 60 times a second).

~~~
falcolas
All that makes sense to me. That said, thanks to goroutines and channels, you
can be writing out that firehose and not have to worry too much about loosing
your 60 FPS (with caveats, of course, but with buffering you can do, say, 6000
log writes a second and not have to worry about overwhelming your disks too
much).

> and pray that it is still there, 60 times a second is 60 times a second

Great thing about memory, most gaming and development machines have gobs of
it. You could allocate upwards of 600mb to the ring buffer and not feel the
pinch. That's a lot of data, especially if you want to get creative and store
pointers in there to other large structures in memory (such as a copy of a
texture or mesh).

Ultimately, though, I agree. Full GDB support would be better; logging like
this is just a stopgap.

~~~
zamalek
> You could allocate upwards of 600mb to the ring buffer and not feel the
> pinch.

Touche.

> Ultimately, though, I agree. Full GDB support would be better; logging like
> this is just a stopgap.

Looks like one of us will need to man up one of these days and make a decent
debugging experience. I'm just worried that the people at the helm of Go
aren't taking this seriously enough. I've seen one or two quotes with them
indicating that they believe printf is enough.

If you want Windows developer mindshare (keeping in mind that a fair amount of
gamedevs are Windows/VS users) you're going to need some competitive tooling,
they will give you tons of leniency, but if you ask them to printf they will
go running back into the arms of the Visual Studio debugger.

------
4ydx
I have also been developing something in my free time with go-gl. The GC issue
does bother me conceptually, but I have yet to make anything so complex that
stop-the-world GC makes me curse the screen. I tried writing something
minecraft-like in c and then c++ a couple of years ago, disregarding the fact
that I don't have deep professional experience in either, but I eventually hit
a place where my confidence dropped off too precipitously. Considering that
learning modern opengl/openal is at the forefront of my concerns, using a
simple language that is compiled felt like a good fit. If I were to choose a
different language, it would probably end up being C#.

Incidentally do you find that compiling is actually disappointingly slow? My
current project takes about 2.5 seconds to build and that feels very long.

------
mseepgood
There are also these debugging tools in the works:
[https://github.com/golang/debug](https://github.com/golang/debug)

------
Thaxll
I already asked here before but what is the status of the GC regarding real-
time games? ( for a gameserver )

~~~
joelgwebber
Here's my take on GC in games, FWIW. I'd be very leery of using a VM with a
garbage collector for the _entirety_ of a game. They can be fine (and are
extremely common) in embedded scripting languages, but it can be far too
difficult to control the size of outlier pauses when everything on the heap is
subject to GC. As mentioned elsewhere in this thread, the JVM GC has had an
_enormous_ amount of effort put into it, and it's still an issue that poses
problems for Minecraft, et al.

I'm far from proving this assertion yet, but I _believe_ that Go's memory
model allows for a middle way that will avoid big GC pauses. As I touch on
briefly in the original post, you can use Go's C-like value types and pointers
to field/elements to avoid generating garbage for large numbers of homogenous
objects (e.g., by implementing a simple pool allocator), just like you'd do in
C[++] to avoid heap fragmentation.

I hope to get more actual data on how this works as I expand my prototype, and
will do follow up posts as I learn more.

~~~
ANTSANTS
I think you're right about Go's memory model helping a lot when compared to
e.g. a typical dynamic language, but I have to ask: if you're going to be
using manual memory management techniques like object pools and avoiding heap
allocation whenever possible, what exactly does a garbage collected language
buy you? I'm more of a C guy myself, but presumably RAII with smart pointers
in C++ would get you most of the productivity benefits of garbage collection
for the parts of the code that "don't matter" with much more reliable soft-
realtime guarantees, while providing you with much greater memory management
controls and optimization opportunities for the parts that do, and having far
superior debugging support to boot.

~~~
joelgwebber
C++ management can of course be workable with enough care. I worked on Chrome
for a bit while at Google, and saw that it more or less holds together with
enough reference counting and smart pointers. At the same time, it still
requires a lot of care, and plenty of bugs have been caused by subtle mistakes
in this kind of code (which is why Chrome uses a sandbox around the actual
rendering engine, because it's far too complex to be trustworthy). But even
Blink/Chrome is moving to a garbage collector
([http://www.chromium.org/blink/blink-gc](http://www.chromium.org/blink/blink-
gc)) for C++ objects because of all the memory management complexity.

What I'm hoping is that you can _have_ a GC that allows you to avoid all these
issues without having to be super-careful all the time, while mitigating the
pause issue by reducing the garbage using pools and similar techniques. My
hypothesis is that most of the little allocations that game engines perform
are homogenous enough that moving them to pools will be fairly easy. And that
this will be sufficient to avoid big pauses. But we'll see how it plays out in
practice when I get some hard data on big scenes.

Finally, memory management isn't the only reason I'd prefer to avoid C++. I'm
particularly sick of long compile times (they could really kill you on a big
project like Chrome), and among other things I believe that Go's concurrency
model will prove a big improvement over C threading.

~~~
ANTSANTS
Interesting, thanks. The idea that Chrome is moving to garbage collected C++
is... a bit surprising to me, though I suppose it makes some sense given their
focus on security.

I can see why you'd want to get away from C++'s compile times, though they're
a lot more manageable if you can avoid templates like the plague. Have you
considered a coroutine library for C or C++? I'm using libco right now for my
hobby game project and much like "goroutines" would, it's significantly
improving the clarity of a lot of systems (though of course I don't get the
"free" parallelism because it doesn't handle scheduling across threads or
anything like that).

~~~
joelgwebber
To be precise, Blink is moving to a GC for stability (including avoiding
leaks), but I don't believe it's for security -- the renderer remains
sandboxed because it's effectively impossible to secure such a huge pile of
C++ code. This presentation (which assumes a lot of familiarity with the
WebKit/Blink smart pointers) goes into some interesting detail:
[https://docs.google.com/presentation/d/1YtfurcyKFS0hxPOnC3U6...](https://docs.google.com/presentation/d/1YtfurcyKFS0hxPOnC3U6JJroM8aRP49Yf0QWznZ9jrk/mobilepresent)

It includes particularly intriguing bits like "You can remove all on-stack
RefPtr<X>'s. This is the biggest reason why Oilpan performs better than the
current reference counting." I don't know whether that always holds true -- as
of the middle of last year, I heard that they'd gotten to the point where most
things perform roughly at parity, some worse, and some better. Keep in mind
that this is an opt-in system -- if you don't use the smart pointers the GC
knows about, it will ignore them (IOW, it's not some crazy conservative beast
like the C++ Boehm collector). Also, my understanding is that, the vast
majority of the time, Oilpan only runs when the event loop goes idle, which
makes perfect sense for a browser, and has an obvious correlate in a game's
simulation/rendering loop. I think they only walk the stack looking for
pointers in rare cases.

It's not hard to imagine a hybrid world where you opt-in to GC'd pointers, but
are free to use different allocators for performance-sensitive bits. This
smells a little like Rust, but without the need to satisfy the lifetime
checker thing.

Thanks for the pointer on libco. I'll definitely have a look at that. I've not
written much C++ (apart from Chrome and a few odds and ends while at Google)
in a long time, so it's quite probable I've missed some significant
improvements on that front.

------
pjmlp
Unity is really making a disservice to games development in safer languages by
bundling that prehistoric Mono runtime.

Anyone that isn't aware of that will assume C# in Unity == other runtimes.

------
niix
Interesting article, and I'm glad to see some progress has been made. I look
forward to the repo when its available.

