

Go Concurrency - jgrahamc
http://www.slideshare.net/jgrahamc/go-oncurrency

======
motter
This talk was given at the Go London User Group yesterday -- if you're in
London, come along to the next one: <http://www.meetup.com/Go-London-User-
Group/>

~~~
jgrahamc
And if you want to speak please write to us.

------
farslan
Good examples, especially where we use close() to start go routines :) However
slide 24 (of 26) is not available.

~~~
jgrahamc
Odd that slide 24 fails. You can download the presentation as a PDF and that
slide is there.

~~~
klapinat0r
func balancer() on slide 23 isn't shown. Would you care to elaborate on that?

Are you creating a buffered channel there? make(chan *job, 10)?

~~~
jgrahamc
It's on the mystery slide 24. Download the file and you can see it, or visit
this gist.

<https://gist.github.com/jgrahamc/5262578>

~~~
klapinat0r
Oh, sorry. Thanks.

Nice to see CloudFront using Go. Good talk on channels, keep up the _Go_ od
work :)

------
shrikant
PDF:
[https://dl.dropbox.com/u/15497688/goconcurrency-130328045002...](https://dl.dropbox.com/u/15497688/goconcurrency-130328045002-phpapp02.pdf)

~~~
jgrahamc
And since people want it, I've added all the code from the slides to a gist:
<https://gist.github.com/jgrahamc/5262578>

------
adamonduty
So correct me if I'm wrong...

Slide 13 shows an example memory recycler. I understand the concept, but it
uses a Go List to keep track of the elements. list.PushFront() allocates a new
object on every invocation (see
<http://golang.org/src/pkg/container/list/list.go> line 138) and list.Remove()
creates garbage for the GC.

This doesn't seem like it saves much GC work.

A while back I wrote a "StackSlice" implementation to recycle memory which
uses a statically allocated slice and pushes pointers to objects onto the
stack. This is nice because pushes and pops don't involve any memory
allocations.

~~~
kisielk
The elements of the doubly-linked list are slices. A slice can be thought of
as a reference to a backing array. The only things being created or collected
by the GC in this case are the Element bookkeeping structures (and possibly
some slice headers) which contains the the slice.

The memory that is being recycled in the example is storage region of the
slice, the size of which is arbitrary. It seems that by default they
preallocate a 100-element byte array.

~~~
adamonduty
Right, I agree with you. Its not that the strategy won't save any memory. But
it won't save any memory allocations because the list still makes them. The
number of allocations can put pressure on a garbage collector just as much as
large allocations. Maybe even more with fragmentation and the like.

In C you explicitly choose stack vs heap allocation. In go, the runtime does
this for you. I had a couple of situations where, after profiling, most of my
CPU time was going to heap memory allocations. Using the strategy on slide 13
probably would have helped some to reduce the size of the allocation, but in
my case only by a few bytes. Using a fixed-size slice that held references to
objects made a big different in performance.

In regards to slide 13 it just seems funny to "recycle" memory by forcing a
whole new allocate/deallocate cycle.

------
damian2000
Can we assume from this that Go is helping to battle against DDOS attacks?

~~~
jgrahamc
I am not involved in the whole CloudFlare/DDoS thing and the Go code I am
writing is not used in that part of the network.

------
vanderZwan
Uh... isn't this a race condition? EDIT: Oh, wait, "die" is unbuffered... But
if you have more than one worker it is a race condition though, unless I'm
overlooking something else as well.

    
    
        func worker(die chan bool) {
        	for {
        		select {
        		// ... do stuff cases
        		case <-die:
        			// ... do termination tasks
        			die <- true
        			return
        		}
        	}
        }
        
        func main() {
        	die := make(chan bool)
        	go worker(die)
        	die <- true
        	<-die
        }

~~~
jgrahamc
You would not want to share a die channel like that with multiple workers
because when doing die <\- true you would not know which one received it.

As I said early on in the presentation using unbuffered channels is 'best'
because it makes reasoning about concurrency easy. Given that die is
unbuffered the die <\- true has to synchronize with something in order to
happen; that something is the case <-die. If die were buffered then this
example would simply not work.

~~~
vanderZwan
I realise that, I just find the example interesting because I often would
expect to use a whole pool of workers, not just _one_. The close(start) idiom
you used earlier makes more sense in that context (but in a close(die) form).

------
paulsmith
Interesting use of goroutines/channels to implement a memory recycler. They
could be used to implement a thread-safe alloc/free pool of objects.

------
ddorian43
Someone who has a slideshare account post the pdf ?

------
brunoqc
Anyone has any idea why the code from slide 5 doesn't print anything?
<http://play.golang.org/p/NtjHL3F1LC>

~~~
jgrahamc
Because I shortened it for the slide. Add something at the end of main() that
waits a bit (say time.Sleep(time.Minute)). What's happening is main()
terminates before those goroutines get to run.

<http://play.golang.org/p/rJ9eE7p7gz>

~~~
brunoqc
Oh thanks! Sorry for the noise.

------
camus
a lot of articles about Go these days , interesting.

~~~
MagicWishMonkey
It looks like an interesting language, but most of the articles are really
shallow. I would love to see an example of something, like, "fetch X records
from a database, encode as json and send an email with the json embedded as an
attachment." That would answer a lot of the questions I have about library
support for basic everyday tasks.

~~~
abraininavat
That's your example of a less shallow demonstration of Go? Something that you
could do in any language given a DB library, a JSON library, and an email
library?

~~~
car54whereareu
Well it's a start at least, and his answer shows go is a programming language.

How about one of these:

How to write an OS (in go)

Maybe an article about why static typing makes for a better compiler despite
what that guy said about JIT and predictive something trees (it was about why
python is slow and it isn't the reason you expect and it was on hn a month
ago)

Malware in go

How to search for go related stuff using google

Something cool about go object files and the linker

More about tooling like the regular grammar, the profiler, etc.

How to write a xen guest OS in go (this is probably smarter than targeting
bare metal)

Writing an exokernel in go instead of OCaml.

------
Evbn
How are you tmusing close to synchronize channels? Close doesn't block.

It looks like a race condition, the workers might try to write their start
messages after main closes the channel.

<http://golang.org/ref/spec#Close>

~~~
jgrahamc
Assuming you are talking about slide 7 I'm not trying to synchronize I'm just
using the closing of the start channel as a signal to the goroutines that they
can start their work. Clearly, it's not guaranteed that all the <\- start's
have failed before close() returns.

