
Go concurrency isn't parallelism: Real world lessons with Monte Carlo sims - soroushjp
http://www.soroushjp.com/2015/02/07/go-concurrency-is-not-parallelism-real-world-lessons-with-monte-carlo-simulations/
======
bkeroack
TLDR: Adjust GOMAXPROCS if you want a speedup from multiple goroutines.

[http://golang.org/pkg/runtime/](http://golang.org/pkg/runtime/)

"GOMAXPROCS sets the maximum number of CPUs that can be executing
simultaneously and returns the previous setting. If n < 1, it does not change
the current setting. The number of logical CPUs on the local machine can be
queried with NumCPU. This call will go away when the scheduler improves."

It will be nice when this requirement is eliminated.

~~~
dingdingdang
Yup, here's what I normally use for these situations (posted it in disqus too
at end of article):

// Initialize to use all available CPU cores

func init() {

    
    
       runtime.GOMAXPROCS(runtime.NumCPU())
    
    }

~~~
malkia
Maybe GOMAXPROCS exists so that the number of hw threads that might be spawn
by the go application can be controlled in a more system-agnostic way. Without
such env varianle, on Windows (for example) you can control this with CPU
affinity (and possibly other better ways), but not sure about Linux/OSX/etc.
So this kind of deals with it upfront.

------
spullara
I think that the default of single threaded is going to bite them hard in the
long run. I've already discovered libraries that were never tested with
MAXGOPROCS > 1 that are not thread safe. They should default it to at least 2
to make sure these bugs are shaken out.

~~~
bdarnell
The go race detector (go test -race) is actually really good at finding these
kinds of issues, regardless of GOMAXPROCS. I've gotten a lot more value from
running my tests with the race detector in a single process than running the
tests with multiple processes and hoping to encounter thread-safety issues in
a debuggable way.

~~~
spullara
Seems like you should probably never run the tests with -race. Is there a
downside from that being the default?

~~~
bdarnell
The only real downside is that it slows things down quite a bit (30s vs 12s
for a full test run in my current project; I've seen both better and worse
ratios in other projects). I've also seen a project that made heavy use of
generated code and a race-enabled build would choke on the very large source
files. But overall I agree that it seems worthwhile to make -race the default.

------
tshadwell
Isn't this a misleading title? The article is essentially the author
forgetting to set GOMAXPROCS, not really a lack of parallelism in Go.

~~~
chinpokomon
I didn't take it as a mistake in forgetting. I think this was more of an
experiment in understanding. Clearly the author was aware of previous
Parallelism vs. Concurrency discussions and this was just an applied test to
witness the difference first hand.

------
intortus
Another error the author made is adding to a sync.WaitGroup in a different
goroutine than the one that waits. This is another rookie mistake that go test
-race would probably catch.

~~~
Twirrim
It does indeed catch it

    
    
      $ go test -race -bench=.
      
      ...
      
      WARNING: DATA RACE
      Write by goroutine 4:
        sync.raceWrite()
            /usr/lib/go/src/pkg/sync/race.go:41 +0x35
        sync.(*WaitGroup).Wait()
            /usr/lib/go/src/pkg/sync/waitgroup.go:120 +0x16d
        _/home/twirrim/monte.GetPiMulti()
            /home/twirrim/monte/monte.go:56 +0x23a
        _/home/twirrim/monte.BenchmarkGetPiMulti()
            /home/twirrim/monte/monte_test.go:17 +0x62
        testing.(*B).runN()
            /usr/lib/go/src/pkg/testing/benchmark.go:119 +0xc0
        testing.(*B).launch()
            /usr/lib/go/src/pkg/testing/benchmark.go:207 +0x1ba
      
      ...
    

And so on.

------
tux1968
OT: The way we use the terms parallel and concurrent in computer science seems
completely backward to me. The dictionary says "concurrent" means at the same
time, and parallel lines need not be drawn at the same moment...

Yet in CS we talk of things being concurrent even if they're executed as
cooperative threads on a single core and parallel only applies when executing
concurrently (at the same time).

~~~
freyr
Perhaps _concurrent_ could be used in place of _parallel_ , but _parallel_
could not be swapped used in place of _concurrent_.

Parallel lines are non-intersecting lines, i.e. lines traveling in identical
directions. This is a nice allusion to the way parallelism works by running
identical processes that do not interact. The fact that these processes can
run simultaneously is a by-product of their parallel structure.

But yeah, the term _concurrent_ is confusing because it can be applied to
things that never actually overlap in time. But I can't think of a better term
off the top of my head.

~~~
coldtea
> _Parallel lines are non-intersecting lines, i.e. lines traveling in
> identical directions. This is a nice allusion to the way parallelism works
> by running identical processes that do not interact._

Maybe, but there's nothing about parallelism that says the parallel processes
have to be (a) identical or (b) not interact, if I am not mistaken.

Two threads running in parallel (and at the same time) might do totally
different things (not identical) and might also interact with each other (e.g.
consumer and producer threads).

~~~
freyr
Good point. In those cases, the analogy breaks down.

Merriam-Webster offers several definitions for "parallel", though the only
definition with a temporal connotation is the definition related to computing:

 _" relating to or being a connection in a computer system in which the bits
of a byte are transmitted over separate channels at the same time"_

Meanwhile, "concurrent" references "parallel" as a synonym. Perhaps the terms
are rather arbitrary after all, and we just need to get comfortable with their
less-than-perfect assignments.

------
replicant
Unrelated question, isn't it a bad idea to update the seed for every sample?

~~~
danbruc
It is in multiple ways.

First, you are wasting cycles and with so little work to do before reseeding
as in the code shown it probably matters quite a bit.

Second, some random number generators need some warm-up time producing lower
quality random numbers at the beginning.

Third, if you are reseeding faster than your seed changes, you will repeatedly
consume the same sequence of random numbers. I am not sure what the resolution
of Now() is, but unless it is on the order of nanoseconds this will heavily
affect the shown implementation.

If the resolution is one millisecond and it took 15 seconds to execute on a
single thread, then the generated random values changed only every 74
iterations.

~~~
omni
It is indeed nanosecond precision.
[http://golang.org/pkg/time/#Time](http://golang.org/pkg/time/#Time)

~~~
danbruc
The value is represented with nanosecond resolution, but that does not imply
that Now() will return a different value every nanosecond.

~~~
omni
You are of course technically correct. It seems the precision returned by Now
may be platform-dependent.
[https://github.com/golang/go/blob/master/src/time/time.go#L7...](https://github.com/golang/go/blob/master/src/time/time.go#L777)

------
SixSigma
No-one has ever claimed it is, in fact they specifically tell you it isn't,
multiple times

A Jan 2013 Go lang blog post

[http://blog.golang.org/concurrency-is-not-
parallelism](http://blog.golang.org/concurrency-is-not-parallelism)

reminding people of the Jan 2012 talk Rob Pike did on the subject

[https://talks.golang.org/2012/waza.slide](https://talks.golang.org/2012/waza.slide)

Feb 2011 : Rob Pike on Parallelism and Concurrency in Programming Languages

[http://www.infoq.com/interviews/pike-
concurrency](http://www.infoq.com/interviews/pike-concurrency)

I'll skip all the intermediate steps from there back to

Tony Hoare

[http://www.usingcsp.com/](http://www.usingcsp.com/)

~~~
soroushjp
Yep, and the title is a reference to Rob's talk on this.

I refer to it in the post:

"Rob Pike, one of the creators of the Go language, dedicates an entire talk,
"Concurrency is not Parallelism" to this ...."

Wrote the post as a real world example to demonstrate Rob's point.

------
cubano
I have just started a Go project for a core aspect of my business, so this
information was well timed, for me at least, and gave me a quick overview of
concurrency/parallelism in Go.

I will be developing both concurrent and parallel threads in my app, so this
was very enlightening.

Thanks to the author for his clear writing style and efforts to educate.

------
pbnjay
Its a decent intro, but I think instead of just jumping into parallel code,
its better to read the docs before hand. There are plenty of references to
GOMAXPROCS and the thread safety of math/rand (and most of the stdlib).

~~~
soroushjp
Completely agree, nothing beats the docs.

------
omni
This is a somewhat trivial suggestion, but it would be much more clear to your
readers what the speedup was if you aligned the values in the benchmark
results.

------
dschiptsov
[http://joearms.github.io/2013/04/05/concurrent-and-
parallel-...](http://joearms.github.io/2013/04/05/concurrent-and-parallel-
programming.html)

Parallel, at least in English, means to have nothing in common with others. On
most OSes it is a _process_ affined to a dedicated CPU which is configured to
access a dedicated physical I/O device only. Everything else is concurrent.

------
nickbauman
Parallelism is like hunting elephants. The languages we use like Java, Ruby,
Python and C++, for example, give you weapons for the hunt on the level of a
very strong toothpick at best. Go has apparently upgraded the situation to a
3" pocket knife. Clojure to the level of a proper spear. But we need languages
that allow us to completely avoid hunting elephants in the first place.

------
kid0m4n
Have sent a PR handling all the wrinkles: [https://github.com/soroushjp/go-
parallelism-monte-carlo-demo...](https://github.com/soroushjp/go-parallelism-
monte-carlo-demo/pull/2)

* There is a race condition where wg.Wait() might execute before the wg.Add(1) runs * The wait group was not necessary as we are already waiting for all the results from the channel

~~~
soroushjp
Thanks again, very helpful and updated all relevant code in the post also.

------
soroushjp
Thanks for all the feedback everyone. Incorporated everything to make the code
as good as possible, want readers to learn as much as possible. Just to be
clear, this article was my way of highlighting Rob Pike's point that
concurrency _isn 't_ parallelism, not to make it seem as if Go didn't support
or was falsely claiming true parallelism.

------
wpeterson
It looks like you didn't allow for using more than a single process by setting
GOMAXPROCS.

Additionally, it looks like you're re-seeding your random engine inside your
sample loop, which is very slow. You only need to seed the engine outside the
loop at the beginning.

~~~
soroushjp
Fixed the issue, thanks wpeterson

------
zzzcpan
I wouldn't call this "real world". In a real world you are better off
distributing these kind of tasks across multiple machines. Multicore
parallelism per se is overrated and overhyped.

