
Concurrency Is Not Parallelism (2013) [video] - tosh
https://www.youtube.com/watch?v=cN_DpYBzKso
======
cultus
Here's a fantastic set of cartoons explaining concurrency and parallelism:

[https://typelevel.org/cats-
effect/concurrency/basics.html](https://typelevel.org/cats-
effect/concurrency/basics.html)

------
yvdriess
After many years of discussions on the topic, my goto line is now the
following:

"Parallelism is for performance, Concurrency is for correctness."

~~~
remify
Clean, simple. I like it

------
millstone
Concurrency and parallelism are properties of the resolution with which you
view your program. View a program at the distributed system level, vs the OS-
scheduled level, vs the microarchitectural superscalar / OoO / SIMD level, and
you will come to different conclusions about whether the program is "parallel"
and/or "concurrent." These are contextual.

~~~
jzoch
To be clear up-front: I agree with you. However, this line of thinking can be
a bit disingenuous as it may turn into convincing someone their single
threaded program in C is actually multithreaded because of OS scheduling or
because it runs on a distributed k8s cluster.

It reminds me of how technically there is space between everything if you go
small enough (ie cellular or atomic) and how some elementary school kids would
use that to convince you that they arent touching you :)

------
turingspiritfly
Doesn't Simon Marlow explain the difference very well? Concurrency is
implementation detail and parallelism is an execution detail?

------
omazurov
Concurrency = logical simultaneity. Parallelism = physical simultaneity. When
the former is mapped to the latter we encounter scalability. Some examples:
Unix commands (cp, grep, awk, sort) do not scale on multi-core systems due to
total lack of concurrency in their implementation. Quick sort scales worse
than merge sort because the latter allows for more concurrency. A parallel
program using a barrier between outer loop iterations (f.e. matrix inversion,
or computational partial differential equations) won't scale well because
barriers are antithetical to concurrency.

~~~
pmarin
>Unix commands (cp, grep, awk, sort) do not scale on multi-core systems due to
total lack of concurrency in their implementation.

This is not true. Unix commands are designed to be used in pipelines. Unix is
actually the easiest way to run things in parallel, so easy that people don't
even notice.

~~~
dspillett
That isn't the individual tools showing concurrency - they are running
concurrently due to the way the OS manages processes and pipelines.

The overall task has multiple things running concurrently but it doesn't have
to[1] and the individual components are doing no concurrency of their own. It
has the same effect in many cases because of the buffering built into how
pipes are handled, for instance passing the output of gzip through gzip
through gzip will happily use most of three cores (if that much CPU time is
free at that moment) but try that with commands that need to block such as
invocations of sort for example and you'll see that break down. One sort
command in your pipeline will tie the whole lot to not scaling much beyond one
core[2]. Also pipes and other trickery like xargs/parallel don't help if the
task is something like "grep over this large file" \- that will only scale
over multiple cores at all if your implementation of grep supports a multi-
threaded search (and your data is in cache or otherwise in RAM so the process
isn't IO-bound).

[1] it _could_ queue each and spool the output between to temporary storage,
it is so chose [2] note that some implementations of sort, such as the GNU
version seen in most (all?) linux distributions, _do_ support their own multi-
core use

------
CoconutPilot
The dictionary definition of concurrency contradicts this video. Concurrency
is defined as "happening or existing at the same time".

~~~
steamer25
I think what he means is that he's defining 'concurrent' as a sort of abstract
parallelism which can run on 'n' CPU cores. It becomes concrete
parallelization when you set n to 2 or more but can still be considered
concurrent software architecture even if someone only ever runs their copy
with n=1.

To address another comment, I agree this distinction has limited utility
(assuming I understand it correctly). This context of discussing architectures
which promote abstract parallelism is probably about the only place it's
useful.

I agree that a strict/true interpretation of 'concurrency' (as distinct from
Pike's definition) would not include time slice multi-tasking since it depends
on the limits of human perception to appear as truly-concurrent/executed-in-
parallel when it's actually fine-grained sequential processing.

------
agumonkey
Isn't parallelism one limit case of concurrency (zero dependencies) and the
opposite of full sequentialism (fully dependent) ?

~~~
jmct
That's true in one direction, but not the other.

The other angle is that concurrent programs are still concurrent on a uni-
processor. Parallel programs are not.

~~~
millstone
What is a "concurrent program?" If concurrency is about program structure,
does that structure have to be explicit? If I rewrite a loop to support OOE is
it thereby concurrent?

A uniprocessor may be pipelined, superscalar, support SIMD, etc which would
seem to satisfy the definition of parallelism. So isn't parallelism achievable
on a uniprocessor?

The more I think about this stuff the less sense the distinction makes.

~~~
gpderetta
> So isn't parallelism achievable on a uniprocessor?

Of course! Normally unqualified parallelism refers to multiprocessor
parallelism, but instruction level parallelism is also a thing. So is
overlapping io or memory access with computation. Even memory accesses
themselves can be parallelized (caches are multi ported, CPUs allow multiple
outstanding cache misses), or IO (multiple disks or network cards).

~~~
agumonkey
From what I recall, Quake existed because of instruction level parallelism
between apu and fpu. This indeed has importance.

------
bottled_poe
There are several misleading comments here making the incorrect assumption
that the term “concurrency” in software has something to do with time. In
software design, when we talk about concurrency, we are talking about
coordinating distinct logical processes (not necessarily OS processes). This
term has limited similarity to the common-knowledge understanding of
concurrent events “happening at the same time”. Logical (not temporal)
ordering of operations is implemented using process synchronisation
techniques. One of the important differences between parallelism and
concurrency is that parallelism is typically restricted to almost identical
process definitions which aim to exploit the physical architecture of
processing units. Concurrency, however, makes fewer assumptions about the
physical architecture and focuses on software design in a more abstract way
with a primary objective of coordinating resource utilisation of competing
processes (especially physical resources).

------
ejanus
What is the core difference between Actor and threadpool (Boss/workers)? I do
know that this is off-topic, please spare me .

------
justin_vanw
A distinction isn't necessarily useful.

~~~
jjtheblunt
no kidding...it's become some convention in the last 15 or so years, but in
the early 1990s working on supercomputers i had never heard of it.

~~~
dboreham
I seem to remember it coming from Golang people.

~~~
dagw
_I seem to remember it coming from Golang people._

The discussion existed around languages like Python, long before Golang showed
up on the scene.

------
realone
@4:05 people with lesser minds ;)

