
Two Performance Walls Approaching (2009) - brassybadger
http://sebastiansylvan.com/2009/11/22/two-performance-walls-approaching/
======
trsohmers
One should also look at Gustafson's Law [0], which is kind of a foil to
Amdahl's law. While the serial portion of a program can not necessarily be
parallelized, who cares if the parallelizable part is what continues to grow
in size year over year?

Take this example: If you are a video rendering house, and do all of your work
during the day and render at night (for say, 8 hours) you can get 'x' amount
of polygons rendered (or whatever other metric). If for your next movie, say
you want to have twice the number of polygons making up your scene.
Gustafson's law says that doubling the data set size (or number of polygons,
flops, or whatever other metric that has parallelizable computation) means you
can actually get '2n' done in the same amount of time if you were to double
the amount of computational performance of a system.

You do run into a bit harder problem today with memory bound (either size or
bandwidth) applications, but there are many approaches to addressing that.

[0]
[https://en.wikipedia.org/wiki/Gustafson%27s_law](https://en.wikipedia.org/wiki/Gustafson%27s_law)

Also: Here is the original 1988 paper by Gustafson... a fantastic read, and
winner of the first Gordon Bell award.
[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.85.6...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.85.6348)

------
chubot
One way to ameliorate this is to write multi-threaded Python extensions --
i.e. with the threads on the C side, not the Python side.

You can use one core for the Python interpreter, and then 63 or 127 cores for
the C/C++ code. Python is the control plane, and C is the data plane. This is
a really useful pattern, but there is not much built-in support for it.

It would be nice if there were Go-like channels between Python and C (i.e. to
communicate between threads, instead of locking and unlocking the GIL and
mutating shared data structures). And this means that you need to
write/rewrite your own extensions too. You need to factor your program
differently, but I find this can make it more modular.

------
marcosdumay
Almdahl’s law is a problem mostly restricted to scientific code and databases
nowadays. Almost everybody else can just get more problems to solve in
parallel, and split computing power between them. (Eg. you just run more
instances of your web-server worker processes.)

And well, databases do not use glue code in high level languages (or at least,
do not lock on it). So, outside of the scientific computing community, that's
an almost complete non-issue.

------
cws
Loved this post. So prescient.

I was exposed to Amdahl's law first through this blog post about how the
concept can be leveraged for competitive advantage. Always love seeing how the
technicals affect the biz side of things.

[https://www.extrahop.com/blog/2013/20gbps-realtime-
transacti...](https://www.extrahop.com/blog/2013/20gbps-realtime-transaction-
analysis-2/)

------
mynegation
So what would be the alternative for the "glue code" language today? From the
top of my head I can think of: (1) Javascript - granted, it is single-threaded
in nature but it acquired enough parts to deal with asynchronicity and
communication with other processes. (2) Clojure - immutable data structure,
refs, atoms and all that (3) Erlang (?)

------
noir_lord
That was a really interesting post, I'd not really considered that as we hit
the limits that the glue layer will be the bottleneck.

In hindsight that makes stuff like Go and Rust look doubly obvious, most of
the benefits of something like Python with the down and dirty speed of a
compiled language.

------
jaimebuelta
Interestingly enough, these days I think that most people won't consider most
Python or Ruby code as "glue code".

------
progmal1
Are there benchmarks that show for simple "glue code" that commonly used
interpreted languages are 20x slower.

