
Bartosz Milewski - The Downfall of Imperative Programming - dons
http://fpcomplete.com/the-downfall-of-imperative-programming/
======
DarkShikari
If only it was that easy: change programming languages, change programming
models, and _poof_! Magical parallelism.

But parallelism is harder than that. It's an algorithm problem, a design
problem, not a language or code problem. While OpenCL might be harder to write
than plain C, for anything except the most embarrassingly parallel problems,
that difficulty _pales_ in comparison to making the solution parallel to begin
with.

For every problem where you can simply split into masses of shared-nothing
tasks, there's a dozen others where you can't. Rate-constrained video
compression. Minimax AI search. Emulation. All of these _can_ be parallelized,
but it requires parallelizing the _algorithm_ and making sacrifices that are
light-years beyond what a dumb compiler that isn't even allowed to change
program output (let alone rewrite half the program) could do.

Modifying an algorithm -- possibly even changing its structure entirely,
changing its output, or even accepting nondeterminism -- is _inherent
complexity_ , to use the terminology of Mythical Man Month. No programming
language or tool can eliminate this complexity. Good tools can make it easier
to implement an algorithm efficiently, but they really can't take a
complicated application and figure out how to change its design, structure,
and behavior. Until we have AI-complete compilers that take program specs as
"code" to be compiled, that's a human job.

~~~
NathanRice
One significant issue is that most programming languages are inherently
temporally inexpressive, having either full order (imperative/impure
functional) or no order (purely functional). Ideally the language would make
it easy to organize statement in such a way that a programmer could indicate a
partial ordering relation without having to use function composition.
Additionally, a big problem with function composition in purely functional
programming is that multiple long function compositions can only be ordered as
monolithic units, however in many cases being able to order at the individual
function level is desirable. If a language had an explicit notion of relative
temporal index for statements (which could be adjusted) it would be a nice win
for writing concurrent code. That would also let compiler writers lift a lot
of the stuff they do up to the program source level as macros (which would be
a HUGE win).

~~~
webjprgm
A futures library solves the case where, in an imperative programming
language, the programmer writes a block to compute A, a block to compute B,
then combines A and B. Pure FP works fine in that case, since writing (+
(compute-A) (compute-B)) does the same ordering.

It seems to me pure FP is fine unless computations for A and B have
interdependencies, which would result in duplicate computation in pure FP
unless you can refactor to a different algorithm. I'm not sure I understand
what you mean by wanting to control ordering at a "the individual functional
level", unless you mean interdependencies between computations done in
"monolithic units".

    
    
        A->(loop B until done)->C
        A->(loop D until done)->E
        
        C-\
          +-> F
        E-/
    

Something like that. Am I one the right track?

No, because I can still express that in pure FP:

    
    
        (let [(A (compute-A))]
          (let [(C (compute-C A)
                (E (compute-E A)]
             (compute-F C E)))
    

Trying again ...

    
    
        A->(loop B until (valid B D))->C
        A->(loop D until (valid D B))->E
        
        C-\
          +-> F
        E-/
    

Is that better? Care to help me understand what you're getting at?

~~~
NathanRice
I was thinking more along the lines of working around data-locality issues.
For instance, imagine that you have code that requires very high latency
fetches, or you are working with a data set that doesn't fit in memory.
Typically, you have to develop modified algorithms that for these scenarios,
but there is no reason that a minimal fiber scheduler couldn't adapt
seamlessly. Even better, if your conditions change (like for instance, mobile
devices moving from low throughput to high throughput links) a scheduler can
adapt, but the hand rolled algorithm must be re-coded.

------
bunderbunder
Mutable state won't eat your children if you know how to handle it in a
disciplined manner. Pure functions and immutable datatypes are definitely
preferable in a variety of cases. However, just because you discover one day
that there are better tools for driving screws than a hammer doesn't mean you
have to start banging your nails in with a screwdriver and wearing a saffron
robe and preaching to your friends that you have achieved liberation and that
they won't be free until they also throw away their hammers.

The softwareverse (and he universe, for that matter) is still full of patterns
and constructs that are inherently stateful, and need to be constructed such
that they behave as if they were stateful. Heck, some constructs simply can't
be made to work efficiently in a purely functional manner. Hash tables come to
mind. Fortunately we already have a tool that's proven to be excellent for
efficiently and effectively modeling state. No, it's not monads. It's _state_.
And it does the job so well that even Haskell was forced to resort to it for
its hash table implementation.

(Though admittedly not until after much sound and fury to the effect of,
"Nobody needs hash tables; trees are Pure and just as good!" I'll say this for
silver bullets - people sure can cling to them.)

~~~
gnuvince
I don't think anyone says that mutable state should be completely eliminated;
the argument is that it should (1) not be the default, and (2) be made
explicit through the type system.

~~~
jinfiesto
While you're right, I think that's kind of obscured by the language that a lot
of functional evangelists use. I myself prefer FP (Lisp first, now Haskell,)
but most people who are coming from an imperative programming languages tend
to see Functional languages as uselessly pure. As a matter of fact, in LYAH,
the author states that Haskell is totally 100% pure. Which of course, is true
in a sense, but false in another, at least in that we still model and use
mutable state. (It's just marked by the type system, as you mentioned, and
it's done in a functional way and usually heavily sugared.) In Haskell, I'm
disregarding the IO monad.

------
NathanRice
While I agree with some things the author says, his conclusions are bullshit.

Very few processes want to be parallel. People who primarily do parallel
programming think everything needs to be parallel, so they assume everyone
must be going through the pain they are going through. This is false. Most
people write sequential software.

Another issue is that purely functional programming is fundamentally flawed,
because it attempts to eschew state. Many functional programmers will call
this a virtue, but the world is stateful, and being able to reason about
necessary state is how you do anything useful. Purely functional programming
is great when you want a convenient test-bed for ideas, but Haskell as a
practical programming language is a horrible idea. Ultimately, what we want is
to eliminate unnecessary state, while reasoning about necessary state. Modern
programming language researchers are doing this under the guise of "effects
systems", however most of the research I've seen far has left me underwhelmed.

Eventually functional programming researchers are going to end up somewhere
that is quite far from where they started, but just a few small steps from
logic programming.

~~~
fleitz
We actually don't know that the world is stateful. Quantum physics would tend
to suggest that the world is a waveform function that collapses on a discrete
value when evaluated. If you think about it the current 'state' of the world
should entirely depend on how the previous functions collapsed to produce the
current 'state'.

Perhaps like in physics there is a sort of Church-Turing duality to it all
that is hard to tease apart.

~~~
NathanRice
The reality is stateful on a macro-scale. I find the idea that the universe is
actually stateful, and we do not completely understand behavior at quantum
level far more parsimonious than the converse.

~~~
fleitz
The double slit experiment confirms the wavelike nature of the universe at
macro scale. A completely stateful universe where a photon discretely moves
through spacetime could not exhibit self interference, only if the photon
takes all paths (suggesting 'functions' that describe its movement) could it
then collapse into a discrete position after being observed (aka. evaluated)

What I'm getting at is that in between emission and observation the photon is
not in any one state, at best it's state could be described as a
superposition.

To take the analogy one step too far, perhaps worst of all this suggests a
lazily evaluated haskellian universe with a giant state monad whose NUMA
architecture updates at the speed of light.

~~~
NathanRice
I hesitate to admit photons into the macro realm, considering they are
elementary particles and essentially massless.

Given the de Broglie hypothesis, I must of course admit macroscopic objects do
have wavelengths, and thus macroscopic uncertainty (your statelessness) is a
nonzero vlue. For all intents and purposes though, the denominator of the
equation causes the wavelength to go very close to 0, as the mass value dwarfs
the Planck constant and velocity is always non-zero due to brownian motion.

Wave function collapse is interesting and I must admit when it is not induced
by momentum, I don't understand it very well.

------
webjprgm
The article starts by mentioning data ownership, but then diverges after
saying that didn't take hold in D and discusses just FP. Yes, FP works well
for concurrency. But if you're complaining about data races, letting a thread
own a piece of data and only be able to share it via channels solves the
concurrency problem too. Thus imperative languages can live on if they adopt
memory models in which data is owned by a single thread and cannot be shared.
I notice Rust has a mechanism for sharing this kind of data by transferring
ownership. That looks pretty cool to me.

Also, there are still plenty of applications where high level scripting
languages like Python and Ruby work great for application logic, and where
single threaded event-based systems work for handling user events. It's the
data processing part that needs to be put in a safe concurrency sandbox.

~~~
bascule
Can we form a club or something? I keep saying this, but it's still a
relatively unpopular idea.

Kilim (lightweight actor-like concurrency for the JVM) implements what it
describes as "linear ownership transfer", to where objects are owned by a
single thread at a time, eliminating any potential data races.

I'd certainly like to see more of that

------
praptak
I am a big believer in FP (reasoning about functional code is much sounder
even in the sequential case) but not in parallel programming. Most algorithms
just do not parallelize that well, regardless of the programming paradigm
used.

Quote Donald Knuth: _'[...] So why should I be so happy about the future that
hardware vendors promise? They think a magic bullet will come along to make
multicores speed up my kind of work; I think it’s a pipe dream. (No—that’s the
wrong metaphor! "Pipelines" actually work for me, but threads don’t. Maybe the
word I want is "bubble.")'_

 _'I won’t be surprised at all if the whole multithreading idea turns out to
be a flop, worse than the "Itanium" approach that was supposed to be so
terrific—until it turned out that the wished-for compilers were basically
impossible to write.'_

~~~
bunderbunder
If you look at classes of problems, sure, only a few of them are easily
parallelizable.

But if you look at what many programmers are actually doing on a day-to-day
basis, it turns out that a huge chunk of it is really basic, straightforward
SIMD. Hence the popularity of for-each loops in modern programming languages.
That kind of stuff is often quite trivially parallelizable. Microsoft
demonstrated that nicely with the introduction of, e.g., Parallel.ForEach() in
.NET 4.

There's also something to be said for tasks which are naturally. . . let's say
concurrent. ETL comes to mind - if you do ETL using purely sequential code,
you're going to end up with software that's immensely slow compared to code
that has enough basic time management skills to realize it could probably be
doing something more productive than twiddling its thumbs while waiting on the
disk controller.

~~~
_delirium
That's true, but for _those_ tasks, it's not clear you need to abandon
imperative languages at all. In cases where you're just doing the same
operation on all elements in an array, with no data dependency, you can do
that in C without fiddling with pthreads by using OpenMP's "parallel for"
construct.

FP promises to make it less error-prone and practical to do more complex
parallelization than these trivial kinds of parallelization, but the skeptical
take (which I partially share) wonders whether that is a big win. In other
words, are the trivial parallelizations just the 10% tip of the iceberg, and
there are 90% wins left from more complex parallel programming (in which case
perhaps FP is what lets us unlock that future), or is trivially parallelizable
stuff more like 90% of the parallelization win?

~~~
bunderbunder
With the question posed that way, I think I fall on yet another skeptics'
side. Regardless of whether it's the majority of the win or not. Suppose, for
the sake of argument, that the non-trivial parallelization stuff is
complicated enough that it's really not feasible for folks in general to be
tackling it outside safety of what's effectively being billed as a linguistic
walled garden. If that's the case, then the previous sentence ended about 13
words too late. Difficult, error-prone tasks like that should really be
handled by a library or run-time.

------
josephg
This is a cute idea, but your computer is almost never CPU pegged. It spends
most of its time waiting for input from you. For 95% of software, faster
execution (through parallelism or whatever) would have no effect.

But don't take my word for it. Look at all the applications you have open
right now and check if a magical 1e100Hz processor would make any meaningful
difference. For me: iTunes: no. Vim: no. Chrome with hackernews: no. VLC: no.
Terminal: no.

"We need to rethink software design to make our programs run faster" – Hardly.
We're actually seeing the opposite trend. People are caring _less_ about CPU
performance now than they used to. One of the effects is that real software is
being written in slow languages like Ruby and Coffeescript, performance be
damned.

There are obviously exceptions (games and scientific simulations). But for the
other 95% of code out there, we simply don't need the parallelism of
functional languages to write good software.

~~~
j_baker
I think it's stretching it to say that 95% of programs won't peg your CPU. Ok,
maybe 95% of all programs written for an actual desktop/laptop that aren't
games are. But we live in a more complex world where mobile devices are slow,
but have multiple cores. And what about games? Game developers are always
trying to squeeze that last little bit of performance out.

And we haven't even gotten into server-side software, nor have we discussed
concurrency due to I/O-boundedness.

~~~
josephg
On mobile devices, you can already be a bit sloppy with your code and your UI
will still be responsive. The fact that Corona and Phonegap are usable is
proof of that. The 'comfort margin' of mobile CPUs will only increase over the
next few years.

You certainly don't need functional languages to write efficient server-side
code for I/O-bound applications. People have solved the C10k problem for every
imperative language I know.

Ryah (the nodejs guy) makes a pretty strong case that functional programming's
parallelism doesn't help you on the server side anyway, because you'll need to
scale across multiple machines. Multiple machines mean multiple processes. If
you're doing that anyway, you may as well avoid threads entirely and
parallelize by spawning a few processes on each server.

Games are the big exception, though I find it curious that I've never heard of
any published game (big or small) that was written in a purely functional
language. If functional programming really does make concurrent code easier to
write, I'd expect at least a couple indie computer game developers to be using
them, somewhere!

Anyway, despite how visible game development is, it makes up a small
percentage of the computer industry. That we will experience 'The Downfall of
Imperative Programming' because some day, I hope game developers use
functional languages to make their games run faster is a bit of a stretch.

The ability of functional programming languages to do parallelization is
really cool, but you're not solving a problem that many programmers actually
have. Its like selling a car by saying it has lots of leg room for tall
people. Its a great pitch to the right customer, but don't expect a
revolution.

~~~
gaius
Shaders are effectively written FP style. The performance-critical bit of a
game isn't waiting for a joystick, it's rendering on the screen.

------
6ren
Hunch: as hardware (multi-core) improves, we will _not_ achieve a sudden
breakthrough that enables us to run conventional software approaches on it
(functional, imperative or otherwise). Clever people have been anticipating
and investigating multi-core for decades, without result, apart from the
"embarrassingly parallelizable". In later decades, actual multi-core hardware
was adopted in leading-edge then mainstream applications: in supercomputers;
in server farms (esp google); in GPUs. Mainstream desktop CPUs have been
multi-core for a decade now; and, today, as the article indicates, even phones
are multi-core. Phones.

In the big picture, multi-core instead will slowly become good enough to solve
problems _suited to it_ , that couldn't be solved easily or cheaply before,
such as massive simulation in non-government/military applications, and
including problems that we did not even see as "problems" before, they seemed
so insoluble.

It will be a new golden age of computation, with utterly different concepts,
approaches and values from today. Our current issues of programming languages
will disappear, pushed down to a lower level, and compared to it, imperative
and functional programming will be as twins.

------
doktrin
While I do agree with the author that FP has in-arguable benefits when it
comes to optimizing for multithreading, I'm not sure I buy the logic
pronouncing the full-fledged demise of imperative languages.

The notion that dev shops will switch to fully-functional languages because a
junior dev is bound to muck up the code base seems a little far fetched.
That's a _huge_ price to pay for what amounts to a warm fuzzy feeling of
immutable certainty (pun intended).

~~~
siavosh
I don't think it's just a matter of a junior programmer messing things up. To
really maintain concurrent code, you have to fully grasp a lot of pieces in
your head, like what's the correct locking order etc. This is huge mental debt
that needs to be transferred between teammates. And if there's no testing for
it, errors creep in. The largest source of heart burn in my career has been
just this.

That said, I agree with you that imperative languages will be around for a
long time to come.

~~~
bunderbunder
In my experience, code for which the phrase "locking order" is applicable is
frequently overdue for a refactoring.

------
ak39
Great article, but does it answer the question it raised: Why has software
evolution lagged behind?

Also, is the focus on parallel computing the only litmus to judge this as the
downfall of every imperative programming language?

To answer the first question, we have to abandon our technical computing hats
in favour of the philosopher's hat.

What went wrong with the promise of reusable objects? Whatever happened to
4GL? Why are frameworks confined to isolated corners of language evolution,
far-removed from the domain problems faced by end-users?

Why, oh is every nerd developing an Order-Entry system from scratch? Why are
nerds doing the same code again and again for decades in every conceivable new
language (now FP).

You see, if we start accepting structural restrictions like those imposed by
FP, then philosophically we should follow that _maxim_ to its logical
conclusion. Restrict to the point where we do not harm ourselves (by
repeating).

Rant over. ;-)

------
CookWithMe
I think this article is a really bad introduction, both to parallel and
functional programming. There is just so much stuff that is badly informed, or
simply wrong, e.g:

> For a functional programmer there is no barrier to concurrency, parallelism,
> or GPU programming.

Amdahls law? And GPUs, while having branch support recently, still only
perform great for problems that are highly data parallel and are not in high
need for branching.

> I may surprise you that the state of the art in parallel programming is
> OpenMP and OpenCL.

This did in fact surprise me a lot! Erlang? Well, if you think about it, a
webserver renders webpages in parallel. If that is not state of the art and
parallel, then what is?

While I am very interested both in parallel and functional programming, I am
actually disappointed this article has made it onto the frontpage of HN...

------
protomyth
I guess I am kind of confused about the title because I was under the
impression Occam counts as imperative. Also, if we are talking general
parallelization strategies then Grand Central Station is something to look at
on the Mac.

~~~
alexbell
Grand Central Dispatch, I think you mean. :)

It's on iOS 4+ as well.

~~~
protomyth
Thanks, I keep doing that with GCD and calling the "First Responder" First
Listener. I am glad they ported it to iOS. It really helps to be able to use
the same programming structure on both platforms.

I still think one of the agent oriented programming variants might not be a
bad way to build parallel programs.

------
6ren
> In fact the most popular language for parallel and distributed programming
> is Erlang — a functional language.

Erlang achieves its parallelism not through immutability (though it has that)
but through a shared-nothing architecture, like smalltalk. You can implement
shared-nothing concurrency imperatively. It circumvents the problem of two
threads accessing the same memory location by restricting access to only one
thread. Of course, then there's the problem when you do need to share data
between threads - Erlang does this with STM (software transactional memory -
like a DB).

------
16s
I'd like to be able to use C++ in a purely functional manner rather than use
Haskell, etc. Has anyone done that, if so, what was it like?

~~~
hristov
You can if you want to. Just make sure you write all your functions in a
purely functional way, and avoid all side effects.

You do not really need changes to the language, just iron discipline.

~~~
ttt_
>> _just iron discipline_

That's where it gets tricky. It's hard enough dealing with your own discipline
ironing, imagine a dev team composed of varied expertise and awareness levels.

~~~
swalsh
imagine the dev team who takes over the project after you leave. Iron
discipline doesn't exist over long periods of time, that's why it needs to be
built into the language.

------
keypusher
No mention of Erlang?

~~~
tedunangst
The article I read mentioned Erlang.

------
ww520
Another silver bullet to chase after. When will people learn? There ain't no
silver bullet. Parallel programming has complexity. Deal with it.

------
keveman
Should we all take a serious look at functional programming? Absolutely.
Should we start learning a functional programming language? Maybe not. C++ is
a multi-paradigm language in which one can write functional programs, and it
lends itself nicely to switching back to stateful imperative programming when
needed.

~~~
jerf
You can not "switch" to an imperative style in C++. You are _always_ in
imperative land by default in C++. You can "switch" to a functional island in
the middle of C++, but it will forever be a foreign body in the greater C++
program, and a mere moment's inattention will quickly re-introduce imperative
side-effects into your purely functional code. Or you'll use some library that
re-introduces it, or somebody else on your team will use your putatively
functional higher-order function to do something imperative.

C++ is a particularly awful language for trying to manage side-effects in or
maintain a purely functional constraint on your code, because it offers so
very, very many ways of performing side effects without meaning to, and only
the thinnest possible compiler support for maintaining those constraints in
the face of temptation (const). To be sure "a = b.c(d, e)" is a purely
functional statement requires understanding arbitrary amounts of code to be
sure that none of reams of code that can invoke accidentally violates your
constraints.

~~~
fferen
I've only been using C++ for a few months, but it's my opinion that all
function parameters should have been "const &" by default (except for maybe
primitive types). Const correctness has helped me spot some nasty bugs, and
it's too easy to accidentally forget the "&" and pass horrendously large
structures by value, which must go through a copy constructor and destructor
with every call. Plus, it gets really tedious typing "const Mat &src" a
million times (guess which library I'm using, lol).

~~~
octopus
In C++11 "type const &name" can be simply replaced with "type name", no
performance penalty involved.

~~~
ajax77
That's simply not the case.. In general, the "no penalty" holds true for
primitive data types, and when passing an r-value instance of a class that has
a move copy constructor defined. In the case where your object is an
expensively constructed l-value, it is preferable to pass by const reference
rather than value (assuming you plan to use the object later... otherwise you
could indeed pass using std::move semantics).

~~~
octopus
In C++11 you are supposed to use move semantics in your own classes. I'm not
100% sure but I think the standard require that the STL use move semantics.

~~~
cageface
Yes but this is one _more_ thing you have to think about when implementing a
class in C++ and one more chance to screw it up.

It makes vanilla C and raw pointers look a lot more appealing, to be honest.

~~~
octopus
Actually I think C++11 is a simpler language than C++03 or C++98. The problem
is that we need to un-learn what we know and learn the new way of doing
things.

~~~
cageface
C++11 is simpler for the casual user of classes but I'm not so sure it's
easier if you're implementing a lot of classes yourself. I disable assignment
and copying in all my classes by default unless I have a really good reason to
implement them and I still pass by const & for types I define myself.

