I have heard several times that the big payoff with function programming comes from parallel processing. Because functions typically have no side effects they can operate on a set of inputs in parallel without modification. This is important because future increases in processing power are expected to come primarily from more cores rather than higher clock speeds as in the past.
Is this correct? The author seems to focus on other real but lesser benefits.
I've done a bunch of both 'classic' and functional programming, and for me the biggest difference is a switch in code-writing mindset.
In imperative languages, generally, I tell the computer what and how it should do - no matter if it's assembly, C, Java or [most of] Ruby.
In FP languages (Haskell or Scala, haven't worked with Lisps), I generally tell the computer what needs to be computed and expect it to figure out how to do it - what order of execution, what grouping of data, what to cache.
If you compare Java code and Scala code for the same algorithm, using the same JVM API - then the main nonsyntactic difference between them will be a pile of missing Java lines detailing the order of processing steps and loops, which probably weren't essential to the algorithm - when writing the Java code I could've written it in opposite way with the same result.
What I lose in FP is the ability to easily explicitly do time/space tradeoffs. Sometimes I need to control that, and then it's a bit trickier. However, it can be fixed with minor refinement of language and libraries - the original article cites a great example on how memoization should be implemented in all great languages.
Parallelism currently is just a nice bonus - say, I lose 3x performance by not detailing a great execution path manually in C++; but I gain 3x performance since most of the code is trivially parallelizable to run on 4 cores. For example, in Haskell it's often literally a one-line change, while in Java it'd be a pain in the butt to make everything safely threaded.
Parallelism currently is just a nice bonus - say, I lose 3x performance by not detailing a great execution path manually in C++; but I gain 3x performance since most of the code is trivially parallelizable to run on 4 cores.
Parallel extensions have been (and are being) added to OOP languages to make it's use not only simpler, but even as a preferred general design.
In order for such extensions to work on most of my code well, my code needs to written in an FP-inspired style that many new additions to OOP languages do - like C++11, python, etc.
As a crude example - there is a big conceptual difference between a for-loop that processes all the elements in a list or array, and a map operation to all the elements in it.
The for-loop specifies that the execution will be sequential and in a specific order, so it can't really be parallelized automagically.
The map-loop says that it might not be such, so you cannot (a) use a mutable state such as counter incrementation inside; and (b) use the 'previous item' in the calculations.
But in practice, you can write most computations in both of these ways - and if you switch from 'idiomatic C' for-loops to map operations (and the equivalents for the many other common data processing tasks), then code is much more parallelizable both in FP and classic OOP languages.
BUT - it means that much of your code needs to be written in FP-style even if the language is not FP, so you need to think in FP-style. If you use these "parallel extensions" and the new "preferred general design of OOP languages" then much of your code will be stateless and w/o sideeffects; much of your code will look quite different than classic/idiomatic code that the OOP langage had earlier.
Thanks for the explanation. Popular "modern" languages like Javascript, Ruby or Python already have many of the "FP" features mentioned in the OP and comments: first class functions (though not so simple Ruby) and high level data processing functions (e.g. map, filter, select).
As best I can tell the big new benefit of FP is the potential, generally not currently realized, for automatic parallelization. The cost is having to think in a new way that is, at least initially, somewhat confusing.
Alternatively non-FP languages could simply import ideas from FP and gain the benefits in an evolutionary way. For example if Mads wanted to he could rewrite the Ruby interpreter to have map, filter and many other methods automatically use multiple cores. Eventually programmers using the new multi-core version would learn which structures were, such as map, or weren't, suc as for loops, going to be parallelized.
There would be many language specific details to work out. For example determining if there are truly no side effects in a particular situation. This is not always so simple in non-FP languages but still not as complex as analyzing every possible for..end loop.
As 4, 8, 16, 32 and more core processors become more and more common the pressure to either change to FP languages, incorporate FP feature in non-FP languages or find some other parallelization methodology (?) will become irresistible.
I heard this too but in my exploration of functional programming I found little to back this up. Automatically deciding whether it's worth running a given function on a different processor or not (i.e. whether the overhead is greater or less than the savings on the wall clock) is not so much different than the halting problem and many implementations either don't try it or don't do it well. Also, every practical functional program needs to deal with the real world, whether it's in a "pure" (Haskell) form or not, and one way or another this introduces ordering constraints that work against parallelization.
I guess I was thinking that the functional language's compiler/interpreter would deal with spreading the work across the cores when appropriate.
For example if a billion item data structure is feed into some function, say a map that reformats a string, it would be sent to as many cores as were available to simultaneously process a portion of the data then recombine the results (kinda like how Hadoop works).
Obviously this is not trivial but certainly seems easier than either doing this automatically in a language with side effects or manually programming the multi-core logic. None of the languages I've uses can do anything like this but I thought Haskell and some of the new generation of languages were headed that way.
I have heard several times that the big payoff with function programming comes from parallel processing. Because functions typically have no side effects they can operate on a set of inputs in parallel without modification. This is important because future increases in processing power are expected to come primarily from more cores rather than higher clock speeds as in the past.
Is this correct? The author seems to focus on other real but lesser benefits.