EDIT: I wrote this comment before watching the video. I stand by this comment, but the video is very good and I wholeheartedly agree with its conclusions.
As someone who writes pure FP for a living at a rather large and well known org, these threads physically hurt me. They're consistently full of bad takes from people who don't like FP, or haven't written a lick of it. Subsequently, you get judgements that are chock full of misconceptions of what FP actually is, and the pros and cons outsiders believe about FP are completely different from its practitioners. It's always some whinge about FP not mapping "to the metal", which is comical given say, Rust's derivation from what is quite functional stock.
My personal belief? We just don't teach it. Unis these days start with Python, so a lot of student's first exposure to programming is a multi-paradigm language that can't really support the higher forms of FP techniques. Sure, there may be a course that covers Haskell or a Lisp, but the majority of the teaching is conducted in C, C++, Java or Python. Grads come out with a 4 year headstart on a non-FP paradigm, why would orgs use languages and techniques that they're going to have to train new grads with from scratch?
And training people in FP is bloody time consuming. I've recorded up to 5 hours of lecture content for devs internally teaching functional Scala, which took quadruple the time to write and revise, plus the many hours in 1-on-1 contact teaching Scala and Haskell. Not a lot of people have dealt with these concepts before, and you really have to start from scratch.
What a coincidence. This sounds exactly like what happens with OOP. Every discussion here gets swarmed with clueless people who think Java is the apex of OO programming, because that's what gets taught in universities these days. They don't understand any of the core concepts that prompted Xerox Park to develop the OO paradigm in the first place. They aren't aware of any of the relevant research. They operate under the assumption that OO was born out complete ignorance of functional programming, even though people who kick-started its rise were keenly aware of Lisp (for example, Alan Kay frequently references McCarthy's work and research papers in his talks). Etc, etc.
Good OOP is good FP - they swim in the same water.
composition, generics/adt's, elegance, encapsulation...
In it's ideal form it's about taming the metal like a trick pony.
I'm nowhere near that level, being a fortran in any language sorta guy lol, but when I see well-built stuff, I take notes. Matryushka dolls, lol
Kay et al were swimming in the same water as Hewitt etc, conceptually. He said that the key takeaway from OOP was objects passing messages (actor model), not so much the inheritance story (the composition)
I think the Smalltalk paradigm is deeply defective and the Actor model (the purest form of OOP to my mind) remedies most of its faults but perpetuates some others. A few flaws:
- Modeling all communication as synchronous message-passing. Some communication (such as evaluating mathematical functions) is naturally modeled as synchronous procedure calls, while communication which is naturally modeled as message-passing should be asynchronous by default (to address unpredictable latency, partial failure, etc.).
- Emphasizing implementation inheritance as the primary means of code reuse. This is now generally acknowledged to be a mistake, so I won't elaborate.
- Deferring all method resolution to runtime. This makes the amazing introspective and dynamic capabilities of Smalltalk possible, but it also makes it impossible to statically verify programs for type-correctness.
- Relying on mutable local state rather than explicit, externalized state. This is controversial, and it's a defect of the Actor model as well (yes, passing new parameters into a tail-recursive message receive loop is equivalent to mutating local state). The partisans of OOP and the Actor model believe this to be a virtue, enabling robust emergent collective behavior from small autonomous software agents, but it makes predicting large-scale behavior difficult and debugging nearly impossible.
There was an article on state in OOP posted here a few days ago that I found very thought-provoking[1]. The blog post and related youtube videos are pretty interesting as well[2][3][4].
All of this has been addressed zillion times. Modern Smalltalk dialects have actor libraries and have code reuse mechanisms that don't involve inheritance. There are ways of doing static analysis on late-bound code. (Obviously, guarantees are not going to be the same. I take that as a reasonable trade-off.) OOP isn't predicated on mutable state and there are ways for the system to manage it anyway. (Although, to be fair - that is one thing from the list that hasn't been fully addressed in any practical OOP system I'm aware of.)
Even if none of the work I mentioned above existed, this sort of criticism is amateurish at best. Real engineering requires considering trade-offs in real-life contexts. For example, compile-time checks aren't going to help you figure out that some vendor supplies incorrect data via a web service. An early prototype, however, can do exactly that.
> Every discussion here gets swarmed with clueless people who think Java is the apex of OO programming, because that's what gets taught in universities these days.
Realistically, Java (or something very much like it) is the apex of OOP, at least as most people will experience it. The Ur-example of OOP might be a beautiful, internally consistent vision of mathematical purity, but most of us will never experience it.
Similarly, Agile-fall is the most common form of Agile that people will experience, which is why we always fall into "no true Scotsman" territory when ~~arguing about~~ discussing it.
There is, I think, a disconnect between people who are primarily concerned with the beauty of software - simple models, elegant algorithms, and so on - and the people who are primarily concerned with getting their feature branch merged to master so their manager will let them go to their kid's soccer game.
The beauty of software is important, and there's value in trying to bring the useful, but more esoteric concepts of CS into the mainstream, but at the same time we need to be aware of the ground truth of software development.
>Realistically, Java (or something very much like it) is the apex of OOP [...]
By this logic Java Streams are the apex of functional programming and anyone who uses them is fully qualified to dismiss the paradigm, even if they don't know anything about proper functional languages.
This makes me appreciate working somewhere that feels kids' soccer games will always take precedence over merging features. I also use hybrid actors/oop extensively. I'd never really considered that these probably go hand in hand.
The thing about these discussions is they seem to have two different questions mixed together. One question is "what's the best way to produce good software in a circumstance where everyone start on top of their game and the right thing" and the other is "what's a sort-of natural, slightly better average, way that programming can be." The answer to the first can be "Good functional Programming" or "Good OOP" or maybe something even more exotic. But it doesn't matter that much for the second question. Because usually the question of "how can this code we have already be better" mean "how do you take a general mess and make salvagable?" I don't know what the FP's answer to this is. I generally get the feel the FP people don't want to have an answer to this because then there'd be a lot of better-but-still-bad installations out there. But there are plenty answers to improvement using OOP - most say just encapsulate everything or similar things. All sorts of things can be thinly encapsulated, better, but still far from good. That seems to me explain the prevalence of OOP.
>My personal belief? We just don't teach it.[...] Grads come out with a 4 year headstart on a non-FP paradigm,
I don't agree the lack of proactive education is the reason FP isn't the norm. Your conclusion doesn't take into account the counterfactuals:
- C Language took off in popularity despite BASIC/Pascal being the language more often taught in schools
- languages like PHP/Javascript/Python/Java all became popular even though prestigious schools like MIT were teaching Scheme/Lisp (before switching to Python in 2009).
You don't need school curricula to evangelize programming paradigms because history shows they weren't necessarily the trendsetters anyway.
On a related note, consider that programmers are using Git DVCS even though universities don't have formal classes on Git or distributed-version-control. How would Git's usage spread with everybody adopting it be possible if universities aren't teaching it? Indeed, new college grads often lament that schools didn't teach them real-world coding practices such as git commands.
Why does Functional Programming in particular need to be taught in schools for it to become a norm but all the other various programming topics do not?
> Why does functional programming need to be taught in schools but all the other various programming topics did not?
Because I think it is harder for people who have programmed with other paradigms - following an inverse law, most things should get easier to learn with experience, not harder. It's foreign, it's weird, it's back to front. It doesn't have an immediately obvious benefit to what people are used to, and the benefits it has come at scale and up against the wall of complexity (in my opinion). It's hard to adopt upfront. At the small scale it's often painful to use. The syntax is weird. It's aggressively polymorphic, reasoning in abstractions rather than concretions. I could go on (and yet I still adore it).
The only reason FP has been successful as it is, is because its evangelists are incredibly vocal, to the point of being fucking annoying sometimes. It's had to be forced down people's throats at times, and frankly, there's no better place to force a paradigm down someone's throats than at a university, where non-compliance comes at academic penalty, and when the mind is most impressionable.
>Because I think it is harder for people who have programmed with other paradigms
I still think there's something missing in your theory of cause-&-effect. A math topic like quaternions is hard and yet programmers in domains like 3d graphics and games have embraced it more than FP.
I also think Deep Learning / Machine Learning / Artificial Intelligence is even more difficult than Functional Programming and it seems like Deep Learning (e.g. Tensorflow, Pytorch, etc) will spread throughout the computer industry much more than FP. Just because the topic is hard can't be the defining reason.
>The only reason FP has been successful as it is, is because its evangelists are incredibly vocal,
But why is FP in particular only successful because of loud evangelists? Why can't FP's benefits be obvious so that it doesn't require evangelists? Hypothetical example:
- Company X's software using FP techniques is 10x smaller code base, 10x less bugs, and 10x faster feature development -- than Company Y. Ergo, this is why Company X is worth $10 billion while Company Y is only worth $1 billion or bankrupt.
If you think the above would be an unrealistic and therefore unfair comparison, keep in mind the above productivity improvement happened with the industry transition from assembly language to C Language. (A well-known example being 1980s WordPerfect being written in pure assembly language while MS Word was written in C Language. MS Word was iterating faster. WordPerfect eventually saw how assembly was holding them back and finally migrated to C but it was too late.) Yes, there's still some assembly language programming but it's niche and overshadowed in use by higher-level languages like C/C++.
If Functional Programming isn't demonstrating a similar real world massive productivity improvement to Imperative Programming, why is that? I don't think it's college classes. (Again, see all the non-PhD enthusiasts jumping on the free FastAI classes and brushing up on Linear Algebra to teach themselves deep learning.)
> Why can't FP's benefits be obvious so that it doesn't require evangelists?
Because there aren't immediate benefits. They only pop out at scale and with complexity, as I said.
> similar real world massive productivity improvement to Imperative Programming
Because there isn't. It's a reasonable benefit, but it's not transformative. I think it's there, enough to commit to FP completely, but the massive productivity improvement doesn't exist, or at least, only exists in specific cases, e.g. the WhatsApp + Erlang + 50 engineers parable (you could argue that this is due to the actor model and BEAM, rather than FP. An argument for a different day).
I feel like this hard + reasonable benefit isn't really efficient utilisation of people's time, especially when there's things like Deep Learning floating around. I think the immediate reaction to a lot of what FP evangelists claim is a shrug and a "I guess, but why bother?"
Exactly. As just an anecdote, my intro do FP class in university was taught by a professor who tended to rant about different levels of purity and elegance between his favorite and least favorite languages. Of course, the favorite was his pet project and we had to spend most of the class using it. I also know that Emacs is partly written in Lisp because it was the only editor he would touch.
FP can't even sell itself well in school as a language where useful things can be done, when the student is stuck in a deep valley of ___morphisms and other alien concepts with claims of aesthetic elegance as the only motivation. I recall the math nerds loved it as relief over C that the rest of the department used, but with me being rather mediocre in math, the weirdness and cult-like vibe from the prof and TA left a really bad taste. The impression was so deep that I have no issues recalling this class a decade later. I've never touched any FP since, unless you count borrowing clever lambda snippets.
The sad part is that this is a common experience - universities have done a bad job at teaching FP. I think there are good pieces of FP education, particularly Learn You a Haskell and https://github.com/data61/fp-course - friends have gone through these have questioned "why wasn't I taught like this the first time around".
> I've never touched any FP since, unless you count borrowing clever lambda snippets.
I'd urge you to give it another shot if you have spare time. Even in spite of all the dogshit things associated with it, it's a paradigm I've bet my career on.
>It's had to be forced down people's throats at times, and frankly, there's no better place to force a paradigm down someone's throats than at a university, where non-compliance comes at academic penalty, and when the mind is most impressionable.
That's also a great way to make people hate it. An example is literature classes with mandatory reading and how they make students hate reading fiction.
I would also say that this might turn off more students from programming. We had functional programming in uni, where we learned Haskell. Maybe a handful of students liked it or were neutral about it, the vast majority seemed to have a negative view of it.
I think that FP is just more difficult to learn. Just look at essentially any entry level programming course and how people understand loops vs recursion.
Okay, so FP is more difficult to learn. Assume for the sake of this argument that FP has a tangible benefit over other paradigms, that manifest themselves at scale. You're tasked with educating students in this paradigm, but they complain that it is more difficult than the techniques that they are used to.
I don't know, because I'm not qualified for it. I had to pass a course on FP, but frankly, I wouldn't be able to do anything with it in practice, let alone teach it. My only personal experiences with it were negative. If it had been Haskell that was the entry level programming course, then I probably would never have learned to program.
Okay, so given this answer here's what I would do:
1) I wouldn't make it the entry level course. It's clearly a paradigm that's used by a minority of people, so it doesn't make sense to start educating students with it.
2) I mandate that all students take it, maybe in their 3rd year. We're going to mandate it because there are tangible benefits (which we've assumed for the sake of this argument). They're going to find it harder and more confusing because its's different to what they're used to. A lot of them may not like it and won't see immediate benefits. Some may even come to dislike it. Frankly, I don't care, some will pick it up and learn about it further. And when the students that disliked it inevitably run it into the future, they sufficiently prepared to deal with it.
We're back to square 1: forcing it down student's throats. If you still think that we shouldn't be forcing students to learn FP in schools, I think you have a problem not with FP but with structured curriculums.
Spark is the quintessential Google-scale FP project - it was even born out of the MapReduce paper by Google!
And there's plenty of other large-scale projects that are arguably in an FP style specifically to deal with the problems associated with scaling them: the Agda/Isabelle proof checkers, the seL4 kernel, the Chisel/FIIRTL project, Erlang/OTP, the Facebook Anti-Spam system (built on Haxl), Jane Street's massive investment into OCaml, Twitter's investment into Scala.
Not all scale problems are distributed problems. Some distributed problems are tackled by FP, and some aren't. Ultimately, these large-scale projects pop up at similar rates to the usage of the language themselves. It's intellectually dishonest to say that FP can't be used to tackle large scale problems, and the problems that occur at scale, because its repeatedly validated that it can.
It is also intellectually dishonest to strawman an argument.
I didn’t say it is impossible to do X with FP - I said it is not necessary to do X in FP.
You can convince yourself of that by looking for larger-scale non-FP counter-examples to the ones you've cherry-picked.
Every single large scale problem is a distributed problem simply because human societies are multi-agent distributed systems and programming languages are human-computer interfaces.
The issues at scale are systemic and arise from the design trade-offs made when your system's requirements bumps against the limits of computation. No language/paradigm can work around those.
The best a language can do is abstract-away the complexities behind a problem - solve it once (in the language/paradigm's preferred way) and give the human an interface/concept to work with.
Okay, I got really confused by this whole mandatory thing. I never said FP should be mandatory at scale. I said it had a moderate benefit at scale. You respond with "well actually, it's not mandatory at Google-scale" so I assumed that you were trying to refute the fact that FP has benefit at scale.
You also followed this up with
> The kind of problems that emerge at scale are not the kind of problems FP tackles.
I cherry picked these examples to demonstrate that you're completely talking out of your ass here.
> I didn’t say it is impossible to do X with FP - I said it is not necessary to do X in FP. You can convince yourself of that by looking for larger-scale non-FP counter-examples to the ones you've cherry-picked.
I never said it wasn't possible to tackle these problems without FP.
You need to get rid or the assumption that "if X is better than Y at task Z, everyone will use X rather than Y for task Z". You've used that line of logic to attempt to invalidate FP's capabilities. It simply does not make sense.
You give them difficult real-world problems where FP is helpful.
But university computer science seems to be specialized from mathematics instead of generalized from engineering, so CS professors most of the time have no idea about real world problems. At least here in Germany, where the problem seems especially bad.
>Subsequently, you get judgements that are chock full of misconceptions of what FP actually is
I put the blame for that squarely on the Haskell cultists. They've caused everyone to have the impression that functional programming needs to have esoteric syntax and lazy evaluation by default.
I think you nailed it. But FP proponents have done a fair bit of harm to the paradigm itself in that some of the more osutspoken proponents have been pretty alienating. They've pushed the language superiority argument to death. They should probably be focused on pushing the style in the hopes that the next round of popular languages begins to implement FP style as first class features. Which is of course what the author alludes to and I think is actually happening. Which would jive with history.
A lot of what became OO language features arose because people were already using the style in non-OO languages. C being a great example of how you can use a ton of C++ like features, you best end up writing a lot of boiler plate to hook up function pointers and the like.
Going back further we see features in C were being implemented by assembly prigrammers as macro assembly. So the pattern the author puts forward has basically held true for multiple shifts in programming paradigms.
Which leaves me with one point of contention with the presenter. That OO dominance is not happenstance. And neither was the fact that lots of people were writing OO style C. There is something about OOP that helped people think about their code easier. That's why they adopted the features. Maybe not everything about it was great and we're learning that. But it genuinely helped people. Just as English language oriented languages helped people over ASM.
Personally, I get frustrated that there seems to be a belief that you can only use FP or OOP, when the reality is both models can be used in conjunction, and there may be reasons to choose one over the other dependent on what you are doing. Not to mention there are other models such as Protocol Oriented Programming. You see this in languages like Swift.
The issues is that you get benefits for sticking to one paradigm, because then everything is made of the same stuff.
If everything is an object, then you can use all your tooling that works with objects, which is everything. If everything is pure, you get easy parallelism. If everything is an actor, you get easy distributability. If everything is a monad or function, you get easy compositionality. The list goes on. Smalltalk, Erlang and Haskell are languages with very dedicated fan bases, which I theorise is because they went all in on their chosen paradigm.
You don't really get easy parallelism, do you? People used to theorise this, but auto-parallelisation never really worked because it's hard to understand - even for a machine - where the cost/benefit tradeoff of parallelism starts to happen. Applying a pure function to a massive list is simply not a common pattern outside of things like multimedia codecs or 3D graphics, where these industries settled on explicitly expressing the data parallelism using SIMD or shaders a long time ago. Functional languages have nothing to add there.
It's true there are benefits to working with in one paradigm, but I find that often you can only do this for so long. This is why you have things like redux-thunk.
As someone who writes pure FP for a living at a rather large and well known org, these threads physically hurt me. They're consistently full of bad takes from people who don't like FP, or haven't written a lick of it. Subsequently, you get judgements that are chock full of misconceptions of what FP actually is, and the pros and cons outsiders believe about FP are completely different from its practitioners. It's always some whinge about FP not mapping "to the metal", which is comical given say, Rust's derivation from what is quite functional stock.
My personal belief? We just don't teach it. Unis these days start with Python, so a lot of student's first exposure to programming is a multi-paradigm language that can't really support the higher forms of FP techniques. Sure, there may be a course that covers Haskell or a Lisp, but the majority of the teaching is conducted in C, C++, Java or Python. Grads come out with a 4 year headstart on a non-FP paradigm, why would orgs use languages and techniques that they're going to have to train new grads with from scratch?
And training people in FP is bloody time consuming. I've recorded up to 5 hours of lecture content for devs internally teaching functional Scala, which took quadruple the time to write and revise, plus the many hours in 1-on-1 contact teaching Scala and Haskell. Not a lot of people have dealt with these concepts before, and you really have to start from scratch.