Good points, although I agree that a lot of them boil down to similar statements.
I have settled on a style of doing "functional-like" programming in Python and C++. It's more about the high level architecture than low level coding details.
It means being very paranoid and rigorous about state -- but still having great tools to express it! For example: Using ZERO mutable globals, and passing state in as a parameter to functions. Using immutable/persistent data structures. These techniques are done very naturally and effectively in Python and C++. You just have to be disciplined.
To me there's no real advantage to expressing something like "split a string by a delimiter" in a purely functional style. Either way, you have a trivial referentially transparent function you can reuse without causing complexity in your program. You might as well do the obvious imperative thing.
However there IS a benefit to threading state explicitly throughout the application, and encapsulating it in OBJECTS (yes objects).
For me, the thing that sealed the deal against functional languages for "real work" was trying to write a production quality sh parser.
I went down a long path of trying to do this first in OCaml and then in Lisp. The VERY first thing that hits you over the head -- lexing -- is heavily and inherently stateful. ocamllex and ocamlyacc to me are evidence of this paucity and poverty of purely functional solutions. They're just transliterating solutions from C. Well I might as well use C then.
Actually, I decided to use C++, which was like my 5th choice as language. Aside from headers and compile times, it's a good choice. I use "functions and data" (a la Rich Hickey).
Except my functions and data are both CLASSES. Data objects are basically structs, except they can do things like print themselves and answer simple queries based on their values, which helps readability (e.g. 1 liners, like word.AsFuncName() ).
Function objects are simply classes with configuration passed to constructors. That usually have a single method, but multiple methods are also often useful. Calling this method is basically equivalent to calling a curried function. But this is supremely useful for both compilers and servers, because often you have config/params that is constant once you reach main(), and then you have params that vary per request or per file processed. Many functions depend on both, and it's cleaner to separate the two kinds of params.
So both "functions and data" are effectively and usefully implemented as classes. The key is to make some classes like functions, and some classes like data. And have more of a bipartite dependency graph, where functions depend on data, and data depends on functions.
When all your classes are an equal mix of data and behavior, that's when they start getting "hard-coded" weird-shaped dependencies, and your program turns into fragile spaghetti. Functions and data are a useful architectural technique, and to me it doesn't have that much to do with Clojure or Lisp, although Hickey is certainly a great advocate.
Yes I agree. It's MUCH easier to do FP in modern OO languages than OOP in functional languages. So with the OO languages, you're simply in a better position to solve real problems.
Learning OCaml really improved my appreciation of C++. C++ code is just all over the place, but you can write it in a pretty clean style (admittedly with considerable effort.)
I actually haven't ever used SmallTalk, although I understand Ruby is considerably influenced by it (blocks, as far as I understand). Any pointers are appreciated though.
I don't really think of C++ algorithm.h as LINQ ? I guess I don't use it that much. To me that is more "functional in the small", e.g. map or filter with lambdas, vs. "functional in the large". But I'd be interested to hear more details on this analogy.
Yes, that is what I was thinking of. The other part, immutable data and such, isn't that possible to practice in large teams. But I do make use of it in personal projects.
As a matter of style, I still prefer using currying over "one trick pony" (single method/function) classes, though. Sometimes this means I have several "layers" of references to the original function, with varying numbers of parameters applied as each layer further specializes (adds configuration). I'm doing most of this type of work in Javascript, though, so YMMV.
(PS - also did some work with Lisp in Uni back in the 80s, and sundry imperative/oop/func ; static/dynamic things since)
If you're using JavaScript, I think that's actually an advantage because your closures can return multiple named methods (in a JS dictionary). To me, closures in Lisp seem a bit impoverished since it's awkward to return more than one method. (Another problem is that I don't like reifying random local variables as program state. State is important; it should be both minimized and made explicit. Classes make it explicit.)
Classes with one method are useful, but so are classes with multiple methods.
I just stumbled across this post again, and HIGHLY agree with it. Yegge is basically outlining why JavaScript is a better language than Emacs Lisp. I had the same experience with hacking on femtolisp and trying to write my shell in it.
OCaml and Lisp both have problems with polymorphic print. There were some recent posts on HN about the expression problem, and the duality between FP and OOP, which I liked. But I have to say that print should be polymorphic, full stop. And if your functional language has problems with that, that's the language's problem and not my problem :)
Back at CSU in the 80s, the Lisp exposure we had was so so. We learned about pure functions and a tiny bit about higher order functions, but that's about it. Nothing much about closures explicitly.
It's funny that one of the other commenters mentioned Smalltalk and blocks. Version 5 of the "Clipper" language I was using at work, which came out around 1990, added (borrowed, stole) blocks to an "XBase" style language. I didn't quite know what to do with them, so I used them to simulate virtual method tables in a non-OOP language :-)
It wasn't until I started tinkering with Ruby around 2006 or so that all the block / closure / lambda stuff REALLY sunk in ("Hmm. I could have been doing this - subroutines as closures, and not just "blank" call-backs - in all the Perl 5 code I've written the last 8 years"), some 20 years or so after I was in college. They did a good job of brainwashing proto-OOP into us back in the mid 80s.
Never used any Lisp macros, but "eval" and dynamic languages sure help shorten many tasks vs static types bondage and discipline. There's another Yegge essay on that sort of thing ("code compression"). The PC "4GL" type languages I used at work in the 80s were dynamic, with runtime types, and quite productive for the jobs they were designed for (pre MS-Windows...). I'm tired of the static type authoritarians punishing us all for the sins of C/C++. (I need to stop before I go into another rant)
Fair enough, but the appeal of Lisp to me is that it's a small axiomatic core, something that not only fits in your head, but the whole implementation fits in your head too. I wanted to use it to bootstrap languages, with no dependencies.
But it turns out that this axiomatic core is too impoverished for a lot of programming. You DO need something like Common Lisp on top. And I'm not really willing to open that can of worms, both for this specific project, and in general.
I would liken it to the language "Factor", which I've also experimented with... you took a specific and elegant paradigm, and tried to extend it to all paradigms, and ended up with a mess. A square peg in a round hole. Factor is basically Forth with OOP and a whole bunch of other stuff.
The other problem I pointed out is that the first problem you hit when writing a language is writing a lexer. I don't see anything that Lisp offers you in that respect. I looked at how Julia does it:
Julia is actually bootstrapped in femtolisp. If you look at the lexer, it just looks like C code written with Scheme syntax. There's nothing helpful about this. It doesn't help you manage state. I might as well just write C.
(And actually I chose the code generator re2c, which is BETTER than C.)
I don't think your opinion about the axiomatic core matches that of all, or even most, Lisp programmers.
And I'm not saying it's not possible to write lexers in Scheme or Common Lisp. I'm saying that there's no real benefit to doing so over C or even Python. You're using the exact same algorithms and just transliterating it into a different language with more awkward syntax for that problem. The code isn't any shorter.
Related to the other commenter as well, the production quality Julia parser in Lisp doesn't use parser combinators. It uses recursive descent. It's C code written in Lisp syntax.
> I don't think your opinion about the axiomatic core matches that of all, or even most, Lisp programmers.
I have never seen anyone developing software with the 'axiomatic core'. But I see Emacs Lisp, Common Lisp, Scheme, etc. developers.
> And I'm not saying it's not possible to write lexers in Scheme or Common Lisp. I'm saying that there's no real benefit to doing so over C or even Python.
Depends on what level you program. With Lisp it is possible to develop a compact syntax, which expresses domain-level concepts, very easily. Interactive development is many times more convenient than in C.
> You're using the exact same algorithms and just transliterating it into a different language with more awkward syntax for that problem. The code isn't any shorter.
I have a surprise for you: it's perfectly legal to write imperative code in Lisp. Lisp is at its heart a multi-paradigm language with the option to add many other paradigms.
With Python you develop mostly object-oriented and in C it's mostly imperative. With something like Common Lisp you can do what you want.
Yes, you are definitely in the right path but don't you sometimes notice that "get-this-stuff-working-by-tomorrow" productivity is dinged by always having to be doing such deep and ordered design, as well as working out all the other typical programming issues?
I went down an almost identical path as you many years ago but found that after a certain point, the maintenance and just being away from the codebase for a couple of months made the time needed to make coherent mods that fit the design quite problematic.
Yes, that's definitely an issue in general. But in this case, my shell has some unique properties, which make it both necessary and possible to do a good job of software design:
1. Shell is a fixed problem domain. It's not changing, so you can spend time getting the components and interactions right, and know that the rug won't be pulled out from under you. (Though I am planning significant enhancements; it's not a "nostalgia" project).
2. But it is a complex problem domain. What I'm writing is a superset of full fledged POSIX shell, i.e. not a toy. You really do need to control complexity... or you'll end up with something like bash (at least 175 K LOC)
3. It's "systems code". I find that systems code needs to be more highly reliable than application code (or at least the market tolerates less sloppy systems code than app code). So good software architecture matters.
4. It's not a commercial product. I am doing all the design by myself, for better or worse, and the only deadlines are my own.
I think my main thing now is designing with ZERO mutable globals. I've probably been doing this for about 5 years now. Once you get in the habit, and get a few idioms down, I find it makes things go a lot faster. Things break less.
My main issue now is C++ headers and compile times. I started the current codebase with C++, but then switched over to Python to figure out the design. There were a lot of nontrivial issues to solve. Admittedly, implementing everything twice is pretty extreme! But maybe not -- I can write Python so fast that it saves time overall.
But that issue is entirely orthogonal to FP vs OOP. As far as that is concerned, Python and C++ are identical -- they have an imperative history, with significant OOP additions, and support a functional-in-the-small style (lambdas, list comprehensions, etc.). functional-in-the-large is already covered by classes, as mentioned.
Can you give me some pointers to some production quality parsers written with this style? By production quality, I mean "used in practice by a significant number of users".
I've looked at dozens of parsers, both for shells and for other programming languages, and none of them use this technique. I've studied many forms of parsing, and written my own lexing and parsing library/language. This included going on a pretty deep dive into PEGs.
I'm going to be a bit rude and say that parser combinators look appropriate for toy programs only. shell is the opposite of a toy language... it's fairly big, and has a very precise spec with all the hairs and scars of evolution.
I have settled on a style of doing "functional-like" programming in Python and C++. It's more about the high level architecture than low level coding details.
It means being very paranoid and rigorous about state -- but still having great tools to express it! For example: Using ZERO mutable globals, and passing state in as a parameter to functions. Using immutable/persistent data structures. These techniques are done very naturally and effectively in Python and C++. You just have to be disciplined.
To me there's no real advantage to expressing something like "split a string by a delimiter" in a purely functional style. Either way, you have a trivial referentially transparent function you can reuse without causing complexity in your program. You might as well do the obvious imperative thing.
However there IS a benefit to threading state explicitly throughout the application, and encapsulating it in OBJECTS (yes objects).
For me, the thing that sealed the deal against functional languages for "real work" was trying to write a production quality sh parser.
I went down a long path of trying to do this first in OCaml and then in Lisp. The VERY first thing that hits you over the head -- lexing -- is heavily and inherently stateful. ocamllex and ocamlyacc to me are evidence of this paucity and poverty of purely functional solutions. They're just transliterating solutions from C. Well I might as well use C then.
Actually, I decided to use C++, which was like my 5th choice as language. Aside from headers and compile times, it's a good choice. I use "functions and data" (a la Rich Hickey).
Except my functions and data are both CLASSES. Data objects are basically structs, except they can do things like print themselves and answer simple queries based on their values, which helps readability (e.g. 1 liners, like word.AsFuncName() ).
Function objects are simply classes with configuration passed to constructors. That usually have a single method, but multiple methods are also often useful. Calling this method is basically equivalent to calling a curried function. But this is supremely useful for both compilers and servers, because often you have config/params that is constant once you reach main(), and then you have params that vary per request or per file processed. Many functions depend on both, and it's cleaner to separate the two kinds of params.
So both "functions and data" are effectively and usefully implemented as classes. The key is to make some classes like functions, and some classes like data. And have more of a bipartite dependency graph, where functions depend on data, and data depends on functions.
When all your classes are an equal mix of data and behavior, that's when they start getting "hard-coded" weird-shaped dependencies, and your program turns into fragile spaghetti. Functions and data are a useful architectural technique, and to me it doesn't have that much to do with Clojure or Lisp, although Hickey is certainly a great advocate.