He won me at:
> ... nor would I expect anybody to switch if they're happy with their current language of choice. For the uninitiated, however, these are the reasons why I love Clojure:
Nice to see a humble post about the language they like without bashing any other language.
Clojure is the most fun programming I've had in years.
It takes quite some time to really kick in (more than one or two pet projects, rather than couple of "hello world"-style lines it took for Ruby), but once it does… walking up and down the ladder of abstraction never felt so good.
I think the path to "Hello world!" ends up being a bit longer and the buy-in cost a little higher since Clojure isn't really a fitting language for scripts. As soon as you've got an application that requires more than a single namespace, though, the languages' strengths start to come into play.
If Clojure was better suited for scripting, that would amazing. It's one area the language could be a bit better about handling. Setting a quick leiningen project to create a "Hello world!" certainly works and is relatively quick, but that includes a lot of leiningen baggage too.
I have always wondered why, if functional programming is such a good deal in terms of better abstractions, entire classes of bugs eliminated, cleaner code, etc., it doesn't come near-universally recommended by the world's most experienced programmers for mid- to high-level tasks. I'm thinking people like Martin Fowler, Donald Knuth, etc., but especially people like Rob Pike, Russ Cox, Guido van Rossum, Yukihiro Matsumoto, who are all incredibly smart and experienced engineers who have dedicated their lives to developing non-FP languages. There must be some trade-offs to FP that are almost never make it into these kinds of "FP has dramatically improved my life" articles.
(BTW I don't buy the "they're just not used to it" or "they're comfortable with what they know" explanation for this kind of people. These are not stodgy Java programmers who are working in programming as a day-job who are resistant to learning new things, they're people who know more about programming than a hundred average programmers combined and spend nearly every waking minute thinking about how to make it better.)
This is pure argument from authority, but I'm gonna take the bait anyway.
> Martin Fowler
What has this guy done except writing books about "the best way to program" without ever designing a full system himself ?
> Guido van Rossum, Yukihiro Matsumoto
Those guys are just language designers, and the language they designed are just as questionnable as the functional ones, so why do they get a pass actually ? Because their languages are more used ? Do you want to follow suit and say that COBOL creators are probably part of the "world's most experienced programmers" ? What about PHP ?
There are some good points and critiques about the practicality of functional languages, but you don't actually touch on any of them here.
I had a conversation with another semi-famous PL designer. Not one of the ones you list, but only a little below them. He primarily worked on languages the C/C++ family, and had no conception of the value of a closure.
There are lots of smart PL designers working systems languages, and there are lots of smart PL designers working on high-level, functional languages. That doesn't mean that either group is necessarily aware of everything the other group is doing, and it doesn't mean they share the same goals, experience, or taste.
Being smart doesn't make you a good engineer, and being a great language or library designer doesn't necessarily make you an authority on language selection for most engineering uses. Finally, take Pike and Cox and their work on Go. Go is a great minimum-change-for-engineers language for Google's purposes. Is it a great language for most? No. But if you're trying to introduce new concepts to thousands of C++ programmers at the same time, Go is a safer bet than trying to go full Haskell.
To further answer your question, "functional programming" isn't always so well-defined. We know, realistically, that pure functional programming isn't going to work for all use cases. Once you're grounded in FP, you think of mutable state (or, in databases, destructive updates and deletes) as an optimization... but sometimes it's an optimization that you need. No language is FP-only because no language can be; even Haskell has the "dirty" IO monad.
I think that most good programmers (like, 99%) recognize the importance of immutability and referential transparency, when possible, and in the function rather than the action being the standard compositional unit for programs. Where there is disagreement is on when, how, and how often to depart from the functional ideal.
The main issue I have with Clojure is the stack trace ... If you've ever hate yourself when an error in Java popup and you have to trace 30 lines of stack trace in Java, it's far worse in clojure.
I find there's a trick to reading stack traces that actually makes them not so bad. If you focus just on lines that reference your classes, there aren't that many lines to look at. From there, you can usually see the point at which the error occurred and ignore all of the lengthy Clojure runtime lines higher up.
This is a completely fair point, though there are tools to help minimize it and/or to make them easier to read. I'm hoping to release something at least partly ameliorating on the subject next month.
clojure.stacktrace is nice for production debugging from time to time, but it's not solving the problem: by the time the code get to production, it normally has enough log (and is reasonably in good shape) that you can figure out what's wrong. It's the development part that's annoying, especially when you're playing around on the REPL - you're not catching all the error manually, so all you get is the normal stacktrace.
I'm hoping to pick up clojure this year. If nothing else just to have a new way of thinking about code - both from a functional perspective and a lisp perspective. I seem to find a lot of these articles - the only bad press i've seen about clojure is from other lisp'rs that think clojure isn't good enough.
I do think it is interesting though that so many people love to extoll the virtues of immutability and yet have such a hard time putting those virtues into words.
I had to think quite hard about how to express the value of immutability in this; did I communicate the point successfully? I'd love feedback on that part of the post.
You did well, but it could be improved by giving an example of how mutability burned you in the past when trying to debug, or how functions are much easier to reason about, or even how significant state mocking code simply isn't needed (as I have heard is necessary in Ruby).
In my own experience, I once built an OO Ruby application that interacted with an internal database to heavily process a CSV file. Dealing with the state of instance variables made debugging maddening. I won't say it had an excellent design, but dealing with shared state made things much worse. If it had been functional and immutable, I wouldn't have had to deal with many bugs I encountered.
I love reading articles like this about Clojure -- each one demystifies some part of the transition from another, more imperative language. (Here, for me, it was how to do unittesting in Clojure coming from Python.)
Something the author touched on is the use of the REPL, in vim, to attach to remote running processes and debug. Does anybody have a quick example of how to do this?
The start-server command in tools.nrepl https://github.com/clojure/tools.nrepl Will open an nREPL (networked REPL) server on some port of your choosing, by default on localhost. Configure your application to allow this to happen on startup.
(having the nREPL port only listen on localhost and then using SSH to access it is somewhat important for security. Someone else who connects to the nREPL port can run arbitrary code with your app's permissions, see any data being processed by your app, etc.)
I installed fireplace.vim when I set up Clojure for the first time (maybe 4, 5 years ago), but didn't really understand much else of Clojure at that point. I'll try it out again, thanks for the tip!
Clojure is fun but starting a Clojure program that just prints hello world takes over 1 second, even worse on a raspberry pi. I suppose command line tools are not its main purpose.
You can make command line tools in Clojure, but in general I'd agree that you'd be better served by a language that can either compile and run faster [Golang] or is interpreted [Python, Ruby, etc.]. There's no real getting away from the JVM start-up cost, sadly.
As a tip to anyone who wants to give Clojure a try, be sure to grab something for your IDE to handle the brackets. For Sublime, I would recommend Paredit. Your sanity will thank you.
For those who don't know that's the thread macro and IMHO it should be a bigger selling point of the language. I think of it like the Unix pipe operator (i.e. "ls | grep X") but for programming.
(-> g ;; input value
(partial a) ;; runs as (partial g a), hands off A
(comp f) ;; runs as (comp A f), hands off B
(map x)) ;; runs as (map B x), this returns a value
or
(-> 5
(< 10)) ;; runs as (< 5 10)
=> true
with a little tom-foolery (lambda functions) you can have the incoming value land anywhere in the function's argument list
heh i am always nervous to use this macro... while i agree it often makes things much simpler & more linear i feel like working without it would probably lead to quicker learning of how to mentally organize/dissect the normal syntax
Given your experiences, would you recommend picking up Python or Clojure for data processing/data science?
I'm currently quite deep into Javascript/Node, and am already comfortable with functional programming. Next I'm looking to gain more data abilities, and had assumed Python would be the natural next step. You're making me wonder whether Clojure would be a smarter step, though I am concerned that most of the jobs still appear to be in Python.
If I was starting a company - Clojure. If I was looking for employment - Python. Clojure lends itself very well to data processing and data science, but the data science ecosystem still isn't nearly as strong as Python's is.
And if you want to mix them, you can try hy (http://docs.hylang.org/en/latest/): a lisp that compiles to the Python AST, allowing you to reuse every Python module under the sun.
Okay, serious question: What is an example of a mid-sized project to really begin hacking on Clojure? Simple programs utilizing multiple namespaces for trivial reasons are great and all, but if I want to really understand what Clojure can do for me in a few weeks, what should I be trying to build?
My first Clojure app fetched large gobs of interconnected data from a graph database (Neo4j). That data needed to be sliced into many different shapes. Clojure and its many standard functions made the task surprisingly easy and fun.
My first Clojure project was a simple chat server with multiple rooms. Built in a couple of days, and convinced me that it is indeed a powerful language.
Do me a favor, please do not write your configuration files in Clojure as well.
As a non-Clojure developer, having to fight with Clojure configuration files for Riemann was a real pain point, and ended up deciding us against using Riemann for anything as a result (trying to convince a team of sysadmins that they would have to learn Clojure for a single piece of their monitoring infrastructure... didn't work).
IMHO, you're welcome to enjoy your language to the fullest extent possible, I would just ask that you don't force others to use it as well.
EDIT: Since it was asked twice in about 30 seconds, here's why having Clojure as a config option can be bad; it encourages things like this:
Please note that this is considered to be the gold standard by Riemann; it's directly referenced in their documentation.
As for my preferred config file format? ini, yaml, json, in that order. They're simple, there are lots of tools out there to read and write them, and friendly to humans.
>Do me a favor, please do not write your configuration files in Clojure as well.
Code is data and data is code. For Lisp programs, configuration is just another Lisp program. I don't want my Lisp programs crippled by a "dumb" config file.
If your configuration file is a Lisp file, then it's not a configuration file. It's a distributed portion of your program which is wide open to being destroyed by a sysadmin. "What do you mean I took down the site by adding a TLS parameter to the config file? How could a config file overwrite a function symbol in your code?!"
What a config file should be is the simplest, most easy to read and fill out configuration file you can possibly think of. It should be tolerant of simple formatting mistakes and be targeted at someone who is not as smart as you are.
It should also never contain executable code, because config files are rarely as well protected on a system as executable code is, and this could lead to some really nasty vulnerabilities if it's exploited.
This is the case with all config files, though, regardless how simple. The incident where Google accidentally blacklisted the entire Internet [1] was because of a one-character typo in a config file.
The lesson you should learn from this is not to never include powerful statements or interpreted strings in config files, it's that you should test your configs as if they were code, because mistakes in them are just as dire. Even if you limit yourself to key/values in a .properties file, you can still bring down the site if you typo a filesystem path or database name.
The most common way to do this in Clojure is to use an EDN file. EDN is similar to Clojure syntax but the EDN reader (intentionally) has no arbitrary runtime execution functionality - it just reads a data structure and returns it.
This is true. It does not follow, however, that the concept of code and data is meaningless. Ask anyone who's had to deal with user data unserializing into executable code whether the truth you speak is a blessing or a curse.
> I don't want my Lisp programs crippled by a "dumb" config file.
Why should a config file not be "dumb?"
The purpose of a config file is to store state. Why should that be Turing complete? What purpose would it serve?
I understand the temptation to add evaluation and expressions to everything, but beyond a certain complexity, you no longer have a config file, you're just exposing variables from an application within an application.
I think it really depends on the context and tools. I've come to believe, for instance, that trying to do logging configuration any way other than just using a standard log4j.properties file is a path to madness; at the same time, Lein profiles are a potent and idiomatic way to do certain types of environment configuration.
The biggest argument in favor of more friendly config files for me is that they're easier to toss in a jar, though. Nobody wants to fight against configuration in their production environment.
Although, slightly outside of this scope, I cannot say enough good things about YAML. For example, we use it for parameterized test cases that must be passed in code written in various languages. i.e. Python server function does the same thing as client JavaScript function.
As for not using JSON, I prefer YAML b/c often times we like to put comments in our test cases. This is really just no doable in JSON (outside of some ugly hacks. e.g. repeated keys (last one wins)).
The problem with using something like YAML to configure (if you even want to call it that) Riemann is that to do anything useful, you're going to end up inventing some convoluted "language", anyways. I'd much rather spend some time learning Clojure which I can take elsewhere than a custom YAML DSL.
Yeah. After digging into it, I figured out that Riemann is more of a framework for building a monitoring tool than a monitoring tool itself.
It provided powerful abstractions handling streams of monitoring data, but it was too unwieldy to be considered unless you started with Riemann and built out your environment on top of it.
I love Clojure. Why? Partly because I am lazy. Larry Wall said a good programmer is lazy, but in my case I am lazy in the wrong ways. If you have enough self-discipline, then you probably don't need immutability. But immutability is a wonderful thing if you are lazy.
I also love immutability because I have had many co-workers who are lazy. Again, not aways the good kind of lazy. I mean the kind of lazy that allows 2 loops with mutable variables into the same function:
howMuchPrizeMoney = 0;
arrayOfMoneyPerCategory =[];
for (i=0; i < users.length; i++) {
u = users[i];
howMuchPrizeMoney += u.prize_money;
}
for (i=0; i < contests.length; i++) {
c = contests[i];
for (j=0; j < categories.length; j++) {
cat = categories[i];
if c.name == cat.name {
arrayOfMoneyPerCategory[cat.name] = u.prize_money;
}
}
}
Wait, now I have a bug! All the categories have the same amount of money, and I know that is wrong. Where is that bug? Hmm, let me look and look and look. And if this example seems easy, I have seen functions larger than this, where finding the bug gets much harder. I am lazy, and my co-workers are lazy, and if we allow ourselves mutable variables, we will abuse them, so I would be happy if we adopted a language that avoided mutable variables. So for instance, even if I had a loop that looped over all the variables, there would still be no risk of one variable effecting another if I did something like this:
Now if I do this, and I stupidly put next a loop inside of a loop, is there any risk of a typo? Sure, but I won't get a confusing number back, instead I'll be told that I'm using a var that doesn't exist.
But what if I made a typo, just like in the first example? What if instead of this:
(first c)
I stupidly wrote:
(first u)
There would be no "u" that was in scope for the whole of the function. The "u" would only exist inside the first (loop).
Needless to say, no one would ever write Clojure code like this. Anyone who puts 3 loops in function, in Clojure, is taken outside and shot at point blank range. But my first example, in Javascript, is something I have seen in real life.
Clojure is definitely my favorite dynamically typed language. I also think that its community has a world-class aesthetic sense that other languages (looking at you, Haskell) could stand to learn from. Take maps as an example: Clojure has {"one" 1, "five" 5} and Haskell has fromList [("one", 1), ("five", 5)]. Clojure's syntax is a lot more attractive. Clojure has hyphens and Haskell has camelCase. Again, Clojure wins that one. Also, Leiningen is probably the best build tool I've ever used. I also think that Clojure has a great UX sensibility: once you get past That One Thing (parentheses, which are not as bad as they're made out to be) it's more readable than Scala and arguably Haskell. You can also make Clojure blazingly fast for a dynlang, although with some loss of aesthetics due to type hinting and array bashing.
There are two issues with Clojure (3 if you consider static typing a hard requirement, because core.typed is brilliant but probably not "there" yet) and both come from the JVM. The debugging experience is still pretty bad; although I dislike debugging in general if it can be avoided, and that's why I've gravitated toward static typing (catch bugs early). The second is that the JVM itself imposes a ceiling on how well you can do performance-wise, and can require a fair amount of configuration in production. Of course, Clojure may be off the JVM in 10 years and, even if not, it's still a great language in very many ways.
> Leiningen is probably the best build tool I've ever used
Leiningen is like an Apple product: it's well-designed and usually 'just works'. But when something goes wrong you realize it's like a black box with only a mystical stack trace to help you.
Isn't Clojure on CLR too? I would think that platform might suit it a bit better, and might get better performance in the long run (the case I can think of off the bat is the possibility of tail-call-optimization).
are there a lot of applications where this matters? maybe for CLI utils or something?
i mostly make web-apps or data-mining apps, never really understood this criticism of long (1 second type of long, maybe 10 for EE server) startup times for jvm since i can't imagine many apps worth putting a ton of development care into that aren't worth a 1 second wait to run.
i'm not trying to be a smart-ass (i dont believe everything should be blindly forced onto jvm), really just asking why this is such a common concern when in my mind it is an edge-case
It matters quite a bit for CLI utils. More than a tenth of a second is not "instantaneous" - which isn't required, and of course won't always be possible, but starting with substantially more than that isn't great. And of course, if you're using that utility in a loop some pipeline, that could even wind up being a bottleneck.
For any single long-running application that doesn't spawn more processes, of course it doesn't matter - but most performance issues only matter for some set of applications.
I think it's such a common concern because 1) people often start out writing CLI utils as they learn their way around a language, 2) people would love to see Clojure be better for this use case, and 3) it's one of the few things to complain about.
thanks -- i didnt realize #1 was the case, & good point about the pipeline loop... I guess since I just use the CLI utils in different ecosystems it doesn't really occur to me how they are written, nor do I really care.
But I understand on principle, for example I work on JVM & hate when ppl bring in random code that make it hard to test across the whole system. Seamless integration of all tools & unified protocols are nice.
Nice to see a humble post about the language they like without bashing any other language.