
Why is Clojure so slow? - Mitt
http://martinsprogrammingblog.blogspot.de/2012/02/why-is-clojure-so-slow.html
======
lukev
Clojure has a slow _startup time_. It is not in itself slow.

It's important to get these distinctions right! Some people, unfortunately,
form impressions from headlines and in this case it's extremely inaccurate. If
you read the article you'll get the truth - there are startup time problems
causing issues with using Clojure for command-line programs.

That's much less damning overall than "Clojure is slow."

~~~
debacle
You should have read the entire article:

> How fast is Clojure at running your code once it finally has got going? ...
> Clojure is on average 4x slower than Java and 2x slower than Scala.

That's pretty freaking slow.

~~~
lukev
And about twice as fast as Erlang, 5x faster than Ruby, Python or PHP and 10x
faster than Perl.[1]

It's all relative.

[1] [http://shootout.alioth.debian.org/u64q/which-programming-
lan...](http://shootout.alioth.debian.org/u64q/which-programming-languages-
are-fastest.php)

~~~
gizzlon
While interesting, those benchmarks are only useful if you implement something
remotely similar to the actual benchmarks. Just glazing over them, they seem
to be "unfairly" targeted at low-level languages.

That said, they are fun to look at. I just had a wtf-moment looking at this:

[http://shootout.alioth.debian.org/u32/performance.php?test=n...](http://shootout.alioth.debian.org/u32/performance.php?test=nbody)

~20 seconds vs ~20 minutes??

~~~
Jach
This sums up my feelings for the benchmarks:

[http://shootout.alioth.debian.org/u64q/benchmark.php?test=fa...](http://shootout.alioth.debian.org/u64q/benchmark.php?test=fannkuchredux&lang=all)

Note that the "alternative" Lisp SBCL and Java 7 programs both outperform
Fortran.

Of course I agree with you on wtf-moments. WTF makes Ruby take an hour, and
SBCL take 10 seconds? That's two orders of magnitude! But no matter, I imagine
a more clever Ruby programmer could reduce that, or just call out to a native
library.

~~~
igouy
>>Note that the "alternative" ...<<

A program that simply switched according the command line arg and then
printed:

    
    
        3968050
        Pfannkuchen(12) = 65
    

:would also out perform :-)

>>I imagine a more clever Ruby programmer could reduce that...<<

Is "a more clever Ruby programmer" some kind of equivalent to "a sufficiently
smart compiler"? :-)

>>or just call out to a native library<<

When is a Ruby program fast? When it's written in C ;-)

~~~
Jach
> A program that simply switched according the command line arg and then
> printed…

Of course making programs do less can improve speed, and a great way of doing
that is compile-time computation via macros! You can finish the program before
it's even run.

> Is "a more clever Ruby programmer" some kind of equivalent to "a
> sufficiently smart compiler"?

No, since we assume human intelligence here. :P As the Graphics Programming
Black Book puts it in the Chapter 1 title, "The Best Optimizer is between Your
Ears".

> When is a Ruby program fast? When it's written in C ;-)

I'm going to use this one.

~~~
igouy
I've spent enough time asking for programs for the benchmarks game in Ruby
forums, to start to doubt whether the "more clever Ruby programmer" will ever
come forward ;-)

Maybe "a more clever Ruby programmer" always drops-down to C?

Maybe there's only _a more clever Rails programmer_ :-)

------
gcv
To Clojure-curious people: please ignore this article. It is misinformed.

Others on this thread have pointed out that the Alioth benchmark takes startup
time into account. Yes, this imposes a startup penalty on Clojure.

More importantly, the implementations of each individual benchmark vary
significantly in performance quality. High-performance Clojure requires a
couple of tricks in type hinting, using unchecked arithmetic, and preferring
native Java arrays.

I looked at a couple of the benchmarks, and the mandelbrot example uses those
tricks. Notice the performance there:
[http://shootout.alioth.debian.org/u64/performance.php?test=m...](http://shootout.alioth.debian.org/u64/performance.php?test=mandelbrot)

Notice that Java 7 and gcc run at about the same speed, and Clojure is only
about 2.2x slower (than C). Scala is about 1.9x slower (than C). gcc is about
1.5x slower than Intel Fortran.

To make the benchmark more fair: (1) all timings should disregard startup
time; and (2) all JVM languages should have the opportunity to run the
benchmark a few thousand times before timing it. Otherwise, it measures JVM
startup time, and then it measures how long it takes the JIT to achieve
maximum optimization. By comparison, the C and Fortran code runs at full speed
almost out of the gate.

All-in-all, considering that Clojure is an extremely high-level language I
consider its performance impressive [1]. Yes, the inner loops need to be coded
in a slightly un-idiomatic manner, but you can do all this in the comfort of
your REPL, which makes the process of making optimizations reasonably
painless.

[1] Don't forget to scroll down the mandelbrot results and look at the stellar
performance of other popular high-level languages.

~~~
igouy
>>"Otherwise, it measures JVM startup time, and then it measures how long it
takes the JIT to achieve maximum optimization."<<

Please take that Clojure mandelbrot program, make repeated timing measurements
without restarting the JVM and then report how those times compare to cold
start on your computer.

The mean "warmed" times for the Java mandelbrot program were actually _slower_
than the reported cold start time for the same program.

<http://shootout.alioth.debian.org/help.php#java>

~~~
gcv
Sure thing. I made it run for 4000 cycles twice. The first (cold) run took
2831.323 ms, and the warmed-up run took 2571.397 ms. About 10% faster.

Judging by the invocation noted at the bottom of
[http://shootout.alioth.debian.org/u64/program.php?test=mande...](http://shootout.alioth.debian.org/u64/program.php?test=mandelbrot&lang=clojure&id=5),
the comments about Java in the FAQ do not apply to the Clojure code. The
benchmark seems to have been invoked straight from the command line.

~~~
igouy
Cool! Now let's check the basics:

\- 2831.323 ms for what workload? The benchmarks game measurements are made at
3 different workloads; but the times that matter are those for the largest
workload, in this case N=16,000. So please show the times for N=16,000.

\- the benchmarks game measurements are made with output redirected to
/dev/null

\- both the clojure and java programs are invoked straight from the command
line, and include start-up. The Help page provides additional "warmed"
measurements for the fastest Java programs, for comparison - because sometimes
the JVM startup costs are larger in the mind than they are when measured :-)

~~~
gcv
Well certainly, in a real-world program which runs for long periods analyzing
tons of data, JVM startup costs and JIT costs amount to nothing, as they are
paid in the first few seconds. But we are talking about short, synthetic
benchmarks here.

I ran mandelbrot for 4000 cycles, just invoking the function which does the
work twice and wrapping each call in Clojure's (time ...) form. This all
happened in an AOT-compiled .class file, which guaranteed a cold start. For
what it's worth, I didn't bother tuning the GC or any other JVM parameters — I
suspect I could have made it run a bit faster by manipulating generation
sizes.

~~~
igouy
>>I ran mandelbrot for 4000 cycles<<

That reduced workload only runs the program for 1/10th the time of the
workload shown on the benchmarks game website.

Run the program for N=16,000 and see that "JVM startup costs and JIT costs
amount to nothing" even for these "short, synthetic benchmarks".

(Incidentally, the "usual" cold start measurement shown on the website is the
best of 6.)

------
yason
Don't compare Clojure to C, comparing it to Python or Ruby makes more sense
because they're all high-level dynamic languages. Sure Clojure runs on the
compiling JVM but that doesn't put it into another category; you can also JIT
compile Python in a few ways.

If we can make working websites and even games in Python, we can make them in
Clojure too. What kills Clojure for small scripts and not-long-running
applications is the startup time and that can mostly be attributed to JVM.
Also, the memory footprint of a JVM process tends to grow a lot over time.

I see Clojure rising above specific platforms, though. JVM is in a slow death
spiral. CLR might have some traction on Windows. If Python got it platform
resettled on something more sophisticated than CPython, that might be a good
ecosystem for Clojure.

~~~
adrianhoward
_Don't compare Clojure to C, comparing it to Python or Ruby makes more sense
because they're all high-level dynamic languages_

For me the more obvious comparison would be with other Lisps. Which, back in
the dim past when I was using 'em, were often _damn_ fast.

~~~
brlewis
If you're looking for a fast JVM-based Lisp the obvious choice would be Kawa
Scheme: <http://per.bothner.com/blog/2010/Kawa-in-shootout/>

But as fogus said, speed is only one piece of the puzzle.
<http://news.ycombinator.com/item?id=4223562>

------
rbanffy
How slow is slow?

If it takes me 3 months to deliver a given program in Clojure and 6 to deliver
its Java equivalent, the Clojure one already has 3 months of lead. Assuming
the Java one is twice as fast, it'll take 45 days to catch up.

Development time is expensive, computers are cheap and get twice as fast every
year or so.

While a long startup time is annoying, it can certainly be optimized out if
someone focuses enough attention to the low level aspects of the runtime.

~~~
spacemanaki
I get what you're saying, and you're sort of right, but it's still interesting
to figure out _why_ that startup time is so slow, or what other things slow it
down. That talk from Daniel Solano Gómez about Clojure on Android was pretty
interesting in that regard. I would want to encourage that kind of
investigation, although this blogpost has a terrifically flamebait title.

> Development time is expensive, computers are cheap and get twice as fast
> every year or so.

It's a bit funny that you're citing Moore's law when Clojure is specifically
designed to overcome its breakdown and get out ahead of that lagging curve.
(Paraphrasing an early talk from Rich Hickey: "The hardware guys are
_punting_!!")

You can turn that right around as fuel for your original point, that Clojure
and it's approach to concurrency buys you tons of developer productivity,
compared to whacking about in the weeds with Java. I totally agree there,
those higher level features are valuable and worth something, but they do not
cost nothing.

~~~
rbanffy
single thread performance isn't increasing that quickly, but machines like the
Xeon Phy should be rather sweet for highly threaded (or processed) apps. Also,
if we can make GPUs run Java bytecode, we would unlock a whole lot of GFLOPS
that are just pushing pixels now.

------
kibwen
_"When using the ClojureScript compiler on a hello word example with advanced
optimisation, we end up with some 100kb of Javascript. [...] The Google
Closure compiler certainly helps here by removing lots of unused code, and the
resulting Javascript file is indeed free from all docstrings etc."_

So does the ClojureScript compiler basically just embed a Clojure interpreter
in every file? I'd be interested to see the code prior to optimization.

~~~
fogus
No, ClojureScript does not interpret ClojureScript at runtime. The Clojure
forms are compiled down to JavaScript directly. More info at
[http://blog.fogus.me/2011/07/21/compiling-clojure-to-
javascr...](http://blog.fogus.me/2011/07/21/compiling-clojure-to-javascript-
pt1/)

~~~
kibwen
I wonder if his figure is simply mistaken, then. Because 100 _kilobytes_ is a
heck of a lot of code for a hello world. The compiled representations in your
blog post seem far more reasonable.

~~~
fogus
100K is a lot of code for a Hello World, so don't use ClojureScript to write
Hello World apps. The cool part is that building a largish app will not
necessarily grow the output JS.

~~~
kibwen
Sure thing, but my curiosity still stands. :) If you're using Closure Compiler
to perform dead code elimination (100kb is the "heavily optimized" number, as
far as I can tell), how is it that `console.log('hello, world!');` requires
100kb of essential (non-eliminatable) scaffolding?

------
m_for_monkey
Hello World execution time benchmark with other languages:

[http://lists.nongnu.org/archive/html/chicken-
users/2011-03/m...](http://lists.nongnu.org/archive/html/chicken-
users/2011-03/msg00070.html)

[http://lists.nongnu.org/archive/html/chicken-
users/2011-03/m...](http://lists.nongnu.org/archive/html/chicken-
users/2011-03/msg00090.html)

<http://news.ycombinator.com/item?id=2698579>

------
martintrojer
Have a look at Rich Hickey's keynote presentation from Conj2011;
<http://blip.tv/clojure/rich-hickey-keynote-5970064>

He pretty much starts off by talking about making Clojure "leaner", faster at
starting up etc.

He mentions stuff like a "production" jar with less metadata, hoisted
evaluator and even some kind of tree shaking ala ProGuard.

------
ivanb
If Clojure could dump the Lisp image like SBCL's save-lisp-and-die function,
the startup time would be greatly reduced. I wonder if JVM itself can dump its
current state and then restore execution.

Another approach to the problem would be similar to FastCGI: keep one Clojure
server process running and execute scripts on it.

~~~
densh
> If Clojure could dump the Lisp image like SBCL's save-lisp-and-die function,
> the startup time would be greatly reduced. I wonder if JVM itself can dump
> its current state and then restore execution.

It's still hard to understand why aren't Oracle working on something like
that. It's not as if they don't care about desktop at all -- JavaFX is going
to be part of Java8 and they are even working on a new packaging tool-chain
for it. JVM's start-up time and inability to allocate memory when needed (as
compared to up-front way it's done now) are the major reasons why Java/JVM is
(still) a bad solution for desktop and cli applications.

~~~
pjmlp
There is something like that as internal research project, but never made into
the mainstream JVM.

[http://java.sun.com/developer/technicalArticles/Programming/...](http://java.sun.com/developer/technicalArticles/Programming/mvm/)

~~~
cmkrnl
Interesting - that paper talks about isolates, which are implemented in Dart.
Dart core team members used to work on HotSpot and other Java tech (CLDC).

------
bitcracker
It's an implementation problem of Java.

I never understood why the JVM folks didn't get along to develop a JIT cache.
That means the first time I start a Java program it would run normally slow.
But from the second run on it would use the native cache and run immediately
fast with native performance. That would eliminate many performance problems
of Java.

I know that there already is a solution which uses a Java server to serve the
application as client which reduces the startup time but this is not very
convenient to use.

The slowness of Clojure is a typical problem of all languages which are based
on JVM. Racket Scheme, for instance, which is a Lisp like language but NOT
based on JVM, needs just 0.062s to print "Hello World" (compiled) on my
system.

~~~
ww520
Did you read the article? The majority of the Clojure startup time is spent on
initializing the Clojure runtime.

"spends 95% of the startup-time loading the clojure.core namespace (the
clojure.lang.RT class in particular) and filling out all the
metadata/docstrings etc for the methods. This process stresses the GC quite a
bit, some 130k objects are allocated and 90k free-d during multiple invokes of
the GC (3-6 times), the building up of meta data is one big source of this
massive object churn."

~~~
bitcracker
> The majority of the Clojure startup time is spent on initializing the
> Clojure runtime.

That's correct but even without this startup time Clojure is significantly
slower than other functional languages. Look at SBCL and Racket in

[http://shootout.alioth.debian.org/u32/which-programming-
lang...](http://shootout.alioth.debian.org/u32/which-programming-languages-
are-fastest.php)

That doesn't mean that I don't like Clojure. I am even considering it for a
business project. But Clojure is definitely unsuitable for small apps (shell
scripts etc.)

Btw the benchmark listing doesn't take LuaJIT into account. This JIT is the
fastest I have ever encountered, way ahead of JVM regarding startup time.

------
fshen
Clojure startup time is slow. AOT helps a little.
<http://clojure.org/compilation>

Clojure runs quite fast, in my experience. I have written an web app
<http://rssminer.net>, in Clojure (and some Java). On a small VPS(512M RAM, 1
core CPU), It can handle about 300 request per second, On my desktop, about
2000 req/s. Which is not slow, at least.

The persistent data structures Clojure use is fast too. I did some test a long
ago, It's roughly the same speed as Java collections.

------
djhworld
I remember seeing somewhere that someone looked into tackling the problem of
the startup time by stripping the clojure core libs of unessential metadata
like docstrings etc

I'm not sure if they went through with it though

~~~
lispm
Why would one do that if that's slow and an image can't be saved? One does not
need to have docstrings in the running Lisp. The typical solution to this
problem is to have a file with docstrings, an index and look up the docstring
for some symbol from the file when needed.

------
Chrix
The Clojure's start-up is slow because the source files are compiled. But once
your program is launched, its perfs are correct and very close to Java. About
the immutable data structure, don't forget each method doesn't return a copy
of the data. It's more clever using changes detection.

You can read a lot of information about this part of Clojure in the book
"Practical Clojure". I'm reading it and I'm learning some stuffs about
Clojure.

To conclude, Clojure is fine for "long" program running.

~~~
seanmcdirmid
As others have stated, the OP was comparing run time separately from startup
time, closure was found only to be about 4 times slower than java after
startup costs are a storied away, which is quite believable given the
benchmarks.

For most programs that aren't compute intensive, you won't notice a penalty,
but the same is true with ruby or python.

------
Mitt
What I would like to see is a `defconstant` macro, which introduces a constant
that can not be changed anymore without restarting the JVM.

(def x 1) ; x = 1 (def x 2) ; x = 2 now

(defconstant y 1) ; y = 1 (defconstant y 2) ; Exception

Also a way of fixing functions would be good, so that no lookup is required.
Calling such a fixed function has no overhead, it would be a direct call.

~~~
simonb
Well, there is defonce [[http://clojure.github.com/clojure/branch-
master/clojure.core...](http://clojure.github.com/clojure/branch-
master/clojure.core-api.html#clojure.core/defonce)]

As for function lookup, I recommend reading this thread:
<http://news.ycombinator.com/item?id=2928285> Also, vars are by default static
(but can be made dynamic with ^:dynamic).

~~~
Mitt
`defonce` unfortunately doesn’t help. Without restarting the JVM I can
overwrite that var. And that is okay, because defonce is just a protection
mechanism, to not delete data when a namespace is repeatedly reloaded. This
reloading occurs 99% at development time. Useful tool.

But I would like to have real constants. A final class with a static final
field (and potentially type information), or something like that. This would
give the optimal lookup time, as the JVM would have the direct address.

When I (defonce x 1) I can still (def x 2), without restarting the JVM. I want
this to not be possible with a defconstant.

------
EternalFury
I don't know if I should laugh or cry. Let's just say rock stars are not
engineers.

