
Observations from a real-world Clojure project - liebke
http://groups.google.com/group/clojure/browse_thread/thread/ffcd4bc722852b4?hl=en
======
richcollins
"Everything* just worked. We took a snapshot at 1.1.0, and we didn't update
it. [* The JVM does SEGV on us occasionally. Internal error, which we have not
diagnosed.]"

How are memory protection errors just a footnote?

~~~
cma
You shouldn't be able to SEGV a JVM no matter what bytecode you feed it
(exempting JNI). So it is a JVM bug.

~~~
wmf
OTOH it doesn't really matter whose "fault" it is; if using Clojure causes
SEGVs but some other Lisp doesn't cause SEGVs, that's a problem.

~~~
yangyang
That depends if you're using it primarily because it is a lisp or primarily
because of its Java interop / the fact it runs on the JVM.

------
aidenn0
I'd like to reiterate the lack of good debuggers for not just Clojure, but
also free CL implementations. I haven't branched out much recently, but is
sbcl/SLIME still state-of-the-art for lisp debugging in the open source world?

~~~
smanek
I personally use SBCL, but I've heard a lot of people use Clozure Common Lisp
(<http://www.clozure.com/clozurecl.html>) because it has better Slime
integration (better stack frame inspector, etc).

The very best tools are probably Allegro CL (w/ its non-emacs IDE) and Lisp
Works (and its own IDE too). I've heard nothing but good things about both
(and have used Allegro CL enough to know it - and its associated libraries -
are very impressive).

As far as I can gather no one really uses them since they are just too
expensive though. They must be making money somewhere, I just don't know where
...

~~~
gruseom
_I've heard a lot of people use Clozure Common Lisp
(<http://www.clozure.com/clozurecl.html>) because it has better Slime
integration (better stack frame inspector, etc)._

Really? I would have said CCL has worse Slime integration in general (Slime
being an epiphenomenon of the SBCL-on-Linux world) and far worse when it comes
to debugging. Am I missing something?

We're using CCL with Slime on OS X. The Clozure people are working on a new
IDE, but I don't know how good its debugging facilities are yet. It would take
a lot to get me off Emacs, but slick debugging might do it.

Alternatively, I often think that maybe we should just take the time to make
sldb do what we want. But it's hard to take precious resources away from one's
main project.

If anyone feels the same way and is interested in hacking on this, email me -
maybe we can work something out.

~~~
smanek
IIRC, ITA Software (one of the biggest Lisp houses in the world ...) uses SBCL
for their main QPX project. A few years ago when they were starting the new
Polaris project, they decided to go with CCL instead of continuing with SBCL.

My understanding is that the two major motivating factors for this decision
were that CCL compiles faster and has better debugging support.

This is all based on my memory of conversations that took place about a year
ago, so it's entirely possible I'm mis-remembering or what used to be true no
longer holds. I have no first hand experience with CCL so if you do, I'll take
your word for it ...

~~~
alec
I currently work on QPX, the air travel search engine. We continue to use
SBCL, primarily because it produces faster code for our workload. We and our
customers are very sensitive to the performance of the application.

Dan Weinreb, another ITA developer, did a talk at Google a few months ago on
ITA's new reservation system (the Polaris project you mention). They do use
CCL, but the code also compiles and runs with SBCL. The primary reason he
cites in that talk for preferring CCL over SBCL (about 22 minutes in) is that
CCL compiles code more quickly than SBCL.

I haven't used CCL much, but I understand SBCL and CCL have different
debugging strengths. An SBCL developer tells me that a significant chunk of
the compilation time for our application is in constraint propagation for
types. That's part of the "generates more efficient code", but it also means
that it's smart about telling you when you're doing something that's not going
to work. For whatever reason, type propagation or otherwise, CCL won't do as
much to help you at compile time. I am told that CCL gives better stack
traces, so SLIME can help you more.

~~~
smanek
Yeah, that vibes with I've seen in SBCL (and been told about CCL). Thanks for
the inside scoop.

------
WalkingDead
_5\. The functionality of the docs hasn't kept up with Clojure.

6\. Debugging facilities also have not kept up with the state of Clojure._

These two are the first thing I check before trying to use any programming
language.

~~~
Raphael_Amiard
That is true only if you use bleeding edge clojure (EG. 1.1.0), clojure 1.0
docs are very mature. And since 1.1.0 is a developper version at the moment,
it appears quite illogical IMO to complain about the state of documentation.

~~~
regularfry
They're mature, but they are also not very good. I still spend ages digging
around trying to figure out what should be relatively simple stuff. The
documentation around ns in particular is difficult to navigate.

------
nearestneighbor
> Example: ~10M records are processed and transformed, various computations
> occur, and ~100K records are spit out. Lots of statistics. One type of run
> takes 12 hours on an 8GB, 4-core Linux box

That seems insanely slow, unless they are doing something insanely complex. In
other words, this doesn't tell us very much.

~~~
dgreensp
My "slow sense" is tingling too. I'm deadly curious to know how fast I could
make that "12-hour" job using Scala or Java.

I've done extensive work with Scala, Java, and bytecodes; writing performant
Scala code requires some knowledge of the JVM and how Scala is compiled, and
requires sacrificing some of its power; but because the static typing and
method dispatching is so similar to Java's, you can get there. From everything
I've read about Clojure, it sounds a whole level less efficient.

Not trying to knock Clojure, but 12 hours just sounds way way too long without
more explanation.

~~~
tensor
How can anyone even approach this with a straight face? You don't even know
what they are computing!

If you want to question clojure's performance for technical reasons, do that.
But let's not use feelings, and instincts in discussions about concrete
scientific topics. There have certainly been several interesting articles on
clojure performance, and work done to improve performance for native data
structures (transients). I, and no doubt many others, would be genuinely
interested in performance studies on more complex concurrency problems I've
seen so far.

~~~
lucifer
Then this discussion between Cliff Click and Rich Hickey may be of interest:
[http://blogs.azulsystems.com/cliff/2008/05/clojure-stms-
vs.h...](http://blogs.azulsystems.com/cliff/2008/05/clojure-stms-vs.html)

Its a bit dated ('08), but sounds like the jury is still out on STMs.

~~~
jbert
That's a great read, thanks very much. It might be worth submitting the link
itself to get a wider audience.

------
Sidnicious
Bah. I still can't forgive Google for wrapping everything sent though Gmail or
Groups to 80 characters.

