

Clojure vs Scala - Anecdote - plinkplonk
http://groups.google.com/group/clojure/browse_thread/thread/b18f9006c068f0a0

======
snprbob86
I'm not so sure this is a Clojure vs Scala story, as much as it is a Write One
To Throw Away story.

"The form of the Clojure code mostly follows the form of the Scala code, most
of the same functions"

Well evidently not. Otherwise, the code would not have seen a 75% reduction in
lines of code.

What specific transformations resulted in this delta?

~~~
seancorfield
I'd have to ask how much Scala and how much Clojure you know to challenge my
statement about the equivalence of the code and the reduction in size? (no
offence intended - a genuine question)

I like Scala. I'd hoped it would become my company's "One True Language". It
has a lot less ceremony than Java. I went thru the very painful 2.7 to 2.8
transition. I attended Scala Lift Off! 2010 and Scala Days 2011. I think Scala
is very impressive - a (much) better Java. I hope it will succeed and I still
follow it closely.

I've always liked functional programming, and Lisp, and Clojure hits a
particular sweet spot for me. It also happens to hit a sweet spot for our
company - our developers like dynamic scripting languages and Clojure fits
that better than Scala. Scala _allows_ you to adopt a functional style but
doesn't enforce it (and, to be honest, makes you work at it) - Clojure makes
you work at deviating from the functional path.

Clojure has less ceremony than Scala, is more functional (or at least more
purely functional), and - in my opinion - far fewer demands on developers in
terms of what you must learn to be productive...

~~~
snprbob86
I've written a substantial prototype in Scala, which included half a dozen
Neo4j-backed implementations of scala.collection traits. As for Clojure, I've
only really toyed with it, but I have written meaty code in several other Lisp
dialects. So while I am by no means an expert in either, I'd say I'm qualified
to be skeptical of your claims. Furthermore, I have no vested interest in
either language. I have a future production project in mind for each of them.

I'm going to ignore the discussion about functional vs OOP, and team sweet
spots, and cognitive load of the Scala type system, and all of that. I agree
with and disagree with several of your points, but they are not relevant.

Scala, being a statically typed language, has some additional "ceremony" over
a Clojure. That's for sure. But, in my experience, that ceremony is a
relatively small fixed cost of around 5 to 10 lines per "big scary top-level
thinggie" in the problem and solution domains.

In general, I'd expect the Clojure source code to be shorter by about 5%, not
75%. In some cases, either Scala or Clojure will be significantly shorter that
the other, depending on whether or not the type system or macros are employed
cleverly.

Regarding your specific claim -- "The form of the Clojure code mostly follows
the form of the Scala code" -- I am beyond skeptical. I do not trust your
account of "mostly the same functions".

Without seeing _concrete_ examples of how the code shrank so significantly,
your anecdote is not useful to any reader.

~~~
seancorfield
Thanx for taking the time to give some background. I posted the anecdote on
the Clojure list just by way of sharing some of the progress we were making
with Clojure. If someone had actually asked me, I'd have probably said "No,
don't post this on HN" because it's an isolated anecdote - not a general case.

A couple of folks have picked on the "Clojure code follows the Scala code"
claim. Since the code embodies a fair bit of proprietary business logic, I
can't share it - which is a shame because it would clearly show the structure
really is very similar with almost all of the same function names.

I'll see if I can find some time to sit down with the two code bases side by
side and produce a more detailed comparison which would answer some of the
more critical voices here.

A couple of folks mentioned library support and part of the increased
conciseness in the Clojure probably comes from that - in particular, in
Clojure, a SQL query result is a sequence of maps whereas I had to construct a
collection wrapper around ResultSet in Scala (remember, the code was written
back 2009 - there are better SQL abstractions available for Scala now as third
party libraries).

Using parallel collections in Scala would more closely mirror the approach we
now use in Clojure - and that would shrink the Scala code a bit - but parallel
collections weren't available when the code was written. However, using agents
in Clojure - which would more closely mirror the old Scala code - would only
add a few lines.

I think the Scala code could be made quite a bit more concise because of the
advances in Scala over the last two years. I suspect I could make the Clojure
code a little bit more concise too. I don't believe I could make the Scala
code more concise than the Clojure code.

------
foobarbazetc
Without seeing the code for both implementations, this posting means
absolutely nothing.

I wish I could unread these types of submissions.

------
abscondment
It's interesting that rewriting in Clojure was "less work and more timely"
than migrating to Akka or waiting for Scala 2.10. Any insight as to why?
Having written no Scala, I'm not sure what the Akka changes would entail.

------
DannoHung
Guess you weren't really using the type system if you were able to translate
your Scala code directly to Clojure.

~~~
seancorfield
Not sure what you mean - if code is statically type safe, it will also be
dynamically type safe. Moving from Scala to Clojure is not really affected by
the type system.

------
jzoidberg
Akka actors are the accepted production solution in Scala.

I wonder how the author translated actor heavy code to clojure - I guess the
closest available concept would be agents.

~~~
seancorfield
The original Scala code was written back in November 2009. Akka wasn't really
the "accepted production solution" back then. There's no question that it is
_today_.

We used actors to create a pool of workers in order to run some slow processes
in parallel. The simplest equivalent in Clojure was to run the processes via
pmap. Cheap, quick and very effective. We didn't have as much control over the
parallelism but it didn't really matter for our use case.

We had about 1,000 lines of Scala that we'd developed as we learned Scala.
Today we have more experience in Clojure than we did in Scala back then.
Writing 260 lines of Clojure with the Scala code as a guide took a couple of
days, including writing a new set of unit tests.

Learning enough about Akka to migrate to it would have taken us some time -
and would still have left us with three languages in the mix. By replacing
this Scala code with Clojure, we only had two languages in the mix, which was
also a win.

~~~
Uchikoma
[edit: See my comment below on how this might be a culture thing]

I wonder how 15 hours in the post become a couple of days here on HN. But of
course, the post was written as an anecdote and should be taken as that.

Could you elaborate what makes Akka "a language" for you? I did assume Akka
actors are for all practical reasons a drop in replacement for scala actors.

More people should btw. read "Anecdotal Evidence and Other Fairy Tales" in
Hacknots book:

<http://www.scribd.com/doc/459372/hacknot-book-a4>

~~~
seancorfield
15 hours = 2 x 7.5 hours - 2 days? (seems like basic math to me :)

Akka is _not_ a language - I'm not sure where you got that impression? (please
explain so I can respond to it - no judgement). Based on the Akka
documentation (pre-1.0), it did not seem to be completely a "drop in
replacement for scala actors" but I hope it's moving that way (and I'm very
pleased that Scala is adopting Akka as the default implementation!).

Heh, yes, anecdotes are meant to be just that....

~~~
lightcatcher
I also thought you were considering Akka as a language. The sentence where you
mention "leaving 3 languages in the mix" gave me this impression.

~~~
seancorfield
Ah, I see. No, we have another language in the mix which is not relevant to
this discussion. The "One True Language" comment was a bit tongue in cheek
since I doubt we'd really end up with just a single language across all tiers.
I had initially hoped Scala would become our primary backend language but it
looks like that "honor" will go to Clojure as we're doing more and more work
with that.

------
Uchikoma
Excellent, the title says anecdote to set the right context. There have been
other discussions on how much the knowledge of writing one solution helps in
writing another solution in a different languages and what is attributable to
the new language itself.

~~~
Uchikoma
Wanted to add one link to this where I did get the idea from the first time:

[http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_swit...](http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_switched_back_to_p_1.html)

"#7 - PROGRAMMING LANGUAGES ARE LIKE GIRLFRIENDS: THE NEW ONE IS BETTER
BECAUSE _YOU_ ARE BETTER"

------
Uchikoma
It takes a little away from the story that it took them a lot of time to
discover a widely published performance and memory problem with scalas actors.

"after a lot of poking around, we became fairly convinced it was due (at least
in part) to the default actor implementation in Scala"

~~~
seancorfield
"widely published performance and memory problems with scala actors" -
remember the code dates back to November 2009 when that was not common
knowledge...

~~~
Uchikoma
The post was probably a little unclear, I did not understand you poked around
to find the oOms in 2009, I just assumed it was more recent as the Clojure
post was from September 2011.

~~~
seancorfield
No, we didn't hit the OoM regularly until we had more traffic as we moved more
sites onto the new system and it wasn't clear that there was an admitted
problem with the Scala actors at first either.

These days folks seem much more comfortable about admitting there's a problem
with the built-in actor implementation.

Is that clear enough for you?

------
rbranson
A story presenting objective truth through measured experimentation and
empirical data.

------
schiptsov
Numbers of lines of code and improved readability alone are worth switching.

As a consequence, less code means less consumption of resources, for Java it
is very true.))

Just by looking at the code one could see that Clojure makes the development
process for JVM less annoying and frustrating while Scala makes it even more
verbose and complicated. ^_^

~~~
ambrosebs
Lisps make it fairly difficult to reason about resource consumption based on
code size. And in general, is not a good metric for such a measurement.

------
vrode
People and their one-liners.

