

Why we are choosing Clojure as our main programming language - gislik
http://appvise.me/2011/03/why-we-are-choosing-clojure-as-our-main-programming-language/

======
jedsmith
I won't commit HN suicide by questioning his choice of Clojure (edit: the
downvotes and only positive replies are interesting, though), but I will focus
on the original impetus: it's interesting to me that a few problems are
explained in Python, including solutions of how to fix them, but the author
doesn't want to use the solutions. Honestly, it sounds like he wrote off
virtualenv without trying it, because he'd have far less complaints about
porting and dependencies if he had.

As for the GIL, the author didn't even consider multiple processes -- he
explained them away as a solution for some workloads. A Web app is one giant
workload where this model makes sense: multiprocessing is the approach you
should be taking with a Web application. Tie a request to one core, and spawn
enough WSGI applications for the number of cores you have (and then some).
That's an elegant solution to this problem, since a request doesn't need to
fart around with other requests in most cases. If your app can't handle
multiple copies of itself running, how do you expect it to scale?

This reads a little bit like a waffling, like a person who considers a
completely different environment and rewrite of the entire stack a solution to
some thorns in Python. I'm all for picking an alternative Web stack, but:

> I had written a database loader to import Apple’s Enterprise Partner Feed
> (EPF) and a web crawler in Python and next up was the web interface.

> It all seemed like a smooth sailing but in the back of my head I was
> beginning to have doubts about my decisions.

> Why we are choosing Clojure as our main programming language

This being the third blog post for an as-yet-unreleased product, I'd say worry
about delivering a product instead of justifying a rewrite of your work thus
far on your blog. If I were considering funding your startup, this blog post
would be a fairly bad sign to me.

At any rate, a computer language is just a tool to implement an idea, and
focusing strongly on the language of choice is busywork itself.

~~~
bad_user
Also related to the GIL -- the "processing" module in Python takes care of any
sharing scenarios you might have and it is mostly API-compatible with the
threading API in Python.

And if that's too heavy, there's always stuff like GEvent. Asynchronous I/O
and multiprocessing solves 80% of all problems you may have.

I do wish that Python / Ruby would be GIL-free, but for my startup I choose
these platforms because they've got no parallel when it comes to rapid-
development of web applications, because no matter how cool this and that
language is, nothing beats the productivity gained by robust, battle-tested
web frameworks (not to mention other stuff, like NLTK or SciPy).

As an early-adopter myself I might try out a couple of projects in Clojure,
but I wouldn't bet my business on it unless I saw a clear need that Clojure
satisfies and that dwarfs all other disadvantages, like platform immaturity.

~~~
swannodette
Having built websites in Ruby and Python, I've found Clojure to hold its own
in terms of productivity. Clojure's library ecosystem is healthy and growing
and you can tap into the battle-tested Java frameworks with ease.

Saying that Asynchronous I/O and multiprocessing solves 80% of the problems is
hand-waving away the complexity such approaches might add to a project.
Clojure simply gives you more and better tools then Ruby and Python currently
do for dealing with concurrency.

You might not bet your business on Clojure, but certainly other people have
for a couple years now and they don't seem to take issue with it.

EDIT: removed the defensive bit. My experience, as this thread shows, is that
people get touchy about the limitations of the GIL.

~~~
bad_user
Stating an opinion is not getting defensive. You can do whatever floats your
boat.

EDIT: RE: Don't take me wrong, I sometimes hate the limitations that the GIL
brings.

I'm continually looking at alternative implementations, like Pypy or Rubinius
or JRuby, or other languages, like Haskell or Clojure, but then I end up doing
a lot of yak shaving and nothing gets done.

~~~
Calamitous
> but then I end up doing a lot of yak shaving

You say that like it's a bad thing. ;)

------
vic_nyc
I have written web applications in Ruby on Rails, PHP and recently Clojure.
While I absolutely love Clojure and what it offers, building web apps with it
has proven painful, because the library ecosystem just isn't there yet. There
is indeed the foundation, as the author mentioned - Ring, the equivalent of
Ruby's Rack, and a few other frameworks such as Compojure. But these are very
early stage projects, parts of them are still being rewritten and they are
relatively poorly documented as of now. If you run into problems along the way
"how do you do this?" you would have a hard time finding adequate info online,
whereas the amount of information that exists out there for RoR, for example,
is incredible (message boards, tutorials, blog posts).

But even if you go through the initial pains of figuring out how things fit
together, many tools are still missing. An easy to use templating library, for
example. (several exist already but they are still early stages /are being
refactored). Also things like the gems and plugins ecosystem in RoR - where
you can pretty much find gems for so many things out there. (e.g.: tagging,
authentication, etc). There simply isn't a comparison.

So, while I really love Clojure and I believe the community around it is
growing - in both size and its contributions, I would say that, let's face it,
writing full web apps in it right now cannot be compared to the productivity
you would get in say RoR, due to the immaturity of the tools. That being said,
I still see Clojure being immensely useful in other scenarios (for example:
A.I. algorithms, high performance data processing, etc) and as such I would
use it mostly in those settings, while integrating it with a more mature
'front-end' framework.

~~~
pico303
I had the exact same experience with a project we started recently.

We jumped from Clojure/Compojure to Erlang/WebMachine with much better success
so far. I did in Erlang in two days what Clojure took me two weeks with
numerous false starts to accomplish, mostly because the libraries were so
poorly documented or incomplete. I spent more time digging through code and
trying to assemble a framework to build upon than I did writing useful
business logic.

I found the Erlang infrastructure to be well-structured and strongly
documented, mostly because of it's maturity. What I surmise is that Clojure is
just too young, and its web framework lacks a strong commercial force driving
it. Rails has 37signals, Webmachine has Basho, Lift has sites like Twitter and
Foursquare. Clojure needs something similar to push it forward.

All arguments and emotions aside, I'd say if you're starting to build a new
web site, and you're not considering Rails or Lift, you're doing yourself a
major disservice. Like others have said, JRuby nicely avoids some of the
issues discussed for scripting languages.

------
iamelgringo
I know this is heresy around here, but have you looked at F#? It's functional,
has type inferencing, even in the dev environment, straight forward parallel
versions of functions (pmap vs map), async IO, amazing dev tools, a REPL
inside of Visual Studio, access to all of the .NET libraries. And, chances
are, it's going to be around for a while.

You're absolutely on the right track. As startups struggle to hire, new
startups should be looking to off loading more and more of their work to more
productive programming languages.

~~~
weego
Is it counter productive in an environment where hiring is difficult to go too
niche though?

Our entire server codebase, baring a few external libraries, is Scala (working
with Lift) which has been an awesome experience but there is a nagging doubt
in my mind that if/when we need to start looking to add in developers we will
either need to invest in cross-training a java dev or end up paying out
probably more than we could/should afford to get a seasoned java dev who
trained themselves in Scala already. If over a short period of time your
choice gets some major traction then it will work in your favour, but if not
then you could be out in the cold or risking employing someone with no real
provable history.

~~~
MichaelGG
Well, Jane Street Capital uses OCaml. According to "Caml trading"[1], hiring
got easier when using OCaml:

"Personnel is one area in which OCaml has been an unmitigated success for us.
Most importantly, using OCaml helps us ﬁnd, hire, and retain great
programmers."

1: <http://www.janestreet.com/minsky_weeks-jfp_18.pdf>

~~~
chollida1
To be fair, Jane Street probably is helped in this regard by being able to pay
great 6 figure salaries and 6 figure bonuses to their developers.

~~~
cygwin98
I think we need to compare apples to apples here: other competing hedge funds
may have to pay a lot more to have those hackers code in C++, Java, or (gasp)
VBA.

~~~
chollida1
Fair point.

------
otterley
It's interesting that the needs of the business don't seem to factor into his
analysis _at all_.

I'll concede that sometimes there are specific cases on which you need to use
a new or non-mainstream language or environment. (I'm 99% certain that this
business is not one of those cases.) I'll concede, too, that being on the
cutting edge is pretty cool and gets you lots of hacker cred.

But if you want your business to survive the departure of the founding team,
you need to consider whether you can solve the problem with a mainstream
environment. If you don't, don't expect your invention (at least in its
current form) to carry any legacy.

Case in point:

People may think Paul Graham is a fucking genius for selling Viaweb to Yahoo!,
despite being written in Lisp, but I assure you none of his code still lives
on there, not even a fork. His brilliance is as a businessman for getting
Yahoo! to fork over $<lots> to him, not for the technical merits of Viaweb
itself.

~~~
eftpotrm
Yahoo bought Viaweb 13 years ago. The challenges then were completely
different to what they are now, as were the underlying technologies at all
levels. Frankly Viaweb could've been written in the most widely used,
predictable technology ever seen and I'd still expect it to have been
rewritten in the last 13 years.

~~~
otterley
Sure, most things are rewritten or replaced in the very long term. But
Viaweb's code didn't even last a few years.

~~~
gridspy
Yes, but it seems lately that Yahoo is where good startups go to die.

Viaweb was purchased because it was successful. According to PG, it was
successful because they could implement major features in a weekend.

~~~
otterley
... and you missed the whole point of my argument.

That the _original author_ can major features quickly does not necessarily
imply that _an ongoing concern after the departure of the original author_
will also be able to implement major features quickly.

Choosing your environment with your successors in mind usually increases the
value of your business (assuming, of course, that your valuation has a
rational basis, which tech companies aren't always good at determining).

~~~
rayiner
You missed the point of his argument.

Consider the Crash Bandicoot folks. Sure after Naughty Dog got bought, Sony
couldn't figure out how to deal with the Lisp codebase and future projects
were in C++. But Lisp let them build the first major platform game on the PSX,
beating Sony itself to the market, and none of the stuff afterwards would've
happened without that.

------
biznickman
"While there are enough programmers that know PHP (which is clearly a plus)
there’s just not enough sex going on here. As with Perl, PHP really wasn’t in
the loop."

How is that a valid argument? "not enough sex going on here" ... if you choose
a language because it's sexy and not because of its utility, you are making a
poor business decision.

Digg spent a long time revamping their system in new languages and on new
databases and look how that turned out (not that the languages were the core
part of their failure, but still)

~~~
JonnieCache
_> "not enough sex going on here"_

I actually thought he meant it in a Darwinian/Dawkinsian(?) sense. That would
have been a pretty neat turn of phrase. But yes you're right, he clearly means
sexy. Stupid word.

------
st3fan
It is a myth that Java _requires_ massive amounts of XML that make your eyes
bleed.

It is perfectly possible to configure a servlet environment once with 15 lines
of XML and then never look at it again.

Look at Google Guice for example, which does awesome Java based configuration
and composition of components.

~~~
ww520
I actually found Java + Play! to be very productive and fun to develop. Have
done 3 projects in that combo already.

------
st3fan
I agree with the Python packaging hell. Things are getting better but it is
really hard to beat a simple WAR file that contains all your dependencies.

Deployment is also a breeze. Place WAR file on a server, let operations people
grab it and drop it in a servlet container. Done.

Python need to reach a point where it is that simple. I see people around me
handle this with Python but it takes a lot of effort to get it right.

~~~
nubela
And why is this hard with virtualenv / pip ? I don't get it. Sure its not as
simple as a single WAR file, but if you are unable to initialiaze a virtualenv
bottle, perform a pip install with a dependency file, ensure that the python
version is the same, you should not be a administrating a server.

------
dylanz
Good read, and I just read this post while attending Erlang Factory in SF, in
the middle of a talk about Lists and Strings :)

I asked the teacher about your comment on String manipulation. Yes, it is
pretty in-efficient. There are libraries to make manipulation easier, and, you
can always go down to binary types, which is much more performant.

We chose to use Erlang for a variety of reasons, and String manipulation isn't
a problem for us (it's definitely a pre-mature optimization point (for us) at
the moment).

~~~
nivertech
Erlang strings is just a syntactic sugar over lists.

    
    
      1> [104,105].
      "hi"
      
      2> [$h,$i].  
      [104,105]
      
      3> "hi".
      "hi"
    

Which is efficient for some uses (i.e. iterating over UTF32 characters) and
inefficient for others (high memory usage).

You can always use:

    
    
      * atoms - for interned strings or enums
      * binaries - for memory efficiency (i.e. UTF8 byte sequences)
      * IO-lists - for efficient appending and IO.
    

What I would like is a per-module compiler directive/pragma, which will turn
every "string" into <<"string">>, while @"string" will remain syntax sugar for
list.

------
enduser
No judgement here, but from reading this it seems possible that the author did
what I and many others have done: overlooked Common Lisp after one or more
encounters with Scheme.

~~~
pavelludiq
I did this too, last year i corrected that mistake by learning common lisp and
now clojure is my second favorite language :D

------
cdavid
He mentioned python issues such as deployment and the GIL. While python is my
language of choise, I agree those are significant pain points. Unfortunately,
he did not mention how clojure solves deployment - is it solved through maven
and other java solutions ? It did not strike me as a nice solution either.

~~~
gislik
Just to collaborate a little bit on this point. With leiningen things are not
too bad.

A typical deployment instruction file (project.clj) looks like this and with
"lein deps" you're all set up in a few seconds.

(defproject leiningen "0.5.0-SNAPSHOT" :description "A build tool designed to
not set your hair on fire." :url "<http://github.com/technomancy/leiningen>
:dependencies [[org.clojure/clojure "1.1.0"] [org.clojure/clojure-contrib
"1.1.0"]] :dev-dependencies [[swank-clojure "1.2.1"]])

~~~
gislik
ouch ... formatting is clearly not preserved :(

But take a look at <https://github.com/technomancy/leiningen>

~~~
dpritchett

        (defproject leiningen "1.5.0-RC1"
          :description "A build tool designed not to set your hair on fire."
          :url "https://github.com/technomancy/leiningen"
          :license {:name "Eclipse Public License"}
          :dependencies [[org.clojure/clojure "1.2.0"]
                         [org.clojure/clojure-contrib "1.2.0"]
                         [lancet "1.0.0"]
                         [jline "0.9.94"]
                         [robert/hooke "1.1.0"]
                         [org.apache.maven/maven-ant-tasks "2.0.10" :exclusions [ant]]]
          :disable-implicit-clean true
          :eval-in-leiningen true)

~~~
cdavid
I am not familiar with leiningen or clojure, but it looks like you are just
adding a list of requirements + version, which is exactly what you get with
pip + requirements.txt file ?

~~~
dpritchett
I believe that's what's going on, yes. Leiningen runs on top of Maven and so
it's downloading the jars you need and storing them in ~/.m2/ where they will
be linked in at compile time.

A "lein uberjar" will roll your whole program up into one .jar file, ready for
deployment.

------
SeanDav
Since the OP enjoys Python, I am quite surprised Stackless Python didn't seem
to enter the equation.

------
dkarl
If library versioning in Python is bad, how would a JVM language be better? In
my experience, there is no widely-used post-build-time system for declaring or
checking dependencies except for OSGi, which is not spreading like wildfire as
I expected it to. The Java model seems to be to bundle all your dependencies
into a single deployable app, performing all dependency checking at build time
and carefully isolating your app from other apps in an application container.
I.e., don't even try to solve the problem of sharing libraries between
applications.

Oddly enough, C has superior runtime checks (dynamic library loading using
major version numbers) and install-time checks (*nix package dependencies)
compared to dynamic languages. It is very, very strange to me that other
language communities have neither embraced alternatives such as OSGi nor
worked to transfer responsibility to native package managers such as dpkg or
RPM. C et al. under Linux have set a standard that other language communities
don't seem interested in matching, much less exceeding. As far as I know, the
standard answer for deploying a security update to a Java library is to
rebuild and redeploy the entire application that depends on it.

~~~
herdrick
Clojure has Leiningen for library versioning / package management / builds.
It's excellent.

~~~
dkarl
It seems to be a compile time only tool. Does it let you safely deploy
application code separately from its dependencies? Does it let different
applications share libraries when possible? Does it come with tools that let
you query the version and dependencies of a deployed library and see how those
dependencies are satisfied? Can you push a security fix for a library to
servers in the field without completely rebuilding and redeploying every
application that uses the library (if you can figure out which ones they are?)

~~~
technomancy
> It seems to be a compile time only tool.

Out-of-the-box it's a development tool, but it has plugins for certain types
of deployment: <https://github.com/technomancy/leiningen/wiki/Plugins>

You can create tar/jar/war files, deploy artifacts to remote mvn repositories,
push to the Google App Engine or Elastic Beanstalk, etc. Deployment to generic
unix servers is handled by Pallet, which integrates well with Leiningen:
<https://github.com/pallet/pallet-lein>

> Does it let different applications share libraries when possible?

This goes strongly against the culture of the JVM for various reasons that are
outside the scope of Clojure itself.

> Can you push a security fix for a library to servers in the field without
> completely rebuilding and redeploying every application that uses the
> library.

Sure, this is pretty easy to do with Swank, but the specifics are going to
vary widely based on the type of deployment.

------
F_J_H
I wonder if discussions like this will someday be analogous to a mechanic
blogging about "Why we are choosing SnapOn instead of Craftsman as our tools
for building NASA cars", or a chef posting on "Why I moved from Cookware X to
Cookware Y".

I guess it is important to discuss the tools of the trade, but I wonder how
long it will be before asking someone what technology they used to build an
app would be like asking a musician what brand her instrument was. ("Hey,
great song - is that a Stratocaster?) At the end of the day, does your app
enable your business to make money? Heck, I've built a business with revenues
in the millions using MS Access as one of the main tools! (A long story
there!)

What I have learned, which other commenters have already pointed out, is that
issues like maintainability after you are gone, finding resources quickly, the
size of the user community, etc. are all equally important and should not be
overlooked.

------
there
_While there are enough programmers that know PHP (which is clearly a plus)
there’s just not enough sex going on here._

wat

~~~
nickik
php sux. Thats what he wants to say in a nice way.

~~~
there
it sounds more like they chose not to use it just because it's not cool, not
because of any technical merits.

~~~
dpritchett
If you're into tea leaves you could check Tiobe's Magic Number charts and see
that PHP apparently peaked last year:

<http://www.tiobe.com/index.php/paperinfo/tpci/PHP.html>

------
hendler
In particular, I appreciate this article as much for "Why not Python". I feel
Python could easily be the lingua franca if it could deal better with the
version issue. Using Scala for similar reasons (the other being general
performance).

Probably the author underestimated node.js though. Using the author's
language, it has a lot of "sex".

~~~
robbles
I'm not sure that a "lingua franca" is even possible. Even if one language was
so far ahead of the others that discussions like this thread were pointless,
there would always be resistance from other developers. People just seem to
get too attached to the idea of "There's one language to write everything in,
and I pick this one."

