Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Who's using Clojure, and to do what?
106 points by gregschlom on June 22, 2010 | hide | past | favorite | 70 comments
Hello everyone,

I've seen a lot of articles on HN recently talking about Clojure. I had a quick look at a few pages (Wikipedia, etc...), and I can't help but think that it sounds like the new cool language, pretty much like Ruby in 2007.

But to get a better idea, I'd like to hear the feedback from the community: Who is using Clojure (in real-life cases), and what are you doing with it?

Bonus points if you can tell what specific features made it a better language for the task.




Another important question to ask is: if you're not using clojure, is there a clojure-specific reason for that? (ie. "I would use clojure but...")

In my case: I reported a bug in the clojure.contrib XML library, on the google group. It's a trivial fix, but not worth the hassle of signing a contributor agreement. So although I'd fixed it myself, I couldn't send a patch. All I could do was report it - and the report was ignored.

Some time later, I reported another, deeper bug that I didn't know how to begin fixing (but did know a workaround for). It got ignored too.

I like the language, and I'm not going to say my experience was typical of the community. I spent some time hanging out on the group and the people there seemed friendly and helpful. But that's one reason I no longer use it. (Another reason is "no reason at all", and I don't know the relative weights of these reasons.)

This feels a little like whining to me, but feedback should be both positive and negative.


It's not just you. Requiring copyright assignment is a powerful way to put the brakes on community involvement by itself, but couple that with a policy of "nothing gets applied until the BDFL personally approves it" and things can get frustrating. I've had bugs with perfectly viable patches linger in the issue tracker for 3-4 months before getting applied. In some cases the original author of the library even personally approved the patch, but it still had to wait for the thumbs up.

Hopefully this will improve with the creation of the Clojure/core group. Having every commit need to be approved by a single person is absolutely unsustainable in the long run, and we're definitely already at to the point where it's causing issues.

That said, it certainly doesn't stop me from using Clojure every day; it just makes me prefer decentralization (using 3rd-party libraries rather than contrib, for example) where possible.


I should add that having Rich review every single commit is very appropriate for some parts of Clojure: I wouldn't want people committing to the STM mechanism or to the implementation of clojure.lang.PersistentVector without his approval. And for a long time Clojure almost entirely consisted of this kind of code--stuff that's really hard to get right.

But now it's had more more pedestrian nuts-and-bolts kind of libraries added like clojure.test and clojure.main. These are included out of necessity, but they are very easy to understand and fairly easy to patch and improve as well. It would be reasonable if the original authors were allowed to improve them and accept patches from the community for them even after they've been promoted from contrib.


Hi Phil,

I went back through the mailing list and think I have found the issues you are referring to here. I have asked Chouser for feedback on the XML issue, and have responded to the "deeper" question myself (http://groups.google.com/group/clojure/browse_thread/thread/...) Let me know (preferably on email) if I found the right ones, and if the answer is satisfactory.


I'm not using Clojure, because I get enough Lispy goodness on the Java platform from the Kawa Scheme compiler already. I haven't been motivated to do anything real with Clojure yet.


I wrote a text classifier in Clojure and I'm currently developing it in to a product. It is fairly good at author identification, with a substantially better success rate than libsvm, the tool most commonly used in academic papers.

I'm looking at several possible applications, the one I find most interesting being a tool to catch students who pay others to write their papers.

Clojure has a few features that have been a big help. The rich set of tree-based data structures is especially helpful because it makes splitting and merging data inexpensive. The ability to inexpensively switch to mutable versions and back where appropriate for performance reasons is also quite nice, and absent from most other functional languages. The fact that its a Lisp has advantages that should be well-known by most people here.

Edit: the fact that it's fast is also important for my application. I need to be able to do classification on a large number of possible categories at web speed.


I work at Etsy. I launched a new search tool powered by Clojure last week:

http://www.etsy.com/explorer

It's used for the autosuggest and facet suggestion on the sidebar.

We'll be posting more about this on our tech blog sometime soon.


Your job postings mention Scala in several places. Are you using both internally or is this Clojure project more of a one-off?


We're using both, but right now this project is the first use of Clojure. We have a lot more Scala code.


Are you using clojure for the front end? Are you using ring/compojure or something else?


The clojure code is a hand-rolled algorithm that sits behind a compojure web service. The front-end is just standard jQuery hackery :)


BackType is a YC (08?) company. They manage Terabytes of real time data using Clojure + Hadoop/Pig:

http://www.backtype.com/opensource


We don't use Pig, we use a Clojure-based query language for Hadoop that we wrote called Cascalog:

http://github.com/nathanmarz/cascalog


My bad. Sorry.


I don't know if you consider this "real-life", but I used Clojure to implement and run stock strategy backtesting trials for my master's thesis project. I defend my thesis on June 29th.

The big reason I used Clojure was that it was easy to parallelize my program; however, running independent trials with many different parameter-sets is one of those "embarrassingly parallel" tasks.


I just thought of this, and I don't know why it didn't come to me earlier.

Not that I had the foresight to pick Clojure because of this, but after I started working with Clojure I noticed that change became much easier than with the languages I was using before (one of which was C++).

When my thesis advisor would tell me, "you should go do X" (fill in the blank), I'd do what he said, and then a week later he'd have changed his mind and he would say "now go do Y", instead of X. Those changes would kill my enthusiasm and productivity when I was using C++, but once I began using Clojure those changes no longer felt like a bombshell. Change became relatively easy.

Now, I'm not saying that change is trivial, but compared to C++ (in my experience), changes are much easier to deal with using Clojure.


Very interesting. Did you consider using R? Why did you choose Clojure for this? I've been thinking of doing something similar myself and I would love to hear your thoughts considering you've already been-there-done-that.

I'd love to read your thesis and code, if they're available.


I had heard of R, but I didn't really know anything about it. I knew it was a statistical package, and I knew it had its own language, but I didn't really care about learning how to use it; I wanted to learn to use Clojure, Scala, Ocaml, or Haskell.

One of the reasons I picked Clojure over the other languages is that I knew that I could use code from the Incanter project if I needed some statistical functions.

A link to my content-complete thesis is: https://docs.google.com/document/edit?id=1kOKjY265a3F5SbN25d...

The thesis is long and boring to read. Let me save you some time. Here it is in a nutshell: Genetic algorithms (GAs) can be used, somewhat successfully, to identify good parameter-sets for technical trading strategies; however, even the "best" GA-identified parameter-sets in my research failed to outperform the buy and hold strategy when trading, with EOD prices, over the course of a randomly picked 1-year period taken from the interval starting January 1, 1985 and ending May 1, 2010.

My code is not available yet because according to the university it belongs to them (ridiculous, I know). I'll have to ask my advisor whether or not I can make it publicly available.


I understand your university's position ... and yours also. If they do decide to make the code publicly available, I'd appreciate a reply to this comment. I'll check back occasionally to look for updates.

Enjoying you're thesis, btw, especially the section on Bollinger bands. The Turtle Traders (http://bit.ly/by1j2M ) seemed to have used this successfully with commodities. Did you reference any of the Turtle traders stuff when you were designing your system?


I just put my thesis project online at github. The URL is http://github.com/davidkellis/stocktrader_clojure.

Also, I bought and read Way of the Turtle. I really enjoyed it. Thanks for the suggestion.


If my advisor tells me that I can make the code available to the public, I'll put it on github, and post a comment here.

Thanks for the comment about my thesis. I hadn't ever heard of The Turtle Traders, but I read a few pages of the preview that you linked to, and it looked interesting enough to buy a copy. It will be an interesting read. Thanks for the tip.


R does have quite a bit of financial libraries, yet I don't remember R has any good support on parallel computing, hmm?


True, but Matlab has good support for distributed computing. It's even possible to do this on an EC2 cluster. Pretty impressive really: http://www.mathworks.com/programs/techkits/ec2_paper.html

edit: Cleaned up my bad grammar.


There is the multicore package:

http://cran.r-project.org/web/packages/multicore/index.html

My worry with parallel loops in R would be inadvertent race conditions. For example, is the random number generator in R thread safe? I doubt it.


Can you tell us more about your data set?


Yes, in the backtesting trials I ran, I tested three technical trading strategies using EOD adjusted closing prices between January 1, 1985 and May 1, 2010. I only conducted trials against current S&P 500 member companies. I licensed the data from Norgate Investor Services. I am not affiliated with that company. They just had a price that I, as a poor graduate student, could afford.


Gotcha.

How do you think it would hold up to larger, say intraday data?


As far a runtime performance goes, my backtesting framework would run slightly slower using intraday price history than it does with EOD data.

There are two reasons why:

1. There would be more data to load into memory, and that process of loading the price history from a CSV file into memory would take a little longer simply because there is more data to deal with.

2. The data structure I'm using to index the historic price information by timestamp is the Java 6 implementation of TreeMap. The TreeMap class is an implementation of a Red-Black tree, and provides a guarantee that lookup operations (even when the key is not present in the collection) are O(log n). Since the time to find a key in the collection is O(log n), there would be a small performance hit when adding more data to the collection.

I actually began using intraday price history that I had downloaded from TradeStation, but I ran into the problem that I only had intraday price history over a 1-year period. I needed to be able to run 1-year trials over the course of a much larger period of time, so I switched to EOD data. Switching to EOD data had the nice side effect of speeding up my program (mainly due to the fact that loading all that intraday price history into memory took several seconds - up to 30 seconds or so).


Clojure's a decent choice for writing a cross-platform UI application, using Swing. Java+Swing sucks, but Clojure+Swing works pretty well. I wrote a dependency graph library, dgraph, which significantly reduces the amount of boilerplate required to keep UI state in sync.

http://github.com/gcv/dgraph

I also wrote an embedded database library for Clojure, based on Berkeley DB.

http://github.com/gcv/cupboard

Which specific features make Clojure a good choice? Interactive development and macros.


I've been considering writing a desktop app. I've been wanting to get into the JVM action, but dimly remember being scared off by Swing years ago. What about Clojure makes it better at using Swing?


Swing can be a lot of repetitive code. Clojure has mechanisms (functions, macros and libraries) that can help you to abstract this away to exactly the way you like to write user interfaces.


I've used Clojure to write clj-peg (http://www.lithinos.com/clj-peg/index.html). I then use clj-peg everytime I've needed to process some consistent structure (RTMP/AMF0/AMF3), anytime I want to create a quick DSL (the source markup for my site), and just for fun (looking into writing a markdown grammar).

Macros were essential to what I wanted in a PEG-based system. I wanted to use as few extra characters as possible to increase readability, since fewer characters meant less clutter around the Non-Terminals. I also wanted a custom syntax that leveraged the basic approach of the parent language to minimize the learning curve. Macros allowed me to easily introduce those features inline with the rest of the language.

I've also wanted to use a language that was moving toward platform independence so that my efforts would last longer. Clojure isn't far (a year or so) from being more easily implemented on platforms other than Java. I'm fairly convinced that they'll make it and clj-peg will take advantage of that if it can.

While Common Lisp, among other languages, meets the macro feature, and I developed in Common Lisp for a while, bringing the Java ecosystem into the picture solves other problems I had had with common lisp.

In addition to clj-peg, there are other projects that are in-process, but I'd rather not bring them up here since they don't have the resources that clj-peg currently has.

To sum up: Macros and the Java ecosystem.


This project looks interesting and although I don't have much time to get into it at the moment I'm mainly commenting to create a personal bookmark for it and to thank you for sharing your project.

Please consider moving the project to a hosting site like github which would make it easier to track updates to it and easier to find in the future. Thanks again!


I'll second the github suggestion. clj-peg looks very interesting.


I'm using Clojure for my PhD thesis project, implementing a distributed computing framework for building P2P applications. I've also been creating an audio synthesis and musical live-coding system, available here:

http://project-overtone.org

Clojure has without a doubt been the most eye opening language I've learned in the last 10 years. I highly recommend giving it a try. It has powerful yet incredibly simple to use constructs for dealing with state in a multi-core world, and I think it far surpasses everything else out there today for parallel programming. The centerpiece being a built-in software transactional memory system that enables composable, deadlock free, multi-threaded coordination.

Beyond solid support for concurrency, the standard library provides an elegant set of programming tools that will make you realize how even expressive languages like python and ruby make you jump through lots of unnecessary hoops. In Clojure you represent everything with basic maps, sets and sequences, and the standard library has extensive tools for manipulating these things. (By the way, your code is also available for manipulation with these same tools.)

Macros let you peel away every last bit of boiler plate, and what you realize is that in the end all of programming is just arithmetic, iteration, branching, and assignment. Everything else is just getting in the way of your core logic. I think Clojure lets you get to the center of your problem with the least overhead possible.


My friend and coworker 'wglb uses it, over my weak and pitious objections, to build network protocol fuzzing code here. Don't worry, we'll beat the Clojure out of him soon enough.


Hmmm. Well, in addition, the other night, I was looking at the java vm spec and mulling over some dissatisfactions with jad. Where could that lead.

Oh, and, as I told "First Blood", quoting from Watchmen, "you think i am in here with you, but, really, you are in here with me."


Out of curiosity, what are your objections?


We do most everything in ruby. It is very cool for the kind of work we do. And being a lisper, I must say that I really do like Ruby.


It scares the crap out of clients.


(Tell them it's just another jar file deployed with the application. Unless they need to see the source code...)


Not sure how good the compiled Clojure code is. Also -- it might be too dynamic to allow any kind of obfuscation so it won't be used for stuff that's sold with licenses for example.


Bollocks. We AOT-compile and then obfuscate closed-source clojure code shipped to commercial customers.


On the flip side, don't they also think you're geniuses to be able to use it?


Writing a baked blogging engine in Clojure, heavily using Enlive for a large chunk of it. Many fewer lines of code than doing it in Python/Ruby.

Plus, honestly, it's a lot of fun to use.

Next up is some data analysis: I'm looking into the excellent Incanter for that.

Great libs, great community and a fun, fun language that's practical and elegant.

EDIT: to expand on what I like about the language; it's a lisp, so everything that comes with that. It's also got some syntax that simply makes it more pleasant to use on a day-to-day basis. And so much more!

(excuse typos: my phone is acting up today/can't handle HN for long comments.)


Warning: irrelevant grammar nazism below.

> Many fewer lines of code than

This grammar bugs me. I feel like this should be "Much fewer" or similar, but I'm not totally sure it's wrong as posted. Opinions?


Many is correct. Many refers to countable items, much refers to uncountable items. Many apples vs much wine, many lines of code vs much text.

I do agree with your assessment of the awkwardness of the phrase though.


"Many fewer" should be technically okay.

"Much fewer" is definitely wrong.

"Far fewer" would be my choice.

Normally, grammar nazis at least know something about, which you clearly do not. Why does someone of such low caliber try to correct others?


"...someone of such low caliber..."? Was that really necessary?


DocuHarvest is a web application for extracting valuable data from documents (just PDFs for now):

https://docuharvest.com

Short reason: JVM perf/libs + persistent data structures + lisp goodness (macros etc.) is a helluva package deal.


I'm building CloudHoist.com, a cross-cloud console that does node provisioning, configuration and administration.

http://cloudhoist.com/

Lisp in general is amazing, and being able to deploy to the JVM is a huge win for Clojure.


I'm using it to verify replication between a big sybase instance and a big oracle instance. This isn't rocket science but performance is much better than it was in Python and the code is about the same size.

I'm also and to liberate data from a MeetingMaker server that's only available via a Java api and make it available in Rails apps. Obviously the superior expressiveness of Clojure makes the code base a lot smaller and makes more sophisticated processing of the data far simpler than it would be in Java.

Hopefully more soon.


I used clojure to build http://elev.at

Clojure code services the HTTP service calls (multimethods are very nice here) and also does the job of converting spreadsheets or CSV files into XML.


We're building semantic web apps for data integration. Clojure is great for it as it's all either symbolic manipulation (query translation and planning) or moving data with as little overhead as possible.


We are an Indian startup and we are building a next-gen finance webapp in Clojure & Python.

We are using Clojure since the early days and we are extremely happy with it.

We used Clojure to build a finance engine which aggregates financial info from various places, stores them in the DB and serves them over a RESTful HTTP API.

Clojure's awesome concurrency semantics and the overall power of Lisp combined with the vast JVM ecosystem has helped us a lot.

We are Infinitely Beta (http://infinitelybeta.com). We are launching in a month, I will announce when we do.


Do you have an announcement mailing list? Or blog RSS feed?


Not yet, but you can follow the team on Twitter - http://twitter.com/ghoseb/infinitely-beta


We have build a large Web-Application with Clojure. It runs on Google App Engine and we are really happy with our decision. You can find it here: http://the-deadline.com

We use the power of lisp to extend the language for specific tasks, like accessing the distributed key/value datastore or building an expert-system shell.


I don't normally correct people's spelling but in this case...you've got an error on your company homepage. Should be "they don’t need to be as daunting as we tend to _perceive_ them."


Is used clojure to build a front-end to an inventory program I wrote for the Library of Congress.

There are a lot things to like about clojure, but the reason I chose it for this task was that it was the only language on the JVM that had a usable (imho) repl.


I've done about 4 non trivial projects with Clojuire. (a) some fairly serious SEO related to RFS1 (b) a news aggregator (c) music-compojure http://github.com/mhowlett/music-compojure (d) some investment portfolio performance calculations.

I really like the language. Am I productive with it? Not really - as a guess, I'm about 3 times faster with .NET / C# (with which I'm very experienced). I might be able to get to the same level of productivity, but it'd certainly take me a while to get there.


do you have the source code for the news-aggregator?


Have you experimented with F# a bit yet?


Not really because I'm fairly keen to move away from the Microsoft platform.


Algorithmic trading. Currently only a GUI for managing my existing trading algorithms in Java and a evolutionary algorithm I use for optimizing parameters, but I am looking at using Clojure to implement the trading algorithms themselves.

Pros: wonderful integration with Java, good support for concurrent programming, conciseness, macros.

Cons: numerically intensive code can be a bit slow (but it is going to improve significantly with 1.2, and you can always do that in Java), prefix notation can be quite hard to read if you are dealing with complicated math expressions.


Can Clojure be used in Android app development? Since Android apps are JVM-based.


Android isn't jvm based. They wrote their own VM with their own bytecode format, and a bytecode translator from java bytecode. Since the translator is written in java, it is possible to run it through itself and use that to run clojure, but it's dog slow. Even when statically compiling a sample "hello world" clojure app for android the bootstrap process takes over a minute.


I'm hoping to port it to the CRIO for FIRST robotics, but I haven't looked into trying yet. All my other projects have been relatively minor so far.


FlightCaster (YC S09) uses it in production: http://www.google.com/search?q=flightcaster+clojure


I'm an soon-to-be-undergraduate just learning the language and Lisp.

I found this great series by Rich Hickey last night: http://clojure.blip.tv/




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: