

Building Clojure Services at Scale - r4vik
http://blog.josephwilk.net/clojure/building-clojure-services-at-scale.html?utm_source=dlvr.it&utm_medium=twitter

======
arohner
One thing I've noticed with my own Clojure service at scale, is that
eventually, I want to significantly reduce my use of futures entirely.

When you're getting started with Clojure, it's easy to do everything that
needs to happen "later" or "in the background" as a future. Once you go to
production, you realize that about half of those futures should have been jobs
going into a queue somewhere, with logging and guaranteed delivery, retrying
on another machine, if necessary.

I haven't gotten to the point of writing that library, but I probably will,
soon.

~~~
adambard
I've had great success using Lamina for in-process message queues in lieu of
futures, although this does nothing towards making your system distributed.

~~~
cgag
In process queues are a good abstraction, but that doesn't really protect you
in case of your app crashing, which I think is the concern more than being
distributed. I may be misunderstanding though.

Edit: I was already a big fan of SoundCloud, very pleased to find out they use
Clojure.

~~~
prospero
I think the point was that it's hard to make something failure-tolerant
without being distributed. A locally persisted queue would avoid certain
failure modes, though.

------
Terretta
With Ghostery on, there's no content on this page unless you unblock the
SoundCloud widget.

In this case, it's definitely worth unblocking. :-)

~~~
josephwilk
Thanks for the note, I've ensured Ghostery only blocks the SoundCloud widget
and not the entire post now.

------
jhspaybar
This is the first I've heard of the "Scala runtime". Isn't it compiled to
bytecode and usable as a jar dropped in anywhere on the JVM possibly with some
name mangling if you don't design your public interface to be pretty for java
based projects?

~~~
jeremyjh
That's right, its really just a library. Also, Finagle already has an
idiomatic Java API so using it from Clojure would not require use of any
mangled names.

~~~
josephwilk
Thanks for those replies. I've updated that section to better reflect Scala
just being a library dependency.

I'm looking forward to playing with Finagles Java API as at some point I'm
going to need to connect our Finagle projects with our Clojure services.

------
dinedal
I've used Hystrix to great effect before, but am now really curious about
mixing it with LMX Disruptor aka Reactor and it's clj bindings (
[https://github.com/clojurewerkz/meltdown](https://github.com/clojurewerkz/meltdown)
). Anyone have any experience doing so?

~~~
jamii
I was recently looking for clojure bindings. Meltdown is the only one I could
find that is currently maintained but it doesn't appear to expose consumer
barriers anywhere. Without them the disruptor is just a lockless ring-buffer.
The barriers are what allow building multi-threaded pipelines on top of it.

