
Software Engineering at Prismatic - fogus
http://blog.getprismatic.com/blog/2012/4/5/software-engineering-at-prismatic.html
======
jshen
"While we make heavy use of the core of Clojure, we don't use its concurrency
primitives (atoms, refs, STM, etc.) because a function like pmap doesn't have
enough fine grained control for our needs. We opt instead to build our own
concurrency abstractions in Clojure on top of the outstanding
java.util.concurrent package."

I wrote a web crawler in clojure at at&t and our team came to this same
conclusion. First we tried to implement a custom atom, but finally using
java.util.concurrent directly.

I'm not sure what this says about the clojure concurrency features, but I've
found myself using them sparingly even when doing a lot of concurrency.

~~~
eternalban
Rich Hickey is borderline genius in my book, and I really enjoy his
philosophical approach to CS. Clojure is a fine language, but I frankly never
bought the "concurrency issues" as a reasonable motivation for Clojure (over
Java).

Fact remains that STMs (as originally developed by the Scala team in (yes)
Java), "outstanding" [OP] concurrency, and idempotent/functional programming
remains /well within reach/ of the seasoned developer in Java.

Also, unless I misunderstand their architecture (and it is single node ..),
then the fact remains that in a distributed system, the "semantics" of
interaction are not bound by the programming language behind box x, and are
entirely a matter of the (remote) API you expose. STMs are a concern of a
shared memory space -- once you go distributed, it is really borderline
snakeoil to talk about the benefits of "functional" languages in creating
remote functions.

~~~
jshen
Our system was both distributed and highly concurrent in the same memory
space. We started off using atoms, but we needed visibility and control over
the queue feeding the thread for the atom which lead us to using
java.util.concurrent directly.

Our use case was fairly odd though.

------
davegauer
"We tend to build our own libraries simply because much of the functionality
of open-source code is locked up into monolithic frameworks and not easily re-
usable."

Wow, I couldn't agree more with this concept of library vs. framework. While
frameworks provide all kinds of up-front time savings, I've never used one
that didn't end up getting in my way and require all kinds of deeply placed
hacks to work how I (or my company/client) wanted.

I sometimes feel out of the loop because I'm not using all of the popular
frameworks everyone else is talking about at the moment. So it's refreshing to
hear I'm not alone in the opinion that time is saved in the long run by
writing one's own small, modular, easily understood and reused libraries if
none are already available.

I do feel as though small, self-contained code bases are becoming easier to
find with the rise in popularity of sites such as GitHub and Bitbucket. So
perhaps the trend towards less monolithic software can become more mainstream.

~~~
gruseom
I had the same reaction. Fully agree with both you and the OP.

------
noelwelsh
I'm most interested to know about the Flop library. This has always seemed a
weak point of the JVM ecosystem. While there are many numeric libraries none
seems dominant and speed suffers next to BLAS. Of course programming and using
such a library in Java is also painful. This is a domain where functions
really are by definition the right abstraction!

------
mattrepl
The CEO, bradfordcross, is amazing at distilling good engineering to
principles. Software engineering for hackers.

------
programnature
Very much looking forward to the rest of the posts, and the release of these
projects :)

------
plam
where's the rss feed to subscribe to?

~~~
aria
I'll put that up right now. Sorry it's a _very_ new blog. Here's a direct link
<http://blog.getprismatic.com/blog/rss.xml>

~~~
spariev
Any chance of getting back contents of Bradford's old blog
<http://measuringmeasures.com/> ? There were a few brilliant articles on
becoming data scientist, learning statistics etc. It would be much
appreciated.

~~~
gtani
that was a good one:

[http://web.archive.org/web/20101102120728/http://measuringme...](http://web.archive.org/web/20101102120728/http://measuringmeasures.com/blog/2010/3/12/learning-
about-machine-learning-2nd-ed.html)

------
sorenmacbeth
nerd crushing so hard this team and their work.

------
davekale
Great post. Very thought-provoking.

