
Yammer moving from Scala to Java - Jd
http://blog.joda.org/2011/11/real-life-scala-feedback-from-yammer.html
======
frio
The big problem I had with Scala, and the reason I stopped actively developing
in it months back, was its similarity to Perl's philosophy: there are too many
ways to skin the same apple. Problems can be resolved functionally,
imperatively, or in some (inevitably, when you come back to work on it, hard
to follow) hybrid of the two; unless you set down clear restrictions and code
guidelines, one man's Scala is entirely different to the next's.

I initially liked the expressivity the language afforded (and some of the
supporting ecosystem - sbt, specs - I found extremely impressive), but came to
abhor it over time. Unfortunately, after maintaining legacy Perl applications
I've developed an aversion to anything that doesn't follow the Zen of Python:
"There should be one-- and preferably only one --obvious way to do it."

~~~
dustingetz

      > I've developed an aversion to anything that doesn't 
      > follow the Zen of Python: "There should be one-- and 
      > preferably only one --obvious way to do it."
    

this is an interesting argument. but surely, given enough rope, a non-expert
can hang himself in any language. I'm starting to wonder that all these
language X vs language Y problems are really a matter of the skills of the
team matched with the difficulty of the problem. joe bloe's blog needs
beginner skills with python. embedded wireless systems need experts and C/asm,
nuclear control systems are prototyped by experts in haskell. bigcos need
large teams of average skill with Java, but as they are driven to become more
competitive, perhaps they will turn to small teams of experts using scala.

~~~
reginaldo
_nuclear control systems are prototyped by experts in haskell_

Do you have any source to back that (honest question)?

~~~
fpgeek
It might be a pun referencing Atom (
<http://hackage.haskell.org/package/atom-1.0.11> ) which is used for control
systems, but not nuclear ones, AFAIK

------
PaulHoule
There's something primal about Java that invites people to write it in styles
that ape other languages. In fact, whenever I write Java I find myself trying
to write FORTRAN or Lisp or something else. I did a stint of work in Scala
that now has me writing Scala in Java. I'm starting to feel that Java is a
better Scala than Scala!

It all started when I had to write Java classes that consumed results from
Scala. There are all these Scala classes like Some that, if you look at the
docs, you realize you can use in Java without any trouble at all.

Many of the patterns that Scala uses can be implemented easily in Java... If
there's something you like about Scala, you can usually do it in Java with a
little thinking.

A good example is all of that Actor BS in Scala. You can screw around with
that and come to realize that you're completely out of control, you don't know
how many threads are involved and exactly how. Java's got ExecutorService,
which for many of the tasks I do, can keep an 8-CPU machine busy with 100%
utilization.

I remember having a discussion with a guy in Germany about his system that
used Scala Actors that revealed that neither of us had any idea of what was
really going on at all. Today I build pure Java systems using Java idioms that
are so fast it's almost scary.

~~~
spaznode
"You can screw around with that and come to realize that you're completely out
of control, you don't know how many threads are involved and exactly how"

I think this was the turning point for me, the day I realized someone had
created this large concurrent system with absolutely no idea or care as to
what was going on. (the not uncoincidentally same day qa discovered some
serious problems making the app become almost useless with extremely high CPU
loads)

------
johnbender
I've been playing around inside the closure-compiler in my free time recently
and the thing that struck me most after years of working in other languages
(Ruby, JavaScript, C#) is that Java is relatively simple. While it's obviously
a very small sample set, I have yet to find myself spending any significant
amount of time wondering at what a given snippet of code does.

I don't have any Scala experience but in general simplicity has major and
obvious benefits (as noted many times in the letter).

~~~
daniel_solano
Yes, Java is generally fairly simple. I generally find that the most difficult
part of Java is generics.

On the other hand, anything that you want to work at a slightly higher level,
such as closures or higher-order functions, Java just gets in your way since
you have to it all by hand.

Thinking about Scala, there are a couple of problems with its
simple/complexity balance:

1\. Some things that appear fairly simple actually explode into a huge mess of
complexity under the hood. Often, that's not a problem. However, sometimes it
becomes a problem because you don't really know what's going on.

2\. Some things that should be simple and straightforward, like iterating
through an array, are impossible. These are cases were the language is
actually _removing_ some of the power of the underlying platform.

~~~
extempore
> Some things that should be simple and straightforward, like iterating
> through an array, are impossible.

Impossible?

    
    
      array foreach f
      array map f
    

Or if you miss java enough:

    
    
      var i = 0
      while (i < array.length) { f(array(i)) ; i += 1 }
    

I can't even imagine what it is you think is impossible.

~~~
daniel_solano
You're right, it's not quite impossible. However, the idiomatic ways of doing
things in Scala tend to be very bad for performance if you're in a tight loop.
For example, if you have two arrays that you need to iterate through
simultaneously, you need to zip them together. In order to regain the
performance, you end up having to do something like the while loop or use
recursion. This is similar to the problem of doing heavily numeric code in
Clojure before release 1.3. You could get Java-like numeric performance out of
Clojure, but you'd have to abandon doing it in any sort of ideal way.

One example where something was impossible in Scala is that to implement a
Parcelable for Android. This requires the creation of a static field, which
Scala doesn't allow. Now, I won't argue that an API that requires such a
construct is _good_ , but it might be necessary for your application. I
believe Scala now contains special code in the compiler just to support this
peculiarity in the Android API.

~~~
EnckePan
In your example of iterating through two arrays, the proper Scala idiom is
(a1,a2).zipped.map((v1,v2) => ...). This avoids doing an actual zip which
would have significant overhead. It also makes it easier to write the map as
the map method on Tuple2.Zipped takes a function of two arguments so you don't
have to deal with pulling values out of a tuple.

I would actually find the details of where Scala performance is poor to be
very interesting. Most significant IMO would be whether they are real problems
or people picking improper implementations. For example I'd like to be able to
answer the following, would using a view have given them the performance they
needed without producing ugly code?

------
sreque
@codahale, was this email from you or someone else in your company?

Judging from your blog post a year ago at
<http://eng.yammer.com/blog/tag/scala>, at one point you thought Scala was
great. As someone who still thinks Scala is great, I would like to know how
long it took you to change your mind and what triggered you to reach this
realization. When it comes down to it, I've never developed Scala as part of a
team or worked on a project more than a few thousand lines of code, so I
definitely lack the experience you probably have with the language and
runtime.

As to the email itself, I find the complaints about performance most
uninsightful, and I say this as someone who has written CPU-intensive NLP code
in Scala. Anyone who writes performant code to run on the JVM knows that the
only way to get C-like performance in Java is to, it turns out, write C-like
code in Java!

That means operating only on primitives and arrays of primitives when in tight
loops. It means avoiding allocations as much as possible. It even means
avoiding generics like the plague, including java.util.HashMap. In my NLP
code, for instance, I got at least a 100x speed up by switching from HashMaps
with string keys to arrays by manually interning each string into an
incremented number. One library I used, called Mallet, relied on GNU trove to
have hash maps without boxing of primitives, because for high performance,
even java.util.HashMap is too slow!

This means that, if you want amazing performance in Scala, you have to write
C-like code in Scala as well. I'd expect anyone who needs top-notch
performance to know this kind of thing already. Writing performant code in any
high-level language in our day and age, even C++, still requires you to
understand how your abstractions are actually getting implemented underneath
the hood. Scala doesn't work miracles here or everyone would probably be using
it already.

Just ask the people who run JGit about performant JVM code.
([http://marc.info/?l=git&m=124111702609723&w=2](http://marc.info/?l=git&m=124111702609723&w=2))
They ran into a ton of cases where they couldn't write the equivalent C-code
in Java and performance suffered as a consequence.

However, the points about the build toolchain, backwards binary compatibility,
and to some extent, initial learning complexity are the ones that I think are
most damning against Scala. I hope Typesafe has a good response to this or
other companies step up to share how they worked around these deficiencies
successfully. Otherwise, I feel like this could be potentially very damaging
to Scala's reputation, and maybe even deservedly so.

~~~
arete
Well said!

I've found Scala to be a fabulous language for writing > 90% of my app(s). The
overhead of Scala's built-in collections, for loops, property
accessors/mutators, closures, etc is simply unimportant outside of critical
hotspots and provides so much in terms of brevity and expressiveness.

Yes, Scala's lack of looping constructs (break, continue, a real indexed for
loop) means that you're better off writing performance-critical things like
crypto or protocol parsing code in Java (or if it really matters, C called via
JNI). But once you're concerned about performance at that level, _every_
language requires you to carefully consider what your code is really
generating, what your allocation patterns are, etc.

Scala is a much more expressive language compared to Java, and the Scala
standard library is excellent. Completely dropping Scala for Java when you're
building large, complex systems seems like chopping your foot off because your
small toe hurts. Use Java or C for the % of code that needs it and Scala for
the rest!

~~~
ShardPhoenix
Part of the original complaint was the issues introduced by using a
combination of Java and Scala rather than just one or the other (with Scala-
only being impractical due to the need to use Java-based libraries).

~~~
arete
I'm guessing you are referring this:

"2. Don't ever use scala.collection.mutable. Replacing a
scala.collection.mutable.HashMap with a java.util.HashMap in a wrapper
produced an order-of-magnitude performance benefit for one of these loops.
Again, this led to some heinous code as any of its methods which took a
Builder or CanBuildFrom would immediately land us with a mutable.HashMap. (We
ended up using explicit external iterators and a while-loop, too.)"

Overall Scala<->Java interop is pretty seamless, and aside from some annoying
things (for loop becomes foreach with a closure, break & continue are done
with exceptions, etc) it's quite easy to treat Scala as a more concise Java
with very similar performance.

Rather than using a Scala collections wrapper for the java.util.HashMap he
should have simply used the Java HashMap as-is. Yes you must do
hashMap.get("key") instead of Scala's shortcut hashMap("key"), but so what?
You're talking performance-critical code already, wouldn't you want to avoid
the wrapper?

~~~
adgar
> You're talking performance-critical code already, wouldn't you want to avoid
> the wrapper?

Trivial wrapper classes that just delegate methods to a final ivar should end
up inlined.

------
Uchikoma
One year ago I wrote on Scala:

"In 15 years of Java I never had those problems. I could focus on writing
production code."

<http://codemonkeyism.com/scala-unfit-development/>

Promises have been made in the comments, things did either not get better or
even got worse since then (Collection API, the nightmare that is SBT 0.1X,
binary compatibility).

------
rayiner
Interesting read. Honestly, Scala looks like gibberish to me. I strongly
distrust a language where I can't hold its conceptual model in my head all at
once. This is true for C, true for JS, true for ML, and hell even true for
Common Lisp (once you realize it's all macros over a simple core). It is not
true for Scala or Haskell, at least for me.

~~~
Peaker
From <http://www.paulgraham.com/avg.html>:

> As long as our hypothetical Blub programmer is looking down the power
> continuum, he knows he's looking down. Languages less powerful than Blub are
> obviously less powerful, because they're missing some feature he's used to.
> But when our hypothetical Blub programmer looks in the other direction, up
> the power continuum, he doesn't realize he's looking up. What he sees are
> merely weird languages. He probably considers them about equivalent in power
> to Blub, but with all this other hairy stuff thrown in as well. Blub is good
> enough for him, because he thinks in Blub.

~~~
rayiner
The power continuum isn't the same as the conceptual simplicity continuum.
C++, for example, is less powerful than Common Lisp while being more complex
as well.

At the same time, is Haskell more powerful? Common Lisp, relative to C++, buys
me the ability to run arbitrary functions at compile-time and generate code.
Haskell takes away that power, and as far as I can tell doesn't add anything
to replace it. Indeed, it lets your code do strictly less thanks to the type
system.

~~~
Peaker
There's also the reliability/error-proneness continuum.

Haskell is probably less powerful than Common Lisp (Template Haskell
technically yields the power, but it is less practical). The type system may
also rule out some useful programs.

However, Haskell yields more reliable, less error-prone programs. The number
of runtime bugs that get through the Haskell compiler is substantially lower
than that of Common Lisp (I venture this is still true if you use CL's
optional typing).

Haskell also has forms of power that as far as I know, Common Lisp does not
have. Namely, type-classes allow expressing various forms of static-type-based
polymorphism that's not expressible via Common Lisp (e.g: Return-type
polymorphism).

In practice, having a smaller emphasis on macros pushed Haskell to abstract
various repeated patterns using means other than macros. Some cases are
covered by laziness. Others are covered by passing around simple functions.
Some are covered by special wrapper types and combinators.

My personal opinion is that the extra composability of these alternatives is
more beneficial than their weaker power is detrimental. The cases where
alternatives to macros cannot be found are relatively rare, so the extra cost
of Haskell's macro system (TH) rarely needs to be paid.

------
va_coder
Are Clojure shops experiencing this? One of the things that makes Clojure
appealing to me is the lack of types and simplicity (IMO) of the language.

~~~
jshen
I wrote a medium sized project in clojure at my last job and it was a good
experience. The team was small (under 5) so the hiring and training wasn't an
issue for us. I never had a problem reading clojure code, and the only major
headache I can remember, which was our fault, was that clojure keywords (think
ruby symbols) used to be turned into interned strings which are stored in
permgen and we were running out of permgen space. I believe this has been
fixed in recent versions of clojure, but I'm not 100% sure about that.

[http://eigenjoy.com/2011/03/02/clojures-keyword-can-fill-
up-...](http://eigenjoy.com/2011/03/02/clojures-keyword-can-fill-up-your-
permgen-space/)

~~~
jes5199
ruby (MRI ruby, even!) has that same bug.

~~~
getsat
So does Erlang, but it's not really a bug. Symbols (atoms, whatever) are
guaranteed to be a unique identifier, so the fact that they're allocated and
exist for the duration of execution isn't really surprising.

It's just something you need to be aware of.

------
protomyth
I look at the talent availability problem as an HR / Mangement issue. It feels
like they want to do checklist hiring instead of hiring great developers who
can learn. I would rather take the time needed in hiring, but at some point it
becomes gears in the machine.

------
squidsoup
There seem to be some questions around how accurate this is; can anyone from
Yammer comment?

~~~
fizx
As a Scala developer at one of the major Scala-using companies, this feels on-
the-money.

~~~
runT1ME
Can you elaborate? Do you mean that the criticism is valid (which I agree is),
or that you agree with the conclusion of the detriments outweighing the
benefits?

~~~
daniel_solano
While I haven't used Scala on an "enterprise" scale, I can certainly agree
with many of the criticisms. However, in my experience, Scala is still worth
it.

I don't deal with training a team of developers, nor do I have to integrate
with enterprise-scale infrastructure, so those things just don't apply to me.

What are the benefits of Scala? I can think of a few, off-hand:

1\. Traits: I really like traits and the way that you can use them to
encapsulate cohesive and reusable functionality. In some ways, traits allow
you to do a form of aspect-oriented programming.

2\. Case classes are a nice, simple way of creating basic data structures. In
Java, creating a data structure that should have a proper equality
implementation and a nice toString is a tedious chore. Granted, I think
Clojures record's are better since they give you immutability, too.

3\. Closures. Other languages also have them, but Java does not. Being able to
use closures can really help deal with things such as handling events. In
Java, you have to resort to creating inner classes, some of which end up
instantiated as members of the class.

4\. Judicious use of implicits can give you some half-decent DSLs.

5\. 3+4: A nice implicit conversion of a one- or two- line block into an
appropriate instance of a ManagerAdapterEventListener can really simplifiy the
code.

6\. The mixed paradigm approach can be a blessing sometimes. I do prefer the
functional approach. However, if you have integrate with a heavily object-
oriented framework, being able to be object-oriented can be helpful.

Overall, I would generally prefer Scala over Java for my projects. However, I
do prefer Clojure over Scala.

~~~
bad_user

         I can certainly agree with many of the 
         criticisms. However, in my experience, Scala
         is still worth it
    

Considering the alternatives, C++ was and is still worth it. However because
of its popularity, it prevented and still prevents other languages from
gaining critical mass, making it the only game in town, with the sole
exception being Obj-C, but that's only because Cocoa was designed for it and
Apple doesn't mind a little technology lockin either.

------
nirav
It is interesting to see how start-ups (twitter, forsquare and now yammer)
adopting or dropping programming languages rains hailstorm on twitter, HN and
blogosphere. As a serious PL designer, start-ups seem to be the prime market
to target your language.

~~~
trimbo
Same goes for the nouveau databases. It's going to be hard to get traction in
an enterprise that can already afford and/or has purchased Oracle.

------
pbreit
"The Rest of the Story": <http://codahale.com/the-rest-of-the-story/>

------
jebblue
At the end of the day, Java still rocks because it makes the most difficult
aspect of programming more manageable...herding cats.

------
SWengineer
I found Neal Gafter's response to Stephen's blog rather telling:
blog.joda.org/2011/11/real-life-scala-feedback-from-
yammer.html?showComment=1322613312501&m=1#c6062812671177626424

------
guard-of-terra
They should consider Bolts as treatment:
<https://bitbucket.org/stepancheg/bolts/wiki/Home>
[https://bitbucket.org/stepancheg/bolts/src/tip/src/main/java...](https://bitbucket.org/stepancheg/bolts/src/tip/src/main/java/ru/yandex/bolts/collection/example/ListFExample.java#cl-22)
(a small Java library with collections wrappers having native performance and
all the FP methods; composable function classes; other niceties like tuples)

