
Technology Change: .Net to Scala - SanderMak
http://randonom.com/blog/2013/10/technology-change-net-to-scala/
======
karterk
I'm part of a team who moved from Java to Scala recently (C# is a nicer
language than Java these days). Apart from me, nobody else had real-world
Scala experience but were generally good programmers and were able to pick up
the functional aspects of Scala fairly quickly. The move was a big success,
and a frequent comment I hear is "I can't imagine moving back to Java".

One of the reasons why the migration worked for us was that one can (with a
bit of discipline) get into idiomatic Scala slowly. You can definitely start
off writing Java-ish Scala and still gain a lot from the decreased verbosity
and nicer APIs. You can safely stay away from over-using implicits, functional
programming etc. till you get a good grasp of the language. With excellent
Java interoperability, you don't have to convert your entire stack to Scala
overnight, but slowly replace old code with Scala.

With respect to performance, if you insist on writing idiomatic Scala, there
is definitely a performance overhead, because the JVM is not suited for large
amounts of small allocations, as is often the case with an immutable style of
programming. However, we run a pretty large scale data crunching system, and
depending on what your code does, CPU bound performance hit pales in
comparison to the network and I/O waits, so for us, it was not a big deal.
When we absolutely needed to optimize a tight loop or something, we wrote
C-like Scala, using while loops and mutable data structures.

Addendum: The greatest win for us was the readability of the code base. Scala
allows you to largely keep your data structures immutable, so it's much easier
to reason about. With parallel collections, Actors and Futures, concurrency
support is excellent as well. You definitely need to pick up a few new
concepts, but the time is well spent as once you get a hang of those things,
your productivity will be 10x - you don't have to wrangle with primitive
concurrency constructs.

~~~
flipchart
Why is it so hard to keep your data structures immutable in C# or Java? C# (my
primary language) has parallel collections as of .Net 4.0 and with .Net 4.5
you have the async keyword support on top of the TPL. Could you elaborate on
the differences between Scala and C# which make Sala more preferable for you?
I'm not trying to start a war on C# vs Scala, I'm genuinely interested in the
differences (I enjoy learning new languages, but for the right reasons)

~~~
bad_user
It's hard because of the type-system. Experiment with Scala's immutable data-
structures and try implementing them in Java. For example, lack of support for
type-classes, not to mention the really shitty support for generics in Java,
designed probably by sado-masochists, makes that really painful to do and
there's a world of difference between Guava and Scala's library.

For a real example, try designing generic data-structures in Java, that
exposes the below functionality in a type-safe way:

    
    
        scala> val list = List(9, 7, 5, 10, 11)
        list: List[Int] = List(9, 7, 5, 10, 11)
    
        scala> list.sum
        res0: Int = 42
    
        scala> list.sorted
        res1: List[Int] = List(5, 7, 9, 10, 11)
    
        scala> val stringList = List("a", "b", "c")
        stringList: List[java.lang.String] = List(a, b, c)
    
        scala> stringList.sum
        <console>:9: error: could not find implicit value for
          parameter num:      Numeric[java.lang.String]
                  stringList.sum
                             ^
        scala> val bitSet = collection.immutable.BitSet(1,2,3,4)
        bitSet: scala.collection.immutable.BitSet = BitSet(1, 2, 3, 4)
    
        scala> bitSet.map(_ + 1) // the result is still a BitSet
        res3: scala.collection.immutable.BitSet = BitSet(2, 3, 4, 5)
    
        scala> bitSet.map(_.toString) // the result is not a BitSet
        res4: scala.collection.immutable.Set[java.lang.String] = Set(1, 2, 3, 4)
    
        scala> map.map(kv => (kv._1 + 1, kv._2 + 1)) // result is still a Map
        res7: scala.collection.immutable.Map[Int,Int] = Map(2 -> 3, 4 -> 5)
    
        scala> map.map(kv => kv._1) // result will not be a map
        res8: scala.collection.immutable.Iterable[Int] = List(1, 3)
    
    

In case you're wondering, you can't.

~~~
curveship
The parent commenter was mostly asking about C# to Scala, so for comparison,
here's you snippet in C# with LINQ. It's a little more verbose -- C# generics
need to be a little more explicit -- but I think you'd agree it's pretty
close. It's equally typesafe, but it is not immutable.

    
    
                var list = new List<int>() { 9, 7, 5, 10, 11 };
                list.Sum();
                list.OrderBy(i => i); // list.Sort() is in-place, so using functional OrderBy instead
    
                var stringList = new List<string>() { "a", "b", "c" };
                stringList.Sum(); // C# signals a typing error
    
                var bitSet = new HashSet<int>() { 1, 2, 3, 4 }; // not sure what a Scala BitSet is, but it looks a collection of ints?
                bitSet.Select(b => b + 1);
                bitSet.Select(b => b.ToString());
    
                // map wasn't defined in the snippet.  Assuming it's like a Dictionary<int, int>?
                var map = new Dictionary<int, int>() { { 1, 2 }, { 3, 4 } };
                map.ToDictionary(kv => kv.Key + 1, kv => kv.Value + 1);
                map.Select(kv => kv.Key);

~~~
bad_user
Thanks @curveship for replying.

C# is more capable and saner than Java and to tell you the truth, C# has been
the first static language that I used with joy. If it weren't for Scala, I
would have prefered C#. Skipping over platform limitations and considering
just the language, I still prefer Scala any day of the week.

Let me outline the differences between the examples I've posted and the ones
that you did.

    
    
        list.sum
    

In Scala this example outlines the usage of the Numeric[T] type-class. A type-
class [1] is like an implemented interface in Java and C#, except that you
don't need the ability to modify the type in order to implement this
interface. See the Expression Problem [2].

In C# or Java, the concept of adding numbers is not being exposed as a
standard interface. In C# the + operator is a static member, so it cannot be
part of an interface (in Scala there is no such thing as static members btw).
In Java the + operator is something special reserved for primitives. In Scala
it doesn't matter how + is exposed, because you can implement a Numeric[T] for
whatever type you want and hence you can treat any type you want as a number
in generic data-structures and algorithms. If you implement your own Complex
type or RomanNumeral type, no problem, as you can sum them up as long as you
also implement Numeric[Complex] and Numeric[RomanNumeral].

In contrast, for your sample to work, the C# engineers had to add extension
method overloads of IEnumerable.Sum() for each number in the system. But what
do you do if you want to add your own number implementation, like a BigDecimal
or something with better performance for your use case. Yes, in Scala it
works, without hacks.

    
    
        list.sorted
    

In Scala, this uses the Ordering[T] type-class. Again, it's an open interface
to which you can add any type, even types whose implementations you don't
control. Also, the C# version is NOT statically type-safe, while the Scala
version is. This compiles just fine, but fails at runtime ...

    
    
        var list = new List<object>() { 1, "2", 3 };
        list.Sort()
    

Want to guess why that is? Because in C# you also can't add extra method usage
restrictions on the generic types of a class. So you can't have a "sorted"
method on a class that only works in case your initialized T is part of the
Ordering type class. I.e. you either have to restrict T to some interface
site-wide (so you can't use your class for anything but things that can be
sorted), or you have to rely on runtime reflection / runtime errors.

There is value in doing this, rather than doing it with an OrderBy that takes
a function for the comparison operation. First of all, many types have
_natural ordering_ and the inability to implement this means that you can't
express _natural ordering_ in your language. Also, these are really simple
examples. You don't see the value of this until you start working with these
tools in the real world in more and more complex problems.

Your examples with the HashSet and the Dictionary are wrong. They aren't doing
the same thing.

In Scala, a BitSet is a Set of integers that stores those integers efficiently
in-memory. It's a different implementation from a regular HashSet, specialized
on storing integers. When mapping over a BitSet, you want the result to still
be a BitSet, otherwise you lose its properties. But the result can't be a
BitSet if you're not returning integers in that mapping function. You missed
the point of my example.

So basically Scala allows you to override the returned type, based on the
given mapping function and it does so in a statically type-safe way, as the
real return type is noticed by the compiler (i.e. it's compile-time ad-hoc
polymorphism, not runtime). And best of all, the mechanism is pluggable so you
can always override the given defaults.

Of course, you can have dozens of utility methods that allow you to escape the
limitations of your language. You can build anything on top of Java and C#
actually.

It's still death by a thousand cuts and in case you can't see it, ask yourself
why you haven't done functional programming in C#, even though it is possible
to do so. And yes I know, some people do it, just like some people are doing
OOP in C.

[1]
[http://en.wikipedia.org/wiki/Type_class](http://en.wikipedia.org/wiki/Type_class)

[2]
[http://en.wikipedia.org/wiki/Expression_problem](http://en.wikipedia.org/wiki/Expression_problem)

~~~
curveship
No, thank _you_ for replying. I'm not sure I deserve it -- a little while
after I posted, I realized that my C# in many cases missed the point, and in a
couple was flat out wrong (BitSet). That led me down a path of researching
more about Scala ... and after some neat reading, my "gotta get back to real
work" timer dinged, and I never got back here to clarify. So thank you for the
insightful and detailed reply. My post's main value was in eliciting yours :).

------
cygwin98
I was being snarky in my another comment and was downvoted as hell. Oh well, I
am OK with that. However, I can't help thinking of the failure of Scala at
Yammer, which caused a big buzz two years ago [1] and [2]. From the blog posts
I've read about Scala and Yammer, they planed the transition to Scala
carefully and with passion, and they switched away from Scala in merely one
year.

Has the situation changed totally? I don't know. Maybe Scala has improved
significantly since then and all the pain points back then have been addressed
nowadays. I'd like to be enlightened on these factors.

[1]
[https://news.ycombinator.com/item?id=3297750](https://news.ycombinator.com/item?id=3297750)
[2]
[https://news.ycombinator.com/item?id=3292555](https://news.ycombinator.com/item?id=3292555)

~~~
bad_user
Joda's impression of Scala is very insightful and it shows that Joda is an
awesome engineer. But first of all, Joda was also careful to explain in his
second post (the one that was meant to go public) that Scala didn't work in
their _own freaking context_ and that all platforms suck depending on
perspective.

Let me address some of the points raised ...

(1) Scala is difficult for Java developers to learn, because Scala has new
concepts in it, like type-classes or monads and these concepts are exposed
within a _static type system_. Are these concepts useful? After using Scala
for the last 2 years, I can bet my life on it. However there's a learning
curve involved and Scala is definitely more expensive to learn than Java.
That's why context matters. If a company has the resources to invest in this
learning curve, then it's worth it, otherwise it isn't.

(2) Scala is not complex as Joda mentions. Not in the original sense of the
word "complex". Scala is in fact pretty simple, as the features exposed are
pretty well thought and orthogonal. But those features are powerful. The real
complexity comes from its interactions with the JVM, as they had to do some
compromises. It's important however to differentiate "easy" versus "simple".
See this awesome talk by Rich Hickey ...
[http://www.infoq.com/presentations/Simple-Made-
Easy](http://www.infoq.com/presentations/Simple-Made-Easy)

(3) As any new community, the Scala community started by doing lots of
experiments, pushing its syntax and its type-system to its limits, going as
far as to use features that aren't fully baked yet. This happens with
languages that try new techniques. But as the community is maturing, those
problematic edge cases get flushed out and there are fewer and fewer libraries
that go nuts, as more and better best practices emerge.

The favorite example of Scala bashers is the Dispatcher library, that
originally went nuts over operator overloading, but that was rewritten [1] and
these days Scala libraries are actually quite sane and elegant.

Also, there's nothing wrong with the existence of experimental libraries, like
Scalaz. Contrary to public opinion, it's not in wide usage in Scala projects,
it's very useful for people that need it and such projects end up exposing
weaknesses and pushing the language forward. The existence of libraries like
Scalaz is a virtue and really, people work on whatever they God-damn please,
you can't blame a whole community for it. Joda used it as an example for the
dramatic effect, in a private email, OK?

(4) SBT's syntax looks awful until you get the hang of it, because it uses
operator overloading to achieve a declarative syntax for building an immutable
data-structure that describes the build process. This syntax will likely get
fixed, but it's also a really pragmatic tool and I now use SBT even for Java
projects and I miss it when working with Python/Ruby. There's also a Maven
plugin as an alternative and Joda mentions the stalled Maven plugin and its
lack of support for incremental compilation, however that's no longer true.

(5) Joda mentioned problems with the upgrade cycle. Scala 2.8 should have been
in fact called Scala 3.0, as the changes in it were dramatic, which is why
when Joda wrote that email, many companies were still on 2.7 and the upgrade
to 2.9 scared the shit out of people. However, things are a lot better these
days. Minor versions no longer introduce backwards compatibility issues, so if
you're using 2.10.3, then you won't have any problems with libraries compiled
for 2.10.x and you can even use many libraries compiled for 2.9. It's much
more stable.

In regards to why is backwards compatibility affected by new versions, well it
happens because the standard library hasn't stagnated. Java has awful pieces
of junk in its standard library that were never pulled out or redesigned, but
as a new and still growing language, Scala cannot afford to freeze its
standard library. And we are not talking about major changes here, just a
simple addition of a method to a base interface can trigger the necessity for
recompilation.

But these days there are a lot of useful libraries with multiple contributers
and that get compiled for newer Scala versions as soon as those versions come
out. And personally I haven't bumped into problems because of it, as upgrades
for the stack we've been using have been really smooth.

(6) At my old startup, we've built a web service that was able to handle
30,000 requests per second, with responses being processed in under 10ms on
average, using only 10 EC2 h1.medium instances. Actually we started by running
that code on 10 Heroku dynos, but then moved to AWS for more control. And the
code was written largely in a functional style, using Scala's standard
immutable data-structures all over the place. People that complain about
performance degradation do have a point in the proper context, but people that
bitch about it based on other people's opinions or based on silly benchmarks,
well such people don't know what they are talking about, especially since
immutable data-structures help a lot with multi-threading woes, giving you
room to efficiently use the resources available.

Actually, I think Joda's actions of eliminating Scala's data-structures or
closures from the whole codebase, were excessive (and I'm trying hard not to
say silly).

(7) It's all about the libraries. If every library you're using is a Java
library like Joda mentioned they did (as Scala makes usage of Java libraries
easy), then you might as well use Java as a language. But if you're using
Scala's strengths and libraries built with Scala, with a Scala-ish interface
and you like those libraries (e.g. Play, Akka), then Scala suddenly becomes
inexpensible.

(8) Stephen Colebourne, for all his talent, is a douche-bag.

[1] [http://code.technically.us/post/54293186930/scala-
in-2007-20...](http://code.technically.us/post/54293186930/scala-in-2007-2013)

~~~
the_watcher
>> Scala is difficult for Java developers to learn, because Scala has new
concepts in it, like type-classes or monads and these concepts are exposed
within a static type system. Are these concepts useful? After using Scala for
the last 2 years, I can bet my life on it. However there's a learning curve
involved and Scala is definitely more expensive to learn than Java. That's why
context matters. If a company has the resources to invest in this learning
curve, then it's worth it, otherwise it isn't.

This sounds similar to some of the stories about Twitter's switch to Scala.
The developers who liked Ruby loved Scala, while the rest of the team stuck
with Java, I believe.

~~~
dxbydt
> The developers who liked Ruby loved Scala, while the rest of the team stuck
> with Java, I believe.

Not true. But hey, its a free country. You can believe whatever you want.

~~~
the_watcher
Or, instead of telling me that it's a free country, you could be productive,
correct me and (ideally) provide a source. I'm happy to learn something new or
have a mistaken belief corrected.

~~~
dxbydt
Its just my sense when I look at the git status of most of the private repos.
Can't provide a source, sorry.

------
V-2
Scala is not that distant from functional aspects of C# such as LINQ...
certainly way more nice than Java itself, even including their latest LINQ me-
too propositions...

Having C# background, I only took a look at Scala and I liked it at first
glance, I wouldn't mind learning it some day.

See
[https://gist.github.com/samskivert/5278311](https://gist.github.com/samskivert/5278311)

And although that's off-topic, while we're at it:
[http://www.digitalmars.com/d/archives/digitalmars/D/Java_str...](http://www.digitalmars.com/d/archives/digitalmars/D/Java_streams_Vs_LINQ_Vs_D_193598.html)

~~~
balakk
As somebody who moved from F# to Scala for a project, my major gripes:

\- Lack of a yield statement equivalent(in c#/f#). I know, there are probably
a dozen alternatives such as continuation passing etc, but none so convenient.

\- It's a massive language. Operator overloading seems to be highly
encouraged, and library authors go to town with it. The Parser combinator
stuff itself is enough to make one go dizzy, if you aren't familiar with it.

\- Tools: Idea is OK, but not a patch on VS and F#. Compilation times are
practically instant for small/medium projects in VS land - here it's glacial,
even on a recent MBA. SBT is great, when it works.

On the + sides:

\- If you know f#, you practically know Scala - well, enough to be instantly
productive.

\- Play 2 is very close to ASP.NET MVC. This was a welcome surprise, and a
relief as well.

\- It's such a relief that whatever you build will run on anything with a JVM.
Xamarin/Mono is awesome, but those guys are still a little obscure, and a
tougher sell than a JVM-based language.

~~~
sandGorgon
I wish that sbt had the ability to work with exploded scala source
dependencies, instead of insisting everything work in the JAR world. I am used
to working with rubygems/pip and love the ability to go and screw around in
the libraries. When I'm ready to package, I can run "sbt assembly" instead and
it can take a little longer to build a packaged JAR.

Clojure/leiningen has the same issue (although there are ways to get the
source itself as well [1], but nowhere as clean as rubygems).

[1] [http://stackoverflow.com/questions/14395413/how-do-i-make-
le...](http://stackoverflow.com/questions/14395413/how-do-i-make-leiningen-
download-the-source-of-a-required-library)

~~~
ssmoot
Could you elaborate what you mean by "insisting everything work in the JAR
world"? SBT works with git submodules, git sources, other URI or File sources
for dependencies. So you must mean something else, but I'm having trouble
parsing.

~~~
sandGorgon
Rubygems and Python packages are source packages. When you used the package
management tools in those languages, these packages are downloaded to
somewhere in your path and used within the code you are writing.

In fact in Rubygems, you can point to a Ruby git repository (like github) and
have dependency management happen from there - no necessity for a centralized
repository (like maven.org). This is how you share a ruby gem -
[http://guides.rubygems.org/publishing/#sharing_source_code](http://guides.rubygems.org/publishing/#sharing_source_code)
. I can now specify in my Gemfile to pull a particular branch/tag/version of
the gem directly from revision control.

The same thing happens with SBT or leiningen, except that they are JARs. I
believe this comes from using maven as the backend. Therefore, either you need
to setup a local repository or publish to Sonatype or some other place. This
is how you share your scala code - [http://www.scala-
sbt.org/0.13.1-SNAPSHOT/docs/Detailed-Topic...](http://www.scala-
sbt.org/0.13.1-SNAPSHOT/docs/Detailed-Topics/Publishing.html) . Remember,
publishing means packaging into a JAR. If you were too lazy to package your
code into a jar and publish it ... well, then it's cumbersome to work directly
from the repo.

I really, really wish scala/clojure would enable the ability to work directly
with source as packages - the big, big reason why it's not is the maven
backend.

~~~
ssmoot
> In fact in Rubygems, you can point to a Ruby git repository (like github)
> and have dependency management happen from there - no necessity for a
> centralized repository (like maven.org).

That's a feature of Bundler, not Rubygems.

> The same thing happens with SBT or leiningen, except that they are JARs. I
> believe this comes from using maven as the backend. Therefore, either you
> need to setup a local repository or publish to Sonatype or some other place.

I don't follow. I mean, you don't need to setup a local repository anymore
than you need to setup a local Rubygems repository.

Checkout this anonymized and trimmed up `build.sbt` for the project I'm
currently working on:
[https://gist.github.com/sam/7162604](https://gist.github.com/sam/7162604)

It's got a number of JAR dependencies, followed by project references to
projects on the local filesystem as well as Github, neither of which are JARs.

In fact, because I don't have much clue how to distribute my own libs on
maven.org, I don't bother. If you want to use it, just drop the URI into your
build.sbt and away you go.

Is this not what you're looking for?

If you want to create a JAR (say for deployment), and you're not working with
a Play application, and you don't want to publish the JAR to a repository,
that's cool too. Just run `package`, or use the sbt-assembly plugin. Just
takes a couple seconds.

SBT is basically Rake + Bundler + RVM + Guard + RDoc all in one. I'm not aware
of anything that those do that SBT can't. It's pretty easy to pick up as well.
People complain about it, but I've personally invested a lot less time
learning SBT than I have in those tools and feel like I'm probably as
proficient at this point.

~~~
sandGorgon
@ssmoot - this is extremely interesting. Thanks for pointing out RootProject.
After doing that, are you able to use them just as seamlessly as declaring it
in libraryDependencies. For example, "sbt assembly", etc. work just as fine ?

~~~
sandGorgon
BTW - I double checked now, your code only works on sbt 0.13. My previous
version of 0.12.4 doesnt work with it.

So, I guess it's cool that this kind of stuff is working now!

~~~
ssmoot
It works as you'd like yes.

The reason my code is SBT 0.13.x specific is just because .sbt file syntax was
robustified in 0.13.x to cover most cases you'd previously need a Build.scala
for.

You can do the same thing easily in 0.12.x, it just requires you do it in a
Scala class that extends sbt.Build instead. In fact, that's what my code was a
few days ago. I only just ported it to the new SBT 0.13.x syntax since the old
syntax continued to work. We'd been using 0.13.x for awhile, I just hadn't got
around to porting the build from being spread across build.sbt and
project/Build.scala to consolidating to just the build.sbt.

The docs for SBT are actually really thorough, but to me at least, a bit hard
to get into. I blame the theme at [http://scala-sbt.org](http://scala-sbt.org)
and the fact that there's no obvious "from basics to mastery" guide. The
website feels more like a manual. Going deeper than you might want pretty
quickly, but before you've got the breadth of it down. If that makes any
sense?

That said, if you end up _having_ to work with SBT for your day-job, you end
up picking everything up pretty quickly once you're forced to. It's a lot more
consistent than anything on the Ruby side. You may have a few syntaxes to
choose from to add a libraryDependency for example (single, Seq, special
operator), but the mechanics are all the same and directly exposed to you.
You're building a definition for the build, not actually performing work, so
as long as that dependency sequence ends up with the graph it needs you'll get
the result you need. There's not a lot of magic to it. Despite peoples
complaints about the operators (which honestly, there's only two or three
you'll run into regularly, and it's not black magic, just look them up or
Command+B in IntelliJ to see what they do).

------
flipchart
I'm more and more confused by everyone that bashes the MS stack because it
doesn't have the OSS community that the other stacks do. I'd really love to
hear what OSS projects there are for the Python/Ruby/JVM/etc languages which
don't have equivalents for .Net. Maybe I'm doing something wrong, but my
experience on .Net has always been such that I can find a project to meet my
needs. Sure my target OS is Windows, so the OS interop is not an issue for me,
but for my day to day needs, there is always a solution to be found in the
.Net OSS community.

~~~
subsection1h
What are the .NET equivalents of The Ruby Toolbox[1] and pluginGeek[2]?

Also, there are more than 11 times as many Ruby projects at GitHub as there
are C# projects[3] (even though .NET developers vastly outnumber Ruby
developers).

[1] [https://www.ruby-toolbox.com/](https://www.ruby-toolbox.com/)

[2] [http://www.plugingeek.com/ruby](http://www.plugingeek.com/ruby)

[3]
[https://github.com/search?q=stars%3A%3E0](https://github.com/search?q=stars%3A%3E0)

~~~
flipchart
To be fair, those appear to be pages which list existing packages, which is
not what I was commenting on in my OP. However, I would suggest one of the
closest equivalents would be nuget[1]. I think that the history of the
languages should be mentioned: Ruby 1.0 was released in 1996[2], while the
beta of .Net was only released in 2000, and 1.0 only in 2002[3]. Thus one
could perhaps say (maybe unfairly, but nevertheless) that Ruby has had more
time for OSS projects to develop. I"m not saying that that is the reason for
the abundance of Ruby projects on Github, but I don't believe that the
relative lack of .Net projects on Github implies that the needs of a .Net
developer is not met by the current OSS offerings.

[1]: [http://www.nuget.org/](http://www.nuget.org/) [2]:
[http://en.wikipedia.org/wiki/Ruby_%28programming_language%29](http://en.wikipedia.org/wiki/Ruby_%28programming_language%29)
[3]:
[http://en.wikipedia.org/wiki/.NET_Framework](http://en.wikipedia.org/wiki/.NET_Framework)

------
DanielBMarkham
Long article. Good stuff for those looking to make a similar change though.

I remain a fan of F# on Mono. Love the FP paradigm, love all the easily
accessible libraries, and love that it's all free.

Scala would probably be a close second choice for a development shop, though.
I personally would move towards pure FP with Haskell or Erlang, but staffing
becomes too much of an issue to scale that out. I don't think they have that
issue with Scala.

~~~
dons
> pure FP with Haskell or Erlang, but staffing becomes too much of an issue to
> scale that out

Does it? We've hired several dozen Haskell developers at this point, and we
always get far more applicants than available positions.

~~~
DanielBMarkham
I'm still collecting data on this. Along with how dev practices scale out in a
purely functional environment.

I try to stay ahead of the game on these things. If I'm still collecting data,
either I'm missing some major sources or the overall market isn't ripe for it
yet.

------
jimbones
The chart included and referenced is just of all the most obscure JVM
languages. Look what happens when you add Java to that list [1].

[1]
[http://www.indeed.com/jobtrends?q=scala%2C+groovy%2C+clojure...](http://www.indeed.com/jobtrends?q=scala%2C+groovy%2C+clojure%2C+jruby%2C+jython%2C+java&l=))

~~~
yelnatz
Wait, what?

[http://www.indeed.com/jobtrends?q=scala%2C+groovy%2C+clojure...](http://www.indeed.com/jobtrends?q=scala%2C+groovy%2C+clojure%2C+jruby%2C+jython%2C+java%2C+go&l=)

~~~
arscan
apparently the matching algorithm isn't particularly smart... those are just
the job postings that have the word "go" in it?

[http://www.indeed.com/jobanalytics/jobtrends?q=scala%2C+groo...](http://www.indeed.com/jobanalytics/jobtrends?q=scala%2C+groovy%2C+clojure%2C+jruby%2C+jython%2C+java%2C+a&l=)

edit: yeah, it isn't intended to be programming specific: _" Indeed.com
searches millions of jobs from thousands of job sites. This job trends graph
shows the percentage of jobs we find that contain your search terms."_

------
jroseattle
I always find the most interesting aspect of "we changed from X to Y" to be
about why change is happening, how it is managed, etc. Context is everything,
but the process described by the OP sounds like standard, top-down decision
making. The OP certainly gets to it by the end of the article.

I've gone through this process (tech stack shift), both as a recipient-of-
change as well as a catalyst-agent-of-change. Remarking on the email that the
OP's CTO sent to everyone, these statements stick out:

    
    
      "...A technology choice will be made appropriately for the needs as we see them..."
      "...no hard decision has been made about our future languages or technologies..."
      "...Less C#/.Net, more Scala/Java/Open Source..."
    

The CTO likely has very good intentions, but this is an implied technical
decision that's already been made. He references "we" in the email to imply
group participation in decision-making, but it is very much a you-are-with-us-
or-you-are-against-us definition. To wit, those who didn't buy in were shown
the door.

Having been the guy in that chair, this is not an easy thing to do. Being
handcuffed by prior technology decisions is not only difficult, it's often
untenable. Usually, the people hired into those leadership roles often have to
resolve the decision making issues that got the company to that point. That
said, here's what I've learned in terms of how to go about these types of
long-term changes:

\- As the CTO, take full responsibility for introducing change. This is not
taking credit, but instead being the owner of the decision and ensuring it
succeeds. Support the team, and encourage them to openly question the
decision, so that those questions can be addressed. The CTO's email above
makes a change decision requisite, but it's up to everyone else to make it
work. To anyone who's worked in a potentially charged environment, that's a
typical setup for scapegoating. As the lead dog, don't put your team in that
position. If your decision comes off as playing-the-executive card, you have
failed.

\- As a team member, embrace change but make it completely objective. Be open
to change, but only as it helps to reach a goal. Evaluate technologies after
using them, comparing them, and finding what's good and what's not-so-good.
Make your personal productivity the least-influential point of analysis. If
you can't find a way to assess something honestly because of your preference
for operation, you're doing yourself a disservice. Be productively critical of
technical decisions -- your own, and others.

~~~
seglo
OP here. Thanks for your feedback.

The Technical Strategy email was sent out about midway through our adoption of
Scala. I didn't explain the dynamics of our larger development department much
in my post, but at that time we were essentially two teams in two different
physical locations. The location I was at had already seen success with a few
projects in Scala (BeetleJuice and the beginning of our API). Our CTO thought
it was time to set a clear direction for the future of development department
after an analysis of our current situation and discussion with members of the
wider team.

Nobody was shown with the door. I'll admit there was some resentment, but
nothing a well adjusted developer couldn't get over.

As I mentioned at the end of the post, one thing I would do differently in the
future is make a stronger effort around consensus building about the decision.
However, the Scala change started off as something small and organically grew
into a legitimate alternative to the Microsoft stack. Going into our first
project (BeetleJuice) we didn't anticipate the impact it would ultimately have
on the organization.

~~~
jroseattle
Cool, thank you for the context (which is everything in these situations.) And
apologies if I implied things that weren't the case; just my
reading/interpretation of the post.

My primary criticism of the CTO's email was that it didn't seem direct, which
I've found is the most important aspect when communicating in that role. With
all the intentions of being inclusive and providing avenues for flexibility,
I've found the interpretations are most often viewed as ambiguous. The higher
up the chain, clarity becomes so much more important. Live and learn, in my
case.

In terms of consensus building, I had to do the same thing. For me, we
accomplished that through objectivity -- we were constantly evaluating. We
asked team members who liked technology A to provide brown-bag sessions on
technology B. We had "skin-the-cat" sessions, where we would have a team of
two or three implement a somewhat basic solution in three different ways (you
know, nine ways to skin a cat....). It took some doing, but essentially what
we did was enable developers who might not be so flexible to considering
alternative technologies to get comfortable with those by spending time using
them and providing an opportunity for them to say "this sucks" or "this is
awesome". The team, as it was constructed, really responded well to this. Not
only did they feel involved, but they were also tacitly preparing themselves
to accommodate a shift, no matter which way it went. I was really surprised at
how well the team became empowered; it was not something I was expecting to
see.

Your CTO definitely had a tricky process and conversion to balance. I
empathize, it's so easy to do a mediocre job with things that one wouldn't
expect to matter quite so much.

------
rodh257
Really interesting article, thanks for writing it. I'd like to hear a bit more
about the reasoning for the whole company swap, what problems you were having
with C#/Microsoft stack. The impression I got was that you tried out something
new for a small project, liked it and liked the idea of OSS in general and
then started doing everything in that way, is that all there was to it? What
other factors came into the decision on a whole?

~~~
300bps
I found the article very interesting as well but honestly it reads like a
nightmare.

Company heavily invested in X. New CTO comes on board, gives the technology
stack a "hard look" which is often a euphemism of, "I wouldn't have chosen
that technology, so you must change." Large portion of the existing developers
leave, new developers (including OP) come in. New developers base technology
decisions on what's best for their career, not necessarily what's in the best
interests of the company.

After all, in the Why Scala section they plainly state they knew little of
Scala and there's a large section about what languages are hiring.

And as you said, if there is time spent on what made them decide to switch
away from the Microsoft stack, it is buried to the point I couldn't find it in
two read-throughs of the very long article.

~~~
taude
I don't see having some of the old-school developers leaving the company and
being replaced by potentially better talent as a bad thing. One signal for me
is that I hire smart people because technology changes. It's a red flag for me
when people stop learning or lose the desire to add to their tool belt. Some
people can't adapt.

But, hey, the world still needs Cobol programers, so...

~~~
300bps
_old-school developer_

With all due respect, you obviously know nothing about the Microsoft stack.
These people were not using VB6. How do I know? From the article:

 _new projects chose a base technology stack of C#, ASP.NET MVC_

ASP.NET MVC is not an "old-school" framework. It was first released in 2009!!
That's four years newer than Ruby on Rails. It's ridiculously actively
developed as well as being open source. It's currently in version 5 which was
released October 17, 2013 which was... OMG 8 days ago.

I continue to be shocked at how little most HN developers know about the
Microsoft stack and how much innovative work has been done since they last
looked at it in 1998.

~~~
taude
Ha, I worked a lot with ASP.NET MVC 2 & 3! By old-school, I was referring more
to the lockin of the MSFT stack than the specificity of a particular library.

(When we moved over to a Python-based solution, we ported from MVC
3/SqlLinq/MS SQL Sever stack)

EDIT: Oh, and were had a lot of good stuff going on using
ServiceStack....loved that set of libraries.

~~~
_random_
"moved over to a Python-based solution"

My condolences...

------
powertower
Reading this thing it just gets worse and worse as you go down.

All they've done was get rid of smart people, create complexity, and embraced
a culture of buzzwords.

A complete disaster in the making.

And for what?

~~~
seglo
I'm sorry you interpreted it that way. It wasn't a disaster and internally the
general consensus was that it was a great success. I've made some amendments
to my post to provide some more context to our initial process change.

Thanks for the feedback.

~~~
powertower
I might have read too fast, and might have some serious bias here because I
actually prefer C# and .NET over anything else (and like leaving performance
issues to be dealt with caching / rather than introducing complexities) - but
it seems like you went from using a familiar, predictable, and very popular
stack to embracing a dozen different smaller and lesser known technologies all
full of interoperability issues, idiosyncrasies, and each requiring very
specific domain knowledge.... All because a single person at the top is
ideological about OSS instead of practical about solutions.

Though maybe that's what was required to get a buyout or merger, in which case
that's what you needed to do due to the environment/culture of the other
party...

[http://www.empathica.com/press-release/mindshare-
technologie...](http://www.empathica.com/press-release/mindshare-technologies-
acquires-empathica/)

~~~
seglo
The process change was top down, yes, I think that's the only way it could
have succeeded IMO. However, the Scala change was bottom up and by developers.

The Mindshare acquisition is a recent event that postdates our technology
change (Sept 19).

~~~
_random_
How did those "developers" choose that change if a lot of them disagreed and
left later on? There had to be 1-2 leaders who happened to be Scala-fans.

~~~
seglo
We had two teams in separate physical locations. We worked together
occasionally, but most of the time projects were fully staffed at one location
or the other. The Scala change was fully supported at one of the office and
not the other.

It wasn't too difficult to convince the CTO that Scala was the right choice.
He's a big OSS advocate. From a technological perspective we were trusted to
make the right decision.

I think what really sold the the rest of the management team was how much
money we could save in licensing fees.

~~~
powertower
> I think what really sold the the rest of the management team was how much
> money we could save in licensing fees.

Come on, ...Microsoft licensing costs are a rounding error compared to the
cost of developers.

~~~
_random_
Well, some people are trying to save money even on free soda for developers -
there were articles on HN.

------
pkmishra
Thanks for Sharing. I also learned scala after working on .net stack for more
than 7 years and I am not disappointed. I really like C# but if I have to
learn JVM based language I would definitely go for scala.

------
Touche
Some people just like complicated things.

~~~
trailfox
Fortunately they saw the light and moved off of the complexity of C# then...

~~~
Touche
From one complicated thing to another.

~~~
trailfox
Java is simpler than both C# and Scala, yet it is more disliked than both (see
recent HN polls).

------
programminggeek
Scala is awesome, but I wish it had a faster compiler. The web dev experience
of 2-5 second code refreshes is painful IMO. If they got it to less than a
second Scala might be my ideal language.

------
mwexler
Besides an interesting read, I laughed all over again at the video on the
bottom. Worth a reminder if you didn't catch it the first time or haven't
"Java 4-Ever" before...

~~~
seglo
Definitely a nugget of Java humour worth preserving and sharing. Glad you
liked the post :)

------
cygwin98
And one year later, a blog post titled "Why we moved to Java from Scala" or
"Why we moved back to .NET" appeared on HN and havested hundreds of votes.

Just kidding.

~~~
CmonDev
It actually makes a lot of sense, because the change to Scala was triggered by
"Simon". Another Simon will come and bring Python or something even worth.

~~~
seglo
That's incorrect. The technology change was bottom-up, by developers. Simon
didn't intervene with his technical direction until we already had a couple of
Scala projects under our belts.

~~~
_random_
I you saying you had a vote and some developers (C#) disagreed and there was a
minority of them?

~~~
seglo
There wasn't really a vote, but we did have a group forum where objections
could be raised. People who weren't onboard with Scala were in the minority
and most of the arguments raised didn't hold much water.

------
taude
It sounds like you might not have worked there yet, but it'd be great to hear
how some of "convincing of management" went. Was it mostly the new CTOs vision
who was able to convince business?

(I speak as someone who moved some web apps off a .NET stack to OSS, but that
was for a small company's single product)

~~~
CmonDev
".NET stack to OSS"

This does not make any sense whatsoever since e.g. ASP.NET MVC is OSS. Why
don't you say ".NET stack to JVM stack"? Because it doesn't make your choice
sound intrinsically better?

~~~
dragonwriter
> This does not make any sense whatsoever since e.g. ASP.NET MVC is OSS.

ASP.NET MVC is an open source framework, sure, but the .NET _platform_ , as
such, is not OSS. (Some parts of the MS .NET ecosystem are open source, and
there are open source alternative implementations for some parts, but that
doesn't make the platform OSS.)

------
kylequest
This statement is pretty funny given the focus on intentional transition to
Scala: "So, no hard decision has been made about our future languages or
technologies, and there are no rules that all new development must happen in
Scala."

------
MrDosu
Why couldn't they have written "Web Application" in the first paragraph...

------
PaulHoule
That's going from the frying pan to the fire.

~~~
woof
Faster and hotter?

------
petermartin
I really liked this article. I am just now starting to learn Scala. Great
post.

