
Scala's version fragility makes the enterprise argument near impossible - LiveTheDream
http://lift.la/scalas-version-fragility-make-the-enterprise
======
stephenjudkins
This is the largest practical issue I've had with developing a Scala project.

One major cause of the problem seems to be that Scala is stuck with the
"precompiled bytecode in JARs" model of software distribution; while this may
be appropriate for Java (where core developers go through _great pains_ to
maintain compatibility between versions) it seems like a poor fit for Scala.
Not only do the core library APIs change between versions, but the conventions
used by the compiler to "link" Scala code together are also altered, for good
reason.

An alternative distribution mechanism that's more resistant to these kinds of
issues might make sense. Even just distributing source code and build files
(like, say, Ruby) could be a better solution. Building dependencies might be
slow at first, but centralized caching could address this. Another solution
might be an intermediate representation (after type-checking and other
compiler phases, but before "linking") that gives the Scala compiler the
knowledge that's missing from compiled class files.

Even namespace-mangling or class-loader magic could be helpful here. Loading
multiple versions of a single dependency should in theory be possible, since
OSGi manages it.

Sadly, there seems to be a pretty large investment by the Scala community in
the whole Java/Maven/JAR ecosystem. It'd take a concerted effort by many
people to create a more robust solution.

~~~
keturn
Thanks for this comment, as not yet having got into Scala myself, I had to
read between the lines to figure out what the problem was here. I thought "Ok,
you don't have binary compatibility, so recompile your dependencies. Sure, it
may extend your build time somewhat, but it shouldn't make it impossible to
use dependencies or test pre-release builds..."

Oh, you don't _have_ the source? Well, there's your problem.

A good reminder of what some of us take for granted in our development
environments, I guess.

~~~
bad_user
You have the source, but the Maven model used by the whole Java ecosystem for
distribution/management of packages doesn't work like that. And replacing it
with something else isn't a small feat.

~~~
technomancy
> You have the source, but the Maven model used by the whole Java ecosystem
> for distribution/management of packages doesn't work like that.

Oh, I don't know about that. Clojure does this; shipping source is default,
and shipping bytecode is only done in cases of certain Java interop forms.

But Maven has a notion of "classifiers" that seem to be a good fit for this.
I've seen them used to specify things like "this is the JDK 1.4-compatible
version", so I wonder why Scala library authors haven't used them to publish
the same artifact compiled against multiple versions of Scala.

~~~
dantheman
Does clojure use maven for shipping source around?

~~~
technomancy
Yes, the source is just stored in jar files, which can contain any kind of
files.

~~~
Luyt
'Jar' files are actually zip files with a '.jar' extension. On my Windows
machine I have .jar associated with a zip application, very handy to quickly
inspect its contents.

------
jzoidberg
David carries quite a lot of weight in my book (we use Lift in production).
Despite the negative title he makes a very good point and points out that this
is entirely solvable. This falls under the "growing pains" category for Scala.
I hope we get this right soon.

------
jbarham
Go is also moving very fast and they often make backwards-incompatible changes
to the libraries (and sometimes the core language) but upgrading code can be
done automatically 95% of the time using gofix, which programmatically
rewrites code to conform to the latest language and library updates.

<http://blog.golang.org/2011/04/introducing-gofix.html>
<http://golang.org/cmd/gofix/>

~~~
angelbob
Wow. So it just rewrites it to the latest version, with (according to what you
just said) about 95% accuracy?

Does that make other people feel queasy, or just me? Like, that feels like a
really bad idea to me. What happens if the result only _mostly_ works?

~~~
sausagefeet
Python has been doing this for awhile, hasn't it?

~~~
DasIch
Python has broken backwards compatibility with Python 3 and tools were written
to translate 2.x code to 3.x but that is not at all a regular thing.

~~~
re
I would say that Python breaks backwards compatibility in small ways for minor
releases semi-regularly, such as adding new keywords (with, yield), removing
little-used standard library modules, and removing support for raising strings
as exceptions, but always provides deprecation warnings for a release first
and (for new keywords or syntax) allows early opt-in with "from __future__
import"

<http://www.python.org/dev/peps/pep-0387/>

------
nobodysbusiness
A recent Typesafe blog post was written in response to the last Scala-
complaint furor: <http://blog.typesafe.com/getting-down-to-work>

One of the points mentioned there is binary compatibility. The Typesafe people
are working on it, so hopefully we'll see some improvement over the next year
or so.

------
gldalmaso
>> An attribute of Scala is that the Scala compiler generates fragile byte-
code.

Can anybody tell me if this would be true to any JVM targeted language, or is
it a specific issue of the Scala compiler?

I'm asking because I just recently decided to abort learning Scala in favor of
Clojure.

~~~
cemerick
As long as you're not AOT-compiling Clojure (i.e. producing class files, which
is generally entirely unnecessary in Clojure except perhaps as the last step
just prior to application delivery / deployment), you'll generally be very
happy.

The 1.2 -> 1.3 transition has had a couple of bumps, but nothing compared to
the pain I recall from my time across Scala versions years ago. FWIW, I have
code I wrote years ago against pre-1.0 Clojure that still sits in jars in my
Maven repository that is loaded and run in 1.4 alphas today. Backwards
compatibility is mostly quite good.

~~~
CountHackulus
In the JIT world, going .java->.class is not AOT. You may call it that, but
it's essentially just source->source conversion.

What AOT really is, is when you first run a java program with certain flags,
and some compilation starts, a special AOT version, which is usually a low
optimization level which is saved to disk with some relocation markers.

This helps with startup time in subsequent runs as the compiler can just load
its baseline pre-compiled classes straight into memory instead of having to do
a bunch of grunt work during startup. Once things get going, the compiler can
much more quickly work its way up to the higher optimization levels on the
sections of code that need it.

THAT is what AOT means from a jitted language perspective.

~~~
gtrak
Java bytecode is what we mean when we say 'compile'. AOT is 'ahead of time'
with regard to clojure, since it compiles to bytecode at run-time normally.

------
Uchikoma
I've raised this issue one year ago

<http://codemonkeyism.com/scala-unfit-development/>

There are two forces at play here:

a.) Binary fragility of Scala

b.) A community that upgrades libraries (your dependencies) to every new RC
candidate - even when only fixing bugs. So if you have a show stopper bug you
need to upgrade to the newest RC breaking all your other dependencies.

Either a. and b. would be no major problem, combined it throws you into a
dependency hell for days hunting for dependencies that work.

Hope they resolve this issue. People promised a solution in the blog comments,
then promised a solution a year later as the issue surfaced again on the
#typesafe blog.

------
donjigweed
Really, it is just becoming increasingly difficult to see how proponents of
Scala can credibly claim any kind of developer productivity gains. I can only
imagine how much time your average Scala developer spends solving problems of
accidental complexity that simply do not exist in more mature languages.

~~~
noelwelsh
As a Scala developer, let me shed some light on this. In the last few months I
have spent exactly zero time on this problem. It was a minor issue when 2.9
came out. No one forces you to upgrade, and we didn't till all our
dependencies worked on 2.9.

~~~
rickmode
And your organization has much more than 3 teams? The article specifically
says this is a large multi-team coordination issue.

~~~
lliiffee
noelwelsh is responding to donjigweed, not the article.

------
Roboprog
I expect this to be exactly the sort of thing that a commercial vendor will
support: here is the language version and library versions that we support.
Don't upgrade any of these until we tell you that you can, or you're on your
own.

I'm just starting to experiment with Scala for scripting on top of existing
Java, so the current state is not a hardship for me. The article is helpful in
pointing out the task (technical as well as P/R) to be faced by any would-be
"golden standard" vendor, though.

------
eweise
A response by J. Suereth to this post
[http://suereth.blogspot.com/2011/12/scala-fresh-is-
alive.htm...](http://suereth.blogspot.com/2011/12/scala-fresh-is-alive.html)

------
latchkey
How about taking some time off from Lift development to implement the ideas
that you and other worked on?

~~~
soc88
That's pretty much what I think every time he spends time on his anti-Scala
rants.

Maybe he should switch the language? Looking at Lift's source code/API this is
pretty much a abuse of the language.

~~~
billmcneale
He's one of the few true Scala experts around (how many can claim they've been
using Scala for more than five years?).

The code he writes is probably as idiomatic Scala as it can get.

Yet another reason why people should be wary.

~~~
dkhenry
No he is a scala early adopter. There is a big difference. I have less and
less respect for the man the more I look at Lift. In fact after I started
working with Play and their scala support I have and even better sense that he
is just a java developer who found a new way to manipulate XML files.

~~~
billmcneale
We're not discussing Lift nor his design skills.

We're talking about his level of expertise in Scala, which is probably one of
the highest in the community.

