
Typelevel Scala and the future of the Scala ecosystem - LiveTheDream
http://typelevel.org/blog/2014/09/02/typelevel-scala.html
======
saryant
Background: I saw Miles' tweet on this last week but didn't follow the
apparent brouhaha that followed. All I know of him was his (hilarious) talk at
Scala Days in June.

As someone who uses Scala daily, I don't think this is a good thing. I fall
squarely into the Typesafe side of the camp—we don't use Scalaz or Shapeless
or much of anything from the Typelevel project but Akka and Play are core
technologies for us. We don't have a Typesafe subscription, we just use Akka
and Play, a lot.

> As I observed earlier, there are already several variants of the language in
> existence and it has been clear for a long time that different sections of
> the community have different interests.

I don't think the author's examples really hold water. Nobody is using the
Dotty compiler commercially, Scala.js is still in its infancy and while I know
large organizations have at times run on minor Scala forks (I think Twitter
used to but is now on 2.10.4), I'm not sure I believe that this is a major
constituency. The official Scala is _the_ Scala.

If this fork gains steam, it's going to be a huge opportunity for FUD around
Scala and will hurt its adoptability, especially if existing Scala teams now
have to choose sides.

That said, if the Haskell side of Scalaland wants to hang out somewhere else,
that might be good too. Too many Scala questions on SO are answered (and
upvoted) with obscure Scalaz solutions that make no sense to someone coming in
from Ruby or Java.

I _like_ functional programming. I _like_ how monads can simplify things. But
I also have to write code that someone without a lot of Scala (or Haskell)
experience can maintain.

~~~
lmm
> I like functional programming. I like how monads can simplify things. But I
> also have to write code that someone without a lot of Scala (or Haskell)
> experience can maintain.

I agree with this, but I think a same replacement for the type lambda syntax
goes a long way towards making those scalaz solutions maintainable.

------
noelwelsh
To people worried about fragmentation, retaining binary compatibility means it
really doesn't matter. Scalaz can be written in the Typelevel fork and you can
use it without issue from code compiled with the standard Scala compiler.

Also, from the announcement:

"As a conservative fork, our aim is to be merge compatible from the Typesafe
Scala compiler, and a key objective is to feed pull requests (many of which
will resolve long standing Scala bugs or feature requests) back to the
Typesafe Scala compiler."

So we aren't looking at radical changes, and hopefully these changes will get
into Typesafe's compiler.

So basically it's an opt-in better Scala compiler with no effect on
compatibility. What's not to like?

------
dkarapetyan
So the elitist community component of Scala that is constantly trying to turn
it into Haskell has decided they need an island of their own. Great, they can
have it. Now I'll know where to go when I need monadic and monoidal libraries
for serving HTTP requests using nothing but type level hacks.

In all seriousness though I don't really know what this is going to
accomplish. The article mentions the academic and non-academic components of
Scala and it is clear the author falls on the academic side but if that's the
case then why not just go and hack on something like Haskell (or Coq), a
perfectly unapologetic academic language that has tried to avoid success at
all costs. If push comes to shove I'm with team Odersky.

~~~
pkinsky
>Now I'll know where to go when I need monadic and monoidal libraries for
serving HTTP requests using nothing but type level hacks.

Monadic libraries with classes like Option, Future, and List? Used in elitist
features like for comprehensions? Anyone who writes Scala on a regular basis
deals with monads constantly. Why not take 15 minutes and learn how they work?

~~~
dkarapetyan
Oh no that is all a little too pedestrian for me. I've moved my sights much
higher now. These days I use template haskell and applicatives to generate
HTML forms: [https://github.com/yesodweb/yesod/wiki/Multi-select-field-
po...](https://github.com/yesodweb/yesod/wiki/Multi-select-field-populated-
from-database-%28using-esqueleto%29).

I'm all for applying theory to enrich the practice of programming but lets be
reasonable here. Lists and Options existed way before anyone added monadic
sugar on top of it as did mapping and filtering collections. I like category
theory as much as the next guy but lets just pull back a little bit. Not
everything needs to be point-free, functorial, and shapeless.

~~~
pkinsky
>I'm all for applying theory to enrich the practice of programming but lets be
reasonable here. Lists and Options existed way before anyone added monadic
sugar on top of it. I like category theory as much as the next guy but lets
just pull back a little bit. Not having needs to be point-free and functorial.

I agree that Spire/Shapeless/Scalaz tend to go far beyond anything I would use
in a production app, but Options/Futures/etc are barely usable without for
comprehensions, which are just syntactic sugar for flatMap. I see the ability
to write

    
    
      for {
        x <- f1() // returns Option[T1]
        y <- f2(x) // returns Option[T2]
        z <- f3(y, x) // returns Option[T3]
      } yield foo(x, y, z)  // for expression returns Option[T4]
    

as a huge part of Scala's success

~~~
dkarapetyan
Sure that's nice and so is LINQ but there are thousands of programmers using
LINQ without worrying about monads. It's like I said, I really don't have any
issues with applying theory to enrich the practice of programming and the new
crop of languages that are more fully grounded in theory is really a breath of
fresh air. The problem I have with some people that enjoy these type level
hacks is that they are not in it to build things that thousands of programmers
can use to make their lives easier. They're usually in it to build monuments
to their own egos. Forking the language is just another instance of that
hubris. They're basically saying what Odersky and friends are doing is not
good enough even though Odersky and friends have poured thousands of hours
into making it something that in my opinion is actually something good.

If this was something that was driven by anything other than ego then they'd
look at library popularity and contribute fixes to the most used and valuable
libraries. Instead, they've decided to fork the language because 3 libraries,
one of which is the author's own, don't have enough syntactic sugar to express
certain type level constraints. Does that sound reasonable to you?

~~~
jroesch
I think it is misguided to call this an act of ego, like Miles says in the
announcement Typesafe has different goals, and milestones that don't
necessarily serve the whole community. There are actually tons of nice small
changes that haven't been implemented for various reasons, and I see this as
an incubator where people can try out experimental extensions of the language
and reasonably package them together, ideally porting them (if they are good)
back to scalac itself. This isn't much different than people reimplementing
Python or Ruby, or adding experimental extensions to Haskell (go look at how
many different Haskell compilers there are). Personally I'm always interested
to see what directions people can push software in when not motivated by
money, or publications.

I think it is also important to remember that these people work on stuff for
free in their free time because they enjoy it. They are free to pick designs
and approaches they like, and have zero obligation to contribute to popular
libraries, regardless of their supposed merit.

~~~
dkarapetyan
How many people write anything in Python or Ruby that is not compatible with
CPython and MRI Ruby? The lowest common denominators in those communities are
still the reference implementations and any production grade library conforms
to that lowest common denominator. As for the Haskell compilers, I'd like to
hear from someone that uses anything other than GHC in production.

I agree they can work on whatever they want but this is a bit less benign than
writing another library that no one is going to care about. Language forks
have never been successful. Please show me one example where a language fork
has succeeded and I'll concede. This is a whole lot of drama that is going to
turn more people away from Scala which in my opinion is one of the better
languages that runs on the JVM.

~~~
tel
Haskell's actually got a pretty great (if informal) standard which has been
used to write a number of different compliant Haskell compilers. GHC is
obviously the most popular open source one, but Standard Chartered Bank has
its own Haskell compiler it uses a great deal of in production, for instance.

~~~
dkarapetyan
What is the point of a proprietary, in-house, custom compiler?

~~~
frowaway001
The Excel backend of course.

------
saosebastiao
My biggest frustration with Scala is the exact opposite of their frustration
with Typesafe's Scala: it moves too fast. Scala's backwards compatibility
problems are obnoxious for the people who don't feel the need to reinvent
everything. I can find a version of Foo that works with 2.11 but the only
version of Bar I can use only has artifacts for <=2.10.

The play framework exemplifies these problems. Documentation doesn't keep up
with releases, blog posts and tutorials don't either. Sometimes I'm looking
for example code for some plugin or library, and the code only works with some
other version of another dependency that I can't use. I totally understand
that people prefer Agile development, but you can't combine Agile development
with non-Agile documentation, unless you want to create a framework that only
experts can use.

~~~
pea
Yeah I've wasted so much time dealing with code for the wrong version of Play.
It's horrendously organised: case in point, the JSON parsing (WTF?)

~~~
Rezo
The documentation for JSON parsing in Play is hilarious.

You read the "The Play JSON library Basics" document, it looks like a lot of
code for something that should be simple, but you go ahead and implement as
explained. After a while, the verbosity starts bugging you, so you read the
"JSON Reads/Writes/Format Combinators", followed by "JSON transformers" and
finally the "JSON Macro Inception" pages.

The punchline in the last section can be summarized as "forget all that other
stuff, just do "implicit val personFmt = Json.format[Person]". I was happy to
be able to delete hundreds of lines of code from my project after learning
this, but couldn't you just have told me this upfront since it probably covers
99% of all cases? The documentation also has an aura of "Isn't this all very
clever?" feeling to it. It's also quite hard to search for anything, as
outdated versions of the docs keep popping up in Google as the first results.

~~~
tormeh
Look at this one: [http://stackoverflow.com/questions/1722726/is-the-
scala-2-8-...](http://stackoverflow.com/questions/1722726/is-the-
scala-2-8-collections-library-a-case-of-the-longest-suicide-note-in-hist)

Scala has a lot of really clever stuff and the "isn't this all really clever"
attitude is definitely there.

EDIT: Come on. I meant clever in a good way as well. I'm just a bit
ambivalent, is all.

~~~
lmm
I think if anything that example shows the cleverness is a good thing. It's
from almost 5 years ago, since when Scala has gone from strength to strength,
so the collections rewrite turned out quite well, and fears that it would stop
new people coming to Scala, or that companies would stay on 2.7 and the
community would schism, were definitely unfounded. The "use case"
simplification of the documentation has been implemented and has shown itself
to work well, suggesting that this kind of complexity can be managed. And a
few years later I've seen Chris use equally complex techniques in his own code
(check out his "practical scalaz" talk).

------
pkinsky
I'm conflicted. Type constructor partial application would be great (the
current type lambda syntax is just awful), but I really don't want to see
Scala fragmented.

~~~
jghn
Agreed on conflicted-ness.

The thing that concerns me the most is that from observation it feels like
many of the high muckity mucks of the Scala world would probably be heading in
this direction. There's nothing inherently wrong with that, but IMO that
increases the chance of serious fragmentation.

------
cryptos
I'm worried about this fork. Scala is far away from being a wildly used
industry language. Yes, there are some well known companies using Scala, but
how many of them develop a really big part of their software with Scala? Not
so much. If you look around you'll see Java, C#, C, C++ (depending on the
business) and actually not so much Scala. The ones who think forking Scala
does help the community are in a reality distortion field.

Take a look and Linux desktops to see that diversity is not always good,
especially if you're in a niche.

~~~
propensive
I think there is a lot of Scala usage you perhaps don't see. As an example
from the corporate world, at least six major international banks (probably
more) are doing a significant amount of their development in Scala, some with
teams in the hundreds. That's a solid market for Typesafe right there, which
isn't going to go away overnight, and won't be stirred by the existence of the
Typelevel fork of Scala. Furthermore, they should benefit from its existence
because it represents a means for more people to be able to contribute to and
enhance the language, without negatively impacting the stable, reliable
releases from Typesafe.

------
robert_tweed
What's the history behind this fork and what is it likely to mean in terms of
fragmentation? In particular what are the irreconcilable beliefs on each side
that have led to this?

I haven't tried Scala yet, but my gut reaction is I should stick with Clojure
until the dust settles. A fragmented community around a relatively niche
language is never good.

~~~
ssmoot
As someone who's been using Scala for the better part of a couple years:

This seems more like a more ambitious version of the Shapeless and ScalaZ
libraries (with some of the same crowd/authors).

I (and I suspect most people who program in Scala for a living) don't use
those libraries. Not because they're in any way bad. But there are trade-offs
using them IME. The biggest being compile-time. So I get by fine without.

I think this is probably generally good news. In the same way that (I believe)
JRuby pushed MRI forward and popularized the case for a multi-threading VM,
more advanced type features in this fork probably benefit the Scala community
as a whole.

I certainly don't see how it could be harmful. Typesafe isn't going away. The
vast majority of businesses will still use their compiler. If you didn't use
the advanced-type-based-libraries before, this news probably isn't going to
impact you anyway.

So from where I'm standing, having made a late 2.9 to 2.10 transition, and now
having made several 2.10 to 2.11 transitions, Scala seems a very mature
language/platform to me. Libraries are generally upgraded fairly quickly, and
source wise, those releases broke _very_ little (2.11 being the biggest
culprit just because of the modularization effort, but that's pretty trivial
to resolve).

I dunno. It's not Java. But it's not Ruby either. It's gotten new features
since 2.9, but it's broken or changed very little. I find working with it,
even working through upgrades, very stable and mostly frustration free.

~~~
runT1ME
As someone involved in the scala community (and scalaz) community, I'd say
about 60% of the companies I talk to at the various meetups use at the very
least #scalaz. I was pleasantly surprised.

~~~
jghn
Could this be a biased sample though? Not saying this _is_ the case but it
seems logical that the people at the scala meetups are cut from a cloth that
would make them more likely to be using things such as scalaz.

I know that my local scala meetup seems to be more heavily oriented towards
the functional end of things than the bulk of the scala developers I interact
with. OTOH, perhaps the bias is with _my_ sample.

------
vorg
When I see all the various Scala compilers out there (Typesafe Scala compiler,
LAMP/EPFL Dotty compiler, Scala.js, Scala Virtualized compiler, Scala.Meta
system, and several research and private variants of the Scala compiler, IDEs
own Scala variants) compared to the lone version of the Groovy one, I realize
just how detrimental to the Groovy ecosystem was Codehaus Groovy's practise of
ignoring creator James Strachan's JSR-241 standardization effort and
redefining "Groovy" to mean the org.codehaus.groovy implementation only. At
times it seemed they were scuttling others attempts to build a variant
compiler, and even cannabalizing the annotation-based addon market like when
they duplicated the Groovy++ static compilation "speed booster".

------
_gkossakowski
I'm somewhat surprised that everybody missed the fact that Scala 2.12 has more
to it than just Java 8 compatibility and stability.

We're working on replacing the whole backend in scalac. We'll ship faster and
easier to maintain optimizer that fixes lots of long-standing bugs in scalac.
In fact, it will allows us to fix the second most voted issue ever:
[https://issues.scala-
lang.org/browse/SI-4633?jql=project%20%...](https://issues.scala-
lang.org/browse/SI-4633?jql=project%20%3D%20SI%20AND%20status%20%3D%20Open%20ORDER%20BY%20votes%20DESC)

More details:
[http://magarciaepfl.github.io/scala/](http://magarciaepfl.github.io/scala/)

\-- Grzegorz, member of Scala team at Typesafe

------
benjaminjackman
I don't understand what features they want to add beyond type lambda's,
singleton types, and literals for bytes / shorts.

1\. I understand if you spend all day making shapeless / scalaz the lack of
sugar for type lambda's is annoying. Why not make a SIP to add some sugar
there?

2\. Singleton types would have use in ScalaJs as well for encoding. There is
already a SIP for this.

3\. Byte / Short literals, seem like again, a SIP could be made for the next
release.

So all this boils down to getting SIPs into the language faster. So I think
the real feature is not having to wait 3+ years for language improvements that
significantly improve the QOL of some library writers.

That is a feature I would like as well. I really miss being able to open the
mailing list and scan for exciting new features, Instead I know even if
someone has a great idea, it'll be years before it sees the light of day. This
sucks and I've moved onto to getting my neat idea -> wow I can use this in my
language fix to ScalaJS[1].

I think (thought?) about Scala as ycombinator for language features. I see
this enterprise mindset coming in now trying to tie down what Scala is and
ultimately I fear that it will kill all the things that has made Scala popular
in the first place. Pushing to a more infrequent release model _might_ help
enterprise customers but it absolutely crushes library developers in the
vanguard working to make the Scala of tomorrow by fighting to get the language
features of today. They need turn around on the improvements listed above and
can't wait 4+ years to get them.

So does Scala take all the strength it has built up from acting as a feature
incubator for the last 10 years and switch and lock down what works, it is in
a prime position to do so? It has a virtual monopoly on it's space.

Or does Scala continue to act as a language feature incubator giving its
coders early access to cutting edge CS idioms in a mainstream language. There
are a lot of CS ideas waiting to get their shot.

Some people would see this situation and say Scala is facing a mutually
exclusive set of outcomes, it must decide now. Nonsense I say! When confronted
with tough choices I think of the famous Yogi Berra quote "When you see a fork
in the road, take it."

I ask myself is there some way we can have our cake and eat it too? Can we go
on trying out new features while still oferring stability to those willing to
wait for it?

What if we moved to model of more frequently releases + less frequent
releases? A controlled fork within the ecosystem, similar to what Ubuntu /
Fedora does.

Publish 2 major short term releases a year that cutting edge library writers
can target and build their libraries against. Vend these are properly tagged
releases in maven central. Stick any old SIP in there, the enterprise
overlords don't have to know about it. Rapid iteration, with rapid fall-off in
support, a fun Scala for those of us that love using new features, bugs and
all, for us the language is about growth.

Then have an LTS version released every other year that picks up the features
that worked and everyone seems to love and that will be supported for years
into the future. Relatively slow iteration with slow fall off in support, vend
this Scala to enterprise users with a coherent multi-year support policy that
just oozes with that 'this is rock solid and will work for eons to come'
professional vibe that makes them happy.

1: I am using ScalaJS and man it feels great, it's like Scala of 6 years ago,
I love opening the mailing list to see if there is a new release with some
awesome new features / improvements. There is a lot of green territory over
there because it's targeting a new platform and that will settle with time,
but already it beats the socks off all the other competitors in the statically
typed compiles to JS camp, just no one knows it yet.

~~~
ddispaltro
+1 on ScalaJS. It's pretty amazing piece of technology (Thank's team!) using
it in combination with David Barri's bindings for ReactJS[1] and Li Haoyi's
uPickle[2]. It makes for elegant solutions with a shared codebase.

[1] [https://github.com/japgolly/scalajs-
react](https://github.com/japgolly/scalajs-react)

[2] [https://github.com/lihaoyi/upickle](https://github.com/lihaoyi/upickle)

~~~
dustingetz
do you know of any large apps using scalajs-react in production?

------
zak_mc_kracken
Regardless of the outcome of this initiative (which has little chance of
succeeding, but best luck to them), this is pretty bad for the Scala ecosystem
overall. There's a general lack of confidence from the community toward Scala
and Typesafe and this certainly contributes greatly to the general uncertainty
about Scala's chances of success.

~~~
frowaway001
I guess it was only a matter of time until the "Scala will surely die!!!
Really!!! This time it will go down!!!" crowd would show up ...

