
Policy – A fork of the Scala compiler - lelf
https://github.com/paulp/policy/blob/master/README.md
======
jendap
I'm so happy to see this! Thank you Paul!

Paul may have been hurting perception of Scala community by his rants a bit
lately. But the fork is a greatly welcomed! All the things Paul said make
sense. The fork can prove the value of those ideas.

Simpler compiler => more compiler developers => more productive => better
compiler => better for the whole community!

As for Typesafe. There are doing great job. Just two releases ago Scala was
not compatible even between minor releases. Stability, documentation,
training, driving the community, good roadmap for 2.12, etc. Great job! Plus
they are build a great framework around akka and play which is gaining
traction.

Policy is actually not really a fork. Forks are, imho, projects which diverge
code base one day and there's really no sharing between them anymore. Paul is
merging from scalac all the time. It really is just vastly simplified scalac
with some wise decisions like modularizing some aspects (like parallel
collections).

BTW: Anybody worried about a friction while trying to push a bold change in a
team of more than one person? Have you ever worked in a team of more than one?
:-) Just do it. Prove it. Dismiss the FUD.

Fist questions on my mind: 1) Future? Is this repo intended to pull from
scalac or diverge more? 2) Dropping reflection? Does it means removal of
macros as well? For the time being or for good? 3) How about leaf node
invariance in collections (and other collection stuff you talked about)?

~~~
extempore
I will continue to merge from scalac for as long as I can, because with no
meaningful specification it is the only way to retain compatibility.

I didn't say I'm dropping reflection, I said I'm dropping the scala-reflect
layer. It's a gigantic and useless abstraction penalty. I'll fold the pseudo-
compiler into the compiler.

We'll see about collections. Here's where it stands:
[https://github.com/paulp/psp-std](https://github.com/paulp/psp-std)

------
charlieflowers
I just watched the talk, and it is full of insight. Highly recommended even if
you've never used Scala (as I have not).

Some samples (some are quotes, some are paraphrased):

* "The compiler [should be] designed around inputs which are Diff[T]s against an in-memory understanding of the last successful compilation.... As a result, both Repl and Debugger are _superfluous as separate tools_." (emphasis mine)

* "I want to programmatically generate ASTs and feed them in"

* The AST (the tree) is fundamental, not the language. For example, if you have an AST you're working with, and you hate curly braces, theoretically you could see the same tree "rendered" as text without those curly braces.

* "True power is in restriction" & "unnecessary expressiveness is the enemy" \-- whatever your language lets you "say", the "other side" (the compiler or interpreter) has to be able to understand. So unlimited language means horribly over-complicated compiler that cannot do nearly as much optimization as you'd like.

* The more free your code is of "over-precision", the more powerful that code is, because the compiler can do more with it. Example: the well known example of an imperative for-loop versus the functional "map".

* "If you want fast, start with comprehensible."

~~~
foolinux
"I want to programmatically generate ASTs and feed them in"

Going by the experience with preprocessors in Ocaml, not the best idea IMO.

~~~
logicchains
Doesn't Ocamlpp4 or whatever its called generate OCaml source code and pass
that to the compiler, not OCaml AST? The new ppx extension in 4.02 allows
direct manipulation of OCaml AST, but that hasn't been used enough yet to
determine its effectiveness.

------
nemothekid
Can someone with some familiarity with Scala community explain why someone who
seems to be the largest contributor to Scala felt the need to create a fork?
It seems completely justified due to his credentials, but he doesn't give any
specific reasons why he created this fork.

~~~
cakoose
Paul Phillips' 2013 rant/talk:
[https://www.youtube.com/watch?v=TS1lpKBMkgg](https://www.youtube.com/watch?v=TS1lpKBMkgg)

~~~
jpgvm
After watching this talk I don't care what he comes up with in the future, I
want to use it.

Excellent talk.

~~~
mdaniel
Are you familiar with JetBrains MPS?

[http://www.jetbrains.com/mps/](http://www.jetbrains.com/mps/)

It sounds very much like what he is describing.

~~~
extempore
Yes, MPS and "language workbenches" in general are chasing something similar.
Ideas like these are entirely in the details though, and so far I haven't
found anything which does it for me. I could easily be missing it, in part
because systems like MPS have a high up-front investment, and I have regretted
such investments more often than not.

------
saosebastiao
Can anyone more familiar with the matter tell me what is wrong with the
parallel collections? Does he plan to reimplement them in a more maintainable
way? Or do away with them altogether?

All the talk about scala makes me wonder what the language would have looked
like if Java compatibility was never a goal. I'm no language designer and I'm
probably wrong but some things that come to mind are 1) no nulls, 2) no
distinction between classes and their companion objects, 3) no distinction
between traits and abstract classes.

~~~
bkirwi
For a short answer to the first question, see his slides on the collection
library[0], particularly number 42. The gist is that the Scala collections
have too many dimensions (mutable/immutable, sequential/parallel,
strict/lazy), all of which share code -- but it turns out that the right
implementation for a lazy immutable parallel collection is not necessarily
good for a strict mutable sequential collection, and juggling all these
variations makes it much trickier to modify or reason about the code. He seems
to be fine with the existence of parallel collections, but probably as
something that lives on top of the standard library instead of being layered
into it.

To your second paragraph -- I believe that #2 might have found its way into a
Scala-from-scratch, but the others certainly wouldn't. I think the big
difference would be in the type system, which could be much simpler if it
didn't have to interop with Java's weird generics.

[0] [http://www.slideshare.net/extempore/a-scala-corrections-
libr...](http://www.slideshare.net/extempore/a-scala-corrections-library)

------
jongraehl
Paul, why do you say it's wrong that Float(Long.max - Int.max) is equal to
Float(Long.max)? Float has less than 32 significant binary digits, and
Long.max - Int.max has the same MSBs as Long.max. You would need to use Double
to see the difference. I'm not sure what implicit conversions/promotions have
to do with that - you want a warning or error when going from Long to Float?
That would be reasonable. You can even lose significant digits between
Int->Float and Long->Double even though they have the same total number of
bits, so you could require those to be explicit. (from a slide in
[https://www.youtube.com/watch?v=TS1lpKBMkgg](https://www.youtube.com/watch?v=TS1lpKBMkgg)
)

~~~
extempore
> Paul, why do you say it's wrong that Float(Long.max - Int.max) is equal to
> Float(Long.max)?

I don't. You have inserted coercions.

I'm aware of how many bits things have. It's hard to make it through five
years on a compiler without gaining some awareness of such details. I consider
the introduction of lossy type conversions into the global scope to be a kind
of insanity. People should be recoiling from such practices, not looking for
ways to justify them.

------
cpeterso
Is the name "Policy" a pun on "Paul SC" (Paul's Scala Compiler)? Or is there
another meaning?

~~~
mjt0229
That's what I thought, and it's clever.

------
vfclists
It appears to be a fork of the _compiler_ not the _language_ itself.

I don't know much about Scala but I believe that most of its facilities are in
the libraries rather than the language itself.

Since he seems to know more about compiling Scala than everyone else I guess
it must be a good thing. Lets hope others get to fill the inevitable gaps.

~~~
zak_mc_kracken
> It appears to be a fork of the compiler not the language itself.

They are the same thing. There is no spec for Scala (no recent one anyway) so
the specification is the Scala compiler. And based on everything that Paul has
been saying these past months, the compiler is very close to being
unmaintainable.

~~~
eeperson
There actually is a recent spec, although I'm not sure if it still needs some
updating. [http://www.scala-
lang.org/files/archive/spec/2.11/](http://www.scala-
lang.org/files/archive/spec/2.11/)

~~~
extempore
There is "a spec" yes. Now let's see someone implement a compatible
implementation based on that. We're not talking about fiddling at the
boundaries here. For instance type inference is completely unspecified. There
is no aspect of scala compilation which is not heavily influenced by how type
inference unfolds, yet there is nothing. And that is the tip of the iceberg.

If you're always hearing about how amazingly small scala's specification is,
now you know why.

------
djrogers
Shouldn't this be called Standard Scala?

Too soon?

~~~
asuffield
Or "Enhanced Scala Compiler System"?

------
ENGNR
Wait.. is it called policy?

That's going to be a nightmare to Google

Almost as bad as 'play' (which has largely fallen back to 'playframework')

~~~
whalesalad
Thank goodness you can search "golang" for things related to the go language.
That's sometimes a nightmare too.

~~~
stefantalpalaru
Unfortunately this SEO justified change has permeated the community and now
people call the language "golang" instead of "Go".

~~~
Tuna-Fish
s/Unfortunately/Fortunately/

Go is a terrible, terrible name.

~~~
gress
Golang is a terrible name.

~~~
Tuna-Fish
Agree. Still vastly better than go.

------
tormeh
Why are parallel collections an "albatross"? I can understand it if he plans
to replace the functionality you get by writing "list.par.map(func).seq" with
"list.parmap(func)" or something, though. Parallel list functions are one of
my favorite features in Scala, and I think it would be a serious loss if they
disappeared.

Also, I wonder what the two Scala forkers are thinking about Dotty (the next-
gen Typesafe scala compiler) and how they imagine merging their ideas in.

~~~
extempore
It's not a question of whether they should exist at all but of whether they
should infiltrate the collections hierarchy to the extent that someone has to
maintain code which looks like this. And there's plenty of it: this is only an
example.

    
    
      trait ParIterableViewLike[+T,
                                +Coll <: Parallel,
                                +CollSeq,
                                +This <: ParIterableView[T, Coll, CollSeq] with ParIterableViewLike[T, Coll, CollSeq, This, ThisSeq],
                                +ThisSeq <: IterableView[T, CollSeq] with IterableViewLike[T, CollSeq, ThisSeq]]
      extends GenIterableView[T, Coll]
         with GenIterableViewLike[T, Coll, This]
         with ParIterable[T]
         with ParIterableLike[T, This, ThisSeq]

~~~
tormeh
Well, that's a pretty good reason to fix something, I agree. But what are your
thoughts about Dotty? I assume it's insufficient for you, but is Typesafe
going in the right direction here, is it not relevant to what you are doing or
will it make things harder?

~~~
extempore
Dotty is unlikely to attain relevance - for me or as a future version of scala
- because there's no reason to expect it to be any less of an implementation
nightmare than is scalac. It is being written by the same person, applying the
same engineering standards and the same checks and balances, so starting with
a less unsound type system isn't going to matter.

~~~
seanmcdirmid
I worked on scalac for two years, so I know most of the same warts and issues;
I also have had my differences with Martin in the past. But you know what: he
has been courageously innovative and has come up with code that works pretty
well considering the risks taken for Scala. Most people would have thought
what Scala was trying to do was impossible (combining FP and OO with type
inference? But semi-unification is undecidable!). But he did it, and you know
what, scalac isn't that bad either. Heck, I've done much worse, but the point
is to just keep trying.

You want everything to be a nice well understood package because its "easier
that way?" But sometimes it isn't easy and the code isn't pretty; not because
the programmer is horrible, but because the problems are hard and not very
well understood. Does that mean we should shy away from the problems and just
do something safe and conservative like C#? If everyone thought like this, we
would make no progress.

Kudos to Martin for being fearless, and let's hope he doesn't cave.

~~~
extempore
That's right, I'm the stodgy conservative one and martin is the fearless
innovator. Where is this version of the movie playing? I'd enjoy seeing it.

Your second paragraph lands somewhere between completely wrong and "not even
wrong" if it is supposed to have some bearing on the matter. You don't appear
to know the first thing about what I've said, what I've done, or what is
really going on here.

Reflexive loyalty of this kind, unhinged from any objective reality, is what
has brought us today's scalac. You should reconsider whether your having
worked on the scala 2.7 eclipse plugin nearly a decade ago qualifies you to
take a position on this. Then to position martin as the scrappy underdog -
"let's hope he doesn't cave" \- just let us know when the novel is finished,
and I hope it's better than the movie.

~~~
seanmcdirmid
You are right, I don't know you beyond these online interactions. I was giving
you the benefit of the doubt, but you sound very vindictive, and I get it, you
know stuff. Building and designing things is a journey, it won't be perfect
unless you set your expectations low.

I didn't just work on the scala plugin, I changed scalac to be completely
incremental at the AST level (better than everything being a Diff[T]), which
was no easy feat (nor did it last, but it worked and became its own tech [1]).
I've dug deeper into scalac front end code (Typers, namers, parsers) than most
scalac devs have; I know Martin's style very well as a result.

[1] work on making scalac incremental eventually led to
[http://research.microsoft.com/en-
us/people/smcdirm/managedti...](http://research.microsoft.com/en-
us/people/smcdirm/managedtime.aspx)

~~~
extempore
I am intimately familiar with your work on scalac, and that's all I can bring
myself to say about that.

~~~
seanmcdirmid
You and miles must be best friends. Its fine. Phew, really glad to be out of
the Scala community at this point, it just wasn't worth it to being around
those people.

~~~
milessabin
Paul and I don't see eye to eye on many things, but on this we're in complete
agreement.

~~~
seanmcdirmid
So is life. Its great to see that you fixed the plugin and it works very well
now. Congrats!

I'm just glad to be out of it. The Scala community really was becoming toxic,
sleeping was hard. Hopefully you guys have all resolved your differences now
and things are more harmonious.

------
cryptos
What is the goal of Paul Philips? He is obviously frustrated by his
experiences at Typesafe, but what does he really want? Show us that he can do
better? Create an alternative Scala? This seems not be realistic and he seems
to be realistic enough to know it. All this sounds like emotional actions from
a disappointed ex-lover.

Paul, if you read this: why don't you do something completely different? Leave
Scala and your personal rage behind.

------
benjaminjackman
Heh +1 to `trailing commas! I think that's the only "feature".` I don't know
why they were removed in the first place.

~~~
frowaway001
Because the one thing Scala is missing is more ways to do the same thing.

~~~
benjaminjackman
Same things? I don't know what you mean, It was a language feature back in the
2.7 days I think? I was a long time ago, hard to remember when they got rid of
it. It's really nice for this:

    
    
      val color = List(
      "Red",
      "Yellow",
      "Green",
      )
    

If I want to add a 4th element I just ctrl+alt+down, I don't have to go back
up and add the comma at the end of the last line, I spend too much time
juggling commas in lists for my liking.

Some programmers work around it like this (which breaks head-inserting, but
that's typically a lot less common) :

    
    
      List("Yellow"
      ,"Red"
      ,"Green"
      )
    

Which is pretty ugly (to me). It's one of those minor QOL improvements where I
don't see what the downside is.

Frankly I wish every language had this feature.

~~~
frowaway001
> It's one of those minor QOL improvements where I don't see what the downside
> is.

That's a nice example why 99.9% of developers out there should never design a
language.

~~~
equup
You really like to insult everyone who disagrees with you, don't you?

------
wittekm
The "A Fork!? You Should Improve Scalac!" section is delightful.

------
morsch
What the hell is going on? We're in the middle of a major investment into
Scala, and now there's forks left and right, both amicable and not-so-
amicable. This doesn't inspire confidence, although I'm still classifying it
as growing pains/healthy lively ecosystem. I really don't want to stick with
Java 8.

~~~
village-idiot
Clojure isn't forking...

~~~
Pacabel
But it also doesn't offer many of the benefits and much of the functionality
that Scala offers. A familiar C-like syntax, static typing, and so on are very
important for a lot of Scala's users.

------
0x434D53
I don't get that outcry and panic inside the Scala community. We should give
the 2 forks a chance and see what it leads to. There is still the chance then
to reunite the community behind "one" scala. Also they could have great
influence on Typesafe Scala. As far as i know the compiler (which is not very
much): The 2 forks will anyway not move away from Typesafe scala at huge
speed.

Let's come here again in 6 months and see where we stand.

------
nimrody
asfq1

