
Scala Native v0.1 - zepolud
https://www.scala-lang.org/blog/2017/03/14/scala-native-0.1-is-here.html
======
scotchmi_st
I would love for there to be a similarly thorough project with Clojure. It
really bothers me there's no good native compiler. Apart from anything else,
it means that Clojure lives and dies by the languages it compiles to, and
while Java is used everywhere still, it probably isn't the thing the kids are
learning these days. Besides, without going into any further rational
arguments for why using the JVM (or another VM) isn't always great, something
about it feels a bit icky to me. On an aesthetic level.

~~~
emmelaich
Forgive my ignorance, but wouldn't one of the existing Schemes be a better or
as good option?

I thought the benefit/raison d'etre to Clojure is the JVM ecosystem.

~~~
mateuszf
JVM is the benefit, but great immutable collections data structures and
respective functions are also very very nice.

------
mark242
This will be huge for getting Scala running on AWS Lambda. The cold-start
times for JVM apps is just ridiculous and makes Lambda/API gateway essentially
unusable for anything written on the JVM.

~~~
sjrd
Actually, there are people using Scala.js to run Scala on AWS lambda,
precisely for that reason.

* [https://github.com/tptodorov/aws-lambda-scalajs](https://github.com/tptodorov/aws-lambda-scalajs)

* [http://underscore.io/blog/posts/2016/03/21/serverless-scale-...](http://underscore.io/blog/posts/2016/03/21/serverless-scale-summit.html)

~~~
danaliv
Sure, but it'd be nice not to have everything in the known universe depend on
JavaScript.

~~~
zepolen
I disagree. Having a common language in many places has led to huge
improvements over the board in the JS ecosystem, it's become the "C" of the
modern era, and you can build a lot of stuff on the foundations that others
have laid.

I still hate JS though.

~~~
pjmlp
Which is anything but positive, as it also shares C's style of unsafety due to
its semantics.

~~~
oldrny
Javascript performs unchecked memory accesses? Or are you alluding to their
both being weakly typed?

~~~
pjmlp
No,but implicit conversions, including from operators into numeric values
isn't much better.

The amount of page differences between "JavaScript the good parts" and the
actual language reference speaks for itself.

~~~
notgood
That book was created before many alternatives came out; now if you are
working in a team project you should look at JavaScript the same way you look
at assembler, you know it will be the final thing but you shouldn't be
creating it directly, but instead "compiling" from TypeScript, ELM, etc.
(langs with no implicit conversions, strong typing, etc)

~~~
pjmlp
Those alternatives are nice in startup land, in enterprise consulting land, we
get to use what the customer IT department sanctions as allowed programming
languages and tools installed on computers for external consultants.

------
cwyers
It seems to me like Scala's biggest benefit and biggest downside are two sides
of the same coin: easy interop with the JVM and Java code. Scala Native just
seems like you're paying all the price of that for none of the benefit.

~~~
dillonb
Well, this is just targeting a different VM, LLVM instead of the JVM. You get
easy interop with all LLVM languages with this, including C.

~~~
the_duke
You should really check your facts before posting a statement like this.

While the VM in LLVM historically was short for Virtual Machine, it really has
nothing to do with that. It's a compiler backend used by Clang (C++ compiler)
and Rust.

~~~
acjohnson55
My understanding is that IR is designed as though there were a VM to run it,
but in practice, IR is immediately used to generate code for a target
architecture.

~~~
dbaupp
That might have been true originally, but I don't think anyone uses LLVM like
a JVM/CLR-esque VM any more. As the parent states, the original Low-Level
Virtual Machine initialism was even retracted, meaning the project's name is
just the "arbitrary" sequence of letters LLVM, with no particular meaning
assigned to them.

------
gbersac
That's a great news ! We are exclusively using scala at work for back end and
I wonder if it could be interesting to switch new projects to scala native.

Did you test scala native against well known and massive open source scala
project ? Did the performance improved or regress ? Did you wrote a brand new
scala compiler for native code ?

~~~
therealmarv
Never touch a running system ;) Scala on JVM is much more tested than the new
shiny thing. Also don't expect improved performance... many people think that
the JVM is bloated and makes programs slower (this is mostly not true). The
downsides of the JVM are more memory consumption/footprint (when you have e.g.
small servers or micro instances) and the cold startup time of the JVM itself
(which is not relevant on a server in comparison to desktop Java apps). Would
be interested to hear if any backend Scala projects like e.g. Play work on
Scala Native.

~~~
algesten
> the cold startup time of the JVM itself (which is not relevant on a server
> in comparison to desktop Java apps).

I disagree somewhat with this.

We found that when we started writing microservices in languages that are not
java, the short startup time changed how we did some error handling.

For errors where we say lose connection to the database, or rabbitmq, we much
rather have the nodejs-process die and restart, than try to construct
reconnect logic.

The problem with reconnect-logic is that it is code that (may) be tested very
rarely. This in turn means it's easy to get strange long term problems there
like very slow memory leak due to some listener being added to a connection
object once the connection is initiated.

We did a 180 on reconnect-logic in our nodejs-processes and let the exceptions
just bubble unhandled and take the entire vm down. With automatic restart
script, the process will be back in seconds anyway, and with docker having
built in back-off timers for auto restart, we don't necessarily overload the
shared resources.

~~~
wtetzner
> For errors where we say lose connection to the database, or rabbitmq, we
> much rather have the nodejs-process die and restart, than try to construct
> reconnect logic.

This sounds like a very Erlang-ish way to handle the problem. Another
advantage is that if the server/process is in some weird state that's causing
problems, killing and restarting it lets you clear out the broken state, and
get back into the state that it's most likely been tested under.

~~~
algesten
Yes. We're almost now risking it going the other way, that some bad
programming goes unchecked for a long time, because overall, the process sort
of does what it should. Even if it restarts like 10 times a day.

~~~
wtetzner
Well, you do still want good reporting, so you know when failures are
happening and can capture a stack trace.

------
lacampbell
Scala is a language I desperately wanted to like - high level, statically
typed pure OO language. But in practice I found it almost unusable. The type
signatures were unreadable and I distinctly recall writing a 100 line or so
program where the type declarations crashed the compiler. And the tools
themselves were huge memory hogs - sbt was a particularly bad offender (though
otherwise quite pleasant).

I also did not get on well with the community, which seemed to have a lot of
people with the attitude - "they won't let me use haskell at work so I'll make
do with this shit". They didn't seem to understand or be interested in OO at
all, and were very fanatical about driving application logic with types,
purity, and the like.

Regardless, a native variant would be something well worth investigating if it
ever reaches "production ready".

~~~
eklavya
It looks like you dislike the haskell like usage of Scala, I wonder how could
you then produce types so complicated/weird that you crashed the compiler in
100 lines (In 4 years I have seen it crash only once in an unreleased version
of pickling a long time ago).

~~~
lacampbell
I never claimed my criticism was fair or consistent :P I think I was more
interested in Haskell-like type driven programming at the time though. I
honestly can't remember what it was.

And it's not so much that I hate type driven programming or purity. I just
view them as tools in my toolbox, not as methodologies or worse - religions. I
still think objects are an amazing concept if you look at them through the
smalltalk lens. Or if you take a moment to step back and realise that an
anonymous function is very similar to an object with an "apply" method, or
that a constructor for an immutable object is like a functor that partially
applies many related functions at once.

~~~
eklavya
In my opinion you are mostly describing Scala. Mutability and impurity is
trivially easy to do in Scala. You may see that as a positive or a negative.
Local mutability is definitely easier. Complexity of Scala comes from mixing
OO and FP worlds and I think it's an amazing engineering feat. It will
undoubtedly look ugly after you do Haskell though.

------
evdev
As a Scala guy on a Scala team, I'd think this would be most immediately
useful on smaller fill-the-gaps sub-projects where we have to integrate with
native code.

~~~
manojlds
Looks like a good fit for the CLI we have been planning for our API bases on a
Scala backend.

------
kentosi
This is extremely exciting. I can't wait to try this out.

On the other hand, I wonder why such an effort was never carried out with Java
itself? Or maybe it was but just never took off?

~~~
twic
It was. Several times.

It didn't take off for two reasons.

Firstly, it's not really any faster than a good JIT, and Java has very good
JITs. The startup time is better, but the startup time of a JVM itself is
actually pretty small ('time java -cp . HelloWorld' prints 'real 0m0.112s' on
my machine), and for the uses Java is usually put to, irrelevant. I'd be very
interested to see some benchmarks for Scala Native.

Secondly, there's enough dynamic code in the Java ecosystem - bytecode
weaving, dynamic proxy generation, plugin classloading, etc - that in
practice, your AOT-compiled application need to have an interpreter along for
the ride anyway. That means you don't even save the space overhead and
distribution complexity of deploying a JVM.

~~~
lmm
> Secondly, there's enough dynamic code in the Java ecosystem - bytecode
> weaving, dynamic proxy generation, plugin classloading, etc - that in
> practice, your AOT-compiled application need to have an interpreter along
> for the ride anyway. That means you don't even save the space overhead and
> distribution complexity of deploying a JVM.

The key advantage of Scala in general is that these things are rarely
necessary - the fancy type system and generally expressive language make it
much easier to address the use cases with "vanilla" Scala instead. So Scala
Native will hopefully be able to bypass this issue.

------
pale-hands
Will macros work with Scala Native, as they do with ScalaJs? (I believe that
compile-time metaprogramming is the way forward, especially if the target
doesn't support reflection or dynamic code loading).

~~~
ddispaltro
"We support the whole language including the more advanced features such as
method dispatch via structural types and even macros."

~~~
pale-hands
D'oh, thanks! Can't believe I missed that.

------
eli_gottlieb
This is _amazing_. Scala is one of my favorite languages to work with, and
getting it native-code support will finally help make it fast enough to
justify using it _everywhere_.

------
wst_
Somewhat relevant: [https://blog.plan99.net/kotlin-
native-310ffac94af2#.ijzik0jx...](https://blog.plan99.net/kotlin-
native-310ffac94af2#.ijzik0jxx)

Title says Kotlin, but it is about JVM languages going native, in general. Or
should they?

~~~
sreque
They should. The article basically throws in some FUD to convince people that
they don't want what they think they want. Except, we do want it! The JVM
ecosystem's general aversion to native code and native system integration is
what gave C# the opportunity to take over as the closest thing we have to a
WORA language. If I want to write code in a higher-level language than C++
that can run on any mobile device or desktop OS, I do it in C#, not Java.

The Java ecosystem is playing huge catch up here, and I don't think it's a bad
thing that people are exploring alternatives, whether that's Avian, the now
defunct RoboVM, or LLVM-based backends.

~~~
pjmlp
The JVM eco-system is full of options to compile Java into native code.

The only thing is that most ignore there is a JVM world outside OpenJDK.

And even those that know that world aren't willing to pay for commercial JVMs.

Thus propagating the myth that Java doesn't support AOT.

Which in a way is also understandable, given Sun's political decision that AOT
compilation was tabu.

Luckily Oracle isn't Sun and has heard the industry. So if all goes as
planned, by Java 10, those that don't want to pay for AOT compilers, will have
one in OpenJDK.

------
speg
I can't get hello_world to work, something about a unresolved dependency:
org.scala-native#sbt-Scalia-native;0.1.0: not found

I'm not a regular Scala user.

~~~
densh
Please drop by our gitter chatroom [1] and we'll try to debug the issue
together. It's likely some sort of environment misconfiguration.

[1] [https://gitter.im/scala-native/scala-native](https://gitter.im/scala-
native/scala-native)

------
jcstauffer
Great! Any benchmarks on whether the Scala compiler runs faster when compiled
to native?

~~~
densh
We're looking into that, but nothing to report yet.

------
mafribe
Nice work, I hope this will eventually be a serious alternative to the JVM
route.

Quick question: does this compile down to DOT before going to LLVM? Or has DOT
not yet arrived in Scala Native?

~~~
densh
DOT [1] is a theoretical calculus that is used as a foundation for our latest
front-end compiler called dotty [2]. Scala Native is mostly a middle-end
technology that bridges front-end compiler and LLVM. We plan to support dotty
as a front-end in the future. 0.1 release is based on older Scala 2.11 front-
end.

[1] [http://www.scala-lang.org/blog/2016/02/03/essence-of-
scala.h...](http://www.scala-lang.org/blog/2016/02/03/essence-of-scala.html)

[2] [https://github.com/lampepfl/dotty](https://github.com/lampepfl/dotty)

~~~
bnjmn
Just what I hoped to hear, thanks!

------
huula
Question: what kind of frameworks can be practically migrated to Scala Native?

~~~
lmm
In Scala it's very normal to write libraries or frameworks in "pure" Scala
(i.e. not using reflection, annotations, proxies or anything like that), and
all of those should be fine to cross-build for Scala Native (assuming their
upstream dependencies do first). The state of libraries for Scala.js should be
a good indication - anything that's cross-compatible with that is likely to
work just as well for Scala Native.

------
gravypod
Does anyone have any example binaries compiled with this? What are the sizes
that you could expect?

They say one of their targets is using this for command-line tools (I'm
guessing for startup speed and needing to be small in memory footprint) but
it's not of much value if an "echo" or "grep" implementation takes up 15 to
30MB on the drive.

~~~
dualogy
> _but it 's not of much value if an "echo" or "grep" implementation takes up
> 15 to 30MB on the drive_

In this day & age? It _might_ be of value if an echo or grep takes ~20MB and a
full-blown HTTP REST server with DB access takes ~25-30MB. Why do I care again
exactly whether there is a fixed portion of bootstrap/runtime core code in the
binary, as long as I'm not writing echo or grep?

(Any helloworld app not written in asm (or lib-less C) will consist largely of
"non-helloworld code" in terms of "bytes occupied in the binary", right?)

Dunno might be of concern eg with embedded/raspberry and such, but for code to
be "moved from a jvm-kinda flair to a go-kinda flair" I'm not seeing the issue
.. yet ;)

~~~
usrusr
When you are writing that full-blown HTTP REST server with DB access you would
probably be served better on the JVM. Scala native is just as garbage
collected as Scala JVM and garbage collection is where the JVM shines
brightly.

In my eyes, the main value proposition of Scala native will be "you don't have
to learn a new language if you need to write an echo or a grep", with "you
don't need to learn a new language if you need to run some of your code on a
platform that does not have a reasonably good JVM" a distant second. The
former would suffer a lot from oversized binaries, the latter depending on the
specifics of the target environment.

~~~
Recurecur
A major value proposition is also "I want to run my code with predictable
latency". This opens Scala to embedded and realtime development.

The LLVM AOT optimizations are also likely to be a lot stronger than JVM JIT
optimizations - granted it's good to compile on the exact target architecture.
Also granted that sometimes dynamic optimization helps a lot.

Nice handle BTW. lol

~~~
usrusr
But how much predictability can you get in presence of garbage collection?
Granted, the JIT can also be an element of surprise that is out of the picture
with AOT and the statically linked GC can never be configured differently from
what you expect, but it's still garbage collection and not Rust or a C.

~~~
gravypod
There's a lot of examples of GC in realtime systems. It's just a different set
of concerns. Rather then the programmer worrying about memory constantly, the
compiler engineers are worrying about memory constantly.

This is a very fun read: [http://www.flownet.com/gat/jpl-
lisp.html](http://www.flownet.com/gat/jpl-lisp.html)

------
c-smile
I've created once custom barebone JavaVM with binary size of around 100k. It
was made as an executable jsmile.exe that was capable to read bytecodes cat'ed
to the executable itself :
[http://www.terrainformatica.com/org/j-smile/index.htm](http://www.terrainformatica.com/org/j-smile/index.htm)

The goal was to create JVM suitable for standalone GUI applications. Project
was abandoned when Sun/MS Java wars started in favor of the Sciter
([https://sciter.com](https://sciter.com)).

As of NativeScala ... I think that approach (binary with nano JVM + attached
class files) may work better and with less effort. Scala needs JVM
infrastructure, GC, etc. as far as I understand.

------
brangalinafoeva
How do the compilation times compare to targeting the JVM?

~~~
densh
There is roughly 1-2s (after sbt warm-up) penalty to perform compilation to
native on iMac (Retina 5K, 27-inch, Late 2015) with 4 GHz Intel Core i7. The
linker is highly parallel so you can throw more hardware at it as your project
grows bigger. Barebones project includes nearly 1000 classes and 2000 methods
in transitive dependencies that are being compiled due to closed-world nature
of the toolchain.

------
0xFFC
so dream comes true!

P.S. I think this is related to rust, in a sense before Rust there was no
serious competitor to C/C++, but after seeing what Rust doing to C/C++ I think
there will be more native language to compete with in low level area.

~~~
usrusr
Between the "scripty" GC agility of Golang and all the impressive low level
benefits that Rust is enjoying from folding memory management into the type
system, I suspect that Scala native will be a difficult sell.

On the other hand, the new targets (js and native) might make Scala more
interesting to Java pragmatists who don't care much about going functional but
would not mind writing Java in a more streamlined syntax.

~~~
Recurecur
Golang is pretty deficient as languages go, really. I find Scala to be far
more appealing.

------
enjoiful
At first glance of this article's title, I thought it would be a terrible idea
to use Scala to write native mobile applications. Imagine using Scala.JS to
write a NativeScript/React Native app. _shutters_

~~~
sjrd
> Imagine using Scala.JS to write a NativeScript/React Native app.

People do [1], and are very happy with it.

[1] [https://github.com/chandu0101/sri](https://github.com/chandu0101/sri)

------
auggierose
It seems like that would open up a nice way of using Scala on iOS also.

------
rainhacker
> This opens the door for Scala to be used in environments where full-blown
> virtual machine is usually an overkill

Not sure if I get this, don't Java VMs support this use case (J2ME) ?

------
Negative1
Important bit: "The project has reached a point of feature completeness in
terms of the coverage of the Scala language. We support the whole language
including the more advanced features such as method dispatch via structural
types and even macros."

It must be frustrating to work on a project like this, see areas where the
language can be improved, but only be able to do the work to make it purely
compatible. Hopefully some good comes out in the form of some good SIPs.

~~~
acjohnson55
Or inspiring! I guess it depends on your outlook.

------
lukax
The generated binary for simple Hello World is 3.45 MB which is quite a lot
for printing one line of text but it can be compressed to 326K using UPX.

~~~
cderwin
Rust's is almost the exact same size (3.46 MB). It's probably just that the
libraries that are statically linked by default (jemalloc, std, etc.).

~~~
densh
That's correct. We statically link all of the transitive Scala dependencies.

------
twic
This is certainly an impressive piece of work. However, i think it's worth
paying attention to the limitations, and the use cases they imply; overall,
this looks less like "compile your existing Scala app to native code!" and
more like "use Scala to interface with existing native libraries!".

On the other hand, it's also worth bearing in mind that this is version 0.1.0;
over time, some of these limitations will lift. What i don't know is whether
Scala Native will develop into a complete version of Scala which compiles to
native code, or evolve into a variant of Scala more tightly adapted to a niche
of talking to native libraries.

Anyway ...

(1) No threading [1]:

 _Scala Native doesn’t yet provide libraries for parallel multi-threaded
programming and assumes single-threaded execution by default. It’s possible to
use C libraries to get access to multi-threading and synchronization
primitives but this is not officially supported at the moment._

So forget about using Akka for now.

(2) NullPointerExceptions are replaced with segfaults (hopefully) [1]:

 _A number of error conditions which are well-defined on JVM are undefined
behavior: Dereferencing null. Division by zero. Stack overflows. Those
typically crash application with a segfault on the supported architectures._

That's not so bad; where Java apps might let nulls flow around and rely on
catching NullPointerExceptions to recover from them, Scala apps are much more
likely to use Optional consistently.

(3) If you do want to talk to a native library, and you need to allocate
memory to do it, you're on your own [2]:

 _Unlike standard Scala objects that are managed automatically by the
underlying runtime system, one has to manage native pointers manually._

 _Scala Native provides a built-in way to perform stack allocations of
unmanaged memory using native.stackalloc function: [...] When using stack
allocated memory one has to be careful not to capture this memory beyond the
lifetime of the method. Dereferencing stack allocated memory after the
method’s execution has completed is undefined behaviour._

 _Scala Native’s library contains a bindings for a subset of the standard libc
functionality. This includes the trio of malloc, realloc and free functions_

Java's traditional JNI is a verbose, slow, pain in the stdout, but it was
designed pretty carefully to avoid problems like this.

(a) Intermission! Check out how they do type-level numbers [1]:

 _Natural numbers are types that are composed of base naturals Nat._0, ...
Nat._9 and an additional Nat.Digit constructor._

That's a new one on me!

(4) Incomplete JDK libraries [3]:

 _Scala Native supports a subset of the JDK core libraries reimplemented in
Scala. Here is the list of currently available classes: [...] This is an
ongoing effort, some of the classes listed here might be partially
implemented._

The list has most of the fundamental stuff - a good chunk of java.io and NIO,
the collections, java.lang, atomics. But no java.text, java.net, concurrency,
regexp, date and time, JDBC, reflection, XML, etc.

They don't mention how much of the Scala libraries they support. I would
imagine that they can build anything that's in pure Scala and depends only on
JDK classes in that list, so you'll get the core language stuff and the
collections. Not sure.

[1] [http://www.scala-native.org/en/latest/user/lang.html](http://www.scala-
native.org/en/latest/user/lang.html)

[2] [http://www.scala-
native.org/en/latest/user/interop.html](http://www.scala-
native.org/en/latest/user/interop.html)

[3] [http://www.scala-native.org/en/latest/lib/javalib.html](http://www.scala-
native.org/en/latest/lib/javalib.html)

------
squar1sm
I wonder if Akka will be a part of scala native? It's sort of considered
stdlib? When our team spiked on Akka, we liked it and got our near-reality
proof of concept to work. It'd be awesome to have such a high level library
like Akka compile to a binary.

------
kelvich
Nice job, thanks! Seems that right now you had implemented some wrappers for
libc and some java classes ported to scala. What plans do you have to further
evolve API? Will you focus on reimplementing java.* or create your own set of
classes?

------
LeanderK
Is Scala Native GCed?

~~~
zerr
Should be, because Scala doesn't have strong vs weak reference distinction.

------
tejasmanohar
Does this mean the future of Scala is off the JVM? I ask because the post
calls the JVM impl. a "reference implementation".

~~~
sjrd
IMO the future of the JVM is highly multi-platform: JVM, native and JS.

------
amelius
Does this provide a garbage collector?

~~~
ddispaltro
Yeah it uses the Boehm garbage collector.

------
anta40
Any Windows pre-built binary to try?

I imagine building this stuff on Windows will be challenging :/

------
NickHoff
Will this mean that we can get rid of type erasure when running native code?

~~~
tormeh
Why is type erasure necessary anyway? Surely this can be worked around
somehow. You could manually do what you do with array length in C - pass the
information along manually. Why can't the compiler do this in a transparent
manner for us? Is it speed?

------
crudbug
Will language support destructors for manual memory management ?

~~~
densh
Please file a feature request on [http://github.com/scala-native/scala-
native](http://github.com/scala-native/scala-native), we can discuss the
details over there.

------
webserg
good idea!

------
mihaela
Most things called native are not.

------
rekado
Unfortunately, this requires an existing Scala compiler to build, so it won't
be useful as a bootstrap compiler for Scala on the JVM. Does anyone here know
of an alternative implementation of Scala that could be used to build the
libraries and tools of the reference implementation from source?

It is a problem that many compilers cannot be bootstrapped from source without
a trusted binary of a previous release.

~~~
ajross
It's a universal truism that _all_ compilers cannot be bootstrapped from
source without a trusted binary. It's true that in the world of standardized
languages (which mostly means "C" in practice) there are compilers (mostly
just gcc and clang) that can bootstrap themselves with a trusted binary of a
previous release of some other compiler.

Is that such a big deal?

~~~
rekado
I wrote "a trusted binary of a previous release" not just "a trusted binary".
There is obviously a difference between having a small set of trusted binaries
to bootstrap and having a bootstrap binary for every language or build system.

There are several compiler implementations that enable bootstrapping from
alternative implementations, which shifts the problem to a simpler language,
which may already have a bootstrap path.

See also [http://bootstrappable.org/best-
practises.html](http://bootstrappable.org/best-practises.html)

