
Swift for Tensorflow 0.3.1 - asparagui
https://groups.google.com/a/tensorflow.org/forum/m/?pli=1#!topic/swift/S6KyX1szLuY
======
sandGorgon
I'm very puzzled about the Swift for Tensorflow investment. I understand
that's primarily because of Chris lattner's personal investment...but given
Google's massive mindshare in Kotlin/Dart/JavaScript, this is a very puzzling
investment.

Kotlin is successful because Google was able to drive the direction of the
language. For example AndroidX Compose is being built with compiler extensions
in Kotlin with Jetbrains. Dart is obviously driven by Google.

Swift is going to be controlled and driven by Apple. From a strategic
perspective, I don't understand why Google would make an investment like this
when Swift-for-Tensorflow has a fundamental need to drive changes in the
language itself.

Now, I understand if Google didn't have an alternative. Are you saying Kotlin
is soooooo worse than Swift that you can't fundamentally take a similar
direction ? Kotlin is currently one of the world's most popular languages,
born out of sheer LOVE from the community (without being pushed by the
giants).

~~~
cwyers
They explain what other languages they looked at:

[https://github.com/tensorflow/swift/blob/master/docs/WhySwif...](https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md)

Presumably Kotlin wouldn't work for the same reasons they excluded Scala and
Java and C#.

(I think that either of C++ or Julia would have probably made better choices,
given what they spell out here, but they put a lot more thought into what
language to use than most HN commenters seem to give them credit for.)

~~~
newen
Swift was chosen because Chris Lattner is leading this project (Chris Lattner
designed Swift). This explanation is clearly just a token justification for
having chosen Swift.

~~~
cwyers
The charitable way of looking at that is Chris Lattner is leading this project
because the project requires certain things Lattner has experience with, and
therefore the language he designed is one that meets all their criteria.

Now, it's possible that they were less objective about the language choice
than they believe themselves to be, but they certainly have thought about it
an awful lot, and they deserve some benefit of the doubt, unless you can
demonstrate what of this justification is untrue.

~~~
FridgeSeal
They full on brushed off Julia with hardly a justification. It’s the strongest
contender and got hardy a mention. They just dismiss it on the grounds of “oh
uuuh there are...aren’t many libraries or something”, nevermind the fact that
Julia’s data science and ML ecosystem is far more developed than Swift’s is.

Moving TF to Julia would have killed 2 birds with one stone: community size
and outright number of packages would have both been fixed as a community grew
up around TF in Julia.

~~~
orbifold
Julia compares very unfavourably to both Python and Swift: Slow startup time
(unpredictable JIT compiler lag), terrible error messages and stack traces,
duck typing misrepresented as type checking. I've tried using it for what it
is meant for (numerical code / ODE solving etc.) multiple times now. It
performs admirably in that domain.

But no ten horses would make me use it for anything else. Machine learning
needs to be fully integrated into the larger language ecosystem. You can
easily imagine developing a large scale software product in Swift, not so much
in Julia (even less so than in Python). Julia is probably a Nightmare to
develop in once you have more than 10 people working on the project.

The internal structure of Julia's compiler is also a mess compared to the
Swift compiler. You could easily imagine building a compile service / large
scale build system / tooling for swift, not so much for Julia. Plus they were
able to easily extend the Swift intermediate language in the way they wanted
to, while there is no comparable abstraction in Julia. The Swift compiler also
fits neatly into the C++ based ecosystem that Google has.

~~~
byt143
So many patently false claims here.

First, julia has it's own IR (comparable to swift IR as mentioned by Chris),
which is used to great effect in the autodiff packages to allow for compile
time zero overhead gradients. The flexibility is actually the other way
around, Julia's compiler allows these sort of compile time initiatives to be
built in Julia rather than hacked into C++ like swift.

Regarding the duck typing vs type checking bit, that's a matter of tooling.
It's much harder to replicate Julia's combination of speed and dynamism (for
example to do flexible multistage programming) than to capitalize on Julia's
the compile time information to build type checking (which are planned)

The compiler lag is going to be gone soon from two ends: more dynamic
(interpreting code that doesn't need to be compiled, this is pretty much there
already) and more static: Compiling and caching entire Julia programs. There
are already fantastic improvements to this situation in 1.2

That leaves the error messages and stack traces, which ...well what's the last
version you have tried? I find them to be ok, but they can and will be
improved.

Regarding general purpose packages in Julia: check out
[http://genieframework.com/](http://genieframework.com/) as an example.

Julia's multimethods are awesome and a clear advantage over swift's methods. A
protocol abstraction is similiar to julia's facility for traits, will at some
point will be likely baked into the language (but even now allows for compile
time resolution of methods just through multi dispatch abstraction , which
again, is only one step away from building type check abstractions!).

If you want to know more about Julia's plans and philosophy in the regards,
see:
[https://github.com/FluxML/Flux.jl/issues/614](https://github.com/FluxML/Flux.jl/issues/614)

~~~
orbifold
I don't think static type checking is a matter of tooling, getting Julia to
perform well amounts to either a lot of trust in the compiler (is static type
inference for the Julia type system even possible?) or manual type annotations
as far as I can tell. And static types are not there in a large program to
make them necesarily faster, but to automatically enforce contracts. For this
to work they have to be mandatory, not added on as an after thought. I don't
want to be able to run a program only to discover it crashes because a method
was not overloaded with this particular combination of parameter types.

Swift has clearly an advantage in that domain, as has C++, C#. In an
enterprise setting such as Google no one actually wants speed and dynamism out
of a language (C++ programmers are not allowed to use rtti for example),
rather consistently boring, predictable, statically guaranteed results with as
minimal fuzz as possible (Go, Java, C++) and enough competent programmers in
that language.

I agree that it is neat that you can hack the Julia compiler in a library to
do compile time automatic differentiation, but my point was that if you build
it into the language like the Swift for Tensorflow authors are doing this will
result in a more conservative, if inflexible, but potentially more performant
solution. In other words precisely because you need a team of people to
implement and integrate this feature in C++ you will get a stable "single
source of truth" implementation in the language, not several (Flux, Zygote)
constantly changing libraries.

Well I've tried DiffEqFlux.jl with Julia 1.1, it works well enough, but the
stack traces are ridiculously long if something goes wrong, a ~200 LOC program
takes seconds to compile and the library and package management story seems
immature compared to rust/swift/python/c++.

~~~
simondanisch
>Swift has clearly an advantage in that domain, as has C++, C#. In an
enterprise setting such as Google no one actually wants speed and dynamism out
of a language (C++ programmers are not allowed to use rtti for example),
rather consistently boring, predictable, statically guaranteed results with as
minimal fuzz as possible (Go, Java, C++) and enough competent programmers in
that language.

That's a pretty weird statement, considering how hard it is to write a complex
C++ program error free. I suppose Swift is much better in this regard - But
compared to C++, Julia is a bliss and one can easily write huge applications
error free, while I had my absolute worst debugging experiences in C++, where
it's very easy to create obscure and hard to find bugs.

If you really need to get predictable results and actually know those results
- you won't get away with a static type checker anyways, but will have to
write & run tests, which then pretty much make it irrelevant whether your
language is dynamic or not.

------
jimfleming
To those that are upvoting this and previous Swift + TF announcements: What
are you excited about, specifically? Why Swift? Why not Julia? Is it the
syntax? Types? Compilation? Performance? Community?

I like Swift and all but our ML/DL/RL/DS tools and libraries are in Python
(and occasionally R). Most are missing for Swift without an awkward Python
compatibility layer and I don't see a compelling reason to adopt it.

~~~
ddragon
I upvoted it not because because I use Swift (or intend to use it) but because
I find it interesting that the concept of differentiable programming is pushed
further and further. Having a host language being used to write and compile a
second language (which is implemented in a third language) just feels
restrictive in many ways.

I do think what is being done with Julia, Cassette, Flux and Zygote more
interesting since it's Julia all the way down (while Tensorflow's backend is
still C++) and the compiler work is focusing on not being specific to one
implementation or technique, but allowing any such language extensions (such
as auto-parallelization and other forms of source to source transformations)
to be done by any 100% Julia library. So if Tensorflow for Swift (regardless
of the actual reasons behind the choice of the language) proves that the
technique is a significant upgrade over what currently exists, it could spark
interest in the competing approaches, and I think Julia can help pushing the
concept even further.

~~~
itsmuriuki
"I like Swift and all but our ML/DL/RL/DS tools and libraries are in Python
(and occasionally R"

the python interoperability allows you to use all python libraries but in
swift

~~~
ddragon
In the perspective of someone who writes Julia, which can also call directly
any Python method (and R, Fortran, C and C++), that's a nice stopgap, but you
really want a true native ecosystem. Not only there is more mental effort
dealing with two languages at the same time (which might lead to people just
use Python in the first place), the whole purpose of Swift for Tensorflow is
having a language with first class differentiation support, which is pointless
when the ecosystem is fragmented in multiple languages.

And there is the risk that the community simply ends up considering that good
enough and just make wrappers (since it needs a lot of work to create
something nearly as good from scratch). Thankfully that didn't happen with the
Julia community, and the key is probably making the creation of the tools much
easier so they can catch up to mature but constantly evolving environments.

------
mrath
Is swift a common language for kind of thing tensorflow is used for? Given
that swift does not work on windows and Linux support is sub par, is this only
because of Chris Lattner?

~~~
eklavya
The way I see it. The technical points have been explained on their GitHub
page. One might not agree with it. From an organisation's perspective it makes
total sense. You need in house expertise and you need a free hand. Not many
languages would have fit both of these things. I mean if it were you or me,
wouldn't you pick what you are most comfortable with given it fit the
constraints?

Linux support for this can't be sub par since the whole stuff will run on
linux like 99.99% of the time.

~~~
bobajeff
Question if Swift works on Linux right now what are the chances that it also
works on Windows's new built in Linux (WSL2)?

~~~
oscargrouch
I used swift on WSL1 and even compiled natively for Windows.

But i guess the pure Windows ecosystem is rough on the edges right now, but
doable. And in WSL(1) things have worked pretty the same as in Linux.

Just download the packaged binary distributed on their site, and you are ready
to go.

(Oh, a note; its not the tensorflow branch, if is thats what you want)

------
mark_l_watson
Glad to see the update. I have experimented with both the macOS and Linux
drops before and they were rough. v0.3.1 is focused on providing a productive
platform. Anyway I am on travel without a laptop so I need to wait to try it.

The idea of LLVM/Swift ‘turtles all the way down’ might really pay off in the
long run. In TensorFlow there is an un-understandable barrier (for me) between
the Python layer and the lower level C++ code. Good to see this Swift effort
along side Julia + Flux, which offers the same implementation language for the
whole system advantage.

------
yeldarb
I took the recent fastai course that had two lessons about S4TF taught by
Chris Lattner and walked away excited.

It’s not ready for prime time yet but the roadmap looks great. Building auto-
differentiation into the language and having an easy way for anyone to
experiment with core architecture changes (without killing performance) will
be a huge win for everyone.

I expect lots of prototyping will be done on S4TF (because it will be easy for
developers to keep things on the GPU with XLA and MLIR) and then the things
that work will be back-ported to other ecosystems.

[https://medium.com/tensorflow/mlir-a-new-intermediate-
repres...](https://medium.com/tensorflow/mlir-a-new-intermediate-
representation-and-compiler-framework-beba999ed18d)

------
gdubs
If you missed it, Chris gave a talk a little while back, and went into detail
regarding the motivation behind the project, what Swift provides, etc. It
answers a lot of the questions being asked here:

[https://www.tensorflow.org/swift](https://www.tensorflow.org/swift)

------
pjmlp
Ctrl+F Windows, oh well back to ML.NET and Julia.

~~~
MiroF
Windows is just not meant for serious DL research

~~~
panpanna
This is a very odd attitude.

Specially when Windows had the best GPU support.

~~~
MiroF
What does this mean? Pretty much everyone uses nvidia gpu and those are
supported by Linux

~~~
panpanna
Not all of them, and the ones supported are not always very stable.

Otherwise gamers would all be switching from win10 to Steam OS.

