
Swift for TensorFlow - gok
https://www.tensorflow.org/community/swift
======
tc
They want to build up the language compiler itself to better support machine
learning. That requires a language with enough type information to support
sophisticated analysis. It should be built on LLVM. It needs an existing
ecosystem of tools and libraries. And having a better performing language than
Python is likely a win.

Swift fits the requirements. As Chris Lattner is driving this, no one could
ask that he choose something else.

But Rust also would have been a plausible choice. As the Rust team is very
interested in applying the language to exciting new use-cases, it's a bit of a
loss for it to miss out on collaboration here. Perhaps this could inspire
similar work on the Rust side; many of the concepts would likely transfer
straightforwardly.

~~~
jpz
The principle reason Python flourishes is because it's highly expressive and
readable. (There are things it misses, such as sensible lambda, any reasonable
multi-threading model, etc.)

I can't see Rust competing on similar merits. Swift however (like Kotlin to
which it is extremely similar) seem to be in the right sweet spot in terms of
languages designed for usability, with a lot of useful f.p. constructs.

~~~
swsieber
It depends on if all your writing cutom functionality. If all you have is one
big main function, rust is pretty readable. Function definitions and structs
are where things get a little messier.

~~~
jpz
I can't agree - there's too much going on in Rust - most people using Python
want something approaching Matlab or R for ease of use.

I'm not saying Rust doesn't have it's sweet spot - but I can't see that as a
general purpose language. (Likewise Haskell and Scala - that's just my opinion
though :))

~~~
tormeh
For better and worse, I wouldn't say anything so absolute about Scala. Scala
contains multitudes ;)

You can write Scala as Java or Haskell or anything in between.

------
KenoFischer
Watching the talk, I thought the problem statement was exactly right. Of
course our answer is a bit different, but I'm glad people are coming around to
seeing ML as a programming language problem. We talked about this a bit in a
blog post a few months ago:
[https://julialang.org/blog/2017/12/ml&pl](https://julialang.org/blog/2017/12/ml&pl).

------
vomjom
The talk is here:
[https://www.youtube.com/watch?v=gplTc2F5Wvk&feature=youtu.be...](https://www.youtube.com/watch?v=gplTc2F5Wvk&feature=youtu.be&t=1h47m20s)

Also see Chris's introductory post for the project:
[https://groups.google.com/a/tensorflow.org/forum/#!topic/swi...](https://groups.google.com/a/tensorflow.org/forum/#!topic/swift/xtXCEvtDe5Q)

~~~
bsaul
Is it me or did the audience had absolutely no clue what that presentation was
about ?

------
J0-nas
I like both Swift and Tensorflow. But how is that going to work?

As far as I understand MacOS has no official Nvidia support (=> no Cuda),
which is (at least) advised if you want to use a GPU for computing. Using
OpenCL instead of CUDA would require building Tenfowlow from source. The
OpenCL support is not as mature as with CUDA, so I imagine you could run into
unexpected (performance) problems.

On windows, you have an excelent CUDA but lackluster Swift support.

Will they add OpenCL as a "first class backend" for Tensorflow or rather
expect a "first class" support of Swift on Linux and Windows? Otherwise who is
going to use it?

~~~
vomjom
People rarely train ML models on macOS for the reason you mentioned. Most
machine learning work happens on Linux, so this should work well there.

TensorFlow supports a standalone server mode where it receives computation
graphs and executes them. This is nice because then you can remotely execute
on any accelerator (Cloud TPU, multi-worker multi-GPU) from your laptop.

In their demo, they did exactly that with a Cloud TPU: it connected to a
TensorFlow server that executed the machine learning training part of the
program.

~~~
J0-nas
I agree, I just had in mind that Apple just now added/announced support for
external GPUs. Besides Image+Video edditing, I though general computing tasks
is a use case they had in mind. It's not like Gaming is big on MacOS.

------
Max-q
Shouldn't it be called "TesnsorFlow for Swift"? Like in "MySQL driver for
Java", not "Java for MySQL driver".

I'm not native English speaker, so I wonder...

~~~
KenoFischer
I think they're making that grammatical choice deliberately because they're
modifying the language to better support TensorFlow integration, as opposed to
just providing bindings for it.

------
riku_iki
Curious why not Go. Google has much more control over Go than Swift.

~~~
relyks
This is actually surprising. Are the machine learning engineers at Google
(besides Chris Lattner) admitting that Swift is a better language than Go? :)
I would be surprised to find out if he was the only one involved in the
decision

~~~
DannyBee
So, i'm the closest thing to Chris's counterpart in language land (I own all
of Google's production and cloud programming languages, including Go). We also
go way back to nearly the beginning of llvm days. Chris and I certainly talked
about this.

I guess i don't _really_ understand why anyone would think this is surprising.
Like, i understand the knee jerk reaction, but I guess i don't understand why
i would try to force him to do something else.

Chris is a high level engineer who decided he wanted to prototype and build a
system in a language he's familiar with and feels is a good match for his
problem space. People generally agree it's a good match for his problem space
and his familiarity/etc makes him more effective in getting things done there.
This doesn't mean anything else is a bad match (general purpose languages are
often good at multiple use cases, hence the reason they are considered general
purpose)

Why would I try to stop him and force him to use another language?

What tangible benefit would such a thing bring Google?

It's not like Google is a bunch of language zealots who think everything must
be written in language X - it already supports a number of production
languages, because different languages tend to have different tradeoffs for
different use cases.

Outside of that framing, i have trouble thinking of a sane goal that would be
served by trying to force him to do it in a different language.

But if i'm missing something, i'd love to know!

~~~
gok
Well initially preferring to use a homegrown solution (like a programming
language) isn’t necessarily a sign of zealotry. Dogfooding is good :)

Go just seems like a much worse match for this problem than Swift. Wrong type
system, wrong compiler architecture, wrong runtime design, wrong language
evolution strategy. This is in so many ways a great match for Swift.

~~~
DannyBee
"Well initially preferring to use a homegrown solution (like a programming
language) isn’t necessarily a sign of zealotry. Dogfooding is good :)"

Sure it's not necessarily.

Building on top of something that is yours, simply because it is yours, is in
fact a form of zealotry.

It is fairly unrelated (IMHO) to building on top of something that is yours so
you can make sure you experience what your customers will experience.

IE Building your own ML on top of your Java thing because you want your
customers to build ML on top of your Java thing would make sense. Building
your own ML on top of your Java thing just because it's yours does not by
itself make sense. (IE it has to provide some particular value over not
building it yourself, etc, otherwise it's pretty clear zealotry)

~~~
netheril96
Regardless of whether it is zealotry or not, Google certainly has a habit of
developing everything (software side) of its own.

~~~
DannyBee
When Google started doing it, there weren't a lot of good options to solve the
problems it has. As that has changed, so has Google (but like any large
company, it changes slowly)

------
deepGem
Very interesting. They are baking auto differentiation/gradient computation
directly into the compiler ? How will this be work, since you invoke TF and TF
computes gradients (at least on the loss functions) using Python/C++ from
first principles.

It'll be nice to use the Xcode build system, but given that Xcode isn't open
source, I wonder how this will play out. Is there a way to use Bazel natively
with Swift ?

Exciting times though !

~~~
chrislattner
The key thing here is that the Swift compiler is computing gradients using SCT
techniques, not TensorFlow. This has a number of advantages that we'll go into
in the white paper we are going to publish in April, so I'd prefer to wait
until that spells it all out rather than go into detail here.

All this works no matter how you choose to build your code: Xcode, Bazel,
SwiftPM, makefiles, whatever. Also, since ML scripts are frequently small, a
simple #! script like we showed in the demo is perfectly reasonable for many
things.

-Chris

~~~
deepGem
Great! Thanks Chris. Look forward to the white paper.

------
practice9
It's not clear what this is.

Is it related to Apple's Swift language?

~~~
harveynick
Yes, it’s language level support for machine learning (specifically
TensorFlow) in the Swift programming language.

~~~
stablemap
Lattner’s post makes this clear:

[https://groups.google.com/a/tensorflow.org/forum/#!topic/swi...](https://groups.google.com/a/tensorflow.org/forum/#!topic/swift/xtXCEvtDe5Q)

------
singularity2001
Am I the only one seeing this is a deliberate April's fools joke? My bet: The
project is real, but "Go for Tensorflow" instead of swift.

Quoth: "Is it me or did the audience had absolutely no clue what that
presentation was about?" One day too early but it couldn't emphasize "April"
more often in the article.

Quite a genius take on Apples openness and and wannabe-AI-ness.

~~~
grzm
I think you're reading too much into it. It was announced at the Developer
Summit.[0][1] Seems like a long ways to go for an April Fool's joke.

[0]: [https://medium.com/tensorflow/highlights-from-tensorflow-
dev...](https://medium.com/tensorflow/highlights-from-tensorflow-developer-
summit-2018-cd86615714b2)

[1]:
[https://www.youtube.com/watch?v=Yze693W4MaU](https://www.youtube.com/watch?v=Yze693W4MaU)

~~~
singularity2001
Will you buy me a beer if they rename it into "Go for Tensorflow" tomorrow
evening?

@Google: Hire me or ask nicely and I'll remove this so that your genius joke
is not spoiled.

~~~
sanxiyn
I am willing to bet 10:1 agaist this being a joke, that is, I give you $20 if
this is a joke, you give me $2 if not. Can pay with cryptocurrency. Mail me
(address on HN profile) if you are interested.

