
Deep learning may need a new programming language - bobjordan
https://venturebeat.com/2019/02/18/facebooks-chief-ai-scientist-deep-learning-may-need-a-new-programming-language/
======
xvilka
What about Julia[1]? Seems like a perfect fit[2].

[1] [https://julialang.org/](https://julialang.org/)

[2] [https://juliacomputing.com/domains/ml-and-
ai.html](https://juliacomputing.com/domains/ml-and-ai.html)

~~~
orbifold
It is not typesave and doesn’t really support programming in the large. I
think the best bet is something like Swift or Microsoft could come out with a
C#/F# compatible thing.

~~~
mark_l_watson
I was curious about Julia, mostly to experiment with the Flux machine learning
library. I tried some ‘general purpose’ Julia programming: network IO,
consuming RDF data, some natural language processing/text processing, etc.

I was specifically interested in the question ‘could Julia be my general
purpose language?’. In reality, I am too much into Lisp languages, Haskell,
etc. to seriously want a new language, so this was just a two evening
experiment.

~~~
dunefox
> In reality, I am too much into Lisp languages, Haskell, etc. to seriously
> want a new language, so this was just a two evening experiment.

But what did you think of Julia?

~~~
mark_l_watson
Julia is a very nice language. Still, I would not use it for a large project
with many developers: similar situation as Python.

------
ovi256
>Python ... the language forms the basis for Facebook’s PyTorch and Google’s
TensorFlow frameworks

The state of journalism today ... their Github repo web pages show that
there's more C++ than Python in both repos.

~~~
throwawaymath
To be fair, while that's true I think what's meant here is that Python is the
language of choice for _using_ Tensorflow and PyTorch. Probably no other
language is used to interface with deep learning libraries and primitives as
much as Python.

~~~
ovi256
>I think what's meant here is that Python is the language of choice for using
Tensorflow and PyTorch

So it's a good editor that's lacking too, besides the fact-checkers.

------
sandGorgon
This is the reason why swift for tensorflow initiative was started by Google -
[https://github.com/tensorflow/swift/blob/master/README.md](https://github.com/tensorflow/swift/blob/master/README.md)

Facebook will obviously go its our own way. Given their investments in the JS
ecosystem, im hoping they end up choosing Typescript for this.

~~~
dtech
Is Facebook adopting Typescript or are they still full-in on Flow, their own
competing typed Javscript alternative?

~~~
sandGorgon
[https://news.ycombinator.com/item?id=18918038](https://news.ycombinator.com/item?id=18918038)

Jest is a Facebook project for testing React. They moved from Flow to
Typescript.

[https://github.com/facebook/jest/pull/7554](https://github.com/facebook/jest/pull/7554)

Dont want this thread to go OT. But I do think Typescript will be an awesome
counterpart to Swift-for-Tensorflow on the ML side.

------
thecatspaw
Im sorry, but this very much feels like a fluff piece. Yes we _may_ need a new
programming language, or we may not.

Half of this article is about hardware, while only a small part of it is about
programming languages.

> There are several projects at Google, Facebook, and other places to kind of
> design such a compiled language that can be efficient for deep learning, but
> it’s not clear at all that the community will follow, because people just
> want to use Python

Are the libraries not implemented in native code? The brief mention that
python gets does not really detail what is wrong with it, aside from not
beeing compiled I guess. But that is no issue if the libraries are native, so
I dont see a reason to move away from python, let alone create a new
programming language for it

~~~
lugg
You don't have to check baby's diaper to know they went number 2.

Python ML code is a bit of a joke. If you're any kind of semi professional
developer who sees deep learning code for the first time and doesn't say "what
the fuck would you do it like that for", I simply don't know what to tell you.

Im not saying a new language will fix this problem but some new primitives and
idiomatic options protecting people from themselves probably wouldn't hurt the
industry.

Likewise, something more ergonomic might actually improve onboarding new
developers into the space.

~~~
ageofwant
poo-poo, people will insult your sensibilities using whatever language you
choose, the "problem" is not them, or the language or its idioms, its you.

If you write ML code for money, you are a pro, the end. Python is not perfect,
but everything else is worse, far worse. What is the product that falls out of
a ML pipeline ? ML pro-tip: its not hand-crafted artisanal code.

~~~
lugg
My complaints about ML prettiness were unclear but to clarify it's usually
criticism aimed at library authors who often come from very academic
backgrounds with very poor sense (lack of exp?) of just how lazy professional
developers are. Case in point, you don't care if it comes out as poop because
you're getting shit done. Hell it seems like you don't even care if you have
to wade in it for another 5 years.

Kudos, but what if i told you if the tooling authors could have ensured you
never wrote poop but still got shit done?

------
amelius
The main problem, imho, is that (for best performance) most libraries require
the programmer to create a data flowgraph, and to think in terms of this
graph. However, this is the perfect job for a compiler. In mainstrain
compilers, dataflow analysis has traditionally been the task of the compiler,
so it seems silly to break with this tradition. A new (compiled) language
could bring us back on track.

~~~
spinningslate
I'd probably argue the opposite. Dataflow is a natural way to think about ML
activities. The problem is that no mainstream languages offer dataflow as a
first class construct; at best it's a set of libraries. That creates some
impedance mismatch between intent and implementation. That compilers already
do dataflow under the covers would hopefully mean that implementing a first-
class dataflow language wouldn't be too hard.

~~~
giornogiovanna
What would a language primitive for "dataflow" look like, and what would it
offer over an implementation as a library?

~~~
spinningslate
There are a few things, nicely covered in "Concepts, Techniques and Models of
Computer Programming" by Van Roy and Heridi [0]. Briefly:

1\. First-class dataflow variables 2\. Syntax for creating flows (c/f pipe in
*nix shell) 3\. Transparent parallelisation

All of these things exist in libraries of various sorts, but can feel
cumbersome compared to native syntax.

[0] [https://mitpress.mit.edu/books/concepts-techniques-and-
model...](https://mitpress.mit.edu/books/concepts-techniques-and-models-
computer-programming)

------
posnet
I know Nim has some interesting working going on here,
[https://github.com/mratsim/Arraymancer](https://github.com/mratsim/Arraymancer)

As well as that esolang that appeared a few months ago on the front page.
[https://github.com/mrakgr/The-Spiral-Language](https://github.com/mrakgr/The-
Spiral-Language)

------
widforss
It would be hilarious if everybody suddenly shifted to a Prolog implementation
with ML functionality ¯\\(°_o)/¯

~~~
XuMiao
I like the idea.

There are two types of ml jobs. One for modeling and another for computation.
The former deals with the communication with human. The latter deals with the
communication with machines. I would prefer that a statistical logic
programing language (maybe extended from prolog) for the modeling part. A
strong compiler tool or service that compiles it into any machines, on cloud
or edge. Most of the time, what human likes to achieve is small and
declarative. But for any small modifications of the program, we have to think
about the computational flow, deal with the type errors and tune the low level
efficiency of CPU and GPU. It's a waste.

I like the settings of SQL where modelers use SQL to mine data while engineers
maintain the performance of the computational engines.

------
Kip9000
There's already Nim ([https://nim-lang.org/](https://nim-lang.org/)), which is
Python like syntax and statically typed, with easy parallelisations etc
already. What's lacking is the adoption as it wasn't hyped up. There's rarely
a need to create yet another language and wait till it becomes mature and
fixed all the issues with the eco system etc. If at all what's required is a
way of translating all the Python libs to Nim or some similar effort.

~~~
dunefox
In my opinion, I'd rather see F# or Julia succeed.

------
pts_
Why did everyone dump C++? Oh wait, they didn't.

~~~
pjmlp
Depends on the context.

On GPGPU programming certainly not, in fact latest NVidia's hardware is
designed explicitly for C++ workloads.

On GUI frameworks, C++ no longer has the spotlight on OS SDKs that it once
had.

~~~
pts_
My point is all AI worth its salt (robots) is C++ based in production.

~~~
pjmlp
There I fully agree with you, specially with NVidia doing C++ in hardware.

~~~
eggy
NVidia is adopting ADA/Spark for autonomous vehicles work [1]. It's service-
proven, compiles to C/C++ speeds, and is safe. I learned Turbo Pascal in
college in the 80s after Basic, C and assembler in the late 70s, early 80s,
and I am attracted to languages like Haskell, Julia, Lisp, and J/APL, yet
after toying with Spark, I think it is probably a good fit to do safe, ML at
C/C++ speeds. It would be easy to hook it into all the C++ of TensorFlow too.

[1] [https://blogs.nvidia.com/blog/2019/02/05/adacore-secure-
auto...](https://blogs.nvidia.com/blog/2019/02/05/adacore-secure-autonomous-
driving/)

~~~
pjmlp
Yeah that as well.

I belong to the same fanboy club, which is kind of why I do like C++ and not
so much about C.

I just don't see many adopting it without legislation enforcement, which is
why the focus is on autonomous vehicles, where Ada already has a good story.

~~~
eggy
If I am not trying to write verbose, safe code with things like Rust or Spark,
I still have a thing for low-level C vs. C++ bloat for fun stuff. I am going
to commit this year to Spark, and probably Rust if Spark doesn't work for me,
and then C++ after Rust. I have not used C++ in years, so I need to look at
the latest and greatest before passing judgement.

------
risubramanian
I wonder what he thinks about Julia. There are lots of projects to turn it
into an "ML language".

~~~
dunefox
I think it was explicitly designed as an "ML Language" from the ground up
already. Do you mean packages like Flux and KNet?

------
anigbrowl
I'd much rather a deep learning system that could handle simplified natural
language and file operations, and then grow a network for specific tasks.
Surely the point of AI is to liberate humans from having to write everything
in code, and especially from needing to learn another language (which he
fairly observes few people want to do).

I do everything flow-based these days. It's not the fastest way (and I also
have the luxury of not having to please anyone but myself), but it allows me
to only think about my domain problem instead of programming language issues.

~~~
mark_l_watson
Maybe not quite what you have in mind, but I have tried saving trained Keras
models, loading them with Racket (a modern Scheme language), and implemented
the required runtime. My idea was to encapsulate trained networks like
‘functions’ for use in other languages.

~~~
dunefox
And?

~~~
mark_l_watson
and... I think this is a neat idea and I thought other people might enjoy
trying it. code (2 repos) is on github

------
melling
Sounds like FB is also working on its own AI chips:

[https://www.bloomberg.com/news/articles/2019-02-18/facebook-...](https://www.bloomberg.com/news/articles/2019-02-18/facebook-
s-ai-chief-researching-new-breed-of-semiconductor?srnd=technology-vp)

[https://www.ft.com/content/1c2aab18-3337-11e9-bd3a-8b2a211d9...](https://www.ft.com/content/1c2aab18-3337-11e9-bd3a-8b2a211d90d5)

------
henning
Fortran is going to make a big comeback. I can feel it.

~~~
enriquto
> Fortran is going to make a big comeback. I can feel it.

It is indeed one of the few existing languages appropriate for generic
experimentation in numeric programming.

For example, computing the product of two matrices in C or Fortran is exactly
as fast by writing three nested loops or by calling a library function. In
python, julia, octave, etc, the difference is abysmal. This is a very sad
state of affairs, that forces a mindset where you are allowed a limited
toolset of fast operations and the rest are either slow or cumbersome. If you
want to compute a variant of the matrix product using a slightly different
formula, in Fortran it is trivial change, you just write the new formula
inside the loop. But in python or julia you are stuck with either an unusably
slow code or you have to write it in another language entirely.

"Vectorized" operations are cool, elegant, and beautiful. But they should not
be the only tool available. As differential geometers often resort to
coordinates to express their tensor operations, so should programmers be able
to.

~~~
cshenton
Have you used Julia recently? Vectorised operations are just as fast relative
to hand written loops as c and FORTRAN, you just need to appropriately
annotate @inbounds and @simd. Sure that’s more work, but removing safety
checks should be explicit.

~~~
enriquto
Not recently, thanks. I will surely try!

My main gripe with the julia interpreter was that it was ridiculously slow to
startup (I was using it as a "calculator" from within a shell loop: each
iteration spawned a julia to perform a simple matrix computation). Does this
performance has improved recently?

By the way, what do you mean by "removing safety checks should be explicit" ?
This sounds like a problem that the language should be able to deal with
itself without bothering the programer (e.g. if the bounds of the loop are
variables of known value, it can be checked beforehand, so the bounds checks
can be safely omitted).

~~~
dunefox
AFAIK it was designed for long running processes, not for quick start up time.
On the other hand, I don't find it that slow for experimentation on the REPL.

------
tzhenghao
We may need a programming language for DL, but I doubt it'll happen soon, if
it even happens. Lindy effect working in favor of Python here, as many data
scientists have prior "big data" experience in it, and typical software
engineers from the "scripting/tooling" world.

People have been calling for the phaseout of C/C++, but even today's most
popular DL frameworks have backends written in C++ in lieu of Rust.

------
qwerty456127
> Deep learning may need a new programming language that’s more flexible and
> easier to work with than Python

Whoever has imagination more rich than mine, how can a programming language
can be easier than Python? What's difficult in it?

The only thing I find inconvenient in Python is you can't simply put every
function (including every member functions of a class - that's what I would
love to do) in a separate file without having to import every one of them
manually.

~~~
4233456
Read it again. He/She did not imply that Python is hard.

I do believe that he/she meant that they need way higher levels of abstraction
than Python.

Let's say Python is current C, they want something X that's like Ruby in
compare to current C.

~~~
qwerty456127
It's just hard for me to imagine a higher level of abstraction. That's why I
invite whoever has imagination more rich than mine to suggest ideas.

~~~
mehh
Have a look at Prolog, that might expand your mind a little.

------
slack3r
Yes. Deep Learning definitely needs a new programming language. Uber's Pyro is
worth checking out. ([https://eng.uber.com/pyro/](https://eng.uber.com/pyro/))

~~~
wenc
Umm that's not a programming language as such, it's a probabilistic
programming DSL built on top of Python/Pytorch. It's one level of abstraction
up.

------
thosakwe
I’m not the biggest Python fan, but I can definitely tolerate it, as long as I
have a decent way to use the models from another language.

------
pi-victor
i always thought Haskell would be amazing for this task.

but python is really popular and easy to grasp. you'll end up writing a new
programming language and people would still use python in the end.

------
kensai
I thought that language was Julia.

------
return0
More likely a markup language

------
Scarbutt
sql

------
known
1\. Features 2. Performance 3. Usability

You can pick only two options;

~~~
awestroke
Or use Rust to get all 3!

~~~
semi-extrinsic
I'm sorry, but Rust certainly doesn't score high on usability. Compare e.g.
the "Guessing Game" intro from the official docs, just look at how much more
complicated both the code and the tutorial is as compared to how you would do
the same in Python, heck even in Fortran:

[https://doc.rust-lang.org/book/ch02-00-guessing-game-
tutoria...](https://doc.rust-lang.org/book/ch02-00-guessing-game-
tutorial.html)

~~~
adrianN
I don't think comparing 100 line programs is a good benchmark for usability.
Most software is much larger and the design choices you have to make for a
programming language should favor programs with at least a few thousand lines
of code.

~~~
pjmlp
Then try to do a few thousand lines of code in Gtk-rs, including custom
widgets, the issues with borrow checker and internal mutable struct data
accessible to callbacks is what lead to the creation of Relm.

~~~
adrianN
The GUI story in Rust is currently terrible, I agree. But writing GUIs in
Python with wrappers to C(++) libraries is not exactly nice either ;)

~~~
pjmlp
To me it looks pretty nice.

[https://www.qt.io/qt-for-python](https://www.qt.io/qt-for-python)

------
nottorp
This guy - or the article's author - hasn't heard of DSLs?

