
Facebook and Microsoft introduce ecosystem for interchangeable AI frameworks - runesoerensen
https://research.fb.com/facebook-and-microsoft-introduce-new-open-ecosystem-for-interchangeable-ai-frameworks/
======
cs702
Translation from _corporatespeak_ : "Our AI frameworks are losing developer
mindshare to TensorFlow, which is controlled by Google, so we are joining
forces."

Corporations like Facebook and Microsoft do not like interoperability... until
and unless it's in their best interest.

The goal here is to make AI frameworks easily interchangeable: build your
model with whatever framework you like; it will run unchanged on all other
frameworks/platforms/stacks.

Regardless of the motivations, I actually think this is GREAT NEWS, and I hope
that Amazon and maybe Apple, and eventually hopefully Google too will
introduce and promote compatibility with this standard.

We all benefit from more interoperability.

~~~
slackoverflower
You got it right. Tensorflow is HUGE in the AI community, Facebook and
Microsoft are scared. I, for one, am a huge fan of Tensorflow and don't plan
on learning anything new to do the work I'm already doing. Facebook and
Microsoft should just join Google's coalition and use Tensorflow company-wide.

~~~
blueyes
Actually, Tensorflow is huge in the data science community, but PyTorch is
taking over ML research. TF is last year's cool thing. That's one reason why
fast.ai just adopted Pytorch for their courses. Tensorflow is surprisingly
low-level and needs libraries like Keras to be used easily. The field of
Python ML tools moves fast...

~~~
lolive
I hope the ML community won't have to suffer the same kind of fatigue syndrom
that has plagued the javascript community for the last decade.

------
pgroves
Predictive Model Markup Language (PMML) was released in 1997 to solve a very
similar problem (but for lots of predictive models). I don't know that it ever
really caught on. That was also promoted by the tool builders but not by the
users.

While checking the wikipedia page, I'm surprised to see that it is still being
developed (with a 2016 release):
[https://en.wikipedia.org/wiki/Predictive_Model_Markup_Langua...](https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language)

Please note that in 1997 it was NOT ridiculous to use XML for this.

~~~
mindcrime
PMML is definitely in use. There are occasional compatibility issues, but for
the present, it is the standard for exchanging models.

------
jeremynixon
Automatic productionizability of PyTorch models through Caffe 2 will speed the
transition from research to production dramatically - this gives the
environment a chance to compete with Tensorflow / Tensorflow Serving.

Frameworks (say, DL4J) have been using Keras as a loose way to share models
between frameworks. It'll be fascinating to see if Theano / Tensorflow / DL4J
/ MXNet walk this path as well.

~~~
Q6T46nT668w6i3m
I was curious about their implementation since PyTorch and Caffe2 semantics
are very different. Unfortuantly, the authors write:

> Currently, our tracer works with many common neural networks, but not some
> of the more advanced programs in PyTorch such as those with dynamic flow
> control. Over time, we will enhance ONNX and the tracer to support these
> programs, so that developers can leverage full flexibility of PyTorch with
> the high-performance robust deployment capabilities of Caffe2.

It is useful, of course. But it's rare for contemporary models to not use
dynamic flow. In fact, PyTorch is popular because it encourages this dynamism.

------
agibsonccc
Hi, Adam from DL4j here. I'm really excited to see this. We will work on
supporting this format as well. We have a new pytorch like api coming out
(dynamic graphs etc), so this fits nicely in to that.

I know a lot of cynics are looking at this and are thinking this is all
politics. Honestly: I am glad something like this exists as a framework
developer.

We would like to build an easy to maintain set of benchmarks for perf
comparisons and have found that to be hard to do. We were originally going to
lean on keras with our model import for this, but if we can get say:

dynet and chainer on board with this as well, I think it will help the
ecosystem as a whole move forward.

(If you're asking "why" it's hard, main reason for us was the primary java
interface, we are fixing this with a new python interface based on pyjnius)

Overall, I don't think the OpenCL folks will be able to pull this off (the
field moves too fast). I feel like this is something the people who actually
build the tools should do.

Ultimately, I would be surprised if tensorflow didn't hop on board at some
point. A lot of ML models are being built in pytorch first now. No one loses
having that easily accessible.

~~~
hedgehog
Hey Adam, I don't know about the "OpenCL folks" but we're able to use OpenCL
to exceed TensorFlow + cuDNN throughput on the same hardware for real
networks. We (Vertex.AI) mostly care about embedded platforms and we currently
use Keras as our research front end but we're open to collaborations.

~~~
agibsonccc
I am talking about their neural net format khronos is working on. We likely
dont overlap. I am mostly focused on banking gov and telco. We are supporting
this out of interest in being able to run these neural nets in our
distribution. We have a few embedded use cases but we will likely write our
own graph executioner for arm and the like.

~~~
hedgehog
Ah, I don't know too much about what Khronos is doing. We have thought about
your work from the standpoint of supporting DL4j on a range of datacenter
accelerators, we do tune for NVIDIA etc with good results. We're pretty
framework and hardware neutral.

~~~
agibsonccc
Feel free to file an issue on nd4j. That is our tensor lib.
[https://github.com/deeplearning4j/nd4j](https://github.com/deeplearning4j/nd4j)

------
electrograv
_> To directly export this code, ONNX would have to support conditionals
[...]_

Does ONNX aim to serve as a human-readable and human-maintainable network
description for experimentation and research, or does ONNX aim to serve only
as a model export interchange format (primarily to be machine-interpreted)?

This could serve very well in the latter case, but would be quite problematic
in the former -- where conditionals, for loops, etc. are _essential_ to a
compact and readable representation of many network architectures.

Reading a static neural network architecture descriptor of a very deep neural
network is akin to reading assembly code from a compiler that does loop
unrolling. Static network descriptions work great when pushing a model to
production, but are much less suitable for the research and development stages
(IMO).

~~~
jsgo
my guess is it is the latter.

If I have my own framework and I want to operate with this standard, I need to
have the ability to map to and from ONNX's format. I don't think it is
intended for people to write directly in ONNX's format to maintain
compatibility. If it were, I'd imagine it would fall flat very quickly as
people wouldn't want to rewrite everything in the hopes that their framework
of choice will adopt it. Seems the intent is more like .odt to me than, say,
Markdown.

------
nicodjimenez
Great move by Facebook to look beyond Caffe's legacy prototext file format
which has become somewhat standard in computer vision, and unite Pytorch and
Caffe2. Unfortunately there are limitations to the neural network logic that
can be represented in configuration files, especially when dealing with
dynamic networks. Most networks used in vision are static so that's where I
expect this will add the most value. My only wish is that Caffe2 followed
Pytorch's interface. After spending 5 minutes with Pytorch it's pretty clear
that they got the interface design right.

------
jamesblonde
Michelangelo, announced by Uber 2 days ago
([https://eng.uber.com/michelangelo/](https://eng.uber.com/michelangelo/)),
use another format for storing models/features and for serving data. It would
be interesting to hear why they didn't use protobufs for storing features
instead of their own system. My guess would be that they have a mix of scikit
learn and tensorflow, and they didn't want to be too tightly coupled to
tensorflow. Still, this move by FB and Microsoft is what happens when you're
desperate - your frameworks are not being adopted, last chance role of the
dice.

------
TCM
Its interesting that this tool has no PATENTS grant.

~~~
jiayq84
Yangqing here from facebook. We consciously made it a MIT license as onnx is
intended to be widely shared by a lot of participants, and MIT seems to be
more widely agreeable among different parties co-owning it. It's also simpler.

~~~
rofrol
Oh, so now there was no danger of this nasty patent trolls, like with react
case?

------
nametube
While this is awesome I'm personally wondering when big companies like FB and
MS are going to standardize on a format for predictive models in general. [0]
PMML exists but there are very few companies that are standing behind it.

[0]
[http://dmg.org/pmml/v4-3/GeneralStructure.html](http://dmg.org/pmml/v4-3/GeneralStructure.html)

------
MikkoFinell
Just imagine the world we will live in, when Facebook or Microsoft wins the
race to general AI.

~~~
aianus
I imagine it will be like the Manhattan Project, impossible to contain to one
company or country for long.

------
protomok
I'm curious if FB/Microsoft considered Khronos Neural Network Exchange Format
(NNEF) - [https://www.khronos.org/nnef](https://www.khronos.org/nnef)?

------
pishpash
So "AI" is equivalent to "neural networks" nowadays, eh?

~~~
jeremynixon
That's not implied by the title. Neural Networks is a member of Machine
Learning, which is a member of AI. PyTorch, Caffe 2 and CNTK are Deep Learning
Frameworks. All Deep Learning Frameworks are AI frameworks, but not all AI
frameworks are neural network / deep learning frameworks.

------
megamindbrian
Why would you want to pile another black box on top of the black box?

~~~
starshadowx2
It's black boxes all the way down.

------
naturalgradient
Any TensorFlow devs who might be able to comment on plans to interoperate with
TensorFlow/reasons why it's not on the cards?

~~~
Q6T46nT668w6i3m
I am a regular TensorFlow (and Keras) contributor and I saw the announcement
when you did. In either situation, Google contributors partcipating without
speaking to the TensorFlow community or the Facebook and Microsoft authors not
reaching out to the TensorFlow community, it's a bad look. However, it's
entirely possible we'll hear more next week at the TensorFlow symposium.

------
sanxiyn
Now, it would be lovely if coremltools could support ONNX and convert ONNX
model to CoreML.

~~~
jiayq84
Yangqing here (created Caffe and Caffe2) - we are much interested in enabling
this path. Historically CoreML has provided Caffe and Keras interfaces, and
having ONNX / CoreML interop would help a lot for everyone to ship models more
easily.

Earlier in the year we provided compatibility between Caffe2 and Qualcomm's
SNPE library, which follows this similar philosophy.

~~~
Q6T46nT668w6i3m
Hi, Yangqing! Nice project. B)

I want to clarify that Apple advertises Keras support for use with CoreML, but
the converter uses a graph from the TensorFlow backend. It begs the question,
have you spoken with anybody from the TensorFlow (or Keras) communities about
collaborating?

~~~
fchollet
CoreML supports Keras but not TensorFlow because Keras models form a well-
structured subset of all possible TensorFlow graphs. It would be quite
difficult to support completely arbitrary TensorFlow graphs, but supporting
every Keras layer is relatively straightforward.

To answer your question: I had no knowledge of this ONIX project before the
public announcement today. Speaking purely for myself, if I wanted to develop
a universal model exchange format, the first step I would take would be to get
in touch with the makers of the frameworks that sum to 80-90% of the market
share -- TF, Keras, MXNet. But maybe such a strategy was thought to be
superfluous in this case -- for instance, because ONIX may not actually be
intended as a universal model exchange format.

~~~
liuliu
To be fair, CNTK (BrainScript) has quite impressive list of features to
support dynamic control structure (in a symbolic fashion, comparing to PyTorch
which delegated much of the dynamic control structure to underlying language
Python). I think Tensorflow and CNTK probably the only two frameworks pursued
such implementation strategy. IMHO, looking back, supporting control
structures may not be that useful (see the recent attention based models, all
of them can be unroll'ed to ordinary graphs), but it is so interesting to
implement!

------
lolive
I have heard about something similar in the open source community:

[https://deepdetect.com](https://deepdetect.com)

Doesn't it sound like a better long-term option than corporate-driven
political initiative?

------
Nocturnals
Does that mean that we can see the AI implementations facebook uses?

They invented the format so I'm sure we'll see everything facebook does in the
new ONNX format. From the placement of ads to whatever...

(sorry for the sarcastic comment)

------
evc123
ONNX plans to support TensorFlow as well:
[https://github.com/onnx/onnx/issues/3](https://github.com/onnx/onnx/issues/3)

------
liuliu
Doesn't seem to have support for control structure (If, While etc.).

------
sandGorgon
Why a new format ? Why couldn't protobuf, hdf5 or the Apache Feather/Arrow
format (which counts Hadley and McKinney as contributors).

I understand there could be something unique in this format, but really keen
to understand what.

~~~
jiayq84
Yangqing here (caffe2 and ONNX). We did use protobuf and we have an extensive
discussion about its versions even, from our experience with the Caffe and
Caffe2 deployment modes. Here is a snippet from the codebase:

// Note [Protobuf compatibility] // ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ // Based on
experience working with downstream vendors, we generally can't // assume
recent versions of protobufs. This means that we do not use any // protobuf
features that are only available in proto3. // // Here are the most notable
contortions we have to carry out to work around // these limitations: // // \-
No 'map' (added protobuf 3.0). We instead represent mappings as lists // of
key-value pairs, where order does not matter and duplicates // are not
allowed.

~~~
sandGorgon
pretty cool and thanks for that reply! Did you look at something like
Arrow/Feather, which is looking to get adopted as the interoperable format in
R/Pandas ... and maybe even Spark. There's been quite a bit of momentum behind
it to optimize it for huge usecases - [https://thenewstack.io/apache-arrow-
designed-accelerate-hado...](https://thenewstack.io/apache-arrow-designed-
accelerate-hadoop-spark-columnar-layouts-data/)

It is based on Google Flatbuffers, but is undergoing enough engineering
specifically from a big data/machine learning perspective. Instead of building
directly over Protobuf, it might be interesting to build it on top of Arrow
(in exactly the same way that Feather is based on top of Arrow
[https://github.com/wesm/feather](https://github.com/wesm/feather)).

~~~
jiayq84
Interesting data, thanks!

We chose protobuf mainly due to a good caffe adoption story and also the track
record of it being compatible with many platforms (mobile, server, embedded,
etc). We actually looked at thrift - which is Facebook owned - and it is
equally nice, but our final decision was mainly to minimize the switching
overhead for existing users such as Caffe and TensorFlow.

To be honest, protobuf is indeed a little bit hard to install (especially if
you have python and c++ version differences). Would definitely be interested
in taking a look at possible solutions - serialization format and the model
standard is mor e or less orthogonal, so one may see a world where we can
convert different serialization formats (JSON <-> protobuf as an overly
simplified example)

------
Markoff
can we send messages through it?

