
Apple to Start Publishing AI Research - rayuela
https://www.bloomberg.com/news/articles/2016-12-06/apple-to-start-publishing-ai-research-to-hasten-deep-learning
======
amelius
Of course! What researcher would want to work for a company that prohibits
scientific publications?

I've always been amazed by their attitude.

Look at Microsoft Research, and their enormous scientific output over the
years. IBM and Google look bleak by comparison, and Apple is not even on the
chart.

~~~
Cyph0n
If you consider the number of papers published over the lifetime of IBM, and
the fact that IBM Research does work in all levels of computing (materials
science -> software), Microsoft's research output looks minuscule in
comparison. But if you're considering only CS research, then you might be
right.

Here is a nice snapshot of the major contributions that came out of IBM
Research over the past 60 years:
[http://www.research.ibm.com/featured/history/](http://www.research.ibm.com/featured/history/).
_Edit:_ the invention of copper interconnects is enough for me to be frank
([http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/copperc...](http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/copperchip/)).

On a similar note, Intel is another company that is very active in research
and has published a significant amount of peer-reviewed publications. Samsung
Research is yet another; they have an amazing presence at circuit design
conferences, for instance.

~~~
j1vms
And all of these, in terms of _quality_ of research output, pale by about an
order of magnitude in comparison to Bell Labs for much of its existence
(particularly in the era ~1940 to mid 1980s). They were the R&D benchmark for
most improvements to IBM Research from the 1960s onward. Additionally IBM
sought out its alumni and populated their ranks with notable management and
research minds from the Bell Labs stable.

~~~
EGreg
I once read a Microsoft research paper. I think its central premise had a
fatal flaw:

[http://research.microsoft.com/en-
us/um/people/lamport/pubs/b...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/buridan.pdf)

The author confused continuity of the DECISION FUNCTION with continuity of the
OUTCOME CURVE.

In other words, an algorithm such as "keep trying to make a decision until you
notice that t > C, after which always pick the left one" will in fact NOT have
a continuous outcome curve, despite the decision function being continuous.

~~~
krenoten
That's a great example of how even some of the most influential minds are
fallible. Lamport is sort of the god of distributed systems, having invented
vector clocks, paxos which is how there is any sanity in modern data centers
(chubby, ZK, etcd, etc... are based on the core ideas of this), TLA+ for
reasoning about correctness, so much great stuff. Check out some of these!
[http://research.microsoft.com/en-
us/um/people/lamport/pubs/t...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/time-clocks.pdf) [http://research.microsoft.com/en-
us/um/people/lamport/pubs/p...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/paxos-simple.pdf)

~~~
EGreg
Amazing work :-)

------
zackkatz
First paper: "Snarky Reply and Joke Generation by a Virtual Assistant Using
Deep Learning"

~~~
themodelplumber
Didn't MS already try that on Twitter and it turned out to be a racist meme
generator?

------
mtgx
My hope is that Apple will continue research on _privacy-protecting_ and
_privacy-enhancing_ machine learning, because out of all the big tech
companies using machine learning, they may just be the only ones to do that.
Some random research for privacy technologies may come out of Google, too, but
they are much less likely to actually use them at scale, _especially_ if they
conflict with ad revenue.

~~~
numbas
Actually Microsoft has been publishing a lot on privacy enhancing Machine
Learning too.

For instance:

[https://www.microsoft.com/en-
us/research/publication/crypton...](https://www.microsoft.com/en-
us/research/publication/cryptonets-applying-neural-networks-to-encrypted-data-
with-high-throughput-and-accuracy/)

[https://www.microsoft.com/en-
us/research/publication/private...](https://www.microsoft.com/en-
us/research/publication/private-computation-on-encrypted-genomic-data/)

[https://youtu.be/xsaXMUelOEA](https://youtu.be/xsaXMUelOEA)

As for Apple: seeing is believing.

~~~
euyyn
And Google started publishing on differential privacy in 2014, way before
Apple used WWDC's marketing to paint themselves as the only ones using it.

~~~
givinguflac
Genuinely curious, where does google actually use differential privacy?

~~~
delroth
Chrome telemetry is one example I know of:
[https://www.chromium.org/developers/design-
documents/rappor](https://www.chromium.org/developers/design-documents/rappor)

------
hota_mazi
> Researchers say among the reasons Apple has failed to keep pace is its
> unwillingness to allow its AI engineers to publish scientific papers,
> stymieing its ability to feed off wider advances in the field.

I don't follow. How would preventing employees from writing papers would stop
them from reading papers?

~~~
leereeves
I think they mean that Apple isn't offering researchers the incentives they
desire, and may have trouble recruiting them.

~~~
freyir
Yes. Researchers like to publish their research because their publication
record the metric they're evaluated on by the outside world. If you can't
publish your research, that will significantly reduce your ability to get
hired by another top research group, industrial or academic, in the future.

------
maciejgryka
I'm just at NIPS and Apple does indeed have a decent presence, which is a good
sign (even if there are no actual publications yet). Super interesting to see
how this will play out compared to their past. There's lots of criticism that
we can throw at big tech companies, but the fact they many of them are so open
about their research and thus are forcing others to do the same is pretty
cool.

------
iverjo
Jokingly, this was the first apple AI research I saw online, about a month
ago: [https://arxiv.org/abs/1610.08120v1](https://arxiv.org/abs/1610.08120v1)

------
deepnotderp
Just goes to show you the market power of ml researchers, even apple has to
bend the knee.

~~~
pinouchon
I completely agree. But still I find it surprising that apple managed to hire
Salakhutdinov. He is a huge name in Deep Learning (with a focus on math,
unsupervised learning and autoencoders)

------
home_boi
The free market wins.

All the AI/ML engineers were going to FB/Google/etc. (and on some rare
occasions M$)

~~~
_1
> and on some rare occasions M$

Are we still doing that?

------
k_lander
So glad that this is happening. It is really inspiring to see the knowledge
sharing spirit become the expected default in the community. This is only
going to be great for progress in the field!

------
dovdovdov
Based on my Siri experience, I imagine a chimp pulling strings in a control
room. :)

------
Animats
Has Apple published anything yet?

~~~
brudgers
Conference slides have been tweeted.

------
deepnotderp
Yay! I never thought I'd ever see this though.

------
turingbook
Actually, Apple is one of sponsors of this year's NIPS conference:
[https://nips.cc/Conferences/2016/Sponsors](https://nips.cc/Conferences/2016/Sponsors)

------
serge2k
> Amazon.com Inc.’s Alexa

So Alexa is really the biggest thing in the market so far right? How many
papers does Amazon publish?

~~~
SEJeff
Not sure how many papers they publish, but I do really use this paper[1] from
James Hamilton (literally the Architect of AWS) as the defacto way of building
large distributed services:

[1] [http://perspectives.mvdirona.com/2014/07/challenges-in-
desig...](http://perspectives.mvdirona.com/2014/07/challenges-in-designing-at-
scale-formal-methods-in-building-robust-distributed-systems/)

------
melling
Story broke 6 hours ago:

[http://www.businessinsider.com/apple-is-finally-going-to-
sta...](http://www.businessinsider.com/apple-is-finally-going-to-start-
publishing-its-artificial-intelligence-research-2016-12?r=UK&IR=T)

Mac site picks it up 5 hours ago and it gets reported on HN:
[https://9to5mac.com/2016/12/06/apple-ai-researchers-can-
publ...](https://9to5mac.com/2016/12/06/apple-ai-researchers-can-publish/)

Bloomberg writes the same story an hour ago and it finally gets huge traction
on HN. My guess is that many people ignore the "new" page and it's all a
matter of luck that 3 or 4 people get it to the front page where a story takes
off.

~~~
JamilD
BusinessInsider and Mac blogs tend to be less reputable sources than
Bloomberg, whose articles regularly make it to the HN front page. Might have
contributed to this particular submission's success.

------
ktamiola
and what would it be? ... What have Apple developed that goes beyond currently
available peer-reviewed work, done by Deep Mind?

~~~
klodolph
That's the whole point of publishing, to share what goes beyond currently-
available work. But keep in mind that DeepMind is only one institution, not
the font of all things AI.

