
Microsoft Research wins image recognition competition - espeed
http://venturebeat.com/2015/12/10/microsoft-beats-google-intel-tencent-and-qualcomm-in-image-recognition-competition/
======
wyc
"Baidu does not show up in this year’s rankings. The company made more
submissions than were permitted and ultimately apologized and fired the team
leader who directed juniors to make the unacceptable submissions."

~~~
sherjilozair
Might be worth noting that it is very likely that Google, Facebook and Baidu
did not even participate in this ImageNet. It is widely believed in the vision
community that this dataset is at the end of its run, i.e. any subsequent
improvements in performance are not due to scientific breakthroughs but
through hyperparameter optimization and ensembling.

~~~
Estragon
Surely that's not true until someone achieves human-level performance with
their entry.

~~~
abrichr
Human level performance was surpassed earlier this year:

[http://arxiv.org/abs/1502.01852](http://arxiv.org/abs/1502.01852)

~~~
natch
I won't be surprised when human level performance is surpassed, but isn't that
paper based on work that has the advantage of knowing the full test dataset in
advance? You can say "yes but they only trained on the training data" but that
doesn't rule out tweaking across several experiments and measuring them
against the known test data, then cherry picking the experiment that was best
overfit to that data, right? I'm not saying there are shenanigans here, just
wondering how you know there are not.

~~~
argonaut
The test labels are not known to them. The test labels are held by a different
team that tells them their test error rate. Furthermore, multiple, separate
teams have surpassed human performance.

FWIW, "human performance" is not as meaningful as most people make it out to
be, because this is such a narrow task.

~~~
tim333
The paper mentions it's unfair to humans to have to tell say a coucal from an
indigo bunting (both blue coloured birds). I've never heard of either so I
guess I would have got that wrong but it's not the fault of my visual system.

~~~
p1esk
But you only have to guess right once in 5 tries!

------
dplarson
It's mentioned in the article, but here's a direct link to the paper on arXiv:
[http://arxiv.org/abs/1512.03385](http://arxiv.org/abs/1512.03385)

And direct link to the PDF:
[http://arxiv.org/pdf/1512.03385v1.pdf](http://arxiv.org/pdf/1512.03385v1.pdf)

------
geomark
I'm disappointed to hear this after watching the video of Andrew Ng's talk. He
was very proud of Baidu's accomplishment in this competition.

------
free2rhyme214
What products does Microsoft use this technology in?

I've seen what Google's doing in Google Photos but I've noticed it's not
accurate all the time.

~~~
jacquesm
> What products does Microsoft use this technology in?

[http://venturebeat.com/2015/11/11/microsoft-launches-
project...](http://venturebeat.com/2015/11/11/microsoft-launches-project-
oxford-apis-for-face-tracking-emotion-speaker-recognition-spell-checking/)

~~~
free2rhyme214
Developer API's. It would be nice to see a consumer use case in Windows.

------
mikeskim
Is there a private one shot hold out in this particular competition?

~~~
smhx
yes. more like 5-shot hold-out, and the held-out set is pretty big as well, so
more legit.

~~~
mikeskim
so you mean there are 5 entries you can select to be scored in the end, and
there is one private leaderboard no one can see but the admins? do you know
the size of the private leaderboard test set?

------
dharma1
Has anyone implemented this yet for Tensorflow or Caffe? Also, any more
details on the MSCOCO segmentation part?

------
robgibbons
I'd like to have seen how IBM would have fared with Watson.

~~~
skadamat
Watson doesn't have much unique computer vision capabilities unfortunately.
They did some good work in NLP a decade ago but not added much since

~~~
douche
Have they not really added much to it since the Jeopardy stunt? We get some
periodic rumblings about integrating it with a customer-service tool, but it
is starting to sound a little Duke Nukem Forever

~~~
papercruncher
Does Chef Watson count? It's actually pretty neat as a consumer tool, it
suggests the weirdest things but somehow they taste fine. No idea about the
inner workings though

[https://www.ibmchefwatson.com/](https://www.ibmchefwatson.com/)

~~~
kristopolous
These are "invented" by the computer right? I just had a suggestion for

shrimp, peanut, olive oil, ginger, garlic, grapefruit juice, blood orange,
dill, coriander seed, turmeric

That's a really odd mix but it honestly seems ok ... although I'd sub the
shrimp for salmon ... apparently I can in the interface. Nice.

Also the ingredient input seems to lack exotic things like frogs,
rattlesnakes, camels, durian, or ostrich. But quinoa and quail eggs are there.

