
Zero-shot learning: Using text to more accurately identify images - pesenti
https://code.fb.com/ai-research/zero-shot-learning/
======
askmike
> WHY IT MATTERS:

> To become more useful, computer vision systems will need to recognize
> objects they have not specifically been trained on. For example, it is
> estimated that there are more than 10,000 living bird species, yet most
> computer vision data sets of birds have only a couple hundred categories.

I think it matters more for Facebook that they can apply these techniques to
learn more about the 4 billion people that are not on Facebook, I don't think
they care that much about birds ;)

~~~
ccozan
However, where do you find a nice and detailed text description of you?

next on facebook: block accounts randomly and reenable them if you can
describe a friend

~~~
cunzo
They have been doing something like that for years: if you get locked out of
your account, they show you pictures of friends and ask you to identify them.
I know this has nothing to do with what the article describes, I'm just
stating this for the record :P

[https://www.androidtipster.com/wp-
content/uploads/2016/08/fa...](https://www.androidtipster.com/wp-
content/uploads/2016/08/facebook_identify_friends.jpg)

(It's an old Mac and an old design of Facebook... Just so you can see how old
this screenshot is!)

------
nl
I'm sorry, but I work in this field and I find this so amazing that I don't
have any sensible comments to make.

I still find the fast.ai devise paper demo[1] amazing, and that is just
working on labelled but unseen classes via the ImageNet hierarchy.

In this case they do really fine grained classification on unseen classes,
with no hierarchy and instead exploit plain (noisy!) text.

This visual pivot thing is something I'm going to spend the next year coming
back to I think.

[1]
[https://github.com/fastai/fastai/blob/master/courses/dl2/dev...](https://github.com/fastai/fastai/blob/master/courses/dl2/devise.ipynb)

