
Facebook’s AI unlocks the ability to search photos by what’s in them - kosei
https://techcrunch.com/2017/02/02/facebooks-ai-unlocks-the-ability-to-search-photos-by-whats-in-them/
======
philsnow
Google photos does this. I just opened Google photos and searched for "food"
and aside from a couple pictures of some jack o lanterns (which are arguable,
but I don't know anybody who eats theirs on Nov 1st), the results were spot
on.

"Unlocks" might not be the right word here.

~~~
david-given
If I search Google Photos for 'dog', the top seven hits are, in order: a fuzzy
toy, a possum, another picture of the same possum, a cat, a pair of goats, a
sheep, and I think it's a pheasant. The dog is #8.

Of course, if I search Google Photos for 'me', the top hit is a frog, so maybe
it just hates me.

 _Update_ : not any more! Now the frog is #2. The top hit if I search for 'me'
is the MV _Isle of Lewis_ car ferry, photographed on the island of Barra in
the Outer Hebrides.

~~~
Analog24
I never used this feature until despite using Google photos extensively (I
have ~60 GB of photos on there, granted that's 60 GB on my computer which
isn't with the same compression/quality that Google uses). I have to say that
I'm extremely impressed. it took multiple attempts to even find a mistake
("mountains", "city", "cat", "guitar", "food"). Although I was a little
disappointed to find no results when I searched for "me". Being a frog is
better than not existing at all.

~~~
maverick_iceman
You can add your name to a photo of yours (or any name to any photo). Google
will then automatically find all of your photos. When you touch search look at
the face of people that shows up below.

------
yunolisten
Seriously, a 51.6MB "gif" that doesn't really add anything to the article?

~~~
iklos55
Really off-topic, but: Are there (tech) news-sites which do not have this
10-100MB bloat with three js framweorks and gifs, ads and whatnot? I'd gladly
pay for a minimalist news source...

~~~
CaptSpify
It's kind of a PIA, but I enjoy using umatrix and ublock origin. Allows me to
only turn on the stuff that I want.

~~~
naranana
[https://github.com/gorhill/uBlock/wiki/Per-site-
switches#no-...](https://github.com/gorhill/uBlock/wiki/Per-site-switches#no-
large-media-elements)

------
googleapologist
I don't see how this is unlocking anything. Google has been doing the same for
the past 5 years.

------
jxy
Google photos does this through their server.

Apple Photos does this through your phone, or your laptop. (Data from these
two are not synced???)

------
planetjones
Must be unlocking the same thing that Amazon Prime Photos is doing. I have to
say that scanning photos automatically and allowing me to search by what is in
the photo is the only solution to cope with the proliferation of photos,
especially from camera phones. I still curate my better photos in Lightroom
and publish them to Flickr albums. But that approach just isn't scalable for
all the "normal" photos I am taking with the iPhone; so privacy concerns
aside, I am happy for a machine to help me find the photos I am interested in.

------
Raphmedia
If you want to see what Facebook tag your photos you can install this chrome
extension: [Show Facebook Computer Vision Tags ]
[https://chrome.google.com/webstore/detail/show-facebook-
comp...](https://chrome.google.com/webstore/detail/show-facebook-computer-
vi/elafbihhbfmfihdflghclaclcilcnmie)

------
newscracker
In the recent times, for some other unrelated reasons, my browsing of FB pages
and people's profiles was a bit slow from the network and download side (I
always use a browser to visit Facebook, not the app). I noticed how accurate
Facebook was in classifying photos when I saw the alt text captions for photos
(before the image loaded). It was impressive, being able to say approximately
how many people were there, whether a person was smiling, whether a photo was
a selfie, whether a person was standing, whether a picture contained a
landscape, trees and many other things.

It was also creepy that Facebook knew so much, because though I use Facebook
for specific purposes and try to limit what I put there, I didn't imagine that
images could convey so much information/metadata that would be useful to the
underlying platform.

------
zaque1213
Recently on my phone I tried to copy and paste a picture from Facebook to my
messaging app. Instead of copying the picture, it copied a text message that
read "picture of dog and three people." I'm sure this is nothing new, but I'd
never seen it before and thought it was interesting.

~~~
hatsunearu
This reminds me, (very unrelated) a few years ago (almost 10), when I dragged
and dropped an image from Wikipedia with Google Chrome to my Windows XP
desktop, and to my surprise it actually saved it with a file name that is the
description of the picture, not some arbitrarily chosen name or the Wikipedia
file name. Sometimes it was really long and descriptive, like "A man wearing
red and a woman wearing blue are playing soccer on a field blah blah blah".

I wondered where that came from, and I looked for that description on that
image's page on Wikipedia and couldn't find the same description anywhere
(keep in mind, I was a middle schooler or something and it could very well be
the case that I overlooked something) I was a little spooked, and I thought it
was some sort of hidden feature in Chrome that labeled images like that.

Does anyone know what was happening? It still bugs me to this day :P

~~~
Kronopath
Could it have been the alt text for the image? I believe the grandparent
comment was also pasting the alt text, since Facebook auto-generates alt text
for blind users using ML (this is mentioned in the article).

------
ijafri
now i know why facebook has been doin this.
[http://i.imgur.com/zPapJHM.png](http://i.imgur.com/zPapJHM.png)

~~~
zapt02
I believe that was for accessibility reasons, ie screen readers.

~~~
mdrzn
but it also allows you to do the opposite things, like search for "sky" and
find that image.

~~~
zapt02
didn't know that, cool!

------
fapjacks
Facebook's search is not useful. It's (obviously) segmented based on your own
network, and generally useless for practical purposes. Perhaps law enforcement
will find some use in this, but most people won't. Or at least, any utility
beyond "Oh, remember that time I took a picture with that weird dog in it?"

~~~
bpodgursky
Your power of imagination is truly inspirational.

~~~
chrischen
He's talking about its current public potential, not idealized potential.

For example go on facebook and use the search and tell me one interesing use
case.

------
justinjlynn
Thinking of just how difficult this is, I'm reminded of
[https://xkcd.com/1425/](https://xkcd.com/1425/).

~~~
neilcarpenter
And see [http://parkorbird.flickr.com/](http://parkorbird.flickr.com/)

~~~
timbre
Also [http://birdsnap.com](http://birdsnap.com) and
[http://merlin.allaboutbirds.org/photo-
id/](http://merlin.allaboutbirds.org/photo-id/)

------
jeffrey-sean
We have a free app that uses AI to find unprofessional Facebook, Instagram,
and Twitter photos used by over 50,000 college students:

[https://www.renpup.com/](https://www.renpup.com/)

