
Facebook’s image outage reveals how the company’s AI tags your photos - saravana85
https://www.theverge.com/2019/7/3/20681231/facebook-outage-image-tags-captions-ai-machine-learning-revealed
======
furyofantares
Lots of comments about how this isn’t new, or how it’s related to
accessibility. The article mentions all this. That’s not what the story is.
It’s not a very deep story, but I don’t think it’s quite so trivial as is
being suggested.

The story is about this info being exposed to users just browsing the site,
and the reaction that elicits. Someone seeing the tags inline in their feed by
a glitch will tend to have a very different response than someone who is asked
to view source and look for it.

I believe this is relevant because a regular conversation around here centers
around the idea that many folks in tech circles find it creepy or unhealthy
for society to have so much info being collected and processed for profit by
giant corporations, and how most people outside of tech circles seem not to
care at all.

~~~
NoodleIncident
That is an incredibly charitable interpretation of the word "reveal" in the
headline. Every piece of information can be "revealed" to someone by that
definition, it's happening all the time, and it wouldn't be a newsworthy
event.

~~~
SmellyGeekBoy
"Reveal" means to show something that was previously hidden from view - what
has happened here is pretty much the textbook definition of the word.

~~~
DougBTX
The implication is that a secret is being revealed, possibly a dark secret to
worry about. But FB and others put content in alt tags for people to read when
they can’t see the images (for example die to blindness), it is much less
dramatic to “reveal” something that was intentionally put there for people to
read.

------
_Wintermute
Maybe I'm used to my slow internet connection, but I've been seeing these tags
before the images load for a couple of years now - I thought it was common
knowledge.

~~~
jug
Besides - both Apple and Google openly reveals that they auto-tag images by
allowing you to search your albums on these sites with the tags as a feature.

~~~
logo2k
And Microsoft too

------
mevile
This stuff is in the HTML, you can see it with web inspector, or I think just
by hovering your mouse over the image because that looks like it's in the
title attribute for the image. This isn't it like something new was exposed
inadvertently. I'm surprised The Verge wrote an article about this.

~~~
sdrothrock
I think it's less about "look at these [creepy] things" and more about "hey
guys, you saw all this text when Facebook broke, this is what it is." At
least, that's how a lot of my non-tech friends seem to be taking it.

~~~
mevile
It's creepy on Facebook, but with any cloud photo storage, like Amazon's or
Microsoft's, being able to search for "ocean" or "plants" in your own photos
is really fucking cool.

~~~
hombre_fatal
Note: You can do this in Apple's Photos.app and the analysis is done 100%
locally on your computer.

------
ablation
You could also see this if you viewed source when it was fully working, of
course. And occasionally before images loaded in if you were on a slow
connection. I don't think it's any great secret, but still interesting to see.

~~~
ahje
I was thinking the same thing. I've noticed this multiple times in the past,
but I guess alt-tags are too archaic for Average Joe.

------
burk96
Not only is this nothing new as other commenters have observed, these tags
have been used by screen readers and other accessibility services for awhile.
While we can talk all we want about Facebook using their all seeing eyes for
evil (which I agree they overstep a lot of boundaries), these tags are very
useful for people who may require them and should not be scrutinized as an
invasion of privacy.

------
pepijndevos
What I think is more interesting about this outage is that Instagram and
WhatsApp were affected. This means your images are stored on the same servers,
and possibly subject to the same AI scanning. I wonder if WhatsApp images are
E2E encrypted...

------
twodave
IMHO by now if you're still on Facebook you've made a conscious decision to
let a business with an awful track-record of respecting its users' privacy
basically have carte blanche over whatever you give to them.

I'm sure there are a few folks out there who don't realize this, but most
people I talk to about this say they a) hate it but b) don't want to leave the
platform because it lets them keep in touch with people. To them, it's worth
it. Not to me.

------
silviumatei
I know for sure that this technology improves life of visual impaired persons
as special software will read respective text.

------
kabwj
It’s funny how what purely is an accessibility feature done to help
“differently advantaged folks” (how these journalists would say) is being
turned against them by the media.

Reminds me how they are vilifying open APIs and explaining them as a way
Facebook “gives away your data”.

~~~
dvtrn
Reading some of the comments right here, it’s interesting watching some eat it
right up and go right along with it.

Facebook is doing a lot with user data I have serious problems with. An
accessibility feature like his probably ranks dead last or close to it.

------
gaucheph
it's not about what it means for this community, where we all know the
shortcut to the dev console in chrome. this is about regular users of Facebook
having their own "they live" moments on social media. where they can see
plainly how Facebook regards them as consumers. "oh just show him pictures of
DRINK and TEXT, that's all he's into anyways".

~~~
lukey_q
"They Live" is a great comparison. Of course this isn't new to someone on HN
who follows this technology and the companies using it for a living, but for
my uncle who just posts pictures of his dog it's a bit shocking to see that
the computer knows that it's (likely) a picture of a dog next to a tree, even
if he was aware on some level that computers these days are capable of that
kind of classification.

------
UglycupRawky
Just alt-text. Granted it is a slight insight though, as I assume it's added
automatically, but I expect they pull a lot more info from people's images
than just what the alt-text shows.

------
codepie
There's also an extension which overlay the alt tags over images:
[https://github.com/ageitgey/show-facebook-computer-vision-
ta...](https://github.com/ageitgey/show-facebook-computer-vision-tags)

------
YeGoblynQueenne
>> Everyone knows the bit in The Matrix when Neo achieves digital messiah
status and suddenly sees “reality” for what it really is: lines of trailing
green code.

Given that the Matrix was a simulation, what it "really is" can't have been
trailing green code. What it "really is" must have been electric charges
stored in memory circuits. What Neo saw was his own mind's interpretation of
the software and hardware underlying the Matrix.

Though of course the point about the green text is that it was a spiffy
effect, and not a supposedly accurate representation of what computer memory
"looks like".

~~~
tehjoker
I believe the green text is also taken from a recipe book but the designer
won't reveal which one.

------
mburst
Looks like alt tags for accessibility purposes. Seems like a win win for all
users. Easier to find photos and the ability for visually impaired users to
get context on what their friends are sharing

------
SpaceInvader
This is nothing new, I've noticed that several months ago and talked about
that with one of my security friends and he confirmed that Facebook is tagging
pictures at least since 2016.

~~~
likpok
Here's the announcement, even: [https://code.fb.com/ios/under-the-hood-
building-accessibilit...](https://code.fb.com/ios/under-the-hood-building-
accessibility-tools-for-the-visually-impaired-on-facebook/)

------
iodbh
I think I've never seen worst GDPR compliance : big "I Consent" button with no
negative counterpart, except a link to the cookie policy where the "opt out"
section invites you to disable cookies in your browser and manually edit local
storage. Is that a joke ?

------
mobilio
If you're using mobile Facebook like - 0.facebook.com then this is well known
for you.

------
gdfasfklshg4
And in other news people using screenreaders have been viewing these texts the
whole time...

------
buboard
"Reveals". God journalists are lazy these days

------
djsumdog
So ... feed images into Facebook to break Captcha?

------
CriticalCathed
Dumb article. It's the alt text.

~~~
gdfasfklshg4
This is true. Maybe this was posted to remind us of how unknowledgable the
average computer user is regarding the web?

------
basgrootarch
Why do they put this data in the alt-tag in human readable format? For what
purpose or use? SEO on a personalized platform with very limited access by
Googlebot? Just to show they can? It makes no sense to me.

~~~
K2L8M11N2
For accessibility?

~~~
basgrootarch
I'd be surprised if they even care about that, but who knows, it's a
possibility

~~~
andreigheorghe
You'd be surprised if the largest content delivery platform in the world cares
about being accessible to the broadest range of users?

[https://code.fb.com/ios/under-the-hood-building-
accessibilit...](https://code.fb.com/ios/under-the-hood-building-
accessibility-tools-for-the-visually-impaired-on-facebook/)

