
How a Pentagon Contract Became an Identity Crisis for Google - lego_bot
https://www.nytimes.com/2018/05/30/technology/google-project-maven-pentagon.html
======
more_corn
The great news is that this is news. Google did a thing they thought was fine.
They drew a line in the Sand and said we won't build a box that kills.
Employees said that's not good enough. Because you're building a box that
targets and makes killing easier.

I'm thrilled that this discussion is coming up and leaking out into the rest
of the world because we need to talk about it.

Because the smart, capable people here need to think about our ethical
obligation not to create terrible weapons. I assure you AI tracking in a
weapon system is a terrible weapon. The test is easy. Imagine yourself on the
looking at the sharp end of the weapon. Does it feel like you're in a
nightmare? That's a terrible weapon. Granted, smart, capable people know not
to misuse a terrible weapon, but the last thing you do when you create
something is the act of relinquishing control. And I assure you the world is
full of people who cannot or choose not to consider the repercussions of their
actions. A weapon once created will be misused.

We all therefore have an obligation not to create such things. Turn your
attention towards creations which create the world you want to live in and
never work on terrible weapons.

I hope that this comment causes a response. I hope that it kicks off a debate.
I trust that my argument is sound and I look forward to exploring the nuances
of it with all of you.

~~~
Chardok
Ultimately I think it was inevitable that Google was to become so heavily
involved in military ([https://wikileaks.org/google-is-not-what-it-
seems/](https://wikileaks.org/google-is-not-what-it-seems/)), but what bugs me
is how intertwined this now military-industrial complex has been in our lives.

It is at best misleading and worst incredibly dishonest for Google to portray
itself as it is, while at the same time using people to contribute to data
that goes into targeting and killing human beings. I am hoping more people
begin to consider the ethics of supporting such a company.

------
cowpig
I'm glad that google's internal culture is healthy enough for this. I wonder
how the internal conversations about affecting politics, issues of privacy,
etc. look..

However, I really think it would be better if these debates happened somewhere
that the public could see. It has to be a biased debate when every participant
is an employee of the company.

~~~
knuththetruth
Well, I think this why Google has been so good at controlling their employees
to stay within the margins of Google’s business interests. It creates a
vibrant illusion of consequential debate and “resistance,” while keeping all
the real power in the hands of a few executives.

Don’t get me wrong, Google employees have ways to put a stop to things like
this project. But they’re not to be found in these pseudo-battles of their
internal forums. Employees have to organize outside of company channels,
withdraw their labor, or quit like those who were already brave enough to do
so.

~~~
more_corn
This is absurd. The true power of democracy is freedom to speak. To disagree
to challenge. Google does this better than any big company I've ever heard of.
I literally challenged the CEO in public on the question of the housing crisis
in the bay area. He responded and the conversation continued. There have been
a dozen times in the last several years when healthy internal conversation at
Google has changed policy. Name one time that happened at IBM.

~~~
knuththetruth
>The true power of democracy is freedom to speak.

The true power of democracy is...democracy. Google doesn’t have a labor union
or worker-ownership, so while executives might be open to appeals, ultimately
what you’re describing is an authoritarian arrangement.

I do think employees and users would be better off if Google was
democratically controlled in either of the ways described above, but
unfortunately that’s not where we are right now.

------
vinceguidry
Companies like Google making company culture such a visible part of their
identity strikes me as the kind of wise foolishness that you find a lot in
mythical narratives. Sure, you will get some very motivated employees that
swallow the kool-aid, but if you think those employees' loyalties are with
you, you better think again. They're loyal to the idea that you sold them on,
to the gospel that you preached.

Once they get it in their heads that the road you're leading them on is going
to hell and not heaven, there's gonna be a reckoning. Now the faithful are
going to start questioning things they wouldn't allow themselves to question
years ago. Is making all this software so that Google can profit off of
collecting people's data worthwhile?

I hope at least one Google exec is forced to sell a yacht over this. But what
I really want is that we find a way to force more accountability onto these
assholes. I'm happy it happened, but I think we're going to start seeing a
shift away from companies making culture such a visible part of their identity
as Google did.

~~~
exelius
I think you’re very wrong.

Companies like Google have trained an entire generation to seek greater
meaning in their work. Even if Google’s promises were hollow, millennials
don’t have the cultural institutions our parents did. Thus we seek self-
fulfillment in the ways we know how: at work.

I work with a lot of C-suites on “the culture issue”. They see statistics like
“40% of people between age 12 and 18 identify as LGBTQ” and they start to
panic because their company culture isn’t so friendly to that. Things that
were taboo only a decade ago are now commonplace: every kid today knows at
least one trans kid, and a over half of teens identify as “somewhere in
between gay and straight”.

The institutions that used to prop up culture in this country (marriage,
church, family, etc) are no longer a reality for most people. Corporations
have lobbied successfully for more power, so the employees of these companies
expect to be able to wield that power from within to make the change they want
to see.

Because the culture at large is so hostile to those ideas, the culture at
companies is only going to become more important as time goes on. Companies
that ignore it are going to have a difficult time attracting the top talent
that can afford to be choosy. Companies that provide a safe, nurturing refuge
will find themselves drowning in qualified applicants.

~~~
brandmeyer
> They see statistics like “40% of people between age 12 and 18 identify as
> LGBTQ” ... and a over half of teens identify as “somewhere in between gay
> and straight”

Citation. Needed.

~~~
dagw
_over half of teens identify as “somewhere in between gay and straight

Citation. Needed._

Possibly referring to this poll: [https://www.independent.co.uk/news/uk/home-
news/yougov-poll-...](https://www.independent.co.uk/news/uk/home-news/yougov-
poll-homosexuality-half-young-people-10458032.html)

If you think about it, it's not too surprising. 12-18 is a weird age with lots
of weird hormonal things happening, and it's hardly unheard of that otherwise
'straight' teenagers in that age bracket experience some sort of same sex
fantasies or experimentation.

It's just that in the past they would identify as "100% totally straight!!!
(except for that time...)" but they're now more comfortable to be more fluid
in their definitions.

~~~
exelius
Exactly. Those kids will eventually reach a company, look around and see it’s
95% cisgender/heterosexual people, and they don’t feel welcome because that’s
not the world they grew up in.

The whole “trans agenda” is really about letting kids know that they are not
weird, and they are not morally bound to the “default” gender they were given.
Turns out when you do that young, a lot of people don’t pick the default
without at least exploring the other options. This blows many adults’ minds,
and it’s going to be a huge shift in the corporate world. You think #MeToo is
a problem? Wait till gender discrimination has a whole lot more variables.

Point is, culture is important. The culture that got you here is not the
culture you’re going to need tomorrow. Google’s culture is pretty close to
what a lot of this generation aspires to though; and doing work to enable
state sponsored surveillance is likely personally threatening to their LGBT
staff given the current president’s agenda.

~~~
tenpies
> Google’s culture is pretty close to what a lot of this generation aspires to
> though; and doing work to enable state sponsored surveillance is likely
> personally threatening to their LGBT staff given the current president’s
> agenda

That brings up a good point about ideology (pro-LGBT) versus politics (Trump
hate). Almost every country on the receiving end of the US military under
Trump can be characterized as a terrible place to be LGBT. The same can be
said about Obama, but I suspect most Google employees would not bad an eye if
this were happening under Obama.

~~~
exelius
It’s not _just_ pro-LGBT; it’s a whole liberal ideology that values people of
all kinds, and looks to transform the culture to include all that diversity.

Trump is very much against this ideology... but he’s also reminded the country
after 8 years of Obama that the government is not your friend. I think the
combination of those two things is what caused the backlash at Google; and
you’re right, I don’t think it would have happened under Obama.

Edit: despite all the crap, the US is still one of the best countries to be
transgender. At least the medical system here doesn’t _prevent_ anyone who
wants care and can afford it from getting it.

~~~
vinceguidry
Okay now I've figured out my objection. I strongly doubt that the slow liberal
ratchet is going to suddenly force companies to become accountable for their
actions in the same way Google has made itself. Google employees were more
than capable of ignoring certain aspects of Google's business in order to work
at Google.

My argument is that Google's professed ethos of "Don't be evil" has directly
led to this sequence of events. You can be LGBTQBBQ-friendly and pay lip
service to the whole host of liberal society acceptance tropes and still be
evil, so long as you don't actually claim that you'll never be evil. Once you
claim that, _and make it a core part of the company mythos_ , then you're
opening yourself up to exactly this sort of thing.

I love the accountability that's being forced onto Google right now, but I
feel it's a rare bird at best.

~~~
exelius
I think companies that don’t have some statement of corporate ethics are going
to find themselves starved for talent while the ones who do eat their lunch.
I’m seeing it happen across the professional services industry; the big 4 are
consistently outperformed by smaller shops that bring a more diverse team.

The “don’t be evil” pledges are a big part of why people feel safe at those
companies. Millennials and especially the generation after them need to be
involved with institutions that reflect their values. They are more than
willing to go create them if they don’t exist.

~~~
vinceguidry
That's a good argument and I hope you're right.

------
V-eHGsd_
> However, he said he thought that it was better for peace if the world’s
> militaries were intertwined with international organizations like Google
> rather than working solely with nationalistic defense contractors.

I remember when sergey said nearly the same thing about china.

~~~
JMCQ87
I wonder what he thinks "international" means.

------
tim333
>However, [Brin] said he thought that it was better for peace if the world’s
militaries were intertwined with international organizations like Google
rather than working solely with nationalistic defense contractors.

You could make an argument that to maximize global wellbeing that that is the
better strategy? Focus on stuff to avoid civilian casualties rather than the
nastier possibilities? Maybe spin it off into a specific division so people
don't have to work there.

AI is going to get used in warfare but we may have a choice between kill all
the humans or take out the weapons while not killing humans.

------
reacweb
Often I hear about people who refuse some contract because of ethical
reserves. But, we should have more people with ethical sensibility working on
military projects, not less. If the only people working on military projects
are sociopaths, what future are we building?

~~~
s_kilk
This presumes that the people with ethical sensibilities have any power to
change the structure of the MIC from the inside. On the contrary, I'd say a
fundamentally rotten institution can't be reformed by a few good apples
joining up to help turn civilians into red streaks on the ground.

~~~
Chardok
Hell, even the "head of Stanford University’s A.I. lab and chief scientist for
A.I. at Google Cloud, one of the search giant’s most promising enterprises"
who literally said “I believe in human-centered AI to benefit people in
positive and benevolent ways. It is deeply against my principles to work on
any project that I think is to weaponize AI.” had little to no effect on
Googles direction towards military applications. She couldn't even get a
publicized email out.

The article certainly supports your theory.

------
WaxProlix
Technologists who fancy themselves pacifists are naive. The history of human
technological advancement is the history of violence and the desire for
control. DARPA funded the early internet, DARPA funded AI and NLP research,
DARPA funded computer vision.

It's not new, it's been going on forever - certainly in our lifetimes.

~~~
_85oz
Hey - I am a soon to be Xoogler who just gave notice in ethical objection to
Project Maven (btw, I hugely appreciate even simple words of support over the
issue from this community).

I don't identify as a pacifist - quite the opposite, I think we should fight
Project Maven like hell. The DoD wanted to fund a communications network?
Cool. A private company recruiting 10k+ engineers on a cool, creative "don't
be evil" brand, and then backdooring their work into use by a defense
department is not cool (I'm not saying the company doesn't have the legal
right to, I'm saying it's a jerk thing to do).

Sharing research, open-source code, etc is fine, and of course defense is free
to poach away top engineers. But they can do that through recruiting, not
their IT budget. Or if Amazon and Microsoft want to do the work, that's their
choice, I never wanted to work at those companies. I think Google is missing a
huge opportunity to define itself uniquely here by not canceling immediately.

I believe true neutrality is to do defense for nobody, I would like there to
be a premier technology company that international colleagues can work at that
serves no military. I wonder how many of my coworkers can't express their
solidarity because of their visa status, because the insane cost of living of
the bay area, etc. I don't know.

~~~
siruncledrew
Good for you for sticking to your moral compass.

Is it going to change anything? Probably not. At the end of the day everyone
is complicit is some respect. Part of those taxes taken out of your paycheck
are consequently going to defense and the military. The US spends 35% of the
world's share of military expenditures[0]. Project Maven was going to happen
with or without Google's help, it's just a matter of when. Who knows, they
might even be using TensorFlow.

At the end of the day, there's little "defining" for Google to do. Google is
not some angel child that fell into dating a bad boy with the DoD. Google does
both good and bad things all the time, even when the consequences are not as
highly publicized as with Project Maven. Google is a for-profit company that
was following the money. Figureheads like Eric Schmidt dove right in to
working with the DoD without even giving it a second thought, and there was
little news from any C-levels on the matter prior to this PR issue for Google,
which I think speaks for itself about the philosophy of the company from its
leadership. Cynically put, Google now cares because their reputation, and by
proxy their fiscal performance for their shareholders, is now threatened and
somebody calculated that the risk is greater than the reward of the government
contract.

[0]
[https://en.wikipedia.org/wiki/List_of_countries_by_military_...](https://en.wikipedia.org/wiki/List_of_countries_by_military_expenditures)

~~~
dmihal
> Is it going to change anything? Probably not.

Google spends incredible amounts of money hiring and retaining people. They
may not lead to any change, but I'm sure Google is keeping track of how many
people are leaving over this issue.

~~~
sleepydog
Googler here. Really good engineers are difficult to replace, our hire/no-hire
ratio for candidates is insanely small and and increasing recruiting efforts
puts more burden on the engineers who do the interviews. Losing people over
this issue definitely hurts.

~~~
nogbit
No need to worry, lots of people you can pick up from the following companies
they may want to shift a few levels up from hell on the morality spectrum.

Lockheed Martin Boeing Raytheon General Dynamics Northrop Grumman United
Technologies L-3 Communications BAE Systems

------
CORDIC
I find this entire situation so odd. None of these people leaving Google over
Project Maven had any issues with working for the largest corporate spy agency
in the world. They were fine with massive data collection in the pursuit of
serving people better ads. But helping analysts find objects in imagery?
That’s a bridge too far.

~~~
grepsign
What's so hard to understand?

I'm an aerospace engineer and I specialize in rockets. I understand that the
nature of my work will almost always 100% benefit the weapons industry because
there's just no avoiding it. When I started my career ~10 years ago I worked
on a defense project as I was a fresh graduate and had no choice, but nowadays
I make it very clear to prospective companies that while I have no issues with
my colleagues working on weapons, I myself will absolutely, under no
circumstances, work on defense funded projects in any capacity. Most companies
from my experience are willing to work with you on this especially because a
lot of them are already set up for segregated access.

The fault in your thinking is that you seem to think it needs to be an "all or
nothing" choice. It's not, and it doesn't need to be. I try to be green and
recycle everything I can, but I don't lose sleep over throwing plastic away in
the garbage because _I do what I can_. I'm 99% vegetarian because of my views
on animal ethics and I honestly can't remember the last time I've bought meat
for my personal consumption, but I have no qualms about going to a friend's
backyard BBQ and eating the same grilled burgers that everyone else is
enjoying because _I do what I can_. For this reason, I choose to not work
directly on weapons, despite the fact that basically all rocket related
technologies can be applied to weapons, because that is something that I can
not change, so _I do what I can_ , which in this case is refusing to work on
defense funded projects.

~~~
hueving
But you're clearly not _doing what you can_. You're just _doing what 's
convenient_.

You still eat meat in one of the most important times not to eat meat (in
front of others) if you are truly driven by ethics for animals. You refuse to
work on defense-funded projects because companies in your field have a pretend
isolation mechanism (here's a hint: unless they are actually separate
companies, they aren't isolated) that allows you to think you aren't
supporting the military despite working for the company supporting the
military.

This type of behavior shocks idealists. Even though it's better than nothing,
just doing little bits here and there where convenient actually makes it
harder to change the status quo for idealists (because you think you're
helping when you're not).

~~~
Melkman
Then, as an idealist, what do you expect from other people to reach your goal
? If 50% of the people do 80% of what is needed you're at 40% of your goal. If
10% of the people do 100% you're only at 10% plus you've turned your goal in
to an "us vs them" instead of "everybody for the greater good". Voting with
your wallet 80% of the time is a valid means to support what you think is
right. An idealistic minority doesn't change the world. The majority does.

~~~
grepsign
Exactly.

I think a more CS way to think about it is shipping out a MVP vs. shipping out
a 100% perfect, 100% feature complete, 100% bug free, 100% documented,
product.

I think everybody would be happy to start a project and ship out that 100%
finished product, but you'll spend your entire life working on this and die
without succeeding because it doesn't exist. So the choices are to 1) not
bother, 2) do what you can (MVP), or 3) work until you die as a failure. OP
would want me to choose 1 or 3, it seems.

------
ggg9990
Why has this gone on so long? Surely the optics are worse than the revenue at
this point. “We hear you, we are canceling this, we deeply apologize,” blah
blah.

~~~
Eridrus
This is the start of a play for the JEDI contract, which is estimated at $10bn
over 2 years. This would make that one contract responsible for ~1/4 of all
Google Cloud revenue.

I also think that while there is a lot of gnashing of teeth, it's not really
clear that the optics here matter. Amazon & Microsoft are already doing these
sorts of projects with no backlash. This is really only a story because there
are people inside Google speaking out.

What's pretty clear from these threads is that people who are already dislike
Google, dislike this too, but it's not clear that this is having an impact on
people's opinions who otherwise have a positive image of Google.

~~~
MichaelMoser123
your mention of a big contract sounds true, also i somehow got the impression
that we did not get all the details in this story. But then JEDI seems to be
about creating a cloud for the DOD [1], whereas this contract seems to be
about image recognition; but what do we know...

But then: why didn't they create an alphabet company for military research ?
(you can even call it Google Mars or Ares). Would have been more
compartmentalized away from the rest of the company (that's how the military
likes it).

also interesting that this item seems to have disappeared from the front page
- is this an editorial practice here with discussions that have gone 'bad'?
(idea for a new application: track HN stories that have been on the front page
but then disappeared from it).

[1] [https://fcw.com/articles/2018/03/07/cloud-jedi-dod-
contract....](https://fcw.com/articles/2018/03/07/cloud-jedi-dod-
contract.aspx)

------
eternalban
[https://youtu.be/Lbjru5CQIW4](https://youtu.be/Lbjru5CQIW4)

[https://en.wikipedia.org/wiki/Information_Awareness_Office](https://en.wikipedia.org/wiki/Information_Awareness_Office)

------
mathinpens
my only comment is that the opportunity to stop weaponized AI is long long
long gone. the barriers to entry for modern machine learning methods are just
so low and the military benefits are there. Near peer states are going to do
this and we frankly cannot stop them. the train has left the station...

sidenote: my impression is the next block of the small diameter bomb has a
semi-autonomous feature where it can glide over a target area without a pre-
sighted target-use some computer vision-recognize a tank- adjust its
trajectory to hit the tank. all without a man-in-the-loop. pretty nifty.

edit: also these jobs are literally hiring right now. you can polish up your
cv and send it to a DOD lab and in a few months you will be using deep
learning to kill. SICK!

~~~
gwenzek
How is using unmanned drone to automatically kill people around the world
different from blind terrorism? It's just a matter of precision.

I agree it's inevitable that someone ill intentioned use ML + drone to create
deadly weapons, I don't understand why USA is doing it, nor what are they
trying to accomplish like that.

Drone strick is already a main argument in terrorist propaganda, I don't see
how increasing the number of attacks and decreasing human intervention is
going to help that.

~~~
mathinpens
Well currently the unmanned drones do not automatically kill anything. Each
drone has `pilot' who runs that drone and I believe only that drone. that
pilot is in arizona somewhere instead of in the plane.

look I am pretty critical of american foreign policy and our current state of
imperial dominance but I mean...calling it blind terrorism misses a huge
amount of nuance.

most drone strikes are very targeted in that they try to kill specific groups
of people in very specific locations. However, in a counter insurgency war,
from 20,000 feet + the fog of war it is quite easy to drone a wedding for
instance.

> I agree it's inevitable that someone ill intentioned use ML + drone to
> create deadly weapons, I don't understand why USA is doing it, nor what are
> they trying to accomplish like that.

Do you really not understand it? People in government think this technology
will have substantial military benefits and they don't see any compelling
reason not to develop the technology.

> Drone strick is already a main argument in terrorist propaganda, I don't see
> how increasing the number of attacks and decreasing human intervention is
> going to help that.

I don't really know if more autonomous weapons will result in more air
strikes/drone strikes or less. I doubt it would have any huge effect on the
number.

Moreover I don't really see increasing automation will have much bearing on
propaganda. I think people in the middle east don't like drones because their
family, friends, and neighbors are killed. The degree of automation in the
kill chain is probably the last thing they are thinking about as they grieve.

------
hugh4life
I have no pacifist objections to Google partnering with the Pentagon, I just
don't think it's very smart for what amounts to be an information platform to
be opening itself too easily to accusations of bias(even though everyone would
know it's there anyway).

One scenario I would plan for if I were Google is if with the emergence of
webassembly that an open source browser-based mobile OS like Firefox OS might
become more viable. I could definitely see some nations(I.E. Russia and
China... but I could see South Korea's phone manufacturers being in favor...
and many EU nations too if it were open source) being very uneasy with a US
Corporation that is a pentagon contractor whose government already threatened
to withhold android software to ZTE. With getting close to the Pentagon and
after the ZTE threat, it will be harder to make a fuss if Russia and China
were to outright ban Android.

------
FrozenVoid
>which uses artificial intelligence to interpret video images and could be
used to improve the targeting of drone strikes.

Is that related to "select all tiles with X" on captcha? It often had other
labels..
[https://twitter.com/lachrob/status/831691552028241921](https://twitter.com/lachrob/status/831691552028241921)

------
finnthehuman
I am amused that _this_ of all things was what it took to get googlers to stop
and take a look at what they're doing.

I work somewhere that has talent bouncing to/from a military contractor and a
common attitude I see is way more comfortable with the military doing "what it
takes" within the rules of engagement to protect the nation than what our own
golden boys in the valley are doing for dollars.

~~~
drivingmenuts
Well, one way of looking at is: Google may or may not make shitty products
that steal your info, but they don't kill people.

The military kills people.

~~~
kinsomo
> Well, one way of looking at is: Google may or may not make shitty products
> that steal your info, but they don't kill people.

> The military kills people.

And, ironically, if the US military is ineffective at killing people, more
people might be killed or other catastrophic human rights compromises might
need to be made. Focusing too much on "the military kills people" is missing
the forest for the trees.

------
xXGwarSlaysXx
How is this any more or less morally objectionable than literally anything
else Google has done, up to and including YouTube?

------
noetic_techy
Funny how the supposed progressive and tolerant won't contribute to the
defense of the society that allows them to be progressive and tolerant.

~~~
zwkrt
There are many places one can be tolerant without having to contribute to the
US government.

~~~
Fins
There's that pretty old saying about people who do not want to feed their own
army ending up feeding a foreign army. It's actually quite accurate.

