Hacker News new | past | comments | ask | show | jobs | submit login
‘We Kill People Based on Metadata’ (nybooks.com)
177 points by Libertatea on May 10, 2014 | hide | past | web | favorite | 22 comments



The N.Y. Review has been very strong in reporting in depth and critically on the post-9/11 security state, and (before the mainstream media caught on) on the thoughtless run-up to the Iraq invasion.

Mark Danner has been writing a series of articles there re-examining the Cheney legacy (two recent ones are here: http://www.nybooks.com/articles/archives/2014/mar/06/darknes... and http://www.nybooks.com/articles/archives/2014/apr/03/dick-ch...). They have been just brutal.


It's odd that in all of what is essentially the fallout from 9/11--two wars, subversion of American rights, spying on Americans, torture, rendition, massive security/surveillance state, etc.--no one seems to have ever seriously asked "why do these people want to kill us".

Beyond the quotable but facile "they hate us for our freedom" line, there hasn't been much real discussion of this sort.

And, here, as we parse the latest round of bills, legal interpretations, infringed freedoms, etc., there is still no such earnest discussion about why we find ourselves in this position in the first place. It's just assumed that this is somehow where we "should" find ourselves, and the only question is how much further we should go in our response to our inevitable position.

EDIT: At this point, I would go beyond the question of why these people want to kill us and ask who and how many are there who actually do want to kill us?


> And, here, as we parse the latest round of bills, legal interpretations, infringed freedoms, etc., there is still no such earnest discussion about why we find ourselves in this position in the first place.

Some of it is possibly history going through its usual cycle. As a corollary to Santayana's legendary quote: the only thing people ever learn from history is that nobody ever learns anything from history.

For an interesting look on the history of modern terrorism, I recommend watching at least the first two parts of this German stand-up: http://www.youtube.com/watch?v=n4H_E8b-qmo . Yes, it's inflammatory, and yes, it's politically slanted. That's the point.

The way to find unexpected things in common wisdom is to look at it from a different angle.


> "they hate us for our freedom"

Besides being not true in principle (they couldn't care less if you're free) even if they did it would stop working as soon as you're no longer free.

In reality it is much simpler than that: The US has (rightly or wrongly) become the de-facto superpower and has been projecting its power all over the globe with abandon. That creates a (very predictable) backlash.

So those that want to kill you want to kill you out of sheer frustration with having their plots foiled and their ability to operate restricted and they figure (rightly) that the way to strike back is to target the home country and hope for the leaders of the country to mess up and alienate themselves from their allies.

From that point of view 911 is mission accomplished.


> no one seems to have ever seriously asked "why do these people want to kill us".

Plenty of people have asked that question. That the answer isn't what you expect doesn't mean the question wasn't asked.

It turns out that the goal is to establish (or rather, re-establish) the Caliphate, or at least some twisted takfiri version of it.

In order to establish the Caliphate you need somewhere to put it -- namely, the Muslim countries of the Middle East.

But the countries already have states running the show. In AQ's viewpoint, they are "the near enemy". The near enemies have supporters too, including the U.S. (which was "the far enemy"). Other enemies include the United Nations (both since the UN Charter is incompatible with takfiri Sharia extremism, but also because the UN recognizes Israel), Shi'a Muslims, etc.

In one of the key decision points for OBL post-Gulf War, he decided to focus on the far enemy first instead of the near enemies.

At this point, the "crime" of the U.S. wasn't drones or bombing brown babies or anything so snide. The U.S.'s only crime was putting infidel soldiers on Arab soil to evict Iraq from Kuwait, and for maintaining regional stability for what I'm sure were completely self-serving reasons that provided no benefit whatsoever to the rest of the world.

Accordingly AQ attacked the U.S.: The 1993 World Trade Center bombings... the 1995 "Bojinka" airline bombing plots (yes, plural) that was only discovered by accident at the last second by an alert Philippine policewoman... the 1998 embassy bombings... the 2000 attack on USS Cole in Aden, Yemen.

The U.S. response was remarkably restrained throughout this entire period. Despite hundreds of Kenyans alone being killed by AQ in the 1998 bombings, the U.S. response was limited to dropping cruise missiles on known AQ training camps in Afghanistan and a chemical production facility in Sudan that showed evidence of working with AQ to produce nerve gas. The evidence was not ironclad, mind, but that was it: No boots on the ground, no roving drone patrols, no NSA massive surveillance efforts.

The problem for AQ wasn't what the U.S. was doing to the rest of the world by 2000, the problem was that the U.S. wasn't doing enough. AQ had somehow failed to get America's serious attention, and had failed to evict America completely from the Middle East, so they decided that they had to go bigger. And they did.

So while the facile "they hate us for our freedom" line is not the answer, it's not far off: They attack because Muslims still live in either approximately free societies, or tyrannies under the wrong dictator (i.e. apostate regimes). There will be no end unless they establish the Caliphate, and even that doesn't necessarily represent the end, since the stated goal of this new caliphate by Ayman al-Zawahiri is to extend jihad against the U.S. and the West.

If you want to research yourself the motives of the various and many insurgent/terrorist groups out there then there's certainly no lack of material, from the very many people who have asked the same question you say has never been asked. I'd recommend going through some of it. Articles asking (and trying to answer) this question have been published since long before 9/11. Huntington's "Clash of Civilizations" article dates to Bosnia.


I appreciate your thoughtful response and I have, in fact, heard this mentioned over the years. I am not sure that it is entirely accepted as the rationale (and, in fact, it has been disputed by some), but it has certainly be suggested.

I should probably have been clearer in my initial question. While there has been some discussion about in different circles regarding the genesis of this "problem", it has largely been muted. At a minimum, the volume and quality of earnest discussion has been remarkably limited relative to the importance of such discussion. Instead, we get pablum such as "they hate us for our freedom", even while we drone strike, destroy, and kill.

Remarkably, the theory that you repeated fits with this pablum as it puts virtually no emphasis on our actions, save for a passing mention or two. Somehow, even with a long and complex history of involvement in the region, we magically have no responsibility for any of wha is happening, save for being free and not Muslim. I find that absolutely incredible and profoundly myopic. Still, even if one can somehow remove the notion of consequences for previous behavior as a cause of 9/11 and what preceeded, and also manage to detach all responsibility from the West, there should be little doubt that our current course of action (yes, including drone strikes), provides indisputable cause for real grievances borne of our actions and, potentially, a generation of terrorists that modern history has yet to see.

Or, to put it in your terms, unless the goal is to kill every single Muslim or sympathizer who could have either answered the call for the Caliphate or been affected by the loss of a loved one who was presumed to have done such, then it seems that our current course is either exceptionally stupid or specifically designed to create perpetual conflict.

But, back to my original point: all of this that we are discussing here is incredibly important and lurks beneath the surface. It has brought us down this path of surveillance, etc., yet it is not questioned or discussed. Think of how fantastically stupid it is that we are in such a reactionary mode that we no longer question key assumptions regarding why we are doing what we are doing in the first place. Even if you believed that it's all about the Caliphate, shouldn't that single word be part of our daily discusion for as long as we ar at war, drone striking, and spying on Americans? Shouldn't we be openly discussing what it means and how we might work to not radicalize members of that community? Shouldn't there be a sober assesment of Western policy and how it might intersect with legitimate complaints and even radical ideas? This, instead of an attitude that "they hate us for our freedom", followed by more drone strikes, spying on Americans, etc.?

Apologies for length. Just hoping to better make my point.


> Somehow, even with a long and complex history of involvement in the region, we magically have no responsibility for any of wha is happening, save for being free and not Muslim. I find that absolutely incredible and profoundly myopic.

U.S. history is not at long as you think, nor as one-sided as you might think. Such history as does exist is due only because the British decided that they could no longer maintain any regional presence in the Middle East with their other Cold War commitments. Sure, you have your 1948 counter-coup in Iran (who are, notably, mostly the opposite sect of Islam than the ones we're talking about in this thread). But there's also the 1956 U.S. invention in favor of Nasser in Egypt that everyone conveniently forgets to mention when they're talking about U.S. meddling in Muslim affairs, to speak nothing of the very Bosnia case I noted in my last comment.

But U.S. involvement in the Middle East was limited otherwise for a long time precisely because there was a strong Iran allied to the West. It wasn't until the Shah fell for good in 1979 that you saw the Carter Doctrine and the invitation for American presence from Gulf states worried about their ability to export oil.

What I find to be incredibly myopic is the idea that if someone, anyone, has a problem with a group, that it is the group's responsibility and even fault.

You could give in and do what they ask for as long as you like, but you'll find at least two things are true: 1) Their demands are rhetoric, not their actual key fault with you (as proven with Bosnia and U.S. support for Afghanistan 1979-1989), and 2) you'll simply "radicalize" some other group who would be put at risk by a worldwide Caliphate.

> Shouldn't we be openly discussing what it means and how we might work to not radicalize members of that community?

Sure. People are. I don't understand why you keep asking why people aren't doing something that's actually being done. President Obama's speech at NDU in May 2013 made almost precisely the same point, and I assure you it's not because the President just had a whim that day.

I certainly hope you don't get all your news about U.S. and the West's foreign policy from HN (or God forbid, cable news), as otherwise I'm at a loss to explain where you get this impression from. The discussion is going on, but you do how to go try to find it, it's not like it's happening in the "To the Editor" section of the New York Times.

But as long as people keep giving credence to the idea that even perceived slights committed by some amorphous politico-cultural nexus are legitimate bases for terrorist retaliation, then I'm not sure what you expect for corrective action.

The way to fix that might be to help educate the masses of people who act as the best recruiting base for Islamist and other extremists, but good luck doing that without scaling up to something even worse than an occupation like Afghanistan's. As it stands now, Islamists control the educational apparatus at these places and they have no incentive whatsoever to teach their youth anything other than that the U.S. and the West are evil, whether it's true or not.


>U.S. history is not at long as you think, nor as one-sided as you might think

Well, I'm pretty sure it's about as long as I think. But, admittedly, it may not be as long as you think I think. :)

The salient point is that there is a history, it is complex, and it is reasonable to consider that it is significantly responsible for disdainful sentiments toward the West. I'm not sure that's in dispute anywhere. Also, note that I frequently reference the West and not just the U.S.

But, really, I think the history lesson itself doesn't really address my point, which is precisely that such historical discussions are not a part of our (the public's) reckoning with regard to the symptomatic issues with which we are dealing.

I know you've expressed bewilderment at that statement, but your reasoning for that bewilderment is, effectively, that the history is known and has been discussed somewhere by someone (even Obama). We agree there, though the degree of causality for everything that we are seeing is arguable. In any case, where we really differ is simply that my view is that such discussions should not just be somewhere between someone, but should absolutely dominate the public discourse. Instead, it is largely absent from the public arena, which really provides cover for policymakers to push through agendas that are orthogonal to solving the real problem, and instead serve other interests. Here, they have moved the goalposts from solving the problem to providing the public with a set of false choices based on the foregone assumption that the problem is well-defined somewhere and otherwise unsolvable.

So, the question is allowed to become not "why do they want to kill us?", but "how much power should the government have in invading our privacy?" At some point, the causal element is so far divorced from the symptomolagy, that any real relationship between the choices we're making and the actual problem is only illusory.

You say I shouldn't look to cable, HN, etc., but should instead presumably be pulling from Foreign Affairs, history books, etc. I say that's the problem I'm describing. I mean, an NDU speech? Should we really be content with confining the discussion to channels such as these? If so, then why is it reasonable to have a discussion about our liberties (e.g. Snowden, etc.) via the various public media, but not one about why such a discussion is neccessary in the first place?

Because, that's the problem that I'm identifying and, by suggesting that I turn to more esoteric channels, you're basically acknowledging the problem and responding that such public discussion is not only missing, but should be missing. So, I suppose the obvious question is why--in your view--is it OK that it's missing from the wider public arena?


You should probably read this first as a starting point: http://www.newstatesman.com/2014/04/challenge-islam


I find this totally hilarious. The NSA can come out and say whatever the hell they want and everyone just buys it because they don't have a choice. What are we going to do, go look at their data logs to make sure they're not lying? Oh no we don't record the contents of your calls. What a bunch of bullshit. You think any spying has stopped since Snowden revealed that NSA is looking up your ass when you're dropping one? This is hilarious because it's so absurd.


> You think any spying has stopped since Snowden

I remember that "internal" NSA memo which tried to reassure the employees, basically saying "don't worry, we'll just winter the storm, it will pass...".

And that's exactly what they're doing: Wait. And it's all that's necessary, because everybody has already shut up and is anxious about the next iPhone.


Okay so why can't we get any reporting with any numeric literacy?

How many people does the average person make contact with in a 5 year time span? Let's call this number X for hop 1.

How many people does each of the people in X then make contact with in the same 5 years? If we assume that on the whole people know ABOUT as many people as others, we can also call that number X for hop 2.

Now of course there's going to be overlap, out of X people somewhere between 0% and probably 70% might overlap between any one person of interest, hop 1 and hop 2.

Let's say that on average there's a 40% overlap. And let's call the Sweep(X) the number of people that get caught up in surveillance as a result of a single suspected terrorist.

Sweep(X) = 0.4 * X (first hop) x 0.4 * X (second hop)

Sweep(X) = 0.16 * X ^ 2

That's quadratic and as anyone who knows math, and especially people who know about Big-Oh notation, it's bad because Sweep(X) grows much faster than X does.

But how big is X? Well we can look to Dunbar's number to give us somewhere to start. Wikipedia says that it's between 100 and 250 but most people use 150 as a reasonable approximation. Of course that's STABLE relationships, not acquaintances, friends-of-friends, or anything like that. I think it's pretty reasonable to assume the 150 number and then assume that you probably make contact with one random person every week. Over 5 years, that's another 250. http://en.wikipedia.org/wiki/Dunbar's_number

So if we assume that X is 150 + 250 = 400 what do we get?

Sweep(400) = 0.16 * 400 * 400 = 25,600 let's just say 25k for short.

Now there are around 300 million people in the US. 300mm / 25k = 12000. That means that the NSA need only find 12,000 people (on average) to get all the call records for everyone in the US for the last 5 years.

It gets worse, though. What happens if you make contact with TWO random people every week? Then X = 150 + 500 = 650

Sweep(650) = 0.16 * 650 * 650 = 67,600. 300mm / 67k = 4477 people suspected of nefarious deeds in order to gain all call records.

As you can see this gets really out of hand really fast. And that's why I'm a bit upset that nobody's doing the math around the "two hops" argument because I assure you that the folks at the NSA aren't rolling over and giving up. This is something that SOUNDS reasonable but in actuality means little/no reform whatsoever.

EDIT: wikipedia link


It is actually much worse than that. You're assuming a uniform distribution over the numbers that people call, but there are some numbers that a significant fraction of the population call. For example, how many people are three hops away from a "target" through Comcast, Time Warner, Dell, etc?

Of course the NSA would probably ignore these spurious connections to generate leads for an actual investigation. But I assume that they would use them to create the largest possible list of people that they are "allowed" to collect and store information on.


Yes I was being conservative so that few reasonable people could challenge the assumptions I made.

But you're completely correct. I bet Amazon gets you the second hop to at least 20% of the whole country. Once you add in all the 800, 866, etc phone numbers I'd imagine that two hops gets you to very nearly every person in the US with a starting pool of perhaps only a few dozen people.

I could easily see someone's job being to figure out which (presumably innocent) people to "target" for the sake of ensuring a full or nearly full data collection. You know, to make things more efficient. "So we don't always have to go crawling to judges and the phone companies every single time we think something bad might be going down."

This two hops thing really is a scam.


"I could easily see someone's job being to figure out which (presumably innocent) people to "target" for the sake of ensuring a full or nearly full data collection."

Twitter makes that job damn easy, don't you think?

[EDIT: s/Twitter/Facebook/WhatsApp/SnapChat/etc.]


It could be because the only people who want to read an article that defines functions and expects them to be familiar with Big-O notation can reach these conclusions without reading the article in the first place.

That being said, it's generally believed to be (or measured) that the social network of the world is a sparse graph. So the fact that the sweep function is quadratic in the degree of a vertex is irrelevant. Using your notation, X is bounded by a constant for all intents and purposes.

In other words, there's nothing inherently bad about the quadratic behavior, and "knowing math" doesn't give you much except perhaps a false sense of intuition. The arithmetic (and your ad-hoc estimates) just works out in favor of your argument. Which is not all that surprising conditioned on the fact that you posted an argument :)


Sure, I'm not suggesting that the author give it the kind of treatment I did. Simply that the notion "oh two hops is better right?!" isn't valid.

Yes social networks are sparse. With 300 million Americans and the capacity to know perhaps only a few thousand, yes, things will be sparse. But that doesn't mean you cannot construct a spanning tree of that sparse graph. My PC isn't directly connected to Amazon or Netflix or really anyone of interest. But in a couple of dozen hops I'm connected to just about anyone in the world. Sparse doesn't mean unconnected.

The "badness" of the quadratic is that the "two hops" restriction SEEMS like a real restriction when in reality, it's not. It's bad in that the "two" in "two hops" sounds linear but the effects are quadratic. The quadratic-ness isn't inherently bad, I would never argue that. But the deception is.


I agree that your point is right, these policy makers are douchebags. I just think your analysis is not much more enlightening with the arithmetic. Change 40% to 10% and your final number goes up to over a million. It would be wonderful to have better estimates than "somewhere between 0% and 70%."


Hah you accidentally caught me in a math error!

Sweep(X) = 0.6 * X * 0.6 * X = 0.36 * X^2

I transposed the "percent of overlap" with "percent not overlapped" and viola problems.

If there's a 40% overlap from one person's social network to another that means that 60% of their social graph isn't included in the first person's and thus it's new for the sake of the two-hops strategy.

NumTargets(X,overlap) = 300mm / Sweep(X,overlap) = 300mm / ((1-overlap)*X)^2

NumTargets(100,99%) = 300mm

NumTargets(1000, 99%) = 3mm

NumTargets(100, 5%) = 33241

NumTargets(1000, 5%) = 332

So it's actually even worse then even I initially expected because the reach of the two hops method is doubly-quadratic. It's quadratic in terms of how much overlap there is from one person's social/business network to the next AND it's quadratic in how many connections each person has.

We're piling assumptions on top of assumptions here but lets suppose that at least 100k is the minimum number of targets we're comfortable with the NSA having to snoop on before they get everyone's data. It's not the right number but it's illustrative.

According to the spreadsheet I just put together we can satisfy this in a number of ways:

overlap = 99%, X just about all realistic numbers

overlap = 90%, X ~ 600 or less

overlap = 80%, X ~ 280 or less

overlap = 60%, X ~ 120 or less

overlap = 40%, X ~ 90 or less

overlap = 20%, X ~ 70 or less

overlap = 5%, X ~ 60 or less

There's no region that's got clearly reasonable assumptions where the NSA can't with a little bit of work get all the call records on all Americans again.

In other words, if it took overlap = 5% and X = 20,000 (to me not that realistic) in order to make it "easy" for the NSA to collect everyone's call records I might be less incensed. But the way this scales there's no reasonable assumptions that make it incredibly difficult for the NSA to vacuum up data on everyone.


Metadata has turned into such a slimy word. Metadata isn't a type of data, it is a description of the relationship between two sets of data.


That is far to narrow: metadata is literally "data about data", be it structural or descriptive.

It's also worth remembering that this is not categorical, particularly for descriptive metadata. In one context a piece of information may be metadata, in another data.


"Stop being so paranoid. Nobody's interested in reading your emails.

Besides, they're only collecting metadata."




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: