
Why spies never discover anything useful - sdoering
http://johnquiggin.com/2013/11/23/why-spies-never-discover-anything-useful/
======
grugq
Arguably WWII was won due to the information provided by a spy, Sorge[1]. His
timely info to Stalin informing him that the Japanese would not invade Russia
allowed Stalin to pull most of his eastern armies back to Moscow. Combined
with the Russian winter, the sudden influx of new troops halted and reversed
the German advance. This was only possible because the Soviet policy makers
understood the __intent__ of the Japanese government. That knowledge was
provided by a spy.

The purpose of a spy is not to "defeat the enemy", but rather to provide the
leadership apparatus with a clear understanding of the capabilities and
intents of the other actors in the game. (This is the theory anyway). Usually
this is referred to as "reducing uncertainty". The intelligence agencies
collect raw information, process it (e.g. determine its veracity, probability
for accuracy, etc.) and combine it with existing knowledge to produce
"product"... the intelligence output that provides the political leadership
with the "best estimate" of what the other actors are capable of doing, and
what they plan on doing. ( _capability_ and _intent_ ).

The author appears to have confused SIGINT, HUMINT and covert action (and also
doesn't seem to have a clear idea on how spies actually operate). SIGINT is
considered particularly useful because it is not subject to the whims,
calculations and forgetfulness of a human agent. HUMINT is much more useful in
the long run, and this is something that the Russians knew from at least the
days of the Okhrana. The Russians wrote the book on HUMINT and have forgotten
more about it than the West will ever know. (See "Technophilic Hubris and
Espionage Styles During the Cold War" [2])

[1]
[http://en.wikipedia.org/wiki/Richard_Sorge](http://en.wikipedia.org/wiki/Richard_Sorge)

[2]
[http://www.jstor.org/stable/pdfplus/10.1086/653104.pdf](http://www.jstor.org/stable/pdfplus/10.1086/653104.pdf)

~~~
TrainedMonkey
Arguably Russian Winter by itself won the war. I do not know if Japanese
actually understood that attacking Russia from their front was futile at that
time due to several reasons.

1\. Russia is big, very big. Traversing Siberia would have taken a long time.
Something that they would have to do without any naval support. Can you
imagine trying to establish supply lines through Siberia?

2\. Add continental winter on top of that and you get no go on Japanese
invading USSR. (Temperatures of 15-20 degrees centigrade colder than people
living near the ocean are used to).

3\. Because China would obviously sit very still, when large portion of
Japanese ground forces would be absent.

4\. Remember no naval support? Attacking Russia would effectively split
Japanese fleet and ground army.

I do not know if Japanese understood those simple facts, but Russians sure
did. European front was way higher priority than Siberia. An if Japanese did
attack, an order of magnitude smaller, more mobile force would had been enough
to harass supply lines and stall any progress. It is not only winter time that
would have been problematic, when snow melts things are worse for a while due
to all the mud.

Similar story with Germans, it is not that Russians in monumental effort
stopped the advance. They merely stalled German blitzkrieg (while taking heavy
losses), and continental winter stopped it. I am not trying to take away from
monumental effort thrown by soviets to stop the advance, but ultimately all it
came down to: Germans could not advance and adequately deliver supplies and
Russians could due to intact railroads.

* I am not saying those were only reasons, but people tend to underestimate importance of supply lines and role of winter.

~~~
tlear
Japanese goal would not have been advance through Siberia towards Moscow. They
would have just grabbed everything on the Pacific coast. Taking it back with
troops being moved from European USSR by 1 rail link.. would be an
impossibility.

Russian Winter is all well and good but there piles of what iffs that would
have won Germans the war earlier. It was a very very close thing and knowing
from inside that Japanese will not strike was of massive value

~~~
philwelch
Honestly, Vladivostok or Moscow? I know which one I would defend if I were the
leader of Russia.

------
shin_lao
Successes are kept secret otherwise you tell your enemies that you can
decipher their communications, which will probably make them switch to a new
system you may not be able to decrypt.

Saying spies never discover anything useful mainly displays a total ignorance
for the topic.

The most famous counter-example is the breaking of Enigma, to give a SIGINT
example
([http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma](http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma)).

In addition, I think the author mixes up the notions of SIGINT and HUMINT. I
think we can all come up with at least one example in History where HUMINT was
key to victory.

~~~
willvarfar
He does mention the enigma in the article. You read it all?

Can you come up with a few humint key to victory examples?

~~~
shin_lao
The allied intoxicated the axis to make them believe they would land somewhere
else (sorry it's another WWII example).

[http://en.wikipedia.org/wiki/Operation_Bodyguard](http://en.wikipedia.org/wiki/Operation_Bodyguard)

There are many examples during the Cold War, but I'm not sure it can be said
"key to victory", if there is any victory to be considered in the first place.

~~~
weichi
Please read the article. This example actually supports his thesis that "as
long as it is possible for counterspies to generate misleading information
most of the time, spies are useless even when their information happens to be
correct."

~~~
shin_lao
It's like saying: "it's useless to have a sword, because my enemy will have a
shield".

~~~
abcd_f
Don't bother.

The crowd just _loves_ the idea of spies being useless morons based on some
game theory musings by a fellow armchair analyst.

~~~
Ygg2
Not, morons, just corrupt criminals posing as doing something useful.

------
walshemj
Author doesn't seem to know what the difference between a sigint organization
NSA/GCHQ and a Humint one is or even what the difference between an officer
and a spy is.

Both GCHQ and NSA can point to successes as can all the major players in the
spy game would Stalin have got the bomb so quickly without spies. Or the
success of the XX committee. Or from the other side the Cambridge 5 where
certainly effective.

And do you think binladen would not have been found and killed if it wasn't
for spies run by CIA officers?

~~~
gwern
> Author doesn't seem to know what the difference between a sigint
> organization NSA/GCHQ and a Humint one is or even what the difference
> between an officer and a spy is.

To the contrary, he specifically contrasts humint with sigint: "

Spying may be worthwhile in cases where it is very hard or very costly to
produce misleading information. Two potential cases are those of code-breaking
in wartime, where the number of messages an enemy needs to send is so large
that their validity can be checked fairly easily, and that of a secret weapon,
where the information produced by spies can be checked by actually making the
weapon."

> Both GCHQ and NSA can point to successes as can all the major players in the
> spy game would Stalin have got the bomb so quickly without spies.

To the contrary, he specifically addressed Stalin's bomb: "On the face of it,
the reaction to the atom spies seemed justified. The atom bomb was a weapon
that could destroy the world (and perhaps still will) and the science on which
it was based was popularly associated with the genius of Albert Einstein.
Surely, the only way the Russians could create such a weapon was to steal the
secrets of the West....But once the existence of the bomb was known, any
competent team of physicists, with access to the right resources, could
duplicate it...The secrets passed by Western spies probably saved them a year
or so in their research program but did not fundamentally change anything. The
Chinese, French, Israelis and others made their bombs without significant
assistance from spies."

> Or the success of the XX committee.

To the contrary, a victory for the British necessarily meant a defeat for the
Nazis, a Nazi defeat only possible _because_ they tried to use humint: "Yet
the actual achievements of these shadowy regiments were unimpressive. In most
cases, espionage agencies can cloak their failures in secrecy, but the defeat
of the Nazis paved the way for a look at the record of one of the most-feared
espionage networks in history. The Hitler regime made numerous attempts to
infiltrate spies into Britain and to recruit British agents. As far as can be
determined from the German records, all were captured and many were ‘turned’,
being induced or forced to transmit disinformation to Berlin."

> And do you think binladen would not have been found and killed if it wasn't
> for spies run by CIA officers?

Sigint, again. Not that bin Laden's death really mattered aside from the
documents captured and the symbolic victory.

~~~
antimagic
Note: the following comment is based as my experience as an engineering
officer in the RAAF, with cryptographic clearance (someone has to load the
keys into the crypto devices!).

Yes, as anyone that has ever had a security clearance knows, the biggest
target of humint is sigint. The level of clearance that is needed to handle
cryptographic material is higher than the clearance to know about the
information sent by the cryptographic system. That's because one piece of
information, no matter how sensitive, is nowhere near as damaging as being
able to read all encrypted material of your enemy.

Clearances aren't just a title that you get - an increase in your security
clearance goes hand in hand with your history coming under heavier scrutiny.
At first you might just get a cursory check to verify that you are who you say
you are - teachers from your school may be contacted, neighbours etc. As you
start gaining access to cryptographic networks, the checks start going into
who you associate with, what your hobbies are, what's happening in your bank
account. And the checks get more discreet - you might learn about your primary
school teacher getting asked questions about you, but you won't learn that
your telephone conversations were listened to for 6 months to make sure you're
not hanging out with $THREAT_DU_JOUR.

So if sigint is valuable, one of the best ways to get your hands on it is to
compromise someone that has access - in other words humint. The two are not
disassociable.

~~~
frandroid
But once again, that is internal and defensive humint, not external humint.
The main point of the article, that spies don't discover anything useful about
foreign enemies, remains.

~~~
antimagic
uh, I think you missed my point. The reason that you get your background
checked so thoroughly is because you're going to be a target of the bad guy's
humint (so that he can use your access to cryptosystems for sigint), so the
good guys vet you first to find the skeletons in your closet. This is most
definately offensive humint.

~~~
onetwofiveten
That's an interesting point. If piece of information X is easy to verify, then
so is piece of information Y where Y is some blackmail material on an
individual who knows X. That does make sense.

------
tlear
USSR point of view:

1\. Elimination of opposition abroad, Trotzkyi assassination and many others.

2\. Discovery that Japan will not enter war. Results in transfer of Siberian
divisions to Moscow that play decisive role in war

3\. Plans for operation Citadel. Key point in war in the east

4\. Nuclear bomb, author discounts 1-3 years.. that had huge effect on
politics around the world

Invasion of USSR failure, one of the reason for lack of reaction was that NKVD
had apparently established set of citeria that would tell them that Germany of
planning an attack. Winter uniforms production, non-freezing oil production
and some others.. since none of those were crossed, detailed and very good
humint they were getting was discounted.

Also he forgets that Nazis also broke a lot of allied codes. If I remember
correctly one of the allowed them to know most of the convoy routes through
atlantic

Also forget that Battle of Midway was basically won because of sigint.

Generating believable crap is not that easy.

~~~
philwelch
Stalin actually had really good intel that Germany was going to invade, he
just refused to believe it and had people executed for spreading
"disinformation".

~~~
tlear
There are many stories on that.. honestly I think a lot of stuff about Stalin
disbelief are pure propaganda aimed at providing an explanation at how bad
Soviet army did initially.

In reality I think it was a combination of Soviet Army just being really bad
initially compared to Germans(remember Germans destroyed French and British
without breaking a sweat just a year before) and legitimate conflicting
assessments of German army no preparing for war in Russia. This lack of
preparation is what cost them in December 1941, if your tank oil freezes..
well you are not going anywhere

~~~
willvarfar
Winston Churchill talks about how the Brits sent info to Stalin at the very
highest diplomatic levels that he was going to be invaded. And later when
Winston met Stalin, Stalin said that he thought he had to do everything to
avoid provoking the Germans as they had started their previous war by
pretending to have been attacked by the Poles. Stalin said that he hoped he
had more time.

So its easy to read into this account that Stalin knew well that he was
probably going to be attacked, and rather than 'not wanting to believe it' in
a wishful way, rather balanced doing nothing to provoke Germany with taking
defensive steps and decided on the former course rationally.

~~~
philwelch
Something I've read a lot of places but is hard to track down--apparently
there was one incident where a German communist actually defected to the USSR,
warned them that Germany was preparing to invade, and was literally executed
for spreading disinformation. He also disappeared to his dacha and reportedly
seemed to have an emotional breakdown when the invasion did happen.

The fact that Stalin tried to play this off to Churchill doesn't really refute
anything. You don't become a legendary dictator and erect statues and
portraits of yourself everywhere without being a little prideful.

------
mathattack
I think the key insight is if enough people provide misinformation, the real
information is useless since it's hard to tell what is true. One could say
that this over-feeding of information contributed to why folks didn't see 9/11
coming. Yes there were drops of clues, but they were lost in the firehose.

I also believe that information needs to be acted on to be relevant. This is
why corporate espionage isn't very helpful. You can throw a monkey wrench into
a competitor's market trials, but IP is very hard to copy. Having a copy of
somoene else's source code only gets you so far - you need people who actually
understand it. Having blueprints to someone's plant only gives you clues - you
actually need someone who understands it enough and can overcome Not Invented
Here syndrome to make value judgments.

I recall an anecdote from the 80s where an American car company was taking
apart a Lexus. They concluded that their cars were superior in every way, and
that all the design decisions were a result of poor Japanese design. Whoops!

~~~
judk
The 9/11 attacks were in the President's daily briefing, meaning Bin Laden's
gang was known as a more severe threat than some Irish rebel or whoever who
might have wanted to cause trouble. It may have been improbable to find the
specific attacking individuals before the attacks, but I have seen no evidence
that they were known but buried in a haystack of extra noise.

------
cstross
Misleading title: the article is really about why spies never discover
anything useful _about other nations_. They're really, really good at
infiltrating internal dissident groups and spy (or terrorist) hunts serve as a
great pretext for domestic repression ...

~~~
7schlaefer
[...] useful [to prevent terrorist attacks]

------
Spooky23
This is really naive. Cherrypicking things like Pearl Harbor where the ball
was dropped is really the exception proving the rule.

The spying that organizations like the NSA against foreign countries, even
allies give key insight into what the leadership there is thinking. If you
think that there haven't been big wins in diplomatic and other areas from this
data, I find that hard to believe.

In warfare, there are all sorts of examples that come to mind where spying
saved the day. How about the utter destruction of the WW2 U-Boat campaign via
interception of communications? Or Patton's "ghost army" of inflatable tanks
that combined with double agents misdirected the German defense of Europe?

------
munin
this article has an active agenda and seems to overlook signals intelligence
successes against militaries and governments. history is filled with (public)
success stories where espionage delivers the goods, ENIGMA, purple, ivy bells,
etc... espionage also played a very large role in deception and
counterintelligence in concealing facts around d-day from the axis powers.

to be fair, espionage and intelligence gets it wrong very often. the goal of
intelligence, though, is to find some truth in an uncertain situation and the
presence of active deception. this is hard enough when you're talking about
figuring out another government.

I think there's an interesting argument around whether or not a "traditional"
government spy agency is "nimble" enough to analyze and understand modern
organized crime and terrorist groups, but it's both an argument that I think
intelligence professionals have been having for quite some time, and an
argument the author of this blog is not interested in considering...

~~~
willvarfar
The article mentions both enigma and purple.

And can you elaborate in which way was ivy bells catch useful?

~~~
munin
the article states:

> was helped by the Poles who had stolen a machine before the outbreak of war,

which is false! the poles actually built an ENIGMA after analyzing encrypted
traffic, see this book:
[http://www.amazon.com/dp/0743217349](http://www.amazon.com/dp/0743217349)

~~~
willvarfar
So you are saying that the small credit he gave humint in the enigma matter
may in fact not be deserved? Surely this makes your argument weaker than ever?

Slightly off topic, but you may enjoy this on my blog:
[http://williamedwardscoder.tumblr.com/post/19946053957/enigm...](http://williamedwardscoder.tumblr.com/post/19946053957/enigma-
spreadsheet)

------
jleyank
Hmm… I would think history suggests spies were pretty successful bootstrapping
the Soviet nuclear program towards the end of WW II. I also recall reading how
various rocket data/details were obtained from foreign countries post-WW II
and there's been a lively, two-sided game of spies for things like submarine
operations and control. And unless we're only talking about spies working for
a government, industrial espionage has been successful in the tech period. As
others have already posted, I assume the original author ignores SIGINT (as
there's a long, colorful history of its success) and other "collection" based
methods.

~~~
willvarfar
Oh come back and comment when you've read the whole article! There are
paragraphs assessing the claims you repeat, even! Geesh.

~~~
wmil
The problem is that he simply states 1-2 years more quickly as if it's an
indisputable fact.

The USSR's first nuclear test was in 1949. The UK had access to some Manhattan
Project data and didn't detonate a test bomb until 1952. France didn't test a
bomb until 1960, and that was arguably the first repeat of the Manhattan
Project without data and designs from the project.

It's entirely reasonable to assume that the USSR wouldn't have had the bomb
until 1964 or so without it's spy network. That would have changed the cold
war dramatically.

~~~
willvarfar
Britain was bankrupt after ww2 and focused on domestic issues and the giving
up of empire. France was even more dire and reconstructing.

But, whilst Russia was shattered, it diverted far bigger resources to gaining
some sense of parity with the US after the war than other countries.

The USSR looked to the west as a threat; the Brits and the French didn't fear
their ally the US, and didn't need to get a bomb of their own with such
urgency (until the USSR got the bomb, that is. Suddenly also having the bomb
was strategically important.)

------
PeterWhittaker
Huh. Perhaps true. But completely ignorant of the impact, of the size, of the
value of what a single "spy" (read also traitor") can discover (read also
"reveal").

Spies and traitors tend to discover/reveal HUGE secrets, e.g., how to make a
nuclear weapon, how to crack air defense, exact plans and order of operations
for critical engagements, etc.

Terrorism doesn't even enter onto the same radar screen as those sorts of
things, because all of the terrorism in all of the 20th C amounts to what, the
effects of one atomic bomb? (Back of the envelope, not terribly accurate, but
gives relevant context.)

I'll stipulate that the vast majority of spies never discover anything useful.
I'll further stipulate that "vast majority" involves several decimal points,
i.e., 99.999% or higher.

And I'll assert that it is still worthwhile having spies for what the 0.001%
or 0.0001% discover - because they don't go for the low hanging fruit, they go
for and get the game changers.

------
asgard1024
I have to say, I also don't believe in much of ability of spies (and domestic
spying) in actually _protecting_ the nation.

In case of war or copying other technology, they may have a lot of merit, as
many commenters have said. But if you actually look at the world, most of the
small nations do just fine without Internet snooping. If survival of a nation
would require to snoop the Internet, all of the small nations (and some are
really really small) would be dead.

So it's probably a case that someone wants to be a bully, so they need the
counterspies to handle all the blowback.

------
milliams
[http://www.bbc.co.uk/blogs/adamcurtis/posts/BUGGER](http://www.bbc.co.uk/blogs/adamcurtis/posts/BUGGER)
makes for relevant reading too.

------
aeonsky
I don't know, Richard Sorge pretty much prevented the downfall of the USSR
during World War II and shaped the current world.

------
mbq
Poles did not stole some single Enigma and smuggled to UK but were
successfully breaking the code (Germans weren't stupid and were improving the
machine so deciphering its cryptograms was a moving target). The whole project
was disclosed and moved to UK when Poland fell under German-Russian
occupation.

------
if_by_whisky
Didn't the soviet union develop a nuke alarmingly quickly after the U.S.?

[http://en.wikipedia.org/wiki/Soviet_atomic_bomb_project](http://en.wikipedia.org/wiki/Soviet_atomic_bomb_project)

------
squozzer
Sun Tzu believed in spies. And what respectable HN groupie doesn't have a copy
of The Art of War, next to a dog-eared copy of The Prince?

Then there is the tale of the Lucy spy ring discovering the impending German
attack at Kursk.

Finally, the Bothans who sacrificed their lives to retrieve the plans for the
Death Star.

~~~
rustynails
You also had the Mongols, arguably the most successful nation in history,
particularly with conflict. They relied heavily on spies and would often
observe for years before the Mongols struck. Ever hear of the Khwarazmian
empire? One of the largest middle eastern empires? Worth reading about. The
Mongol's success depended largely on spies. Success in this instance means
"wiping the empire from the face of the earth", which is obviously subjective.

~~~
Ygg2
You also miss that the times have changed in last thousand years, and that its
easier to fake information now, than back then.

------
robterrell
Just gonna add one more to the pile:
[http://en.wikipedia.org/wiki/Zimmermann_Telegram](http://en.wikipedia.org/wiki/Zimmermann_Telegram)

------
NN88
The best spies are the ones you don't hear about so...

