
Don't trust me: I might be a spook - cperciva
http://www.daemonology.net/blog/2013-09-10-I-might-be-a-spook.html
======
cs702
_" If you can't see anything because you can't get the source code... well,
who knows what they might be hiding?"_

Actually, it's worse than that. Even if you can review the source code, you
STILL won't know for sure. As Ken Thompson put it three decades ago, "no
amount of source-level verification or scrutiny will protect you from using
untrusted code."[1] This is because the tools we all use to process or
transform source code (e.g., already-compiled-to-binary compilers, assemblers,
loaders, etc.) may already contain secret back-doors. Ditto for even lower-
level tools like hardware microcode.

While open-source software tools -- and open hardware too -- are much less
likely to have secret back-doors embedded in them, there are no 100%
guarantees. Ultimately we have little choice but to trust the people who
create, package, and distribute all the software and hardware we use but
didn't create ourselves from scratch.

\--

EDITS: Added last paragraph. Made minor edits to second paragraph so it more
accurately conveys my thoughts.

\--

[1] [http://cm.bell-labs.com/who/ken/trust.html](http://cm.bell-
labs.com/who/ken/trust.html)

~~~
cperciva
It's not quite as bad as it sounds: I don't think even the NSA has people
smart enough to insert code into a compiler which will add backdoors to
software which didn't yet exist when the compiler was written.

If you're paranoid, use software written in the past 5 years and compile it on
a system running FreeBSD 4.1 off a CD which you've had in your closet for the
past decade.

~~~
sillysaurus2
_I don 't think even the NSA has people smart enough to insert code into a
compiler which will add backdoors to software_

Really? Why wouldn't they? That's one of the most valuable attack vectors,
precisely because it's so difficult.

I wouldn't have guessed the NSA had people smart enough to break Windows
Update's encryption using a brand-new cryptographic technique that also
required several hundred thousand dollars of machine time to execute, but it
happened nonetheless.

We have to start thinking ahead, e.g. by making deterministic builds a
standard procedure: [https://blog.torproject.org/blog/deterministic-builds-
part-o...](https://blog.torproject.org/blog/deterministic-builds-part-one-
cyberwar-and-global-compromise)

~~~
anigbrowl
_Really? Why wouldn 't they? That's one of the most valuable attack vectors,
precisely because it's so difficult._

You're assuming that because the NSA would _like_ to have such a capability
(as would most programmers), that they actually do, because they have abundant
money to throw around. Likewise, I'm sure the Air Force, Army etc. would all
like to have antigravity generators, and they have abundant money to throw at
the problem, so they must have them, right?

Of course not. 'A compiler that adds backdoors to software' implies a compiler
that _knows_ which routines are for security and which are not. How exactly is
it to distinguish between

    
    
      get_string(super_sekrit_password)
    

and

    
    
      get_string(pretty_background_color)
    

for example? What if the background color is actually a way of revealing
hidden messages, do you think it can see that coming? You're asking for a
system which not only inserts unwanted code seamlessly into an application,
but which can actually _model the intention of the programmer_ and make
decisions about how to compromise the code. From within the compiler running
on a standard desktop or laptop, without noticeably extending compile times.
I'm sorry, but given what a poor job _humans_ do at turning specifications
into code, the idea that there's a super-clever anti-security demon lurking
inside every compiler is just laughable.

And that's before the other obvious objection that looking at compiled code in
a debugger/disassembler is going to reveal lumps of code that were not put
there by the original author. Invisible on a sufficiently large project? Sure,
but an encryption/decryption program doesn't need to be very large to begin
with - all it has to do is to reliably transform a block of data into
scrambled form and back again. This is more amenable to proof than most
computer programs (not least because it has no need to be interactive).
Furthermore, we can easily imagine test cases that are very very short; ROT-13
is a lousy cipher, but it _is_ a cipher, and one that can be implemented in
~20 lines of (non-obfuscated) code. Now suppose we make a variant that asks
the user how many places to rotate by (eg 14) and that number functions as our
'secret' key. Still hopelessly insecure to anyone over the age of 10, but what
of it? Wouldn't your hypothesized 'insecurity demon' need to put a backdoor in
anyway, because it is an encryption tool, be it ever so primitive? And
wouldn't that block of code show up in a debugger? If your answer is no,
you're now positing hidden functionality that not only divines programmers'
intentions and subverts those that are intended to add security, but _also_
looks at the quality of the security algorithm and only sticks in a backdoor
if it passes a certain threshold of cleverness.

I _wish_ compilers had those smarts built in! Think how much
tedious/craptacular code could be automated away by simply labeling things as
'sekrit!!' and having the NSA module inside the compiler generate lightweight,
reliable code with no real penalty! It would take the pain out of unit testing
for ever!

~~~
sillysaurus2
You're presenting your argument as if that's the only kind of malicious binary
we have to worry about, and since it's (probably) impossible, then therefore
we don't have to worry. But there are more ways to compromise a binary than
via the compiler automatically backdooring them. Deterministic builds protect
against compromised binaries of any kind, so we need to use it.

The attack to be worried about goes like this: You build "Firefox Setup
23.0.1.exe" and intentionally insert a backdoor into the setup process. You
make sure the setup process appears to function exactly the same as the clean
installer (not hard). You then replace Firefox Setup 23.0.1.exe on various
distribution websites with your malicious version. Or you MITM the
distribution websites in order to send your malicious version in place of the
one the user expects to be downloading.

Deterministic builds defend against that attack vector, while also defending
against any hypothetical compiler-backdoor-autoinjector. You get both defense
layers for free, just by using deterministic builds. This is a necessary step
for the future, not an optional security layer.

~~~
anigbrowl
I responded to the claim you made. If you wanted to talk about something else
in the first place, maybe you should have done that instead.

 _Deterministic builds protect against compromised binaries of any kind, so we
need to use it._

Unless I'm a spook (or group of spooks) any my binary includes a backdoor by
design from the outset. Or unless I have backdoors built into the chips (a far
more likely possibility than magic compiler demons). ISTM you're yo-yoing
between treating the NSA as omniscient/omnipotent one moment and then holding
up things like this as silver bullets the next.

~~~
sillysaurus2
What is up with you? We're on the same side here, and you're trying to play a
game of superior-nerd.

Deterministic builds aren't a silver bullet. But they're an important defense
layer.

------
anon1385
>We need an army of software developers on the look out for potential NSA back
doors — to borrow a phrase, if you see something, say something. And if you
can't see anything because you can't get the source code... well, who knows
that they might hiding?

Of course this rules out all 'cloud' software other than simple dumb data
storage. RMS was right again[1].

[1] "Cloud computing is a trap, warns GNU founder Richard Stallman"
[http://www.theguardian.com/technology/2008/sep/29/cloud.comp...](http://www.theguardian.com/technology/2008/sep/29/cloud.computing.richard.stallman)

~~~
cperciva
Cloud software is fine as long as you don't need to trust it. I use Google
Calendar, for example, because it's convenient and I really don't care if the
NSA knows when my next dentist appointment is.

~~~
alan_cx
No, but the NSA might care about your movements, who you know, who you deal
with, and so on.

Remember that video that was posted of a Lawyer given a talk to law students
about never speaking to policemen? Remember how innocent things can be
completely twisted and used against you?

Yeah, I'd worry about the NSA snooping in my calendar.

~~~
tedunangst
How do you know the NSA isn't snooping on your dentist's calendar?

~~~
alan_cx
Well, we dont. And if they are, that's data connections. Maybe the dentist is
connected to some one scary, a client. Now "you" are connected to some one
scary. That's a terror network...

~~~
mpyne
Even in the braindead idiocy of government, which I can confirm from firsthand
experience, that would not constitute a "terror network".

~~~
girvo
I'm personally more frightened about someone like the IRS being handed data
from the NSA, than the NSA thinking I'm a terrorist, and I pay my taxes and
everything. I'm just frightened about being nailed for any tiny mistakes that
I have made in the past, that this data capture could glean, if that makes
sense?

FWIW, I'm not even from or in the US of A, but I'm worried about my government
doing something exactly like that considering how much it's cooperated with
the NSA directly in the past and present.

------
parennoob
Sadly, for most users (read: people who don't program, or read HN, and even
those who do), "Read the code and find bugs in it" is probably an impractical
defense, although it is the logically correct one from the cryptographic point
of view.

An easier way for _most_ people (Read: People who have never heard of
cryptography. People who think 'DES' is a Government agency. People who think
of an actual python when they hear 'python'. People who think Perfect Forward
Secrecy means no one sees your Mom's embarrassing email forwards. There are a
lot of such people, probably far more of them than software developers who
speak C.) is probably to lobby for change in the draconian laws that authorize
and encourage this sort of spookery. It is _not_ a perfect solution, but in
the long run, will probably make things easier for the average person.

~~~
Tyrannosaurs
When it comes to crypto most people who read HN aren't close to being
qualified to read the code and spot issues.

The things the NSA is doing isn't putting in "IF password = 'Joshua" then
LetMeIn()", it's subtle flaws in random number generators. The reality is that
open source may be our best defence but it's still a pretty poor defence
because of the complexity of what's been compromised.

The more I think about it the more I think the actual best thing you can do is
to pick obscure technologies that were probably too small to warrant NSA
attention.

~~~
cperciva
I've dealt with a lot of bugs in FreeBSD, and most of them were found by non-
security-experts. The people finding said bugs didn't necessarily know enough
to be able to exploit the bugs, or even enough to be certain that there _was_
a bug; but there were a lot of "this code looks weird" reports turning out to
expose vulnerabilities.

I don't think you need to be a security or cryptography expert to find
vulnerabilities the NSA might try to exploit.

~~~
Tyrannosaurs
While you're right that fewer issues means fewer potential exploits, all
you're really doing is getting rid of weaknesses that should never have been
there in the first place, but you're doing nothing to address the systematic
undermining of encryption standards and hardware and software implementations
of them.

To use a metaphor, it's great that you're making sure your windows are shut
but the locksmith may have sold you a deliberately weak front door lock.

~~~
bad_user
When speaking of encryption and "systematic undermining of encryption
standards" you have to be aware of what that means.

What people fear is the deliberate weakening of the algorithm such that it can
be broken (implementations can be fixed, the real nightmare being subverted
algorithms), where broken doesn't mean what many people think it means.

Say you've got a 256 bits key, to brute force it you'd normally need to try
2^256 combinations of bits, right? Well, if you found a flaw that permitted
you to brute force it in 2^200 attempts, then that's a massive improvement and
you can consider the algorithm broken, but guess what - that's still
exponential complexity with the issue being solvable by simply making the key
bigger. And there are people working on these standards that are not on NSA's
payroll and outside the US jurisdiction, people that aren't idiots, so bigger
flaws than this aren't feasible.

This is why, even if they've introduced subtle flaws in current standards,
that doesn't mean they have the capability of breaking the encryption - e.g.
it is possible that they are able to break RSA-1024 keys, but RSA-2048 is an
entirely different problem. And RSA-4096 keys will likely stay unbreakable,
unless a huge breakthrough happens.

People give them more credit than they deserve: yes, they have cash and
authority and can coerce companies and individuals and they can also plan for
the long term, etc, etc, but let's be realistic about their abilities.

------
alan_cx
"The only solution is to read source code and look for anything suspicious.
Linus's Law states that "given enough eyeballs, all bugs are shallow": If
enough people read source code, we will find the bugs — including any which
the NSA was hoping to exploit in order to spy on us. "

I understand the point about software, but what about hardware? If the NSA is
corrupting software, then it must be corrupting chips too. How can chips be
verified NSA spy free?

~~~
privong
_I understand the point about software, but what about hardware? If the NSA is
corrupting software, then it must be corrupting chips too. How can chips be
verified NSA spy free?_

In principle, an analagous "Linus's Law" exists for open-sourced hardware. So
if you have the code/design for chips, that could be verified as well.

~~~
solarexplorer
And how do you verify that the design was not modified before the chip went
into production?

~~~
sliverstorm
So, basically, design your own chip on paper (because you can't trust the CAD
tools or the compiler that made the CAD tools), build your own fab and make
your chip at home, build your own computer, write your own compiler, write
your own operating system... Yeah, this'll be good.

~~~
mpyne
And also, while you're doing all that, don't leave in any exploitable breaks.

Don't accidentally hardware RANDU as your PRNG, for instance.

------
parley
Every time these issues come up for discussion, my mind always returns to some
sort of imagined utopia where our software is much more modular, composable
and compartmentalizable (grammar?) than todays "mainstream" platforms.

It could enable several different things, like running systems where most
modules have access to much less resources of different kinds, allowing less
mischief (microkernels, hierarchical resource mgmt frameworks like GenodeOS,
etc).

Greater modularity would also mean that basic or common modules would need
less changes, requiring new audits less regularly, thus decreasing that load
on the community. Of course many vulnerabilities are the result of unexpected
interactions between modules, but then a certain composition of certain
modules could be audited and hopefully not require any changes for some time.

Being a software engineer I'm not kidding myself with regards to the huge
software engineering problems inherent in achieving that, and of course the
open source community already performs a lot of code reuse. Many will argue
that an argument for more reuse is an argument against fragmentation and thus
an argument against experimenting and forking. I guess that's true, but I
still feel that more could be split and shared while still experimenting on
many other things. There will also always be politics, personalities and the
will to reinvent wheels for many reasons, like self-education or
implementation pet peeves.

Also, different languages and/or runtimes/VMs and their differing suitability
for different environments affect fragmentation greatly. We probably won't
ever end up with a single "winner", no matter how much many
C/C++/JS/Go/Rust/Haskell/ATS/theorem provers we go through, and for reasons
(only some of which are mentioned above) we probably don't want to.

Dreaming is nice, though.

~~~
chubot
Definitely. One thing I've noticed is that constant code churn causes a huge
amount of instability in many software domains (cloud, desktop, phone). The
problem is only getting worse and will continue to get worse as software runs
more of our lives and the world. Apple is sending down updates all the time;
Ubuntu is; Google is, etc. Nobody can keep track of what the hell they're
running. It's a security nightmare.

There are so many components to systems now -- and not all of them need to
change all the time. What I imagine, and what you seem to be hinting it, is
that we have to do is factor software into processes which vary by the amount
of code churn. Instead of having 1000 modules being patched every week, have
500 modules which change every 5 years, 300 which change every year, ... and
maybe 2 or 3 which change every day. This would let us maintain the pace of
software innovation.

Unix does this to some extent -- think about when you would have a REAL need
to upgrade "grep"? Almost never. It's basically hardware at this point.

Likewise, with the architecture of Chrome and the Quark browser yesterday on
HN, you could just update the rendering engine, or the JS engine, which run in
restricted processes. Browser kernel updates could be separate, and verified
separately.

Related paper I found interesting:

[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.180....](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.180.6737)

We are moving closer and closer to "ubiquitous computing", and I agree with
that paper in that software upgrade has to be treated like a first class
problem. Right now even the best vendors are too sloppy about upgrades and
modularity. I am constantly being nagged for updates on every device, and at
work I am constantly dealing with shifting sands underneath my code.

~~~
parley
Thanks for the paper link, it looks interesting.

Yes, factoring software into different processes and enforcing proper
sandboxing and resource restrictions is definitely one aspect of it.
Hierarchies of such processess where resource allocation can be delegated
makes it even more interesting.

Another aspect is composing any one process of - where possible - shared,
audited modules that haven't changed for a good while _and_ that you're able
to verify individually before building your binary. It requires proper code
signing and delegates the problem to the domains of who to trust (and of
course issues like Thompson's famous Trusting Trust) and once again I'm not
kidding myself about the difficulties of that either. But that shouldn't stop
us from working towards such a goal if we find it desireable.

This of course helps closed source software production, but even more
important is for any individual or FOSS organisation to be able to build their
own system of (a myriad of) components that can be cryptographically verified
to not have changed since some specific audit event, provided that these
signatures of single modules and signatures of compositions of modules can be
kept by a public and decently trustworthy actor or preferrably actors, like
Linux distributions or FSF or whoever. In a world where the loyalties of
individuals are easily purchased we need to keep each other honest.

This extends from Joe building his own doubly-linked lists and red-black trees
to crypto modules that take side-channel attacks into account to kernels and
drivers and everything. Many will argue that layers and abstractions can kill
software with complexity. I certainly agree that they can if used poorly (or,
quite frankly, if not used with great skill) but they are also one of the only
things that can make complex problems manageable, and - as you say - our
systems are now hilariously complex.

------
shin_lao
_The only solution is to read source code and look for anything suspicious.
Linus 's Law states that "given enough eyeballs, all bugs are shallow_

Well. It's probably easier to break into your home, put an eavesdropping
device and get your keys. I think it's important to audit code but let's be
realistic, you won't defeat an agency that has you on your radar because you
use "open source software".

And by the way, Linus Law doesn't make any sense, simply because some bugs
cannot be seen just by looking at the source code and also because the
bandwidth between eyeballs that don't belong to the same brain is extremely
limited.

~~~
cperciva
_It 's probably easier to break into your home, put an eavesdropping device
and get your keys._

If they were only targeting one person, maybe. But getting a back door added
to a piece of widely-used software is much easier than bugging the homes of
everybody who uses that software.

~~~
shin_lao
The backdoor can be anywhere between the two computers (most likely at the
extremities though).

I agree with you in the sense that software should be made as secure as
possible, my comment is just a remainder that one should be realistic about
what level of security he can get.

If you didn't audit the hardware your software is running on and your local
isn't physically secure spending so much time on software auditing isn't very
useful.

You'd even have to thoroughly double check that the software you have it the
one you expect, audit your compiler, audit your os and recompile everything
you have.

You know better than me that there is no such thing as absolute security, so
you're secure relative to a menace. If you try to secure yourself against a
1st world intelligence agency, I say... "Good luck". ;)

------
lhnz
The most damaging thing is that I literally have no idea what is safe now -
even good actors might unwittingly have been backdoored at the hardware level.

I'm taking all private communication completely off the internet to minimise
my exposure - I do not have enough knowledge or experience to make decisions
on cryptography, nor sufficient need to waste time on this.

~~~
pyre
But where you are taking them? SMS? Phone calls?

Maybe a private mesh network setup through line-of-sight directed antennas?

~~~
lhnz
Face-to-face is the only way.

All emails or SMS or phone calls have to be just stuff like "Hey, how are you
are doing? Do you wanna meet up and have a beer."

~~~
jmccree
Do you remove the battery from your phone before talking in person? The NSA
could very well listen in to your phone's mic/s. Your phone probably has a
camera or two. Even if you have no phone, everyone around you likely does. If
you live in an urban area, your entire walk to the bar is likely on camera,
the bar has cameras, your cab ride home has cameras, the transit bus has
multiple cameras, the transit station has cameras, if you live in a high rise
it likely has cameras, in a suburb atleast one of your neighbors have cameras.
Most all of these cameras are now connected to internet enabled DVRs with
questionable at best security. This is reality now. Privacy seems to be on
it's last legs.

~~~
lhnz
And I am happen to live in the UK where we have CCTV. ...you are right, we
have woken to a truly disturbing reality.

------
genecavanaugh
I think everyone is overlooking a significant threat. If we have information
that is shared, we know when it is changed. The NSA has data that is not
necessarily shared, so if the top people decide to "do you in", such as Obama
and Romney, they have only to "edit" the data - and you have no way of proving
they didn't. Meanwhile, we know the FISC will accept whatever NSA gives them
as the absolute truth.

------
educating
> If the NSA can break 2048-bit RSA, it would be a Big Deal

2048-bit is nothing. Here's why:

1\. A great mathematician can find shortcuts. Tons of great mathematicians
working in parallel will find them faster.

2\. A fast processor can eventually brute force a key. Millions of fast
processors working in parallel will find it faster.

You are talking about a well-known method with a determinate algorithm. May as
well just hand them a cake, because they are going to eat that up.

Why use encryption mechanisms they've trained us to believe in and use? And
how could be beat them at a game where they hold all of the talent and power?
The only answer is to have dark encryption methods that are not shared, and
that change frequently, frequently enough so there is no group of
mathematicians with millions of servers that could determine the constantly
changing algorithms.

~~~
qu4z-2
So you're suggesting we should just frequently change almost-certainly-flawed
algorithms, but rely on their obscurity for security? I feel like there's a
term for this strategy, but I can't think what it is...

------
linhat
I do not know where this op-ed piece was intended for, but I think you should
have written it anyway, stating just this _feeling_ of yours. There
was/is/will be so much media frenzy about this, every article (even if you
think you _really had nothing useful to say_ , which you actually do), op-ed
or not, that does not resolve to sensationalists headlines is just really
helpful.

~~~
cperciva
I seriously considered it -- I even started trying to write a "there's nothing
new here, the NSA has always been spying on us" editorial -- but I just
couldn't make it work.

------
minimax
"I also pay for bugs people find in Tarsnap, as well as scrypt, kivaloo, and
spiped, right down to the level of typographical errors in comments."

I think these bug bounties are interesting. Is there something like a market
price for one of these bugs? I.e. are other people out there besides Colin
willing to pay for bugs in the tarsnap client?

~~~
shabble
I imagine there could be a similar reputation market along the lines of the
write-my-college-paper services, in which you sell the bug to someone who can
then take credit for it.

Certainly I'm sure you could find some people who'd be prepared to pay for a
valid Knuth error, although exactly how you'd advertise/seek those who have
such things would risk wrecking the value of the thing you seek to acquire.

Stepping up from that, there are clearly existing markets for exploits, and
you might be able to sell potentially exploitable bugs in the same way.

------
tylerkahn
I don't see what the NSA revelations have to do with this line of reasoning.

How do we know you're not (and have not been) selling tarsnap data to a
corporation or non-US intelligence agency? This has always been a possibility
and it has roughly equal probability to you being an NSA agent even after the
Snowden leaks.

~~~
marcosdumay
I don't see how you don't see a relation.

You assume he is, then you read the source of the client that is sending the
data, and verify that it's not. You trust the code, not the person. That's the
TLDR of the article.

~~~
derefr
Note that anyone who can't read code has to trust the person. I guess they
should be trusting a person who has read the code? What happens if you don't
personally know any software developers?

~~~
tedunangst
What happens if you don't know any civil engineers? Do you stop driving over
bridges?

~~~
jasomill
Bridges don't secretly collapse. And an overt collapse due to a serious,
government-mandated secret weakness in design or construction leading to the
revelation that a large portion of a "free" country's transportation
infrastructure suffered from similar secret weaknesses in the name of national
security would be a scandal of government-shattering proportions.

------
mbq
Indeed: [http://www.daemonology.net/blog/2011-01-18-tarsnap-
critical-...](http://www.daemonology.net/blog/2011-01-18-tarsnap-critical-
security-bug.html)

------
deadfece
I'm glad I kept reading past the product plug.

~~~
qu4z-2
Come on, that was clearly on-topic.

------
jheriko
thank you. common sense ftw.

someone who doesn't seem generally surprised that the NSA are involved in
espionage of all things... :)

"Despite all the above, it is still possible that I am working for the NSA,
and you should not trust that I am not trying to steal your data."

now if you can just get people to pay attention to issues that result in dead
children and reduced quality of /real/ life to people instead of overhyped and
questionable civil liberties violations... XD

