
How Lavabit Melted Down - jeanbebe
http://www.newyorker.com/online/blogs/elements/2013/10/how-lavabit-edward-snowden-email-service-melted-down.html
======
jobu
The integrity and bravery he has shown in this fight is impressive. He has
definitely earned enough "cred" to restart this business outside the US and be
very successful.

~~~
moxie
We should celebrate Ladar for making the decision to put himself at risk in
order to protect his users, but I think we should be careful not to forget
that Ladar was forced to make that decision because the security of Lavabit
was all a total handwave.

This wasn't untested water, either. The exact same thing happened to Hushmail
for the exact same reason, and should have been evidence enough that the model
isn't viable.

So I think we should definitely support Ladar as a person, but we also need to
be careful not to confuse that with supporting Lavabit, which was a very real
danger that should never be repeated again (again).

~~~
anologwintermut
Unless he actually used properly implemented forward secure SSL for every
connection, which I doubt all of either his customers browsers or the SMTP
servers he talked to supported, didn't his choices actually put his customers
in more danger?

He could have complied with one of the several valid court orders that
requested he give the FBI data on a specific account but stopped short if
installing FBI code or devices on his system or handing over the keys. Had he
done so, it would have stop there.

Instead, it escalated to the point where he actually was forced to expose all
his users. Anyone who has transcripts of those connections (e.g the NSA), can
now read them, get the passwords, and decrypt any mail they got form the
server. It seems like a boneheaded move unless his only goal was to protect
Snowden at all costs.

~~~
angersock
According to the article, the FBI jumped straight to "give us all the SSL keys
for everything", and would not let him to that selective warrant.

He rightly observed that those leaked keys would then get into the hands of
God-only-knows-who.

~~~
anologwintermut
The story, as far as I have read from this article and others, was they asked
for data(probably with an NSL), he said no. They got a court order. He said
no. At some-point he was willing to cooperate, but by that point, they didn't
care because they thought he was jerking them around.They then requested the
SSL keys. This article is more clear about the exact sequence of events[0],
but the the posted one says so as well. The initial request was not for the
SSL keys.

From the newyorker article : "On June 10th, the government secured an order
from the Eastern District of Virginia. The order, issued under the Stored
Communications Act, required Lavabit to turn over to the F.B.I. retrospective
information about one account, widely presumed to be that of Snowden. (The
name of the target remains redacted, and Levison could not divulge it.) The
order directed Lavabit to surrender names and addresses, Internet Protocol and
Media Access Control addresses, the volume of each and every data transfer,
the duration of every “session,” and the “source and destination” of all
communications associated with the account. It also forbade Levison and
Lavabit from discussing the matter with anyone. "

Sometime after his initial refusal and then offer to comply with some caveats
that the fed's interpreted as stalling:

"Prior to the hearing on July 16th, the U.S. Attorney filed a motion for civil
contempt, requesting that Levison be fined a thousand dollars for every day
that he refused to comply with the pen-register order. EARLIER IN THE DAY,
Hilton issued a search-and-seizure warrant, authorizing law enforcement to
seize from Lavabit “all information necessary to decrypt communications sent
to or from [the account], including encryption keys and SSL keys,” and “all
information necessary to decrypt data stored in or otherwise associated with
[the account].” (emphasis mine)

[O][http://www.wired.com/threatlevel/2013/10/lavabit_unsealed/](http://www.wired.com/threatlevel/2013/10/lavabit_unsealed/)

~~~
trobertson
This is an inaccurate account of events. If you read the actual documents [1],
you can see that the FBI had exactly 2 demands: A pen register device,
attached to his servers; and his SSL private key. That is the sum total of
what they wanted: complete, near-real-time access to all of Lavabit's data. A
physical device to copy the server traffic and send it to the FBI, and the SSL
key, to decrypt that traffic.

The stated use of these two things was to get information concerning a single
person, but they never wanted just that information. On page 100, Levison
states that he can manage to get the information the FBI is looking for,
without providing the FBI with Lavabit's encryption keys. Someone
(AUSA[censored]) says that the proposed solution does not satisfy the
subpoenas and court orders, because it would not provide real-time access to
the data.

\---

[1]: [http://cryptome.org/2013/10/lavabit-
orders.pdf](http://cryptome.org/2013/10/lavabit-orders.pdf)

~~~
anologwintermut
It's entirely possible it's an inaccurate account of events. I haven't read
all of the primary documents, just secondary sources.

In your linked documents,Exhibit 1 is the original June 10th order. Attachment
A of it(page 4 of the PDF) details what he was order to hand over. It does not
mention SSL keys at all. Instead it asks for a bunch of meta-data. In fact, it
explicitly doesn't even cover communication contents. It also doesn't specify
how Lavabit has to execute the order, just that it must provide the data.

This was the order Lavabit apparently initially refused.

Can you point to the first point they demanded the SSL keys? The stuff on page
100 looks like it pertains to the July 16th order. Which is, again,
considerably after the June 10th order that originally asked for the data and
after Lavabit refused that order. Also, totally inline with narrative of
events as I presented it.

Regardin pen-registers: a pen-register can be done in software and is
typically done by the service provider, not the government. The term is an
anacranism dating back to telgraphs. It doesn't necessarly mean government
hardware or software[0]. Hence the discussion page 99 of the pdf about
"implementing the pen-trap device" in section d. So that's not blanket access

[0][http://en.wikipedia.org/wiki/Pen_register](http://en.wikipedia.org/wiki/Pen_register)

~~~
scintill76
The June 10th order is on page 2, and seems to be only for the Target's
account details (not metadata on messages, AFAICT.) Page 19 (and again on 97)
says "Mr. Levison provided very little of the information sought by the June
10,2013 order." This sounds like he did not refuse it, and may have actually
not had much data to turn over since part of his business niche was to not
collect that kind of stuff. (Page 98 says "Levison claimed 'we don't record
this data'" although in context "this data" appears to be non-content message
data, which would not apply to the June 10th order.)

The June 28th order ("pen register/trap and trace order", page 7) is the one
he started refusing, then tried to negotiate on later. I think the order "that
Lavabit shall furnish agents from the Federal Bureau of Investigation,
forthwith, all information, facilities, and technical assistance necessary to
accomplish the installation and use of the pen/trap device" includes keys
implicitly. The June 28th Order Compelling Compliance Forthwith (to the
earlier order on the same day) notes, "To the extent any information,
facilities, or technical assistance are under the control of Lavabit are
needed to provide the FBI with the unencrypted data, Lavabit shall provide
such information, facilities, or technical assistance forthwith."

The first explicit order referring to keys seems to be the July 16th search
warrant, specifically Attachment B on page 36. According to page 98, FBI
agents discussed encryption keys with Levison as early as June 28th.

------
smsm42
The most scary quote in the whole article is this:

THE COURT: You want to do it in a way that the government has to trust you
/.../ THE COURT: And you won’t trust the government. So why would the
government trust you?

It was that the whole idea on which US is built on - the Constitution and
other founding ideas - was based on trusting the government only with very
little that is necessary for it to function and no more, and having the
ultimate power reside in the hands of the citizens. Now it comes to trust in
the government being implied and if the citizen doesn't trust the government,
he is not to be trusted and must be subjected to coercion. And that's coming
from courts, that are supposed to be protecting the constitutional rights.
America has come a long and very sad way since its noble origins.

~~~
SwellJoe
That quote made me feel sick to my stomach. I mean, I knew it had gotten that
bad...I've been involved in Restore The Fourth organizing, and before that
I've been paying close attention to all the previous leaks about the
surveillance state. But, knowing it and seeing a judge state it outright is
two very different things. It used to be under cover. It only happened in the
darkness of secret documents and agencies. Now, it's come out into the light
of day...and they're getting away with it. Not even getting away with it,
really...they're wearing it proudly, as though _they_ are the people in the
right; they honestly believe they are the people who have nothing to hide or
be ashamed of.

It's astonishing that more of our reps aren't standing up and shouting about
this. So many of the people in power are complicit, it feels hopeless at
times.

------
at-fates-hands
The fact the government wanted the SSL keys is obvious they wanted to get at
all his customers, not just the one they were targeting.

Levison offered multiple times to write a specific script for the single user
that would do what they wanted and at a minimal cost to the government - and
they refused. A pretty clear indication they wanted unfettered access to his
client base and his network.

Then you add in the lack of ANY oversight on either Lavabit's or the
government's, and you have to praise him for what he did.

~~~
65_196_127_226
Do you really consider the judicial warrant system a lack of ANY oversight?

After Levison's lack of cooperation, could the investigators really trust
Levison to hand over all the information?

~~~
lhc-
Why should Levison trust a government who has proven to be untrustworthy when
it comes to data collection? Levison didn't lie or mislead anyone, he even
offered to get the data for them as long as it was targeted. The government
has basically zero credibility in matters like this, and yet he was expected
to trust them with no oversight (in the article, he was told there was no
independent audit of their use of the data)?

~~~
MustBeAShill
What is this monolithic government you speak of? Are you saying that the FBI
and the NSA are synonymous? I'm sure plenty of FBI investigators would take
exception to an accusation like that.

I can play this game too. Dread Pirate Roberts used StackExchange, so
therefore StackExchange users cannot be trusted to build websites that don't
host drug deals and supposed hitmen.

~~~
jlgreco
Why the constant influx of throwaways?

------
ck2
I am blown away by the bravery, I know I'd never be so bold.

Also confused why he didn't end up in prison on mysterious "pervert" charges
out of the blue or even dead. And don't lecture me that is far fetched after
this past year.

~~~
tomp
Well, if they killed him, they probably wouldn't be able to get the keys. And
they probably had to keep the bigger "punishment", imprisonment, looming over
his head in case he reveals confidential information about the case.

~~~
jobu
It wouldn't surprise me if they had some sort of back door with Verisign or
other certificate companies for this.

~~~
alexwright
CA like Verisign don't have the key though, this misconception is too common.
If you're _doing it right_ the CA is just signing a cert you've generated,
they never see the key.

~~~
nitrogen
Having CA access _does_ allow them to create a silent MITM in the absence of
certificate pinning.

~~~
alexwright
With a different key though.

Once you've got this new cert. you can MITM, but you can't use it to decrypt
the traffic already captured. Also anyone paying attention sees the cert.
fingerprint change out of the blue.

~~~
delinka
A) Law enforcement doesn't need to decrypt previously-captured traffic; they
either want to fish for criminal activity or they'll allow their target to
build up new incriminating evidence. B) Who pays attention?

~~~
alexwright
A) That's what they were after though: “all information necessary to decrypt
data stored in or otherwise associated with [the account].” A rogue cert and
MITM would get the password for the account though, unless _B_.

B) Anyone who knows what they're doing and has something they really want to
keep secret? Maybe if someone had such a secret they'd learn to check the
cert, maybe even install an extension that would highlight unexpected changes.

------
lmm
The more I read the more sympathy I have for the government here. They had a
(presumably lawfully obtained) warrant against a specific user; it's not they
who designed lavabit such that it was impossible to execute this without
obtaining access to every other user. The proposal that Levison would extract
the information himself rather than turning over the keys strikes me as
completely unrealistic - any information so obtained would be quite rightly
thrown out of court, because there's no reliable evidentiary chain, only (in
effect) Levison's word. Even if he had turned over the SSL keys, the US still
has a fairly strong "fruit of the poison tree" doctrine: any information the
government happened to obtain on other users would be invalid for prosecution
because it wouldn't be covered by their search warrant.

~~~
fennecfoxen
> it's not [the government] who designed lavabit such that it was impossible
> to execute this without obtaining access to every other user.

That's true, but they're still essentially implying that services which are
explicitly designed to omit backdoor capabilities for the government to spy on
you -- that is, services offering actual cryptographically guaranteed privacy,
not just "no one has looked yet, and if they did, it'll all turn out okay in
the end trust us" \-- are broadly illegal and will get you criminal contempt.

~~~
smoyer
CALEA requires that all telephone companies (and now mobile phone and cable
companies) provide a means of "tapping" a phone line. To my knowledge, there's
nothing similar that says a data service has to provide the ability to
retrieve unencrypted data.

~~~
SideburnsOfDoom
What they required was more like the means to tap all the phone lines
simultaneously.

------
RyanMcGreal
> While he opposes the bulk collection of domestic communications, he has no
> such strong feelings about the N.S.A.’s foreign-surveillance efforts.

As a non-American, I have a problem with this seemingly widespread idea even
among privacy advocates in the USA that only Americans are entitled to the
protection of their rights from the American government.

~~~
betterunix
To be fair, _your_ government should be working to protect you from foreign
threats like this. You should not rely on foreign powers to protect you.

~~~
RyanMcGreal
Believe me, I'm not happy that my government is all-in on the American mass
surveillance game. But my specific concern here is with Americans who think
it's okay for the US government to conduct mass surveillance on the rest of
the planet, just not on Americans.

~~~
drivingmenuts
Because we never know where the next threat will come from, or perhaps the
threat after that.

Your country may be perfectly at peace with the US now, but there is no way to
guarantee that peace unless we have people to continually watch for potential
threats. Even that is, in itself, no guarantee, but it's better than nothing.

Maybe someday, mankind will be able to share universal goodwill and peace, but
until that time, trust, but verify, at a minimum.

~~~
jruthers
I call "Bullshit" on that.

For all of the talk about "all men created equal" and "do to others as you
would have them do to you", fundamentally Americans are brought up to believe
they are different or "exceptional" to other humans. This belief let's them
distort reality so that spying on innocent foreigners is ok but spying on
innocent Americans is an abomination. Hypocrisy.

Timothy McVeigh and others prove that the domestic threat to the US is as
serious as the foreign.

------
angersock
Of wonderful note:

 _At approximately 1:30 p.m. CDT on August 2, 2013, Mr. Levison gave the
F.B.I. a printout of what he represented to be the encryption keys needed to
operate the pen register. This printout, in what appears to be four-point
type, consists of eleven pages of largely illegible characters. To make use of
these keys, the F.B.I. would have to manually input all two thousand five
hundred and sixty characters, and one incorrect keystroke in this laborious
process would render the F.B.I. collection system incapable of collecting
decrypted data._

I tip my hat to this magnificent bastard.

EDIT:

The core issue is summed up nicely thereafter:

 _Levison believes that when the government was faced with the choice between
getting information that might lead it to its target in a constrained manner
or expanding the reach of its surveillance, it chose the latter._

------
selmnoo
For the fortitude he has shown in fighting the good fight, please consider
donating to his defense fund: [http://lavabit.com/](http://lavabit.com/) (link
at the end).

~~~
frenger
wtf? The site's [[https://lavabit.com](https://lavabit.com)] SSL cert been
revoked: [http://d.pr/i/sc71](http://d.pr/i/sc71) [IMG]

~~~
computer
Great!

It's because the private certificate was forcibly supplied to the government.
No longer secure => revoke it.

~~~
frenger
Ah, makes sense. I donated, anyway.

------
gregd
There is a huge disconnect between the "justice" system and technology which
needs to end. You've seen it before if you're in IT, that glazed eyes look
when explaining why their Word document is missing…

Anyone with judicial experience know if judges have trusted advisory panels
that can help wrap their heads around technology to better rule on cases such
as this?

~~~
frossie
You mean like Judge Alsup, who taught himself Java so that he could rule that
Oracle's APIs are not copyrightable? Or Judge Wells, who ordered SCO to show
him the code and then threw the case out when the failed to do so? We haven't
done so bad on tech judges recently - it seems to me that the problem with the
lavabit/NSA cases is not so much the technical side, but the classic one of
government powers, and the fact that there is no explicit constitutional
protections of privacy.

------
kbart
I still don't get one thing about this story:

>> To make use of these keys, the F.B.I. would have to manually input all two
thousand five hundred and sixty characters, and one incorrect keystroke in
this laborious process would render the F.B.I. collection system incapable of
collecting decrypted data

Don't FBI have some ultra DPI scanner with advanced OCR software? Let's say
they live under a rock, it's still not so hard to manually type ~2k characters
using magnifying glass. If so, what was the point to shut down Lavabit AFTER
turning in printed keys?

P.S. I still highly respect Lavabit and people behind it, but this point in a
story doesn't make sense at all.

~~~
andylei
what really happened is that the court then ordered him to turn over the key
on a CD, or continue to face the $5k / day fine. so he eventually did turn
over the key on a CD

~~~
adolph
Was it a bmp file on the CD?

I should read the article...

~~~
tadfisher
The order was to supply the key in PEM format.

------
jedbrown
News outlets keep repeating "11 pages of 4-point type totaling 2560
characters", which just doesn't match up since that number of characters fits
on one page in a fairly normal font size. Also, RSA keys just aren't that big,
so the 11 pages must have either been many keys or some other data.

As I understand Lavabit's architecture, there is no "master" key. Instead,
incoming mail is encrypted using an asymmetric per-user key. All the key pairs
were created by Lavabit and stored on-site, but locked by a password to be
provided over TLS. Since Levison probably didn't compromise his system to
store users' passwords, presumably the keys that he was handing over in
4-point type were still locked with a password.

~~~
CamperBob2
I don't understand why, if he was making a principled stand, he would have
bothered with the printout in 4-point type. That was sort of a juvenile move,
one that could only serve to justify the government's attitude towards him.
Weird.

------
danielweber
I've been skeptical of LavaBit, chalking it up to the general deification that
HN gives to its heroes du jour, but he really seems to have made a highly
principled stand while still allowing the government to intercept any
individual for which it had a warrant.

------
CamperBob2
Demanding the SSL keys to the entire database was clearly an insane overreach
on the FBI's part, a mistake that they compounded if it's true that they
refused to work with Lavar on the more targeted approach he suggested. I would
like to kick in some bucks towards Ladar's defense, but I'd rather do it
through the EFF (where I'm already a member) rather than rally.org, which I've
never heard of.

Does anyone have any experience with (or thoughts about) rally.org -- or, for
that matter, any knowledge of why the EFF isn't running point on this case?

------
smoyer
Is anyone else thinking that their systems should include a self-destruct
button? (for LavaBit I'd imagine a process that e-mailed each user the SSL key
used to encrypt their mailbox, then deleted the key from the system. A user
could still decrypt their mailbox by downloading it and using the key).

~~~
betterunix
The problem is that services like Lavabit want to do something that is
technically not possible: give you access to your encrypted mail from _any_
computer i.e. the convenience of webmail. If I can just download a key and
keep it on my computer, why would I not just _generate_ the key on my computer
by e.g. using PGP or S/MIME?

~~~
smoyer
No ... I meant a "red" button that could be used just prior to wiping the
servers clean. Your point is completely valid while the service is running.

~~~
krapp
Here's a Defcon talk about more or less that:
[http://www.youtube.com/watch?v=1M73USsXHdc](http://www.youtube.com/watch?v=1M73USsXHdc)

~~~
dublinben
Thanks for sharing that link. I couldn't find that talk last time the question
of emergency data destruction came up.

------
65_196_127_226
The amount of support for Levison and ire toward the government in this case
is absurd. The FBI followed the Constitutional process of obtaining a warrant
for the information of the "one user".

I suspect that the only reason anyone cares about this case is because Lord
Snowden the Infallible deigned to grace Lavabit with his email traffic.

Would the internet outrage be the same if the targeted user was found out to
be a Goldman Sachs executive or a Westboro Baptist Church minister?

~~~
zentiggr
Well, if the hallowed FBI et al had actually taken the reasonable offers of
access to one user and not escalated it into full access to the entire
service's and customer's content/traffic, there might be a lot less to be
outraged about, hmm?

