
Analyzing the FBI’s Explanation of How They Located Silk Road - nikcub
https://www.nikcub.com/posts/analyzing-fbi-explanation-silk-road/
======
dmix
Came across a discussion on Twitter pointing out reddit users who apparently
discovered IP addresses in error messages printing out in the HTTP response:

26 Mar 2013:
[http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the...](http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the_silk_road_revealed_its_public_ip_last/)

26 Mar 2013:
[http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_s...](http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_silk_road_users_sr_is_leaking_their/)

03 May 2013:
[http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_b...](http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_be_worried_showing_on_login_page/)

~~~
cookiecaper
Wow, I love how those threads are full of people who insist the OP is full of
shit and that Silk Road doesn't even have a public IP address.

Maybe we'd have fewer incidents like these if keyboard jockeys were a bit more
teachable. Even super 1337 h4xx0r5 make mistakes, often obvious ones. Software
is complicated. Let this be a lesson to any of us who may dismiss a correction
too quickly.

~~~
pyre
Yes. They all tend to talk in certainties about something that they are only
speculating about:

    
    
      > you would have to go out of your way to expose
      > your public IP via Tor.
    
      > However, the hidden service you connect to,
      > would be the load balancer. This service would
      > have zero knowledge of its "external" ip
      > address, as it would be running in a VM
      > pointing to a port on the loopback device.
    

(because _obviously_ nothing else is possible)

    
    
      > - Server is almost 100% certainly running in a
      > VM and doesn't have a public IP.
      >
      > - The entire harddrive for the server (which
      > includes all the the information such as
      > private messages and addresses) is almost 100%
      > encrypted.
      > 
      > This is the motherfucking SilkRoad. They've
      > been operating so successfully for so long
      > because they're not amateurs. OP is a liar.
    

... etc ...

~~~
nrser
the funny part is: the people i know that used the service (who were far down
towards 'non' on the technical spectrum) complained regularly about it's
terrible design, stability and security, and were under the impression it was
administrated by parties that had little idea what they were doing. they
theorized it was a big scheme to go Gox* on them once enough coins were
present in the system.

it's interesting that these people, who have no idea what an ip or load
balancer or VM is, were able to pick up on this. i would guess that the
h4xx0r5 actually spent little time using the service or had the bit of their
own necks being exposed taint their assessment.

* yeah, i know no one really knows exactly what happened, but it's fun to poke fun.

------
hueving
Whoever wrote this article either very poorly understands network traffic or
is being intentionally obtuse to create doubt. He states that there is no way
that a public IP would show up in a packet capture over a Tor network. This is
blatantly false and idiotic to think.

HTTP servers, caching proxies, TLS terminators and load balancers have a habit
of putting extra HTTP headers in the HTTP response to make the path of a
request more clear. It's very possible that under some error condition one of
them included an X-FORWARDED-FOR header or something similar that included the
IP. This is all application layer data and can contain IPs.

Even ignoring headers, they could have triggered an error page that just dumps
all PHP server variables, one of which includes local IP addresses.

Edit: It's infuriating that this article author doesn't understand that a
packet sniffer can see contents of the HTTP protocol.

~~~
michael_storm
_He states that there is no way that a public IP would show up in a packet
capture over a Tor network._

No, he does not. He says that there's no way a public IP would show up at
layers 3 or 4 of the OSI model. The author takes the position that the IP leak
must have come from layer 7 (quoting from the article):

 _[...] and it is at the application layer that the FBI uncovered the IP
address._

Depending on how one interprets the FBI's wording, this impossibility
contradicts their accounting of how they discovered the IP. Let's look at what
the FBI said:

 _Upon examining the individual packets of data being sent back from the
website,3 [sic] we noticed that the headers of some of the packets reflected a
certain IP address not associated with any known Tor node as the source of the
packets._

Note that they're talking about headers _of the packets_ , not _HTTP_ headers.
This is the FBI, though, so it's possible they confused "packets" with "HTTP
requests". Taking their words at face value, though, the FBI seem to say that
they got the IP address from the IP headers on packets received from Silk
Road, or at least traffic generated by their browser on behalf of Silk Road.
That's what the author argues is impossible.

Nobody, least of all the author, disputes that Silk Road could have leaked its
IP in HTTP headers, or via some other misconfiguration. But if they did, why
didn't the FBI just say that? Why claim that they leaked it in the IP layer?

Two possibilities come to mind. The first is that the FBI got the IP from the
application layer, as everyone believes is possible, and misattributed the
leak to the IP layer due to terminology confusion. The second is that they got
the IP from another source, attempted to deceptively misattribute the source,
and accidentally picked a fake source the leak could not have come from. If
the second option is the case, we're looking at an example of parallel
construction.

Edit: Grammar. Gotta keep up appearances.

~~~
MaulingMonkey
_Note that they 're talking about headers of the packets, not HTTP headers._

HTTP headers are _in_ the packets, and I could see myself very very very
easily writing "of the packets" in lieu of "in the packets". When
proofreading, I'd be much more likely to catch the extra "3" in that same
sentence, than to catch the s/in/of/ as introducing confusion.

~~~
ssmoot
My gut reaction to that is skepticism.

I've done basic switch configuration, looked at packets, had to troubleshoot
MTU and MRU misconfiguration in commercial switches, etc.

Conceptually stuff at that layer is a long ways from the actual content.

It's like pointing to a guy walking his dog at the park after noticing a
parked car with it's lights on, and saying "there's the owner _in_ the car" vs
"there's the owner _of_ the car".

It feels, to me, like those sorts of linguistic mistakes must be exceedingly
rare?

Or maybe the person writing up the statement wasn't the person who generated
the notes and they just had a transcription error. I'd buy that I guess.

~~~
diminoten
Your gut reaction is to lean towards conspiracy rather than a simple
typo/transposition in the mind of the FBI agent who wrote the report?

~~~
ssmoot
That's awful snarky, and misses the point entirely.

My gut reaction is that if I, as a civilian, were to ask an FBI agent for
details about a case, I'm as likely to receive half-truths and lies by
omission as anything at all though. So if you want to be pedantic I guess so.

I did point out that I'd believe it could just be a transcription error by
someone unfamiliar with the subject matter though.

~~~
diminoten
The point here is that this is nothing more than a simple writing mistake on
the part of the FBI. Pretending like there's some conspiracy or parallel
construction taking place here is just nonsense and the folks suggesting it
are doing so out of irrational fear more so than any actual evidence.

Your gut reaction seems to place you in the camp of the conspiracy theorists.
That should raise alarm bells for you.

~~~
ssmoot
> The point here is that this is nothing more than a simple writing mistake on
> the part of the FBI.

You say that like it's fact. But you're assuming.

I thought it was noteworthy that I can't recall ever seeing that specific
error in the wild. Maybe I just missed it. Maybe it's a lot more common than I
imagine. Maybe not. I don't know. Once it was brought to my attention I
thought it was noteworthy though.

> Pretending

I'm not pretending anything. I gave an additional explanation I felt might be
plausible. I'm sure there are others. I just doubt the plausibility of the one
brought forward.

> irrational fear

Tomato tomato. There's no evidence it was a writing mistake, but you're
presenting it as fact.

I guess it all boils down to this: Given domestic spying revelations I think
you're extremely naive if you think it's _irrational_ to think that the FBI
might be telling less than the whole truth.

Your gut reaction puts you in the camp of people who would volunteer
information to a LEO. Which I think should raise a lot more alarm bells than
me shrugging my shoulders and suggesting I'd need actual evidence before
taking the FBI at it's word.

The very idea that you would is blissfully naive IMO. But good luck with that.

And if you have an point to make, a little less ad-hom and snark (aka
Reddittude) would be appreciated. My original comment was a few wonderings-
out-loud, including posing a couple questions for discussion. Nothing in there
intended to be snarky. If you want to discuss further, it would be nice if you
could approach the conversation with the same level of basic respect.

~~~
diminoten
Sidestepping your bruised ego entirely, the fact that _you_ haven't seen
someone mistake an IP header with an HTTP header is a terrible standard with
which to judge whether or not such a mistake is possible.

This isn't really about you, it's about showing the other folks reading this
how nuts you are, and luckily you're doing that for me.

~~~
ssmoot
That's a weird argument (that I never made). Though I have to wonder about
that. IPv4 headers and HTTP headers don't much look the same IME (feel free to
hit up Wikipedia if you're unfamiliar).

As far as "how nuts your are". You seem to be taking this way too seriously.

"My gut reaction is to be skeptical" somehow equates to, in your words:

    
    
      * Nuts
      * Conspiracy Theorist
      * Nonsense
      * Irrational
    

Expressing _skepticism_ that the FBI would "tell the whole truth and nothing
but", despite high ranking intelligence officials outright lying to congress,
and the history of the FBI being a bit checkered... _That_ you think should
raise alarm bells. Skepticism.

If that's all it takes to be a conspiracy theorist in your book, well sign me
up I guess.

The outright dismissal that it might just be possible you don't have all the
facts (and I'm certainly not claiming I do)... That's an idea you dismiss
outright. I have to wonder how you get there. Because if anything here seems
_irrational_ , that's what I'd put my finger on. But maybe that's just me.

------
asuffield
So we have a pretty good guess of how they did it, and while their affidavit
doesn't give a lot of important details, ultimately they were just interacting
with a web service in the way it is supposed to be interacted with - by
sending it http requests - and the buggy site revealed itself to them because
the guy who wrote it wasn't a very good web developer.

I can understand why they wouldn't want to give more details than they have
to; it makes the prosecution's work harder. But I and many others have been
arguing for years that there's nothing illegal about sending messages to a
server and looking at the responses, if you aren't trying to damage anything.
I can't see any reason why it would be less legal for the FBI to do it.

~~~
zackbrown
_"...they were just interacting with a web service in the way it is supposed
to be interacted with - by sending it http requests"_

That line of reasoning sounds a lot like the Andre 'Weev' Auernheimer case
[0], where he gathered AT&T customers' email addresses by interacting with a
server simply by 'sending it HTTP requests.' The FBI made its position clear
on that case, prosecuting Weev for "conspiracy to gain unauthorized access to
computers" and ultimately getting the guy sentenced to three years in prison.

The overarching circumstances are clearly different but undeniably parallel.
It seems curious to me that the FBI could use these some sort of apparently
'criminal' tactics (by their own precedent) as legal grounds in their case
against DPR.

[0] [http://www.huffingtonpost.com/2013/03/18/andrew-weev-
auernhe...](http://www.huffingtonpost.com/2013/03/18/andrew-weev-
auernheimer_n_2900387.html)

~~~
res0nat0r
The slight difference is the FBI is investigating the source of the Silk Road
(not illegal), vs. trying to obtain private information of folks from ATT
(illegal).

~~~
jonknee
There is no difference if they did not have a warrant. The FBI has the ability
to break into your house, but if they do that without having a warrant they
can't use anything they found against you.

~~~
res0nat0r
I wouldn't call this breaking in, more like surveillance.

~~~
tatterdemalion
And with some very obvious exceptions, warrants are required to commit
surveillance that violate certain privacy and property rights.

~~~
res0nat0r
The thing is I don't believe the court will find this to be violating any
laws.

~~~
tatterdemalion
I think you're probably right, but the argument up-thread was that the justice
system is hypocritical regarding the permissibility of actions taken by the
government compared to individual people.

------
colinbartlett
All of these analyses of how they might have located the server are great. I'm
pleased to see folks trying to prove or disprove that the FBI is not trying
some parallel construction here.

But what worries me is that all of this will have to be eventually dumbed down
to an explanation that a jury of average joes can understand. The increasingly
technical complexity of evidence in some of these cases worries me that our
system doesn't have an adequate way to deal with it.

~~~
cookiecaper
I share that concern in general (and in particular with regard to software
patent cases, which often cover highly technical material and where the jury
has no context about the novelty of the patented inventions), but I think this
case is pretty much a slam dunk however you approach it. Regardless of how
they obtained the IP, they got to the box to find it running Silk Road code,
they got to the box's owner to find him holding Silk Road bitcoins, etc.

This isn't a case that hinges on a technical nuance. It will be practically
impossible for DPR to convince a jury that there is any reasonable doubt as to
his guilt.

~~~
roel_v
If the evidence was obtained using illegal methods, all that evidence can't be
presented at trial and the jurors can't take that information into account
when deciding guilty or not. There are many people who are acquitted for
things they obviously did, but for which there is no _formal legal_ way of
proving it.

~~~
spindritf
_If the evidence was obtained using illegal methods, all that evidence can 't
be presented_

Isn't that for the judge to decide? Judge decides what can be presented to the
jury, and jury renders the verdict, right?

Then the argument doesn't need to be dumbed down for the jury. They won't see
it at all.

Although the argument will still probably have to be dumbed down for the
judge.

~~~
gambiting
Nope. A judge cannot decide to admit an illegally obtained evidence, no matter
what it is.

~~~
spindritf
OK but he gets to decide if it was illegally obtained.

~~~
_archon_
This is decided in pretrial hearings. There have been entire cases based on
the admissibility of evidence. If the judge rules it is or is not admissible,
there is some manner of recourse or escalation for the usurped party. IANAL
YMMV BBQ

------
zachrose
Is the constitutional issue here that exploiting a security hole (SQL
injection, remote execution to dump the $_SERVER variable, what have you)
constitutes an illegal search?

If so, is their any guidance from case law about the difference between some
sort of legal poking and prodding vs illegal hacking?

~~~
downandout
In general in order to break the law you must have a warrant. For example, a
cop entering private property without a warrant or probable cause is nothing
more than a trespasser. These agents likely violated the CFAA by doing what
they did. Because they didn't have a warrant, IMO it _probably_ constitutes an
illegal search, but the courts may well have a very different opinion.
Regardless of which side loses this argument, this will likely find its way to
the appellate courts and possibly the Supreme court, as it seems to be an
unanswered question of great importance.

~~~
maxerickson
From the CFAA:

 _(f) This section does not prohibit any lawfully authorized investigative,
protective, or intelligence activity of a law enforcement agency of the United
States, a State, or a political subdivision of a State, or of an intelligence
agency of the United States._

I'm not sure what "lawfully authorized" means there, but my current belief is
that one form of it is permission from the Attorney General.

~~~
downandout
But I wonder if computer hacking (which this likely would fall under) can be
"lawfully authorized" outside the confines of a search warrant. The Attorney
General could order the FBI to break into the computers of a hedge fund
suspected of insider trading, but that clearly wouldn't be lawful unless
authorized via search warrant. This is very similar.

~~~
maxerickson
Well, I don't think they broke into the computer. But I guess my point was
more that the CFAA doesn't have any answers, you have to figure out the scope
of the authority possessed by the Department of Justice.

------
siliconc0w
I think it's more likely by header the FBI meant HTTP response header not IPV4
header.

Legally I don't know if we should allow "fuzzing" without a warrant. They've
put people in jail for doing less.

~~~
Igglyboo
Weev was put in prison for wayyyy less than what the FBI did.

~~~
meowface
I don't think Weev should have been tried criminally at all, but I kind of
disagree with this purely from a technical standpoint.

The FBI discovered a flaw that let them see the IP address of a web server.

Weev discovered a flaw that let him see private information of AT&T customers.

Customer information is generally considered much more sensitive than an IP
address.

~~~
totony
Privacy is relative. In Weev's case, AT&T considered the information of its
users to be sensible, private information, while, in SR's care, the IP address
is a very sensible and private information, considering that tor was used to
hide it.

------
kevinpet
I'm suspicious of anyone who seems to have trouble understanding the FBI's
claims. It's clear to me that what the FBI is claiming is that some part of
the CAPTCHA was linked to the server directly, rather than its TOR address.

~~~
istjohn
From the article:

Even in the hypothetical case where – for some unrealistic reason – the Silk
Road hidden site was including an image on an external server by referencing
its IP address or hostname, the agents would still observe this traffic as
having come from Tor. There is no magic way that the traffic from a real IP
embedded within the HTML of a hidden service would find its way directly to a
client without passing over the Tor network and through Tor nodes. Were this
the case, it would be a huge vulnerability in Tor, as it would allow the
administrator of a hidden site to uncover visitors by including an element
that is served directly to the client over clearnet (thankfully it isn’t and
this doesn’t work – try it).

~~~
thefreeman
I don't think they are presenting it accurately. I think what they are
indicating is it was something like this:

    
    
        Login page
        ...
        $code_to_embed_captcha
        ...
    

I believe $code_to_embed_captcha was returning something along the lines of
[http://REAL_SERVER_IP/captcha.jpg](http://REAL_SERVER_IP/captcha.jpg) instead
of [http://ONION_ADDRESS/captcha.jpg](http://ONION_ADDRESS/captcha.jpg)

This wouldn't allow you to identify users, the request for captcha.jpg is
still routed through TOR. However it does reveal the true IP of the server.

~~~
patio11
This is my guess as well. That or the captcha had a query parameter like
?nonce=1234&redirect_url=$HOSTNAME/login

------
batbomb
We just barely have some solid ground protecting phones you have on-person
from unwarranted search, and, AFAIK, there's no real laws protecting your
rights when you have data residing on third party servers in foreign
countries.

I'm definitely not a lawyer, but the server had an unknown provenance and it
was physically located in Iceland, and that probably makes for some fuzzy
ground at best as to whether or not what _anything_ the FBI did to gain some
sort of access to the physical server, even if it was made possible by
parallel construction, would even be illegal.

------
s_q_b
Look, this is a great chance to see if anything is amiss. We can simply
directly test our hypotheses:

Start up the Sill Road server with a tor circuit in front of it (isolated from
the main network.) Then have the FBI demonstrate how to get the server to
cough up it's IP. It's easy to prove who's telling the truth here.

~~~
malka
not really, it could be parallel construction : they found the flaw after
having seized the server.

------
coriny
Well, I can't say much about the reality of the FBI approach, y'all seems to
have that better covered.

Instead, I'm led to ask at what point will people stop using "myself" as a
formal version of "me"? It isn't, it actually has a completely different
usage.

In this case: "the SR Server was located by myself and another member of the
CY-2". Here 'by myself' means 'on my own', not 'by me', so it directly
confuses the meaning of this sentence. To me the obvious meaning is that they
both work independently and found the same thing.

About the only other time you can use '-self' is in the reflexive, as in "I
did this to myself", "you do this to yourself". I will _never_ do something to
yourself, and you will _never_ do something to myself.

So, I know this is completely out of place, but if I can get one person to not
do this in the future, I will make my life a bit happier. Also if I can
persuade that small group of nameless people (you know who you are) that
"could care less" actually means the opposite of what they think it does; and
that "alot" is not a word.

I'm all for the English language evolving and expanding, but let's weed out
the bad ideas which only increase confusion and bring nothing good. And
seriously, "alot"? "Lot" is a three letter word, it's not hard to spell. As
for "a" ...

~~~
_archon_
I understand your confusion, but this is a common usage case for the
nominative first person of "me" in formal speech. This is one of a gajillion
special case word choice situations. This particular sentence structure
(Direct object, past perfect verb, subject) suggests "myself". However, the
writer should have put in the other person's name before "myself", as is
common practice, leading to confusion.

Consider: "the SR Server was located by another member of the CY-2 and
myself".

I'm not claiming that "me" wouldn't parse, but that using the current rules
properly would not cause confusion in the first place. Someone is trying to
sound intelligent, while putting their name first so as to emphasize their
contribution.

~~~
coriny
So, I'm definitely not confused :)

I do appreciate it's very common in what I'd call pseudo-formal speech, mostly
used when trying to impress someone one perceives to be of higher social
station. Estate agents do it a lot when talking to clients. And so it does
have the "well, lots of people do it" support.

However, if you go back to the roots (Latin), "myself" is specifically the
reflexive form of the first person. When the first person is the object or the
dative, it is "me/to me". So to have correct agreement with the verb, your
example should end in "and me".

While your example is easy enough to parse, it doesn't actually follow the
rules - though maybe the common practice. And if you rearrange the two object
nouns (i.e. back to the original) the sentence loses clarity. Using "me" never
loses clarity, and the ordering of object nouns shouldn't matter beyond
providing emphasis.

Basically, that example is ok in Perl, but not in Java ;)

And ultimately, using the reflexive form incorrectly never makes a sentence
clearer, and it's longer to write and say.

Anyway, this is so far off the OP's topic, which is actually much more
important & interesting, I do apologise to him/her.

------
hadoukenio
If they obtained a warrant, is fuzzing/hacking Silk Road the equivalent of
kicking down a door during a raid?

~~~
ascorbic
They didn't obtain a warrant.

------
praptak
If I ever set up a hidden service I'll make sure it lives on a box that has no
public IP whatsoever. Preferably a physical box.

~~~
nwf
Unfortunately, the technology you need to do that is not yet finished:
[https://trac.torproject.org/projects/tor/ticket/9498](https://trac.torproject.org/projects/tor/ticket/9498)

ETA: I should have said "one possible technology"; there may be others, but I
am pretty sure that an IP-less Tor node requires that you play the Tor Bridge
game and stream the Tor protocol over a non-IP link. I have had a prototype of
this design running, but have yet to have it to a point I consider robust.

~~~
icebraining
praptak didn't say IP-less, but no _public_ IP. If the server has an IP in the
192.168.1.X range, that doesn't tell an attacker much, supposedly.

------
meowface
While I agree that the FBI's explanation as they gave it is impossible, I see
2 possible explanations for how their explanation would make sense. I don't
know why they would glaze over these details, but:

1\. They could have meant simply viewing all the HTTP response headers with
something like Chrome's dev tools, Firebug, or possibly a proxy like Burp
Suite.

2\. They may have set up Tor to essentially use no hops and use only their own
host as an exit node. That way they would be able to directly observe traffic
from a clearweb IP address in a PCAP. Or they could set just a single hop and
specified another host they control as an exit node. This seems way overboard
just to find an exposure like this, but it's possible that the FBI or NSA have
such test environments to probe for other Tor-related vulnerabilities and
misconfigurations.

------
ufmace
The thing that struck me as the most odd about this article was the assertion
that keeping a hidden service securely hidden must be easy, because the TOR
website's Wiki article on it is so short. I think it's clear that there isn't
that much proven experience out there in keeping a hidden service secure
against a determined and well-funded adversary, and what experience there is
probably hasn't made it to the TOR Wiki page for various reasons. Just the
last HN article on this generated dozens of comments with various best-
practices for securing such a service that clearly weren't followed by Silk
Road, and we're mostly amateurs at this.

------
ianstallings
The guy was _not_ a good developer and they caught him because of it. It
really does appear to be that simple. No parallel reconstruction needed when
you're dumping server vars to the public.

------
Eleopteryx
Slightly off topic: could you please increase the font size on your site? It's
actually below 12pt. The blockquotes are more readable than the body of the
post.

~~~
DougBTX
It is 17px for me, but most browsers support zooming into a page, that will
make text easier to read on almost any site, but could make it more difficult
if accidentally zoomed out.

~~~
wcummings
Every browser has supported this since forever

------
BadDev
Why is using var_dump debugging bad practice? Or is he insinuating that using
var_dump while testing on a live site is bad practice?

~~~
cookiecaper
I think he infers that both are bad practice, but for the record I don't think
there's anything wrong with var_dump debugging. Sure, a real interactive
debugger is nicer, but it can be a pain to set up with PHP (I know, having set
these up several times). Print-based debugging is fine as long as you don't
expose sensitive information to end users.

~~~
girvo
Difficult to set up? I disagree, Xdebug with connectback is amazingly easy to
set up in a Linux VM! On windows, yes, it's hard, but Linux and OS X its very
simple nowadays. I should write blog post on how to do it (having done it
dozens of times over the past 6 months)

------
gcb4
1\. get warranty to track someone id'ed as recipient of usps drug shipment.

2\. exploit client tor missconfig

3\. repeat 1-2 until you have a net so wide of tor users that only node out is
the service

------
m3talridl3y
Nothing will keep you from getting hacked by a determined government, but
using Apache/PHP isn't exactly trying to make their life difficult. But at
least they didn't use Ruby on Rails. :P

~~~
jradd
So you are saying Rails might make intruders' lives less difficult? Of course,
the _implementation_ is what would be more or less secure, but i'm a little
curious what you meant.

