
Reading the Silk Road configuration - philip1209
http://blog.erratasec.com/2014/10/reading-silk-road-configuration.html
======
nisa
So - how to do it correctly? The onion server should not be able to access the
internet directly and should never know it's own IP-address. So putting it
inside a virtual machine/container and redirecting outgoing traffic through
Tor (on another VM) should fix that.

As for the backend servers - if all machines are connected through OpenVPN
with TLS-Auth they can use private IP space and are not accessible from
outside. The OpenVPN tunnels are configured outside of the onion VM and Tor VM
and the backend machines are also in a VM that pipes all outgoing traffic
(except OpenVPN traffic) to the internet through Tor.

This setup should at least omit the problem of leaking public IP addresses on
compromise or did I overlook something? Let's ignore VM outbreak exploits
(ksplice/unattenend-upgrades should help here if it's not the NSA).

With TLS-Auth and a default DROP firewall rule all machines should not even
appear in scans on the internet. So any outside contact to any machines is
only possible if you know the OpenVPN TLS-Key + Certificates. An intruder
would only see local IP addresses even if he manages to compromise a backend
server. All created outgoing traffic to find the machines would be routed
through Tor.

As the Tor onion VM is separate from the webserver only a Tor remote exploit
or a VM outbreak exploit would be critical. Maybe encryption can help here.

Another point is using dedicated hardware with Full Disk Encryption and just
enter the passphrase via the provider console. There are cold boot attacks but
VPS servers allow to dump a memory image while running - that's not so easy
with dedicated hardware.

~~~
tokenizerrr
Say you have your own hardware, what would you need to access the display and
keyboard securely remotely over the network? Are there such devices available?
I imagine you can have the OS disable its display and keyboard input after
boot for added security, and lock down the boot process, but it still seems
like it's hard to do this securely. I'm trying to think of a scenario where
you can have:

1) Full Disk Encryption

2) Two physical machines owned by you, perhaps stored in some basement

3) Be able to boot them back on without physical access (perhaps this is
simply a Bad Idea?)

4) One machine with two NICs running Tor, exposing only Tor to one of the NICs

5) The other machine running a VM host with 1 VM for each of your services.
The host is connected to the NIC of the first machine, thus only has access to
the internet through Tor

~~~
teddyh
> _Full Disk Encryption_

[…]

> _Be able to boot them back on without physical access (perhaps this is
> simply a Bad Idea?)_

This is _exactly_ the problem which our project solves:

[http://www.recompile.se/mandos](http://www.recompile.se/mandos)

Introduction:

[http://www.recompile.se/mandos/man/intro.8mandos](http://www.recompile.se/mandos/man/intro.8mandos)

Summary:

 _The computers run a small client program in the initial RAM disk environment
which will communicate with a server over a network. All network communication
is encrypted using TLS. The clients are identified by the server using an
OpenPGP key; each client has one unique to it. The server sends the clients an
encrypted password. The encrypted password is decrypted by the clients using
the same OpenPGP key, and the password is then used to unlock the root file
system, whereupon the computers can continue booting normally._

~~~
ryan-c
Mandos is pretty cool. Debian and Ubuntu can also embed dropbear into the
initramfs so you can ssh in and unlock the disk.

~~~
teddyh
Dropbear (SSH access) still requires you to manually log in and unlock the
disks; a primary feature of Mandos is that it can boot completely _unattended_
(unless configured otherwise).

~~~
ryan-c
Yup, I've used Mandos, just stating that there are other options if you don't
want automatic.

~~~
teddyh
Mandos can be configured to be not automatic by setting the
“approved_by_default” option for a client to “False” in the clients.conf file;
See mandos-clients.conf(5): ([http://www.recompile.se/mandos/man/mandos-
clients.conf.5](http://www.recompile.se/mandos/man/mandos-clients.conf.5)).

You also need to adjust the “approval_delay” setting.

------
jackgavigan
There are some jigsaw pieces missing here.

If the nginx configuration for port 443 did indeed _not_ restrict access to
[star].php, then that means that index.php would have been accessible to the
Internet at large (although HTML elements with other suffixes - e.g. .jpg,
.css, .js - would not have been served).

If the CAPTCHA element's URL also ended in .php, then it's not beyond the
realms of possibility that Tarbell could type the IP address, followed by
/index.php and end up seeing a screwed-up version of the SR home page, _with_
the CAPTCHA as he describes in his testimony.

The log file entries cited are for port 80, whereas the SR webserver ran on
port 443.

If the defence already have all the log files, they should grep for
199.170.71.133 in the 443 logs and/or search for a group of log file entries
with simultaneous successful serves of anything ending in .php, with
"permission denied" failures for things _not_ ending in .php

Incidentally, the May 3, 2013 webserver IP leak referred to in footnote 5 to
Tarbell's testimony syncs up nicely with the date of this thread on Reddit:
[https://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_...](https://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_be_worried_showing_on_login_page/)

 _Credit to Michael Koziarski for the Reddit
link:[http://krebsonsecurity.com/2014/10/silk-road-lawyers-poke-
ho...](http://krebsonsecurity.com/2014/10/silk-road-lawyers-poke-holes-in-
fbis-story/comment-page-1/#comment-296697) _

------
BuildTheRobots
Am I being extremely thick here?

The evidence submitted by the FBI [1] shows the phpmyadmin page running on
address 192.168.1.24. Are they claiming to have connected over the internet to
a public facing RFC1918 address and it was routable?

(Saying that, I'm now realising that I could knock up a similar screenshot
showing me connecting to _any_ "real-world" IP address anyway; it seems to
prove nothing.

[1] [http://krebsonsecurity.com/wp-
content/uploads/2014/10/70-8.p...](http://krebsonsecurity.com/wp-
content/uploads/2014/10/70-8.pdf)

~~~
samcrawford
Just a guess, but I imagine that's a clone of the original server (using the
server image they took) running on a local network.

~~~
BuildTheRobots
It's no longer a clone if they've made config changes... o_0

------
comex
Talk about burying the lede! Even though he starts off by saying the data
confirms the "parallel construction" theory, he proceeds to poke a hole in the
_defense 's_ claim - that someone wouldn't have been able to access the login
page (I am too tired to determine whether or not he's correct about this), and
then proposes a possible mechanism (scanning the entire Internet for pages
containing a term) which, although it would technically constitute "parallel
construction" in that it wasn't mentioned by the prosecution, need not have
anything to do with the NSA and could have been done (with enough time) by
anyone.

If he's right that the quoted configuration file doesn't do what the defense
said it did as part of their accusations of lying, they've seriously messed
up, no matter what the truth is.

------
samcrawford
> The second problem is that some of the details are impossible, such as
> seeing the IP address in the "packet headers".

If the frontend server in Germany is reverse proxying through to the backend
server in Iceland, then sure, a user is not going to see the Icelandic
server's IP in the source IP field of the packets. But I don't see this as
definitive proof of the FBI's assertion being a flat out lie. The IP could
easily have been exposed in the packet body.

What happens if you visited the captcha URL with a HTTP/1.0 request without
Host header? If the resulting URL generated any self-referential links, what
did they use as the hostname? If the Host header is available the norm is to
use this, but if not then the script may use the server's FQDN or IP address.
If it sent a 301/302 redirect in the HTTP response headers, then that _must_
contain a hostname according to the RFC (it shouldn't be relative), so what
was used there? There's nothing in the nginx config that rewrote such response
headers.

What happens if you make malformed requests to the captcha URL? Do you get an
error page with the IP address embedded, or something that references an
object hosted on the IP?

These are just two possibilities, and yes, neither would lead to the IP being
exposed in the 'packet headers'. But it's very feasible for it to be exposed
in the packet body, so it seems silly to hang the entire argument on the basis
that one word is correct, without considering the alternatives.

~~~
forgotAgain
In both the cases you site would there be a trail of those requests in the
server log files? If so they could have been (or will be) pointed to by the
prosecution. Since the prosecution has an image of the server they should be
able to demonstrate the point if needed. If they can't then I think it casts
doubt on their story.

~~~
samcrawford
You'll see the requests in nginx's access logs, but it might be very hard to
isolate them. These would not show the response data (only the length and the
response code).

But you're right, I'd expect that whatever tactic they used, the prosecution
should be able to demonstrate in far more depth how the IP leaked.

------
thornofmight
I know for a fact that there was a source disclosure MySQL injection bug on
many of the pages in SR.

There was also other MySQL Injection bugs. You could even look through the SR
forum archive and find people talking about how the search field at one point
was exploitable by the standard "' or 'a'='a" and was disclosing customer's
names and addresses.

------
peteretep
I don't know how the US legal system works - but is there any chance of the
DPR actually walking away a free man if the FBI are lying?

~~~
downandout
It would be a big step forward. The government would be unable to use anything
that it obtained from the server in their prosecution. This may make it
impossible to prove that he ran Silk Road. I think he has several other
charges though, including multiple counts of solicitation to commit murder.
Those may well involve evidence from outside the server that would be
unaffected by this.

Given this, the odds of DPR ever seeing the light of day, even if they win
this suppression motion, are quite small. He is looking at multiple life
sentences even without Silk Road specific charges. But his lawyers are going
through piece by piece, hoping to convince prosecutors that it will be
difficult enough to get convictions that they offer him a substantial, but
less than life, prison sentence in exchange for a plea. Even then we're
talking about decades in prison.

~~~
icebraining
_I think he has several other charges though, including multiple counts of
solicitation to commit murder._

Nope, most were dropped before the indictment. Only one (in Maryland) remains:
[http://freeross.org/correction-of-our-report-on-the-
indictme...](http://freeross.org/correction-of-our-report-on-the-indictment/)

~~~
downandout
Still, one is probably enough for a decades-long or life sentence.

------
NoMoreNicksLeft
I thought it was a foregone conclusion that the NSA uses its massive
surveillance network to do traffic analysis on all entrance and exit nodes.

What we're seeing here is called "parallel construction". The FBI was given
this illegally obtained surveillance data, made the arrest, and then needed to
make up a lie as to how they really found him.

------
dolzenko
Fun, failing to count how many times he said "I am an expert", I assume thats
the basic tactic to survival in security consultant space but still.

~~~
meowface
This is actually a legal convention, not arrogance. Look at criminal
complaints for other computer crimes (child pornography, hacking, etc.) and
the language is identical.

~~~
drdaeman
Hm. "As an expert I can attest DPR sucked at opsec"¹ is a bit unconventional
for a legal convention. ;)

¹) Not an exact quote, I know, I glued two sentences together and cut some
parts.

~~~
meowface
Nearly all infosec experts can agree that DPR was awful at OPSEC, and expert
testimony is a real legal thing. :)

~~~
drdaeman
Yeah, I was only a bit curious about the wording of the statement.

------
us0r
The Iceland/Reykjavik bits are pretty interesting[1].

I wonder if they just said here you go? It sure seems like it.

[1] [http://krebsonsecurity.com/wp-
content/uploads/2014/10/70-4.p...](http://krebsonsecurity.com/wp-
content/uploads/2014/10/70-4.pdf)

------
denzil_correa
Is it time to have special judges/jury who understand the technical details of
such cases? I can't seem to even think as to how a judge/jury with no
technical expertise would understand the fine print of the details being given
out by the lawyers.

~~~
Yardlink
I think expert witnesses serve that purpose. There have been complicated
technical details in jury trials before the internet and this seems to be an
adequate way to solve the problem.

~~~
spacemanmatt
The problem with expert witnesses is that the skill required to tell which
expert is more credible is the same skill that each expert claims to represent
best. And then you have the outright liars who have made their careers being
expert liars in court, like Dr. Michael West, who are not backed up by the
medical community, but by prosecutors.

------
CPLX
Can someone who knows things about things address this item in the errattasec
article:

> BTW: one plausible way of having discovered the server is to scan the entire
> Internet for SSL certificates, then correlate information in those
> certificates with the information found going across the Tor onion
> connection.

Would this be considered parallel construction, or would this be a legitimate
way to attempt to figure out who was involved in the Silk Road, and is it
plausibly the way the FBI might have zeroed in on the server?

~~~
frontline
parallel construction is retroactively finding an alternate,
legal/unclassified explanation for an evidence trail, but one which is
inherently incorrect; if the ssl cert scanning or other repeatable method was
the route taken, nsa probably wouldn't want to tip their hand. it's probably
worth noting that the only examples of parallel construction that we have
confirmation of were drug cases

~~~
CPLX
Yes I know what it is, but what I'm curious about is would that tactic fit the
definition. Scanning IP addresses for publicly served SSL certs and comparing
them with one served over TOR isn't obviously an unwarranted search is it?

~~~
sp332
If the original TOR cert was discovered in an unlawful search, and then they
did a scan of the public internet to find it again in a legal search, that's
parallel construction.

~~~
AlyssaRowan
And let's not forget: NSA already have an almost-irresistibly useful database
designed for _exactly_ this purpose (selecting and correlating on attributes
of SIGINT-captured SSL/TLS sessions, such as certificates - and they could
easily just put a selector on the CommonName or the certificate fingerprint).

You don't have to be the NSA to make a database like that, but it helps. I
could build a database broadly like that for certificates/ciphersuites/other
metadata myself with active scanning and zmap (and it might make a good
weekend project, to examine and contrast RC4 proliferation amongst TLS-
encrypted web and mail servers) - but they have a near-realtime-updating
passively-constructed one. If the FBI asked them for help, they'd definitely
use that.

------
kerkeslager
It's hard enough to explain this kind of stuff to one intelligent
businessperson--there's just so much background knowledge that goes into
understanding this. Explaining this to 12 jurors, half of whom will likely be
of below average intelligence, is a pretty much impossible task. Sadly I think
this case will be won on the charisma of the lawyers rather than facts which
will go over most jurors' heads.

------
Evolved
This may have something to do with discovering the real IP.

[https://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_...](https://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_be_worried_showing_on_login_page/)

------
plumeria
How much time would it take to browse all public IP addresses?

~~~
adamnemecek
Depends on your definition of 'browse'. But possibly 45 minutes
[https://zmap.io](https://zmap.io).

~~~
huuu
Note that in one of his talks Morgan Marquis-Boire said he had to change IP
some times because he was blocked generating too much traffic while scanning.

But he managed to map a lot of secret Trojan servers used by govs.

------
pbhjpbhj
Isn't it contempt of court to bear false witness; don't people caught doing
that get charged and put in prison? So, which person was in the witness stand
and will USA prove to be a democratic nation under the rule-of-law and charge
them?

~~~
spacemanmatt
The government (executive) lies to other branches of the government
(judiciary) with impunity. Once in a while, a judge will get angry about it,
but prosecutors and expert witnesses issue whopping lies into the official
record every day.

~~~
pbhjpbhj
Aren't USA judges voted in? Why do the people vote for judges that will sit
quiet whilst someone provably lies in court?

We found them by simple looking at the IP that the packets of the captcha were
served from. The captcha was served over tor, based on the evidence that's
impossible. Either you lied or you need to add further evidence. If you
can't/won't add further evidence then as the court sees it you lied and are in
contempt and will be punished in line with minimum sentencing??

This is of the order of a "he robbed me at home at exactly 2pm", "your
submitted cell-phone evidence says you were at work from 1pm until 4pm".

If the defendant is convicted on some other counts can they, or anyone else,
do anything or is the allowance of officers of the FBI to lie in open court
somehow embedded in the USA constitution.

As a foreigner to the USA this sort of thing just undermines the entire
foreign policy rhetoric of bringing democracy to the world. Bring some damned
democracy to the USA first: government by the people for the people, my arse.

~~~
derekp7
Any time you have two opposite sides in court, one says yes the other says no,
one of them is lying. So, in addition to a regular guilty / not-guilty
verdict, do you want to also see an automatic contempt of court verdict for
the losing party?

~~~
pbhjpbhj
If they provably and knowingly lied it seems reasonable. Here there seems like
~0% chance that the FBI expert in computer forensics (which they'd need to be
to be running such an investigation, surely) can't possibly know that he's not
lying?

If you can't tell the truth in court then keep your mouth shut or you'll be
punished accordingly seems like the exact message a country intending to
operate under the rule-of-law should be promoting.

------
iaw
I wish they'd have made the FBI's story a little more believable. I mean, it
seems _slightly_ more legitimate than what you see on TV and movies about
hacking.

~~~
junto
I have a feeling that they don't actually care. They know that judges and
juries can't handle these kinds of technical details.

~~~
AlyssaRowan
That's what experts are for.

I don't do forensics. But I am a reverse-engineer and I am familiar with the
techniques: more familiar than Tarbell, it seems. (That's really his name?
Tarballs from Tarbell? My goodness.) Tarbell's declaration reads to me more
like a textbook demonstration of (bad) parallel construction in action.

They _could_ have done it legitimately, without compromising the server and
potentially tainting the evidence any way they wanted: DPR indeed made a few
rookie mistakes that would potentially provide for that. But the logs don't
seem to actually have evidence supporting that, which is _very_ unusual and at
this time not explained? The declarations filed so far do not really seem to
support that either, which is very odd and strongly suggests that we don't
have the whole picture here: and we _really should_.

(Of course, we don't have the whole image, so we don't have the whole picture
here. BTW: They used tar, not dd or ddfldd? Boo.)

------
wrboyce
Why not link the original Krebs article?
[http://krebsonsecurity.com/2014/10/silk-road-lawyers-poke-
ho...](http://krebsonsecurity.com/2014/10/silk-road-lawyers-poke-holes-in-
fbis-story/)

~~~
dang
Ok, we changed to that from [http://blog.erratasec.com/2014/10/reading-silk-
road-configur...](http://blog.erratasec.com/2014/10/reading-silk-road-
configuration.html), which points to it.

~~~
dchest
Please, change it back. These are completely different articles, and most of
comments here are about the first one.

~~~
dang
Ok, done.

~~~
colinbartlett
Thanks for all the work you do to make this place a little bit better every
day.

~~~
dang
You're welcome! I appreciate your saying so.

------
tuxidomasx
The FBI owes me 1 bitcoin.

------
snowy
I am a foremost Internet expert on this sort of thing..........As an expert in
such topics as sniffing passwords.........As an expert, I know the Tarbell
declaration is gibberish. As an expert reading the configuration and
logs........

HA!

~~~
Luc
It is somewhat comical, but he's not lying. At least he's not hiding the fact
that he's arguing from authority (his own).

~~~
jessaustin
Arguments from authority might be a bad thing on internet fora where everyone
is assumed to have infinite time (and ability) to dig into details. In society
as a whole, and especially in the courts, however, these arguments are a
useful short-cut.

------
thomaslutz
"myphpadmin"? "200 error code"? "As an expert[...]" ... no comment.

