
The Looting of ShapeShift - danielvf
https://news.bitcoin.com/looting-fox-sabotage-shapeshift/
======
danielvf
This is certainly the worst case scenario - your security officer installing
remote access software on developers machines, stealing bitcoins from
production, then selling the company source code, access credentials and
access to the internal network to a Russian hacker.

Building a security system to handle this level of attack is a whole level
beyond stopping even determined external attackers. Are there any best
practices guides on this?

One thing that the article showed is the importance of external security
review to deal with the threat of internal incompetence or evil.

~~~
illumin8
It's extremely expensive. Many banks and companies in the finance industry
(hedge funds) do this:

Hire at least 2 or 3 people for every job. Have them watch each other whenever
touching systems that connect to production or deploying code to production.
Never trust any one of them with the private keys or passwords to anything -
they can only get half of a secret and their co-worker gets the other half.

To do this effectively, you have to build a zero trust environment, and rely
on surveillance to ensure that nobody is a bad actor. It really makes CIA/NSA
level security look somewhat weak.

It's also very expensive, as you can imagine.

~~~
rodgerd
> Hire at least 2 or 3 people for every job.

This is actually the key one - the more people who need to be corrupt, the
harder it is to get away with being a crook. A surprising amount of internal
fraud can be completed simply by requiring people to take solid blocks (2
weeks plus) of leave every year.

That "inefficiency" the "lean 10x disruptors" congratulate themselves over
does not always end well.

~~~
Terr_
> solid blocks (2 weeks plus) of leave

I've heard that's because a lot of those scams/tricks fall apart without
constant gradual intervention, so an important part of it is that the employee
is _prevented_ from accessing most work-resources during that time.

------
quanticle

        We learn some more things. Bob has prior police records in Florida, where he’s from.
    

So they didn't even do a background check before hiring "Bob"? For a position
where he would have access to systems that handled financial data? That's just
grossly incompetent, in my book. I've worked for 5-man startups and Fortune
500 companies. In every case, the offer letter has stated that the offer is
conditional on the successful completion of a background check, and none of
the positions I've held have been remotely as sensitive as the position that
Bob was hired for.

~~~
cloudjacker
They don't do the background check.

~~~
randomfool
They do- one company found a warrant on me that I didn't know I had for an
unpaid traffic ticket.

~~~
omarchowdhury
Do you know where one would go to perform an accurate background check on
oneself?

~~~
exhilaration
I use [https://www.e-renter.com](https://www.e-renter.com) to screen potential
tenants along with
[http://www.experian.com/connect/](http://www.experian.com/connect/) for
credit checks.

------
tekromancr
Facinating. More interesting reading seems to be available here:
[https://www.patrolx.com/wp-
content/uploads/2016/04/309591980...](https://www.patrolx.com/wp-
content/uploads/2016/04/309591980-ShapeShift-Postmortem.pdf)

------
marco_salvatori
For those interested in security engineering in the financial industry a good
reading source is the banking section of:
[http://www.cl.cam.ac.uk/~rja14/book.html](http://www.cl.cam.ac.uk/~rja14/book.html)
One interesting and relevant take away (of many) -- The greatest fraud threat
to a financial firm are insiders. About 1% of staff across the industry is
fired every year due to fraud. Within the fraud incidents, the most damaging
fraud is perpetrated by senior and trusted individuals.

------
koolba
Not the best writing but it reads like a modern hardboiled[1]. Fun piece.

[1]:
[https://en.wikipedia.org/wiki/Hardboiled](https://en.wikipedia.org/wiki/Hardboiled)

------
felixgallo
I'm surprised that anyone would take this at face value. The cryptic black hat
responding to e-mail with "one word: bob" is straight out of the most teenage
of hacker fanfics.

------
jessaustin
It seems odd to me that "Bob" hasn't been outed. It almost makes me suspect
that someone isn't sure how much of "Bob's" role as portrayed in TFA is true
and how much is a frame-job by an untouchable hacker [EDIT:] or wishful
thinking by a frustrated executive.

~~~
adekok
> It seems odd to me that "Bob" hasn't been outed.

Since Bob hasn't been criminally charged with anything, "outing" him is
legally "libel". He could sue, and win.

~~~
jessaustin
Why didn't you quote the next sentence, in which I directly addressed the
possibility that TFA is wrong about "Bob"?

Let me guess, YANAL? In USA, it ain't libel unless it's _provably_ false. If
"Bob" could prove that, why did he run?

------
keithpeter
_" We had changed almost everything, but hadn’t scrapped our personal
computers used while Bob had been part of the team. Would that have been the
paranoid thing to do? Yes."_

At my humble and refreshingly drama-free place of work we have standard client
images. Anything weird and the techies re-image the client. Assuming 'Bob'
wasn't in charge of the images, would such a procedure have sorted the rdp?

~~~
adekok
Maybe.

The larger question was why did Bob have root access to people's individual
laptops? He could have done a "snowden", grabbed their SSH keys including
passphrases. That would have been much harder to detect.

~~~
azernik
That attack would have been prevented by their immediate SSH key rotation
after the initial attack.

------
Nutmog
Let's not lose sight of the fact that their cold wallet was untouched and all
they lost was on the order of a hour's worth of turnover. That's more than can
be said for a lot of the other bitcoin hacks.

------
a_small_island
I thought this was a great story to read. Explained in laymans terms when
needed and kept my attention throughout.

Bob sucks.

------
oh_sigh
What's the status of civil/criminal charges against Bob?

------
buttershakes
This reads like a case study in pure incompetance at every possible level.
Lack of vetting, no third party auditing, poor segregation of customer funds.
It's a total shit show. This should permanently damage their business and
reputation, but the Bitcoin community has always been forgiving of people who
lose their money. Fool me once...

------
Ontheflyflyfly
When being accused of bigotry, it's never a good sign to use "social serfdom
number" in your post mortem.

~~~
tekromancr
That's just some shit that ancaps like to say. Also note the constant
references to fiat currency. I don't think he was actually being racist with
that.

~~~
Ontheflyflyfly
I agree, I don't think "serfdom" is racist, but it doesn't shine a good light
on you to use, and erodes the trust people put in you by default.

~~~
tekromancr
Agreed.

------
Ontheflyflyfly
"Very quickly, we realize he is pretty much useless. "

When you hire IT people, have no clue how distinguish between a good one and a
fake one, in other words have no clue, this happens.

Also not enough oversight and auditing admins when money is involved is a bad
sign.

------
cortesoft
Man, calling a social security number a "social serfdom number" is really dumb
and off putting. So is the continual reference to 'fiat money' constantly.

I always love the irony of people so against the basic social contract are
always so quick to turn to authorities when things predictably go wrong.

~~~
cloudjacker
You might appreciate

[http://www.newyorker.com/humor/daily-shouts/l-p-d-
libertaria...](http://www.newyorker.com/humor/daily-shouts/l-p-d-libertarian-
police-department)

~~~
ikeboy
[http://www.theatlantic.com/politics/archive/2014/04/nlpd-
non...](http://www.theatlantic.com/politics/archive/2014/04/nlpd-non-
libertarian-police-department/360224/)

~~~
bduerst
While I think both of these are great, it still doesn't explain calling it
"social serfdom number".

~~~
walrus01
it's a blunt dog whistle for "hey libertarian gold hoarders we are your sort
of people"

------
floordaemon
When things are handled professionally, you'll have a lot less drama.

Why is the author being so nice to the theif over and over again? Even the
last sentence!

Eric brought this all on himself. Obviously 'scared' and intimidated to
discuss important things WHEN they need to be discussed.

Doing everything out of order -- like the background checks -- is a classic
rookie move.

You are basically running an Online Banking Website. You need to be aware of
immense risks...

...it just goes on and on. But its clear why there is so much drama, crime and
corruption and this company.

Sad really. Who is to blame?

------
fasteo
Am I the only one to think that all this narrative to blame Bob is pathetic ?

This is pure and simple Mr. Voorhees (CEO) incompetency. After all, Bob is a
criminal and he was just doing his "job".

~~~
danielvf
The article seemed pretty open about major mistakes that ShapeShift made and
lessons learned. It's a good postmortem to learn from, and far more open than
most would have posted.

~~~
illumin8
One of the striking things in this article was when he said they might have
been compromised by their "CloudCo" (Cloud Provider). If I'm going to build
any systems that handle money or bitcoin in a cloud provider, I will make damn
sure I don't trust the cloud provider with anything.

Everything should be fully encrypted such that even a breach of trust from the
hosting provider would not compromise your data/funds. I know this is hard to
do, but it's mandatory when you're handling digital currency.

~~~
gridspy
Encryption won't protect you - the cloud provider has access to executables
(in ram and perhaps on disc), your keys (ram and disc) and the data both pre
and post encryption (in ram).

Because they control the hypervisor, they control everything. That means they
have as much access and authority as the code that you are running on their
servers have. So the only way to protect yourself from them is to limit what
your servers (deployed on their cloud) can actually do.

So for instance you could have a secure backend server on a dedicated host in
a trusted environment, with the cloud servers using an API to the backend
server. If the API is suitably secure then the cloud servers could be
compromised without allowing them to directly issue invalid commands in the
same way the backend server could. Then you could use the cloud to scale out
your web frontend without compromising yourself.

The same is true of hardware on the dedicated host (such as the "Trusted
Computing" Module) that you do not control. If that (or the BIOS) gets
compromised you might not even know that your host is no longer secure.

~~~
illumin8
You can use hardware security modules in datacenter space you physically
control to store the private keys used to encrypt your data at "CloudCo".
Amazon even offers this service and calls it Cloud HSM.

There's always the in-memory vulnerability, which is harder to mitigate, but
requires an attacker with physical access to the hypervisor, so it's much more
difficult to execute (as most meat-space hacks are).

