

BrowserStack Post Mortem (Shell shocked) - dmak
http://www.browserstack.com/attack-and-downtime-on-9-November

======
gnur
I cannot really give a good explanation, but I feel like this isn't the whole
story. They make it sound like it isn't a big deal, but in the post detailing
the "breach" there was some talk about people stating that they have seen
other peoples session in progress. If their story on this page is true that
really should have been impossible.

~~~
jacquesm
Nice reminder to patch _all_ your servers, not just your 'active' ones (can't
believe they didn't do that).

This totally misses out on 'motive', that mail was far to cleverly crafted to
be just some drive-by hacker that targeted browserstack for the fun of it.

Good they did a write-up though, so points for them. It must be very hard to
draw the line on transparency when you write a document like this, I tend to
think that it is better to spill all the beans, even the uncomfortable parts
but for business reasons they may decide to keep some of the information to
themselves.

"Both the passwords mentioned, ‘nakula’ and ‘c0stac0ff33’, were indeed in use
a couple of years ago during our prototyping phase, and thus were present in
the old prototype machine that was hacked. ‘nakula’ was previously our VNC
password, and was hashed. However, unlike the hash used for the user
passwords, this hash is much weaker. This was due to a limitation in VNC
protocol, and we had overcome this liability by regenerating a new password
for every session, and thus ‘nakula’ has not been in use for years.
‘c0stac0ff33’ was one of our system user passwords on the prototype machine,
before we moved to key-based authentication."

Still points very strongly to a bad leaver. No outsider would have that info,
I'd be digging through my older personnel records for suspects at this stage.

Still waiting for that updated postmortem from
[http://www.codespaces.com/](http://www.codespaces.com/)

------
shawabawa3
> All user passwords are salted, and hashed with the powerful bcrypt
> algorithm, which creates an irreversible hash which cannot be cracked

While using bcrypt is great, it is not uncrackable.

It is still vulnerable to brute force attacks. If you have a strong password
it wont be cracked, but most people don't

~~~
bgdam
Can you explain this in more detail? As a crypto noob I have always used
bcrypt to hash the password concatenated with a unique salt for that password
and stored the hash in my db. Is this no longer sound advice?

~~~
shawabawa3
No, it's still best practice.

The problem is it's impossible to protect weak passwords, because attackers
can just run a password database through your hashing algorithm and see if
there are any matches.

i.e. Just because you can't reverse a hash to get the password, doesn't mean
you can't hash a bunch of passwords to figure out which leads to which hash

~~~
wastedhours
Thanks for making that point - that's been going around my head for a few
weeks whilst pondering on password encryption. Rainbow table attacks are still
more than possible with all of these algorithms when used on their own?

So salts are certainly the way to go, as, although they're "known" (appended
to hash, or stored elsewhere in DB), the hash can't be compared to a generic
table, it'd need to be a full dictionary table of hashes already encrypted
with that salt (which theoretically/hopefully is used only once)?

------
robotrobot
Hmm. Another one for your security audit. I was able to change the registered
email address on our account

1) Without re-entering my creds following login 2) With no notification to the
old email address that this had been done.

------
bdcravens
One of the most detailed, and candid, post-mortems I've read. Kudos.

~~~
MisterNegative
"All user passwords are salted, and encrypted with the powerful bcrypt
algorithm, which creates an irreversible hash which cannot be cracked."

~~~
chill1
I assume you mean to point out that it is disingenous to say that _any_
hashing algorithm "cannot be cracked"?

------
bscanlan
It's good that this was published, some tough lessons in there. It's missing a
timeline though, which is important to learn from - when was it realised that
user data was accessed, how long did it take to figure out that keys were
stolen, exactly when & what actions were taken to protect customer
information, etc.

Other interesting details not covered was whether an incident response plan
existed or was followed, how many unpatched Internet-facing servers were
sitting around and whether any existing security measures did actually work in
slowing down the attacker.

------
KhalPanda
This is probably(/definitely!) sensitive information, but I'm always
interested to know what impact this sort of thing has on a company's customer
base. Customers/potential customers being put off by the malicious
email/downtime vs. content of post-mortem and how professionally the incident
is handled.

Even a negative incident like this has put BrowserStack in the spotlight for a
short time ("no such thing as bad publicity" and all that).

------
robotrobot
Another thing to mention here - we've received no direct contact from BS at
all following this - if we hadn't been following this on HN (and we were a
little naive) we might still not know this email was not genuine. The blog
post mortem is great, but I think it's a poor show not to reach out to
affected customers directly.

~~~
neotek
I just got an email from them containing the text of this blog post, you'll
presumably get one too, it's just that it takes time to email hundreds of
thousands of people.

------
robotrobot
Is anyone from BrowserStack monitoring this thread? We received the email so
we can assume our account was accessed by the hacker. Passwords crypted and
salted (ok - changed) but what about the BrowserStack automate username and
token, generated by BS themselves, that we are unable to edit or regen?

~~~
adimania
that is in a different table which was not compromised.

~~~
robotrobot
OK, I want to make sure though. How can these creds be regenerated? What
happens if _we_ had leaked them accidentally, there seems no way to regen
them?

~~~
robotrobot
Thanks (can't reply direct to the reply for some reason)

~~~
Jgrubb
There's some kind of algorithm that prevents two users ping ponging comments
too quickly. Not sure exactly how it works, but I've had experienced that
before.

------
jtreminio
> All user passwords are salted, and hashed with the powerful bcrypt algorithm

It is so refreshing when you come across a company that takes user password
handling seriously.

------
coldcode
If only everyone who is hacked provided such detail, we might learn something
to avoid being the next provider of such detail.

------
robotrobot
Me again.

"he was only able to reach less than 1% (our estimate is 5,000 users)"

So you have 500k registered customers then? Surely not.

~~~
mercuti0
[http://www.browserstack.com/growth](http://www.browserstack.com/growth)

"475,000 registered developers"

~~~
travem
The Last-Modified header for the image is from 18 July 2014 so they could
certainly have passed 500K over the last few months.

