

Ask HN: How should a startup handle a security breach? - doh

Imagine you have a startup. You are working few last months incredibly hard every day. Things are going well. The press noticed you and they love your startup. Your user acquisition is crazy high and they love you.<p>First scenario:
In one very nice day you will receive an email that someone downloaded all your users data and he will publish everything if you will not pay.<p>What would you do? Will you pay? Will you call authorities? If he will share this information with the press they will eat you alive.<p>Second scenario:
You just noticed something weird in your system. Someone else was logging on your servers as a root. You are freaking out. Asking everybody in your company, if they know the IP address. Unfortunately, nobody was loggin in from Washington, because everybody lives in California.<p>What would you do? Should you inform users? What will you tell them?<p>I'm just curious, because almost nobody talks about security in startups and what to do if something really bad will happen.
======
mappu
Yikes... i don't know what i'd do. I hope with enough preventative measures
i'll never have to deal with something like that.

First scenario: First priority is to branch and patch the hole asap. There's
no point dealing with this if someone else is going to put you back in this
position a month later. Make sure everyone in the company is on the same page,
and then unfortunately i'd probably attempt to verify that the request was
legitimate, and ask for proof with the pretence of being willing to pay.. Try
and think of some way to make the information useless to the perpetrator, but
in the end, i probably would pay up (under some contract) and maybe offer
employment as well.

Could possibly whip up a bug reporting program, with well-defined rewards in
the vicinity of what was being asked, and give the perpetrator a nondescript
shrug toward it. That way the perpetrator gets an easy, moral way out, which
is almost as important.

Second scenario: Security audit, change all passwords, IP restrict logins, re-
encrypt all user passwords on login. Hopefully there was no sensitive data -
then i would wait it out to see if it turns into scenario one. It would be
embarassing to send out apologies + service credits if it wasn't malicious
(not just e.g. the hosting provider doing maintenance, someone using a proxy,
someone working out at a client's site, a contractor's ip you've overlooked,
etc)

It always feels like someone's done the right thing when you read stories
about services notifying all their users, forcing password changes, and so on.
But you have to keep up your business, and it's not like you can just shut
everything down, rewind time and never program again. Damage control and move
on.

~~~
sharth
You would actually offer employment to someone who just blackmailed you?

~~~
mappu
Maybe? I don't know. It's important to act reasonably and not to spook them
into releasing all the data. My company could probably do with an extra pair
of hands, especially someone who's proven they know their way around debian /
SQL (probable attack vectors), and now they have the DB schema to study, so...
not the worst candidate... Plus, the company would by now be busy with the
task of patching all the exploits.

You almost have to look at it from their point of view. Their options are (a)
sign NDA, take employment, money, an honourable way out of a difficult
situation, or (b) go on the offensive, release the data for little or no
personal gain, and suffer through a police investigation and potentially jail-
time or a permanent criminal record. Option (A) works out better for your
company, your product, and for them, so you might as well aim for that.

I think this is a pretty common approach. Although, it's been used in stings
in the past, where a fake offer of employment is enough to get the person in
the flesh where the police can take over. There's one very interesting story
from Valve about the leak of the Half-Life 2 beta[1] - and of course, Owen
Walker[2] found employment after a high-profile hacking case.

It would be interesting to see what other companies in similar situations
ended up considering...

__________________________

1\. <http://www.theregister.co.uk/2008/11/14/half_life_sting/>

2\.
[https://secure.wikimedia.org/wikipedia/en/wiki/Owen_Thor_Wal...](https://secure.wikimedia.org/wikipedia/en/wiki/Owen_Thor_Walker)

------
praxprasanna
I have worked in a startup myself as the lone IT & Sec guy for quite sometime.
Generally the attitude towards securing one's IP, Infrastructure, etc is lax
atleast in the beginning days and comes into focus if something nasty happens.

My inputs wrt second scenario. 1\. Try and figure out from the log as to what
changes were made.

2\. Take a backup

3\. Ask your employees if anyone indeed logged in from that suspicious remote
IP (You never know if any of your employees used a proxy, etc.....)

4\. Review the permissions given and harden as necessary. Change Credentials

5\. If cost is a constraint, there are plenty of open source
utilities/applications that can be used for relatively lower costs than the
ones from the big companies

6\. If something as important as Intellectual Property, Sensitive information
is stolen, I guess you are obligated to inform LEA & make a disclosure.

7\. Mistakes happen!

8\. Document everything!!! I had an experience where a rogue dhcp server
popped into our LAN from nowhere! Fortunately I had documented every systems
MAC address in our LANs previously and was able to identify the rogue machine
and also created a blog post on the same. you can read here:

[http://virtualthoughts.org/2006/best-practices-network-
outag...](http://virtualthoughts.org/2006/best-practices-network-outages-and-
resolutions/)

------
jnorthrop
I wrote about this yesterday... <http://jnorthrop.me/2012/04/7/preparing-data-
breach/>

The short answer is call the authorities, then get a lawyer to help you
understand all of the legal obligations. Then expect to notify your customers,
third-party services and the credit bureaus.

------
benologist
I guess it depends on the nature of the data ... if it can be used against me
/ my users then I'd call the FBI and start sending apologies.

I wouldn't bother emailing the bad person in any case.

