
Why LivingSocial’s 50 million password breach is graver than you may think - darxius
http://arstechnica.com/security/2013/04/why-livingsocials-50-million-password-breach-is-graver-than-you-may-think/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+arstechnica%2Findex+%28Ars+Technica+-+All+content%29
======
mistercow
>LivingSocial engineers should be applauded for adding cryptographic salt

Should they? Should they really? Should we also give them a cookie for
properly dressing themselves in the morning?

Uniquely salting and hashing passwords is the bare minimum for acceptable
security practices. It is not a sign of expertise, but of competence.

------
swanson
I don't get why they used SHA-1. I mean, LivingSocial has been sending people
to speak and attend a lot of developer conferences. They hire a lot of bright
people. Did none of the Rails devs raise an alert of like "Uhh hey guys, why
don't we use Rails-community standard bcrypt?".

I suppose it could be corporate bureaucracy or something, but it seems weird
that a company with at least 30+ developers (based on public members on the
github org) no one would point out that this is a Bad Idea.

~~~
rmrfrmrf
I could think of a few reasons:

1\. user management is a pretty trivial thing to code and one of the first
things you do when you set up any site. It could have been a simple oversight
before a rushed launch.

2\. Playing off of 1), once you have any users in a database using a
particular system, you have to go through a bit of a process to convert people
over (you can't convert from one hashing algo to another yourself since you
don't have the original passwords). That involves creating a concurrent
password table, creating login hooks that force the user to upgrade their
password on login (especially a pain if they're using native apps -- i'm not
sure if they are), etc. Making sure that 50 million users migrate over without
a hitch is a potentially site-breaking undertaking that could have serious
impact on the site's success. Given the rocky climate LS currently faces,
you'd have to have nerves of steel to lead a n initiative like that.

3\. Depending on how LivingSocial has their code management structure set up,
there may only be a few devs who actually have access to the user login system
(you really don't want an intern or freelancer to be able to modify those).
Given that the same devs who have that access probably have more immediate
issues to deal with day-to-day, something like the current hashing algorithm
might not be on the top of their priority list.

Not that any of these are very good excuses, but I could see how it could
happen.

~~~
rurounijones
I do not agree with your #2 regarding making users re-create their password.

Assuming you haven't been hacked yet the old password is still secret.
Therefore you modify your login system so that when the user next logs in:

* Check password against old hash

* If old hash exists and matches user's password then re-hash the password using new hash and save new hash in new field

* If old hash does not exist then just check against new hash only.

Run the above code for X time (X being the time it takes Y% (Defined by
company) of your users to login at least once.

After X time THEN you start forcing the remainder to switch over.

Done properly for the vast majority of your users should not even know you
have changed anything.

[EDIT] - With some extra code you could probably avoid having to create a new
field for the new hash by just checking against one hash field using old
algorithm then new algorithm. This would avoid a potentially costly DB
migration.

~~~
jaf656s
It's much safer to just hash twice: the first time using your old hash
function and again using a secure hash. Then to migrate you only have to
update all your customer's password hashes in place. Additionally, if you lose
your database, you don't lose all the old, weak password hashes.

~~~
rmrfrmrf
Ooo, I hadn't thought of that! For some reason, I had it in my mind that some
insecure hashing algorithm as the base would result in an insecure resulting
hash, but of course that all goes out the window when your whole password
table is out in the open.

I'm glad you mentioned "safer", because for some reason rurounijones's
approach made me nervous, but I couldn't quite put my finger on it.

Here's some more discussion on the issue:
[http://crypto.stackexchange.com/questions/2945/is-this-
passw...](http://crypto.stackexchange.com/questions/2945/is-this-password-
migration-strategy-secure) (I found that from here:
[http://stackoverflow.com/questions/3955223/password-
hashing-...](http://stackoverflow.com/questions/3955223/password-hashing-how-
to-upgrade)).

------
FooBarWidget
Hearing about all these recent hacks (LinkedIn, RubyGems, LivingSocial) really
makes me nervous and wonder how I can avoid hacks. Are there any details on
how these systems were hacked in the first place, so that the rest of us can
learn from the lessons? I already use as many security good practices as I can
(bcrypt, processes and apps running under different user accounts and database
accounts, read-only accounts, principle of least privilege, firewall,
fail2ban, key-only SSH logins, file modification monitoring, forwarding logs
to a log host, etc) but I can't shake off the feeling that I may be forgetting
something.

~~~
larrys
"and wonder how I can avoid hacks"

(I'll leave it to others more knowledgeable to answer this but keep what I say
next in mind.)

My question to you is, how big of a target are you?

I mean the security you would use if you are Madonna is not the same as a "B"
list actor or the person working in the local coffee shop.

You can put untold hours into this but unfortunately security is a full time
job and can also detract from other things you need to be concerned with that
might be more important for your particular situation.

"I may be forgetting something"

Sure but the same concern is in play when taking a trip or with anything. What
are you forgetting? Where are you going? How easy will it be to solve the
problem and how much impact will forgetting dental floss have? Are you going
to an uninhabited island or Manhattan where you can buy almost anything 24/7?

~~~
laumars
What you're advocating is bordering dangerously into the realm of "security
through obscurity" - a very bad principle to base any kind of security on.

While the former member might not be a high profile target for the moment (and
let's assume that he isn't for the sake of devils advocate), who's to say that
next week a story hits the news about his wife or children or even a company
he's affiliated with and then reporters decide to start poking about on forums
and social media for cheap information to write sensationalist reports on? Or
that hackers use that news as an excuse to target the poor guy?

Or even who's to say that next month he annoys someone on HN who decides to
play some pranks?

While you are right that that security is a full time job, you should never
advocate the numbers game because as remote as the odds might be, people do
win the lottery.

~~~
rmrfrmrf
I think both of you and your parent post's points are well-taken. IMO you
should do all that you can to secure your site within practical reasoning.

The LivingSocial password hashing debacle is an example of a security failure
that pretty much every website in the known universe could improve on by
changing a single line of code -- that's what's so frustrating about it in the
first place.

There are cases, though, where certain security measures are probably not
needed. For example, I would probably not recommend that my mom use a keyfob
security token to log into her blog admin page (although I'd probably offer to
make one for her with an Arduino as a weekend project lol), or hire armed
guards to guard her database servers. Enterprises certainly need these
measures along with many others.

~~~
laumars
What you're discussing is how valuable is the data. eg if your mum lost her
blog (and kudos to her being tech-savvy enough to run one - my mum certainly
isn't!), it would be frustrating, but not really a big problem. She'd just
just restore from a back up, change the passwords then back to business as
usual. But then you're comparing it against examples where there's
confidential information (eg user passwords) which, if stolen, is a massive
problem.

I'm assuming from the OPs tone, that his box sits somewhere between the two
extremes. However the very fact that he's got an SSH account on it makes the
box a much more favorable target than the millions of free blogs on shared
hosting services - so even if his box doesn't contain 3rd party passwords
and/or credit card information, and even if he can easily restore the box from
a back up, he still has the problem that a break in could see his box used to
send spam and what not. Many hosting providers will just kill your
subscription if that happens.

So while you're right that there's a balance between security and usability,
security most definitely comes first if you're administrating your own VPS /
dedicated hardware.

------
ExpiredLink
I wonder why password tables are readable at all. An application only needs to
call a stored procedure that compares the computed hash value against the
stored. This stored procedure needs the rights to read the data. No
application, no administrator should to be able to read that data.

~~~
Drakim
The issue is how you enforce this. Obviously if somebody has root on your
server they can just disregard any rules for how the database should be
accessed.

Maybe by having the database on external hardware with an API, but I can
imagine this is hard to scale. What if you need to sync it across several load
balancing servers, or what if you want to make backups?

~~~
ExpiredLink
> _Obviously if somebody has root on your server ..._

... then it's game over for you. But I guess intruders typically use either
SQL-injection or somehow get the application password to access the password
table. In both cases an unreadable table would thwart the intruder.

~~~
Drakim
But is it?

If the only way your production server can check passwords is by sending
hash(salt+pw+salt) and getting a true\false reply from a piece of external
hardware...then even if an intruder got complete control though some exploit
or injection, he wouldn't be able to dump the database for later cracking with
rainbow tables.

~~~
shabble
You may have just re-invented the Hardware Security Module(HSM)[1] :-)

See also [2] for a cute implementation using an Arduino to secure an AWS API
key

[1] <https://en.wikipedia.org/wiki/Hardware_security_module>

[2] [http://stefan.arentz.ca/signing-aws-requests-with-your-
ardui...](http://stefan.arentz.ca/signing-aws-requests-with-your-arduino.html)

------
hashmymustache
Are tech companies that store a fair amount of personally identifying
information on millions of people ever subject to security audits? Not saying
it would help, but it seems like as attacks get more prevalent and old
security practices become obsolete that users might be able sue companies for
negligence by not using optimal data encryption. Of course the tech company
would counter sue for stupid passwords. And even if they want to restrict
users from dictionary passwords with symbol replacements, hey, it's patented.

~~~
jaredmcateer
I think in N. America if you do credit card processing you have to conform to
PCI standards, unfortunately those standards are loosely audited and relies
mostly on self-reporting. More than a few of the security protocols are cargo
culty and even detrimental to security.

------
jgrahamc
One way they could have upgraded their database to make it more secure:
[http://blog.jgc.org/2012/06/one-way-to-fix-your-rubbish-
pass...](http://blog.jgc.org/2012/06/one-way-to-fix-your-rubbish-
password.html)

~~~
kyrra
I would vote for PBKDF2-HMAC-SHA512. The SHA512 part makes it a little harder
to do in GPU as key lengths are 64-bit vs 32-bit for sha256.

Pbkdf2 has been recommended by NIST, while the main crypto algorithm for
bcrypt hasn't seen as much attention (though it has been around for a while
and is probably safe).

Edit: Though, it looks like PBKDF2-HMAC-SHA512 is not FIPS compliant.

~~~
tptacek
So, this isn't a bad recommendation, in that PBKDF2-XXX for any XXX is _much_
better than "salted hashes", but I'm confused about this meme that recommends
PBKDF2 above bcrypt and scrypt, both of which are better, marginally in
bcrypt's case, and significantly in scrypt's.

SHA2-512 is designed to be fast on hardware.

~~~
kyrra
True true, all of them are better than just salted hashes. It is much harder
to brute-force bcrypt and scrypt in hardware (ASIC, GPUs, etc...).

It's just important to understand the differences and trust behind the
different algorithms. PBKDF2 is NIST recommended, which I would say speaks to
its strength (unlikely it will be easily cracked). It can also be used with
any input size. Though, calculation requires little memory, so it can easily
be implemented in hardware.

bcrypt has been around since 1999 has is still going strong, but is limited to
input sizes of 55-characters.

scrypt is fairly new and I don't feel has been wrong long enough to be vetted
by the security community.

~~~
tptacek
You say it's "unlikely it will be easily cracked", but, obviously, it's
demonstrably easier to crack than bcrypt and scrypt.

~~~
kyrra
SHA-1 and SHA-256 use 32-bit word sizes which makes them very easy to
implement on current GPUs. CUDA and OpenCL allow for computation on 32-bit
floats very quickly, but neither can easily do 64-bit floats right now. Which
is why I said originally that SHA-512 would be preferable right now (due to
64-bit word sizes for its keys).

I don't follow GPU tech enough to know if they plan on implementing 64-floats
any time soon, so commodity hardware to easily brute-force SHA-512 may not be
around for a while. Though, they can still implement SHA-512 calculations on a
GPU a lot easier than they can for scrypt or bcrypt.

Now, if you use PBKDF2 with a large iteration count (lets say 20000), it will
make the time to calculate the final solution a lot harder. You could give an
iteration count to PBKDF2 to have it take the same amount of time as bcrypt
would take to run. The only thing you can't do with PBKDF2 is adjust the
amount of memory required to calculate the solution.

~~~
tptacek
_Though, they can still implement SHA-512 calculations on a GPU a lot easier
than they can for scrypt or bcrypt._

------
casca
Everyone should be salting and hashing correctly - it's trivially easy to
implement and the cost is trivial. However, once the user database is stolen
in its entirety, there are other factors that become as important:

\- Did the user use a common passwords? If they did, then it will quick to
brute-force.

\- Did the user reuse the same credentials on other sites? If they did then
the value of recovering the password becomes worth plowing through well-salted
scrypt.

The real problem is that far too many people use the same credentials for
LivingSocial and their Gmail.

~~~
misnome
It was my understanding that hashing and salting is now obsolete, and doesn't
really increase security that much. AFAIK, the best 'recommended' way
currently is to use bcrypt.

~~~
UnoriginalGuy
Your understand is completely wrong.

Plus bcrypt is used as a hashing function and includes a salt. So even if what
you said was true, it would make bcrypt obsolete.

Yes, I know, bcrypt uses a block cipher but block cipher can be easily used
for cryptographic hashes (see DES for one example).

~~~
misnome
Yes, bcrypt is used as a hashing function, and includes a salt; but it isn't
what is usually meant when "Hashed and salted" are suggested - the salting
part becomes part of the algorithm used, rather than something you
specifically have to implement yourself.

From the perspective of the person using it, it's just a single hash. Saying
"Hashing and salting" works is not really any better than what people do
already, because, e.g. the passwords in the case discussed in this article
WERE hashed and salted, but with a relatively weak hash.

------
mattberg
Sorry if this has been asked, but I haven't seen anything on it yet. What
about users that only signed in via Facebook? Any actions I need to take?

------
CoachRufus87
Does anyone actually know how to close a livingsocial account? I scoured their
site but saw nothing that pointed me in the right direction.

