
Bit.ly compromised – More Details - shdon
http://blog.bitly.com/post/85260908544/more-detail
======
dewey
"If you registered, logged in or changed your password after January 8th,
2014, your password was converted to be hashed with BCrypt and HMAC using a
unique salt. Before that, it was salted MD5."

Are they saying the passwords were hashed with MD5 before January? Because
that'd be a bit scary.

~~~
KMag
It's still a bit scary that they're apparently using some ad-hoc KDF (key
derivation function) built from bcrypt and HMAC. bcrypt is a decent KDF, but
there are several ways to make a bad KDF from a combination of bcrypt and
HMAC. For instance, concatenating bcrypt(salt, pw) and HMAC-SHA512(salt, pw)
makes guessing passwords way too fast.

I'm hoping that all they've done is use bcrypt(salt, HMAC-SHA256(salt, pw)) to
get around the bcrypt 72 character limit. However, using scrypt would have
been better.

~~~
nialo
why does concatenating bcrypt(salt, pw) with HMAC(salt, pw) make guessing
fast?

Intuitively it seems like you'd still need to guess the bcrypt portion, making
it no worse than just bcrypt.

~~~
euank
Why not just ignore the bcrypt portion? "Intuitively" it seems to me like you
could just un-concatonate the two strings and guess at them separately,
working on whichever you know to be easier.

------
michaelfeathers
I've never been clear about what value-add registration brings to a link-
shortener, so I've just used bit.ly without registration.

I'm kind of happy about that now.

~~~
edraferi
it allows you to track stats more easily. for example, you can link bit.ly to
Buffer and get a unique bit mark for each link/platform pair. then you can
track engagement separately for each platform.

~~~
brownbat
Couldn't you just have an IFTTT pull anything you bitly into a bucket of links
to watch stats on? Why does bitly need to know who you are, or what links you
submitted, if you can just remember (maybe through automation) which links are
yours, and bitly or buffer can monitor each link individually for stats?

My guess is that logins probably help bitly track usage patterns, improve the
product, and keep an eye out for ways to increase their value by measuring
engagement with their userbase. Maybe it was just "everyone's adding a social
layer" peer pressure. Not sure.

There are security costs to "use passwords everywhere, for everything," but
they are more diffuse. We've ended up with a system where security best
practices would recommend using hundreds of distinct passwords that are each
too complex to memorize (so you can dump them in a locker or support one login
services, each creating a single point of failure, or you can use a similar
password or mental password algorithm for each site, it's all tradeoffs).

I don't think there's an obviously better solution, I think it's just a
tragedy of the commons we were probably inevitably going to trip over sooner
or later.

~~~
peteforde
What percentage of the internet would even understand what you just suggested,
much less know how to do it?

Also, you'd have to see a massive amount of value in not signing up to Bitly
to offset the hours you'd spend creating a vastly inferior hack job on your
own.

~~~
brownbat
> "[Your ideas are] vastly inferior hack jobs, 0/10 stars..." [ very roughly
> :) ]

All I'm claiming is that bitly could have offered stats without a login
system. IFTTT was just a proof of that concept, a gratuitous example, not some
idealized form of stat tracking that everyone should run out and implement.

You're probably right that the solution I thought of in around ten seconds
sucks. I've spent more time on worse ideas. My real point is that login
systems, despite their ubiquity, have their own share of downsides. Some of
these downsides should be painfully evident today given the article.

------
ForHackernews
Link-shorteners are a cancer. We have enough of a problem with historical
links going dead, without interposing a third party.

When bit.ly goes down, what happens to all their links?

~~~
barsonme
True, but for example if I'm posting an article I wrote it's much more
efficient to save space using my erclgrgn.us short link because it's usually
~20 chars which is less than half of what the full URL is.

Unfortunately, sometimes brevity is a necessity.

edit: original comment mentioned twitter, then I remembered twitter
automatically reduces a URL's chars to 20 or so.

~~~
ForHackernews
Yeah, or websites could just allow actual links. I don't see why hackernews
(and twitter, etc.) can't just render <a
href="[http://somereallylongdomain.com/stupidlylongpath/?what=thisi...](http://somereallylongdomain.com/stupidlylongpath/?what=thisisgettingridiculous">I'm)
short!</a>

~~~
maxerickson
Because of griefer trolls. Forcing links to show the destination is basically
0 effort on the system side and pretty much completely undermines them.

(that only answers the HN part, I'm guessing that is much of the reasoning, I
don't know it for a fact)

------
druiid
Chef supports encrypted data-buckets where you decrypt only on the server.
Puppet supports hiera data with encryption as well (via plugins). There's no
reason anyone should be storing password information in source, especially API
keys which I'm guessing this was (S3 information more than likely?). I think
about the extent of private information I'd be okay with storing in source is
DB user/pass information because that will generally require compromising the
machine more generally first and by that point they already had access to your
DB even if the password/username wasn't stored in source.

Essentially this will serve as a wake-up call to the Bitly guys and hopefully
they'll take some steps to deploy this stuff via CM or similar in the future.

~~~
thefreeman
I read it a bit differently. I don't think they store their credentials
directly in their source code repository. I think they are saying they have a
separate repository specifically for credentials.

It seems like a bit of a strange way to manage sensitive information, but not
really any worse then a lot of other things I've seen.

------
tedd4u
For security stuff, if you can't find a well-tested implementation to re-use,
I always suggest referring to the OWASP Cheat Series [1] - there's a nice one
just for password storage [2].

[1]
[https://www.owasp.org/index.php/OWASP_Cheat_Sheet_Series](https://www.owasp.org/index.php/OWASP_Cheat_Sheet_Series)
[2]
[https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet](https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet)

------
kator
Is the root issue here that they stored remote server database passwords in
their source code?

I'm always hesitant to have source code with hard coded passwords, I know at
some point you have to have them "somewhere" but anything you can do to reduce
attack vectors for them is helpful.

What do people think should be best practices for storing sensitive
configuration data? I know some people will have just a config repo with very
limited access but that still seems dangerous at scale when that ends up being
a lot of people etc.

~~~
nknighthb
I have yet to find any solution to this problem that I like. A separate config
repo is least bad of the various bad options. The others have all been error-
prone and failed to play nicely with version control.

If someone could make git store all on-disk data in encrypted form with an
ssh-agent-like solution for on-the-fly (de|en)cryption, that would seem ideal.

~~~
jarrett
> If someone could make git store all on-disk data in encrypted form with an
> ssh-agent-like solution for on-the-fly (de|en)cryption, that would seem
> ideal.

You could probably roll your own. A decent middleground would be to keep the
data in the central repo encrypted, but keep it decrypted on the engineers'
workstations. Though a better approach is just to not check in sensitive
values at all.

~~~
nknighthb
"Roll your own" is exactly what causes trouble. It's never right and always
painful. We need a general, widely-vetted solution that doesn't make me want
to yank my hair out.

"Don't check in sensitive values" is not an option. Credentials must be kept
track of and maintained somewhere. Storing them in an ad-hoc manner without
version control doesn't just invite trouble, it demands it.

We have password managers for desktops and mobile devices. We need an
equivalent for servers that accounts for the fact that the credentials must be
highly available, multiple people must be able to use them (whether or not
they get to directly _view_ them), some of them will be technicians with
limited training following someone else's procedures, and regardless of
knowledge and skill, there will be half-asleep people taking emergency actions
at 3AM when shit has hit the fan.

------
theboss
"Security Team"?.

I'm honestly curious. I've never seen startups that aren't security startups
hiring a security team. Is this a real team or just their developers doing an
extra job?

~~~
bullseye
I don't fault them for it if they don't. "The Bitly security team learned of
the potential compromise..." sounds much better than "our programmer, Joe, got
an email..."

~~~
addandsubtract
Good guy Joe. Developer by day, security team by night.

------
nknighthb
Missing step: Asymmetric encryption of off-site backups with passphrase-
protected private keys kept offline.

