
Slack was hacked - trustfundbaby
http://slackhq.com/post/114696167740/march-2015-security-incident-and-launch-of-2fa
======
FiloSottile
I hate to be the negative guy, and they were hashing passwords better than 90%
of the sites, but it would be SO easy to completely neutralize password
leakage when the attacker only has access to the database.

[https://blog.filippo.io/salt-and-pepper/](https://blog.filippo.io/salt-and-
pepper/)

tl;dr: Hardcode a second salt in your application code or in an environment
variable. Then a database dump is not enough anymore to do any kind of
bruteforce.

It's simple, free and you can retroactively apply it.

EDIT: I addressed some of the points raised in this thread here
[https://blog.filippo.io/salt-and-
pepper/#editedtoaddanoteonr...](https://blog.filippo.io/salt-and-
pepper/#editedtoaddanoteonrotation)

~~~
randunel
I generally disagree with hardcoded salts, you should assume everything is
compromised in a successful attack. But I'm actually commenting here because I
don't see how you can retroactively apply the second salt to a hashed string.
Could you please elaborate or share a link?

Later edit: I'm referring to your example in your link:

    
    
        salt = urandom(16)
        pepper = "oFMLjbFr2Bb3XR)aKKst@kBF}tHD9q"  # or,
        getenv('PEPPER')
        hashed_password = scrypt(password, salt + pepper)
        store(hashed_password, salt)
    

How do you retroactively apply this?

~~~
FiloSottile
In incident response you assume the worst, but in system design you try to
minimize impact of common attacks like SQL injection.

There's nothing wrong with nesting algorithms (see the Facebook hash onion),
so you can use the following scheme:

    
    
        bcrypt(bcrypt(password, salt), pepper)
    

And do a pass on all your database entries like

    
    
        bcrypt(old_hash, pepper)

~~~
randunel
In your website's example, you have

    
    
        bcrypt(password, salt+pepper)
    

Observe the difference between that and the rehash you just posted

    
    
        bcrypt(bcrypt(password, salt), pepper)

~~~
mbrubeck
> bcrypt(password, salt+pepper)

I hope it's obvious that no one should never do this, since the output would
contain the "salt+pepper" bits in cleartext alongside the hash, defeating the
entire point of the "pepper":

[https://www.usenix.org/legacy/event/usenix99/provos/provos_h...](https://www.usenix.org/legacy/event/usenix99/provos/provos_html/node5.html)

In fact, this is a perfect illustration of why it's bad to put secret bits
into a crypto function in a place that's not designed to take secret bits.
Bcrypt does _not_ treat the salt parameter as a cryptographic secret, and
other algorithms might not either. And they might leak it in more subtle ways.

~~~
rmrfrmrf
One would think it's obvious, yet this is what the person in the linked
article suggests doing :/

------
bitsweet
> No financial or payment information was accessed or compromised in this
> attack.

This wouldn't be my first concern. It would be all of the confidential
communication that happens within slack.

~~~
fillskills
My concern are the usernames, emails and phone numbers that were probably not
encrypted

~~~
ipedrazas
Exactly!!!

Encrypting user data should be a common practice like hashing passwords.

~~~
scrollaway
Third party authentication should be the norm. Leaving authentication to
providers that _absolutely know their shit_ , just like we leave _payments_ to
third party services.

Of course, that requires a decent protocol, and Mozilla is doing the world a
disservice in not marketing Persona better seeing as it's the right
solution....

~~~
mike_hearn
You mean like how Authy specialised in two-factor authentication, but still
managed to have basic string concatenation bugs that rendered their entire 2FA
system bypassable?

~~~
yellowapple
Huh? This is the first I've heard about this, and searching for "Authy
concatenation bug" isn't turning up anything useful.

~~~
zaroth
Here's the write-up from Homokov. The guy is a pen-testing genius:
[http://sakurity.com/blog/2015/03/15/authy_bypass.html](http://sakurity.com/blog/2015/03/15/authy_bypass.html)

But if you just want the money shot:
[http://sakurity.com/img/smsauthy.png](http://sakurity.com/img/smsauthy.png)

Yes. Typing '../sms' in the field bypassed the 2nd factor. Just, wow.

~~~
yellowapple
Huh. Well now I know. Thanks!

Amazing what you can do with improperly-implemented input sanitation :)

This probably could've been prevented by disallowing non-number inputs, no?

~~~
zaroth
"In fact the root of the problem was default Sinatra dependency 'rack-
protection'".

They _were_ doing the input sanitation, but it wasn't the very first thing in
the processing pipeline, since "best practice" was to pipe everything through
'rack-protection' first.

Homokov was first to state, this was really a black-swan type bug which 99.9%
of the time makes it into production. Apparently, they were doing the "right
thing" and still got burned.

~~~
homakov
The parent meant "This probably could've been prevented by disallowing non-
number inputs" in SDK libraries. Yes, if SDK would cast everything to digits
it wouldn't be possible. It is also quite obvious security-in-depth for a 2FA
API. Now they do it.

*HomAkov

~~~
yellowapple
Or even just input validation on the form itself before passing on to the API,
which is more of what I was getting at. I don't know about the details of
Authy's setup, but I know that AJAX (for example) supports enforcement of
specific value types in text fields.

Basically, the form itself could have (and maybe even should have) required
numeric-only values, seeing as Authy's codes are either 6 or 7 digits long and
contain no alphabetical or special characters.

------
elchief
Assuming (no evidence, it's just very common) that this was a SQL Injection,
here are some ways to protect yourself:

* Use [http://en.wikipedia.org/wiki/Database_activity_monitoring](http://en.wikipedia.org/wiki/Database_activity_monitoring). If you don't list users on your site and you get a query that would return more than one user record, it's a hacker

* Add some [http://en.wikipedia.org/wiki/Honeytoken](http://en.wikipedia.org/wiki/Honeytoken) s to your user table, and sound the alarm if they leave your db

* Use Row-Level Security

* Database server runs on own box in own network zone

* Send logs via write-only account to machine in different network zone. Monitor logs automatically, and have alerts.

* Pepper your passwords (HMAC them with a key in an HSM on the web server (then bcrypt). Don't store key in db). [https://blog.mozilla.org/webdev/2012/06/08/lets-talk-about-p...](https://blog.mozilla.org/webdev/2012/06/08/lets-talk-about-password-storage/)

* Use a WAF that looks for SQL injections

* [Use real database authentication, per user. Not one username for everyone connecting to db. Yes, this is bad for connection pooling]

~~~
azinman2
Why would you assume that? There are plenty of ways to hack into stuff without
sql injections.

~~~
elchief
It's the most common vulnerability.
[https://www.owasp.org/index.php/Top_10_2013-A1-Injection](https://www.owasp.org/index.php/Top_10_2013-A1-Injection)

~~~
scrollaway
It's the most common vulnerability _on the web_. It's certainly not the most
common vulnerability in projects built under popular non-php frameworks. Under
that model, it's harder to create a situation where a SQL injection is
_possible_ than not.

Edit: Slack's in PHP, I thought it was in RoR for some reason. Oops.

~~~
alexhill
Slack is a web service written in PHP, so I'd say elchief's assumption is
reasonable.

------
_trb_
I am an application security professional, and I created this account in order
to make this post after reading many of the comments on this thread.

Many of the comments have great suggestions. However, very few talk about the
most important part of creating mitigations and designing key
management/crypto. What is the security target?

Before throwing new designs at a problem, the attackers and attack vectors
must be defined. If you don't know who you are guarding against and what they
will do (and what data they will steal), then how can you possibly say what is
a good mitigation??

One might argue that the threat is obvious, but I'll guarantee you that there
are dozens of threats here. List them. Prioritize them. Then mitigate them. It
is helpful to fully understand the problem/solution space before jumping in
with pepper's, salt's, extra databases, and solutions.

------
rudolf0
It's refreshing to 1) see a breach notification including the actual password
hashing algorithm, 2) see they're using a strong one like bcrypt (presumably
with a reasonable cost factor).

Regardless, this is an example of why cloud communication (and ticketing and
database off-loading [see MongoHQ] and...) systems probably won't ever become
commonplace in most of the government space and the finance and health
sectors.

~~~
jedberg
I think this just goes to show exactly why these systems will become more
commonplace. There are only so many security experts to go around. Having all
the very best concentrated on a smaller set of services seems like it makes
more sense than trying to get a security expert for every service.

~~~
sbov
I have a friend who works security, previously for the government, and now for
Visa. In his opinion: if you haven't been hacked, you just aren't an
interesting enough target for the right people.

I don't know how common this line of thought is in security. But if it is
common, then if you're a small company, aren't you better off not hosting my
stuff at these large companies, because it's putting information you collect
with someone who is more likely to be "interesting" to the right people?

~~~
sounds
I think a more correct assessment would be:

If you think you haven't been hacked, you probably have (or you are so small
that you may have only been probed by bots).

If you haven't actually been hacked yet, it is only a matter of time. Ideally
you start designing security layers now, before you are compromised.

~~~
freehunter
As the famous quote goes: there's two types of companies - those who have been
hacked and those who will be. Actually, there is a third kind: those that will
back hacked again.

------
rattray
How does one discover that they were hacked? The post states that the breach
occurred during February, and this is the end of March... did it just take
them a long time to react and write a post about it, or did they likely
discover after the fact? If so, how?

~~~
rectang
True story:

Log into server. Why is server slow? Run `top`. Hmm, `./exploit` is consuming
99% CPU...

~~~
atmosx
./exploit ...? Seriously??? LOL

------
nvk
Host your own IRC if you care about the privacy and security of your
communication.

There is no reason why you can't take 10min to setup a IRC with SSL on your
own.

Yes, Slack is awesome, lots of features, but it's not yours!

~~~
programmarchy
Agree with you in one sense of being responsible for your own security, but by
this logic I should keep all my money under the mattress instead of the bank,
no?

~~~
enraged_camel
Not the same thing. Banks are insured against robbery and theft, so if
something like that happens, customers don't lose their money. In addition,
there's an insane amount of fraud protection in the banking industry, and
billions of dollars of vested interests to make sure criminals are caught and
prosecuted.

Can you say the same about cloud services?

~~~
programmarchy
Good point. Even in the worst case scenario, banks can fail and get bailed out
by the public. But once information leaks, there's no "bail out" remedy
possible... you can't really put the "information" genie back in the bottle.

------
mirashii
Surprisingly, they didn't force a password reset on all accounts. Even though
the passwords are hashed and salted, targeting a couple users and checking for
weak passwords can now be done offline, with no rate-limiting or network calls
necessary. In breaches like these, it should still be mandatory to issue
service wide password resets. Anything less is unacceptable.

~~~
kannonboy
I was curious about this too. I don't use Slack, do they enforce a password
complexity and use a bcrypt work factor high enough to justify not requiring a
reset?

~~~
mirashii
There isn't a work factor high enough or password complexity high enough that
it doesn't justify a reset, in my opinion. People will use bad passwords, and
we can't really stop that. Allowing offline attacks against them will never be
okay.

~~~
tempestn
I'm definitely not an expert, but my understanding is that a per-user salt,
which Slack was using, protects against brute-forcing bad passwords. So
assuming a reasonable work factor, it really should be computationally
unfeasible to retrieve these passwords. That said, I'll change mine, because
why not.

~~~
mirashii
A per-user salt doesn't prevent brute forcing bad passwords. The purpose of a
per-user salt is to prevent brute forcing all of the passwords simultaneously.
Without it, you could just hash a password once and check against all the
passwords. With the salt, you have to hash each guess for each user. It
certainly increases the work, but with bad passwords, its likely you'll always
crack a few in a large dump.

If you have a strong password, a high work factor should prevent brute-forcing
it. But really, if it's computationally infeasible to crack a bad password,
its also probably computationally infeasible to log in.

In cases like these, even if the risk of password recovery is minimal, a reset
should still be forced because the attacks are offline and only become easier
as time goes on. Forcing a rotation discourages people from continuing to work
on cracking passwords for the next n years until they finally get an
interesting account.

~~~
tempestn
Of course, thanks. Guess I wrote that without actually thinking about it. Now,
a _pepper_ would prevent brute forcing weak passwords, assuming it wasn't
compromised, but that's not relevant here. And regardless I agree with your
logic regarding forcing a reset.

------
chatmasta
If I were Slack, I would pretend to get hacked. Slack critics often point to
its centralized architecture as a weak point, because rational corporations
should not entrust security of their internal communications to a third party.
Particuarly when that third party aggregates communications of its many
clients, it becomes a target of hacking. Why hack a single corporation when
you can hack Slack and get all their clients at once?

This is a valid criticism. Slack can do all it can to mitigate security risk.
But at the end of the day, there is always at least one vulnerability,
somewhere.

As Slack matures as a company, it needs an answer to this criticism. Because
security is so naturally unpredictable, it would be disingenuous for Slack to
respond with anything resembling "our security is perfect." Because, of
course, as we see time and time again, no security is perfect.

Now that Slack has captured the low-hanging-fruit of the market, it needs to
pick the _high_ -hanging-fruit. The most profitable clients for slack will be
the largest, conservative, enterprise clients who will join the Slack platform
and then never leave. The long term survivability prospects of Slack depend on
capturing these large enterprise customers.

Strategically, Slack needs to find a response to the criticism that its
security is prohibitively weak, so that it can convince these large
enterprises to join its platform.

Perhaps, the best response to security criticism is that "we got hacked, but
our internal policies mitigated any cascading effects and customer data
remains safe." [0] [1] So would it be in Slack's best interest to stage a hack
on itself? Or to report a hack occurred when it really didn't?

It seems feasible that by setting precedent for its reaction to a hack, Slack
has a chance to demonstrate the competence of its security team. Now investors
can point to this incident as one handled well by the security team. In a
world where, unfortunately, corporations will always get hacked, Slack was
able to survive with some dignity.

[0] or, as safe as it can possibly be according to computer science.

[1] debatable.

~~~
rubyn00bie
Note: My comments and thoughts are geared specifically to Slack or a very
similar entity (tech start up, which sells a service).

I strongly disagree with this idea:

"Now that Slack has captured the low-hanging-fruit of the market, it needs to
pick the high-hanging-fruit. The most profitable clients for slack will be the
largest, conservative, enterprise clients who will join the Slack platform and
then never leave. The long term survivability prospects of Slack depend on
capturing these large enterprise customers."

I think this is a large misconception held by people. Often times, and I say
this from experience, enterprise customers will demand a large amount of
support/coddling because of the gross amount they're paying.

That "$1 million" a year contract sounds less and less good when you realize
it requires constant attention of two, now dedicated, engineers, two support
reps, sales person, and on occasion an executive... Compare that to 1,000
individual customers/subscribers who require next to no special attention, or
dedicated engineers. Large, conservative, enterprise customers distort profit
margins, and should be avoided.

Think about this: some enterprise customers are so large they may hire a
person (or persons) to _only_ deal with your product. Now you've got a team of
cogs (people) at your enterprise client, working, to take up your time and
cost you money.

What's worse is that in most corporate cultures, partners/suppliers/providers
are not looked at as "family" but as disposable. There's no vested interest in
your success or failure. So there is little to no concern over how your
smaller company may be abused by a larger one.

~~~
chatmasta
This is why humans invented contracts. No two businesses enter into a $1
million deal together without a contract to describe the expectations of the
partnership. For this reason, no service provider should be "surprised" by the
level of commitment required to fulfill its end of an enterprise deal, because
all the expectations should be in the original contract (literally, the
service level agreement). There are no surprises.

Also, (pedantic) in your example, $1m a year is beyond sufficient to cover the
costs of two engineers and some support staff. Besides, realisically the
biggest support issues will be problems with availability, which presumably
will be network wide and not limited to one client. I highly doubt slack needs
to hire dedicated engineers for each new enterprise client. Instead they can
reallocate existing engineers when needed, and grow their total labor capacity
as it becomes constrained.

------
matdrewin
So a database gets hacked, they add MFA and people are arguing about peppering
passwords. What about the part on how the hackers got access to the database
in the first place?

Passwords are not the only sensitive info that can be stored in a database and
most of the time, that info isn't hashed.

------
zuck9
I would be more interested in how the hacker got access to their DB and
nothing else. Maybe the DB is remotely accessible (unlikely) or there is SQLi
vuln. in Slack.

------
kstop
One thing that bugged me about this today was that after I changed my password
on desktop, my mobile session wasn't invalidated. Apparently it's an option
for mass password resets, but it really should be mandatory.

------
racontour
My team logs with our Google accounts. It's not addressed in the disclosure,
but should we be deauthorising Slack. Have our tokens been breached?

~~~
jturolla
Possibly no because the token is app-specific and it's not likely that the
slack key was compromised, but I also agree that they should have disclosed
that.

------
eranation
> Slack’s hashing function is bcrypt with a randomly generated salt per-
> password which makes it computationally infeasible that your password could
> be recreated from the hashed form.

I'm happy to hear they didn't just use MD5 with no salt as this would be the
same as storing it in plane text...

bcrypt + random salt sounds to me like the best practice nowadays, is it still
holding? or are there some advanced in GPU cluster costs on EC2 that make even
bcrypt hackable. I think I heard something that it has a way to "adapt" to
advances in computing, is that by simply adding more iterations based on the
current CPU speed or something? how does that work?

~~~
sdevlin
There are some thoughts on the matter here:
[http://chargen.matasano.com/chargen/2015/3/26/enough-with-
th...](http://chargen.matasano.com/chargen/2015/3/26/enough-with-the-salts-
updates-on-secure-password-schemes.html).

But I would say yes, bcrypt is still best practice. Other commenters are right
that bad passwords will still be recoverable, but using one of
bcrypt/scrypt/PBKDF2 is due diligence. I would use whichever one is most
easily available on your platform.

~~~
ntucker
Why do articles talking about this always talk about specific work factors and
choosing a correct work factor, as if it's fixed? I thought good practice was
to choose the work factor dynamically so it's calibrated to whatever hardware
you're running today.

~~~
jjarmoc
Uhh.. it doesn't? While it does give some specific values as 'a reasonable
starting point' the article suggests tuning these to the specific environment.

"Each of these algorithms requires setting of an appropriate work factor. This
should be tuned to make digest computation as difficult as possible while
still performing responsively enough for the application. For concurrent user
logins, you may need <0.5ms response times on whatever hardware is performing
the validation. For something like single user disk or file encryption, a few
seconds might be fine. As a result, the specific work factor values
appropriate for your application are dependent on the hardware running your
application and the use case of the application itself. Thomas Pornin gave
some great guidance on determining these values in a post to stackexchange"

The stackexchange post (which is linked in the article) can be found at
[http://security.stackexchange.com/questions/3959/recommended...](http://security.stackexchange.com/questions/3959/recommended-
of-iterations-when-using-pkbdf2-sha256/3993#3993)

~~~
ntucker
Yeah, what I'm getting at is that those posts don't actually come right out
and say your code should be calibrating itself regularly. They talk about
selecting a work factor by benchmarking your current hardware, but they leave
it there, and one might come to the conclusion that once they've measured
their hardware, they put "13" in a config file and call it done. I'm
advocating for advice like "Don't think about work factors, think about time.
Your configuration should be a _time_ value, and when your app starts up, it
should compute its own work factor based on this time value." If your code
does this, there's no need for rules of thumb like "11 is a good starting
point for a bcrypt work factor."

~~~
saurik
This doesnt make any sense: the goal of this work factor is to make things
slower for the _opponent_ , not my own servers. My estimation of the
computational power available to my opponent has nothing to do with how large
the web servers I chose to use happen to be.

~~~
ntucker
The recommendation is to make it as slow as you can tolerate, which _is_ based
on how fast your web servers are.

------
Klinky
Was an admin account compromised in a situation where 2FA could have prevented
the unauthorized access? If that's not what happened, then 2FA seems a bit
hand wavy if it's not directly related to this security incident.

~~~
BrainInAJar
I actually am coming to the conclusion that 2FA is dangerous. It's recommended
any time there's any kind of breach as if it were some kind of panacea for
security, when in most cases the cause of a security breach is a database
intrusion.

2FA keys are stored unencrypted so if the db is breached they're already
cleartext (no boil the oceans bcrypt collision needed) and the extra layer of
security theatre from 2FA only encourages bad practices like password reuse &
easy passwords.

~~~
jest3r-
2FA can prevent future password phishing attempts when your users table is
compromised.

It's a bit of a redirect in terms of what happened with Slack though and not
worth mentioning after the fact.

------
rdl
I wonder how many people send sensitive credentials or other operational
details through Slack. It'd definitely be a target (along with mail systems)
if you want to attack better-protected customer systems.

~~~
sergiotapia
I think that's the point of slack, being able to communicate sensitive
information. Where would you relay something like an Amazon AWS Access Key?

~~~
aianus
PGP encrypted email.

------
colordrops
We've been waiting for over a year for Slack to create a self-hosted version
that we can deploy to our intranet specifically because we can't expose
ourself to things like this. They've kept insisting that it's around the
corner but it doesn't seem to be happening. Hopefully this will spur them to
prioritize self-hosted Slack.

------
ocdtrekkie
Literally was arguing with someone like two days ago that using Slack for
sensitive data was a bad idea, guaranteed to blow up in your face sooner or
later.

Nothing sweeter than "I told you so".

~~~
eswat
We haven’t seen anything blow up yet other than Slack itself. While it sucks
that the user table was compromised, I think the actual affect on businesses
is more benign than they’d like to believe (ie: sensitive chatlogs that could
be used for blackmailing).

------
larsiusprime
Slack encourages 2-factor authentication:

> Download and install either the Google Authenticator or Duo Mobile apps on
> your phone or tablet.

Hey Slack, I don't have a smartphone. What am I supposed to do?

~~~
larsiusprime
While I appreciate all the downvotes and "get with the times" comments, a
significant portion of the population does not have a smartphone, assuming
everyone has a smartphone or will instantly know what to do when presented
with official instructions that only mention smartphones/tablets seems like a
bit of a security oversight on Slack's part, no?

Those pointing out PC-enabled authentication apps: thanks. That's USEFUL
feedback!

~~~
drcoopster
The vast majority of the population that comprises Slack customers will have a
smartphone.

~~~
larsiusprime
And only focusing on the "vast majority" is a GREAT security solution. There
is some significant, non-zero number of people who use Slack and do not own a
smart device. I am one of them. (reasons for choosing not to own a smart
device are many and I will not go into them here)

I'm not trying to rake them over the coals, just point out this is a very real
blind-spot and they should promptly update the notice with instructions for
people who don't have smart-phones.

~~~
geofft
Can you expand just enough on the reasons for not owning a smartphone, so we
can give constructive advice? I don't want to suggest things like "buy a
Yubikey" or "buy a tablet or iPod Touch with no cell radio" if you're
objecting to buying new things, for instance.

I think it's hard to give generic advice about this situation.

------
kolev
Can we go back to IRC now, please! Slack is not only distracting, proprietary,
but it is also pretty expensive. Let the mere mortals use it, but we should
stay away!

~~~
matthewmacleod
I don't think IRC quite offers the feature set that Slack users would be
expecting, does it?

~~~
kolev
I'm not talking about users who care about custom Emojis and animated gifs.
That's why I said - let the mere mortals use it; we can do better!

~~~
raphael_l
Is this actually the sort of people that get attracted to HN? Thinking of
others as "mere mortals" and yourselves as gods?

Sad picture.

~~~
tomjen3
Probably not in that condescending way, but it is useful to keep in mind that
people are fundamentally different from you. The trouble is when you think
they are inferior beings (their tech skills on the other hand are inferior, no
reason to not state that plainly).

For what it is worth I have coded Erlang at work and am quite partial to cat
pictures, myself.

~~~
kolev
Well, let's talk about the reality now. By definition, most people are
mediocre, not very smart, not very healthy, not very pretty, etc. Most! That's
whom Slack targets - most.

------
globaloptima
Out of interest, where where the per-user salts stored I wonder? Where would
people normally store this if not next to the hashed password in the same
table?

~~~
drinchev
Yeah it's Usually the same table, but salt is used to prevent hackers using
already generated hash map of popular passwords to diff with your password
hash directly without computing.

~~~
why-el
I am slightly confused. How can they have use their "hash map of popular
passwords" to diff against your table if they do not have the "pepper" used
originally for bcrypting the passwords?

~~~
swhipple
Pre-computed mappings of popular passwords (rainbow tables) aren't really used
nowadays due to parallelization being more cost effective, but the idea in
both cases is that you want collisions with known values: the popular password
hashes in the case of the rainbow table, or the computed values in parallel
enumeration.

If the password hashes each have a stored unique salt (bcrypt will), you have
to compute the hash per salt per password that you test. Instead of computing
the hash for "password" (+ stored work factor for bcrypt) to be X, and
checking all database entries for X, they instead have to calculate each hash
per entry.

For a table of 1000 users, it would take around 1000 times as much work to
determine the users with "password" as their password. If you just wanted to
target a single user though, the salt doesn't really matter for enumeration
(though if you were using rainbow tables, you likely wouldn't have those
specific hashes computed).

------
3princip
Lot's of hype (IMO) around Slack, but lot's of money thrown at them so I kept
thinking that I'm missing something! Just being skeptical as usual. The other
day an invitation arrives to use Slack. Great! Let's see it, this killer
feature or killer combination of features. What have these smart people come
up with that hasn't been done countless times in the same space to make them
so successful?

It's literally nothing. I can't believe that's the product.

Anyway, on top of a completely underwhelming experience comes this news. I
can't see why a company would use them, to be honest. But then I haven't built
a billion dollar company, so not many people will be asking me for an opinion.

~~~
tracker1
You could do something similar with an IRC server and bots with webhooks...
Just the same, they did do this, and offer a web based interface that people
are more comfortable with (no need for an irc client), and does more than IRC
clients do.

IT's not that anyone else couldn't do this, it's that they've done it
relatively well.

~~~
chralieboy
Absolutely. It's just so much better than any IRC client. People looked at the
iPhone and said "but all of this technology already existed!" That isn't the
point. It had never been put together in a user-friendly way.

Slack hasn't invented anything new, but the entire product is a delight to
use.

------
glesica
"As part of our investigation we detected suspicious activity affecting a very
small number of Slack accounts. We have notified the individual users and team
owners who we believe were impacted and are sharing details with their
security teams."

Assuming the password hashes can't be reasonably reversed, what would have
caused suspicious activity on some user accounts? Is this a situation where
certain users may have been targeted specifically, meaning that only a couple
hashes needed to be reversed, making the task feasible?

------
Zolmeister0
Dear All - Your passwords should be considered compromised. Hashing is merely
a deterrent, it does not prevent cracking.

~~~
elchief
6500 bcrypt(5) hashes/second with custom FPGA:

[http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=703252...](http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7032529&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7032529)

~~~
Zolmeister0
I didn't say it would be easy.

But dictionary and password list-based attacks are expected to be quite
effective anyways.

[http://www.openwall.com/presentations/Passwords13-Energy-
Eff...](http://www.openwall.com/presentations/Passwords13-Energy-Efficient-
Cracking/)

------
sergiotapia
So they didn't get the password, ok. But they still have my name, my email, my
skype.

This completely sucks.

~~~
balabaster
Yeah, when they call, can you tell them I have a whole flock of ducks that
need cleaning? :P

------
opragel
Hope the Slack team will properly implement SSO two-factor authentication
policy - right now none of the Slack apps request re-authentication after
thirty days once an account is signed into via SSO. Sign in once on OS X,
Windows, iOS, Android and no further authentication needed. Looking at Google
Apps SSO specifically but am assuming it affects other authentication
providers.

Might as well plug my tiny Slack auto-install/update script for the OS X app
while I'm here - hope someone else finds it helpful:

[https://github.com/opragel/install_latest_slack_osx_app](https://github.com/opragel/install_latest_slack_osx_app)

------
balabaster
Can anyone from Slack elaborate as to where this hack originated if known?

------
martin_
Slack has now sent me 6 e-mails about this, to the same address :(

They have different names associated in each one (i.e. some have my last name,
some have an alias, etc), but all to the same target e-mail address.

~~~
vram22
See my other comment on the same point in this thread.

~~~
martin_
Also see that mine predates yours by an hour :)

------
mead5432
Can we please reference to this event as the "Shlacking"?

------
vsync
Interesting that the title says "March 2015 Security Incident" but it turns
out it was in February. Also interesting they don't say _which_ days in
February.

------
meritt
Why is everyone worried about what hashing algorithm was used and not if their
company's private chat logs are about to become public knowledge?

~~~
amckenna
Because obtaining a hash of a password does not mean the attacker can access
the accounts and the chat logs.

If the passwords were not hashed or hashed in a weak manner then that means
the attacker could figure out the account passwords, then gain access to the
accounts, which would be bad.

~~~
meritt
I completely understand the risks associated with user passwords and hashing
algorithms. But you know what? That shit gets leaked all the time. It's not
uncommon and unfortunately, users are accustomed to it by now. It's relatively
easy to fix.

You know what's a big deal? Having your company's internal private
communications suddenly public or sold to the highest bidder. That's the sort
of thing that doesn't have a "reset password" simple fix.

------
homakov
If you have a reset_token you still can log in, even when passwords are
hashed. Likely reset tokens were stored in the same table

------
eridius
> Slack’s hashing function is bcrypt with a randomly generated salt per-
> password which makes it computationally infeasible that your password could
> be recreated from the hashed form.

Is this true even when the attacker is specifically focusing on a single
account, or is it only computationally infeasible to recover passwords for
accounts in general?

~~~
NobleLie
Coincidentally, I was just looking into your question; this should answer your
concern.

"Since every user has their own unique random salt, two users who happen to
have the same password get different salted hashes. [If] the dictionary attack
is foiled, the attacker cannot compute the hashes of every word in a
dictionary once and then check every hash in the table for matches anymore.
Rather, the attacker is going to have to re-hash the entire dictionary anew
for every salt. A determined attacker who has compromised the server will have
to mount an entire new dictionary attack against every user's salted hash,
rather than being able to quickly scan the list for known hashes." [0]

[0]: [http://www.developerfusion.com/article/4679/you-want-salt-
wi...](http://www.developerfusion.com/article/4679/you-want-salt-with-that/3/)

~~~
eridius
Yeah, salts mean you can't use rainbow tables. But you can still attack a
single user. The question is what "computationally infeasible" actually means.
How much computing power would it take to crack a single user's password? How
about if it's a weak one? A strong one?

If the answer is "it would take $1000 worth of Amazon EC2 computing to crack a
single weak password", well, that's certainly feasible to do if you have a
specific target in mind.

~~~
Xylakant
The answer depends on the tuning parameters used for the bcrypt hashing
(cost/work factor) and the length of the user password. Obviously a
1-character password will fall even with a high workfactor and a dictionary
word will probably fall as well. This paper
[http://www.emsec.rub.de/media/crypto/veroeffentlichungen/201...](http://www.emsec.rub.de/media/crypto/veroeffentlichungen/2014/10/22/reconfig14_bcrypt.pdf)
tags some numbers on breaking passwords with a low cost factor. Assuming a
cost factor if 5 (12 would be more "real" world) and an 8 character password
with an alphabet of 62 chars (uppercase, lowecase, 10 digits) the estimated
cost to break a password within a month is in the millions with dedicated
hardware designed for brcypt breaking (Fig 5). With a work factor of 12, an 8
character password will not break on EC2.

~~~
eridius
I knew bcrypt was tunable, and I guess I was hoping someone from Slack would
actually pop up and say how their bcrypt was tuned (which I guess means what
the cost factor is). But you do have some good info, I wasn't aware of what
expected cost factors were and how secure a single password would be at those
cost factors.

I'm assuming that your "8 character password" is assuming a randomly-generated
password. I'm curious what the expected cost would be of breaking a
particularly weak one (e.g. a human-generated password, based on dictionary
words although perhaps with mnemonic devices or letter/number substitutions).
The paper you linked says their hardware computed 6511 passwords per second at
a cost factor of 5, and based on the cycle costs listed, I'm thinking it does
52 passwords per second at a cost factor of 12. Assuming a particularly weak
password, I don't know how many passwords a brute-forcer would expect to have
to try before hitting the correct one.

------
hoodoof
It would be good to know exactly how this sort of thing happens so others can
try to prevent it in their own systems.

------
rcarmo
Well, I can't even reset my password. I suspect that's because my registered
e-mail address uses the "foo+bar@gmail.com" format (which I use for easier
filtering), and something on Slack doesn't like sending those (no e-mail on my
inbox, spam folder or filter matches)

------
cheshire137
Looks like they require Google Authenticator or Duo Mobile app to do two-
factor auth. I'm not interested. Why can't they be like Github and just send
me a text message? I don't want a dependency on some other company's product
to make Slack more secure.

~~~
modeless
I hate to break it to you but SMS is some other company's product. Google
Authenticator is not a cloud service; it's simply an implementation of the
TOTP RFC and there are others available if you hate Google so much that you
don't trust a purely local app written by them.

~~~
__david__
An open source one at that!

[https://github.com/google/google-
authenticator/](https://github.com/google/google-authenticator/)

------
getdavidhiggins
Remember kids, it's 2FA before the fact, not after. 2FA is not a magic bullet
though, and neither is salting. Salting makes it _expensive_, but not
impossible for pass recovery. Always aim for impossible. You want to be able
to throw away the key during an incident.

~~~
tomjen3
The only impossible system I know of would be one that uses client side
certificates, but I would be surprised if anybody those in a successful
business.

~~~
zrail
US Government systems typically use client certificates as a component of
access control in the form of the Common Access Card[1]. Not that that's a
business, or even a success from the point of view of the users, but it's one
of the largest deployments of client side certs out there.

[1]:
[http://en.m.wikipedia.org/wiki/Common_Access_Card](http://en.m.wikipedia.org/wiki/Common_Access_Card)

~~~
tomjen3
Unfortunately the only reason they can do that is that they are the
government.

------
EGreg
For those that store the user tables in the same db as everything else, what
is the big deal about protecting passwords in particular? The attacker already
has much more. If it is because of password re-use then that is what should be
prevented.

------
vram22
Got the email from them about the issue, as a Slack user. Also got 2 other
duplicate emails from them (which went into the same Gmail conversation, since
same subject), that were empty.

Seen this a few times for other services, not sure why it happens.

~~~
fxthea
They probably send a message to you for every team you're on.

~~~
vram22
Could be, thanks.

------
homakov
This is why you need to hash reset tokens too

[http://sakurity.com/blog/2015/03/27/slack_or_reset_token_has...](http://sakurity.com/blog/2015/03/27/slack_or_reset_token_hashing.html)

~~~
grey-area
In devise at least reset tokens expire, so they'd need to have been set in the
last day to be useful, which narrows that attack considerably doesn't it?

~~~
homakov
Why wait and not use it right away? If you have read access now you can
exploit now.

~~~
grey-area
Oh I see - you mean they have read access, then trigger password reset, then
use the token straight away? That does mean they'd be firing off emails which
would alert users though.

~~~
homakov
It would. They didn't do it probably because they didn't try this trick. But
they could i think.

------
zobzu
ppl gotta stop thinking encryption in a db saves you from compromise. Its
being very naive or ignorant. It only reduces your exposure to data leak very
lightly - in some circumstances, which are generally not even likely (like
make a dump and post it publicly)

Its like 'but it says AES on the box so its secure right?' and shouldn't be a
thing among developers anymore.

Obviously the database data has to be decrypted for the app to use it, and
generally, you hack the app, not the db.

------
vayarajesh
What tangible things were hacked? apart from password was the communication
history, files share etc were hacked too? We sometimes share code blocks and
zip files etc..

------
paimpozhil
If someone was able to get access to the user table I would believe it is
trivial for them to download the chatlogs/ mine other information from
database.

May be slack wants us to believe only a small part of the data is hacked , I
dont know .

We have been using Slack for many projects over last year and it helps improve
productivity

------
alfredxing
Slack _is_ growing their security team, if you want to help them improve the
security of the product: [https://slack.com/jobs/dfd75111/security-
engineer](https://slack.com/jobs/dfd75111/security-engineer)

------
scottmcdot
"So yeah, we got hacked..." \- Landing Page

------
dutchbrit
How many rounds of hashing did they use?

~~~
elchief
PHP default is 10 if that helps

------
thspimpolds
Wait a second, you got hacked 27 DAYS ago (at least), you know they got data,
and you are NOW telling people?

Dong, Dong, Dong, that is the death bell of a startup

~~~
tomjen3
Sadly the universe isn't fair.

Nobody gives a shit about security, but yeah it should be illegal to sit on
the information more than 48 hours. If you can't determine the complete and
full extend of the damage before that you need to be out of business.

------
automathematics
They added team-wide password/security stuff.... but it's paid users only.
Lame.

~~~
imc
Only owners can see it, not admins apparently.

[https://twitter.com/SlackHQ/status/581544119677374464](https://twitter.com/SlackHQ/status/581544119677374464)

------
tomjen3
Why the fuck am I reading this on hacker news and not from a notification in
slack?

~~~
jivid
> If you have not been explicitly informed by us in a separate communication
> that we detected suspicious activity involving your Slack account, we are
> very confident that there was no unauthorized access to any of your team
> data (such as messages or files)

------
benshanjiang
uc3dp

------
curiously
is there anything that slack does you can't do with skype?

I find lot of these new startups are just creative ways of reinventing the
wheel and convincing you need it to appear cool & hip....kind of like fashion
for high schoolers

~~~
minitech
Skype is really bad for text chat.

\- It is closed-source, obfuscated, and untrustworthy overall
([https://www.eff.org/secure-messaging-scorecard](https://www.eff.org/secure-
messaging-scorecard)).

\- The supported (Windows and Mac) versions have obnoxious ads, and the Linux
version is buggy and 32-bit-only.

\- Since it’s P2P-ish (which makes sense for audio and video calls), you can’t
send a message to someone who’s offline and expect them to be notified (say,
via e-mail, which Slack does). Worse, if you go offline, they won’t get it
until you’re both on at the same time!

\- Files shared on Skype don’t last. You have to use an external service.

\- There is no permanent archive; logs are client-side and easy to lose. (You
can get messages that are a few days old from peers, but there’s a limit – I
don’t know exactly where.)

~~~
zkhalique
_\- It is closed-source, obfuscated, and untrustworthy overall
([https://www.eff.org/secure-messaging-scorecard](https://www.eff.org/secure-
messaging-scorecard))._

and Streak is open source?

 _\- The supported (Windows and Mac) versions have obnoxious ads, and the
Linux version is buggy and 32-bit-only._

I have the Mac version of Skype, where are the ads?

 _\- Since it’s P2P-ish (which makes sense for audio and video calls), you
can’t send a message to someone who’s offline and expect them to be notified
(say, via e-mail, which Slack does). Worse, if you go offline, they won’t get
it until you’re both on at the same time!_

This part is true, it's pretty limited.

 _\- Files shared on Skype don’t last. You have to use an external service._

Skype is more P2P

 _\- There is no permanent archive; logs are client-side and easy to lose.
(You can get messages that are a few days old from peers, but there’s a limit
– I don’t know exactly where.)_

Skype is for encrypted P2P communication.

Why not just use IRC?

~~~
minitech
Yes, I know. What point are you trying to make here? curiously asked:

> is there anything that slack does you can't do with skype?

So I named some things. Mostly being not P2P. I also prefer IRC over Slack,
although you do need to have some system in place for staying connected all
the time or receiving logs to make it workable for everyone.

Anyway, re: your points:

> and Streak is open source?

Slack? No, but I don’t have to allow arbitrary code to run free on my computer
to use it.

> I have the Mac version of Skype, where are the ads?

Maybe they’re gone. Windows certainly still has them.

> etcetera

Sure, it’s P2P. That’s a fine excuse, but it also doesn’t make Skype _good_.

------
tn13
I dont really care. Never shared any critical information on slack any ways.

------
dmazin
Why do I have to install Google Authenticator some sort of other app for
2factor here? Why can't you send me a text like everyone else does?

EDIT: Slack responded that they do not support SMS _yet_.

~~~
dragonwriter
Because SMS is not even remotely a secure communication channel, and the point
of two factors is _proof_ of possession of two different classes of things
(that you _have_ some device and that you _know_ some secret), and using an
insecure channel to send a message weakens the proof of the _have_ half of
that, in much the same way as using a weak or published password weakens the
_know_ half.

Ideally, you want as strong as practical proofs of each half within the
constraints set by UX considerations.

~~~
BrainInAJar
> Because SMS is not even remotely a secure communication channel

Neither is 2FA, it's a key stored completely unencrypted in both your phone
and the the host's db.

~~~
dragonwriter
That's not particularly weak as proof to the person who controls the DB that
you have access to the phone, which is what the device factor of a 2FA scheme
is intended to prove, since you have to have access to either the DB or the
phone to get the key.

Conversely, SMS's weakness is in the communication channel, which can be
compromised without compromising the things for which the factor is intended
as proof.

~~~
BrainInAJar
Phones get hacked too. If you ever access a site with your phone, it's not two
factor auth. Malware can read your 2FA key _and_ read your password as you
type it in.

~~~
mirashii
And thus why we have Yubikeys and other hardware tokens.

------
sarciszewski
This sounds like a half-assed hack if they were able to limit the damage. 2/10
try harder next time, skiddos.

------
gkya
What the flippin' heck is Slack? I received an email today telling me of this
event. Then I had to recover my _team_ and my password, then deactivate my
account, and I have never seen this website until I saw that email. Looking at
my _team_ , it seems to be created by the guys at <pyistanbul.org>, or someone
using that website as a base for accounts. That one has a people page and
there are links to various places. Now, someone has created an account in my
name using my main e-mail address and I have not been e-mail-notified for a
confirmation of account, or at least not have been notified that an account is
created. Apparently either these lot cannot write proper emails that can pass
through spam filters or they expressly allow this sort of _malicious_ account
creation. Not surprising that they got hacked.

