
SendGrid employee’s account was compromised and used to access internal systems - prostoalex
https://sendgrid.com/blog/update-on-security-incident-and-additional-security-measures/
======
rubbingalcohol
> evidence suggests that the cyber criminal accessed servers that contained
> some of our customers’ recipient email lists/addresses and customer contact
> information. We have not found any forensic evidence that customer lists or
> customer contact information was stolen.

Which doesn't mean the customer list data _wasn 't_ stolen. The wording on
this is too cute. Accessing data from privileged accounts doesn't necessarily
leave evidence, depending on the type of system access or server logging
involved.

I think SendGrid should clarify what _could have happened_ here, worst case.

~~~
noir_lord
Thought exactly the same, they lost me as a customer over the wording on their
released statement and email.

Hell it wasn't instantly clear why you needed to reset your password and the
link to make it clear was way smaller than the link to do anything else,
pretty dirty pool.

------
colinbartlett
The customer targeted has got to be Coinbase, right? There was an email sent
around a few weeks ago, clearly spam/scan but seemingly sent by legitimate
coinbase.com servers. I received something like this:

    
    
      subject:      colin, We've got a message for You
      mailed-by:    em.coinbase.com
      signed-by:    coinbase.com

~~~
ikeboy
Yes. See [http://bits.blogs.nytimes.com/2015/04/09/sendgrid-email-
brea...](http://bits.blogs.nytimes.com/2015/04/09/sendgrid-email-breach-was-
used-to-attack-coinbase-a-bitcoin-exchange/)

------
nwenzel
Two Factor Authentication. Please, all companies that have any of my data,
don't just give me the ability to use 2FA, please force all of your employees
to use it on their email, GitHub repos, everything. It should be a requirent
right alongside https.

My personal favorite service is Authy, now part of Twilio.

~~~
shard972
> Two Factor Authentication. Please

My understanding is that Sendgrid isn't too keen on dongles.

~~~
mtrimpe
Maybe they should just publicly fire someone to make this all go away.

~~~
BhavdeepSethi
How will that solve anything? The damage is already done. No point someone
losing their job over this.

~~~
cheshire137
It was a reference to this drama from a couple years back:
[http://arstechnica.com/tech-policy/2013/03/21/how-dongle-
jok...](http://arstechnica.com/tech-policy/2013/03/21/how-dongle-jokes-got-
two-people-fired-and-led-to-ddos-attacks/)

------
dantiberian
What was their password hashing algorithm? The lack of specificity isn't
encouraging. The vague "hashed and salted" could mean anything from MD5
upwards.

~~~
compbio
I'd hope it would be PBKDF2. But then again, they could have just specified
that. The lack of specificity makes me think otherwise.

------
mr337
Did anyone get an email about this from SendGrid. We have been searching and
haven't found a "We got hacked, please reset your passwords".

I had to find out on HN, also on twitter they seem to be downloading it too.

~~~
akarambir
I got an email on 28th and a reminder yesterday on both of my account emails
to reset my password.

~~~
ZoF
Same here.

------
yuvadam
> CYBERCYBERCYBER

How about you just give me the details with resorting to propaganda?

~~~
sneak
Spread the meme: Saying "cyber" means "I have no idea what I'm doing or
talking about".

~~~
specto
Because that's what the government and DoD calls it?

------
compbio
> used to access several of our internal systems on three separate dates in
> February and March 2015.

> On April 8, the SendGrid account of a Bitcoin-related customer was
> compromised

If I can gather this right: SendGrid was fully hacked for 3 months on end. At
least that is what they were able to recover from forensics, it may have been
longer.

This sounds illogical:

> We have not found any forensic evidence that customer lists or customer
> contact information was stolen. However, as a precautionary measure, we are
> implementing a system-wide password reset.

How would a password reset help combat information that was stolen before the
reset? The password reset is because the systems accessed contained password
hashes. Also it may be to upgrade the hashing mechanism to be more secure than
"salted and iteratively hashed".

Sendgrid's privacy policy is cookie cutter, but it contains this:

> For example, our policy is that only those individuals who need your
> personally identifiable information to perform a specific job are granted
> access to that personally identifiable information.

Apparently the employee that was hacked needed access to the data of all his
colleagues and all users.

> Upon discovery, we took immediate actions to block unauthorized access and
> deployed additional processes and controls to better protect our customers,
> our employees, and our platform.

Then the Privacy Policy again:

> We will use at least industry standard security measures on the Site to
> protect the loss, misuse and alteration of the information under our
> control. While there is no such thing as "perfect security" on the Internet,
> we will take all reasonable steps to insure the safety of your personal
> information.

So apparently there were still some reasonable steps left to take, which were
forced by this hack, not by 'industry standard security measures'.

> Two-Factor Authentication: We encourage all of our customers to enable two-
> factor authentication, which can effectively prevent unauthorized logins.

I think you can better encourage (or force) your employees to enable this, so
you can prevent unauthorized logins into superuser accounts.

From the Privacy Policy you'd expect they already did this:

> Likewise, all employees and contractors are kept up-to-date on our security
> and privacy practices.

Then the unspecified hashing mechanism. Should you worry about the chance of
account compromise again?

> salts and iteratively hashes passwords

It would be a breath of fresh air if these companies would just say 'We use
bcrypt' in their privacy policy.

> Our Ongoing Commitment to Security

Your 3 month struggle with hackers. Also, three reasonable steps follow that
could have been taken before this hack, like your Privacy Policy promised us.

> NOTE: We require passwords to be a minimum of 8 alpha-numeric characters.
> Make sure any new passwords you set conform to this requirement.

Before or after this hack? Why should the customer make sure his password
conforms to this requirement? Is it even possible to set a shorter password?

> Security update: Please reset your SendGrid account passwords today.
> Beginning today, and in line with standard practice, we are requesting that
> all of our customers reset their passwords to all of their SendGrid account
> access points.

Why not force this? Asking nicely? Standard practice would be to force this
upon next log-in and temp disable accounts that have not changed their
password yet.

------
kordless
Time to decentralize our data, people!

~~~
bdcravens
Which means what in terms of using a service for transactional email?

~~~
davideous
My company, www.drh.net, provides an SAAS or on-premises Mail Transfer Agent,
GreenArrow Engine.

You can run it in your network and tightly lock down the system. We can take
care of all of the email deliverability setup and operations tasks, if you
want.

One advantage of running licensed software on your network is that you don't
pay per-message fees, so for higher volume it's really economical compared to
a service like SendGrid.

~~~
Karunamon
Legitimate bulk email is not a thing you want to have decentralized - the
anti-spam efforts of the past 20 years or so have made that all but
impossible.

~~~
davideous
Not true. We have plenty of clients using their own decentralized email
servers and getting great inbox delivery of legitimate bulk email.

To get good results on your own server, you need:

(a) To be sending email that people want to receive.

(b) To have the technology setup correctly. It's not hard; we do it all the
time for our clients.

(c) It's greatly beneficial to above ~20k messages/day on your own server. Not
required, but being above that point lets the ISPs gather statistical data on
your complaint and open rates, which allows reputation filters to click in.

