
DATADOG got hacked? - ssalat
We&#x27;ve faced a nightmare weekend and ran into a locked account at AWS. After more than two days they told us that our key at DATADOG got compromised?<p>--<p>Hello XXX,<p>Thank you for reaching out for your patience. I’m reaching out to you in behalf of my colleague David as he is currently off shift. Regarding this issue the EC2 team have new information, and they mentioned that spot instance requests are block due to a compromised key in your account.
An email regarding this issue was sent on Tuesday, July 12, 2016 at 4:26 PM PDT from no-reply-aws@amazon.com to the email X@X.X whit the following Subject “Informational Message Regarding Security Incident at Third Party ‘Datadog’ [AWS Account: X]”<p>Here is the information regarding the compromised key in your account XXX:
Type: Access Key Pair
Credential: XXX
IAM User: datadog<p>To rotate access keys, you should follow these steps:
1. Create a second access key in addition to the one in use.
2. Update all your applications to use the new access key and validate that the applications are working.
3. Change the state of the previous access key to inactive.
4. Validate that your applications are still working as expected.
5. Delete the inactive access key.<p>Here are some resources that you might find useful: 
[1] https:&#x2F;&#x2F;blogs.aws.amazon.com&#x2F;security&#x2F;post&#x2F;Tx15CIT22V4J8RP&#x2F;How-to-rotate-access-keys-for-IAM-users
[2] http:&#x2F;&#x2F;docs.aws.amazon.com&#x2F;IAM&#x2F;latest&#x2F;UserGuide&#x2F;best-practices.html#rotate-credentials<p>Once these actions are taken please update the case so we can reach out to the EC2 team and they can remove the blocking for spot EC2 instances. If you have any questions or concerns regarding this issue please let us know we will happy to further assist you. Thank you!<p>XXX
Amazon Web Services<p>--<p>Are there anybody else out there with the same issue? Maybe the DataDog team could provide their perspective? We&#x27;re now sitting on the additional AWS costs.
======
irabinovitch1
Ilan with Datadog here.

On Friday, September 30th a number of Datadog-AWS shared customers received an
erroneous email notification from AWS about compromised AWS access keys
associated with Datadog. AWS intended to reach out to shared customers that
had been lax in deactivating and deleting access keys associated with the
7/8/2016 Datadog breach ([https://www.datadoghq.com/blog/2016-07-08-security-
notice/](https://www.datadoghq.com/blog/2016-07-08-security-notice/)). AWS
accidentally sent out a standard compromise notification to the original list
of shared customers. False positives abound! You can validate this error by
contacting your AWS support contact.

As recommended in our original communication, if you have not you should
deactivate _and delete_ any service integration credential shared with Datadog
on or prior to 7/8/2016 immediately and for AWS transition to Role Delegation
as outlined here: [https://help.datadoghq.com/hc/en-
us/articles/210376966-Revok...](https://help.datadoghq.com/hc/en-
us/articles/210376966-Revoking-AWS-keys-and-enabling-Role-Delegation-for-the-
Datadog-AWS-Integration)

The AWS communication was made in error. If you received the initial message
you should have also received a retraction. Customers that have been lax in
deactivating and deleting these access keys will receive additional
communication directly from AWS.

------
subie
Did you make that key public at any time? Its fairly easy to leak keys on
github that get picked up by automated bots. AWS is very good about covering
charges from account 'hacks'

I wouldn't jump to conclusions on Datadog unless you can verify they only had
access to the key.

(Just went through a key leak with AWS)

~~~
ssalat
100% definitely not! It's a specific IAM key only for the use of DATADOG.

