SaaS-based monitoring and analytics platform Datadog has reset all stored passwords following unauthorized activity on some infrastructure servers.
Chief Security Officer Andrew Becherer opened up to users about the incident over the weekend:
“Last night we sent email notifications regarding a security incident that took place within our server infrastructure on 2016-07-08. While our team is working on the technical and forensics aspects of the incident response, we want to be fully transparent with you regarding our current status and help you protect your own infrastructure.”
The Datadog security team detected unauthorized activity associated with several production infrastructure servers, including a database that stores user credentials.
A user also reported someone unsuccessfully attempted to leverage their AWS credentials shared with Datadog.
The incident did not affect the platform’s service. As of this writing, Datadog rebuilt all compromised systems and infrastructure as well as mitigated all vulnerabilities. It also determined that any agents running on users’ servers were not unaffected.
The cloud monitoring solutions provider stores all passwords using bcrypt, a strong cryptographic algorithm which takes time and resources to break. But to be on the safe side, Datadog sent out a security notice to all admin users urging them to rotate/revoke stored credentials, and it invalidated all stored passwords (Google Auth and SAML users aren’t affected).
Users can reset their passwords here.
Becherer explains the company is still working to figure out what happened and that they’ll provide more information in the coming weeks:
“We’re still piecing together the attack and we have brought in third party incident response and forensics experts. We expect forensics to continue well into next week. A post-mortem and longer term plans will follow.”
As it continues with its investigation, Datadog recommends AWS users employ Identity and Access Management Role Delegation, which prevents individuals from sharing security credentials between accounts.
This announcement follows a few months after databases containing 3.8 million Naughty America users’ usernames and passwords, many of which were also protected using bcrypt, went up for sale on a dark web marketplace.