A few days ago, Docker discovered that a database holding the credentials of some 190,000 Docker Hub accounts was exposed to unauthorized access (about 5% of all Docker Hub accounts). We’ve been getting questions from customers on this, so I wanted to set the record straight on what we know and what we recommend doing.
I’d like to clarify that Aqua’s accounts on Docker Hub were not among those that were compromised. In any event, we do not store Aqua’s own product images on Docker Hub.
What Happened
The full disclosure was provided by Docker’s customer success team. The TL;DR version is that a database was accessed by an unauthorized party. The data on it included usernames and hashed passwords as well as GitHub and Bitbucket tokens for Docker autobuilds. All these tokens have been revoked. Docker notified the affected account owners.
Was this an intentional breach? We don’t know. The unauthorized access doesn’t necessarily mean that it was an intentional act by malicious parties. Since that possibility exists, Docker took action to disclose and notify, as any responsible vendor would do.
The risk in such a breach would be that the compromised access credentials could be used to read private images, and with read/write access also potentially replace them with poisoned images, ones that contain malware, for example. However, from the information provided, there’s no evidence to suggest that exposed credentials were actually used.
It should also be clear that this type of incident has no bearing whatsoever on container security, per se. It doesn’t pertain to any element in the container stack, nor to its structural integrity. It’s purely an issue with this specific Docker SaaS offering.
What You Should Do
First, change your Docker Hub password and tokens. Even if your account isn’t one of the compromised ones, it’s good practice to do so periodically, and certainly after an incident. If you used the same or similar passwords on other online services, change those as well. (Try to use different, randomly generated, strong passwords for each service.)
If you were among the affected parties, it’s also advised to check your logs and image history to verify that no suspicious access or tampering took place. Check if a collaborator was added to any of the repositories.
Beyond the Registry: How to Protect Your Images
This incident points to a broader issue that we’ve highlighted before. We already know that the “hard shell, soft center” security model creates a single point of failure. Applying layers of security is always a good idea, and registries (not just Docker Hub) are often protected with access controls only. Some have multi-factor authentication, which offers further assurance. However, once you’re in, you’re in, and you have access to everything. This is problematic, especially since teams often share credentials. (Come on, you know you’ve done it.)
In Aqua 3.5, released late last year, we introduced the ability to encrypt container images, making them readable only with a decryption key. We use Aqua MicroEnforcer to decrypt a container when it is instantiated.
With image encryption, even if intruders gain access to your registry, they won’t be able to read nor run encrypted images. Naturally, the decryption keys are stored outside the registry in a key management system (KMS) or vault.
In the era of DevOps, developer environments are becoming as sensitive to attack as production environments and protecting them should become more of a priority. Perhaps this incident will serve as a wake-up call. There should be better education, monitoring, and tooling.