This isn’t a story about a Docker vulnerability; it’s a story about how hackers are looking for unsecured Docker deployments where they can mine cryptocurrency. You shouldn’t leave your Docker daemon unsecured any more than you would leave your mail server unsecured.
We’ve heard many accounts of attempted (sometimes successful) cryptocurrency mining attacks on container environments and decided to investigate and analyze an anatomy of such an attack.
While not the most malicious of attacks (data exfiltration or DDoS often have more devastating effects), illicit use of cloud compute resources for mining cryptocurrencies should be regarded as a big warning sign on the security posture of the mined environments – because if an attacker can run a rogue container that mines for bitcoin, they can probably run containers that do worse things. On the other hand, because coin mining is not the most sophisticated attack, it’s rather popular among “script-kiddies” and can be used to learn about methods that are used out there.
Before elaborating on what happened, a little background is required.
Docker Daemon Network Access
Docker works in a client-server architecture. A daemon runs in the background as a service, and is responsible for container management. It creates, runs, and stops, the containers, and operates the many other function that docker supplies.
Executing a docker command invokes a client that sends the requested command to the server (docker daemon) through Docker’s REST API.
On Linux hosts, docker daemon listens to a unix socket that it creates at /var/run/docker.sock.
In order to make the daemon accessible and manageable over the network, one can configure it to be exposed via TCP socket. The default ports are:
- Port 2375 for plain HTTP connections
- Port 2376 for trusted HTTPS connections
Docker daemon replaces the unix socket on Windows hosts with a named pipe.
If Docker engine must be exposed, it is generally recommended to configure it as HTTPS only, and with enforced authentication.
The exposure of docker daemon to untrusted sources without authentication poses a major threat to the host.
Enter the Honeypot
Attacker methodologies share one objective, targeting a victim. They achieve that goal in different ways, singular targeting via social engineering, group targeting by physical location, or a wide area network scanning to find alive services and other.
The objective of research honeypots is to learn an attacker’s goals and methods. These honeypots are built to make an attacker believe they are actual targets, lure them in and make them spend as much time as possible without realizing it’s actually a trap.
While an attacker is in the honeypot, the honeypot detects and logs activities, which can later be processed and analyzed.
Our Setup
In order to examine the container attack surface and the container security posture, we have decided to target scanners of docker daemons.
As mentioned earlier, docker can be configured to be exposed over the network by TCP sockets. We deployed a virtual machine, installed docker on it and exposed it to the internet.
To monitor the honeypot environment we deployed the Aqua Enforcer on the same VM. We wanted to focus on how an attacker might deploy a container with the intent of running it, but without actually running it, so we configured Aqua CSP to block containers from running.
From there on we just had to wait and collect all event logs and other related information from the honeypot environment.
It took roughly two days until we noticed interesting traffic – someone took the bait.
Story Time
Non-production systems, whether they’re test environments, development, or otherwise, tend to be less protected. Those systems are touched by many more people than production environments, developers, QA, presale engineers etc., which leads to a much greater risk of human error. Another factor is that for practical reasons, engineers may take configuration shortcuts to make their work easier.
There is a tendency to underestimate the security needs of such systems, leading to potential disaster – in many cases an attacker is able to carve his way from lower, less important systems to the company network and into true production environments.
Our story is about a system that we “accidentally” exposed.
Let the games begin, and may the odds be ever in your favor!
Note: During this research we found some sources that were exploited to distribute malicious services. We contacted the relevant parties to disclose those sources.
Weird Traffic
So coming back from our weekend, having had our morning coffee, we noticed that hundreds of suspicious actions were logged by the Aqua Enforcer. Undoubtedly, many of them were created automatically by a bot that didn’t manage to cause any trouble.
The attacker attempted to execute a variety of docker commands for image and container management. After several hours of attempts , it seemed that the attacker just gave up.
Behavior Analysis
The attacker’s first objective was to identify the available Docker API version. This can easily be done by invoking a command for an old version. The response would specify the minimum supported API version:
Handler for GET /v1.16/version returned error: client version 1.16 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version
The attacker’s first objective was to identify the running Docker version, i.e., invoking the docker version command. The result would help him learn about the system he should target, and to know the API version he should use.
After successful information gathering about the victim, the next step would be to execute his scheme – running his code on the target host by running containers.
First, he had to inject his malicious image into the docker host. The popular and conventional way to do this is by pushing the image to a registry (Docker Hub is the natural place), and pull it from the victim host.
Unexpectedly, the method we saw was a bit different. The attacker used docker import functionality for image injection. Docker import can be used to download an image from a local/remote tar file via path or URL. The name of the imported image contained “xmr”. XMR is the code of the cryptography coin Monero, which can be mined with almost any computing power. The attacker’s goal was to use the target’s resources for his own profit.
After a successful image injection, the attacker would start mining. When using the docker CLI client, you would typically invoke the docker run command. Behind the scenes, running a container (detached mode) invokes two REST API calls:
- Create a container instance via docker create. The result grants the invoker the created container ID. This ID is used to manipulate the container.
- Once a container is created, it can be deployed via docker start, using the container ID.
Since we did not allow to run containers, the attacker failed at the docker create stage. He repeated the steps many times with no success.
The attacker switched his method to inject the image, then summoned docker build for the rescue. Similarly to docker import, the docker build command can create an image from a URL of a git repository / pre-packaged tarball / plain text file. There was an attempt to build an image, named “xminer”, from a tarball URL.
After many repeated attempts that failed miserably, the attacker decided to use a different Monero miner – Minergate official image. Needless to say, it failed as well.
Visiting The Attacker
The source of the traffic was, ostensibly, from somewhere in the Netherlands.
Our assumption was that the attacker might be hosting other services, maybe malicious services or legitimate ones for shadowing his true purpose.
We saw a web server running there. The root page contained a JavaScript snippet that used the Coin Hive platform for Monero mining. The browser miner reported to a control hub that coordinated the miners using WebSockets.
It looks like a complicated system that targets victims and utilizes their computing power.
Summary
Docker daemon enables its usage by exposing itself to the network so that it can be managed remotely. There is risk in configuring docker to listen to unauthenticated HTTP traffic, therefore, a good practice is to disallow such a connection.
An attacker can use such a configuration to execute arbitrary code on the victim host, inside containers.
Nowadays, attackers can target docker as a service to attack, alongside services like HTTP servers, Mail services, etc.
Mining anonymous cryptocurrencies is very popular these days, and the attack we observed is in line with this trend. The attacker was persistent and attempted all known methods to inject a malicious container, when a simple image pull hadn’t worked.
Aqua CSP detects and blocks such abnormal and unwanted behaviors, which in the case allowed us to observe and learn the various attack vectors, while preventing the attacker from achieving his goal of exploiting compute power to mine for cryptocurrencies.