Securing Container Deployments on Bluemix with Aqua Security
The security of enterprise software has never been more important. As more and more enterprises move to container-based deployments in the public or hybrid cloud, there are increasing numbers of security professionals who must be confident that they are minimizing and controlling risk. In this post we’ll explore using Aqua Security to protect a containerized deployment on the IBM Bluemix Container Service from attacks in real time. As an example, we’ll run the sample Car Dashboard app for the Watson Conversation Service and take a look at it under Aqua.
You can divide security exploits into two categories: the ones we know about and for which there are patches, and the ones that haven’t been fixed yet - perhaps they haven’t even been invented yet!
Scanning container images for known vulnerabilities allows us to know whether those images have issues from that first category, but what can we do about that second group?
In any deployment, you can limit the attack surface by reducing the set of things that the deployed software is allowed to do. Containers can help with this, especially in a microservices architecture. A given microservice has a limited set of things that it should do - for example, a product search microservice needs read access to a product database, but probably has no business writing to any files on disk. If we can catch code in the act of doing something unexpected, we might be able to stop a new exploit from taking effect.
It’s possible to limit what a container can do using security profiles (e.g. AppArmor or seccomp), and tools like Aqua make it much more straightforward and manageable. As well as scanning your images for known vulnerabilities, Aqua helps you define and police a run-time profile for each of your containers. Let’s see that in action.
We install Aqua in Bluemix just as we do in any Kubernetes environment. The Aqua components include a containerized Web service that’s exposed externally, and Gateway and Database services that are internal only. In addition, an Aqua Enforcer (also a container) runs on every node, which is achieved in Kubernetes using a DaemonSet.
Installing the Watson Conversation
There is a great Blumix recipe showing how to set up the Watson Conversation service, bind it to Kubernetes and deploy the sample containerized application. After running through the steps in that article, we can see two pods running the bot-son app in our deployment.
View Running Containers in Aqua
Aqua automatically spots any container running on the cluster. That includes the ones that are built in automatically as part of the Bluemix cluster deployment, and there are quite a few being used to run Kubernetes itself and its networking. Fortunately we can filter by name, and just find the bot-son containers.
Scanning Images for Known Vulnerabilities
We can see these containers are both running an image jcantosz/watson:latest. This images hasn’t yet been scanned, but we can easily add it to the set of images we’re interested in. The Aqua scanner checks the image against known vulnerabilities.
In this case, we can see that the image actually has quite a few known vulnerabilities. It’s not at all surprising, because I’ve chosen an example image that’s clearly intended for experimentation and hasn’t been hardened for production use. (In fact for the purposes of this demonstration that was exactly what I’m looking for!)
Expanding on the image shows us details about the issues that are flagging concerns.
We can get even more information about a particular vulnerability - I’ll pick the one related to curl as an example.
Hardening Container Images
Looking at the Dockerfile for this container image, we’ll see that it uses curl to install application code, but as we can see from the description in the Aqua console, this application is a cause for serious concern. For a production environment, it would be a good idea to simply remove the curl application from the container image once it has done its job. If the code is not in the image, that vulnerability can’t be exploited by the bad guys.
We can run curl inside this container, like this:
$ kubectl exec -it bot-son-3825905317-nb1md curl
curl: try 'curl --help' or 'curl --manual' for more information
As I didn’t supply a URL this didn’t do anything much, but it’s sufficient to show that curl is installed and can be executed inside this container.
Removing unnecessary code like this from container images is a sensible step towards security hardening your container images.
To see what the bot-son app does under normal behaviour we run it under Aqua’s runtime profiler. I had a brief conversation with the car dashboard.
While this conversation was happening, the Aqua Enforcer noted the executables and other resources that the container needed to use.
I stored this profile for the container image, and applied it so that Aqua will ensure that any instances of this container can’t do anything not defined as part of the profile.
Try an “Exploit”
We know from the image scan that this particular image has curl installed, but because it hasn’t been used within the container in normal use, it’s not permitted by the runtime profile. We can test this out by trying to run curl again inside the bot-son container.
$ kubectl exec -it bot-son-3825905317-nb1md curl
As you can see, the Aqua Enforcer is limiting the container so that it can only run executables that are permitted by the profile.
The Enforcer also allows for limiting any given container’s access to certain network addresses and ports, environment variables and to run under certain user IDs.
This is a very powerful way to add an extra layer of defense so that even if your container image running on Bluemix contains a vulnerability, exploits can be stopped in their tracks with runtime profiling.
Kubernetes + Aqua on IBM Bluemix
Now that the IBM Bluemix Container Service supports Kubernetes, it offers a consistent cloud environment for deploying your containers, and it’s easy to link your containerized apps with the cognitive capabilities available from Watson. Add Aqua’s security capabilities, and you have an attractive solution for secure, intelligent, containerized applications.