In the spectrum of deployment options available for cloud native applications, the most widely used option, at least presently, is running containers on VMs that use the Linux operating system (or less frequently, bare-metal servers running Linux).
Since containers share the host OS Linux kernel, securing the host and securing the containerized applications that run on it are inseparable. A vulnerable container can be used to attack the host, but a vulnerable host OS will make the containerized application vulnerable to attack as well.
There have been two unanimously adopted best practices for running containers on Linux:
- Always run containers on dedicated hosts, and don’t mix containerized and non-containerized workloads on the same machine: The logic here is simple. The two types of applications have very different update cycles, immutability vs. mutability and different stack architectures. Using both on the same machine would negate the security benefits of using containers, and make it very difficult to secure both types of workloads adequately.
- For containers, use a “Thin OS”, i.e. a minimal Linux distribution: There are many options for Linux distros, and some of them are “heavy” distributions made to support enterprise applications with many requirements. As recommended by Gartner, if you’re only going to run containers on a machine, it’s better to use a “thin OS” such as RedHat CoreOS, VMware Photon or Rancher OS, which provide the support needed to run containers but not the extraneous features that aren’t needed in a highly orchestrated environment.
“Thin OS” Security Considerations
In addition to performance, cost, and manageability as considerations for using a thin OS to run containers, security is front and center. Here’s why:
- Containers are immutable: Once you run a container, its code doesn’t change. There’s no patching in runtime. Instead, the container image is updated, and fresh instances deployed replacing old ones. So it makes sense to apply the same logic to the host OS itself – which is what some of those minimal Linux distros do. This means that any attempted change to a running OS will be blocked, making the attacker’s job much harder.
- Containers are orchestrated: Especially in production environments, the assumption is that containers are governed by an orchestrator such as Kubernetes. It is the orchestrator that determines when and where containers run, and it is the orchestrator (in conjunction with CI/CD) that switches out old containers for new ones. This “single entry point” model makes it easy to lock down access to such actions so that only the orchestrator is allowed to perform them.
By cutting out many of the capabilities provided in larger Linux distributions, a thin OS provides a much smaller attack surface, with fewer packages and fewer configurations to worry about. This, coupled with immutability and the ability to enforce “single user” access for the orchestrator makes the job of securing container hosts much more effective.
Security Controls for Thin Hosts
“No man is an island entire of itself; every man is a piece of the continent, a part of the main” wrote John Donne in his famous poem.
The same holds true for hosts in an orchestrated environment. Yes, thin OS hosts are more secure to begin with, but they can never be completely isolated. There are networking requirements, and instances such as “break glass” situations or upgrades, when a human admin will need to login to a host and make changes, etc., etc. There may also be vulnerabilities in the OS itself, with packages that may be exploited, or poor configuration that leaves the host vulnerable.
It’s therefore recommended to keep the OS patched and updated, and use hardening methods as appropriate for running containers. There are best practices from the likes of the Center for Internet Security (CIS) and NIST that can and should be applied.
As with other areas of security, using preventive measures alone is insufficient. You should also monitor and constantly check your environment for vulnerabilities and anomalies, as well as track user and application activity for compliance purposes.
The good news is many of the resource-intensive security tools used to secure traditional hosts such as HIPS (host-based intrusion prevention) are overkill for thin OS hosts, if not completely redundant. Most of the protections they provide are simply not relevant to a thin OS. You don’t need to use two different solutions to protect containers and the host OS they run on. That’s where Aqua can help.
How Aqua Protects Thin Hosts
At Aqua we’ve implemented a series of security and compliance controls intended to protect thin OS hosts, along with the containerized applications running on those hosts. This is part of the Aqua platform and can be used to manage full stack security and compliance.
First, we provide automated checks of your hosts against the CIS Benchmark for Docker and Kubernetes, which include best practices for keeping your hosts patched and hardened, as well as specific minimal privileges for hosts running containers, such as seccomp profiles. You get a very clear picture of which checks passed or failed.
Aqua also scans container hosts for vulnerabilities. This a separate process from scanning for vulnerabilities in container images, or notifying you when you’re running vulnerable containers — it’s scanning for vulnerabilities in the host OS itself. We will alert you if you’re running a vulnerable host OS, which allow you to update your OS and remain compliant.
Additionally, Aqua checks hosts for logins and login attempts, and logs them as audit events, a key requirement for many regulatory compliance mandates. It provides evidence of attempts to tamper with the host and makes it possible to enforce orchestrator-only access. As with all other audit events in Aqua, you can automatically send the event stream to 3rd party SIEM and analytics tools (e.g. Splunk) where you can define alerts for certain types of events.
Adapting Security to “Thin OS”
If you plan to “put your hosts on a diet”, adapting your security to “thin OS” hosts has never been easier. In environments where organizations are constantly required to add more and more layers of security, it’s good to know that minimal configurations and adherence to the least privilege model can also present an opportunity to simplify and reduce layers of security that are needed for “fat” hosts.