In August, the National Security Agency (NSA) and the Cybersecurity and Infrastructure Security Agency (CISA) released their Kubernetes Hardening Guidance. As Kubernetes continues to rapidly gain adoption, it’s good to see government organizations, such as the NSA, providing guidance on how to secure this critical technology. This blog post takes a high-level look at the most important points in the guide while also providing commentary on how to implement them.
Target audience
While the guidance is ostensibly for system administrators and developers who work on national security systems, it's applicable and valuable to a wide range of Kubernetes deployments. If you're operating a Kubernetes deployment or planning to build one, take heed of this guidance to ensure that your environment is resilient to attack.
Focus on hardening
Kubernetes is both powerful and complex, and few practitioners know all the nuances of securing Kubernetes environments. By default, Kubernetes leaves many components vulnerable to exploitation. The NSA and CISA guidance outlines the benefits and importance of hardening Kubernetes deployments to ensure that they are safe for a production environment.
Areas of focus
The guidance covers a wide set of controls for Kubernetes environments. It covers preventative controls, such as network separation and authentication, and detective controls, such as logging and monitoring, that can be used to detect and respond to attacks. Using both types of controls is critical to reducing your Kubernetes attack surface while still responding to threats quickly when they gain a foothold in the environment.
Some of the guide’s highlights include:
Network separation
This is an important control for clusters that host multiple applications. Key recommendations include guidance for default policies for each Kubernetes namespace to restrict ingress and egress traffic. This “default deny” policy approach is a valuable way of ensuring that only required network traffic is allowed in the cluster. It is easiest to implement in new clusters, so that applications can be designed with it in place, rather than trying to retrofit the policies, which requires analysis of network flows to understand the required network policies to put in place.
Authentication
On authentication, the guidance provided is quite high level, noting the importance of using of an external authentication service, because none of the built-in Kubernetes services are suitable for production use. Another control to note in this section is the requirement to remove anonymous authentication from the cluster to reduce the risk of attackers gaining unauthorized access to cluster resources.
Logging
The guide’s coverage of logging and monitoring includes several important areas for cluster administrators to consider when designing their containerized environment. On container logging, the importance of centralizing logs to allow for secure storage and analysis is noted along with analysis of cluster traffic to look for anomalies that might indicate an attack.
Auditing
The NSA and CISA guide also echoes other Kubernetes security guides in recommending that Kubernetes’ auditing facility be enabled, which allows for all activities done via the Kubernetes API to be reviewed. Care should be taken, however, with the sample Kubernetes audit policy that the guide provides in Appendix L because it's likely to generate a large quantity of traffic. Some tuning will likely be required.
Control plane and worker coverage
When architecting and implementing Kubernetes, it's critical to configure the control pane correctly to prevent vulnerabilities while also implementing controls to ensure that workloads deployed into the cluster are properly secured.
Main threats to Kubernetes
This document is a good opportunity to see what threats the NSA sees targeting Kubernetes environments. The NSA recommends keeping an eye for the following threats:
Supply chain attacks
In these attacks, threat actors attempt to compromise any element that makes up a system. These elements include third-party software and vendors used to create and manage a Kubernetes cluster.
Malicious threat actors
These are your more traditional threat actors who aim to compromise a system, usually from a remote location, by exploiting vulnerabilities in the Kubernetes system or the components that support it, or in the application that the system supports. Malicious threat actors may attack systems for monetary gain, to cause damage, or on behalf of a hostile nation state.
Insider threats
Employees and other insiders often have privileged access to your systems. Malicious insiders will abuse their access privileges, special knowledge, and other benefits to compromise a system for their own personal gain.
What to watch out for
These guidelines, like many others in container security, show the difficulty in keeping up with the latest developments. Simply speaking, DevOps moves so quickly that guidelines often are already outdated when they are released – and the NSA and CISA hardening guide is no exception. The recommendations don't specify what Kubernetes version they apply to, which makes it hard to determine when recommendations are outdated — and in fact, some already are.
The guidelines contain several recommendations involving the Pod Security Policy Kubernetes feature, which has been deprecated. Instead, using the Pod Security Standards (PSS) from Kubernetes is a better choice because they cover more of the available options. All workloads in a cluster should use the baseline PSS, and wherever possible, you should use the restricted PSS for greater security. In addition, the guidelines refer to old, insecure versions of the controller manager and scheduler ports.
How Aqua Security can help
Kubernetes security
Aqua’s Kubernetes security includes a number of capabilities that will help with or entirely satisfy the NSA and CISA hardening recommendations. Kubernetes security capabilities allow you to:
- Scan containers and pods for vulnerabilities and misconfigurations
- Use strong authentication and authorization to ensure that no one has overly permissive access
- Run containers and pods with the least privileges possible
- Use vulnerability scans to identify risks and ensure that they are accounted for and patched
Runtime and workload protection
Aqua’s runtime protection and workload security capabilities will enable you to:
- Enforce network separation to reduce an attacker’s reach in the event of a breach
- Restrict unnecessary network connectivity
- Ensure that network communications are properly encrypted
Conclusion
There’s never been a more critical point for a strong security baseline, so the NSA and CISA guidance comes at the right time. Considering that cloud native security is such a rapidly changing field, the guidance is thorough and examines the benefits of a wide set of controls for Kubernetes environments, covering both preventive controls, like network segregation and authentication, and detective controls, like logging and monitoring, which can be used to detect and respond to attacks.
While the hardening guidelines provide valuable guidance for securing your Kubernetes environment, they aren't perfect. Some of the information is outdated, and the guidance provides no Kubernetes version numbers, which is important because recommendations may apply only to a certain version.
To learn more about Kubernetes security best practices, get a free Gartner report Best Practices for Running Containers and Kubernetes in Production.
