Hybrid Cloud security

Securing Hybrid Cloud Workloads on Google Anthos

There are major shifts happening around container technology. We have seen ongoing improvements in automation tooling and changes as fully containerized workloads move into production. Companies are often running more than one production Kubernetes cluster at a time.  DevOps and operations teams, subsequently, have an additional burden because they must control and manage these clusters, and configuring Kubernetes isn’t easy. Operations teams are looking for tools that utilize a centralized approach to manage this dynamic infrastructure.

However, as we layer multiple sets of tools and infrastructures to support these clusters, automation can become inflated, incredibly complex, and prone to error. We need an easier way to manage these resources and keep them secure at the same time.

Google Anthos is a solution that enables a true hybrid cloud experience for Kubernetes clusters. In this scenario, the burden of managing the configuration state of multiple Kubernetes clusters can be drastically reduced with Config Management. Config Management is a component of the Google Anthos solution that enables a centralized, declarative configuration control with a GitOps approach. In other words, a centralized Git repository is responsible for managing the state of all “registered” clusters. It’s a one-stop-shop for config changes.

Aqua Security, as an enterprise security platform, can leverage a centralized configuration tool such as Config Management, and deploy it across all GKE or GKE On-Premises, and still have the desired granular control, depending on the type of resources which need to be secured.

By using Config Management to control the deployment and configuration of Aqua Security, we can easily achieve:

CI/CD Scanning Integration for any cluster (permanent or ephemeral)

Automatic distribution of Enforcers across all GKE and GKE On-Premises nodes, providing:

  • Host vulnerability management
  • Image vulnerability management
  • Runtime enforcement
Centralized secrets management with distributed access, providing:
  • Real-time secret updates without pod restarts
  • Integration with secrets providers such as HashiCorp Vault

Centralized dashboards, providing:

  • Policy configuration
  • Audit logging
  • Cross-cluster visibility

Google Cloud Platform integration, providing;

  • Google Container Registry inventory and scanning
  • Google Stackdriver logging integration
  • Google SAML integration for user account management

Here's an Environment Overview: 

1_OK

Deploying Aqua Security Components with Anthos Config Management

Note: While the content below illustrates all Aqua Security configuration via the web console, the aquactl CLI tool is a new addition to v4. You can use it to automate the configuration of your Aqua environment.

Ultimately, the daemonset is the most important component to place into Anthos Config Management, since it ensures that all managed clusters receive Enforcers. For this lab, however, all components are deployed via Anthos Config Management.

In order to leverage Anthos Config Management to deploy the Aqua Security components, check the following:

  • You utilized a single aqua namespace to hold any applicable Aqua components
  • All server/database/gateway components were deployed to a single cluster using a ClusterSelector
  • Two separate daemonset configurations were created, using a ClusterSelector, to target different GKE clusters with different Aqua Enforcer Policies

How it works

The aqua Config Management directory looks like this:

  │   ├── namespaces
    │   │   ├── aqua
    │   │   │   ├── csp-registry-secret.yml
    │   │   │   ├── db-deployment.yaml
    │   │   │   ├── db-password-secret.yaml
    │   │   │   ├── db-pvc.yaml
    │   │   │   ├── db-service.yaml
    │   │   │   ├── enforcer-daemonset-ephemeral-cluster.yml
    │   │   │   ├── enforcer-daemonset-gke-op-cluster.yml
    │   │   │   ├── gate-deployment.yaml
    │   │   │   ├── gate-service.yaml
    │   │   │   ├── namespace.yml
    │   │   │   ├── serviceaccount.yaml
    │   │   │   ├── web-deployment.yaml
    │   │   │   └── web-service.yaml

The server/database/gateway components are annotated with the proper cluster selector for the GKE cluster with the label env: core (The labeling is decided upon arbitrarily by the operations team.)

The metadata annotation to select the correct cluster

metadata:
      annotations:
          configmanagement.gke.io/cluster-selector: selector-env-core

Separate daemonsets to align with different Enforcer groups. Ephemeral clusters are selected as follows:

metadata:  
      annotations:
          configmanagement.gke.io/cluster-selector: selector-env-ephemeral

Long running production clusters are selected as follows:

metadata:
          annotations:
              configmanagement.gke.io/cluster-selector: selector-gke-op
              configmanagement.gke.io/cluster-selector: selector-env-core

Once the base configuration of the Aqua Server is completed, create two separate Enforcer groups, which generate the appropriate daemonset manifest.
Aqua Cloud Native Security

Here are the two separate Enforcer groups:
Container security

Aqua Integration with the Google Container Registry

Aqua Security supports all the popular container registries, including Google Container registry. The advantage is that the Aqua scanner visualizes and scans all images within the registry. It is not limited only to the images that are scanned via the CI pipeline, or to images that are pulled down to the worker nodes, and maintains continuous visibility into the security posture of images. This simple integration requires the service account JSON key or an access token to access the appropriate repository.

Under System > Integrations > Image Registries

Aqua Cloud native security

Once all images are scanned, the scanning results and vulnerabilities information are presented in this dashboard:

Aqua Cloud Native Security

Integrations with CI/ CD Pipelines

Aqua capabilities include scanning the CI/CD pipeline, which is a powerful feature with many integration options. In this demo environment, leveraging the Aqua Security Scanner Plugin for Jenkins is the de facto strategy, enabling developers to receive vulnerability feedback at any stage in the CI pipeline as outlined in this OpenShift Container security blog. The Aqua MicroScanner is also a powerful tool for scanning local builds, or scanning during the build process, as outlined in this MicroScanner blog.

In this demo environment, an ephemeral GKE cluster is created, our application is built and deployed, and then it is scanned for relevant results. The Enforcers are acting in Audit mode only.

7_OK

Secrets Management Access

Another advantage of running Aqua Enforcers in every GKE and GKE On-Premises cluster is that the Enforcers have access to secrets managed by Aqua Security. (Secret access can have a fine-grained access policy applied to it.) As long as an Enforcer is present, Aqua can serve up any secret as an environment variable, or as a tmpfs file within a desired pod, regardless of the cluster or location. This functionality means that developers don’t need to create or synchronize secrets into each new cluster that they deploy. Furthermore, it is often a simpler method of proxying secret access from a solution such as HashiCorp’s Vault, instead of deploying vault-agent sidecars or writing in direct API calls into the application.

More Integrations on the Horizon

We showed that the Google Anthos solution enables a true hybrid cloud experience for Kubernetes  clusters. Aqua can leverage centralized configuration tools and use Config Management to control the deployment and configuration of Kubernetes clusters.

Future blogs will explore more details of secret distribution as well as additional Logging and Google SCC integrations with Aqua.

This guest post by Shea Stewart was first published on the Arctiq Blog

Picture of Guest post by Shea Stewart

Guest post by Shea Stewart

Shea is a partner at Arctiq: Intelligent Architecture. With the bulk of his career focused on infrastructure solution architecture and implementation, Shea now brings that knowledge into the development space. Working with software development and infrastructure teams, Shea focuses on improving technology processes through automation and collaboration tooling and culture changes. Being a technologist, Shea is happiest when found exploring emerging technologies and applying them to real-world customer challenges. Outside of the office Shea can be found out with his friends enjoying a Jays game, being active, or hunting down a new craft brew.

Kubernetes Security, CI/CD, Hybrid Cloud Security, Google Cloud Platform

Related Posts