Aqua Blog

Kubernetes Metadata and Manifesto

Kubernetes Metadata and Manifesto

We recently released the first prototype of Manifesto, a tool for storing metadata alongside a container image. Metadata can hold information about software that can change even if the software itself doesn’t, and we can use this information to create powerful automation tools.

Here’s a simple example use case that we’ll demonstrate in this article. Suppose that a fictional organization – we’ll call it CompanyX – has a number of different components owned by different teams. If one component starts behaving badly, for example generating error logs, the ops team needs to know who to contact.

CompanyX can store the contact information in the form of container image metadata, query it, and keep it up-to-date using Manifesto.

Storing Contact Information as Metadata

Alex Developer works at Company X as the lead on Component A. She adds her contact details, along with a webhook to the team’s Slack channel, into a text file.

cat contact.file
{
email: [email protected]
phone: 0123 456 789
slackwebhook: hooks.slack.com/services/abc/def/123
}

This is associated with the image for Component A using manifesto:

./manifesto put companyx/componenta contact contact.file
Storing metadata 'contact' for image 'companyx/componenta:latest'
Username: alexd
Password:
Metadata 'contact' for image 'companyx/componenta:latest' stored at sha256:6a39686306556ab512ec1e0842dc5134f8e5e879080e9cc9e21fecf78d95106eAdding 'contact' metadata to manifesto for
'companyx/componenta:latest'

If Alex moves to a different team, the contact file can be updated with the new team lead’s email and phone number, and stored against the container image with another manifesto put (without having to re-publish the container image for Component A).

This step can be automated as part of a CI/CD pipeline that updates runs manifesto put whenever contact files are changed.

We now have the metadata stored alongside the container image. The next step is to apply it to Kubernetes resources.

Adding Metadata to a Kubernetes Resource

Kubernetes resources can have metadata associated with them through Labels and Annotations. Both are key-value pairs, but they are used for different purposes:

  • Labels are typically used to indicate some attribute of the resource that might be used to identify it, or to select is as a member of some logical grouping of resources – for example, mark whether it’s part of a production, staging or development.
  • Annotations are more free-form and are intended for arbitrary metadata.

For the contact information in our example, Annotations are more appropriate.

The kubectl annotate command takes one or more KEY=VALUE parameters to define the annotations being added. Let’s apply the contact information to the deployment for Component A with the latest contact data for the container.

kubectl annotate deployment componenta contact="$(./manifesto get companyx/componenta contact)"
deployment "componenta" annotated

We can take this a step further by first querying what the container image is for the deployment.

IMAGE=$(kubectl get deployment componenta -o=custom-columns=:.spec.template.spec.containers[0].image --no-headers)
> kubectl annotate deployment componenta contact="$(./manifesto get $IMAGE contact)"
deployment "componenta" annotated

In this way we can easily add contact information (or any other metadata) in the form of annotations, to all the running software in our cluster.  If the contact information changes, this can be updated simply by overwriting the annotation.

Querying the Annotation

When something goes wrong and we need to find out who to contact about Component A, we can simply read the information out of the annotation.

$ kubectl get deployment componenta -o=custom-columns=:.metadata.annotations.contact --no-headers
{
email: [email protected]
phone: "0123 456 789"
slackwebhook: hooks.slack.com/services/abc/def/123
}

With this information we know who to contact about this component. This could easily be further automated, for example to send a message to the team’s Slack channel.

Other Use Cases for Metadata

There are many more scenarios where container metadata could be applied. Puppet have shown how to use Lumogon and Manifesto to store and query information about container image contents, and we have a list of use cases on the Manifesto Readme. Thought of another example? We welcome pull requests!

 

Liz Rice
Liz Rice was the VP of Open Source Engineering at Aqua Security.