Tracee: Tracing Containers with eBPF

This week at Velocity Berlin, I’ll be giving a talk called A Beginner’s Guide to eBPF. To coincide with it, we’re opening up a new Aqua Security open source project called Tracee, which uses eBPF to trace events in containers. This isn’t something that most developers need to do on a day-to-day basis, but for those of us interested in the details of implementing container security, this can be a helpful tool.

eBPF is a kernel technology that lets you run custom programs within the kernel itself. It is typically used for powerful observability tools, such as bpftrace.

Tracee Focuses on Container Events

Using eBPF to trace events such as system calls is not new. You can see me live-coding an example in my eBPF Superpowers talk at DockerCon earlier this year. In putting together this talk, I was standing on the shoulders of giants such as Brendan Gregg. What Tracee brings to the party is the ability to see only events generated within containers, without being cluttered by events from other processes running on the host.

As you may know, from the host machine’s perspective, a container is really just a set of Linux processes. If there is no real difference between a containerized process and a regular one, how can Tracee detect if an event comes from a container or not?

A container is usually started with its own process ID (PID) namespace. Inside that namespace, the first process looks as though it has the ID 1. Tracee monitors for new binaries being executed, checking the process ID from the perspective of the process’s PID namespace. Whenever it finds a process that appears to have an ID of 1, Tracee knows that it’s the first process within a new PID namespace, so it starts tracing. Any other process that shares the same PID namespace will also get traced. Tracee will catch child processes within the container, as well as other containers in the same Kubernetes pod which share the PID namespace.

This approach means that any pre-existing containers won’t get traced – you need to run Tracee first and then start the container in order for it to be detected.

Get Tracee

Tracee was written by Yaniv Agman in Aqua’s research team, and it’s currently experimental. It supports the tracing of the most common system calls, as well as other events like cap_capable, where the kernel checks if the process has the capabilities it needs.

Check out the README on GitHub for instructions and examples of Tracee’s output

We’d love your feedback on it!