This week at Velocity Berlin, I’ll be giving a talk called A Beginner’s Guide to eBPF. To coincide with it, we’re opening up a new Aqua Security open source project called Tracee, which uses eBPF to trace events in containers. This isn’t something that most developers need to do on a day-to-day basis, but for those of us interested in the details of implementing container security, this can be a helpful tool.
eBPF is a kernel technology that lets you run custom programs within the kernel itself. It is typically used for powerful observability tools, such as bpftrace.
Using eBPF to trace events such as system calls is not new. You can see me live-coding an example in my eBPF Superpowers talk at DockerCon earlier this year. In putting together this talk, I was standing on the shoulders of giants such as Brendan Gregg. What Tracee brings to the party is the ability to see only events generated within containers, without being cluttered by events from other processes running on the host.
As you may know, from the host machine’s perspective, a container is really just a set of Linux processes. If there is no real difference between a containerized process and a regular one, how can Tracee detect if an event comes from a container or not?
A container is usually started with its own process ID (PID) namespace. Inside that namespace, the first process looks as though it has the ID 1. Tracee monitors for new binaries being executed, checking the process ID from the perspective of the process’s PID namespace. Whenever it finds a process that appears to have an ID of 1, Tracee knows that it’s the first process within a new PID namespace, so it starts tracing. Any other process that shares the same PID namespace will also get traced. Tracee will catch child processes within the container, as well as other containers in the same Kubernetes pod which share the PID namespace.
This approach means that any pre-existing containers won’t get traced – you need to run Tracee first and then start the container in order for it to be detected.
Tracee was written by Yaniv Agman in Aqua’s research team, and it’s currently experimental. It supports the tracing of the most common system calls, as well as other events like cap_capable, where the kernel checks if the process has the capabilities it needs.
Check out the README on GitHub for instructions and examples of Tracee’s output
We’d love your feedback on it!
Aqua Security is the largest pure-play cloud native security company, providing customers the freedom to innovate and accelerate their digital transformations. The Aqua Platform provides prevention, detection, and response automation across the entire application lifecycle to secure the build, secure cloud infrastructure and secure running workloads, wherever they are deployed.
Aqua customers are among the world’s largest enterprises in financial services, software, media, manufacturing and retail, with implementations across a broad range of cloud providers and modern technology stacks spanning containers, serverless functions and cloud VMs.