Detecting Drovorub's File Operations Hooking with Tracee

This blog was co-authored by Itamar Maouda

Two years ago, the NSA (the United States' National Security Agency) revealed that Drovorub, an advanced Russian malware created by the GRU 85th GTsSS team, had been discovered targeting Linux systems. Drovorub works by introducing advanced techniques which can manipulate the Linux operation system. It has an advanced kernel rootkit that hooks several kernel functions. In this blog we’ll take a deep dive into a small part of the Drovorub kernel rootkit and examine how it uses hooks to hide processes, files, and network connections. We will then introduce Tracee’s (Aquas’ eBPF open-source Runtime Security and Forensics tool) new features that can alert on those hooks.

How Rootkits Hide Files and Processes

One purpose of a rootkit is to hide itself and the malicious activities performed by the threat actor. Rootkits often aim to hide files, directories, and processes.

The most common way rootkits hide files and directories is by hooking syscalls that are used by the operation system (OS) to list directories.

Since everything in Linux is a file, processes can be hidden in a similar manner. The Linux OS lists processes by iterating over the procfs (/proc) directory where each process is represented by its PID (Process Id) as its own directory. By hiding the PID directories from the procfs, threat actors are able to hide processes.

Below are the syscalls that are used by the OS to list files, directories, and processes:

• getdents
• getdents64
• old_readdir

In a previous blog we wrote about syscall hooking and explained in detail how Diamorphine, a kernel rootkit, uses syscall hooking to hook the getdents and getdents64 syscalls in order to hide files, directories, and processes. To sum this up, threat actors can hide files, directories, and processes in the system by hooking to these 3 syscalls.

How Does Drovorub Rootkit Hide Files and Processes

We began our research by reading the NSA’s advisory which describes how Drovorub works in detail. According to their summary, processes are hidden by hooking d_lookup(), iterate_dir(), or vfs_readdir(). The last two are also used to hide files.

“Hiding processes from the proc filesystem is achieved by hooking multiple kernel functions, which may include d_lookup(), iterate_dir(), or vfs_readdir() depending on the Linux kernel version”  

(Mind that according to the Linux manuals  vfs_readdir()was removed in version 4.1 of the kernel.)
Next, we found a blogpost by Yassine Tioual, aka Nisay, that describes the same technique that is used by Drovorub. This article focuses on iterate_dir as it provides a PoC source code that hooks to the iterate_shared file operation member, which is a function pointer that is used by the iterate_dir kernel function.

In the rest of the blog, we will provide an in-depth analysis of this technique.

When listing files or directories, the following happens:

1. The original execution flow starts with getdents, getdetns64, or old_readdir syscalls.
2. All 3 syscalls call the function iterate_dir.
3. A file_operations struct determines the next stage by holding a flag which indicates whether the target directory is “shared” or not.
4. As mentioned above, the flag determines which function is called by iterate_dir. It either calls iterate_shared or iterate functions.
5. Both of the functions in 4 above use the dir_context struct provided as a parameter to call its “actor” member which is a pointer to a filldir_t function.

Below you can see each step as it appears in the kernel source code.

iterate_dir: In line #45 you can see the shared flag which is set by the f_op member (file_ operations struct) of the directory (file struct). This flag determines whether the iterate _shared function or the iterate function will be called (line #64).

file operations (f_op) struct: Before we move on, let’s further dig into the f_op in the condition function in line #45. The f_op is a member of the file struct and contains pointers to functions that allow common action on files like read, write, and more.

The struct holds pointers to functions where each member represents an action the user may ask to perform. In lines #1973 and #1974 in the screenshot of the struct below, you can see that the functions iterate and iterate_shared are defined. Mind that both receive the same parameters since they are intended to perform the same task of iterating over a directory.

iterate_shared/iterate: The iterate_shared and iterate members of the struct are function pointers. That means that any function could be pointed by them and there are multiple iterate_shared and iterate implementation functions in the kernel according to the listed directory architecture. As an example of how this function is used and what it does we found the file_operations struct of procfs.

This struct led us to the implementation of the iterate_shared function of procfs – proc_readdir (line #344 in the image above), which is used to list procfs directories. This function receives a file struct and a dir_context struct as its parameters. It then calls the proc_readdir_de function with those parameters along with the file system information of the file after some sanity checks.

In the screenshot below, you can see the dir_context struct, which reveals that this structs’ “a member of type filldir_t is a function pointer that is used to specify the requested layout for directory listing. That “actor” function is responsible for producing the list of files in the iteration process of directories. This is wonderful news for an attacker because an attacker can replace the pointer to the filldir_t function and return a modified list of files and directories which are present under the target directory. The modified list could have files or directories removed by the attackers and therefore keep them hidden from the users.

So What Does the Hooking Process Look Like?

1. As before, the original execution flow starts with getdents, getdetns64, or old_readdir syscalls.
2. As before, all 3 syscalls call the function iterate_dir function.
3. The file_operations struct is modified to point to a malicious iterate or iterate_shared function. The flag which determines which function will be called is defined by the presence of the iterate_shared function pointer.
4. As mentioned above, the flag determines which function is called by iterate_dir. When the iterate_dir function is invoked the malicious iterate or iterate_shared will be called accordingly.
5. Either of the two functions in 4 above modifies the “actor” member of the dir_context struct to a pointer that points to a malicious filldir_t function.
6. The malicious filldir_t function calls the original filldir_t function to get the list of files and directories present under the target directory and modifies it as it will be removing files and directories that needs to be hidden.
7. The modified list will be returned to the user mode application that invoked the original syscalls (either getdents, getdetns64, or old_readdir) and be displayed as the content of the target directory.

Below is a snippet from the PoC code that demonstrates the process by overwriting both iterate_shared and filldir_t function pointers then replacing them with new malicious function pointers to list directories.

Detection with Tracee

This new hooking technique poses a great risk and does a great job avoiding detection. Luckily, we added a detection feature to Tracee that creates an alert upon this hooking technique. The hooked_proc_fops event enables Tracee to detect those kinds of hooks at runtime. It works by fetching the address of the file_operations struct of procfs and its function pointers (iterate_shared and iterate) each time someone tries to access a file under /proc. Tracee then compares the addresses to the memory boundary to check if it‘s in the original source of the kernel, as we did in the syscall detection event.

You can run tracee-ebpf with the event by the following command line:
sudo tracee-ebpf -t e=hooked_proc_fops

You can also get an alert if that technique is used by running Tracee

More instructions and documentation are available in Tracees’ GitHub repository.

You can learn more about this technique and other malicious behaviors or kernel rootkits in our BlackHat Arsenal 2022 session.

In Conclusion

As threat actors dwell deeper and deeper within the kernel and its internal functions, they find more ways to perform hooking and hide malicious artifacts. Our goal is to detect those advance methods of obscuring rootkits. Those detections are available both in Tracee open-source and Aquas’ CNDR.


This blog was co-authored by:

   Itamar Maouda
Itamar is a Security Researcher at Team Nautilus, Aqua’s research team. He focuses on researching malware and threats in cloud native environments. Outside of work, Itamar is a professional long-distance runner and BA student at the Open University of Israel.