The past couple of years have seen a rise in software supply chain attacks, with the most salient example being the Solarwinds attack. As production environments have gained multiple layers of protection, and much of the attention of security teams, malicious actors have set their sights on “poisoning the well”, i.e., target where applications are developed or their building block components. But what does this mean for cloud native? Are cloud native applications more susceptible to supply chain attacks? Let’s try to answer these questions, using examples from Aqua’s Team Nautilus cyber research findings from attacks observed in the wild.
Cloud Native Supply Chain Threats
There are several characteristics of cloud native application development environments that make them a lucrative target for attackers looking to embed malicious code into the supply chain.
First, cloud native application development is characterized by widespread use of open source components, often obtained from public registries. While many open source projects are well-governed and scrutinized under many eyes, some are not, and this allows malicious actors to masquerade as contributors. The higher up in the supply chain that attackers can go, the better their chances of achieving widespread dissemination of their code, which is why the more advanced attackers look for widely used packages that might reappear in many different applications.
Additionally, today’s dev environments are not scrutinized as closely as production environments, and container images, functions and packages are updated frequently using CI/CD pipelines, creating multiple opportunities for attackers to embed themselves into the process. Team Nautilus has detected and analyzed attacks on CI SaaS environments that abused the CI process itself to gain access to cloud CPU time. From there it’s a relatively short hop into the artifacts being built in those CI pipeline.
Observed Attack Vectors
Our cyber research on attacks in the wild that target containers and Kubernetes environments has shown varying levels of sophistication and evasion techniques. Starting from the relatively simple to the more advanced, we’re seeing:
- Placing malicious images in public registries, either for pulling them into compromised environments or in the hope someone will use them.
- Hiding malicious code inside benign, “vanilla” images, in the hope developers will use them not knowing what lurks inside them. We’ve also seen “typo-squatting” used in this regard, where a popular image name is deliberately misspelled – for example tesnorflow instead of tensorflow.
- Evading static analysis of malware by using polymorphic, evolving malware that isn’t recognizable based on its signature, and calls in additional components after an image has been deployed and run as a container.
- Circumventing the controls in the CI/CD pipeline by building images directly on hosts
- Abusing cloud-based CI tools, in which the attackers take advantage of free SaaS offerings to run crypto-currency mining under the guise of building an application.
Figure 1: AVScan attack tree showing gradual branching and malware downloads
(Source: Team Nautilus, Aqua)
At various stages of these attacks, the malicious actors would perform gradual, increasingly severe actions to attain their objectives and establish persistence in the environment – i.e., enable them to continue an attack beyond the specific container, host, or even cluster.
Any Good News in This Scenario?
Thankfully, yes. There are inherent characteristics of cloud native applications that make them more resilient to attack, and enable to limit the damage of an attack that could slip through in a way that’s more robust than with traditional, monolithic applications:
- The high degree of automation in the pipeline allows to embed controls that would potentially detect or at least make it harder for supply chain attacks to succeed (more about that in the next section).
- Due to the orchestrated and often ephemeral nature of cloud native workloads, it is extremely hard to for the attacker to persist. When an attack like the Solarwinds attack happens, and infects a Windows server somewhere, that Windows server might be running with the same permissions, same IP address, and no downtime or upgrades for many months, sometimes even years. By contrast, a serverless function typically runs for seconds, and a maximum of 15 minutes in most cloud environments, which would limit the potential damage.
- Observability in cloud native is vastly superior. Thanks to open source tools like Tracee, we can get very detailed real-time information about every process, every network connection, every event. It’s easier to detect anomalies, and easier to isolate them at a process level, function level, or container level.
How to Protect Against Supply Chain Attacks
Eliminating the risk of supply chain attacks is virtually impossible – but there are measures that DevOps and security teams can take to reduce that risk:
Control access to public registries and open source components
Reduce the risk by limiting the number of people who can access public images and packages, and create a curated internal registry for base images. Implement a process to ensure that only trusted images are used.
Digitally sign images or use other methods of maintaining dev-to-prod image integrity
Ensure that the images you’re deploying are the same ones you vetted and prevent the deployment of images outside the predefined pipeline. In the Aqua platform, we automatically digitally fingerprint and track all scanned images, which allows us to detect and prevent the use of non-compliant or unrecognized images or functions in your environment.
Isolate your dev/staging environments
If your supply chain is successfully penetrated, attackers might try to jump into adjacent environments, even without targeting production environments, for things like admin credentials, sensitive IP, or source-code. Be sure to authenticate access and limit it to specific IP addresses (or VPN). Within the Aqua platform we allow customers to maintain strict separation of duties (SoD) between teams and different roles, as well as use flexible scoping for policies so that different ones can be applied to development versus production environments.
Scan images for malware, using both static and dynamic analysis
While there’s some value scanning images and functions for malware using static, signature- or pattern-based tools, the more sophisticated attacks now evade such techniques by phasing their attacks – embedding small, innocent-looking components in the image, and then downloading and running additional malware only when the image is run as a container.
Such sophisticated attacks are missed by static vulnerability and malware scanning and can only be detected by dynamic analysis of the container behavior as it’s running. You need to use tools like Aqua DTA (Dynamic Threat Analysis) that identify malicious elements in images by running the image in a secure sandbox to observe its behavior.
Automate and embed security controls in the CI/CD pipeline
With the fast pace of release cycles, it is crucial to embed security into your development lifecycle right from the start. To catch a supply chain attack early on, you need to be able to detect unexpected code execution drifts in your pipeline. For this purpose, you can use Tracee, open source runtime security tool, which can also identify malicious activity during the build, thereby helping you secure your applications early in the development lifecycle.
Monitor your runtime environment
If your supply chain is penetrated despite all of the above, this will ultimately show up as abnormal activity in your production environment. Real-time runtime protection should be an element in any strategy intended to protect against both supply chain attacks as well as “normal” attacks on production infrastructure and applications.
Conclusion
The reality of the cloud native software supply chain is that organizations are heavily relying on third-party code, potentially introducing vulnerable and risky components into their applications. This provides plenty of opportunities for attackers to target legitimate software through third parties and embed themselves into the development process. Supply chain attacks are here to stay and evolve, and organizations should adjust their security practices to detect, identify and mitigate them.
Learn how Aqua DTA helps protect against supply chain attacks.