A Brief Guide to Supply Chain Security Best Practices

With the rise in attacks targeting the supply chain of cloud native applications, it’s important to understand how you can prepare for and stifle risks that enter your environments through third-party packages and tools. This post outlines the top software supply chain security best practices that should be included in any organization’s cloud native strategies.

Empower DevOps teams while fostering a DevSecOps culture

This recommendation comes as teams begin working in ways that are often articulated with various versions of the infinity loop diagram that represents DevOps. The key to realizing value from the artistic vision that is the DevOps loop is ensuring clear, accurate, and actionable security insight gets communicated to developers as part of that closed-loop workflow. You might have heard this described as “shifting security left.”

To make wise decisions about the packages and libraries they consume, or the steps they must take to mitigate risks, developers need to be aware of risks and educated on how to avoid or address them. This also encompasses the idea of identifying barriers to efficiency, baking security into established workflows, and getting that security risk information directly into the hands and tools of the people who need to do something about it.

But, as we already know: the path of least resistance is the one most often pursued. Security cannot impede productivity. If it does, it becomes far too easy for individual contributors to task others with the security of projects that are getting pushed into production. When this happens, we’re subject to the tragedy of the commons, and one’s short-sighted self-interest becomes the burden of the team.

Automate and embed security controls in the CI pipeline

It is not uncommon for security automation to be thought of as a result or consequence of some other initiative. In fact, embedding security to the extent at which it can be automated (at least in part) is a deliberate process, and it doesn’t just happen by changing a setting on some of the tools that Dev teams, Security, or DevOps are using.

It’s important to acknowledge that many security professionals express some concern about automation, commonly around concerns of sacrificing control or over-engineering conditions that impede workflows or permit security issues to make their way through the pipeline. But these two ideas – security and automation – can work in concert when you have the appropriate tools and success/failure criteria in place to act as the security team’s proxy in automated systems.

Recommendation
Establish proper policies and controls to facilitate secure automation, being cautious not to craft overly specific or generalized conditions for those policies to take effect. Craft these with scalability in mind and clearly define consequent steps that must happen upon conditional failure – this is critical to scaling that control and for compliance to keep up with faster shipping cycles.

Establish risk tolerance and security standards for the supply chain

When crafting policies and controls, it is important to engage all relevant stakeholders – those who assess security risks, those who determine failure to comply with policies, those who must fix security risks, and those responsible for security in production environments. Together, these parties should define the security risk tolerance for the software artifacts moving through the pipeline to detect, prioritize, and fix security risks before they become issues in production.

When considering the collection of artifacts, packages, and libraries that come in via the software supply chain, however, this exercise can become significantly more complicated.

This is because most organizations relinquish some level of control to the supply chain in exchange for greater agility, scalability, or efficiency. Risk identification may not be possible by traditional application security testing methods for resources ingested from third parties, and remediation may not always be a task that your in-house development teams can achieve. Therefore, your risk tolerance and security standards for supply chain packages will probably differ from what you have for custom code, for example.

Recommendation
Facilitate a conversation with contributors and security stakeholders. Provide prompts like: “What’s the review process for risks in code vs. risks that enter through the supply chain?” and “Which response and remediation steps happen automatically, and who is notified?”
When the software supply chain is involved, the answers to those two may involve people outside your team or organization, so your risk tolerance and consequent activities should reflect that. Issue resolution, in these scenarios, might only come as a result of a longer process of notifying the vendor of an issue and may be subject to their SLAs for response.

Eliminate “trust” from the security strategy

You might recall from our previous blog about supply chain attacks some examples of how trust is being used as an attack vector or an entry point for exploitation. Despite being able to provide vulnerability information to developers and integrate security testing across the SDLC and CI/CD pipelines, the software supply chain introduces a lot of additional potential points of failure, where those security procedures may not be invoked or may be ineffective.

Recommendation
Engender the understanding across your organization that a trusted vendor might not approach software security with the same rigor and standards that you ascribe to your own internal Security, DevOps, and Dev teams. You don’t want your success or failure to depend on the security adequacy of someone else, and your cloud native security toolkit and DevSecOps processes need to reflect this. Only once this is understood by all internal stakeholders, can the three previous stages have the greatest chance at being effective and consistent.

How to structure a software supply chain security plan

These four stages can coalesce into a sound approach to addressing software supply chain risks in cloud native applications. This, ultimately, means expanding your security approach to include the types of sophisticated attacks we discussed previously so you can trace and visualize the attack kill chain before an attack is carried out.

In general, for the greatest chance at success, this is going to require running the software. Of course, you don’t want the first time you run this software to be in production. Run the containers in a secure sandbox environment and be certain that you have a security solution prepped and ready to go to identify and track malicious behaviors that only manifest at runtime.

The malicious or anomalous activity you detect should trigger precautionary measures to block the malicious artifacts from being deployed into production and start that cascade of notification and response. It is important to automate this as part of the build process or CI/CD pipeline whenever possible, that way everything is being tested in pre-production without adding hurdles for your security and DevOps teams or increasing the risk of missing a shipping deadline for unnecessary reasons (if no risks are detected).

With this setup, organizations can carry out standard workflows and then get a detailed report at the end of malicious, risky, or anomalous behaviors and artifacts discovered during analysis. The results of this analysis get shared with anyone who needs that insight to take appropriate action. In keeping with best practices, be sure that whatever security solution you implement for this can support policies, which will facilitate automation requirements and still give you control over security risk exposure.

Remember that the standards and risk tolerances that are baked into policies for supply chain security risks may be different from those of traditional software vulnerabilities (e.g., CVEs from open source components developers select, CWEs in custom code), due to the lack of control over remediation for risks ingested via the software supply chain.

Additionally, it’s important to examine all the behaviors and actions that occur when the application is running in the pre-production sandbox. Identify all indicators of compromise and kill chain activities – of course, because they’re being revealed in a secure non-production sandbox, you will be able to observe and document them without risking sensitive production assets. Anything being used in the container is going to have some sort of behavior, and that behavior might be malicious; that’s just an inherent risk of today’s software supply chains.

Lastly, support SecOps and forensics by automatically classifying those detected behaviors by standards like the MITRE ATT@CK framework. This is going to help assimilate the insight you gather into the greater story of your organization’s software security risk posture, so you’re not comparing apples to oranges when your security team looks at the risk profiles of your cloud native apps, legacy apps, internal software, and commercial technologies in aggregate.

Including an automated mechanism – such as Aqua’s Dynamic Threat Analysis – that accomplishes these best practices into your cloud native security strategy helps to drastically reduce your exposure to supply chain risks without adding additional burden on your security or DevOps teams. This helps to:

• Avoid surprises for your Ops teams who must attest to the security of software in production,
• Satisfy Security teams’ needs to fully evaluate risk,
• Allay some of the concerns for Developers when sourcing third-party or vendor-supplied packages,
• Establish an impactful way to support forensics requirements and compliance standards.

For more on this topic, you can check out the full discussion of Supply Chain Security in the Cloud Native Stack in an on-demand webinar.