Over the past few years, attackers have embraced cryptomining as a fast revenue source, easily converting compute power into digital coins. Unlike other types of cybercrime, cryptomining is perceived by the attacker as relatively harmless and reversible, with a low footprint and an immediate payoff. Last year, bad actors switched from attacking unpatched servers to abusing the free-tier offerings of popular cloud CI/CD platforms. In this blog, I’ll share some of the techniques and tactics that attackers use to maximize their gain while maintaining a low footprint on the platform.
How the abuse of CI/CD free plans works
The mining process is effective only if it runs for a long time, in contrast to the stages in a CI/CD pipeline, which are meant to run for short periods. This is why CI/CD platforms have introduced some fail-safe features, such as maximum runtime periods or parallelism limitations. Those features are intended to protect the system from abuse and to ensure that technical bugs won’t lead to infinite runs.
Bad actors stretch these limits by creating multiple build steps that run in parallel within a single plan. As you can see in the screenshot below, each job runs for six hours, which is the maximum runtime allowed by GitHub:
These runs are described by the workflow declaration below:
Techniques used to bypass CI/CD limits and detection
Further research revealed a few unique practices:
- The bad actor set max-parallel to 20, which optimizes the concurrent job up to the limit as mentioned in this GitHub doc.
- Fail-fast is set to false to ensure that if one step fails, the rest would still run.
- The bad actor chose to run on a Windows image with Windows subsystem for Linux (WSL) so that he could run bash scripts and ELF binaries. Apparently, this was meant to create another sandboxing layer to make it harder to audit the platforms and detect these activities.
- The actor used git.io as a short link solution to mask the actual miner address (used ccminer for Ubuntu), which is also stored in a GitHub repository. Again, this practice is intended to keep the activities under the radar in case of a network telemetry audit.
To examine the profit that this actor can achieve, we traced his wallet activity (although it’s not made public by the crypto network) by using a built-in feature in mining pools that allows worker machines to check the current hash power and the pending reward.
As seen below, the actor still has active cryptominers:
In total, they provide a solid reward of 287 Verus coins, which is about $200 as of the current market rate.
More evading tactics against GitHub Actions
In another campaign, targeting gradient.run and GitHub Actions, the actor installed a NPM package called node-process-hider, which aims to hide the mining process by a known dynamic linker hijacking technique:
As part of the build job preparation, installing node modules is common and legitimate. However, the actor used it to avoid dropping any binaries from suspicious domains that might be restricted or audited by the platform providers. The miner itself is also stored in another GitHub repository, which again reduces the chances of being detected.
The actor seems to operate multiple GitHub accounts simultaneously by forking and duplicating the same repositories. We believe the goal is to reduce the noise per account or repository and to have a backup in case some of them get banned.
The risks of CI/CD abuse
Attackers are continually looking for ways to abuse unsecured CI/CD platforms. This can lead to multiple risks:
- Financial loss: By consuming compute time, attackers drain compute tokens, which can result in significant revenue loss.
- Denial of service (DoS): When all build runners are busy with cryptomining, none are left for legitimate CI/CD runs. This can lead to denial of service and slow down your development team.
- Compromise of business-critical assets: With a foothold in your CI/CD environment, attackers can try to pivot to higher-risk assets, such as source code, as happened with Mercedes. They can also leverage secrets that are stored in your CI/CD environment as a path into your artifact repositories, databases, Kubernetes clusters, and APIs — like in the incidents with Travis CI and spell-check.js.
How to protect against CI/CD cryptomining
The good news is that most of the major CI providers are already aware of free-tier abuse and are trying to prevent and block such activity. For example, GitHub hardened the way that forked pull requests could run, GitLab started requiring stronger verification, Azure DevOps stopped offering free compute, and CircleCI banned violated accounts. However, as we saw, bad actors still manage to overcome some of these restrictions.
That’s why it’s crucial to make sure you consistently run a secured software supply chain stack and audit for any anomalies. This will dramatically reduce your attack surface and help detect issues earlier.
Here are some best practices to improve security:
Fix permissive rights and any misconfigurations in your CI/CD platform
- Enforce two-factor authentication for all users.
- Restrict users with permissive access.
- Set branch protection restriction.
- Restrict forked pull requests to run on your CI.
- Restrict CI runner token with permissive access.
Fix known vulnerabilities
- Find and fix any known CVEs in your open source dependencies and artifacts. To scan them for vulnerabilities, you can use the open source scanner Trivy.
- If you run an on-premises source code management solution, make sure to run the latest security version, such as the one recently published by GitLab.