Table of Contents
June 10, 2021
|4m
How Attackers Abused Kubeflow To Run Crypto Mining Campaigns Targeting Kubernetes Clusters
Researchers on 8 June 2021 reported a new large-scale active crypto-mining campaign targeting Kubernetes clusters abusing Kubeflow deployments to run malicious cryptocurrency mining containers.
What Is Kubernetes?
Kubernetes is a portable and open-source container orchestration platform for managing containerized workloads and services. This platform is used to automate the manual processes involved in deploying, managing, and scaling containerized applications. In other words, this platform allows you to create a customized cluster of groups of hosts running Linux containers and helps you easily and efficiently manage those clusters.
What Is Kubeflow?
Kubeflow is an open-source framework developed to run machine learning (ML) workflows on Kubernetes clusters. Watch this video created by Kubeflow to have a better idea of what it is?
What Is TensorFlow?
TensorFlow is an open-source machine learning library. This library has a comprehensive, flexible ecosystem of tools, libraries, and community resources, which can be used mostly in building and training machine learning models. TensorFlow was developed for internal usage by the Google Brain team, and later it was released under the Apache License 2.0 in 2015.
What Do We Know About This New Crypto-Mining Campaign?
It’s been observed a spike in the number of deployments of TensorFlow pods on Kubernetes clusters in May 2021. All these deployments occurred around the same time. This clearly indicates that threat actors have done some homework before they launch the campaign. Probably they might have scanned the clusters and identified the targets to launch attacks.
Fig #1: Kubeflow Dashboard
Initially, attackers have abused publicly exposed Kubeflow dashboards to launch the crypto mining campaign. They have been doing mass deployment of TensorFlow pods on Kubernetes clusters as part of the crypto mining campaign. In this crypto mining campaign, they have been using TensorFlow pods from legitimate TensorFlow images published on the official Docker Hub account to cover the detection. However, cybersecurity researchers revealed that the container images were configured to mine cryptocurrencies.
Primarily, there are two different TensorFlow images were used. The first one is the latest version of TensorFlow (tensorflow/tensorflow:latest), and the second one is the latest version of TensorFlow image with GPU support (tensorflow/tensorflow:latest-gpu).
How Attackers Abused Kubeflow To Run This Crypto Mining Campaign Targeting Kubernetes Clusters?
Fig #2: POC of crypto-mining campaign
Attackers will gain access to the publicly exposed Kubeflow centralized dashboards.
After which, they create a new pipeline (clusters running TensorFlow images which configured to mine cryptocurrencies). “Pipeline is a series of steps, each one of them is an independent container, and together they form an ML workflow. The image of the container that runs in each step is written in the pipeline configuration.”
Each cluster will have two pods, one for CPU mining (tensorflow/tensorflow:latest) and the other for GPU mining (tensorflow/tensorflow:latest-gpu). XMRIG miner is used in CPU and Ethminer miner in GPU containers
As part of this crypto mining campaign, attackers also deployed a reconnaissance container that queries GPU and CPU information from the environment.
Precautions To Take Against This Crypto Mining Campaign:
Avoid exposing the centralized dashboard to the Internet. If your business needs that, use secure channels like a VPN to access the dashboard over the Internet.
Configure authentication to access the centralized dashboard.
Implement SSL certificates to access the dashboard over the web.
Run this command to get all the pods running in the cluster: kubectl get pods –all-namespaces -o json
Inspect all the entry points of the TensorFlow images if you are running now.
At last, calculate the file fingerprints of the TensorFlow images you have downloaded so far. Remove the images if your file fingerprint matches the shared IOCs.
Conclusion:
This is not the first time attackers used legitimate container images for running their malicious code. A similar crypto-mining campaign targeting Kubernetes clusters was reported in June 2020. Please see the detailed report, which clearly described how attackers targeted the Kubeflow dashboard to deploy the malicious container. The crypto mining campaign is still active. New Kubernetes clusters that run Kubeflow are still getting compromised.
Thanks for reading this threat post. We request you to share this with all who are using Kubernetes and Kuberflow in their environment and ask them to take the required measures written in the last section of this post.
You may also like these articles:
Be Aware of This New Windows Container Malware "Siloscape" Targeting Kubernetes Clusters
How Can You Protect Your Linux Infrastructure From XorDdos Malware
What is Redigo Malware? How to Protect Your Redis Server from Redigo Malware?
How DarkRadiation Ransomware Attacks Targets Linux And Docker Instances?
A New MSBuild Fileless Malware Campaign in Which Threat Actors Used MSBuild to Deliver RATs
Arun KL
Arun KL is a cybersecurity professional with 15+ years of experience in IT infrastructure, cloud security, vulnerability management, Penetration Testing, security operations, and incident response. He is adept at designing and implementing robust security solutions to safeguard systems and data. Arun holds multiple industry certifications including CCNA, CCNA Security, RHCE, CEH, and AWS Security.
