www.darkreading.com Open in urlscan Pro
2606:4700::6810:ddab  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Dark Reading is part of the Informa Tech Division of Informa PLC
Informa PLC|ABOUT US|INVESTOR RELATIONS|TALENT
This site is operated by a business or businesses owned by Informa PLC and all
copyright resides with them. Informa PLC's registered office is 5 Howick Place,
London SW1P 1WG. Registered in England and Wales and Scotlan. Number 8860726.

Black Hat NewsOmdia Cybersecurity

Newsletter Sign-Up

Newsletter Sign-Up

Cybersecurity Topics

RELATED TOPICS

 * Application Security
 * Cybersecurity Careers
 * Cloud Security
 * Cyber Risk
 * Cyberattacks & Data Breaches
 * Cybersecurity Analytics
 * Cybersecurity Operations
 * Data Privacy
 * Endpoint Security
 * ICS/OT Security

 * Identity & Access Mgmt Security
 * Insider Threats
 * IoT
 * Mobile Security
 * Perimeter
 * Physical Security
 * Remote Workforce
 * Threat Intelligence
 * Vulnerabilities & Threats


World

RELATED TOPICS

 * DR Global

 * Middle East & Africa

See All
The Edge
DR Technology
Events

RELATED TOPICS

 * Upcoming Events
 * Podcasts

 * Webinars

SEE ALL
Resources

RELATED TOPICS

 * Library
 * Newsletters
 * Podcasts
 * Reports
 * Videos
 * Webinars
 * Whitepapers

 * 
 * 
 * 
 * 
 * Partner Perspectives:
 * > Microsoft

SEE ALL


 * Сloud Security
 * Cyberattacks & Data Breaches
 * Threat Intelligence
 * Vulnerabilities & Threats


CRITICAL FLAW IN REPLICATE AI PLATFORM EXPOSES PROPRIETARY DATA

The finding underscores the challenges of protecting data from multiple
customers across AI-as-a-service solutions, especially in environments that run
AI models from untrusted sources.

Elizabeth Montalbano, Contributing Writer

May 23, 2024

4 Min Read
Source: Ole CNX via Shutterstock


A critical vulnerability in the Replicate AI platform could have allowed
attackers to execute a malicious AI model within the platform for a cross-tenant
attack — allowing access to the private AI models of customers and potentially
exposing proprietary knowledge or sensitive data.

Researchers at Wiz discovered the flaw as part of a series of partnerships with
AI-as-a-service providers to investigate the security of their platforms. The
discovery of the flaw demonstrates the difficulty of tenant separation across
AI-as-a-service solutions, especially in environments that run AI models from
untrusted sources.



"Exploitation of this vulnerability would have allowed unauthorized access to
the AI prompts and results of all Replicate's platform customers," and
potentially alter those results, Wiz's Shir Tamari and Sagi Tzadik wrote in a
blog post published today. Previously, Wiz researchers found flaws that led to a
similar outcome in the HuggingFace AI platform.

"As we saw in the results of our work with Hugging Face and now in Replicate,
two leading AI-as-a-service providers, when running AI models in cloud
environments, it is crucial to remember that AI models are actually code," Ami
Luttwak, Wiz CTO and co-founder, tells Dark Reading. "Like all code, the origin
must be verified, and content-scanned for malicious payloads."



Indeed, the flaw presents an immediate threat to AI-as-a-service providers, who
often allow their customers to execute untrusted code in the form of AI models
in shared environments - where there is other customers' data. It also can
impact AI teams, who could be affected when they adopt AI models from untrusted
sources and run them on their workstation or company servers, the researchers
noted.



Wiz Research responsibly disclosed the vulnerability to AI model-sharing vendor
Replicate in January 2023; the company promptly mitigated the flaw so that no
customer data was compromised. At this time, no further action is required by
customers.


EXPLOITING THE FLAW

The flaw lies in achieving remote code execution on Replicate's platform by
creating a malicious container in the Cog format, which is a proprietary format
used to containerize models on Replicate. After containerizing a model using
Cog, users can upload the resulting image to Replicate's platform and start
interacting with it.

Wiz researchers created a malicious Cog container and uploaded it to the
platform and then, with root privileges, used it to execute code on the
Replicate infrastructure.  



"We suspect this code-execution technique is a pattern, where companies and
organizations run AI models from untrusted sources, even though these models are
code that could potentially be malicious," the researchers wrote in the post. A
similar technique was used to exploit flaws found on the HuggingFace platform.

This exploitation allowed the researchers to investigate the environment move
laterally out and ultimately outside of the node on which they were running,
which was inside a Kubernetes cluster hosted on Google Cloud Platform. Though
the process was challenging, they eventually were able to conduct a cross-tenant
attack that allowed them to query other models and even modify the output of
those models.

"The exploitation of this vulnerability would have posed significant risks to
both the Replicate platform and its users," the researchers wrote. "An attacker
could have queried the private AI models of customers, potentially exposing
proprietary knowledge or sensitive data involved in the model training process.
Additionally, intercepting prompts could have exposed sensitive data, including
personally identifiable information (PII)."



Indeed, this ability to alter prompts and responses of an AI model poses a
severe threat to the functionality of AI applications, giving attackers a way to
manipulate AI behavior and compromise the decision-making processes of these
models.

"Such actions directly threaten the accuracy and reliability of AI-driven
outputs, undermining the integrity of automated decisions and potentially having
far-reaching consequences for users dependent on the compromised models," the
researchers wrote.


NEW FORMS OF MITIGATION REQUIRED

Currently there is no easy way to validate a model's authenticity, or to scan it
for threats, so malicious AI models present a new attack surface for defenders
that needs other forms of mitigation, Luttwak says.

The best way to do this is to ensure that production workloads only use AI
models in secure formats, like so-called safetensors. "We recommend that
security teams monitor for usage of unsafe models and work with their AI teams
to transition into safetensors or similar formats," he says.

Using only safe AI formats can the attack surface "dramatically," as "these
formats are designed to prevent attackers from taking over the AI model
instance," Luttwak says.  

Further, cloud providers who run their customers' models in a shared environment
should enforce tenant-isolation practices to ensure that a potential attacker
who managed to execute a malicious model cannot access the data of other
customers or the service itself, he adds.




ABOUT THE AUTHOR(S)

Elizabeth Montalbano, Contributing Writer



Elizabeth Montalbano is a freelance writer, journalist, and therapeutic writing
mentor with more than 25 years of professional experience. Her areas of
expertise include technology, business, and culture. Elizabeth previously lived
and worked as a full-time journalist in Phoenix, San Francisco, and New York
City; she currently resides in a village on the southwest coast of Portugal. In
her free time, she enjoys surfing, hiking with her dogs, traveling, playing
music, yoga, and cooking.


See more from Elizabeth Montalbano, Contributing Writer
Keep up with the latest cybersecurity threats, newly discovered vulnerabilities,
data breach information, and emerging trends. Delivered daily or weekly right to
your email inbox.

Subscribe

You May Also Like

--------------------------------------------------------------------------------

Сloud Security

Fortinet, Ivanti Keep Customers Busy With Yet More Critical Bugs
Сloud Security

Google Kubernetes Clusters Suffer Widespread Exposure to External Attackers
Сloud Security

CISA: AWS, Microsoft 365 Accounts Under Active 'Androxgh0st' Attack
Сloud Security

Attackers Target Microsoft Accounts to Weaponize OAuth Apps
More Insights
Webinars

 * Extending Access Management: Securing Access for all Identities, Devices, and
   Applications
   
   June 4, 2024

 * Assessing Software Supply Chain Risk
   
   June 6, 2024

 * Preventing Attackers From Wandering Through Your Enterprise Infrastructure
   
   June 19, 2024

More Webinars
Events

 * Black Hat USA - August 3-8 - Learn More
   
   August 3, 2024

 * Cybersecurity's Hottest New Technologies: What You Need To Know
   
   March 21, 2024

More Events



EDITOR'S CHOICE

A hooded white ghost in the dark woods
Cyberattacks & Data Breaches
Novel EDR-Killing 'GhostEngine' Malware Is Built for StealthNovel EDR-Killing
GhostEngine Malware Is Built for Stealth
byElizabeth Montalbano, Contributing Writer
May 22, 2024
4 Min Read

database connection security hand finger pointing
Cybersecurity Analytics
Picking the Right Database Tech for Cybersecurity DefensePicking the Right
Database Tech for Cybersecurity Defense
byRobert Lemos, Contributing Writer
May 21, 2024
5 Min Read
Digital orb graphic
Cybersecurity Operations
Chinese 'ORB' Networks Conceal APTs, Render Static IoCs IrrelevantChinese 'ORB'
Networks Conceal APTs, Render Static IoCs Irrelevant
byNate Nelson, Contributing Writer
May 22, 2024
5 Min Read

Reports

 * Elastic named a Leader in The Forrester Wave™: Security Analytics Platforms,
   Q4 2022

 * 2023 Global Threat Report

 * EMA: AI at your fingertips: How Elastic AI Assistant simplifies cybersecurity

 * Industrial Networks in the Age of Digitalization

 * How Enterprises Assess Their Cyber-Risk

More Reports
White Papers

 * Leveling Up Cyber-Threat Intelligence Maturity for More Value and Better
   Insights

 * Generative AI Gifts

 * ESG E-Book: Taking a Holistic Approach to Securing Cloud-Native Application
   Development

 * The Cloud Threat Landscape: Security learnings from analyzing 500+ cloud
   environments

 * The Future of Cloud Security: Attack Paths & Graph-based Technology

More Whitepapers
Events

 * Black Hat USA - August 3-8 - Learn More
   
   August 3, 2024

 * Cybersecurity's Hottest New Technologies: What You Need To Know
   
   March 21, 2024

More Events





DISCOVER MORE WITH INFORMA TECH

Black HatOmdia

WORKING WITH US

About UsAdvertiseReprints

JOIN US


Newsletter Sign-Up

FOLLOW US



Copyright © 2024 Informa PLC Informa UK Limited is a company registered in
England and Wales with company number 1072954 whose registered office is 5
Howick Place, London, SW1P 1WG.

Home|Cookie Policy|Privacy|Terms of Use

Cookies Button


WE CARE ABOUT YOUR PRIVACY

We and our 830 partners store and/or access information on a device, such as
unique IDs in cookies to process personal data. You may accept or manage your
choices by clicking below, including your right to object where legitimate
interest is used, or at any time in the privacy policy page. These choices will
be signaled to our partners and will not affect browsing data.Privacy Policy


WE AND OUR PARTNERS PROCESS DATA TO PROVIDE:

Use precise geolocation data. Actively scan device characteristics for
identification. Store and/or access information on a device. Personalised
advertising and content, advertising and content measurement, audience research
and services development. List of Partners (vendors)

I Accept
Essential Only
Show Purposes



ABOUT YOUR PRIVACY

We process your data to deliver content or advertisements and measure the
delivery of such content or advertisements to extract insights about our
website. We share this information with our partners on the basis of consent and
legitimate interest. You may exercise your right to consent or object to a
legitimate interest, based on a specific purpose below or at a partner level in
the link under each purpose. These choices will be signaled to our vendors
participating in the Transparency and Consent Framework.
More information
Allow All


MANAGE CONSENT PREFERENCES

PERFORMANCE COOKIES

Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and
improve the performance of our site. They help us to know which pages are the
most and least popular and see how visitors move around the site.    All
information these cookies collect is aggregated and therefore anonymous. If you
do not allow these cookies we will not know when you have visited our site, and
will not be able to monitor its performance.

Cookies Details‎

FUNCTIONAL COOKIES

Functional Cookies

These cookies enable the website to provide enhanced functionality and
personalisation. They may be set by us or by third party providers whose
services we have added to our pages.    If you do not allow these cookies then
some or all of these services may not function properly.

Cookies Details‎

STRICTLY NECESSARY COOKIES

Always Active

These cookies are necessary for the website to function and cannot be switched
off in our systems. They are usually only set in response to actions made by you
which amount to a request for services, such as setting your privacy
preferences, logging in or filling in forms.    You can set your browser to
block or alert you about these cookies, but some parts of the site will not then
work. These cookies do not store any personally identifiable information.

Cookies Details‎

TARGETING COOKIES

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may
be used by those companies to build a profile of your interests and show you
relevant adverts on other sites.    They do not store directly personal
information, but are based on uniquely identifying your browser and internet
device. If you do not allow these cookies, you will experience less targeted
advertising.

Cookies Details‎

STORE AND/OR ACCESS INFORMATION ON A DEVICE 670 PARTNERS CAN USE THIS PURPOSE

Store and/or access information on a device

Cookies, device or similar online identifiers (e.g. login-based identifiers,
randomly assigned identifiers, network based identifiers) together with other
information (e.g. browser type and information, language, screen size, supported
technologies etc.) can be stored or read on your device to recognise it each
time it connects to an app or to a website, for one or several of the purposes
presented here.

List of IAB Vendors‎ | View Illustrations 

PERSONALISED ADVERTISING AND CONTENT, ADVERTISING AND CONTENT MEASUREMENT,
AUDIENCE RESEARCH AND SERVICES DEVELOPMENT 794 PARTNERS CAN USE THIS PURPOSE

Personalised advertising and content, advertising and content measurement,
audience research and services development

 * USE LIMITED DATA TO SELECT ADVERTISING 606 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Advertising presented to you on this service can be based on limited data,
   such as the website or app you are using, your non-precise location, your
   device type or which content you are (or have been) interacting with (for
   example, to limit the number of times an ad is presented to you).
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * CREATE PROFILES FOR PERSONALISED ADVERTISING 492 PARTNERS CAN USE THIS
   PURPOSE
   
   Switch Label
   
   Information about your activity on this service (such as forms you submit,
   content you look at) can be stored and combined with other information about
   you (for example, information from your previous activity on this service and
   other websites or apps) or similar users. This is then used to build or
   improve a profile about you (that might include possible interests and
   personal aspects). Your profile can be used (also later) to present
   advertising that appears more relevant based on your possible interests by
   this and other entities.
   
   View Illustrations 

 * USE PROFILES TO SELECT PERSONALISED ADVERTISING 487 PARTNERS CAN USE THIS
   PURPOSE
   
   Switch Label
   
   Advertising presented to you on this service can be based on your advertising
   profiles, which can reflect your activity on this service or other websites
   or apps (like the forms you submit, content you look at), possible interests
   and personal aspects.
   
   View Illustrations 

 * CREATE PROFILES TO PERSONALISE CONTENT 217 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information about your activity on this service (for instance, forms you
   submit, non-advertising content you look at) can be stored and combined with
   other information about you (such as your previous activity on this service
   or other websites or apps) or similar users. This is then used to build or
   improve a profile about you (which might for example include possible
   interests and personal aspects). Your profile can be used (also later) to
   present content that appears more relevant based on your possible interests,
   such as by adapting the order in which content is shown to you, so that it is
   even easier for you to find content that matches your interests.
   
   View Illustrations 

 * USE PROFILES TO SELECT PERSONALISED CONTENT 190 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Content presented to you on this service can be based on your content
   personalisation profiles, which can reflect your activity on this or other
   services (for instance, the forms you submit, content you look at), possible
   interests and personal aspects. This can for example be used to adapt the
   order in which content is shown to you, so that it is even easier for you to
   find (non-advertising) content that matches your interests.
   
   View Illustrations 

 * MEASURE ADVERTISING PERFORMANCE 707 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information regarding which advertising is presented to you and how you
   interact with it can be used to determine how well an advert has worked for
   you or other users and whether the goals of the advertising were reached. For
   instance, whether you saw an ad, whether you clicked on it, whether it led
   you to buy a product or visit a website, etc. This is very helpful to
   understand the relevance of advertising campaigns.
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * MEASURE CONTENT PERFORMANCE 357 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information regarding which content is presented to you and how you interact
   with it can be used to determine whether the (non-advertising) content e.g.
   reached its intended audience and matched your interests. For instance,
   whether you read an article, watch a video, listen to a podcast or look at a
   product description, how long you spent on this service and the web pages you
   visit etc. This is very helpful to understand the relevance of
   (non-advertising) content that is shown to you.
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * UNDERSTAND AUDIENCES THROUGH STATISTICS OR COMBINATIONS OF DATA FROM
   DIFFERENT SOURCES 447 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Reports can be generated based on the combination of data sets (like user
   profiles, statistics, market research, analytics data) regarding your
   interactions and those of other users with advertising or (non-advertising)
   content to identify common characteristics (for instance, to determine which
   target audiences are more receptive to an ad campaign or to certain
   contents).
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * DEVELOP AND IMPROVE SERVICES 532 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information about your activity on this service, such as your interaction
   with ads or content, can be very helpful to improve products and services and
   to build new products and services based on user interactions, the type of
   audience, etc. This specific purpose does not include the development or
   improvement of user profiles and identifiers.
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * USE LIMITED DATA TO SELECT CONTENT 120 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Content presented to you on this service can be based on limited data, such
   as the website or app you are using, your non-precise location, your device
   type, or which content you are (or have been) interacting with (for example,
   to limit the number of times a video or an article is presented to you).
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

List of IAB Vendors‎

USE PRECISE GEOLOCATION DATA 258 PARTNERS CAN USE THIS SPECIAL FEATURE

Use precise geolocation data

With your acceptance, your precise location (within a radius of less than 500
metres) may be used in support of the purposes explained in this notice.

List of IAB Vendors‎

ACTIVELY SCAN DEVICE CHARACTERISTICS FOR IDENTIFICATION 119 PARTNERS CAN USE
THIS SPECIAL FEATURE

Actively scan device characteristics for identification

With your acceptance, certain characteristics specific to your device might be
requested and used to distinguish it from other devices (such as the installed
fonts or plugins, the resolution of your screen) in support of the purposes
explained in this notice.

List of IAB Vendors‎

ENSURE SECURITY, PREVENT AND DETECT FRAUD, AND FIX ERRORS 504 PARTNERS CAN USE
THIS SPECIAL PURPOSE

Always Active

Your data can be used to monitor for and prevent unusual and possibly fraudulent
activity (for example, regarding advertising, ad clicks by bots), and ensure
systems and processes work properly and securely. It can also be used to correct
any problems you, the publisher or the advertiser may encounter in the delivery
of content and ads and in your interaction with them.

List of IAB Vendors‎ | View Illustrations 

DELIVER AND PRESENT ADVERTISING AND CONTENT 492 PARTNERS CAN USE THIS SPECIAL
PURPOSE

Always Active

Certain information (like an IP address or device capabilities) is used to
ensure the technical compatibility of the content or advertising, and to
facilitate the transmission of the content or ad to your device.

List of IAB Vendors‎ | View Illustrations 

MATCH AND COMBINE DATA FROM OTHER DATA SOURCES 351 PARTNERS CAN USE THIS FEATURE

Always Active

Information about your activity on this service may be matched and combined with
other information relating to you and originating from various sources (for
instance your activity on a separate online service, your use of a loyalty card
in-store, or your answers to a survey), in support of the purposes explained in
this notice.

List of IAB Vendors‎

LINK DIFFERENT DEVICES 327 PARTNERS CAN USE THIS FEATURE

Always Active

In support of the purposes explained in this notice, your device might be
considered as likely linked to other devices that belong to you or your
household (for instance because you are logged in to the same service on both
your phone and your computer, or because you may use the same Internet
connection on both devices).

List of IAB Vendors‎

IDENTIFY DEVICES BASED ON INFORMATION TRANSMITTED AUTOMATICALLY 477 PARTNERS CAN
USE THIS FEATURE

Always Active

Your device might be distinguished from other devices based on information it
automatically sends when accessing the Internet (for instance, the IP address of
your Internet connection or the type of browser you are using) in support of the
purposes exposed in this notice.

List of IAB Vendors‎
Back Button


COOKIE LIST



Search Icon
Filter Icon

Clear
checkbox label label
Apply Cancel
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label

Confirm My Choices