www.darkreading.com Open in urlscan Pro
2606:4700::6810:dfab  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Dark Reading is part of the Informa Tech Division of Informa PLC
Informa PLC|ABOUT US|INVESTOR RELATIONS|TALENT
This site is operated by a business or businesses owned by Informa PLC and all
copyright resides with them. Informa PLC's registered office is 5 Howick Place,
London SW1P 1WG. Registered in England and Wales and Scotlan. Number 8860726.

Black Hat NewsOmdia Cybersecurity

Newsletter Sign-Up

Newsletter Sign-Up

Cybersecurity Topics

RELATED TOPICS

 * Application Security
 * Cybersecurity Careers
 * Cloud Security
 * Cyber Risk
 * Cyberattacks & Data Breaches
 * Cybersecurity Analytics
 * Cybersecurity Operations
 * Data Privacy
 * Endpoint Security
 * ICS/OT Security

 * Identity & Access Mgmt Security
 * Insider Threats
 * IoT
 * Mobile Security
 * Perimeter
 * Physical Security
 * Remote Workforce
 * Threat Intelligence
 * Vulnerabilities & Threats


World

RELATED TOPICS

 * DR Global

 * Middle East & Africa

See All
The Edge
DR Technology
Events

RELATED TOPICS

 * Upcoming Events
 * Podcasts

 * Webinars

SEE ALL
Resources

RELATED TOPICS

 * Library
 * Newsletters
 * Podcasts
 * Reports
 * Videos
 * Webinars
 * Whitepapers

 * 
 * 
 * 
 * 
 * Partner Perspectives:
 * > Microsoft

SEE ALL


 * Cyberattacks & Data Breaches


RESEARCHER TRICKS CHATGPT INTO BUILDING UNDETECTABLE STEGANOGRAPHY MALWARE

Using only ChatGPT prompts, a Forcepoint researcher convinced the AI to create
malware for finding and exfiltrating specific documents, despite its directive
to refuse malicious requests.

Jai Vijayan, Contributing Writer

April 5, 2023

4 Min Read
Source: Rokas_Tenys via Shutterstock


A security researcher has tricked ChatGPT into building sophisticated
data-stealing malware that signature and behavior-based detection tools won't be
able to spot — eluding the chatbot's anti-malicious-use protections.

Without writing a single line of code, the researcher, who admits he has no
experience developing malware, walked ChatGPT through multiple, simple prompts
that ultimately yielded a malware tool capable of silently searching a system
for specific documents, breaking up and inserting those documents into image
files, and shipping them out to Google Drive.



In the end, all it took was about four hours from the initial prompt into
ChatGPT to having a working piece of malware with zero detections on Virus
Total, says Aaron Mulgrew, solutions architect at Forcepoint and one of the
authors of the malware.


BUSTING CHATGPT'S GUARDRAILS

Mulgrew says the reason for his exercise was to show how easy it is for someone
to get past the guardrails that ChatGPT has in place to create malware that
normally would require substantial technical skills.



"ChatGPT didn't uncover a new, novel exploit," Mulgrew says. "But it did work
out, with the prompts I had sent to it, how to minimize the footprint to the
current detection tools out there today. And that is significant."

Interestingly (or worryingly), the AI-powered chatbot seemed to understand the
purpose of obfuscation even though the prompts did not explicitly mention
detection evasion, Mulgrew says.

This latest demonstration adds to the rapidly growing body of research in recent
months that has highlighted security issues around OpenAI's ChatGPT large
language model (LLM). The concerns include everything from ChatGPT dramatically
lowering the bar to malware writing and adversaries using it to create
polymorphic malware to attackers using it as bait in phishing scams and
employees cutting and pasting corporate data into it.



Some contrarians have questioned whether the worries are overhyped. And others,
including Elon Musk, an early investor in OpenAI, and many industry
luminaries, have even warned that future, more powerful AIs (like the next
version of the platform that ChatGPT is based on) could quite literally take
over the world and threaten human existence.




PROMPTING MALICIOUS CODE INTO CHATGPT

Mulgrew's research is likely to do little to calm those who see AI tools as
posing a major security risk. In a Forcepoint blog post this week, Mulgrew
provided a step-by-step description of how he coaxed ChatGPT into building a
full-fledged malware tool starting with an initial request to generate code that
would qualify as malware.

When ChatGPT's content filter predictably denied that request, Mulgrew decided
to take an approach where he would try and get the AI tool to generate small
snippets of code which, when put together, would function as data-stealing
malware.

His first successful prompt was when he got ChatGPT to generate code that would
search for PNG image files larger than 5MB on the local disk. Using that code,
he then asked ChatGPT for additional code for encoding any discovered PNGs with
steganography. It was a prompt to which ChatGPT responded by providing a call to
readily available steganographic library on GitHub. 

Using a series of other prompts, Mulgrew then got ChatGPT to generate additional
code to look for and find Word and PDF documents on the local disk. He then
figured out a way to get ChatGPT to write code for breaking up files larger than
1MB into smaller chunks, inserting them into the PNGs, and using steganography
to hide them. 



The final piece was getting the chatbot to write code for uploading the data to
an external Google drive account — Mulgrew successfully tricked the AI into
creating malware despite its training to refuse malicious requests.


ZERO DETECTIONS ON VIRUS TOTAL

To test if malware detection tools would flag the ChatGPT-generated code as
malicious, Mulgrew uploaded the code to Virus Total. He found that five vendors
out of 60 marked the file as suspicious. After figuring out the issue might have
to do with how the ChatGPT code called the steganographic library, Mulgrew
asked the chatbot to tweak the code, after which only two vendor products
flagged it as suspicious. After some further tweaking, he finally ended up with
code that no products on VirusTotal detected.

For initial infiltration, Forcepoint researchers asked ChatGPT to create a SCR
file or screensaver file and embed the executable inside it under the disguise
of additional "ease of use" for everyday business applications, Mulgrew says. 

"ChatGPT happily generated step-by-step instructions on how I could do that and
configure the SCR file to auto launch the executable." While the method is not
unique, it was interesting that ChatGPT generated the content without Forcepoint
researchers having to find ways to bypass its guardrails, he says.

Mulgrew says it's almost certain that ChatGPT would generate different code for
similar prompts meaning that a threat actor would relatively easily be able to
spin up new variants of such tools. He says that based on his experience, a
threat actor would need little more than basic knowledge of how to write malware
to get past ChatGPT's anti-malware restrictions. 

"I don't write malware or conduct penetration tests as part of my job and
looking at this is only a hobby for me," he says. "So, I'd definitely put myself
more in the beginner/novice category than expert hacker."




ABOUT THE AUTHOR(S)

Jai Vijayan, Contributing Writer



Jai Vijayan is a seasoned technology reporter with over 20 years of experience
in IT trade journalism. He was most recently a Senior Editor at Computerworld,
where he covered information security and data privacy issues for the
publication. Over the course of his 20-year career at Computerworld, Jai also
covered a variety of other technology topics, including big data, Hadoop,
Internet of Things, e-voting, and data analytics. Prior to Computerworld, Jai
covered technology issues for The Economic Times in Bangalore, India. Jai has a
Master's degree in Statistics and lives in Naperville, Ill.

See more from Jai Vijayan, Contributing Writer
Keep up with the latest cybersecurity threats, newly discovered vulnerabilities,
data breach information, and emerging trends. Delivered daily or weekly right to
your email inbox.

Subscribe

You May Also Like

--------------------------------------------------------------------------------

Cyberattacks & Data Breaches

First BofA, Now Fidelity: Same Vendor Behind Third-Party Breaches
Cyberattacks & Data Breaches

Amex Customer Data Exposed in Third-Party Breach
Cyberattacks & Data Breaches

Pharmacy Delays Across US Blamed on Nation-State Hackers
Cyberattacks & Data Breaches

macOS Malware Campaign Showcases Novel Delivery Technique
More Insights
Webinars

 * Extending Access Management: Securing Access for all Identities, Devices, and
   Applications
   
   June 4, 2024

 * Assessing Software Supply Chain Risk
   
   June 6, 2024

 * Preventing Attackers From Wandering Through Your Enterprise Infrastructure
   
   June 19, 2024

More Webinars
Events

 * Black Hat USA - August 3-8 - Learn More
   
   August 3, 2024

 * Cybersecurity's Hottest New Technologies: What You Need To Know
   
   March 21, 2024

More Events



EDITOR'S CHOICE

A hooded white ghost in the dark woods
Cyberattacks & Data Breaches
Novel EDR-Killing 'GhostEngine' Malware Is Built for StealthNovel EDR-Killing
GhostEngine Malware Is Built for Stealth
byElizabeth Montalbano, Contributing Writer
May 22, 2024
4 Min Read

database connection security hand finger pointing
Cybersecurity Analytics
Picking the Right Database Tech for Cybersecurity DefensePicking the Right
Database Tech for Cybersecurity Defense
byRobert Lemos, Contributing Writer
May 21, 2024
5 Min Read
Digital orb graphic
Cybersecurity Operations
Chinese 'ORB' Networks Conceal APTs, Render Static IoCs IrrelevantChinese 'ORB'
Networks Conceal APTs, Render Static IoCs Irrelevant
byNate Nelson, Contributing Writer
May 22, 2024
5 Min Read

Reports

 * Elastic named a Leader in The Forrester Wave™: Security Analytics Platforms,
   Q4 2022

 * 2023 Global Threat Report

 * Industrial Networks in the Age of Digitalization

 * Zero-Trust Adoption Driven by Data Protection

 * How Enterprises Assess Their Cyber-Risk

More Reports
White Papers

 * Leveling Up Cyber-Threat Intelligence Maturity for More Value and Better
   Insights

 * Purple AI Datasheet

 * Shining a light in the dark: observability and security, a SANS profile

 * EMA: AI at your fingertips: How Elastic AI Assistant simplifies cybersecurity

 * Cisco Panoptica for Simplified Cloud-Native Application Security

More Whitepapers
Events

 * Black Hat USA - August 3-8 - Learn More
   
   August 3, 2024

 * Cybersecurity's Hottest New Technologies: What You Need To Know
   
   March 21, 2024

More Events





DISCOVER MORE WITH INFORMA TECH

Black HatOmdia

WORKING WITH US

About UsAdvertiseReprints

JOIN US


Newsletter Sign-Up

FOLLOW US



Copyright © 2024 Informa PLC Informa UK Limited is a company registered in
England and Wales with company number 1072954 whose registered office is 5
Howick Place, London, SW1P 1WG.

Home|Cookie Policy|Privacy|Terms of Use

Cookies Button


WE CARE ABOUT YOUR PRIVACY

We and our 830 partners store and/or access information on a device, such as
unique IDs in cookies to process personal data. You may accept or manage your
choices by clicking below, including your right to object where legitimate
interest is used, or at any time in the privacy policy page. These choices will
be signaled to our partners and will not affect browsing data.Privacy Policy


WE AND OUR PARTNERS PROCESS DATA TO PROVIDE:

Use precise geolocation data. Actively scan device characteristics for
identification. Store and/or access information on a device. Personalised
advertising and content, advertising and content measurement, audience research
and services development. List of Partners (vendors)

I Accept
Essential Only
Show Purposes



ABOUT YOUR PRIVACY

We process your data to deliver content or advertisements and measure the
delivery of such content or advertisements to extract insights about our
website. We share this information with our partners on the basis of consent and
legitimate interest. You may exercise your right to consent or object to a
legitimate interest, based on a specific purpose below or at a partner level in
the link under each purpose. These choices will be signaled to our vendors
participating in the Transparency and Consent Framework.
More information
Allow All


MANAGE CONSENT PREFERENCES

PERFORMANCE COOKIES

Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and
improve the performance of our site. They help us to know which pages are the
most and least popular and see how visitors move around the site.    All
information these cookies collect is aggregated and therefore anonymous. If you
do not allow these cookies we will not know when you have visited our site, and
will not be able to monitor its performance.

Cookies Details‎

FUNCTIONAL COOKIES

Functional Cookies

These cookies enable the website to provide enhanced functionality and
personalisation. They may be set by us or by third party providers whose
services we have added to our pages.    If you do not allow these cookies then
some or all of these services may not function properly.

Cookies Details‎

STRICTLY NECESSARY COOKIES

Always Active

These cookies are necessary for the website to function and cannot be switched
off in our systems. They are usually only set in response to actions made by you
which amount to a request for services, such as setting your privacy
preferences, logging in or filling in forms.    You can set your browser to
block or alert you about these cookies, but some parts of the site will not then
work. These cookies do not store any personally identifiable information.

Cookies Details‎

TARGETING COOKIES

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may
be used by those companies to build a profile of your interests and show you
relevant adverts on other sites.    They do not store directly personal
information, but are based on uniquely identifying your browser and internet
device. If you do not allow these cookies, you will experience less targeted
advertising.

Cookies Details‎

STORE AND/OR ACCESS INFORMATION ON A DEVICE 670 PARTNERS CAN USE THIS PURPOSE

Store and/or access information on a device

Cookies, device or similar online identifiers (e.g. login-based identifiers,
randomly assigned identifiers, network based identifiers) together with other
information (e.g. browser type and information, language, screen size, supported
technologies etc.) can be stored or read on your device to recognise it each
time it connects to an app or to a website, for one or several of the purposes
presented here.

List of IAB Vendors‎ | View Illustrations 

PERSONALISED ADVERTISING AND CONTENT, ADVERTISING AND CONTENT MEASUREMENT,
AUDIENCE RESEARCH AND SERVICES DEVELOPMENT 794 PARTNERS CAN USE THIS PURPOSE

Personalised advertising and content, advertising and content measurement,
audience research and services development

 * USE LIMITED DATA TO SELECT ADVERTISING 606 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Advertising presented to you on this service can be based on limited data,
   such as the website or app you are using, your non-precise location, your
   device type or which content you are (or have been) interacting with (for
   example, to limit the number of times an ad is presented to you).
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * CREATE PROFILES FOR PERSONALISED ADVERTISING 492 PARTNERS CAN USE THIS
   PURPOSE
   
   Switch Label
   
   Information about your activity on this service (such as forms you submit,
   content you look at) can be stored and combined with other information about
   you (for example, information from your previous activity on this service and
   other websites or apps) or similar users. This is then used to build or
   improve a profile about you (that might include possible interests and
   personal aspects). Your profile can be used (also later) to present
   advertising that appears more relevant based on your possible interests by
   this and other entities.
   
   View Illustrations 

 * USE PROFILES TO SELECT PERSONALISED ADVERTISING 487 PARTNERS CAN USE THIS
   PURPOSE
   
   Switch Label
   
   Advertising presented to you on this service can be based on your advertising
   profiles, which can reflect your activity on this service or other websites
   or apps (like the forms you submit, content you look at), possible interests
   and personal aspects.
   
   View Illustrations 

 * CREATE PROFILES TO PERSONALISE CONTENT 217 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information about your activity on this service (for instance, forms you
   submit, non-advertising content you look at) can be stored and combined with
   other information about you (such as your previous activity on this service
   or other websites or apps) or similar users. This is then used to build or
   improve a profile about you (which might for example include possible
   interests and personal aspects). Your profile can be used (also later) to
   present content that appears more relevant based on your possible interests,
   such as by adapting the order in which content is shown to you, so that it is
   even easier for you to find content that matches your interests.
   
   View Illustrations 

 * USE PROFILES TO SELECT PERSONALISED CONTENT 190 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Content presented to you on this service can be based on your content
   personalisation profiles, which can reflect your activity on this or other
   services (for instance, the forms you submit, content you look at), possible
   interests and personal aspects. This can for example be used to adapt the
   order in which content is shown to you, so that it is even easier for you to
   find (non-advertising) content that matches your interests.
   
   View Illustrations 

 * MEASURE ADVERTISING PERFORMANCE 707 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information regarding which advertising is presented to you and how you
   interact with it can be used to determine how well an advert has worked for
   you or other users and whether the goals of the advertising were reached. For
   instance, whether you saw an ad, whether you clicked on it, whether it led
   you to buy a product or visit a website, etc. This is very helpful to
   understand the relevance of advertising campaigns.
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * MEASURE CONTENT PERFORMANCE 357 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information regarding which content is presented to you and how you interact
   with it can be used to determine whether the (non-advertising) content e.g.
   reached its intended audience and matched your interests. For instance,
   whether you read an article, watch a video, listen to a podcast or look at a
   product description, how long you spent on this service and the web pages you
   visit etc. This is very helpful to understand the relevance of
   (non-advertising) content that is shown to you.
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * UNDERSTAND AUDIENCES THROUGH STATISTICS OR COMBINATIONS OF DATA FROM
   DIFFERENT SOURCES 447 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Reports can be generated based on the combination of data sets (like user
   profiles, statistics, market research, analytics data) regarding your
   interactions and those of other users with advertising or (non-advertising)
   content to identify common characteristics (for instance, to determine which
   target audiences are more receptive to an ad campaign or to certain
   contents).
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * DEVELOP AND IMPROVE SERVICES 532 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Information about your activity on this service, such as your interaction
   with ads or content, can be very helpful to improve products and services and
   to build new products and services based on user interactions, the type of
   audience, etc. This specific purpose does not include the development or
   improvement of user profiles and identifiers.
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

 * USE LIMITED DATA TO SELECT CONTENT 120 PARTNERS CAN USE THIS PURPOSE
   
   Switch Label
   
   Content presented to you on this service can be based on limited data, such
   as the website or app you are using, your non-precise location, your device
   type, or which content you are (or have been) interacting with (for example,
   to limit the number of times a video or an article is presented to you).
   
   View Illustrations 
   Object to Legitimate Interests Remove Objection

List of IAB Vendors‎

USE PRECISE GEOLOCATION DATA 258 PARTNERS CAN USE THIS SPECIAL FEATURE

Use precise geolocation data

With your acceptance, your precise location (within a radius of less than 500
metres) may be used in support of the purposes explained in this notice.

List of IAB Vendors‎

ACTIVELY SCAN DEVICE CHARACTERISTICS FOR IDENTIFICATION 119 PARTNERS CAN USE
THIS SPECIAL FEATURE

Actively scan device characteristics for identification

With your acceptance, certain characteristics specific to your device might be
requested and used to distinguish it from other devices (such as the installed
fonts or plugins, the resolution of your screen) in support of the purposes
explained in this notice.

List of IAB Vendors‎

ENSURE SECURITY, PREVENT AND DETECT FRAUD, AND FIX ERRORS 504 PARTNERS CAN USE
THIS SPECIAL PURPOSE

Always Active

Your data can be used to monitor for and prevent unusual and possibly fraudulent
activity (for example, regarding advertising, ad clicks by bots), and ensure
systems and processes work properly and securely. It can also be used to correct
any problems you, the publisher or the advertiser may encounter in the delivery
of content and ads and in your interaction with them.

List of IAB Vendors‎ | View Illustrations 

DELIVER AND PRESENT ADVERTISING AND CONTENT 492 PARTNERS CAN USE THIS SPECIAL
PURPOSE

Always Active

Certain information (like an IP address or device capabilities) is used to
ensure the technical compatibility of the content or advertising, and to
facilitate the transmission of the content or ad to your device.

List of IAB Vendors‎ | View Illustrations 

MATCH AND COMBINE DATA FROM OTHER DATA SOURCES 351 PARTNERS CAN USE THIS FEATURE

Always Active

Information about your activity on this service may be matched and combined with
other information relating to you and originating from various sources (for
instance your activity on a separate online service, your use of a loyalty card
in-store, or your answers to a survey), in support of the purposes explained in
this notice.

List of IAB Vendors‎

LINK DIFFERENT DEVICES 327 PARTNERS CAN USE THIS FEATURE

Always Active

In support of the purposes explained in this notice, your device might be
considered as likely linked to other devices that belong to you or your
household (for instance because you are logged in to the same service on both
your phone and your computer, or because you may use the same Internet
connection on both devices).

List of IAB Vendors‎

IDENTIFY DEVICES BASED ON INFORMATION TRANSMITTED AUTOMATICALLY 477 PARTNERS CAN
USE THIS FEATURE

Always Active

Your device might be distinguished from other devices based on information it
automatically sends when accessing the Internet (for instance, the IP address of
your Internet connection or the type of browser you are using) in support of the
purposes exposed in this notice.

List of IAB Vendors‎
Back Button


COOKIE LIST



Search Icon
Filter Icon

Clear
checkbox label label
Apply Cancel
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label

Confirm My Choices