www.helpnetsecurity.com Open in urlscan Pro
52.40.114.121  Public Scan

Form analysis
1 forms found in the DOM

POST

<form id="mc4wp-form-1" class="mc4wp-form mc4wp-form-244483 mc4wp-ajax" method="post" data-id="244483" data-name="Footer newsletter form">
  <div class="mc4wp-form-fields">
    <div class="hns-newsletter">
      <div class="hns-newsletter__top">
        <div class="container">
          <div class="hns-newsletter__wrapper">
            <div class="hns-newsletter__title">
              <i>
                        <svg class="hic">
                            <use xlink:href="#hic-plus"></use>
                        </svg>
                    </i>
              <span>Cybersecurity news</span>
            </div>
          </div>
        </div>
      </div>
      <div class="hns-newsletter__bottom">
        <div class="container">
          <div class="hns-newsletter__wrapper">
            <div class="hns-newsletter__body">
              <div class="row">
                <div class="col">
                  <div class="form-check form-control-lg">
                    <input class="form-check-input" type="checkbox" name="_mc4wp_lists[]" value="520ac2f639" id="mcs1">
                    <label class="form-check-label text-nowrap" for="mcs1">Daily Newsletter</label>
                  </div>
                </div>
                <div class="col">
                  <div class="form-check form-control-lg">
                    <input class="form-check-input" type="checkbox" name="_mc4wp_lists[]" value="d2d471aafa" id="mcs2">
                    <label class="form-check-label text-nowrap" for="mcs2">Weekly Newsletter</label>
                  </div>
                </div>
              </div>
            </div>
            <div class="form-check form-control-lg mb-3">
              <input class="form-check-input" type="checkbox" name="_mc4wp_lists[]" value="28abe5d9ef" id="mcs3">
              <label class="form-check-label" for="mcs3">(IN)SECURE - monthly newsletter with top articles</label>
            </div>
            <div class="input-group mb-3">
              <input type="email" name="email" id="email" class="form-control border-dark" placeholder="Please enter your e-mail address" aria-label="Please enter your e-mail address" aria-describedby="hns-newsletter-submit-btn" required="">
              <button class="btn btn-dark rounded-0" type="submit" id="hns-newsletter-submit-btn">Subscribe</button>
            </div>
            <div class="form-check">
              <input class="form-check-input" type="checkbox" name="AGREE_TO_TERMS" value="1" id="mcs4" required="">
              <label class="form-check-label" for="mcs4">
                <span>I have read and agree to the <a href="https://www.helpnetsecurity.com/newsletter/" target="_blank" rel="noopener" class="d-inline-block">terms &amp; conditions</a>
                </span>
              </label>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div><label style="display: none !important;">Leave this field empty if you're human: <input type="text" name="_mc4wp_honeypot" value="" tabindex="-1" autocomplete="off"></label><input type="hidden" name="_mc4wp_timestamp"
    value="1715610416"><input type="hidden" name="_mc4wp_form_id" value="244483"><input type="hidden" name="_mc4wp_form_element_id" value="mc4wp-form-1">
  <div class="mc4wp-response"></div>
</form>

Text Content

 * News
 * Features
 * Expert analysis
 * Videos
 * Events
 * Whitepapers
 * Industry news
 * Product showcase
 * Newsletters

 * 
 * 
 * 


Please turn on your JavaScript for this page to function normally.
Dane Sherrets, Solutions Architect, HackerOne
May 13, 2024
Share


RED TEAMING: THE KEY INGREDIENT FOR RESPONSIBLE AI



Developing responsible AI isn’t a straightforward proposition. On one side,
organizations are striving to stay at the forefront of technological
advancement. On the other hand, they must ensure strict compliance with ethical
standards and regulatory requirements.



Organizations attempting to balance this thin line between rapid innovation and
increasing regulatory requirements will need to employ a standardized approach
to development, ensuring they remain compliant and competitive in an
increasingly crowded market.


AI INNOVATION AT RISK

Many businesses are already struggling to decipher an increasingly tangled knot
of regulations, including the (upcoming) Cyber Resilience Act and Data Act.

Although the recent EU AI Act has taken a significant step towards AI safety,
the law has also created additional bureaucracy. It has sparked calls from the
European Parliament to make compliance with the Act easier by simplifying
administration requirements and clarifying grey legal areas. Plus, there are
requests for better funding of AI research and support to help small businesses
get to grips with the legislation. Without these adjustments to the act, there
are genuine concerns that the EU will be unable to establish itself as a
front-runner in the field and lose out to the US and China.

The UK government has taken a more pro-innovation stance. Rather than
introducing new laws, its AI white paper proposes five high-level principles for
existing regulators to apply within their jurisdictions, focusing on safety,
fairness, transparency, accountability, and user rights. These broader
principles are less prescriptive than the EU’s Act. In fact, they align well
with the goals of red teaming, an already trusted ingredient of IT security
testing procedures.


AI RED TEAMING: DEFINING AND REDUCING RISK, WITHOUT STIFLING INNOVATION

To regulate a technology, you must understand it. Part of the challenge with
overly rigid regulation is that it assumes we already know how to limit the
risks of AI from both a safety and security perspective — but that’s not the
case.

We’re still regularly discovering new weaknesses in models from a traditional
security perspective, like AI models leaking data, and safety perspectives, like
models producing unintended and harmful imagery or code. These risks are still
being discovered and defined by the global researcher community so until we
better understand and define these challenges, the best course of action is to
remain diligent in stress-testing AI models and deployments.

Red teaming exercises are one of the best ways to find novel risk, making them
ideal for finding security and safety concerns in emerging technologies like
generative AI. This can be done using a combination of penetration testing,
time-bound offensive hacking competitions, and bug bounty programs. The result
is a comprehensive list of issues and actionable recommendations, including
remediation advice.

With this clear focus on safety, security, and accountability, red teaming
practices are likely to be considered favorably by regulators worldwide, as well
as aligning with the UK government’s vision for responsible AI development.

Another advantage of setting up red teaming as a method of AI testing is that it
can be used for both safety and security. However, the execution and goals are
different.

For safety issues, the focus is on preventing AI systems from generating harmful
information; for example, blocking the creation of content on how to construct
bombs or commit suicide and preventing the display of potentially upsetting or
corrupting imagery, such as violence, sexual activity, and self-harm. Its aim is
to ensure responsible use of AI by uncovering potential unintended consequences
or biases, guiding developers to proactively address ethical standards as they
build new products.

A red teaming exercise for AI security takes a different angle. Its objective is
to uncover vulnerabilities to stop malicious actors from manipulating AI to
compromise the confidentiality, integrity, or availability of an application or
system. By quickly exposing flaws, this aspect of red teaming helps identify,
mitigate, and remediate security risks before they are exploited.

For a real-world indication of its capabilities, the launch of Bard’s Extensions
AI feature provides a valuable example. This new functionality enabled Bard to
access Google Drive, Google Docs, and Gmail, but within 24 hours of going live,
ethical hackers identified issues demonstrating it was susceptible to indirect
prompt injection.

It put personally identifiable information (PII) at severe risk, including
emails, drive documents, and locations. Unchecked, this vulnerability could have
been exploited to exfiltrate personal emails. Instead, ethical hackers promptly
reported back to Google via their bug bounty program, which resulted in $20,000
in rewards – and a potential crisis averted.


TALENT DIVERSITY MAKES A DIFFERENCE

This quality of red teaming relies on carefully selected and diverse skill sets
as the foundation for effective assessments. Partnering with the ethical hacking
community through a recognized platform is a reliable way of ensuring talent is
sourced from different backgrounds and experiences, with relevant skills
necessary for rigorously testing AI.

Hackers are renowned for being curiosity-driven and thinking outside of the box.
They offer organizations external and fresh perspectives on ever-changing
security and safety challenges.

It’s worth noting that when red teaming members are given the opportunity to
collaborate, their combined output becomes even more effective, regularly
exceeding results from traditional security testing. Therefore, facilitating
cooperation across teams is a key consideration. Getting a blend of individuals
with a variety of skills and knowledge will deliver the best results for AI
deployments.


DEVISING THE BEST BUG BOUNTY PROGRAMS

Tailoring the incentive model for an ethical hacking program is vital, too. The
most efficient model includes incentivizing hackers according to what is most
impactful to an organization, in conjunction with bounties for achieving
specific safety outcomes.

Building on the established bug bounty approach, this new wave of red teaming
addresses the novel security and safety challenges posed by AI that businesses
must address before launching new deployments or reviewing existing products.

Targeted offensive testing that harnesses the collective skills of ethical
hackers proficient in AI and LLM prompt hacking will help strengthen systems and
processes alike. It will guard against potential vulnerabilities and unintended
outcomes missed by automated tools and internal teams. Importantly, it ensures
the creation of more resilient and secure AI applications that uphold the
principles of “responsible AI.”





More about
 * artificial intelligence
 * bug bounty
 * cybersecurity
 * hacker
 * HackerOne
 * opinion
 * red team
 * regulation

Share


FEATURED NEWS

 * Black Basta target orgs with new social engineering campaign
 * Red teaming: The key ingredient for responsible AI
 * Establishing a security baseline for open source projects

Download: The Ultimate Guide to the CISSP



SPONSORED

 * eBook: Do you have what it takes to lead in cybersecurity?
 * eBook: Why CISSP?
 * The CISO’s guide to reducing the SaaS attack surface




DON'T MISS

 * Black Basta target orgs with new social engineering campaign
 * Red teaming: The key ingredient for responsible AI
 * Establishing a security baseline for open source projects
 * How AI affects vulnerability management in open-source software
 * Download: The Ultimate Guide to the CISSP




Cybersecurity news
Daily Newsletter
Weekly Newsletter
(IN)SECURE - monthly newsletter with top articles
Subscribe
I have read and agree to the terms & conditions
Leave this field empty if you're human:

© Copyright 1998-2024 by Help Net Security
Read our privacy policy | About us | Advertise
Follow us
×