www.technologyreview.com Open in urlscan Pro
192.0.66.184  Public Scan

Form analysis
2 forms found in the DOM

<form><span class="fieldset">
    <p><input type="checkbox" value="check" id="chkMain" checked="checked" class="legacy-group-status optanon-status-checkbox"><label for="chkMain">Active</label></p>
  </span></form>

<form class="stayConnected__form--Z3UCp"><label for="email" class="stayConnected__formWrap--2vvXW">
    <div class="stayConnected__labelWrap--3QWaM"><span class="stayConnected__label--2AaJN">Enter your email</span></div>
    <div class="stayConnected__inputWrap--235RV"><input type="email" id="email" name="email" class="stayConnected__input--3FalV" required=""><input type="text" id="name" name="name" class="stayConnected__hp--3W-Zi" tabindex="-1" autocomplete="off">
      <div class="stayConnected__submitWrap--p6VoQ"><button type="submit" class="stayConnected__submit--3WVqk">
          <div class="stayConnected__arrow--1NvTP"><svg width="17" height="15" fill="none">
              <path stroke="#000" d="M0 8.149h15.66M9.532 1L16 7.809l-6.468 6.468"></path>
            </svg></div>
          <div class="stayConnected__check--mKfmD"><svg width="18" height="16" viewBox="0 0 18 16" fill="none">
              <path data-name="check" d="M1 8.32502L6.5 14.2766L17 1" stroke="black"></path>
            </svg></div>
        </button></div>
    </div><a href="/privacy/" class="stayConnected__privacyLink--28aJn">Privacy Policy</a>
  </label>
  <div class="stayConnected__responseMessage--2vOEt">
    <div class="stayConnected__labelWrap--3QWaM">
      <p class="stayConnected__label--2AaJN stayConnected__centered--1Wcng">Thank you for submitting your email!</p>
    </div><a class="stayConnected__btn--dvz0W" href="/newsletter-preferences?email_address=undefined">Explore more newsletters</a>
  </div>
  <div class="stayConnected__responseMessage--2vOEt">
    <div class="stayConnected__labelWrap--3QWaM">
      <p class="stayConnected__label--2AaJN stayConnected__error--128Xk stayConnected__errorTitle--1gz0B">It looks like something went wrong.</p>
    </div>
    <p class="stayConnected__errorMessage--1rm4n"> We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at
      <a href="mailto:customer-service@technologyreview.com" class="stayConnected__link--2mrtZ">customer-service@technologyreview.com</a> with a list of newsletters you’d like to receive.</p>
  </div>
</form>

Text Content

Cookie Notice

We use cookies to offer you a better browsing experience, analyze site traffic,
personalize content, and serve targeted advertisements. By using our website,
you agree to the use of cookies as described in our Cookie Statement.


Close
Accept Cookies


 * Your Privacy

 * Strictly Necessary Cookies

 * Performance Cookies

 * Functional Cookies

 * Targeting Cookies

 * More Information

Privacy Preference Centre

Active

Always Active



Save Settings

Allow All

You need to enable JavaScript to view this site.
Skip to Content
MIT Technology Review


FeaturedTopicsNewslettersEventsPodcasts
Sign in
Subscribe
MIT Technology Review


FeaturedTopicsNewslettersEventsPodcasts
Sign in
Subscribe
Artificial intelligence


THE FINANCIAL WORLD WANTS TO OPEN AI’S BLACK BOXES

Some of the most powerful machine-learning techniques work in mysterious ways,
which is a problem if you need to explain a decision to customers.
By
 * Will Knightarchive page

April 13, 2017


Powerful machine-learning methods have taken the tech world by storm in recent
years, vastly improving voice and image recognition, machine translation, and
many other things.

Now these techniques are poised to upend countless other industries, including
the world of finance. But progress may be stymied by a significant problem: it’s
often impossible to explain how these “deep learning” algorithms reach a
decision (see “The Dark Secret at the Heart of AI”).

Adam Wenchel, vice president of machine learning and data innovation at Capital
One, says the company would like to use deep learning for all sorts of
functions, including deciding who is granted a credit card. But it cannot do
that because the law requires companies to explain the reason for any such
decision to a prospective customer. Late last year Capital One created a
research team, led by Wenchel, dedicated to finding ways of making these
computer techniques more explainable.

“Our research is to ensure we can maintain that high bar for explainability as
we push into these much more advanced, and inherently more opaque, models,” he
says.

Deep learning emerged in the last five years as a powerful way of mimicking
human perceptual abilities. The approach involves training a very large neural
network to recognize patterns in data. It is loosely inspired by a theory about
the way neurons and synapses facilitate learning. Although each simulated neuron
is simply a mathematical function, the complexity of these interlinked functions
makes the reasoning of a deep network extremely difficult to untangle.

Some other machine-learning techniques, including those that outperform deep
learning in certain scenarios, are a lot more transparent. But deep learning,
which allows for sophisticated analytics that are useful to the finance
industry, can be very difficult to interrogate.

Some startups aim to exploit concerns over the opacity of existing algorithms by
promising to use more transparent approaches (see “An AI-Fueled Credit Formula
Might Help You Get a Loan”).

This issue could become more significant over the next few years as deep
learning becomes more commonly used and as regulators turn their attention to
algorithmic accountability. Starting next year, under its General Data
Protection Regulation, the European Union may require any company to be able to
explain a decision made by one of its algorithms.

The problem has also caught the attention of the Defense Advanced Research
Projects Agency, which does research for the U.S. Department of Defense. Last
year DARPA launched an effort to fund approaches to making machine learning less
opaque (see “The U.S. Military Wants Autonomous Machines to Explain
Themselves”). The 13 projects selected for funding show a variety of approaches
for making algorithms more transparent.

The hope is that deep learning can be used to go beyond just matching human
perceptual capabilities. A credit card company might, for example, feed credit
history and other financial data into a deep network and train it to recognize
people who might default on their credit card payments.

Capital One is also looking at deep learning as a way of automatically detecting
fraudulent charges more reliably, Wenchel says, although the company is wary of
trusting such a system when its reasoning can’t be examined. “We operate in a
heavily regulated industry,” he says. “We need to be able to explain both
internally as well as to people why we’re making decisions. And make sure we’re
making decisions for the right reasons.”

“Deep learning is a very big buzzword right now, and there’s been great progress
in computer vision and natural language processing,” says Trevor Darrell, a
professor at UC Berkeley who is leading one of the projects selected for funding
from DARPA. “But [deep-learning systems] are criticized because it’s sometimes
hard to figure out what’s going on inside of them.”

For the DARPA project, Darrell’s group is developing several new deep-learning
approaches, including more complex deep networks capable of learning several
things simultaneously. There are also approaches that include an explanation in
the training data: in the case of image captioning, for example, an image
classified as a cat would be paired with an explanation for why it was
classified as such. The same approach could be used in classifying credit card
charges as fraudulent. “All these things get us to more interpretable deep
networks,” Darrell says.

hide



BY WILL KNIGHT


SHARE

 * linkedinlink opens in a new window
   
 * twitterlink opens in a new window
   
 * facebooklink opens in a new window
   
 * emaillink opens in a new window
   


 * 


 * POPULAR
   
    1. How SpaceX’s massive Starship rocket might unlock the solar system—and
       beyond
    2. The therapists using AI to make therapy better
    3. We reviewed three at-home covid tests. The results were mixed.
    4. Isaac Asimov Asks, “How Do People Get New Ideas?”


DEEP DIVE


ARTIFICIAL INTELLIGENCE


A HORRIFYING NEW AI APP SWAPS WOMEN INTO PORN VIDEOS WITH A CLICK

Deepfake researchers have long feared the day this would arrive.

By
 * Karen Haoarchive page


HOW AI IS REINVENTING WHAT COMPUTERS ARE

Three key ways artificial intelligence is changing what it means to compute.

By
 * Will Douglas Heavenarchive page


HOW AI DIGITAL TWINS HELP WEATHER THE WORLD’S SUPPLY CHAIN NIGHTMARE

Just-in-time shipping is dead. Long live supply chains stress-tested with AI
digital twins.

By
 * Will Douglas Heavenarchive page


THESE WEIRD VIRTUAL CREATURES EVOLVE THEIR BODIES TO SOLVE PROBLEMS

They show how intelligence and body plans are closely linked—and could unlock AI
for robots.

By
 * Will Douglas Heavenarchive page


STAY CONNECTED

Illustration by Rose Wong


GET THE LATEST UPDATES FROM
MIT TECHNOLOGY REVIEW

Discover special offers, top stories, upcoming events, and more.

Enter your email

Privacy Policy

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and
updating them one more time. If you continue to get this message, reach out to
us at customer-service@technologyreview.com with a list of newsletters you’d
like to receive.





MIT TECHNOLOGY REVIEW

Our mission is to bring about better-informed and more conscious decisions about
technology through authoritative, influential, and trustworthy journalism.

Subscribe to support our journalism.


 * ABOUT
   
    * About us
    * Careers
    * Custom content
    * Advertise with us
    * International Editions
    * Republishing
    * MIT News


 * HELP
   
    * Help & FAQ
    * My subscription
    * Editorial guidelines
    * Privacy policy
    * Cookie statement
    * Terms of Service
    * Contact us

 * twitterlink opens in a new window
   
 * facebooklink opens in a new window
   
 * instagramlink opens in a new window
   
 * rsslink opens in a new window
   
 * linkedinlink opens in a new window
   

Cover Art by Saiman Chow

© 2021 MIT Technology Review

Back to top


Path


CONTENT METER NOTICE

2 free stories remaining

You know good tech journalism when you see it. Subscribe for unlimited access.

 * Subscribe
 * Sign in