www.eejournal.com Open in urlscan Pro
68.168.102.15  Public Scan

Form analysis
2 forms found in the DOM

GET https://www.eejournal.com

<form role="search" method="get" class="search-form" action="https://www.eejournal.com">
  <label>
    <span class="screen-reader-text">Search for:</span>
    <input type="search" class="search-field" placeholder="Search …" value="" name="s">
  </label>
  <button type="submit" class="search-submit"><span class="screen-reader-text">Search</span></button>
</form>

POST https://techfocus.activehosted.com/proc.php

<form method="POST" action="https://techfocus.activehosted.com/proc.php" id="_form_64B15A9756F05_" class="_form _form_1000 _inline-form  _dark" novalidate="" data-styles-version="0">
  <input type="hidden" name="u" value="64B15A9756F05" data-name="u">
  <input type="hidden" name="f" value="1000" data-name="f">
  <input type="hidden" name="s" data-name="s">
  <input type="hidden" name="c" value="0" data-name="c">
  <input type="hidden" name="m" value="0" data-name="m">
  <input type="hidden" name="act" value="sub" data-name="act">
  <input type="hidden" name="v" value="2" data-name="v">
  <input type="hidden" name="or" value="0ae685f04901f28cf76f2a3212a46d1a" data-name="or">
  <div class="_form-content">
    <div class="_form_element _x95353959 _full_width ">
      <label for="email" class="_form-label">Email*</label>
      <div class="_field-wrapper">
        <input type="text" id="email" name="email" placeholder="" required="" data-name="email">
      </div>
    </div>
    <div class="_form_element _x95484604 _full_width ">
      <label for="firstname" class="_form-label">First Name</label>
      <div class="_field-wrapper">
        <input type="text" id="firstname" name="firstname" placeholder="Type your first name" data-name="firstname">
      </div>
    </div>
    <div class="_form_element _x88705123 _full_width ">
      <label for="lastname" class="_form-label">Last Name</label>
      <div class="_field-wrapper">
        <input type="text" id="lastname" name="lastname" placeholder="Type your last name" data-name="lastname">
      </div>
    </div>
    <div class="_form_element _x23199688 _full_width ">
      <label for="ls" class="_form-label">Enter the following to confirm your subscription*</label>
      <div class="g-recaptcha" data-sitekey="6LcwIw8TAAAAACP1ysM08EhCgzd6q5JAOUR1a0Go" id="recaptcha_0">
        <div style="width: 304px; height: 78px;">
          <div><iframe title="reCAPTCHA"
              src="https://www.google.com/recaptcha/api2/anchor?ar=1&amp;k=6LcwIw8TAAAAACP1ysM08EhCgzd6q5JAOUR1a0Go&amp;co=aHR0cHM6Ly93d3cuZWVqb3VybmFsLmNvbTo0NDM.&amp;hl=en&amp;v=khH7Ei3klcvfRI74FvDcfuOo&amp;size=normal&amp;cb=l6iq4r3c0thf"
              width="304" height="78" role="presentation" name="a-6w9v1ks9pvr5" frameborder="0" scrolling="no"
              sandbox="allow-forms allow-popups allow-same-origin allow-scripts allow-top-navigation allow-modals allow-popups-to-escape-sandbox"></iframe></div><textarea id="g-recaptcha-response" name="g-recaptcha-response"
            class="g-recaptcha-response" style="width: 250px; height: 40px; border: 1px solid rgb(193, 193, 193); margin: 10px 25px; padding: 0px; resize: none; display: none;"></textarea>
        </div><iframe style="display: none;"></iframe>
      </div>
    </div>
    <div class="_button-wrapper _full_width"><button id="_form_1000_submit" class="_submit" type="submit">Subscribe</button></div>
    <div class="_clear-element"></div>
  </div>
  <div class="_form-thank-you" style="display:none;"></div>
  <input type="hidden" name="pum_form_popup_id" value="74267">
</form>

Text Content

Search for: Search

Menu
 * Articles
 * News
 * Fish Fry
 * Chalk Talks
 * Blogs
 * Categoriesexpand child menu
   * Category List
 * Media Kit



Skip to content

feature article


June 22, 2023


AI-AUGMENTED EARBUDS THAT READ YOUR MIND!

by Max Maxfield

I was 16 or so years old when I built my first brainwave amplifier circa 1973.
This was prior to the widespread availability of microcontrollers. As far as I
know, it was also before anyone had even coined the term DSP (digital signal
processing), because—to the best of my knowledge—all signal processing at that
time was ASP (analog signal processing), which was performed using analog
components and techniques.

My humble brainwave amplifier artifact required a cornucopia of electrodes to be
clamped around my cranium, buffered with conductive paste, and connected to my
noggin by surprisingly sticky sticky tape. Why is it that such tape is always at
its stickiest when you don’t want it to be while managing to be at its least
sticky when you need it the most? Suffice it to say that once the sensors were
firmly attached, I didn’t feel a great urge to take them off again in a hurry.
The idea was to detect and filter out theta and alpha brain waves, which
increase when you enter a meditative state, amplify them, and use them to
stimulate a pink noise generator into outputting a relaxing “chuff, chuff,
chuff…” sound a bit like an old-fashioned steam locomotive (see also Colors of
Noise).

Theoretically, after practicing for only a few weeks, it should have been
possible for me to out-meditate a Shaolin monk. In practice… well, that’s a tale
that remains to be told. Suffice it to say that my brainwave amplifier did not
distinguish itself on either the “small” or “lightweight” fronts. So, you can
only imagine my surprise to discover that various groups are currently working
on creating AI-augmented earbuds that can monitor your brainwaves. “Why on earth
would anyone want to do this?” I hear you cry. Well, I shall expound, explicate,
and elucidate shortly, but first…

I was just chatting with Martin Croome from GreenWaves Technologies, which is a
fabless chip manufacturer that was founded in 2014 and is based in Grenoble,
France. Martin dons a strange double hat in the company—on the one hand, he’s
the VP of marketing; on the other hand, he’s the chief architect for the
company’s neural network development tools (and I thought my life was
confusing).

The core mission for the folks at GreenWaves is to design, develop, and market
extreme processors for energy-constrained devices. In this context, an
energy-constrained device may be something tiny, like an earbud that must keep
running for a day or more on a single charge, or it may be something like an IoT
sensor node that has larger batteries but that must truck along for years or
even decades.

Their initial product offering, the GAP8, has been in production since 2020. As
one of the very first AI-enabled RISC-V-based microcontrollers on the market,
the GAP8 is currently deployed in a wide variety of products. In the case of
their second-generation product, the GAP9, which is billed as an ultra-low-power
AI-and-DSP-enabled RISC-V-based microcontroller, they’ve been sampling since the
beginning of last year, they currently have production wafers in their hands,
and their customers will be launching GAP9-based products later this year.

Just to set the scene, the GAP8 boasts 22.65 giga operations per second (GOPS)
at 4.24mW/GOP, while the GAP9 flaunts 150.8 GOPS at 0.33mW/GOP. An example
GAP8-based application that has been shipping for some time is shown below.



Ceiling-mounted people detector (Source: GreenWaves)

Using a GreenWaves’ GAP8 as its brains, this little beauty—a ceiling mounted
infrared (IR) people detecting sensor—was created by a company called Kontakt.io
(there’s little wonder I have so many problems with spelling). The sensor wakes
up once a minute to count the number of people in the room and convey this
information “upstairs.” Each sensor is also a Bluetooth mesh node. The sensors
pass their data from node to node until it reaches a gateway (e.g., Wi-Fi or
cellular) that can communicate the data to a local fog or a global cloud.

There are many applications for this sort of thing, such as meeting room
utilization. For example, is this meeting room really getting used? And, if it
is being used, how many people typically use it at any particular time? (If it’s
a 10-person room but 99% of the time there are no more than 5 people in it, then
maybe it should be divided into two meeting rooms.) Other applications include
things like cafeteria utilization (“Hey, this would be a good time for you to go
to the cafeteria because there aren’t many people in it”) and smart cleaning
(“No one has used these toilets since they were last cleaned, so there’s no
point in cleaning them again”), and so forth.

As I pen these words, GreenWaves’ partners are feverishly developing products
based on GAP9 samples. The folks at GreenWaves say they can’t say (no pun
intended) who their GAP9 customers are, but they can say that they’ve already
had 10+ design wins, and this is for a device that’s not yet in production,
which is rather exciting. 



The GAP9 next to a Euro cent (Source: GreenWaves)



Let’s take a slightly deeper dive into this little rascal. In a crunchy
nutshell, the GAP9 essentially embodies three different processing capabilities.
The first is real-time streamed autonomous time domain digital signal
processing, which is predominantly focused on audio. This employs an in-house
developed IP called the smart filtering unit (SFU), which we will discuss in
more detail a little later.

The second processing capability is general-purpose digital signal processing
(DSP) like codecs, FFTs, frequency-based filtering, frequency domain filtering…
that kind of stuff. This is performed in what they call a multi-core RISC-V
computational cluster.

Last, but certainly not least, the third processing capability involves the
combination of the aforementioned computational cluster with a neural network
accelerator called the NE16.

In line with our earlier discussions, all these processing capabilities are
focused on delivering extreme performance whilst consuming very, very low power
for energy-constrained devices. Martin told me that, in addition to offering
lower latency and lower power consumption to pretty much anything else on the
market, a key differentiator is that “GAP processors are easy to program, as
compared to most of the alternatives.” Martin also says, “GAP processors are
very flexible, which allows them to cope with accelerating the types of things
that are happening today, not the things which happened last year.” By this he
means that these processors can implement all the latest and greatest types of
neural networks, such as recurrent, convolutional, transformer, etc.

So, what does the GAP9 look like inside? Well, as we see in the diagram below,
there are fundamentally two parts to the chip that we might call “the left side”
and “the right side” (stop me if I’m getting too technical).



High level block diagram of the GAP9 (Source: GreenWaves)

The left side looks fundamentally like a regular MCU. What they call the “Fabric
Controller” (for historical or hysterical reasons, I forget Martin’s exact
wording) is basically a RISC-V core. They’ve exploited the fact that you can
extend the instruction set architecture (ISA) of the RISC-V, and they’ve
employed this capability to add their own custom instructions for things like
DSP bit manipulation, lightweight vectorization in the cores, and what they call
transprecisional floating-point, which means they can do 32-bit floating point
along with two different 16-bit floating point representations in the form of
IEEE FP16 and BF16 (these formats were discussed in more detail in my Mysteries
of the Ancients: Binary Coded Decimal (BCD) column). 

Meanwhile, on the right-hand side we find an additional nine RISC-V cores (this
is the “computational cluster” we were waffling about earlier). This means that
there are 10 RISC-V cores on the chip, all identical, not only in terms of their
instruction sets, but also in terms of their memory map. In turn, this means you
can write a function, compile it once, and run it anywhere on the chip.

Adjacent to the computational cluster on one side is the NE16 AI accelerator. On
the other side, we find a hardware synchronization block, which implements all
kinds of synchronization primitives—forks, joins, barriers, mutexes, so on and
so forth—in hardware. This facilitates extremely fine-grained parallelism,
allowing multiple cores to be working on different facets of the same task (like
an FFT) or in a pipelined fashion, or on completely different tasks.

In addition to clock gating all over the device (there are multiple clock and
voltage domains across the chip), the GAP9 also supports dynamic voltage and
frequency scaling (DVFS), with all this power control taking place inside the
chip. 

Also, there’s an abundance of external interfaces, including a MIPI interface
for a camera, three different serial audio interfaces with time-division
multiplexing (TDM) support for up to 16 channels on each. Three pulse-density
modulation (PDM) inputs and one PDM output, which can support a total of nine
PDM microphones in and three PDM sources or syncs out.

And then there’s the smart filtering unit (SFU) that we mentioned earlier.
Located on the left-hand-side of the device, this is a completely autonomous
unit that can process from interface-to-interface, or interface-to-memory, or
memory-to-interface, or memory-to-memory (it can also be working on multiple
streams simultaneously).



Block-ish diagram of the GAP9’s smart filtering unit (Source: GreenWaves)

One key aspect to this block is its interface-to-interface capability. This
comes into play when we want to do something like active noise cancellation
(ANC), which demands extremely low latency. One of the big markets for GAP
processors is hearables, and one of the key areas for hearables is ANC. Why
would you want to put ANC in a chip that can implement neural networks? Well,
because you can use the neural networks to steer the ANC. To do this, the
primary path of the ANC must be fast enough that—between the time the signal
takes to go from the error microphone into the user’s ear—we can generate the
anti-noise to cancel it. Of course, this technology can be used for lots of
other things, including heavy-duty filtering, spatialization, sound effects,
etc.

So, what are the results of all this? Well, I’m just about to show you.
MLCommons is an open engineering consortium with a mission to benefit society by
accelerating innovation in machine learning (ML). As part of this, MLCommons
publishes the MLPerf Tiny benchmarks for embedded devices (these benchmarks are
available from GitHub). The results from two of these benchmarks are shown
below.





GAP9 vs. competitor benchmark results (Source: GreenWaves)

The results were that the GAP9 consumed ~2 to 3X less energy than a highly
specialized analog neuromorphic chip and ~2 to 4X less energy than a specialized
neural network accelerator chip, all while providing the lowest latency in all
the tests.

The image above reflects only two of the four benchmarks. Martin notes that the
GAP9 provided the best latency and best energy out of all the contenders on all
four benchmarks except the anomaly detection benchmark. However, the company
whose device reported the best latency on this benchmark didn’t report energy,
which must be viewed as being a tad suspicious. So, if we limit ourselves to
those companies that reported both energy and latency values, the GAP9 was the
best in latency and energy on all benchmarks across the board.

All of which brings us back to thinking what can be achieved with GAP9
processors being used to power true wireless stereo (TWS) earbuds.



GAP9 deployed in TWS earbuds (Source: GreenWaves)

In turn, this brings us back to my meandering mutterings at the start of this
column when I made mention of the fact that various groups are currently working
on creating AI-augmented earbuds that can monitor your (well, our) brainwaves. I
just had a quick Google while no one was looking and immediately found a couple
of somewhat related articles: Google Spinoff Working on Earbuds That Spy on Your
Brain Signals and Earbuds That Read Your Mind.

Martin tells me that the guys and gals at GreenWaves are working with another
company to create earbuds for the 21st century. In addition to acting as regular
TWS earbuds and/or hearing aids, these will have sensors involving flexible
contacts on the rubber that you put in your ear, thereby allowing them to act
like an electroencephalogram (EEG) to pick up your brainwaves.

There are obvious medical applications for this sort of thing, like epilepsy
detection. But the thing that piqued my interest came in the context of hearing
aids that can address the “cocktail party problem,” which involves trying to
pick out one voice when multiple people are speaking at the same time, like at a
party, for example.

I know several people who are hard of hearing. Can you imagine earbuds with
processing power sufficient to disassemble the sound space into individual
voices? Now, take this one step further and imagine using AI to monitor the
brainwave signals to determine which of these voices is of particular interest
to the earbud owner (well, wearer), and to selectively boost that voice whilst
fading back any extraneous sounds including any other speakers.

All I can say is “Wowsers!” (and I mean that most sincerely). The more I think
about this, the more applications come to mind, like gaming, for example (your
earbuds could detect changes in your brainwave patterns and signal information
to your game controller—I’m not sure where I’m going with this, but it’s both
exhilarating and scary at the same time).

Now I’m thinking about my recent The End of the Beginning of the End of
Civilization as We Know It? column. Supposing a generative AI like ChatGPT were
using the signals from your EEG earbuds to monitor your brainwaves and… I’ll let
your imagination run wild from here. As always, please do share any thoughts you
have on anything you’ve read here in the comments below (as quickly as you can
before ChatGPT gets to hear about it—again, no pun intended).

 * 
 * 
 * 
 * 
 * 


RELATED

AGILE ANALOG MAKES THE ANALOG PARTS OF SOC DESIGN EASY PEASY LEMON SQUEEZY

I sometimes joke that I’d bend over backwards to be flexible and that I’d give
my right arm to be ambidextrous (I didn’t say they were good jokes). Now,
following a recent excursion into wibbly-wobbly analog space (where no one can
hear you scream), I’m tempted to say that I’d…

SAY HOWDY TO HIGH FIDELITY SOFTWARE-DEFINED SENSING FOR THE 21ST CENTURY

There are many things I don’t know much (if anything) about. So many, in fact,
that I could write a book about them… or not, as the case might be. An example
of one of the things I know very little about would be touchscreens, which makes
it somewhat paradoxical…

NEXT-GEN AI VISION PROCESSORS TARGET EDGE APPLICATIONS

It’s been a long time coming, but I have to say that artificial intelligence
(AI) is finally starting to make my life easier in meaningful ways. I know,
right? I’m as surprised to hear myself saying this as I’m sure you are to be
reading it. As a brief aside,…

Share




LEAVE A REPLY CANCEL REPLY

You must be logged in to post a comment.

 * 
 * 
 * 
 * 
 * 

featured blogs
Joules RTL Design Studio: Accelerating Fully Optimized RTL
Jul 14, 2023
Cadence announced Joules RTL Design Studio today at CadenceLIVE Japan 2023, a
new product that expands upon Cadence's existing Joules RTL Power Solution. The
solution will address all aspects of physical design by adding visibility beyond
just power into performance, area, an...
More from Cadence...
Synopsys & Georgia Tech Team Up on Reinforcement Learning-Driven Research—Win
DAC 2023 Best Paper Award
Jul 13, 2023
We're experimenting with deep reinforcement learning in chip design alongside
Georgia Tech ECE; see why our research won the DAC 2023 Best Paper Award.The
post Synopsys & Georgia Tech Team Up on Reinforcement Learning-Driven
Research—Win DAC 2023 Best Paper Award ...
More from Synopsys...
Self-Solving Levitating Rubik’s Cube
Jul 11, 2023
Just when I thought I'd seen everything, I ran across a video of a Rubik's cube
that solves itself while levitating above the table....
More from Max's Cool Beans...



FEATURED VIDEO


SYNOPSYS 224G ETHERNET PHY IP INTEROP AT TSMC SYMPOSIUM 2023

Sponsored by Synopsys

At TSMC Symposium 2023 we showcased a successful 224G Ethernet PHY IP interop
demonstration with backplane channels. Watch the various plots, ADC histogram
and excellent eye diagrams results.

Learn More




FEATURED PAPER


SHIFT LEFT WITH CALIBRE TO OPTIMIZE IC DESIGN FLOW PRODUCTIVITY, DESIGN QUALITY,
AND TIME TO MARKET

Sponsored by Siemens Digital Industries Software

Every IC designer strives to create a “clean,” or error-free, cell, block,
chiplet, SoC, or 3DIC assembly before passing their work downstream for full
signoff verification. However, waiting until signoff verification to find out
how well you did is probably the least efficient approach to achieving
production-ready layouts, impacting engineer productivity, project schedules,
and hardware resources.

Click to read more


FEATURED CHALK TALK

High-Voltage Isolation for Robust and Reliable System Operation
Sponsored by Mouser Electronics and Texas Instruments
In this episode of Chalk Talk, Amelia Dalton and Luke Trowbridge from Texas
Instruments examine the benefits of isolation in high voltage systems. They also
explore the benefits of TI’s integrated transformer technology and how TI’s high
voltage isolations can help you streamline your design process, reduce your bill
of materials, and ensure reliable and robust system operation.
Click here for more information about Texas Instruments ISOW774x/ISOW774x-Q1
Quad-Ch Digital Isolators
Apr 27, 2023
9,651 views

All material on this site copyright © 2003 - 2023 techfocus media, inc. All
rights reserved.
Privacy Policy Media Kit Contact Us Log In



Subscribe to EE Journal Daily
Email*

First Name

Last Name

Enter the following to confirm your subscription*

Subscribe


×


We'd like to show you notifications for the latest news and updates.


AllowCancel