Brain Computer Interface: A Primer
BCI #1: A paradigm for understanding BCIs
What if we could combine the amazing capabilities of brains with the computational power of computers?
This tantalizing idea of merging biological and artificial intelligence is the core idea behind brain-computer interfaces (BCIs). Creating a real-life cyborg, if you will — part human, part machine.
Many experts believe that the 2020s will see a boom in the development and use of neurotechnologies, like BCIs, because new technologies, materials, computational power, and algorithms are making once-impossible things now possible. Some suggest we shouldn’t be too surprised if we start seeing an increase in the availability of such devices. These devices might promise a range of benefits, from rehabilitation for those with sensory or motor impairments to personal enjoyment and cognitive enhancement.
Over the coming weeks, I want to explore the latest in the world of BCI research. What are the scientists doing? What might the future of humans look like? What is possible? And what is hype? What are the risks? And what are the challenges?
I’ll discuss some of the work my last lab group was involved in, including my experiences trying out some of these technologies. But I’ll also explore plenty of other amazing work by colleagues from around the world. Any discussion on BCI should also address the ethical considerations relevant to merging biological intelligence with artificial systems. So, we’ll discuss that too.
But this week, I want to lay the groundwork by answering three questions:
What is a brain-computer interface?
How does a cochlear implant work? (Understanding how cochlear implants work will be useful in understanding the different approaches and challenges BCIs face).
What are the different approaches BCI researchers are taking?
1. What is a brain-computer interface?
As the name suggests, a brain-computer interface (BCI) is a system that allows direct communication between a brain and a computing device or machine.
BCIs have recently received some increased attention, with Neuralink making headlines after implanting their device in a human for the first time. Despite the recent Neuralink hype, BCIs have been around for a while. Arguably, the cochlear implant, which was first developed in the 1950s, is one of the most widely used and successful BCI systems today.
It's worth reviewing how the cochlear implant works simply because it provides a useful example for comparing the different methods and approaches that BCI systems use.
2. How does a cochlear implant work?
Normally, we hear because sound waves cause the eardrum and three tiny earbones to vibrate. These vibrations are then transmitted to the inner ear's cochlear, which contains hair cells that transform these vibrations into electrical signals that can be sent to the brain via the hearing nerve.
Specific regions of the cochlea contain hair cells tuned to different frequencies of sound waves. In the basal region, hair cells detect higher-pitch sounds, while in the apical region, they detect lower-pitch sounds.
Hearing loss often happens because the tiny hair cells are damaged, but the hearing nerve is completely fine. Without working hair cells, the hearing nerve, and therefore the brain, simply can’t receive signals from the ear.
That’s where the cochlear implant comes in. The device has two parts — an outer part and an inner part. The outer part is usually worn behind the ear and communicates with the inner part. The outer part has a microphone that picks up sound, a speech processor that prioritises audible speech among ambient sounds, a transmitter coil that converts the audio signals into electrical impulses, and a battery.
The inner part has a receiver secured to the skull under the skin behind the ear and an electrode array surgically implanted into the cochlear. This electrode array is a long thin cable that usually contains 24 electrodes distributed along its length. Each electrode stimulates different locations in the cochlear. The inner receiver obtains signals from the outer part and sends those signals to the electrode array, stimulating the hearing nerve.
Sometimes, people confuse cochlear implants with hearing aids, but the two devices work very differently. A hearing aid works by amplifying sound waves and is only helpful for those who still have some ability to hear. A cochlear implant bypasses the eardrum, ear bones, and damaged hair cells and directly stimulates the hearing nerve to restore hearing for those with profound or total hearing loss.
Experiences after receiving a cochlear implant can vary. But for most new recipients, sounds initially seem weird and unintelligible. This is partly because the device needs to be adjusted to suit the user, but the main reason is that the brain needs time to learn how to interpret the new incoming signals. The brain is plastic; with time, it can learn to interpret the new signals as intelligible sounds.
The following clip gives a fantastically clear description of what it’s like to receive a cochlear implant.
3. What are the different approaches BCI researchers are taking?
There are many different approaches to BCI research. Most scientific papers about BCIs categorise them as either invasive or non-invasive.
Invasive BCIs require putting something into the brain or body. Cochlear implants and Neuralink's chip would be considered invasive examples.
As you might expect, noninvasive BCIs are systems that measure and interact with the brain from outside the body without any implants or surgical components. You may have seen examples of noninvasive BCIs in which EEG sensors are placed on the scalp to record electrical activity from the brain.
While the invasive versus non-invasive categorisation of BCIs can provide a useful distinction, it doesn't fully capture the most compelling differences between the various types of BCI technologies. So, I'd like to highlight three other key distinctions that better illustrate the differences in approaches that researchers take when experimenting with BCIs.
Getting signals in vs getting signals out
Do you remember Mr. Potato Head from Toy Story? He was a potato that could be decorated by sticking body parts into a potato to create — well, Mr Potato Head.
Very loosely, we can think of Mr Potato Head as being a bit like us. In this analogy, the potato body would be our brain, and our sense organs, like eyes, ears, nose, mouth, and limbs, plug into our brains.
The sense organs and limbs that plug into our brains have two important roles. One is to get signals from the world into our brain, and the other is to get signals from our brain out to the sense organs and limbs.
Current BCIs tend to focus on either getting signals into the brain or getting signals out of the brain.
The cochlear implant is a getting-signals-into-the-brain type of BCI. It is designed to take signals from the world and send them to the brain by stimulating the hearing nerve.
Neuralink, on the other hand, is a getting-signals-out-of-the-brain type of BCI. It records signals from the brain and sends them to a device in the physical world.
Notice how BCIs that are the getting-signals-into-the-brain type are about stimulating the brain, and the getting-signals-out-of-the-brain type is about recording and decoding brain activity.
These two different types of BCIs have different sorts of challenges.
Let’s start with:
The challenge of getting signals into the brain.
If we think again about the Mr Potato Head analogy, to get signals into the brain, we simply use the bits and pieces we plug into our brains as peripheral devices that convert signals from our peripheral world into signals for our brains. In this scenario, the brain is doing a lot of the hard work.
Recall the cochlear implant patients from above. Initially, the recipient reports that sounds seem weird and unintelligible. But with time, the brain learns how to interpret these signals.
All brains are constantly changing. The connections between neurons are constantly being strengthened or weakened; we lose unused connections and create new ones. Every time we try something new, experience a powerful emotion, or recall a vivid memory, our brains change — and I don’t mean figuratively; I mean they literally physically change. New synapses are formed, or existing ones are pruned and reorganised. It’s these physical changes that allow someone to go from thinking the input from a cochlear implant is weird and unintelligible to thinking the signals are interpretable and sensible.
The fascinating thing is that the cochlear implant isn’t the important part. It doesn’t really matter what signals the brain is fed. As long as those signals are structured signals that provide information about the outside world, the brain will figure out how to extract what’s important and what’s not important from that signal. (By structured signals, I mean that the signals contain patterns and regularities that map onto things in the physical world rather than just random noise).
In this series on BCI, we’ll discover many devices designed to provide the brain with structured signals for it to interpret. Many of these BCIs are designed to replace lost functions, but some BCIs promise an augmented reality, where signals are input in one modality, let’s say touch, but interpreted as another, say sight. Other BCIs are designed to provide the receiver with superhuman abilities or entirely new sensations.
The challenge of getting signals out of the brain.
To get signals out of the brain, we need a way to accurately record and measure brain activity and then a method to decode that brain activity into useful outputs.
Neuralink is an example of a BCI designed to do just that.
Neuralink's approach involves implanting electrodes into the brain. These electrodes detect neurons firing in a specific brain area (the motor cortex). After some amplification and filtering of these signals, a computer program is used to decode these signals. So, instead of the brain figuring out the signals from the device or computer (as in the getting-signals-into-the-brain type of BCI), getting signals out of the brain requires that the computer program do the figuring out (although in the Neuralink example, the brain does some figuring out too). As with a cochlear implant, this figuring out requires time and training.
Open-loop vs. closed-loop
The second key distinction between BCIs is open-loop vs closed-loop systems.
In an open-loop system, the BCI does not provide feedback to either the BCI system itself or to the user's brain. For example, an electrode array may be implanted in the recipient's brain, and the computer system receives signals from the brain under different conditions. Perhaps the recipient views images of different objects while the BCI records their neural activity. The computer system learns to decode and interpret the patterns of brain signals that correspond to each image type. However, the BCI and the recipient receive no performance or brain activity feedback.
In contrast, a closed-loop BCI system incorporates feedback, allowing the BCI and/or the brain to receive and respond to inputs from the BCI. Closed-loop systems allow for adaptation. For instance, a closed-loop BCI might record neural signals associated with intended movements and also provide sensory feedback to the brain based on the actual movements produced. Over time, the recipient’s brain adapts to this feedback by refining its signals.
Hardware vs. software
The final key distinction that is important to make is between hardware and software.
By software, I mean the computer programs and algorithms used to process signals from the world, like sound waves, and signals from the brain, like neural spike patterns. Over the past 10 years, we have seen large leaps in software innovation. The improvements have been so great that some researchers predict that non-invasive BCIs will soon perform with similar accuracy and fidelity to invasive methods. The rapid progress in software development is largely attributable to recent breakthroughs in machine learning techniques, particularly deep learning algorithms.
Progress in hardware has not been as great as that in software. However, there have been some noteworthy advancements that we will discover in some future articles.
I'm looking forward to writing this series — it's a fascinating field of research. In the comments, let me know if you've discovered any recent BCI innovations that piqued your interest.
Thanks so much for reading this article.
I want to take a small moment to thank the lovely folks who have reached out to say hello and joined the conversation here on Substack.
If you'd like to do that, too, you can leave a comment, email me, or send me a direct message. I’d love to hear from you. If reaching out is not your thing, I completely understand. Of course, liking the article and subscribing to the newsletter also help the newsletter grow.
If you would like to support my work in more tangible ways, you do that in two ways:
You can become a paid subscriber
or you can support my coffee addiction through the “buy me a coffee” platform.
I want to personally thank those of you who have decided to financially support my work. Your support means the world to me. It's supporters like you who make my work possible. So thank you.
That's a very cool topic I'd love to read more about.
I really enjoyed the several minutes of explanation from the guy who received the cochlear implant. I honestly naively assumed that cochlear implants give you a crude but generally directly mappable soundscape that you can quite quickly adjust to (sort of like hearing people talk while you're submerged in the water). It was a revelation that the stuff you hear is initially unintelligible.
I wonder what would happen if you'd ever had a patient who suddenly regained the ability to hear and then removed the cochlear implant. Would they be able to immediately pick up speech (because their brain was once exposed to it) or would they need to relearn because their brain has now physically adjusted to the distorted world of the cochlear implant?
My guess is that it's the latter, knowing what we know about George Stratton's "inverted sight" experiment where the brain adjusted to seeing the world upside down, but then had to re-adjust again when the "inverted goggles" were taken off.
By the way, love your new "When Life Gives You A BrAIn" logo - neat play on words and it better reflects the more all-encompassing nature of your newsletter.
An excellent breakdown of a very complex and non-intuitive topic. I love reading and researching about BCIs keeping the whole neurallink hopium aside. I definitely think it's one of the scientific frontiers that are going to see massive breakthroughs within a decade or so and could be as revolutionary as AI itself in terms of impact.