The Unity of Consciousness and The Binding Problem

How does the Brain Create a Seamless World?

Consciousness seems to be a unified, seamless experience.

We perceive the world around us, we think our thoughts, and we feel our emotions. And we do so as part of a seemingly coherent unified stream of awareness. This apparent unity is so fundamental to our experience that we rarely question it.

But there is a problem with this intuitive view of consciousness — the unity of consciousness can break down.

When we think about a conscious experience — let’s say our conscious experience of a blue mug — we assume that our conscious experience of its shape cannot be separated from its colour. The colour is inherently tied to the mug.

But it turns out that we can assign colours to the wrong shapes — even when we have normal, healthy vision.

How could this happen?

We know that different brain areas are specialised to process different types of information. Our brain receives sensory information from the world and, in a way, breaks that information up into its different features — like colour, shape, edges and motion. While some parts of the brain will respond strongly to the blue colour of the mug, others will respond strongly to its edges.

But all the brain's sources and all the brain's cells don't seem to put the blue colour and the mug’s edges back together again — at least not in the way we might expect.

So, why does it feel like it does? Why does our conscious experience seem unified and seamless when our brain doesn't seem to unify its processing into one specific time and place?

This is the binding problem. And it’s a key problem that is frequently discussed in consciousness research.

This week, let’s explore the binding problem by asking three questions:

1. How does the visual system process our visual world?

2. What do people with brain damage tell us about the binding problem? and,

3. How do scientists think the brain solves the problem?

---

But first, a quick caveat: Accurately explaining how the brain works is an impossible task. In an attempt to explain things clearly, my explanations here will inevitably involve some simplifications and omissions.

---

Q1: How Does the Visual System Process Our Visual World?

Let’s start at the beginning. With a blue coffee mug.

Imagine you're at your favourite café. The busy barista has made a variety of coffees, each served in different coloured cups. With a smile, he extends his hand, offering you your order in a deep, blue mug.

Sounds like the perfect start to the day! Doesn’t it!?

It also seems like a simple, everyday moment for your brain to process. But a lot is going on in your head to process this scene.

To state the obvious, all visual information (aka light) comes into your brain through your eyes. From there, neurons extend to an area deep in the brain called the lateral geniculate nucleus (LGN). And then it’s a direct axon step onto the visual cortex — where things get interesting.

The visual cortex is where the brain seems to break down the visual scene into its fundamental components. It is often divided into several distinct areas, for example, V1, V2, V3, V4, and MT. Each region specialises in processing different aspects of vision. For example, V1 (or the primary visual cortex) is primarily involved in detecting basic visual features, with a particular emphasis on lines and edges.

Image adapted from: https://doi.org/10.3389/fnhum.2021.689912

So, if we consider the café scene, V1 would respond strongly to the basic lines and edges you see. Specific groups of neurons in your V1 would detect the edges of the coffee cups, the lines of the counter, and even the contours of the barista's face.

But the café scene doesn’t just have information about edges and lines. Each cup is a different colour and a slightly different shape. Neurons in area V4 respond most strongly to this type of input. So, neurons in V4 would respond preferentially to each cup's colour. The deep blue of your mug would strongly activate one set of neurons, while the bright red of another cup would strongly activate a different group.

To complicate things further, real cafés aren’t static images — in real cafés, baristas hand you coffee in deep, blue mugs (apparently). The area known as MT (or V5) contains neurons that are particularly responsive to motion. So, these cells allow you to track the moving mug so you know how fast the barista is moving the mug towards you.

Do you see the problem?

Your brain is processing all these different aspects of the scene — lines, shapes, colours, and motion in different areas of the visual cortex. But somehow, you don’t see the lines and edges of the coffee mug separately from the mug’s blue colour. And you don’t see the mug’s blue colour separately from the mug’s motion. And when you blink your eyes, the deep blue of your mug isn’t suddenly changed to red. Somehow, you see the lines and edges of the mug, its colour, and its motion as one single, unified mug that is stable over time.

How are you doing that?

We might think that there is someplace in the brain—maybe somewhere towards the front of the brain—that receives all the separate signals from the visual cortex and combines them into one unified visual perception. But no. There is no place like that.

In fact, things seem to get even more separated. From the visual cortex, neurons extend along two distinct pathways:

1. The ‘what’ pathway, which extends downward to the temporal lobe, and

2. The ‘where’ pathway, which extends upwards to the parietal lobe at the top of the brain.

Unsurprisingly, the 'what' pathway is involved in identifying objects. So, it helps us recognise what we see, whether that be faces, places, words, or any other type of object or living thing.

The ‘where’ pathway helps us interact with our world by guiding our actions — our bodies and limbs, as well as our eyes, to where objects are located.

A lot of what we know about the specialisation of different areas of the brain comes from technology such as MRI machines and techniques such as single-cell recordings. But some of the most striking evidence that specific parts of the brain are specialised for different functions comes from studying people with damage to these specific areas.

Q2: What Can We Learn From Studying People with Brain Damage?

Before we dive into this question, two short caveats:

One

I’ve described the brain areas above as specialised, but it’s important to note that this does not mean they operate in isolation. The brain is highly interconnected, and neurons are continuously influenced by other neurons that are downstream and upstream in the brain.

Two

While studies of brain damage are valuable, we should be careful when we interpret these cases. Damage to the brain often affects multiple areas, not just the area we are interested in, and damage also varies between patients.

It’s also important to remember that while a lack of function might suggest an area is necessary for certain conscious experiences, it doesn't prove that the area alone is sufficient for, or the sole site of, that conscious experience. It just means that the area is likely an essential part of the brain's network that gives us the conscious experience.

Damage to area V4

As mentioned above, colour primarily stimulates neurons in area V4. People who have damage to this area are commonly colour-blind. But this type of colour-blindness is not the same as colour blindness caused by defective photoreceptors in the retina. People with V4 damage can still detect that colours differ, but they have no conscious experience of colour.

So, in the café, they might be able to tell that each cup is a different shade, but they couldn't tell you which cup is blue, red, or any other specific colour. Importantly, they don’t lose their conscious experience of other features like the mug’s shape or its motion.

Damage to Area MT

Neurons in area MT, are primarily responsive to motion. Damage to this area commonly leads to ‘motion blindness’ (akinetopsia). People with this type of brain damage describe moving objects as a series of still snapshots rather than fluid motion. As you can imagine, this would make everyday tasks like pouring a drink or crossing a street extremely challenging.

In the café, people with damage to this area would likely have trouble accurately gauging the speed or direction of the barista's hand as he extends the mug. They might see the mug in one position and, then, suddenly in another without having any conscious experience of the smooth movement between these positions.

But as long as the other areas of their visual cortex are intact, people with motion blindness will continue to experience things like colour and shape normally.

Damage to the ‘what’ pathway

Someone with damage to the ‘what’ pathway will have difficulty identifying and recognising objects.

In the café, people with damage to this pathway might correctly reach for the mug but not be able to name what they are reaching for or describe how to use it.

People with damage in this pathway often have deficits that are specific to categories of objects. For example, damage to one area might result in difficulty recognising places. Damage to a different area might result in difficulty recognising faces — a condition called prosopagnosia.

Some individuals may struggle to bind facial features into a complete face. For instance, they might say, ‘I can see the eyes, nose, and mouth, but these don’t add up to a face.’

In more severe cases, the individual may not even recognise faces as faces, as was the case with Dr. P, a case famously recounted by Oliver Sacks in his book, The Man Who Mistook his Wife for a Hat.

Damage to the ‘where’ pathway

Someone with damage to the ‘where’ pathway often has difficulty accurately reaching for objects. These individuals can clearly perceive and identify objects but struggle to interact with them precisely (this condition is called optic ataxia).

In the café, someone with this condition might see the mug being handed to them and even understand its trajectory of motion, but they will struggle to make the right type of movements required to grasp the mug.

Some people with extensive damage to the 'where' pathway might develop a rare condition known as Balint's syndrome. People with this syndrome have a severely distorted perception of space. They typically can only perceive one object at a time but struggle to describe where that object is in space. While they don't completely lose their sense of 'where', their understanding of how objects relate to space is profoundly impaired. Their concept of space seems limited to the object they are currently seeing. So, instead of seeing a coherent scene, they perceive the world as a series of isolated objects with little context. As you can imagine, this makes navigating the world extremely challenging.

---

For those with normal, healthy vision, collecting coffee at the café isn’t an experience of isolated patches of colour, disconnected edges, and independent motions. Instead, the experience is one of a unified, coherent, and stable whole. The blue mug, its shape, its motion as the barista moves it, and its location relative to other objects all come together seamlessly.

Yet, given what we know about the specialised nature of visual processing in the brain, we might wonder how our brain brings all these elements together into a coherent whole.

Q3: How do Scientists Think the Brain Solves the Binding Problem?

Although scientists are still figuring out how the brain solves the binding problem (or whether it solves it at all), they have some clues about what might be going on.

The brain might use spatial position to keep track

When light from the world enters our eyes, that light is projected onto the retinas, similar to how light is projected onto the image sensor (or film) in a camera.

Each area in the visual cortex then acts like a specialised map of what the retina ‘sees’.

For example, neurons next to each other in V1, will respond to locations that are next to each other on the retina. This is true for the other areas in the visual cortex, too. So, we say that areas in the visual cortex are (mostly) retinotopically mapped.

Some cool experiments have been done to show this. For example, in one study, people looked at flashing letters while lying in an MRI scanner. The researchers could see the visual cortex 'light up' in patterns that corresponded to the shape of the letters.

Image source: doi:10.1016/j.neuroimage.2010.05.005

Some scientists think retinotopic mapping might have an important function—it might provide a common frame of reference between areas of the visual cortex, which could be used to solve the binding problem.

The brain might use neural synchrony

Some researchers suggest that for binding to occur, neurons that respond to features of the same object must synchronise their firing. For example, to see the blue mug moving as a coherent object, the neurons in V4 and MT that respond to the mug must fire at the same time or in a coordinated rhythm.

This idea was very popular for a while, but new findings have some scientists questioning whether neural synchronisation is really the answer to the binding problem. It remains an active area of research, so our understanding will likely change as we discover more.

Attention might have something to do with it

In the café, when the barista hands you that deep blue mug, your attention to that moment is likely critical for bringing all those visual features together.

When people are distracted, or their attention is overloaded, they can make what we call ‘conjunction errors’.

To show this effect, a researcher might quickly show you a blue letter H and a red letter T, for example. Then, the researcher might ask you what you saw. If you are able to pay attention to the letters, you will say, I saw a blue letter H and a red letter T.

But, if the researcher were to overload or divert your attention, you are just as likely to say you saw a blue T and a red H. Your brain would get the features right — you would report seeing something blue, something red, the letter H and the letter T, but you might mix up how they combine.

It's as if, without attention, the brain struggles to stick the right features together.

If you're wondering how the brain does this? And what precisely it means for a brain to ‘pay attention’, then you are not alone. Many scientists, myself included, are interested in answering these sorts of questions. While we've made progress, there's still a lot we don’t know.

---

Thank you.

I want to take a small moment to thank the lovely folks who have reached out to say hello and joined the conversation here on Substack.

If you'd like to do that, too, you can leave a comment, email me, or send me a direct message. I’d love to hear from you. If reaching out is not your thing, I completely understand. Of course, liking the article and subscribing to the newsletter also help the newsletter grow.

If you would like to support my work in more tangible ways, you do that in two ways:

• You can become a paid subscriber

• or you can support my coffee addiction through the “buy me a coffee” platform.

I want to personally thank those of you who have decided to financially support my work. Your support means the world to me. It's supporters like you who make my work possible. So thank you.