Hello? Is There Anybody in There?

John Searle's Chinese Room Thought Experiment

Lately, us humans have been talking to machines.

Some of us might even have a favourite — ChatGPT, Google’s Gemini, or maybe Claude? The LLMs seem helpful, polite, articulate, endlessly patient, and sometimes, weirdly insightful. Which makes them eerily human-like. So naturally, some of us humans have started to wonder:

Is there anybody in there?

This is not a new question. Ever since the early days of computing, us humans have been wondering whether digital computers don’t just seem to understand, but whether they do (or some day could) actually understand. Like real understanding!

Back in 1980, philosopher John Searle labeled this claim strong AI: the idea that a digital computer running the right kind of program doesn’t just simulate a mind — it is a mind.¹ And he introduced a thought experiment to push back against this idea. The thought experiment is known as:

The Chinese Room.

You might be thinking, haven’t we spilled enough ink on this thought experiment already? Fair point — I hear you.

But this little thought experiment raises some big questions, and it just so happens that these questions are exactly the kinds of questions I want to explore in a new series on The Trouble with Meaning. So, it’s a great place to start.

Searle’s goal with the Chinese Room was to show why strong AI can’t be right — why, no matter how good a computer’s outputs are, something will always be missing.

But what exactly is missing?

Well… meaning. Meaning is missing.

At least, that’s what Searle thinks.²

So this week, let’s explore Searle’s argument. Can his imagined room tell us anything meaningful about meaning, minds, and real understanding?

To find out, we’ll explore three questions:

1. What is Searle’s Chinese Room thought experiment?

2. Why might we find the Chinese Room argument convincing?

3. Why might we reject Searle’s argument?

---

Before We Step Inside the Room…

Let’s step back and consider Searle’s goals here. Yes, he is attacking the idea of strong AI. But the idea of strong AI — is based on the broader philosophical theory of mind:

functionalism.

Functionalism says that what matters about the mind isn’t what it’s made of, but what it does. Just like a computer can run the same software on different hardware, a mind— according to functionalism — could be implemented in a biological brain, or in a machine, or in something else entirely. The key is the function, not the material. Searle doesn’t buy this. And the Chinese Room is his way of pushing back.

To understand that push, there’s one more distinction we need to clear up:

syntax vs. semantics.

In everyday language, syntax usually just means grammar. But Searle uses it more broadly. For him, syntax is about form — the shape of letters and symbols, the order they appear in, and the rules for moving them around.³

If you see the symbol 猫, and you don’t understand Japanese or Mandarin, then, to you, that symbol is just syntax — a shape on a page with no meaning attached.

Semantics, on the other hand, is about meaning — what the symbols and their arrangements stand for, what they refer to, what they represent. In this case, the symbol 猫 means cat in both Mandarin (māo) and Japanese (neko).

Finally, a quick word about AI.

Historically, AI research has followed two major approaches. One focuses on building symbolic systems — programs that manipulate explicit symbols using rules (like, if this, then that). The other focuses on connectionist systems, which use networks of simple units to learn patterns — this is the approach behind most modern AI systems, like large language models (LLMs).⁴ Searle designed the Chinese Room thought experiment with symbolic AI — the first kind — in mind.⁵

Q1: What is Searle’s Chinese Room thought experiment?

Imagine you’re trapped in a room with no idea what’s going on outside.

Someone keeps slipping notes through a slot in the wall. The notes have strange squiggles on them. You don’t know what these squiggles mean. Maybe they’re characters from another language — Mandarin, maybe? Or Japanese? Could be alien, for all you know. Or totally made up.

Turns out, they’re Chinese Mandarin symbols. But you have no way of knowing that.

In this room you have two things — a giant book of rules written in English (which you do understand). The rulebook is full of entries in the form of ‘if this, then do that’. The other thing in the room is a basket full of cards, each marked with different symbols.

So, when a new card is slipped into your room you flip through the rulebook and find the matching entry. You follow the rules, which asks you to find certain cards in the basket and arrange them according to the rules. Then once everything is in order, you slide those cards back out the slot.

Outside the room. The someone who has been slipping the symbols into your room, actually understands what those symbols mean. So, it seems to them that they have been chatting with someone fluent in Mandarin. But you don’t understand a word of it. You’ve just been following instructions in a book.

According to Searle, you — the person that is inside the Room — are doing everything a digital computer does when it runs a program. You’ve followed instructions. You’ve rearranged symbols. You’ve produced the right outputs. And you’ve done it so well, that you may have convinced someone outside the room that you understand Mandarin.

In other words, you might have just passed the Turing Test.⁶

But you don’t really understand. The symbols are just strange squiggles to you.

Searle argues, if going through a rulebook is not enough to give you an understanding of Mandarin, then going through a rulebook — or a computer program — is not enough to give any digital computer an understanding either.

There are two ways we can break down Searle’s argument. Let’s set out the intuitive gloss here, and then we’ll look at a more precise version later.

• Premise 1: If strong AI is true, then the person in the Room understands Mandarin.

• Premise 2: The person in the Room does not understand Mandarin.

• Conclusion: Therefore, strong AI is false.

Q2: Why might we find the Chinese Room argument convincing?

Let’s start with Premise 1 of The Chinese Room Argument:

This premise is a conditional statement — if P then Q

If [strong AI is true], then [the person in the Room understands Mandarin.]

It’s not meant to be a controversial claim — strong AI claims that a digital computer running the right kind of program doesn’t just simulate a mind — it is a mind. So, Searle says, if that’s true, if the person the room is executing the right kind of program, then they too should understand Mandarin.

Before publishing the Chinese Room, Searle presenting his idea in talks and lectures. And during these talks and lectures he didn’t dwell on this premise — he took it as a given. But it’s worth noting that many of the most common criticisms of the Chinese Room are a push back against this premise. So I will address these concerns and Searle’s response to them, in Q3.

For now, let’s turn to Premise 2:

The person in the Room does not understand Mandarin.

This is the part of the argument that feels, to many people, immediately obvious.

You — as the person in the Room — don’t know that you’re communicating in Mandarin. You don’t know what the characters mean. You can’t tell a question from a command, a poem from a shopping list. You’re just flipping through a rulebook and following instructions. You might get really good at this matching game, but it’s not a real conversation. It’s all syntax, and no semantics.

It’s not just that you don’t understand Mandarin right now — it’s that, on Searle’s view, there’s no possible way for you to ever understand Mandarin. There’s nothing in the system that could get you from symbol manipulation to meaning.

Remember, the instruction book doesn’t tell you what the symbols mean — it just tells you which symbols to pull out of the basket and in what order to arrange them. It’s kind of like trying to learn Mandarin with only a Chinese–Chinese dictionary. You look up one unfamiliar character, and it points you to another unfamiliar character, and then another. At no point does meaning break through — it’s all symbols pointing to other symbols, with no anchor to anything you actually grasp.

This is what Searle wants us to see. Understanding isn’t simply about producing the right responses so that we seem to understand. It’s about actually understanding.

So if all a digital computer ever does is follow instructions and manipulate symbols —then, according to Searle, there’s no way it could ever understand anything. Not now. Not later. Not even in principle.

Q3: Why might we reject Searle’s argument?

The Chinese Room thought experiment is a classic in the philosophy of mind — which means it has sparked plenty of debate. But not all objections to Searle’s argument are made for the same reasons.

There have been many critical responses over the years. Let’s review four of them —and why Searle didn’t find them persuasive.

1. The Systems Reply

This is the most common — and probably the most famous — objection. It says: sure, the person in the Room doesn’t understand Mandarin. But the person plus the room plus the rulebook plus the symbols plus the process — in other words, the whole system — does understand.

The Systems Reply is a rejection of Premise 1: If strong AI is true, then the person in the Room understands Mandarin.⁷

As I mentioned earlier, Premise 1 is a conditional statement — if P, then Q.

P = strong AI is true
Q = the person in the room understands Mandarin

Logically, if you want to deny the conclusion of a conditional argument, you can reject the assumption (the P), or you can reject the consequence (the Q). The Systems Reply rejects the consequence (the Q). It says: no, the person doesn’t understand. But that doesn’t mean the systems reply also rejects the condition (the P). They argue:

If strong AI is true, then it’s not the person alone who understands, it’s the whole system.

Essentially, they are suggesting that what Searle is proposing is that strong AI claims that parts of a digital computer will understand. But no one who is a proponent of strong AI would ever suggest that part of a digital computer could understand.

Searle, raises several pointed objections to the systems reply. His most serious response requires him to revise his thought experiment.

He says: okay, suppose I memorise the whole rulebook. I do all the symbol manipulation in my head. No paper, no baskets, no room. Just me. In that case, I am the system — and, he claims, he still wouldn’t understand a word of Mandarin. So adding up a bunch of non-understanding parts doesn’t somehow produce understanding.

Some critics push back by saying: hang on — memorising all those rules and symbols isn’t nothing. It’s actually a pretty serious cognitive task. Who’s to say you wouldn’t come to understand? Maybe the real problem, they suggest, is a failure of imagination on Searle’s part — an unwillingness to consider that understanding might come from the very process he’s describing.

2. The Robot Reply

This reply says the problem isn’t that a digital program couldn’t understand — it’s that a digital program that doesn’t have body can’t understand. The person in the Room is disconnected from the world. And because the words and symbols we use are tied to things in the world, if we want an AI to truely understand it would need to have access to that world. In other words, it would need to be a robot (or like a robot). It would need to do things like move through space, see, touch, listen, and respond.

In other words, understanding isn’t just about symbol manipulation — it’s about being embedded in the world. That’s where meaning comes from.

The robot reply is also a rejection of Premise 1.

Searle’s answer? He revises his thought experiment.

He asks us to imagine we put the entire Room inside the head of a robot — one with cameras and sensors that take in the outside world. But inside the room, nothing has changed. There’s still a little person sitting in the Room, flipping through rulebooks and manipulating symbols. The robot might have eyes, but the person inside the room doesn’t know that. They are still just receiving symbols and following instructions, with no idea what any of it means.

Searle also suggests, this shows that the robot reply actually concedes his point. Understanding cannot come from syntax — from simply running a syntactical program.

Philosopher Jerry Fodor thinks Searle has missed the point.⁸

Fodor says: sure, if there’s just a little guy in a box shuffling symbols, of course there’s no understanding. But that’s not what a real robot would be like. Real robots form connections between the symbols they process and what’s happening in the world. In other words, they build causal links between their inputs and their actions. The problem, Fodor says, is that Searle imagines the wrong kind of system — a symbolic bottleneck mediated by a human, one that can’t build or update any of those links.

3. The Wrong Model Reply

As I mentioned earlier, when Searle designed the Chinese Room thought experiment, he had symbolic AI in mind — the classic systems that manipulate symbols by following explicit rules, like if this, then do that — much like the person in the Room. But most modern AI doesn’t work that way. Instead, today’s systems are built using neural networks. Neural nets don’t use fixed rules. Instead rely on patterns learned from data, which is much closer to how we think the brain works.

This has led some critics to argue that Searle’s thought experiment is targeting the wrong kind of system. It doesn’t resemble anything like what advocates of strong AI think a real artificial mind would look like.

So, what is Searle’s reply to the Wrong Model Reply?

You guessed it — he revises the thought experiment again.

This time, instead of just one person in the Room, he asks us to imagine a whole gym full of people: dozens or even hundreds of people, each following part of the rulebook, working together to simulate the program. It’s meant to mirror the kind of distributed processing you get in a neural network.⁹

But for Searle, it still doesn’t change anything. Even if you scale it up and spread it out, no one in the gym understands Mandarin. And if no one understands, then — he claims — understanding just isn’t happening anywhere in the system.

But notice how much less convincing this argument is. Part of what made the original thought experiment seem so powerful was that you could imagine yourself in the Room. You could picture what it would be like to shuffle those symbols and realise that you didn’t understand a word. But once it becomes a whole gym full of people, that first-person perspective starts to break down. It’s harder to say what the system as a whole does or doesn’t understand — because there is no clear ‘you’ inside it anymore.

4. The Begging the Question Reply

Thought experiments aren’t always designed to discover something new about the world. Most of the time, they’re designed to persuade — to nudge us toward a certain feeling.

That’s how Daniel Dennett sees the Chinese Room. He thinks what Searle is really saying is: “Come on, you know this guy doesn’t understand Mandarin, right?” He’s counting on your gut reaction.

And maybe that gut reaction is right. But gut reactions aren’t arguments. They’re not grounded in logic or evidence. They’re just... intuitions.

When we read Searle’s papers, one phrase comes up again and again: syntax is not enough for semantics. It sounds right. It feels right. So it often goes unquestioned.

Searle doesn’t offer much in the way of empirical evidence to support his claim. He relies instead on logical reasoning.

So let’s take a closer look the the axiom derivation Searle uses.

• Premise 1: Computer programs are syntactic — they operate on symbols based on shape, structure, or position.

• Premise 2: Minds have semantics — real understanding is about meaning, not just structure.

• Premise 3: Syntax alone isn’t enough for semantics.

• Conclusion: Therefore, running a program — even a perfect one — can’t produce real understanding.

Here’s the criticism: the third premise — syntax alone isn’t enough for semantics — is exactly what strong AI is trying to question. It’s the core issue under debate. So, if Searle simply asserts it as one of his premises, then he’s not proving anything — he’s just assuming the conclusion he wants.

That’s what philosophers mean when they say an argument begs the question. It puts the very thing it’s supposed to prove into the setup.

Notice how Premise 1 and Premise 2 on their own don’t quite get us to the conclusion. Searle needs Premise 3. But it is precisely Premise 3 that needs defending.

The Sum Up

The point, according to many critics, isn’t that Searle is definitely wrong. He might be right. It might turn out that real understanding (whatever that turns out to be) really does require more than symbol manipulation. But this argument — the Chinese Room — doesn’t get us there. It just gives us a feeling. And hopes we don’t notice the shortcut.

If Searle thinks symbol manipulation isn’t enough to give us meaning, then we might want to ask… what is enough?

That’s the challenge raised by philosophers like Jerry Fodor. He insists that Searle must answer this question. And he points to decades of empirical research in cognitive science and work done in linguistics that supports the idea that symbol manipulation is something the brain does.

So, if Searle is right and meaning does not come from symbol manipulation, then the real question is still wide open:

Where does meaning come from?

1. Searle introduced the strong/weak AI distinction in his 1980 paper “Minds, Brains and Programs.” John Haugeland later popularised a parallel contrast when he coined GOFAI (Good Old-Fashioned AI) in 1985.

2. Searle is commonly interpreted as being a dualist. He denies this claim in the following:

• Is the Brain a Digital Computer? (Proc. AISB, 1990) – where Searle clarifies his biological naturalism.

• Why I Am Not a Property Dualist (2002) – where he spells out why he thinks all intentionality is biologically grounded.

3. In analytic philosophy: syntax = formal structure; semantics = truth-conditional or intentional content. In contemporary linguistics, semantics often just means word- and sentence-level meaning, not necessarily aboutness.

In psychology, semantic memory is Endel Tulving’s term for our storehouse of general facts (e.g. Paris is the capital of France), as opposed to episodic memory, which is the record of personally experienced events. This use of the word semantic is orthogonal to the philosophy-of-language sense, where semantics means aboutness — the content that symbols refer to. It’s the same word, but it has separate technical traditions.

4. LLMs are neural networks (connectionist) but they are trained on symbolic text and often operate on discrete tokens at the first layer. So, they are somewhat of a hybrid and are not symbol-free.

5. He was targeting Schank & Abelson’s story-understanding program — archetypal symbolic AI.

In the early 1980s (when Searle developed his famous thought experiment) symbolic AI was popular. But by the late 1980s symbolic AI had stalled, and parallel distributed processing (connectionism) became fashionable.

6. Originally called the imitation game in Alan Turing’s 1950 paper “Computing Machinery and Intelligence.” It is strictly a behavioural test.

7. The systems reply accepts strong-AI but shifts understanding to the whole system.

8. Fodor’s 1980 commentary is the standard robot reply.

9. The Chinese Gym (or Chinese Nation) was first used by Ned Block (1978); Searle later adopted a variant of Block’s argument against connectionism in the 1990s.