The Chinese Room has already been surpassed by LLMs, which have shown to contain neurons that activate in such high correlation to abstract concepts like “formal text” or “positive sentiment”, that tweaking them is one of the options that LLM based chatbots are presenting to the user.
Analyzing the activation space, it’s also been shown that LLMs categorize and cluster sequences of text representing similar concepts closer to each other, which allows them to present reasonably accurate zero shot responses that have never been in the training set (that “weren’t in the book” for the Chinese Room).
I don’t understand what you mean by “The Chinese Room has already been surpassed by LLMs”. It’s not a test that can be surpassed. It’s just a thought experiment.
In any case, you do bring up a good point. Perhaps this understanding is in the organization of the information. So if you have a Chinese room where all the query-response pairs are in arbitrary orders, then maybe you wouldn’t consider that to be understanding. But if you have the data organized such that similar queries/responses are close to each other and this person in the room doing the answering can make mistakes such as accidentally copying out the response next to the correct response and still make sense, then maybe we can consider this system to have better understanding.
The Chinese Room is really a thought experiment about the inner workings of a partner in a Turing test. Externally they have the same pitfalls, but the Chinese Room also reveals itself completely if one can observe in detail the inner workings of the room/partner.
LLMs are still mostly black boxes, but we can have enough of a glimpse inside to reveal that they aren’t “following some rails” like a simple algorithm.
make mistakes such as accidentally copying out the response next to the correct response and still make sense
Precisely. This is another part that we can see with LLMs: at runtime, the models get applied a “temperature” parameter, which intentionally introduces a certain level of mistakes. With “temperature = 0”, the output is a “stochastic parrot”, and quickly turns into nonsense. With a higher temperature, the randomness increases and the output becomes a total mess. But setting it just right, to a sweet spot of “very little, but not zero”, turns out to produce the outputs that we see in ChatGPT and similar.
Knowing that the concept space of LLMs has similar concepts clustered, it makes sense that these errors would force the LLM to sometimes make associations on the fly between close concepts, associations that it didn’t have trained for before, and which “derail” it into a close, but not exactly the same, train of thought.
This behavior also seems to be what we call “intelligence” in humans: the ability to solve problems not seen before (zero shot).
A further extension would be the ability to constantly learn from every interaction. Right now LLMs have a “context” of some length, that changes dynamically, but has no influence over the pre-trained network.
Interestingly, this has a parallel in “crystallized intelligence” vs. “fluid intelligence” in humans.
So… maybe LLMs are not full AGIs yet, but they are showing many of the behaviors that we would expect from an AGI, while at the same time giving or confirming insights into the workings of the human mind itself.
The Chinese Room has already been surpassed by LLMs, which have shown to contain neurons that activate in such high correlation to abstract concepts like “formal text” or “positive sentiment”, that tweaking them is one of the options that LLM based chatbots are presenting to the user.
Analyzing the activation space, it’s also been shown that LLMs categorize and cluster sequences of text representing similar concepts closer to each other, which allows them to present reasonably accurate zero shot responses that have never been in the training set (that “weren’t in the book” for the Chinese Room).
I don’t understand what you mean by “The Chinese Room has already been surpassed by LLMs”. It’s not a test that can be surpassed. It’s just a thought experiment.
In any case, you do bring up a good point. Perhaps this understanding is in the organization of the information. So if you have a Chinese room where all the query-response pairs are in arbitrary orders, then maybe you wouldn’t consider that to be understanding. But if you have the data organized such that similar queries/responses are close to each other and this person in the room doing the answering can make mistakes such as accidentally copying out the response next to the correct response and still make sense, then maybe we can consider this system to have better understanding.
The Chinese Room is really a thought experiment about the inner workings of a partner in a Turing test. Externally they have the same pitfalls, but the Chinese Room also reveals itself completely if one can observe in detail the inner workings of the room/partner.
LLMs are still mostly black boxes, but we can have enough of a glimpse inside to reveal that they aren’t “following some rails” like a simple algorithm.
Precisely. This is another part that we can see with LLMs: at runtime, the models get applied a “temperature” parameter, which intentionally introduces a certain level of mistakes. With “temperature = 0”, the output is a “stochastic parrot”, and quickly turns into nonsense. With a higher temperature, the randomness increases and the output becomes a total mess. But setting it just right, to a sweet spot of “very little, but not zero”, turns out to produce the outputs that we see in ChatGPT and similar.
Knowing that the concept space of LLMs has similar concepts clustered, it makes sense that these errors would force the LLM to sometimes make associations on the fly between close concepts, associations that it didn’t have trained for before, and which “derail” it into a close, but not exactly the same, train of thought.
This behavior also seems to be what we call “intelligence” in humans: the ability to solve problems not seen before (zero shot).
A further extension would be the ability to constantly learn from every interaction. Right now LLMs have a “context” of some length, that changes dynamically, but has no influence over the pre-trained network.
Interestingly, this has a parallel in “crystallized intelligence” vs. “fluid intelligence” in humans.
So… maybe LLMs are not full AGIs yet, but they are showing many of the behaviors that we would expect from an AGI, while at the same time giving or confirming insights into the workings of the human mind itself.