The Chinese Broom - offmark - Obsidian Publish

Version 0.2 Let this be interpreted as a light-heartedly jokey, but also seriously critical modification of the original [[The Chinese Room|chinese room thought experiment]]. Modified parts are underlined, while the rest remain the same, as originally proposed by Searle in his book [Minds, Brains, and Programs (1980)](https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/minds-brains-and-programs/DC644B47A4299C637C89772FACC2706A). ![[kd0yh30w8ole1.jpeg]] 💭Remains of a broom. Gansu, China, Han dynasty, 200 BC-200 AD ------------ Suppose that I’m locked in a room and given a large batch of Chinese writing. Suppose furthermore (as is indeed the case) that I know no Chinese, either written or spoken, and that I’m not even confident that I could recognize Chinese writing as Chinese writing distinct from, say, Japanese writing or meaningless squiggles. To me, Chinese writing is just so many meaningless squiggles. Now suppose further that after this first batch of Chinese writing I am given a second batch of Chinese script together with a set of rules for correlating the second batch with the first batch. The rules are in English, and I understand these rules as well as any other native speaker of English. They enable me to correlate one set of formal symbols with another set of formal symbols, and all that ‘formal’ means here is that I can identify the symbols entirely by their shapes. Now suppose also that I am given a third batch of Chinese symbols together with some instructions, again in English, that enable me to correlate elements of this third batch with the first two batches, and these rules instruct me how to give back certain Chinese symbols with certain sorts of shapes in response to certain sorts of shapes given me in the third batch. Unknown to me, the people who are giving me all of these symbols call the first batch “a script,” they call the second batch a “story,” and they call the third batch “questions.” Furthermore, they call the symbols I give them back in response to the third batch “answers to the questions.” and the set of rules in English that they gave me, they call “the program.” Now just to complicate the story a little, imagine that these people also give me stories in English, which I understand, and they then ask me questions in English about these stories, and I give them back answers in English. Suppose also that after a while I get so good at following the instructions for manipulating the Chinese symbols and the programmers get so good at writing the programs that from the external point of view that is, from the point of view of somebody outside the room in which I am locked—my answers to the questions are absolutely indistinguishable from those of native Chinese speakers. Nobody just looking at my answers can tell that I don’t speak a word of Chinese. Let us also suppose that my answers to the English questions are, as they no doubt would be, indistinguishable from those of other native English speakers, for the simple reason that I am a native English speaker. From the external point of view— from the point of view of someone reading my “answers”—the answers to the Chinese questions and the English questions are equally good. But in the Chinese case, unlike the English case, I produce the answers by manipulating uninterpreted formal symbols. As far as the Chinese is concerned, I simply behave like a simple computer program; I perform computational operations on formally specified elements, and the processing may not even require Turing completeness, it could be a simple very large lookup table. For the purposes of the Chinese, I am simply an instantiation of the computer program. Now let's suppose, that people from outside send me questions colored in two distinct ways: green and red. Whenever I encounter a green question, I do things as mentioned above. However, when I encounter a red question, I do things a bit differently by following [[Inference Engine|this algorithm instead]]. The algorithm completely ignores the original "program". Instead, it allows us to create our own program, which will in turn generate a "dictionary". We are allowed to modify the "dictionary" according to the new program every time we receive a body of Chinese text, be it a new story or question of any color. The moment before we start receiving input, "the dictionary" will be empty, because we haven't had the chance to write anything to it yet. After the first input, our dictionary will be absolutely ambiguous - every Chinese symbol or combination of symbols we've encountered so far could be mapped to any English concept. As we receive more inputs and work on our "dictionary", we will minimize ambiguity, and our answers to red questions will result in significantly less gibberish. It is expected, that at some point we will arrive at an almost perfect mapping between Chinese and English symbols, where such mappings exist. At that point we will not need any more training, and we'll just enjoy reducing the complexity of the program to a simple lookup table. What will people outside observe at different times, and is there any understanding generated? 1. After the first question: - Answers to green questions will be perfect; No understanding is generated. - Answers to red questions will be horrid; No understanding is yet generated, because we haven't inferred anything yet. Our dictionary is completely ambiguous at this point. 2. After the second question: - Answers to green questions will be perfect; No understanding is generated. - Answers to red questions will still be horrid; Some understanding is generated because ambiguity is reduced. Since we understand English by definition, it is safe to assume that we now understand a bit of Chinese as well. The less ambiguity we have in the dictionary, the more understanding is generated. 3. After the Nth question: - Answers to green questions will be perfect; No understanding is generated. - Answers to red questions will be almost perfect; Almost full understanding is generated. In the rare cases of Chinese concepts that have no equivalence in English ambiguity will remain until we visit China and fill up our ontology with more real-life experience. Alternatively, we can also use an "imaginary" concept as described [[How daltonists understand red|here]], but this has no real relevance to the experiment at hand. Now here's the kicker. If, by chance or otherwise, we eventually arrive at the same instructions for responding as the original program had, can it be said that the initial program had understanding all along? In one case we use a program supplied from outside, designed allegedly by humans, and all we are doing is executing the program. In the second case we use a program supplied from inside, designed by us (a human), and all we are doing after all training is complete, is just executing the program. End of experiment. ------------ Now lets analyze this in depth: Q: Can we correlate the quality of answers with the existence of understanding? A: Absolutely not! Q: Can we correlate the quality of answers with the existence of intelligence? A: No. A kid learning a new language will do many mistakes. This has no relation to their potential to learn, reason, and become intelligent. Q: Can we say, that following formal instructions can't produce understanding? A: Absolutely not! It is not the formal instructions that impede understanding. It is the lack of ontology! If you read my algorithm you will notice, that the algorithm essentially ["links"](https://en.wikipedia.org/wiki/Linker_(computing)) with the human ontology of the operator. This is the missing link in Searle's experiment - he takes away the operator's ability to construct semantic meaning, and then states that understanding is impossible (duh), but for the wrong reasons. He wrongly attributes the lack of understanding to the utilization of formal instructions. Q: But you admit he successfully demonstrated that there's something missing? A: No. First, he fails to acknowledge, that understanding has always been there, just not available to the operator, because it was used at design-time in the creation of the program the operator uses. He made it seem as if the formal instructions are the reason for the lack of understanding, which is fundamentally wrong. He avoided to specify what the missing link actually is, and left it to people's imagination, allowing many religiously-inclined people to entertain some epiphenomenological reasons, or quantum mumbo-jumbo. He introduced more noise, instead of reducing it. His experiment doesn't touch on the subject of whether an [ontology](https://en.wikipedia.org/wiki/Semantics#Formal_semantics) can be formalized or not. [Computationalists](https://en.wikipedia.org/wiki/Computational_theory_of_mind) believe that it can, which would suggest that you can replace the human operator with a precompiled human ontology instead. Only thing Searle is right about, is that there still has to be a human element, but it is required only in the creation of the system, not for "running" it. But this is just stating the obvious - there is no conceivable way an AGI can manifest itself out of nothing. Evolution is the missing link when it comes to creating a mind, both real and artificial. But evolution is not a requirement for __running__ a mind, real or artificial. Evolution made us capable of thinking, but doesn't necessarily participate in it. Q: So, if we're allowed to generate our own program, and we arrive at the same formal rules, and we start simply executing them when training is done, could it be said that we have understanding? A: Yes. We understand English, and all we did is create a mapping from English to Chinese. Therefore we understand Chinese as well. Maybe not perfectly, but in a not-so-perfect human (realistic) way. Q: But when using rules generated by someone else, we don't have understanding? A: Yes and no. Searle really wants this to be true, but no. If we have arrived to the same rules through understanding, then we would have understanding about these rules in general, no matter where they physically come from. I've heard polyglots say that it gets easier after they learn the fifth language - they start seeing reusable patterns, and each new language maps ever more neatly onto their previous experience. If the operator is a polyglot + computational linguist capable of seeing the reasons behind the program's instructions, then it is safe to assume some understanding is evident. Q: So it all boils down to whether human ontology is formalizable or not? A: Yes. Ontology not being expressible in finite terms is a much better reason for the impossibility of AGI. It all comes down to whether the human mind generates ontology by using natural processes directly, or after them being modulated, reduced and "virtualized". See [[Why virtualization is an important component of intelligence]] to see why evidence suggests the latter. It is likely, that our model of the world is already formalized the moment it enters our mind, in the sense that it already becomes a finite representation of the outside world, and is as complex as our capabilities to comprehend it. Then it is a question of just another mapping function. Searle didn't clarify how plausible that is, even when he had the chance in his robot reply. Q: What would cause the human ontology to not be formalizable? A: It would be because the human mind isn't virtualized and uses fundamental physical processes to operate. Then it would be [computationally irreducible](https://en.wikipedia.org/wiki/Computational_irreducibility). All we could ever arrive at will be approximations, just the same way our physical simulations are always just approximations - utterly devoid of any real physical events, except the ones that happen in the computing networks themselves (there's still the chance that the computing networks can replicate the phenomena, but this is out of the scope of this discussion). It is important to state, however, that this turn of events is highly unlikely, as current evidence suggests the opposite.