5 ms·

Just some food for thought: you’re anthropomorphizing a machine, it doesn’t understand like humans do It’s not going to learn more ‘with time,’ either

kilgnad 4y ago
It does. Did you follow the link in my original post. https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read it to the end. The end is quite mind blowing. I'm not sure how someone can finish this article and not realize that chatGPT actually completely understands what you're telling it.
- jodrellblank 4y ago
  In some ways it's amazing, in other ways it's like saying that if I gave my mother a cue card saying "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver" and she read that back to me when I said "nvidia-smi" then she must be running a virtual machine in her head and querying it for a GPU. Or if she read "ping reply 14ms" off a notepad she must be simulating ICMP echo and understand networking. She clearly wouldn't be. ChatGPT has zillions of web pages as cue cards. It will have many examples of ping and reply, of first ten prime numbers in Python, of Docker output. Was it actually building a Docker file inside it in any way? I asked it a similar prompt, I'm amazed at how much it can get right, I've never seen any other program or computer do what it does. At the same time, it's not implementing the things behind the scenes that we're asking it to and more complicated prompts makes that visible - it's not actually a CPython runtime or a Linux shell or curl'ing a web page; this is the disconnected lever fallacy I mentioned earlier. "If I copy the interface to a warp drive, that will give me a warp drive, if I copy the interface to a Linux machine, that's the same as being a Linux machine". It's also akin to Searle's Chinese Room - at some point it may be good enough to be indistinguishable from a Linux machine and that would render this argument irrelevant.
  - scarface74 4y ago
    I’ve given it much more complicated prompts to create a specific Python script with very specific requirements and it did pretty well. I’ve even given it a Python script I wrote that took in a JSON file And asked it to make sone changes to the script. It does far more than just mindlessly return results.
  - kilgnad 4y ago
    There's no example cue cards of itself. In, the ending of that article, chatGPT curled the internet from the virtual machine. It specifically curled the chatGPT api and talked to itself. Then it asked itself to create another virtual machine, and the virtual chatGPT on the virtual internet on the virtual machine created another virtual machine.... The end... It's not actually creating any of these things. It's imitating these things indicating it Understands what it is. It imitated itself. Indicating an awareness of self. aka self awareness.
    - anonymouskimmer 4y ago
      > "It imitated itself. Indicating an awareness of self. aka self awareness." The mirror test is a stronger indicator of self-awareness than imitation of a like entity.
      - kilgnad 4y ago
        [dead]
    - threeseed 4y ago
      > It imitated itself. Indicating an awareness of self. aka self awareness. We create algorithms all the time that can identify and imitate other algorithms. That doesn't make them self aware in the normal use of the term.
      - kilgnad 4y ago
        [dead]
- LoganDark 4y ago
  > I'm not sure how someone can finish this article and not realize that chatGPT actually completely understands what you're telling it. ChatGPT is a predictive language model. It understands nothing. It simply tries to mimic its training data. It produces output that mimics the output of someone who understands, but does not understand itself. To clarify, there is no understanding happening. That is why language models hallucinate so convincingly. Because they are able to create convincing output without understanding.
  - jodrellblank 4y ago
    Or maybe they hallucinate so convincingly because they do understand, but they don't understand much? What is this distinction you make "output that mimics the output of someone who understands, but does not understand itself." ? Imagine you learning a foreign language, the Common European Framework of Reference for Languages (CEFR) grades people at their skill from A1 (beginner) through A2, B1, B2, C1, to C2 (fluent). At the start you are repeating phrases you don't understand imitating someone who does, but you cannot change the phrases at all because you don't know more words and cannot change the grammar because you don't understand it. Call this a chatbot with hard coded strings it can print. After a while, you can fit some other words in the basic sentences. Call this a chatbot which has template strings and lists of nouns it can substitute in, Eliza style "tell me about {X}" where X is a fixed list of {your mother, your childhood, yourself}. After a bit longer you can make basic use of grammar. If you get to fluent you can make arbitrary sentences with arbitrary words and see new words you have never seen before and guess how they will conjugate, whether they are polite or slang from the context, what they might mean from other languages, and use them probably correctly. ChatGPT can make new sentences in English, new words, it can make plausible explanations of words it has never seen before - make up a word like "carstool" and it can say something like that word does not exist but if it did it could be a compound word of 'car' and 'stool' like 'carseat', a car with a stool for a seat. Ask it to make up new compound words and it can say English does not have compound words made of four words, but if it had some examples might be trainticketprintingmachine (a machine for printing train tickets). Something that a complaete beginner in a foreign language could never do until they gained some understanding. Something that an Eliza chatbot could never do.
ukj 4y ago
But do humans "really" understand? When you claim that you "understand" what is it really that you are claiming about yourself? Is the claim testbale/falsifiable? I know humans understands me because they behave in a way that 's congruent with my expectations of "understanding". e.g compliance, cooperation etc. But then Alexa also complies and cooperates with me. I also know when humans don't understand - when they are combative, uncooperative, argumentative etc. even when I say incredibly sensible and obvious things.
- kilgnad 4y ago
  We can't even define the word. It's just a point of comparison. Humans are doing this sort of thing we don't fully comprehend called understanding. The main question here is that if LLMs are doing something that can be described as isomorphic to the concept of understanding. Hence what I mean by point of comparison.
nextaccountic 4y ago
> It’s not going to learn more ‘with time,’ either Perhaps not this model, but future models will be better.

kilgnad 4y ago

It does. Did you follow the link in my original post. https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read it to the end. The end is quite mind blowing. I'm not sure how someone can finish this article and not realize that chatGPT actually completely understands what you're telling it.

jodrellblank 4y ago
In some ways it's amazing, in other ways it's like saying that if I gave my mother a cue card saying "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver" and she read that back to me when I said "nvidia-smi" then she must be running a virtual machine in her head and querying it for a GPU. Or if she read "ping reply 14ms" off a notepad she must be simulating ICMP echo and understand networking. She clearly wouldn't be. ChatGPT has zillions of web pages as cue cards. It will have many examples of ping and reply, of first ten prime numbers in Python, of Docker output. Was it actually building a Docker file inside it in any way? I asked it a similar prompt, I'm amazed at how much it can get right, I've never seen any other program or computer do what it does. At the same time, it's not implementing the things behind the scenes that we're asking it to and more complicated prompts makes that visible - it's not actually a CPython runtime or a Linux shell or curl'ing a web page; this is the disconnected lever fallacy I mentioned earlier. "If I copy the interface to a warp drive, that will give me a warp drive, if I copy the interface to a Linux machine, that's the same as being a Linux machine". It's also akin to Searle's Chinese Room - at some point it may be good enough to be indistinguishable from a Linux machine and that would render this argument irrelevant.
- scarface74 4y ago
  I’ve given it much more complicated prompts to create a specific Python script with very specific requirements and it did pretty well. I’ve even given it a Python script I wrote that took in a JSON file And asked it to make sone changes to the script. It does far more than just mindlessly return results.
- kilgnad 4y ago
  There's no example cue cards of itself. In, the ending of that article, chatGPT curled the internet from the virtual machine. It specifically curled the chatGPT api and talked to itself. Then it asked itself to create another virtual machine, and the virtual chatGPT on the virtual internet on the virtual machine created another virtual machine.... The end... It's not actually creating any of these things. It's imitating these things indicating it Understands what it is. It imitated itself. Indicating an awareness of self. aka self awareness.
  - anonymouskimmer 4y ago
    > "It imitated itself. Indicating an awareness of self. aka self awareness." The mirror test is a stronger indicator of self-awareness than imitation of a like entity.
    - kilgnad 4y ago
      [dead]
  - threeseed 4y ago
    > It imitated itself. Indicating an awareness of self. aka self awareness. We create algorithms all the time that can identify and imitate other algorithms. That doesn't make them self aware in the normal use of the term.
    - kilgnad 4y ago
      [dead]
LoganDark 4y ago
> I'm not sure how someone can finish this article and not realize that chatGPT actually completely understands what you're telling it. ChatGPT is a predictive language model. It understands nothing. It simply tries to mimic its training data. It produces output that mimics the output of someone who understands, but does not understand itself. To clarify, there is no understanding happening. That is why language models hallucinate so convincingly. Because they are able to create convincing output without understanding.
- jodrellblank 4y ago
  Or maybe they hallucinate so convincingly because they do understand, but they don't understand much? What is this distinction you make "output that mimics the output of someone who understands, but does not understand itself." ? Imagine you learning a foreign language, the Common European Framework of Reference for Languages (CEFR) grades people at their skill from A1 (beginner) through A2, B1, B2, C1, to C2 (fluent). At the start you are repeating phrases you don't understand imitating someone who does, but you cannot change the phrases at all because you don't know more words and cannot change the grammar because you don't understand it. Call this a chatbot with hard coded strings it can print. After a while, you can fit some other words in the basic sentences. Call this a chatbot which has template strings and lists of nouns it can substitute in, Eliza style "tell me about {X}" where X is a fixed list of {your mother, your childhood, yourself}. After a bit longer you can make basic use of grammar. If you get to fluent you can make arbitrary sentences with arbitrary words and see new words you have never seen before and guess how they will conjugate, whether they are polite or slang from the context, what they might mean from other languages, and use them probably correctly. ChatGPT can make new sentences in English, new words, it can make plausible explanations of words it has never seen before - make up a word like "carstool" and it can say something like that word does not exist but if it did it could be a compound word of 'car' and 'stool' like 'carseat', a car with a stool for a seat. Ask it to make up new compound words and it can say English does not have compound words made of four words, but if it had some examples might be trainticketprintingmachine (a machine for printing train tickets). Something that a complaete beginner in a foreign language could never do until they gained some understanding. Something that an Eliza chatbot could never do.

ukj 4y ago

But do humans "really" understand? When you claim that you "understand" what is it really that you are claiming about yourself? Is the claim testbale/falsifiable? I know humans understands me because they behave in a way that 's congruent with my expectations of "understanding". e.g compliance, cooperation etc. But then Alexa also complies and cooperates with me. I also know when humans don't understand - when they are combative, uncooperative, argumentative etc. even when I say incredibly sensible and obvious things.

kilgnad 4y ago
We can't even define the word. It's just a point of comparison. Humans are doing this sort of thing we don't fully comprehend called understanding. The main question here is that if LLMs are doing something that can be described as isomorphic to the concept of understanding. Hence what I mean by point of comparison.

nextaccountic 4y ago

> It’s not going to learn more ‘with time,’ either Perhaps not this model, but future models will be better.