11 ms·

You are so narrowly fixated on this one specific domain which is an edge case with LLMs. ChatGPT was trained specifically to solve "natural language understand

You are so narrowly fixated on this one specific domain which is an edge case with LLMs. ChatGPT was trained specifically to solve "natural language understanding tasks". You are missing the forest for the trees and ignoring everything else it's good at solving outside of its training set. The emergent properties of neural networks should not be so casually dismissed. You're arguing that since its not perfect at arithmetic reasoning then its not capable of any degree of reasoning in any domain. That is an oversimplification and just doesn't make logical sense.

PaulDavisThe1st 3y ago
Actually, I'm not narrowly fixated at all. I do not believe that any part of any current or future LLM (i.e. using the same fundamental architecture) is capable of reasoning, or in fact, capable of anything other than, essentially, doing a really, really, really good job of generating the next word in a response. I am all about emergent properties of neural networks, but I absolutely do not believe that LLMs have them, specifically because of the way they are designed. However, as to the specifics, people who seem to believe otherwise claim that they can reason, and so that's merely one specific angle of attack: to show that they cannot reason, and that in fact, everything they do is implicitly contained in their training set. As I've already said, what they can do is enormously powerful and in many (most?) ways entirely unexpected, so I still regard the advent of LLMs as extremely significant, both from a practical but also a scientific point of view. I think it may force a revision in the most basic aspects of understanding human speech behavior, for example. Nevertheless, I do not believe that anybody is served by believing that these systems can do things that they cannot. I do not understand why the seemingly magic results of these systems is leading so many into a denial of what they actually do.
- Mike_12345 3y ago
  > I am all about emergent properties of neural networks, but I absolutely do not believe that LLMs have them, specifically because of the way they are designed. Even when faced with evidence that contradicts your beliefs and proves that you are wrong? LLMs are a type of neural network. What fundamentally prevents LLMs from having emergent abilities while other neural networks do have them? How do you explain the emergent abilities that we have actually observed in LLMs?
  - PaulDavisThe1st 3y ago
    You don't have any evidence, and neither does any paper I've read claiming as such. The level of naievete around this stuff is quite incredible. BEFORE: "We have no idea really what a system like this will do" ... amazing results ... AFTER: "Wow, look at those emergent properties"
    - Mike_12345 3y ago
      What do you mean we don't have evidence? It has already been presented to you. You choose to reject it for reasons I truly don't understand. Search on Google Scholar if you want a more academic explanation. You can try it out on ChatGPT right now and see it yourself. > The level of naievete around this stuff is quite incredible. "Emergent properties" has a formal definition in the literature. In machine learning we don't explicitly program the machine to understand anything. It automatically learns patterns in the data. That's the entire point of machine learning. With such a large neural network and training set obviously it's hard to predict all of its capabilities due to the sheer scale of it all. Of course we cannot predict exactly how it will model things. Take this for example. No one programmed it to understand Go. It learned by itself and became the world champion. That's what deep learning is capable of. https://en.wikipedia.org/wiki/AlphaGo_Zero https://en.wikipedia.org/wiki/AlphaGo_Zero "The neural network initially knew nothing about Go beyond the rules."
    - Mike_12345 3y ago
      Here is an example of an emergent ability of ChatGPT that you can try yourself right now. Give it this prompt: "Write a short play where "Karen" (who behaves as the Karen meme) is on a romantic date with Hunter S. Thompson. They are eating at a Chinese restaurant." It is able to script their interaction and dialog in a way that makes sense in the context of the setting and their personalities, including an absurd meme character. That is an emergent ability that it was not trained to do and is certainly not in its training set. Try it. Add different characters. Ask it to rewrite the play using pirate metaphors. You can go deep into its "mind" and see the emergent abilities at play. Just apply some creativity and skip the boring arithmetic problems, as that's a well accepted weakness of this type of model.
- Mike_12345 3y ago
  Solving natural language understanding tasks requires reasoning, by definition. I think you are sticking to a narrow definition of reasoning that is not very technical. Formally there are many types of reasoning in AI.
  - PaulDavisThe1st 3y ago
    But LLMs do not solve natural language understanding in any of the meanings that the phrase meant before LLMs. Instead, they throw a completely new technique at it that completely sidesteps the need for language understanding and what do you know? For the purposes of responding in meaningful, generally sensible ways, it works amazingly well. And that is incredibly cool. But it doesn't solve the (all) problem(s) that more historical approaches to machine language "understanding" were concerned with. But there is no world representation inside an LLM, only text (words, letters) representations, so nothing the LLM does can be based on reasoning in a traditional sense. I would wager that if we build an LLM based on a training data set collection, and then we rebuild it with a heavily edited version of the data set that explicitly excludes certain significant areas of human discourse, the LLM will be severely impaired in its apparent ability to "reason" about anything connected with the excluded areas. That sounds as if it ought to surprise you, since you think they are capable of reasoning beyond the training set. It wouldn't surprise me at all, since I do not believe that it what they are doing. LLMs contain a model of human speech (really text) behavior that is almost unimaginably more complex than anything we've built before. But by itself that doesn't mean very much with respect to general reasoning ability. The fact that LLMs can convince you otherwise points, to me, to the richness of the training data in suitable responses to almost any prompt,suitable, that is, for the purpose of persuading you that there is some kind of reasoning occuring. But there is not. The fact that neither you nor I can really build a model (hah!) of what the LLM is actually doing doesn't change that.
    - Mike_12345 3y ago
      > But LLMs do not solve natural language understanding in any of the meanings that the phrase meant before LLMs. Are you saying that NLP as a field of research did not exist before LLMs? This is a continuation of research that has been in progress for decades. > But there is no world representation inside an LLM, only text (words, letters) representations, so nothing the LLM does can be based on reasoning in a traditional sense. Not true. The model has learned a representation of semantic relationships between words and concepts at multiple levels of abstraction. That is the entire point. That's what is was trained to do. It's a vast and deep neural network with a very high dimensional representation of the data. Those semantic/meaning relations are automatically learned and encoded in the model.

PaulDavisThe1st 3y ago

Actually, I'm not narrowly fixated at all. I do not believe that any part of any current or future LLM (i.e. using the same fundamental architecture) is capable of reasoning, or in fact, capable of anything other than, essentially, doing a really, really, really good job of generating the next word in a response. I am all about emergent properties of neural networks, but I absolutely do not believe that LLMs have them, specifically because of the way they are designed. However, as to the specifics, people who seem to believe otherwise claim that they can reason, and so that's merely one specific angle of attack: to show that they cannot reason, and that in fact, everything they do is implicitly contained in their training set. As I've already said, what they can do is enormously powerful and in many (most?) ways entirely unexpected, so I still regard the advent of LLMs as extremely significant, both from a practical but also a scientific point of view. I think it may force a revision in the most basic aspects of understanding human speech behavior, for example. Nevertheless, I do not believe that anybody is served by believing that these systems can do things that they cannot. I do not understand why the seemingly magic results of these systems is leading so many into a denial of what they actually do.

Mike_12345 3y ago
> I am all about emergent properties of neural networks, but I absolutely do not believe that LLMs have them, specifically because of the way they are designed. Even when faced with evidence that contradicts your beliefs and proves that you are wrong? LLMs are a type of neural network. What fundamentally prevents LLMs from having emergent abilities while other neural networks do have them? How do you explain the emergent abilities that we have actually observed in LLMs?
- PaulDavisThe1st 3y ago
  You don't have any evidence, and neither does any paper I've read claiming as such. The level of naievete around this stuff is quite incredible. BEFORE: "We have no idea really what a system like this will do" ... amazing results ... AFTER: "Wow, look at those emergent properties"
  - Mike_12345 3y ago
    What do you mean we don't have evidence? It has already been presented to you. You choose to reject it for reasons I truly don't understand. Search on Google Scholar if you want a more academic explanation. You can try it out on ChatGPT right now and see it yourself. > The level of naievete around this stuff is quite incredible. "Emergent properties" has a formal definition in the literature. In machine learning we don't explicitly program the machine to understand anything. It automatically learns patterns in the data. That's the entire point of machine learning. With such a large neural network and training set obviously it's hard to predict all of its capabilities due to the sheer scale of it all. Of course we cannot predict exactly how it will model things. Take this for example. No one programmed it to understand Go. It learned by itself and became the world champion. That's what deep learning is capable of. https://en.wikipedia.org/wiki/AlphaGo_Zero https://en.wikipedia.org/wiki/AlphaGo_Zero "The neural network initially knew nothing about Go beyond the rules."
  - Mike_12345 3y ago
    Here is an example of an emergent ability of ChatGPT that you can try yourself right now. Give it this prompt: "Write a short play where "Karen" (who behaves as the Karen meme) is on a romantic date with Hunter S. Thompson. They are eating at a Chinese restaurant." It is able to script their interaction and dialog in a way that makes sense in the context of the setting and their personalities, including an absurd meme character. That is an emergent ability that it was not trained to do and is certainly not in its training set. Try it. Add different characters. Ask it to rewrite the play using pirate metaphors. You can go deep into its "mind" and see the emergent abilities at play. Just apply some creativity and skip the boring arithmetic problems, as that's a well accepted weakness of this type of model.
Mike_12345 3y ago
Solving natural language understanding tasks requires reasoning, by definition. I think you are sticking to a narrow definition of reasoning that is not very technical. Formally there are many types of reasoning in AI.
- PaulDavisThe1st 3y ago
  But LLMs do not solve natural language understanding in any of the meanings that the phrase meant before LLMs. Instead, they throw a completely new technique at it that completely sidesteps the need for language understanding and what do you know? For the purposes of responding in meaningful, generally sensible ways, it works amazingly well. And that is incredibly cool. But it doesn't solve the (all) problem(s) that more historical approaches to machine language "understanding" were concerned with. But there is no world representation inside an LLM, only text (words, letters) representations, so nothing the LLM does can be based on reasoning in a traditional sense. I would wager that if we build an LLM based on a training data set collection, and then we rebuild it with a heavily edited version of the data set that explicitly excludes certain significant areas of human discourse, the LLM will be severely impaired in its apparent ability to "reason" about anything connected with the excluded areas. That sounds as if it ought to surprise you, since you think they are capable of reasoning beyond the training set. It wouldn't surprise me at all, since I do not believe that it what they are doing. LLMs contain a model of human speech (really text) behavior that is almost unimaginably more complex than anything we've built before. But by itself that doesn't mean very much with respect to general reasoning ability. The fact that LLMs can convince you otherwise points, to me, to the richness of the training data in suitable responses to almost any prompt,suitable, that is, for the purpose of persuading you that there is some kind of reasoning occuring. But there is not. The fact that neither you nor I can really build a model (hah!) of what the LLM is actually doing doesn't change that.
  - Mike_12345 3y ago
    > But LLMs do not solve natural language understanding in any of the meanings that the phrase meant before LLMs. Are you saying that NLP as a field of research did not exist before LLMs? This is a continuation of research that has been in progress for decades. > But there is no world representation inside an LLM, only text (words, letters) representations, so nothing the LLM does can be based on reasoning in a traditional sense. Not true. The model has learned a representation of semantic relationships between words and concepts at multiple levels of abstraction. That is the entire point. That's what is was trained to do. It's a vast and deep neural network with a very high dimensional representation of the data. Those semantic/meaning relations are automatically learned and encoded in the model.