11 ms·

I can, right now, have GPT 4 write me sonnets on any topic. We are already beyond mere parroting. People say "it's just parroting what it's heard"! But did GPT

I can, right now, have GPT 4 write me sonnets on any topic. We are already beyond mere parroting. People say "it's just parroting what it's heard"! But did GPT4's training data really include a sonnet about Minecraft creepers? So they're not parroting. They're parroting PLUS something else. Plus some kind of analysis and transformation. Your claim is that this transformation is superficial and won't become deeper as the neural networks become deeper. We'll see.

lm28469 3y ago
Yes you can cherry pick things that work very well, in the meantime it still can't rhyme in French, which is 12 years old type of intelligence It's wrong > 50% of the time for anything semi complex and semi niche, that's not AGI, no one in the field of AI argue it's AGI, only people on twitter and reddit with absolutely no background in tech and/or AI argue it's AGI It's a very very very good chatbot, built with very very impressive technology and amount of data, which is already quite impressive and definitely a big milestone, it's not AGI, not even close It's like saying the leather auger of my swiss army knife is a step towards an aerospace grade titanium cutting CNC machine
- dandellion 3y ago
  I can't rhyme in French either. A lot of these comments just remind me of that Alan Kay quote: “Artificial intelligence is what we don’t know how to do yet.” In the past year we've got models that can draw, write poetry and code better than a significant percentage of people, and now the goal posts are moving so fast they're even hard to see. I'd probably agree that we're still a few breakthroughs away from something that could pass as AGI, but the current models are already better than just "parroting". And even if progress stales for the next 10 or 15 years like it has happened frequently with AI, everything that it has learned to do in the last couple years can be fine-tuned and made a lot more efficient, which I expect will have a big impact already.
  - lm28469 3y ago
    I have a really really really hard time thinking we're even remotely close to AGI when I see stuff like: https://imgur.com/eukW66W https://imgur.com/eukW66W You must have a really convoluted definition of AGI to come to the conclusion gpt is AGI when it proves times and times again it quite literally have no understanding of what it writes
    - sharemywin 3y ago
      see it's already passed the Turing test: https://sports.yahoo.com/24-most-hilariously-bad-spelling-191602032.html https://sports.yahoo.com/24-most-hilariously-bad-spelling-19...
    - dandellion 3y ago
      I said that "I think we are a few breakthroughs away from something could pass as AGI". That means, first, that I don't think we're there and so I don't consider chatgpt AGI, and second that even something that "passes" as AGI at first glance might not necessarily be so under close inspection. But at the same time, I won't call what chatgpt is doing parroting. It's not just repeating input data as-is, at least it's capable of applying some transformations on that data. Most of the time the transformation is useful, occasionally it's pretty borked. Parroting to me would be just repeating the training data without much transformation. > it quite literally have no understanding of what it writes Even though I DON'T think gpt AGI, I don't consider that a deciding factor. Humans can also show no understanding of what we write or read, and I consider us "GI".
marcosdumay 3y ago
> But did GPT4's training data really include a sonnet about Minecraft creepers? The entire internet is one hell of a large knowledge base. It does surely look like it's doing some transformation. But I don't know how to get certainty on this.

lm28469 3y ago

Yes you can cherry pick things that work very well, in the meantime it still can't rhyme in French, which is 12 years old type of intelligence It's wrong > 50% of the time for anything semi complex and semi niche, that's not AGI, no one in the field of AI argue it's AGI, only people on twitter and reddit with absolutely no background in tech and/or AI argue it's AGI It's a very very very good chatbot, built with very very impressive technology and amount of data, which is already quite impressive and definitely a big milestone, it's not AGI, not even close It's like saying the leather auger of my swiss army knife is a step towards an aerospace grade titanium cutting CNC machine

dandellion 3y ago
I can't rhyme in French either. A lot of these comments just remind me of that Alan Kay quote: “Artificial intelligence is what we don’t know how to do yet.” In the past year we've got models that can draw, write poetry and code better than a significant percentage of people, and now the goal posts are moving so fast they're even hard to see. I'd probably agree that we're still a few breakthroughs away from something that could pass as AGI, but the current models are already better than just "parroting". And even if progress stales for the next 10 or 15 years like it has happened frequently with AI, everything that it has learned to do in the last couple years can be fine-tuned and made a lot more efficient, which I expect will have a big impact already.
- lm28469 3y ago
  I have a really really really hard time thinking we're even remotely close to AGI when I see stuff like: https://imgur.com/eukW66W https://imgur.com/eukW66W You must have a really convoluted definition of AGI to come to the conclusion gpt is AGI when it proves times and times again it quite literally have no understanding of what it writes
  - sharemywin 3y ago
    see it's already passed the Turing test: https://sports.yahoo.com/24-most-hilariously-bad-spelling-191602032.html https://sports.yahoo.com/24-most-hilariously-bad-spelling-19...
  - dandellion 3y ago
    I said that "I think we are a few breakthroughs away from something could pass as AGI". That means, first, that I don't think we're there and so I don't consider chatgpt AGI, and second that even something that "passes" as AGI at first glance might not necessarily be so under close inspection. But at the same time, I won't call what chatgpt is doing parroting. It's not just repeating input data as-is, at least it's capable of applying some transformations on that data. Most of the time the transformation is useful, occasionally it's pretty borked. Parroting to me would be just repeating the training data without much transformation. > it quite literally have no understanding of what it writes Even though I DON'T think gpt AGI, I don't consider that a deciding factor. Humans can also show no understanding of what we write or read, and I consider us "GI".

marcosdumay 3y ago

> But did GPT4's training data really include a sonnet about Minecraft creepers? The entire internet is one hell of a large knowledge base. It does surely look like it's doing some transformation. But I don't know how to get certainty on this.