8 ms·

Just because it's newly created doesn't mean that the structure of the language and the concepts it represents are actually new. It's clear that whatever tests

Just because it's newly created doesn't mean that the structure of the language and the concepts it represents are actually new. It's clear that whatever tests he writes cover well established and understood concepts. This is where I believe people are missing the point. GPT4 is not a general intelligence. It is a highly overfit model, but it's overfit to literally every piece of human knowledge. Language is humanities way of modelling real world concepts. So GPT is able to leverage the relationships we create through our language to real world concepts. It's just learned all language up until today. It's an incredible knowledge retrieval machine. It can even mimick how our language is used to conduct reasoning very well. It can't do this efficiently, nor can it actually stumble upon a new insight because it's not being exposed in real time to the real world. So, this professors 'new' test is not really new. It's just a test that fundamentally has already been modelled.

xyzzy123 3y ago
Don't students prepare for tests by studying past instances of them? "Teaching the test" (aka overfitting of human students at the expense of "real" learning) is a common complaint about our current education system. Do you think it doesn't "deserve" an A here?
- rafiki6 3y ago
  Did I say that? The OP's post was saying it's somehow able to solve something new. It's showing a severe misunderstanding how how language modelling works.
famouswaffles 3y ago
Watching posts shift in real time is very entertaining. First it's not generally intelligent because it can't tackle new things then when it obviously does its not generally intelligent because it's overfit. You've managed to essentially say nothing of substance. So it passes because structure and concepts are similar. okay. are students preparing for tests working with alien concepts and structures then because i'm failing to see the big difference here. A model isn't overfit because you've declared it so. and unless GPT-4 is several trillion parameters, general overfitting is severely unlikely. But i doubt you care about any of that. Can you devise a test to properly asses what you're asserting ?
- greesil 3y ago
  I agree with the parent post. I can get ChatGPT to solve a basic world problem but if I add a small wrinkle to it that a human would understand it fails hard. Overfitted seems apt. Yeah it's amazing, but it's not AGI.
  - anonylizard 3y ago
    Stop confusing ChatGPT with GPT-4. Most common rookie mistake. GPT-4 is way stronger at 'solving problems' than ChatGPT. I was baiting ChatGPT with basic logical or conversion problems, I stopped doing that with GPT-4, since it would take too much effort to beat it.
    - kelseyfrog 3y ago
      It's trivially easy even with gpt-4. > Please respond with the number of e's in this sentence. > There are 8 "e" characters in the sentence "Please respond with the number of e's in this sentence."
      - bumbledraven 3y ago
        Dealing with words on the level of their constituent letters is a known weakness of OpenAI’s current GPT models, due to the kind of input and output encoding they use. The encoding also makes working with numbers represented as strings of digits less straightforward than it might otherwise be. In the same way that GPT-4 is better at these things than GPT-3.5, future GPT models will likely be even better, even if only by the sheer brute force of their larger neural networks, more compute, and additional training data. (To see an example of the encoding, you can enter some text at https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer. The input is presented to GPT as a series of integers, one for each colored block.)
        xyzzy123 3y ago
        Almost like it has a kind of dyslexia when it comes to "looking inside" tokens. If you instead ask it to write a Python program to do the same job, it will do it perfectly.
Kranar 3y ago
This is an unusual comment to say the least. It suggests that unless GPT4 can somehow independently derive facts entirely on its own, then it's nothing more than an overfit model, almost as if to say that it's basically just a kind of sophisticated search engine on top of a glorified Wikipedia. Of course that's not actually true, people don't independently invent knowledge either. People study from books or from teachers or other sources of knowledge and internalize it and relate it to other concepts as well, and no one considers that to be a form of overfitting.
tehf0x 3y ago
Ah the good old "it's not me it's the test" argument. These systems are not just next token predictors, they learn complex algorithms and can perform general computation, its just so happens that by asking them to next-token predict the internet they learn a bunch of smart ways to compress everything, potentially in a way similar to how we might use a general concept to avoid memorizing a lookup table. Please have a look at https://arxiv.org/pdf/2211.15661 https://arxiv.org/pdf/2211.15661 and https://mobile.twitter.com/DimitrisPapail/status/1620834409275609088 https://mobile.twitter.com/DimitrisPapail/status/16208344092.... We don't understand everything that's going on yet but it would be foolish to discount anything at this stage, or to state much of anything with any degree of confidence (and that stands for both sides of the opinion spectrum). Also these systems aren't exposed to the real world today, but this will be untrue very soon https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal-language.html?m=1 https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal...
- rafiki6 3y ago
  I never said: - "it's not me it's the test" - "These systems are not just next token predictors" None of the papers or blogs you've shared offer any points that actually rebutt what I'm saying. And yes, we will eventually have them work in real time. Can't wait.
flangola7 3y ago
What would a "new" test look like then?
- qgin 3y ago
  I would certainly be peeved if I showed up to a midterm that asked questions outside of existing human knowledge.
  - flangola7 3y ago
    "I didn't make it into the university I wanted because I didn't invent enough new mathematics during the entry exams."
KyeRussell 3y ago
Given that OpenAI were THEMSELVES surprised by how even GPT-3 ended up, it’s always funny to see HN know-it-alls pipe up with all the answers. These sorts of poorly formed faux-philosophical arguments against LLMs have become the new domain of people that confuse blindly acting skeptical with actual intelligence. Ironic. This latest generation of AI quite rightfully raises questions and challenges assumptions about what it means to be intelligent. It quite rightfully challenges our assumptions about what can be accomplished with language. And, thank God, it quite rightfully challenges assumptions many have made about what sets humanity apart from everything else.
- PaulDavisThe1st 3y ago
  > poorly formed faux-philosophical arguments against LLMs There's a misunderstanding here. The post you're replying to is not an argument against LLMs. It's an argument about what LLMs can and cannot do, what their fundamental capabilities are, and so forth. It's very clear that if you need a system to provide answers based on a substantial body of human writing, LLMs are totally awesome. But that doesn't mean, in and of itself, that they can X or that they can Y.
- Eji1700 3y ago
  > Given that OpenAI were THEMSELVES surprised by how even GPT-3 ended up, Yeah and they have 0 incentive to overhype their takes. OpenAI has already slanted already impressive data in the past to make it more "hype building" for the general public, when a more scientific study style reading is "this is really cool, here's where it still fails". I am very confident shit like that is the same.
qgin 3y ago
I think the hallucinations show that it's not simply overfit to all of human knowledge. To hallucinate, there is a certain amount of generalization and information overlap that is necessary.
pama 3y ago
I’m working in a related area and I’m rather curious about this point. In what way is GPT-4 overfit? Does overfit in this context mean the conventional: validation loss went up with additional training, or something special?
- rafiki6 3y ago
  More specifically validation loss is irrelevant when you can't even sample out of distribution anymore.

xyzzy123 3y ago

Don't students prepare for tests by studying past instances of them? "Teaching the test" (aka overfitting of human students at the expense of "real" learning) is a common complaint about our current education system. Do you think it doesn't "deserve" an A here?

rafiki6 3y ago
Did I say that? The OP's post was saying it's somehow able to solve something new. It's showing a severe misunderstanding how how language modelling works.

famouswaffles 3y ago

Watching posts shift in real time is very entertaining. First it's not generally intelligent because it can't tackle new things then when it obviously does its not generally intelligent because it's overfit. You've managed to essentially say nothing of substance. So it passes because structure and concepts are similar. okay. are students preparing for tests working with alien concepts and structures then because i'm failing to see the big difference here. A model isn't overfit because you've declared it so. and unless GPT-4 is several trillion parameters, general overfitting is severely unlikely. But i doubt you care about any of that. Can you devise a test to properly asses what you're asserting ?

greesil 3y ago
I agree with the parent post. I can get ChatGPT to solve a basic world problem but if I add a small wrinkle to it that a human would understand it fails hard. Overfitted seems apt. Yeah it's amazing, but it's not AGI.
- anonylizard 3y ago
  Stop confusing ChatGPT with GPT-4. Most common rookie mistake. GPT-4 is way stronger at 'solving problems' than ChatGPT. I was baiting ChatGPT with basic logical or conversion problems, I stopped doing that with GPT-4, since it would take too much effort to beat it.
  - kelseyfrog 3y ago
    It's trivially easy even with gpt-4. > Please respond with the number of e's in this sentence. > There are 8 "e" characters in the sentence "Please respond with the number of e's in this sentence."
    - bumbledraven 3y ago
      Dealing with words on the level of their constituent letters is a known weakness of OpenAI’s current GPT models, due to the kind of input and output encoding they use. The encoding also makes working with numbers represented as strings of digits less straightforward than it might otherwise be. In the same way that GPT-4 is better at these things than GPT-3.5, future GPT models will likely be even better, even if only by the sheer brute force of their larger neural networks, more compute, and additional training data. (To see an example of the encoding, you can enter some text at https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer. The input is presented to GPT as a series of integers, one for each colored block.)
      - xyzzy123 3y ago
        Almost like it has a kind of dyslexia when it comes to "looking inside" tokens. If you instead ask it to write a Python program to do the same job, it will do it perfectly.

Kranar 3y ago

This is an unusual comment to say the least. It suggests that unless GPT4 can somehow independently derive facts entirely on its own, then it's nothing more than an overfit model, almost as if to say that it's basically just a kind of sophisticated search engine on top of a glorified Wikipedia. Of course that's not actually true, people don't independently invent knowledge either. People study from books or from teachers or other sources of knowledge and internalize it and relate it to other concepts as well, and no one considers that to be a form of overfitting.

tehf0x 3y ago

Ah the good old "it's not me it's the test" argument. These systems are not just next token predictors, they learn complex algorithms and can perform general computation, its just so happens that by asking them to next-token predict the internet they learn a bunch of smart ways to compress everything, potentially in a way similar to how we might use a general concept to avoid memorizing a lookup table. Please have a look at https://arxiv.org/pdf/2211.15661 https://arxiv.org/pdf/2211.15661 and https://mobile.twitter.com/DimitrisPapail/status/1620834409275609088 https://mobile.twitter.com/DimitrisPapail/status/16208344092.... We don't understand everything that's going on yet but it would be foolish to discount anything at this stage, or to state much of anything with any degree of confidence (and that stands for both sides of the opinion spectrum). Also these systems aren't exposed to the real world today, but this will be untrue very soon https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal-language.html?m=1 https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal...

rafiki6 3y ago
I never said: - "it's not me it's the test" - "These systems are not just next token predictors" None of the papers or blogs you've shared offer any points that actually rebutt what I'm saying. And yes, we will eventually have them work in real time. Can't wait.

flangola7 3y ago

What would a "new" test look like then?

qgin 3y ago
I would certainly be peeved if I showed up to a midterm that asked questions outside of existing human knowledge.
- flangola7 3y ago
  "I didn't make it into the university I wanted because I didn't invent enough new mathematics during the entry exams."

KyeRussell 3y ago

Given that OpenAI were THEMSELVES surprised by how even GPT-3 ended up, it’s always funny to see HN know-it-alls pipe up with all the answers. These sorts of poorly formed faux-philosophical arguments against LLMs have become the new domain of people that confuse blindly acting skeptical with actual intelligence. Ironic. This latest generation of AI quite rightfully raises questions and challenges assumptions about what it means to be intelligent. It quite rightfully challenges our assumptions about what can be accomplished with language. And, thank God, it quite rightfully challenges assumptions many have made about what sets humanity apart from everything else.

PaulDavisThe1st 3y ago
> poorly formed faux-philosophical arguments against LLMs There's a misunderstanding here. The post you're replying to is not an argument against LLMs. It's an argument about what LLMs can and cannot do, what their fundamental capabilities are, and so forth. It's very clear that if you need a system to provide answers based on a substantial body of human writing, LLMs are totally awesome. But that doesn't mean, in and of itself, that they can X or that they can Y.
Eji1700 3y ago
> Given that OpenAI were THEMSELVES surprised by how even GPT-3 ended up, Yeah and they have 0 incentive to overhype their takes. OpenAI has already slanted already impressive data in the past to make it more "hype building" for the general public, when a more scientific study style reading is "this is really cool, here's where it still fails". I am very confident shit like that is the same.

qgin 3y ago

I think the hallucinations show that it's not simply overfit to all of human knowledge. To hallucinate, there is a certain amount of generalization and information overlap that is necessary.

pama 3y ago

I’m working in a related area and I’m rather curious about this point. In what way is GPT-4 overfit? Does overfit in this context mean the conventional: validation loss went up with additional training, or something special?

rafiki6 3y ago
More specifically validation loss is irrelevant when you can't even sample out of distribution anymore.