7 ms·

I appreciate this paper for relatively clearly stating what "human-like" might entail, which in this case involves "reasoning about the causes behind other peop

I appreciate this paper for relatively clearly stating what "human-like" might entail, which in this case involves "reasoning about the causes behind other people's behavior" which is "critical to navigate the social world" as outlined in this citation: https://www.sciencedirect.com/science/article/abs/pii/S0010028520300633?via%3Dihub https://www.sciencedirect.com/science/article/abs/pii/S00100... I get frustrated often when people argue "well, it isn't really intelligent" and then give examples that are clearly dependent on our brain's chemical state and our bodies' existence in-the-physical-world. I get the feeling that when/if we are all enslaved by a super-intelligent AI that we do not understand its motives, we will still argue that it is not intelligent because it doesn't get hungry and it can't prove to us that it has Qualia. This paper argues that gpts are bad at understanding human risk/reward functions, which seems like a much more explicit way to talk about this, and also casts it in a way that could help reframe the debate about how human evolution and our physical beings might be significantly responsible for the structure of our rational minds.

swatcoder 3y ago
The underlying problem is that "intelligence" is itself a crappy, poorly defined word with a fraught and inconsistent history. It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted (but not scientifically evidenced) as some general measure of mental aptitude. At that point it shifts from something qualitative (which mental tasks might someone be good at) to something quantitative (how much more might one personal excel at all mental tasks than another), and the burgeoning field of modern American psychology goes "Aha! A quantitative measure! Here's our meal ticket to being recognized as a science instead of those quacks from Vienna", with far too much at stake to either question the many assumptions at play or the inconsistent history of usage. Momentum takes hold and the public takes the word into its everyday vernacular, even while it's still not a clear and sound concept in its technical domain. [Most of this is history is more academically covered in Danziger's 1987 "Naming the Mind" which is excellent, and critical foundational reading to contextualize recent hot discussions in AI] The way you're using it when you worry about "super-intelligence" is in the sense of intelligence being some universal, unbounded, quantitative independent variable along the lines of "the more intelligent something is, the more cunningly it can pursue some rationalized goal" -- some master strategist. That's fine, and you're not alone in that, but there's not really any sound scientific groundwork to establish that there exists some quality of the world that scales like that. You're fear, and what you try to distinguish conceptually from what the paper addresses, is an inductive leap made from highly unstable ground. It's in the same invented, purely abstract idea-space of "omnipotence" or "omniscience" where one takes a practical idea like "power to influence" or "ability to know fact" and inductively draws a line from these practical senses towards some abstract infinite/incomprehensible version of that thing. But that inductive leap a Platonic logician's parlor trick and ends up raising all kinds of abstract paradoxes, as well countless physical impracticalities about how such things could exist. So a lot of people (academic and lay) just aren't with you in taking that framing of intelligence very seriously. For many, an "super-intelligent" software whose "motives" we don't understand is just a program that produces incorrect outputs and ought to be debugged or retired, and the more interesting questions around machine "intelligence" are practical ones like "what tasks are these programs well-suited for". Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind. Knowing the answer to that kind of question reaches back to the earliest and least disputable sense of the word, where we saw that some new students and soldiers excelled at certain tasks and struggled with others, and wanted to understand how best to educated/assign them. And likewise, as we look at these tools, the pressing question for engineers and businesses is "what are they good for and what are they not good for" rather than the fantastical "what if we make a broken program and it wants to kill everyone and we don't notice and forget to shut it off"
- famouswaffles 3y ago
  There does seem to be a general factor of intelligence in humans though that is the single biggest indicator of performance. Yes there are other factors too. >Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind. Not good at tasks that benefit from a theory of mind extracted from visual data.
  - throwanem 3y ago
    "Seem" is doing a lot of work here. So is the implicit claim that theory of mind in general can be demonstrated by current-gen foundation models, and only those aspects dependent on vision cannot.
    - famouswaffles 3y ago
      I say seem but it's stronger than that. all evidence and testing points towards a general factor of intelligence. The better you perform at one "kind" of intelligence task, the better you will perform at them all. The shift in defining intelligence didn't come from nowhere. Yes, It's easy to think that there are multiple different mutually exclusive-ish kinds of intelligences and that you can excel in one and it has no bearing on performance on the other but that's not really true. all indication point otherwise. I'm not saying there aren't other factors but generally, that's what you can expect. Yes theory of mind can be demonstrated. Make up whatever bespoke story you can with characters having varying levels if intention and knowledge. Then query GPT-4 about the state of the characters. What i'm saying is that the vision component introduces another point of error, is it a lack of theory of mind ? or being yet unable to extract the necessary features from visual data ? They rapidly learn to but Blind people who could recognize squares by feel do not have the ability to recognize squares by sight upon gaining vision. https://www.projectprakash.org/_files/ugd/2af8ef_5a0c6250cc3a445db0d15c09615b6dba.pdf https://www.projectprakash.org/_files/ugd/2af8ef_5a0c6250cc3...
    - pegasus 3y ago
      "Seem" only seems to be doing lots of work. Here's a relevant article: https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)
mistermann 3y ago
> I get frustrated often when people argue "well, it isn't really intelligent" and then give examples that are clearly dependent on our brain's chemical state and our bodies' existence in-the-physical-world. A big part of the problem is that "is" has a wide variety of inconsistent meanings, and that this fact is sub-perceptual, and that it is culturally very inappropriate to comment on aspects of our culture like this, preventing knowledge of the problem from spreading. /u/Swatcoder makes essentially the same point but in much more detail, though regarding less important words.

swatcoder 3y ago

The underlying problem is that "intelligence" is itself a crappy, poorly defined word with a fraught and inconsistent history. It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted (but not scientifically evidenced) as some general measure of mental aptitude. At that point it shifts from something qualitative (which mental tasks might someone be good at) to something quantitative (how much more might one personal excel at all mental tasks than another), and the burgeoning field of modern American psychology goes "Aha! A quantitative measure! Here's our meal ticket to being recognized as a science instead of those quacks from Vienna", with far too much at stake to either question the many assumptions at play or the inconsistent history of usage. Momentum takes hold and the public takes the word into its everyday vernacular, even while it's still not a clear and sound concept in its technical domain. [Most of this is history is more academically covered in Danziger's 1987 "Naming the Mind" which is excellent, and critical foundational reading to contextualize recent hot discussions in AI] The way you're using it when you worry about "super-intelligence" is in the sense of intelligence being some universal, unbounded, quantitative independent variable along the lines of "the more intelligent something is, the more cunningly it can pursue some rationalized goal" -- some master strategist. That's fine, and you're not alone in that, but there's not really any sound scientific groundwork to establish that there exists some quality of the world that scales like that. You're fear, and what you try to distinguish conceptually from what the paper addresses, is an inductive leap made from highly unstable ground. It's in the same invented, purely abstract idea-space of "omnipotence" or "omniscience" where one takes a practical idea like "power to influence" or "ability to know fact" and inductively draws a line from these practical senses towards some abstract infinite/incomprehensible version of that thing. But that inductive leap a Platonic logician's parlor trick and ends up raising all kinds of abstract paradoxes, as well countless physical impracticalities about how such things could exist. So a lot of people (academic and lay) just aren't with you in taking that framing of intelligence very seriously. For many, an "super-intelligent" software whose "motives" we don't understand is just a program that produces incorrect outputs and ought to be debugged or retired, and the more interesting questions around machine "intelligence" are practical ones like "what tasks are these programs well-suited for". Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind. Knowing the answer to that kind of question reaches back to the earliest and least disputable sense of the word, where we saw that some new students and soldiers excelled at certain tasks and struggled with others, and wanted to understand how best to educated/assign them. And likewise, as we look at these tools, the pressing question for engineers and businesses is "what are they good for and what are they not good for" rather than the fantastical "what if we make a broken program and it wants to kill everyone and we don't notice and forget to shut it off"

famouswaffles 3y ago
There does seem to be a general factor of intelligence in humans though that is the single biggest indicator of performance. Yes there are other factors too. >Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind. Not good at tasks that benefit from a theory of mind extracted from visual data.
- throwanem 3y ago
  "Seem" is doing a lot of work here. So is the implicit claim that theory of mind in general can be demonstrated by current-gen foundation models, and only those aspects dependent on vision cannot.
  - famouswaffles 3y ago
    I say seem but it's stronger than that. all evidence and testing points towards a general factor of intelligence. The better you perform at one "kind" of intelligence task, the better you will perform at them all. The shift in defining intelligence didn't come from nowhere. Yes, It's easy to think that there are multiple different mutually exclusive-ish kinds of intelligences and that you can excel in one and it has no bearing on performance on the other but that's not really true. all indication point otherwise. I'm not saying there aren't other factors but generally, that's what you can expect. Yes theory of mind can be demonstrated. Make up whatever bespoke story you can with characters having varying levels if intention and knowledge. Then query GPT-4 about the state of the characters. What i'm saying is that the vision component introduces another point of error, is it a lack of theory of mind ? or being yet unable to extract the necessary features from visual data ? They rapidly learn to but Blind people who could recognize squares by feel do not have the ability to recognize squares by sight upon gaining vision. https://www.projectprakash.org/_files/ugd/2af8ef_5a0c6250cc3a445db0d15c09615b6dba.pdf https://www.projectprakash.org/_files/ugd/2af8ef_5a0c6250cc3...
  - pegasus 3y ago
    "Seem" only seems to be doing lots of work. Here's a relevant article: https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)

mistermann 3y ago

> I get frustrated often when people argue "well, it isn't really intelligent" and then give examples that are clearly dependent on our brain's chemical state and our bodies' existence in-the-physical-world. A big part of the problem is that "is" has a wide variety of inconsistent meanings, and that this fact is sub-perceptual, and that it is culturally very inappropriate to comment on aspects of our culture like this, preventing knowledge of the problem from spreading. /u/Swatcoder makes essentially the same point but in much more detail, though regarding less important words.