9 ms·

I tried for a few months getting ChatGPT4 to work with a MUD. In my experience it's not very good at that particular task. One problem I ran into was its abil

I tried for a few months getting ChatGPT4 to work with a MUD. In my experience it's not very good at that particular task. One problem I ran into was its ability to logically connect rooms. In a MUD you navigate by going north, south, east, west, up, or down. Not every room lets you go any direction. And usually, if you go east, in your new room, you can go west. Rarely a level creator will make this not be true. ChatGPT4 was pretty bad at it though. Another problem was descriptions. It might mention a mountain in the distance once. But then never again. So this giant landmark was described in a single room. It was also difficult to get it to create a fair quantity of secrets in logical places. Lots of times it would just chain together multiple secrets in a single place. If you have more than one, you want to spread it around And finally, room layout. It tended to not be very good at this. Lots of linear layouts. It didn't have an eye towards when details should be complex rooms and when it can just be a line in a description. So it could do it, but it created levels that weren't very fun or particularly creative, even when it came to room descriptions.

danjc 2y ago
Somewhat related, Yann LeCun posted a few months back about how many concepts aren't understood through language and therefore can't be modeled through it (which is why LLM's are terrible with things like position and direction). https://x.com/ylecun/status/1768353714794901530?s=46 https://x.com/ylecun/status/1768353714794901530?s=46
- Folcon 2y ago
  To people who claim that "thinking and reasoning require language", here is a problem: Imagine standing at the North Pole of the Earth. Walk in any direction, in a straight line, for 1 km. Now turn 90 degrees to the left. Walk for as long as it takes to pass your starting point. Have you walked: 1. More than 2xPi km 2. Exactly 2xPi km 3. Less than 2xPi km 4. I never came close to my starting point. Think about how you tried to answer this question and tell us whether it was based on language. Just quoting this here in case anything happens to the tweet... I agree with this, however I have one tiny nitpick, feel free to tell me if you think I'm wrong or being overly nitpicky, but the knowledge of the situation in which the phenomenon that he's describing occurs, I learned about entirely from language. So my ability to answer the question is based on that knowledge. I'm aware that the reasoning problem itself doesn't utilise it, but a position and direction system in and of itself arguably also suffers from being insufficient. I suppose I'm wondering if "setup" counts as needing language model?
  - dragonwriter 2y ago
    I don't think the example is a good rebuttal of "thinking and reasoning require language". It may be a decent challenge, probably still not an actual rebuttal, of "language is sufficient for all thinking and reasoning", but "X is required for Y" and "X is sufficient for everything encompassed by Y" are very different claims.
    - Folcon 2y ago
      That's fair, it's not supposed to be a rebuttal of that :)... Though now that you said it, I'm really thinking about the original statement. Honest question, can you give me an example of thinking or reasoning that happens fully independently of reasoning via symbols or their manipulation? I feel like I'm missing something obvious, but nothing is coming to mind right now :)... Just thinking about the underlying statement and simplifying language down to symbolic expression. (I was originally going to say manipulation, but it doesn't feel like it quite fits...)
      - dragonwriter 2y ago
        > Honest question, can you give me an example of thinking or reasoning that happens fully independently of reasoning via symbols or their manipulation? I don't think we have anything but subjective, indirect understanding of how thinking happens, but I think that, at a minimum, what we describe as "reasoning" specifically is tightly conceptually related to, if not a subset of, manipulation of abstract symbols to which concrete experiences may be approximately mapped. I'm not sure I'd say this is the same thing as language, but there's at a minimum a shared common symbol-manipulation underlying both. Do the capacities always come together? I'm not sure how we would know that, I think our ability to recognize reasoning is tied to it being mapped to language, and are ability to distinguish something as language rather than nonlinguistic signaling or mimicry of something else that is using language is tied to independent expression of reasoning through it.
  - erik_seaberg 2y ago
    If the brain actually used language to represent ideas in use, we should have succeeded in finding its https://en.wikipedia.org/wiki/Universal_grammar https://en.wikipedia.org/wiki/Universal_grammar. Instead it seems more like language is a kind of lossy compression we keep reinventing for ideas, to export them from a brain in some tangible way and get them into another (even in the future).
- maxglute 2y ago
  This interaction down the chain is interesting: Q: yann do you have an internal monologue? A: Not that I know of.
- TeMPOraL 2y ago
  My bet is that this is wrong, and at the same time, language isn't required - just sufficient. I see concepts as defined only through associations with other concepts (which can be modeled as proximity in high-dimensional space, and that's precisely what LLMs are doing) and, sometimes, through memorized sensory data - the latter isn't the typical case, but it's needed to make the recursive definition (concepts defined in terms of concepts) stay anchored to reality. From that follows that written language is enough to build that structure of concepts (latent space in ML terms). So is spoken language. So is vision in general, or hearing in general[0]. The brain will build one concept space out of all inputs available; it is necessary and sufficient to have at least one, but none of them alone is itself necessary. -- [0] - Languages are higher-level regularities used for communication, growing on top of those senses, but not strictly necessary for understanding the real world. I'd use people who have no perceptible inner voice and high visualization skills as a counter to the idea that concepts need to be thought of symbolically in something resembling a written or spoken language.
- ninetyninenine 2y ago
  It’s all about the training. Llms can have better understanding of space then humans to the point where they can draw things better than us. Don’t restrict LLMs to text. If you train one with images and text you’ll get one that understands position.
TillE 2y ago
I think the best thing to do in a scenario like that is standard procgen where you randomly, logically generate the rooms with a bunch of descriptive tags, and then LLM-ify the room descriptions, with some context for what the world/area is supposed to be like.
- adastra22 2y ago
  It can even dynamically extend by using prompts to generate new rooms as the user enters them, but keeping track of generated world outside of the LLM. Which is the same way nearly all procedurally generated games work.
Waterluvian 2y ago
I think the issue might be that people like to throw the tool at the whole problem when it’s not the right tool for a lot of it. Don’t use LLMs for logic. Use it for colour and flavour. Generate your own layout, populate the rooms in a manner befitting the context, difficulty, story arc, and then whenever a user makes an action, update the state then pass to the LLM the state and ask it to describe the room and make a few other smaller decisions.
immibis 2y ago
Did you try AI Dungeon 2, in 2020 when it was a thing?
- lolinder 2y ago
  It's still running now: https://aidungeon.com/ https://aidungeon.com/
  - immibis 2y ago
    Apparently still with a version of the NSFW filter that made everyone abandon it in 2020.
    - infotainment 2y ago
      Really? I was under the impression that was an OpenAI limitation, which they eventually worked around by deploying their own models.
      - lolinder 2y ago
        It's both. The game started as a side project that went crazy viral as the first real way people could access an LLM (GPT-2 at the time). That of course immediately led to people using it for NSFW content, and they were really the first project to have to deal with the question of what to do about that content. Their initial reaction was to deploy a filter, which worked but was unpopular with many in their early audience. I haven't followed the project for a while, but my understanding was that they rolled back the filter at least for paid accounts. But when they had the filter it wasn't just an OpenAI restriction because their filter actually predated GPT-3 and OpenAI's APIs.
        immibis 2y ago
        "worked" as in "made OpenAI happy" but not as in "retained their customers". People were getting banned for using words like "melon" (racist implication, said the filter).
segmondy 2y ago
Looks more like a prompting and coding issue. ChatGPT4 is not strong AGI. It's an AGI, but you need to guide it.

danjc 2y ago

Somewhat related, Yann LeCun posted a few months back about how many concepts aren't understood through language and therefore can't be modeled through it (which is why LLM's are terrible with things like position and direction). https://x.com/ylecun/status/1768353714794901530?s=46 https://x.com/ylecun/status/1768353714794901530?s=46

Folcon 2y ago
To people who claim that "thinking and reasoning require language", here is a problem: Imagine standing at the North Pole of the Earth. Walk in any direction, in a straight line, for 1 km. Now turn 90 degrees to the left. Walk for as long as it takes to pass your starting point. Have you walked: 1. More than 2xPi km 2. Exactly 2xPi km 3. Less than 2xPi km 4. I never came close to my starting point. Think about how you tried to answer this question and tell us whether it was based on language. Just quoting this here in case anything happens to the tweet... I agree with this, however I have one tiny nitpick, feel free to tell me if you think I'm wrong or being overly nitpicky, but the knowledge of the situation in which the phenomenon that he's describing occurs, I learned about entirely from language. So my ability to answer the question is based on that knowledge. I'm aware that the reasoning problem itself doesn't utilise it, but a position and direction system in and of itself arguably also suffers from being insufficient. I suppose I'm wondering if "setup" counts as needing language model?
- dragonwriter 2y ago
  I don't think the example is a good rebuttal of "thinking and reasoning require language". It may be a decent challenge, probably still not an actual rebuttal, of "language is sufficient for all thinking and reasoning", but "X is required for Y" and "X is sufficient for everything encompassed by Y" are very different claims.
  - Folcon 2y ago
    That's fair, it's not supposed to be a rebuttal of that :)... Though now that you said it, I'm really thinking about the original statement. Honest question, can you give me an example of thinking or reasoning that happens fully independently of reasoning via symbols or their manipulation? I feel like I'm missing something obvious, but nothing is coming to mind right now :)... Just thinking about the underlying statement and simplifying language down to symbolic expression. (I was originally going to say manipulation, but it doesn't feel like it quite fits...)
    - dragonwriter 2y ago
      > Honest question, can you give me an example of thinking or reasoning that happens fully independently of reasoning via symbols or their manipulation? I don't think we have anything but subjective, indirect understanding of how thinking happens, but I think that, at a minimum, what we describe as "reasoning" specifically is tightly conceptually related to, if not a subset of, manipulation of abstract symbols to which concrete experiences may be approximately mapped. I'm not sure I'd say this is the same thing as language, but there's at a minimum a shared common symbol-manipulation underlying both. Do the capacities always come together? I'm not sure how we would know that, I think our ability to recognize reasoning is tied to it being mapped to language, and are ability to distinguish something as language rather than nonlinguistic signaling or mimicry of something else that is using language is tied to independent expression of reasoning through it.
- erik_seaberg 2y ago
  If the brain actually used language to represent ideas in use, we should have succeeded in finding its https://en.wikipedia.org/wiki/Universal_grammar https://en.wikipedia.org/wiki/Universal_grammar. Instead it seems more like language is a kind of lossy compression we keep reinventing for ideas, to export them from a brain in some tangible way and get them into another (even in the future).
maxglute 2y ago
This interaction down the chain is interesting: Q: yann do you have an internal monologue? A: Not that I know of.
TeMPOraL 2y ago
My bet is that this is wrong, and at the same time, language isn't required - just sufficient. I see concepts as defined only through associations with other concepts (which can be modeled as proximity in high-dimensional space, and that's precisely what LLMs are doing) and, sometimes, through memorized sensory data - the latter isn't the typical case, but it's needed to make the recursive definition (concepts defined in terms of concepts) stay anchored to reality. From that follows that written language is enough to build that structure of concepts (latent space in ML terms). So is spoken language. So is vision in general, or hearing in general[0]. The brain will build one concept space out of all inputs available; it is necessary and sufficient to have at least one, but none of them alone is itself necessary. -- [0] - Languages are higher-level regularities used for communication, growing on top of those senses, but not strictly necessary for understanding the real world. I'd use people who have no perceptible inner voice and high visualization skills as a counter to the idea that concepts need to be thought of symbolically in something resembling a written or spoken language.
ninetyninenine 2y ago
It’s all about the training. Llms can have better understanding of space then humans to the point where they can draw things better than us. Don’t restrict LLMs to text. If you train one with images and text you’ll get one that understands position.

TillE 2y ago

I think the best thing to do in a scenario like that is standard procgen where you randomly, logically generate the rooms with a bunch of descriptive tags, and then LLM-ify the room descriptions, with some context for what the world/area is supposed to be like.

adastra22 2y ago
It can even dynamically extend by using prompts to generate new rooms as the user enters them, but keeping track of generated world outside of the LLM. Which is the same way nearly all procedurally generated games work.

Waterluvian 2y ago

I think the issue might be that people like to throw the tool at the whole problem when it’s not the right tool for a lot of it. Don’t use LLMs for logic. Use it for colour and flavour. Generate your own layout, populate the rooms in a manner befitting the context, difficulty, story arc, and then whenever a user makes an action, update the state then pass to the LLM the state and ask it to describe the room and make a few other smaller decisions.

immibis 2y ago

Did you try AI Dungeon 2, in 2020 when it was a thing?

lolinder 2y ago
It's still running now: https://aidungeon.com/ https://aidungeon.com/
- immibis 2y ago
  Apparently still with a version of the NSFW filter that made everyone abandon it in 2020.
  - infotainment 2y ago
    Really? I was under the impression that was an OpenAI limitation, which they eventually worked around by deploying their own models.
    - lolinder 2y ago
      It's both. The game started as a side project that went crazy viral as the first real way people could access an LLM (GPT-2 at the time). That of course immediately led to people using it for NSFW content, and they were really the first project to have to deal with the question of what to do about that content. Their initial reaction was to deploy a filter, which worked but was unpopular with many in their early audience. I haven't followed the project for a while, but my understanding was that they rolled back the filter at least for paid accounts. But when they had the filter it wasn't just an OpenAI restriction because their filter actually predated GPT-3 and OpenAI's APIs.
      - immibis 2y ago
        "worked" as in "made OpenAI happy" but not as in "retained their customers". People were getting banned for using words like "melon" (racist implication, said the filter).

segmondy 2y ago

Looks more like a prompting and coding issue. ChatGPT4 is not strong AGI. It's an AGI, but you need to guide it.