11 ms·

You can also continue to chat with it in each of its answers and ask it why it said this or that and see how it works out corrections and clarifications which t

You can also continue to chat with it in each of its answers and ask it why it said this or that and see how it works out corrections and clarifications which to me is so much more significant as that's how people converse. One-shot answers are for google.

simonw 3y ago
Asking it why it said something isn't a great pattern, because it can't answer that truthfully: each interaction with the bot resets its "memory" entirely. You're effectively asking it to invent a rationale for what you are telling it was the thing it told you last time round. Asking it to "think out loud" during its initial answer is a better way to get insight into why it answered in a particular way (and also often causes it to provide better answers). Asking it follow-up questions is almost always useful, it's just the "why did you say X?" pattern that I'm recommending against.
- earthboundkid 3y ago
  Yes. I strongly feel that reinforcement learning should be applied to punish the LLMs for speculating about their past behavior. They should respond along the lines of “I’m sorry, I don’t know why I said 3 + 5 is 9, but I will try to answer again.”
- dwighttk 3y ago
  it is always inventing rationale
- tyingq 3y ago
  >each interaction with the bot resets its "memory" entirely. Completely separate interaction maybe. You can certainly ask a question then ask it to rephrase the answer, etc. Like "say five words", followed by "reverse the answer"...produces the same five words in reverse. Asking a question then following with "why did you say that?" usually cites the previous context reasonably.
  - simonw 3y ago
    Sure, but the thing you have to understand is that there's no hidden state from the previous answer that can be revealed through extra questions. It's lost all of that state the moment it spits out the reply tokens.
- travisjungroth 3y ago
  > You're effectively asking it to invent a rationale for what you are telling it was the thing it told you last time round. There’s lots of evidence that this is what people do, too. Asked about their reasoning, people will do things like include information they didn’t have at the time. The part of the mind that moves the fastest doesn’t encode the process to memory[0] but we’re really uncomfortable saying “I don’t know” about our own decisions. So, we rationalize. If you want to be more accurate, you need to do the same thing of “thinking out loud”, either subvocalization, voice or paper. Even then it’s not perfect. Easy to make up your mind and justify it after. [0] Which, of course it wouldn’t. You don’t log every line of your fastest piece of code. Then it’s not fast.
sebringj 3y ago
I think didn't do a good comment here because I don't mean that exactly. I don't necessarily care so much about it telling me how it got there. Rather I like it when I ask someone about something then I can ask them... "Tell me more about this or that or explain that in more detail" which I can never do in google.

simonw 3y ago

Asking it why it said something isn't a great pattern, because it can't answer that truthfully: each interaction with the bot resets its "memory" entirely. You're effectively asking it to invent a rationale for what you are telling it was the thing it told you last time round. Asking it to "think out loud" during its initial answer is a better way to get insight into why it answered in a particular way (and also often causes it to provide better answers). Asking it follow-up questions is almost always useful, it's just the "why did you say X?" pattern that I'm recommending against.

earthboundkid 3y ago
Yes. I strongly feel that reinforcement learning should be applied to punish the LLMs for speculating about their past behavior. They should respond along the lines of “I’m sorry, I don’t know why I said 3 + 5 is 9, but I will try to answer again.”
dwighttk 3y ago
it is always inventing rationale
tyingq 3y ago
>each interaction with the bot resets its "memory" entirely. Completely separate interaction maybe. You can certainly ask a question then ask it to rephrase the answer, etc. Like "say five words", followed by "reverse the answer"...produces the same five words in reverse. Asking a question then following with "why did you say that?" usually cites the previous context reasonably.
- simonw 3y ago
  Sure, but the thing you have to understand is that there's no hidden state from the previous answer that can be revealed through extra questions. It's lost all of that state the moment it spits out the reply tokens.
travisjungroth 3y ago
> You're effectively asking it to invent a rationale for what you are telling it was the thing it told you last time round. There’s lots of evidence that this is what people do, too. Asked about their reasoning, people will do things like include information they didn’t have at the time. The part of the mind that moves the fastest doesn’t encode the process to memory[0] but we’re really uncomfortable saying “I don’t know” about our own decisions. So, we rationalize. If you want to be more accurate, you need to do the same thing of “thinking out loud”, either subvocalization, voice or paper. Even then it’s not perfect. Easy to make up your mind and justify it after. [0] Which, of course it wouldn’t. You don’t log every line of your fastest piece of code. Then it’s not fast.

sebringj 3y ago

I think didn't do a good comment here because I don't mean that exactly. I don't necessarily care so much about it telling me how it got there. Rather I like it when I ask someone about something then I can ask them... "Tell me more about this or that or explain that in more detail" which I can never do in google.