8 ms·

I've given up on these LLMs. The amount of fatigue I get having to determine if what they tell me are fact is just too much. I'm sure someone will tell me my

I've given up on these LLMs. The amount of fatigue I get having to determine if what they tell me are fact is just too much. I'm sure someone will tell me my experience should be similar with generic web search, but at least I'm in control of what websites to read through to determine sources. However, I'll agree with most that state it is helpful for creative purposes, or perhaps with coding.

boringuser2 3y ago
[flagged]
- kranner 3y ago
  I asked it for references about Hafez Shirazi’s abandoned journey to India and it suggested a very specific Encyclopaedia Iranica entry which seemed perfect, and of course did not exist.
  - renewiltord 3y ago
    I asked it with this prompt: > Provide me with references about Hafez Shirazi’s abandoned journey to India. Answer "I don't know" if you are not 95% certain they exist. It said "I don't know". I asked it with the same prompt but for a thing I know has references and those were real. Not guaranteed to work but better results if you want greater certainty.
    - addingnumbers 3y ago
      > Answer "I don't know" if you are not 95% certain they exist. Technically, this could be read as == instead of >=, meaning it should answer "I don't know" when it is 99% or 100% certain...
      - klyrs 3y ago
        Technically, GPT is gobshite at anything resembling numeracy.
      - renewiltord 3y ago
        Ultimately, it's just a tool, so if the tool needs you to hold it this way and twist, you hold it and twist. And this seems to do the trick. Since it does answer with references for other situations, we needn't concern ourselves with the details.
- Crash0v3rid3 3y ago
  > GPT-4 doesn't hallucinate all that much. What data do you have to back this up? From my own experience GPT-4 hallucinates quite a bit, enough to make it unusable for my use cases.
  - ToValueFunfetti 3y ago
    The technical report[1] makes that claim at least: >GPT-4 significantly reduces hallucinations relative to previous GPT-3.5 models (which have them- selves been improving with continued iteration). GPT-4 scores 19 percentage points higher than our latest GPT-3.5 on our internal, adversarially-designed factuality evaluations [1] https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 (text from page 10)
    - simonw 3y ago
      "reduces hallucinations" and "doesn't hallucinate all that much" aren't quite the same thing.
      - ToValueFunfetti 3y ago
        I interpreted "all that much" as "close to as much as the earlier model", but yours is probably a more fair reading.
- simonw 3y ago
  It's much, much better than ChatGPT 3.5... in particular, if I ask it for biographical information about non-celebrity but internet-famous people I know 3.5 tends to make up all sorts of details while 4 is almost entirely correct. It still makes things up though, just in less obvious ways. So the trap is very much still there for people to fall into - if anything it's riskier, because the fact it lies less means people are more likely to assume that it doesn't ever.
- flanked-evergl 3y ago
  How does it determine if its generated text is factual?
  - simonw 3y ago
    The short answer is it can't. It's arguable whether anyone can - for a human being, determining if text is "factual" can be incredibly difficult. A better answer: if a fact is present many, many times in training data - "Paris is the capital of France" for example, it's much more likely to be true. Also influential: RLHF - Reinforcement Learning from Human Feedback. This is the process by which human labellers rate answers from LLMs - if they consistently rate up "facts" the models have a better chance of outputting factual information, at least if they can relate it to the up-voted responses somehow.
    - flanked-evergl 3y ago
      > The short answer is it can't. It's arguable whether anyone can - for a human being, determining if text is "factual" can be incredibly difficult. Yet, most adults I deal with don't make false things up out of whole cloth as much as ChatGPT does, and it really does not seem like it is that difficult for them. Children do this quite often though, and some adults do, but most don't. > A better answer: if a fact is present many, many times in training data - "Paris is the capital of France" for example, it's much more likely to be true. I think it is quite expected that it is biased to generating output that represent its training data, but this seems like it is not really a solution to the problem. Furthermore, sometimes I want ChatGPT to make things up which is not identical to training data. How do you get it to recognize that it is operating in the realm of fact or not? I'm not sure larger models with more parameters gets you to where you want to go. I think many people overstate the problem, I think it is not that serious, but I think a lot of people also try and just dismiss the issue.
- FeistySkink 3y ago
  It still can't explain standard CS algorithms most of the time. I've just tried asking it to explain deleting a non-root node from a max heap with examples. And both attempts were either plain wrong (random nodes disappearing) or poor (deleting a leaf node which is not very illustrative). Edit: I then asked who a certain deceased person _is_ and it gave me a completely wrong answer about a different person who's still alive and happens to share the last name. Both people have multiple website, books, publications and Wikipedia entries older than 2021 (which seems to be the cut-off). Edit 2: Looks like I'm still on 3.5, so disregard the above.
- alfalfasprout 3y ago
  Multiple times a day for me. And they're tricky to spot. I can't trust it with anything serious without thorough review.
simonw 3y ago
Your experience absolutely shouldn't be similar to generic web search. The idea that they are an effective replacement for that is one of the most widespread misunderstandings. They're good at SO MUCH OTHER STUFF. The challenge is figuring out what that other stuff is. (I have a few examples here: https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-or-help-on https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-... )
- isthiseasymode 3y ago
  That explanation makes me think of blockchain. I do think AI is already more useful that block chain has ever been, however.
  - roflc0ptic 3y ago
    Blockchain is good for one narrow thing most people don’t care about: reconciliation in multiparty transactions. LLMs appear to have myriad uses, today, no Twitter .eth con men required.
  - shagie 3y ago
    I find it very useful for doing zero shot and few shot classifications of natural language input. The "use it as a chat companion" is an interesting technology demo that demonstrates some emergent processes that make me wish I was back in college on the philosophy / linguistics / computer science intersection (though I suspect the hype would make grad school there rather unpleasant).
- quantiq 3y ago
  > They're good at SO MUCH OTHER STUFF. The challenge is figuring out what that other stuff is. I’m getting Déjà vu
  - simonw 3y ago
    The difference this time is that we've figured out all kinds of stuff that this is useful for. The challenge genuinely is helping people learn how to use it, not finding those applications in the first place.
kolbe 3y ago
I've had your same experience. I've found them mostly to be an error-prone search engine, with somehow less accountability than the open internet, because it hides its sources. At least with Stack Exchange answers, we have who wrote it, what responses there were, what the upvoting behavior around it was. And for the most part, I've found ChatGPT will transcribe often times wrong answers very poorly. One small example, I asked it to solve the heat equation (i useded the mathematical definition, and not "the heat equation") with dirac initial conditions on an infinite domain. It did a good job of recognizing which stack exchange answer to plagiarize, but did so incorrectly, and after a mostly correct derivation, declared the answer was "zero everywhere."
- simonw 3y ago
  Somewhat surprisingly, language models are TERRIBLE at mathematical or logic puzzles.
  - svachalek 3y ago
    It's kind of interesting that our science fiction projected traditional computing's strengths, math and logic, into the AI future with overly logical and mechanical AI characters. But our first creation of fully communicative AI has elementary school strength in these areas while it's probably better than the average adult at writing poetry or an inspiring speech.
  - kolbe 3y ago
    That's a whole other topic. I was mostly commenting on how it just plagiarized a correct answer off of Stack Exchange, except it took an incorrect hard right turn at the end to make up a solution.
    - simonw 3y ago
      What makes you think it was copying information from Stack Exchange in this case?
      - 3y ago
fferen 3y ago
I've found they serve almost exactly the opposite purpose as search engines. When I want reliable info and don't need hand-holding: search. When I have no idea what to search, or want a quick intro to something: ChatGPT. Together, they are very powerful complementary tools.
pmarreck 3y ago
Are you speaking of ChatGPT 3.5 or 4?
Sivart13 3y ago
With Google search going steadily downhill, I find it really tough to verify anything that ChatGPT authoritatively states is true Everyone on here is so enthusiastic about AI gobbling up the entire software landscape, I would just like a search engine that has any chance of telling me if something is factual
- alfalfasprout 3y ago
  It's going to be even worse when search results and training data from these LLMs is just output from other LLMs.
- klyrs 3y ago
  Product idea: the original PageRank over the Wayback Machine dataset pre-2022, with a mechanism to establish trust in users to moderate and cull SEO.
- eastbound 3y ago
  SciHub ;) Not joking, any search engine will pervert the results, like Google, like OpenAI said they would, “to protect the children”.
- ChatGTP 3y ago
  Buy an encyclopaedia and put it on your desk because it doesn’t sound like it’s going to get better anytime soon.
typon 3y ago
I use it for thing's that don't really matter if they're exactly correct. For example, coming up with a travel itinerary for a country I have never visited. Rewriting a work email with better English. Summarizing a news article. There are lot of things that don't require ultimate precision. I feel like people expect these models to do something they aren't really designed for - and the mismatch in expectations causes people to be let down. They are just tools - not "mildly conscious beings" like OpenAI founders wants you to believe.
broast 3y ago
It's like dealing with electricity (or maybe the internet). Early skeptics believe it is a curiosity with little application. People see how it can jump all over the place and create disasters that they can't imagine having engineered systems to finely control its behavior and create reliable complex functions and become the bedrock for computing.
- pstorm 3y ago
  I think there is also an aspect of willful disregard. This technology may change a lot, and it may be easier to dismiss that idea rather than process it.
  - ChatGTP 3y ago
    Do you think there might be the opposite going on ? Wanting to believe something that isn’t there because you won’t have to do as much work, feel smarter etc ? Because it’s really hard not to anthropomorphize it ? Gloss over all the incredible dangers we might be exposing our world too just because it’s “fun to play with” and see what AutoGPT can do to the Internet ?
baq 3y ago
They aren’t search engines or knowledge databases. They’re language computers. Use them for computing on language.

boringuser2 3y ago

[flagged]

kranner 3y ago
I asked it for references about Hafez Shirazi’s abandoned journey to India and it suggested a very specific Encyclopaedia Iranica entry which seemed perfect, and of course did not exist.
- renewiltord 3y ago
  I asked it with this prompt: > Provide me with references about Hafez Shirazi’s abandoned journey to India. Answer "I don't know" if you are not 95% certain they exist. It said "I don't know". I asked it with the same prompt but for a thing I know has references and those were real. Not guaranteed to work but better results if you want greater certainty.
  - addingnumbers 3y ago
    > Answer "I don't know" if you are not 95% certain they exist. Technically, this could be read as == instead of >=, meaning it should answer "I don't know" when it is 99% or 100% certain...
    - klyrs 3y ago
      Technically, GPT is gobshite at anything resembling numeracy.
    - renewiltord 3y ago
      Ultimately, it's just a tool, so if the tool needs you to hold it this way and twist, you hold it and twist. And this seems to do the trick. Since it does answer with references for other situations, we needn't concern ourselves with the details.
Crash0v3rid3 3y ago
> GPT-4 doesn't hallucinate all that much. What data do you have to back this up? From my own experience GPT-4 hallucinates quite a bit, enough to make it unusable for my use cases.
- ToValueFunfetti 3y ago
  The technical report[1] makes that claim at least: >GPT-4 significantly reduces hallucinations relative to previous GPT-3.5 models (which have them- selves been improving with continued iteration). GPT-4 scores 19 percentage points higher than our latest GPT-3.5 on our internal, adversarially-designed factuality evaluations [1] https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 (text from page 10)
  - simonw 3y ago
    "reduces hallucinations" and "doesn't hallucinate all that much" aren't quite the same thing.
    - ToValueFunfetti 3y ago
      I interpreted "all that much" as "close to as much as the earlier model", but yours is probably a more fair reading.
simonw 3y ago
It's much, much better than ChatGPT 3.5... in particular, if I ask it for biographical information about non-celebrity but internet-famous people I know 3.5 tends to make up all sorts of details while 4 is almost entirely correct. It still makes things up though, just in less obvious ways. So the trap is very much still there for people to fall into - if anything it's riskier, because the fact it lies less means people are more likely to assume that it doesn't ever.
flanked-evergl 3y ago
How does it determine if its generated text is factual?
- simonw 3y ago
  The short answer is it can't. It's arguable whether anyone can - for a human being, determining if text is "factual" can be incredibly difficult. A better answer: if a fact is present many, many times in training data - "Paris is the capital of France" for example, it's much more likely to be true. Also influential: RLHF - Reinforcement Learning from Human Feedback. This is the process by which human labellers rate answers from LLMs - if they consistently rate up "facts" the models have a better chance of outputting factual information, at least if they can relate it to the up-voted responses somehow.
  - flanked-evergl 3y ago
    > The short answer is it can't. It's arguable whether anyone can - for a human being, determining if text is "factual" can be incredibly difficult. Yet, most adults I deal with don't make false things up out of whole cloth as much as ChatGPT does, and it really does not seem like it is that difficult for them. Children do this quite often though, and some adults do, but most don't. > A better answer: if a fact is present many, many times in training data - "Paris is the capital of France" for example, it's much more likely to be true. I think it is quite expected that it is biased to generating output that represent its training data, but this seems like it is not really a solution to the problem. Furthermore, sometimes I want ChatGPT to make things up which is not identical to training data. How do you get it to recognize that it is operating in the realm of fact or not? I'm not sure larger models with more parameters gets you to where you want to go. I think many people overstate the problem, I think it is not that serious, but I think a lot of people also try and just dismiss the issue.
FeistySkink 3y ago
It still can't explain standard CS algorithms most of the time. I've just tried asking it to explain deleting a non-root node from a max heap with examples. And both attempts were either plain wrong (random nodes disappearing) or poor (deleting a leaf node which is not very illustrative). Edit: I then asked who a certain deceased person _is_ and it gave me a completely wrong answer about a different person who's still alive and happens to share the last name. Both people have multiple website, books, publications and Wikipedia entries older than 2021 (which seems to be the cut-off). Edit 2: Looks like I'm still on 3.5, so disregard the above.
alfalfasprout 3y ago
Multiple times a day for me. And they're tricky to spot. I can't trust it with anything serious without thorough review.

simonw 3y ago

Your experience absolutely shouldn't be similar to generic web search. The idea that they are an effective replacement for that is one of the most widespread misunderstandings. They're good at SO MUCH OTHER STUFF. The challenge is figuring out what that other stuff is. (I have a few examples here: https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-or-help-on https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-... )

isthiseasymode 3y ago
That explanation makes me think of blockchain. I do think AI is already more useful that block chain has ever been, however.
- roflc0ptic 3y ago
  Blockchain is good for one narrow thing most people don’t care about: reconciliation in multiparty transactions. LLMs appear to have myriad uses, today, no Twitter .eth con men required.
- shagie 3y ago
  I find it very useful for doing zero shot and few shot classifications of natural language input. The "use it as a chat companion" is an interesting technology demo that demonstrates some emergent processes that make me wish I was back in college on the philosophy / linguistics / computer science intersection (though I suspect the hype would make grad school there rather unpleasant).
quantiq 3y ago
> They're good at SO MUCH OTHER STUFF. The challenge is figuring out what that other stuff is. I’m getting Déjà vu
- simonw 3y ago
  The difference this time is that we've figured out all kinds of stuff that this is useful for. The challenge genuinely is helping people learn how to use it, not finding those applications in the first place.

kolbe 3y ago

I've had your same experience. I've found them mostly to be an error-prone search engine, with somehow less accountability than the open internet, because it hides its sources. At least with Stack Exchange answers, we have who wrote it, what responses there were, what the upvoting behavior around it was. And for the most part, I've found ChatGPT will transcribe often times wrong answers very poorly. One small example, I asked it to solve the heat equation (i useded the mathematical definition, and not "the heat equation") with dirac initial conditions on an infinite domain. It did a good job of recognizing which stack exchange answer to plagiarize, but did so incorrectly, and after a mostly correct derivation, declared the answer was "zero everywhere."

simonw 3y ago
Somewhat surprisingly, language models are TERRIBLE at mathematical or logic puzzles.
- svachalek 3y ago
  It's kind of interesting that our science fiction projected traditional computing's strengths, math and logic, into the AI future with overly logical and mechanical AI characters. But our first creation of fully communicative AI has elementary school strength in these areas while it's probably better than the average adult at writing poetry or an inspiring speech.
- kolbe 3y ago
  That's a whole other topic. I was mostly commenting on how it just plagiarized a correct answer off of Stack Exchange, except it took an incorrect hard right turn at the end to make up a solution.
  - simonw 3y ago
    What makes you think it was copying information from Stack Exchange in this case?
    - 3y ago

fferen 3y ago

I've found they serve almost exactly the opposite purpose as search engines. When I want reliable info and don't need hand-holding: search. When I have no idea what to search, or want a quick intro to something: ChatGPT. Together, they are very powerful complementary tools.

pmarreck 3y ago

Are you speaking of ChatGPT 3.5 or 4?

Sivart13 3y ago

With Google search going steadily downhill, I find it really tough to verify anything that ChatGPT authoritatively states is true Everyone on here is so enthusiastic about AI gobbling up the entire software landscape, I would just like a search engine that has any chance of telling me if something is factual

alfalfasprout 3y ago
It's going to be even worse when search results and training data from these LLMs is just output from other LLMs.
klyrs 3y ago
Product idea: the original PageRank over the Wayback Machine dataset pre-2022, with a mechanism to establish trust in users to moderate and cull SEO.
eastbound 3y ago
SciHub ;) Not joking, any search engine will pervert the results, like Google, like OpenAI said they would, “to protect the children”.
ChatGTP 3y ago
Buy an encyclopaedia and put it on your desk because it doesn’t sound like it’s going to get better anytime soon.

typon 3y ago

I use it for thing's that don't really matter if they're exactly correct. For example, coming up with a travel itinerary for a country I have never visited. Rewriting a work email with better English. Summarizing a news article. There are lot of things that don't require ultimate precision. I feel like people expect these models to do something they aren't really designed for - and the mismatch in expectations causes people to be let down. They are just tools - not "mildly conscious beings" like OpenAI founders wants you to believe.

broast 3y ago

It's like dealing with electricity (or maybe the internet). Early skeptics believe it is a curiosity with little application. People see how it can jump all over the place and create disasters that they can't imagine having engineered systems to finely control its behavior and create reliable complex functions and become the bedrock for computing.

pstorm 3y ago
I think there is also an aspect of willful disregard. This technology may change a lot, and it may be easier to dismiss that idea rather than process it.
- ChatGTP 3y ago
  Do you think there might be the opposite going on ? Wanting to believe something that isn’t there because you won’t have to do as much work, feel smarter etc ? Because it’s really hard not to anthropomorphize it ? Gloss over all the incredible dangers we might be exposing our world too just because it’s “fun to play with” and see what AutoGPT can do to the Internet ?

baq 3y ago

They aren’t search engines or knowledge databases. They’re language computers. Use them for computing on language.