6 ms·

"Pretty neat, but definitely watch out for hallucinations." We'd never hire someone who just makes stuff up (or at least keep them employed for long). Why are

"Pretty neat, but definitely watch out for hallucinations." We'd never hire someone who just makes stuff up (or at least keep them employed for long). Why are we okay with calling "AI" tools like this anything other than curious research projects? Can't we just send LLMs back to the drawing board until they have some semblance of reliability?

oldstrangers 2y ago
> Can't we just send LLMs back to the drawing board until they have some semblance of reliability? Well at this point they've certainly proven a net gain for everyone regardless of the occasional nonsense they spew.
- aiono 2y ago
  No, from the research around it the findings are mixed. There is no consensus that it's net gain.
- DanHulton 2y ago
  That is... debatable. You may be entirely inside the bubble, there.
  - orangepanda 2y ago
    You overestimate the importance of being correct
  - taikahessu 2y ago
    Not sure if this was posted as humour, but I don't feel that way. In today's world, where I certainly would consider taking the blue pill, I'm having a blast with LLMs! It has helped me learn stuff incredibly faster. Especially I find them useful for filling the gaps of knowledge and exploring new topics in my own way and language, without needing to wait an answer from a human (that could also be wrong). Why does it feel, that "we are entirely inside the bubble" for you?
    - dingnuts 2y ago
      >It has helped me learn stuff incredibly faster. Especially I find them useful for filling the gaps of knowledge and exploring new topics in my own way and language and then you verify every single fact it tells you via traditional methods by confirming them in human-written documents, right? Otherwise, how do you use the LLM for learning? If you don't know the answer to what you're asking, you can't tell if it's lying. It also can't tell if it's lying, so you can't ask it. If you have to look up every fact it outputs after it does, using traditional methods, why not skip to just looking things up the old fashioned way and save time? Occasionally an LLM helps me surface unknown keywords that make traditional searches easier, but they can't teach anything because they don't know anything. They can imagine things you might be able to learn from a real authority, but that's it. That can be useful! But it's not useful for learning alone. And if you're not verifying literally everything an LLM tells you.. are you sure you're learning anything real?
      - kardos 2y ago
        The Gell-Mann amnesia effect applies to LLMs as well! https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect
      - taikahessu 2y ago
        I guess it all depends on the topic and levels of trust. How can I be certain that I have a brain? I just have to take something for granted, don't I? Of course I will "verify" the "important stuff", but what is important? How can I tell? Most of the time only thing I need is a pointer in the right direction. Wrong advice? I know when I get there I suppose. I can remember numerous things I was told while growing up, that aren't actually true. Either by plain lies and rumours or because of the long list of our cognitive biases. > If you have to look up every fact it outputs after it does, using traditional methods, why not skip to just looking things up the old fashioned way and save time? What is the old fashioned way? I mean people learn "truths" these days from Tiktok and Youtube. Some of the stuff is actually very good, you just have to distill it based on the stuff I was being taught at school. Nonody has yet declared LLMs as a subtitute for schools, maybe they soon will, but neither "guarantees" us anything. We could as well be taught political agendas. I could order a book about construction, but I wouldn't build a house without asking a "verified" expert. Some people build anyway and we get some catastrofic results. Levels of trust, it's all games and play until it gets serious, like what to eat or doing something that involves life threatening physics. I take it as playing with a toy. Surely something great have come up from only a few piece of legos? > And if you're not verifying literally everything an LLM tells you.. are you sure you're learning anything real? I guess you shouldn't do it that way. But really, so far the topics I've rigorously explored with ChatGPT for example, have been better than your average journalism. What is real?
  - deleted 2y ago
    [deleted]
- kees99 2y ago
  "Occasional nonsense" doesn't sound great, but would be tolerable. Problem is - LLMs pull answers from their behind, just like a lazy student on the exam. "Halucinations" is the word people use to describe this. Those are extremely hard to spot - unless you happen to know the right answer already, at which point - why ask? And those are everywhere. One example - recently there was quite a discussion about llm being able to understand (and answer) base16 (aka "hex") encoding on the fly, so I went on to try base64, gzipped base64, zstd-compressed base64, etc... To my surprise, LLM got most of those encoding/compressions right, decoded/uncompressed the question, and answered it flawlessly. But with few encodings, LLM detected base64 correctly, got compression algorithm correctly, and then... instead of decompressing, made up a completely different payload, and proceeded to answer that. Without any hint of anything sinister going. We really need LLMs to reliably calculate and express confidence. Otherwise they will remain mere toys.
  - oldstrangers 2y ago
    Yeah, what you said represents a 'net gain' over not having any of that at all.
- majormajor 2y ago
  I think as these things get more integrated into customer service workflows - especially for things like insurance claims - there's gonna start being a lot more buyer's remorse on everyone's part. We've tried for decades to turn people into reliable robots, now many companies are running to replace people robots with (maybe less reliable?) robot-robots. What could go wrong? What are the escalation paths going to be? Who's going to be watching them?
- hawaiianbrah 2y ago
  A net gain for everyone? Tell that to the artists its screwing over!
deleted 2y ago
[deleted]
throwing_away 2y ago
> We'd never hire someone who just makes stuff up (or at least keep them employed for long). This is contrary to my experience.
- kadushka 2y ago
  Our president begs to differ! Or pretty much any elected official for that matter.
  - Gud 2y ago
    Not everything is politics. Already your president gets too much media spotlight.
Gerardo1 2y ago
> Why are we okay with calling "AI" tools like this anything other than curious research projects? Because they are a way to launder liability while reducing costs to produce a service. Look at the AI-based startups y-combinator has been funding. They match that description.
deeviant 2y ago
Yeah, I used to hire people, but then one of them made a mistake, now I'm done with them forever, they are useless. It is not I, who is directing the workers, who cannot create a process that is resistant to errors, it's definitely the fact that all people are worthless until they make no errors as there truly is no other way of doing things other than telling your intern to do a task then having them send it directly to the production line.
ramon156 2y ago
3k a month vs ~500 dollars a month. That's all u need to know. Not saying its as good, but its all some managers care about
kenjackson 2y ago
You can use them for whatever you like, or not use them. Everyone has a different bar for when technology is useful. My dad doesn't think EVs are useful due to the long charge times, but there are others who find it fully acceptable.
dumbfounder 2y ago
Why not just verify the output? It’s faster than generating the entire thing yourself. Why do you need perfection in a productivity tool?
- toasteros 2y ago
  At that point why not just... I dunno, do the research yourself?
  - tuckerman 2y ago
    Perhaps because the time to proofread/correct is less than to do it from scratch? That would still make it a valuable tool
    - Yoric 2y ago
      But is it?
    - toasteros 2y ago
      How? It's given you some information and now you have to seek out a source to verify that it's correct. Finding information is hard work. It's why librarian is a valuable skilled profession. What you've done by suggesting that I should "verify" or "proofread" what a glorified, water-wasting Markov chain has given me now entails me looking up that information to verify that it's correct. That's...not quite doubling the work involved but it's adding an unnecessary step. I could have searched for the source in the first instance. I could have gone to the library and asked for help. We spent time coming up with a question ("prompt engineering"! hah!), we used up a bunch of electricity for an answer to be generated and now you...want me to search up that answer to find the source? Why did we do the first step? People got undergraduate degrees - hell, even PhDs - before generative AI. Look up the tweet from someone who said "Sometimes when coming up with a good prompt for ChatGPT, I sometimes come up with the answer myself without needing to submit".
      - murloc_oracle 2y ago
roflyear 2y ago
> We'd never hire someone who just makes stuff up We do all the time - of course we do, all the time.
nomel 2y ago
LLM are "great" in some use cases, "ok" in others, and "laughable" in more. Some people might find $500 worth of value, in their specific use case, in those "great" and "ok" categories, where they get more value than "lies" out of it. A few verifiable lies, vs hours of time, could be worth it for some people, with use cases outside of your perspective.
rybosome 2y ago
This doesn’t make LLMs worthless, you just need to structure your processes around fallibility. Much like a well designed release pipeline is built with the expectation that devs will write bugs that shouldn’t ship.

oldstrangers 2y ago

> Can't we just send LLMs back to the drawing board until they have some semblance of reliability? Well at this point they've certainly proven a net gain for everyone regardless of the occasional nonsense they spew.

aiono 2y ago
No, from the research around it the findings are mixed. There is no consensus that it's net gain.
DanHulton 2y ago
That is... debatable. You may be entirely inside the bubble, there.
- orangepanda 2y ago
  You overestimate the importance of being correct
- taikahessu 2y ago
  Not sure if this was posted as humour, but I don't feel that way. In today's world, where I certainly would consider taking the blue pill, I'm having a blast with LLMs! It has helped me learn stuff incredibly faster. Especially I find them useful for filling the gaps of knowledge and exploring new topics in my own way and language, without needing to wait an answer from a human (that could also be wrong). Why does it feel, that "we are entirely inside the bubble" for you?
  - dingnuts 2y ago
    >It has helped me learn stuff incredibly faster. Especially I find them useful for filling the gaps of knowledge and exploring new topics in my own way and language and then you verify every single fact it tells you via traditional methods by confirming them in human-written documents, right? Otherwise, how do you use the LLM for learning? If you don't know the answer to what you're asking, you can't tell if it's lying. It also can't tell if it's lying, so you can't ask it. If you have to look up every fact it outputs after it does, using traditional methods, why not skip to just looking things up the old fashioned way and save time? Occasionally an LLM helps me surface unknown keywords that make traditional searches easier, but they can't teach anything because they don't know anything. They can imagine things you might be able to learn from a real authority, but that's it. That can be useful! But it's not useful for learning alone. And if you're not verifying literally everything an LLM tells you.. are you sure you're learning anything real?
    - kardos 2y ago
      The Gell-Mann amnesia effect applies to LLMs as well! https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect
    - taikahessu 2y ago
      I guess it all depends on the topic and levels of trust. How can I be certain that I have a brain? I just have to take something for granted, don't I? Of course I will "verify" the "important stuff", but what is important? How can I tell? Most of the time only thing I need is a pointer in the right direction. Wrong advice? I know when I get there I suppose. I can remember numerous things I was told while growing up, that aren't actually true. Either by plain lies and rumours or because of the long list of our cognitive biases. > If you have to look up every fact it outputs after it does, using traditional methods, why not skip to just looking things up the old fashioned way and save time? What is the old fashioned way? I mean people learn "truths" these days from Tiktok and Youtube. Some of the stuff is actually very good, you just have to distill it based on the stuff I was being taught at school. Nonody has yet declared LLMs as a subtitute for schools, maybe they soon will, but neither "guarantees" us anything. We could as well be taught political agendas. I could order a book about construction, but I wouldn't build a house without asking a "verified" expert. Some people build anyway and we get some catastrofic results. Levels of trust, it's all games and play until it gets serious, like what to eat or doing something that involves life threatening physics. I take it as playing with a toy. Surely something great have come up from only a few piece of legos? > And if you're not verifying literally everything an LLM tells you.. are you sure you're learning anything real? I guess you shouldn't do it that way. But really, so far the topics I've rigorously explored with ChatGPT for example, have been better than your average journalism. What is real?
- deleted 2y ago
  [deleted]
kees99 2y ago
"Occasional nonsense" doesn't sound great, but would be tolerable. Problem is - LLMs pull answers from their behind, just like a lazy student on the exam. "Halucinations" is the word people use to describe this. Those are extremely hard to spot - unless you happen to know the right answer already, at which point - why ask? And those are everywhere. One example - recently there was quite a discussion about llm being able to understand (and answer) base16 (aka "hex") encoding on the fly, so I went on to try base64, gzipped base64, zstd-compressed base64, etc... To my surprise, LLM got most of those encoding/compressions right, decoded/uncompressed the question, and answered it flawlessly. But with few encodings, LLM detected base64 correctly, got compression algorithm correctly, and then... instead of decompressing, made up a completely different payload, and proceeded to answer that. Without any hint of anything sinister going. We really need LLMs to reliably calculate and express confidence. Otherwise they will remain mere toys.
- oldstrangers 2y ago
  Yeah, what you said represents a 'net gain' over not having any of that at all.
majormajor 2y ago
I think as these things get more integrated into customer service workflows - especially for things like insurance claims - there's gonna start being a lot more buyer's remorse on everyone's part. We've tried for decades to turn people into reliable robots, now many companies are running to replace people robots with (maybe less reliable?) robot-robots. What could go wrong? What are the escalation paths going to be? Who's going to be watching them?
hawaiianbrah 2y ago
A net gain for everyone? Tell that to the artists its screwing over!

deleted 2y ago

[deleted]

throwing_away 2y ago

> We'd never hire someone who just makes stuff up (or at least keep them employed for long). This is contrary to my experience.

kadushka 2y ago
Our president begs to differ! Or pretty much any elected official for that matter.
- Gud 2y ago
  Not everything is politics. Already your president gets too much media spotlight.

Gerardo1 2y ago

> Why are we okay with calling "AI" tools like this anything other than curious research projects? Because they are a way to launder liability while reducing costs to produce a service. Look at the AI-based startups y-combinator has been funding. They match that description.

deeviant 2y ago

Yeah, I used to hire people, but then one of them made a mistake, now I'm done with them forever, they are useless. It is not I, who is directing the workers, who cannot create a process that is resistant to errors, it's definitely the fact that all people are worthless until they make no errors as there truly is no other way of doing things other than telling your intern to do a task then having them send it directly to the production line.

ramon156 2y ago

3k a month vs ~500 dollars a month. That's all u need to know. Not saying its as good, but its all some managers care about

kenjackson 2y ago

You can use them for whatever you like, or not use them. Everyone has a different bar for when technology is useful. My dad doesn't think EVs are useful due to the long charge times, but there are others who find it fully acceptable.

dumbfounder 2y ago

Why not just verify the output? It’s faster than generating the entire thing yourself. Why do you need perfection in a productivity tool?

toasteros 2y ago
At that point why not just... I dunno, do the research yourself?
- tuckerman 2y ago
  Perhaps because the time to proofread/correct is less than to do it from scratch? That would still make it a valuable tool
  - Yoric 2y ago
    But is it?
  - toasteros 2y ago
    How? It's given you some information and now you have to seek out a source to verify that it's correct. Finding information is hard work. It's why librarian is a valuable skilled profession. What you've done by suggesting that I should "verify" or "proofread" what a glorified, water-wasting Markov chain has given me now entails me looking up that information to verify that it's correct. That's...not quite doubling the work involved but it's adding an unnecessary step. I could have searched for the source in the first instance. I could have gone to the library and asked for help. We spent time coming up with a question ("prompt engineering"! hah!), we used up a bunch of electricity for an answer to be generated and now you...want me to search up that answer to find the source? Why did we do the first step? People got undergraduate degrees - hell, even PhDs - before generative AI. Look up the tweet from someone who said "Sometimes when coming up with a good prompt for ChatGPT, I sometimes come up with the answer myself without needing to submit".
    - murloc_oracle 2y ago

roflyear 2y ago

> We'd never hire someone who just makes stuff up We do all the time - of course we do, all the time.

nomel 2y ago

LLM are "great" in some use cases, "ok" in others, and "laughable" in more. Some people might find $500 worth of value, in their specific use case, in those "great" and "ok" categories, where they get more value than "lies" out of it. A few verifiable lies, vs hours of time, could be worth it for some people, with use cases outside of your perspective.

rybosome 2y ago

This doesn’t make LLMs worthless, you just need to structure your processes around fallibility. Much like a well designed release pipeline is built with the expectation that devs will write bugs that shouldn’t ship.