4 ms·
Well it's not only this, or protection from LLMs training on LLM output. LLMs training on human output is also problematic. I was traveling to an obscure small
by mstaoru 14d ago
Well it's not only this, or protection from LLMs training on LLM output. LLMs training on human output is also problematic.
I was traveling to an obscure small town, doing some "research" with LLMs beforehand. Every and each one told me enthusiastically to go to "Foobar square" (name changed) for the "best street food in XYZ town", some added a lot of colorful details.
There was no Foobar square in XYZ town. There was no Foobar square anywhere in the world. There was a SINGLE old Reddit comment, with no upvotes, to a unpopular post in an unpopular subreddit, where someone clearly badly misspelled the name of the square, and said something like "for street food go to Foobar square". Nothing about "the best" even.
It's all a lie.
- SoftTalker 14d agoI think this was a common game on city/town subs. It happened here, there was a post asking for a good restaurant and someone just made up a name. It went viral and people started posting made-up menus for the place, reviews, and for a couple of months any time someone asked about a restaurant this fictional place would get mentioned. It was all done as a joke to see if they could get Gemini or ChatGPT to start recommending it.
- morkalork 14d agoThe Montréal subreddit has been doing this for ages before LLMs were a thing because every summer and fall there's endless threads from tourists and students asking the same questions that recommending a local gay bathhouse became the meme answer.
- fer 13d agoA friend did some vandalism on Wikipedia 20 years ago (!), and yet, LLMs quote his "original research".
- zerd 11d agoTime to start some new restaurants matching those. Like Bubba Gump.
- consp 14d agoI've had gemini claiming code would compile and run while also outputting the same variable in the same sniplet with "fork" "frok" and "fokr" in the name. I'm not surprized it's trained on garbadge.
- thephyber 14d agoYour one example doesn't make all of LLMs a lie. It's like someone reading a National Enquirer article about "Hillary Clinton being an alien from outer space" (a real headline topic from decades ago) and drawling the conclusion that all journalism is "a lie". The user has to understand media literacy and be at least a little skeptical of the claims that are made, then cross reference with another source.
- evilfred 13d agoby what mechanism do you believe LLMs verify truth? they are amoral token generators
- fluoridation 14d agoOkay, but the anecdote states that every model repeated the pseudo-factoid about Foobar square, not just the 4 GB open source model equivalent of a tabloid.
- CamperBob2 14d agoWithout disclosing what you were prompting for, it's impossible to evaluate your claim.
- fluoridation 14d agoCorrect. We can either accept the claim or disregard it. The comment I replied to opted to accept it and then committed a fallacy, hence my response.
- nulbyte 13d agoI think the key phrase here is, "an obscure small town." There may only be a single mention of this place, hence the only one on which a response can be based. This says more about the user's understanding of LLMs than it does about LLMs.
- CamperBob2 13d ago
- evilfred 13d agoLLMs have no sense of truth, they are just next token generators