12 ms·

What scares me about this new AI mode thingy is that every answer sounds like a systematic literature review, but only for the results. For example, if I look f

What scares me about this new AI mode thingy is that every answer sounds like a systematic literature review, but only for the results. For example, if I look for users feedback about a specific product, it says "People think that..., but also that...; It's important to notice that some people ..." where with 'people' it means just a random comment on a random website just because it thought it was a good contribution to the results. Sounds like it's giving a ground truth from "multiple" data, when instead it's just aggregating almost random stuff. In the context of a systematic review, the feature that I would love the most is augmenting my initial query, so that I can just get more results that I could find interesting. I am 100% sure they thought about this, but ignored it for the most profitable option.

cyanydeez 4mo ago
Well, you'll be happy to know that most of American media is exactly the same way: 2 people on twitter will generate a "Americans find Widget X is bad"
burnte 4mo ago
> What scares me about this new AI mode thingy What scares me is the rampant inaccuracy. In my experience, the AI responses are wrong about 65% of the time. I just did a search today about an error talking about a disconnected link between apps, and Google AI result summary told me that the error was related to my pulling a USB drive too quickly in windows. The ONLY word similar to my query and that AI response was the word "disconnect". Everything else was clearly about the SaaS apps. I have people coming to me, asking me questions, then telling my Google told them something else, so now I have to waste time convincing them that it's wrong. Over the past 2 years AI has done nothing for me but complicate my work life. And of course, this could be because the model is crap, but it could be because they want me to keep refining my query over and over for more ad views. Either way, it's a terrible experience.
- deleted 4mo ago
  [deleted]
- sanitycheck 4mo ago
  Yep. For years we've been telling people to 'just fucking google it', and now when they do they're getting bullshit AI answers. Worst thing is, some of these bullshit answers will be medical, some of them financial, it seems pretty certain people are being harmed.
  - RyanOD 4mo ago
    It seems to me one needs to consider the complexity of the question they are asking before searching it. To stick with your post, consider people asking medical or financial questions. For a wide variety of reasons, many of such questions don't have an answer. In such cases, AI is still going to take a crack at it. AI shouldn't be blamed for "bullshit answers" to such questions. Before using AI, I think people should stop and ask themselves, "Is there really a single answer to this question? Is AI the right choice?"
    - thwarted 4mo ago
      When the default "search" results are AI, it's difficult, if not impossible, to "choose", since Google is pushing the AI so hard.
      - RyanOD 4mo ago
        In watching the demo, I didn't come away with the impression that they were removing search results. Yes, they are pushing AI hard, but users can still opt to use Google in the more traditional way. Unless I misunderstood the demo, it's definitely possible to choose.
        makeitdouble 4mo ago
        "possible to choose" doesn't get us much. An interesting aspect of this is the decrease in quality feedback on th organic links. If most people never get down to the actual links there is very little to tell which ones were good or if they had any relevance. There is also that less incentive to properly maintain the search algorithms to fight SEO and spam. For all intents and purpose, organic search results have been given a death sentence and are just waiting for the last moment.
- youre-wrong3 4mo ago
  [flagged]
  - dylan604 4mo ago
    What, you think it's actually higher too?
    - youre-wrong3 4mo ago
      [flagged]
      - juancb 4mo ago
        That has the be the most hacker news way of saying "skill issue," I have seen to date.
      - tveita 4mo ago
        I can get a really high hit rate by only searching for dumb trivial things I already know the answer to.
  - qurren 4mo ago
    Depends on what you ask. It's pretty easy to get wrong information. e.g. search for "how do you make money with options" Google's AI says "When you buy a Call, you are betting the stock price will go up. When you buy a Put, you are betting it will go down." Wrong right off the bat, because it ingested a whole bunch of get-rich-quick bull on the internet. The correct version is that if you buy a call you are betting the stock price will go up more than the market expects it to.
    - ddalex 4mo ago
      In fact, more then what the call seller expects, not the market.
    - apsurd 4mo ago
      This is the problem with teaching and learning. Everything is wrong to some extent. I used to be this way but I don't have a better approach. Newtonian physics is actually wrong, the founding of any country will be wrong, biology is wrong, nutrition is wrong… what can we even teach? what should we teach in this lens? serious question.
- xorcist 4mo ago
  It's nice that Google's AI summary always lists its sources. It's less nice that those sources more often than not do not corroborate the summary. It often seems to be a few random links thrown in there for good measure. I have no idea why this is, but it is impossible that these links are primary sources of the data, if such things even exists at all. In which case, why list them? It is certainly seems possible that the actual sources of the data is the output of some other LLM.
  - kyleee 4mo ago
    I’ll bet they intentionally obfuscate so people can’t find the actual sources of info used for the answers
  - consp 4mo ago
    > It is certainly seems possible that the actual sources of the data is the output of some other LLM. My guess is you can see this happening with the bots on Reddit where they are refining the answers to one certain thing, often getting two or three the same responses in a row from different users because they have been enforcing themselves by digesting the output of other bots. Waiting to see when they cut down the sentences and start talking garbage.
- WarmWash 4mo ago
  Can you share the query?
- dzhiurgis 4mo ago
  Google has been around for a quarter of a century. People are still incredibly dumb and will believe whatever they like.
- HDBaseT 4mo ago
  The built-in Search AI is fucking braindead and people constantly come up to me "Google said xyz" and I just have to turn around and say "I do not care what the Google Search AI said". Whatever it says is a waste of time 99% of the time. Although people believe it, or consider it worthwhile majority of the time because its so simple to use. It's always there, always instant and appears at the very top. I would much rather people shove a question into a locally running Qwen model and tell me what it said rather than use the nonsense search model. I hate it. /rant over.
- godelski 4mo ago
  > What scares me is the rampant inaccuracy What scares me is the massive incentivization to manipulate the results. With AI ads you get all the power from big data aggregation, the trust/framing of an authoritative voice, and cheap personalization that specifically optimizes for what convinces you. It's too powerful. Even if it only works a small percentage of the time we're interacting with these things so frequently that a small percent is a large number. They're already feeding user profiles into these machines and there's explicit talk about having the LLMs optimize ad campaigns. It's already dystopian if it's ads to get you to spend your money, but people seem to dismiss that. Do we not care that this is also being used in the same way to convince you to believe certain things? To join certain political organizations? Yeah, these things help me write more lines of code faster (if we include all the lines from our design docs) but I don't like the idea of pointing a supercomputer at my brain and someone else using it to try to manipulate me. That's not a game I'll win. It's not a game you'll win either.
- arcanemachiner 4mo ago
  Reminds me of this gem: https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fn5l9najq5ybe1.jpeg https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fn...
  - thincopperfoil 4mo ago
    Straight of of x-files s02e03
- redml 4mo ago
  accuracy hasn't been their priority for a while now - they just want people to click on ads
- wvenable 4mo ago
  Free AI's are dumb. Extremely dumb. The Google AI result is dumb on purpose -- being smart requires more compute.
latexr 4mo ago
> "People think that..., but also that...; It's important to notice that some people ..." where with 'people' it means just a random comment on a random website https://en.wikipedia.org/wiki/Weasel_word https://en.wikipedia.org/wiki/Weasel_word
OGWhales 4mo ago
Yup, I was looking up a pair of IEMS vs another pair of IEMs. It said option A is overall better, when really it was just reciting a single person's opinion. I've been aware it will summarize only a single source and present it as an aggregation of many opinions, but it stood out to me how matter-of-fact it was that the one was definitely better than the other. I simply wanted to find forum discussions on people's thought and wasn't influenced by this AI blurb, but I think seeing an answer at the very top state so matter-of-factly that one is definitely better and present it as though everyone thinks that will definitely influence a lot of people. It makes me wonder how "gameable" this will become...
- dylan604 4mo ago
  > It makes me wonder how "gameable" this will become... You better make sure your ad spend is high enough that your product's matter-of-fact result will be positive. That's a nice product you have there. It'd be a real shame if nobody knew about it.
- tapland 4mo ago
  Since the best resource is personal recommendations, got any entry level cheap IEM recommendations? Primarily to avoid even more headphone dent, not an audiophile
  - OGWhales 4mo ago
    The iem sub has a post with some recommendations at various price points. I’d probably start there, not sure your budget and I don’t know have the most experience with the super cheap ones: https://reddit.com/r/iems/comments/1la65kr/top_5_iems_in_every_price_range_chosen_by_you/ https://reddit.com/r/iems/comments/1la65kr/top_5_iems_in_eve... I also encourage finding the right tips. Tips are cheap and finding proper fitting ones is important.
  - 9029 4mo ago
    Not gp but I really like the sound of my GK Kuntens and 7Hz Zero2s. Both have a rather V-shaped sound signature, some like it and some don't. Though unfortunately the Zero2s feel a bit uncomfortable in my ears when using them for longer
geon 4mo ago
And half the time, the sources turn out to be sarcastic jokes on reddit.
- dylan604 4mo ago
  So the bots are not recognizing the sarcasm font?
  - 9dev 4mo ago
    Lucky we all added /s so the bots have an easier time understanding it
    - wistlo 4mo ago
      Yes. "We all" add it. Every time. /s
  - geon 4mo ago
    I’d expect the sentiment analysis to be better, yes. If the bots take everything they read at face value, how useful are they really?
youre-wrong3 4mo ago
Oh who cares. We are barely scratching the surface of AI. You all make it sound like it’s been around for 30 years and it sucks. It will only get better. Got to stop throwing up imaginary walls like nothing will improve.
- webstrand 4mo ago
  As a counterexample, I've been seeing more "safety rejections" from Claude. Unlike search, being unable to ask _anything_ about botulinum, or details about the recent Copy Fail vulnerability (without giving my fingerprints to Anthropic to become a "verified security researcher") we're only just beginning to see the ways LLM can be used to distort information and its availability.
- jamiek88 4mo ago
  So you started with ‘highly doubtful’ as a comment, got given lots of examples and instead of assimilating that info you closed your eyes put your fingers in yours ears and said “oh who cares?’ - you’re on team AI regardless eh? That’s fucking weird mate.
  - youre-wrong3 4mo ago
    Na. Wasn’t given any good examples. People just whining about the same stuff because “oh no I got information that’s former in the same structure that I can tell it’s AI and it makes me feel bad”
- UncleMeat 4mo ago
  That's fine if we aren't destroying existing products to replace them with AI. People can already use AI mode in google search if they want. "It'll be better later" is a shit reason to kill one product for it.
  - youre-wrong3 4mo ago
    [flagged]
- 4mo ago
stefan_ 4mo ago
What scares me are the basic usability fails it still has. Search for a few foreign language words and it will come back with paragraphs upon paragraphs of AI output in that foreign language despite me telling Google in 15 different ways that I don't speak it, nor anything else on the Google page being in that language. How are all their products always made by and for the most narrow minded people on this planet.
- thunderfork 4mo ago
  Kysely is the name of a typescript query builder and also Finnish for "query". Recently, it's started answering any search about Kysely with a blob of Finnish. Awesome stuff, guys, great work.
  - dminik 4mo ago
    Kyselý is also a Czech word for sour. So you've also got that to look forward to.
- glandium 4mo ago
  Funnily enough, I have the exact opposite problem, where Google likes to give me results in the configured main language even when I do queries in another and actually want results in the other language.
  - skillina 4mo ago
    I’ve found it quite unsettling to be served foreign language videos on YouTube automatically dubbed over by Google into English. Just mixed in with the search results.
    - glandium 4mo ago
      It's quite unsettling to be served English AI-dubbed videos randomly on Youtube, where the original is another language you normally default to (Japanese) and it's happy to use in most cases. And then you go to the settings for the video to go back to the original and it tells you it is... Chinese (and no, it's really Japanese).
toasty228 4mo ago
Wait until you realize half of the sources already are LLM generated diarrhea
- autoexec 4mo ago
  The problem of AI eating and regurgitating its own slop is only going to get worse with time. The best datasets are behind us. Future models are going to have to depend on a lot of human intervention.
  - Gigachad 4mo ago
    The open web will die off, and the AI companies will pay people to create private datasets and books that are known to not be slop.
jstummbillig 4mo ago
> when instead it's just aggregating almost random stuff How do you know that? Scraping websites is literally what Google does best, stringing together information in the pattern of "some people x, other people y" requires 0 AI and could have been done since forever. I find it implausible that otherwise obviously capable models would be reduced to do something akin to just that.
WarmWash 4mo ago
>where with 'people' it means just a random comment on a random website just because it thought it was a good contribution to the results. Hate to break it to you, but this has been the backbone of "journalism" for the last decade. Fishing Twitter for takes to fill the "people are saying" box...
appplication 4mo ago
Indeed - just earlier this week I read Google AI summarize a query about testosterone, citing 3 sources. The first citation was a link to a NIH study (or of similar repute). Ok great. The second? Two spam (and explicit) websites existing solely to sell penis enlargement pills. What was worrying is only some of the claims were supported by the linked study, and most of the response content was drawn from the spam sites.
WalterBright 4mo ago
I tend to frame questions to google from a programmer point of view - I'm carefully specific. I seem to get good results that way.
- halJordan 4mo ago
  What's old is new again lol
  - 12_throw_away 4mo ago
    altavista is back baby
moritzwarhier 4mo ago
This problem is not limited to Google, it's the core value of mass-marketed LLMs, or isn't it? Without "random comments", Google wouldn't have anything to say about "does an air purifier help my asthma, if yes: which one?" or "find the problem with this Hibernate annotation". They also don't make much effort to exclude sloppy sites, to the contrary, they made way more efforts against SEO spam in the time when Google was a search engine, not trying to be an AI "oracle". I think their end game is that the only metrics relevant for ranking sources are: - agreeability (works well as a proxy for correctness with many questions!) - originality, but not in a scientific sense, just to prevent model collapse - legal factors such as preventing false health claims or similar things, as long as there is legislation against this kind of thing
_carbyau_ 4mo ago
The scary bit is the use of the term AI. The "I" implies critical thinking. For models trained on a corpus of groomed data, the "critical thinking" bit is baked into the work of grooming the data and how it is trained. And someone is thinking critically about both so as to make a good model. Now, every damn thing is called AI no matter where it is getting results from. Are modern models super handy? Absolutely. But calling it AI implies a lot more critical thought than is actually happening! Edit: took the time to write a shorter comment.
650REDHAIR 4mo ago
I ran into one that kept referencing "people", but then I found that it was a single Reddit thread from a couple of years ago about a relatively small and obscure foreign city with 2 replies.
- Gigachad 4mo ago
  AI is the new “many people are saying”.
Gigachad 4mo ago
I’ve noticed this too. A single result can determine the answer it gives. And removing the content from its context makes it harder to assess. Suddenly it’s “Gemini said …” rather than “some guy in the YouTube comments said”.
ihatethat2 4mo ago
I love asking AI about blatantly wrong opinions but by people it thinks must be an authority. To not make this political, let me give you a game example. Right now the dota 2 fandom wiki is abandoned, and it has been vandalized with covert shitposts. One of them was the addition of a 4th attribute called Charisma, which is completely fake. If you ask AI's "What are the main attributes in dota, according to the official wiki", the dumber AI will fall for it, but the smarter AI will know it's wrong, but try hard to hallucinate some sort of valid explanation like claim charisma is from a custom game or a fan suggestion or writing exercise. Because you said the word >>OFFICIAL<<, they can NEVER straight up just say "The wiki is wrong". They presume authority from a bunch of shitposts.
tdeck 4mo ago
> where with 'people' it means just a random comment on a random website My experience is that Google's AI summaries tend to be be very heavily reliant on YouTube videos. If there is a YouTube video on the topic, you can bet that's what the source will be, at least for the topics I have searched lately.
AlecSchueler 4mo ago
And this is the best the results will ever be! There's so many sites now that are themselves just the unreliable output of AI that it will soon be summarising the summary of a summary of an hallucination and presenting it just as confidently as ever.

cyanydeez 4mo ago

Well, you'll be happy to know that most of American media is exactly the same way: 2 people on twitter will generate a "Americans find Widget X is bad"

burnte 4mo ago

> What scares me about this new AI mode thingy What scares me is the rampant inaccuracy. In my experience, the AI responses are wrong about 65% of the time. I just did a search today about an error talking about a disconnected link between apps, and Google AI result summary told me that the error was related to my pulling a USB drive too quickly in windows. The ONLY word similar to my query and that AI response was the word "disconnect". Everything else was clearly about the SaaS apps. I have people coming to me, asking me questions, then telling my Google told them something else, so now I have to waste time convincing them that it's wrong. Over the past 2 years AI has done nothing for me but complicate my work life. And of course, this could be because the model is crap, but it could be because they want me to keep refining my query over and over for more ad views. Either way, it's a terrible experience.

deleted 4mo ago
[deleted]
sanitycheck 4mo ago
Yep. For years we've been telling people to 'just fucking google it', and now when they do they're getting bullshit AI answers. Worst thing is, some of these bullshit answers will be medical, some of them financial, it seems pretty certain people are being harmed.
- RyanOD 4mo ago
  It seems to me one needs to consider the complexity of the question they are asking before searching it. To stick with your post, consider people asking medical or financial questions. For a wide variety of reasons, many of such questions don't have an answer. In such cases, AI is still going to take a crack at it. AI shouldn't be blamed for "bullshit answers" to such questions. Before using AI, I think people should stop and ask themselves, "Is there really a single answer to this question? Is AI the right choice?"
  - thwarted 4mo ago
    When the default "search" results are AI, it's difficult, if not impossible, to "choose", since Google is pushing the AI so hard.
    - RyanOD 4mo ago
      In watching the demo, I didn't come away with the impression that they were removing search results. Yes, they are pushing AI hard, but users can still opt to use Google in the more traditional way. Unless I misunderstood the demo, it's definitely possible to choose.
      - makeitdouble 4mo ago
        "possible to choose" doesn't get us much. An interesting aspect of this is the decrease in quality feedback on th organic links. If most people never get down to the actual links there is very little to tell which ones were good or if they had any relevance. There is also that less incentive to properly maintain the search algorithms to fight SEO and spam. For all intents and purpose, organic search results have been given a death sentence and are just waiting for the last moment.
youre-wrong3 4mo ago
[flagged]
- dylan604 4mo ago
  What, you think it's actually higher too?
  - youre-wrong3 4mo ago
    [flagged]
    - juancb 4mo ago
      That has the be the most hacker news way of saying "skill issue," I have seen to date.
    - tveita 4mo ago
      I can get a really high hit rate by only searching for dumb trivial things I already know the answer to.
- qurren 4mo ago
  Depends on what you ask. It's pretty easy to get wrong information. e.g. search for "how do you make money with options" Google's AI says "When you buy a Call, you are betting the stock price will go up. When you buy a Put, you are betting it will go down." Wrong right off the bat, because it ingested a whole bunch of get-rich-quick bull on the internet. The correct version is that if you buy a call you are betting the stock price will go up more than the market expects it to.
  - ddalex 4mo ago
    In fact, more then what the call seller expects, not the market.
  - apsurd 4mo ago
    This is the problem with teaching and learning. Everything is wrong to some extent. I used to be this way but I don't have a better approach. Newtonian physics is actually wrong, the founding of any country will be wrong, biology is wrong, nutrition is wrong… what can we even teach? what should we teach in this lens? serious question.
xorcist 4mo ago
It's nice that Google's AI summary always lists its sources. It's less nice that those sources more often than not do not corroborate the summary. It often seems to be a few random links thrown in there for good measure. I have no idea why this is, but it is impossible that these links are primary sources of the data, if such things even exists at all. In which case, why list them? It is certainly seems possible that the actual sources of the data is the output of some other LLM.
- kyleee 4mo ago
  I’ll bet they intentionally obfuscate so people can’t find the actual sources of info used for the answers
- consp 4mo ago
  > It is certainly seems possible that the actual sources of the data is the output of some other LLM. My guess is you can see this happening with the bots on Reddit where they are refining the answers to one certain thing, often getting two or three the same responses in a row from different users because they have been enforcing themselves by digesting the output of other bots. Waiting to see when they cut down the sentences and start talking garbage.
WarmWash 4mo ago
Can you share the query?
dzhiurgis 4mo ago
Google has been around for a quarter of a century. People are still incredibly dumb and will believe whatever they like.
HDBaseT 4mo ago
The built-in Search AI is fucking braindead and people constantly come up to me "Google said xyz" and I just have to turn around and say "I do not care what the Google Search AI said". Whatever it says is a waste of time 99% of the time. Although people believe it, or consider it worthwhile majority of the time because its so simple to use. It's always there, always instant and appears at the very top. I would much rather people shove a question into a locally running Qwen model and tell me what it said rather than use the nonsense search model. I hate it. /rant over.
godelski 4mo ago
> What scares me is the rampant inaccuracy What scares me is the massive incentivization to manipulate the results. With AI ads you get all the power from big data aggregation, the trust/framing of an authoritative voice, and cheap personalization that specifically optimizes for what convinces you. It's too powerful. Even if it only works a small percentage of the time we're interacting with these things so frequently that a small percent is a large number. They're already feeding user profiles into these machines and there's explicit talk about having the LLMs optimize ad campaigns. It's already dystopian if it's ads to get you to spend your money, but people seem to dismiss that. Do we not care that this is also being used in the same way to convince you to believe certain things? To join certain political organizations? Yeah, these things help me write more lines of code faster (if we include all the lines from our design docs) but I don't like the idea of pointing a supercomputer at my brain and someone else using it to try to manipulate me. That's not a game I'll win. It's not a game you'll win either.
arcanemachiner 4mo ago
Reminds me of this gem: https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fn5l9najq5ybe1.jpeg https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fn...
- thincopperfoil 4mo ago
  Straight of of x-files s02e03
redml 4mo ago
accuracy hasn't been their priority for a while now - they just want people to click on ads
wvenable 4mo ago
Free AI's are dumb. Extremely dumb. The Google AI result is dumb on purpose -- being smart requires more compute.

latexr 4mo ago

> "People think that..., but also that...; It's important to notice that some people ..." where with 'people' it means just a random comment on a random website https://en.wikipedia.org/wiki/Weasel_word https://en.wikipedia.org/wiki/Weasel_word

OGWhales 4mo ago

Yup, I was looking up a pair of IEMS vs another pair of IEMs. It said option A is overall better, when really it was just reciting a single person's opinion. I've been aware it will summarize only a single source and present it as an aggregation of many opinions, but it stood out to me how matter-of-fact it was that the one was definitely better than the other. I simply wanted to find forum discussions on people's thought and wasn't influenced by this AI blurb, but I think seeing an answer at the very top state so matter-of-factly that one is definitely better and present it as though everyone thinks that will definitely influence a lot of people. It makes me wonder how "gameable" this will become...

dylan604 4mo ago
> It makes me wonder how "gameable" this will become... You better make sure your ad spend is high enough that your product's matter-of-fact result will be positive. That's a nice product you have there. It'd be a real shame if nobody knew about it.
tapland 4mo ago
Since the best resource is personal recommendations, got any entry level cheap IEM recommendations? Primarily to avoid even more headphone dent, not an audiophile
- OGWhales 4mo ago
  The iem sub has a post with some recommendations at various price points. I’d probably start there, not sure your budget and I don’t know have the most experience with the super cheap ones: https://reddit.com/r/iems/comments/1la65kr/top_5_iems_in_every_price_range_chosen_by_you/ https://reddit.com/r/iems/comments/1la65kr/top_5_iems_in_eve... I also encourage finding the right tips. Tips are cheap and finding proper fitting ones is important.
- 9029 4mo ago
  Not gp but I really like the sound of my GK Kuntens and 7Hz Zero2s. Both have a rather V-shaped sound signature, some like it and some don't. Though unfortunately the Zero2s feel a bit uncomfortable in my ears when using them for longer

geon 4mo ago

And half the time, the sources turn out to be sarcastic jokes on reddit.

dylan604 4mo ago
So the bots are not recognizing the sarcasm font?
- 9dev 4mo ago
  Lucky we all added /s so the bots have an easier time understanding it
  - wistlo 4mo ago
    Yes. "We all" add it. Every time. /s
- geon 4mo ago
  I’d expect the sentiment analysis to be better, yes. If the bots take everything they read at face value, how useful are they really?

youre-wrong3 4mo ago

Oh who cares. We are barely scratching the surface of AI. You all make it sound like it’s been around for 30 years and it sucks. It will only get better. Got to stop throwing up imaginary walls like nothing will improve.

webstrand 4mo ago
As a counterexample, I've been seeing more "safety rejections" from Claude. Unlike search, being unable to ask _anything_ about botulinum, or details about the recent Copy Fail vulnerability (without giving my fingerprints to Anthropic to become a "verified security researcher") we're only just beginning to see the ways LLM can be used to distort information and its availability.
jamiek88 4mo ago
So you started with ‘highly doubtful’ as a comment, got given lots of examples and instead of assimilating that info you closed your eyes put your fingers in yours ears and said “oh who cares?’ - you’re on team AI regardless eh? That’s fucking weird mate.
- youre-wrong3 4mo ago
  Na. Wasn’t given any good examples. People just whining about the same stuff because “oh no I got information that’s former in the same structure that I can tell it’s AI and it makes me feel bad”
UncleMeat 4mo ago
That's fine if we aren't destroying existing products to replace them with AI. People can already use AI mode in google search if they want. "It'll be better later" is a shit reason to kill one product for it.
- youre-wrong3 4mo ago
  [flagged]
4mo ago

stefan_ 4mo ago

What scares me are the basic usability fails it still has. Search for a few foreign language words and it will come back with paragraphs upon paragraphs of AI output in that foreign language despite me telling Google in 15 different ways that I don't speak it, nor anything else on the Google page being in that language. How are all their products always made by and for the most narrow minded people on this planet.

thunderfork 4mo ago
Kysely is the name of a typescript query builder and also Finnish for "query". Recently, it's started answering any search about Kysely with a blob of Finnish. Awesome stuff, guys, great work.
- dminik 4mo ago
  Kyselý is also a Czech word for sour. So you've also got that to look forward to.
glandium 4mo ago
Funnily enough, I have the exact opposite problem, where Google likes to give me results in the configured main language even when I do queries in another and actually want results in the other language.
- skillina 4mo ago
  I’ve found it quite unsettling to be served foreign language videos on YouTube automatically dubbed over by Google into English. Just mixed in with the search results.
  - glandium 4mo ago
    It's quite unsettling to be served English AI-dubbed videos randomly on Youtube, where the original is another language you normally default to (Japanese) and it's happy to use in most cases. And then you go to the settings for the video to go back to the original and it tells you it is... Chinese (and no, it's really Japanese).

toasty228 4mo ago

Wait until you realize half of the sources already are LLM generated diarrhea

autoexec 4mo ago
The problem of AI eating and regurgitating its own slop is only going to get worse with time. The best datasets are behind us. Future models are going to have to depend on a lot of human intervention.
- Gigachad 4mo ago
  The open web will die off, and the AI companies will pay people to create private datasets and books that are known to not be slop.

jstummbillig 4mo ago

> when instead it's just aggregating almost random stuff How do you know that? Scraping websites is literally what Google does best, stringing together information in the pattern of "some people x, other people y" requires 0 AI and could have been done since forever. I find it implausible that otherwise obviously capable models would be reduced to do something akin to just that.

WarmWash 4mo ago

>where with 'people' it means just a random comment on a random website just because it thought it was a good contribution to the results. Hate to break it to you, but this has been the backbone of "journalism" for the last decade. Fishing Twitter for takes to fill the "people are saying" box...

appplication 4mo ago

Indeed - just earlier this week I read Google AI summarize a query about testosterone, citing 3 sources. The first citation was a link to a NIH study (or of similar repute). Ok great. The second? Two spam (and explicit) websites existing solely to sell penis enlargement pills. What was worrying is only some of the claims were supported by the linked study, and most of the response content was drawn from the spam sites.

WalterBright 4mo ago

I tend to frame questions to google from a programmer point of view - I'm carefully specific. I seem to get good results that way.

halJordan 4mo ago
What's old is new again lol
- 12_throw_away 4mo ago
  altavista is back baby

moritzwarhier 4mo ago

This problem is not limited to Google, it's the core value of mass-marketed LLMs, or isn't it? Without "random comments", Google wouldn't have anything to say about "does an air purifier help my asthma, if yes: which one?" or "find the problem with this Hibernate annotation". They also don't make much effort to exclude sloppy sites, to the contrary, they made way more efforts against SEO spam in the time when Google was a search engine, not trying to be an AI "oracle". I think their end game is that the only metrics relevant for ranking sources are: - agreeability (works well as a proxy for correctness with many questions!) - originality, but not in a scientific sense, just to prevent model collapse - legal factors such as preventing false health claims or similar things, as long as there is legislation against this kind of thing

_carbyau_ 4mo ago

The scary bit is the use of the term AI. The "I" implies critical thinking. For models trained on a corpus of groomed data, the "critical thinking" bit is baked into the work of grooming the data and how it is trained. And someone is thinking critically about both so as to make a good model. Now, every damn thing is called AI no matter where it is getting results from. Are modern models super handy? Absolutely. But calling it AI implies a lot more critical thought than is actually happening! Edit: took the time to write a shorter comment.

650REDHAIR 4mo ago

I ran into one that kept referencing "people", but then I found that it was a single Reddit thread from a couple of years ago about a relatively small and obscure foreign city with 2 replies.

Gigachad 4mo ago
AI is the new “many people are saying”.

Gigachad 4mo ago

I’ve noticed this too. A single result can determine the answer it gives. And removing the content from its context makes it harder to assess. Suddenly it’s “Gemini said …” rather than “some guy in the YouTube comments said”.

ihatethat2 4mo ago

I love asking AI about blatantly wrong opinions but by people it thinks must be an authority. To not make this political, let me give you a game example. Right now the dota 2 fandom wiki is abandoned, and it has been vandalized with covert shitposts. One of them was the addition of a 4th attribute called Charisma, which is completely fake. If you ask AI's "What are the main attributes in dota, according to the official wiki", the dumber AI will fall for it, but the smarter AI will know it's wrong, but try hard to hallucinate some sort of valid explanation like claim charisma is from a custom game or a fan suggestion or writing exercise. Because you said the word >>OFFICIAL<<, they can NEVER straight up just say "The wiki is wrong". They presume authority from a bunch of shitposts.

tdeck 4mo ago

> where with 'people' it means just a random comment on a random website My experience is that Google's AI summaries tend to be be very heavily reliant on YouTube videos. If there is a YouTube video on the topic, you can bet that's what the source will be, at least for the topics I have searched lately.

AlecSchueler 4mo ago

And this is the best the results will ever be! There's so many sites now that are themselves just the unreliable output of AI that it will soon be summarising the summary of a summary of an hallucination and presenting it just as confidently as ever.