10 ms·

The particularly worrying thing here is that they're now going to be gathering training data for a conversational model on _how to influence people effectively

by svieira 4mo ago

The particularly worrying thing here is that they're now going to be gathering training data for a conversational model on _how to influence people effectively even when they already know they are being influenced_. Even more than RLFH already does. "We had to build the Torment Nexus so our children could eat" is not a good reason to build the Torment Nexus. The fact that they are not committing to not doing this tells me that either no one thought about what else this could be used for, or the short term gains are all they are thinking about.

cyanydeez 4mo ago
the rationale is theyre a business and selling is valuable. acting like theyre naively building influence peddling is how we got here. theyre now "dont not be evil"
thrance 4mo ago
I'm really worried about political ads. Google had no problem algorithmically bubbling up disinformation in the past, will Gemini now peddle straight lies for anyone willing to pay enough for them?
- Henchman21 4mo ago
  Obviously, yes.
- hightrix 4mo ago
  "You're absolutely right. The trump administration has caused the lowest gas prices in US history and lowered drug costs by 1300%. Click here to donate to your nearest MAGA politician." 2026 may not see it too bad, but 2028 will be an absolutely nasty election cycle.
deleted 4mo ago
[deleted]
globalnode 4mo ago
Its not worrying for me. This is what I expect from ad companies so I walk in prepared. What I'm more interested in is how we are going to take this new technology and give it back to people without trying to rip into them, for free. We need a Stallman / Linus of AI.
janalsncm 4mo ago
While this is true, it is less effective than you might think in practice. Researchers studied something similar six months ago: https://arstechnica.com/science/2025/12/researchers-find-what-makes-ai-chatbots-politically-persuasive/ https://arstechnica.com/science/2025/12/researchers-find-wha... Of course it is possible Google will create something orders of magnitude better but I doubt it. Amazon is already doing product recommendation on their front page and while it certainly drives sales it doesn’t turn people into zombies buying products they don’t want. What did work really well was the “1 click buy” button. Reducing the friction for people who already want to buy is usually a much higher ROI than persuading people who previously weren’t interested.
- majormajor 4mo ago
  My favorite example of why advertising/marketing is important but not overwhelming/unstoppable is in the hit-based industries: If the industry had a "make a superstar" button or a "make a blockbuster movie" button they could just keep mashing, they'd be mashing it constantly. You wouldn't see franchises go from top-of-the-world to decline like Marvel or Fast and Furious. You wouldn't see expensive new bombs or failure-to-launch reboots. There is countless chatter out there about industry plants, organic vs PR-shilling word of mount campaigns, etc. But... if that's all it took, it would be wayyyyy more constant. A lot of words spilled recently about a band, Geese, and how their buzz wasn't as organic as it seemed... for a band that's still, at the end of the day, quite niche and small-in-audience... It's very hard to get a huge hit without marketing - even great word of mouth benefits from amplification - but it's also near-impossible to force a hit into an audience that isn't vibing with it. The highest-grossing movies, or highest-listened pop music, is a combo of marketing + accurately hitting extremely-common/trendy tastes. See also iPhone marketing vs Windows Phone marketing. I thought Windows Phone was better; none of my friends or coworkers was convinced even after I showed them my phones. The mass media consumer may not have thought much about their tastes or tried very hard to be more adventurous, but that doesn't mean they don't like the stuff they're eating. I think this is still in many ways bad - at the very least, it's incredibly inefficient to have a billion-dollar zero-sum "pick this one over that one" industry. But I don't think it's a deadly threat. (See also any number of "tons of money and star-power behind them" failed political candidates too...)
jgalt212 4mo ago
Parenting advice updated for the AI age. If all of your chatbots jumped off a bridge, would you do it too?
- lucketone 4mo ago
  Yes, because in all likelihood an artificial train is coming
utopiah 4mo ago
Yes... but that's Google from the moment they started monetizing. When it was a research project at Stanford in 1996 it was genuinely about making the World information accessible. The day in 2000 when they decided to sell advertisement is exactly when this process started, not just now in 2026. Since that day they transformed from a knowledge management project to an advertising company that used knowledge management, among other tools, to influence people effectively. It's the same for Meta except they had Google business model as an example. Those are advertising company that use tech, not real tech companies like e.g. ASML or IBM.
- mxmilkiib 4mo ago
  something something about liberalism always ending up siding with economic advantage over social justice/liberation

by svieira 4mo ago

cyanydeez 4mo ago

the rationale is theyre a business and selling is valuable. acting like theyre naively building influence peddling is how we got here. theyre now "dont not be evil"

thrance 4mo ago

I'm really worried about political ads. Google had no problem algorithmically bubbling up disinformation in the past, will Gemini now peddle straight lies for anyone willing to pay enough for them?

Henchman21 4mo ago
Obviously, yes.
hightrix 4mo ago
"You're absolutely right. The trump administration has caused the lowest gas prices in US history and lowered drug costs by 1300%. Click here to donate to your nearest MAGA politician." 2026 may not see it too bad, but 2028 will be an absolutely nasty election cycle.

deleted 4mo ago

[deleted]

globalnode 4mo ago

Its not worrying for me. This is what I expect from ad companies so I walk in prepared. What I'm more interested in is how we are going to take this new technology and give it back to people without trying to rip into them, for free. We need a Stallman / Linus of AI.

janalsncm 4mo ago

While this is true, it is less effective than you might think in practice. Researchers studied something similar six months ago: https://arstechnica.com/science/2025/12/researchers-find-what-makes-ai-chatbots-politically-persuasive/ https://arstechnica.com/science/2025/12/researchers-find-wha... Of course it is possible Google will create something orders of magnitude better but I doubt it. Amazon is already doing product recommendation on their front page and while it certainly drives sales it doesn’t turn people into zombies buying products they don’t want. What did work really well was the “1 click buy” button. Reducing the friction for people who already want to buy is usually a much higher ROI than persuading people who previously weren’t interested.

majormajor 4mo ago
My favorite example of why advertising/marketing is important but not overwhelming/unstoppable is in the hit-based industries: If the industry had a "make a superstar" button or a "make a blockbuster movie" button they could just keep mashing, they'd be mashing it constantly. You wouldn't see franchises go from top-of-the-world to decline like Marvel or Fast and Furious. You wouldn't see expensive new bombs or failure-to-launch reboots. There is countless chatter out there about industry plants, organic vs PR-shilling word of mount campaigns, etc. But... if that's all it took, it would be wayyyyy more constant. A lot of words spilled recently about a band, Geese, and how their buzz wasn't as organic as it seemed... for a band that's still, at the end of the day, quite niche and small-in-audience... It's very hard to get a huge hit without marketing - even great word of mouth benefits from amplification - but it's also near-impossible to force a hit into an audience that isn't vibing with it. The highest-grossing movies, or highest-listened pop music, is a combo of marketing + accurately hitting extremely-common/trendy tastes. See also iPhone marketing vs Windows Phone marketing. I thought Windows Phone was better; none of my friends or coworkers was convinced even after I showed them my phones. The mass media consumer may not have thought much about their tastes or tried very hard to be more adventurous, but that doesn't mean they don't like the stuff they're eating. I think this is still in many ways bad - at the very least, it's incredibly inefficient to have a billion-dollar zero-sum "pick this one over that one" industry. But I don't think it's a deadly threat. (See also any number of "tons of money and star-power behind them" failed political candidates too...)

jgalt212 4mo ago

Parenting advice updated for the AI age. If all of your chatbots jumped off a bridge, would you do it too?

lucketone 4mo ago
Yes, because in all likelihood an artificial train is coming

utopiah 4mo ago

Yes... but that's Google from the moment they started monetizing. When it was a research project at Stanford in 1996 it was genuinely about making the World information accessible. The day in 2000 when they decided to sell advertisement is exactly when this process started, not just now in 2026. Since that day they transformed from a knowledge management project to an advertising company that used knowledge management, among other tools, to influence people effectively. It's the same for Meta except they had Google business model as an example. Those are advertising company that use tech, not real tech companies like e.g. ASML or IBM.

mxmilkiib 4mo ago
something something about liberalism always ending up siding with economic advantage over social justice/liberation