7 ms·

PhysicsForums and the Dead Internet Theory

264 points by TheCog 2y ago

deleted 2y ago
[deleted]
Terr_ 2y ago
Ooof. The idea--or reality--that humans' accounts would be hijacked by site-owners to make impersonating slop (presumably to bring in ad-revenue) is somehow both infuriating and energy-sapping-depressing. Issues of trust and attribution have always existed for the web, but for many reasons it feels so much worse now--how bad must it get before some kind of sea-change can occur? I'm not sure what the solution would be here. * Does one need to establish a friggin' trademark for their own name/handle [0], just so they can threaten to sue using money they probably don't have? * Is it finally time for PKI and everybody signs their posts with private keys and wastes CPU cycles verifying signatures to check for impersonation? * Is there some set of implied collective expectations which need to be captured and formalized into the global patchworks of law? [0] Ex: By establishing a small but plausible "business" selling advice and opinions under that name, and going after the impersonator for harming that brand.
- UltraSane 2y ago
  I exchange public keys with close friends in person. A large scale solution would be very Orwellian. You would need a national ID that is a smart card to connect to an ISP and possible biometric verification.
  - hooverd 2y ago
    Do you exchange public keys with your non-computer-toucher close friends?
    - arccy 2y ago
      if you convince them to use signal that's close enough...
  - afpx 2y ago
    Could I buy a physical device like RSA SecurID from my bank branch or post office and log into a closed VPN-like network where all the servers are run by verified users? I know there are problems with that idea.
  - bawolff 2y ago
    We already have e-passports and zero knowledge proofs to show you have one without revealing who you are. If all else fails, there is always the web of trust (i think web of trust has a lot of issues, but establishing soneone is human seems like a much lower bar than establishing identity)
    - UltraSane 2y ago
      I hadn't though of zero-knowledge proofs. That is an interesting idea.
      - bawolff 2y ago
        The blockchain people have been experimenting with it https://ethresear.ch/t/zero-knowledge-proofs-of-identity-using-electronic-passports/19263 https://ethresear.ch/t/zero-knowledge-proofs-of-identity-usi... (im no fan of blockchain but i assume you can reuse the ideas without the blockchain part)
    - UltraSane 2y ago
      Web of trust would be interesting if phones could automatically trust other phones they spend enough time nearby.
- m463 2y ago
  It is sad. I have been putting a copyright notice on my resume at the bottom to prevent some nonsense. I have always wondered if people could attach some sort of cryptographic marker to their posts, that could link to an archive somewhere. Mostly I was thinking of backups of posts to yelp that couldn't be taken down, but I wonder if it would work that posts someone never made.
  - Terr_ 2y ago
    > I have been putting a copyright notice on my resume at the bottom to prevent some nonsense. I expect the bad-actors will feed it into an LLM and say: "Rephrase this slightly", and they will get away with it because the big-money hucksters will have already convinced courts to declare it transformative or fair-use.
- scotty79 2y ago
  Shouldn't we invent a protocol that keeps the content you produce under your control so that places like forums or facebook are only discovery devices and interaction facilitators, but not custodians of all communication? Being able to independantly reach the source of piece of information is increasingly important.
  - hooverd 2y ago
    That's what ATProto is trying to solve, funny enough.
- hooverd 2y ago
  Don't sign your posts!
  - Terr_ 2y ago
    Are you saying nothing should be key-signed because you want some kind of deniability later? Or do you mean people should avoid using an pseudonym in favor of posts that are anonymous, so that there's never any created identity to exploit/defend?
    - hooverd 2y ago
      -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Sorry, it was a bad joke, there's a phrase "don't sign your posts" used when someone ends one with an insult. I support signing your posts with digital signatures if you want. -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQC37hdRRO1LtrTQY8AXxvbqjG5KgUCZ5QRXwAKCRAAXxvbqjG5 Kth4AQCccNygglcSyEiMAqQyw6cXH54fnqBT9rJO9TSIqH14rgEAyUwxiQlV05XV Du2ftMk3DwiUZLKDxVI+ODCn4osf2wM= =XZhX -----END PGP SIGNATURE-----
- bee_rider 2y ago
  Impersonating somebody to make it look like they said something they didn’t really ought to be considered defamation or something. Also there’s something really uncomfortable about the phrasing of a lot of those answers. I mean, even as somebody with an engineering degree, I try not to ever answer a question “as a <field> engineer” because when screwing around online I haven’t done the correct amount of analysis to provide answers “as an engineer” ethically (acknowledging the irony of using the phrase here, but, clearly this is not a technical statement so I think it is fine). The bot doesn’t seem to have this compunction. This ravenprp guy was an engineering student a couple years ago. I guess it’s less of a thing because he wasn’t commenting under his real name. But it seems like this site, given the type of content it hosts, could easily end up impersonating somebody “as an engineer” in the field they work and have a professional reputation in. And the site even has a historical record of them asking and answering questions through their education, so it does a really good job of misleading people into thinking an engineer is answering their questions. I know the idea of an individual professional reputation has taken a beating in the modern hyper-corporate world. But the more I think of it, the more I think… this seems incredibly shitty and actually borderline dangerous, right?
Rodeoclash 2y ago
The ShackNews forum: https://www.shacknews.com/chatty https://www.shacknews.com/chatty was similar - go back in time on on it and you can find posts about 9/11 unfolding.
- tomrod 2y ago
  Ars Technica started with comms forums + this new idea to report tech news. The forums are still there but not nearly the camaraderie of the early days.
  - r58lf 2y ago
    Same with siliconinvestor.com It was an early stock discussion forum. It grew rapidly when search engines started indexing everything and this forum had a URL for each message that was easily indexable. It's still around, but nothing like the old days.
    - 0xDEAFBEAD 2y ago
      I find these old school forums fascinating. How does that even work, to have a thread of 192,211 posts about Qualcomm? https://www.siliconinvestor.com/subject.aspx?subjectid=36035 https://www.siliconinvestor.com/subject.aspx?subjectid=36035 Suppose the average post is about 1 paragraph long. One paragraph is about 150 words. So 192211 * 150 = about 29 million words. For comparison, the Lord of the Rings trilogy is only around half a million words. It wouldn't surprise me if there are more words about Qualcomm in that thread than the total amount of internal and external documentation and financial guidance that Qualcomm itself has ever produced. Surely users aren't expected to read the entire thread before adding a post? But I think I remember seeing old forums where that basically is the expectation. And honestly... that's pretty cool. It seems better than the new social media, where we keep having low-effort recurring debates. I like the idea of adding to an enormous pile of scholarship in cyberspace. A Ship of Theseus discussion which may outlive any individual participant, but has a semblance of continuity all the same, like an undergraduate college society with a 100+ year history. Time for a cyberpunk revival. Retro-cyberpunk, we could call it.
EA-3167 2y ago
I don't quite understand the issue of "back-dating" or hijacking accounts. How is this being done exactly? I came away from this article wondering if I was missing something.
- roywiggins 2y ago
  Whoever runs the site/database is just inserting rows with fake datestamps under existing (presumably abandoned) account names.
  - aaron_m04 2y ago
    How could anyone possibly think it'd be OK to impersonate real humans?
    - jeremyjh 2y ago
      They don't give a fuck if its "ok". They are just trying to scrape up some additional ad revenue, like 99% of the rest of the internet.
      - simplicio 2y ago
        I don't really get the revenue angle though. The AI posts don't seem to be trying to drive traffic to ads or anything. I really don't understand the point of auto-generating a bunch of AI gibberish under the name of old users on ones own site?
        roywiggins 2y ago
        A misguided attempt at SEO?
- Evidlo 2y ago
  The last section mentions that the PhysicsForums admins are experimenting with LLM-generated responses, so I think the site owners are responsible. > We reached out to Greg Bernhardt asking for comment on LLM usage in PhysicsForums, and he replied: > "We have many AI tests in the works to add value to the community. I sent out a 2024 feedback form to members a few weeks ago and many members don’t want AI features. We’ll either work with members to dramatically improve them or end up removing them. We experimented with AI answers from test accounts last year but they were not meeting quality standards. If you find any test accounts we missed, please let me know. My documentation was not the best." Why they would recycle old human accounts as AI "test accounts", I have no idea.
deleted 2y ago
[deleted]
segasaturn 2y ago
Money quote: > There’s also a social contract: when we create an account in an online community, we do it with the expectation that people we are going to interact with are primarily people. Oh, there will be shills, and bots, and advertisers, but the agreement between the users and the community provider is that they are going to try to defend us from that, and that in exchange we will provide our engagement and content. This is why the recent experiments from Meta with AI generated users are both ridiculous and sickening. When you might be interacting with something masquerading as a human, providing at best, tepid garbage, the value of human interaction via the internet is lost. It is a disaster. I have no idea how to solve this issue, I can't see a future where artificially generated slop doesn't eventually overwhelm every part of the internet and make it unusable. The UGC era of the internet is probably over.
- lumost 2y ago
  I suspect that the honest outcome will be that platforms where AI content is allowed/encouraged will begin to appear like a video game. If everyone in school is ai-media famous - then no one is. There is most assuredly a market for a game where you are immediately influencer famous, but it's certainly much smaller than the market for social media.
- thatguy0900 2y ago
  Invite only forums or forums with actual identity checking of some sort. Google and Facebook are in prime position to actually provide real online identity services to other websites, which makes Facebook itself developing bots even funnier. Maybe we'll eventually get bank/government issued online identity verification.
  - segasaturn 2y ago
    Online identity verification is the obvious solution, the only problem is that we would lose the last bits of privacy we have on the internet. I guess if everyone was forced to post under our real name and identity, we might treat each other with better etiquette, but...
Scoundreller 2y ago
> It had fairly steady growth until 2012, before petering out throughout the 2010s and 2020s in lieu of more centralized sites like StackExchange, and by 2025, only a small community was left This timeline tracks with my own blogging. Google slowly stopped ranking traditional forum posts and blogs as well around that time, regardless of quality, unless it was a “major”. > But, unlike so many other fora from back in the early days, it went from 2003 to 2025 without ever changing its URLs, erasing its old posts, or going down altogether. I can also confirm if you have a bookmark to my blog from 2008, that link will still work! The CMS is no longer, it's all static now... which too few orgs take the short amount of time to bother with when "refreshing" their web presence :(
- Lammy 2y ago
  > Google slowly stopped ranking traditional forum posts and blogs as well around that time IMO the true inflection point was 2014 when Google first hid (from the UI) and then fully removed (no longer accessible by magic URL) the “Blogs” and especially the “Discussions” filters. Some contemporary discussions on “Discussions”: - https://techcrunch.com/2014/01/23/googles-search-filters-now-update-dynamically-based-on-your-queries/ https://techcrunch.com/2014/01/23/googles-search-filters-now... - http://googlesystem.blogspot.com/2014/03/bring-back-forum-search-recipe-search.html http://googlesystem.blogspot.com/2014/03/bring-back-forum-se... (details the briefly-working magic URLs) - https://www.ghacks.net/2014/01/23/search-discussions-blogs-places-google-now/ https://www.ghacks.net/2014/01/23/search-discussions-blogs-p... - https://www.seroundtable.com/google-search-filters-gone-17993.html https://www.seroundtable.com/google-search-filters-gone-1799... - https://www.webmasterworld.com/google/4687960.htm https://www.webmasterworld.com/google/4687960.htm - https://www.thecoli.com/threads/i-cant-google-search-by-discussions-anymore.185088/ https://www.thecoli.com/threads/i-cant-google-search-by-disc... - https://www.neogaf.com/threads/anyone-else-annoyed-google-removed-the-search-by-discussions-filter.775463/ https://www.neogaf.com/threads/anyone-else-annoyed-google-re... - https://webapps.stackexchange.com/questions/57249/has-the-option-of-search-in-discussions-been-removed-from-google-search https://webapps.stackexchange.com/questions/57249/has-the-op... - https://www.bladeforums.com/threads/how-to-do-google-discussion-search-now-that-it-has-been-disabled.1207584/ https://www.bladeforums.com/threads/how-to-do-google-discuss... - https://browsermedia.agency/blog/alternatives-discussion-search-filter/ https://browsermedia.agency/blog/alternatives-discussion-sea...
COAGULOPATH 2y ago
Something I'm increasingly noticing about LLM-generated content is that...nobody wants it. (I mean "nobody" in the sense of "nobody likes Nickelback". ie, not literally nobody.) If I want to talk to an AI, I can talk to an AI. If I'm reading a blog or a discussion forum, it's because I want to see writing by humans. I don't want to read a wall of copy+pasted LLM slop posted under a human's name. I now spend dismaying amounts of time and energy avoiding LLM content on the web. When I read an article, I study the writing style, and if I detect ChatGPTese ("As we dive into the ever-evolving realm of...") I hit the back button. When I search for images, I use a wall of negative filters (-AI, -Midjourney, -StableDiffusion etc) to remove slop (which would otherwise be >50% of my results for some searches). Sometimes I filter searches to before 2022. If Google added a global "remove generative content" filter that worked, I would click it and then never unclick it. I don't think I'm alone. There has been research suggesting that users immediately dislike content they perceive as AI-created, regardless of its quality. This creates an incentive for publishers to "humanwash" AI-written content—to construct a fiction where a human is writing the LLM slop you're reading. Falsifying timestamps and hijacking old accounts to do this is definitely something I haven't seen before.
- robswc 2y ago
  100%. So far (thankfully) I've noticed this stuff get voted down on social media but it is blowing my mind people think pasting in a ChatGPT response is productive. I've seen people on reddit say stuff like "I don't know but here's what ChatGPT said." Or worse, presenting ChatGPT copy-paste as their own. Its funny because you can tell, the text reads like an HR person wrote it.
  - Trasmatta 2y ago
    I've noticed the opposite actually, clearly ChatGPT written posts on Reddit that get a ton of upvotes. I'm especially noticing it on niche subreddits. The ones that make me furious are on some of the mental health subreddits. People are asking for genuine support from other people, but are getting AI slop instead. If someone needs support from an AI (which I've found can actually help), they can go use it themselves.
elashri 2y ago
It is sad that this is happening to PhysicsForums. It was one of first websites I was using frequently 15 years ago when I started my physics passion (later career). I was active reader and contributed on few occasions but I still remember some members who I thought that one day I will be smart and knowledgeable like them. With years and the move to social media following Arab spring things started to change (as part of the overall transition from forum being the dominant place for discussions). But I stopped visiting it around 2018 unless I came through google search (later kagi). I still find the archive useful to answer some questions and I would disagree with author of article that because no one is sharing links on twitter that means no one care.
LordShredda 2y ago
Don't give out your real name online, the server admin might change your posts.
- eddd-ddde 2y ago
  They can also just, set a random account under your name. So no difference.
paulpauper 2y ago
Dead internet theory is one of those ideas that keeps resurfacing or being revied with articles like this, even though the evidence is only limited to confirmation bias. It ignores that there are huge parts of the internet that are not dead. I think it's more like the quality of discourse has fallen for reasons that are not clear.
- NotYourLawyer 2y ago
  > there are huge parts of the internet that are not dead Such as?
- datadrivenangel 2y ago
  The article looked at the PhysicsForums and found that 92% of the text is AI or machine generated...
  - paulpauper 2y ago
    The internet is way bigger than PhysicsForums. That was my point, but your response seems to confirm what I said about discourse declining though.
inasio 2y ago
Talk about burying the lede! Near the bottom of the story the site owner confirms that it was him that added the backdated AI comments (perhaps it should have been obvious...)
- firesteelrain 2y ago
  I couldn’t find it. He was trying to seed the site ?
  - pbronez 2y ago
    Experimenting with using AI bots to respond to questions that had been open for a long time with no response.
  - inasio 2y ago
    This is what he replied when asked about the backdated comments: > The backdated answers were an internal test. We conceived of a bot that would provide a quality answer to a thread without a reply after 1+ years. That too also failed. Instead, I’m considering pruning all threads without a reply as they clutter up the forums.
    - MichaelZuo 2y ago
      He’s an SEO guy at shopify, so he likely doesn’t care about the consequences of doing shady SEO experiments with old user’s accounts…
    - naijaboiler 2y ago
      Thats some really shady stuff. Its one thing to provide answers with AI bot. Its totally disingenuous to 1) backdate it 2) assume the alias of a real person
- advisedwang 2y ago
  The investigation of the events on this particular website are just a tool to illustrate a much broader point about internet content, identity and LLMs.
majgr 2y ago
It is interesting, that this forum might be „AI poisoned” for other AI bots, because training AI on content generated by AI = garbage.
Slava_Propanei 2y ago
[dead]
Slava_Propanei 2y ago
[dead]
econ 2y ago
I like the assumption that it was a real account originally. It all seems so unthinkable but when running a forum or a blog with an active comment section.. what would you do/think if your users show up, browse around and not say anything for a week? You start out by making topics in your own name, write helpful replies.. until you look like an idiot talking to yourself. Forums with good traffic and lots of spammy advertisement no doubt consider it when visitors leave because nothing new happened. I once upon a time, on a rather stale forum, created two similarly named accounts from the same ip and argued with myself. At first I thought the owner or one of the other users would notice but I quickly learned that no behaviour is weird enough for it to be ever considered.
- BlueTemplar 2y ago
  I think I would rather post alone than my current experience (on 2 forums already) of other posters being overwhelmingly spam bots.
  - econ 2y ago
    A blog is more suitable for that. You could mirror the posts on a forum and look sophisticated.
secretsatan 2y ago
I suspect a manager i work with has started using llms, i’ve sat next to him long enough before he went manager to know he’s incompetent, and now in chat, suddenly he spouts out of character plausible, but off for him explanations. I work in a company where for most english is not their first language so i’m not sure if anyone else has picked up on it.
hintymad 2y ago
A trajectory observation: People from the US and maybe Europe seem love having all kinds of niche forums, like PhysicsForum. On the contrary, Chinese people seemed love having a centralized place. Case in point, Zhihu(zhihu.com) was a clone of Quora, yet now Zhihu is the largest site for finding deep discussion of practically everything, maths, machine learning, history, physics, engineering, general science, just name a few. Tons of researchers and experts and hobbyists are there to share their insights. In contrasts, the quality of Quora seems have been deteriorating over the years, and most of the experts are in all kinds of niche forums anyway. I wonder why there is such a difference.
upghost 2y ago
I'm also noticing there's a dead economy theory or at least dead job market theory. AI sending resumes, AI reading/rejecting resumes. Humans wanting to talk to real humans, but all we get is slop. If we ever develop AGI I guess we'll have to go back to completely offline just to talk to real humans.

264 points by TheCog 2y ago

deleted 2y ago

[deleted]

Ooof. The idea--or reality--that humans' accounts would be hijacked by site-owners to make impersonating slop (presumably to bring in ad-revenue) is somehow both infuriating and energy-sapping-depressing. Issues of trust and attribution have always existed for the web, but for many reasons it feels so much worse now--how bad must it get before some kind of sea-change can occur? I'm not sure what the solution would be here. * Does one need to establish a friggin' trademark for their own name/handle [0], just so they can threaten to sue using money they probably don't have? * Is it finally time for PKI and everybody signs their posts with private keys and wastes CPU cycles verifying signatures to check for impersonation? * Is there some set of implied collective expectations which need to be captured and formalized into the global patchworks of law? [0] Ex: By establishing a small but plausible "business" selling advice and opinions under that name, and going after the impersonator for harming that brand.

UltraSane 2y ago
I exchange public keys with close friends in person. A large scale solution would be very Orwellian. You would need a national ID that is a smart card to connect to an ISP and possible biometric verification.
- hooverd 2y ago
  Do you exchange public keys with your non-computer-toucher close friends?
  - arccy 2y ago
    if you convince them to use signal that's close enough...
- afpx 2y ago
  Could I buy a physical device like RSA SecurID from my bank branch or post office and log into a closed VPN-like network where all the servers are run by verified users? I know there are problems with that idea.
- bawolff 2y ago
  We already have e-passports and zero knowledge proofs to show you have one without revealing who you are. If all else fails, there is always the web of trust (i think web of trust has a lot of issues, but establishing soneone is human seems like a much lower bar than establishing identity)
  - UltraSane 2y ago
    I hadn't though of zero-knowledge proofs. That is an interesting idea.
    - bawolff 2y ago
      The blockchain people have been experimenting with it https://ethresear.ch/t/zero-knowledge-proofs-of-identity-using-electronic-passports/19263 https://ethresear.ch/t/zero-knowledge-proofs-of-identity-usi... (im no fan of blockchain but i assume you can reuse the ideas without the blockchain part)
  - UltraSane 2y ago
    Web of trust would be interesting if phones could automatically trust other phones they spend enough time nearby.
m463 2y ago
It is sad. I have been putting a copyright notice on my resume at the bottom to prevent some nonsense. I have always wondered if people could attach some sort of cryptographic marker to their posts, that could link to an archive somewhere. Mostly I was thinking of backups of posts to yelp that couldn't be taken down, but I wonder if it would work that posts someone never made.
- Terr_ 2y ago
  > I have been putting a copyright notice on my resume at the bottom to prevent some nonsense. I expect the bad-actors will feed it into an LLM and say: "Rephrase this slightly", and they will get away with it because the big-money hucksters will have already convinced courts to declare it transformative or fair-use.
scotty79 2y ago
Shouldn't we invent a protocol that keeps the content you produce under your control so that places like forums or facebook are only discovery devices and interaction facilitators, but not custodians of all communication? Being able to independantly reach the source of piece of information is increasingly important.
- hooverd 2y ago
  That's what ATProto is trying to solve, funny enough.
hooverd 2y ago
Don't sign your posts!
- Terr_ 2y ago
  Are you saying nothing should be key-signed because you want some kind of deniability later? Or do you mean people should avoid using an pseudonym in favor of posts that are anonymous, so that there's never any created identity to exploit/defend?
  - hooverd 2y ago
    -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Sorry, it was a bad joke, there's a phrase "don't sign your posts" used when someone ends one with an insult. I support signing your posts with digital signatures if you want. -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQC37hdRRO1LtrTQY8AXxvbqjG5KgUCZ5QRXwAKCRAAXxvbqjG5 Kth4AQCccNygglcSyEiMAqQyw6cXH54fnqBT9rJO9TSIqH14rgEAyUwxiQlV05XV Du2ftMk3DwiUZLKDxVI+ODCn4osf2wM= =XZhX -----END PGP SIGNATURE-----
bee_rider 2y ago
Impersonating somebody to make it look like they said something they didn’t really ought to be considered defamation or something. Also there’s something really uncomfortable about the phrasing of a lot of those answers. I mean, even as somebody with an engineering degree, I try not to ever answer a question “as a <field> engineer” because when screwing around online I haven’t done the correct amount of analysis to provide answers “as an engineer” ethically (acknowledging the irony of using the phrase here, but, clearly this is not a technical statement so I think it is fine). The bot doesn’t seem to have this compunction. This ravenprp guy was an engineering student a couple years ago. I guess it’s less of a thing because he wasn’t commenting under his real name. But it seems like this site, given the type of content it hosts, could easily end up impersonating somebody “as an engineer” in the field they work and have a professional reputation in. And the site even has a historical record of them asking and answering questions through their education, so it does a really good job of misleading people into thinking an engineer is answering their questions. I know the idea of an individual professional reputation has taken a beating in the modern hyper-corporate world. But the more I think of it, the more I think… this seems incredibly shitty and actually borderline dangerous, right?

Rodeoclash 2y ago

The ShackNews forum: https://www.shacknews.com/chatty https://www.shacknews.com/chatty was similar - go back in time on on it and you can find posts about 9/11 unfolding.

tomrod 2y ago
Ars Technica started with comms forums + this new idea to report tech news. The forums are still there but not nearly the camaraderie of the early days.
- r58lf 2y ago
  Same with siliconinvestor.com It was an early stock discussion forum. It grew rapidly when search engines started indexing everything and this forum had a URL for each message that was easily indexable. It's still around, but nothing like the old days.
  - 0xDEAFBEAD 2y ago
    I find these old school forums fascinating. How does that even work, to have a thread of 192,211 posts about Qualcomm? https://www.siliconinvestor.com/subject.aspx?subjectid=36035 https://www.siliconinvestor.com/subject.aspx?subjectid=36035 Suppose the average post is about 1 paragraph long. One paragraph is about 150 words. So 192211 * 150 = about 29 million words. For comparison, the Lord of the Rings trilogy is only around half a million words. It wouldn't surprise me if there are more words about Qualcomm in that thread than the total amount of internal and external documentation and financial guidance that Qualcomm itself has ever produced. Surely users aren't expected to read the entire thread before adding a post? But I think I remember seeing old forums where that basically is the expectation. And honestly... that's pretty cool. It seems better than the new social media, where we keep having low-effort recurring debates. I like the idea of adding to an enormous pile of scholarship in cyberspace. A Ship of Theseus discussion which may outlive any individual participant, but has a semblance of continuity all the same, like an undergraduate college society with a 100+ year history. Time for a cyberpunk revival. Retro-cyberpunk, we could call it.

EA-3167 2y ago

I don't quite understand the issue of "back-dating" or hijacking accounts. How is this being done exactly? I came away from this article wondering if I was missing something.

roywiggins 2y ago
Whoever runs the site/database is just inserting rows with fake datestamps under existing (presumably abandoned) account names.
- aaron_m04 2y ago
  How could anyone possibly think it'd be OK to impersonate real humans?
  - jeremyjh 2y ago
    They don't give a fuck if its "ok". They are just trying to scrape up some additional ad revenue, like 99% of the rest of the internet.
    - simplicio 2y ago
      I don't really get the revenue angle though. The AI posts don't seem to be trying to drive traffic to ads or anything. I really don't understand the point of auto-generating a bunch of AI gibberish under the name of old users on ones own site?
      - roywiggins 2y ago
        A misguided attempt at SEO?
Evidlo 2y ago
The last section mentions that the PhysicsForums admins are experimenting with LLM-generated responses, so I think the site owners are responsible. > We reached out to Greg Bernhardt asking for comment on LLM usage in PhysicsForums, and he replied: > "We have many AI tests in the works to add value to the community. I sent out a 2024 feedback form to members a few weeks ago and many members don’t want AI features. We’ll either work with members to dramatically improve them or end up removing them. We experimented with AI answers from test accounts last year but they were not meeting quality standards. If you find any test accounts we missed, please let me know. My documentation was not the best." Why they would recycle old human accounts as AI "test accounts", I have no idea.

deleted 2y ago

[deleted]

segasaturn 2y ago

Money quote: > There’s also a social contract: when we create an account in an online community, we do it with the expectation that people we are going to interact with are primarily people. Oh, there will be shills, and bots, and advertisers, but the agreement between the users and the community provider is that they are going to try to defend us from that, and that in exchange we will provide our engagement and content. This is why the recent experiments from Meta with AI generated users are both ridiculous and sickening. When you might be interacting with something masquerading as a human, providing at best, tepid garbage, the value of human interaction via the internet is lost. It is a disaster. I have no idea how to solve this issue, I can't see a future where artificially generated slop doesn't eventually overwhelm every part of the internet and make it unusable. The UGC era of the internet is probably over.

lumost 2y ago
I suspect that the honest outcome will be that platforms where AI content is allowed/encouraged will begin to appear like a video game. If everyone in school is ai-media famous - then no one is. There is most assuredly a market for a game where you are immediately influencer famous, but it's certainly much smaller than the market for social media.
thatguy0900 2y ago
Invite only forums or forums with actual identity checking of some sort. Google and Facebook are in prime position to actually provide real online identity services to other websites, which makes Facebook itself developing bots even funnier. Maybe we'll eventually get bank/government issued online identity verification.
- segasaturn 2y ago
  Online identity verification is the obvious solution, the only problem is that we would lose the last bits of privacy we have on the internet. I guess if everyone was forced to post under our real name and identity, we might treat each other with better etiquette, but...

Scoundreller 2y ago

> It had fairly steady growth until 2012, before petering out throughout the 2010s and 2020s in lieu of more centralized sites like StackExchange, and by 2025, only a small community was left This timeline tracks with my own blogging. Google slowly stopped ranking traditional forum posts and blogs as well around that time, regardless of quality, unless it was a “major”. > But, unlike so many other fora from back in the early days, it went from 2003 to 2025 without ever changing its URLs, erasing its old posts, or going down altogether. I can also confirm if you have a bookmark to my blog from 2008, that link will still work! The CMS is no longer, it's all static now... which too few orgs take the short amount of time to bother with when "refreshing" their web presence :(

Lammy 2y ago
> Google slowly stopped ranking traditional forum posts and blogs as well around that time IMO the true inflection point was 2014 when Google first hid (from the UI) and then fully removed (no longer accessible by magic URL) the “Blogs” and especially the “Discussions” filters. Some contemporary discussions on “Discussions”: - https://techcrunch.com/2014/01/23/googles-search-filters-now-update-dynamically-based-on-your-queries/ https://techcrunch.com/2014/01/23/googles-search-filters-now... - http://googlesystem.blogspot.com/2014/03/bring-back-forum-search-recipe-search.html http://googlesystem.blogspot.com/2014/03/bring-back-forum-se... (details the briefly-working magic URLs) - https://www.ghacks.net/2014/01/23/search-discussions-blogs-places-google-now/ https://www.ghacks.net/2014/01/23/search-discussions-blogs-p... - https://www.seroundtable.com/google-search-filters-gone-17993.html https://www.seroundtable.com/google-search-filters-gone-1799... - https://www.webmasterworld.com/google/4687960.htm https://www.webmasterworld.com/google/4687960.htm - https://www.thecoli.com/threads/i-cant-google-search-by-discussions-anymore.185088/ https://www.thecoli.com/threads/i-cant-google-search-by-disc... - https://www.neogaf.com/threads/anyone-else-annoyed-google-removed-the-search-by-discussions-filter.775463/ https://www.neogaf.com/threads/anyone-else-annoyed-google-re... - https://webapps.stackexchange.com/questions/57249/has-the-option-of-search-in-discussions-been-removed-from-google-search https://webapps.stackexchange.com/questions/57249/has-the-op... - https://www.bladeforums.com/threads/how-to-do-google-discussion-search-now-that-it-has-been-disabled.1207584/ https://www.bladeforums.com/threads/how-to-do-google-discuss... - https://browsermedia.agency/blog/alternatives-discussion-search-filter/ https://browsermedia.agency/blog/alternatives-discussion-sea...

COAGULOPATH 2y ago

Something I'm increasingly noticing about LLM-generated content is that...nobody wants it. (I mean "nobody" in the sense of "nobody likes Nickelback". ie, not literally nobody.) If I want to talk to an AI, I can talk to an AI. If I'm reading a blog or a discussion forum, it's because I want to see writing by humans. I don't want to read a wall of copy+pasted LLM slop posted under a human's name. I now spend dismaying amounts of time and energy avoiding LLM content on the web. When I read an article, I study the writing style, and if I detect ChatGPTese ("As we dive into the ever-evolving realm of...") I hit the back button. When I search for images, I use a wall of negative filters (-AI, -Midjourney, -StableDiffusion etc) to remove slop (which would otherwise be >50% of my results for some searches). Sometimes I filter searches to before 2022. If Google added a global "remove generative content" filter that worked, I would click it and then never unclick it. I don't think I'm alone. There has been research suggesting that users immediately dislike content they perceive as AI-created, regardless of its quality. This creates an incentive for publishers to "humanwash" AI-written content—to construct a fiction where a human is writing the LLM slop you're reading. Falsifying timestamps and hijacking old accounts to do this is definitely something I haven't seen before.

robswc 2y ago
100%. So far (thankfully) I've noticed this stuff get voted down on social media but it is blowing my mind people think pasting in a ChatGPT response is productive. I've seen people on reddit say stuff like "I don't know but here's what ChatGPT said." Or worse, presenting ChatGPT copy-paste as their own. Its funny because you can tell, the text reads like an HR person wrote it.
- Trasmatta 2y ago
  I've noticed the opposite actually, clearly ChatGPT written posts on Reddit that get a ton of upvotes. I'm especially noticing it on niche subreddits. The ones that make me furious are on some of the mental health subreddits. People are asking for genuine support from other people, but are getting AI slop instead. If someone needs support from an AI (which I've found can actually help), they can go use it themselves.

elashri 2y ago

It is sad that this is happening to PhysicsForums. It was one of first websites I was using frequently 15 years ago when I started my physics passion (later career). I was active reader and contributed on few occasions but I still remember some members who I thought that one day I will be smart and knowledgeable like them. With years and the move to social media following Arab spring things started to change (as part of the overall transition from forum being the dominant place for discussions). But I stopped visiting it around 2018 unless I came through google search (later kagi). I still find the archive useful to answer some questions and I would disagree with author of article that because no one is sharing links on twitter that means no one care.

LordShredda 2y ago

Don't give out your real name online, the server admin might change your posts.

eddd-ddde 2y ago
They can also just, set a random account under your name. So no difference.

paulpauper 2y ago

Dead internet theory is one of those ideas that keeps resurfacing or being revied with articles like this, even though the evidence is only limited to confirmation bias. It ignores that there are huge parts of the internet that are not dead. I think it's more like the quality of discourse has fallen for reasons that are not clear.

NotYourLawyer 2y ago
> there are huge parts of the internet that are not dead Such as?
datadrivenangel 2y ago
The article looked at the PhysicsForums and found that 92% of the text is AI or machine generated...
- paulpauper 2y ago
  The internet is way bigger than PhysicsForums. That was my point, but your response seems to confirm what I said about discourse declining though.

inasio 2y ago

Talk about burying the lede! Near the bottom of the story the site owner confirms that it was him that added the backdated AI comments (perhaps it should have been obvious...)

firesteelrain 2y ago
I couldn’t find it. He was trying to seed the site ?
- pbronez 2y ago
  Experimenting with using AI bots to respond to questions that had been open for a long time with no response.
- inasio 2y ago
  This is what he replied when asked about the backdated comments: > The backdated answers were an internal test. We conceived of a bot that would provide a quality answer to a thread without a reply after 1+ years. That too also failed. Instead, I’m considering pruning all threads without a reply as they clutter up the forums.
  - MichaelZuo 2y ago
    He’s an SEO guy at shopify, so he likely doesn’t care about the consequences of doing shady SEO experiments with old user’s accounts…
  - naijaboiler 2y ago
    Thats some really shady stuff. Its one thing to provide answers with AI bot. Its totally disingenuous to 1) backdate it 2) assume the alias of a real person
advisedwang 2y ago
The investigation of the events on this particular website are just a tool to illustrate a much broader point about internet content, identity and LLMs.

majgr 2y ago

It is interesting, that this forum might be „AI poisoned” for other AI bots, because training AI on content generated by AI = garbage.

Slava_Propanei 2y ago

[dead]

Slava_Propanei 2y ago

[dead]

econ 2y ago

I like the assumption that it was a real account originally. It all seems so unthinkable but when running a forum or a blog with an active comment section.. what would you do/think if your users show up, browse around and not say anything for a week? You start out by making topics in your own name, write helpful replies.. until you look like an idiot talking to yourself. Forums with good traffic and lots of spammy advertisement no doubt consider it when visitors leave because nothing new happened. I once upon a time, on a rather stale forum, created two similarly named accounts from the same ip and argued with myself. At first I thought the owner or one of the other users would notice but I quickly learned that no behaviour is weird enough for it to be ever considered.

BlueTemplar 2y ago
I think I would rather post alone than my current experience (on 2 forums already) of other posters being overwhelmingly spam bots.
- econ 2y ago
  A blog is more suitable for that. You could mirror the posts on a forum and look sophisticated.

secretsatan 2y ago

I suspect a manager i work with has started using llms, i’ve sat next to him long enough before he went manager to know he’s incompetent, and now in chat, suddenly he spouts out of character plausible, but off for him explanations. I work in a company where for most english is not their first language so i’m not sure if anyone else has picked up on it.

hintymad 2y ago

A trajectory observation: People from the US and maybe Europe seem love having all kinds of niche forums, like PhysicsForum. On the contrary, Chinese people seemed love having a centralized place. Case in point, Zhihu(zhihu.com) was a clone of Quora, yet now Zhihu is the largest site for finding deep discussion of practically everything, maths, machine learning, history, physics, engineering, general science, just name a few. Tons of researchers and experts and hobbyists are there to share their insights. In contrasts, the quality of Quora seems have been deteriorating over the years, and most of the experts are in all kinds of niche forums anyway. I wonder why there is such a difference.

upghost 2y ago

I'm also noticing there's a dead economy theory or at least dead job market theory. AI sending resumes, AI reading/rejecting resumes. Humans wanting to talk to real humans, but all we get is slop. If we ever develop AGI I guess we'll have to go back to completely offline just to talk to real humans.