8 ms·

OpenAI shuts down its AI Classifier due to poor accuracy

• atleastoptimal 3y ago
  Even if AI detectors were 99% effective, anyone could just iterate over an AI produced piece of writing until it's in the 1% that isn't detected and submit it.
• seeknotfind 3y ago
  Watermarking is a more tractable approach, but the cat is out of the bag.
  • sydon 3y ago
    How would you go about watermarking AI written text?
    • mepian 3y ago
      If it's generated by a SaaS, the service could sign all output with a public key.
      • meandmycode 3y ago
        This isn't a watermark though, the idea of a watermark is that it's inherently embedded in the data itself while not drastically changing the data
      • hackernewds 3y ago
        So other text should not be tagged as AI generated?
        • deleted 3y ago
          [deleted]
      • csmpltn 3y ago
        Why is this comment being downvoted? OpenAI can internally keep a "hash" or a "signature" of every output it ever generated. Given a piece of text, they should then be able to trace back to either a specific session (or a set of sessions) through which this text was generated in. Depending on the hit rate and the hashing methods used, they may be able to indicate the likelihood of a piece of text being generated by AI.
        • pixl97 3y ago
          Why would they want to is my question. A single character change would break it. Then you have database costs of storing all that data forever. Moreso, it's only for openAI, I don't think it will be too long before other gpt4 level models are around and won't give two shits about catering to the AI identification police.
          • csmpltn 3y ago
            > A single character change would break it. That depends on how they hash the data, right? They can use various types of Perceptual Hashing [1] techniques which wouldn't be susceptible to a single-character change. [1] https://en.wikipedia.org/wiki/Perceptual_hashing https://en.wikipedia.org/wiki/Perceptual_hashing > Then you have database costs of storing all that data forever. A database of all textual content generated by people? That sounds like a gold mine, not a liability. But as I've mentioned earlier, they don't need to keep the raw data (a perceptual hash is enough). > won't give two shits about catering to the AI identification police I'm sure there will be customers willing to pay for access to these checks, even if they're only limited to OpenAI's product (universities and schools - for plagiarism detection, government agencies, intelligence agencies, police, etc).
    • thewataccount 3y ago
      https://arxiv.org/pdf/2301.10226.pdf https://arxiv.org/pdf/2301.10226.pdf Here's a decent paper on it. It covers private watermarking (you can't detect it exists without a key), resistance to modifications, etc. Essentially you wouldn't know it was there and you can't make simple modifications to fool it. OpenAI could already be doing this, and they could be watermarking with your account ID if they wanted to. The current best countermeasure is likely paraphrasing attacks https://arxiv.org/pdf/2303.11156.pdf https://arxiv.org/pdf/2303.11156.pdf
    • doctorpangloss 3y ago
      I don't know. I suppose hosted solutions like ChatGPT could offer an API where you copy some text in, and it searches its history of generated content to see if anything matches. > bUt aCtuAlLy... It's not like I don't know the bajillion limitations here. There are many audiences for detection. All of them are XY Problems. And the people asking for this stuff don't participate on Hacker News aka Unpopular Opinions Technology Edition. There will probably be a lot of "services" that "just" "tell you" if "it" is "written by an AI."
    • taneq 3y ago
      type=text/chatgpt :P
    • merlincorey 3y ago
      Invisible characters in a specific bit-pattern. Pretty common steganographic technique, really.
      • philipov 3y ago
        So, all I have to do is copy-paste it into a text editor with remove-all-formatting to circumvent that?
      • nonethewiser 3y ago
        Can you elaborate on "invisible?" The only invisible character I can imagine is a space. It seems like any other character either isn't invisible or doesn't exist (ie, isnt a character). Additionally, if I copy-paste text like this are the invisible characters preserved? Are there a bunch of extra spaces somewhere?
        • lucasmullens 3y ago
          There's a bunch of different "spaces", one is a "zero-width space" which isn't visible but still gets copied with the text. https://en.wikipedia.org/wiki/Zero-width_space https://en.wikipedia.org/wiki/Zero-width_space
          • pixl97 3y ago
            And the second site students will go to is zerospaceremover.com or whatever will show up to strip the junk.
        • michaelt 3y ago
          When students try to evade plagiarism detectors, they will swap characters like replacing spaces with nonbreaking spaces, replacing letters with lookalikes (I vs extended Cyrillic Ӏ etc), and inserting things like the invisible 'Combining Grapheme Joiner' IMHO it isn't a feasible way of watermarking text though - as someone would promptly come up with a website that undid such substitutions.
          • 3y ago
    • cateye 3y ago
      One way is trying to sneak in a specific structure/pattern that is difficult for a human to notice when reading, like using a particular sentence length, paragraph length, or punctuation pattern. Or use certain words in the text that may not be frequently used by humans etc. Watermarking needs to be subtle enough to be unnoticeable to opposing parties, yet distinctive enough to be detectable. So, this is an arms race especially because detecting it and altering it based on the watermark is also fun :)
      • nonethewiser 3y ago
        > One way is trying to sneak in a specific structure/pattern that is difficult for a human to notice when reading This seems like a total non-starter. That can only negatively impact the answers. A solution needs to be totally decoupled from answer quality.
        • yttribium 3y ago
          You can do this by injecting non visible unicode (LTR / RTL markers, zero width separators, the various "space" analogs, homographs of "normal" characters) but it can obviously be stripped out.
        • thewataccount 3y ago
          The paper I linked in the parent's comment as the "Simple proof of concept" on page 2, and like you said outlines it's limitations as both negative to performance and also easily detectable and determinable. Their improved method instead only replaces tokens when there's many good choices available, and skips replacing tokens when there are few good choices. "The quick brown fox jumps over the lazy dog" - "The quick brown" is not replaceable because it would severely harm the quality. Essentially it's only replacing tokens where it won't harm the performance. It's worth noting that any watermarking will likely harm the quality to some degree - but it can be minimized to the point of being viable.
    • brucethemoose2 3y ago
      Make half of the tokens (the AI's "dictionary") slightly more likely. This would not impact output quality much, but it would only work for longish outputs. And the token probability "key" could probsbly be reverse engineered with enough output.
      • pixl97 3y ago
        It would be pretty easy to figure out against standard word probability in average datasets. Even then the longer this system runs the more likely it is to pollute its own dataset by people learning to write from gpt itself.
  • cwkoss 3y ago
    only tractable for closed source hosted LLMs
• deleted 3y ago
  [deleted]
• capableweb 3y ago
  I'm glad that they did, although they should obviously done an announcement for it. The amount of people in the ecosystem who thinks it's even possible to detect if something is AI written or not when it's just a couple of sentences is staggering high. And somehow, people in power seems to put their faith in some of these tools that guarantee a certain amount of truthfulness when in reality it's impossible they could guarantee that, and act on whatever these "AI vs Human-written" tool tell them to. So hopefully this can serve as another example that it's simply not possible to detect if a bunch of characters were outputted by an LLM or not.
  • catboybotnet 3y ago
    There's also the post going around about how it can (and does) falsely flag human posts as AI output, particularly among some autistic people. About as useful as a polygraph, no?
    • capableweb 3y ago
      Both false-positives are as useful as the other one, flagged "human" but actually "LLM" vs flagged "LLM" but actually "human". As long as no one put too much weight on the result, no harm would have been done, in either case. But clearly, people can't stay away from jumping to conclusions based on what a simple-but-incorrect tool says.
      • frumper 3y ago
        A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process. There is no way to know when it’s wrong so you’ll either use it to help justify what you want, or ignore it. Edit: this tool is as reliable as a magic 8-ball
        • dontreact 3y ago
          If you were trying to predict the direction a stock will move (up or down) and it was right 99.9% of the time, would you use it or not?
• hospitalJail 3y ago
  Not to say "I can detect chatgpt" but it sure seems to have a similar way of talking even when I say things like: Talk like a "Millennial male who is obsessed with Zelda, their name is bob zelenski" Now the topic isnt about anything millennial or Zelda related, but I'd think that the language model would select sentence and paragraph phrasing differently. Maybe I need to switch to the API.
  • post-it 3y ago
    I've also noticed that ChatGPT tends to respond to short prompts, especially questions, in a predictable format. There are a few characteristics. First, it tends to print a five-paragraph essay, with an introduction, three main points, and a conclusion. Second, it signposts really well. Each of the body paragraphs is marked with either a bullet point or a number or something else that says "I'm starting a new point." Third, it always reads like a WikiHow article. There's never any subtle humour or self-deprecation or ironic understatement. It's very straightforward, like an infographic. It's definitely easy to recognize a ChatGPT response to a simple prompt if the author hasn't taken any measures to disguise it. The conclusion usually has a generic reminder that your mileage may vary and that you should always be careful.
    • SquareWheel 3y ago
      I have to admit I'm struggling to tell if this was done ironically, but your comment is exactly a five paragraph essay with an introduction, three main points, and a conclusion. If so, nice meta-commentary.
      • post-it 3y ago
        Thank you, it was intentional!
• hackernewds 3y ago
  Good. If it is not reliable, it's a further harm than good if it exists as a false security. An analogous example: my local pizza delivery (where I worked) would shut the box with a safety sticker, to avoid tampering / dipping by the delivery boys. Now, sometimes they would forget to do this for various logistical reasons. Every one of the non-stickered ones started getting returned as customers worried a pepperoni stolen. They stopped doing it shortly after.
  • pixl97 3y ago
    Eh, I'd consider that a failure of employee training and reverse the situation by giving out a weekly bonus to shifts that did not fail to put the security stickers on. Kinda like if they forgot to put the security seal on your aspirin, I'm not going to take them all off because someone forgot to run production with all the bottles sealed.
    • frumper 3y ago
      The bottle of Aspirin goes through many hands between the manufacturer and you including sitting on an unattended shelf open to the public. The person making the pizza is working for the same company as the person delivering it, or may even be the same person. If you can't trust the pizza co delivery person then you probably shouldn't trust the person making it either.
      • Too 3y ago
        Nowadays food delivery often goes through third party. Eg Uber eats, Foodora, Bolt, Grab, etc.
        • frumper 3y ago
          All the pizza places here have their own drivers still.
      • pixl97 3y ago
        Your right, don't eat at that pizza place either This is the brown m&m principle in effect.
• PeterStuer 3y ago
  It's an open ended red-queen problem. You can't win. Besides, even if they did win, they would still lose by shooting their own foot.
• chakintosh 3y ago
  This tool's been fueling tons of false accusations in academia. Wife is doing her PhD and she often tells me stories about professors falsely accusing students of using ChatGPT.
  • practice9 3y ago
    Lots of stories on Reddit about school teachers unfairly accusing students of using ChatGPT for assignments too
  • VeninVidiaVicii 3y ago
    Eh I am doing my PhD and I use ChatGPT all the time!
    • gigglesupstairs 3y ago
      And? The tool in question was used for AI text detection not generation.
      • KeplerBoy 3y ago
        Not all people might be accused wrongly. Then again does it matter if you use ChatGPT for inspiration?
        • gigglesupstairs 3y ago
          > does it matter if you use ChatGPT for inspiration? Absolutely not.
  • paxys 3y ago
    It isn't just this one. There are a hundred different "AI detectors" sold online that are all basically snake oil, but overzealous professors and school administrators will keep paying for them regardless.
  • Qqqwxs 3y ago
    A group of students at my university were claiming their papers were being marked by a LLM. They cited a classifier like the OP which they used on their feedback comments.
• stormed 3y ago
  Interesting. I was under the impression this tool was effective because of some sort of hidden patterns generated in sentences. I guess my assumption was way complex than what it actually is
• TheCaptain4815 3y ago
  I'm in the SEO game and I've spoken with some 'heavy players' who believe a "Google Ai Update" is in the works. As it currently stands, the search engine results will be completely overtaken by Ai content in the near future without this. From my understanding, this is a fools play in the long run, but there are current Ai Classifier Detectors that can successfully detect ChatGPT and other models (Originality.ai being a big one) on longish content. Their process is fairly simple, they create a classification model after generating tons of examples from all the major models (ChatGPT, GPT4, Laama, etc). One obvious downside to their strategy is the implementation of Finetuning and how that changes the stylistic output. This same 'heavy hitter' has successfully bypassed Originalities detector using his specified finetuning method (which he said took months of testing and thousands of dollars).
  • bearjaws 3y ago
    Google needs to do a full 180, and only the most succinct website that answers search queries should be elevated. The current state of Google is a disaster, everything is 100 paragraphs per article, the answer you are looking for buried half way in to make sure you spend more time and scroll to appease the algorithm. I cannot wait for them to sink all these spam websites.
    • pixl97 3y ago
      Waiting for Google to do that won't happen, they'd lose too many ad links.
• 13years 3y ago
  "Half a year later, that tool is dead, killed because it couldn’t do what it was designed to do." This was my conclusion as well testing the image detectors. Current automated detection isn’t very reliable. I tried out Optic’s AI or Not , which boasts 95% accuracy, on a small sample of my own images. It correctly labeled those with AI content as AI generated, but it also labeled about 50% of my own stock photo composites I tried as AI generated. If generative AI was not a moving target I would be optimistic such tools could advance and become highly reliable. However, that is not the case and I have doubts this will ever be a reliable solution. from my article on AI art - https://www.mindprison.cc/p/ai-art-challenges-meaning-in-a-world https://www.mindprison.cc/p/ai-art-challenges-meaning-in-a-w...
  • danuker 3y ago
    > but it also labeled about 50% of my own stock photo composites I tried as AI generated Could it be that a large proportion of the source stock photos were actually AI generated?
    • 13years 3y ago
      No, they were older images. However, that is now becoming a problem. Some stock photo sites now have AI images and they are not labeled. I'm able to distinguish most for now because at hires the details contain obvious errors. This is really painful, because for some of my work I need high quality images suitable for print. Now I can't just look at the thumbnail and say "this will work". I now have to examine it taking more of my time.
• kylecordes 3y ago
  There is inherent conflict in having both an AI tool business and an AI tool detection business. If the first does a good job, the second fails. And vice versa. (On the other hand, maybe there is a lot of money to be made selling both, to different groups?)
  • sebzim4500 3y ago
    I don't think this follows. If they wanted, they could crypographically bias the sampling to make the output detectable without decreasing capabilities at all. Only people using it deceptively would be affected. No idea what portion of ChatGPT's users that is, would be very interested to know.
    • zarzavat 3y ago
      There’s a much more effective way: store hashes of each output paragraph (with a minimum entropy) that has ever been generated, and allow people to enter a block of text to search the database. It wouldn’t beat determined users but it would at least catch the unaware.
      • sebzim4500 3y ago
        Changing every 10th word defeats that strategy but doesn't defeat a cryptographic bias. Also the cost of storing every paragraph hash might eventually add up even if at the moment it would be negligable compared to the generation cost.
        • doliveira 3y ago
          They literally already store the whole conversation...
        • explaininjs 3y ago
          One solution is to store a hash of every n-gram for n from 2 to whatever, then report what percent of ngrams of various lengths were hits. Did someone say Bloom Filter??
• sakopov 3y ago
  Humorously, in my experience, if a response from ChatGPT ever got classified as AI generated by tools like ZeroGPT or similar, all I had to do was adjust the prompt to tell the model not to sound like it was AI generated and that bypassed all detection with a very high success rate. Additionally, I also found that if you prompt it to make the response be in the style or some known writer for example, it often made responses 100% human written by most AI detection models.
  • klabb3 3y ago
    “Can’t you just try to blend in and be a little more cool? The bouncer is gonna notice.” Starts talking like Shakespeare
  • muzani 3y ago
    This didn't work half a year ago. They'd still accuse the rewritten text to be AI generated. I think some of the recent updates have changed the tone of ChatGPT so significantly that they no longer register on the radar.
• hayd 3y ago
  Was this what Stack Overflow were using to detect automated answers?
  • Ukv 3y ago
    None were officially built into the site so it'll vary from moderator to moderator, but the one that mods had a browser script made for to help streamline moderation was RoBERTa Base OpenAI Detector from 2019, created prior to the existence of GPT-3, GPT-3.5 (ChatGPT free), or GPT-4 (ChatGPT pro). It'll be far worse than the 2023 one this article is about.
• mercurialsolo 3y ago
  I wonder why we need this very thing of AI generated. It's a luddite view of AI. Much like the need to distinguish between handcrafted versus machined products - is there a real utility to knowing this? For educators looking at evaluating students, essays and the like - we possibly need different ways of evaluation rather than on written asynchronous content for communicating concepts and ideas.
  • klabb3 3y ago
    I believe you’re exactly right. It would be similar to detecting that math homework used wolfram alpha or even a calculator.
  • pixl97 3y ago
    >is there a real utility to knowing this? For civics, I would say yes. Imagine you were talking to an online group about a design project for a local neighborhood. Based on the plurality of voices it seemed like mist people wanted a brown and orange design. But later when you talk to actual people in real life, you could only find a few that actually wanted that. Virtual beings are a great addition to the bot nets that generate false consensus.
• rootusrootus 3y ago
  Good. And I think watermarking AI output is also a dead end. Better that we simply assume that all content is fake unless proven otherwise. To the extent that we need trustworthy photos, it seems like a better idea to cryptographically sign images at the hardware level when the photo is taken. Voluntarily watermarking AI content is completely pointless.
  • cwkoss 3y ago
    Cryptography cant save us here, people will figure out how to send AI images to the crypto hardware to get it signed in months. Just would be another similar layer of false security.
    • baby_souffle 3y ago
      > Cryptography cant save us here, people will figure out how to send AI images to the crypto hardware to get it signed in months. Possibly (who am I kidding. *PROBABLY*!) will use chatGPT to help them design the method :)
    • malfist 3y ago
      That's not how cryptographic signing works. Cryptographic signing means "I wrote this" or "I created this". Sure you could sign an AI generated image as yourself. But you could not sign an image as being created by Getty or NYT
      • esclerofilo 3y ago
        I believe that's not what they're saying. It's signing hardware, like a camera that signs every picture you take, so not even you can tamper with it without invalidating that signature. Naively, then, a signed picture would be proof that it was a real picture taken of a real thing. What GP is saying is that people would inevitably get the keys from the cameras, and then the whole thing would be pointless.
        • cwkoss 3y ago
• squarefoot 3y ago
  How could we have both the AI that is indistinguishable from humans and the AI that can detect it with good accuracy? That would imply the race on both sides for an AI that is more intelligent than an high IQ human.
  • jerf 3y ago
    Text is a very high dimensional space, n-dimensional in fact. There is plenty of room for an AI to leave a fingerprint that can be detected in some ways but not others. In fact it doesn't take much text to distinguish between two human beings. The humanly-obvious version is that someone that habitually speaks in one dialect and someone else in another must be separate, but even without such obvious tells humans separate themselves into characterizeable subsets of this space fairly quickly. I'm skeptical about generalized AI versus human detection in the face of the fact that it is adversarial. But a constant, unmoving target of some specific AI in some particular mode would definitely be detectable; e.g., "ChatGPT's current default voice" would certainly be detectable, "ChatGPT when instructed to sound like Ernest Hemmingway" would be detectable, etc. I just question whether ChatGPT in general can be characterized.
• JohnMakin 3y ago
  We shattered the turing test and now we want to put it back into pandora’s box because we don’t like the repercussions.
  • pixl97 3y ago
    We asked the question 'can we beat the turning test', not what would happen when we did.
  • callalex 3y ago
    “We shattered the Turing test” The Turing test is typically an interactive back-and-forth, no? Detecting if paragraphs of pre-written LLM output and giving a yes-or-no answer is not the same thing.
  • CatWChainsaw 3y ago
    The techbros were so focued on whether or not they could that they accused anyone who asked if they should of being a luddite. And dinner that night was crow, and dessert was humble pie.
• al_be_back 3y ago
  low accuracy is certainly a good reason to drop a project, especially when dealing with small text (<1000 chars), this is where most social media post/mini-blogs fall under. bigger text e.g. reports, thesis etc are probably easier & cheaper to verify by humans, with help of A.I. tools (ref checking, searching...)
• rhyme-boss 3y ago
  This should have been rejected as an idea just on its face. False positives are really problematic. And if it performs unexpectedly well (accuracy is high) then it just becomes a training tool for reinforcement learning.
• bilater 3y ago
  As a joke I built a simple tool that swaps random words for their synonym and it did the trick in throwing off any distribution matching (came out with gibberish but still lol) https://www.gptminus1.com/ https://www.gptminus1.com/
  • system2 3y ago
    Fascinating. I tried zerogpt.com for testing a text I generated. Got 91% AI generated response. Used gptminus1.com and now zerogpt.com shows 0% AI. These detectors are garbage.
• vorticalbox 3y ago
  Could we not add invisible characters into the text a bit like a water mark?
  • nomel 3y ago
    Yes, and we could just as easily remove them.
    • vorticalbox 3y ago
      True but I feel it would catch a few people out?
  • tester457 3y ago
    We could but doing so would lobotomize it.
• lacker 3y ago
  Smart of OpenAI to shut down a tool that basically doesn't work before the school year starts and students start to get in trouble based on it. I think this upcoming school year is going to be a wakeup call for many educators. ChatGPT with GPT-4 is already capable of getting mostly A's on Harvard essay assignments - the best analysis I have seen is this one: https://www.slowboring.com/p/chatgpt-goes-to-harvard https://www.slowboring.com/p/chatgpt-goes-to-harvard I'm not sure what instructors will do. Detecting AI-written essays seems technologically intractable, without cooperation from the AI providers, who don't seem too eager to prioritize watermarking functionality when there is so much competition. In the short term, it will probably just be fairly easy to cheat and get a good grade in this sort of class.
  • woeirua 3y ago
    Nah, everything is just going to be proctored exams on paper in the future. Sucks for the pro take-home project crowd, but they ruined it for themselves.
• spandrew 3y ago
  The only way to prevent AI from answering questions in digital platforms is to develop a ML db on the typing style of every student across their tenure at an institution. Good luck getting that approved — departments can't even access grade or demo data without a steering group going through a 3-deep committee process. ¯\_(ツ)_/¯ try paper I guess. Time to brush up on our OCR.
  • humanistbot 3y ago
    If AI can replicate linguistic patterns in a way that is undetectable for both humans and models, then it seems even easier for a ML model to emulate a natural typing style, rhythm, and cadence in a way that is undetectable for both humans and models. But you know who has more real-world data on typing style? Google, Microsoft, Meta, and everyone else who runs SaaS docs, emails, or messaging. I imagine a lot of students write their essays on Google Docs, Word, or the like, and submit them as attachments or copy-paste into a textbox.
• jillesvangurp 3y ago
  This kind of thing strikes me in any case as something that's only good for the generation of AI it's been trained against. And with the exponential improvements happening almost on a monthly basis, that becomes obsolete pretty quickly and a bit of a moving target. Maybe a better term would be Superior Intelligence (SI). I sure as hell would not be able to pass any legal or medical exams without dedicating the next decade or so to getting there. Nor do I have any interest in doing so. But chat gpt 4 is apparently able to wow its peers. Does that pass the Turing test because it's too smart or too stupid? Most of humanity would fail that test.
• yk 3y ago
  Funny incentive problem, OpenAI obviously has an incentive to use it's best AI detection tool for adversarial training, with the result that it's detection tool will not be very good against chatGPT generated text because it is trained to defeat the detection tool.
  • mitthrowaway2 3y ago
    If they wanted to disguise their content as coming from a human, they probably wouldn't keep saying "As an AI chatbot, I can't..."
    • yk 3y ago
      I'm not saying that they are trying to disguise, I'm saying that their goal is natural language and a way you can distinguish chatGPT (aside from trivial things like "as an ai chatbot...") from human speech is most likely a way to improve chatGPT, because the goal is speaking like a human.
• jedberg 3y ago
  Latest I heard is that teachers are requiring homework to be turned in in Google Docs so that they can look at the revision history and see if you wrote the whole thing or just dumped a fully formed essay into GDocs and then edited it. Of course the smart student will easily figure out a way to stream the GPT output into Google Docs, perhaps jumping around to make "edits". A clever and unethical student is pretty much undetectable no mater what roadblocks you put in their way. This just stops the not clever ones. :)
  • justrealist 3y ago
    Well, there is one way, which is timed, proctored exams. Which sucks, because take-home projects are evaluating a different skill set, and some people thrive on one vs the other. But it is what it is.
  • ribosometronome 3y ago
    Retyping the essay from the chatgpt while actively rewording the occassional sentence seems like it would do it.
    • wmeredith 3y ago
      It seems like that's nearing the sweet spot of fraud prevention, where committing the act of fraud is as much work as doing the real thing.
      • frumper 3y ago
        It sounds a lot easier to retype what you see rather than to create it.
      • Nickersf 3y ago
        The intellectual labor of thinking about an essay, drafting it, editing, and revising it is much higher than strategically re-typing a ChatGPT output. One requires experience, knowledge, understanding and creativity, the other one requires basic functioning of motor skills and senses. You could program a robot to re-type the ChatGPT output into a different word processor and feed it parameters to make the duration between keystrokes and backspaces fluctuate over time. You could even have it stop, come back later, copy and paste sections and re-organize as it moves through and end up with the final essay from ChatGPT.
• wouldbecouldbe 3y ago
  If the goal is students, then the best would be a tool not only detects AI. But where you can submit previous writing and see how likely it is they wrote a similar text, not so much if it was llm generated.
• kmeisthax 3y ago
  The idea that OpenAI was intentionally watermarking its output to avoid training data back-contamination should be thoroughly discredited now.
• janalsncm 3y ago
  On a related note, I built this site as a PSA: https://isthiswrittenbyai.surge.sh https://isthiswrittenbyai.surge.sh
• deleted 3y ago
  [deleted]
• bloppe 3y ago
  Rather than try to detect if pieces were generated by AI, why not just check if they're plagiarized off the outputs of a bunch of popular models? We already have traditional plagiarism analysis methods, we just need a corpus of most of the recent outputs from the most popular LLM services to check against. OpenAI, Google, Anthropic, and any other LLM-as-a-service companies could profitably sell access to these corpora to third-party analysis services that compile them and offer search.
  • ShamelessC 3y ago
    One reason would be that LLM's don't tend to output verbatim pieces from their dataset (unless explicitly prompted to do so). This is made further complicated by the "temperature" setting allowing users to make output even more "creative" than its dataset. In OpenAI's case, its writing style usually comes from OpenAI's in-house dataset they used for RLHF. This is what gives it the ability to chat and respond with its signature (perhaps overly formal and apologetic) tone. Although it can be used to write in other styles, sometimes it will refuse to because of this.
    • bloppe 3y ago
      You misunderstand me. I'm not talking about detecting when LLM's plagiarize humans, but when humans "plagiarize" (copy from without attributing) LLM's.
• deleted 3y ago
  [deleted]
• ChatGTP 3y ago
  We're entering a wild world. Our information space is up for the slaughter.
• cutler 3y ago
  I urge anyone who values data privacy to refuse to use sites such as this which employ third party data services, ie. popups, which purport to allow you to "Manage" your consent but which hide a long list of opt-out "Legitimate Interest" flags in a "Vendors" list concealed at the bottom of another scrolling list.
• judcnegri 3y ago
  I think educators got away for too long of handling the problem of umotivated students and uninteresting classes subjects... Not the educators fault tough, more like the system is bad. My point is that given knowledge is mostly free and available, the system should teach the students to think rather than using tools or remembering facts
• partiallypro 3y ago
  If we can't tell what is written by AI, how will we know if AI is just stuck in a feedback loop in which it references itself (and is wrong)?
• BigElephant 3y ago
  No official announcement is lame
  • system2 3y ago
    No lamer than dropping davinci.
• albert_e 3y ago
  Many comments here seem to suggest that practically it is going to be impossible to classify text as Human generated versus AI generated -- given the many different ways in which such attempts can be foiled in a never ending game of cat-and-mouse. If we accept this ... The challenge I am foreseeing is this: We are only at the very beginning of the AI revolution -- and if LLMs need to get more sophisticated and powerful in future they will need good-quality human-generated / curated training data at a scale that is likely impossible to do manual curation/cleansing/quality-checks on. And there is no doubt that evey medium is going to get bombarded and spammed with AI-generated content in coming years. How then, are we going to filter the data -- to separate the real data from AI generated noise -- to train future LLMs on -- and really push them to their potential. This problem has been bugging me for a while and I commented here previously as well, tentatively calling it 'Data Pollution' for the lack of a better word. Curious to hear other perspectives on this.
  • kobalsky 3y ago
    curation? I mean of the content is considered good, does it matter if it was produced by an LLM?
• thepaulthomson 3y ago
  Wow, OpenAI pulling the plug on its AI Classifier tool really shows how tough it is to spot the difference between human and AI-generated text. It's a bummer that it didn't work out as planned, but hey, that's how we learn, right?
  • ShamelessC 3y ago
    No disrespect, but...is this comment AI generated?
• bcherny 3y ago
  Timing seems related to Altman launching Worldcoin (whose goal is to reliably differentiate between human and AI generated content). https://www.reuters.com/technology/openais-sam-altman-launches-worldcoin-crypto-project-2023-07-24/ https://www.reuters.com/technology/openais-sam-altman-launch...
• anshumankmr 3y ago
  I read a post on LinkedIn by Timnit Gebru, where she shared an anonymous post by a student who said that their paper was being rejected due to having seemingly high AI written content. Their paper was evaluated by Turnitin who claim to have a built an AI detector. The student mentioned it was putting their career at risk, and they hadn't actually taken the help of any AI and was actually a 4.0 student throughout their college career. So assuming all that to be true, how can the likes of Turnitin claim to be an authority for AI writing detection. When I graduated a few years back, they used to offer plag check only.
  • ShamelessC 3y ago
    > how can the likes of Turnitin claim to be an authority for AI writing detection Pretty easy - they lie to people.
• Roark66 3y ago
  I wonder if this entire "AI cheating" stuff will end up as calculators did when I was growing up. When I was a young child in school calculators were forbidden. You had to learn multiplications in your head. By the time I got to the final exam at the end of our "high school" equivalent we could use "dumb" calculators in math exams. Few years later graphing calculators became accepted in schools (sometimes even required). It is important humans learn to express themselves in writing. The only way I think this will happen is if kids do their writing at school supervised.
• torginus 3y ago
  There's an exceedingly simple way of doing this that's pretty much bullet proof: Just check the given text against the database of stored generations (which they no doubt keep) and you'll have a pretty much perfect result. Logistically, searching untold terabytes of text might be challenging but to say there's fundamental difficulties is just not accurate imo.
  • Grimblewald 3y ago
    that's the problem - no you won't. The problem is you've bought too much into the stochastic parrot view of LLM's to a point where you are viewing it as a fancy content storage/retrieval system.
• islammidov 3y ago
  It's very interesting actually. I think from business perspective SEO is the most exposed. Very curious how Google will (or may be now) solve the issue?
• nektro 3y ago
  it was doomed to fail from the start. being detectable is precisely how the bot is trained. so the moment a tool figures out a classifier, a new generation that undetectable springs up.