4 ms·

I'd counter with an anecdote; I had a colleague that boasted how he memorized a classmate's SSN in college and would greet him by SSN when seeing him years late

I'd counter with an anecdote; I had a colleague that boasted how he memorized a classmate's SSN in college and would greet him by SSN when seeing him years later. Is the goal of AI to replicate the entirety of the human experience (including social pressures, norms, and shame) or a tool to complement human decision making? While, yes, you can argue the slippery slope, it may be advantageous to flag certain training material as exempt. We as humans often make decisions without perfect knowledge, and "knowing more" isn't a guarantee that it produces better outcomes, given the types of information consumed.

lmm 1y ago
Knowing more might not improve your accuracy but it's not going to harm it. Forcibly forgetting true parts of your knowledge seems far more likely to have unintended consequences.
- conception 1y ago
  Counterpoint: There are plenty examples of breakthroughs from folks who are ignorant of the “right” way to go about it. A fresh take isn’t always bad.
- Dylan16807 1y ago
  I disagree. Actively fighting against your memory will slow you down in any context where some memorized idea is similar to what you're doing but you shouldn't be using the memorized idea.
- lou1306 1y ago
  One obvious consequence: the model might still produce copyright infringement because it thinks its creative ideas are novel.
  - genewitch 1y ago
    If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation, and also it ought devalue the original work, moreso if the work is more recent. Maybe. I get that "first to publish" matters to a lot of people, but, say 5 unrelated people are writing unique screenplays about a series of events that seems important to them or culture or whatever; if they all come up with very similar plots and locations and scenes, it just means that the idea is more obvious than non-obvious. Please, argue. I haven't fully reconciled a lot of this to myself, but off the cuff this'll do. The logic being - if an AI without taint produces some other work, that work drew on the same information the model did, and came to the same "conclusion" - which means with a time machine, you could wipe the LLM, go back to the period of the original work, train the LLM, and produce the work contemporaneous to the original. Hope that made sense.
    - lmm 1y ago
      > If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation You can't claim it's a clean room without actually doing the legwork of making a clean room. Not including the copyrighted work verbatim isn't enough, you would need to show that the AI hadn't seen anything derived from that copyrighted work, or that it had seen only non-copyrightable pieces.
    - lou1306 1y ago
      > The logic being - if an AI without taint produces some other work, that work drew on the same information the model did, and came to the same "conclusion" - which means with a time machine, you could wipe the LLM, go back to the period of the original work, train the LLM, and produce the work contemporaneous to the original. Hope that made sense. This logic would immediately get shot down by an "Objection, speculation" in an actual litigation. Besides, the technicalities of how the work was produced don't really play a role in assessing infringement. PK Dick wrote "The man in the high castle" by extensively using the I Ching, but if I use it and recreate the novel by complete accident I would still be infringing. By the way, I highly suggest Borges's "Pierre Menard, Author of the Quixote" as a great story on the topic of authorship :)
      - genewitch 1y ago
        > PK Dick wrote "The man in the high castle" by extensively using the I Ching, but if I use it and recreate the novel by complete accident I would still be infringing. I touched on this, with the comment that we love "first to market." That multiple people coming up with the same output may mean that the idea isn't that novel. whether that matters or not isn't really relevant to me. The part you quoted was just a thought experiment to explain why i compared it to a "clean room implementation" - note it also avoids this argument from a sibling comment: >need to show that the AI hadn't seen anything derived from that copyrighted work since there could not possibly be any derived work prior to the "original" work being published. For the sake of argument.
    - kgwgk 1y ago
      I highly suggest Borges's "Pierre Menard, Author of the Quixote" as a great story on the topic of authorship :)
lynx97 1y ago
The goal of AI is to make money. All the moralisation is very human, but also extremely naive. BTW, I don't really understand what "social pressure" and "shame" has to do with your story? In my book, the person with a good memory isn't to blame. They're just demonstrating a security issue, which is a good thing.
- falcor84 1y ago
  In that example, the mnemonist should be demonstrating the security issue to the government, and not to their friend. We have social taboos for this reason. As an extreme example, I wouldn't greet a person by their penis size after noticing it in the locker room - some information should still be considered private, regardless of how we came to obtain it. Same with an LLM, when it got sensitive information in its weights, regardless of how it obtained it, I think we should apply pressure/shame/deletion/censorship (whatever you call it) to stop it from using that information in any future interactions.
  - lynx97 1y ago
    I am probably too autistic to recognize remembering a personal datum as a taboo. However, I am totally on your side regarding LLMs learning data they shouldn't have seen in the first place. IMO, we as a society are too much chicken to act on the current situation. Its plain insane that everyone and their dog knows that libgen has been used to train models, and the companies who did this experiencing NO consequences at all. After that, we shouldn't be surpised if things go downhill from here on.

lmm 1y ago

Knowing more might not improve your accuracy but it's not going to harm it. Forcibly forgetting true parts of your knowledge seems far more likely to have unintended consequences.

conception 1y ago
Counterpoint: There are plenty examples of breakthroughs from folks who are ignorant of the “right” way to go about it. A fresh take isn’t always bad.
Dylan16807 1y ago
I disagree. Actively fighting against your memory will slow you down in any context where some memorized idea is similar to what you're doing but you shouldn't be using the memorized idea.
lou1306 1y ago
One obvious consequence: the model might still produce copyright infringement because it thinks its creative ideas are novel.
- genewitch 1y ago
  If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation, and also it ought devalue the original work, moreso if the work is more recent. Maybe. I get that "first to publish" matters to a lot of people, but, say 5 unrelated people are writing unique screenplays about a series of events that seems important to them or culture or whatever; if they all come up with very similar plots and locations and scenes, it just means that the idea is more obvious than non-obvious. Please, argue. I haven't fully reconciled a lot of this to myself, but off the cuff this'll do. The logic being - if an AI without taint produces some other work, that work drew on the same information the model did, and came to the same "conclusion" - which means with a time machine, you could wipe the LLM, go back to the period of the original work, train the LLM, and produce the work contemporaneous to the original. Hope that made sense.
  - lmm 1y ago
    > If the copyrighted content is not in the training data, and I mean explicitly, and the AI produces a copyrighted output, I'd argue it's a clean room re-implementation You can't claim it's a clean room without actually doing the legwork of making a clean room. Not including the copyrighted work verbatim isn't enough, you would need to show that the AI hadn't seen anything derived from that copyrighted work, or that it had seen only non-copyrightable pieces.
  - lou1306 1y ago
    > The logic being - if an AI without taint produces some other work, that work drew on the same information the model did, and came to the same "conclusion" - which means with a time machine, you could wipe the LLM, go back to the period of the original work, train the LLM, and produce the work contemporaneous to the original. Hope that made sense. This logic would immediately get shot down by an "Objection, speculation" in an actual litigation. Besides, the technicalities of how the work was produced don't really play a role in assessing infringement. PK Dick wrote "The man in the high castle" by extensively using the I Ching, but if I use it and recreate the novel by complete accident I would still be infringing. By the way, I highly suggest Borges's "Pierre Menard, Author of the Quixote" as a great story on the topic of authorship :)
    - genewitch 1y ago
      > PK Dick wrote "The man in the high castle" by extensively using the I Ching, but if I use it and recreate the novel by complete accident I would still be infringing. I touched on this, with the comment that we love "first to market." That multiple people coming up with the same output may mean that the idea isn't that novel. whether that matters or not isn't really relevant to me. The part you quoted was just a thought experiment to explain why i compared it to a "clean room implementation" - note it also avoids this argument from a sibling comment: >need to show that the AI hadn't seen anything derived from that copyrighted work since there could not possibly be any derived work prior to the "original" work being published. For the sake of argument.
  - kgwgk 1y ago
    I highly suggest Borges's "Pierre Menard, Author of the Quixote" as a great story on the topic of authorship :)

lynx97 1y ago

The goal of AI is to make money. All the moralisation is very human, but also extremely naive. BTW, I don't really understand what "social pressure" and "shame" has to do with your story? In my book, the person with a good memory isn't to blame. They're just demonstrating a security issue, which is a good thing.

falcor84 1y ago
In that example, the mnemonist should be demonstrating the security issue to the government, and not to their friend. We have social taboos for this reason. As an extreme example, I wouldn't greet a person by their penis size after noticing it in the locker room - some information should still be considered private, regardless of how we came to obtain it. Same with an LLM, when it got sensitive information in its weights, regardless of how it obtained it, I think we should apply pressure/shame/deletion/censorship (whatever you call it) to stop it from using that information in any future interactions.
- lynx97 1y ago
  I am probably too autistic to recognize remembering a personal datum as a taboo. However, I am totally on your side regarding LLMs learning data they shouldn't have seen in the first place. IMO, we as a society are too much chicken to act on the current situation. Its plain insane that everyone and their dog knows that libgen has been used to train models, and the companies who did this experiencing NO consequences at all. After that, we shouldn't be surpised if things go downhill from here on.