5 ms·

> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually p

by joshka 15d ago

> Apple argues that when trade secret information is fed into an AI agent or model that learns from it, that learning “may create irreversible and continually propagating uses of the trade secret.” This is somewhat of a high impact argument to test. I wonder if the case will eventually get to working this point out.

fantasizr 15d ago
the fact that llms can spit out books they've been trained on near verbatim lends some credence to this.
- fantasizr 15d ago
  "However, we also find that Llama 3.1 70B memorizes some books, like Harry Potter and the Sorcerer’s Stone and 1984, almost entirely. In fact, Harry Potter is so memorized that, using a seed prompt consisting of just the first line of chapter 1, we can deterministically generate the entire book near-verbatim. " https://reglab.stanford.edu/publications/extracting-memorized-pieces-of-copyrighted-books-from-open-weight-language-models/ https://reglab.stanford.edu/publications/extracting-memorize...
  - keeda 15d ago
    A more important quote from that link: > With our specific experiments, we find that the largest LLMs don’t memorize most books–either in whole or in part. This technique has only ever been made to work with a vanishingly small number of extremely popular works, probably because they are so overrepresented in the training data set. The risk with IP, however, is a lot more grave. You may not even need to memorize the details of the IP verbatim, just the broad idea may be enough. It may lurk encoded in the weights forever, just waiting to be activated by the right prompt to start a chain of thought that unlocks further details. Heck, it may even appear as if the model suggested the idea itself.
rjsw 15d ago
A related argument could be whether the output of an LLM that has been trained on GPL code should also be considered to be under a GPL licence.
- m4rtink 15d ago
  I would not have a problem with that. :)
KingMachiavelli 15d ago
Not sure why people are latching on to this. I'm 95% sure the AI agent was not "trained" on this intellectual property but rather the IP was just analyzed and it's usage and details were recorded to the AI agents memory (typically markdown/JSON summary of previous sessions). The defense IMO is just trying to muddy the waters between "irreversible AI training that updates weights" and "AI learning by just storing secrets in text files". Even in the 5% chance this AI model actually was fine-tuned or fully-trained (i.e the weights were updated), the employee did this while employed at Apple and used Apple IP as the training data. Even if the output is under fair-use (because it's transformative), (1) any IP created while employed during work duties almost always belongs to your employer and (2) the act of accessing Apple's IP would be (trade secret or regular) theft if done outside normal work duties.
- joshka 14d ago
  Not sure why the downvotes on this - it's a pretty reasonable take. I suspect that you're right that it's unlikely that the tainted items would be used to update the weights here and are more likely to be something that would be in sessions / memories rather than future model weights.
ChrisRR 15d ago
This is why many companies don't allow the use of AI programming, because it's currently unknown whether allowing AI to access your code can allow parts of it to be replicated elsewhere When a company finally challenges it legally, it might finally answer that unknown for many companies
Jtsummers 15d ago
There's already some precedent when dealing with humans reverse engineering systems. We use "cleanroom" approaches where the ones doing the reverse engineering are segregated from those doing the later design and development work. Instead of producing a design document based on the thing being studied, the reverse engineers produce a specification from which a new design, untainted by knowledge of the original design, is produced. People using ML to try and reverse-engineer and create a "clean" version of things will likely need to use a similar approach. You can't ask one LLM to take in a circuit design as input and produce a specification and design within the same context. The resulting design will be at least partially informed by knowledge of the original design. The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. At least then you know your LLM was not aware of the original design. Of course, LLM agents "cheat", so you'll also want to be careful to ensure a clean environment if you're using an agent that does not provide access to the original design material.
- kodablah 15d ago
  > The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. I have seen Claude literally suggest using a Sonnet sub agent to read source the main agent shouldn't, and have the sub agent describe the "facts and ideas" via markdown to the parent agent thereby maintaining that it is clean room and not, for example, GPL encumbered. This seems as acceptable as asking a human to do the same. Otherwise, taken to its logical conclusion, if any LLM was trained on GPL software, it cannot be used for non-GPL authoring (and IIUC Claude does offer indemnity for enterprise plans if this is challenged).
  - michaelmrose 15d ago
    [dead]
  - hananova 15d ago
    Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.

by joshka 15d ago

fantasizr 15d ago

the fact that llms can spit out books they've been trained on near verbatim lends some credence to this.

fantasizr 15d ago
"However, we also find that Llama 3.1 70B memorizes some books, like Harry Potter and the Sorcerer’s Stone and 1984, almost entirely. In fact, Harry Potter is so memorized that, using a seed prompt consisting of just the first line of chapter 1, we can deterministically generate the entire book near-verbatim. " https://reglab.stanford.edu/publications/extracting-memorized-pieces-of-copyrighted-books-from-open-weight-language-models/ https://reglab.stanford.edu/publications/extracting-memorize...
- keeda 15d ago
  A more important quote from that link: > With our specific experiments, we find that the largest LLMs don’t memorize most books–either in whole or in part. This technique has only ever been made to work with a vanishingly small number of extremely popular works, probably because they are so overrepresented in the training data set. The risk with IP, however, is a lot more grave. You may not even need to memorize the details of the IP verbatim, just the broad idea may be enough. It may lurk encoded in the weights forever, just waiting to be activated by the right prompt to start a chain of thought that unlocks further details. Heck, it may even appear as if the model suggested the idea itself.

rjsw 15d ago

A related argument could be whether the output of an LLM that has been trained on GPL code should also be considered to be under a GPL licence.

m4rtink 15d ago
I would not have a problem with that. :)

KingMachiavelli 15d ago

Not sure why people are latching on to this. I'm 95% sure the AI agent was not "trained" on this intellectual property but rather the IP was just analyzed and it's usage and details were recorded to the AI agents memory (typically markdown/JSON summary of previous sessions). The defense IMO is just trying to muddy the waters between "irreversible AI training that updates weights" and "AI learning by just storing secrets in text files". Even in the 5% chance this AI model actually was fine-tuned or fully-trained (i.e the weights were updated), the employee did this while employed at Apple and used Apple IP as the training data. Even if the output is under fair-use (because it's transformative), (1) any IP created while employed during work duties almost always belongs to your employer and (2) the act of accessing Apple's IP would be (trade secret or regular) theft if done outside normal work duties.

joshka 14d ago
Not sure why the downvotes on this - it's a pretty reasonable take. I suspect that you're right that it's unlikely that the tainted items would be used to update the weights here and are more likely to be something that would be in sessions / memories rather than future model weights.

ChrisRR 15d ago

This is why many companies don't allow the use of AI programming, because it's currently unknown whether allowing AI to access your code can allow parts of it to be replicated elsewhere When a company finally challenges it legally, it might finally answer that unknown for many companies

Jtsummers 15d ago

There's already some precedent when dealing with humans reverse engineering systems. We use "cleanroom" approaches where the ones doing the reverse engineering are segregated from those doing the later design and development work. Instead of producing a design document based on the thing being studied, the reverse engineers produce a specification from which a new design, untainted by knowledge of the original design, is produced. People using ML to try and reverse-engineer and create a "clean" version of things will likely need to use a similar approach. You can't ask one LLM to take in a circuit design as input and produce a specification and design within the same context. The resulting design will be at least partially informed by knowledge of the original design. The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. At least then you know your LLM was not aware of the original design. Of course, LLM agents "cheat", so you'll also want to be careful to ensure a clean environment if you're using an agent that does not provide access to the original design material.

kodablah 15d ago
> The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. I have seen Claude literally suggest using a Sonnet sub agent to read source the main agent shouldn't, and have the sub agent describe the "facts and ideas" via markdown to the parent agent thereby maintaining that it is clean room and not, for example, GPL encumbered. This seems as acceptable as asking a human to do the same. Otherwise, taken to its logical conclusion, if any LLM was trained on GPL software, it cannot be used for non-GPL authoring (and IIUC Claude does offer indemnity for enterprise plans if this is challenged).
- michaelmrose 15d ago
  [dead]
- hananova 15d ago
  Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.