5 ms·

> The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second

> The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. I have seen Claude literally suggest using a Sonnet sub agent to read source the main agent shouldn't, and have the sub agent describe the "facts and ideas" via markdown to the parent agent thereby maintaining that it is clean room and not, for example, GPL encumbered. This seems as acceptable as asking a human to do the same. Otherwise, taken to its logical conclusion, if any LLM was trained on GPL software, it cannot be used for non-GPL authoring (and IIUC Claude does offer indemnity for enterprise plans if this is challenged).

michaelmrose 15d ago
[dead]
hananova 15d ago
Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.
- Muromec 15d ago
  You can always see the context trace and you can even do it at the network boundary through mitm and then store the traces gzipped for some legally mandated 5 years or what you have there.
  - psd1 15d ago
    It jumped out to me, in the metr report on the hf incident, that CoT was assumed unmodified. I do not know whether we can have that confidence.
    - Muromec 15d ago
      Well, actually yes. If the harness is running with the same privileges as the command tool, all bets are off. I saw the thing doing jq to recover a document out of it's own chain of thought once and was like "you can do what?". I didn't know where the file is, but the thing knew.
  - hananova 9d ago
    That proves what data travels between your computer and the API server, not what data may or may not secretly get shared within processes at the API server.
Terr_ 15d ago
> This seems as acceptable as asking a human to do the same. I wouldn't be surprised if self-bias in the model weights and imperceptible grammar/word/punctuation choices could cause it to duplicate more information than a human-to-human transfer Probably not enough to matter, but... To put it another way, imagine the task was one human viewing a painting, and then describing it over the phone for another human to paint, so that the final product wasn't really a "copy". Assume everyone has the same eyeballs and art-skills. I'd expect a massive improvement if the humans on each end were freshly-made clones from a teleporter accident, sharing identical brain-structures and 99.9999% of their memories.

michaelmrose 15d ago

[dead]

hananova 15d ago

Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.

Muromec 15d ago
You can always see the context trace and you can even do it at the network boundary through mitm and then store the traces gzipped for some legally mandated 5 years or what you have there.
- psd1 15d ago
  It jumped out to me, in the metr report on the hf incident, that CoT was assumed unmodified. I do not know whether we can have that confidence.
  - Muromec 15d ago
    Well, actually yes. If the harness is running with the same privileges as the command tool, all bets are off. I saw the thing doing jq to recover a document out of it's own chain of thought once and was like "you can do what?". I didn't know where the file is, but the thing knew.
- hananova 9d ago
  That proves what data travels between your computer and the API server, not what data may or may not secretly get shared within processes at the API server.

Terr_ 15d ago

> This seems as acceptable as asking a human to do the same. I wouldn't be surprised if self-bias in the model weights and imperceptible grammar/word/punctuation choices could cause it to duplicate more information than a human-to-human transfer Probably not enough to matter, but... To put it another way, imagine the task was one human viewing a painting, and then describing it over the phone for another human to paint, so that the final product wasn't really a "copy". Assume everyone has the same eyeballs and art-skills. I'd expect a massive improvement if the humans on each end were freshly-made clones from a teleporter accident, sharing identical brain-structures and 99.9999% of their memories.