6 ms·

How would you re-introduce correlations on top of a CSPRNG without a cryptographic break?

by kalkin 25d ago

rcxdude 25d ago
Because in order to not break the token selection process you are making decisions biased by the LLM's output, and in order to make the watermark detectable without the entire context, you are making those decisions based on a relatively small number of preceding tokens. To give an example in the extreme: if you make the bias total and only make that decision based on the preceding token, there are certain token pairs that your model will never output, and this will be pretty obvious even to people just reading the text (because of any given common two-token phrase, there's a 50% chance you would just disappear in watermarked text). You can make this less extreme and more hidden by increasing the window and reducing the bias, but at the cost of reducing the signal. I don't know exactly what the tradeoff curve looks like, so it might be that you can reach set of parameters where the bias is in principle undetectable without the key but still reliably detectable for realistic lengths of text segments, but I would not assume that this is definitely the case.
- kalkin 24d ago
  This doesn't change the temperature of the model. It's not going to make something that's a 70% chance suddenly a 100% chance. It also doesn't change the context length. At most, it seems like it may reduce some of the variation between different requests sampled from the same prompt with nonzero temperature, if I'm reading this right from the Google paper that Anthropic says they're implementing: > For our experiments, we configure SynthID-Text to be single-sequence non-distortionary; this preserves text quality and provides good detectability, while having some reduction to inter-response diversity. We call this configuration ‘non-distortionary SynthID-Text’ (and where not otherwise specified, ‘SynthID-Text’ also refers to this). https://www.nature.com/articles/s41586-024-08025-4 https://www.nature.com/articles/s41586-024-08025-4
  - rcxdude 24d ago
    That approach is pretty much what I was describing. The example in the paper uses the previous 4 tokens to derive the bias for the next token, so some 5 token sequences become more likely and others becomes less likely compared to what the LLM would normally produce. The context and temperature of the LLM stays the same, but the sampling process is biased by something that depends on far shorter runs of text.

How would you re-introduce correlations on top of a CSPRNG without a cryptographic break?

by kalkin 25d ago

How would you re-introduce correlations on top of a CSPRNG without a cryptographic break?

Because in order to not break the token selection process you are making decisions biased by the LLM's output, and in order to make the watermark detectable without the entire context, you are making those decisions based on a relatively small number of preceding tokens. To give an example in the extreme: if you make the bias total and only make that decision based on the preceding token, there are certain token pairs that your model will never output, and this will be pretty obvious even to people just reading the text (because of any given common two-token phrase, there's a 50% chance you would just disappear in watermarked text). You can make this less extreme and more hidden by increasing the window and reducing the bias, but at the cost of reducing the signal. I don't know exactly what the tradeoff curve looks like, so it might be that you can reach set of parameters where the bias is in principle undetectable without the key but still reliably detectable for realistic lengths of text segments, but I would not assume that this is definitely the case.

kalkin 24d ago
This doesn't change the temperature of the model. It's not going to make something that's a 70% chance suddenly a 100% chance. It also doesn't change the context length. At most, it seems like it may reduce some of the variation between different requests sampled from the same prompt with nonzero temperature, if I'm reading this right from the Google paper that Anthropic says they're implementing: > For our experiments, we configure SynthID-Text to be single-sequence non-distortionary; this preserves text quality and provides good detectability, while having some reduction to inter-response diversity. We call this configuration ‘non-distortionary SynthID-Text’ (and where not otherwise specified, ‘SynthID-Text’ also refers to this). https://www.nature.com/articles/s41586-024-08025-4 https://www.nature.com/articles/s41586-024-08025-4
- rcxdude 24d ago
  That approach is pretty much what I was describing. The example in the paper uses the previous 4 tokens to derive the bias for the next token, so some 5 token sequences become more likely and others becomes less likely compared to what the LLM would normally produce. The context and temperature of the LLM stays the same, but the sampling process is biased by something that depends on far shorter runs of text.