6 ms·

Watermarking: The Enshittification of Closed Models Has Begun

16 points by PhilKunz 1mo ago

2001zhaozhao 1mo ago
I think this is overblown, the watermarking is just enforcing a particular recognizable output text style. The existing Claudisms like "smoking gun" and "it's not x it's y" evidently haven't impacted the model's agentic capabilities much, and i doubt the new ones will either.
- PhilKunz 1mo ago
  But it means we are optimizing for something else than the best outcome or not? It is like with mp3 vs flac: mp3 is fine and optimizes for amount of data to represent a certain waveform, but lossless is the real deal for audiophiles and the only one capturing the real thing at a certain resolution. Same might be true for complex code or the best solution to a compression algorithms. Can the question really be answered if optimizing for marking does still produce the best result? Is it not like an author wanting to say something one way, but being forced to use certain words to get there?
  - cyanydeez 1mo ago
    none of the large AI labs know what the best outcome is because lab science never fully predict large scale implementation. We've had decades of grade A research on social improvement programs and it's clear to everyone that implementation cannot be simulated or easily predicted. So the "optimizing" thing is a fallacy. It's also found in highly racially charged pseudo science and eugenics; selecting for trains on small scales do not equal large scale benefits. But people's mental model are predisposed to small scales while also being blissfully ignorant of cognitive biases to the predisposition. Even the idea that we can just keep throwing compute, context, power is a fallacy when you see chinese models making smaller and just as capable models with limited resources. so, optimizing is something we should be putting into a democratic process because anything else will be lopsided, much like when eugenics was tried.
gonight 1mo ago
So is k3 on openrouter the current meta?
rbtms 1mo ago
I wonder what watermarks on an LLM output would look like. Say you have a draft for an email you have written and would like the LLM to correct typos and such. Would it be forced to change it substantially? If you tell it to reword it even so slightly, would the watermark still be valid?
- bionhoward 1mo ago
  I think it’s a bunch of hidden UTF-8 character substitutions so the text looks the same but the underlying bytes are different
  - rbtms 1mo ago
    Thanks, that makes a lot of sense. I wonder however how can this be done without giving a lot of problems when copypasting or sending the input to other LLMs (not even thinking about code here).
    - bionhoward 1mo ago
      I was wrong here, it’s more about the RNG they use for the word choices [1]. Although I wouldn’t be surprised to see UTF-8 substitutions also. [1] https://www.anthropic.com/news/claude-text-watermark https://www.anthropic.com/news/claude-text-watermark

16 points by PhilKunz 1mo ago

2001zhaozhao 1mo ago

I think this is overblown, the watermarking is just enforcing a particular recognizable output text style. The existing Claudisms like "smoking gun" and "it's not x it's y" evidently haven't impacted the model's agentic capabilities much, and i doubt the new ones will either.

PhilKunz 1mo ago
But it means we are optimizing for something else than the best outcome or not? It is like with mp3 vs flac: mp3 is fine and optimizes for amount of data to represent a certain waveform, but lossless is the real deal for audiophiles and the only one capturing the real thing at a certain resolution. Same might be true for complex code or the best solution to a compression algorithms. Can the question really be answered if optimizing for marking does still produce the best result? Is it not like an author wanting to say something one way, but being forced to use certain words to get there?
- cyanydeez 1mo ago
  none of the large AI labs know what the best outcome is because lab science never fully predict large scale implementation. We've had decades of grade A research on social improvement programs and it's clear to everyone that implementation cannot be simulated or easily predicted. So the "optimizing" thing is a fallacy. It's also found in highly racially charged pseudo science and eugenics; selecting for trains on small scales do not equal large scale benefits. But people's mental model are predisposed to small scales while also being blissfully ignorant of cognitive biases to the predisposition. Even the idea that we can just keep throwing compute, context, power is a fallacy when you see chinese models making smaller and just as capable models with limited resources. so, optimizing is something we should be putting into a democratic process because anything else will be lopsided, much like when eugenics was tried.

gonight 1mo ago

So is k3 on openrouter the current meta?

rbtms 1mo ago

I wonder what watermarks on an LLM output would look like. Say you have a draft for an email you have written and would like the LLM to correct typos and such. Would it be forced to change it substantially? If you tell it to reword it even so slightly, would the watermark still be valid?

bionhoward 1mo ago
I think it’s a bunch of hidden UTF-8 character substitutions so the text looks the same but the underlying bytes are different
- rbtms 1mo ago
  Thanks, that makes a lot of sense. I wonder however how can this be done without giving a lot of problems when copypasting or sending the input to other LLMs (not even thinking about code here).
  - bionhoward 1mo ago
    I was wrong here, it’s more about the RNG they use for the word choices [1]. Although I wouldn’t be surprised to see UTF-8 substitutions also. [1] https://www.anthropic.com/news/claude-text-watermark https://www.anthropic.com/news/claude-text-watermark