6 ms·

I am pretty sure they did A/B testing to show it didn't. I could gave sworn they even released a quiz were the user has to try and guess which answer is waterma

I am pretty sure they did A/B testing to show it didn't. I could gave sworn they even released a quiz were the user has to try and guess which answer is watermarked or not and it was impossible to tell.

sejje 15d ago
That's not the metric, though. You either output the best version, or you output something else. You can't do both.
- frabcus 15d ago
  That's not the case, because LLMs are non-deterministic. It only alters outputs when the last layer of the neural network give significant weights to multiple tokens, and it would anyway have picked a random answer. Instead it picks a non-random one, but non-random in such a way that you can't tell without the private key of the watermarking. This mostly adds randomness these days for branches in syntax that make no difference, and the model has no reason to believe make a difference. Anything that matters, it is much more confident in the last layer of weights on the token to use.
  - SkyBelow 15d ago
    >That's not the case, because LLMs are non-deterministic. That feels a bit like a lie. At the core, they are deterministic. We found that adding some ability to randomly pick the second or third best tokens made for better output, so we added temperature. And then we started running them in optimized ways where your answer is deterministic only if the batch of tokens are the same (not your input tokens, but other tokens in another batch being processed), and in practice those are never the same. Lastly, we use harnesses that do things like adding IDs and timestamps to the context, which means the same exact text from the user does not lead to the same text hitting the AI. The final result is that, in practice, you are right (unless you run a model fully locally, where you can seed temperature and turn off all these other features). But strictly calling it non-deterministic makes it sound like the underlying algorithm is itself non-deterministic (and I've seen many people with that misunderstanding) rather than it being a result of how we purposefully changed the algorithm for better results. A bit like saying path finding is non-deterministic, because having the best pathfinding makes for poor gameplay, so we added some randomness to NPC path finding to make it more realistic. The given implementation is non-deterministic, but the underlying algorithm isn't.
- NewsaHackO 15d ago
  I feel as though you are overlooking simple statistics/confidence intervals. It absolutely possible for two different works to be not have a distinguishable difference in quality.
  - sejje 15d ago
    Agreed, but not if one of them was altered to contain a secret message. That one will have a distinguishable difference in quality. Maybe (almost certainly) negligible, but still there.
    - joshuamorton 15d ago
      This...isn't true. Others have already said this, but the watermarking is something like "when the model flips a coin picking between two values, always choose heads". It was already flipping a coin. You're not choosing a less good result, you're just using a deterministic process when it was stochastic before. This will have some impact on outputs, but unless you have some reason to believe that always picking tails was better than always picking heads (in which case, you should be working at one of these companies in model training!) it won't have any impact on output quality.
      - theshrike79 15d ago
        And in this case the model is flipping a million coins. Just one of them is always set to "heads". People saying they can tell from the output are just huffing glue.
- itemize123 15d ago
  that's fair. but it outputted non-best version in the first place sometimes too. Now it still output non-best version sometimes.
  - sejje 15d ago
    Now it outputs the non-best version every time.

sejje 15d ago

That's not the metric, though. You either output the best version, or you output something else. You can't do both.

frabcus 15d ago
That's not the case, because LLMs are non-deterministic. It only alters outputs when the last layer of the neural network give significant weights to multiple tokens, and it would anyway have picked a random answer. Instead it picks a non-random one, but non-random in such a way that you can't tell without the private key of the watermarking. This mostly adds randomness these days for branches in syntax that make no difference, and the model has no reason to believe make a difference. Anything that matters, it is much more confident in the last layer of weights on the token to use.
- SkyBelow 15d ago
  >That's not the case, because LLMs are non-deterministic. That feels a bit like a lie. At the core, they are deterministic. We found that adding some ability to randomly pick the second or third best tokens made for better output, so we added temperature. And then we started running them in optimized ways where your answer is deterministic only if the batch of tokens are the same (not your input tokens, but other tokens in another batch being processed), and in practice those are never the same. Lastly, we use harnesses that do things like adding IDs and timestamps to the context, which means the same exact text from the user does not lead to the same text hitting the AI. The final result is that, in practice, you are right (unless you run a model fully locally, where you can seed temperature and turn off all these other features). But strictly calling it non-deterministic makes it sound like the underlying algorithm is itself non-deterministic (and I've seen many people with that misunderstanding) rather than it being a result of how we purposefully changed the algorithm for better results. A bit like saying path finding is non-deterministic, because having the best pathfinding makes for poor gameplay, so we added some randomness to NPC path finding to make it more realistic. The given implementation is non-deterministic, but the underlying algorithm isn't.
NewsaHackO 15d ago
I feel as though you are overlooking simple statistics/confidence intervals. It absolutely possible for two different works to be not have a distinguishable difference in quality.
- sejje 15d ago
  Agreed, but not if one of them was altered to contain a secret message. That one will have a distinguishable difference in quality. Maybe (almost certainly) negligible, but still there.
  - joshuamorton 15d ago
    This...isn't true. Others have already said this, but the watermarking is something like "when the model flips a coin picking between two values, always choose heads". It was already flipping a coin. You're not choosing a less good result, you're just using a deterministic process when it was stochastic before. This will have some impact on outputs, but unless you have some reason to believe that always picking tails was better than always picking heads (in which case, you should be working at one of these companies in model training!) it won't have any impact on output quality.
    - theshrike79 15d ago
      And in this case the model is flipping a million coins. Just one of them is always set to "heads". People saying they can tell from the output are just huffing glue.
itemize123 15d ago
that's fair. but it outputted non-best version in the first place sometimes too. Now it still output non-best version sometimes.
- sejje 15d ago
  Now it outputs the non-best version every time.