4 ms·

Why do you claim that? There is no reason why there has to be a negative effect of text watermarking.

• pkulak 17d ago
  It literally re-weights the output tokens from what the LLM would otherwise have chosen. It _has_ to. It can't be positive, because then that's not watermarking, it's a better LLM.
  • frabcus 17d ago
    It's a very unintuitive algorithm, and is pretty clever. I recommend reading up on it: https://www.nature.com/articles/s41586-024-08025-4 https://www.nature.com/articles/s41586-024-08025-4 But no, it only ever picks tokens that are in the probability distribution of the last layer, and it might have picked anyway.
    • throwuxiytayq 17d ago
      What if the next token represents a wrong or low-quality answer, but would have only been picked 10% of the time, but now it's picked 20% of the time? Doesn't that obviously decrease the model quality, even though "it might have picked that token anyway"?
      • dan-robertson 17d ago
        It would be picked 10% of the time with watermarking. The randomness properties of the PRNG will be very similar to other random number generators, it is just chosen to be vulnerable to a particular cryptanalytic attack (that requires a private key known only to anthropic). I think of it like the Dual_EC_DRGB generator rather than a biased coin.
        • throwuxiytayq 16d ago
          It is absolutely possible that it would not continue to be picked 10% of the time with a given fixed watermark key. The implementation literally labels tokens using a keyed hash and then modifies their scores. The entire point of the watermarking system is to bias certain tokens against others, and - as you would expect - this reportedly results in a reduced response diversity.
          • deleted 16d ago
            [deleted]
      • wasabi991011 16d ago
        What if the token represents a high-quality answer, but would have only been picked 80% of the time, but now it's picked 90% of the time? I'm not entirely sure (haven't read the original synthID proposal), but I believe that the re-weighing is set to make both your scenarios and mine equally likely, averaging out to net Zero effect on quality.
    • dbdr 17d ago
      To add nuance, that article does say one of the two versions does reduce text quality (and the other is worse at detectability): > SynthID-Text can be configured to be non-distortionary (preserving text quality) or distortionary (improving watermark detectability at the cost of text quality).
  • qgin 17d ago
    Unless you’re running at temperature 0, there’s not one single token that the model definitely would have chosen each time.
  • brokencode 17d ago
    Unless you’re at 0 temperature, there is no single token it would have chosen. It’s always picking one of multiple randomly according to a probability distribution.
    • northzen 17d ago
      Give me an example how would you watermark a single short sentence like "I like turtles"?
      • qbit42 17d ago
        Watermarking just alters the pseudorandom number generator. If "I like turtles" was previously the response to your prompt with probability 100%, it will still be so. This is why watermarking is only effective for long strings of text
        • northzen 17d ago
          It's like the sudden change of a language style and its verbosity didn't happen recently. To random words you pick and provide a sufficient amount of text to vary with random number without losing its meaning you need a text with high entropy.
          • qlte 17d ago
            Nothing about watermarking would require padding the response length with pseudo-intelligible Claudese. Regular filler would work fine. Also, it would probably provide higher entropy to write normal human-sounding English instead of reusing a repetitive grab bag of load-bearing phrases. This theory doesn't really make any sense.
  • WithinReason 17d ago
    No, it basically uses a fixed seed for the random number generator to generate tokens. The generated random sequence is just as random as it would be with a random seed.