33 ms·
No, I mean that when constrained (like discussing a specific bug in code) the options are limited. SynthID specifically mentions something similar: "It is hard
by sroussey 14d ago
No, I mean that when constrained (like discussing a specific bug in code) the options are limited.
SynthID specifically mentions something similar: "It is harder to watermark factual answers because the model has fewer alternative word choices available without altering accuracy."
In the codebase, I do see the models finding alternative word choices -- and I hate them. Already in a highly technical latent space, it reaches for other highly technical word choices (which may be more accurate, but ones I am unfamiliar with -- like terms in ERP systems as it felt that was close but bring just technical jargon that i have to google what it is saying because i don't understand -- and i have to google the phase sentence as the words themselves are ok, just not how they are put together).
- a2ff6eeb0 14d agoIt's been making weird word choices for a long time; what trials have you done to check if you can actually tell, or it's just picking weird words?
- sroussey 14d agoAnthropic in particular has been moving the level of watermarking since the beginning of the year. You didn’t think it just went from off to on one day did you? It needs time to get plastered over the internet, see what people do that break it, how Google summaries erase it (or not, or add their own). All of this takes at least months if not a year.
- sroussey 14d agoI can tell as it uses what are super accurate words but i only know that after i lookup the phase. Today, i was stumped as it substituted a work with a word from another language! "The four items the ledger had标 standing open..."