Trinicode

5 ms·

1. 1.▲
   by Trinicode 7mo ago

   A letter is not a token, is it? Redundancy could hit 75% in long sentences, but Shannon was not predicting tokens or words, he was predicting letters (characters).
2. 2.▲
   Low-entropy tokens tolerate substitution with and 0.1 PPL cost across models

   2 points by Trinicode 7mo ago|0 comments