5 ms·
How do you determine that the model was reasoning in Chinese in layer X? I would think the middle layers do not map into any tokens.
by ydj 4mo ago
How do you determine that the model was reasoning in Chinese in layer X? I would think the middle layers do not map into any tokens.
- s314 4mo agoUsing a logit lens (prior art: https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreti...)