5 ms·
I highly question this simplistic idea of high vector magnitude = high influence.
by sva_ 8d ago
I highly question this simplistic idea of high vector magnitude = high influence.
- ifz 8d agoI don't disagree with that. I did add an entire caveat paragraph there. To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.
- smallmancontrov 8d agoYou get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.html https://transformer-circuits.pub/2025/attention-qk/index.htm...
- apnabhidu47 8d agoSame I dont get it just, could you clarify it