ifz

4 ms·

1. 1.▲
   by ifz 6d ago

   Oh, I actually didn't know about Bertviz before. That looks interesting as well!
2. 2.▲
   by ifz 6d ago

   Thanks for trying it out! You can hover / click / tap on any word to focus it. I actually intended for this to be main way to use the visualization, I just added the auto-playback later on so it's more eye-catching on page lo
3. 3.▲
   by ifz 8d ago

   Hmm, I might try to add some controls to limit which layers get summed up. It might be able to reveal more patterns. Right now only simple correlations are visible.
4. 4.▲
   by ifz 8d ago

   It's really simple, basically just the magnitude of the value vector, weighted by QK dot product, summed across all attention heads and layers. When I started, I expected I'd have to experiment a lot to find something comprehensib
5. 5.▲
   by ifz 8d ago

   I don't disagree with that. I did add an entire caveat paragraph there. To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some intere
6. 6.▲
   Show HN: LLM Attention Visualization(ishamf.dev)

   175 points by ifz 8d ago|32 comments
7. 7.▲
   Show HN: Timezone Converter in Map Form(ishamf.dev)

   2 points by ifz 1y ago|0 comments
8. 8.▲
   by ifz 3y ago

   Nowadays, you can get a free VM from some providers (GCP) and there are lots of sites where you can deploy a static website for free (github pages). I'd argue deploying something for free is easier than ever.