6 ms·
On a related note: recently, I released a visualization of all MLP neurons inside the llama3 8B model. Here is an example "derivative" neuron which is triggere
by xcodevn 2y ago
On a related note: recently, I released a visualization of all MLP neurons inside the llama3 8B model. Here is an example "derivative" neuron which is triggered when talking about the derivative concept.
https://neuralblog.github.io/llama3-neurons/neuron_viewer.html#24,2362 https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...
- vpj 2y agoPretty cool. The tokens are highlighted based on the activation?
- xcodevn 2y agoYes, you're correct. The tokens are highlighted based on the neuron activation value, which is scaled to a range of 0 to 10.
- skulk 2y agoThis is insanely fun to just flip through. I found a "sex" neuron. https://neuralblog.github.io/llama3-neurons/neuron_viewer.html#21,911 https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...