PaulPauls

8 ms·

1. 1.▲
   by PaulPauls 2y ago

   Thank you, I too am a big believer and enjoyer of open research. The actual code has clarity that complex research papers were never able to convey to me as well as the actual code could. Regarding the cost I would probably sum it up to rou
2. 2.▲
   by PaulPauls 2y ago

   Thank you very much! I included a section at the bottom that provides a sample bibtex citation. I didn't expect this much attention so I didn't even bother with a License but I'll include a MIT license later today and release
3. 3.▲
   by PaulPauls 2y ago

   Just bad wording from me, trying to combine too much information in 1 sentence. The auxiliary loss is supposed to prevent dead latents from occuring in the first place - therefore "prevent dead latents" - and it is also supposed t
4. 4.▲
   by PaulPauls 2y ago

   I'm very happy you appreciate it - particularly the documentation. Writing the documentation was much harder for me than writing the code so I'm happy it is appreciated. I furthermore downloaded your paper and will read through it
5. 5.▲
   by PaulPauls 2y ago

   So evaluating SAEs - determining which SAE is better at creating the most unique features while being as sparse as possible at the same time - is a very complex topic that is very much at the heart of the current research into LLM interpret
6. 6.▲
   by PaulPauls 2y ago

   Not sure yet to be honest. I'll definitely consider it but I'll reorient myself and what I plan to do next in the coming week. I also planned on maybe starting a simpler project and maybe showing people how to create the full mode
7. 7.▲
   by PaulPauls 2y ago

   Thank you for saying that! I have a much, much harder time documenting everything and writing out each decision in continuous text than actually writing the code. So it took a look time for me to write all of this down - so I'm happy y
8. 8.▲
   Show HN: Llama 3.2 Interpretability with Sparse Autoencoders(github.com)

   579 points by PaulPauls 2y ago|99 comments