5 ms·
reminds me of the anthropic's recent work on identifying the neuron sets that correlate to various semantic concepts in Claude: https://news.ycombinator.com/ite
by 100ideas 2y ago
reminds me of the anthropic's recent work on identifying the neuron sets that correlate to various semantic concepts in Claude: https://news.ycombinator.com/item?id=40429540 https://news.ycombinator.com/item?id=40429540 "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet"
- cabidaher 2y agoIn the same vein, Refusal in LLMs is mediated by a single direction: https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in-llms-is-mediated-by-a-single-direction https://www.lesswrong.com/posts/jGuXSZgv6qfdhMCuJ/refusal-in...
- szvsw 2y agoOpenAI also just published similar work, though Anthropic did beat them to the punch. https://openai.com/index/extracting-concepts-from-gpt-4/ https://openai.com/index/extracting-concepts-from-gpt-4/ https://news.ycombinator.com/item?id=40599749 https://news.ycombinator.com/item?id=40599749