4 ms·

“ Wafer level faults probably won't matter though - neural nets are resistant to a few missing or wrong weights.” Brain science people “love” traumatic brain i

“ Wafer level faults probably won't matter though - neural nets are resistant to a few missing or wrong weights.” Brain science people “love” traumatic brain injury cases because it can help explore what happens when bits of the “brain wafer” get damaged. We’ve learned a lot from such things. I wonder if people are intentionally “destroying” parts of the model weights to learn more about what happens? Like could you strategically wipe a gig of the model so it’s “all zeros” and see what happens? I have to wonder

zurfer 3mo ago
This is called mechanistic interpretability. There is lots of fascinating insights already since you can do basically everything down to the neuron or weight level thousands of times. The human brain is many orders of magnitude harder to make sense of.
- sometimelurker 3mo ago
  well its actually called ablation, and its one way to do mech interp. anthriopics got a bunch of work on mech interp here https://transformer-circuits.pub/ https://transformer-circuits.pub/, like SAEs and NLAs
Computer0 3mo ago
Reminds me of Golden Gate Claude (https://www.anthropic.com/news/golden-gate-claude https://www.anthropic.com/news/golden-gate-claude)
Cantinflas 3mo ago
Somehow related: https://github.com/elder-plinius/OBLITERATUS https://github.com/elder-plinius/OBLITERATUS
mdp2021 3mo ago
Of course tampering with chunks or nodes in the NNs is a way to study the "spawned" (through gradient descent etc.) configuration and "reverse-engineer the black box" to get "AI transparency". Anthropic published an important work around one year and a half ago.
- mdp2021 3mo ago
  > Anthropic published an important work around one year and a half ago > #Tracing the thoughts of a large language model# https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language... https://news.ycombinator.com/item?id=43495617 https://news.ycombinator.com/item?id=43495617 (27 March 2025)

zurfer 3mo ago

This is called mechanistic interpretability. There is lots of fascinating insights already since you can do basically everything down to the neuron or weight level thousands of times. The human brain is many orders of magnitude harder to make sense of.

sometimelurker 3mo ago
well its actually called ablation, and its one way to do mech interp. anthriopics got a bunch of work on mech interp here https://transformer-circuits.pub/ https://transformer-circuits.pub/, like SAEs and NLAs

Computer0 3mo ago

Reminds me of Golden Gate Claude (https://www.anthropic.com/news/golden-gate-claude https://www.anthropic.com/news/golden-gate-claude)

Cantinflas 3mo ago

Somehow related: https://github.com/elder-plinius/OBLITERATUS https://github.com/elder-plinius/OBLITERATUS

mdp2021 3mo ago

Of course tampering with chunks or nodes in the NNs is a way to study the "spawned" (through gradient descent etc.) configuration and "reverse-engineer the black box" to get "AI transparency". Anthropic published an important work around one year and a half ago.

mdp2021 3mo ago
> Anthropic published an important work around one year and a half ago > #Tracing the thoughts of a large language model# https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language... https://news.ycombinator.com/item?id=43495617 https://news.ycombinator.com/item?id=43495617 (27 March 2025)