6 ms·

> processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations If the workings of those

> processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations If the workings of those circuits are obvious to you, I'd really like to learn. Do you mean the level of analysis at https://transformer-circuits.pub/ https://transformer-circuits.pub/ ? (That looks like good work but not a deep understanding.) Hofstadter referenced this back in the day as a promising beginning: https://en.wikipedia.org/wiki/Sparse_distributed_memory https://en.wikipedia.org/wiki/Sparse_distributed_memory which sounds kind of similar in style to the embeddings you bring up.

HarHarVeryFunny 5d ago
The predictive nature of our cortex seems even more direct that that, although perhaps obvious is overstating it. To read about the 6-layer circuit of our cortex, google for "cortical microcircuit". Basically the repeating structure of our cortex (across all regions) is this vertical connectivity pattern ("circuit") between the 6 layers, and also the connections coming into and out of the cortex at the top and bottom of those layers. There are also horizontal connections within layers of the cortex, but these are short-distance/localized likely for coordinating activity with a local region (cortical column). Our cortex is connected in a loop with our thalamus (cf recent talk of looped transformers!), which can be separated into thalamic core and matrix. Basically our senses feed into the thalamic core, and cortical outputs (predictions) feed into the thalamic matrix. The thalamic connections/loop seem to act a bit like a patch panel connecting different regions of the cortex. The cortical microcircuit is a real physical circuit - a pattern of synaptic connections between layers and inputs/outputs. Those Transformer circuits are more functional than physical. We know how a Transformer is physically connected, with attention heads etc, and this "circuit" analysis then describes how it functions, e.g. with attention heads in adjacent transformer layers effectively combining to form "induction heads" that copy data across embeddings.

HarHarVeryFunny 5d ago

The predictive nature of our cortex seems even more direct that that, although perhaps obvious is overstating it. To read about the 6-layer circuit of our cortex, google for "cortical microcircuit". Basically the repeating structure of our cortex (across all regions) is this vertical connectivity pattern ("circuit") between the 6 layers, and also the connections coming into and out of the cortex at the top and bottom of those layers. There are also horizontal connections within layers of the cortex, but these are short-distance/localized likely for coordinating activity with a local region (cortical column). Our cortex is connected in a loop with our thalamus (cf recent talk of looped transformers!), which can be separated into thalamic core and matrix. Basically our senses feed into the thalamic core, and cortical outputs (predictions) feed into the thalamic matrix. The thalamic connections/loop seem to act a bit like a patch panel connecting different regions of the cortex. The cortical microcircuit is a real physical circuit - a pattern of synaptic connections between layers and inputs/outputs. Those Transformer circuits are more functional than physical. We know how a Transformer is physically connected, with attention heads etc, and this "circuit" analysis then describes how it functions, e.g. with attention heads in adjacent transformer layers effectively combining to form "induction heads" that copy data across embeddings.