7 ms·

TopoNets: High performing vision and language models with brain-like topography

225 points by mayukhdeb 2y ago

LZ_Khan 2y ago
Shouldn't there be a comparison in performance on common benchmarks to other models? Like a 7B toponet model vs a 7B Llama model? As a layperson I don't understand why topology is a thing to optimize for.
- TOMDM 2y ago
  The only potential benefit shown in the paper is the topologically local models seem to be more resilient after pruning. So you may be able to prune a 7B model down to 6B while maintaining most of the capability.
  - mayukhdeb 2y ago
    > The only potential benefit Other benefits: 1. Significantly lower dimensionality of internal representations 2. More interpretable (see: https://toponets.github.io https://toponets.github.io) > 7B model down to 6B We remove ~80% of the parameters in topographic layers and retain the same performance in the model. The drop in parameter count is not significant because we did not experiment with applying TopoLoss in all of the layers of the model (did not align with the goal of the paper) We are currently performing those strong sparsity experiments internally, and the results look very promising!
akokanka 2y ago
[dead]
slama 2y ago
The title here doesn't seem to match. The paper is called "TopoNets: High Performing Vision and Language Models with Brain-Like Topography" Even with their new method, models with topography seem to perform worse than models without.
- dang 2y ago
  Submitted title was "Inducing brain-like structure in GPT's weights makes them parameter efficient". We've reverted it now in keeping with the site guidelines (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html). Since the submitter appears to be one of the authors, maybe they can explain the connection between the two titles? (Or maybe they already have! I haven't read the entire thread)
  - mayukhdeb 2y ago
    Thanks for clarifying your reason for renaming the title. The explanation for the original title is this plot from our publication in ICLR 2025: https://toponets.github.io/webpage_assets/FigureEfficiencyNanoGPTAppendix.png https://toponets.github.io/webpage_assets/FigureEfficiencyNa... You can find more details on the website: https://toponets.github.io https://toponets.github.io (see section: "Toponets deliver sparse, parameter-efficient language models") We find out that inducing topographic structure in the weights of GPTs made them compressible (during inference) without losing out on performance. I encourage you to revert the name if you find it justified after looking into the evidence I've shown here. Thanks.
energy123 2y ago
The main reason topography emerges in physical brains is because spatially distant connections are physically difficult and expensive in biological systems. Artificial neural nets have no such trade-off. So what's the motivation here? I can understand this might be a very good regularizer, so it could help with generalization error on small-data tasks. But hard to see why this should be on the critical path to AGI. As compute and data grows, you want less inductive bias. For example, CNN will beat ViT on small data tasks, but that flips with enough scale because ViT imposes less inductive bias. Or at least any inductive bias should be chosen because it models the structure of the data well, such as with causal transformers and language.
- xpl 2y ago
  > So what's the motivation here? Better interpretability, I suppose. Could give insights into how cognition works.
  - energy123 2y ago
    That's true, and interpretability is helpful for AI safety.
    - mayukhdeb 2y ago
      Indeed. What's cool is that we were able to localize literal "regions" in the GPTs which encoded toxic concepts related to racism, politics, etc. A similar video can be found here: https://toponets.github.io https://toponets.github.io More work is being done on this as we speak.
      - cwillu 2y ago
        Was it toxicity though as understood by the model, or just a cluster of concepts that you've chosen to label as toxic? I.e., is this something that could (and therefore, will) be turned towards identifying toxic concepts as understood by the chinese or us government, or to identify (say) pro-union concepts so they can be down-weighted in a released model, etc?
light_hue_1 2y ago
They bury the part where inducing brain like structure hurts performance! This is a method to just hurt your network in exchange for nothing useful at all aside from some sketchy story that this is "brain like".
- mayukhdeb 2y ago
  Our goal was never to optimize for performance. There's a long standing hypothesis that topographic structure in the human brain leads to metabolic efficiency. Thanks to topography in ANNs, we were able to test out this hypothesis in a computational setting. > sketchy story this is "brain like". we reproduce the hallmarks of functional organization seen in the visual and language cortex of the brain. I encourage you to read the paper before making such comments
  - light_hue_1 2y ago
    I did read the paper. I really hope I don't get assigned to be a reviewer for it. You don't reproduce anything about the functional organization of the visual or language cortex. You make a pretty picture with blobs in it. And one that's trivial to get from current methods. If you think "the functional organization of the visual or language system" means random blobs of activation/connectivity, well, then it's time for a class on neuroscience. I cannot imagine what neuroscientist would let this fly reviewing the paper. The whole "we don't optimize for performance" is nonsense. Take any modern method that prunes weights and beats your approach with ease. Then smooth its output a bit to make nice blobs. The performance loss from smoothing will still beat your method and look "brain-like" by your definition. There you go. Your experiments don't show anything at all aside from the fact that a bad method performs poorly. You didn't think through controls or alternative hypotheses. You didn't take into account a decade of research on methods to prune networks. You don't take seriously what we know about functional organization in the brain. All sorts of bad papers make it through reviewing these days. But.. you can definitely do better. Good luck!
- chaumaha 2y ago
  [dead]
vessenes 2y ago
I hate to dog on research papers. They’re work to write. That said, I think this paper is not likely to be of interest to AI researchers — instead it may be of interest to Neuroscience folks or other brain research types. The lede — adding topography worsens networks at similar weights — is not only buried, it’s obscured with statements claiming that topo networks show less upheaval when scaled down, e.g. they are more efficient than similar weight networks. It’s hard for me to see how both these things can be true — the graphs show the more topography is added, the worse the networks perform at the trained model sizes. To have the second statement “They compress better and are therefore more efficient” also be true, I think you’d need to show a pretty remarkable claim, which is that while a model trained at the same scale as a llama architecture is worse, when you scale them both down, this model becomes not only better than the scaled down llama, but also better than a natively trained model at the new smaller scale. There is no proof of this in the paper, and good reason to be skeptical of this idea based on the data presented. That said, like a lot of ideas in AI, this .. works! You can train a model successfully imposing these outside structures on it, and that model doesn’t even suck very much. Which is a cool statement about complexity theory and the resilience of these architectures, in my opinion. But I don’t think it says much else about either the brain or underlying AI ‘truths’.
- deleted 2y ago
  [deleted]
michalsustr 2y ago
The blurring in the sheets and the topo loss reminded me of https://arxiv.org/abs/2408.05446 https://arxiv.org/abs/2408.05446
FrereKhan 2y ago
This paper imports an arbitrarily-chosen aspect of cortical architecture — topological maps of function — and ignores every other aspect of biological neural tissue. The resulting models show lower performance for the same number of parameters — not surprising, since they are more constrained compared with baseline. They may be slightly more robust against pruning — not surprising, since they are more regularised. The figures show individual seeds, presumably, with no statistical analysis in the performance or pruning comparisons, so the null hypothesis is there is no difference between toponets and baseline. I would never let this paper be submitted by my team. We haven't learned anything about the brain, or about ANNs.
- brrrrrm 2y ago
  this paper plays into some popular fantasy about the aesthetic of ANNs. it’s not scientifically useful
  - mayukhdeb 2y ago
    If by popular fantasy you mean replicating the functional profiles of the visual and language cortex of the brain, then yes. These ideas in neuroscience are popular, but not fantasy. I encourage you to read up on functional organization in the brain, it's very fascinating. > it’s not scientifically useful Having structured weights in GPTs enables us to localize and control various concepts and study stuff like polysemanticity, superposition, etc. Other scientific directions include sparse inference (already proven to work) and better model editing. Turns out, topographic structure also helps these models better predict neural data, which is yet another direction we're exploring in computational neuroscience.
    - photon_lines 2y ago
      I love the paper - don't read into the negative comments. I find that a lot of online feedback (more so on Reddit and much less so on HN (usually)) tends to be opinionated and misinformed by quite a bit these days. Fantastic work and fantastic read.
devmor 2y ago
Is this "brain-like" in any functional way, or "brain-like" in the same way that a tall rectangle is "door-like" even if it doesn't share any functions with a door? I know quite a bit about machine learning, but very little to nothing about neuroscience and human cognition, so I am curious how an expert (that didn't work on the paper) would describe it. (Forgive me for the pre-emptive negativity but I am so utterly exhausted by dishonest comparisons to sapient thought in the field of artificial intelligence that it has nearly drained me of the incredible amount of enthusiasm I used to carry for it.)
- mayukhdeb 2y ago
  It is indeed brain-like in a functional way. Topographic structure is what enables the brain to have low dimensionality and metabolic efficiency. We find that inducing such structure in neural nets made them have significantly lower dimensionality and also more parameter efficient (After training, we could take advantage of the structure to remove ~80% of the weights in topographic layers without sacrificing performance)
  - devmor 2y ago
    >After training, we could take advantage of the structure to remove ~80% of the weights in topographic layers without sacrificing performance This is really interesting to me. Is it that the structure clustered the neurons in such a way that they didn't need to be weighted because their function were grouped by similar black box properties?
    - mayukhdeb 2y ago
      > Is it that the structure clustered the neurons in such a way that they didn't need to be weighted Yep. Because of the structure, we did not have to compute the output of each weight column and simply copied the outputs of nearby weight columns whose outputs were computed.
      - devmor 2y ago
igleria 2y ago
This is excellent. Since reading https://books.google.de/books/about/Models_of_the_Mind.html?id=4NqPEAAAQBAJ&redir_esc=y https://books.google.de/books/about/Models_of_the_Mind.html?... I've been expecting someone to start looking back into biology to try to move forward. I guess the poster is one of the authors. Kudos!
- deleted 2y ago
  [deleted]
- mayukhdeb 2y ago
  Thank you for your kind words! Indeed. The problem with most AI research today is they simply do trial and error with large amounts of compute. No room for taking inspiration from nature, which is requires more thought and less FLOPS.
gzer0 2y ago
I spent time working with Andrej and the rest of the FSD team back in 2020/2021, and we had plenty of conversations on how human visual processing maps onto our neural network architectures. Our approach—transformer-based attention blocks, multi-scale feature extraction, and temporal fusion—mirrors elements of the biological visual cortex (retina → LGN → V1 → V2 → V4 → IT) which break down raw inputs and integrate them over time. It’s amazing how closely this synthetic perceptual pipeline parallels the way our own brains interpret the world. The key insight we discovered was that explicitly enforcing brain-like topographic organization (as some academic work attempts - such as this one here) isn't necessary - what matters is having the right functional components that parallel biological visual processing. Our experience showed that the key elements of biological visual processing - like hierarchical feature extraction and temporal integration - emerge naturally when you build architectures that have to solve real visual tasks. The brain's organization serves its function, not the other way around. This was validated by the real-world performance of our synthetic visual cortex in the Tesla FSD stack. Link to the 2021 Tesla AI day talk: https://www.youtube.com/live/j0z4FweCy4M?t=3010s https://www.youtube.com/live/j0z4FweCy4M?t=3010s
- chaumaha 2y ago
  [dead]
- lukan 2y ago
  "It’s amazing how closely this synthetic perceptual pipeline parallels the way our own brains interpret the world." It is amazing, that the synthetic pipeline, that was build to mimick the brain, seems to mimick the brain? That sounds a bit tautological and otherwise I doubt we have really understood how our brain exactly interprets the world. In general this is definitely interesting research, but worded like this, it smells a bit hyped to me.
  - trhway 2y ago
    The convolutional kernels in the first levels do converge to Gabors like the ones in V1 (and there were math works in the 90-ies, in neuro research, about optimality of such kernels) so it wouldn't be surprising if higher levels would converge to something that is similar to the higher levels of visual cortex (like hierarchical feature aggregation that is nicely illustrated by deep dreaming and also feels like it can be optimal under reasonable conditions and thus would be expected to emerge).

225 points by mayukhdeb 2y ago

Shouldn't there be a comparison in performance on common benchmarks to other models? Like a 7B toponet model vs a 7B Llama model? As a layperson I don't understand why topology is a thing to optimize for.

TOMDM 2y ago
The only potential benefit shown in the paper is the topologically local models seem to be more resilient after pruning. So you may be able to prune a 7B model down to 6B while maintaining most of the capability.
- mayukhdeb 2y ago
  > The only potential benefit Other benefits: 1. Significantly lower dimensionality of internal representations 2. More interpretable (see: https://toponets.github.io https://toponets.github.io) > 7B model down to 6B We remove ~80% of the parameters in topographic layers and retain the same performance in the model. The drop in parameter count is not significant because we did not experiment with applying TopoLoss in all of the layers of the model (did not align with the goal of the paper) We are currently performing those strong sparsity experiments internally, and the results look very promising!

akokanka 2y ago

[dead]

slama 2y ago

The title here doesn't seem to match. The paper is called "TopoNets: High Performing Vision and Language Models with Brain-Like Topography" Even with their new method, models with topography seem to perform worse than models without.

dang 2y ago
Submitted title was "Inducing brain-like structure in GPT's weights makes them parameter efficient". We've reverted it now in keeping with the site guidelines (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html). Since the submitter appears to be one of the authors, maybe they can explain the connection between the two titles? (Or maybe they already have! I haven't read the entire thread)
- mayukhdeb 2y ago
  Thanks for clarifying your reason for renaming the title. The explanation for the original title is this plot from our publication in ICLR 2025: https://toponets.github.io/webpage_assets/FigureEfficiencyNanoGPTAppendix.png https://toponets.github.io/webpage_assets/FigureEfficiencyNa... You can find more details on the website: https://toponets.github.io https://toponets.github.io (see section: "Toponets deliver sparse, parameter-efficient language models") We find out that inducing topographic structure in the weights of GPTs made them compressible (during inference) without losing out on performance. I encourage you to revert the name if you find it justified after looking into the evidence I've shown here. Thanks.

energy123 2y ago

The main reason topography emerges in physical brains is because spatially distant connections are physically difficult and expensive in biological systems. Artificial neural nets have no such trade-off. So what's the motivation here? I can understand this might be a very good regularizer, so it could help with generalization error on small-data tasks. But hard to see why this should be on the critical path to AGI. As compute and data grows, you want less inductive bias. For example, CNN will beat ViT on small data tasks, but that flips with enough scale because ViT imposes less inductive bias. Or at least any inductive bias should be chosen because it models the structure of the data well, such as with causal transformers and language.

xpl 2y ago
> So what's the motivation here? Better interpretability, I suppose. Could give insights into how cognition works.
- energy123 2y ago
  That's true, and interpretability is helpful for AI safety.
  - mayukhdeb 2y ago
    Indeed. What's cool is that we were able to localize literal "regions" in the GPTs which encoded toxic concepts related to racism, politics, etc. A similar video can be found here: https://toponets.github.io https://toponets.github.io More work is being done on this as we speak.
    - cwillu 2y ago
      Was it toxicity though as understood by the model, or just a cluster of concepts that you've chosen to label as toxic? I.e., is this something that could (and therefore, will) be turned towards identifying toxic concepts as understood by the chinese or us government, or to identify (say) pro-union concepts so they can be down-weighted in a released model, etc?

light_hue_1 2y ago

They bury the part where inducing brain like structure hurts performance! This is a method to just hurt your network in exchange for nothing useful at all aside from some sketchy story that this is "brain like".

mayukhdeb 2y ago
Our goal was never to optimize for performance. There's a long standing hypothesis that topographic structure in the human brain leads to metabolic efficiency. Thanks to topography in ANNs, we were able to test out this hypothesis in a computational setting. > sketchy story this is "brain like". we reproduce the hallmarks of functional organization seen in the visual and language cortex of the brain. I encourage you to read the paper before making such comments
- light_hue_1 2y ago
  I did read the paper. I really hope I don't get assigned to be a reviewer for it. You don't reproduce anything about the functional organization of the visual or language cortex. You make a pretty picture with blobs in it. And one that's trivial to get from current methods. If you think "the functional organization of the visual or language system" means random blobs of activation/connectivity, well, then it's time for a class on neuroscience. I cannot imagine what neuroscientist would let this fly reviewing the paper. The whole "we don't optimize for performance" is nonsense. Take any modern method that prunes weights and beats your approach with ease. Then smooth its output a bit to make nice blobs. The performance loss from smoothing will still beat your method and look "brain-like" by your definition. There you go. Your experiments don't show anything at all aside from the fact that a bad method performs poorly. You didn't think through controls or alternative hypotheses. You didn't take into account a decade of research on methods to prune networks. You don't take seriously what we know about functional organization in the brain. All sorts of bad papers make it through reviewing these days. But.. you can definitely do better. Good luck!
chaumaha 2y ago
[dead]

vessenes 2y ago

I hate to dog on research papers. They’re work to write. That said, I think this paper is not likely to be of interest to AI researchers — instead it may be of interest to Neuroscience folks or other brain research types. The lede — adding topography worsens networks at similar weights — is not only buried, it’s obscured with statements claiming that topo networks show less upheaval when scaled down, e.g. they are more efficient than similar weight networks. It’s hard for me to see how both these things can be true — the graphs show the more topography is added, the worse the networks perform at the trained model sizes. To have the second statement “They compress better and are therefore more efficient” also be true, I think you’d need to show a pretty remarkable claim, which is that while a model trained at the same scale as a llama architecture is worse, when you scale them both down, this model becomes not only better than the scaled down llama, but also better than a natively trained model at the new smaller scale. There is no proof of this in the paper, and good reason to be skeptical of this idea based on the data presented. That said, like a lot of ideas in AI, this .. works! You can train a model successfully imposing these outside structures on it, and that model doesn’t even suck very much. Which is a cool statement about complexity theory and the resilience of these architectures, in my opinion. But I don’t think it says much else about either the brain or underlying AI ‘truths’.

deleted 2y ago
[deleted]

michalsustr 2y ago

The blurring in the sheets and the topo loss reminded me of https://arxiv.org/abs/2408.05446 https://arxiv.org/abs/2408.05446

FrereKhan 2y ago

This paper imports an arbitrarily-chosen aspect of cortical architecture — topological maps of function — and ignores every other aspect of biological neural tissue. The resulting models show lower performance for the same number of parameters — not surprising, since they are more constrained compared with baseline. They may be slightly more robust against pruning — not surprising, since they are more regularised. The figures show individual seeds, presumably, with no statistical analysis in the performance or pruning comparisons, so the null hypothesis is there is no difference between toponets and baseline. I would never let this paper be submitted by my team. We haven't learned anything about the brain, or about ANNs.

brrrrrm 2y ago
this paper plays into some popular fantasy about the aesthetic of ANNs. it’s not scientifically useful
- mayukhdeb 2y ago
  If by popular fantasy you mean replicating the functional profiles of the visual and language cortex of the brain, then yes. These ideas in neuroscience are popular, but not fantasy. I encourage you to read up on functional organization in the brain, it's very fascinating. > it’s not scientifically useful Having structured weights in GPTs enables us to localize and control various concepts and study stuff like polysemanticity, superposition, etc. Other scientific directions include sparse inference (already proven to work) and better model editing. Turns out, topographic structure also helps these models better predict neural data, which is yet another direction we're exploring in computational neuroscience.
  - photon_lines 2y ago
    I love the paper - don't read into the negative comments. I find that a lot of online feedback (more so on Reddit and much less so on HN (usually)) tends to be opinionated and misinformed by quite a bit these days. Fantastic work and fantastic read.

devmor 2y ago

Is this "brain-like" in any functional way, or "brain-like" in the same way that a tall rectangle is "door-like" even if it doesn't share any functions with a door? I know quite a bit about machine learning, but very little to nothing about neuroscience and human cognition, so I am curious how an expert (that didn't work on the paper) would describe it. (Forgive me for the pre-emptive negativity but I am so utterly exhausted by dishonest comparisons to sapient thought in the field of artificial intelligence that it has nearly drained me of the incredible amount of enthusiasm I used to carry for it.)

mayukhdeb 2y ago
It is indeed brain-like in a functional way. Topographic structure is what enables the brain to have low dimensionality and metabolic efficiency. We find that inducing such structure in neural nets made them have significantly lower dimensionality and also more parameter efficient (After training, we could take advantage of the structure to remove ~80% of the weights in topographic layers without sacrificing performance)
- devmor 2y ago
  >After training, we could take advantage of the structure to remove ~80% of the weights in topographic layers without sacrificing performance This is really interesting to me. Is it that the structure clustered the neurons in such a way that they didn't need to be weighted because their function were grouped by similar black box properties?
  - mayukhdeb 2y ago
    > Is it that the structure clustered the neurons in such a way that they didn't need to be weighted Yep. Because of the structure, we did not have to compute the output of each weight column and simply copied the outputs of nearby weight columns whose outputs were computed.
    - devmor 2y ago

igleria 2y ago

This is excellent. Since reading https://books.google.de/books/about/Models_of_the_Mind.html?id=4NqPEAAAQBAJ&redir_esc=y https://books.google.de/books/about/Models_of_the_Mind.html?... I've been expecting someone to start looking back into biology to try to move forward. I guess the poster is one of the authors. Kudos!

deleted 2y ago
[deleted]
mayukhdeb 2y ago
Thank you for your kind words! Indeed. The problem with most AI research today is they simply do trial and error with large amounts of compute. No room for taking inspiration from nature, which is requires more thought and less FLOPS.

gzer0 2y ago

I spent time working with Andrej and the rest of the FSD team back in 2020/2021, and we had plenty of conversations on how human visual processing maps onto our neural network architectures. Our approach—transformer-based attention blocks, multi-scale feature extraction, and temporal fusion—mirrors elements of the biological visual cortex (retina → LGN → V1 → V2 → V4 → IT) which break down raw inputs and integrate them over time. It’s amazing how closely this synthetic perceptual pipeline parallels the way our own brains interpret the world. The key insight we discovered was that explicitly enforcing brain-like topographic organization (as some academic work attempts - such as this one here) isn't necessary - what matters is having the right functional components that parallel biological visual processing. Our experience showed that the key elements of biological visual processing - like hierarchical feature extraction and temporal integration - emerge naturally when you build architectures that have to solve real visual tasks. The brain's organization serves its function, not the other way around. This was validated by the real-world performance of our synthetic visual cortex in the Tesla FSD stack. Link to the 2021 Tesla AI day talk: https://www.youtube.com/live/j0z4FweCy4M?t=3010s https://www.youtube.com/live/j0z4FweCy4M?t=3010s

chaumaha 2y ago
[dead]
lukan 2y ago
"It’s amazing how closely this synthetic perceptual pipeline parallels the way our own brains interpret the world." It is amazing, that the synthetic pipeline, that was build to mimick the brain, seems to mimick the brain? That sounds a bit tautological and otherwise I doubt we have really understood how our brain exactly interprets the world. In general this is definitely interesting research, but worded like this, it smells a bit hyped to me.
- trhway 2y ago
  The convolutional kernels in the first levels do converge to Gabors like the ones in V1 (and there were math works in the 90-ies, in neuro research, about optimality of such kernels) so it wouldn't be surprising if higher levels would converge to something that is similar to the higher levels of visual cortex (like hierarchical feature aggregation that is nicely illustrated by deep dreaming and also feels like it can be optimal under reasonable conditions and thus would be expected to emerge).