5 ms·

The issue is that no one fully understands why synaptic pruning occurs in biology. Large language models have no direct connection to biological systems, and pr

The issue is that no one fully understands why synaptic pruning occurs in biology. Large language models have no direct connection to biological systems, and pruning in LLMs is no exception.

zug_zug 2y ago
Really? It seems obvious to me. During the learning stage we want input from every variable so that we are sure that we don't omit a variable that turns out to be essential for the calculation. However in any calculation a human does 99.9999% of variables are irrelevant (e.g. what day of the week it is, am I sleepy, etc), so of course the brain wouldn't use resources to keep connections that aren't relevant to a given function. Imagine what a liability it would be if we have had excessive direct connections from our visual processing system to the piece of our brain that controls heartrate.
- idiotsecant 2y ago
  We can convince ourselves of a lot of things that 'seem obvious'. The pesky thing is that sometimes those obvious facts have the temerity to be untrue. That's why we try to understand systems instead of believing obvious things.
- xpuente 2y ago
  As far as I know, pruning is related to age. At birth, we have a massive number of silent synapses. As we grow older, those that remain unused (i.e., inactive) tend to disappear. This process involves a delicate mechanism, including components of the immune system. The unfortunate reality is that no one truly understands how memory works. Many theories are floating around, but the fundamental components remain elusive. One thing is certain: it is quite different from backpropagation. Thankfully, our brains do not suffer from catastrophic forgetting.
zamalek 2y ago
In ANNs pruning helps prevent over-fitting. With the discovery that transformers lack reasoning capabilities this research really comes at a great time. It's a miniscule chance, but we might see this improve performance over the long term and further research.
- Workaccount2 2y ago
  >With the discovery that transformers lack reasoning capabilities The only paper I have seen claiming this studied only lightweight open-source models (<27B, mostly 2B and 8B). The also included o1 and 4o for reference, which kind of broke their hypothesis, but they just left that part out of the conclusion. Not even kidding, their graphs show o1 and 4o having strong performance in their benchmarks, but the conclusion just focuses on 2B and 7B models like gemma and qwen.
  - zamalek 2y ago
    https://arxiv.org/abs/2410.05229 https://arxiv.org/abs/2410.05229 An 18% drop in accuracy (figure 8) is not insignificant. Even 4o suffered 10% loss (figure 6), and 4o isn't a small llm. Competent performance should have near zero performance loss. The simplest benchmark merely changes things like "john had 4 apples" to "Mary had 4 oranges." Performance loss due to inconsequential tokens changing is the very definition of over-fitting.
    - Workaccount2 2y ago
      It's clear though that as the models get bigger and more advanced, their "reasoning" benchmark results improve. The conclusion though just focuses on the bottom tier models. The fact they even set out to create an LLM benchmark and only focus on bottom tier models itself is ridiculous. The authors did the equivalent of "Lets design a human intelligence benchmark, and use a bunch of 12 year olds as reference points" I will eat my hat if the authors rescind the paper in a year or so if their benchmarks show no difference on SOTA models.
    - jdietrich 2y ago
      I just don't see how anyone can see a study comparing the reasoning abilities of various LLMs, see that large LLMs have better reasoning abilities and conclude that LLMs can't reason. LLMs don't have human-like reasoning abilities, but it's just obviously true that they have some capacity for reasoning; that ability seems to scale roughly linearly with model size and training FLOPs.
d0mine 2y ago
A number of things that work for biological systems (humans) work for LLMs too: - after the answer, ask it "are you sure?" (from the office tv series: "is it a stupid thing to do? if it is, don't do it") - chain of thought, step-by-step thinking - different hats (godfather style: piecetime vs. wartime consigliere): looking at the problem from different points of view (at the same time or in stages). For example, first draft: stream of consciousness answer, second iteration: critic/editor/reviewer (produces comments), third (address comments), repeat for some time - collaborative work of different experts(MoE), delegate specific tasks to specialists - [deliberate] practice with immediate feedback

The issue is that no one fully understands why synaptic pruning occurs in biology. Large language models have no direct connection to biological systems, and pruning in LLMs is no exception.

zug_zug 2y ago

Really? It seems obvious to me. During the learning stage we want input from every variable so that we are sure that we don't omit a variable that turns out to be essential for the calculation. However in any calculation a human does 99.9999% of variables are irrelevant (e.g. what day of the week it is, am I sleepy, etc), so of course the brain wouldn't use resources to keep connections that aren't relevant to a given function. Imagine what a liability it would be if we have had excessive direct connections from our visual processing system to the piece of our brain that controls heartrate.

idiotsecant 2y ago
We can convince ourselves of a lot of things that 'seem obvious'. The pesky thing is that sometimes those obvious facts have the temerity to be untrue. That's why we try to understand systems instead of believing obvious things.
xpuente 2y ago
As far as I know, pruning is related to age. At birth, we have a massive number of silent synapses. As we grow older, those that remain unused (i.e., inactive) tend to disappear. This process involves a delicate mechanism, including components of the immune system. The unfortunate reality is that no one truly understands how memory works. Many theories are floating around, but the fundamental components remain elusive. One thing is certain: it is quite different from backpropagation. Thankfully, our brains do not suffer from catastrophic forgetting.

zamalek 2y ago

In ANNs pruning helps prevent over-fitting. With the discovery that transformers lack reasoning capabilities this research really comes at a great time. It's a miniscule chance, but we might see this improve performance over the long term and further research.

Workaccount2 2y ago
>With the discovery that transformers lack reasoning capabilities The only paper I have seen claiming this studied only lightweight open-source models (<27B, mostly 2B and 8B). The also included o1 and 4o for reference, which kind of broke their hypothesis, but they just left that part out of the conclusion. Not even kidding, their graphs show o1 and 4o having strong performance in their benchmarks, but the conclusion just focuses on 2B and 7B models like gemma and qwen.
- zamalek 2y ago
  https://arxiv.org/abs/2410.05229 https://arxiv.org/abs/2410.05229 An 18% drop in accuracy (figure 8) is not insignificant. Even 4o suffered 10% loss (figure 6), and 4o isn't a small llm. Competent performance should have near zero performance loss. The simplest benchmark merely changes things like "john had 4 apples" to "Mary had 4 oranges." Performance loss due to inconsequential tokens changing is the very definition of over-fitting.
  - Workaccount2 2y ago
    It's clear though that as the models get bigger and more advanced, their "reasoning" benchmark results improve. The conclusion though just focuses on the bottom tier models. The fact they even set out to create an LLM benchmark and only focus on bottom tier models itself is ridiculous. The authors did the equivalent of "Lets design a human intelligence benchmark, and use a bunch of 12 year olds as reference points" I will eat my hat if the authors rescind the paper in a year or so if their benchmarks show no difference on SOTA models.
  - jdietrich 2y ago
    I just don't see how anyone can see a study comparing the reasoning abilities of various LLMs, see that large LLMs have better reasoning abilities and conclude that LLMs can't reason. LLMs don't have human-like reasoning abilities, but it's just obviously true that they have some capacity for reasoning; that ability seems to scale roughly linearly with model size and training FLOPs.

d0mine 2y ago

A number of things that work for biological systems (humans) work for LLMs too: - after the answer, ask it "are you sure?" (from the office tv series: "is it a stupid thing to do? if it is, don't do it") - chain of thought, step-by-step thinking - different hats (godfather style: piecetime vs. wartime consigliere): looking at the problem from different points of view (at the same time or in stages). For example, first draft: stream of consciousness answer, second iteration: critic/editor/reviewer (produces comments), third (address comments), repeat for some time - collaborative work of different experts(MoE), delegate specific tasks to specialists - [deliberate] practice with immediate feedback