9 ms·

I've been following the whole thing low key since the 2nd wave of neural networks in the mid 90s - and made a very very minor contribution to the field which ha

I've been following the whole thing low key since the 2nd wave of neural networks in the mid 90s - and made a very very minor contribution to the field which has applications these days back then too. My observation is that every wave of neural networks has resulted in a dead end. In my view, this is in large part caused by the (inevitable) brute force mathematical approach used and the fact that this can not map to any kind of mechanistic explanation of what the ANN is doing in a way that can facilitate intuition. Or as put in the article "Current AI systems have no internal structure that relates meaningfully to their functionality". This is the most important thing. Maybe layers of indirection can fix that, but I kind of doubt it. I am however quite excited about what LLMs can do to make semantic search much easier, and impressed at how much better they've made the tooling around natural language processing. Nonetheless, I feel I can already see the dead end pretty close ahead.

pat64 2y ago
Given you just mentioned semantic search (a term I haven’t heard in over 15 years) and the other breadcrumbs in this comment, you wouldn’t by chance be an English lecturer living in Ireland would you?
- singingfish 2y ago
  me? No. Ex trainee neruopsychologist and failed academic who was in the right place at the right time back in the mid 90s who didn't pick up computers for professional interest until the mid-late 2000s after getting excited by Neil Stephenson's Cryptonomicon when I was looking for a career change. These days I identify as an international computer hacker, but mainly to take the piss (due to the tiny element of truth sitting underneath)
ninetyninenine 2y ago
If we ever hit agi this overall point: “ the fact that this can not map to any kind of mechanistic explanation of what the ANN is doing in a way that can facilitate intuition.” Will remain true imho. We will never fully intuit AI or understand it outside of some brute force abstraction like a token predictor or best fit curve.
steve_adams_86 2y ago
I didn’t see this at first, and I was fairly shaken by the potential impact on the world if their progress didn’t stop. A couple generations showed meaningful improvements, but now it seems like you’re probably correct. I’ve used these for years quite intensively to aid my work and while it’s a useful rubber duck, it doesn’t seem to yield much more beyond that. I worry a lot less about my career now. It really is a tool that creates more work for me rather than less.
- iman453 2y ago
  Would this still hold true in your opinion if models like O3 become super cheap and bit better over time? I don't know much about the AI space, but as a vanilla backend dev also worry about the future :)
  - tempodox 2y ago
    No degree of cheapness will be able to offset the “creates more work for me rather than less” part.
  - steve_adams_86 2y ago
    Generally speaking, I think it would. I’m open to being wrong. I think there is a non-trivial amount of hype around O3, and while it would certainly be interesting if it was cheap, I don’t think it would address important issues that AI currently doesn’t seem to even begin to accommodate in its current capacity to recognize or utilize contexts. For example, I have little to no expectation that it will handle software architecture well. Especially refactoring legacy code, where two enormous contexts need to be held in mind at once.
  - root_axis 2y ago
    Let's see how O3 pans out in practice before we start setting it as the standard for the future.
    - varelse 2y ago
      Mamba-ish models are the breakthrough to cheap inference if they pan out. Calling a dead-end already is just silly.
hammock 2y ago
What are your thoughts on neuro-symbolic integration (combining the pattern-recognition capabilities of neural networks with the reasoning and knowledge representation of symbolic AI) ?
- bionhoward 2y ago
  Seems like the symbolic aspect is poorly defined and it’s too unclear to be useful. Always sounds cool, but what exactly are we talking about?
  - hammock 2y ago
    I’m not an AI expert, but from my armchair I might draw a comparison between functional (symbolic rule- and logic-based AI) and declarative (LLM) programming languages
gizmo 2y ago
Previous generations of neural nets were kind of useless. Spotify ended up replacing their machine learning recommender with a simple system that would just recommend tracks that power listeners had already discovered. Machine learning had a couple of niche applications but for most things it didn't work. This time it's different. The naysayers are wrong. LLMs today can already automate many desk jobs. They already massively boost productivity for people like us on HN. LLMs will certainly get better, faster and cheaper in the coming years. It will take time for society to adapt and for people to realize how to take advantage of AI, but this will happen. It doesn't matter whether you can "test AI in part" or whether you can do "exhaustive whole system testing". It doesn't matter whether AIs are capable of real reasoning or are just good enough at faking it. AI is already incredibly powerful and with improved tooling the limitations will matter much less.
- jfengel 2y ago
  From what I have seen, most of the jobs that LLMs can do are jobs that didn't need to be done at all. We should turn them over to computers, and then turn the computers off.
  - kube-system 2y ago
    They're good at processing text. Processing text is a valuable thing that sometimes needs to be done. We still use calculators even though the profession we used to call "computer" was replaced by them.
    - jonasced 2y ago
      But here reliability comes in again. Calculators are different since the output is correct as long as the input is correct. LLMs do not guarantee any quality in the output even when processing text, and should in my opinion be verified before used in any serious applications.
      - rtsil 2y ago
        Neither are humans, that's why we have proofreaders and editors. That doesn't make them any less useful. And a translator will not write the same exact translation for a text longer than a couple of sentences, that does not mean translation is a dead end. Ironically, it's LLMs that made translation a dead end.
dlkf 2y ago
> Current AI systems have no internal structure that relates meaningfully to their functionality In what sense is the relationship between neurons and human function more “meaningful” than the relationship between matrices and LLM function? You’re correct that LLMs are probably a dead end with respect to AGI, but this is completely the wrong reason.
- imtringued 2y ago
  The neurons are always learning whereas the matrices don't change.
  - dlkf 2y ago
    I mean, the matrices obviously change during training. I take it your point is that LLMs are trained once and then frozen, whereas humans continuously learn and adapt to their environment. I agree that this is a critical distinction. But it has nothing to do with “meaningful internal structure.”
- singingfish 2y ago
  Yeah the internal representation of organic neural networks are also weird - check out the signal processing that occurs between the retina and the various parts of the visual cortex before any decent information can emerge from the signal - David Marr's 1980s book Vision is a mathematically chewy treatise on this. This leads me to start thinking that human intuition may well caused by different neural network subsystems feeding processed data into other subsystems where consciousness and thus intuition and explanation emerges. Organic neural networks are pretty energy efficient in comparison- although still decently inefficient compared to other body systems - so there is the capacity to build things out to the scale required, assuming my read on what's going on there is correct, that is. So it's not clear to me that the energy inefficiency of ANNs can be sufficiently resolved to enable these multiple quasi-independent subsystems to be built at the scale required. Not even if these interesting looking trinomial neural nets which are matrix addition based rather than multiplication come to dominate the ANN scene. While I was thinking this comment through I realised there's a possible interpretation wherin human activity induced climate change is an emergent property of the relative energy inefficiency of neural architecture.
ftlisnotftl 2y ago
[dead]
busyant 2y ago
> "Current AI systems have no internal structure that relates meaningfully to their functionality". I'm curious as to why you feel this needs to be true? Or to put it another way, what would an AI structure look like to be more meaningfully connected to its function? Not trying to flame. I always feel that I can't think quite deeply enough about these issues, so I'm worried that I'm missing something 'obvious'.
- singingfish 2y ago
  The reasoning is quite subtle, and because I'm not a very coherent guy I have problems expressing it. In the LLM space there are a whole bunch of pitfalls around overfit (largely solvable with pretty standard statistical methods) and inherent bias in training material which is a much harder to problem to solve. The fact that the internal representation gives you zero information on how to handle this bias means the tool can itself not be used to detect or resolve the problem. I found this episode of the nature podcast - "How AI works is often a mystery — that's a problem": https://www.nature.com/articles/d41586-023-04154-4 https://www.nature.com/articles/d41586-023-04154-4 - very useful in a 'thank goodness someone else has done the work of being coherent so I don't have to' way.
  - busyant 2y ago
    Thank you. That's a really interesting (and understandable) explanation.
- fragmede 2y ago
  AlphaGo had an artificial neural network that was specifically trained in best moves and winning percentages. An LLM trained on text has some data on what constitutes winning at go, but internally doesn't have a ANN specifically for the game of go.
  - deleted 2y ago
    [deleted]
  - busyant 2y ago
aoeusnth1 2y ago
Do your kids have internal structures which relate meaningfully to their functionality, which allow a mechanistic explanation of what they learned in school?
- intended 2y ago
  Not sure if this is satirical, but absolutely yes. Heck we have everything from fields of study, to professions that cover this. Neurology, psychology, counseling, teaching, amongst a few. All things being equal, If a kid didn’t pick up a concept, I can sit with them and figure out what happened, and we can both work towards making sure its cleared up.

pat64 2y ago

Given you just mentioned semantic search (a term I haven’t heard in over 15 years) and the other breadcrumbs in this comment, you wouldn’t by chance be an English lecturer living in Ireland would you?

singingfish 2y ago
me? No. Ex trainee neruopsychologist and failed academic who was in the right place at the right time back in the mid 90s who didn't pick up computers for professional interest until the mid-late 2000s after getting excited by Neil Stephenson's Cryptonomicon when I was looking for a career change. These days I identify as an international computer hacker, but mainly to take the piss (due to the tiny element of truth sitting underneath)

ninetyninenine 2y ago

If we ever hit agi this overall point: “ the fact that this can not map to any kind of mechanistic explanation of what the ANN is doing in a way that can facilitate intuition.” Will remain true imho. We will never fully intuit AI or understand it outside of some brute force abstraction like a token predictor or best fit curve.

steve_adams_86 2y ago

I didn’t see this at first, and I was fairly shaken by the potential impact on the world if their progress didn’t stop. A couple generations showed meaningful improvements, but now it seems like you’re probably correct. I’ve used these for years quite intensively to aid my work and while it’s a useful rubber duck, it doesn’t seem to yield much more beyond that. I worry a lot less about my career now. It really is a tool that creates more work for me rather than less.

iman453 2y ago
Would this still hold true in your opinion if models like O3 become super cheap and bit better over time? I don't know much about the AI space, but as a vanilla backend dev also worry about the future :)
- tempodox 2y ago
  No degree of cheapness will be able to offset the “creates more work for me rather than less” part.
- steve_adams_86 2y ago
  Generally speaking, I think it would. I’m open to being wrong. I think there is a non-trivial amount of hype around O3, and while it would certainly be interesting if it was cheap, I don’t think it would address important issues that AI currently doesn’t seem to even begin to accommodate in its current capacity to recognize or utilize contexts. For example, I have little to no expectation that it will handle software architecture well. Especially refactoring legacy code, where two enormous contexts need to be held in mind at once.
- root_axis 2y ago
  Let's see how O3 pans out in practice before we start setting it as the standard for the future.
  - varelse 2y ago
    Mamba-ish models are the breakthrough to cheap inference if they pan out. Calling a dead-end already is just silly.

hammock 2y ago

What are your thoughts on neuro-symbolic integration (combining the pattern-recognition capabilities of neural networks with the reasoning and knowledge representation of symbolic AI) ?

bionhoward 2y ago
Seems like the symbolic aspect is poorly defined and it’s too unclear to be useful. Always sounds cool, but what exactly are we talking about?
- hammock 2y ago
  I’m not an AI expert, but from my armchair I might draw a comparison between functional (symbolic rule- and logic-based AI) and declarative (LLM) programming languages

gizmo 2y ago

Previous generations of neural nets were kind of useless. Spotify ended up replacing their machine learning recommender with a simple system that would just recommend tracks that power listeners had already discovered. Machine learning had a couple of niche applications but for most things it didn't work. This time it's different. The naysayers are wrong. LLMs today can already automate many desk jobs. They already massively boost productivity for people like us on HN. LLMs will certainly get better, faster and cheaper in the coming years. It will take time for society to adapt and for people to realize how to take advantage of AI, but this will happen. It doesn't matter whether you can "test AI in part" or whether you can do "exhaustive whole system testing". It doesn't matter whether AIs are capable of real reasoning or are just good enough at faking it. AI is already incredibly powerful and with improved tooling the limitations will matter much less.

jfengel 2y ago
From what I have seen, most of the jobs that LLMs can do are jobs that didn't need to be done at all. We should turn them over to computers, and then turn the computers off.
- kube-system 2y ago
  They're good at processing text. Processing text is a valuable thing that sometimes needs to be done. We still use calculators even though the profession we used to call "computer" was replaced by them.
  - jonasced 2y ago
    But here reliability comes in again. Calculators are different since the output is correct as long as the input is correct. LLMs do not guarantee any quality in the output even when processing text, and should in my opinion be verified before used in any serious applications.
    - rtsil 2y ago
      Neither are humans, that's why we have proofreaders and editors. That doesn't make them any less useful. And a translator will not write the same exact translation for a text longer than a couple of sentences, that does not mean translation is a dead end. Ironically, it's LLMs that made translation a dead end.

dlkf 2y ago

> Current AI systems have no internal structure that relates meaningfully to their functionality In what sense is the relationship between neurons and human function more “meaningful” than the relationship between matrices and LLM function? You’re correct that LLMs are probably a dead end with respect to AGI, but this is completely the wrong reason.

imtringued 2y ago
The neurons are always learning whereas the matrices don't change.
- dlkf 2y ago
  I mean, the matrices obviously change during training. I take it your point is that LLMs are trained once and then frozen, whereas humans continuously learn and adapt to their environment. I agree that this is a critical distinction. But it has nothing to do with “meaningful internal structure.”
singingfish 2y ago
Yeah the internal representation of organic neural networks are also weird - check out the signal processing that occurs between the retina and the various parts of the visual cortex before any decent information can emerge from the signal - David Marr's 1980s book Vision is a mathematically chewy treatise on this. This leads me to start thinking that human intuition may well caused by different neural network subsystems feeding processed data into other subsystems where consciousness and thus intuition and explanation emerges. Organic neural networks are pretty energy efficient in comparison- although still decently inefficient compared to other body systems - so there is the capacity to build things out to the scale required, assuming my read on what's going on there is correct, that is. So it's not clear to me that the energy inefficiency of ANNs can be sufficiently resolved to enable these multiple quasi-independent subsystems to be built at the scale required. Not even if these interesting looking trinomial neural nets which are matrix addition based rather than multiplication come to dominate the ANN scene. While I was thinking this comment through I realised there's a possible interpretation wherin human activity induced climate change is an emergent property of the relative energy inefficiency of neural architecture.

ftlisnotftl 2y ago

[dead]

busyant 2y ago

> "Current AI systems have no internal structure that relates meaningfully to their functionality". I'm curious as to why you feel this needs to be true? Or to put it another way, what would an AI structure look like to be more meaningfully connected to its function? Not trying to flame. I always feel that I can't think quite deeply enough about these issues, so I'm worried that I'm missing something 'obvious'.

singingfish 2y ago
The reasoning is quite subtle, and because I'm not a very coherent guy I have problems expressing it. In the LLM space there are a whole bunch of pitfalls around overfit (largely solvable with pretty standard statistical methods) and inherent bias in training material which is a much harder to problem to solve. The fact that the internal representation gives you zero information on how to handle this bias means the tool can itself not be used to detect or resolve the problem. I found this episode of the nature podcast - "How AI works is often a mystery — that's a problem": https://www.nature.com/articles/d41586-023-04154-4 https://www.nature.com/articles/d41586-023-04154-4 - very useful in a 'thank goodness someone else has done the work of being coherent so I don't have to' way.
- busyant 2y ago
  Thank you. That's a really interesting (and understandable) explanation.
fragmede 2y ago
AlphaGo had an artificial neural network that was specifically trained in best moves and winning percentages. An LLM trained on text has some data on what constitutes winning at go, but internally doesn't have a ANN specifically for the game of go.
- deleted 2y ago
  [deleted]
- busyant 2y ago

aoeusnth1 2y ago

Do your kids have internal structures which relate meaningfully to their functionality, which allow a mechanistic explanation of what they learned in school?

intended 2y ago
Not sure if this is satirical, but absolutely yes. Heck we have everything from fields of study, to professions that cover this. Neurology, psychology, counseling, teaching, amongst a few. All things being equal, If a kid didn’t pick up a concept, I can sit with them and figure out what happened, and we can both work towards making sure its cleared up.