7 ms·

fortunately i wrote an entire post about what the difference is between the parts of this that it is easy to make sense of and the parts of it that it is prohib

fortunately i wrote an entire post about what the difference is between the parts of this that it is easy to make sense of and the parts of it that it is prohibitively difficult to make sense of and it was posted on hackernews

sirwhinesalot 1y ago
Your article, unlike the bizarre desperate take from the poster above, is actually very good. We do not understand the features the neural net learns, that's 100% true (and really the whole point of them in the first place). For small image recognition models we can visualize them and get an intuition for what they are doing, but it doesn't really matter. For even smaller models, we can translate them to a classical AI model (like a mixed integer program as an example) and actually do various "queries" on the model itself to, e.g., learn that the network recognizes the number "8" by just checking 2 pixels in the image. None of this changes the fact that we know what these things are and how they work, because we built them. Any comparisons to our lack of knowledge of the human brain are ridiculous. LLMs are obviously not conscious, they don't even have real "state", they're an approximated pure function f(context: List<Token>) -> Token, that's run in a loop. The only valid alarmist take is that we're using black box algorithms to make decisions with serious real-world impact, but this is true of any black box algorithm, not just the latest and greatest ML models.
- dpoloncsak 1y ago
  Its a complex adaptive system, right? Isn't that the whole idea? We know how each part of the system works by itself. We know all inputs, and can measure outputs. I still (even if I actually understood the math) cannot tell you 'If you prompt 'x', the model will return 'y' with 100% confidence.
  - sirwhinesalot 1y ago
    > If you prompt 'x', the model will return 'y' with 100% confidence. We can do this for smaller models. Which means it's a problem of scale/computing power rather than a fundamental limitation. The situation with the human brain is completely different. We know neurons exchange information and how that works, and we have a pretty good understanding of the architecture of parts of the brain like the visual cortex, but we have no idea of the architecture as a whole. We know the architecture of an LLM. We know how the data flows. We know what it is the individual neurons are learning (cuts and bends of a plane in a hyperdimensional space). We know how the weights are learned (backpropagation). We know the "algorithm" the LLM as a whole is approximating (List<Token> -> Token). Yes there are emergent properties we don't understand but the same is true of a spam filter. Comparing this to our lack of understanding of the human brain and discussing how these things might be "conscious" is silly.
    - ninetyninenine 1y ago
      >Comparing this to our lack of understanding of the human brain and discussing how these things might be "conscious" is silly. Don't call my claim silly. I'm sick of your attitude. Why can't you have a civil discussion? Literally we don't know. You can't make a claim that it's silly when you can't even define what consciousness is. You don't know how human brains work, you don't know how consciousness forms, you don't know how emergence in LLMs work. So your claim here is logically just made up out of thin air. Sure we "understand" LLMs from the curve fitting perspective. But the entirety of why we use LLMs and what we use it for arises from the emergence which is what we don't understand. Curve fitting is like 1% of the LLM, it is the emergent properties we completely don't get (99%) and take advantage of on a daily basis. Curve fitting is just a high level concept that allows us to construct the algorithm which is the actual thing that does the hard work of wiring up the atomic units of the network. >Yes there are emergent properties we don't understand but the same is true of a spam filter. Yeah and so? Your statement proves nothing. It just illustrates a contrast in sentiment. The spam filter is a trivial thing, the human brain is not. We don't understand the spam filter. And this is the most interesting part of it all is that the SAME scaling problem that prevents us from understanding the spam filter can be characterized as the reason that prevents us from understanding BOTH the LLM and the human brain. Your statement doesn't change anything. It's just using sentiment to try to re-characterize a problem in a different light.
      - Suppafly 1y ago
        >Don't call my claim silly. Not the other guy, but your claims are silly. People can't have a civil discussion with you because you post nonsense.
        ninetyninenine 1y ago
        Civil discussion is part of human decency. It has nothing to do with the other party. Whether you're a good person or a shit head has nothing to do with whether some 3rd part is talking nonsense. People have different opinions all the time and people all the time think the other part is nonsense and often times end up being right or wrong, that is irrelevant. Let me spell it out for you, when you are rude to other people, it speaks to your character. In short, you're a shit head. Doesn't matter what my argument was. And you're comment here contributes nothing to the discussion. You just wanted to be a shit head by making a comment here that contributed nothing other then being rude and displaying how low your moral character is.
- ninetyninenine 1y ago
  https://youtu.be/qrvK_KuIeJk?t=284 https://youtu.be/qrvK_KuIeJk?t=284 I don’t appreciate your comments. Especially rude to call me desperate and bizarre. Take a look at the above video where Geoffrey Hinton basically the god father of AI directly contradicts your statement. I sincerely hope you self reflect and are able to realize that you’re the one completely out of it. Realistically the differences get down to sort of a semantic issue. We both agree that there are things we don’t understand and things that we do understand. It’s just the overall aggregate generalization of this in your opinion comes down to: “we overall do understand” and mine is “we don’t understand shit” Again. Your aggregate is wrong. Utterly. Preeminent Experts are on my side. If we did understand LLMs we’d be able to edit the individual weights of each neuron to remove hallucinations. But we can’t. Like literally we know a solution to the hallucination problem exists. It’s in the weights. We know a certain configuration of weights can remove the hallucination. But even for a single prompt and answer pair we do not know how to modify the weights such that the hallucinations go away. We can’t even quantify, formally define or model what an hallucination is. We describe LLMs in human terms and we manipulate the thing through prompts and vague psychological methods like “chain of thought”. You think planes can work like this? Do we psychologically influence planes to sort of fly correctly? Literally. No other engineered system like this exists on earth where sheer lack of understanding is this large.
  - sirwhinesalot 1y ago
    Sorry for the remark, it was indeed unnecessarily rude and I apologise. That said your appeal to authority means nothing to me. I can simply counter with another appeal to authority like Yann LeCun who thinks LLMs are an evolutionary dead end (and I agree). It matters not that we cannot comprehend them (in the sense of predicting what output it'll give for a given input). Doing that is their job. I also can't comprehend why a support vector machine ends up categorizing spam the way it does. In both cases we understand the algorithms involved which is completely different from our understanding of the brain and its emergent properties.
    - ninetyninenine 1y ago
      Apology not accepted. Your initial take was to cast me as out of touch. With my appeal to authority you now realize that my stance occupies a very valid place even though you disagreed. Like it wasn’t just rude, statements like “bizarre” are inaccurate given that Geoffrey agrees with me. So even if I’m not offended by your rudeness it’s not a bizarre take at all. It’s a valid take. That being said. Yan Lecunn is not in agreement with you either. His only claim is that LLMs are not agi and that hallucinations on LLMs can never be removed. The debate here isn’t even about that. The debate here is that we don’t understand LLMs. Whether the LLM is agi or whether we can remove or never remove the hallucinations is COMPLETELY orthogonal. So you actually can’t counter with another appeal to authority. Either way I didn’t “just” appeal to authority. I literally logically countered every single one of your statements as well.
polotics 1y ago
Coud you provide a link so we can follow your thread of thought on this? It appears your article got submitted by another user than you.

sirwhinesalot 1y ago

Your article, unlike the bizarre desperate take from the poster above, is actually very good. We do not understand the features the neural net learns, that's 100% true (and really the whole point of them in the first place). For small image recognition models we can visualize them and get an intuition for what they are doing, but it doesn't really matter. For even smaller models, we can translate them to a classical AI model (like a mixed integer program as an example) and actually do various "queries" on the model itself to, e.g., learn that the network recognizes the number "8" by just checking 2 pixels in the image. None of this changes the fact that we know what these things are and how they work, because we built them. Any comparisons to our lack of knowledge of the human brain are ridiculous. LLMs are obviously not conscious, they don't even have real "state", they're an approximated pure function f(context: List<Token>) -> Token, that's run in a loop. The only valid alarmist take is that we're using black box algorithms to make decisions with serious real-world impact, but this is true of any black box algorithm, not just the latest and greatest ML models.

dpoloncsak 1y ago
Its a complex adaptive system, right? Isn't that the whole idea? We know how each part of the system works by itself. We know all inputs, and can measure outputs. I still (even if I actually understood the math) cannot tell you 'If you prompt 'x', the model will return 'y' with 100% confidence.
- sirwhinesalot 1y ago
  > If you prompt 'x', the model will return 'y' with 100% confidence. We can do this for smaller models. Which means it's a problem of scale/computing power rather than a fundamental limitation. The situation with the human brain is completely different. We know neurons exchange information and how that works, and we have a pretty good understanding of the architecture of parts of the brain like the visual cortex, but we have no idea of the architecture as a whole. We know the architecture of an LLM. We know how the data flows. We know what it is the individual neurons are learning (cuts and bends of a plane in a hyperdimensional space). We know how the weights are learned (backpropagation). We know the "algorithm" the LLM as a whole is approximating (List<Token> -> Token). Yes there are emergent properties we don't understand but the same is true of a spam filter. Comparing this to our lack of understanding of the human brain and discussing how these things might be "conscious" is silly.
  - ninetyninenine 1y ago
    >Comparing this to our lack of understanding of the human brain and discussing how these things might be "conscious" is silly. Don't call my claim silly. I'm sick of your attitude. Why can't you have a civil discussion? Literally we don't know. You can't make a claim that it's silly when you can't even define what consciousness is. You don't know how human brains work, you don't know how consciousness forms, you don't know how emergence in LLMs work. So your claim here is logically just made up out of thin air. Sure we "understand" LLMs from the curve fitting perspective. But the entirety of why we use LLMs and what we use it for arises from the emergence which is what we don't understand. Curve fitting is like 1% of the LLM, it is the emergent properties we completely don't get (99%) and take advantage of on a daily basis. Curve fitting is just a high level concept that allows us to construct the algorithm which is the actual thing that does the hard work of wiring up the atomic units of the network. >Yes there are emergent properties we don't understand but the same is true of a spam filter. Yeah and so? Your statement proves nothing. It just illustrates a contrast in sentiment. The spam filter is a trivial thing, the human brain is not. We don't understand the spam filter. And this is the most interesting part of it all is that the SAME scaling problem that prevents us from understanding the spam filter can be characterized as the reason that prevents us from understanding BOTH the LLM and the human brain. Your statement doesn't change anything. It's just using sentiment to try to re-characterize a problem in a different light.
    - Suppafly 1y ago
      >Don't call my claim silly. Not the other guy, but your claims are silly. People can't have a civil discussion with you because you post nonsense.
      - ninetyninenine 1y ago
        Civil discussion is part of human decency. It has nothing to do with the other party. Whether you're a good person or a shit head has nothing to do with whether some 3rd part is talking nonsense. People have different opinions all the time and people all the time think the other part is nonsense and often times end up being right or wrong, that is irrelevant. Let me spell it out for you, when you are rude to other people, it speaks to your character. In short, you're a shit head. Doesn't matter what my argument was. And you're comment here contributes nothing to the discussion. You just wanted to be a shit head by making a comment here that contributed nothing other then being rude and displaying how low your moral character is.
ninetyninenine 1y ago
https://youtu.be/qrvK_KuIeJk?t=284 https://youtu.be/qrvK_KuIeJk?t=284 I don’t appreciate your comments. Especially rude to call me desperate and bizarre. Take a look at the above video where Geoffrey Hinton basically the god father of AI directly contradicts your statement. I sincerely hope you self reflect and are able to realize that you’re the one completely out of it. Realistically the differences get down to sort of a semantic issue. We both agree that there are things we don’t understand and things that we do understand. It’s just the overall aggregate generalization of this in your opinion comes down to: “we overall do understand” and mine is “we don’t understand shit” Again. Your aggregate is wrong. Utterly. Preeminent Experts are on my side. If we did understand LLMs we’d be able to edit the individual weights of each neuron to remove hallucinations. But we can’t. Like literally we know a solution to the hallucination problem exists. It’s in the weights. We know a certain configuration of weights can remove the hallucination. But even for a single prompt and answer pair we do not know how to modify the weights such that the hallucinations go away. We can’t even quantify, formally define or model what an hallucination is. We describe LLMs in human terms and we manipulate the thing through prompts and vague psychological methods like “chain of thought”. You think planes can work like this? Do we psychologically influence planes to sort of fly correctly? Literally. No other engineered system like this exists on earth where sheer lack of understanding is this large.
- sirwhinesalot 1y ago
  Sorry for the remark, it was indeed unnecessarily rude and I apologise. That said your appeal to authority means nothing to me. I can simply counter with another appeal to authority like Yann LeCun who thinks LLMs are an evolutionary dead end (and I agree). It matters not that we cannot comprehend them (in the sense of predicting what output it'll give for a given input). Doing that is their job. I also can't comprehend why a support vector machine ends up categorizing spam the way it does. In both cases we understand the algorithms involved which is completely different from our understanding of the brain and its emergent properties.
  - ninetyninenine 1y ago
    Apology not accepted. Your initial take was to cast me as out of touch. With my appeal to authority you now realize that my stance occupies a very valid place even though you disagreed. Like it wasn’t just rude, statements like “bizarre” are inaccurate given that Geoffrey agrees with me. So even if I’m not offended by your rudeness it’s not a bizarre take at all. It’s a valid take. That being said. Yan Lecunn is not in agreement with you either. His only claim is that LLMs are not agi and that hallucinations on LLMs can never be removed. The debate here isn’t even about that. The debate here is that we don’t understand LLMs. Whether the LLM is agi or whether we can remove or never remove the hallucinations is COMPLETELY orthogonal. So you actually can’t counter with another appeal to authority. Either way I didn’t “just” appeal to authority. I literally logically countered every single one of your statements as well.

polotics 1y ago

Coud you provide a link so we can follow your thread of thought on this? It appears your article got submitted by another user than you.