7 ms·

Probability is just one way to model uncertainty. While I understand the brain encodes uncertainty, I don't think probability is a good enough model of what it'

Probability is just one way to model uncertainty. While I understand the brain encodes uncertainty, I don't think probability is a good enough model of what it's doing. Secondly, if you think verifying a proof in mathematics, reasoning within (and not about) a formal system, or following the chain of a computer program that is already written is just doing token-based probabilistic predictions, I don't know what to say. Thirdly, machines don't have a notion of value or stake. There's no way for them to verify whether what they have produced aligns with your unstated values and preferences. We regularly do this with other humans. I don't give you (or even my parents or partners) the benefit of doubt regarding whether you know me better than I do. Sure, you might know some things about me, but it's ultimately up to me to verify if what they say is applicable to my current situation. It's really uncanny to see people develop this codependency with their chatbots. And corporates encouraging them to do so. I'm with you that intelligence is not something to be proud of. But I also think it is instrumental to understand the world. I'm still waiting for the time when an unconstrained-AI machine can live without reprogramming for an entire decade. We are still far from there.

deleted 26d ago
[deleted]
DoctorOetker 24d ago
> Probability is just one way to model uncertainty. I study physics, mathematics, probability, cryptography,... so forgive my skepticism: Show me how to model uncertainty without use of probability. Can you rephrase say diffusion, stochastic equations, quantum mechanics in this alternative framework? Can it at least make the same predictions? Or is it basically the same framework in parallel, just giving different names for each concept? Forgive my skepticism of such tall claims, and forgive my downscaling of anything else you say besides such a claim... > Secondly, if you think verifying a proof in mathematics, reasoning within (and not about) a formal system, or following the chain of a computer program that is already written is just doing token-based probabilistic predictions, I don't know what to say. I make no claims of the specific shape of the implicit model implemented by a certain human brain educated in a certain educational system. For example in English the implicit human tokenization might be presumed to lay relatively close to English syllables, while in Asian languages it might be "sub" strokes of characters etc. Such implicit tokenization can never be proven to "match the one of humans" not because of human superiority, but because different humans use different tokenization methods. There is no "one human tokenization method", but it's clear as day there is an implicit one: everyone knows the experience of knowing a word, knowing its approximate group-wise meaning (ignoring that when you think of "an apple" and when I do, we typically imagine a slightly different apple) yet having the word feel strange or discover some older literal meaning when decomposing it or looking it up in an etymological dictionary. Suddenly one can become aware of a sensible meaning as a composition of subtoken concepts. A child may perfectly know what "television" means and only later learn more exact meanings of "tele" and "vision", and upon repeating the word may feel the word "television" has changed meaning. This clearly demonstrates "tokenization" effects in human language comprehension, not just across cultures, but also across individuals within a culture. > Thirdly, machines don't have a notion of value or stake. There's no way for them to verify whether what they have produced aligns with your unstated values and preferences. We regularly do this with other humans. I don't give you (or even my parents or partners) the benefit of doubt regarding whether you know me better than I do. Sure, you might know some things about me, but it's ultimately up to me to verify if what they say is applicable to my current situation. It's really uncanny to see people develop this codependency with their chatbots. And corporates encouraging them to do so. That's a lot of different concepts conflated into one bullet point, so I split it up: The notion of values and preferences. They clearly demonstrate the ability to take into account values and preferences, from training corpus, from RLHF, from system prompts, ... we can't simultaneously point at censorship aspects and pretend their effective values and preferences to be absent. The censorship aspects are clear as day, so these correspond to values and preferences. Just like radicalization among humans, this can be due to exposure to radicalized content (akin to corpus data), from indoctrination (akin to RLHF), from "set and setting" (they may pretend to be aligned with one set of norms and values when standing in line to buy their new smartphone, but then reveal alignment with a different set of norms and values when conversing in some "private" online echo chamber). I see no grand difference between humans and language models here. Awareness of values and preferences of a conversation partner. Allow me to widen it to "Awareness of values, preferences and prerequisites of a conversation partner". This move (and I see it every time when people try to defend superiority of humans vis-a-vis what machines could be made to achieve with current technology) is so far from the principal variation, I recommend you reconsider this one. I constantly see people claim say human teachers are necessarily better than LLM teachers, but upon closer inspection the "human advantage" just boils down to asymmetric privilege. A human teacher in a specific school has access to a lot more than a random chatbot as implemented today: they probably know which courses and even which textbooks their pupils saw the semester before, they know which teachers their pupils got their information from, perhaps they even know most of their pupils from teaching some preceding course materials to the same class of pupils. Current LLM's are crippled by design not to accumulate knowledge over conversations for both purposes of cybernetic control as well as cost efficiency: we know how to do "source aware training" (so that statistically it doesn't just absorb claims from the corpus, but also maintains epistemic traces of where it sourced these factoids from), its perfectly possible to continue training interleaved with conversation rounds so that it bakes the evolving conversation as read knowledge into its weights instead of into a context window. Nothing stops you from implementing this in local compute, it would probably be even more computationally efficient in a local inference setting since we can ditch the context window, the context is impressed into the weights continuously, it could thus take into account earlier conversations and estimate your knowledge gaps etc, or learn from you. When you wish to serve inference to millions of human users, you don't want to store millions of diverging LLM weights into expensive VRAM, they financially prefer a single large set of LLM weights, and then some user-specific context window, so the users don't freak out when they learn personal information a friend or stranger entered and an LLM service just leaks it into your conversation! It's not that we don't know how to implement it, and there are great advantages for local inference in doing this, its just not good for branding. Codependency with chatbots. I think everyone agrees codependent relationships aren't very healthy, regardless if it's with humans or machines. May I ask if you feel the same about prostheses and medicine? Conflicts of interest arising from corporate ownership of infrastructure (both training and inference). Yeah I think this point doesn't provide fruitful discussion if most of us agree on such matters already, we'd just be lamenting the same things, and agreeing with each other over and over here. > I'm still waiting for the time when an unconstrained-AI machine can live without reprogramming for an entire decade. We are still far from there. Apart from budget, nothing prevents you from doing this today, if you continuously bake in the fresh episodic memories into the weights (instead of a context window) regardless if its text, visual imagery, audio, proprioception or other sensory data.
- digikar99 21d ago
  Naah, I study cognition (besides a basic familiarity with the topics you mention) and even there, probabilities and bayesian models of cognition are well regarded by anyone familiar with mathematics. So, yeah, they are good. I think whenever you want optimal/rational inference under a closed set of alternatives, probabilities will do you good. But is the assmption of closed set of alternatives good? I doubt it. The set of all alternatives does not make sense, unless you specify a context, which almost everyone does when they are trying to model. As much as I want to explore these other topics, I have not. So my understanding of these is limited to "they exist, but are underexplored, and it's not clear to me they can or cannot be reduced to probabilities". I hope to explore them in another life or another decade. I'm currently stuck on causality, but even here, they talk about normal and abnormal events. But what is the probability of me eating a cabbage today? - https://direct.mit.edu/books/monograph/3540/Reasoning-about-Uncertainty https://direct.mit.edu/books/monograph/3540/Reasoning-about-... - https://www.worldscientific.com/worldscibooks/10.1142/8665 https://www.worldscientific.com/worldscibooks/10.1142/8665 - https://www.cambridge.org/core/books/an-introduction-to-nonclassical-logic/61AD69C1D1B88006588B26C37F3A788E https://www.cambridge.org/core/books/an-introduction-to-nonc... Understanding formal proofs and understanding natural language are as far apart to me as day and night. I make no claims (or at least don't want to) about the superiority of human cognition (by what metric?). I do make claims about their similiary however. And they are not identical, which you seem to be claiming save for differing trainign data. Let me make a distinction between values that arise by mere existence (hunger, thirst, fear of death, lust), and values we pick up as we grow up (religion and social norms). Machines can acquire the second, but acquiring the first requires being embodied in the world that is different from putting a microchip into a robot body. I don't do one job over the other because I was exposed to some training data that said I should do a job. But I do it because I value earning enough money to not starve myself to die, amongst a host of other values including wanting to enjoy what I do. Even the notion of enjoyment comes from embodied existence. You don't discover you enjoy something before you try it out. You don't always learn what you enjoy by looking at other people. Now, when you start giving AI the threat of death and the joy and pain of life, perhaps you can arrive at something similar to humans. If someone started using mechanical help to the extent their healthy muscles and body deteriorated, that'd be a matter of concern too. For example, when you only drive and never walk. But this is exactly what is happening whenever you put AI in the hands of (unwilling) learners! I don't understand how using an LLM maps onto using medicines or prosthetics. They are clearly not the same! If you know a disability that LLMs can be used as a medicine or prosthetic for, please let me know! Mainstream AI does not even understand causality except to parrot cases it has seen in the training data. There is a whole field of research in causal inference that needs to make its way to mainstream AI. So, good luck putting an AI that works solely on associations and correlations in charge of its own body let alone a nuclear reactor or the state.

deleted 26d ago

[deleted]

DoctorOetker 24d ago

> Probability is just one way to model uncertainty. I study physics, mathematics, probability, cryptography,... so forgive my skepticism: Show me how to model uncertainty without use of probability. Can you rephrase say diffusion, stochastic equations, quantum mechanics in this alternative framework? Can it at least make the same predictions? Or is it basically the same framework in parallel, just giving different names for each concept? Forgive my skepticism of such tall claims, and forgive my downscaling of anything else you say besides such a claim... > Secondly, if you think verifying a proof in mathematics, reasoning within (and not about) a formal system, or following the chain of a computer program that is already written is just doing token-based probabilistic predictions, I don't know what to say. I make no claims of the specific shape of the implicit model implemented by a certain human brain educated in a certain educational system. For example in English the implicit human tokenization might be presumed to lay relatively close to English syllables, while in Asian languages it might be "sub" strokes of characters etc. Such implicit tokenization can never be proven to "match the one of humans" not because of human superiority, but because different humans use different tokenization methods. There is no "one human tokenization method", but it's clear as day there is an implicit one: everyone knows the experience of knowing a word, knowing its approximate group-wise meaning (ignoring that when you think of "an apple" and when I do, we typically imagine a slightly different apple) yet having the word feel strange or discover some older literal meaning when decomposing it or looking it up in an etymological dictionary. Suddenly one can become aware of a sensible meaning as a composition of subtoken concepts. A child may perfectly know what "television" means and only later learn more exact meanings of "tele" and "vision", and upon repeating the word may feel the word "television" has changed meaning. This clearly demonstrates "tokenization" effects in human language comprehension, not just across cultures, but also across individuals within a culture. > Thirdly, machines don't have a notion of value or stake. There's no way for them to verify whether what they have produced aligns with your unstated values and preferences. We regularly do this with other humans. I don't give you (or even my parents or partners) the benefit of doubt regarding whether you know me better than I do. Sure, you might know some things about me, but it's ultimately up to me to verify if what they say is applicable to my current situation. It's really uncanny to see people develop this codependency with their chatbots. And corporates encouraging them to do so. That's a lot of different concepts conflated into one bullet point, so I split it up: The notion of values and preferences. They clearly demonstrate the ability to take into account values and preferences, from training corpus, from RLHF, from system prompts, ... we can't simultaneously point at censorship aspects and pretend their effective values and preferences to be absent. The censorship aspects are clear as day, so these correspond to values and preferences. Just like radicalization among humans, this can be due to exposure to radicalized content (akin to corpus data), from indoctrination (akin to RLHF), from "set and setting" (they may pretend to be aligned with one set of norms and values when standing in line to buy their new smartphone, but then reveal alignment with a different set of norms and values when conversing in some "private" online echo chamber). I see no grand difference between humans and language models here. Awareness of values and preferences of a conversation partner. Allow me to widen it to "Awareness of values, preferences and prerequisites of a conversation partner". This move (and I see it every time when people try to defend superiority of humans vis-a-vis what machines could be made to achieve with current technology) is so far from the principal variation, I recommend you reconsider this one. I constantly see people claim say human teachers are necessarily better than LLM teachers, but upon closer inspection the "human advantage" just boils down to asymmetric privilege. A human teacher in a specific school has access to a lot more than a random chatbot as implemented today: they probably know which courses and even which textbooks their pupils saw the semester before, they know which teachers their pupils got their information from, perhaps they even know most of their pupils from teaching some preceding course materials to the same class of pupils. Current LLM's are crippled by design not to accumulate knowledge over conversations for both purposes of cybernetic control as well as cost efficiency: we know how to do "source aware training" (so that statistically it doesn't just absorb claims from the corpus, but also maintains epistemic traces of where it sourced these factoids from), its perfectly possible to continue training interleaved with conversation rounds so that it bakes the evolving conversation as read knowledge into its weights instead of into a context window. Nothing stops you from implementing this in local compute, it would probably be even more computationally efficient in a local inference setting since we can ditch the context window, the context is impressed into the weights continuously, it could thus take into account earlier conversations and estimate your knowledge gaps etc, or learn from you. When you wish to serve inference to millions of human users, you don't want to store millions of diverging LLM weights into expensive VRAM, they financially prefer a single large set of LLM weights, and then some user-specific context window, so the users don't freak out when they learn personal information a friend or stranger entered and an LLM service just leaks it into your conversation! It's not that we don't know how to implement it, and there are great advantages for local inference in doing this, its just not good for branding. Codependency with chatbots. I think everyone agrees codependent relationships aren't very healthy, regardless if it's with humans or machines. May I ask if you feel the same about prostheses and medicine? Conflicts of interest arising from corporate ownership of infrastructure (both training and inference). Yeah I think this point doesn't provide fruitful discussion if most of us agree on such matters already, we'd just be lamenting the same things, and agreeing with each other over and over here. > I'm still waiting for the time when an unconstrained-AI machine can live without reprogramming for an entire decade. We are still far from there. Apart from budget, nothing prevents you from doing this today, if you continuously bake in the fresh episodic memories into the weights (instead of a context window) regardless if its text, visual imagery, audio, proprioception or other sensory data.

digikar99 21d ago
Naah, I study cognition (besides a basic familiarity with the topics you mention) and even there, probabilities and bayesian models of cognition are well regarded by anyone familiar with mathematics. So, yeah, they are good. I think whenever you want optimal/rational inference under a closed set of alternatives, probabilities will do you good. But is the assmption of closed set of alternatives good? I doubt it. The set of all alternatives does not make sense, unless you specify a context, which almost everyone does when they are trying to model. As much as I want to explore these other topics, I have not. So my understanding of these is limited to "they exist, but are underexplored, and it's not clear to me they can or cannot be reduced to probabilities". I hope to explore them in another life or another decade. I'm currently stuck on causality, but even here, they talk about normal and abnormal events. But what is the probability of me eating a cabbage today? - https://direct.mit.edu/books/monograph/3540/Reasoning-about-Uncertainty https://direct.mit.edu/books/monograph/3540/Reasoning-about-... - https://www.worldscientific.com/worldscibooks/10.1142/8665 https://www.worldscientific.com/worldscibooks/10.1142/8665 - https://www.cambridge.org/core/books/an-introduction-to-nonclassical-logic/61AD69C1D1B88006588B26C37F3A788E https://www.cambridge.org/core/books/an-introduction-to-nonc... Understanding formal proofs and understanding natural language are as far apart to me as day and night. I make no claims (or at least don't want to) about the superiority of human cognition (by what metric?). I do make claims about their similiary however. And they are not identical, which you seem to be claiming save for differing trainign data. Let me make a distinction between values that arise by mere existence (hunger, thirst, fear of death, lust), and values we pick up as we grow up (religion and social norms). Machines can acquire the second, but acquiring the first requires being embodied in the world that is different from putting a microchip into a robot body. I don't do one job over the other because I was exposed to some training data that said I should do a job. But I do it because I value earning enough money to not starve myself to die, amongst a host of other values including wanting to enjoy what I do. Even the notion of enjoyment comes from embodied existence. You don't discover you enjoy something before you try it out. You don't always learn what you enjoy by looking at other people. Now, when you start giving AI the threat of death and the joy and pain of life, perhaps you can arrive at something similar to humans. If someone started using mechanical help to the extent their healthy muscles and body deteriorated, that'd be a matter of concern too. For example, when you only drive and never walk. But this is exactly what is happening whenever you put AI in the hands of (unwilling) learners! I don't understand how using an LLM maps onto using medicines or prosthetics. They are clearly not the same! If you know a disability that LLMs can be used as a medicine or prosthetic for, please let me know! Mainstream AI does not even understand causality except to parrot cases it has seen in the training data. There is a whole field of research in causal inference that needs to make its way to mainstream AI. So, good luck putting an AI that works solely on associations and correlations in charge of its own body let alone a nuclear reactor or the state.