4 ms·
Since the times GPT-2 was reimplemented inside Minecraft - its quite obvious LLMs are just math. Nothing else, by nature. Modern LLMs have the same math as in G
by search_facility 5mo ago
Since the times GPT-2 was reimplemented inside Minecraft - its quite obvious LLMs are just math. Nothing else, by nature. Modern LLMs have the same math as in GPT-2 - just bigger and with extra stuff around - and math is the only area of human knowledge with perfect flawless reductionism, straight to the roots. It was build that way since the beginning, so philosophy have no say in this :) And because of that flawless reductionism, complexity adds nothings to the nature of math things, this is how math working by design - so it can be proven there are no anything like consciousness simply because conciousness was not implented in the first place, only perfect mimicry.
And the real secret is in the data, not math. Math (and LLMs running it through billions of weights) is just a tool.
- SuperV1234 5mo agoWe are not fundamentally different. Chemical reactions are just math.
- rellfy 5mo agoWell, (in our current understanding) yes, but there may be underlying aspects of physics and the universe that we do not understand that could be the reason consciousness kicks in. It could turn out that LLMs do work similarly to how humans think, but as an abstracted system it does not have the low level requirements for consciousness.
- baggy_trough 5mo ago> it does not have the low level requirements for consciousness. What is the evidence for this?
- rellfy 5mo agoI didn’t mean it as fact. “Could turn out that …”
- vidarh 5mo agoWe do not know what the "low level requirements for consciousness" are. We do not know how to measure whether consciousness is present in an entity - even other humans - or whether it is just mimicry, nor whether there is a distinction between the two.
- unparagoned 5mo agoI think we know enough that consciousness acts at the neuron/brain level new physics isn’t going to provide any more insights
- ekianjo 5mo agoAmusing statement since we are far from being able to understand chemical reactions in depth. Most of our knowledge in chemistry is empirical. Nothing like math.
- petters 5mo agoWe have a very good idea of all math behind chemistry. But the equations are very difficult to solve.
- ekianjo 5mo agoWe are not talking about the same thing. Not all chemical reactions are predictable like math is. Organic chemistry is full of lucky findings. Just look at how catalysts are discovered.
- slopinthebag 5mo agoNo, math is a tool that we can use to describe something more fundamental. Don't mistake the map for the territory!
- kbrkbr 5mo ago"The universe is fundamentally just a complicated clockwork" Unknown Ptolemy disciple
- BoredomIsFun 5mo ago> Chemical reactions are just math. No, it is quantum mechanics. Physical world is not reducible to math, it has been long proven since early 20th century.
- unparagoned 5mo agoQm is described by maths. You have fully deterministic qm interpretations, which are fully determined by maths
- XMPPwocky 5mo agoYup- the question is "can math be conscious?" (If you've engaged w/ the literature here, it's quite hard to give a confident "yes". it's also quite hard to give a confident "no"! so then what the heck do we do)
- search_facility 5mo agoImho no, math itself have no conciousness. Quite confidently its a helpful tool that does not act by himself.
- XMPPwocky 5mo agoHm, say more about what your opinion's based on here?
- solid_fuel 5mo agoTake a piece of paper, write two numbers on it, let me know when they start to reproduce.
- nandomrumber 5mo agoThe math isn’t the ink on the page.
- SwellJoe 5mo agoNot just any math: Matrix multiplication. Can matrix multiplication be conscious? And, I don't see how it can be. It is deterministic, when all variables are controlled. You can repeat the output over and over, if you start it with the same seed, same prompt, and same hardware operating in a way that doesn't introduce randomness. At commercial scale, this is difficult, as the floating point math on GPUs/TPUs when running large batches is non-deterministic, as I understand it. But, in a controlled lab, you can make a model repeat itself identically. Unless the random number generator is "conscious", I don't see a place to fit consciousness into our understanding of LLMs.
- canjobear 5mo agoYou could simulate your own brain in Minecraft. What do you conclude from this?
- search_facility 5mo agoI can not simulate my brain, it's a huge stretch to imply this. But with LLMs - anyone can simulate LLM. LLM can be simulated without any uncertainties in pen and paper and a lot of time. Does it mean that 100 tons of paper plus 100 years of time (numbers are just examples) calculating long formulae makes this pile of paper consiousness? Imho answer is definitive no.
- thrownthatway 5mo agoI don’t think anyone is arguing the silicon is conscious. Similarly the paper. What about the agent doing the calculations. He may be conscious. Or anyway, we can’t rule it out.
- search_facility 5mo agoWith both cases the agent doing something is human. And human beings are indeed conscious. Outside human needs LLM are useless. Math, as a tool, is just a proxy for people using LLMs, as well as GPUs spending cycles on calculating the math
- NiloCK 5mo agoThe whole is composed of parts, ergo there is no whole. This seems incorrect to me. We too are amalgamations of inanimate components - emerged superstructures. Just cells. Just molecules. Just atoms.
- solid_fuel 5mo agoThis is such a weird comment. > Since the times GPT-2 was reimplemented inside Minecraft - its quite obvious LLMs are just math. This was obvious since LLMs were first invented. They published papers with all the details, you don't need to see something implemented in Minecraft to realize that it's just math. You could simply read the paper or the code and know for certain. [0] > math is the only area of human knowledge with perfect flawless reductionism, straight to the roots Incorrect, Kurt Gödel showed with his Incompleteness Theorems in 1931 [1] that it is impossible to find a complete and consistent set of axioms for mathematics. Math is not perfectly reducible and there is no single set of "roots" for math. > It was build [sic] that way since the beginning, This is a serious misunderstanding of what mathematics is. Math is discovered as much as it is built. No one sat down and planned out what we understand as modern mathematics - the math we know is the result of endless amounts of logical reasoning and exploration, from geometric proofs to calculus to linear algebra to everything else that encompasses modern mathematics. > And because of that flawless reductionism, complexity adds nothings to the nature of math things, this is how math working by design This sentence means nothing, because math is not reducible in that way. > so it can be proven there are no anything like consciousness simply because conciousness [sic] was not implented [sic] in the first place, only perfect mimicry. Even if the previous sentence held, this does not follow, because while we are conscious the current consensus is that LLMs are not and most AI experts who are not actively selling a product recognize that LLMs will not lead to human-equivalent general intelligence. [3] [0] https://github.com/openai/gpt-2 https://github.com/openai/gpt-2 [1] https://en.wikipedia.org/wiki/G%C3%B6del's_incompleteness_theorems https://en.wikipedia.org/wiki/G%C3%B6del's_incompleteness_th... [2] https://www.cambridge.org/core/journals/think/article/mathematics-discovery-or-invention/D95CA7FFC636B147C9BD17F1409BAD36 https://www.cambridge.org/core/journals/think/article/mathem... [3] https://deepmind.google/research/publications/231971/ https://deepmind.google/research/publications/231971/
- search_facility 5mo agoMath used in LLMs is perfectly reducible and Gödel have nothing to do with it - inside commonly used axioms (which sufficient for LLM to exist and outside of Kurt Gödel scope) there are ZERO questions/uncertainties how it works, it's just a fact :)
- IshKebab 5mo agoThis is likely wrong because you're assuming there is some secret sauce in biological brains that lets them do something that cannot be simulated. That both seems very unlikely and also we've never found anything remotely like that by observation. Of course it would be extremely difficult to simulate a human brain but as far as we know there's no fundamental physics preventing it. And yes that does have super weird consequences about consciousness. But consciousness is clearly super weird already so I suppose that's not too surprising.