12 ms·
I'm not sure if this really says the truth is more complex? It is still doing next-token prediction, but it's prediction method is sufficiently complicated in t
by ceh123 1y ago
I'm not sure if this really says the truth is more complex? It is still doing next-token prediction, but it's prediction method is sufficiently complicated in terms of conditional probabilities that it recognizes that if you need to rhyme, you need to get to some future state, which then impacts the probabilities of the intermediate states.
At least in my view it's still inherently a next-token predictor, just with really good conditional probability understandings.
- Mahn 1y agoAt this point you have to start entertaining the question of what is the difference between general intelligence and a "sufficiently complicated" next token prediction algorithm.
- dontlikeyoueith 1y agoA sufficiently large lookup table in DB is mathematically indistinguishable from a sufficiently complicated next token prediction algorithm is mathematically indistinguishable from general intelligence. All that means is that treating something as a black box doesn't tell you anything about what's inside the box.
- int_19h 1y agoWhy do we care, so long as the box can genuinely reason about things?
- chipsrafferty 1y agoWhat if the box has spiders in it
- dontlikeyoueith 1y ago:facepalm: I ... did you respond to the wrong comment? Or do you actually think the DB table can genuinely reason about things?
- int_19h 1y agoOf course it can. Reasoning is algorithmic in nature, and algorithms can be encoded as sufficiently large state transition tables. I don't buy into Searle's "it can't reason because of course it can't" nonsense.
- zeroonetwothree 1y agoIt can do something but I wouldn’t call it reasoning. IMO a reasoning algorithmic must be more complex than a lookup table.
- int_19h 1y agoWe were talking about a "sufficiently large" table, which means that it can be larger than realistic hardware allows for. Any algorithm operating on bounded memory can be ultimately encoded as a finite state automaton with the table defining all valid state transitions.
- dontlikeyoueith 1y agoThis is such a confusion of ideas that I don't even know how to respond any more. Good luck.
- jermaustin1 1y agoBut then so are we? We are just predicting the next word we are saying, are we not? Even when you add thoughts behind it (sure some people think differently - be it without an inner monologue, or be it just in colors and sounds and shapes, etc), but that "reasoning" is still going into the act of coming up with the next word we are speaking/writing.
- thomastjeffery 1y agoWe are really only what we understand ourselves to be? We must have a pretty great understanding of that thing we can't explain then.
- hadlock 1y agoHumans and LLMs are built differently, it seems disingenuous to think we both use the same methods to arrive at the same general conclusion. I can inherently understand some proofs of pythagorean's theorem but an LLM might apply different ones for various reasons. But the output/result is still the same. If a next token generator run in parallel can generate a performant relational database that doesn't directly imply I am also a next token generator.
- wetpaws 1y ago[dead]
- spookie 1y agoThis type of response always irks me. It shows that we, computer scientists, think of ourselves as experts on anything. Even though biological machines are well outside our expertise. We should stop repeating things we don't understand.
- BobaFloutist 1y agoWe're not predicting the next word we're most likely to say, we're actively choosing the word that we believe most successfully conveys what we want to communicate. This relies on a theory of mind of those around us and an intentionality of speech that aren't even remotely the same as "guessing what we would say if only we said it"
- dymk 1y agoLike the old saying goes, a sufficiently complex next token predictor is indistinguishable from your average software engineer
- johnthewise 1y agoA perfect next token predictor is equivalent to god
- lanstin 1y agoNot really - even my kids knew enough to interrupt my stream of words with running away or flinging the food from the fork.
- Tadpole9181 1y agoThat's entirely an implementation limitation from humans. There's no reason to believe a reasoning model could NOT be trained to stream multimodal input and perform a burst of reasoning on each step, interjecting when it feels appropriate. We simply haven't.
- lanstin 1y agoNot sure training on language data will teach how to experiment with the social system like being a toddler will, but maybe. Where does the glance of assertive independence as the spoon turns get in there? Will the robot try to make its eyes gleam mischeviously as is written so often.
- Tadpole9181 1y agoBut then this classifier is entirely useless because that's all humans are too? I have no reason to believe you are anything but a stochastic parrot. Are we just now rediscovering hundred year-old philosophy in CS?
- BalinKing 1y agoThere's a massive difference between "I have no reason to believe you are anything but a stochastic parrot" and "you are a stochastic parrot".
- ToValueFunfetti 1y agoIf we're at the point where planning what I'm going to write, reasoning it out in language, or preparing a draft and editing it is insufficient to make me not a stochastic parrot, I think it's important to specify what massive differences could exist between appearing like one and being one. I don't see a distinction between this process and how I write everything, other than "I do it better"- I guess I can technically use visual reasoning, but mine is underdeveloped and goes unused. Is it just a dichotomy of stochastic parrot vs. conscious entity?
- Tadpole9181 1y agoThen I'll just say you are a stochastic parrot. Again, solipsism is not a new premise. The philosophical zombie argument has been around over 50 years now.
- deleted 1y ago[deleted]