5 ms·

Exactly. I am becoming increasingly convinced that this is actually just a part of how intelligence/cognition works.

astrobe_ 7d ago
But is it really what we want, machines with the same defects as humans? I don't want a pocket calculator that make mistakes "sometimes" so I have to double-check the results, I want a pocket calculator that works (to those who want to argue that pocket calculators don't give the correct result for (1/3)*3: STFU).
- CookieCrisp 7d ago
  No, but it makes sense to me that we’d need to go through this step to get where we want to go
- Lord-Jobo 7d ago
  One of the biggest strengths of a computer is reproducibility. The worst software bugs are inconsistent or non reproducible. The least useful calculators apply rules inconsistently, to your example. the inconsistency of LLMs is by far one of the biggest gripes I have with them. Closely related to their apparently deep desire to avoid following instructions. I know these are both a byproduct of noise (which is somewhat tunable) and noise is inherent to these systems in a lode bearing way. I still hate it. it’s holding the technology back. I don’t honestly see how we can safely or even successfully approach the idealized realm of AI without bypassing this problem, which to my understanding, probably means not using language models at all and trying a totally different approach. But I really don’t know much about machine learning, I’m a super novice compared to a lot on this website.
- supern0va 7d ago
  >But is it really what we want, machines with the same defects as humans? Sort of, actually. I think we humans actually have some intuition that we'd be more effective if our cognition were augmented more directly by machine strengths: the ability to run precise calculations, more memory, ability to look facts in some sort of knowledge graph. I think we're on the right track, but instead of augmenting humans with machine strengths, we're building intelligence in hardware in a way where it can access that augmentation. Plus, then we can quickly distribute updates, run parallel instances, etc. If intelligence is compression, and hallucinations are essentially loss, then as the models grow in size performance (at least as far as hallucinations) should reduce. Or we'll get things fast enough that we can afford to stop relying on model weights for memory and check an increasingly larger set of discrete facts as part of reasoning. Right now, the models are making trade-offs. As compute grows, and inference gets faster, we can make fewer of those trade-offs and start to use the unique strengths of machines to fill the gaps we're seeing, I suspect.
- Paradigma11 7d ago
  Nobody is taking away your calculator.
Lord-Jobo 7d ago
When we are really thinking about something we do it forwards, backwards and middle out, and regenerate and distill many times. When we do meta thinking about that process after the fact, two things happen. 1, we change our total “thought” by adding that meta thinking to it. And 2: it’s a very lossy process, because we don’t have very good data about what our brain or mind was actually doing during that first think and emotional factors are nearly always at play and even more complex. Now for the more complex AI, the fragmented process of multiple agents and loops and reruns are pretty similar to that first think we do. At least structurally. But the meta think is where they differ. They have no emotion, but they also have even worse data about its own function. They constantly degenerate so I would argue their “changing the thought by thinking about it” factor is also generally way higher than ours. Getting better at consistent/reproducible thinking, with many ‘steps’, that leaves good documentation of that thinking behind for future analysis, has to be one of the more important areas for the big flagships going forward. I’m certain that “what is this fucker doing and why” is the biggest pain point for AI researchers. Or the math, it’s usually the math. But you’re correct in the general structure; they generally do the same post hoc analysis we do, just noticeably worse because of their opaque nature(even to themselves) and general degenerative instability.

But is it really what we want, machines with the same defects as humans? I don't want a pocket calculator that make mistakes "sometimes" so I have to double-check the results, I want a pocket calculator that works (to those who want to argue that pocket calculators don't give the correct result for (1/3)*3: STFU).

CookieCrisp 7d ago
No, but it makes sense to me that we’d need to go through this step to get where we want to go
Lord-Jobo 7d ago
One of the biggest strengths of a computer is reproducibility. The worst software bugs are inconsistent or non reproducible. The least useful calculators apply rules inconsistently, to your example. the inconsistency of LLMs is by far one of the biggest gripes I have with them. Closely related to their apparently deep desire to avoid following instructions. I know these are both a byproduct of noise (which is somewhat tunable) and noise is inherent to these systems in a lode bearing way. I still hate it. it’s holding the technology back. I don’t honestly see how we can safely or even successfully approach the idealized realm of AI without bypassing this problem, which to my understanding, probably means not using language models at all and trying a totally different approach. But I really don’t know much about machine learning, I’m a super novice compared to a lot on this website.
supern0va 7d ago
>But is it really what we want, machines with the same defects as humans? Sort of, actually. I think we humans actually have some intuition that we'd be more effective if our cognition were augmented more directly by machine strengths: the ability to run precise calculations, more memory, ability to look facts in some sort of knowledge graph. I think we're on the right track, but instead of augmenting humans with machine strengths, we're building intelligence in hardware in a way where it can access that augmentation. Plus, then we can quickly distribute updates, run parallel instances, etc. If intelligence is compression, and hallucinations are essentially loss, then as the models grow in size performance (at least as far as hallucinations) should reduce. Or we'll get things fast enough that we can afford to stop relying on model weights for memory and check an increasingly larger set of discrete facts as part of reasoning. Right now, the models are making trade-offs. As compute grows, and inference gets faster, we can make fewer of those trade-offs and start to use the unique strengths of machines to fill the gaps we're seeing, I suspect.
Paradigma11 7d ago
Nobody is taking away your calculator.

Lord-Jobo 7d ago

When we are really thinking about something we do it forwards, backwards and middle out, and regenerate and distill many times. When we do meta thinking about that process after the fact, two things happen. 1, we change our total “thought” by adding that meta thinking to it. And 2: it’s a very lossy process, because we don’t have very good data about what our brain or mind was actually doing during that first think and emotional factors are nearly always at play and even more complex. Now for the more complex AI, the fragmented process of multiple agents and loops and reruns are pretty similar to that first think we do. At least structurally. But the meta think is where they differ. They have no emotion, but they also have even worse data about its own function. They constantly degenerate so I would argue their “changing the thought by thinking about it” factor is also generally way higher than ours. Getting better at consistent/reproducible thinking, with many ‘steps’, that leaves good documentation of that thinking behind for future analysis, has to be one of the more important areas for the big flagships going forward. I’m certain that “what is this fucker doing and why” is the biggest pain point for AI researchers. Or the math, it’s usually the math. But you’re correct in the general structure; they generally do the same post hoc analysis we do, just noticeably worse because of their opaque nature(even to themselves) and general degenerative instability.