5 ms·
How is it solved? LLMs cannot think new things, they can only cobble something together if it's in their training set.
by rschiavone 3mo ago
How is it solved? LLMs cannot think new things, they can only cobble something together if it's in their training set.
- nashashmi 3mo agoI am not sure if claude had powerpc scripts in its training.
- tock 3mo agoNew things are made by cobbling together existing things.
- varjag 3mo agoThat "only" part used to be the hardest. Getting the ideas was never the hard part. I think someone here even wrote an essay on that.
- qsera 3mo agoIt does not think at all. It vibes based on its training and any additional bolted on constraints. It is a quite simple automation that only works by huge amount of existing data. Modern man has grown quite dumb. He only seems to be able to "invent" by massive scaling things that are decades or centuries old..
- brookst 3mo agoOut of curiousity, can you share a human invention that is not merely scaling things that were decades or centuries old at the time? Fire? The wheel? Archimedes screw, maaaaybe?
- qsera 3mo agoElectricity, RF communication, LASERS, Transistors....
- brookst 3mo agoWhat specific inventions in those areas? Electricity runs from simple batteries (600 BCE) to today’s power grids. RF was predicted but not demonstrated by Maxwell in the 1860’s. His work built on Faraday’s (1840’s) and Coulomb’s (1780’s). Coulomb built on Franklin and Newton, among others. Or do you mean Marconi and Tesla, who merely implemented what Maxwell predicted? The same is true for lasers and transistors but it’s tedious. There was no single “back in the day people invented things from whole cloth” moment.
- skydhash 3mo agoThose are not merely scaling. I can get “build upon other works”, but there’s a lot of scientific insights needed for observing and modeling a phenomena. It may even requires a boost of creativity to theorize an effect based on that model and then make it possible in an experiment.
- qsera 3mo agoThose are the specific inventions.
- addaon 3mo ago> Those are the specific inventions. In what way is electricity an invention? Electricity is a physical phenomenon. Various machines for doing work with electrical energy, storing electrical energy, converting other types of energy to electrical energy, etc. are certainly inventions... Heck, rubbing an amber rod with a fur is an invention. The static charge transferred is not.
- qsera 3mo agoInvention, discovery..does not matter. There was a point in time when humanity was oblivious to the phenomenon and then there was a point when we were not and we could generate and use it.
- 3mo ago
- pixl97 3mo ago>Modern man has grown quite dumb Ah yes, that's why it only took 50 years instead of 100,000 years for homosapients to reach the space age.... Dude, there was no glorious past, we've always sucked.
- qsera 3mo agoWe have always sucked, but there used to be individual brilliance. Now everyone is the same shade of grey...
- pixl97 3mo agoIndividual brilliance has always been rare. If I booted your ass back 1000 years you'd have seen the same shade of grey in the pig farmers and peasants along with most of the nobles and kings. It turns out that in general the world is a shade of grey. Hell, looking back at the brilliant individuals, they had many, many grey areas.
- qsera 3mo ago>Hell, looking back at the brilliant individuals, they had many, many grey areas. That is exactly the point. Back then that could exist. Now, we would just cancel them for one of the grey areas.
- brookst 3mo agoWhat? This is a massive misunderstanding. It’s easy to get truly novel ideas from LLMs, unless your definition of “new things” is so strict that no human can do so either. The training set is about patterns, not facts or specific configurations. Yes, it’s possible to extract (some) of the training set verbatim, but that doesn’t mean it’s all you can do.
- NoMoreNicksLeft 3mo ago>unless your definition of “new things” is so strict that no human can do so either. Humans rarely think of new things. We're a weak hivemind species. One or two individuals figure something out, and the rest of the troop of monkeys imitates. Brains are too fuel hungry for every brain to be innovating, "innovate and copy to the other brains" is the norm.
- antonvs 3mo agoI take it that your training cutoff was early 2023.
- rschiavone 3mo agothe trope that everyone else you are talking to on the Internet is a bot is getting tiring real fast
- antonvs 3mo agoI wasn't saying you were a bot. I was pointing out two things: first, your understanding of LLM capabilities is very outdated; and second, that in this respect, you're behaving much like an LLM with a training cutoff. That further touches on the idea that the differences between you and an LLM may not be as large as you imagine. In particular, "cobbling something together if it's in their training set" is pretty much what all humans do.
- bitwize 3mo agoThey can think and reason better than most humans. Most problems they're pointed at are not in their training set, but in certain ways they resemble things that are—maybe there are a few different resemblances to different problems in its training set—so it's able to pull these disparate similarities together and apply the patterns it finds to come up with a solution. Much like human brains do.