7 ms·

what basis do you have for assuming an LLM is fundamentally incapable of doing this?

dvt 4mo ago
Because by definition LLMs are permutation machines, not creativity machines. (My premise, which you may disagree with, is that creativity/imagination/artistry is not merely permutation.)
- KoolKat23 4mo ago
  It pretty much is, otherwise it is randomness or entropy.
- nh23423fefe 4mo ago
  god of the gaps
  - iwontberude 4mo ago
    non overlapping magisteria
- fnordpiglet 4mo ago
  I prefer to think of it as they’re interpolation machines not extrapolation machines. They can project within the space they’re trained in, and what they produce may not be in their training corpus, but it must be implied by it. I don’t know if this is sufficient to make them too weak to create original “ideas” of this sort, but I think it is sufficient to make them incapable of original thought vs a very complex to evaluate expected thought.
  - drdeca 4mo ago
    People keep saying this, but if you try to interpret this at all literally, it just doesn’t work. Like, it’s phrased like it should have a precise meaning, right? Like, people even mention convex hulls when talking about it. But if you actually try to take a convex hull of, some encoding of sentences as vectors? It isn’t true. The outputs are not in the convex hull of the training data. I guess it’s supposed to be a metaphor and not literal, but in that case it’s confusing. Especially seeing as there are contexts in machine learning where literal interpolation vs literal extrapolation, is relevant. So, please, find a better way to say it than saying that “it can only interpolate”?
    - Muromec 4mo ago
      If it's all just points in the multidimensional space, why would the thing be restricted to some operations and not others. I'm not buying the argument
      - drdeca 4mo ago
        Sorry, I don't understand what you mean. Are you agreeing or disagreeing with me? If it can only interpolate in a literal sense, that means that it only produces good outputs on convex combinations of inputs that appear in the training set. That's what interpolation means. But, if you take the embedding vectors of sentences/prompts, and then take the convex hull of these, it is not typical for new sentences not in the training set to have its embedding vectors be in the convex hull of these.
        fnordpiglet 4mo ago
        I’m not sure I follow your end to end reasoning. In an n dimensional space interpolation along and within the convex hull is pretty much what they’re doing. How can it possibly not be? How would it interpolate a point that’s not within its vector space? Yes, it’s very complex with non linear transformations and a very high dimensionality, and residuals and other features create more complexity in the shape of the hull. But an LLM can not infer a concept to which it has no information channel. That’s clearly nonsense. The fact that they do bounded, learned, nonlinear compositional generalizations over a representational space induced by training -is by nature interpolation- not extrapolation. I’m sorry, but I believe their immense power has you confusing math with magic.
        drdeca 4mo ago
        A convex hull is a different thing than the linear span. It is smaller. And, my point is that the inputs it is often fed are not in the convex hull of the inputs in the training data. When the input space is very high dimensional, this is a common outcome. I’m not denying that the outputs are causally downstream from the training data. Of course it is. I’m saying that the inference time inputs aren’t in the convex hull of the training time inputs. This isn’t about saying that the output isn’t because of the training data. Of course it is. But when you have very high dimensional input space, then even with many inputs in the training data, it is still common for inference time inputs to not be in the convex hull of the train time inputs. This has nothing to do with the complexities of how the models work after the initial embedding of the tokens as vectors. It’s just about the inputs that appear during training, and the inputs that appear at inference time. > But an LLM can not infer a concept to which it has no information channel. Of course! And nothing I said implies otherwise. Really, the point I’m making doesn’t even depend on what the model outputs! If I took a best fit line from 1 parameter to a 1D output, and then provided that linear model an output that was outside the range of inputs the best fit line was obtained from, that would not be interpolation, it would be extrapolation. It is similar here, except instead of the input being outside the convex hull due to being further away, it is outside the convex hull due to, like, the shape of the convex hull of training inputs just doesn’t include the point in question.
- lajamerr 4mo ago
  LLMs by themselves are not able to but you are missing a piece here. LLMs are prompted by humans and the right query may make it think/behave in a way to create a novel solution. Then there's a third factor now with Agentic AI system loops with LLMs. Where it can research, try, experiment in its own loop that's tied to the real world for feedback. Agentic + LLM + Initial Human Prompter by definition can have it experiment outside of its domain of expertise. So that's extending the "LLM can't create novel ideas" but I don't think anyone can disagree the three elements above are enough ingredients for an AI to come up with novel ideas.
  - awesome_dude 4mo ago
    You're proving the GP's argument - LLMs aren't creative you say as much, it's the driving that is the creative force
    - Barbing 4mo ago
      If that’s a requirement, aren’t LLMs driven by pretraining which was human driven? Who decides at which the last point it’s OK to provide text to the model in order to be able to describe it as creative? (non-rhetorical)
    - lajamerr 4mo ago
      You can tell an agentic system. "Go and find a novel area of math that has unresolved answers and solve it mathematically with verified properties in LEAN. Verify before you start working on a problem that no one has solved this area of math" That's not creative prompt. That's a driving prompt to get it to start its engine. You could do that nowadays and while it may spend $1,000 to $100,000 worth of tokens. It will create something humans haven't done before as long as you set it up with all its tool calls/permissions.
      - awesome_dude 4mo ago
        Let me know when the Fields medal arrives in the mail. It won't because even though it looks clever to you, people who /do/ understand math and LLMs understand that LLMs /are/ regurgitating Why does your LLM need you to tell it to look in the first place? Why isn't just telling us all the answers to unsolved conjectures known and unknown? Why isn't the LLM just telling us all the answers to all the problems we are facing? Why isn't the LLM telling us, step by step with zero error, how to build the machine that can answer the ultimate question?
- lukol 4mo ago
  This "new math" might be a recombination of things that we already know - or an obvious pattern that emerges if you take a look at things from a far enough distance - or something that can be brute-forced into existence. All things LLMs are perfectly capable of. In the end, creativity has always been a combination of chance and the application of known patterns in new contexts.
  - dvt 4mo ago
    > This "new math" might be a recombination of things that we already know If you know anything about the invention of new math (analytic geometry, Calculus, etc.), you'd know how untrue this is. In fact, Calculus was extremely hand-wavy and without rigorous underpinnings until the mid 1800s. Again: more art than science.
    - baq 4mo ago
      And yet nowadays you can restate all of it using just combinations of sets of sets and some logic operators.
    - jfyi 4mo ago
      Newton and Leibniz were "hand-waving"? If anything, they were fighting an uphill battle against the perception of hand-waving by their contemporaries.
      - dvt 4mo ago
        > Newton and Leibniz were "hand-waving"? Yes, and it's pretty common knowledge that Calculus was (finally) formalized by Weierstrass in the early 19th century, having spent almost two centuries in mathematical limbo. Calculus was intuitive, solved a great class of problems, but its roots were very much (ironically) vibes-based. This isn't unique to Newton or Leibniz, Euler did all kinds of "illegal" things (like playing with divergent series, treating differentials as actual quantities, etc.) which worked out and solved problems, but were also not formalized until much later.
        4mo ago
- satvikpendem 4mo ago
  What is creativity if not permutation? A brain has some model of the world and recombines concepts to create new concepts.
  - d3ffa 4mo ago
    [flagged]
    - rowanG077 4mo ago
      This is really not an acceptable reply. How about actually engaging with the point the commenter made instead of stamping your foot and throwing a tantrum.
    - anthk 4mo ago
      Innovation it's just another word for the term 'enhanced copy'. Everything it's a copy, except for nature.
truncate 4mo ago
What's your basis for assuming LLM is capable of doing this? I honestly don't know personally either way. Based on my limited understanding of how LLMs work, I don't see them be making the next great song or next great book and based on that reasoning I'm betting that it probably wont be able to do whatever next "Descartes, Newton, Leibnitz, Gauss, Euler, Ramanujan, Galois" are going to do. Of course AI as a wider field comes up with something more powerful than LLM that would be different.
- redsocksfan45 4mo ago
  [dead]
- dist-epoch 4mo ago
  LLMs are already making the next great songs. Just check out the Billboard charts.
  - truncate 4mo ago
    I'm sorry, I don't consider them "great songs". Obviously, different people have different taste.
- EMM_386 4mo ago
  "I don't see them be making the next great song" Meanwhile, songs are hitting number one on some charts on Spotify that people think are humans and are actually AI. And Spotify has to start labelling them as such. One AI "band" had an entire album of hits. Also - music is a subjective. Mathematics isn't. And in this case, an LLM discovered a new way to reason about a conjecture. I don't know how much proof is needed - since that is literally proof that it can be done.
  - truncate 4mo ago
    >> Meanwhile, songs are hitting number one on some charts on Spotify that people think are humans and are actually AI. And Spotify has to start labelling them as such. One AI "band" had an entire album of hits. There is quite some questions around that. Music is subjective and obviously different people have different taste, but I wouldn't call any of them to be actual good music / real hits. >> LLM discovered a new way to reason about a conjecture I wasn't questioning LLMs ability to prove things. Parent threads were talking about building new kind of maths , or approaching it in a creative/artistic way. Thats' what I was referring to. I can't speak for maths of hard science as I'm not trained in that, but the creativity aspect in code is definitely lacking when it comes to LLMs. May not matter down the line.
blueone 4mo ago
> what basis do you have for assuming an LLM is fundamentally incapable of doing this? because I have no basis for assuming an LLM is fundamentally capable of doing this.
- sswatson 4mo ago
  Good on you for spelling out this reasoning, but it is manifestly unsound. For a wide variety of values of X, people a few years ago had no reason to expect that LLMs would be capable of X. Yet here we are.
  - TheOtherHobbes 4mo ago
    In 1989, Gary Kasparov said that it was "ridiculous!" to suggest a computer would ever beat him at chess. "Never shall I be beaten by a machine!” In 1997 he lost to Deep Blue.
    - FartyMcFarter 4mo ago
      Yeah, and back then people moved the goal posts too, saying Deep Blue was just "brute-forcing" chess (which isn't even true since it's not a pure minimax search).
      - bananaflag 4mo ago
        Deep Blue was brute forcing chess in the sense that AlphaGo wasn't brute forcing Go.
        FartyMcFarter 4mo ago
        Both of them contained a search algorithm that explored some moves from each considered position, usually not all moves. Both of them contained logic (learned or programmed) to evaluate moves and/or positions. The differences between them are many, but brute force doesn't enter into it in either case.
    - 4mo ago

dvt 4mo ago

Because by definition LLMs are permutation machines, not creativity machines. (My premise, which you may disagree with, is that creativity/imagination/artistry is not merely permutation.)

KoolKat23 4mo ago
It pretty much is, otherwise it is randomness or entropy.
nh23423fefe 4mo ago
god of the gaps
- iwontberude 4mo ago
  non overlapping magisteria
fnordpiglet 4mo ago
I prefer to think of it as they’re interpolation machines not extrapolation machines. They can project within the space they’re trained in, and what they produce may not be in their training corpus, but it must be implied by it. I don’t know if this is sufficient to make them too weak to create original “ideas” of this sort, but I think it is sufficient to make them incapable of original thought vs a very complex to evaluate expected thought.
- drdeca 4mo ago
  People keep saying this, but if you try to interpret this at all literally, it just doesn’t work. Like, it’s phrased like it should have a precise meaning, right? Like, people even mention convex hulls when talking about it. But if you actually try to take a convex hull of, some encoding of sentences as vectors? It isn’t true. The outputs are not in the convex hull of the training data. I guess it’s supposed to be a metaphor and not literal, but in that case it’s confusing. Especially seeing as there are contexts in machine learning where literal interpolation vs literal extrapolation, is relevant. So, please, find a better way to say it than saying that “it can only interpolate”?
  - Muromec 4mo ago
    If it's all just points in the multidimensional space, why would the thing be restricted to some operations and not others. I'm not buying the argument
    - drdeca 4mo ago
      Sorry, I don't understand what you mean. Are you agreeing or disagreeing with me? If it can only interpolate in a literal sense, that means that it only produces good outputs on convex combinations of inputs that appear in the training set. That's what interpolation means. But, if you take the embedding vectors of sentences/prompts, and then take the convex hull of these, it is not typical for new sentences not in the training set to have its embedding vectors be in the convex hull of these.
      - fnordpiglet 4mo ago
        I’m not sure I follow your end to end reasoning. In an n dimensional space interpolation along and within the convex hull is pretty much what they’re doing. How can it possibly not be? How would it interpolate a point that’s not within its vector space? Yes, it’s very complex with non linear transformations and a very high dimensionality, and residuals and other features create more complexity in the shape of the hull. But an LLM can not infer a concept to which it has no information channel. That’s clearly nonsense. The fact that they do bounded, learned, nonlinear compositional generalizations over a representational space induced by training -is by nature interpolation- not extrapolation. I’m sorry, but I believe their immense power has you confusing math with magic.
        drdeca 4mo ago
        A convex hull is a different thing than the linear span. It is smaller. And, my point is that the inputs it is often fed are not in the convex hull of the inputs in the training data. When the input space is very high dimensional, this is a common outcome. I’m not denying that the outputs are causally downstream from the training data. Of course it is. I’m saying that the inference time inputs aren’t in the convex hull of the training time inputs. This isn’t about saying that the output isn’t because of the training data. Of course it is. But when you have very high dimensional input space, then even with many inputs in the training data, it is still common for inference time inputs to not be in the convex hull of the train time inputs. This has nothing to do with the complexities of how the models work after the initial embedding of the tokens as vectors. It’s just about the inputs that appear during training, and the inputs that appear at inference time. > But an LLM can not infer a concept to which it has no information channel. Of course! And nothing I said implies otherwise. Really, the point I’m making doesn’t even depend on what the model outputs! If I took a best fit line from 1 parameter to a 1D output, and then provided that linear model an output that was outside the range of inputs the best fit line was obtained from, that would not be interpolation, it would be extrapolation. It is similar here, except instead of the input being outside the convex hull due to being further away, it is outside the convex hull due to, like, the shape of the convex hull of training inputs just doesn’t include the point in question.
lajamerr 4mo ago
LLMs by themselves are not able to but you are missing a piece here. LLMs are prompted by humans and the right query may make it think/behave in a way to create a novel solution. Then there's a third factor now with Agentic AI system loops with LLMs. Where it can research, try, experiment in its own loop that's tied to the real world for feedback. Agentic + LLM + Initial Human Prompter by definition can have it experiment outside of its domain of expertise. So that's extending the "LLM can't create novel ideas" but I don't think anyone can disagree the three elements above are enough ingredients for an AI to come up with novel ideas.
- awesome_dude 4mo ago
  You're proving the GP's argument - LLMs aren't creative you say as much, it's the driving that is the creative force
  - Barbing 4mo ago
    If that’s a requirement, aren’t LLMs driven by pretraining which was human driven? Who decides at which the last point it’s OK to provide text to the model in order to be able to describe it as creative? (non-rhetorical)
  - lajamerr 4mo ago
    You can tell an agentic system. "Go and find a novel area of math that has unresolved answers and solve it mathematically with verified properties in LEAN. Verify before you start working on a problem that no one has solved this area of math" That's not creative prompt. That's a driving prompt to get it to start its engine. You could do that nowadays and while it may spend $1,000 to $100,000 worth of tokens. It will create something humans haven't done before as long as you set it up with all its tool calls/permissions.
    - awesome_dude 4mo ago
      Let me know when the Fields medal arrives in the mail. It won't because even though it looks clever to you, people who /do/ understand math and LLMs understand that LLMs /are/ regurgitating Why does your LLM need you to tell it to look in the first place? Why isn't just telling us all the answers to unsolved conjectures known and unknown? Why isn't the LLM just telling us all the answers to all the problems we are facing? Why isn't the LLM telling us, step by step with zero error, how to build the machine that can answer the ultimate question?
lukol 4mo ago
This "new math" might be a recombination of things that we already know - or an obvious pattern that emerges if you take a look at things from a far enough distance - or something that can be brute-forced into existence. All things LLMs are perfectly capable of. In the end, creativity has always been a combination of chance and the application of known patterns in new contexts.
- dvt 4mo ago
  > This "new math" might be a recombination of things that we already know If you know anything about the invention of new math (analytic geometry, Calculus, etc.), you'd know how untrue this is. In fact, Calculus was extremely hand-wavy and without rigorous underpinnings until the mid 1800s. Again: more art than science.
  - baq 4mo ago
    And yet nowadays you can restate all of it using just combinations of sets of sets and some logic operators.
  - jfyi 4mo ago
    Newton and Leibniz were "hand-waving"? If anything, they were fighting an uphill battle against the perception of hand-waving by their contemporaries.
    - dvt 4mo ago
      > Newton and Leibniz were "hand-waving"? Yes, and it's pretty common knowledge that Calculus was (finally) formalized by Weierstrass in the early 19th century, having spent almost two centuries in mathematical limbo. Calculus was intuitive, solved a great class of problems, but its roots were very much (ironically) vibes-based. This isn't unique to Newton or Leibniz, Euler did all kinds of "illegal" things (like playing with divergent series, treating differentials as actual quantities, etc.) which worked out and solved problems, but were also not formalized until much later.
      - 4mo ago
satvikpendem 4mo ago
What is creativity if not permutation? A brain has some model of the world and recombines concepts to create new concepts.
- d3ffa 4mo ago
  [flagged]
  - rowanG077 4mo ago
    This is really not an acceptable reply. How about actually engaging with the point the commenter made instead of stamping your foot and throwing a tantrum.
  - anthk 4mo ago
    Innovation it's just another word for the term 'enhanced copy'. Everything it's a copy, except for nature.

truncate 4mo ago

What's your basis for assuming LLM is capable of doing this? I honestly don't know personally either way. Based on my limited understanding of how LLMs work, I don't see them be making the next great song or next great book and based on that reasoning I'm betting that it probably wont be able to do whatever next "Descartes, Newton, Leibnitz, Gauss, Euler, Ramanujan, Galois" are going to do. Of course AI as a wider field comes up with something more powerful than LLM that would be different.

redsocksfan45 4mo ago
[dead]
dist-epoch 4mo ago
LLMs are already making the next great songs. Just check out the Billboard charts.
- truncate 4mo ago
  I'm sorry, I don't consider them "great songs". Obviously, different people have different taste.
EMM_386 4mo ago
"I don't see them be making the next great song" Meanwhile, songs are hitting number one on some charts on Spotify that people think are humans and are actually AI. And Spotify has to start labelling them as such. One AI "band" had an entire album of hits. Also - music is a subjective. Mathematics isn't. And in this case, an LLM discovered a new way to reason about a conjecture. I don't know how much proof is needed - since that is literally proof that it can be done.
- truncate 4mo ago
  >> Meanwhile, songs are hitting number one on some charts on Spotify that people think are humans and are actually AI. And Spotify has to start labelling them as such. One AI "band" had an entire album of hits. There is quite some questions around that. Music is subjective and obviously different people have different taste, but I wouldn't call any of them to be actual good music / real hits. >> LLM discovered a new way to reason about a conjecture I wasn't questioning LLMs ability to prove things. Parent threads were talking about building new kind of maths , or approaching it in a creative/artistic way. Thats' what I was referring to. I can't speak for maths of hard science as I'm not trained in that, but the creativity aspect in code is definitely lacking when it comes to LLMs. May not matter down the line.

blueone 4mo ago

> what basis do you have for assuming an LLM is fundamentally incapable of doing this? because I have no basis for assuming an LLM is fundamentally capable of doing this.

sswatson 4mo ago
Good on you for spelling out this reasoning, but it is manifestly unsound. For a wide variety of values of X, people a few years ago had no reason to expect that LLMs would be capable of X. Yet here we are.
- TheOtherHobbes 4mo ago
  In 1989, Gary Kasparov said that it was "ridiculous!" to suggest a computer would ever beat him at chess. "Never shall I be beaten by a machine!” In 1997 he lost to Deep Blue.
  - FartyMcFarter 4mo ago
    Yeah, and back then people moved the goal posts too, saying Deep Blue was just "brute-forcing" chess (which isn't even true since it's not a pure minimax search).
    - bananaflag 4mo ago
      Deep Blue was brute forcing chess in the sense that AlphaGo wasn't brute forcing Go.
      - FartyMcFarter 4mo ago
        Both of them contained a search algorithm that explored some moves from each considered position, usually not all moves. Both of them contained logic (learned or programmed) to evaluate moves and/or positions. The differences between them are many, but brute force doesn't enter into it in either case.
  - 4mo ago