6 ms·

I was interested in this question so I trained NanoGPT from scratch to sort lists of random numbers. It didn't take long to succeed with arbitrary reliability,

I was interested in this question so I trained NanoGPT from scratch to sort lists of random numbers. It didn't take long to succeed with arbitrary reliability, even given only an infinitesimal fraction of the space of random and sorted lists as training data. Since I can evaluate the correctness of a sort arbitrarily, I could be certain that I wasn't projecting my own beliefs onto its response, and reading more into the output than was actually there. That settled this question for me.

manmal 2y ago
Have you considered that the nature of numeric characters is just so predictable that they can be sorted without actually understanding their numerical value?
- jbay808 2y ago
  Can you say more precisely what you mean?
  - manmal 2y ago
    I mean that maybe gradient descent is a passable sorting algorithm, once the weights have been learned to properly describe ordering. It may be a speciality of transformers that they can sort things well. Which wouldn’t tell us that much about whether they are mentalists or not.
dartos 2y ago
I don’t really understand what you’re testing for? Language, as a problem, doesn’t have a discrete solution like the question of whether a list is sorted or not. Seems weird to compare one to the other, unless I’m misunderstanding something. What’s more, the entire notion of a sorted list was provided to the LLM by how you organized your training data. I don’t know the details of your experiment, but did you note whether the lists were sorted ascended or descended? Did you compare which kind of sorting was most common in the output and in the training set? Your bias might have snuck in without you knowing.
- tossandthrow 2y ago
  Commenter is merely saying that LLMs indeed are able to approximate arbitrary functions exemplified through sorting. It is nothing new and has been well established in the literature since the 90s. The shared article really is not worth the read and mostly uncovers an author who does not know what he write about.
  - dartos 2y ago
    You’re talking specifically about perceptrons and feed forward neural networks. LLMs didn’t exist in then. Attention only came out in 2017…
    - tossandthrow 2y ago
      Yes? Are you saying that attention is less expressive?
      - dartos 2y ago
        I’m saying that LLMs (models trained on language specifically) are not automatically capable of the same generic function solving. The network itself can be trained to solve most functions (or all, I forget precisely if NNs can solve all functions) But the language model is not necessarily capable of solving all functions, because it was already trained on language.
- IshKebab 2y ago
  A large number of commenters are under the illusion that LLMs are "just" stochastic parrots and can't generalise to inputs not seen in their training data. He was proving that that isn't the case.
  - dartos 2y ago
    Not saying I disagree with the thesis, but I don’t think this proves anything. If every pair of digits appears sorted in the dataset, then that could still be “just” a stochastic parrot. I’m kind of interested to see if an LLM can sort when the dataset specifically omits comparisons between certain pairs of numbers. Also I don’t think OC was responding to commenters, but the article

manmal 2y ago

Have you considered that the nature of numeric characters is just so predictable that they can be sorted without actually understanding their numerical value?

jbay808 2y ago
Can you say more precisely what you mean?
- manmal 2y ago
  I mean that maybe gradient descent is a passable sorting algorithm, once the weights have been learned to properly describe ordering. It may be a speciality of transformers that they can sort things well. Which wouldn’t tell us that much about whether they are mentalists or not.

dartos 2y ago

I don’t really understand what you’re testing for? Language, as a problem, doesn’t have a discrete solution like the question of whether a list is sorted or not. Seems weird to compare one to the other, unless I’m misunderstanding something. What’s more, the entire notion of a sorted list was provided to the LLM by how you organized your training data. I don’t know the details of your experiment, but did you note whether the lists were sorted ascended or descended? Did you compare which kind of sorting was most common in the output and in the training set? Your bias might have snuck in without you knowing.

tossandthrow 2y ago
Commenter is merely saying that LLMs indeed are able to approximate arbitrary functions exemplified through sorting. It is nothing new and has been well established in the literature since the 90s. The shared article really is not worth the read and mostly uncovers an author who does not know what he write about.
- dartos 2y ago
  You’re talking specifically about perceptrons and feed forward neural networks. LLMs didn’t exist in then. Attention only came out in 2017…
  - tossandthrow 2y ago
    Yes? Are you saying that attention is less expressive?
    - dartos 2y ago
      I’m saying that LLMs (models trained on language specifically) are not automatically capable of the same generic function solving. The network itself can be trained to solve most functions (or all, I forget precisely if NNs can solve all functions) But the language model is not necessarily capable of solving all functions, because it was already trained on language.
IshKebab 2y ago
A large number of commenters are under the illusion that LLMs are "just" stochastic parrots and can't generalise to inputs not seen in their training data. He was proving that that isn't the case.
- dartos 2y ago
  Not saying I disagree with the thesis, but I don’t think this proves anything. If every pair of digits appears sorted in the dataset, then that could still be “just” a stochastic parrot. I’m kind of interested to see if an LLM can sort when the dataset specifically omits comparisons between certain pairs of numbers. Also I don’t think OC was responding to commenters, but the article