4 ms·

Phrases and ideas repeat, because the universe has patterns and structure.

• tgv 2y ago
  But it also suggests that the neural networks are not the miracle as some proclaim. They get their knowledge from compressing a very large amount of text, but add little in terms of inference. Its success is then a function of its size, but that size cannot grow much. It also ties in with the idea that the current ANNs don't discriminate text, and repeat bad input as readily as anything else.
  • naasking 2y ago
    Compressing nearly all of human knowledge into a desktop PC that can converse in natural language and translate between languages is the miracle.
    • tgv 2y ago
      It's miraculous, but it's not nearly nearly all of human knowledge. Ask anything slightly specialist, and it'll give wrong answers. ChatGPT's translation is also not well developed. Llama doesn't have any, I believe.