4 ms·
The style you refer to is the "container" of the writing. The medium. Like the specific encoding of the message. What @arctic-true was talking about was the "co
by ivansavz 10d ago
The style you refer to is the "container" of the writing. The medium. Like the specific encoding of the message. What @arctic-true was talking about was the "content" of the writing, which is bounded from above[1] by the information content of the prompt.
So, I'm not sure if it's a question of time at all: if a LLM text contains some piece of information beyond the information that went into the prompt, where does this "extra" information come from? [Note, I'm not thinking about facts which could trivially come from the training corpus, I'm thinking specifically as information in the sense of intended message from sender (author) to receiver (reader)]
[1] cf. this comment where I explain this analogy between LLMs and noise channel in communication theory: https://news.ycombinator.com/item?id=49510244 https://news.ycombinator.com/item?id=49510244