5 ms·

I was so optimistic about using LLMs for "write once, read many" English language documents, but the more I've used the tools, the more pessimistic I get. More

by ahepp 11d ago

I was so optimistic about using LLMs for "write once, read many" English language documents, but the more I've used the tools, the more pessimistic I get. More and more, I try to ask it for low prose responses because its writing just seems like such a low signal to noise ratio I'm curious about why LLM writing fails. Particularly whether LLM writing is fundamentally flawed, or if it's just distinctive and since it often reflects low effort, that distinctive voice is associated with low quality. I find its reliance on extremely consistent rhetorical patterns concerning. The fact that it always finds a way to talk about how "It's not the X, it's the Y Z" no matter what topic you feed it, makes me concerned that the tail is wagging the dog

classified 11d ago
> I'm curious about why LLM writing fails. Apart from the tasteless manipulations of the providers, it's mostly training data. LLMs output the average of their training data, and the overwhelming majority of humans are bad writers.
- HappyPanacea 11d ago
  This argument doesn't work, the average majority of humans are bad at math but recent LLM aren't.
  - bfbf 10d ago
    But people who are bad at maths are unlikely to be writing about maths. A crude example might be if you search for “2+2=” in the training data, you’re much more likely to find “4” as the next character. Obviously llms are far more complex than this, but I think this proves the point. The fact you had to add the “recent” qualifier there highlights that llms in general were bad and had to be provided with corrective targeted training data to improve. (And they still can’t count the R’s in strawberry!)
    - rmunn 10d ago
      > And they still can’t count the R’s in strawberry! Really? I do not have the time to survey the modern LLMs to see if your assertion is correct, but if it is then I'm surprised; I would have thought that that one would have shown up so often in their training data that they would be able to answer that question, even if they would then be unable to (for example) count the R's in raspberry, or in some other word where "count the R's in _____" was not widely found in recent online discussion.
      - bfbf 10d ago
        I mean a very quick check in ChatGPT got it wrong today, yeah! I’m not sure what model I was using, but it proves the point!
    - classified 9d ago
      > they still can’t count the R’s in strawberry! My resident Qwythos-9B counts 3 "r"s in "strawberry". So even locally hosted models are catching up.
- dwaltrip 10d ago
  Nah, the deepest problem is the lack of intent. LLMs don’t have a message they are trying to convey or a clear picture of who the intended audience is. They don’t know what you want to say solely based off a prompt, as it can’t possibly convey enough detail. And they can’t read your mind to fill in the gaps. If it was just training data, that would actually be a much easier problem to solve.
grey-area 10d ago
Lack of taste, intent and intelligence. This is ok in domains if you can train against known good answers and make sure the machine generates conforming text most of the time. It falls apart in fuzzier domains where training is much harder and intent is required (i.e. having something to say). LLM writing is generally ok in factual domains where it can regurgitate bits of wikipedia or answers to questions, they are terrible at long form writing, in particularly in literary styles, because of a lack of intelligence and taste. I don't think the answer lies in the data or in their training. It seems we've had a few years for this problem to be solved, but nobody seems to have worked out an answer to it.
hackeraccount 10d ago
I think it's somewhere in the process it was asked "what's the most compelling written text?" The answer was things from great speeches "Ask not what you ..." and so on. And that really is great and compelling. However. Great and compelling is not what I'm looking for when my question is, "Systemd-networkd is pulling an ip address for a bonded interface that only exist as a 802.1Q trunk. How do I make it stop that?"
- ahepp 9d ago
  Not the main point at all, but what was it? Pattern matching rule nestled deep in /lib/systemd/network? Some kind of mysterious netplan / NetworkManager compat layer?
armchairhacker 10d ago
Maybe ask it to summarize, improve phrasing, and remove tropes a couple times? Otherwise you have the LLM analogue of a first draft. I doubt it will be as good as humans, because LLMs don't seem to have "taste" (RVLR doesn't work, RHLF is unreliable and inconsistent because the graders don't have your taste or really know what they prefer themselves, especially when overworked and rushed). But I expect it to be better.
voidhorse 10d ago
When human beings write we do so with a particular perspective with an intention to communicate to a particular audience. Even the driest scientific writing is partially informed by the experiences of the researchers, no matter how hard they try to remove it. Human writing has a viewpoint. LLM generated text always lacks these elements. It's always some bland, shapeless sequence of words pulled from the ether. I'm sure with time and effort you could combat this and give LLM text more of a human feeling by giving the LLM enough context about your communication, but that would take work, which in many cases would defeat the purpose of using the LLM instead of doing the writing yourself. And if you care that much about your communication with others (and you should!) you'd probably just find using LLMs frustrating to begin with.

by ahepp 11d ago

classified 11d ago

> I'm curious about why LLM writing fails. Apart from the tasteless manipulations of the providers, it's mostly training data. LLMs output the average of their training data, and the overwhelming majority of humans are bad writers.

HappyPanacea 11d ago
This argument doesn't work, the average majority of humans are bad at math but recent LLM aren't.
- bfbf 10d ago
  But people who are bad at maths are unlikely to be writing about maths. A crude example might be if you search for “2+2=” in the training data, you’re much more likely to find “4” as the next character. Obviously llms are far more complex than this, but I think this proves the point. The fact you had to add the “recent” qualifier there highlights that llms in general were bad and had to be provided with corrective targeted training data to improve. (And they still can’t count the R’s in strawberry!)
  - rmunn 10d ago
    > And they still can’t count the R’s in strawberry! Really? I do not have the time to survey the modern LLMs to see if your assertion is correct, but if it is then I'm surprised; I would have thought that that one would have shown up so often in their training data that they would be able to answer that question, even if they would then be unable to (for example) count the R's in raspberry, or in some other word where "count the R's in _____" was not widely found in recent online discussion.
    - bfbf 10d ago
      I mean a very quick check in ChatGPT got it wrong today, yeah! I’m not sure what model I was using, but it proves the point!
  - classified 9d ago
    > they still can’t count the R’s in strawberry! My resident Qwythos-9B counts 3 "r"s in "strawberry". So even locally hosted models are catching up.
dwaltrip 10d ago
Nah, the deepest problem is the lack of intent. LLMs don’t have a message they are trying to convey or a clear picture of who the intended audience is. They don’t know what you want to say solely based off a prompt, as it can’t possibly convey enough detail. And they can’t read your mind to fill in the gaps. If it was just training data, that would actually be a much easier problem to solve.

grey-area 10d ago

Lack of taste, intent and intelligence. This is ok in domains if you can train against known good answers and make sure the machine generates conforming text most of the time. It falls apart in fuzzier domains where training is much harder and intent is required (i.e. having something to say). LLM writing is generally ok in factual domains where it can regurgitate bits of wikipedia or answers to questions, they are terrible at long form writing, in particularly in literary styles, because of a lack of intelligence and taste. I don't think the answer lies in the data or in their training. It seems we've had a few years for this problem to be solved, but nobody seems to have worked out an answer to it.

hackeraccount 10d ago

I think it's somewhere in the process it was asked "what's the most compelling written text?" The answer was things from great speeches "Ask not what you ..." and so on. And that really is great and compelling. However. Great and compelling is not what I'm looking for when my question is, "Systemd-networkd is pulling an ip address for a bonded interface that only exist as a 802.1Q trunk. How do I make it stop that?"

ahepp 9d ago
Not the main point at all, but what was it? Pattern matching rule nestled deep in /lib/systemd/network? Some kind of mysterious netplan / NetworkManager compat layer?

armchairhacker 10d ago

Maybe ask it to summarize, improve phrasing, and remove tropes a couple times? Otherwise you have the LLM analogue of a first draft. I doubt it will be as good as humans, because LLMs don't seem to have "taste" (RVLR doesn't work, RHLF is unreliable and inconsistent because the graders don't have your taste or really know what they prefer themselves, especially when overworked and rushed). But I expect it to be better.

voidhorse 10d ago

When human beings write we do so with a particular perspective with an intention to communicate to a particular audience. Even the driest scientific writing is partially informed by the experiences of the researchers, no matter how hard they try to remove it. Human writing has a viewpoint. LLM generated text always lacks these elements. It's always some bland, shapeless sequence of words pulled from the ether. I'm sure with time and effort you could combat this and give LLM text more of a human feeling by giving the LLM enough context about your communication, but that would take work, which in many cases would defeat the purpose of using the LLM instead of doing the writing yourself. And if you care that much about your communication with others (and you should!) you'd probably just find using LLMs frustrating to begin with.