6 ms·

Yea. The bad thing as non native English speaker and spending whole days with LLMs at this point I can’t event notice that’s something sounding bad. Any hints

Yea. The bad thing as non native English speaker and spending whole days with LLMs at this point I can’t event notice that’s something sounding bad. Any hints what should I look at to avoid this? I hope you like the concept of the ox at least ;)

derefr 26d ago
That second sentence you wrote there? That's a good, normal English-language sentence. But LLMs never generate that kind of sentence if they can help it; they break it up into a bunch of tiny "flat" top-level sentences. If you think about how you speak in your native language, it probably has a certain rhythm of long and short sentences, with some "shallow" sentences that just say one thing, but then sentences that nest other clauses that could be whole sentence of their own, and then a hanging sentence fragment that makes sense in context, etc. As far as I know, every spoken language, English or otherwise, looks like that when humans are writing it. The human mind's "buffer of verbalization" seems to be quite short, basically around one grammatical "clause" in size. So humans, when writing (or speaking) "off the cuff", generally only try to keep "a non-verbalized concept of what they want to say" plus "the verbal pattern for the current grammatical clause" buffered in their heads. A human speaker will only start deciding how to glue the next clause onto the current clause—whether to make it a new sentence, or use some preposition or conjunction, or to "verbally backtrack" / "interrupt themselves" to add detail "before" what they said—when they get to near the end of speaking/writing a clause. Much of the "reason" for the grammars of spoken languages to be structured the way they are, is to allow for this kind of narrow-buffered "streaming" composition. Human written language can look different when someone has sat down and taken this "off the cuff" writing as a first draft, and intensively edited and rearranged and polished it. But the result of doing this still usually retains a lot of the original positive qualities of the "off the cuff" writing that went in. (Editors are told to not over-edit, because doing so will remove the "author's voice" from the writing. The particular grammatical gymnastics a speaker/writer uses to connect their thoughts can be a large part of this "author's voice.") LLMs, despite "streaming" in a much more literal sense than humans do, seem to avoid "off the cuff" generation of successive grammatical clauses using "whatever grammatical glue works to get to the next thought." Instead, they seem to have been forced by their training into favoring particular sentence structures that allow them to never end up needing to reach for artful just-in-time grammatical connections in the first place. Mainly, they like using sequences of short sentences that each say exactly one thing. (I hypothesize they like these forms because, in some internal layer of the model, these "simple" sentences can be represented all-at-once as plans [with that same plan getting reconstructed on each successive inference-step during emission of the sentence]; and so this kind of sentence can be emitted in a token order that results in the "polished, edited writing" style rather than the "off-the-cuff speaking" style. Much of the base-model training dataset — the stuff that made the model understand language and writing at all — came from polished, edited writing rather than casual/conversational writing. However much the model is trained to adopt a casual style, it's doing so on top of a language-generation "module" that learned to write by trying to emit "polished, edited writing" one token at a time. And the only way it managed to do that was by limiting itself to constructing sentences that could look like "polished, editing writing" despite a bounded ability to plan.)
- bobowski 26d ago
  Wooow. I never really thought about it this way, but kind of it makes sense. Humans more intend to stream concepts in form of words when LLM just stream words that relate to the current concepts. It’s almost identical but the first approach makes the reader feel the writer wants to arrive somewhere
arjie 26d ago
I actually don’t know what the concept is and I’m not going to look to see if it’s real. There is a lot of spam like this now on the Internet. Sorry, you might be a real Nigerian prince in urgent need of my bank account details but I can’t really know. I use HTMX and like it so I might have read if it didn’t look like spam post. Good luck with your project. Your English fluency is fine.
- bobowski 26d ago
  I meant the headline - because I understand that’s what you refer to. I’ll try to make is less like spam post next time. Just headlining as „built tiny js lib that works like HTMX live or datastar but also slightly different” Anyhow, thanks for commenting - will try to improve for next time Thanks! PS: send your bank details via email, not here in public ;)
dundercoder 26d ago
When I learned Spanish and had the opportunity to speak with native Spanish speaking friends I told them “if you love me, correct my Spanish” I know of no other way, because there are lots of technically grammatically correct ways to say things in English, but sounds awkward to a native speaker. You can’t know really on your own.
- bobowski 26d ago
  LLMs now will tell you that all your farts smell amazing - I kind of feel my English degrading over the past years. Gosh I have to work on it more intentiaonnly
- goodmythical 26d ago
  There are lots of funny "why do you speak like a man/woman" or "why do you speak like a children's cartoon/radio announcer/train conductor" examples in language learning forums. It always comes out that the learner speant to long with or only used one source for their learning. As a counter-example, chinese ESL learners will often look for "American Business English" because our office speak is different enough from common speach and that is their intended use case for the language they're learning. They don't want to sound like they're announcing the news or acting in a movie, they want to sound like business professionals.
  - arjie 26d ago
    I get a lot of that from my wife and in-laws who find my zài nǎr amusing rather than the zài nǎlǐ they'd say. You always know who's a Pimsleur boy.
  - bobowski 26d ago
    „I am groot”
hiccuphippo 24d ago
Read a lot of pre-ai novels.

derefr 26d ago

That second sentence you wrote there? That's a good, normal English-language sentence. But LLMs never generate that kind of sentence if they can help it; they break it up into a bunch of tiny "flat" top-level sentences. If you think about how you speak in your native language, it probably has a certain rhythm of long and short sentences, with some "shallow" sentences that just say one thing, but then sentences that nest other clauses that could be whole sentence of their own, and then a hanging sentence fragment that makes sense in context, etc. As far as I know, every spoken language, English or otherwise, looks like that when humans are writing it. The human mind's "buffer of verbalization" seems to be quite short, basically around one grammatical "clause" in size. So humans, when writing (or speaking) "off the cuff", generally only try to keep "a non-verbalized concept of what they want to say" plus "the verbal pattern for the current grammatical clause" buffered in their heads. A human speaker will only start deciding how to glue the next clause onto the current clause—whether to make it a new sentence, or use some preposition or conjunction, or to "verbally backtrack" / "interrupt themselves" to add detail "before" what they said—when they get to near the end of speaking/writing a clause. Much of the "reason" for the grammars of spoken languages to be structured the way they are, is to allow for this kind of narrow-buffered "streaming" composition. Human written language can look different when someone has sat down and taken this "off the cuff" writing as a first draft, and intensively edited and rearranged and polished it. But the result of doing this still usually retains a lot of the original positive qualities of the "off the cuff" writing that went in. (Editors are told to not over-edit, because doing so will remove the "author's voice" from the writing. The particular grammatical gymnastics a speaker/writer uses to connect their thoughts can be a large part of this "author's voice.") LLMs, despite "streaming" in a much more literal sense than humans do, seem to avoid "off the cuff" generation of successive grammatical clauses using "whatever grammatical glue works to get to the next thought." Instead, they seem to have been forced by their training into favoring particular sentence structures that allow them to never end up needing to reach for artful just-in-time grammatical connections in the first place. Mainly, they like using sequences of short sentences that each say exactly one thing. (I hypothesize they like these forms because, in some internal layer of the model, these "simple" sentences can be represented all-at-once as plans [with that same plan getting reconstructed on each successive inference-step during emission of the sentence]; and so this kind of sentence can be emitted in a token order that results in the "polished, edited writing" style rather than the "off-the-cuff speaking" style. Much of the base-model training dataset — the stuff that made the model understand language and writing at all — came from polished, edited writing rather than casual/conversational writing. However much the model is trained to adopt a casual style, it's doing so on top of a language-generation "module" that learned to write by trying to emit "polished, edited writing" one token at a time. And the only way it managed to do that was by limiting itself to constructing sentences that could look like "polished, editing writing" despite a bounded ability to plan.)

bobowski 26d ago
Wooow. I never really thought about it this way, but kind of it makes sense. Humans more intend to stream concepts in form of words when LLM just stream words that relate to the current concepts. It’s almost identical but the first approach makes the reader feel the writer wants to arrive somewhere

arjie 26d ago

I actually don’t know what the concept is and I’m not going to look to see if it’s real. There is a lot of spam like this now on the Internet. Sorry, you might be a real Nigerian prince in urgent need of my bank account details but I can’t really know. I use HTMX and like it so I might have read if it didn’t look like spam post. Good luck with your project. Your English fluency is fine.

bobowski 26d ago
I meant the headline - because I understand that’s what you refer to. I’ll try to make is less like spam post next time. Just headlining as „built tiny js lib that works like HTMX live or datastar but also slightly different” Anyhow, thanks for commenting - will try to improve for next time Thanks! PS: send your bank details via email, not here in public ;)

dundercoder 26d ago

When I learned Spanish and had the opportunity to speak with native Spanish speaking friends I told them “if you love me, correct my Spanish” I know of no other way, because there are lots of technically grammatically correct ways to say things in English, but sounds awkward to a native speaker. You can’t know really on your own.

bobowski 26d ago
LLMs now will tell you that all your farts smell amazing - I kind of feel my English degrading over the past years. Gosh I have to work on it more intentiaonnly
goodmythical 26d ago
There are lots of funny "why do you speak like a man/woman" or "why do you speak like a children's cartoon/radio announcer/train conductor" examples in language learning forums. It always comes out that the learner speant to long with or only used one source for their learning. As a counter-example, chinese ESL learners will often look for "American Business English" because our office speak is different enough from common speach and that is their intended use case for the language they're learning. They don't want to sound like they're announcing the news or acting in a movie, they want to sound like business professionals.
- arjie 26d ago
  I get a lot of that from my wife and in-laws who find my zài nǎr amusing rather than the zài nǎlǐ they'd say. You always know who's a Pimsleur boy.
- bobowski 26d ago
  „I am groot”

hiccuphippo 24d ago

Read a lot of pre-ai novels.