10 ms·
I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and
by npilk 1mo ago
I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want.
On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense.
Even good human writers would probably struggle with this constraint. It would be like someone interrupting your writing to tell you the next word MUST be such-and-such, and then you have to try and make it work as best you can first try, without going back to edit. The result would probably be a little clunky. (Maybe it’s impressive LLMs write as well as they do.)
- scuppernong 1mo agoauto-oulipo
- Alive-in-2025 1mo agoToday I learned a new word, "Oulipo". Interesting. But what about the general idea that they can watermark results to tell where they came from. The next step is tracking down which user got a result. I hate both of these things. Must everything we do be tracked? Next altering wikipedia results so they can tell who looked at the page or something? I'd like "the best answer" from an llm and don't want to be tracked, but this isn't for me, it is for them. I understand llm results are already using a varying statistical input so they aren't always the same. But I really hate watermarking and likely tracking too.
- hyusap 1mo agoautoregressive generation doesn’t mean the model is myopic. the next-token distribution can already reflect a longer horizon plan for the output sequence.
- npilk 1mo agoSure, but mightn’t there be several plausible long horizon plans? Here’s an example: I had asked Claude for some music recommendations in a certain style. Part of its output was: — *Long journey tracks* Clinic — “The Return of Evil Bill” Guided by Voices — not really, wrong band Silver Apples — “Oscillations”. Proto-everything, deeply repetitive, hypnotic. — So at some point there, the next token produced was “Guided” or “Guide” or whatever, and then because it can’t go back, it had to correct itself after the fact. Reasoning/CoT have helped a lot, but I feel like small versions of this still happen all the time. Human writing is like 90% editing.
- inigyou 1mo agoWould be fun to run an LLM on fake output from itself. Like just force the first N tokens to say the beginning of something really stupid, and then see how it finishes the sentence. "You're absolutely right! Human feces is actually the most effective engine coolant because $<completion>"
- Gracana 1mo agoSome LLM interfaces allow you to modify and “continue” an agent response. It’s very useful for guidance, including jailbreaking. Need the model to go in a certain direction? Got a refusal that you want to bypass? Just start it off in the appropriate direction and then have it continue from there.
- inigyou 1mo agoIt can but it is limited because it's only got a single pass through the network to fit the entire "longer horizon plan".
- mholm 1mo agoThis was true in the ChatGPT era. Now we're in a world with reasoning tokens, where a model can thoroughly plan out the response it wants to make. If anything, it makes the style worse.
- tomrod 1mo agoIsn't this just chain-of-thought though, doing the same thing multiple times without necessarily defining one path?
- brookst 1mo agoChain of thought attempts to guide a single forward pass. Reasoning tokens with tool calling tell the model to loop on a one phase of a question and call a tool to indications completion when done. Related, but not the same thing.
- npilk 1mo agoYes, models can reason and plan, which helps them write more coherently. But when they write the final output, it’s still a single generation. It would be like letting a human make notes and write an outline, but not let them use the backspace once they start typing their response. Presumably you could use the same reasoning trace, run multiple generations, and get different outputs (if the temperature is >0). But now I’m interested in playing more with Cowork or Claude Code/Codex for prose writing to see if the set of tools there affects outputs at all. I guess you might need a more custom “writing” harness.
- mholm 1mo agoThere's been a lot of effort into the writing space, and the models genuinely prefer this style. You can let them iterate on the same idea 100 times, rewrite sentences, determine what works best — and they'll still verb the noun, do rule of 3, and keep the same monotonous structure.
- cush 1mo agoModels can easily do multiple passes
- brookst 1mo agoThis is the classic misunderstanding that LLMs only pick the next token at a time. Really, they are coalescing the probabilities of a range of tokens at a time. There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.
- doctorpangloss 1mo agoeveryone in this thread is saying something kind of imprecise and reductive and varies between models and even modifications among the models
- npilk 1mo agoBut on some level there is uncertainty, right? Even if it’s not token-specific but at the word- or phrase-level? Otherwise what does the temperature setting do? Or has architecture changed significantly in the background?
- moralestapia 1mo agoThere are diffusion-based models and transformer-based models (and many other "architectures"), so your comment does not make sense.
- inigyou 1mo agoAre there any diffusion-based or otherwise non-transformer-based models in mainstream use?
- moralestapia 1mo agoIf by mainstream you mean "ChatGPT" or "Claude", then no. But also, no one really knows as they're closed.
- Sharlin 1mo agoIf you include non-language models, yes.
- disillusioned 1mo agoIt's a bit like trying to finish a sentence when you're really stoned... you vaguely remember the preceding couple of words you've said but don't really know how you got there and now you're wandering in the forest trying to stumble on coherency. Well, I suppose it's nearly the opposite of that experience, upon further review. But for some reason, that's where my head jumped.
- inigyou 1mo agoYou're thinking of Markov chains.
- robocat 1mo agoAfter "stoned" you triggered into a different state. You be a human who's brain shifted into LLM mode (chainneling Markov?). Or perhaps you're an LLM impersonating humanity. I often wonder how much LLMs are just mirroring our own brain's patterns.
- danofsteel32 1mo agoI tried brainstorming what an agent harness for writers would look like. https://chainofbranches.com/conversations/2/branches/20/ https://chainofbranches.com/conversations/2/branches/20/ I’m not convinced it’s possible. A good nights sleep and a notepad in a quiet room still feels like the state of the art toolchain for writers.
- halJordan 1mo agoHumans already do struggle with this constraint. Good examples are JRR Martin, Tolkien, and Rothfuss. You cant describe the struggle of picking the next word and then act like humans don't sit at the table struggling to pick the next word.
- lazide 1mo agoHuman writers do better because they can think, and adjust, based on context. They are also usually worse (which is often better!) because they are usually lazy and don’t want to spend effort they do not have too, to accomplish their goals. Their goals are often complex and nuanced. None of this is true of LLMs.
- lesostep 1mo agoI also really liked the quote "Your existence is not impossible, but it's also not very likely" from the Night Vale podcast. I feel like the existence of good writing is also not impossible but not very likely, and so of course LLM can only write mediocrity, even when taught only on great writing.
- zimpenfish 1mo ago> Even good human writers would probably struggle with this constraint. But that would be a fun writing exercise, I think. Thoroughly in the oulipo wheelhouse. Maybe generate a Markov chain table over all of Project Gutenberg and then say every 10th word is whatever the Markov Chain thinks it should be at that point? Or every Nth word has a P% possibility to be constrained by the chain? Optionally with the possibility building for each skipped word to guarantee it happens at some point. Bonus with this approach is that the human can't game the words leading up to the constraint because you don't know when it will happen.
- eru 1mo ago> [...] and go back and restructure sentences and paragraphs if we want. That's what LLMs in reasoning mode do, too, to the text they present to you.