6 ms·

I like the term I've seen someone use for LLMs: a "thought partner." Truly it's the duck programming duck I've always wanted.

Brian_K_White 2mo ago
I say that LLM's are a pacifier. LLM output is to conscious output as a silicone nipple is to a breast. Sure, at some level, there are some people who at least some times, cannot pecieve a difference. That does not prove that there is no difference or that the difference doesn't matter.
- WorldMaker 2mo ago
  I think that LLM's are a broken mirror or maybe disco ball. What you get back is almost always a reflection of what you put in, just garbled through and with the reflections of so many other people. The best parts that I often get back are mostly just the stuff that I already put in it. I think that's also where LLMs get the most dangerous. That ELIZA effect of telling you exactly what you told it but in a slightly rephrased way makes you think that you think that your ideas are genius. You've got someone "actively listening" and telling you whatever idea you just gave it was the best idea it ever heard. That's not what you want from a serious writing partner. You want a writing partner to challenge you, to point out your mistakes and flaws, to make you better, to offer a truly different perspective to your own. You don't want a dark reflection from a "yes man" to spiral into.
  - foobarian 2mo ago
    I think it is a net positive for me, because I am super unconfident by nature and the agreeable, almost sycophantic output from major LLMs give me just enough of a boost to debias me. They are certainly better than a physical rubber duck.
    - WorldMaker 2mo ago
      I feel it is a net negative for me. One of my hobbies has always been fiction writing. The LLM reflections of my own writing become slot machines to me for cheap dopamine hits. It's not because LLMs are good at fiction writing, far from it, it's that LLMs are good at reflecting my own writing back to me in fun house mirror ways that amuse me and frustrate me and pleasantly surprise me. That slot machine addiction gives me a lot of perspective on what LLMs are good at in coding versus what they appear to be good at. It's easy to characterize them as "over-eager junior developers" or "powerful rubber duck debuggers" but their biases are still your biases so long as you are the one writing the prompts. A lot of the "surprises" aren't intelligence but fun house mirror versions of the same biases you input into the system. I keep referring to their coding capabilities as "legacy code as a service" and the addiction to the fictional equivalent only underscores that understanding of LLMs. You don't write the next great American novel getting a word salad of past great novels back full of the best and worst of your own writing. You don't write the next great American software getting a legacy code jumble of past great software back full of the best and worst of your own prompting. At the end of the day I'm still a pragmatist and using the LLMs in my programming efforts, but I don't trust them as a partner, I use them as a tool. (A biased tool that I know needs a careful eye.)
      - bsenftner 1mo ago
        > but their biases are still your biases so long as you are the one writing the prompts. That is if you only work with your LLMs / Agents in a "straight ahead" manner, versus using knowledge of the bias baked into AIs and purposefully invoking a specific type of subject matter expert for problems that require specialized knowledge. It is similar to using people skilled at what you need, invoke through the vocabulary and language of your prompts specialized skills that are not tapped until using the terminology only those specialists use, and like magic their skills are suddenly in-context and the AI acts accordingly.

Brian_K_White 2mo ago

I say that LLM's are a pacifier. LLM output is to conscious output as a silicone nipple is to a breast. Sure, at some level, there are some people who at least some times, cannot pecieve a difference. That does not prove that there is no difference or that the difference doesn't matter.

WorldMaker 2mo ago
I think that LLM's are a broken mirror or maybe disco ball. What you get back is almost always a reflection of what you put in, just garbled through and with the reflections of so many other people. The best parts that I often get back are mostly just the stuff that I already put in it. I think that's also where LLMs get the most dangerous. That ELIZA effect of telling you exactly what you told it but in a slightly rephrased way makes you think that you think that your ideas are genius. You've got someone "actively listening" and telling you whatever idea you just gave it was the best idea it ever heard. That's not what you want from a serious writing partner. You want a writing partner to challenge you, to point out your mistakes and flaws, to make you better, to offer a truly different perspective to your own. You don't want a dark reflection from a "yes man" to spiral into.
- foobarian 2mo ago
  I think it is a net positive for me, because I am super unconfident by nature and the agreeable, almost sycophantic output from major LLMs give me just enough of a boost to debias me. They are certainly better than a physical rubber duck.
  - WorldMaker 2mo ago
    I feel it is a net negative for me. One of my hobbies has always been fiction writing. The LLM reflections of my own writing become slot machines to me for cheap dopamine hits. It's not because LLMs are good at fiction writing, far from it, it's that LLMs are good at reflecting my own writing back to me in fun house mirror ways that amuse me and frustrate me and pleasantly surprise me. That slot machine addiction gives me a lot of perspective on what LLMs are good at in coding versus what they appear to be good at. It's easy to characterize them as "over-eager junior developers" or "powerful rubber duck debuggers" but their biases are still your biases so long as you are the one writing the prompts. A lot of the "surprises" aren't intelligence but fun house mirror versions of the same biases you input into the system. I keep referring to their coding capabilities as "legacy code as a service" and the addiction to the fictional equivalent only underscores that understanding of LLMs. You don't write the next great American novel getting a word salad of past great novels back full of the best and worst of your own writing. You don't write the next great American software getting a legacy code jumble of past great software back full of the best and worst of your own prompting. At the end of the day I'm still a pragmatist and using the LLMs in my programming efforts, but I don't trust them as a partner, I use them as a tool. (A biased tool that I know needs a careful eye.)
    - bsenftner 1mo ago
      > but their biases are still your biases so long as you are the one writing the prompts. That is if you only work with your LLMs / Agents in a "straight ahead" manner, versus using knowledge of the bias baked into AIs and purposefully invoking a specific type of subject matter expert for problems that require specialized knowledge. It is similar to using people skilled at what you need, invoke through the vocabulary and language of your prompts specialized skills that are not tapped until using the terminology only those specialists use, and like magic their skills are suddenly in-context and the AI acts accordingly.