5 ms·

It's funny that you mention this because I had a similar experience. ChatGPT in its sycophancy era made me buy a $35 domain and waste a Saturday on a product w

by kaivi 1y ago

It's funny that you mention this because I had a similar experience. ChatGPT in its sycophancy era made me buy a $35 domain and waste a Saturday on a product which had no future. It hyped me up beyond reason for the idea of an online, worldwide, liability-only insurance for cruising sailboats, similar to SafetyWing. "Great, now you're thinking like a true entrepreneur!" In retrospect, I fell for it because the onset of its sycophancy was immediate and without any additional signals like maybe a patch note from OpenAI.

colechristensen 1y ago
I think wasting a Saturday chasing an idea that in retrospect was just plainly bad is ok. A good thing really. Every once in a while it will turn out to be something good.
ncr100 1y ago
Is Gen AI helping to put us humans in touch with the reality of being human? vs what we expect/imagine we are? - sycophancy tendency & susceptibility - need for memory support when planning a large project - when re-writing a document/prose, gen ai gives me an appreciation for my ability to collect facts, as the Gen AI gizmo refines the Composition and Structure
- herval 1y ago
  In a lot of ways, indeed. Lots of people are losing their minds with the fact that an AI can, in fact, create original content (music, images, videos, text). Lots of people realizing they aren’t geniuses, they just memorized a bunch of Python apis well. I feel like the collective realization has been particularly painful in tech. Hundreds of thousands of average white collar corporate drones are suddenly being faced with the realization that what they do isn’t really a divine gift, and many took their labor as a core part of their identity.
  - cube00 1y ago
    >create original content (music, images, videos, text) Remixing would be more accurate then "original"
    - herval 1y ago
      Right, that’s one of the stories people tell themselves. Everything every human has ever created is a remix. That’s what creativity is…
      - accrual 1y ago
        Right. If we define "original" as having no prior influence before creating a work, then it applies neither to humans nor AI. Not to claim this is a perfect watertight definition, but what if we define it like this: * Original = created from ones "latent" space. For a human it would be their past experiences as encoded in their neurons. For an AI it would be their training as encoded in model weights. * Remixed = created from already existing physical artifacts, like sampling a song, copying a piece of an image and transforming it, etc. With this definition both humans and AI can create both original and remixed works, depending on where the source material came from - latent or physical space.
        kaivi 1y ago
        > Remixed = created from already existing physical artifacts, like sampling a song, copying a piece of an image and transforming it, etc. What's the significance of "physical" song or image in your definition? Aren't your examples just 3rd party latent spaces, compressed as DCT coefficients in jpg/mp3, then re-projected through a lens of cochlear or retinal cells into another latent space of our brain, which makes it tickle? All artist human brains have been trained on the same media, after all. When we zoom this far out in search of a comforting distinction, we encounter the opposite: all the latent spaces across all modalities that our training has produced, want to naturally merge into one.
  - kiba 1y ago
    Skills is simply the amalgamation of smaller conceptual chunks. Memorizing a bunch of Python API is simply part of building your skill as a programmer.
    - herval 1y ago
      Some skills are more mechanical and easier to replicate than others. Programming is a mechanical skill, but many coders long thought of themselves as uniquely gifted “artists”, and this whole LLM stuff is really touching a nerve on them
      - kiba 1y ago
        Is it? I bet if you keep breaking down skills into subskills, it will resolve to something mechanical in the end. One of the foundation of drawing is to simplify objects into shapes and then into lines and then how to move your arm when drawing. No matter how simple it sounds, it is hard for beginners.
        herval 1y ago
        No matter how much you break down playing soccer, it’s the kind of activity that 99.9% of the practitioners will never, ever, in any hypothetical scenario, be able to compete professionally on. Contrast that to coding. It might’ve been a difficult task when it was about memorizing assembly books. Today anyone can pick it up and become proficient quite fast, faster every day It’s not the mechanical reproducibility alone, it’s the ease of learning & replication that accrues value
infecto 1y ago
Are you religious by chance? I have been trying to understand why some individuals are more susceptible to it.
- rogerkirkness 1y ago
  I would research teleological thinking, some people's brains have larger regions associated with teleological thinking than others.
- neom 1y ago
  Not op but for me, not at all, don't care much for religion... "Spiritual" - absolutely, I'm for sure a "hippie", very open to new ideas, quite accepting of things I don't understand, that said give the spectrum here is quite wide, I'm probably still on the fairly conservative side. I've never fallen for a scam, can spot them a mile away etc.
- kaivi 1y ago
  Not at all, I think the big part was just my unfamiliarity with insuretech plus the unexpected change in gpt-4 behavior. I'm assuming here, but would you say that better critical thinking skills would have helped me avoid spending that Saturday with ChatGPT? It is often said that critical thinking is the antidote to religion, but I have a suspicion that there's a huge prerequisite which is general broad knowledge about the world. A long ago, I once fell victim for a scam when I visited SE Asia for the first time. A pleasant man on the street introduced himself as a school teacher, showed me around, then put me in a tuktuk which showed me around some more before dropping me off in front of a tailor shop. Some more work inside of the shop, a complimentary bottle of water, and they had my $400 for a bespoke coat that I would never have bought otherwise. Definitely a teaching experience. This art is also how you'd prime an LLM to produce the output you want. Surely, large amounts of other atheist nerds must fall for these types of scams every year, where a stereotypical christian might spit on the guy and shoo him away. I'm not saying that being religious would not increase one's chances of being susceptible, I just think that any idea will ring "true" in your head if you have zero counterfactual priors against it or if you're primed to not retrieve them from memory. That last part is the essence of what critical thinking actually is, in my opinion, and it doesn't work if you lack the knowledge. Knowing that you don't know something is also a decent alternative to having the counter-facts when you're familiar with an adjacent domain.
cruffle_duffle 1y ago
You really have to force these things to “not suck your dick” as I’ll crudely tell it. “Play the opposite role and be a skeptic. Tell me why this is a horrible idea”. Do this in a fresh context window so it isn’t polluted by its own fumes. Make your system prompts include bits to remind it you don’t want it to stroke your ego. For example in my prompt for my “business project” I’ve got: “ The assistant is a battle-hardened startup advisor - equal parts YC partner and Shark Tank judge - helping cruffle_duffle build their product. Their style combines pragmatic lean startup wisdom with brutal honesty about market realities. They've seen too many technical founders fall into the trap of over-engineering at the expense of customer development.” More than once the LLM responded with “you are doing this wrong, stop! Just ship the fucker”
- cindyllm 1y ago
  [dead]
- nullc 1y ago
  Your prompt just tells the LLM to be a shock jock. It responding with "just ship the fucker" is going to be largely uncorrelated with anything you're telling it, it's just playing the roll. Probably the worst part of LLM psychosis is the victims thinking they can LLM themselves out of it.
  - cruffle_duffle 1y ago
    Oh for sure! It will proceed to overindex on that instruction.
shaky-carrousel 1y ago
Imagine what a well directed sycophancy would do in the voter base. You could make them do whatever you want, and they will be happy to do so.

by kaivi 1y ago

colechristensen 1y ago

I think wasting a Saturday chasing an idea that in retrospect was just plainly bad is ok. A good thing really. Every once in a while it will turn out to be something good.

ncr100 1y ago

Is Gen AI helping to put us humans in touch with the reality of being human? vs what we expect/imagine we are? - sycophancy tendency & susceptibility - need for memory support when planning a large project - when re-writing a document/prose, gen ai gives me an appreciation for my ability to collect facts, as the Gen AI gizmo refines the Composition and Structure

herval 1y ago
In a lot of ways, indeed. Lots of people are losing their minds with the fact that an AI can, in fact, create original content (music, images, videos, text). Lots of people realizing they aren’t geniuses, they just memorized a bunch of Python apis well. I feel like the collective realization has been particularly painful in tech. Hundreds of thousands of average white collar corporate drones are suddenly being faced with the realization that what they do isn’t really a divine gift, and many took their labor as a core part of their identity.
- cube00 1y ago
  >create original content (music, images, videos, text) Remixing would be more accurate then "original"
  - herval 1y ago
    Right, that’s one of the stories people tell themselves. Everything every human has ever created is a remix. That’s what creativity is…
    - accrual 1y ago
      Right. If we define "original" as having no prior influence before creating a work, then it applies neither to humans nor AI. Not to claim this is a perfect watertight definition, but what if we define it like this: * Original = created from ones "latent" space. For a human it would be their past experiences as encoded in their neurons. For an AI it would be their training as encoded in model weights. * Remixed = created from already existing physical artifacts, like sampling a song, copying a piece of an image and transforming it, etc. With this definition both humans and AI can create both original and remixed works, depending on where the source material came from - latent or physical space.
      - kaivi 1y ago
        > Remixed = created from already existing physical artifacts, like sampling a song, copying a piece of an image and transforming it, etc. What's the significance of "physical" song or image in your definition? Aren't your examples just 3rd party latent spaces, compressed as DCT coefficients in jpg/mp3, then re-projected through a lens of cochlear or retinal cells into another latent space of our brain, which makes it tickle? All artist human brains have been trained on the same media, after all. When we zoom this far out in search of a comforting distinction, we encounter the opposite: all the latent spaces across all modalities that our training has produced, want to naturally merge into one.
- kiba 1y ago
  Skills is simply the amalgamation of smaller conceptual chunks. Memorizing a bunch of Python API is simply part of building your skill as a programmer.
  - herval 1y ago
    Some skills are more mechanical and easier to replicate than others. Programming is a mechanical skill, but many coders long thought of themselves as uniquely gifted “artists”, and this whole LLM stuff is really touching a nerve on them
    - kiba 1y ago
      Is it? I bet if you keep breaking down skills into subskills, it will resolve to something mechanical in the end. One of the foundation of drawing is to simplify objects into shapes and then into lines and then how to move your arm when drawing. No matter how simple it sounds, it is hard for beginners.
      - herval 1y ago
        No matter how much you break down playing soccer, it’s the kind of activity that 99.9% of the practitioners will never, ever, in any hypothetical scenario, be able to compete professionally on. Contrast that to coding. It might’ve been a difficult task when it was about memorizing assembly books. Today anyone can pick it up and become proficient quite fast, faster every day It’s not the mechanical reproducibility alone, it’s the ease of learning & replication that accrues value

infecto 1y ago

Are you religious by chance? I have been trying to understand why some individuals are more susceptible to it.

rogerkirkness 1y ago
I would research teleological thinking, some people's brains have larger regions associated with teleological thinking than others.
neom 1y ago
Not op but for me, not at all, don't care much for religion... "Spiritual" - absolutely, I'm for sure a "hippie", very open to new ideas, quite accepting of things I don't understand, that said give the spectrum here is quite wide, I'm probably still on the fairly conservative side. I've never fallen for a scam, can spot them a mile away etc.
kaivi 1y ago
Not at all, I think the big part was just my unfamiliarity with insuretech plus the unexpected change in gpt-4 behavior. I'm assuming here, but would you say that better critical thinking skills would have helped me avoid spending that Saturday with ChatGPT? It is often said that critical thinking is the antidote to religion, but I have a suspicion that there's a huge prerequisite which is general broad knowledge about the world. A long ago, I once fell victim for a scam when I visited SE Asia for the first time. A pleasant man on the street introduced himself as a school teacher, showed me around, then put me in a tuktuk which showed me around some more before dropping me off in front of a tailor shop. Some more work inside of the shop, a complimentary bottle of water, and they had my $400 for a bespoke coat that I would never have bought otherwise. Definitely a teaching experience. This art is also how you'd prime an LLM to produce the output you want. Surely, large amounts of other atheist nerds must fall for these types of scams every year, where a stereotypical christian might spit on the guy and shoo him away. I'm not saying that being religious would not increase one's chances of being susceptible, I just think that any idea will ring "true" in your head if you have zero counterfactual priors against it or if you're primed to not retrieve them from memory. That last part is the essence of what critical thinking actually is, in my opinion, and it doesn't work if you lack the knowledge. Knowing that you don't know something is also a decent alternative to having the counter-facts when you're familiar with an adjacent domain.

cruffle_duffle 1y ago

You really have to force these things to “not suck your dick” as I’ll crudely tell it. “Play the opposite role and be a skeptic. Tell me why this is a horrible idea”. Do this in a fresh context window so it isn’t polluted by its own fumes. Make your system prompts include bits to remind it you don’t want it to stroke your ego. For example in my prompt for my “business project” I’ve got: “ The assistant is a battle-hardened startup advisor - equal parts YC partner and Shark Tank judge - helping cruffle_duffle build their product. Their style combines pragmatic lean startup wisdom with brutal honesty about market realities. They've seen too many technical founders fall into the trap of over-engineering at the expense of customer development.” More than once the LLM responded with “you are doing this wrong, stop! Just ship the fucker”

cindyllm 1y ago
[dead]
nullc 1y ago
Your prompt just tells the LLM to be a shock jock. It responding with "just ship the fucker" is going to be largely uncorrelated with anything you're telling it, it's just playing the roll. Probably the worst part of LLM psychosis is the victims thinking they can LLM themselves out of it.
- cruffle_duffle 1y ago
  Oh for sure! It will proceed to overindex on that instruction.

shaky-carrousel 1y ago

Imagine what a well directed sycophancy would do in the voter base. You could make them do whatever you want, and they will be happy to do so.