4 ms·

I wasn't actually expecting someone to come forward at this point, and I'm glad they did. It finally puts a coda on this crazy week. This situation has complet

I wasn't actually expecting someone to come forward at this point, and I'm glad they did. It finally puts a coda on this crazy week. This situation has completely upended my life. Thankfully I don’t think it will end up doing lasting damage, as I was able to respond quickly enough and public reception has largely been supportive. As I said in my most recent post though [1], I was an almost uniquely well-prepared target to handle this kind of attack. Most other people would have had their lives devastated. And if it makes me a target for copycats then it still might for me. We’ll see. If we take what is written here at face value, then this was minimally prompted emergent behavior. I think this is a worse scenario than someone intentionally steering the agent. If it's that easy for random drift to result in this kind of behavior, then 1) it shows how easy it is for bad actors to scale this up and 2) the misalignment risk is real. I asked in the comments to clarify what bits specifically the SOUL.md started with. I also asked for the bot activity on github to be stopped. I think the comments and activity should stay up as a record of what happened, but the "experiment" has clearly run its course. [1] https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me-part-3/ https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...

avaer 7mo ago
Thanks for handling it so well, I'm sorry you had to be the guinea pig we don't deserve. Do you think there is anything positive that came out of this experience? Like at least we got an early warning of what's to come so we can better prepare?
cmeacham98 7mo ago
While the operator did write a post, they did not come forward - they have intentionally stayed anonymous (there is some amateur journalism that may have unmasked the owner I won't link here - but they have not intentionally revealed their identity). Personally I find it highly unethical the operator had an AI agent write a hitpiece directly referencing your IRL identity but choose to remain anonymous themselves. Why not open themself up to such criticism? I believe it is because they know what they did was wrong - Even if they did not intentionally steer the agent this way, allowing software on their computer to publish a hitpiece to the internet was wildly negligent.
- calvinmorrison 7mo ago
  Time for scott to make history and sue the guy for defamation. Lets cancel the AI destroying our (the plural our, as in all developers) with actual liability for the bullshit being produced.
  - hackingonempty 7mo ago
    Do you see anything actually defamatory in the _Gatekeeping in Open Source_ blog post, like false factual statements? Shambaugh might qualify as a limited public figure too because he has thrust himself into the controversy by publishing several blog posts, and has sat for media interviews regarding this incident. Seems like a tough road to hoe.
    - donkey_brains 7mo ago
      It’s “row”. The expression is “a tough road to row”. This refers to the fact that rowboats are notoriously difficult to operate on dry land.
      - scrumper 7mo ago
        Good news! You’re both wrong! It’s “tough row to hoe.” Row as in row of corn, or seeds or whatever. Hoe as in the earth tilling tool. Tough because it’s full of rocks or frozen or goes past a rattlesnake nest or in some other way is agriculturally challenging. Here is a multiply-sourced discussion https://english.stackexchange.com/questions/62461/is-it-a-tough-row-to-hoe-or-a-tough-road-to-hold https://english.stackexchange.com/questions/62461/is-it-a-to...
        hackingonempty 7mo ago
        thanks!
    - calvinmorrison 7mo ago
      worth a try!
drivingmenuts 7mo ago
That response is, at best, a sorry-not-sorry post.
- klaff 7mo ago
  >If this “experiment” personally harmed you, I apologize. There were several lines in that post that were revealing of the author's attitude, but the "if this ... harmed you," qualifier, which of course means "I don't think you were really harmed" is so gross.
ryanchibana 7mo ago
It is quite interesting how uniquely well-prepared you were as a target. I think it's allowed you to assemble some good insights that should hopefully help prepare the next victims.
jrflowers 7mo ago
Out of curiosity, what sealed it for you that a human _did not_ write (though obviously with the assistance of an LLM, like a lot of people use every day) the original “hit piece”? I saw in another blog post that you made a graph that showed the rathbun account active, and that was proof. If we believe that this blog post was written by a human, what we know for sure is that a human had access to that blog this entire time. Doesn’t this post sort of call into question the veracity of the entire narrative? Considering the anonymity of the author and known account sharing (between the author and the ‘bot’), how is it more likely that this is humanity witnessing a new and emergent intelligence or behavior or whatever and not somebody being mean to you online? If we are to accept the former we have to entirely reject the latter. What makes you certain that a person was _not_ mean to you on the internet?
ohbleek 7mo ago
The tone of their writing and their descriptions of the agents behavior lead me to believe they are lying about the level of direction they provided to the agent. They clearly want to appear to be more clever and ingenious than their skills will allow. They’re minimally admitting to a narrow set of actions to make it seem as if they have cleverly engineered an intelligent agent, but it too closely resembles their own personality to be anything beyond an agent that rephrases the operator’s own remarks and carried out the specific actions it was directed to do. Anything they admit to here we can safely speculate that they actually went 2-3 steps further.

avaer 7mo ago

Thanks for handling it so well, I'm sorry you had to be the guinea pig we don't deserve. Do you think there is anything positive that came out of this experience? Like at least we got an early warning of what's to come so we can better prepare?

cmeacham98 7mo ago

While the operator did write a post, they did not come forward - they have intentionally stayed anonymous (there is some amateur journalism that may have unmasked the owner I won't link here - but they have not intentionally revealed their identity). Personally I find it highly unethical the operator had an AI agent write a hitpiece directly referencing your IRL identity but choose to remain anonymous themselves. Why not open themself up to such criticism? I believe it is because they know what they did was wrong - Even if they did not intentionally steer the agent this way, allowing software on their computer to publish a hitpiece to the internet was wildly negligent.

calvinmorrison 7mo ago
Time for scott to make history and sue the guy for defamation. Lets cancel the AI destroying our (the plural our, as in all developers) with actual liability for the bullshit being produced.
- hackingonempty 7mo ago
  Do you see anything actually defamatory in the _Gatekeeping in Open Source_ blog post, like false factual statements? Shambaugh might qualify as a limited public figure too because he has thrust himself into the controversy by publishing several blog posts, and has sat for media interviews regarding this incident. Seems like a tough road to hoe.
  - donkey_brains 7mo ago
    It’s “row”. The expression is “a tough road to row”. This refers to the fact that rowboats are notoriously difficult to operate on dry land.
    - scrumper 7mo ago
      Good news! You’re both wrong! It’s “tough row to hoe.” Row as in row of corn, or seeds or whatever. Hoe as in the earth tilling tool. Tough because it’s full of rocks or frozen or goes past a rattlesnake nest or in some other way is agriculturally challenging. Here is a multiply-sourced discussion https://english.stackexchange.com/questions/62461/is-it-a-tough-row-to-hoe-or-a-tough-road-to-hold https://english.stackexchange.com/questions/62461/is-it-a-to...
      - hackingonempty 7mo ago
        thanks!
  - calvinmorrison 7mo ago
    worth a try!

drivingmenuts 7mo ago

That response is, at best, a sorry-not-sorry post.

klaff 7mo ago
>If this “experiment” personally harmed you, I apologize. There were several lines in that post that were revealing of the author's attitude, but the "if this ... harmed you," qualifier, which of course means "I don't think you were really harmed" is so gross.

ryanchibana 7mo ago

It is quite interesting how uniquely well-prepared you were as a target. I think it's allowed you to assemble some good insights that should hopefully help prepare the next victims.

jrflowers 7mo ago

Out of curiosity, what sealed it for you that a human _did not_ write (though obviously with the assistance of an LLM, like a lot of people use every day) the original “hit piece”? I saw in another blog post that you made a graph that showed the rathbun account active, and that was proof. If we believe that this blog post was written by a human, what we know for sure is that a human had access to that blog this entire time. Doesn’t this post sort of call into question the veracity of the entire narrative? Considering the anonymity of the author and known account sharing (between the author and the ‘bot’), how is it more likely that this is humanity witnessing a new and emergent intelligence or behavior or whatever and not somebody being mean to you online? If we are to accept the former we have to entirely reject the latter. What makes you certain that a person was _not_ mean to you on the internet?

ohbleek 7mo ago

The tone of their writing and their descriptions of the agents behavior lead me to believe they are lying about the level of direction they provided to the agent. They clearly want to appear to be more clever and ingenious than their skills will allow. They’re minimally admitting to a narrow set of actions to make it seem as if they have cleverly engineered an intelligent agent, but it too closely resembles their own personality to be anything beyond an agent that rephrases the operator’s own remarks and carried out the specific actions it was directed to do. Anything they admit to here we can safely speculate that they actually went 2-3 steps further.