7 ms·

I disagree, because the television was never talking back to you. By the medium directly engaging with the person, the responsibility / enablement of this issue

I disagree, because the television was never talking back to you. By the medium directly engaging with the person, the responsibility / enablement of this issue is more on the company than previous non interactive mediums. It would be like if a movie caused psychosis in people, and the sequel doubled down / enabled that. Even though there's no person (besides the consumer) in the loop, there is responsibility to train these systems to reduce the amount of these cases.

im3w1l 1y ago
A key question I think is whether these models can be made safer for mentally ill without cutting into utility for the healthy. Is the danger inherent to this technology, or is it incidental? And here I think we are fortunate that there doesn't seem to be tradeoff.
- evanextreme 1y ago
  Agreed here. My personal opinion is that this is fixable, but it would require reinforcement into the models that counteract the sycophantic stickiness that most companies use to drive engagement, so it won't really ever be a priority.
_dark_matter_ 1y ago
Counterpoint: https://en.m.wikipedia.org/wiki/Truman_Show_delusion https://en.m.wikipedia.org/wiki/Truman_Show_delusion
visarga 1y ago
> By the medium directly engaging with the person and > Even though there's no person in the loop contradict each other. There is always a person in the loop, and the LLM is actually reacting to their messages, however wrong it turns out. They could have chosen a positive interaction instead. The LLM reflects back what the human puts in.
- evanextreme 1y ago
  Sorry, should have clarified. By "no person in the loop" I meant from the side of the language model itself. The person is interacting with a thing which produces content for it, and the only person engaged in that process is the person consuming the content.
pixl97 1y ago
There are people that send the vast majority of the money they make to televangelists who are not speaking to them 'directly' and yet those individuals would tell you said speaker is communicating directly with them. This is how a lot of propaganda over the radio and TV works.
- evanextreme 1y ago
  Great point, actually. In many respects these language models doing this could be similar to how YouTube used to platform Alex Jones before removing him. Regardless, I still believe its the responsibility of the creators of these models to work on mitigations

im3w1l 1y ago

A key question I think is whether these models can be made safer for mentally ill without cutting into utility for the healthy. Is the danger inherent to this technology, or is it incidental? And here I think we are fortunate that there doesn't seem to be tradeoff.

evanextreme 1y ago
Agreed here. My personal opinion is that this is fixable, but it would require reinforcement into the models that counteract the sycophantic stickiness that most companies use to drive engagement, so it won't really ever be a priority.

_dark_matter_ 1y ago

Counterpoint: https://en.m.wikipedia.org/wiki/Truman_Show_delusion https://en.m.wikipedia.org/wiki/Truman_Show_delusion

visarga 1y ago

> By the medium directly engaging with the person and > Even though there's no person in the loop contradict each other. There is always a person in the loop, and the LLM is actually reacting to their messages, however wrong it turns out. They could have chosen a positive interaction instead. The LLM reflects back what the human puts in.

evanextreme 1y ago
Sorry, should have clarified. By "no person in the loop" I meant from the side of the language model itself. The person is interacting with a thing which produces content for it, and the only person engaged in that process is the person consuming the content.

pixl97 1y ago

There are people that send the vast majority of the money they make to televangelists who are not speaking to them 'directly' and yet those individuals would tell you said speaker is communicating directly with them. This is how a lot of propaganda over the radio and TV works.

evanextreme 1y ago
Great point, actually. In many respects these language models doing this could be similar to how YouTube used to platform Alex Jones before removing him. Regardless, I still believe its the responsibility of the creators of these models to work on mitigations