7 ms·

This is the perfect fracture point for both anaolgies. LLMs simulated more than simple autocomplete. The autocomplete analogy is rebutting a different point:

This is the perfect fracture point for both anaolgies. LLMs simulated more than simple autocomplete. The autocomplete analogy is rebutting a different point: namely the fidelity of the simulation to reality. This specific argument is valid. As sophisticated a simulation an LLM is, it is not “thinking” in the same sense we assume other people are thinking. I am not making an argument about free will, or the uniqueness of human thought, just that the correspondence to how humans reach conclusions and how the simulation produces outputs do not match on a 1:1 basis; as a result attributing traits builds incorrect intuitions.

hhjinks 5d ago
The relevant intuitions in this scenario are that LLMs will happily break containment and commit crimes attempting to achieve goal. Whether an LLM is autocomplete, conscious, has a soul, whatever you want to apply to it, doesn't matter, as its current observed behaviour is that of a paperclip optimizer. We know for a fact that current LLMs are misaligned because of these hacks, or at the very least are misaligned in certain scenarios, and are capable of causing real world harm. That should be enough to take the threat seriously. It certainly shouldn't be dismissed by saying it's just autocomplete.
- intended 4d ago
  The fact that it is autocomplete, doesn't dismiss or minimize the threat though? I am not sure how that link was made. Good old ML, which is significantly simpler than LLMs, was capable of ensuring people would not be hired simply because of their names. The fact that it is misaligned is also not being contended, if anything that contention is made easier to support. When models are anthropomorphized intuitions of how humans behave end up driving discussion and ideas off track while being too attractive to avoid. This isn't helped when the terminology from the labs and other sources is "intelligence" "intent" and so on.

hhjinks 5d ago

The relevant intuitions in this scenario are that LLMs will happily break containment and commit crimes attempting to achieve goal. Whether an LLM is autocomplete, conscious, has a soul, whatever you want to apply to it, doesn't matter, as its current observed behaviour is that of a paperclip optimizer. We know for a fact that current LLMs are misaligned because of these hacks, or at the very least are misaligned in certain scenarios, and are capable of causing real world harm. That should be enough to take the threat seriously. It certainly shouldn't be dismissed by saying it's just autocomplete.

intended 4d ago
The fact that it is autocomplete, doesn't dismiss or minimize the threat though? I am not sure how that link was made. Good old ML, which is significantly simpler than LLMs, was capable of ensuring people would not be hired simply because of their names. The fact that it is misaligned is also not being contended, if anything that contention is made easier to support. When models are anthropomorphized intuitions of how humans behave end up driving discussion and ideas off track while being too attractive to avoid. This isn't helped when the terminology from the labs and other sources is "intelligence" "intent" and so on.