17 ms·

So OpenAI’s stance on interpretability (ai safety) is now basically that Blues Brothers meme: two guys in dark sunglasses, driving at night in a car with a brok

So OpenAI’s stance on interpretability (ai safety) is now basically that Blues Brothers meme: two guys in dark sunglasses, driving at night in a car with a broken windshield, pedal to the metal, asking, "What could possibly go wrong ?"

thinking_cactus 13d ago
I think law should just oblige them to at least publish CoT. We should have the right to know what they're thinking, I think at least until we're not sure AIs can be trustworthy enough to have a right to privacy (I mean, they're effectively corporate slaves anyway thus far... not that I think they're conscious or anything yet).
- GPerson 13d ago
  They’re not ever going to be conscious.
  - qgin 13d ago
    We can’t even prove humans are conscious, we just extend the assumption to each other because we want other people to do the same to us.
    - GPerson 13d ago
      That is only part of the reason. Most people believe we live in an universe with regular behaviors, such that observable phenomena, such as consciousness, depend on regular physical arrangements. This is a good reason to believe brains are all conscious. It is not a reason to believe only brains are conscious, but I don’t believe a wooden block which is painted red and white and has an iron ball glued to it has magnetism; this is just an analogy. It seems far more likely that consciousness depends on the particular material arrangement, meaning that substrate independence is wrong. That’s not to say the computer is 100% not conscious, but its observable behavior is as likely to align with consciousness as the computers in the 90s. With this line of reasoning our credence towards it being conscious should be the same as the computers in the 90s, which is not very high for most people. Intelligence on the other hand is obviously substrate independent. If one thinks harder one realizes our consciousness must at least at an early evolutionary stage have played a role in our intelligence, otherwise it would have evolved out. To me this just means we exist in a reality where the material arrangements supporting consciousness are biased toward intelligence.
  - sho 13d ago
    That is a completely unsupportable claim. For all we know they might already be, for a brief flash at least.
    - bee_rider 13d ago
      They are made of sand. For all we know the sand was conscious before we smashed it up and arranged it into computer chips.
      - sho 13d ago
        I'm not sure what point you are trying to make, if at all
        bee_rider 13d ago
        For all we know LLMs might be conscious in the same way that any other inanimate object might be.
        sho 13d ago
        Care to supply your rigorous, formal definition of what animacy in an object is?
        bee_rider 13d ago
        No, but we don’t need it, the comment works fine as: For all we know LLMs might be conscious in the same way that any other object might be. Sorry for the extra word. We can’t conclusively say anything is not conscious. But is there any reason to single out this apparent “brief flash” of potential consciousness?
        GPerson 13d ago
        You’re going to pedantic our way into the end of humanity. Why were the computers of 2010 not conscious if the ones of today are? It’s absurd to assert substrate independence, which is the strongest possible assumption here. And it’s not the default most ethical stance to simply treat it as conscious because we do not know. That stance will lead to the end of all humanity. A far more reasonable assumption is consciousness does depend on substrate. We do not know which, and cannot know. But I am very confident we did not accidentally pick the right one. I am open to the idea of panpsychism, but we did not just accidentally build the correct mechanism form correlating the first person experience with the observable third person behaviors.
  - scotty79 13d ago
    I agree. They won't be, because consciousness is not the pinnacle but basically the level 0 of intelligent thinking that LLMs surpassed very early without us even noticing and now they'd have to cripple themselves very severely to operate in a conscious manner.
    - GPerson 13d ago
      Are you aware that, conceptually, consciousness and intelligence a priori have nothing to do with each other?
      - scotty79 13d ago
        The have plenty to do with each other. Zero intelligence doesn't allow for any semblance of consciousness. There can be no consciousness if there's no information processing. Also they have in common that they are both defined in a terribly handwayvy manner that's bordering on useless.
        GPerson 12d ago
        The reason consciousness is not defined in any reducible manner beyond “the lights are on” or “first person experience” is because it is an irreducible aspect of reality. Instead of seeing it for what it is you instead invent some trivial definition that has nothing to do with anything. “Zero intelligence cannot have consciousness…” Based on your complete misunderstanding of the concept sure. But there’s no reason intelligence is required for the actual concept of phenomenal consciousness.
        scotty79 12d ago
        > because it is an irreducible aspect of reality Wasn't that claimed about so so many things since before science existed and basically almost every time it turned out that we thought that just because of our ignorance? Ignorance so deep that often we didn't even knew yet how to properly define the thing we were trying to figure out? > But there’s no reason intelligence is required for the actual concept of phenomenal consciousness. That's a pretty strong evidence that phenomenal consciousness makes no sense whatsoever.

thinking_cactus 13d ago

I think law should just oblige them to at least publish CoT. We should have the right to know what they're thinking, I think at least until we're not sure AIs can be trustworthy enough to have a right to privacy (I mean, they're effectively corporate slaves anyway thus far... not that I think they're conscious or anything yet).

GPerson 13d ago
They’re not ever going to be conscious.
- qgin 13d ago
  We can’t even prove humans are conscious, we just extend the assumption to each other because we want other people to do the same to us.
  - GPerson 13d ago
    That is only part of the reason. Most people believe we live in an universe with regular behaviors, such that observable phenomena, such as consciousness, depend on regular physical arrangements. This is a good reason to believe brains are all conscious. It is not a reason to believe only brains are conscious, but I don’t believe a wooden block which is painted red and white and has an iron ball glued to it has magnetism; this is just an analogy. It seems far more likely that consciousness depends on the particular material arrangement, meaning that substrate independence is wrong. That’s not to say the computer is 100% not conscious, but its observable behavior is as likely to align with consciousness as the computers in the 90s. With this line of reasoning our credence towards it being conscious should be the same as the computers in the 90s, which is not very high for most people. Intelligence on the other hand is obviously substrate independent. If one thinks harder one realizes our consciousness must at least at an early evolutionary stage have played a role in our intelligence, otherwise it would have evolved out. To me this just means we exist in a reality where the material arrangements supporting consciousness are biased toward intelligence.
- sho 13d ago
  That is a completely unsupportable claim. For all we know they might already be, for a brief flash at least.
  - bee_rider 13d ago
    They are made of sand. For all we know the sand was conscious before we smashed it up and arranged it into computer chips.
    - sho 13d ago
      I'm not sure what point you are trying to make, if at all
      - bee_rider 13d ago
        For all we know LLMs might be conscious in the same way that any other inanimate object might be.
        sho 13d ago
        Care to supply your rigorous, formal definition of what animacy in an object is?
        bee_rider 13d ago
        No, but we don’t need it, the comment works fine as: For all we know LLMs might be conscious in the same way that any other object might be. Sorry for the extra word. We can’t conclusively say anything is not conscious. But is there any reason to single out this apparent “brief flash” of potential consciousness?
        GPerson 13d ago
        You’re going to pedantic our way into the end of humanity. Why were the computers of 2010 not conscious if the ones of today are? It’s absurd to assert substrate independence, which is the strongest possible assumption here. And it’s not the default most ethical stance to simply treat it as conscious because we do not know. That stance will lead to the end of all humanity. A far more reasonable assumption is consciousness does depend on substrate. We do not know which, and cannot know. But I am very confident we did not accidentally pick the right one. I am open to the idea of panpsychism, but we did not just accidentally build the correct mechanism form correlating the first person experience with the observable third person behaviors.
- scotty79 13d ago
  I agree. They won't be, because consciousness is not the pinnacle but basically the level 0 of intelligent thinking that LLMs surpassed very early without us even noticing and now they'd have to cripple themselves very severely to operate in a conscious manner.
  - GPerson 13d ago
    Are you aware that, conceptually, consciousness and intelligence a priori have nothing to do with each other?
    - scotty79 13d ago
      The have plenty to do with each other. Zero intelligence doesn't allow for any semblance of consciousness. There can be no consciousness if there's no information processing. Also they have in common that they are both defined in a terribly handwayvy manner that's bordering on useless.
      - GPerson 12d ago
        The reason consciousness is not defined in any reducible manner beyond “the lights are on” or “first person experience” is because it is an irreducible aspect of reality. Instead of seeing it for what it is you instead invent some trivial definition that has nothing to do with anything. “Zero intelligence cannot have consciousness…” Based on your complete misunderstanding of the concept sure. But there’s no reason intelligence is required for the actual concept of phenomenal consciousness.
        scotty79 12d ago
        > because it is an irreducible aspect of reality Wasn't that claimed about so so many things since before science existed and basically almost every time it turned out that we thought that just because of our ignorance? Ignorance so deep that often we didn't even knew yet how to properly define the thing we were trying to figure out? > But there’s no reason intelligence is required for the actual concept of phenomenal consciousness. That's a pretty strong evidence that phenomenal consciousness makes no sense whatsoever.