4 ms·

I think we might be largely in agreement. LLM's should be used to speed up the drudgery, explore creative/evolutionary ideas, but.... > What's the dreaded imp

I think we might be largely in agreement. LLM's should be used to speed up the drudgery, explore creative/evolutionary ideas, but.... > What's the dreaded impossible-to-replicate human advantage? Responsibility. Ultimately LLM's are a tool. Someone has to be responsible for the output. You don't get to use it to create a car/plane/submarine/bridge/building/childs-toy/whatever and then if it bursts into flame, blame the LLM and walk away. Ideally, the person signing off would "know" what they are signing off on. Which requires an engineer.

somesortofthing 16d ago
What happens when the thing they have to be responsible for takes multiple lifetimes to understand on even the most basic level, let alone know deeply enough to feel comfortable taking responsibility for it?
- _carbyau_ 16d ago
  Again, engineers know what needs to be tested, how, to what degree, with which materials, and with what safety margin. Your widget that mankind may never understand. Well, that's just like anything else in the world. Engineers didn't understand building foundations to anywhere like the degree we do now. Or how to build earthquake proof buildings for a long time. It didn't stop stuff getting built. But it sure affected the confidence in it. And when they honestly don't know, they don't have to approve it. I cited the Titan sub example because one engineer refused to sign off on it and was fired for it. Whoever signed off on it has deaths to weigh on their conscience/reputation - for whatever societal good that is worth. If the latest electric vehicle car never needs to charge but might catastrophically explode randomly between 2-15 years, then the car company doesn't get to take money for the car but handwave away the deaths caused. Legally someone is on the hook. In this case, the engineer who signed off on the magic involved - whether they understood it or not - is at fault. But honestly, people sign off on shit they don't understand all the time. It's a matter of: 1. did their gamble not pay off? 2. were they negligent? If your answer to 2. is "the LLM made a mistake I never checked for in regulation approved ways", then you are negligent.
  - somesortofthing 15d ago
    You can write human-comprehensible regulations(and the AI will helpfully beat whatever it's building into the shape of those regulations, probably against the spirit of them, in ways you can't detect) and you can have humans go through the checklist, but it won't accomplish much. To use your foundation example, it'd be like trying to make go/no go decisions using engineers that don't and can't understand why the foundations go below the building rather than on top of it. Past a certain complexity level, a human approval process is just noise.
    - _carbyau_ 15d ago
      You are absolutely correct that the LLM vs Regulation relationship would be adversarial. This is why there is the saying: regulations are written in blood. Mankind has been on a blast improving the understanding and engineering of stuff in our society for a while now. The regulations are already written in an adversarial sense. Will LLMs take it up a notch? Yeah. And then the people making the regulations in their respective expert field, will have to deal with it. What they do?! I'm not qualified to say. The whole point is that engineers are the people we deem qualified in their field. Whatever LLMs do, we will still need them.

somesortofthing 16d ago

What happens when the thing they have to be responsible for takes multiple lifetimes to understand on even the most basic level, let alone know deeply enough to feel comfortable taking responsibility for it?

_carbyau_ 16d ago
Again, engineers know what needs to be tested, how, to what degree, with which materials, and with what safety margin. Your widget that mankind may never understand. Well, that's just like anything else in the world. Engineers didn't understand building foundations to anywhere like the degree we do now. Or how to build earthquake proof buildings for a long time. It didn't stop stuff getting built. But it sure affected the confidence in it. And when they honestly don't know, they don't have to approve it. I cited the Titan sub example because one engineer refused to sign off on it and was fired for it. Whoever signed off on it has deaths to weigh on their conscience/reputation - for whatever societal good that is worth. If the latest electric vehicle car never needs to charge but might catastrophically explode randomly between 2-15 years, then the car company doesn't get to take money for the car but handwave away the deaths caused. Legally someone is on the hook. In this case, the engineer who signed off on the magic involved - whether they understood it or not - is at fault. But honestly, people sign off on shit they don't understand all the time. It's a matter of: 1. did their gamble not pay off? 2. were they negligent? If your answer to 2. is "the LLM made a mistake I never checked for in regulation approved ways", then you are negligent.
- somesortofthing 15d ago
  You can write human-comprehensible regulations(and the AI will helpfully beat whatever it's building into the shape of those regulations, probably against the spirit of them, in ways you can't detect) and you can have humans go through the checklist, but it won't accomplish much. To use your foundation example, it'd be like trying to make go/no go decisions using engineers that don't and can't understand why the foundations go below the building rather than on top of it. Past a certain complexity level, a human approval process is just noise.
  - _carbyau_ 15d ago
    You are absolutely correct that the LLM vs Regulation relationship would be adversarial. This is why there is the saying: regulations are written in blood. Mankind has been on a blast improving the understanding and engineering of stuff in our society for a while now. The regulations are already written in an adversarial sense. Will LLMs take it up a notch? Yeah. And then the people making the regulations in their respective expert field, will have to deal with it. What they do?! I'm not qualified to say. The whole point is that engineers are the people we deem qualified in their field. Whatever LLMs do, we will still need them.