Because contrary to your example, this intelligence could « understand » the flaws of our systems and lock us down if it wants to. For example by creating a mas

Because contrary to your example, this intelligence could « understand » the flaws of our systems and lock us down if it wants to. For example by creating a massive ransomware and exploiting 0-days because it’s much better and faster than us to analyze code. What happens then ? If it’d like to do harm for any reason (it might not be harm for an AI but just the « right thing » according to its criterias) what would we do ? Wouldn’t it be a good thing to know how it works inside precisely ? Or « whatever happens, happens » ?