I might be missing something here, but where is the passphrase negotiated? Apart from calling or talking to the other person, the only way to define this common

I might be missing something here, but where is the passphrase negotiated? Apart from calling or talking to the other person, the only way to define this common key that I can see would be electronically. Isn't this a bit of a problem?