This would be just to allow you to connect to the server. If there was a vulnerable sshd on port 22, an adversary would have to know the port knocking sequence

This would be just to allow you to connect to the server. If there was a vulnerable sshd on port 22, an adversary would have to know the port knocking sequence to connect to sshd and run the exploit.