I was wondering the same and asked gpt to explain it to me the other day. Basically it’s all one process, there aren’t 2 modules (understand and generate). The

I was wondering the same and asked gpt to explain it to me the other day. Basically it’s all one process, there aren’t 2 modules (understand and generate). The “understanding” is perhaps like the generating in reverse. Rather than generating the next word from a statistical process, it’s defining the statistical process from the next words of the input. But as I explain here, even reviewing that conversation we had, I think maybe gpt and I were both hallucinating and feigning understanding. Cuz I still don’t get it. It still seems far too magical to fit this explanation. So my next best guess: we’re a simulation, GPT is magic, and our creators are just having fun introducing more and more magic into the system.