You also need a shitton of data. People are saying that synthetic data is widely used, but pretty sure that is on top of the organic data. Both for pretraining

You also need a shitton of data. People are saying that synthetic data is widely used, but pretty sure that is on top of the organic data. Both for pretraining and posttraining. That said, I do think we will see more open and collaborative approaches over time.