You'll probably find this talk [1] interesting. They control all the training data for small LLMs and then perform experiments (including reasoning experiments)

You'll probably find this talk [1] interesting. They control all the training data for small LLMs and then perform experiments (including reasoning experiments). [1] Physics of LLMs: https://www.youtube.com/watch?v=yBL7J0kgldU&t=7s https://www.youtube.com/watch?v=yBL7J0kgldU&t=7s