If the results of the distillation are released freely for everyone to download, I wouldn't even call it a wrong. Big models are built by scraping the recorded

If the results of the distillation are released freely for everyone to download, I wouldn't even call it a wrong. Big models are built by scraping the recorded thoughts of everyone, so giving everyone a chance to run a distilled small model is just going full circle. Obviously the underlying motivations aren't 100% altruistic, but I'll still take it.