5 ms·
You can also have another model "mentor" a new model you are teaching to speed up training. You don't have to start from scratch with zero knowledge. This is do
by cooljoseph 3y ago
You can also have another model "mentor" a new model you are teaching to speed up training. You don't have to start from scratch with zero knowledge. This is done a lot in what are called distillations.
- fnordpiglet 3y agoThis came out a little bit ago, my open question is if this approach can be used to port weights between architectures like this. https://arxiv.org/abs/2402.13144 https://arxiv.org/abs/2402.13144
- eru 3y agoYou can also re-use a lot of the infrastructure. Eg you can re-use your training data.