7 ms·
No. Distillation trains on a teach model's logits or output tokens.
by mrfox321 2mo ago
No. Distillation trains on a teach model's logits or output tokens.
- naveen99 2mo agoHow do you have teacher model output anything without it being output tokens (or embeddings / intermediate logits / activations)?