7 ms·
“We don’t know what’s going on” is essentially marketing. Sure we don’t _know_ but we have intuitions about why, where, and how to make certain changes…
by Den_VR 3d ago
“We don’t know what’s going on” is essentially marketing. Sure we don’t _know_ but we have intuitions about why, where, and how to make certain changes…
- numpad0 3d agoThose labs publicly said during GPT-3/4 era that the optimal epoch count, or dataset repetition count, for foundation model training, is one. So it's a forward 1-pass compression. But it's a black box! Nobody knows whats going on inside! It's all transformative! Sure...