5 ms·

The way these systems work feel massively inefficient. We don't re-evaluate our astrophysics models when reading a cooking book.

• DavidSJ 3y ago
  Neither does GPT-4 or other sparse mixtures of experts, such as e.g. switch transformers [1]. [1] https://arxiv.org/abs/2101.03961 https://arxiv.org/abs/2101.03961
• rini17 3y ago
  Speak for yourself :)