6 ms·
The common method for choosing the next output token for an LLM is sampling from a Boltzmann distribution. If you have seen the term "temperature" in the contex
by hangsi 2y ago
The common method for choosing the next output token for an LLM is sampling from a Boltzmann distribution. If you have seen the term "temperature" in the context of language models, that is a direct link to the statistical gas mechanics.
- whimsicalism 2y agoi don't find the connection between softmax and boltzmann really all that deep tbh (compared to say, the connection between field theory/ising models and EBM)