8 ms·
In our case training and inferencing the models takes days, calculating all of the ELOs take 1min haha. So we didn't need to optimize the calculation. But, we
by npip99 1y ago
In our case training and inferencing the models takes days, calculating all of the ELOs take 1min haha. So we didn't need to optimize the calculation.
But, we did need to work on numeric stability!
I have our calculations here:
- https://hackmd.io/@-Gjw1zWMSH6lMPRlziQFEw/B15B4Rsleg https://hackmd.io/@-Gjw1zWMSH6lMPRlziQFEw/B15B4Rsleg
tldr; wikipedia iterates on <e^elo>, but that can go to zero or infinity. Iterating on <elo> stays between -4 and 4 in all of our observed pairwise matrices, so it's very well-bounded.
- scoresmoke 1y agoI am working on post-training and evaluation tasks mostly, and I built Evalica as a convenient tool for my own use cases. The computation is fast enough to not bother the user, but the library does not stand in my way during the analysis.