oxidi

5 ms·

1. 1.▲
   by oxidi 1y ago

   Good point. Though sampling generally happens on the CPU in a linear way. What you describe might influence the raw output logits from a single LLM step, but since the differences are only tiny, a well designed sampler could still make the
2. 2.▲
   by oxidi 1y ago

   I think lots of people misunderstand that the "non-deterministic" nature of LLMs come from sampling the token distribution, not from the model itself.
3. 3.▲
   by oxidi 1y ago

   I was referring to the "creative syntax" and it wasn't meant to be an attack on Python. We cannot deny that Python has some interesting solutions, such as the std lib namedtuple implementation. It's basically a code temp
4. 4.▲
   by oxidi 1y ago

   I think the point was that Python syntax is simpler than e.g. borrow checking. Although Python has some seriously PERLesque YOLO moments, like "#"*3 == "###". This is admittedly useful, but funny nonetheless.