sohojoe

6 ms·

1. 1.▲
   by sohojoe 10mo ago

   > someone corrected me above, it does seem to matter more then I thought if you llm agent takes different decisions from the same prompt, then you have to deal with it 1) your benchmarks become stochastic so you need multiple samples to
2. 2.▲
   by sohojoe 10mo ago

   yeah, stochastic is there because we give up control of order of operations for speed so the order in which floating-point additions happen is not fixed because of how threads are scheduled, how reductions are structured (tree reduction vs