9 ms·
What? No. Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a si
by leodavi 14d ago
What? No. Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.
- munificent 14d ago> Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too. Floating point math is 100% deterministic, but different hardware/OS have different but deterministic behavior in some corners. The same code run on the same hardware with the same inputs (including access to timers, peripherals, etc.) will behave the same way, unless you're talking about cosmic rays flipping bits or something.
- stratos123 14d agoIt's certainly possible to write LLM inference on the GPU in a deterministic way, but it's somewhat nontrivial and trades off against performance, so by default most LLM inference engines aren't deterministic even at zero temperature. The classic post about that is https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in...