34 ms·

Can you elaborate? Is it because the models are inherently not deterministic? But then, will it not get better with time? I mean arent we just at the beginning

• troupo 6d ago
  > Is it because the models are inherently not deterministic This is the main reason (and yes, many modern compilers and CPUs carry some non-determinism which actually quite well explained and specified). > But then, will it not get better with time? No, it won't. Bacuse that is that is the actual literal limitation/feature of LLMs. > I mean arent we just at the beginning of the research here? In general? Yes. With LLMs? We can reasonably say that they will never be deterministic. There is a way to get a non-determenistic output: first question on a temp 0 local model on a completely new session will give you the same answer. The second answer in that same session will already be different on every session (even if it's the same question).
• eudamoniac 6d ago
  It is because English is an ambiguous input unlike a high level programming language. The output cannot help but be wrong sometimes when the input is ambiguous. See my comment here https://news.ycombinator.com/item?id=49491797 https://news.ycombinator.com/item?id=49491797 Determinism isn't the issue; it's that the prompt does not contain enough information to know the correct way to do the thing, even if it did the thing the same way with the same inputs every time.