10 ms·
> because their output is non-deterministic by design. It isn't. At least not by design, even though in practice it often can be. If you do greedy decoding (or
by kouteiheika 12d ago
> because their output is non-deterministic by design.
It isn't. At least not by design, even though in practice it often can be. If you do greedy decoding (or use a preset seed) and deterministically compute everything (e.g. only use integer math) then it will be 100% always deterministic.
- kennywinker 12d agoThat’s true, but not true-true. Sure, every time you prompt “what is the weather in kansas” you’ll get the same output, but if you prompt “what is the weather in kansas right now” you’ll get a different output, and then “what is the weather in kansas today” gets a different output. Language being language, there are infinite ways to say things, so there are infinite variations in what the llm can output in response to very similar prompts. This tool has a finite amount of outputs for an infinite amount of inputs. Which is different from an llm based tool.
- skeledrew 12d agoI think the point being made is that given a particular input string, you can get a deterministic output string back from the LLM.
- kennywinker 12d agoYes i think I acknowledged that, but is that useful for making a tool that can be trusted to safely run shell commands when asked arbitrary questions? No. It’s not.