There's nothing inherently biased or intentionally wrong with greedy decoding. Why would there be? Let's say you're trying to predict what an expert doctor woul

There's nothing inherently biased or intentionally wrong with greedy decoding. Why would there be? Let's say you're trying to predict what an expert doctor would say to a patient with cancer. You think there's a 99% chance the doctor would say "start chemotherapy" and a 1% chance the doctor would say "don't worry about it". If you have to pick one to output, you'll pick "start chemotherapy". It would be crazy to say the best thing to do is to roll a 100-sided die to determine your answer. A language model is literally in that exact situation.