Your question is very interesting. Going over the comments the only plausible explanation I could see is KV cache being extremely useful - don't know if this i

Your question is very interesting. Going over the comments the only plausible explanation I could see is KV cache being extremely useful - don't know if this is really just the case. Would love to know the true answer to the question.