6 ms·
I faced latency challenges on llama2. So I haven’t used it. Usually OpenAI response is within 2 seconds, but llama2 takes 20+ seonds for the same. How are the
by dilipray 2y ago
I faced latency challenges on llama2. So I haven’t used it.
Usually OpenAI response is within 2 seconds, but llama2 takes 20+ seonds for the same.
How are they tackling the performance?