5 ms·
Time to first token is as important to know for many use cases, rarely are people reporting it
by bkitano19 2y ago
Time to first token is as important to know for many use cases, rarely are people reporting it
- Gcam 2y agoSee here for our TTFT metric benchmarks: https://artificialanalysis.ai/models/llama-3-1-instruct-70b/providers#latency https://artificialanalysis.ai/models/llama-3-1-instruct-70b/...