10 ms·
quite cool! haven't tried it yet, but what's the latency on hot-loading a model? (for instance, loading `stabilityai/stable-diffusion-2-1` for the first API cal
by julien_c 4y ago
quite cool! haven't tried it yet, but what's the latency on hot-loading a model? (for instance, loading `stabilityai/stable-diffusion-2-1` for the first API call)
- nikola_borisof 4y agoBecause this is popular model and many people use it, you will not experience the cold-start latency most likely. But in general it is <10s.