6 ms·
Cloudflare AI and Replicate are great for running off-the-shelf models, but anything custom is going to incur a 10+ minute cold start. For running custom fine-
by Mernit 3y ago
Cloudflare AI and Replicate are great for running off-the-shelf models, but anything custom is going to incur a 10+ minute cold start.
For running custom fine-tuned models on serverless, you could look into https://beam.cloud https://beam.cloud which is optimized for serving custom models with extremely fast cold start (I'm a little biased since I work there, but the numbers don't lie)
- NicoJuicy 3y agoWhy would it incur a cold start of 10 minutes on cloudflare? :O Any proof?
- Nischalj10 3y agoThanks! Looks promising from the outside. Will surely check out