You need to validate what providers are actually serving. Add benchmarks, properly showcase what quantization they are serving on the model and KV cache, etc.

You need to validate what providers are actually serving. Add benchmarks, properly showcase what quantization they are serving on the model and KV cache, etc. Until that happens, your service is doomed to be shitty.