4 ms·
Does this mean you have to retrain routing rules every time a new model gets released? I imagine since the price/token (or rather the amount of work that can be
by RuiWang0811 2mo ago
Does this mean you have to retrain routing rules every time a new model gets released? I imagine since the price/token (or rather the amount of work that can be done per token) does not monotonically increase with new models, that the routing logic has to change all the time
- Mzzzzz 2mo agotest
- rohaga 2mo agoYeah this is a great observation--we have to collect some more data to understand new models as they come out. However, by having this composable architecture that separates "confidence prediction" and the actual "routing layer," this makes it a lot easier to add new models/restrict the models rather than re-training from scratch.