6 ms·
Exactly. Why does this not exist yet?
by aazo11 1y ago
Exactly. Why does this not exist yet?
- byyoung3 1y agoits an if statement on whether the model has downloaded or not
- aazo11 1y agoA better solution would train/finetune the smaller model from the responses of the larger model and only push to the inference to the edge if the smaller model is performant and the hardware specs can handle the workload?
- monoid73 1y agoyeah, that'd b nice, some kind of self-bootstrapping system where you start with a strong cloud model, then fine-tune a smaller local one over time until it’s good enough to take over. tricky part is managing quality drift and deciding when it's 'good enough' without tanking UX. edge hardware's catching up though, so feels more feasible by the day.