7 ms·
Where we've had some success is with heterogeneous agents with some cheap quantised/local models performing certain tasks extremely cheaply that are then overse
by rando1234 6mo ago
Where we've had some success is with heterogeneous agents with some cheap quantised/local models performing certain tasks extremely cheaply that are then overseen or managed by a more expensive model.
- timcobb 6mo agoI've played with this type of thing and I couldn't justify it vs just using a premium model, which seems more direct and error proof. Cheap models in my experience could really consume tokens and generate cost