7 ms·
Don't forget that you're also spending much more electricity because it takes so long to run inference.
by iamsaitam 5mo ago
Don't forget that you're also spending much more electricity because it takes so long to run inference.
- Wowfunhappy 5mo agoGiven current hardware prices I wouldn't expect this to tip the scales. Mind you, for me local models are a fun experiment, for anything serious I would use a frontier model. So I'm not using this for hours a day.