5 ms·
Decent vs best-money-can-buy. Further, a self-hosted LLM will be much slower.
by dmos62 3mo ago
Decent vs best-money-can-buy. Further, a self-hosted LLM will be much slower.
- VBprogrammer 3mo agoI think we're all past the "bet-money-can-buy" stage. The most expensive models are an order of magnitude more expensive than the middle ground ones, so you need to be selective about what you run where. And with a bit of careful routing - there isn't a lot stopping you sending the hard stuff to a cloud model and the average stuff to an on prem model.
- dmos62 3mo agoOnly people who do pay-per-use optimize this. Most heavy users have their use covered by an employer.
- VBprogrammer 3mo agoI have my use covered by my employer but we also have budgets and limits.