6 ms·
I think it's been pretty much proven by now that there are no cases where local inferencing is better than remote inferencing, unless absolute privacy is a hard
by mike_hearn 9d ago
I think it's been pretty much proven by now that there are no cases where local inferencing is better than remote inferencing, unless absolute privacy is a hard requirement. The efficiencies that come with datacenter scale and hw can't be beaten.
- akmarinov 8d agoYeah, but data centers don't usually host abliterated models, hence the point of the article.
- big-and-small 8d agoYou can rent B300 at hourly rate and run whatever model you want.