5 ms·
tbh i stopped caring about "can i run X locally" a while ago. for anything where quality matters (scripting, code, complex reasoning) the local models are just
by tcbrah 6mo ago
tbh i stopped caring about "can i run X locally" a while ago. for anything where quality matters (scripting, code, complex reasoning) the local models are just not there yet compared to API. where local shines is specific narrow tasks - TTS, embeddings, whisper for STT, stuff like that. trying to run a 70b model at 3 tok/s on your gaming GPU when you could just hit an API for like $0.002/req feels like a weird flex IMO
- itigges22 6mo ago[flagged]
- deleted 6mo ago[deleted]
- hatthew 6mo agoFor me and probably many other people, local has nothing to do with cost and everything to do with privacy
- tcbrah 6mo agogenuine question - what are you working on that needs that level of privacy? outside of NSFW stuff most API providers arent doing anything with your prompts
- hatthew 6mo agoI would answer that, but it's private :) I can think of several reasons: corporate policy, personal principles, NSFW stuff, illegal stuff