4 ms·
Exactly, thank you, we are on the same page! It's great to be able to use our own devices and not have their compute coopted by a third party. I'd rather not h
by steno132 4mo ago
Exactly, thank you, we are on the same page! It's great to be able to use our own devices and not have their compute coopted by a third party.
I'd rather not have intensive compute needed shifted onto my personal machine which I want to use for something else.
- Zambyte 4mo agoI am not a "third party" on my own computer.
- satvikpendem 4mo agoBy that logic, any software you run that isn't fully built by yourself is "third party" therefore you shouldn't run anything at all on your machine, thus obviating the need for it entirely.
- steno132 4mo agoBut practically AI inference requires substantial local computing resources. It's not some web app, it's a order of magnitude more compute needed
- deleted 4mo ago[deleted]
- Zambyte 4mo agoHopefully now you understand why people want smaller models.
- satvikpendem 4mo agoNot really, I run a production service on a basic server using these Gemma models, the server is weaker than my MacBook. Most people's laptops and even phones actually can run local models, most simply don't know how. Run Unsloth Studio and you'll see how easy it is. As the sibling says this is why people want smaller but still performant models.