5 ms·
Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy
by a12k 2y ago
Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy so just went that route.
When I see people bring up the sketchiness most of the time the creator responds with the equivalent of shrugs, which imo increases the sketchiness.
- nialv7 2y agoIt's fully open source. I mean yes it uses llama.cpp without giving it credit. But why run it in a VM?
- a12k 2y agoIt severely over-permissions itself on my Mac.
- deleted 2y ago[deleted]
- super_mario 2y agoCan you please elaborate? How are you running ollama? I just build it from source and have written a shell script to start/stop it. It runs under my local user account (I should probably have its own user) and is of course not exposed outside localhost.
- celeritascelery 2y agoI have never had it request any permissions
- _w1tm 2y agoJust install it from Brew and run the service in a separate terminal tab.
- instagary 2y agoIsn't there a clause in MIT that says you're required to give credit? Also, I didn't know a YC company which started it: https://www.ycombinator.com/companies/ollama https://www.ycombinator.com/companies/ollama.
- krowek 2y ago> But why run it in a VM? Because you don't execute untrusted code in your machine without containerization/virtualization. Don't you?
- adastra22 2y agoEvery single day. There's just too much good software out there, and life is too short to be so paranoid.
- Aurornis 2y agoThe question was asking why it’s untrusted code, not why you run untrusted code in a VM. There are a lot of open-source tools that we have to trust to get anything done on a daily basis.
- buyucu 2y agoollama advertising llama.cpp features as their own is very dishonest in my opinion.
- portaouflop 2y agoThat’s the curse and blessing of open source I guess? I have billion dollar companies running my oss software without giving me anything - but do I gripe about it in public forums? Yea maybe sometimes but it never helps to improve the situation.
- sitkack 2y agoAre they a wrapper with a similar name? You, like I, do gripe in public forums.
- kgwgk 2y agoThe similar name in this case doesn’t originate in the wrapped thing either.
- portaouflop 2y agoNo it’s totally different from this case - but the software is a super important part of their stack - if it were to stop working they can shut down their business
- weinzierl 2y agoIt's the curse of permissively licensed open source. Copyleft is not the answer to everything but against companies leeching and not giving back it is effective.
- portaouflop 2y agoWell we had this conversation but once you change the license to something like that you lose the trust of the oss community instantly and as elasticsearch etc. show it also doesn’t really help with monetisation
- nicce 2y ago> but VMs are pretty easy so just went that route. Don’t you need at least 2 GPUs in that case and put kernel level passthrough?
- a12k 2y agoI don’t use GPU. Works fine, but the large Mixtral models are slow.
- bdhcuidbebe 2y agoi pass through my dGPU to VM and use iGPU for desktop
- n144q 2y agoCare to elaborate what "sketchy" refers to here?