7 ms·

how does one do that from their comfy chair?

pixelatedindex 13d ago
I for one am running what I can on my aging 1080Ti(!!), namely a Qwen2.5 14B (4-bit quantized). It’s not great, and the only other card I have is a 3070Ti but I need that for gaming. :( What a terrible card that 3070Ti is. Mad regrets buying it because I wanted to save $400 compared to a 3080Ti.
- pixelatedindex 12d ago
  I really don’t understand why this is being downvoted? Did I say something wrong / controversial? Or is it because I’m not using my resources the “optimal” way? Is it because my models aren’t open weight? I was only trying to contribute my experience :(
oefrha 13d ago
By sending some cash (and maybe some non-sensitive training data too) the way of labs creating open models? At work we have expensive Anthropic and OpenAI subscriptions but also have some in house workflows plugged into DeepSeek and GLM APIs.
LtWorf 13d ago
By not harassing projects with AI generated pull requests.
- dv_dt 13d ago
  A pull request is a communication to the project. AI or no AI, a poorly communicated request is a burden. IMHO AI prs are fine as long as the submitter has done the work to refine that communication to make it easy to read and assess.
  - LtWorf 13d ago
    True, the problem is that AI ones are 500x longer and at a first glance look fine, so they waste much more time. Moreover a human contributor can learn and do better in the next PR, while the AI won't, unless the prompt is changed, but the operator won't learn.
    - dv_dt 13d ago
      A large submission without a breakdown or explanation, or a pass to minimize the code change is junk. The operator can literally ask the ai to compose it better, refactor in to smaller PRs, or do it themself. The annoyance is to have to communicate and enforce expectations to drive by submitters. In the end every project has to figure out what they accept. If, at a glance the PR is drive by junk being thrown over the wall, a response of this doesn't meet our pr submission standards delete, or try again, to me is not overly burdensome.
  - owebmaster 12d ago
    AI PRs aren't fine. Open a issue and propose a change, the maintainers decide what to do next with their own AI if they want.
nucleative 13d ago
1. Download ollama and give it a try. It doesn't have to be that one but it's an easy on-ramp. Just get a feel for what open source/open weights is capable of 2. Mention it whenever it comes up. Most people have no clue this is a thing and I think it's useful to make people aware. We need access to uncensored / unbiased LLM models - information wants to be free but there are plenty of businesses gunning for regulatory capture as these are very powerful tools. 3. If you're in a position of developing any project that makes use of AI in any form, check the open source models first, unless you absolutely require the best of the best, these other models are pretty dang capable of almost everything any commercial model can do. 4. If your state or {insert legal jurisdiction here} attempts to regulate access to open source tools for this, oppose it with your vote and your voice. 5. Opposed laws that grant commercial AI suppliers any priority or premium access under government purchasing programs. I'm sure others will chime in, those are a few that pop the mind.
- KronisLV 13d ago
  > Download ollama and give it a try Weren't people pissed off at them for a variety of reasons? Not giving attribution etc. In case anyone wants other good alternatives, there's llama.cpp and vLLM, each harder to setup but opens the door for squeezing more performance out of your hardware. I've also heard okay things about LM Studio and I think Unsloth had their own thing as well: https://unsloth.ai/docs/new/studio https://unsloth.ai/docs/new/studio > Mention it whenever it comes up. You could also vote with your valley and support orgs that release open weights of their near-SOTA models, though nowadays that means giving cash to primarily Chinese companies (e.g. Moonshot and Z.ai). For what it's worth, people are also complaining about them decreasing usage quotas on their subscription offerings as well so seems like the squeeze is everywhere, even DeepSeek raised their prices (which is better than them going broke, I guess).
  - wongarsu 13d ago
    sglang is also worth mentioning. Fills a similar role as vLLM, but with less fiddling at the knobs to get a working setup For a GUI experience that also serves an OpenAI-compatible API, Unsloth studio or LM Studio is probably the way to go
athrowaway3z 13d ago
Unsubscribe from Anthropic because they block third party harness on subscriptions. As long as you keep providers replaceable things will be fine. Google, Facebook, Apple etc were much better deals around 2010 before they became entrenched and irreplaceable, and thus able to extract & enshitify without people leaving.
- intrasight 13d ago
  I agree on Google and Apple being irreplaceable. But Facebook is easy to leave.
  - wasmitnetzen 13d ago
    For you, sure. But there are a lot of communities running entirely there. Our local farmer's market is preorder only, and is entirely run off Facebook. I would have to skip going there altogether. And it's not viable to propose any alternative. There is no platform, closed or open, where there are enough people registered to be actually usable, and nobody will make a separate account for this.
  - Pavilion2095 13d ago
    In many countries Meta platforms—Facebook, Instagram, WhatsApps—are primary means of communication. If you want to find interesting events happening this weekend, you need to be on Facebook. In this regard, Google and Apple are much easier to leave because they offer personalised services, and you can always move your email and photos elsewhere. But you cannot move other people from Facebook or Instagram.
    - sajithdilshan 13d ago
      It’s not that easy to leave Google or Apple. As an example I wanted to leave iCloud and stop paying storage tax for Apple. But they have made it almost impossible to migrate the photos. There is no API to download the photos programmatically and the only way is to sync all the photos to MacBook and then move it. However, the photos are close to 1TB of storage and my MacBook doesn’t have enough storage to sync all those photos locally. Same goes for Gmail. I have used my Gmail address for countless services and it’s almost part of my official identity. The benefits of moving away from Gmail doesn’t justify the effort I have to put in
znite 12d ago
OpenCode+OpenRouter is probably one of the easiest combos
xenodium 12d ago
Support vendor agnostic tools offering agent integrations through the likes of Agent Client Protocol https://agentclientprotocol.com https://agentclientprotocol.com ps. I built Emacs integration through ACP https://github.com/xenodium/agent-shell https://github.com/xenodium/agent-shell

I for one am running what I can on my aging 1080Ti(!!), namely a Qwen2.5 14B (4-bit quantized). It’s not great, and the only other card I have is a 3070Ti but I need that for gaming. :( What a terrible card that 3070Ti is. Mad regrets buying it because I wanted to save $400 compared to a 3080Ti.

pixelatedindex 12d ago
I really don’t understand why this is being downvoted? Did I say something wrong / controversial? Or is it because I’m not using my resources the “optimal” way? Is it because my models aren’t open weight? I was only trying to contribute my experience :(

oefrha 13d ago

By sending some cash (and maybe some non-sensitive training data too) the way of labs creating open models? At work we have expensive Anthropic and OpenAI subscriptions but also have some in house workflows plugged into DeepSeek and GLM APIs.

LtWorf 13d ago

By not harassing projects with AI generated pull requests.

dv_dt 13d ago
A pull request is a communication to the project. AI or no AI, a poorly communicated request is a burden. IMHO AI prs are fine as long as the submitter has done the work to refine that communication to make it easy to read and assess.
- LtWorf 13d ago
  True, the problem is that AI ones are 500x longer and at a first glance look fine, so they waste much more time. Moreover a human contributor can learn and do better in the next PR, while the AI won't, unless the prompt is changed, but the operator won't learn.
  - dv_dt 13d ago
    A large submission without a breakdown or explanation, or a pass to minimize the code change is junk. The operator can literally ask the ai to compose it better, refactor in to smaller PRs, or do it themself. The annoyance is to have to communicate and enforce expectations to drive by submitters. In the end every project has to figure out what they accept. If, at a glance the PR is drive by junk being thrown over the wall, a response of this doesn't meet our pr submission standards delete, or try again, to me is not overly burdensome.
- owebmaster 12d ago
  AI PRs aren't fine. Open a issue and propose a change, the maintainers decide what to do next with their own AI if they want.

nucleative 13d ago

1. Download ollama and give it a try. It doesn't have to be that one but it's an easy on-ramp. Just get a feel for what open source/open weights is capable of 2. Mention it whenever it comes up. Most people have no clue this is a thing and I think it's useful to make people aware. We need access to uncensored / unbiased LLM models - information wants to be free but there are plenty of businesses gunning for regulatory capture as these are very powerful tools. 3. If you're in a position of developing any project that makes use of AI in any form, check the open source models first, unless you absolutely require the best of the best, these other models are pretty dang capable of almost everything any commercial model can do. 4. If your state or {insert legal jurisdiction here} attempts to regulate access to open source tools for this, oppose it with your vote and your voice. 5. Opposed laws that grant commercial AI suppliers any priority or premium access under government purchasing programs. I'm sure others will chime in, those are a few that pop the mind.

KronisLV 13d ago
> Download ollama and give it a try Weren't people pissed off at them for a variety of reasons? Not giving attribution etc. In case anyone wants other good alternatives, there's llama.cpp and vLLM, each harder to setup but opens the door for squeezing more performance out of your hardware. I've also heard okay things about LM Studio and I think Unsloth had their own thing as well: https://unsloth.ai/docs/new/studio https://unsloth.ai/docs/new/studio > Mention it whenever it comes up. You could also vote with your valley and support orgs that release open weights of their near-SOTA models, though nowadays that means giving cash to primarily Chinese companies (e.g. Moonshot and Z.ai). For what it's worth, people are also complaining about them decreasing usage quotas on their subscription offerings as well so seems like the squeeze is everywhere, even DeepSeek raised their prices (which is better than them going broke, I guess).
- wongarsu 13d ago
  sglang is also worth mentioning. Fills a similar role as vLLM, but with less fiddling at the knobs to get a working setup For a GUI experience that also serves an OpenAI-compatible API, Unsloth studio or LM Studio is probably the way to go

athrowaway3z 13d ago

Unsubscribe from Anthropic because they block third party harness on subscriptions. As long as you keep providers replaceable things will be fine. Google, Facebook, Apple etc were much better deals around 2010 before they became entrenched and irreplaceable, and thus able to extract & enshitify without people leaving.

intrasight 13d ago
I agree on Google and Apple being irreplaceable. But Facebook is easy to leave.
- wasmitnetzen 13d ago
  For you, sure. But there are a lot of communities running entirely there. Our local farmer's market is preorder only, and is entirely run off Facebook. I would have to skip going there altogether. And it's not viable to propose any alternative. There is no platform, closed or open, where there are enough people registered to be actually usable, and nobody will make a separate account for this.
- Pavilion2095 13d ago
  In many countries Meta platforms—Facebook, Instagram, WhatsApps—are primary means of communication. If you want to find interesting events happening this weekend, you need to be on Facebook. In this regard, Google and Apple are much easier to leave because they offer personalised services, and you can always move your email and photos elsewhere. But you cannot move other people from Facebook or Instagram.
  - sajithdilshan 13d ago
    It’s not that easy to leave Google or Apple. As an example I wanted to leave iCloud and stop paying storage tax for Apple. But they have made it almost impossible to migrate the photos. There is no API to download the photos programmatically and the only way is to sync all the photos to MacBook and then move it. However, the photos are close to 1TB of storage and my MacBook doesn’t have enough storage to sync all those photos locally. Same goes for Gmail. I have used my Gmail address for countless services and it’s almost part of my official identity. The benefits of moving away from Gmail doesn’t justify the effort I have to put in

znite 12d ago

OpenCode+OpenRouter is probably one of the easiest combos

xenodium 12d ago

Support vendor agnostic tools offering agent integrations through the likes of Agent Client Protocol https://agentclientprotocol.com https://agentclientprotocol.com ps. I built Emacs integration through ACP https://github.com/xenodium/agent-shell https://github.com/xenodium/agent-shell