4 ms·

This is why I'm so hesitant to use AI. There are other reasons (for example: the journey is the fun part, not the destination; and I want to learn not have some

This is why I'm so hesitant to use AI. There are other reasons (for example: the journey is the fun part, not the destination; and I want to learn not have something done for me), but the pressure AI puts on memory and storage is too much for me. I do use it occasionally, mostly to keep tabs on it and for small problem solving (how do I do this one piece of a query in MySQL, since I'm not a MySQL DBA), but that's it for now, until this memory/storage issue is fixed.

rickydroll 1mo ago
Unfortunately, I can't not use AI because it provides the best speech recognition I've ever used and, going along with hand problems, it now allows me to write code after a three-year absence from that skill. I do miss it. I'd never do it again professionally in the large, but I have missed writing code a lot. My solution is to see what four or five-year-old equipment I can buy that will let me run local LLMs. I may only get six or seven tokens per second out of an i7, but it's a start. And best of all, I can turn the machine off when I'm not using it. IMO, Migrating to small-scale local LLMs would be a significant improvement over using data centers.
- BeetleB 1mo ago
  > Unfortunately, I can't not use AI because it provides the best speech recognition I've ever used and, going along with hand problems, it now allows me to write code after a three-year absence from that skill. I'll second this. The combination of Whisper + LLM makes speech recognition fantastic. I occasionally have arm pain from typing, and this is a Godsend. I don't use it to write code - but in my experience stuff like emails + docs was the greater source of pain (one generally types slower while coding).
- sfRattan 1mo ago
  > Migrating to small-scale local LLMs would be a significant improvement over using data centers. This shift is probably inevitable, but it will vary significantly by region depending on prices for electricity. Look at, for example, the difference in fundamental homelab build recommendations between Germans and just about anyone else. Electricity prices in Germany are so high that even a now expensive Raspberry Pi or other ARM board is often preferred over Intel/AMD builds due to low power draw (especially low idle power draw), an effect that adds up for a machine running all the time over years. With local LLMs and the GPUs to run it, especially if you want a model available to you all the time and can remote into your local network to use it whenever you want, there's no escaping much higher power draws, even at idle. Wherever electricity is expensive, the electric bill can be a prohibitive barrier.
  - pixl97 1mo ago
    Smaller solar setups might become a hedge against high power bills in the future it seems.
  - butvacuum 1mo ago
    No-Load losses on the power conversion seems to be the real issue and is rarely mentioned and understood even less often. Plus the conversion eff. curve. Eg: I shaved ~40W off the idle load on a server (250->210W) by doung nothing more than removing the redundant supply
    - rmunn 1mo ago
      You just reminded me of a funny story about redundant power supply. A college buddy used to work at Motorola (I'm naming the company because they wouldn't mind this story being shared) back in the late 90's or early 2000's. They had redundant power to their campus, bought from two different companies, coming in on opposite sides of the campus, so that even if some backhoe operator cut a ground-based power line somewhere, they wouldn't lose power. And yet, one morning, the power went off all across their campus. After a little investigation, they sent pretty much all their employees home at noon and told them "take the afternoon off, don't come back until tomorrow, you wouldn't be able to do any work anyway". Turns out that although the power lines came in at opposite sides of their campus, somewhere a few miles away both of the power lines feeding their campus ended up running through the same underground conduit. And yes, a backhoe had managed to cut that conduit and break both of the lines they depended on at the same time. They had a single, VERY non-obvious, point of failure, and the backhoe had unerringly homed in on that SPoF.
- Aurornis 1mo ago
  > IMO, Migrating to small-scale local LLMs would be a significant improvement over using data centers. LLM serving is most efficient when you batch a lot of parallel requests together. Data center solutions also have the advantage of collecting queries from around the globe, so the hardware can be utilized around the clock. Having everyone serve their own local LLMs would produce a lot more memory demand. Not less. The same memory would be idle most of the time, and when it was used it would be used for 1 person instead of a batch of requests. There are other reasons to run local LLMs, but solving hardware demand problems is not one of them.
  - theLiminator 1mo ago
    Yeah, local LLMs are probably an order of magnitude less efficient at a fixed level of "intelligence" if not more.
    - tjwebbnorfolk 1mo ago
      Inefficient per watt, yes, but local inference capacity is greatly underutilized in aggregate. If a model can run on a machine that already exists, that's a bunch of additional chips that don't need to be built.
      - rickydroll 1mo ago
        You picked up on what I didn't write existing machines don't have to be built, you just use them. It occurred to me much later that local machines could become a shared resource of a small co-op. Or it only runs off solar or energy harvested at times of excess generation (too much daytime solar, lots of wind, or overnight when demand naturally drops). Exploring how to partition inference across many machines and shed load for idle sections of hardware would be a problem that could be solved by a community of smart people like us here
        tjwebbnorfolk 1mo ago
        yea right now local models are like the swing producer. When claude throws 503s or 429s at me, or I run out of quota, I fire up a local model. It's also resilience. I can still do AI stuff if the internet goes down.
- globular-toast 1mo ago
  You can not use it. You spent your life minus a couple of years not using it. You could totally get back into coding but LLMs have you trapped like a tethered elephant.
  - folkrav 1mo ago
    Missed the part where they had stopped entirely because of chronic pain? Are you somehow aware of the levels of pain they deal with daily to be able to say they could “totally” do it without?
    - globular-toast 1mo ago
      They didn't say anything about chronic pain, but with that extra context I do understand now.
      - folkrav 1mo ago
        Re-reading they indeed didn’t say “chronic pain”, that was me extrapolating. However I think it’s still relatively safe to assume “hand problems” prevalent enough to prevent them from typing code for many years to be both painful and chronic.
        globular-toast 1mo ago
        You're right. I misread it. I'd hate to lose my ability to type and jump at the chance to be able to use a computer again.
        rickydroll 1mo ago
        You're right, and problems were chronic and painful and as I said elsewhere, have mutated over the past 30 odd years. Speech recognition error misrecognized 30 years as 3 years.
  - rickydroll 1mo ago
    Speech recognition got the number of years wrong. It's more like 30 years. I was forced to stop programming in 1994. And hand problems was an aggregate thing. first chronic pain then fine motor control and tremors. The pain flares up again if I try to use my hands too much. I don't see the llms have me trapped like a tethered elephant anymore than the other layers of entrapment that we live with. I would argue entrapment started when we learned the secret was banging rocks together. Or did you mean some other kind of entrapment?
- deleted 1mo ago
  [deleted]
- apple4ever 1mo ago
  That's definitely fair, and for people like you, it makes sense. And yes, running local ones actually are the best of both worlds I think, because it puts less pressure on the memory/storage needs as a whole.
1718627440 1mo ago
First, I am kinda the same. > how do I do this one piece of a query in MySQL, since I'm not a MySQL DBA Honestly for most things, the actual documentation is the best resource. I needed to learn to stop looking for solutions to my issue on various fora (like SO) and just read the primary documentation from the vendor. Have a look at dev.mysql.com or mariadb.com/docs/server
- Anamon 1mo ago
  One of the problems with this is that documentation is increasingly also written with LLMs by the vendors, and now is often so vague and riddled with errors to be basically unusable, if not counterproductive by sending you the wrong way. Microsoft has a great racket going on there. Vibecode shitty, unreliable services, offer wrong information in the slop documentation, then charge an arm and a leg for premium support which, in many cases, will just be an employee regurgitating more wrong AI slop information to you.
- apple4ever 1mo ago
  I do agree, and generally that's what I do. Because then I actually learn instead of fire and forget. In that example specifically, it was a bit of "let me see how this works and how good it is" too.

rickydroll 1mo ago

Unfortunately, I can't not use AI because it provides the best speech recognition I've ever used and, going along with hand problems, it now allows me to write code after a three-year absence from that skill. I do miss it. I'd never do it again professionally in the large, but I have missed writing code a lot. My solution is to see what four or five-year-old equipment I can buy that will let me run local LLMs. I may only get six or seven tokens per second out of an i7, but it's a start. And best of all, I can turn the machine off when I'm not using it. IMO, Migrating to small-scale local LLMs would be a significant improvement over using data centers.

BeetleB 1mo ago
> Unfortunately, I can't not use AI because it provides the best speech recognition I've ever used and, going along with hand problems, it now allows me to write code after a three-year absence from that skill. I'll second this. The combination of Whisper + LLM makes speech recognition fantastic. I occasionally have arm pain from typing, and this is a Godsend. I don't use it to write code - but in my experience stuff like emails + docs was the greater source of pain (one generally types slower while coding).
sfRattan 1mo ago
> Migrating to small-scale local LLMs would be a significant improvement over using data centers. This shift is probably inevitable, but it will vary significantly by region depending on prices for electricity. Look at, for example, the difference in fundamental homelab build recommendations between Germans and just about anyone else. Electricity prices in Germany are so high that even a now expensive Raspberry Pi or other ARM board is often preferred over Intel/AMD builds due to low power draw (especially low idle power draw), an effect that adds up for a machine running all the time over years. With local LLMs and the GPUs to run it, especially if you want a model available to you all the time and can remote into your local network to use it whenever you want, there's no escaping much higher power draws, even at idle. Wherever electricity is expensive, the electric bill can be a prohibitive barrier.
- pixl97 1mo ago
  Smaller solar setups might become a hedge against high power bills in the future it seems.
- butvacuum 1mo ago
  No-Load losses on the power conversion seems to be the real issue and is rarely mentioned and understood even less often. Plus the conversion eff. curve. Eg: I shaved ~40W off the idle load on a server (250->210W) by doung nothing more than removing the redundant supply
  - rmunn 1mo ago
    You just reminded me of a funny story about redundant power supply. A college buddy used to work at Motorola (I'm naming the company because they wouldn't mind this story being shared) back in the late 90's or early 2000's. They had redundant power to their campus, bought from two different companies, coming in on opposite sides of the campus, so that even if some backhoe operator cut a ground-based power line somewhere, they wouldn't lose power. And yet, one morning, the power went off all across their campus. After a little investigation, they sent pretty much all their employees home at noon and told them "take the afternoon off, don't come back until tomorrow, you wouldn't be able to do any work anyway". Turns out that although the power lines came in at opposite sides of their campus, somewhere a few miles away both of the power lines feeding their campus ended up running through the same underground conduit. And yes, a backhoe had managed to cut that conduit and break both of the lines they depended on at the same time. They had a single, VERY non-obvious, point of failure, and the backhoe had unerringly homed in on that SPoF.
Aurornis 1mo ago
> IMO, Migrating to small-scale local LLMs would be a significant improvement over using data centers. LLM serving is most efficient when you batch a lot of parallel requests together. Data center solutions also have the advantage of collecting queries from around the globe, so the hardware can be utilized around the clock. Having everyone serve their own local LLMs would produce a lot more memory demand. Not less. The same memory would be idle most of the time, and when it was used it would be used for 1 person instead of a batch of requests. There are other reasons to run local LLMs, but solving hardware demand problems is not one of them.
- theLiminator 1mo ago
  Yeah, local LLMs are probably an order of magnitude less efficient at a fixed level of "intelligence" if not more.
  - tjwebbnorfolk 1mo ago
    Inefficient per watt, yes, but local inference capacity is greatly underutilized in aggregate. If a model can run on a machine that already exists, that's a bunch of additional chips that don't need to be built.
    - rickydroll 1mo ago
      You picked up on what I didn't write existing machines don't have to be built, you just use them. It occurred to me much later that local machines could become a shared resource of a small co-op. Or it only runs off solar or energy harvested at times of excess generation (too much daytime solar, lots of wind, or overnight when demand naturally drops). Exploring how to partition inference across many machines and shed load for idle sections of hardware would be a problem that could be solved by a community of smart people like us here
      - tjwebbnorfolk 1mo ago
        yea right now local models are like the swing producer. When claude throws 503s or 429s at me, or I run out of quota, I fire up a local model. It's also resilience. I can still do AI stuff if the internet goes down.
globular-toast 1mo ago
You can not use it. You spent your life minus a couple of years not using it. You could totally get back into coding but LLMs have you trapped like a tethered elephant.
- folkrav 1mo ago
  Missed the part where they had stopped entirely because of chronic pain? Are you somehow aware of the levels of pain they deal with daily to be able to say they could “totally” do it without?
  - globular-toast 1mo ago
    They didn't say anything about chronic pain, but with that extra context I do understand now.
    - folkrav 1mo ago
      Re-reading they indeed didn’t say “chronic pain”, that was me extrapolating. However I think it’s still relatively safe to assume “hand problems” prevalent enough to prevent them from typing code for many years to be both painful and chronic.
      - globular-toast 1mo ago
        You're right. I misread it. I'd hate to lose my ability to type and jump at the chance to be able to use a computer again.
      - rickydroll 1mo ago
        You're right, and problems were chronic and painful and as I said elsewhere, have mutated over the past 30 odd years. Speech recognition error misrecognized 30 years as 3 years.
- rickydroll 1mo ago
  Speech recognition got the number of years wrong. It's more like 30 years. I was forced to stop programming in 1994. And hand problems was an aggregate thing. first chronic pain then fine motor control and tremors. The pain flares up again if I try to use my hands too much. I don't see the llms have me trapped like a tethered elephant anymore than the other layers of entrapment that we live with. I would argue entrapment started when we learned the secret was banging rocks together. Or did you mean some other kind of entrapment?
deleted 1mo ago
[deleted]
apple4ever 1mo ago
That's definitely fair, and for people like you, it makes sense. And yes, running local ones actually are the best of both worlds I think, because it puts less pressure on the memory/storage needs as a whole.

1718627440 1mo ago

First, I am kinda the same. > how do I do this one piece of a query in MySQL, since I'm not a MySQL DBA Honestly for most things, the actual documentation is the best resource. I needed to learn to stop looking for solutions to my issue on various fora (like SO) and just read the primary documentation from the vendor. Have a look at dev.mysql.com or mariadb.com/docs/server

Anamon 1mo ago
One of the problems with this is that documentation is increasingly also written with LLMs by the vendors, and now is often so vague and riddled with errors to be basically unusable, if not counterproductive by sending you the wrong way. Microsoft has a great racket going on there. Vibecode shitty, unreliable services, offer wrong information in the slop documentation, then charge an arm and a leg for premium support which, in many cases, will just be an employee regurgitating more wrong AI slop information to you.
apple4ever 1mo ago
I do agree, and generally that's what I do. Because then I actually learn instead of fire and forget. In that example specifically, it was a bit of "let me see how this works and how good it is" too.