9 ms·

Local models also enable: - not being locked into a provider - not being forced to have your prompts saved by a possible competitor - an alternative to the d

Local models also enable: - not being locked into a provider - not being forced to have your prompts saved by a possible competitor - an alternative to the duolopy we quickly see forming - offline access I fear for the future without local models, much more than the future with them, and would rather everyone had access to a local model than be certain we catch everyone copy+pasting LLM responses. Watermarking would be cool, but it's not worth losing local for.

dTal 1mo ago
To be clear, we agree. The problem is that unless local AIs become "normie friendly" real damn quick, we're gonna lose em, because they're damned inconvenient to power. That is what I fear.
- dpoloncsak 1mo ago
  My apologies, I misunderstood your original comment to mean "We shouldn't have local models if it breaks watermarking" but yes, I think we're on the same page now
- bethekidyouwant 1mo ago
  I don’t get how you’re gonna lose them how is a guy from Brussels gonna come to your house and turn off your computer?
  - dTal 1mo ago
    In the short term, governments can suppress free distribution of open weight LLMs. For instance, they could "ask" Hugginface to require an account with a university affiliated email address to download anything. It won't stop you outright, but it will force it underground, slowing everything to a crawl. In the long run... well, general purpose computing is under threat anyway. Platforms with locked bootloaders and mandatory digital signatures outnumber those that don't. Even on "PCs", the openness is merely a cultural norm observed for a particular market segment[0] by Apple and Microsoft - for they are the only ones with true root keys. Even a brand new motherboard comes with Microsoft signing keys pre-flashed, and you are permitted to enroll your own only by grace. The technical infrastructure to flip a switch and lock it all down is now in place, ready to go at the stroke of a pen. Will you be able to get Llama.cpp from "the app store"? I hazard not. If you think this sounds hyperbolic, look at what is happening to Android. [0] They don't even segment it the same! Apple segments by "does it have a keyboard", and Microsoft segments it by "does it have a native x86 processor". Apple runs different software on the same basic hardware, Microsoft runs the same basic software on different hardware, but both have created artificially restricted second class computing categories.
    - bethekidyouwant 1mo ago
      Well I guess all your points are true for Europe but I don’t see how for America. (because freedom) it will certainly cost more but also it already does.
      - dTal 1mo ago
        Hah. Apple, Microsoft, and Google are all American companies.
        bethekidyouwant 1mo ago
        you are suggesting that linux will be banned somehow... which is a take for sure. but not one I share. (understandably I hope.. )?
        dTal 1mo ago
        Not "banned" per se, but functionally impossible to run on hardware that's powerful enough to be useful. It's not a stretch to imagine GPU drivers that refuse to run unless installed on a fully "attested" system. We're already there with video games, it's called "anti-cheat". They'll call it something like "AI abuse protection".
        bethekidyouwant 1mo ago
        Yes of course all of the servers in all the data centres will suddenly start running windows…
    - graemep 1mo ago
      They also do not really care if a few geeks get around the system.
      - dTal 1mo ago
        That's precisely the point - with AI, "a few geeks" turns into "everyone".
- dabbz 1mo ago
  I was going to interject that very sentiment. The upfront costs of running any model of value to the average consumer is so large to be basically non-existent. We need hardware to become commoditized and capable of handling LLMs at the scale necessary to enjoy these agentic workflows.
  - dTal 1mo ago
    Mmm, software too though. Even those with capable hardware (Macbook Pro for instance) are usually incapable of getting over the hump of assembling an inference framework, agentic harness etc. If that ever becomes easy, watch out! Governments will not appreciate the user/developer dichotomy being blown away.

dTal 1mo ago

To be clear, we agree. The problem is that unless local AIs become "normie friendly" real damn quick, we're gonna lose em, because they're damned inconvenient to power. That is what I fear.

dpoloncsak 1mo ago
My apologies, I misunderstood your original comment to mean "We shouldn't have local models if it breaks watermarking" but yes, I think we're on the same page now
bethekidyouwant 1mo ago
I don’t get how you’re gonna lose them how is a guy from Brussels gonna come to your house and turn off your computer?
- dTal 1mo ago
  In the short term, governments can suppress free distribution of open weight LLMs. For instance, they could "ask" Hugginface to require an account with a university affiliated email address to download anything. It won't stop you outright, but it will force it underground, slowing everything to a crawl. In the long run... well, general purpose computing is under threat anyway. Platforms with locked bootloaders and mandatory digital signatures outnumber those that don't. Even on "PCs", the openness is merely a cultural norm observed for a particular market segment[0] by Apple and Microsoft - for they are the only ones with true root keys. Even a brand new motherboard comes with Microsoft signing keys pre-flashed, and you are permitted to enroll your own only by grace. The technical infrastructure to flip a switch and lock it all down is now in place, ready to go at the stroke of a pen. Will you be able to get Llama.cpp from "the app store"? I hazard not. If you think this sounds hyperbolic, look at what is happening to Android. [0] They don't even segment it the same! Apple segments by "does it have a keyboard", and Microsoft segments it by "does it have a native x86 processor". Apple runs different software on the same basic hardware, Microsoft runs the same basic software on different hardware, but both have created artificially restricted second class computing categories.
  - bethekidyouwant 1mo ago
    Well I guess all your points are true for Europe but I don’t see how for America. (because freedom) it will certainly cost more but also it already does.
    - dTal 1mo ago
      Hah. Apple, Microsoft, and Google are all American companies.
      - bethekidyouwant 1mo ago
        you are suggesting that linux will be banned somehow... which is a take for sure. but not one I share. (understandably I hope.. )?
        dTal 1mo ago
        Not "banned" per se, but functionally impossible to run on hardware that's powerful enough to be useful. It's not a stretch to imagine GPU drivers that refuse to run unless installed on a fully "attested" system. We're already there with video games, it's called "anti-cheat". They'll call it something like "AI abuse protection".
        bethekidyouwant 1mo ago
        Yes of course all of the servers in all the data centres will suddenly start running windows…
  - graemep 1mo ago
    They also do not really care if a few geeks get around the system.
    - dTal 1mo ago
      That's precisely the point - with AI, "a few geeks" turns into "everyone".
dabbz 1mo ago
I was going to interject that very sentiment. The upfront costs of running any model of value to the average consumer is so large to be basically non-existent. We need hardware to become commoditized and capable of handling LLMs at the scale necessary to enjoy these agentic workflows.
- dTal 1mo ago
  Mmm, software too though. Even those with capable hardware (Macbook Pro for instance) are usually incapable of getting over the hump of assembling an inference framework, agentic harness etc. If that ever becomes easy, watch out! Governments will not appreciate the user/developer dichotomy being blown away.