7 ms·

we just have to download the weights and use piratebay, or even emule, just like old times

eurekin 3y ago
Yes, but companies will have no incentive to publish open source models anymore. Or, it could be so difficult/beaurocratic no one will bother and keep it close source
- krunck 3y ago
  Then we'll create a distributed infrastructure for the creation of models. Run some program and donate spare GPU cycles to generate public AI tools that will be made available to all.
  - eurekin 3y ago
    I really really would like to believe this could work in practice. Given current data volume used during the training phase (tb/s), I highly doubt it's possible without two, magnitude changing, breakthroughs at once
- moralestapia 3y ago
  What will actually happen is that innovation there will just move somewhere else (and it has partly done so). This proposal is the US doing that bicycle/stick meme, it will backfire spectacularly.
  - lukeschlather 3y ago
    The innovation is largely happening within the megacorps anyway, this is solely intended to make sure the innovation cannot move somewhere else.
    - eurekin 3y ago
      Still, one of those corporations wants to capture the market and has monopolistic attitude. Meta clearly chose the other direction, when publishing their models and allowing us all to participate in.
    - v3ss0n 3y ago
      Mistral and Falcon is not from megacorps and not even US.and many other opensource chinese models . And both are based models that means they are totally organic outside of US.
    - renewiltord 3y ago
      That’s what they told us. Turns out Google stopped innovating a long time ago. They could say stuff like this when Bard wasn’t out but now we have Mistral and friends to compare to Llama. Now it turns out they were just bullshitting at Google.
      - smoldesu 3y ago
        > Now it turns out they were just bullshitting at Google. I don't think Google was bullshitting when they wrote, documented and released Tensorflow, BERT and flan-t5 to the public. Their failure to beat OpenAI in a money-pissing competition really doesn't feel like it reflects on their capability (or intentions) as a company. It certainly doesn't feel like they were "bullshitting" anyone.
        renewiltord 3y ago
        Everyone told us they had secret tech that they were keeping inside. But then Bard came out and it was like GPT-3. I don’t know man. The proof of the pudding is in the eating. > The innovation is largely happening within the megacorps anyway That was the part I was replying to. Whichever megacorp this is, it’s not Google.
        smoldesu 3y ago
        Hey, feel free to draw your own conclusions. AI quality is technically a subjective topic. For what it's worth though, Google's open models have benched quite competitively with GPT-3 for multiple years now: https://blog.research.google/2021/10/introducing-flan-more-generalizable.html https://blog.research.google/2021/10/introducing-flan-more-g... The flan quantizations are also still pretty close to SOTA for text transformers. Their quality-to-size ratio is much better than a 7b Llama finetune, and it appears to be what Apple based their new autocorrect off of.

Yes, but companies will have no incentive to publish open source models anymore. Or, it could be so difficult/beaurocratic no one will bother and keep it close source

krunck 3y ago
Then we'll create a distributed infrastructure for the creation of models. Run some program and donate spare GPU cycles to generate public AI tools that will be made available to all.
- eurekin 3y ago
  I really really would like to believe this could work in practice. Given current data volume used during the training phase (tb/s), I highly doubt it's possible without two, magnitude changing, breakthroughs at once
moralestapia 3y ago
What will actually happen is that innovation there will just move somewhere else (and it has partly done so). This proposal is the US doing that bicycle/stick meme, it will backfire spectacularly.
- lukeschlather 3y ago
  The innovation is largely happening within the megacorps anyway, this is solely intended to make sure the innovation cannot move somewhere else.
  - eurekin 3y ago
    Still, one of those corporations wants to capture the market and has monopolistic attitude. Meta clearly chose the other direction, when publishing their models and allowing us all to participate in.
  - v3ss0n 3y ago
    Mistral and Falcon is not from megacorps and not even US.and many other opensource chinese models . And both are based models that means they are totally organic outside of US.
  - renewiltord 3y ago
    That’s what they told us. Turns out Google stopped innovating a long time ago. They could say stuff like this when Bard wasn’t out but now we have Mistral and friends to compare to Llama. Now it turns out they were just bullshitting at Google.
    - smoldesu 3y ago
      > Now it turns out they were just bullshitting at Google. I don't think Google was bullshitting when they wrote, documented and released Tensorflow, BERT and flan-t5 to the public. Their failure to beat OpenAI in a money-pissing competition really doesn't feel like it reflects on their capability (or intentions) as a company. It certainly doesn't feel like they were "bullshitting" anyone.
      - renewiltord 3y ago
        Everyone told us they had secret tech that they were keeping inside. But then Bard came out and it was like GPT-3. I don’t know man. The proof of the pudding is in the eating. > The innovation is largely happening within the megacorps anyway That was the part I was replying to. Whichever megacorp this is, it’s not Google.
        smoldesu 3y ago
        Hey, feel free to draw your own conclusions. AI quality is technically a subjective topic. For what it's worth though, Google's open models have benched quite competitively with GPT-3 for multiple years now: https://blog.research.google/2021/10/introducing-flan-more-generalizable.html https://blog.research.google/2021/10/introducing-flan-more-g... The flan quantizations are also still pretty close to SOTA for text transformers. Their quality-to-size ratio is much better than a 7b Llama finetune, and it appears to be what Apple based their new autocorrect off of.