8 ms·

RE: Open Source models, there is the AnthropicAI model making the rounds on Twitter[1] and Stability.ai (makers of Stable Diffusion) are working on one [2]. If

RE: Open Source models, there is the AnthropicAI model making the rounds on Twitter[1] and Stability.ai (makers of Stable Diffusion) are working on one [2]. If we use recent history as an example, OpenAI announced DALL-E in Jan 5, 2021 [3], announced v2 and a waitlist for public use in July 20, 2022, and Stable Diffusion shipped an open source model on August 22, 2022 [4] using ~$600K of compute (at retail prices on AWS) [5]. I don't see how it's likely that any company can acquire a durable technology moat here. There are scale barriers to entry, but even VC sized funding can overcome that. [1] https://twitter.com/goodside/status/1611556749726605312 https://twitter.com/goodside/status/1611556749726605312 [2] https://humanloop.com/blog/stability-ai-partnership https://humanloop.com/blog/stability-ai-partnership [3] https://en.wikipedia.org/wiki/DALL-E https://en.wikipedia.org/wiki/DALL-E [4] https://en.wikipedia.org/wiki/Stable_Diffusion https://en.wikipedia.org/wiki/Stable_Diffusion [5] https://twitter.com/emostaque/status/1563870674111832066?lang=en https://twitter.com/emostaque/status/1563870674111832066?lan...

orra 4y ago
Stable Diffusion's code and model weights can be downloaded, but it's proprietary, not Open Source.
- localhost 4y ago
  This is the text of the license from this repo [1]. Seems pretty open to me? About this license The Responsible AI License allows users to take advantage of the model in a wide range of settings (including free use and redistribution) as long as they respect the specific use case restrictions outlined, which correspond to model applications the licensor deems ill-suited for the model or are likely to cause harm. [1] https://huggingface.co/CompVis/stable-diffusion-v-1-4-original https://huggingface.co/CompVis/stable-diffusion-v-1-4-origin...
  - schoen 4y ago
    Compare https://opensource.org/osd https://opensource.org/osd This isn't a very minor point, as this was an explicit discussion and is also OSI's translation of Debian's translation of Richard Stallman's "freedom 0". That is, it's an important, and explicit, tradition/consensus in FOSS that users aren't restricted in the purposes for which they may use the software.
    - localhost 4y ago
      I don't have an opinion on this one way or another, but if the RAIL license concerns you, then perhaps you can take it up with the organization behind it? https://www.licenses.ai/ https://www.licenses.ai/
      - orra 4y ago
        There's no point. RAIL are well aware their licenses are proprietary (see their FAQ), and they are happy with that.
    - optimalsolver 4y ago
      Doesn't Stallman's "free" license dictate what you can and cannot do with the software?
      - orra 4y ago
        No, the GNU licences place copyleft obligations on distribution/conveyance. But they allow you to run the programs for any purpose, without field of endeavour restrictions, or moral police. You don't even need to accept a licence to run a GNU program.
        frabcus 4y ago
        Out of interest are there any copyleft style neural network licenses - eg that require fine tuned model weights are published? (And Affero GPL style in terms of servers and distribution meaning these days)
        orra 4y ago
        I am not an expert, but I'd just use the the normal LGPL/GPL/AGPL licenses for the models.
        optimalsolver 4y ago
        >place copyleft obligations on distribution/conveyance Ok, so a major restriction on what you can do with the software.
  - satvikpendem 4y ago
    That is not open source. That's at most source available. Consider, what will they do if they think you're using ill-suited use cases? How would they even determine something like that?
espadrine 4y ago
Does the AnthropicAI model have published weights? It is listed as closed in HELM[0]. [0]: https://crfm.stanford.edu/helm/v0.1.0/?models=1 https://crfm.stanford.edu/helm/v0.1.0/?models=1
- localhost 4y ago
  Not that I know of. But my understanding is that's their intent.
  - rewq4321 4y ago
    What source is that understanding based on? I was not under the impression that Anthropic was intending to open source their models at all.
localhost 4y ago
This tweet has a lot of additional details with timelines for releasing weights of LLMs: https://twitter.com/swyx/status/1579551634832883712?s=20 https://twitter.com/swyx/status/1579551634832883712?s=20

orra 4y ago

Stable Diffusion's code and model weights can be downloaded, but it's proprietary, not Open Source.

localhost 4y ago
This is the text of the license from this repo [1]. Seems pretty open to me? About this license The Responsible AI License allows users to take advantage of the model in a wide range of settings (including free use and redistribution) as long as they respect the specific use case restrictions outlined, which correspond to model applications the licensor deems ill-suited for the model or are likely to cause harm. [1] https://huggingface.co/CompVis/stable-diffusion-v-1-4-original https://huggingface.co/CompVis/stable-diffusion-v-1-4-origin...
- schoen 4y ago
  Compare https://opensource.org/osd https://opensource.org/osd This isn't a very minor point, as this was an explicit discussion and is also OSI's translation of Debian's translation of Richard Stallman's "freedom 0". That is, it's an important, and explicit, tradition/consensus in FOSS that users aren't restricted in the purposes for which they may use the software.
  - localhost 4y ago
    I don't have an opinion on this one way or another, but if the RAIL license concerns you, then perhaps you can take it up with the organization behind it? https://www.licenses.ai/ https://www.licenses.ai/
    - orra 4y ago
      There's no point. RAIL are well aware their licenses are proprietary (see their FAQ), and they are happy with that.
  - optimalsolver 4y ago
    Doesn't Stallman's "free" license dictate what you can and cannot do with the software?
    - orra 4y ago
      No, the GNU licences place copyleft obligations on distribution/conveyance. But they allow you to run the programs for any purpose, without field of endeavour restrictions, or moral police. You don't even need to accept a licence to run a GNU program.
      - frabcus 4y ago
        Out of interest are there any copyleft style neural network licenses - eg that require fine tuned model weights are published? (And Affero GPL style in terms of servers and distribution meaning these days)
        orra 4y ago
        I am not an expert, but I'd just use the the normal LGPL/GPL/AGPL licenses for the models.
      - optimalsolver 4y ago
        >place copyleft obligations on distribution/conveyance Ok, so a major restriction on what you can do with the software.
- satvikpendem 4y ago
  That is not open source. That's at most source available. Consider, what will they do if they think you're using ill-suited use cases? How would they even determine something like that?

espadrine 4y ago

Does the AnthropicAI model have published weights? It is listed as closed in HELM[0]. [0]: https://crfm.stanford.edu/helm/v0.1.0/?models=1 https://crfm.stanford.edu/helm/v0.1.0/?models=1

localhost 4y ago
Not that I know of. But my understanding is that's their intent.
- rewq4321 4y ago
  What source is that understanding based on? I was not under the impression that Anthropic was intending to open source their models at all.

localhost 4y ago

This tweet has a lot of additional details with timelines for releasing weights of LLMs: https://twitter.com/swyx/status/1579551634832883712?s=20 https://twitter.com/swyx/status/1579551634832883712?s=20