5 ms·

Note the announcement at the end, that they're moving away from the non-commercial only license used in some of their models in favour of Apache: We’re renewin

by asb 2y ago

Note the announcement at the end, that they're moving away from the non-commercial only license used in some of their models in favour of Apache: We’re renewing our commitment to using Apache 2.0 license for our general purpose models, as we progressively move away from MRL-licensed models

diggan 2y ago
Note that this seems to be about the weights themselves, AFAIK, the actual training code and datasets (for example) aren't actually publicly available. It's a bit like developing a binary application and slapping a FOSS license on the binary while keeping the code proprietary. Not saying that's wrong or anything, but people reading these announcements tend to misunderstand what actually got FOSS licensed when the companies write stuff like this.
- mcraiha 2y ago
  The binary comparison is a bit bad, since binary can have copyrights. Weights cannot.
  - diggan 2y ago
    Has that actually been tried in court, or is that your guess? Because you seem confident, but I don't think this has been tried (yet)
    - badsectoracula 2y ago
      It is a guess (not the same author) but it'd make sense: weights are machine output so if the output of AI is not under copyright because it is machine output (which seems to be something that is pretty much universally agreed upon), then the same would apply for the weights themselves. I'm not sure how someone would argue (in good faith) that training on copyrighted materials does not cause the weights to be a derivative of those materials and the output of their AI is not protected under copyright but the part in the middle, the weights, does fall under copyright. Note that this would be about the weights (i.e. the numbers), not their container.
      - wongarsu 2y ago
        Photographs are machine output too and are famously subject to copyright. The movie Toy Story is also machine output, but I'm confident Disney is enforcing copyright on that. The opinion that AI output isn't copyrightable derives from the opinion of the US Copyright Office, which argues that AI output is more like commissioning an artist than like taking a picture. And since the artist isn't human they can't claim copyright for their work. It's not at all obvious to me that the same argument would hold for the output of AI training. Never mind that the above argument about AI output is just the opinion of some US agency and hasn't been tested in court anywhere in the world.
        michaelt 2y ago
        A photograph is often subject to copyright - but there's actually some nuance here; some countries also require a certain level of creative input by a human. https://en.wikipedia.org/wiki/Threshold_of_originality https://en.wikipedia.org/wiki/Threshold_of_originality Areas of dispute include photographs of famous paintings (is it more in the character of a photocopy?), photographs taken by animals (does the human get copyright if they deliberately created the situation where the animal would take a photograph?), and videos taken automatically (can a CCTV video have an author?) Historically, the results are all over the place.
        badsectoracula 2y ago
        As mentioned by the other comment, the difference lies in how much human effort was put - something by its nature is not possible to ask in a black and white manner that applies everywhere. But in cases which are close to the extremes, it is easy to answer: even though the Toy Story renders are machine output, these renders are the result of a lot of human effort by the human artists that made the 3D scenes, materials, models, animation sequences, etc for the purpose of being used in those renders. So Disney can claim copyright on that sort of "machine output". Similarly, claiming copyright on AI output is like claiming copyright on something like `init_state(42, &s); for (int i=0; i < count; i++) output[i] = next_random(&s);`. While there is a bit of (theoretical) effort involved into choosing 42 as a starting input, ultimately you can't really claim copyright on a bunch of random numbers because you chose the initial seed value. Of course you can claim copyright in the code, but doing the same on the output makes no sense: even the if the idea of owning random numbers isn't absurd enough, consider what would happen if -say- 10000 people did the same thing (and to make things even more clear, what if `init_state` used only 8bits of the given number, therefore making sure that there would be a lot of people ending up with the same numbers). AI is essentially `init_state` and `next_random`, just with more involved algorithms than a random number generator.
    - realusername 2y ago
      If you make them copyrightable, then it means they are a derivative of the training dataset. The only defense these AI companies have is making the weights machine output and thus not copyrightable. But then again that's the theory, the copyright system follows money and it wouldn't be surprising to have contradicting ideas being allowed.
    - conradev 2y ago
      The copyright office is actively figuring it out. From yesterday: https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-2-Copyrightability-Report.pdf https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...
- jacooper 2y ago
  > Note that this seems to be about the weights themselves, AFAIK, the actual training code and datasets (for example) aren't actually publicly available. Like every other open source / source available LLM?
  - amoss 2y ago
    https://github.com/huggingface/open-r1 https://github.com/huggingface/open-r1
    - pridkett 2y ago
      While I hope the HuggingFace is successful here, a plan for building a model is a long way from releasing a model. Mistral has models out there - they allow you to modify them. Yeah, it’s now like what we’re used to. It probably needs something else, but people are doing some great things with them.
  - diggan 2y ago
    Like every other Open Source LLM weights, yes. But looking around, there are models that are 100% FOSS, like OLMo (https://github.com/allenai/OLMo https://github.com/allenai/OLMo). Also, I don't buy the argument that because many in the ecosystem mislabel/mislead people about the licensing, makes it ethically OK for everyone else to do so too.
- eldenring 2y ago
  Its not the exact same since you can still finetune it, you can modify the weights, serve it with different engines, etc. This kind of purity test mindset doesn't help anyone. They are shipping the most modifiable form of their model.
  - deleted 2y ago
    [deleted]
  - diggan 2y ago
    Agree that it's not exactly the same, all analogies have holes, they're simplifications after all. I guess I'm vary of the messaging because I'm a developer 99% thanks to FOSS, and being able to learn from FOSS projects how to build similar stuff myself. Without FOSS, I probably wouldn't have been able to "escape" the working-class my family was "stuck in" when I grew up. I want to do whatever I can to make sure others have the same opportunity, and it doesn't matter if the weights themselves are FOSS or not, others cannot learn how to create their own models based on just looking at the weights. You need to be able to learn the model architecture, training and what datasets models are using too, otherwise you won't get very far. > This kind of purity test mindset doesn't help anyone. They are shipping the most modifiable form of their model. It does help others who might be stuck in the same situation I was stuck in, that's not nothing nor is it about "purity". They're not shipping the most open model they can, they could have done something like OLMo (https://github.com/allenai/OLMo https://github.com/allenai/OLMo) which can teach people how to build their own models from scratch.
    - fzzzy 2y ago
      Keep fighting the good fight. Saying Llama is open source is straight up lying. It's open weights.
      - diggan 2y ago
        Thank you, sometimes it feels weird to argue against people who are generally pro-FOSS but somehow for LLMs are fine with misleading statements. I'm glad at least one other person can see through it, encouraging I'm on the right track :) I'm not sure I'd even call Llama "open weights". For me that would mean I can download the weights freely (you cannot download Llama weights without signing a license agreement) and use them freely, you cannot use them freely + you need to add a notice from Meta/Llama on everything that uses Llama saying: > prominently display “Built with Llama” on a related website, user interface, blogpost, about page, or product documentation. https://www.llama.com/llama3_2/license/ https://www.llama.com/llama3_2/license/ Not sure what the correct label is, but it's not open source nor open weights, as far as I can tell.
- crawshaw 2y ago
  It's not the same as slapping an open source license on a binary, because unencumbered weights are so much more generally useful than your typical program binary. Weights are fine-tunable and embeddable into a wide range of software. To consider just the power of fine tuning: all of the press DeepSeek have received is over their R1 model, a relatively tiny fine-tune on their open source V3 model. The vast majority of the compute and data pipeline work to build R1 was complete in V3, while that final fine-tuning step to R1 is possible even by an enthusiastic dedicated individual. (And there are many interesting ways of doing it.) The insistence every time open sourced model weights come up that it is not "truly" open source is tiring. There is enormous value in open source weights compared to closed APIs. Let us call them open source weights. What you want can be "open source data" or somesuch.
  - diggan 2y ago
    > The insistence every time open sourced model weights come up that it is not "truly" open source is tiring. There is enormous value in open source weights compared to closed APIs. Let us call them open source weights. What you want can be "open source data" or somesuch. Agree that there is more value in open source weights than closed APIs, but what I really want to enable, is people learning how to create their own models from scratch. FOSS to me means being able to learn from other projects, how to build the thing yourself, and I wrote about why this is important to me here: https://news.ycombinator.com/item?id=42878817 https://news.ycombinator.com/item?id=42878817 It's not a puritan view but purely practical. Many companies started using FOSS as a marketing label (like what Meta does) and as someone who probably wouldn't be a software developer without being able to learn from FOSS, it fucking sucks that the ML/AI ecosystem is seemingly OK with the term being hijacked.
    - crawshaw 2y ago
      It's not just a marketing label. The term is not being hijacked. Open source models, open source weights, the license chosen, these are all exetrmely valuable concepts. The thing you want, open source model data pipelines, is a different thing. It's existence in no way invalidates the concept of an open source model. Nothing has been hijacked.
- zamalek 2y ago
  Binaries can do arbitrary things, like report home to a central server. Weights cannot.
  - actualwitch 2y ago
    Depending on format, they might.
    - microtonal 2y ago
      Virtually all models are now distributed as Safetensors/gguf/etc. (which are just metadata + data), not pickled Python classes. Many libraries also don't even load pickled checkpoints anymore unless you add an argument explicitly stating that you want to load an unsafe checkpoint.
  - hiq 2y ago
    Why is that relevant regarding the FOSS aspects of weights / binaries? If I run a binary within a VM and only consider its output and prevent any side-effect host, just like I could just consider the output of an LLM, my binary is still not any closer to being FOSS, is it?
- dismalaf 2y ago
  But the weights can be modified. Also the real key is that you can host it yourself, fine tune and make money from it without restriction. That's what it's really about. No one (well, few) cares about recreating it because if they could they'd simply have made one from scratch themselves.
  - diggan 2y ago
    The same is true for FOSS in general. You're arguing that because no one (almost) builds their own X from scratch, there is therefore no value in having resources available for how to build your own X from scratch. For someone who basically couldn't become a developer with FOSS, this way of thinking is so backwards, especially on Hacker News. I thought we were pro-FOSS in general, but somehow LLMs get a pass because "they're too complicated and no one would build one from scratch".
    - dismalaf 2y ago
      They get a pass because we know what these companies train on (proprietary or private data) but they can't admit it, but they're still giving away multi million dollar models for free. Yes, it'd be nice if it was open and reproducible from start to finish. But let's not let perfect be the enemy of good.
      - diggan 2y ago
        > Yes, it'd be nice if it was open and reproducible from start to finish. But let's not let perfect be the enemy of good. "Let's not let companies exploit well-known definitions for their own gain" is what I'm going for, regardless if we personally gain from it or not.
      - hiq 2y ago
        I still don't understand why they have to mix definitions to confuse developers, and we on top of that apparently have to give up on the true meaning of FOSS. What's so hard about using the term "open weights" or some new term instead of trying to reuse FOSS terms they don't abide to?
youssefabdelm 2y ago
I guess since they're not ahead anymore they decide to go back to open source.
mythz 2y ago
Happy to seem them back to releasing OSS models, we used a lot of their OSS models early last year before they were eclipsed by better models and never bothered to try any of their large models which IMO weren't great value.
dismalaf 2y ago
They must have realized they were becoming irrelevant... I know I forgot about them and have been using other models locally. Openness is a huge win, even if I am using Mistral's hosting service I want to know I can always host it myself too, to protect my business against rug pulls and the like. No one's going to pay for an inferior closed model...
littlestymaar 2y ago
I wonder if that's a consequence of the Deepseek distill release: fine-tuned Qwen and Llama were both released by Deepseek, but not Mistral, and that's was a missed PR opportunity for them for no good reason.
globular-toast 2y ago
What does an Apache licence even mean in this context? It's not software. Is it even copyrightable?

by asb 2y ago

diggan 2y ago

Note that this seems to be about the weights themselves, AFAIK, the actual training code and datasets (for example) aren't actually publicly available. It's a bit like developing a binary application and slapping a FOSS license on the binary while keeping the code proprietary. Not saying that's wrong or anything, but people reading these announcements tend to misunderstand what actually got FOSS licensed when the companies write stuff like this.

mcraiha 2y ago
The binary comparison is a bit bad, since binary can have copyrights. Weights cannot.
- diggan 2y ago
  Has that actually been tried in court, or is that your guess? Because you seem confident, but I don't think this has been tried (yet)
  - badsectoracula 2y ago
    It is a guess (not the same author) but it'd make sense: weights are machine output so if the output of AI is not under copyright because it is machine output (which seems to be something that is pretty much universally agreed upon), then the same would apply for the weights themselves. I'm not sure how someone would argue (in good faith) that training on copyrighted materials does not cause the weights to be a derivative of those materials and the output of their AI is not protected under copyright but the part in the middle, the weights, does fall under copyright. Note that this would be about the weights (i.e. the numbers), not their container.
    - wongarsu 2y ago
      Photographs are machine output too and are famously subject to copyright. The movie Toy Story is also machine output, but I'm confident Disney is enforcing copyright on that. The opinion that AI output isn't copyrightable derives from the opinion of the US Copyright Office, which argues that AI output is more like commissioning an artist than like taking a picture. And since the artist isn't human they can't claim copyright for their work. It's not at all obvious to me that the same argument would hold for the output of AI training. Never mind that the above argument about AI output is just the opinion of some US agency and hasn't been tested in court anywhere in the world.
      - michaelt 2y ago
        A photograph is often subject to copyright - but there's actually some nuance here; some countries also require a certain level of creative input by a human. https://en.wikipedia.org/wiki/Threshold_of_originality https://en.wikipedia.org/wiki/Threshold_of_originality Areas of dispute include photographs of famous paintings (is it more in the character of a photocopy?), photographs taken by animals (does the human get copyright if they deliberately created the situation where the animal would take a photograph?), and videos taken automatically (can a CCTV video have an author?) Historically, the results are all over the place.
      - badsectoracula 2y ago
        As mentioned by the other comment, the difference lies in how much human effort was put - something by its nature is not possible to ask in a black and white manner that applies everywhere. But in cases which are close to the extremes, it is easy to answer: even though the Toy Story renders are machine output, these renders are the result of a lot of human effort by the human artists that made the 3D scenes, materials, models, animation sequences, etc for the purpose of being used in those renders. So Disney can claim copyright on that sort of "machine output". Similarly, claiming copyright on AI output is like claiming copyright on something like `init_state(42, &s); for (int i=0; i < count; i++) output[i] = next_random(&s);`. While there is a bit of (theoretical) effort involved into choosing 42 as a starting input, ultimately you can't really claim copyright on a bunch of random numbers because you chose the initial seed value. Of course you can claim copyright in the code, but doing the same on the output makes no sense: even the if the idea of owning random numbers isn't absurd enough, consider what would happen if -say- 10000 people did the same thing (and to make things even more clear, what if `init_state` used only 8bits of the given number, therefore making sure that there would be a lot of people ending up with the same numbers). AI is essentially `init_state` and `next_random`, just with more involved algorithms than a random number generator.
  - realusername 2y ago
    If you make them copyrightable, then it means they are a derivative of the training dataset. The only defense these AI companies have is making the weights machine output and thus not copyrightable. But then again that's the theory, the copyright system follows money and it wouldn't be surprising to have contradicting ideas being allowed.
  - conradev 2y ago
    The copyright office is actively figuring it out. From yesterday: https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-2-Copyrightability-Report.pdf https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...
jacooper 2y ago
> Note that this seems to be about the weights themselves, AFAIK, the actual training code and datasets (for example) aren't actually publicly available. Like every other open source / source available LLM?
- amoss 2y ago
  https://github.com/huggingface/open-r1 https://github.com/huggingface/open-r1
  - pridkett 2y ago
    While I hope the HuggingFace is successful here, a plan for building a model is a long way from releasing a model. Mistral has models out there - they allow you to modify them. Yeah, it’s now like what we’re used to. It probably needs something else, but people are doing some great things with them.
- diggan 2y ago
  Like every other Open Source LLM weights, yes. But looking around, there are models that are 100% FOSS, like OLMo (https://github.com/allenai/OLMo https://github.com/allenai/OLMo). Also, I don't buy the argument that because many in the ecosystem mislabel/mislead people about the licensing, makes it ethically OK for everyone else to do so too.
eldenring 2y ago
Its not the exact same since you can still finetune it, you can modify the weights, serve it with different engines, etc. This kind of purity test mindset doesn't help anyone. They are shipping the most modifiable form of their model.
- deleted 2y ago
  [deleted]
- diggan 2y ago
  Agree that it's not exactly the same, all analogies have holes, they're simplifications after all. I guess I'm vary of the messaging because I'm a developer 99% thanks to FOSS, and being able to learn from FOSS projects how to build similar stuff myself. Without FOSS, I probably wouldn't have been able to "escape" the working-class my family was "stuck in" when I grew up. I want to do whatever I can to make sure others have the same opportunity, and it doesn't matter if the weights themselves are FOSS or not, others cannot learn how to create their own models based on just looking at the weights. You need to be able to learn the model architecture, training and what datasets models are using too, otherwise you won't get very far. > This kind of purity test mindset doesn't help anyone. They are shipping the most modifiable form of their model. It does help others who might be stuck in the same situation I was stuck in, that's not nothing nor is it about "purity". They're not shipping the most open model they can, they could have done something like OLMo (https://github.com/allenai/OLMo https://github.com/allenai/OLMo) which can teach people how to build their own models from scratch.
  - fzzzy 2y ago
    Keep fighting the good fight. Saying Llama is open source is straight up lying. It's open weights.
    - diggan 2y ago
      Thank you, sometimes it feels weird to argue against people who are generally pro-FOSS but somehow for LLMs are fine with misleading statements. I'm glad at least one other person can see through it, encouraging I'm on the right track :) I'm not sure I'd even call Llama "open weights". For me that would mean I can download the weights freely (you cannot download Llama weights without signing a license agreement) and use them freely, you cannot use them freely + you need to add a notice from Meta/Llama on everything that uses Llama saying: > prominently display “Built with Llama” on a related website, user interface, blogpost, about page, or product documentation. https://www.llama.com/llama3_2/license/ https://www.llama.com/llama3_2/license/ Not sure what the correct label is, but it's not open source nor open weights, as far as I can tell.
crawshaw 2y ago
It's not the same as slapping an open source license on a binary, because unencumbered weights are so much more generally useful than your typical program binary. Weights are fine-tunable and embeddable into a wide range of software. To consider just the power of fine tuning: all of the press DeepSeek have received is over their R1 model, a relatively tiny fine-tune on their open source V3 model. The vast majority of the compute and data pipeline work to build R1 was complete in V3, while that final fine-tuning step to R1 is possible even by an enthusiastic dedicated individual. (And there are many interesting ways of doing it.) The insistence every time open sourced model weights come up that it is not "truly" open source is tiring. There is enormous value in open source weights compared to closed APIs. Let us call them open source weights. What you want can be "open source data" or somesuch.
- diggan 2y ago
  > The insistence every time open sourced model weights come up that it is not "truly" open source is tiring. There is enormous value in open source weights compared to closed APIs. Let us call them open source weights. What you want can be "open source data" or somesuch. Agree that there is more value in open source weights than closed APIs, but what I really want to enable, is people learning how to create their own models from scratch. FOSS to me means being able to learn from other projects, how to build the thing yourself, and I wrote about why this is important to me here: https://news.ycombinator.com/item?id=42878817 https://news.ycombinator.com/item?id=42878817 It's not a puritan view but purely practical. Many companies started using FOSS as a marketing label (like what Meta does) and as someone who probably wouldn't be a software developer without being able to learn from FOSS, it fucking sucks that the ML/AI ecosystem is seemingly OK with the term being hijacked.
  - crawshaw 2y ago
    It's not just a marketing label. The term is not being hijacked. Open source models, open source weights, the license chosen, these are all exetrmely valuable concepts. The thing you want, open source model data pipelines, is a different thing. It's existence in no way invalidates the concept of an open source model. Nothing has been hijacked.
zamalek 2y ago
Binaries can do arbitrary things, like report home to a central server. Weights cannot.
- actualwitch 2y ago
  Depending on format, they might.
  - microtonal 2y ago
    Virtually all models are now distributed as Safetensors/gguf/etc. (which are just metadata + data), not pickled Python classes. Many libraries also don't even load pickled checkpoints anymore unless you add an argument explicitly stating that you want to load an unsafe checkpoint.
- hiq 2y ago
  Why is that relevant regarding the FOSS aspects of weights / binaries? If I run a binary within a VM and only consider its output and prevent any side-effect host, just like I could just consider the output of an LLM, my binary is still not any closer to being FOSS, is it?
dismalaf 2y ago
But the weights can be modified. Also the real key is that you can host it yourself, fine tune and make money from it without restriction. That's what it's really about. No one (well, few) cares about recreating it because if they could they'd simply have made one from scratch themselves.
- diggan 2y ago
  The same is true for FOSS in general. You're arguing that because no one (almost) builds their own X from scratch, there is therefore no value in having resources available for how to build your own X from scratch. For someone who basically couldn't become a developer with FOSS, this way of thinking is so backwards, especially on Hacker News. I thought we were pro-FOSS in general, but somehow LLMs get a pass because "they're too complicated and no one would build one from scratch".
  - dismalaf 2y ago
    They get a pass because we know what these companies train on (proprietary or private data) but they can't admit it, but they're still giving away multi million dollar models for free. Yes, it'd be nice if it was open and reproducible from start to finish. But let's not let perfect be the enemy of good.
    - diggan 2y ago
      > Yes, it'd be nice if it was open and reproducible from start to finish. But let's not let perfect be the enemy of good. "Let's not let companies exploit well-known definitions for their own gain" is what I'm going for, regardless if we personally gain from it or not.
    - hiq 2y ago
      I still don't understand why they have to mix definitions to confuse developers, and we on top of that apparently have to give up on the true meaning of FOSS. What's so hard about using the term "open weights" or some new term instead of trying to reuse FOSS terms they don't abide to?

youssefabdelm 2y ago

I guess since they're not ahead anymore they decide to go back to open source.

mythz 2y ago

Happy to seem them back to releasing OSS models, we used a lot of their OSS models early last year before they were eclipsed by better models and never bothered to try any of their large models which IMO weren't great value.

dismalaf 2y ago

They must have realized they were becoming irrelevant... I know I forgot about them and have been using other models locally. Openness is a huge win, even if I am using Mistral's hosting service I want to know I can always host it myself too, to protect my business against rug pulls and the like. No one's going to pay for an inferior closed model...

littlestymaar 2y ago

I wonder if that's a consequence of the Deepseek distill release: fine-tuned Qwen and Llama were both released by Deepseek, but not Mistral, and that's was a missed PR opportunity for them for no good reason.

globular-toast 2y ago

What does an Apache licence even mean in this context? It's not software. Is it even copyrightable?