7 ms·

The weights is the source code. You are looking for design docs or something.

swatcoder 3mo ago
The "open" in "open source" is traditionally about respecting a user's right to modify a library/application to suit their needs. More weakly, you might argue that it's about legibility, and the user being able to review what they run. The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you. Do weights provide either of those qualities?
- zamadatix 3mo ago
  You don't need the previous training material to customize the weights.
  - sokoloff 3mo ago
    I don’t need the source code to randomly change bytes in the compiled Linux kernel binary either.
    - ben_w 3mo ago
      Fine-tuning weights is easier than retraining the foundation model from scratch with a different corpus.
    - 0-_-0 2mo ago
      Source code makes it easier to modify a binary, training data doesn't make it easier to modify a trained network.
- ben_w 3mo ago
  > The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you. > Do weights provide either of those qualities? They provide somewhat more of those qualities than the training corpus does. Not a lot, especially for "understanding", but more.
dcrazy 3mo ago
That’s not true at all. The weights are the outputs of training. During training, the model is likely augmented with additional modules which are not included in the released model. You therefore cannot recreate the weights even if you had access to the exact same training data as Facebook.
BoingBoomTschak 3mo ago
Same way that a freeware is open source because you can see the bytes, right?
mschuster91 3mo ago
> The weights is the source code. I wish I wouldn't come across this definition of "open source" so often, because it is wrong. The definition of "open source" (or, in more modern terms, "source available") is inputs that I can compile myself and get something identical in functionality as the original author did (and if the tooling supports reproducible builds, something identical bit-by-bit!). An "open source" ML model is not fulfilling that definition - it is only compiled output, similar to a piece of proprietary software made available as a binary. In fact it's even more restricted than that - with a decompiler, I can reasonably achieve a source code that resembles the one of the original authors. With an ML model, there is no way of reversing the "training" process. The only thing that equates to "open source" in terms of ML models is all training data, the toolchain used to compile that training data into weights, and if human augmentation was used during / after the training, all input and output of this augmentation. But no one of the large players will ever release that. First of all, the training data is heavily contaminated. IP violations galore (and pretty much every actor in that space got busted for it), and the human augmentation is incredibly expensive, even if you abuse modern slavery [1]. [1] https://www.theguardian.com/technology/article/2024/jul/06/mercy-anita-african-workers-ai-artificial-intelligence-exploitation-feeding-machine https://www.theguardian.com/technology/article/2024/jul/06/m...
- 0-_-0 2mo ago
  Even if you had all that you would get completely different weights out at the end, and you also don't have the resources to "compile" an LLM because the compilation can cost $100M. If you were given the training data but not the weights, would you consider that open source?

The "open" in "open source" is traditionally about respecting a user's right to modify a library/application to suit their needs. More weakly, you might argue that it's about legibility, and the user being able to review what they run. The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you. Do weights provide either of those qualities?

zamadatix 3mo ago
You don't need the previous training material to customize the weights.
- sokoloff 3mo ago
  I don’t need the source code to randomly change bytes in the compiled Linux kernel binary either.
  - ben_w 3mo ago
    Fine-tuning weights is easier than retraining the foundation model from scratch with a different corpus.
  - 0-_-0 2mo ago
    Source code makes it easier to modify a binary, training data doesn't make it easier to modify a trained network.
ben_w 3mo ago
> The idea is that you have what you need to make some bespoke change to the "source", or that you can at least analyze the source to understand the hows and whys of its behavior, to make sure it suits you. > Do weights provide either of those qualities? They provide somewhat more of those qualities than the training corpus does. Not a lot, especially for "understanding", but more.

dcrazy 3mo ago

That’s not true at all. The weights are the outputs of training. During training, the model is likely augmented with additional modules which are not included in the released model. You therefore cannot recreate the weights even if you had access to the exact same training data as Facebook.

BoingBoomTschak 3mo ago

Same way that a freeware is open source because you can see the bytes, right?

mschuster91 3mo ago

> The weights is the source code. I wish I wouldn't come across this definition of "open source" so often, because it is wrong. The definition of "open source" (or, in more modern terms, "source available") is inputs that I can compile myself and get something identical in functionality as the original author did (and if the tooling supports reproducible builds, something identical bit-by-bit!). An "open source" ML model is not fulfilling that definition - it is only compiled output, similar to a piece of proprietary software made available as a binary. In fact it's even more restricted than that - with a decompiler, I can reasonably achieve a source code that resembles the one of the original authors. With an ML model, there is no way of reversing the "training" process. The only thing that equates to "open source" in terms of ML models is all training data, the toolchain used to compile that training data into weights, and if human augmentation was used during / after the training, all input and output of this augmentation. But no one of the large players will ever release that. First of all, the training data is heavily contaminated. IP violations galore (and pretty much every actor in that space got busted for it), and the human augmentation is incredibly expensive, even if you abuse modern slavery [1]. [1] https://www.theguardian.com/technology/article/2024/jul/06/mercy-anita-african-workers-ai-artificial-intelligence-exploitation-feeding-machine https://www.theguardian.com/technology/article/2024/jul/06/m...

0-_-0 2mo ago
Even if you had all that you would get completely different weights out at the end, and you also don't have the resources to "compile" an LLM because the compilation can cost $100M. If you were given the training data but not the weights, would you consider that open source?