7 ms·
Open source != open weight. Big difference and it bugs me that nobody seems to care about using the right words in only this context.
by s0ss 1mo ago
Open source != open weight.
Big difference and it bugs me that nobody seems to care about using the right words in only this context.
- semiquaver 1mo agoWords mean what people use them to mean. Ship has sailed whether you approve or not.
- s0ss 1mo agoYou can choose to use the wrong words all you want. Doing so intentionally is an interesting choice.
- tripleee 1mo agoWe're not talking about standard code here. It's totally reasonable to change the meaning depending on context.
- s0ss 1mo agoI vehemently disagree. There's no source and it's not open, the licenses are often abysmal too.
- ben_w 1mo agoCommunication is not a one-player game. For example, if I was being pedantic: > nobody seems to care about using the right words in only this context. "nobody" would include you. And then I might complain about we use the word "weight" for something massless, or how "bugs me" is *ento*mologically incorrect: https://xkcd.com/1012/ https://xkcd.com/1012/ This is of course not a good use of time. I wonder if illustrating the point about how language is dynamic and meanings are descriptive not proscriptive, was a good use?
- s0ss 1mo agoyeah, I'm not sure what your goal is pedantically picking apart my argument when the subject at hand is clearly not "open source". love xkcd though. :P
- ben_w 1mo agoThe point is to give you empathy for those you disagree with here.
- throw0101d 1mo ago> And then I might complain about we use the word "weight" for something massless, or how "bugs me" is entomologically incorrect: https://xkcd.com/1012/ https://xkcd.com/1012/ "Communicating": https://xkcd.com/1860/ https://xkcd.com/1860/
- ben_w 1mo agoNot sure how the quoted examples fits with your link, TBH. https://www.merriam-webster.com/grammar/descriptive-vs-prescriptive-defining-lexicography https://www.merriam-webster.com/grammar/descriptive-vs-presc... Prescriptively, a "model weight" weighs nothing. Descriptively, this is stupid: a new meaning was created by common use. *Common* use says the weights you can download and run locally are "open source". May not like it, but it's not one person acting alone; and neither are the objectors to the shifting language, it's just the former is more "common" than the latter.
- JumpCrisscross 1mo agoYup. An extremist wing of the FOSS movement ceded the debate early on by trying to insist open source required full access to the training data. Philosophically, not wrong. But practically fucked, so the word evolved. Within tech circles, open weight != open source. Outside them, they’re synonyms.
- noosphr 1mo agoTraining data without the training regime is useless. A cake is not open source because they list the ingredients.
- JumpCrisscross 1mo agoOpen source captures a practical utility as well as a philosophy. When those two cease to converge, the practical prerogative wins. The correct battle would have been weights + regime. But extremists insisted on data, too, which left Meta as the only other real voice arguing with anything practical. They had open weights. I think eventually open use was negotiated and that closed the case except for the folks still arguing about how to pronounce GIF.
- deleted 1mo ago[deleted]
- glitchc 1mo agoIsn't open source not the ingredients but the recipe?
- Der_Einzige 1mo agoWrong. The data (and compute) matters infinitely more than the training regime/architecture. This is the Bitter lesson. I could rerun Kimi K3 with full transformers instead of the hybrid mamba architecture and it would only get better.
- sterlind 1mo agoalso, you can do a lot to finetune or repurpose an open weight model. much more easily than you can mod closed source binaries.
- deleted 1mo ago[deleted]
- deleted 1mo ago[deleted]
- dragonwriter 1mo agoIf the weight, training and inference code, and training data are all released under of Open Source (OSI definition) license, the it is unmistakably “open source”. As you drift from that it becomes less clearly so, and when you get to no training data, and the model weights license having extensive limitations on allowed uses, the use of even “open weights” becomes deceptive.
- maleldil 1mo agoIs there any relevant model that meets that "open source" definition?
- crabmusket 1mo agoIt depends on if you count https://allenai.org/ https://allenai.org/ as relevant?
- maleldil 1mo agoI'm familiar with Ai2. I've used their resources extensively over the years. However, no one is using Olmo for "serious" work, and the name is only known to a small subset in academia.
- dragonwriter 1mo agoNo major language model I am aware of meets the polar extreme that I describe as unmistakably open source, because even those with transparent training data (like IBM Granite) generally do kot use exclusively training data that either they own and can control the license, are under an open license, or are public domain. OTOH, to the extent that the original model trainers rely on training on certain data not requiring a license from the copyright holder, there is at least an argument that with an open source licenses for the weights and training and inference code, a transparent training corpus to which the original trainer has relied on no special permissions not granted to the general public to train on it, to the extent that the legal theory behind the original trainer believing that it is free to train on the data is correct, provides all of the essential features of open source. At the same time, there are things portrayed as open weights where training data is undisclosed and the weights have a license which limits purpose of use and other aspects of use; the models are free-of-charge (for limited uses) but not meaningfully open.
- JoshTriplett 1mo agoAlso, llama isn't even open source; it's proprietary, with the source available.