4 ms·
It's about the claim of whether these companies could develop a similarly powerful model without larger companies building their own first, which is an importan
by make3 2mo ago
It's about the claim of whether these companies could develop a similarly powerful model without larger companies building their own first, which is an important point, and it's likely not the case.
It's also about the larger companies explaining why they can't be as efficient, of course they can't, they're not just ripping the outputs of another model that someone else invested billions to train.
- PaulHoule 2mo agoSimply knowing it is possible to do something makes it easier to do.
- cindyllm 2mo ago[dead]
- doctoboggan 2mo agoYeah agreed, from one standpoint I couldn't care less that they did a "distillation attack", but I am interested in knowing if China is able to develop open weight frontier models without the prior existence of a huge model to distill from.
- IncreasePosts 2mo agoWhy would that matter? OpenAI or whatever frontier lab couldn't have built their frontier models without the entirety of humanity unknowingly developing their training set for 5000 years. It would be one thing if Moonshot was breaking into OpenAI servers and stealing trade secrets, but the only thing they are doing is looking at the output of the program, which is exactly the service that OpenAI offers. So, at best, this is a ToS violation. Sucks for the frontier labs I suppose, but live by the sword - die by the sword.
- mrhottakes 2mo ago> they're not just ripping the outputs of another model that someone else invested billions to train. True, they're simply ripping the inputs that humanity invested thousands of years and trillions of dollars to produce.
- make3 2mo agono argument from me here
- xcf_seetan 2mo ago> they're not just ripping the outputs of another model that someone else invested billions to train. If they payed for inference, doesn't they own the output? So if I pay for a model to generate code, isn't that code mine to do with it whatever I want? Just curious.
- make3 2mo agoNot arguing for the morality of it, but if we're going by the law because that's what you're using in your comment ("don't I own" which only matters wrt the law), then you explicitly accepted a Terms of Use which excludes distillation as a use case. Now of course they themselves trained on the whole Internet for free, etc.