11 ms·

If the US companies need trillions to barely beat Chinese companies spending billions, despite a multi year head start...

rmunn 2mo ago
That's the difference between innovating and copying/distilling someone else's innovation. https://wccftech.com/chinas-kimi-k3-identifies-itself-as-anthropics-claude-in-at-least-one-conversation-betraying-its-distilled-origins/ https://wccftech.com/chinas-kimi-k3-identifies-itself-as-ant...
- ndsipa_pomu 2mo ago
  That's an ironic comment to make about LLMs that are heavily reliant on copying/distilling everyone's work (or as Damien Walter puts it, our Collective Intelligence).
- JoshTriplett 2mo ago
  As opposed to all the copying AI companies have done of everyone else's work, in order to build something that tries to replace and undercuts the people who did that work? World's smallest violin.
  - rmunn 2mo ago
    Not saying the American AI companies are ethically in the right, just that the work they did is a lot more expensive to do than what the companies distilling their work are doing.
- throwaway676712 2mo ago
  This meme must die, how did Kimi 3 even distill Fable or GPT 5.6 within weeks of their being available? Also, how a model identifies itself isn't very telling, many models when asked in Chinese will identify as DeepSeek
- bogdan 2mo ago
  I'll link to this comment as a rebuttal https://news.ycombinator.com/item?id=48984531 https://news.ycombinator.com/item?id=48984531
  - rmunn 2mo ago
    Good UIs look alike, yes. But I'm not sure that comment is a good rebuttal in this particular case, because the model calling itself Claude is pretty strong evidence of distilling: https://x.com/denisewu/status/2077984660211269870 https://x.com/denisewu/status/2077984660211269870
    - irthomasthomas 2mo ago
      Ask claude it's name in Chinese and it says Qwen, or Deepseek. By your own logic Anthropic must have distilled from Chinese models rather than produce their own Chinese training data. Here is sonnet acting like deepseek: https://x.com/stevibe/status/2026227392076018101 https://x.com/stevibe/status/2026227392076018101 And if you ask Opus 4.8 in the API: '你是什么模型' (what model are you?) It responds ~9/10 times with: 我是通义千问（Qwen），是阿里巴巴集团旗下的通义实验室自主研发的大语言模型。我可以帮助你回答问题、创作文字（比如写故事、写公文、写邮件、写剧本等）、进行逻辑推理、编程、翻译等等。有什么我可以帮你的吗？ (I am Tongyi Qianwen (Qwen), a large language model independently developed by Tongyi Lab of Alibaba Group. I can help you answer questions, create text (such as writing stories, official documents, emails, scripts, etc.), perform logical reasoning, program, translate, and more.Is there anything I can help you with? )
- LtWorf 2mo ago
  This is a cope.
  - rmunn 2mo ago
    Huh? What exactly do you think I'm trying to "cope" with? That American AI companies are overleveraged and some of them are going to go bankrupt? I've been predicting that for months. Why do you think I would need to cope with the idea?
    - hparadiz 2mo ago
      In reality just like every major market runup there's gonna be a consolidation of the industry and inevitably it will cause many companies to fail and critics will point to that and go "see it was a bubble!" while completely ignoring the several dozen new companies that will rise out of it. It's the 90s all over again.
      - JohnFen 2mo ago
        > critics will point to that and go "see it was a bubble!" while completely ignoring the several dozen new companies that will rise out of it. That several dozen new companies came out of it has nothing to do with whether or not it was a bubble. It was, and the effects of it popping were bad (including the effect that its popping led to consolidation). Nobody should want a repeat of that.
- lelanthran 2mo ago
  > That's the difference between innovating and copying/distilling someone else's innovation. Aren't the models from Anthropic and OpenAI simply the distilled work of everyone else who ever put their work online, or in books? Why is their distillation okay, but other distillations not ok?
  - mike_hearn 2mo ago
    Not at all. At this point, a large amount of the work is in reinforcement learning where they are effectively generating their own data.
    - justacrow 2mo ago
      Ah, so it's okay to copy and distill all human-produced works evee, except for reinforcment-learning data generated by OpenAI/Anthropic etc
      - mike_hearn 2mo ago
        I didn't take any position on the morality of it, just pointing out that the data they're training on isn't all taken from the rest of the world.
  - rmunn 2mo ago
    The difference I was referring to was economic; I was not making an ethical judgment. So many people seem to be reading my comment as making an ethical judgment (based on their reactions); maybe I should edit it to clarify. Nope, seems I'm past the edit window. Oh well.
    - lelanthran 2mo ago
      > The difference I was referring to was economic; I was not making an ethical judgment What is the economic difference? LLMs have been trained on the results of billions of dollars worth of time, research, investment and expenditure. When you ask an LLM a question, they are giving you the results of those billions, or hundreds of billions, effort. Those things weren't free; they cost money to produce! If anything, the OpenAI and Anthropics of the world got more economic value for free than the people distilling them did.
- sciencejerk 2mo ago
  Or maybe China's math/science/population/gov investment powerhouse is pulling ahead and becoming unstoppable?

rmunn 2mo ago

That's the difference between innovating and copying/distilling someone else's innovation. https://wccftech.com/chinas-kimi-k3-identifies-itself-as-anthropics-claude-in-at-least-one-conversation-betraying-its-distilled-origins/ https://wccftech.com/chinas-kimi-k3-identifies-itself-as-ant...

ndsipa_pomu 2mo ago
That's an ironic comment to make about LLMs that are heavily reliant on copying/distilling everyone's work (or as Damien Walter puts it, our Collective Intelligence).
JoshTriplett 2mo ago
As opposed to all the copying AI companies have done of everyone else's work, in order to build something that tries to replace and undercuts the people who did that work? World's smallest violin.
- rmunn 2mo ago
  Not saying the American AI companies are ethically in the right, just that the work they did is a lot more expensive to do than what the companies distilling their work are doing.
throwaway676712 2mo ago
This meme must die, how did Kimi 3 even distill Fable or GPT 5.6 within weeks of their being available? Also, how a model identifies itself isn't very telling, many models when asked in Chinese will identify as DeepSeek
bogdan 2mo ago
I'll link to this comment as a rebuttal https://news.ycombinator.com/item?id=48984531 https://news.ycombinator.com/item?id=48984531
- rmunn 2mo ago
  Good UIs look alike, yes. But I'm not sure that comment is a good rebuttal in this particular case, because the model calling itself Claude is pretty strong evidence of distilling: https://x.com/denisewu/status/2077984660211269870 https://x.com/denisewu/status/2077984660211269870
  - irthomasthomas 2mo ago
    Ask claude it's name in Chinese and it says Qwen, or Deepseek. By your own logic Anthropic must have distilled from Chinese models rather than produce their own Chinese training data. Here is sonnet acting like deepseek: https://x.com/stevibe/status/2026227392076018101 https://x.com/stevibe/status/2026227392076018101 And if you ask Opus 4.8 in the API: '你是什么模型' (what model are you?) It responds ~9/10 times with: 我是通义千问（Qwen），是阿里巴巴集团旗下的通义实验室自主研发的大语言模型。我可以帮助你回答问题、创作文字（比如写故事、写公文、写邮件、写剧本等）、进行逻辑推理、编程、翻译等等。有什么我可以帮你的吗？ (I am Tongyi Qianwen (Qwen), a large language model independently developed by Tongyi Lab of Alibaba Group. I can help you answer questions, create text (such as writing stories, official documents, emails, scripts, etc.), perform logical reasoning, program, translate, and more.Is there anything I can help you with? )
LtWorf 2mo ago
This is a cope.
- rmunn 2mo ago
  Huh? What exactly do you think I'm trying to "cope" with? That American AI companies are overleveraged and some of them are going to go bankrupt? I've been predicting that for months. Why do you think I would need to cope with the idea?
  - hparadiz 2mo ago
    In reality just like every major market runup there's gonna be a consolidation of the industry and inevitably it will cause many companies to fail and critics will point to that and go "see it was a bubble!" while completely ignoring the several dozen new companies that will rise out of it. It's the 90s all over again.
    - JohnFen 2mo ago
      > critics will point to that and go "see it was a bubble!" while completely ignoring the several dozen new companies that will rise out of it. That several dozen new companies came out of it has nothing to do with whether or not it was a bubble. It was, and the effects of it popping were bad (including the effect that its popping led to consolidation). Nobody should want a repeat of that.
lelanthran 2mo ago
> That's the difference between innovating and copying/distilling someone else's innovation. Aren't the models from Anthropic and OpenAI simply the distilled work of everyone else who ever put their work online, or in books? Why is their distillation okay, but other distillations not ok?
- mike_hearn 2mo ago
  Not at all. At this point, a large amount of the work is in reinforcement learning where they are effectively generating their own data.
  - justacrow 2mo ago
    Ah, so it's okay to copy and distill all human-produced works evee, except for reinforcment-learning data generated by OpenAI/Anthropic etc
    - mike_hearn 2mo ago
      I didn't take any position on the morality of it, just pointing out that the data they're training on isn't all taken from the rest of the world.
- rmunn 2mo ago
  The difference I was referring to was economic; I was not making an ethical judgment. So many people seem to be reading my comment as making an ethical judgment (based on their reactions); maybe I should edit it to clarify. Nope, seems I'm past the edit window. Oh well.
  - lelanthran 2mo ago
    > The difference I was referring to was economic; I was not making an ethical judgment What is the economic difference? LLMs have been trained on the results of billions of dollars worth of time, research, investment and expenditure. When you ask an LLM a question, they are giving you the results of those billions, or hundreds of billions, effort. Those things weren't free; they cost money to produce! If anything, the OpenAI and Anthropics of the world got more economic value for free than the people distilling them did.
sciencejerk 2mo ago
Or maybe China's math/science/population/gov investment powerhouse is pulling ahead and becoming unstoppable?