7 ms·

> DeepSeek does not "do for $6M what cost US AI companies billions". I can only speak for Anthropic, but Claude 3.5 Sonnet is a mid-sized model that cost a few

> DeepSeek does not "do for $6M what cost US AI companies billions". I can only speak for Anthropic, but Claude 3.5 Sonnet is a mid-sized model that cost a few $10M's to train (I won't give an exact number). Also, 3.5 Sonnet was not trained in any way that involved a larger or more expensive model (contrary to some rumors). Wow!

jddj 2y ago
The narrative is running away in every direction. There are popular infographics floating around today which explain to the layman that deepseek invented MoE.
- xtracto 2y ago
  Shit... I was just looking at the DeepSeekMoE paper, almost exactly from 1 year ago: https://arxiv.org/html/2401.06066v1 https://arxiv.org/html/2401.06066v1 The red star in the first chart is so funny.
Palmik 2y ago
Anthropic is, according to themselves, using RLAIF... which is basically using LLM as a judge / reward model. So maybe he means that the models they use for RLAIF are not (much?) more expensive than Sonnet 3.5 (e.g. previous Sonnet or Haiku 3 :)).
- highfrequency 2y ago
  Do you have a link to Anthropic saying they use RLAIF?
  - Palmik 2y ago
    https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback https://www.anthropic.com/research/constitutional-ai-harmles...
gadtfly 2y ago
^ This is publicly new information, and the 2nd part especially contradicts consequential rumours that had been all-but-cemented in closely-following outsiders' understanding of Sonnet and Anthropic. Completely aside from anything else in this article.
- gadtfly 2y ago
  Also, though it's not "new information": "Making AI that is smarter than almost all humans at almost all things [...] is most likely to happen in 2026-2027." continues to sail over everybody's head, not a single comment about it, even to shit on it. People will continue to act as though they are literally blind to this, as though they literally don't see it.
  - suraci 2y ago
    we're getting used to it and, personally, i think, if any CEO in this industry dare to say "we won't get a super AI in 2028" many people will be disappointed, some people will be scared, and one prisident be pissed off
  - disgruntledphd2 2y ago
    I find that really, really, really hard to believe, given the current approaches.
  - dragonwriter 2y ago
    > People will continue to act as though they are literally blind to this, as though they literally don't see it. Or like they see it and have learned the appropriate weight to give unsupported predictions of this type from people with a vested interest in them being perceived as true. It not only not new information, its not information at all.

jddj 2y ago

The narrative is running away in every direction. There are popular infographics floating around today which explain to the layman that deepseek invented MoE.

xtracto 2y ago
Shit... I was just looking at the DeepSeekMoE paper, almost exactly from 1 year ago: https://arxiv.org/html/2401.06066v1 https://arxiv.org/html/2401.06066v1 The red star in the first chart is so funny.

Palmik 2y ago

Anthropic is, according to themselves, using RLAIF... which is basically using LLM as a judge / reward model. So maybe he means that the models they use for RLAIF are not (much?) more expensive than Sonnet 3.5 (e.g. previous Sonnet or Haiku 3 :)).

highfrequency 2y ago
Do you have a link to Anthropic saying they use RLAIF?
- Palmik 2y ago
  https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback https://www.anthropic.com/research/constitutional-ai-harmles...

gadtfly 2y ago

^ This is publicly new information, and the 2nd part especially contradicts consequential rumours that had been all-but-cemented in closely-following outsiders' understanding of Sonnet and Anthropic. Completely aside from anything else in this article.

gadtfly 2y ago
Also, though it's not "new information": "Making AI that is smarter than almost all humans at almost all things [...] is most likely to happen in 2026-2027." continues to sail over everybody's head, not a single comment about it, even to shit on it. People will continue to act as though they are literally blind to this, as though they literally don't see it.
- suraci 2y ago
  we're getting used to it and, personally, i think, if any CEO in this industry dare to say "we won't get a super AI in 2028" many people will be disappointed, some people will be scared, and one prisident be pissed off
- disgruntledphd2 2y ago
  I find that really, really, really hard to believe, given the current approaches.
- dragonwriter 2y ago
  > People will continue to act as though they are literally blind to this, as though they literally don't see it. Or like they see it and have learned the appropriate weight to give unsupported predictions of this type from people with a vested interest in them being perceived as true. It not only not new information, its not information at all.