9 ms·

Interview with DeepSeek Founder: We're Done Following. It's Time to Lead

126 points by oli5679 2y ago

oli5679 2y ago
I think this project is awesome and am quite disappointed with some cynical commentary from large American labs. Researcher at Meta or OpenAI spending hundreds of millions on compute, and being paid millions themselves, whilst not publishing any of their learnings openly, here a bunch of very smart, young Chinese researchers have had some great ideas, proved they work, and published details that allow everyone else to replicate. "No “inscrutable wizards” here—just fresh graduates from top universities, PhD candidates (even fourth- or fifth-year interns), and young talents with a few years of experience." "If someone has an idea, they can tap into our training clusters anytime without approval. Additionally, since we don’t have rigid hierarchical structures or departmental barriers, people can collaborate freely as long as there’s mutual interest."
- fngjdflmdflg 2y ago
  Why did you group Meta with OpenAI here?
infecto 2y ago
Maybe worth adding that the interview is from July of last year. This is not a recent interview. Still interesting but was not what I was expecting.
- tobr 2y ago
  On the other hand, if you release something innovative in January, you probably had to already be on the right track in July.
newbie578 2y ago
Doesn't matter if and how much they used OpenAI's models. The only important thing that matters is that they managed to disrupt the status quo, Silicon Valley will need to be more aware going forward.
ben30 2y ago
Its success stems from a refreshingly unconventional approach to innovation. Liang Wenfeng's philosophy of maintaining a flat organizational structure where researchers have unrestricted access to computing resources and can collaborate freely. What's particularly striking is their deliberate choice to stay lean and problem-focused, avoiding the bureaucratic bloat that often plagues AI departments at larger companies. By hiring people driven primarily by curiosity and technical challenges rather than career advancement, they've created an environment where genuine innovation can flourish. AI development doesn't necessarily require massive resources - it's more about fostering the right culture of open collaboration and maintaining focus on the core technical challenges.
- CharlieDigital 2y ago
  The model you described probably works great (not just in AI) as long as it's not your primary and direct source of revenue with which you must pay back investors. Once it becomes your primary and direct source of revenue and you must generate some returns for investors or meet some revenue targets, then whatever you're doing somehow has to align with that revenue stream (often ruining the fun).
  - dundarious 2y ago
    Have you seen the "returns" for OpenAI, etc.? All cutting edge research is subsidized by government or megacorps in USA.
    - CharlieDigital 2y ago
      They are not profitable. The problem is that they have to find their way to profitability because investors and shareholders need to be paid back. And because they have to do that, you could say that it "compromises" on objectives that would more rapidly advance the field like openly sharing their reasoning architecture.
  - bko 2y ago
    You're describing a lot of tech companies like Google that had all these different orgs that were money sinks not related to a direct source of revenue and funded by dominance in search and high margins. And these programs didn't necessarily yield great creative products. Quite the opposite. Whereas if you have some objective measure that's driving your decisions, like revenue or customer engagement (proxy for usefulness), you can drive great results. I think either method can work if you have the right culture.
eduction 2y ago
I think this was super interesting, it sounds like he’s leaning more into “open” than openai is. “In disruptive tech, closed-source moats are fleeting. Even OpenAI’s closed-source model can’t prevent others from catching up. “Therefore, our real moat lies in our team’s growth—accumulating know-how, fostering an innovative culture. Open-sourcing and publishing papers don’t result in significant losses. For technologists, being followed is rewarding. Open-source is cultural, not just commercial. Giving back is an honor, and it attracts talent.”
- halfmatthalfcat 2y ago
  Inspiring and what the Valley (at least in part) use to represent. People doing cool shit as an end, not a means.
- rfoo 2y ago
  Says someone who personally has 50-100 billion USD. And no, it's not net worth through corp shares. The guy is essentially his own LP.
  - rfoo 2y ago
    Correction: I meant 5-10 billion USD, too drunk yesterday
cchance 2y ago
Is that why if you ask it... it says it's based on ChatGPT4 ?
- durumu 2y ago
  Most LLMs do this due to the proliferation of ChatGPT-generated content in the training data.
wouldbecouldbe 2y ago
They are nice words, ironically though their product is an exact clone of a US product (apart from the data stealing discussion). You could argue the cheaper aspect is innovating, but that's what China has been doing for many products.
- falcor84 2y ago
  To the best of my knowledge, there's nothing quite like R1-Zero released by OpenAI or others, they seem to really be pushing innovation. Relevant ARC-Prize post and discussion from yesterday: https://arcprize.org/blog/r1-zero-r1-results-analysis https://arcprize.org/blog/r1-zero-r1-results-analysis https://news.ycombinator.com/item?id=42868390 https://news.ycombinator.com/item?id=42868390
  - wouldbecouldbe 2y ago
    An iteration at best, some Chinese electric cars have a better battery then Tesla and are cheaper. Yet would hardly call them innovative as he claims the chinese should become. It's actually what the Chinese have been doing, copying and making it cheaper with slightly different features.
jgord 2y ago
At the heart of all progress is the mantra that "best idea wins". Maybe DeepSeeks creative use of RL within LLMs will open up founder and VC interest in using RL to solve real problems - I expect to see a cambrian explosion of high growth applied RL startups in engineering,logistics,finance,medicine
helf 2y ago
[dead]
falcor84 2y ago
It's a great interview throughout, but I was thrown off by this strange question (which I found to be much more interesting than the answer): > An Yong: What do you envision as the endgame for large AI models? I don't know if it has a different meaning/connotation in Chinese, but reading this metaphor with a Chess connotation scared me. If there is a game, who are the players? what is the victory condition? will there be a static stalemate, or a definitive win? and most importantly, will there be an opportunity for future games after it, or is this the final game we get to play?
- LelouBil 2y ago
  Isn't "endgame" a common expression to mean "the end", "the place where there's no progress anymore" etc ?
  - falcor84 2y ago
    Sorry, am I the only one who finds this sort of formulation in regard to large AI models existentially intimidating?
    - andrekandre 2y ago
      i guess? but what are you hinting at exactly?
      - falcor84 2y ago
        I am legitimately confused at other people around me thinking that this exponentially evolving technological explosion will end at a steady state that will be at all familiar to us. It's a worn-out metaphor, but I can't help but think of horses marvelling at this mechanical carriage thing, wondering what's the endgame of that.
        andrekandre 2y ago
        > will end at a steady state that will be at all familiar to us. admittedly, it probably wont, but i think a lot of replies like mine above are because your hinting at something vague... what's the concrete worry? > horses marvelling at this mechanical carriage so you mean to say, the end-result of all this is humans will be out-of-the-job so to speak?
mythz 2y ago
Didn't expect to be cheering for Chinese AI companies and Facebook over mega funded US tech corps, but here we are. Were fortunate that not all SOTA AI models are controlled by US Tech corps. Right now they're in the "maximum marketshare at all costs" stage, but they'll be looking for their ROI after achieving a dominant share. I trust OpenAI the least, it's still early on in the AI age and they look like the company that they were formed to prevent. Can only hope that DeepSeek, Facebook, Qwen and Mistral continue to release open models. Unfortunately if a companies motivation is ROI from cloud hosting then they're going to be incentivised to stop releasing their models as OSS to prevent competition which we've seen with Mistral's best models although in their latest model released today under Apache 2.0 the CEO is saying they’re renewing their commitment to Open Source [1], so we’ll have to see how long that holds. We're also starting to see that from Alibaba whose latest Qwen2.5-Max model is only available through their Alibaba Cloud. Luckily Facebook business model isn't reliant on cloud hosting so we should continue to expect Open models from them. So far efficiency seems to be DeepSeek's competitive advantage as despite being OSS they're still the cheapest hosting provider [2] despite other hosting providers not having to recoup any R&D and training costs. [1] https://x.com/arthurmensch/status/1884972984202338450 https://x.com/arthurmensch/status/1884972984202338450 [2] https://openrouter.ai/deepseek/deepseek-r1 https://openrouter.ai/deepseek/deepseek-r1
- yieldcrv 2y ago
  Its open source vs closed source, not China vs US but this new dimension of geopolitical competition is now sidelining the cautionary anti-AGI populace, which was honestly probably saving us a few years
magwa101 2y ago
[dead]
deleted 2y ago
[deleted]
ecret 2y ago
[dead]
deleted 2y ago
[deleted]
walterbell 2y ago
2023 and 2024 interviews, https://www.lesswrong.com/posts/kANyEjDDFWkhSKbcK/two-interviews-with-the-founder-of-deepseek https://www.lesswrong.com/posts/kANyEjDDFWkhSKbcK/two-interv... > Liang Wenfeng is a very rare person in China's AI industry who has abilities in “strong infrastructure engineering, model research, and also resource mobilization”, and “can make accurate high-level judgments, and can also be stronger than a frontline researcher in the technical details”. He has a “terrifying ability to learn” and at the same time is “less like a boss and more like a geek”.

126 points by oli5679 2y ago

I think this project is awesome and am quite disappointed with some cynical commentary from large American labs. Researcher at Meta or OpenAI spending hundreds of millions on compute, and being paid millions themselves, whilst not publishing any of their learnings openly, here a bunch of very smart, young Chinese researchers have had some great ideas, proved they work, and published details that allow everyone else to replicate. "No “inscrutable wizards” here—just fresh graduates from top universities, PhD candidates (even fourth- or fifth-year interns), and young talents with a few years of experience." "If someone has an idea, they can tap into our training clusters anytime without approval. Additionally, since we don’t have rigid hierarchical structures or departmental barriers, people can collaborate freely as long as there’s mutual interest."

fngjdflmdflg 2y ago
Why did you group Meta with OpenAI here?

infecto 2y ago

Maybe worth adding that the interview is from July of last year. This is not a recent interview. Still interesting but was not what I was expecting.

tobr 2y ago
On the other hand, if you release something innovative in January, you probably had to already be on the right track in July.

newbie578 2y ago

Doesn't matter if and how much they used OpenAI's models. The only important thing that matters is that they managed to disrupt the status quo, Silicon Valley will need to be more aware going forward.

ben30 2y ago

Its success stems from a refreshingly unconventional approach to innovation. Liang Wenfeng's philosophy of maintaining a flat organizational structure where researchers have unrestricted access to computing resources and can collaborate freely. What's particularly striking is their deliberate choice to stay lean and problem-focused, avoiding the bureaucratic bloat that often plagues AI departments at larger companies. By hiring people driven primarily by curiosity and technical challenges rather than career advancement, they've created an environment where genuine innovation can flourish. AI development doesn't necessarily require massive resources - it's more about fostering the right culture of open collaboration and maintaining focus on the core technical challenges.

CharlieDigital 2y ago
The model you described probably works great (not just in AI) as long as it's not your primary and direct source of revenue with which you must pay back investors. Once it becomes your primary and direct source of revenue and you must generate some returns for investors or meet some revenue targets, then whatever you're doing somehow has to align with that revenue stream (often ruining the fun).
- dundarious 2y ago
  Have you seen the "returns" for OpenAI, etc.? All cutting edge research is subsidized by government or megacorps in USA.
  - CharlieDigital 2y ago
    They are not profitable. The problem is that they have to find their way to profitability because investors and shareholders need to be paid back. And because they have to do that, you could say that it "compromises" on objectives that would more rapidly advance the field like openly sharing their reasoning architecture.
- bko 2y ago
  You're describing a lot of tech companies like Google that had all these different orgs that were money sinks not related to a direct source of revenue and funded by dominance in search and high margins. And these programs didn't necessarily yield great creative products. Quite the opposite. Whereas if you have some objective measure that's driving your decisions, like revenue or customer engagement (proxy for usefulness), you can drive great results. I think either method can work if you have the right culture.

eduction 2y ago

I think this was super interesting, it sounds like he’s leaning more into “open” than openai is. “In disruptive tech, closed-source moats are fleeting. Even OpenAI’s closed-source model can’t prevent others from catching up. “Therefore, our real moat lies in our team’s growth—accumulating know-how, fostering an innovative culture. Open-sourcing and publishing papers don’t result in significant losses. For technologists, being followed is rewarding. Open-source is cultural, not just commercial. Giving back is an honor, and it attracts talent.”

halfmatthalfcat 2y ago
Inspiring and what the Valley (at least in part) use to represent. People doing cool shit as an end, not a means.
rfoo 2y ago
Says someone who personally has 50-100 billion USD. And no, it's not net worth through corp shares. The guy is essentially his own LP.
- rfoo 2y ago
  Correction: I meant 5-10 billion USD, too drunk yesterday

cchance 2y ago

Is that why if you ask it... it says it's based on ChatGPT4 ?

durumu 2y ago
Most LLMs do this due to the proliferation of ChatGPT-generated content in the training data.

wouldbecouldbe 2y ago

They are nice words, ironically though their product is an exact clone of a US product (apart from the data stealing discussion). You could argue the cheaper aspect is innovating, but that's what China has been doing for many products.

falcor84 2y ago
To the best of my knowledge, there's nothing quite like R1-Zero released by OpenAI or others, they seem to really be pushing innovation. Relevant ARC-Prize post and discussion from yesterday: https://arcprize.org/blog/r1-zero-r1-results-analysis https://arcprize.org/blog/r1-zero-r1-results-analysis https://news.ycombinator.com/item?id=42868390 https://news.ycombinator.com/item?id=42868390
- wouldbecouldbe 2y ago
  An iteration at best, some Chinese electric cars have a better battery then Tesla and are cheaper. Yet would hardly call them innovative as he claims the chinese should become. It's actually what the Chinese have been doing, copying and making it cheaper with slightly different features.

jgord 2y ago

At the heart of all progress is the mantra that "best idea wins". Maybe DeepSeeks creative use of RL within LLMs will open up founder and VC interest in using RL to solve real problems - I expect to see a cambrian explosion of high growth applied RL startups in engineering,logistics,finance,medicine

helf 2y ago

[dead]

falcor84 2y ago

It's a great interview throughout, but I was thrown off by this strange question (which I found to be much more interesting than the answer): > An Yong: What do you envision as the endgame for large AI models? I don't know if it has a different meaning/connotation in Chinese, but reading this metaphor with a Chess connotation scared me. If there is a game, who are the players? what is the victory condition? will there be a static stalemate, or a definitive win? and most importantly, will there be an opportunity for future games after it, or is this the final game we get to play?

LelouBil 2y ago
Isn't "endgame" a common expression to mean "the end", "the place where there's no progress anymore" etc ?
- falcor84 2y ago
  Sorry, am I the only one who finds this sort of formulation in regard to large AI models existentially intimidating?
  - andrekandre 2y ago
    i guess? but what are you hinting at exactly?
    - falcor84 2y ago
      I am legitimately confused at other people around me thinking that this exponentially evolving technological explosion will end at a steady state that will be at all familiar to us. It's a worn-out metaphor, but I can't help but think of horses marvelling at this mechanical carriage thing, wondering what's the endgame of that.
      - andrekandre 2y ago
        > will end at a steady state that will be at all familiar to us. admittedly, it probably wont, but i think a lot of replies like mine above are because your hinting at something vague... what's the concrete worry? > horses marvelling at this mechanical carriage so you mean to say, the end-result of all this is humans will be out-of-the-job so to speak?

mythz 2y ago

Didn't expect to be cheering for Chinese AI companies and Facebook over mega funded US tech corps, but here we are. Were fortunate that not all SOTA AI models are controlled by US Tech corps. Right now they're in the "maximum marketshare at all costs" stage, but they'll be looking for their ROI after achieving a dominant share. I trust OpenAI the least, it's still early on in the AI age and they look like the company that they were formed to prevent. Can only hope that DeepSeek, Facebook, Qwen and Mistral continue to release open models. Unfortunately if a companies motivation is ROI from cloud hosting then they're going to be incentivised to stop releasing their models as OSS to prevent competition which we've seen with Mistral's best models although in their latest model released today under Apache 2.0 the CEO is saying they’re renewing their commitment to Open Source [1], so we’ll have to see how long that holds. We're also starting to see that from Alibaba whose latest Qwen2.5-Max model is only available through their Alibaba Cloud. Luckily Facebook business model isn't reliant on cloud hosting so we should continue to expect Open models from them. So far efficiency seems to be DeepSeek's competitive advantage as despite being OSS they're still the cheapest hosting provider [2] despite other hosting providers not having to recoup any R&D and training costs. [1] https://x.com/arthurmensch/status/1884972984202338450 https://x.com/arthurmensch/status/1884972984202338450 [2] https://openrouter.ai/deepseek/deepseek-r1 https://openrouter.ai/deepseek/deepseek-r1

yieldcrv 2y ago
Its open source vs closed source, not China vs US but this new dimension of geopolitical competition is now sidelining the cautionary anti-AGI populace, which was honestly probably saving us a few years

magwa101 2y ago

[dead]

deleted 2y ago

[deleted]

ecret 2y ago

[dead]

deleted 2y ago

[deleted]

walterbell 2y ago

2023 and 2024 interviews, https://www.lesswrong.com/posts/kANyEjDDFWkhSKbcK/two-interviews-with-the-founder-of-deepseek https://www.lesswrong.com/posts/kANyEjDDFWkhSKbcK/two-interv... > Liang Wenfeng is a very rare person in China's AI industry who has abilities in “strong infrastructure engineering, model research, and also resource mobilization”, and “can make accurate high-level judgments, and can also be stronger than a frontline researcher in the technical details”. He has a “terrifying ability to learn” and at the same time is “less like a boss and more like a geek”.