5 ms·

I mean, I don’t know how long the NVIDIA moats can hold. With this much money at stake, others will challenge their dominance especially in a market as diverse

I mean, I don’t know how long the NVIDIA moats can hold. With this much money at stake, others will challenge their dominance especially in a market as diverse and fragmented as advanced semiconductors. That’s not to say I’m brave enough to short NVDA.

dworks 1y ago
"Your margin is my opportunity" as someone said. Certainly Google must have plans to sell its chips externally with this much up for grabs?
- mark_l_watson 1y ago
  I was also wondering if Google would try to make profit from selling TPUs, but they probably won’t because: At least for me, Google has some real cachet and deserves kudos for not losing money selling Gemini services, at least I think it is plausible that they are already profitable, or soon will be. In the US, I get the impression that everyone else is burning money to get market share, but if I am wrong I would enjoy seeing evidence to the contrary. I suspect that Microsoft might be doing OK because of selling access to their infrastructure (just like Google).
  - alephnerd 1y ago
    There's no point selling TPUs when you can bundle TPU access as part of much more profitable training services. The margins are much higher providing a service as part of GCP versus selling.
    - mark_l_watson 1y ago
      I agree. Amazon and I think Microsoft are also working on their own NVIDIA replacement chips - it will be interesting to see if any companies start selling chips, or stick with services.
      - alephnerd 1y ago
        From what I'm hearing in my network, the name of the game is custom chips hyperoptimized for your own workloads. A major reason Deepseek was so successful margins wise was because the team heavily understood Nvidia, CUDA, and Linux internals. If you have an understanding of the intricacies of your custom ASIC's architecture, it's easier for you to solve perf issues, parallelize, and debug problems. And then you can make up the cost by selling inference as a service. > Amazon and I think Microsoft are also working on their own NVIDIA replacement chips Not just them. I know of at least 4-5 other similar initiatives (some public like OpenAI's, another which is being contracted by a large nation, and a couple others which haven't been announced yet so I can't divulge). Contract ASIC and GPU design is booming, and Broadcom, Marvell, HPE, Nvidia, and others are cashing in on it.
        coredog64 1y ago
        I wouldn't be surprised if a fair portion of Amazon's Bedrock traffic is being served by Inferentia silicon. Their margins on Anthropic models are razor thin and there's a lot of traffic, so there's definitely an incentive. Additionally, every model that's served by Inferentia frees up Nvidia capacity for either models that can't be so served or for selling to customers.
  - Mistletoe 1y ago
    Do you have a link or references showing Google isn’t losing money on Gemini?
    - mark_l_watson 1y ago
      Earning report does not break out profit from Gemini separately, but this is still useful https://abc.xyz/assets/34/fa/ee06f3de4338b99acffc5c229d9f/2025q1-alphabet-earnings-release.pdf?utm_source=chatgpt.com https://abc.xyz/assets/34/fa/ee06f3de4338b99acffc5c229d9f/20... A long time ago I worked as a contractor at Google, and that experience taught me that they don’t like things that don’t scale or are inefficient.
      - brazukadev 1y ago
        That's the same as saying that Google is winning the AI race because they don't like losing. They won't win anything if we are in a bubble that burst tho
        GoatInGrey 1y ago
        A hypothetical AI bubble bursting doesn't mean that every single AI vendor fails completely. Like the Dot-Com Bubble, the market value drops precipitously and many companies fold, but because the market value does not fall to zero, the survivors (i.e. Amazon) still win.
        noduerme 1y ago
        Websites were still mostly selling goods and services in 2001. Not giving away hot takes and hallicinated summaries in exchange for eyeballs. In other words, after stuff like pets.com collapsed, people still found it useful to have pet food delivered, and the business model evolved. LLMs, on the other hand, don't seem to have a lot of public appeal. Most of the use cases are being shoved down the public's throat. Their appeal is to corporations as cost saving replacements for workers. But an AI bubble bursting would look like corporations rolling back their exuberance for the AI craze. What's already only speculatively profitable and requires enormous capex would probably become too toxic for anyone to try again for a generation.
- hiddencost 1y ago
  Fabrication is the bottle neck. They can't even meet internal demand.
- heavyset_go 1y ago
  They make more money using them themselves or renting out their time to others.
xbmcuser 1y ago
google has already started offering its TPUs to other neocloud providers
- xnx 1y ago
  I hadn't heard that. Source?
  - xbmcuser 1y ago
    https://www.datacenterdynamics.com/en/news/google-offers-its-tpus-to-ai-cloud-providers-report/ https://www.datacenterdynamics.com/en/news/google-offers-its...
    - xnx 1y ago
      Interesting. I read that as Google is using colocation to host its TPUs. I don't think Google is selling its TPUs like Nvidia sells H100s.
mrktf 1y ago
As long as only TMSC is only top performance chip producer and it is possible to reserve all it manufacturing capacity for one two clients the NVIDIA will hold without problem... My opinion, the problems for NVIDIA will start when China ramp up internal chip manufacturing performance enough to be in same order of magnitude as TMSC.
- TSiege 1y ago
  They are currently doing this. It’s part of their Made in China 2025 plan
- user34283 1y ago
  I'm not knowledgeable about this, but I wonder how important performance really is here. Wont it be enough to just solder on a large amount of high bandwidth memory and produce these cards relatively cheaply?
  - TylerE 1y ago
    Isn’t memory production relatively limited also?
  - alephnerd 1y ago
    > but I wonder how important performance really is here. Perf is important, but ime American MLEs are less likely to investigate GPU and OS internals to get maximum perf, and just throw money at the problem. > solder on a large amount of high bandwidth memory and produce these cards relatively cheaply HBM is somewhat limited in China as well. CXMT is around 3-4 years behind other HBM vendors. That said, you don't need the latest and most performant GPUs if you can tune older GPUs and parallelize training at a large scale. ----------- IMO, Model training is an embarrassingly parallel problem, and a large enough cluster leveraging 1-2 generation older architectures that is heavily tuned should be able to provide similar performance to train models. This is why I bemoan America's failures at OS internals and systems education. You have entire generations of "ML Engineers" and researchers in the US who don't know their way around CUDA or Infiniband optimization or the ins-and-outs of the Linux kernel. They're just boffins who like math and using wrappers. That said, I'd be cautious to trust a press release or secondhand report from CCTV, especially after the Kirin 9000 saga and SMIC. But arguably, it doesn't matter - even if Alibaba's system isn't comparably performant to an H20, if it can be manufactured at scale without eating Nvidia's margins, it's good enough.
mark_l_watson 1y ago
I think that NVIDIA’s moat is the US government. Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world? I am a long time fan of Dave Sacks and the All In podcast ‘besties’ but now that he is ‘AI czar’ for our government it is interesting what he does not talk about. For example on a recent podcast he was pumping up AI as a long term solution to US economic woes, but a week before that podcast, a well known study was released that showed that 95% of new LLM/AI corporate projects were fails. Another thing that he swept under the rug was the recent Stanford study that 80% of US startups are saving money using less expensive Chinese (and Mistral, and Google Gemma??) models. When the Stanford study was released, I watched All In material for a few weeks, expecting David Sack’s take on the study. Not a word from him. Apologies for this off-topic rant but I am really concerned how my country is spending resources on AI infrastructure. I think this is a massive bubble, but I am not sure how catastrophic the bubble will be.
- heavyset_go 1y ago
  > Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world? The US is burning good will at an alarming rate, how long will countries keep paying a premium to be spied on by the US instead of China?
  - mark_l_watson 1y ago
    I think the answer to your question is ‘not for very long.’ I frequently have breakfast with a friend who is a retired math professor and he is an avid investor in the stock market. We talk a lot about how long the US stock market will keep increasing in value. We don’t know the answer about the stock market, but it is fun to talk about. We both want to start easing out of the stock market.
  - rsynnott 1y ago
    The main competitors to Huawei in cell network stuff are mostly European (Nokia and friends), not American.
- anonymousDan 1y ago
StopDisinfo910 1y ago
> That’s not to say I’m brave enough to short NVDA. Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky.
- re-thc 1y ago
  > Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky. They've been trying really hard to pivot and find new growth areas. They've taken their "inflated" stock price as capital to invest in many other companies. If at least some of these bets pay off it's not so bad.

dworks 1y ago

"Your margin is my opportunity" as someone said. Certainly Google must have plans to sell its chips externally with this much up for grabs?

mark_l_watson 1y ago
I was also wondering if Google would try to make profit from selling TPUs, but they probably won’t because: At least for me, Google has some real cachet and deserves kudos for not losing money selling Gemini services, at least I think it is plausible that they are already profitable, or soon will be. In the US, I get the impression that everyone else is burning money to get market share, but if I am wrong I would enjoy seeing evidence to the contrary. I suspect that Microsoft might be doing OK because of selling access to their infrastructure (just like Google).
- alephnerd 1y ago
  There's no point selling TPUs when you can bundle TPU access as part of much more profitable training services. The margins are much higher providing a service as part of GCP versus selling.
  - mark_l_watson 1y ago
    I agree. Amazon and I think Microsoft are also working on their own NVIDIA replacement chips - it will be interesting to see if any companies start selling chips, or stick with services.
    - alephnerd 1y ago
      From what I'm hearing in my network, the name of the game is custom chips hyperoptimized for your own workloads. A major reason Deepseek was so successful margins wise was because the team heavily understood Nvidia, CUDA, and Linux internals. If you have an understanding of the intricacies of your custom ASIC's architecture, it's easier for you to solve perf issues, parallelize, and debug problems. And then you can make up the cost by selling inference as a service. > Amazon and I think Microsoft are also working on their own NVIDIA replacement chips Not just them. I know of at least 4-5 other similar initiatives (some public like OpenAI's, another which is being contracted by a large nation, and a couple others which haven't been announced yet so I can't divulge). Contract ASIC and GPU design is booming, and Broadcom, Marvell, HPE, Nvidia, and others are cashing in on it.
      - coredog64 1y ago
        I wouldn't be surprised if a fair portion of Amazon's Bedrock traffic is being served by Inferentia silicon. Their margins on Anthropic models are razor thin and there's a lot of traffic, so there's definitely an incentive. Additionally, every model that's served by Inferentia frees up Nvidia capacity for either models that can't be so served or for selling to customers.
- Mistletoe 1y ago
  Do you have a link or references showing Google isn’t losing money on Gemini?
  - mark_l_watson 1y ago
    Earning report does not break out profit from Gemini separately, but this is still useful https://abc.xyz/assets/34/fa/ee06f3de4338b99acffc5c229d9f/2025q1-alphabet-earnings-release.pdf?utm_source=chatgpt.com https://abc.xyz/assets/34/fa/ee06f3de4338b99acffc5c229d9f/20... A long time ago I worked as a contractor at Google, and that experience taught me that they don’t like things that don’t scale or are inefficient.
    - brazukadev 1y ago
      That's the same as saying that Google is winning the AI race because they don't like losing. They won't win anything if we are in a bubble that burst tho
      - GoatInGrey 1y ago
        A hypothetical AI bubble bursting doesn't mean that every single AI vendor fails completely. Like the Dot-Com Bubble, the market value drops precipitously and many companies fold, but because the market value does not fall to zero, the survivors (i.e. Amazon) still win.
        noduerme 1y ago
        Websites were still mostly selling goods and services in 2001. Not giving away hot takes and hallicinated summaries in exchange for eyeballs. In other words, after stuff like pets.com collapsed, people still found it useful to have pet food delivered, and the business model evolved. LLMs, on the other hand, don't seem to have a lot of public appeal. Most of the use cases are being shoved down the public's throat. Their appeal is to corporations as cost saving replacements for workers. But an AI bubble bursting would look like corporations rolling back their exuberance for the AI craze. What's already only speculatively profitable and requires enormous capex would probably become too toxic for anyone to try again for a generation.
hiddencost 1y ago
Fabrication is the bottle neck. They can't even meet internal demand.
heavyset_go 1y ago
They make more money using them themselves or renting out their time to others.

xbmcuser 1y ago

google has already started offering its TPUs to other neocloud providers

xnx 1y ago
I hadn't heard that. Source?
- xbmcuser 1y ago
  https://www.datacenterdynamics.com/en/news/google-offers-its-tpus-to-ai-cloud-providers-report/ https://www.datacenterdynamics.com/en/news/google-offers-its...
  - xnx 1y ago
    Interesting. I read that as Google is using colocation to host its TPUs. I don't think Google is selling its TPUs like Nvidia sells H100s.

mrktf 1y ago

As long as only TMSC is only top performance chip producer and it is possible to reserve all it manufacturing capacity for one two clients the NVIDIA will hold without problem... My opinion, the problems for NVIDIA will start when China ramp up internal chip manufacturing performance enough to be in same order of magnitude as TMSC.

TSiege 1y ago
They are currently doing this. It’s part of their Made in China 2025 plan
user34283 1y ago
I'm not knowledgeable about this, but I wonder how important performance really is here. Wont it be enough to just solder on a large amount of high bandwidth memory and produce these cards relatively cheaply?
- TylerE 1y ago
  Isn’t memory production relatively limited also?
- alephnerd 1y ago
  > but I wonder how important performance really is here. Perf is important, but ime American MLEs are less likely to investigate GPU and OS internals to get maximum perf, and just throw money at the problem. > solder on a large amount of high bandwidth memory and produce these cards relatively cheaply HBM is somewhat limited in China as well. CXMT is around 3-4 years behind other HBM vendors. That said, you don't need the latest and most performant GPUs if you can tune older GPUs and parallelize training at a large scale. ----------- IMO, Model training is an embarrassingly parallel problem, and a large enough cluster leveraging 1-2 generation older architectures that is heavily tuned should be able to provide similar performance to train models. This is why I bemoan America's failures at OS internals and systems education. You have entire generations of "ML Engineers" and researchers in the US who don't know their way around CUDA or Infiniband optimization or the ins-and-outs of the Linux kernel. They're just boffins who like math and using wrappers. That said, I'd be cautious to trust a press release or secondhand report from CCTV, especially after the Kirin 9000 saga and SMIC. But arguably, it doesn't matter - even if Alibaba's system isn't comparably performant to an H20, if it can be manufactured at scale without eating Nvidia's margins, it's good enough.

mark_l_watson 1y ago

I think that NVIDIA’s moat is the US government. Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world? I am a long time fan of Dave Sacks and the All In podcast ‘besties’ but now that he is ‘AI czar’ for our government it is interesting what he does not talk about. For example on a recent podcast he was pumping up AI as a long term solution to US economic woes, but a week before that podcast, a well known study was released that showed that 95% of new LLM/AI corporate projects were fails. Another thing that he swept under the rug was the recent Stanford study that 80% of US startups are saving money using less expensive Chinese (and Mistral, and Google Gemma??) models. When the Stanford study was released, I watched All In material for a few weeks, expecting David Sack’s take on the study. Not a word from him. Apologies for this off-topic rant but I am really concerned how my country is spending resources on AI infrastructure. I think this is a massive bubble, but I am not sure how catastrophic the bubble will be.

heavyset_go 1y ago
> Remember our government’s efforts to prevent the use of Huawei cell infrastructure in Europe and around the world? The US is burning good will at an alarming rate, how long will countries keep paying a premium to be spied on by the US instead of China?
- mark_l_watson 1y ago
  I think the answer to your question is ‘not for very long.’ I frequently have breakfast with a friend who is a retired math professor and he is an avid investor in the stock market. We talk a lot about how long the US stock market will keep increasing in value. We don’t know the answer about the stock market, but it is fun to talk about. We both want to start easing out of the stock market.
- rsynnott 1y ago
  The main competitors to Huawei in cell network stuff are mostly European (Nokia and friends), not American.
anonymousDan 1y ago

StopDisinfo910 1y ago

> That’s not to say I’m brave enough to short NVDA. Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky.

re-thc 1y ago
> Their multiples don't seem sustainable so they are likely to fall at some point but when is tricky. They've been trying really hard to pivot and find new growth areas. They've taken their "inflated" stock price as capital to invest in many other companies. If at least some of these bets pay off it's not so bad.