7 ms·

If you're NVIDIA then open-weight models are a classic example of commoditizing your complements; cheaper models mean more people buying GPU's to run them. [1]

by seanc 2mo ago

If you're NVIDIA then open-weight models are a classic example of commoditizing your complements; cheaper models mean more people buying GPU's to run them. [1] Your guess is as good as mine for China though. [1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/ https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

elictronic 2mo ago
Nvidia selling more GPU's at the cost of it's datacenter business is pretty close to Kodak selling digital cameras at the cost of film. The data center side is so bloated anything that eats into it is a huge negative. Their data center business brings in 20x the gpu market. Local open weight models will be what pops the bubble and China will do anything in it's power to enable that pop.
- bee_rider 2mo ago
  I wonder what the thinking inside NVIDIA is at the moment. They have countless examples to learn from here, about the danger of not being willing to cannibalize your high end products. But, of course, there’s a reason that there are lots of examples of this sort of failure. There’s plenty of competition that would be happy to attack them from below, though…
  - fridder 2mo ago
    I also wonder what this moment will mean for chip design going forward. The fact that we are so constrained on the GPU side along with the success of unified memory with apple silicon does make me wonder what chips in 3-5 years will look like.
    - Danox 2mo ago
      Those that can will incorporate memory into their designs, and the Chinese will be a bigger part of the memory market worldwide going into the future, that alone will drive more tech companies to look at memory/ssd design differently than what they did before. Market conditions have changed you either change/adapt with it or you die. I think memory and ssd design will end up being incorporated into the overall design of the SOC chip. And several companies that can probably will sponsor a existing company or build a foundry going forward. Once again, change or die.
    - bigyabai 2mo ago
      Unified Memory simply isn't viable at the scale that a full Nvidia cluster operates at, and it would hamstring the parallelism that gives these clusters an edge. The current solution of Mellanox-style interconnect is probably still going to be the status quo in 5 or 10 years.
  - SecretDreams 2mo ago
    Make money while there's money to be made then pivot to the next scheme? Gaming > Bitcoin > LLMs > Robotics Jensen's job is to just be one step ahead of the market dynamics to keep the investor dollars flowing.
    - rad-b 2mo ago
      That might seem like obvious pivots in hindsight , but there’s no way to make such easy predictions going forward. “Jensen’s job is to just be one step ahead of the market” — how is that a “just”? That’s arguably one of the most difficult things to do reliably (judging by how much payout it can give you). Feels like saying he’s making “just” billions of dollars.
      - SecretDreams 2mo ago
        He's effectively setting the pace of the trends. Objectively, Bitcoin was a grift and money laundering scheme. LLM valuations vs what they've made so far is probably also quite a bit more towards the grift side (yes, I use them and yes they do good work in good hands). Robotics will probably be valued similarly, despite not actually having anywhere near the needed ROIs to make sense. The only time where revenue was actually grounded behind a sensible industry was gaming. But it doesn't matter because Jensen sells the shovels and, in the background, tells people where to dig. Dude's basically a genius at making money.
        bee_rider 2mo ago
        > But it doesn't matter because Jensen sells the shovels and, in the background, tells people where to dig. Dude's basically a genius at making money. Being the default high-throughput device is an incredibly strong position. We could say he’s a genius at making money, but really that’s downstream of the ability to keep CUDA relevant.
- wiremine 2mo ago
  > Nvidia selling more GPU's at the cost of it's datacenter business is pretty close to Kodak selling digital cameras at the cost of film. This assumes a) AI is a zero-sum game, and b) we're actually talking about on-prem AI will replace cloud-based AI. I think neither statements are true. AI is like compute: we'll need all sorts of it, in various sizes, everywhere. I'm sure Nvidia whats to own all the workloads. On the other hand, I do think open weight, like open source, will win in general.
  - elictronic 2mo ago
    It will take about 35 years for digital camera sales to reach Kodaks profit margins. The market growing and companies raising the economy are not zero-sum. Nvidia/Kodak killing the golden goose is zero-sum. When the difference is your valuation crashes that's zero sum for the company, just not for everyone else. My only concern is if we can limit the economic impact from lowering investments and causing a 40% market collapse circa 2008/9.
  - Danox 2mo ago
    But they won’t with the Chinese around long-term and other tech companies that have capabilities. This is a short term bottleneck.
- ak_111 2mo ago
  Hmm this sounds like an incumbent missing a paradigm shift because they didn't want it to disrupt their core (usually enterprise) business, although riding the shift would have ultimately delivered an order magnitude larger business. Classical example is Microsoft actively undermining mobile because it threatened selling Windows or enterprise licenses. Or Yahoo fighting Google's model because the latter model's didn't depend on taking enterprise deals to rank results.
  - elictronic 2mo ago
    Thanks generic LLM model response. Please upgrade to a higher tier model if you are going to waste our time.
    - ak_111 2mo ago
      lol honest this wasn’t AI, I guess I need to work on my style
- TeMPOraL 2mo ago
  > Nvidia selling more GPU's at the cost of it's datacenter business is pretty close to Kodak selling digital cameras at the cost of film. You mean they resisted the idea trying to protect their legacy business, and it ended up all but killing them?
  - elictronic 2mo ago
    They resisted the idea because it did kill their legacy business. Kodaks peak was 16 billion in revenue in 1996. The digital camera market will be approaching that level of value inflation adjusted around 2030. 35 years to get back to status quo. Putting a bullet in a golden goose is often considered a bad idea. Everyone is happy they are dead, but if you can't understand why they might try to keep the corpse alive you have never looked at the numbers.
    - davkan 2mo ago
      Kodak was definitely mismanaged in many ways but you are correct in your analysis. If they had pivoted hard to digital they could have been what today? Nikon? Fujifilm? These are known brands that are bigger than Kodak is today but they are nothing compared to what Kodak was. At its peak Kodak was one of the (the?) most well known brands on the entire planet. Anywhere in the world if someone snapped a photo Kodak was more than likely taking a cut, coming and going. They employed as many people in Rochester alone as any digital camera company does today internationally. Sure it could have been managed better but they were always doomed for a fall. They were a chemical company entering a digital age. Does anybody even care who makes the sensors in iPhones?
      - BoorishBears 2mo ago
        > If they had pivoted hard to digital they could have been what today? Having the first portable-ish digital camera they could have seen the true value of Fairchild's CCD business, got a stake/bought it/replicated done whatever it took to push the frontier of digital imaging and became the Kodak (old, film-era Kodak) of electronic imaging? There's no comparable business today because all the things they could have invested in ended up being taken up by different companies, they had the R&D culture, revenue, and distribution. I don't see why they couldn't have been category defining.
- maxerickson 2mo ago
  Their data center business is selling gpus. They wrap them in a complete platform, but that's what they are selling. They do report it separately from consumer and business sales of gpus used in PCs.
  - elictronic 2mo ago
    The data center business is selling Tensor core GPU's which are used for Large language models nearly exclusively. I don't think of them as GPU's anymore considering a GPU is a graphics processing unit and this is not the primary function of those cards anymore. A GPU will always be a card you put in your computer to play a game in my mind. I know a tomato is a fruit but will still be annoyed when someone is pedantic about it because that's dumb. When the mobile phone is eventually integrated into the human body or some other silly application in the future that will annoy me as well. Congrats on being pedantic.
    - maxerickson 2mo ago
      Talking about their product segments the same way they do is not being pedantic. Insisting on your personal definition is!
      - elictronic 2mo ago
        You sound like someone who believes online betting markets aren’t gambling because the company says so. Cryptocurrency is commonly used for personal transactions. You might not ever admit your argument is dumb but tomatoes are fruit is stupid.
        maxerickson 2mo ago
        I'm perfectly happy to acknowledge that tomatoes are a vegetable in the kitchen. If you want to understand their reproductive biology, you are stuck understanding them as fruit. It's maybe telling that you bring up a confusion that is harmful instead of explaining why grouping massively parallel processors under the term "gpu" is harmful.
- taftster 2mo ago
  Could you put that in units of HP Printers and Toner please?
- spunker540 2mo ago
  Could you explain what you mean by this? I thought all of their insane profitability and returns are from crazy margins on their GPUs. I know they started/partnered/invested in some data center businesses, but I thought they were fledgling
- russli1993 2mo ago
  You need 64 H200 super-node for inference for kimi k3. You will not do inference locally. What might pop the western hardware bubble is Chinese GPU, memory, networking companies. But even in China, these AI centric hardware is not cheap.
  - overfeed 2mo ago
    > But even in China, these AI centric hardware is not cheap Definitely not cheap for individuals, but well within SME territory. There are countless small-town, family-owned businesses that had higher startup costs than a hypothetical Kimi-R-Us, Inc.
lyjackal 2mo ago
There’s export controls on nvidia. According to Jensen, they expect that Chinese models will start being optimized for Huawei https://www.dwarkesh.com/p/jensen-huang https://www.dwarkesh.com/p/jensen-huang
- rjzzleep 2mo ago
  They already are.
russli1993 2mo ago
No, Nvidia is worried about Chinese hardware stack. China is under sanction, what are they training and inferencing these models on? Even if this model might still be Nvidia chips, what about the next model. Everyone knows model and hardware companies are working together. There are a number of very competitive companies in China in this space. After they got this area sorted out, China will do training, inference and tokens entirely on their stack and export their entire stack.

by seanc 2mo ago

elictronic 2mo ago

Nvidia selling more GPU's at the cost of it's datacenter business is pretty close to Kodak selling digital cameras at the cost of film. The data center side is so bloated anything that eats into it is a huge negative. Their data center business brings in 20x the gpu market. Local open weight models will be what pops the bubble and China will do anything in it's power to enable that pop.

bee_rider 2mo ago
I wonder what the thinking inside NVIDIA is at the moment. They have countless examples to learn from here, about the danger of not being willing to cannibalize your high end products. But, of course, there’s a reason that there are lots of examples of this sort of failure. There’s plenty of competition that would be happy to attack them from below, though…
- fridder 2mo ago
  I also wonder what this moment will mean for chip design going forward. The fact that we are so constrained on the GPU side along with the success of unified memory with apple silicon does make me wonder what chips in 3-5 years will look like.
  - Danox 2mo ago
    Those that can will incorporate memory into their designs, and the Chinese will be a bigger part of the memory market worldwide going into the future, that alone will drive more tech companies to look at memory/ssd design differently than what they did before. Market conditions have changed you either change/adapt with it or you die. I think memory and ssd design will end up being incorporated into the overall design of the SOC chip. And several companies that can probably will sponsor a existing company or build a foundry going forward. Once again, change or die.
  - bigyabai 2mo ago
    Unified Memory simply isn't viable at the scale that a full Nvidia cluster operates at, and it would hamstring the parallelism that gives these clusters an edge. The current solution of Mellanox-style interconnect is probably still going to be the status quo in 5 or 10 years.
- SecretDreams 2mo ago
  Make money while there's money to be made then pivot to the next scheme? Gaming > Bitcoin > LLMs > Robotics Jensen's job is to just be one step ahead of the market dynamics to keep the investor dollars flowing.
  - rad-b 2mo ago
    That might seem like obvious pivots in hindsight , but there’s no way to make such easy predictions going forward. “Jensen’s job is to just be one step ahead of the market” — how is that a “just”? That’s arguably one of the most difficult things to do reliably (judging by how much payout it can give you). Feels like saying he’s making “just” billions of dollars.
    - SecretDreams 2mo ago
      He's effectively setting the pace of the trends. Objectively, Bitcoin was a grift and money laundering scheme. LLM valuations vs what they've made so far is probably also quite a bit more towards the grift side (yes, I use them and yes they do good work in good hands). Robotics will probably be valued similarly, despite not actually having anywhere near the needed ROIs to make sense. The only time where revenue was actually grounded behind a sensible industry was gaming. But it doesn't matter because Jensen sells the shovels and, in the background, tells people where to dig. Dude's basically a genius at making money.
      - bee_rider 2mo ago
        > But it doesn't matter because Jensen sells the shovels and, in the background, tells people where to dig. Dude's basically a genius at making money. Being the default high-throughput device is an incredibly strong position. We could say he’s a genius at making money, but really that’s downstream of the ability to keep CUDA relevant.
wiremine 2mo ago
> Nvidia selling more GPU's at the cost of it's datacenter business is pretty close to Kodak selling digital cameras at the cost of film. This assumes a) AI is a zero-sum game, and b) we're actually talking about on-prem AI will replace cloud-based AI. I think neither statements are true. AI is like compute: we'll need all sorts of it, in various sizes, everywhere. I'm sure Nvidia whats to own all the workloads. On the other hand, I do think open weight, like open source, will win in general.
- elictronic 2mo ago
  It will take about 35 years for digital camera sales to reach Kodaks profit margins. The market growing and companies raising the economy are not zero-sum. Nvidia/Kodak killing the golden goose is zero-sum. When the difference is your valuation crashes that's zero sum for the company, just not for everyone else. My only concern is if we can limit the economic impact from lowering investments and causing a 40% market collapse circa 2008/9.
- Danox 2mo ago
  But they won’t with the Chinese around long-term and other tech companies that have capabilities. This is a short term bottleneck.
ak_111 2mo ago
Hmm this sounds like an incumbent missing a paradigm shift because they didn't want it to disrupt their core (usually enterprise) business, although riding the shift would have ultimately delivered an order magnitude larger business. Classical example is Microsoft actively undermining mobile because it threatened selling Windows or enterprise licenses. Or Yahoo fighting Google's model because the latter model's didn't depend on taking enterprise deals to rank results.
- elictronic 2mo ago
  Thanks generic LLM model response. Please upgrade to a higher tier model if you are going to waste our time.
  - ak_111 2mo ago
    lol honest this wasn’t AI, I guess I need to work on my style
TeMPOraL 2mo ago
> Nvidia selling more GPU's at the cost of it's datacenter business is pretty close to Kodak selling digital cameras at the cost of film. You mean they resisted the idea trying to protect their legacy business, and it ended up all but killing them?
- elictronic 2mo ago
  They resisted the idea because it did kill their legacy business. Kodaks peak was 16 billion in revenue in 1996. The digital camera market will be approaching that level of value inflation adjusted around 2030. 35 years to get back to status quo. Putting a bullet in a golden goose is often considered a bad idea. Everyone is happy they are dead, but if you can't understand why they might try to keep the corpse alive you have never looked at the numbers.
  - davkan 2mo ago
    Kodak was definitely mismanaged in many ways but you are correct in your analysis. If they had pivoted hard to digital they could have been what today? Nikon? Fujifilm? These are known brands that are bigger than Kodak is today but they are nothing compared to what Kodak was. At its peak Kodak was one of the (the?) most well known brands on the entire planet. Anywhere in the world if someone snapped a photo Kodak was more than likely taking a cut, coming and going. They employed as many people in Rochester alone as any digital camera company does today internationally. Sure it could have been managed better but they were always doomed for a fall. They were a chemical company entering a digital age. Does anybody even care who makes the sensors in iPhones?
    - BoorishBears 2mo ago
      > If they had pivoted hard to digital they could have been what today? Having the first portable-ish digital camera they could have seen the true value of Fairchild's CCD business, got a stake/bought it/replicated done whatever it took to push the frontier of digital imaging and became the Kodak (old, film-era Kodak) of electronic imaging? There's no comparable business today because all the things they could have invested in ended up being taken up by different companies, they had the R&D culture, revenue, and distribution. I don't see why they couldn't have been category defining.
maxerickson 2mo ago
Their data center business is selling gpus. They wrap them in a complete platform, but that's what they are selling. They do report it separately from consumer and business sales of gpus used in PCs.
- elictronic 2mo ago
  The data center business is selling Tensor core GPU's which are used for Large language models nearly exclusively. I don't think of them as GPU's anymore considering a GPU is a graphics processing unit and this is not the primary function of those cards anymore. A GPU will always be a card you put in your computer to play a game in my mind. I know a tomato is a fruit but will still be annoyed when someone is pedantic about it because that's dumb. When the mobile phone is eventually integrated into the human body or some other silly application in the future that will annoy me as well. Congrats on being pedantic.
  - maxerickson 2mo ago
    Talking about their product segments the same way they do is not being pedantic. Insisting on your personal definition is!
    - elictronic 2mo ago
      You sound like someone who believes online betting markets aren’t gambling because the company says so. Cryptocurrency is commonly used for personal transactions. You might not ever admit your argument is dumb but tomatoes are fruit is stupid.
      - maxerickson 2mo ago
        I'm perfectly happy to acknowledge that tomatoes are a vegetable in the kitchen. If you want to understand their reproductive biology, you are stuck understanding them as fruit. It's maybe telling that you bring up a confusion that is harmful instead of explaining why grouping massively parallel processors under the term "gpu" is harmful.
taftster 2mo ago
Could you put that in units of HP Printers and Toner please?
spunker540 2mo ago
Could you explain what you mean by this? I thought all of their insane profitability and returns are from crazy margins on their GPUs. I know they started/partnered/invested in some data center businesses, but I thought they were fledgling
russli1993 2mo ago
You need 64 H200 super-node for inference for kimi k3. You will not do inference locally. What might pop the western hardware bubble is Chinese GPU, memory, networking companies. But even in China, these AI centric hardware is not cheap.
- overfeed 2mo ago
  > But even in China, these AI centric hardware is not cheap Definitely not cheap for individuals, but well within SME territory. There are countless small-town, family-owned businesses that had higher startup costs than a hypothetical Kimi-R-Us, Inc.

lyjackal 2mo ago

There’s export controls on nvidia. According to Jensen, they expect that Chinese models will start being optimized for Huawei https://www.dwarkesh.com/p/jensen-huang https://www.dwarkesh.com/p/jensen-huang

rjzzleep 2mo ago
They already are.

russli1993 2mo ago

No, Nvidia is worried about Chinese hardware stack. China is under sanction, what are they training and inferencing these models on? Even if this model might still be Nvidia chips, what about the next model. Everyone knows model and hardware companies are working together. There are a number of very competitive companies in China in this space. After they got this area sorted out, China will do training, inference and tokens entirely on their stack and export their entire stack.