5 ms·
Continue to believe that Cerebras is one of the most underrated companies of our time. It's a dinner-plate sized chip. It actually works. It's actually much fas
by pjs_ 7mo ago
Continue to believe that Cerebras is one of the most underrated companies of our time. It's a dinner-plate sized chip. It actually works. It's actually much faster than anything else for real workloads. Amazing
- arcanemachiner 7mo agoJust wish they weren't so insanely expensive...
- azinman2 7mo agoThe bigger the chip, the worse the yield.
- moralestapia 7mo agoThis comment doesn't make sense.
- DocJade 7mo agoBigger chip = more surface area = higher chance for somewhere in the chip to have a manufacturing defect Yields on silicon are great, but not perfect
- moralestapia 7mo agoDoes that mean smaller chips are made from smaller wafers?
- Sohcahtoa82 7mo agoNope. They use the same size wafers and then just put more chips on a wafer.
- moralestapia 7mo agoSo, does a wafer with a huge chip has more defects per area than a wafer with 100s of small chips?
- dgfl 7mo agoThere’s an expected amount of defects per wafer. If a chip has a defect, then it is lost (simplification). A wafer with 100 chips may lose 10 to defects, giving a yield of 90%. The same wafer but with 1000 smaller chips would still have lost only 10 of them, giving 99% yield.
- moralestapia 7mo agoAs another comment referenced in this thread states, Cerebras seems to have solved by making their big chip a lot of much smaller cores that can be disposed of if they have errors.
- dgfl 7mo agoIndeed, the original comment you replied to actually made no sense in this case. But there seemed to be some confusion in the thread, so I tried to clear that up. I hope I’ll get to talk with one of the cerebras engineers one day, that chip is really one of a kind.
- moralestapia 7mo agoYes, amazing tech. You should join their Discord, it's pretty active these days!
- wat10000 7mo agoThey can be made from large wafers. A defect typically breaks whatever chip it's on, so one defect on a large wafer filled with many small chips will still just break one chip of the many on the wafer. If your chips are bigger, one defect still takes out a chip, but now you've lost more of the wafer area because the chip is bigger. So you get a super-linear scaling of loss from defects as the chips get bigger. With careful design, you can tolerate some defects. A multi-core CPU might have the ability to disable a core that's affected by a defect, and then it can be sold as a different SKU with a lower core count. Cerebras uses an extreme version of this, where the wafer is divided up into about a million cores, and a routing system that can bypass defective cores. They have a nice article about it here: https://www.cerebras.ai/blog/100x-defect-tolerance-how-cerebras-solved-the-yield-problem https://www.cerebras.ai/blog/100x-defect-tolerance-how-cereb...
- azinman2 7mo agoSure it does. If it’s many small dies on a wafer, then imperfections don’t ruin the entire batch; you just bin those components. If the entire wafer is a single die, you have much less tolerance for errors.
- dekhn 7mo agoAlthough, IIUC, Cerebras expects some amount of imperfection and can adjust the hardware (or maybe the software) to avoid those components after they're detected. https://www.cerebras.ai/blog/100x-defect-tolerance-how-cerebras-solved-the-yield-problem https://www.cerebras.ai/blog/100x-defect-tolerance-how-cereb...
- pertymcpert 7mo agoYou can just do dynamic binning.
- Sohcahtoa82 7mo agoOne wafer will turn into multiple chips. Defects are best measured on a per-wafer basis, not per-chip. So if if your chips are huge and you can only put 4 chips on a wafer, 1 defect can cut your yield by 25%. If they're smaller and you fit 100 chips on a wafer, then 1 defect on the wafer is only cutting yield by 1%. Of course, there's more to this when you start reading about "binning", fusing off cores, etc. There's plenty of information out there about how CPU manufacturing works, why defects happen, and how they're handled. Suffice to say, the comment makes perfect sense.
- snovv_crash 7mo agoThat's why you typically fuse off defective sub-units and just have a slightly slower chip. GPU and CPU manufacturers have done this for at least 15 years now, that I'm aware of.
- louiereederson 7mo agoYou say this with such confidence and then ask if smaller chips require smaller wafers.
- speedgoose 7mo agoI suggest to read their website, they explain pretty well how they manage good yield. Though I’m not an expert in this field. I does make sense and I would be surprised if they were caught lying.
- thunderbird120 7mo agoCerebras has effectively 100% yield on these chips. They have an internal structure made by just repeating the same small modular units over and over again. This means they can just fuse off the broken bits without affecting overall function. It's not like it is with a CPU.
- moregrist 7mo agoI think what you’re saying is that every wafer is usable, but won’t have the same performance characteristics, depending on how many bits are broken. Doesn’t that just bucket wafers based on performance? Which effectively gives a yield to each bucket.
- zozbot234 7mo agoIt's "dinner-plate sized" because it's just a full silicon wafer. It's nice to see that wafer-scale integration is now being used for real work but it's been researched for decades.
- femiagbabiaka 7mo agoyep
- latchkey 7mo agoNot for what they are using it for. It is $1m+/chip and they can fit 1 of them in a rack. Rack space in DC's is a premium asset. The density isn't there. AI models need tons of memory (this product annoucement is case in point) and they don't have it, nor do they have a way to get it since they are last in line at the fabs. Their only chance is an aquihire, but nvidia just spent $20b on groq instead. Dead man walking.
- p1esk 7mo agoThe real question is what’s their perf/dollar vs nvidia?
- xnx 7mo agoOr Google TPUs.
- latchkey 7mo agoTPUs don't have enough memory either, but they have really great interconnects, so they can build a nice high density cluster. Compare the photos of a Cerebras deployment to a TPU deployment. https://www.nextplatform.com/wp-content/uploads/2023/07/cerebras-g42-cg1-system-logo.jpg https://www.nextplatform.com/wp-content/uploads/2023/07/cere... https://assets.bwbx.io/images/users/iqjWHBFdfxIU/iOLs2FEQxQvM/v0/-1x-1.webp https://assets.bwbx.io/images/users/iqjWHBFdfxIU/iOLs2FEQxQv... The difference is striking.
- p1esk 7mo agoOh wow the cabling in the first link is really sloppy!
- latchkey 7mo agoExactly. They won't ever tell you. It is never published. Let's not forget that the CEO is an SEC felon who got caught trying to pull a fast one.
- zozbot234 7mo ago
- xnx 7mo agoCerebras is a bit of a stunt like "datacenters in spaaaaace". Terrible yield: one defect can ruin a whole wafer instead of just a chip region. Poor perf./cost (see above). Difficult to program. Little space for RAM.
- the_duke 7mo agoThey claim the opposite, though, saying the chip is designed to tolerate many defects and work around them.
- deleted 7mo ago[deleted]
- deleted 7mo ago[deleted]
- onlyrealcuzzo 7mo agoNvidia seems cooked. Google is crushing them on inference. By TPUv9, they could be 4x more energy efficient and cheaper overall (even if Nvidia cuts their margins from 75% to 40%). Cerebras will be substantially better for agentic workflows in terms of speed. And if you don't care as much about speed and only cost and energy, Google will still crush Nvidia. And Nvidia won't be cheaper for training new models either. The vast majority of chips will be used for inference by 2028 instead of training anyway. Nvidia has no manufacturing reliability story. Anyone can buy TSMC's output. Power is the bottleneck in the US (and everywhere besides China). By TPUv9 - Google is projected to be 4x more energy efficient. It's a no-brainer who you're going with starting with TPUv8 when Google lets you run on-prem. These are GW scale data centers. You can't just build 4 large-scale nuclear power plants in a year in the US (or anywhere, even China). You can't just build 4 GW solar farms in a year in the US to power your less efficient data center. Maybe you could in China (if the economics were on your side, but they aren't). You sure as hell can't do it anywhere else (maybe India). What am I missing? I don't understand how Nvidia could've been so far ahead and just let every part of the market slip away.
- whism 7mo agoI believe they licensed smth from groq
- sailingparrot 7mo ago> let every part of the market slip away. Which part of the market has slept away, exactly ? Everything you wrote is supposition and extrapolation. Nvidia has a chokehold on the entire market. All other players still exist in the small pockets that Nvidia doesn’t have enough production capacity to serve. And their dev ecosystem is still so far ahead of anyone else. Which providers gets chosen to equip a 100k chips data center goes so far beyond the raw chip power.
- onlyrealcuzzo 7mo ago> Nvidia has a chokehold on the entire market. You're obviously not looking at expected forward orders for 2026 and 2027.
- dalemhurley 7mo agoYet investors keep backing NVIDIA.
- vimda 7mo agoAt this point Tech investment and analysis is so divorced from any kind of reality that it's more akin to lemmings on the cliff than careful analysis of fundamentals
- tiffanyh 7mo agoIf history has taught us anything, “engineered systems” (like mainframes & hyper converged infrastructure) emerge at the start of a new computing paradigm … but long-term, commodity compute wins the game.
- pjs_ 7mo agoI think that was true when you could rely on good old Moore’s law to make the heavy iron quickly obsolete but I also think those days are coming to an end
- alecco 7mo agoChips and RAM grew in capacity but latency is mostly flat and interconnect power consumption grew a lot. So I think the paradigm changed. Even with newer ones like NVlink. For 28 years Intel Xeon chips come with massive L2/L3. Nvidia is making bigger chips with last being 2 big chips interconnected. Cerebras saw the pattern and took it to the next level. And the technology is moving 3D towards stacking layers on the wafer so there is room to grow that way, too.
- mzl 7mo agoTechnically, Cerebras solution is really cool. However, I am skeptical that it will be economically useful for models that are larger in size, as the requirements on the number of racks scales with the the size of the model to fit the weights in SRAM.
- h14h 7mo agoI'm fascinated by how the economy is catching up to demand for inference. The vast majority of today's capacity comes from silicon that merely happens to be good at inference, and it's clear that there's a lot of room for innovation when you design silicon for inference from the ground up. With CapEx going crazy, I wonder where costs will stabilize and what OpEx will look like once these initial investments are paid back (or go bust). The common consensus seems to be that there will be a rug pull and frontier model inference costs will spike, but I'm not entirely convinced. I suspect it largely comes down to how much more efficient custom silicon is compared to GPUs, as well as how accurately the supply chain is able to predict future demand relative to future efficiency gains. To me, it is not at all obvious what will happen. I don't see any reason why a rug pull is any more or less likely than today's supply chain over-estimating tomorrow's capacity needs, and creating a hardware (and maybe energy) surplus in 5-10 years.