6 ms·
Cerebras have special techniques to work around etching errors / bad cores on their wafers. This is possible since their wafers are effectively hundreds of iden
by briansm 1mo ago
Cerebras have special techniques to work around etching errors / bad cores on their wafers. This is possible since their wafers are effectively hundreds of identical copies of redundant cores. Can't do that for a globally unique model.
Etching failure in that situation would be like brain-damage in a human, all sorts of weird effects would start appearing.
- fulafel 1mo agoThere's several ways to engineer around that as the errors are detectable. There's a big literature on how to trade off speed or transistors for error correction. [1] (Is Cerebras doing something novel? CPUs and memory blocks have been doing those things for a long time too, since the error rate is otherwise too high for normal size chips as well) [1] see eg https://www.vlsimentor.com/dft/redundancy-bisr https://www.vlsimentor.com/dft/redundancy-bisr to get some basic concepts
- phonon 1mo agoA few hundred bad bits/transistors in a trillion+ parameter model would compromise its abilities not one iota...the models are inherently lossy and resistant to "brain damage"...