6 ms·
- Interconnect between WSE-2's chips in the cluster was 150GB/s, much lower than NVIDIA's 900GB/s. - non-sparse fp16 in WSE-2 was 7.5 tflops (about 8 H100s, 10
by asdfasdf1 3y ago
- Interconnect between WSE-2's chips in the cluster was 150GB/s, much lower than NVIDIA's 900GB/s.
- non-sparse fp16 in WSE-2 was 7.5 tflops (about 8 H100s, 10x worse performance per dollar)
Does anyone know the WSE-3 numbers? Datasheet seems lacking loads of details
Also, 2.5 million USD for 1 x WSE-3, why just 44GB tho???
- Tuna-Fish 3y ago44GB is the SRAM on a single device, comparable to the 50MB of L2 on the H100. There is also a lot of directly attached DRAM.
- terafo 3y agoNo, it's comparable to 230Mb of SRAM on Groq chip, since both of them are SRAM-only chips that can't really use external memory.
- bee_rider 3y agoIs that 150GB/s between elements that expect to run tightly coupled processes together? Maybe the bandwidth between chips is less important. I mean, in a cluster you might have a bunch of nodes with 8x GPUs hanging off each, if this thing replaces a whole node rather than a single GPU, which I assume is the case, it is not really a useful comparison, right?
- xcv123 3y ago>> why just 44GB tho??? You can order one with 1.2 Petabytes of external memory. Is that enough? "External memory: 1.5TB, 12TB, or 1.2PB" https://www.cerebras.net/press-release/cerebras-announces-third-generation-wafer-scale-engine https://www.cerebras.net/press-release/cerebras-announces-th... "214Pb/s Interconnect Bandwidth" https://www.cerebras.net/product-system/ https://www.cerebras.net/product-system/
- acchow 3y agoI can't find the memory bandwidth to that external memory. Did they publish this?
- deleted 3y ago[deleted]
- terafo 3y agoBecause SRAM stopped getting smaller with recent nodes.