Actually they wouldnt spend the money if it were cheaper. HBM has way higher bandwidth and its not all about flops. Also the FP4 flops (inference) are so mind

Actually they wouldnt spend the money if it were cheaper. HBM has way higher bandwidth and its not all about flops. Also the FP4 flops (inference) are so mind bogglingly high on these things. Lastly what you fail to consider is the chip to chip bandwidth which is critical. the people running these know that networking is just as critical. all reduce etc. they wouldnt pay if they could get something better value.