6 ms·
What LLM-specific hardware improvements should one expect? Seems to me that LLM inference is simple architecturally (matmul et al) so most scaling in hardware s
by brausepulver 1mo ago
What LLM-specific hardware improvements should one expect? Seems to me that LLM inference is simple architecturally (matmul et al) so most scaling in hardware should come from general improvements (memory BW, packaging, interconnect, power).
- dorkypunk 1mo agoWhat you describe is basically Cerebras case, at the bottom it's just a really big die (about x28 an NVIDIA GB200) with a lot of work to reduce memory latency and improve throughput. What it's actually amazing is how can they make a chip so big and still have a decent yield to be commercially viable.