12 ms·
Yes, but running out of RAM is impractical due to low memory bandwidth. According to the article/Samsung RAM dies inside can support way higher bandwidth than
by petu 19d ago
Yes, but running out of RAM is impractical due to low memory bandwidth.
According to the article/Samsung RAM dies inside can support way higher bandwidth than they expose, they're limited by external interface / bus width:
> Together, they can utilize the chip’s internal bandwidth across all 16 banks, which comes out to 614 GB/s. For comparison, regular DRAM accesses can hit two banks in parallel and max out at 76.8 GB/s.
And that's just for single 64-bit IC. So way faster and more power efficient.
- dannyw 19d agoYou can scale with more memory channels. Workstation/server platforms go up to 12 or 16 channels if I remember correctly. Consumer platforms have been stuck at dual channel for decades; most of it I attribute to intentional product segmentation. I'm hoping that LLMs might change eventually for an upcoming consumer platforms; going to 4 channel would be really nice.
- zerd 18d agoHow do you scale past 16 channels though? 16 channels give you around 614 GB/s, while PIM can do that per chip, so it can achieve 58TB/s.