4 ms·
I think the key AI improvement is that these chips double the memory bandwidth and this has been the main limit for AI performance recently, became LLMs do not
by bhouston 6d ago
I think the key AI improvement is that these chips double the memory bandwidth and this has been the main limit for AI performance recently, became LLMs do not fit into cache.
- aurareturn 6d agoThey should use LPDDR6 which is a sizable improvement but the other thing they can do is increase prompt processing even further.