5 ms·
Understanding LLM Compute Requirements
- pgspaintbrush 3y agoA guide for the non-technical
- brucethemoose2 3y agoInference is not that flop intense for modern IGPs... the bigger issue is the sheer memory requirements. Training requires a ridiculous amount of both, but again the raw compute requirements would be much lower with huge memory pools.