5 ms·
My experience with Mac Studio is that memory bandwidth matters more than raw cores for reasonable LLM throughput locally; curious what others find for models >1
by b_brief 7mo ago
My experience with Mac Studio is that memory bandwidth matters more than raw cores for reasonable LLM throughput locally; curious what others find for models >13B parameters?