6 ms·
Check out this guide for some recommendations: https://www.hardware-corner.net/guides/computer-to-run-llama-ai-model/ https://www.hardware-corner.net/guides/com
by hellsten 3y ago
Check out this guide for some recommendations:
https://www.hardware-corner.net/guides/computer-to-run-llama-ai-model/ https://www.hardware-corner.net/guides/computer-to-run-llama...
You can easily experiment with smaller models, for example, Mistral 7B or Phi-2 on M1/M2/M3 processors. With more memory, you can run larger models, and better memory bandwidth (M2 Ultra vs. M2 base model) means improved performance (tokens/second).