5 ms·
@simonw what is the token/s like on your 64gb m2 mbp?
by joseferben 2y ago
@simonw what is the token/s like on your 64gb m2 mbp?
- simonw 2y agoWith MLX: Prompt: 49 tokens, 95.691 tokens-per-sec Generation: 723 tokens, 10.016 tokens-per-sec Peak memory: 32.685 GB
- joseferben 2y agoso quite usable, thanks!