4 ms·
Even at 30 t/s, 3.8 thinks so long, even on medium, it still takes 3x or more longer than any cloud model, in my testing.
by SwellJoe 8d ago
Even at 30 t/s, 3.8 thinks so long, even on medium, it still takes 3x or more longer than any cloud model, in my testing.
- lowbloodsugar 8d agoI've got an M1 Max 64GB too. It's just not an LLM-class workstation. Give it a year and buy an M7 and you'll be laughing. Right now is a really bad time to invest in anything - using the cloud is the cheapest option, especially for open weight models.