5 ms·
What model? And what hardware do you run it on? I find these style of models are great, but fail hard, and fail randomly. I'd be hesitant to use it for a daily
by digdugdirk 2mo ago
What model? And what hardware do you run it on?
I find these style of models are great, but fail hard, and fail randomly. I'd be hesitant to use it for a daily driver, but I'm using dual 3060s, so it's not like I'm quantizing a frontier model here.
How do you find the overall experience? And do you have any special sauce or recommendations for going this route?
- kamranjon 2mo agoI’m using DeepSeek V4 Flash on 128gb mbp - it’s a bit different using a 200b+ param model. It’s MoE so performance is acceptable. It will still malform a tool call every now and then, but the capabilities are so far ahead anything else that the majority of the time it works really well and solves really complex problems.
- apitman 2mo agoI've got dual 3060s as well. What's the best models you've found for this setup?