taalas is great for llama 3.x 8B models, really bad for one board serving Kimi K3, it seems like you would bottleneck at a few hundred tokens no matter what you

taalas is great for llama 3.x 8B models, really bad for one board serving Kimi K3, it seems like you would bottleneck at a few hundred tokens no matter what you do.... spreading the big model against multiple cards seems the only way to get into the 1k+ tok/sec range. Another thing taalas is doing is masking the model weights into the silicon itself, not a flashable firmware, which would increase latency....