4 ms·
Thanks for posting this. I did a lot of research yesterday into the model card and the implications of the architecture changes they made and I was hoping to fi
by rpdillon 4d ago
Thanks for posting this. I did a lot of research yesterday into the model card and the implications of the architecture changes they made and I was hoping to find discussion about that here. I haven't seen any besides your comment.
V4.1 Flash seems very clearly to be a model optimized for agentic tool calling at the expense of both context window and knowledge. It uses a variety of tricks to absolutely minimize the size of the KV cache and due to the use of only 8 billion active parameters for pre-fill is definitely optimized to ingest lots of tokens and produce a moderate number of them, which aligns well with the agentic use case.
I think the core insight is that they wanted something that was cheap to host and could respond quickly, and so by increasing the total parameter count and decreasing the active parameter count, they wanted the capability but didn't want to pay for it in FLOPS. It's a super clever architecture and I like the direction they're going, but I feel like they were playing around a little bit by versioning it as version 4.1. It seems like a dramatically different beast than Deepseek V4 Flash.