6 ms·

Jesus, this is a whole nother beast, and a different architecture from their previous flash. Lots of goodies here. > Causal Encoder-Decoder (CED) architecture:

Jesus, this is a whole nother beast, and a different architecture from their previous flash. Lots of goodies here. > Causal Encoder-Decoder (CED) architecture: a 40-layer Transformer organized as a 20-layer causal encoder followed by a 20-layer decoder. With CED, the decoder's global KV cache is projected from the final encoder hidden states rather than derived from each decoder layer's own hidden states. This allows the model to activate only 8B parameters per token during prefill and 16B during decode, substantially improving cost efficiency for input-heavy agentic workloads. > these designs reduce the global KV cache footprint to 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash. Faster prefill, lower kv cache (~1GB / 1m context is insane). > The model supports a continuously controllable reasoning effort setting (integer 1–100) that trades inference cost for accuracy. Benchmarks are benchmarks, to be seen if they translate to real-world use, but they seem to have focused a lot on post-training with "agentic" scores looking good. "world knowledge" is obviously lower than higher param models.

rpdillon 4d ago
Thanks for posting this. I did a lot of research yesterday into the model card and the implications of the architecture changes they made and I was hoping to find discussion about that here. I haven't seen any besides your comment. V4.1 Flash seems very clearly to be a model optimized for agentic tool calling at the expense of both context window and knowledge. It uses a variety of tricks to absolutely minimize the size of the KV cache and due to the use of only 8 billion active parameters for pre-fill is definitely optimized to ingest lots of tokens and produce a moderate number of them, which aligns well with the agentic use case. I think the core insight is that they wanted something that was cheap to host and could respond quickly, and so by increasing the total parameter count and decreasing the active parameter count, they wanted the capability but didn't want to pay for it in FLOPS. It's a super clever architecture and I like the direction they're going, but I feel like they were playing around a little bit by versioning it as version 4.1. It seems like a dramatically different beast than Deepseek V4 Flash.

rpdillon 4d ago

Thanks for posting this. I did a lot of research yesterday into the model card and the implications of the architecture changes they made and I was hoping to find discussion about that here. I haven't seen any besides your comment. V4.1 Flash seems very clearly to be a model optimized for agentic tool calling at the expense of both context window and knowledge. It uses a variety of tricks to absolutely minimize the size of the KV cache and due to the use of only 8 billion active parameters for pre-fill is definitely optimized to ingest lots of tokens and produce a moderate number of them, which aligns well with the agentic use case. I think the core insight is that they wanted something that was cheap to host and could respond quickly, and so by increasing the total parameter count and decreasing the active parameter count, they wanted the capability but didn't want to pay for it in FLOPS. It's a super clever architecture and I like the direction they're going, but I feel like they were playing around a little bit by versioning it as version 4.1. It seems like a dramatically different beast than Deepseek V4 Flash.