coder543

5 ms·

31.
by coder543 3mo ago
> I have come to consider Gemma 4 31b the best model I can self-host I'm confused. Your own results show that Gemma 4 26B A4B and Qwen3.6-27B did better in these tests? I really like Gemma 4 31B, especially with how exceptionally go
32.
by coder543 3mo ago
Is a recipe useful if no one likes it? There are equally open, much more useful models out there: https://artificialanalysis.ai/?models=nvidia-nemotron-3-ultr...
33.
by coder543 3mo ago
Claude Sonnet 4.6 identified itself as DeepSeek repeatedly: https://www.reddit.com/r/DeepSeek/comments/1rd5jw7/claude_so... I tested this myself a few months ago, and confirmed that it was really happeni
34.
by coder543 3mo ago
dsv4 flash has 284 billion parameters, not 158 billion. Huggingface's little parameter count badge seems unreliable.
35.
by coder543 3mo ago
I agree completely. It's also annoying that OpenCode doesn't even try to support local LLMs properly. Getting OpenCode to work is possible, but extremely manual and clunky to configure. I have written a script to automate converti
36.
by coder543 3mo ago
Qwen3.6-27B supports a 1 million token context window. Of course, you have to have the right hardware to be able to run with a context window like that, as it takes about 100GB of memory on my DGX Spark to do that with full f16 KV cache on
37.
by coder543 3mo ago
Don't forget the open weight model they could release: Free Verse.
38.
by coder543 3mo ago
Yep, Unsloth already did, as linked in the comment at the top of this thread
39.
by coder543 3mo ago
> meaning Google quantized the model to 4 bit and stored the result in BF16 format for compatibility and convenience to downstream packers. You also misunderstand what is happening. Google did not do that. Google further trained the orig
40.
by coder543 4mo ago
Closed source? IntelliJ: https://github.com/JetBrains/intellij-community PyCharm: https://github.com/JetBrains/intellij-community/tree/master/... Android Studio: https://
41.
by coder543 4mo ago
I have never ever seen Windows provide this warning even once just because there is a faster port on the machine and the user plugged the device into the wrong one. Please provide a source for this claim that you are making. Citation absolu
42.
by coder543 4mo ago
The computer pops up a warning if you plug a fast device into the slow port, which is a lot more informative for the average user than a tiny label that most users wouldn’t even read. Labels would be nice, I guess, but their absence is hard
43.
by coder543 5mo ago
That's great news. That has not been the case with other MTP implementations like Qwen3.5, but I see the section in the article saying Google introduced some architectural optimizations to make this possible.
44.
by coder543 5mo ago
MTP requires a separate KV cache, so there is more memory overhead than just the weights of the MTP model, but it's a manageable amount.
45.
by coder543 5mo ago
Your “benchmark” is invalid. Penalizing the model because the hosting environment is being DDoSed by users a few hours after launch is utter nonsense. I see that you tried to justify this lower in the thread, but no… it completely invalidat
46.
by coder543 5mo ago
The description specifically says: "Kimi-K2.6 adopts the same native int4 quantization method as Kimi-K2-Thinking."
47.
by coder543 5mo ago
From the page: > Import from anywhere. Start from a text prompt, upload images and documents (DOCX, PPTX, XLSX), or point Claude at your codebase. You can also use the web capture tool to grab elements directly from your website so proto
48.
by coder543 5mo ago
No… seriously. Every model release is accused. Including Opus, GPT-5.4, whatever. And yes, including smaller models that are not the top in every benchmark. My own experiences with Gemma 4 have been quite mediocre: https://www.re
49.
by coder543 5mo ago
Every model release gets accused of that, including the flagship models.
50.
by coder543 5mo ago
Artificial Analysis hasn't posted their independent analysis of Qwen3.6 35B A3B yet, but Alibaba's benchmarks paint it as being on par with Qwen3.5 27B (or better in some cases). Even Qwen3.5 35B A3B benchmarks roughly on par with
51.
by coder543 5mo ago
Not true. With a MoE, you can offload quite a bit of the model to CPU without losing a ton of performance. 16GB should be fine to run the 4-bit (or larger) model at speeds that are decent. The --n-cpu-moe parameter is the key one on llama-s
52.
by coder543 5mo ago
That is an extremely strange article, in my opinion. They test Gemma 4 31B, but they use Qwen3 32B, DeepSeek R1, and Kimi K2, which are all outdated models whose replacements were released long before Gemma 4? Qwen3.5 27B would have done fa
53.
by coder543 5mo ago
Gemma 4 31B has now wiped out several of those models from the pareto frontier, now that it has pricing. Gemma 4 26B A4B has an Elo, but no pricing, so it still isn't on that chart. The Gemma 4 E2B/E4B models still aren't on
54.
by coder543 6mo ago
If you search the model card[0], there is a section titled "Code for processing Audio", which you can probably use to test things out. But, the model card makes the audio support seem disappointing: > Audio supports a maximum l
55.
by coder543 6mo ago
The E2B and E4B models support 128k context, not 256k, and even with the 128k... it could take a long time to process that much context on most phones, even with the processor running full tilt. It's hard to say without benchmarks, but
56.
by coder543 6mo ago
Reducing the expert count after training causes catastrophic loss of knowledge and skills. Cerebras does this with their REAP models (although it is applied to the total set of experts, not just routing to fewer experts each time), and it c
57.
by coder543 6mo ago
That rule of thumb was invented years ago, and I don’t think it is relevant anymore, despite how frequently it is quoted on Reddit. It is certainly not the "current" rule of thumb. For the sake of argument, even if we take that ol
58.
by coder543 6mo ago
It was just an example of a bug, not that it was the only bug. I’ve personally reported at least one other for Gemma 4 on llama.cpp already. In a few days, I imagine that Gemma 4 support should be in better shape.
59.
by coder543 6mo ago
That Pareto plot doesn't seem include the Gemma 4 models anywhere (not just not at the frontier), likely because pricing wasn't available when the chart was generated. At least, I can't find the Gemma 4 models there. So, no
60.
by coder543 6mo ago
There are issues with the chat template right now[0], so tool calling does not work reliably[1]. Every time people try to rush to judge open models on launch day... it never goes well. There are ~always bugs on launch day. [0]: https:&#x2F

More ›