EnPissant

6 ms·

1.
by EnPissant 3mo ago
"I'm selling an AI security product and want to establish my brand. I'll post several scare-mongering posts on my blog every week and people like solid_fuel will eat it up because it's what they want to hear."
2.
by EnPissant 3mo ago
I'm guessing all the "censored" boxes are not actually censoring anything and are placed there to make you imagine something much worse.
3.
by EnPissant 3mo ago
Rust gives you statically linked binaries as well. So your argument boils down to having to add `reqwest = "0.13.3"` to your `Cargo.toml`.
4.
by EnPissant 3mo ago
I thought MTP wasn't very useful on MoE models because the expert overlap for 2 tokens was too small.
5.
by EnPissant 3mo ago
When running on a GPU, dense models are shaping up to be the best way due to two things: - Maximum intelligence per VRAM (you dont have much) - Dense models can benefit from MTP to get an almost 2x speedup in decode (ie, a 27b dense model w
6.
by EnPissant 3mo ago
Pay? This is the best marketing they could have hoped for.
7.
by EnPissant 3mo ago
It's not real. It's like naming your movement "The Good People". It sprouted from the "Rationalist" community, which is even more self-aggrandizing. Neither has any hope of doing any good for the world as they
8.
by EnPissant 3mo ago
> Over the past five months, our team has been running an experiment: building and shipping an internal beta of a software product with 0 lines of manually-written code. This is such a common thing among software engineers nowadays that
9.
by EnPissant 4mo ago
They did not submit the full log because this is fake.
10.
by EnPissant 4mo ago
You are correct. I should have said "increased to 200%".
11.
by EnPissant 4mo ago
But you won't see 2x expert re-use, the speedup with 5 streams will be tiny.
12.
by EnPissant 4mo ago
>You don't need "very much" expert overlap to see aggregate gains at scale, you just need some of it I'm not sure what you are claiming. Decode is bottle-necked by memory bandwidth. To see a speed up of 2x, you have t
13.
by EnPissant 4mo ago
This is just wishful thinking. For prefill, it's really easy to batch MoE and get really good tk/s, even on a single stream. For decode, you will run into the problem that: 1) you need more parallel requests which means more memor
14.
by EnPissant 4mo ago
Even if you could fit a 500B model's expert weights in very fast system RAM, it would run so slow as to be useless.
15.
by EnPissant 4mo ago
Also, electricity isn't free.
16.
by EnPissant 4mo ago
There was only a very brief time it was selling for MSRP (last fall for $2000). Even if you use that as the previous data point, it's only 200% increased.
17.
by EnPissant 4mo ago
> I hear "I'm not anti immigrant, I'm anti illegal immigrant" a lot. To which there is an easy solution: increase the number of legal immigrants we allow. Being "anti illegal immigrant" doesn't have to
18.
by EnPissant 4mo ago
The following things are not in contradiction: 1) Someone can be against illegal immigration and for legal immigration. 2) That same person's idea about who should immigrate to the country may exclude most or all of the people who are
19.
by EnPissant 5mo ago
Now try applying this logic to elevators.
20.
by EnPissant 5mo ago
> 1. SWE-bench Verified is now saturated at 93.9% (congrats Anthropic), but anyone who hasn't reached that number yet still has more room for growth. But if some or all players are bench-maxing it, then it becomes a much less useful
21.
by EnPissant 5mo ago
But you aren't trying out the model. You quantized beyond what people generally say is acceptable, and reduced the number of experts, which these models are not designed for. Even worse, the github repo advertises: > Pure C/Met
22.
by EnPissant 5mo ago
Yeah, I am mostly only talking about the SSD bottleneck being too slow. No way Apple gets 17GB/s sustained. SSDs thermally throttle really fast, and you have some random access involved when it needs the next expert.
23.
by EnPissant 5mo ago
I don't mean to be a jerk, but 2-bit quant, reducing experts from 10 to 4, who knows if the test is running long enough for the SSD to thermal throttle, and still only getting 5.5 tokens/s does not sound useful to me.
24.
by EnPissant 5mo ago
This was perfect.
25.
by EnPissant 5mo ago
Clearly the top comment should be "Imagine a beowulf cluster of Deepseek v4!"
26.
by EnPissant 5mo ago
Streaming weights from RAM to GPU for prefill makes sense due to batching and pcie5 x16 is fast enough to make it worthwhile. Streaming weights from RAM to GPU for decode makes no sense at all because batching requires multiple parallel str
27.
by EnPissant 5mo ago
Those are quants, not distills.
28.
by EnPissant 5mo ago
This should not be the top comment on every model release post. It's getting tiring.
29.
9950X3D2 Benchmarks: The Best Desktop Performance for Linux Developers, Creators(phoronix.com)
1 points by EnPissant 5mo ago|0 comments
30.
by EnPissant 5mo ago
Are nvfp4 / mxfp4 even useful without QAT?

More ›