dot_treo

10 ms·

1.
by dot_treo 2mo ago
It is actually an insurance kind of thing as far as I understand. I've had plenty of cases where the bus couldn't continue due to being in a traffic jam, and people who would have happily get off the bus couldn't either.
2.
by dot_treo 4mo ago
Reminds me very much of https://music.ishkur.com/ which is the same kind of thing but for electronic music.
3.
by dot_treo 4mo ago
By the looks of it, it will take a couple more follow up PRs to clean things up a bit and get the most performance from MTP. I hope that by that point it will be easier to add more spec decoding types. In the meantime I've benchmarked
4.
by dot_treo 4mo ago
And it also looks like the original authors are working on qwen 3.5 too: https://github.com/chiennv2000/orthrus/issues/1#issuecomment...
5.
by dot_treo 4mo ago
I would probably treat the (3 GatedDeltaNet + 1 GatedAttention) Blocks as one transformer block, when generating next steps one would therefore use the kv cache for the gated attention and skip the entire delta nets.
6.
by dot_treo 4mo ago
I've tried MTP, and that got me about 1.5x on average with a very spec friendly benchmark. I didn't run the full benchmark with the demo code, just picked up a single prompt from it. The prompt is about 1300 token, the response is
7.
by dot_treo 4mo ago
Just to get it into a GGUF file would be fairly trivial. But using that GGUF file would need a bunch of additional things. One would need to create a new architecture derived from Qwen3, and then probably adapt the speculative decoding func
8.
by dot_treo 4mo ago
It is all about moving the bottleneck. During prompt processing everything can be calculated in parallel, while during token generation you create a single token at a time. For example, using an RTX 4000 Ada, I'm getting 2700 t/s
9.
by dot_treo 4mo ago
Do you plan on releasing the training code?
10.
by dot_treo 5mo ago
True, but how many 18 year olds do you know that will just randomly get their balls checked?
11.
by dot_treo 5mo ago
It used to be that way, and probably will be that way again. I know of a few of people who got an early testicular cancer diagnosis that way. So it seems that there is a medical use for it.
12.
by dot_treo 6mo ago
I don't care too much about the article being written with LLM support. There is actual work being done that is being showcased here. I'd rather read an LLM version of it, rather that not learning about those things at all.
13.
by dot_treo 6mo ago
The linguistic argument is fascinating. One particular thing, unrelated to the linguistic argument itself, stood out to me. In the PCA visualisation, we can see that some sequences of layers have particularly tight and stationary clusters.
14.
Do LLMs Break the Sapir-Whorf Hypothesis?(dnhkng.github.io)
15 points by dot_treo 6mo ago|7 comments
15.
by dot_treo 6mo ago
Looks like we discovered it at essentially the same time, and in essentially the same way. If the pth file didn't trigger a fork-bomb like behavior, this might have stayed undiscoverd for quite a bit longer. Good thinking on asking Cla
16.
by dot_treo 6mo ago
It actually wasn't. That was one of the reasons why I looked into what was changed. Even 1.82.6 is only at an RC release on github since just before the incident. So the fact that 1.82.7 and then 1.82.8 were released within an hour of
17.
by dot_treo 6mo ago
Yeah, that release has the base64 blob, but it didn't contain the pth file that auto triggers the malware on import.
18.
by dot_treo 6mo ago
Even just having an import statement for it is enough to trigger the malware in 1.82.8.
19.
Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised(github.com)
938 points by dot_treo 6mo ago|500 comments
20.
by dot_treo 6mo ago
The main reason is just how hard it is to actually create anything that integrates with Teams. You have to jump through so meany hoops, wade through so many deprecated APIs, guess through so many half-way-wrong-by-now documentation pages. A
21.
by dot_treo 8mo ago
In the past I've been trying to adopt the stoic mindset, but always struggled. But I continued to read and learn about it. Unrelatedly, I came across a recomendation for David Burns "Feeling Good" here on hackernews a couple