lopuhin

7 ms·

1.
by lopuhin 5mo ago
Qwen3.5 is able to output click coordinates and bounding boxes just fine, as values normalized to 0..1000, I’d hope Qwen3.6 didn’t loose this capability.
2.
by lopuhin 7mo ago
For that you only need high throughput which is much easier to achieve compared to high latency, thanks to batching -- assuming the log lines or chunks can be processed independently. You can check TensorRT-LLM benchmarks ( https:/&#x2
3.
by lopuhin 8mo ago
On whether this accounts only the final output layer -- once the first token is generated (i.e. selected according to the modified sampling procedure), and assuming a different token is selected compared to standard sampling, then all layer
4.
by lopuhin 9mo ago
That's impressive how you figured out the reason for the difference in list of floats vs list of ints container size, framed as an interview question that would have been quite difficult I think
5.
by lopuhin 9mo ago
Context window size of 400k is not new, gpt-5, 5.1, 5-mini, etc. have the same. But they do claim they improved long context performance which if true would be great.
6.
by lopuhin 1y ago
you can rent them for less then $2/h in a lot of places (maybe not in the drawer)
7.
by lopuhin 1y ago
I find OpenAI's new flex processing more attractive, as it has the same 50% discount, but allows to use the same API as regular chat mode, so you can still do stuff where Batch API won't work (e.g. evaluating agents), and in pract
8.
by lopuhin 1y ago
it's pretty difficult to package native python dependencies for wasmtime or other wasi runtimes, e.g. lxml
9.
Visualize LLM Token Probabilities and Confidence with ELI5(eli5.readthedocs.io)
2 points by lopuhin 1y ago|0 comments
10.
by lopuhin 1y ago
Crazy amount of breakage... Here is a PR which reverts this: https://github.com/pypa/setuptools/pull/4911 Interesting that maintainers of setuptools still only postpone the depreciation date for a year, so we
11.
by lopuhin 2y ago
Congrats on the launch! How much does it cost? And what is the sandboxing technology?
12.
by lopuhin 2y ago
I find it strange that the author is really happy with the quality of string comparison here https://pgaleone.eu/ai/coding/2025/01/26/using-ai-for-coding... and while it would kind of work, it'
13.
by lopuhin 2y ago
It's a 600B+ mixture of experts and yes it's described in the paper, GitHub, etc.
14.
by lopuhin 2y ago
Why is this doubtful, did you spot any suspicious things in their paper? They make the weights and a lot of training details open as well, which leaves much less room for making stuff up, e.g. you could check training compute requirements f
15.
by lopuhin 2y ago
With distilled models being released, it's very likely they'd be soon served by other providers at a good price and perf, unlike the full R1 which is very big and much harder to serve efficiently.
16.
by lopuhin 2y ago
I don't think so, what they show on CS video is exactly the Dust2 map, not just something similar/inspired by it.
17.
by lopuhin 2y ago
I think GraalPython does have a GIL, see https://github.com/oracle/graalpython/blob/master/docs/contr... - and if by "there is no such thing on those platforms" you mean JVM/CLR not h
18.
by lopuhin 2y ago
Curious which model was used? Sorry if I missed that. Looks like an important detail to mention when doing an evaluation.
19.
by lopuhin 2y ago
Also I don't think you can use NIM packages in production without a subscription, and I wasn't able to find the cost without signing up. Also NIM package for Mistral Nemo is not yet available anyways.
20.
by lopuhin 2y ago
The README says they plan to add llama.cpp support which should cover a lot of targets, also they have tinygrad already integrated I think.
21.
by lopuhin 2y ago
Not quite the same, OpenAI was initially quite open, while Ilia is currently very explicitly against opening or open-sourcing research, e.g. see https://www.theverge.com/2023/3/15/23640180/openai-gpt-4-la
22.
by lopuhin 2y ago
yes looks like a bug in the example to me, feel free to report to https://github.com/fchollet/ARC-AGI/issues :)
23.
by lopuhin 2y ago
Really nice to see no-GIL Python become closer and closer to reality.
24.
by lopuhin 2y ago
I think they mean "chemical composition", e.g. how much hydrogen / helium / other elements there are in the star, deduced from their spectra
25.
by lopuhin 2y ago
Then the date of 5th July 2023 looks likely as this is the date from which overall privacy policy is in effect: https://slack.com/intl/en-gb/trust/privacy/privacy-policy Interesting choice of date btw
26.
by lopuhin 2y ago
And also article is testing on a different task (Needle in a Needlestack which is kind of similar to Needle in a Haystack), compared to finding a difference between two documents. For sure it's useful to know that the model does ok in
27.
by lopuhin 2y ago
Similar or greater inference wins are achieved with speculative decoding which is already widely used, so while this is really interesting (and was tried before with less success AFAIK), it's not yet clear how impactful it would be.
28.
by lopuhin 3y ago
If convergence were a matter of luck, it would look completely different, like white noise, but it clearly has well-defined structure. The reason for high learning rate is that they used full batched training (see the first cell in https:&
29.
by lopuhin 3y ago
Strange that they don't mention the performance, how long does it take to do one step, and how does it compare to a similarly priced GPU cluster? Sure simple code is good, but it needs to also be useful.
30.
by lopuhin 3y ago
Yes it might. "Reduction of number of neurons" is not static here, unlike traditional pruning approaches, here they still keep all weights, but the network dynamically selects which sub-portion of them to use. There is a related d

More ›