liuliu

33 ms·

1.
by liuliu 19d ago
Yes, I heard you! I think one of the issue Draw Things inherited is the baggage of supporting too much models. Once you settled on a model, then it is just "try recommended settings", and prompt. The model part is unfortunate, but
2.
by liuliu 19d ago
What you get with Draw Things: 1. Download the app from Mac AppStore; 2. Download the model; 3. Tap "Try recommended settings", have the guarantee that for whatever model it supports, it is the fastest in Mac ecosystem, no need to
3.
by liuliu 19d ago
The local LLM scene needs a Draw Things equivalent for Mac. Too much fiddle for things that doesn't make sense (Qwen 3.8 27B should be exactly the same speed as Qwen 3.6 27B). It feels like that I am teasing (I am the author of Draw Th
4.
by liuliu 27d ago
I agree. But I think the DFlash2 case is just that 1/1000 invalid syntax failure case from sampling rather than a bug.
5.
by liuliu 28d ago
Correct. I am trying to explain why even it is "exact", the generated text is different from the with / without DFlash2 runs, and potentially why the DFlash2 run will contain the invalid Python syntax.
6.
by liuliu 28d ago
Yes, it doesn’t impact the probability distribution due to verifier. However, remember how you use PRNG and effectively due to the drafter is sampled from a different distribution initially, a separate rejection sampling won’t be able to re
7.
by liuliu 28d ago
Only if you do greedy sampling. With probabilisitic sampling (categorical sampling), you will end up with different trajectory just “mathematically equivalent”.
8.
by liuliu 1mo ago
1. The syntax surface is smaller, allowing less LLM "creativity; 2. The error handling is mechanical, which LLM clearly prefers (LLM is already trigger happy about writing tons of throw / try...catch.. in other languages, doing t
9.
by liuliu 1mo ago
They ship a complete checkpoint for easily management (inference & training) in their own infrastructure. Moving to a LUT would make training on these layers impossible. BTW, these are not useful for lightweight fine-tuning, but might s
10.
by liuliu 1mo ago
One thing similar would be projecting both the head.weight and the final LLM activations into a smaller vector space, since that is basically just cosine similarity ranking step (so that would reduce the head.weight size). But again, it mus
11.
by liuliu 1mo ago
One thing is not obvious to me is how ConvRot can be applicable beyond diffusion models. Especially for LLM decoding, as each ConvRot would be more expensive for a given decoding vector, and it is required now, so you cannot easily get the
12.
by liuliu 1mo ago
It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different f
13.
by liuliu 2mo ago
Well, it is a "platform of Balancing Budget", you don't need to actually work on that or do anything. The harder part is just letting people believing in miracles. Look no further than the current administration. Lying has no
14.
by liuliu 2mo ago
DS4 is designed to do real-work. Gemma 4 is not going to cut it.
15.
by liuliu 2mo ago
> training an LLM takes more resources and expertise than distilling from an existing LLM This is not automatically true. Training and distillation use the same underlying infra and method and there is no intrinsic differences in between
16.
by liuliu 2mo ago
It is disabled because it doesn't work :) Try it and see the doom loop it gets itself in.
17.
by liuliu 2mo ago
Note that 3.5 9B cannot do thinking (while 3.6 27B can, pretty effectively, quite verbosely).
18.
by liuliu 2mo ago
You also need to pay close attention to BFCLv3 multi-turn result, that helps you to get a sense how frequently these quants will be in a doom loop.
19.
by liuliu 2mo ago
The problem, of course, is if you run the UD_Q2 variant (Unsloth) which does only post-training, the number is pretty close to 1-bit model here and the 5% drop in tool-call is significant than it suggests in real-life use cases.
20.
by liuliu 2mo ago
This should be the first comment here. Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok Build team to have a toggle from the server to let
21.
by liuliu 3mo ago
Realistically, you need to experiment with any user prompt + a good amount of system prompt (at least > 1000 tokens, but realistically, in the range of 3000 tokens probably good). llama.cpp includes tools for that, what you are looking a
22.
by liuliu 3mo ago
It is a “research note”. It might not pan out, and you might say it doesn’t deserve the attention on the internet. But it did suggest something that resembles of compression, just no experiment done for that.
23.
by liuliu 4mo ago
I actually agree. At some point, a RSI system has to interact with real-world, and that imposes serialization constraints. It is harder to know how much that slow-down would be and how much speed-up we will get before that. But a RSI cannot
24.
by liuliu 4mo ago
Except the two (GPT-Image-2 and Nano Banana Pro), anything displayed here can run on the 16 GiB MacBook (including the FLUX.2 [dev]): https://tests.drawthings.ai/generate
25.
by liuliu 4mo ago
> To our knowledge, Bonsai Image 4B is the first image model in its parameter class to run directly on an iPhone. This is wrong. But they worded it carefully to be not entirely wrong. FLUX.2 [klein] 4B (the same parameter class, basicall
26.
by liuliu 4mo ago
It solves part of the download issue if they actually delivers a 1-bit whole package (currently their download is around 3.5GiB, still not ideal since FLUX.2 [klein] 4B you can get a package including text encoder ~6 GiB). For speed, no. Dr
27.
by liuliu 4mo ago
One thing people seems not to acknowledge, and this post made it super clear is that NVIDIA kept their lead extremely well in a few years of very high growth. The TFLOPs, the bandwidth, the interconnect mentioned in this post continues to g
28.
by liuliu 4mo ago
Probably not really. For gaming, I think probably just need to have a better way to explain visual and what the problem is (collision not done correctly, ways to feedback to LLM's experimentation loop how that should be checked and why
29.
by liuliu 4mo ago
Only if you think B is an important thing. He is easily > $100M from Tesla.
30.
by liuliu 4mo ago
Since the frontier is only 8-month ahead of DeepSeek, it is hard to see how model training can be a moat as all the tricks are available from open labs in China. You really just need <100m to bootstrap at this point.

More ›