diyer22

7 ms·

1.
by diyer22 7mo ago
MathArena uses newly released competition sets and evaluates models close to the event. They also mark models released after the competition date as potential contamination. On Feb 6, the just-concluded AIME 2026 I, Step 3.5 Flash take firs
2.
Open model StepFun-3.5 is #1 on MathArena, an uncheatable math benchmark(twitter.com)
3 points by diyer22 7mo ago|2 comments
3.
A new local LLM king: Step-3.5-Flash-int4(old.reddit.com)
2 points by diyer22 8mo ago|1 comments
4.
by diyer22 8mo ago
StepFun has open-sourced Step-3.5-Flash: 196 B total parameters, 11 B active, 256 K context length. Strong performance, with speed as the highlight—blazing fast, peaking at 350 tokens/s. It’s currently in promotion and free on OpenRout
5.
Got burned by an Apple ICLR paper – it was withdrawn after my Public Comment(twitter.com)
6 points by diyer22 10mo ago|3 comments
6.
by diyer22 10mo ago
A record of getting burned by a “sloppy, low-quality, and irresponsible” paper from Big Tech.
7.
by diyer22 11mo ago
Thanks for all the great feedback! I've created a Twitter thread to discuss future development and share updates. Would love to connect with you all there: https://x.com/diyerxx/status/1978531040068321766 Get
8.
by diyer22 11mo ago
I believe DDN is capable of handling TTS (text-to-speech) tasks, because with the text condition, the generation space is significantly reduced. And it's recommended to combine it with an autoregressive model (GPT) for more powerful mo
9.
by diyer22 11mo ago
During neural network training, the ground truth (GT) must be known to compute the loss. In DDN, the GT is only used to calculate the loss and guide sampling; it never becomes an input to the model.
10.
by diyer22 11mo ago
Exactly what i think! - The DDN single-shot generator architecture is more efficient than diffusion. - DDN is fully end-to-end differentiable, allowing for more efficient optimization when integrated with discriminative models or reinforcem
11.
by diyer22 11mo ago
Thanks for the idea, but DDN and flow can’t be flipped into each other that easily. 1. DDN doesn’t need to be invertible. 2. Its latent is discrete, not continuous. 3. As far as I know, flow keeps input and output the same size so it ca
12.
by diyer22 11mo ago
It does seem that way — we’re both trying to overcome the limitations imposed by LLM tokenization to achieve a truly end-to-end model. And, their work is far more polished; I’ve only put together a quick GPT+DDN proof-of-concept. Thank you
13.
by diyer22 11mo ago
Yes, there is a transform that make final size of stem features remains unchanged
14.
by diyer22 11mo ago
Even in the earliest stages of the DDN concept, we had already decided to pass features down to the next layer. I never even ran an ablation that disabled the stem features; I assume the network would still train without them, but since the
15.
by diyer22 11mo ago
It's just a coincidence—the guided images used for ZSCG all come from Celeb-A, whereas the DDN model was trained only on FFHQ. Besides, I feel the red shoulder strap/blob is reconstructed rather poorly.
16.
by diyer22 11mo ago
This understanding is incorrect. The video samples all the leaf nodes of the entire tree only to visualize the distribution in latent space. In normal use, only the L outputs along a single path are generated.
17.
by diyer22 11mo ago
1x1 convolution is the most lightweight operator for transforming features into outputs. 3x3 convolution is the most common operator used to provide basic computational power.
18.
by diyer22 11mo ago
I understand that by "discrete number" you mean the selected output of each layer. Both the "feature" and the "selected output" are designed to be passed to the next layer.
19.
by diyer22 11mo ago
That's right! The second paragraph on OPTIMIZATION WITH SPLIT-AND-PRUNE in the original paper: > Inspired by the theory of *evolution and genetic algorithms*, we propose the Split-and-Prune algorithm to address the above issues, a
20.
by diyer22 11mo ago
I agree with @ActivePattern and thank you for your help in answering. Supplement for @f_devd: During training, the K outputs share the stem feature from the NN blocks, so generating the K outputs costs only a small amount of extra computati
21.
by diyer22 11mo ago
In DDN, 1×1 convolutions are used only in the output layers of the Discrete Distribution Layer (DDL). The NN blocks between DDLs, which supply the fundamental computational power and parameter count, adopt standard 3×3 convolutions.
22.
by diyer22 11mo ago
We provide the source code and weights along with a Docker environment to facilitate reproducing the experimental results. The original paper’s EXPERIMENTS section mentions the hardware configuration (8× RTX 2080 Ti).
23.
by diyer22 11mo ago
Exactly! The paragraph on Efficient Data Compression Capability in the original paper also highlights: > To our knowledge, Taiji-DDN is the first generative model capable of directly transforming data into a semantically meaningful bin
24.
by diyer22 11mo ago
Thank you very much for your interest. 1. The comparison with GANs and the issue of mode collapse are addressed in Q2 at the end of the blog: https://github.com/Discrete-Distribution-Networks/Discrete-D... 2. Regardi
25.
by diyer22 11mo ago
I believe it is the novelty. Here I would like to quote Reviewer r4YK’s original words: > Many high rated papers would have been done by someone else if their authors never published them or were rejected. However, if this paper is not p
26.
by diyer22 11mo ago
Thank you for your appreciation. I will update the future work on both GitHub and Twitter. https://github.com/DIYer22 https://x.com/diyerxx
27.
by diyer22 11mo ago
I believe DDN is exceptionally well-suited to the “generative models for discriminative tasks” paradigm for object detection. Much like DiffusionDet, which applies diffusion models to detection, DDN can adopt the same philosophy. I expect
28.
by diyer22 11mo ago
No, DDN and VQ-VAE are clearly different. Similarities: - Both map data to a discrete latent space. Differences: - VQ-VAE needs an external prior over code indices (e.g. PixelCNN or a hierarchical prior) to model distribution. DDN build
29.
by diyer22 11mo ago
The first version of DDN was developed in less than three months, almost entirely by one person. Consequently, the experiments were preliminary and the results far from SoTA. The current goal in research is scaling up. Here are some thought
30.
by diyer22 11mo ago
Yes, it's absolutely possible—just like how diffusion LLMs work, we can do the same with DDN LLMs. I made an initial attempt to combine [DDN with GPT]( https://github.com/Discrete-Distribution-Networks/Discrete-D...

More ›