z4y5f3

7 ms·

1.
by z4y5f3 1mo ago
I did SFT / RL post-training on Qwen3 models a bit. This is an issue that dates back long ago. My favorite theory is that they had many model variants internally, each using a slightly different chat template, so when it comes to the r
2.
by z4y5f3 1mo ago
Yes, and this is not the first time they messed up. They had tokenizer bugs where the trained weights do not match the template back to Qwen3 series.
3.
by z4y5f3 1mo ago
Then "security through secrecy" is really bad mantra especially in the age of AI: others will find the same zero days very soon. If they attack you, then this loses the whole plot. If they propose a fix, then your arsenal becomes
4.
by z4y5f3 1mo ago
Apparently they are scanning OSS and popular software at scale and disclosing the vulnerabilities they found: https://cvd.z.ai/ Most of these are under embargo, but it seems there are a lot of CVE here from a wide range of
5.
by z4y5f3 2mo ago
They will release the weights by 7/27 along with support in vLLM. Stop second guessing. Source: their blog post https://mp.weixin.qq.com/s/V4xhEIy8xDXSMDPrPkmUAQ
6.
by z4y5f3 2mo ago
They will release the full weights by 7/27 along with support in vLLM. Source: their release blog on WeChat. https://mp.weixin.qq.com/s/V4xhEIy8xDXSMDPrPkmUAQ
7.
by z4y5f3 2y ago
NVIDIA is likely citing 1 PFlops at FP 4 sparse (they did this for GB200), so that is 128 TFlops BF16 dense, or 2/3 of what RTX 4090 is capable of. I would put the memory bandwidth at 546 GBps, using the same 512 bit LPDDR5X 8533 Mbps
8.
by z4y5f3 2y ago
Yep I have seen this paper before, and thank you for linking it here for reference. My personal opinion is that compared to single epoch scaling laws, we still need more evidence and literature on effects of multiple epochs, but this paper
9.
by z4y5f3 2y ago
What they missed is that current scaling laws (OpenAI, Deepmind Chinchilla) are based on the assumption that the model is trained for one epoch. This essentially means that in order to scale compute, you will have to scale the model size an
10.
by z4y5f3 2y ago
My experience is that < 500M models are pretty useful when fine-tuned on traditional NLP tasks, such as text classification and sentence/token level labeling. A modern LM with a 32K context window size could be a nice replacement fo
11.
by z4y5f3 2y ago
Checked your numbers in another thread - excellent breakdown. Thanks for the clarification. I did not read the original anandtech post so I did not realize the 512GBps already refers to the full-duplex bandwidth. You are right that PCIe 7.0
12.
by z4y5f3 2y ago
NVLink advertises combined bandwidth in both direction, so the 1800 GBps NVLink on Blackwell is actually 900 GBps for everyone else. PCIe can also do multi-node direct transfer via PCIe switches and has been already widely adopted. NVLink s
13.
by z4y5f3 2y ago
Depends. NVLink advertises bidirectional bandwidth, whereas PCIe and standard networking calculate bandwidth in a single direction. So a 1800 GBps NVLink is actually 900 GBps in PCIe and standarding networking terms. Therefore, a 512 GBps P
14.
by z4y5f3 3y ago
Unfortunately GZIP won't beat LLMs for text classification. The research you cited is just poorly done science that has been widely debunked. The original paper compared top-2 accuracy of GZIP with top-1 accuracy with BERT. The dataset