7 ms·
Many newer Chinese lab models are releasing with int4 native weights. Latest NVIDIA generation GPUs have a hard time with this and can actually be slower than p
by bradfa 3mo ago
Many newer Chinese lab models are releasing with int4 native weights. Latest NVIDIA generation GPUs have a hard time with this and can actually be slower than previous generations. This may make Blackwell depreciate faster than other recent generations.
- BoorishBears 3mo agoThat's not a real problem, hardly any 3rd party was running native weights anyways so they'll get quantized to NVFP4