The article mentions the 8 bit quantization, I believe this is about training in fp8 as native format. The latest GPUs provide huge flops for those, Tim Dettmer

The article mentions the 8 bit quantization, I believe this is about training in fp8 as native format. The latest GPUs provide huge flops for those, Tim Dettmers updated his gpu article and he talks about this, the claim is 0.66 PFLOPS for an RTX 4090.