7 ms·That would be 1.38 bits per weight on average, which I can confidently guess would not perform well.by vletal 2y agoThat would be 1.38 bits per weight on average, which I can confidently guess would not perform well.Lerc 2y agoThe model card says the 70B is 16 bit so I think you have twice thatqeternity 2y agoBitNet is functional at 1.58 bpw.