8 ms·
Ha made me chuckle. For those wondering seriously about this, it’s not a viable optimization because weights are not readily compressible via JPEG/DCT, and ther
by refibrillator 1y ago
Ha made me chuckle. For those wondering seriously about this, it’s not a viable optimization because weights are not readily compressible via JPEG/DCT, and there are a limited number of these units on the chip which bottlenecks throughout, meaning speed is dwarfed by simply reading uncompressed weights from HBM.
- moralestapia 1y agoYeah, but they could be. I won an GPU hackathon back in 2019 doing something very similar to this; although the other way around, I was compressing weights using hardware modules.
- heavyset_go 1y agoHave a link to this?
- moralestapia 1y agoUnfortunately no. I have cool picture, though!
- heavyset_go 1y agoI will have to settle for a picture then :)
- moralestapia 1y agoSend email (see profile), I'll gladly share more details ^^.
- jhoho 1y agoIt seems like this is indeed possible using video codecs: https://arxiv.org/abs/2407.00467v1 https://arxiv.org/abs/2407.00467v1
- touisteur 1y agoGood fun. Now I wish RT cores would be programmable with some form of PTX, but for now it's Optix or die. Managed to do fun stuff with it but it's like pulling teeth.