7 ms·
It there anything similar for RTX 3090 and RTX 4090?
by pulse7 1mo ago
It there anything similar for RTX 3090 and RTX 4090?
- mmlkrx 1mo agoI'm not sure about a 4090 but there is a fork for 3090s: https://github.com/Don-Chad/ninfer-3090 https://github.com/Don-Chad/ninfer-3090
- mirekrusin 1mo ago[dead]
- ixaxaar 1mo agoI'm running it using a 4090 on using llama.cpp with Q5_K_S and its running at ~33 t/s