It's not that GPUs don't support fp64, it's that for domestic gamer GPUs fp64 arithmetic is normally ~1:32 performance of fp32 arithmetic. e.g. 1080gtx FP

It's not that GPUs don't support fp64, it's that for domestic gamer GPUs fp64 arithmetic is normally ~1:32 performance of fp32 arithmetic. e.g. 1080gtx FP16 (half) performance 138.6 GFLOPS (1:64) FP32 (float) performance 8.873 TFLOPS FP64 (double) performance 277.3 GFLOPS (1:32) e.g. 3090rtx FP16 (half) performance 35.58 TFLOPS (1:1) FP32 (float) performance 35.58 TFLOPS FP64 (double) performance 556.0 GFLOPS (1:64) Only generally 'tesla' class cards targeted at super computers have a 1:2 ratio (e.g. v100, A100, Titan V). Note, I believe Titan V is the only Titan series GPU with good double performance, as the Volta architecture was never available to Geforce GPUs. https://www.techpowerup.com/gpu-specs/geforce-gtx-1080.c2839 https://www.techpowerup.com/gpu-specs/geforce-gtx-1080.c2839 https://www.techpowerup.com/gpu-specs/geforce-rtx-3090.c3622 https://www.techpowerup.com/gpu-specs/geforce-rtx-3090.c3622 https://www.techpowerup.com/gpu-specs/tesla-v100-sxm3-32-gb.c3472 https://www.techpowerup.com/gpu-specs/tesla-v100-sxm3-32-gb.... https://www.techpowerup.com/gpu-specs/a100-sxm4-80-gb.c3746 https://www.techpowerup.com/gpu-specs/a100-sxm4-80-gb.c3746 https://www.techpowerup.com/gpu-specs/titan-v.c3051 https://www.techpowerup.com/gpu-specs/titan-v.c3051