6 ms·
What are the pros of using CUDA-enabled devices for inference?
by alienthrowaway 1y ago
What are the pros of using CUDA-enabled devices for inference?
- bitexploder 1y agoMy limited understanding is that CUDA wins on smaller batches and jobs but TPU wins on larger jobs. It is just easier to use and better at typical small workloads. At some point for bigger ML loads and inference TPU starts making sense.