5 ms·
Not the guy you replied to, but here are some improvements that feel obvious: 1. Memory indexing. It's a pain to avoid banking conflicts, and implement coopera
by programjames 1y ago
Not the guy you replied to, but here are some improvements that feel obvious:
1. Memory indexing. It's a pain to avoid banking conflicts, and implement cooperative loading on transposed matrices. To improve this, (1) pop up a warning when banking conflicts are detected, (2) make cooperative loading solved by the compiler. It wouldn't be too hard to have a second form of indexing memory_{idx} that the compiler solves a linear programming problem for to maximize throughput (do you spend more thread cycles cooperative loading, or are banking conflicts fine because you have other things to work on?)
2. Why is there no warning when shared memory is unspecified? It isn't hard to check if you're accessing an index that might not have been assigned a value. The compiler should pop out a warning and assign it to 0.0, or maybe even just throw an error.
3. Timing - doesn't exist. Pretty much the gold standard is to run your kernel 10_000 times in a loop and subtract the time from before and after the loop. This isn't terribly important, I'm just getting flashbacks to before I learned `timeit` was a thing in Python.
- jacquesm 1y agoThose are good and actionable suggestions. Have you passed these on to NVIDIA? https://forums.developer.nvidia.com/c/accelerated-computing/cuda/cuda-programming-and-performance/7 https://forums.developer.nvidia.com/c/accelerated-computing/... They regularly have threads asking for such suggestions. But I don't think they rise to the general conclusion that the tooling is bad.