7 ms·
Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
- cataPhil 2y agoI recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
- quibono 2y agoIs your work publicly available anywhere? I'd love to have a look if possible.
- lagrange77 2y agoAwesome, thank you for publishing it. I was planning the same, but didn't find the time yet.
- coolThingsFirst 2y agoLovely project, what did you find as the beat place to learn about cuda?
- cataPhil 2y agoThanks! I highly recommend the book Programming Massively Parallel Processors
- st380752143 2y agoThis is the second time this week that people have recommended this boot while I was searching for information about CUDA. Thanks!
- lairv 2y agoIf you ever feel like improving your matmul kernel you can check this blog https://siboehm.com/articles/22/CUDA-MMM https://siboehm.com/articles/22/CUDA-MMM
- cataPhil 2y agoThanks! I will probably do that this weekend
- suhacker256 2y agoVery cool!!
- zer0zzz 2y agoCool
- kosolam 2y agoAwesome. Thanks