roanakb

7 ms·

1. 1.▲
   What Neptune.ai Got Right (and How to Keep It)(trainy.ai)

   2 points by roanakb 7mo ago|0 comments
2. 2.▲
   Show HN: Pluto – open-source Experiment Tracker for Neptune users(github.com)

   2 points by roanakb 7mo ago|0 comments
3. 3.▲
   by roanakb 2y ago

   Unfortunately, SM efficiency is not accessible via nvidia-smi. The best methods to track it would be to: 1. Profile your model with Pytorch Profiler 2. Export metrics with Nvidia DCGM
4. 4.▲
   by roanakb 2y ago

   oh this looks great, thank you for bringing this up! I'll have to give it a try, but seems like the FSDP limitation on torch.compile might carry over?
5. 5.▲
   by roanakb 2y ago

   Yup, you'll see 100% utilization on a kernel over a time period if it's considered active, which includes just having a single thread executing [1]. SM occupancy is great but can be a little difficult to interpret since you'r
6. 6.▲
   by roanakb 2y ago

   Nice, seems like ML Productivity Goodput is a pretty well thought-out metric to understand the overall efficiency of your cluster. I'll consider adding this into our cluster management platform. Only potential drawbacks I'd guess
7. 7.▲
   by roanakb 2y ago

   Agreed, roofline plots would be quite powerful in this context. From a quick search, seems like the only way to create a roofline plot for your model would be to use Nsight [1]? Would be interested to know if there are any simpler tools, si
8. 8.▲
   by roanakb 2y ago

   Yup, similar to SM efficiency in that sense too. If you aren't seeing >80%, there is certainly time left on the table. But getting a high SM efficiency value doesn't guarantee you're making good use of the hardware as well
9. 9.▲
   GPU utilization can be a misleading metric(trainy.ai)

   144 points by roanakb 2y ago|36 comments
10. 10.▲
    by roanakb 2y ago

    this is a good one for debugging rdma: https://docs.redhat.com/en/documentation/red_hat_enterprise_...
11. 11.▲
    by roanakb 2y ago

    Looks great, you guys made it really easy to integrate!
12. 12.▲
    Show HN: GPU Health Controller(trainy.ai)

    2 points by roanakb 2y ago|0 comments
13. 13.▲
    GPT-4 can catch edge cases in your code using unit tests(docs.sweep.dev)

    3 points by roanakb 3y ago|0 comments
14. 14.▲
    by roanakb 3y ago

    Thanks! Let me know if there are any features you'd like to see added.
15. 15.▲
    Monitor and Optimize your large-scale model training(trainy.ai)

    13 points by roanakb 3y ago|2 comments
16. 16.▲
    by roanakb 3y ago

    Looks really cool! Nice work.
17. 17.▲
    by roanakb 6y ago

    Location: Fremont, CA Remote: Yes Willing to relocate: No Technologies: Python, R, Pytorch, Keras, SciPy, Machine Learning, Deep Learning, Computer Vision, NLP, Javascript, MERN Stack Résumé/CV: on request Linkedin: https://
18. 18.▲
    Adding a Voice Assistant to a React.js Hacker News Clone(medium.com)

    3 points by roanakb 6y ago|0 comments