7 ms·Speedup Megatron-LM from 10% to 30% with Zero Bubble Pipeline Parallelism4 points by yzh 3y agoufotalent 3y agoAn interesting playground to understand the scheduler is here https://huggingface.co/spaces/sail/zero-bubble-pipeline-parallellism https://huggingface.co/spaces/sail/zero-bubble-pipeline-para...QPHutu 3y agoCool!