6 ms·
Is this a general issue rather than unsloth specific. How wide is this problem? Sounds wild if it has been affecting everyones training.
by ejddhbrbrrnrn 2y ago
Is this a general issue rather than unsloth specific. How wide is this problem? Sounds wild if it has been affecting everyones training.
- danielhanchen 2y agoUnfortunately it's not an Unsloth issue but a general issue affecting nearly all trainers which use grad accum. We worked with Huggingface so their trainers should be fixed now though in the main branch