6 ms·
How do you deal with how poorly rclone handles rate limits? It doesn't honor dropbox's retry-after header and just adds an exponential back off that, in my migr
by plasticsoprano 8mo ago
How do you deal with how poorly rclone handles rate limits? It doesn't honor dropbox's retry-after header and just adds an exponential back off that, in my migrations, has resulted in a pause of days.
I've adjusted threads and the various other controls rclone offers but I still feel like I'm not see it's true potential because the second it hits a rate limit I can all but guarantee that job will have to be restarted with new settings.
- coreylane 8mo agoI honestly haven't used it with Dropbox before, have you tried adjusting --tpslimit 12 --tpslimit-burst 0 flags? Are you creating a dedicated api key for the transfer? Rate limits may vary between Plus/Advanced forum.rclone.org is quite active you may want to post more details there.
- plasticsoprano 8mo agoI have made a dedicated api application and I have adjusted the tps flags. I’ve scanned the forums a few times but I’ve yet to inquire there.
- darthShadow 8mo ago> doesn't honor dropbox's retry-after header That hasn't been true for more than 8 years now. Source: https://github.com/rclone/rclone/blob/9abf9d38c0b8009430228150c8f1f0a50e52505b/backend/dropbox/dropbox.go#L418 https://github.com/rclone/rclone/blob/9abf9d38c0b80094302281... And the PR adding it: https://github.com/rclone/rclone/pull/2622 https://github.com/rclone/rclone/pull/2622
- plasticsoprano 8mo agoInteresting. I’ve been through 4 large transfers to Dropbox in the last 3 years and never once has it honored that header.