10 ms·

I'm an occasional contributor for Rclone. Rclone holds a special place for me as the first open source project that I got seriously involved in and made me far

I'm an occasional contributor for Rclone. Rclone holds a special place for me as the first open source project that I got seriously involved in and made me far more comfortable with working in open source. Rclone is now very prevalent in my infrastructure. Almost all my websites are updated by a CI job that builds the website from a repo and pushes it up to the hosting server. There's an encrypted Rclone config in the repo and the password for it is a really long randomly generated string that gets saved in the CI as a secret. Rclone with Restic is how most of my servers get backed up, Rclone is how I access my Nextcloud and Google Drive, I have a containerized s3 compatible storage system that actually stores it's data on a Rclone remote (I hope `serve s3` gets implemented soon this setup can be simpler [1]), and much more. I'm using it so much that I'm running up against the Google Drive API limit even though I'm using my own key. * [1] https://github.com/rclone/rclone/issues/3382 https://github.com/rclone/rclone/issues/3382

leotaku 6y ago
I did not know that Restic was built on top of Rclone, but of course that makes a lot of sense. Thanks for contributing to a project that has made my life much easier!
- tankenmate 6y ago
  rclone is just one of the backends you can use for restic. personally i find it far more performant to use a local disk for your restic repo (and keep the desired number of snapshots, etc there) and then use rclone to clone that repo to a cloud provider. personally i used backblaze's b2 api as it is generally cheaper. restic is somewhat sensitive to the latency of the backend it uses.
  - anderspitman 6y ago
    I do the exact same thing.
DelightOne 6y ago
Is it better to integrate with the go-api or to write files to a local folder and run rclone on that to upload files? I'm planning to upload files from my app and let an rclone cronjob occasionally sync between my backends.
aspenmayer 6y ago
You may benefit from rotating service accounts to sidestep the API limits. I haven't implemented this on my own setup yet, but I know that it is scriptable. Implementation differences prevent one overarching tutorial, but I left some breadcrumbs for you below. https://github.com/l3uddz/cloudplow https://github.com/l3uddz/cloudplow https://github.com/Rhilip/AutoRclone https://github.com/Rhilip/AutoRclone https://hub.docker.com/r/hotio/rclone https://hub.docker.com/r/hotio/rclone

leotaku 6y ago

I did not know that Restic was built on top of Rclone, but of course that makes a lot of sense. Thanks for contributing to a project that has made my life much easier!

tankenmate 6y ago
rclone is just one of the backends you can use for restic. personally i find it far more performant to use a local disk for your restic repo (and keep the desired number of snapshots, etc there) and then use rclone to clone that repo to a cloud provider. personally i used backblaze's b2 api as it is generally cheaper. restic is somewhat sensitive to the latency of the backend it uses.
- anderspitman 6y ago
  I do the exact same thing.

DelightOne 6y ago

Is it better to integrate with the go-api or to write files to a local folder and run rclone on that to upload files? I'm planning to upload files from my app and let an rclone cronjob occasionally sync between my backends.

aspenmayer 6y ago

You may benefit from rotating service accounts to sidestep the API limits. I haven't implemented this on my own setup yet, but I know that it is scriptable. Implementation differences prevent one overarching tutorial, but I left some breadcrumbs for you below. https://github.com/l3uddz/cloudplow https://github.com/l3uddz/cloudplow https://github.com/Rhilip/AutoRclone https://github.com/Rhilip/AutoRclone https://hub.docker.com/r/hotio/rclone https://hub.docker.com/r/hotio/rclone