5 ms·
I actually worked on gsutil a while ago! I added in flight compression -J/j. Glad to see gsutil is on its way out.
by mooman219 4y ago
I actually worked on gsutil a while ago! I added in flight compression -J/j. Glad to see gsutil is on its way out.
- titanomachy 4y agoHow did you choose the letter "j"? were all the more obvious ones just taken?
- rwiggins 4y agoMaybe the inspiration was `-j` for `tar`, which enables bzip2 compression?
- hinkley 4y agoNow we just have to explain why -j means bzip2 for 'tar'...
- mooman219 4y agoThe original intention for most of the flags was to keep them aligned with rsync. The feature only compressed the data in flight, as opposed to -Z/z which stored the compressed version. This didn't directly align to anything in rsync, so I picked an unused letter in rsync (-J/j). In retrospect, it might have had a better home as some option on -Z/z, or if I had the tool figure out if you were bottlenecked on bandwidth, spare compute, and your data compressed well, and apply it automatically.
- hinkley 4y agoI don't know how long this flag has been there, but -z does mean compression in-flight in rsync. > -z, --compress compress file data during the transfer
- mooman219 4y agoYou're totally right! If anything the gsutil's -Z/z doesn't align with rsync's -Z/z, and gsutil's -J/j is a better match for what rsync's -Z/z does. I added -J/j well after -Z/z was there so it's something that we have to life with unfortunately.